andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
a4396f9218
rocksdb/tools
History
Shobhit Dayal b45b1cde3e Feature for sampling and reporting compressibility (#4842) 
Summary:
This is a feature to sample data-block compressibility and and report them as stats. 1 in N (tunable) blocks is sampled for compressibility using two algorithms:
1. lz4 or snappy for fast compression
2. zstd or zlib for slow but higher compression.

The stats are reported to the caller as raw-bytes and compressed-bytes. The block continues to be compressed for storage using the specified CompressionType.

The db_bench_tool how has a command line option for specifying the sampling rate. It's default value is 0 (no sampling). To test the overhead for a certain value, users can compare the performance of db_bench_tool, varying the sampling rate. It is unlikely to have a noticeable impact for high values like 20.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/4842

Differential Revision: D13629011

Pulled By: shobhitdayal

fbshipit-source-id: 14ca668bcab6499b2a1734edf848eb62a4f4fafa
2019-03-18 12:15:34 -07:00
..
advisor Rules Advisor: some fixes to support fetching stats from ODS (#4223) 2018-08-02 15:42:42 -07:00
dump fix gflags namespace 2017-12-01 10:42:05 -08:00
rdb Fix /bin/bash shebangs 2017-08-03 15:56:46 -07:00
analyze_txn_stress_test.sh WritePrepared: script to analyze stress test failures (#5033) 2019-03-01 09:18:40 -08:00
auto_sanity_test.sh Suppress lint in old files 2018-01-29 12:56:42 -08:00
benchmark_leveldb.sh Suppress lint in old files 2018-01-29 12:56:42 -08:00
benchmark.sh Updated benchmark script (#4134) 2018-12-17 16:34:30 -08:00
blob_dump.cc comment unused parameters to turn on -Wunused-parameter flag 2018-04-12 17:59:16 -07:00
check_format_compatible.sh Avoid using kInAtomicGroup tag for single-cf op (#4981) 2019-02-13 18:33:42 -08:00
CMakeLists.txt cmake support for linux and osx (#1358) 2016-09-28 11:53:15 -07:00
db_bench_tool_test.cc add whole key bloom filter support in memtables (#4985) 2019-02-19 12:15:39 -08:00
db_bench_tool.cc Feature for sampling and reporting compressibility (#4842) 2019-03-18 12:15:34 -07:00
db_bench.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_crashtest.py exercise WAL recycling in crash test (#5070) 2019-03-15 12:03:26 -07:00
db_repl_stress.cc Update all unique/shared_ptr instances to be qualified with namespace std (#4638) 2018-11-09 11:19:58 -08:00
db_sanity_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_stress.cc exercise WAL recycling in crash test (#5070) 2019-03-15 12:03:26 -07:00
dbench_monitor Fix /bin/bash shebangs 2017-08-03 15:56:46 -07:00
Dockerfile adding docker build script and dockerfile 2015-05-22 16:03:39 -07:00
generate_random_db.sh Fix /bin/bash shebangs 2017-08-03 15:56:46 -07:00
ingest_external_sst.sh Add compatibility test of SST ingestion (#4310) 2018-08-24 14:27:43 -07:00
ldb_cmd_impl.h Add SST ingestion to ldb (#4205) 2018-08-09 14:29:11 -07:00
ldb_cmd_test.cc tools: use provided options instead of the default (#4839) 2019-01-03 11:23:49 -08:00
ldb_cmd.cc ldb: set total_order_seek for scans (#5066) 2019-03-12 13:10:39 -07:00
ldb_test.py Add SST ingestion to ldb (#4205) 2018-08-09 14:29:11 -07:00
ldb_tool.cc Add SST ingestion to ldb (#4205) 2018-08-09 14:29:11 -07:00
ldb.cc comment unused parameters to turn on -Wunused-parameter flag 2018-04-12 17:59:16 -07:00
pflag Fix /bin/bash shebangs 2017-08-03 15:56:46 -07:00
reduce_levels_test.cc Per-thread unique test db names (#4135) 2018-07-13 17:27:39 -07:00
regression_test.sh Suppress lint in old files 2018-01-29 12:56:42 -08:00
report_lite_binary_size.sh Legocastle job to report lite build binary size to scuba 2018-02-15 17:27:24 -08:00
rocksdb_dump_test.sh Suppress lint in old files 2018-01-29 12:56:42 -08:00
run_flash_bench.sh Fix /bin/bash shebangs 2017-08-03 15:56:46 -07:00
run_leveldb.sh Fix /bin/bash shebangs 2017-08-03 15:56:46 -07:00
sample-dump.dmp First version of rocksdb_dump and rocksdb_undump. 2015-06-19 16:24:36 -07:00
sst_dump_test.cc Feature for sampling and reporting compressibility (#4842) 2019-03-18 12:15:34 -07:00
sst_dump_tool_imp.h tools: use provided options instead of the default (#4839) 2019-01-03 11:23:49 -08:00
sst_dump_tool.cc Feature for sampling and reporting compressibility (#4842) 2019-03-18 12:15:34 -07:00
sst_dump.cc comment unused parameters to turn on -Wunused-parameter flag 2018-04-12 17:59:16 -07:00
trace_analyzer_test.cc Apply modernize-use-override (2nd iteration) 2019-02-14 14:41:36 -08:00
trace_analyzer_tool.cc Add the -try_process_corrupted_trace option to trace_analyzer (#5067) 2019-03-14 20:03:01 -07:00
trace_analyzer_tool.h Add unique key number changing statistics to Trace_analyzer (#4646) 2018-11-12 08:26:50 -08:00
trace_analyzer.cc RocksDB Trace Analyzer (#4091) 2018-08-13 11:44:02 -07:00
verify_random_db.sh tools/check_format_compatible.sh to cover forward option reading too (#3994) 2018-06-15 11:12:29 -07:00
write_external_sst.sh correct mistyped msg. (#4341) 2018-09-13 14:57:38 -07:00
write_stress_runner.py Suppress lint in old files 2018-01-29 12:56:42 -08:00
write_stress.cc Compilation fixes for powerpc build, -Wparentheses-equality error and missing header guards 2018-02-09 14:12:43 -08:00