andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
6a40ee5eb1
rocksdb/tools
History
Maysam Yabandeh 6a40ee5eb1 Refresh snapshot list during long compactions (2nd attempt) (#5278) 
Summary:
Part of compaction cpu goes to processing snapshot list, the larger the list the bigger the overhead. Although the lifetime of most of the snapshots is much shorter than the lifetime of compactions, the compaction conservatively operates on the list of snapshots that it initially obtained. This patch allows the snapshot list to be updated via a callback if the compaction is taking long. This should let the compaction to continue more efficiently with much smaller snapshot list.
For simplicity, to avoid the feature is disabled in two cases: i) When more than one sub-compaction are sharing the same snapshot list, ii) when Range Delete is used in which the range delete aggregator has its own copy of snapshot list.
This fixes the reverted https://github.com/facebook/rocksdb/pull/5099 issue with range deletes.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/5278

Differential Revision: D15203291

Pulled By: maysamyabandeh

fbshipit-source-id: fa645611e606aa222c7ce53176dc5bb6f259c258
2019-05-03 17:30:22 -07:00
..
advisor Rules Advisor: some fixes to support fetching stats from ODS (#4223) 2018-08-02 15:42:42 -07:00
dump fix gflags namespace 2017-12-01 10:42:05 -08:00
rdb Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
analyze_txn_stress_test.sh Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
auto_sanity_test.sh Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
benchmark_leveldb.sh Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
benchmark.sh Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
blob_dump.cc comment unused parameters to turn on -Wunused-parameter flag 2018-04-12 17:59:16 -07:00
check_format_compatible.sh Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
CMakeLists.txt cmake support for linux and osx (#1358) 2016-09-28 11:53:15 -07:00
db_bench_tool_test.cc add whole key bloom filter support in memtables (#4985) 2019-02-19 12:15:39 -08:00
db_bench_tool.cc Improve explicit user readahead performance (#5246) 2019-04-26 21:24:10 -07:00
db_bench.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_crashtest.py Refresh snapshot list during long compactions (2nd attempt) (#5278) 2019-05-03 17:30:22 -07:00
db_repl_stress.cc Update all unique/shared_ptr instances to be qualified with namespace std (#4638) 2018-11-09 11:19:58 -08:00
db_sanity_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_stress.cc multiget: fix memory issues due to vector auto resizing (#5279) 2019-05-03 15:58:43 -07:00
dbench_monitor Fix /bin/bash shebangs 2017-08-03 15:56:46 -07:00
Dockerfile adding docker build script and dockerfile 2015-05-22 16:03:39 -07:00
generate_random_db.sh Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
ingest_external_sst.sh Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
ldb_cmd_impl.h Add SST ingestion to ldb (#4205) 2018-08-09 14:29:11 -07:00
ldb_cmd_test.cc ldb command parsing: allow option values to contain equals signs (#5088) 2019-03-25 13:23:11 -07:00
ldb_cmd.cc Avoid double-compacting data in bottom level in manual compactions (#5138) 2019-04-16 23:32:20 -07:00
ldb_test.py Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
ldb_tool.cc Add SST ingestion to ldb (#4205) 2018-08-09 14:29:11 -07:00
ldb.cc comment unused parameters to turn on -Wunused-parameter flag 2018-04-12 17:59:16 -07:00
pflag Fix /bin/bash shebangs 2017-08-03 15:56:46 -07:00
reduce_levels_test.cc Per-thread unique test db names (#4135) 2018-07-13 17:27:39 -07:00
regression_test.sh Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
report_lite_binary_size.sh Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
rocksdb_dump_test.sh Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
run_flash_bench.sh Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
run_leveldb.sh Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
sample-dump.dmp First version of rocksdb_dump and rocksdb_undump. 2015-06-19 16:24:36 -07:00
sst_dump_test.cc Feature for sampling and reporting compressibility (#4842) 2019-03-18 12:15:34 -07:00
sst_dump_tool_imp.h tools: use provided options instead of the default (#4839) 2019-01-03 11:23:49 -08:00
sst_dump_tool.cc Feature for sampling and reporting compressibility (#4842) 2019-03-18 12:15:34 -07:00
sst_dump.cc comment unused parameters to turn on -Wunused-parameter flag 2018-04-12 17:59:16 -07:00
trace_analyzer_test.cc Apply modernize-use-override (2nd iteration) 2019-02-14 14:41:36 -08:00
trace_analyzer_tool.cc Add the -try_process_corrupted_trace option to trace_analyzer (#5067) 2019-03-14 20:03:01 -07:00
trace_analyzer_tool.h Add unique key number changing statistics to Trace_analyzer (#4646) 2018-11-12 08:26:50 -08:00
trace_analyzer.cc RocksDB Trace Analyzer (#4091) 2018-08-13 11:44:02 -07:00
verify_random_db.sh Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
write_external_sst.sh Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
write_stress_runner.py Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
write_stress.cc Compilation fixes for powerpc build, -Wparentheses-equality error and missing header guards 2018-02-09 14:12:43 -08:00