andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
930bfa5750
rocksdb/include/rocksdb
History
Maysam Yabandeh 6a40ee5eb1 Refresh snapshot list during long compactions (2nd attempt) (#5278) 
Summary:
Part of compaction cpu goes to processing snapshot list, the larger the list the bigger the overhead. Although the lifetime of most of the snapshots is much shorter than the lifetime of compactions, the compaction conservatively operates on the list of snapshots that it initially obtained. This patch allows the snapshot list to be updated via a callback if the compaction is taking long. This should let the compaction to continue more efficiently with much smaller snapshot list.
For simplicity, to avoid the feature is disabled in two cases: i) When more than one sub-compaction are sharing the same snapshot list, ii) when Range Delete is used in which the range delete aggregator has its own copy of snapshot list.
This fixes the reverted https://github.com/facebook/rocksdb/pull/5099 issue with range deletes.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/5278

Differential Revision: D15203291

Pulled By: maysamyabandeh

fbshipit-source-id: fa645611e606aa222c7ce53176dc5bb6f259c258
2019-05-03 17:30:22 -07:00
..
utilities Option string/map/file can set env from object registry (#5237) 2019-04-25 11:35:09 -07:00
advanced_options.h Use creation_time or mtime when file_creation_time=0 (#5184) 2019-04-18 22:39:34 -07:00
c.h Refresh snapshot list during long compactions (2nd attempt) (#5278) 2019-05-03 17:30:22 -07:00
cache.h Make adaptivity of LRU cache mutexes configurable (#5054) 2019-03-20 12:33:44 -07:00
cleanable.h Support pragma once in all header files and cleanup some warnings (#4339) 2018-09-05 18:13:31 -07:00
compaction_filter.h Run automatic formatter against public header files (#5115) 2019-03-27 13:24:25 -07:00
compaction_job_stats.h Add a new CPU time counter to compaction report (#4889) 2019-01-29 17:24:00 -08:00
comparator.h Run automatic formatter against public header files (#5115) 2019-03-27 13:24:25 -07:00
concurrent_task_limiter.h Run automatic formatter against public header files (#5115) 2019-03-27 13:24:25 -07:00
convenience.h Run automatic formatter against public header files (#5115) 2019-03-27 13:24:25 -07:00
db_bench_tool.h Change RocksDB License 2017-07-15 16:11:23 -07:00
db_dump_tool.h Fix some typos in comments and docs. 2018-03-08 10:27:25 -08:00
db.h DB::Close() to fail when there are unreleased snapshots (#5272) 2019-05-01 10:17:30 -07:00
env_encryption.h Run automatic formatter against public header files (#5115) 2019-03-27 13:24:25 -07:00
env.h Optionally wait on bytes_per_sync to smooth I/O (#5183) 2019-04-22 11:51:39 -07:00
experimental.h Change RocksDB License 2017-07-15 16:11:23 -07:00
filter_policy.h Remove a couple of non-public includes from public header file (#5219) 2019-04-19 11:10:33 -07:00
flush_block_policy.h Run automatic formatter against public header files (#5115) 2019-03-27 13:24:25 -07:00
iostats_context.h Add a new CPU time counter to compaction report (#4889) 2019-01-29 17:24:00 -08:00
iterator.h Revert "BaseDeltaIterator: always check valid() before accessing key(… (#4744) 2018-12-03 23:38:27 -08:00
ldb_tool.h Run automatic formatter against public header files (#5115) 2019-03-27 13:24:25 -07:00
listener.h Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
memory_allocator.h JemallocNodumpAllocator: option to limit tcache memory usage (#4736) 2018-11-29 17:33:40 -08:00
memtablerep.h Run automatic formatter against public header files (#5115) 2019-03-27 13:24:25 -07:00
merge_operator.h Run automatic formatter against public header files (#5115) 2019-03-27 13:24:25 -07:00
metadata.h Run automatic formatter against public header files (#5115) 2019-03-27 13:24:25 -07:00
options.h Refresh snapshot list during long compactions (2nd attempt) (#5278) 2019-05-03 17:30:22 -07:00
perf_context.h Run automatic formatter against public header files (#5115) 2019-03-27 13:24:25 -07:00
perf_level.h Introduce a CPU time counter in perf_context (#4741) 2018-12-20 12:03:44 -08:00
persistent_cache.h Change RocksDB License 2017-07-15 16:11:23 -07:00
rate_limiter.h Fix typos in comments (#4819) 2018-12-26 09:43:56 -08:00
slice_transform.h Run automatic formatter against public header files (#5115) 2019-03-27 13:24:25 -07:00
slice.h Run automatic formatter against public header files (#5115) 2019-03-27 13:24:25 -07:00
snapshot.h Change RocksDB License 2017-07-15 16:11:23 -07:00
sst_dump_tool.h tools: use provided options instead of the default (#4839) 2019-01-03 11:23:49 -08:00
sst_file_manager.h Smooth the deletion of WAL files (#5116) 2019-03-28 15:17:13 -07:00
sst_file_reader.h Get CompactionJobInfo from CompactFiles 2018-12-13 14:21:24 -08:00
sst_file_writer.h Run automatic formatter against public header files (#5115) 2019-03-27 13:24:25 -07:00
statistics.h Introduce a new MultiGet batching implementation (#5011) 2019-04-11 14:28:26 -07:00
stats_history.h add GetStatsHistory to retrieve stats snapshots (#4748) 2019-02-20 15:52:54 -08:00
status.h Run automatic formatter against public header files (#5115) 2019-03-27 13:24:25 -07:00
table_properties.h Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
table.h Add BlockBasedTableOptions::index_shortening (#5174) 2019-04-22 08:20:35 -07:00
thread_status.h Run automatic formatter against public header files (#5115) 2019-03-27 13:24:25 -07:00
threadpool.h Run automatic formatter against public header files (#5115) 2019-03-27 13:24:25 -07:00
trace_reader_writer.h Add the max trace file size limitation option to Tracing (#4610) 2018-11-27 14:27:05 -08:00
transaction_log.h Run automatic formatter against public header files (#5115) 2019-03-27 13:24:25 -07:00
types.h WriteUnPrepared: less virtual in iterator callback (#5049) 2019-04-02 14:47:16 -07:00
universal_compaction.h Run automatic formatter against public header files (#5115) 2019-03-27 13:24:25 -07:00
version.h Update history and version for future 6.2.0 (#5270) 2019-04-30 15:09:36 -07:00
wal_filter.h Run automatic formatter against public header files (#5115) 2019-03-27 13:24:25 -07:00
write_batch_base.h Fix WriteBatchBase::DeleteRange API comment (#4935) 2019-01-31 14:43:40 -08:00
write_batch.h refactor SavePoints (#5192) 2019-04-19 20:33:04 -07:00
write_buffer_manager.h WriteBufferManger doens't cost to cache if no limit is set (#4695) 2018-11-18 16:55:43 -08:00