andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
3ae0047278
rocksdb/include/rocksdb
History
Andrew Kryczka 3ae0047278 skip CompactRange flush based on memtable contents 
Summary:
CompactRange has a call to Flush because we guarantee that, at the time it's called, all existing keys in the range will be pushed through the user's compaction filter. However, previously the flush was done blindly, so it'd happen even if the memtable does not contain keys in the range specified by the user. This caused unnecessarily many L0 files to be created, leading to write stalls in some cases. This PR checks the memtable's contents, and decides to flush only if it overlaps with `CompactRange`'s range.

- Move the memtable overlap check logic from `ExternalSstFileIngestionJob` to `ColumnFamilyData::RangesOverlapWithMemtables`
- Reuse the above logic in `CompactRange` and skip flushing if no overlap
Closes https://github.com/facebook/rocksdb/pull/3520

Differential Revision: D7018897

Pulled By: ajkr

fbshipit-source-id: a3c6b1cfae56687b49dd89ccac7c948e53545934
2018-02-27 17:12:44 -08:00
..
utilities WritePrepared Txn: optimize SubBatchCnt 2018-02-22 18:12:26 -08:00
advanced_options.h pass key/value samples through zstd compression dictionary generator 2017-11-02 22:56:36 -07:00
c.h add support for allow_ingest_behind in C API 2018-01-08 17:26:31 -08:00
cache.h Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
cleanable.h PinnableSlice move assignment 2017-10-12 18:28:24 -07:00
compaction_filter.h Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
compaction_job_stats.h Change RocksDB License 2017-07-15 16:11:23 -07:00
comparator.h Move ~Comparator define to comparator.h 2017-10-17 09:58:13 -07:00
convenience.h Delete files in multiple ranges at once 2018-01-30 13:56:39 -08:00
db_bench_tool.h Change RocksDB License 2017-07-15 16:11:23 -07:00
db_dump_tool.h Change RocksDB License 2017-07-15 16:11:23 -07:00
db.h skip CompactRange flush based on memtable contents 2018-02-27 17:12:44 -08:00
env_encryption.h Change RocksDB License 2017-07-15 16:11:23 -07:00
env.h Fix the Logger::Close() and DBImpl::Close() design pattern 2018-02-23 13:57:26 -08:00
experimental.h Change RocksDB License 2017-07-15 16:11:23 -07:00
filter_policy.h Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
flush_block_policy.h Change RocksDB License 2017-07-15 16:11:23 -07:00
iostats_context.h Change RocksDB License 2017-07-15 16:11:23 -07:00
iterator.h Add rocksdb.iterator.internal-key property 2018-02-20 19:12:09 -08:00
ldb_tool.h Change RocksDB License 2017-07-15 16:11:23 -07:00
listener.h Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
memtablerep.h Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
merge_operator.h Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
metadata.h Fix coverity issues in include/rocksdb 2017-11-02 17:56:48 -07:00
options.h Add delay before flush in CompactRange to avoid write stalling 2018-02-12 15:42:47 -08:00
perf_context.h Add lock wait time as a perf context counter 2017-11-06 10:57:19 -08:00
perf_level.h Change RocksDB License 2017-07-15 16:11:23 -07:00
persistent_cache.h Change RocksDB License 2017-07-15 16:11:23 -07:00
rate_limiter.h Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
slice_transform.h Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
slice.h Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
snapshot.h Change RocksDB License 2017-07-15 16:11:23 -07:00
sst_dump_tool.h Change RocksDB License 2017-07-15 16:11:23 -07:00
sst_file_manager.h Make trash-to-DB size ratio limit configurable 2017-11-17 11:58:17 -08:00
sst_file_writer.h Support skipping bloom filters for SstFileWriter 2018-01-22 14:42:18 -08:00
statistics.h Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
status.h Change RocksDB License 2017-07-15 16:11:23 -07:00
table_properties.h Added support for differential snapshots 2017-11-01 18:56:43 -07:00
table.h Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
thread_status.h fix ThreadStatus for bottom-pri compaction threads 2017-12-14 14:57:49 -08:00
threadpool.h Change RocksDB License 2017-07-15 16:11:23 -07:00
transaction_log.h Change RocksDB License 2017-07-15 16:11:23 -07:00
types.h Fix WriteBatch rep_ format for RangeDeletion records 2018-01-23 12:57:32 -08:00
universal_compaction.h Change RocksDB License 2017-07-15 16:11:23 -07:00
version.h Switch version to 5.10 2017-12-11 15:42:01 -08:00
wal_filter.h Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
write_batch_base.h Change RocksDB License 2017-07-15 16:11:23 -07:00
write_batch.h Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
write_buffer_manager.h Change RocksDB License 2017-07-15 16:11:23 -07:00