andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
0302da47a7
rocksdb/db
History
Andrew Kryczka a22b9cc6fe overlapping endpoint fixes in level compaction picker 
Summary:
This diff addresses two problems. Both problems cause us to miss scheduling desirable compactions. One side effect is compaction picking can spam logs, as there's no delay after failed attempts to pick compactions.

1. If a compaction pulled in a locked input-level file due to user-key overlap, we would not consider picking another file from the same input level.
2. If a compaction pulled in a locked output-level file due to user-key overlap, we would not consider picking any other compaction on any level.

The code changes are dependent, which is why I solved both problems in a single diff.

- Moved input-level `ExpandInputsToCleanCut` into the loop inside `PickFileToCompact`. This gives two benefits: (1) if it fails, we will try the next-largest file on the same input level; (2) we get the fully-expanded input-level key-range with which we can check for pending compactions in output level.
- Added another call to `ExpandInputsToCleanCut` inside `PickFileToCompact`'s to check for compaction conflicts in output level.
- Deleted call to `IsRangeInCompaction` in `PickFileToCompact`, as `ExpandInputsToCleanCut` also correctly handles the case where original output-level files (i.e., ones not pulled in due to user-key overlap) are pending compaction.
Closes https://github.com/facebook/rocksdb/pull/2615

Differential Revision: D5454643

Pulled By: ajkr

fbshipit-source-id: ea3fb5477d83e97148951af3fd4558d2039e9872
2017-07-19 20:42:00 -07:00
..
builder.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
builder.h Change RocksDB License 2017-07-15 16:11:23 -07:00
c_test.c support merge and delete in file ingestion 2017-05-26 12:11:21 -07:00
c.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
column_family_test.cc Fix column_family_test with LITE build 2017-07-17 15:08:24 -07:00
column_family.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
column_family.h Change RocksDB License 2017-07-15 16:11:23 -07:00
compact_files_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
compacted_db_impl.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
compacted_db_impl.h Change RocksDB License 2017-07-15 16:11:23 -07:00
compaction_iteration_stats.h Change RocksDB License 2017-07-15 16:11:23 -07:00
compaction_iterator_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
compaction_iterator.cc Remove some left-over BSD headers 2017-07-18 11:56:57 -07:00
compaction_iterator.h Remove some left-over BSD headers 2017-07-18 11:56:57 -07:00
compaction_job_stats_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
compaction_job_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
compaction_job.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
compaction_job.h Change RocksDB License 2017-07-15 16:11:23 -07:00
compaction_picker_test.cc overlapping endpoint fixes in level compaction picker 2017-07-19 20:42:00 -07:00
compaction_picker_universal.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
compaction_picker_universal.h Change RocksDB License 2017-07-15 16:11:23 -07:00
compaction_picker.cc overlapping endpoint fixes in level compaction picker 2017-07-19 20:42:00 -07:00
compaction_picker.h Change RocksDB License 2017-07-15 16:11:23 -07:00
compaction.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
compaction.h Change RocksDB License 2017-07-15 16:11:23 -07:00
comparator_db_test.cc Remove some left-over BSD headers 2017-07-18 11:56:57 -07:00
convenience.cc Remove some left-over BSD headers 2017-07-18 11:56:57 -07:00
corruption_test.cc CodeMod: Prefer ADD_FAILURE() over EXPECT_TRUE(false), et cetera 2017-07-16 21:26:02 -07:00
cuckoo_table_db_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_basic_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_block_cache_test.cc CodeMod: Prefer ADD_FAILURE() over EXPECT_TRUE(false), et cetera 2017-07-16 21:26:02 -07:00
db_bloom_filter_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_compaction_filter_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_compaction_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_dynamic_level_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_encryption_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_filesnapshot.cc Remove some left-over BSD headers 2017-07-18 11:56:57 -07:00
db_flush_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_impl_compaction_flush.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_impl_debug.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_impl_experimental.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_impl_files.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_impl_open.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_impl_readonly.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_impl_readonly.h Change RocksDB License 2017-07-15 16:11:23 -07:00
db_impl_write.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_impl.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_impl.h Change RocksDB License 2017-07-15 16:11:23 -07:00
db_info_dumper.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_info_dumper.h Change RocksDB License 2017-07-15 16:11:23 -07:00
db_inplace_update_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_io_failure_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_iter_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_iter.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_iter.h Change RocksDB License 2017-07-15 16:11:23 -07:00
db_iterator_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_log_iter_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_memtable_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_merge_operator_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_options_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_properties_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_range_del_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_sst_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_statistics_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_table_properties_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_tailing_iter_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_test2.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_test_util.cc enable PinnableSlice for RowCache 2017-07-17 15:08:30 -07:00
db_test_util.h enable PinnableSlice for RowCache 2017-07-17 15:08:30 -07:00
db_test.cc enable PinnableSlice for RowCache 2017-07-17 15:08:30 -07:00
db_universal_compaction_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_wal_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_write_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
dbformat_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
dbformat.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
dbformat.h Change RocksDB License 2017-07-15 16:11:23 -07:00
deletefile_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
event_helpers.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
event_helpers.h Change RocksDB License 2017-07-15 16:11:23 -07:00
experimental.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
external_sst_file_basic_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
external_sst_file_ingestion_job.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
external_sst_file_ingestion_job.h Change RocksDB License 2017-07-15 16:11:23 -07:00
external_sst_file_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
fault_injection_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
file_indexer_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
file_indexer.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
file_indexer.h Change RocksDB License 2017-07-15 16:11:23 -07:00
filename_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
flush_job_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
flush_job.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
flush_job.h Change RocksDB License 2017-07-15 16:11:23 -07:00
flush_scheduler.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
flush_scheduler.h Change RocksDB License 2017-07-15 16:11:23 -07:00
forward_iterator_bench.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
forward_iterator.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
forward_iterator.h Change RocksDB License 2017-07-15 16:11:23 -07:00
internal_stats.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
internal_stats.h Change RocksDB License 2017-07-15 16:11:23 -07:00
job_context.h Change RocksDB License 2017-07-15 16:11:23 -07:00
listener_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
log_format.h Change RocksDB License 2017-07-15 16:11:23 -07:00
log_reader.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
log_reader.h Change RocksDB License 2017-07-15 16:11:23 -07:00
log_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
log_writer.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
log_writer.h Change RocksDB License 2017-07-15 16:11:23 -07:00
malloc_stats.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
malloc_stats.h Change RocksDB License 2017-07-15 16:11:23 -07:00
managed_iterator.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
managed_iterator.h Change RocksDB License 2017-07-15 16:11:23 -07:00
manual_compaction_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
memtable_list_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
memtable_list.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
memtable_list.h Change RocksDB License 2017-07-15 16:11:23 -07:00
memtable.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
memtable.h Change RocksDB License 2017-07-15 16:11:23 -07:00
merge_context.h Change RocksDB License 2017-07-15 16:11:23 -07:00
merge_helper_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
merge_helper.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
merge_helper.h Change RocksDB License 2017-07-15 16:11:23 -07:00
merge_operator.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
merge_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
options_file_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
perf_context_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
pinned_iterators_manager.h Change RocksDB License 2017-07-15 16:11:23 -07:00
plain_table_db_test.cc Remove some left-over BSD headers 2017-07-18 11:56:57 -07:00
prefix_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
range_del_aggregator_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
range_del_aggregator.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
range_del_aggregator.h Change RocksDB License 2017-07-15 16:11:23 -07:00
repair_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
repair.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
snapshot_impl.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
snapshot_impl.h Change RocksDB License 2017-07-15 16:11:23 -07:00
table_cache.cc enable PinnableSlice for RowCache 2017-07-17 15:08:30 -07:00
table_cache.h Change RocksDB License 2017-07-15 16:11:23 -07:00
table_properties_collector_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
table_properties_collector.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
table_properties_collector.h Change RocksDB License 2017-07-15 16:11:23 -07:00
transaction_log_impl.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
transaction_log_impl.h Change RocksDB License 2017-07-15 16:11:23 -07:00
version_builder_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
version_builder.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
version_builder.h Change RocksDB License 2017-07-15 16:11:23 -07:00
version_edit_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
version_edit.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
version_edit.h Change RocksDB License 2017-07-15 16:11:23 -07:00
version_set_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
version_set.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
version_set.h Change RocksDB License 2017-07-15 16:11:23 -07:00
wal_manager_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
wal_manager.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
wal_manager.h Change RocksDB License 2017-07-15 16:11:23 -07:00
write_batch_base.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
write_batch_internal.h Change RocksDB License 2017-07-15 16:11:23 -07:00
write_batch_test.cc CodeMod: Prefer ADD_FAILURE() over EXPECT_TRUE(false), et cetera 2017-07-16 21:26:02 -07:00
write_batch.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
write_callback_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
write_callback.h Change RocksDB License 2017-07-15 16:11:23 -07:00
write_controller_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
write_controller.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
write_controller.h Change RocksDB License 2017-07-15 16:11:23 -07:00
write_thread.cc Moving static AdaptationContext to outside function 2017-07-18 16:58:22 -07:00
write_thread.h Change RocksDB License 2017-07-15 16:11:23 -07:00