rocksdb/db/compaction
Andrew Kryczka a1aa3f8385 Disable manual compaction during ReFitLevel() (#7250)
Summary:
Manual compaction with `CompactRangeOptions::change_levels` set could
refit to a level targeted by another manual compaction. If
force_consistency_checks were disabled, it could be possible for
overlapping files to be written at that target level.

This PR prevents the possibility by calling `DisableManualCompaction()`
prior to `ReFitLevel()`. It also improves the manual compaction disabling
mechanism to wait for pending manual compactions to complete before
returning, and support disabling from multiple threads.

Fixes https://github.com/facebook/rocksdb/issues/6432.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/7250

Test Plan:
crash test command that repro'd the bug reliably:

```
$ TEST_TMPDIR=/dev/shm python tools/db_crashtest.py blackbox --simple -target_file_size_base=524288 -write_buffer_size=1048576 -clear_column_family_one_in=0 -reopen=0 -max_key=10000000 -column_families=1 -max_background_compactions=8 -compact_range_one_in=100000 -compression_type=none -compaction_style=1 -num_levels=5 -universal_min_merge_width=4 -universal_max_merge_width=8 -level0_file_num_compaction_trigger=12 -rate_limiter_bytes_per_sec=1048576000 -universal_max_size_amplification_percent=100 --duration=3600 --interval=60 --use_direct_io_for_flush_and_compaction=0 --use_direct_reads=0 --enable_compaction_filter=0
```

Reviewed By: ltamasi

Differential Revision: D23090800

Pulled By: ajkr

fbshipit-source-id: afcbcd51b42ce76789fdb907d8b9ada790709c13
2020-08-14 11:29:52 -07:00
..
compaction_iteration_stats.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
compaction_iterator_test.cc Revert "Update googletest from 1.8.1 to 1.10.0 (#6808)" (#6923) 2020-06-03 15:55:03 -07:00
compaction_iterator.cc Disable manual compaction during ReFitLevel() (#7250) 2020-08-14 11:29:52 -07:00
compaction_iterator.h Disable manual compaction during ReFitLevel() (#7250) 2020-08-14 11:29:52 -07:00
compaction_job_stats_test.cc Replace reinterpret_cast with static_cast_with_check (#7067) 2020-07-02 19:25:41 -07:00
compaction_job_test.cc Store FileSystemPtr object that contains FileSystem ptr (#7180) 2020-08-12 17:31:23 -07:00
compaction_job.cc Disable manual compaction during ReFitLevel() (#7250) 2020-08-14 11:29:52 -07:00
compaction_job.h Disable manual compaction during ReFitLevel() (#7250) 2020-08-14 11:29:52 -07:00
compaction_picker_fifo.cc Make max_subcompactions dynamically changeable (#7159) 2020-07-22 18:32:52 -07:00
compaction_picker_fifo.h Make max_subcompactions dynamically changeable (#7159) 2020-07-22 18:32:52 -07:00
compaction_picker_level.cc Make max_subcompactions dynamically changeable (#7159) 2020-07-22 18:32:52 -07:00
compaction_picker_level.h Make max_subcompactions dynamically changeable (#7159) 2020-07-22 18:32:52 -07:00
compaction_picker_test.cc Fix delete triggered compaction for single level universal (#7224) 2020-08-10 12:19:17 -07:00
compaction_picker_universal.cc Fix delete triggered compaction for single level universal (#7224) 2020-08-10 12:19:17 -07:00
compaction_picker_universal.h Make max_subcompactions dynamically changeable (#7159) 2020-07-22 18:32:52 -07:00
compaction_picker.cc Make max_subcompactions dynamically changeable (#7159) 2020-07-22 18:32:52 -07:00
compaction_picker.h Make max_subcompactions dynamically changeable (#7159) 2020-07-22 18:32:52 -07:00
compaction.cc SST Partitioner interface that allows to split SST files (#6957) 2020-07-24 13:44:49 -07:00
compaction.h SST Partitioner interface that allows to split SST files (#6957) 2020-07-24 13:44:49 -07:00
sst_partitioner.cc SST Partitioner interface that allows to split SST files (#6957) 2020-07-24 13:44:49 -07:00