rocksdb/db/compaction
sdong c3ff14e2c1 Hint temperature of bottommost level files to FileSystem (#8222)
Summary:
As the first part of the effort of having placing different files on different storage types, this change introduces several things:
(1) An experimental interface in FileSystem that specify temperature to a new file created.
(2) A test FileSystemWrapper,  SimulatedHybridFileSystem, that simulates HDD for a file of "warm" temperature.
(3) A simple experimental feature ColumnFamilyOptions.bottommost_temperature. RocksDB would pass this value to FileSystem when creating any bottommost file.
(4) A db_bench parameter that applies the (2) and (3) to db_bench.

The motivation of the change is to introduce minimal changes that allow us to evolve tiered storage development.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/8222

Test Plan:
./db_bench --benchmarks=fillrandom --write_buffer_size=2000000 -max_bytes_for_level_base=20000000  -level_compaction_dynamic_level_bytes --reads=100 -compaction_readahead_size=20000000 --reads=100000 -num=10000000

followed by

./db_bench --benchmarks=readrandom,stats --write_buffer_size=2000000 -max_bytes_for_level_base=20000000 -simulate_hybrid_fs_file=/tmp/warm_file_list -level_compaction_dynamic_level_bytes -compaction_readahead_size=20000000 --reads=500 --threads=16 -use_existing_db --num=10000000

and see results as expected.

Reviewed By: ajkr

Differential Revision: D28003028

fbshipit-source-id: 4724896d5205730227ba2f17c3fecb11261744ce
2021-05-03 13:34:04 -07:00
..
compaction_iteration_stats.h Update compaction statistics to include the amount of data read from blob files (#8022) 2021-03-04 00:43:48 -08:00
compaction_iterator_test.cc Fix some typos in comments (#8066) 2021-03-25 21:18:08 -07:00
compaction_iterator.cc Fix some typos in comments (#8066) 2021-03-25 21:18:08 -07:00
compaction_iterator.h Use SystemClock* instead of std::shared_ptr<SystemClock> in lower level routines (#8033) 2021-03-15 04:34:11 -07:00
compaction_job_stats_test.cc Add more tests to ASSERT_STATUS_CHECKED (3), API change (#7715) 2021-01-06 14:15:02 -08:00
compaction_job_test.cc Remove Legacy and Custom FileWrapper classes from header files (#7851) 2021-01-28 22:10:32 -08:00
compaction_job.cc Hint temperature of bottommost level files to FileSystem (#8222) 2021-05-03 13:34:04 -07:00
compaction_job.h Add internal compaction API for Secondary instance (#8171) 2021-04-22 13:02:28 -07:00
compaction_picker_fifo.cc Rename variables in ImmutableCFOptions to avoid conflicts with ImmutableDBOptions (#8227) 2021-04-26 12:43:45 -07:00
compaction_picker_fifo.h Make max_subcompactions dynamically changeable (#7159) 2020-07-22 18:32:52 -07:00
compaction_picker_level.cc Make max_subcompactions dynamically changeable (#7159) 2020-07-22 18:32:52 -07:00
compaction_picker_level.h Make max_subcompactions dynamically changeable (#7159) 2020-07-22 18:32:52 -07:00
compaction_picker_test.cc Fix some typos in comments (#8066) 2021-03-25 21:18:08 -07:00
compaction_picker_universal.cc Rename variables in ImmutableCFOptions to avoid conflicts with ImmutableDBOptions (#8227) 2021-04-26 12:43:45 -07:00
compaction_picker_universal.h Make max_subcompactions dynamically changeable (#7159) 2020-07-22 18:32:52 -07:00
compaction_picker.cc Rename variables in ImmutableCFOptions to avoid conflicts with ImmutableDBOptions (#8227) 2021-04-26 12:43:45 -07:00
compaction_picker.h Make max_subcompactions dynamically changeable (#7159) 2020-07-22 18:32:52 -07:00
compaction.cc Fix some typos in comments (#8066) 2021-03-25 21:18:08 -07:00
compaction.h Refactor: use TableBuilderOptions to reduce parameter lists (#8240) 2021-04-29 07:00:50 -07:00
sst_partitioner.cc SST Partitioner interface that allows to split SST files (#6957) 2020-07-24 13:44:49 -07:00