rocksdb/db/compaction
Levi Tamasi b0e7834100 Integrate blob file writing with the flush logic (#7345)
Summary:
The patch adds support for writing blob files during flush by integrating
`BlobFileBuilder` with the flush logic, most importantly, `BuildTable` and
`CompactionIterator`. If `enable_blob_files` is set, large values are extracted
to blob files and replaced with references. The resulting blob files are then
logged to the MANIFEST as part of the flush job's `VersionEdit` and
added to the `Version`, similarly to table files. Errors related to writing
blob files fail the flush, and any blob files written by such jobs are immediately
deleted (again, similarly to how SST files are handled). In addition, the patch
extends the logging and statistics around flushes to account for the presence
of blob files (e.g. `InternalStats::CompactionStats::bytes_written`, which is
used for calculating write amplification, now considers the blob files as well).

Pull Request resolved: https://github.com/facebook/rocksdb/pull/7345

Test Plan: Tested using `make check` and `db_bench`.

Reviewed By: riversand963

Differential Revision: D23506369

Pulled By: ltamasi

fbshipit-source-id: 646885f22dfbe063f650d38a1fedc132f499a159
2020-09-14 21:11:43 -07:00
..
compaction_iteration_stats.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
compaction_iterator_test.cc Integrate blob file writing with the flush logic (#7345) 2020-09-14 21:11:43 -07:00
compaction_iterator.cc Integrate blob file writing with the flush logic (#7345) 2020-09-14 21:11:43 -07:00
compaction_iterator.h Integrate blob file writing with the flush logic (#7345) 2020-09-14 21:11:43 -07:00
compaction_job_stats_test.cc Replace reinterpret_cast with static_cast_with_check (#7067) 2020-07-02 19:25:41 -07:00
compaction_job_test.cc Store FileSystemPtr object that contains FileSystem ptr (#7180) 2020-08-12 17:31:23 -07:00
compaction_job.cc Integrate blob file writing with the flush logic (#7345) 2020-09-14 21:11:43 -07:00
compaction_job.h Store FSWritableFilePtr object in WritableFileWriter (#7193) 2020-09-08 10:56:08 -07:00
compaction_picker_fifo.cc Make max_subcompactions dynamically changeable (#7159) 2020-07-22 18:32:52 -07:00
compaction_picker_fifo.h Make max_subcompactions dynamically changeable (#7159) 2020-07-22 18:32:52 -07:00
compaction_picker_level.cc Make max_subcompactions dynamically changeable (#7159) 2020-07-22 18:32:52 -07:00
compaction_picker_level.h Make max_subcompactions dynamically changeable (#7159) 2020-07-22 18:32:52 -07:00
compaction_picker_test.cc Add missing ComputeCompactionScore() for a new universal manual compaction (#7281) 2020-08-19 17:42:08 -07:00
compaction_picker_universal.cc Fix wrong level args (#7346) 2020-09-03 23:17:37 -07:00
compaction_picker_universal.h Make max_subcompactions dynamically changeable (#7159) 2020-07-22 18:32:52 -07:00
compaction_picker.cc Add missing ComputeCompactionScore() for a new universal manual compaction (#7281) 2020-08-19 17:42:08 -07:00
compaction_picker.h Make max_subcompactions dynamically changeable (#7159) 2020-07-22 18:32:52 -07:00
compaction.cc SST Partitioner interface that allows to split SST files (#6957) 2020-07-24 13:44:49 -07:00
compaction.h SST Partitioner interface that allows to split SST files (#6957) 2020-07-24 13:44:49 -07:00
sst_partitioner.cc SST Partitioner interface that allows to split SST files (#6957) 2020-07-24 13:44:49 -07:00