rocksdb/db/compaction
Levi Tamasi a7a04b6898 Integrate BlobFileBuilder into the compaction process (#7573)
Summary:
Similarly to how https://github.com/facebook/rocksdb/issues/7345
integrated blob file writing into the flush process,
the patch adds support for writing blob files to the compaction logic.
Namely, if `enable_blob_files` is set, large values encountered during
compaction are extracted to blob files and replaced with blob indexes.
The resulting blob files are then logged to the MANIFEST as part of the
compaction job's `VersionEdit` and added to the `Version` alongside any
table files written by the compaction. Any errors during blob file building fail
the compaction job.

There will be a separate follow-up patch to perform blob garbage collection
during compactions.

In addition, the patch continues to chip away at the mess around computing
various compaction related statistics by eliminating some code duplication
and by making the `num_output_files` and `bytes_written` stats more consistent
for flushes, compactions, and recovery.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/7573

Test Plan: `make check`

Reviewed By: riversand963

Differential Revision: D24404696

Pulled By: ltamasi

fbshipit-source-id: 21216af3a172ad3ce8f85d11cd30923784ae426c
2020-10-26 13:51:55 -07:00
..
compaction_iteration_stats.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
compaction_iterator_test.cc Allow compaction iterator to perform garbage collection (#7556) 2020-10-23 22:59:46 -07:00
compaction_iterator.cc Allow compaction iterator to perform garbage collection (#7556) 2020-10-23 22:59:46 -07:00
compaction_iterator.h Allow compaction iterator to perform garbage collection (#7556) 2020-10-23 22:59:46 -07:00
compaction_job_stats_test.cc Add is_full_compaction to CompactionJobStats, cleanup (#7451) 2020-10-01 12:52:58 -07:00
compaction_job_test.cc Integrate BlobFileBuilder into the compaction process (#7573) 2020-10-26 13:51:55 -07:00
compaction_job.cc Integrate BlobFileBuilder into the compaction process (#7573) 2020-10-26 13:51:55 -07:00
compaction_job.h Integrate BlobFileBuilder into the compaction process (#7573) 2020-10-26 13:51:55 -07:00
compaction_picker_fifo.cc Make max_subcompactions dynamically changeable (#7159) 2020-07-22 18:32:52 -07:00
compaction_picker_fifo.h Make max_subcompactions dynamically changeable (#7159) 2020-07-22 18:32:52 -07:00
compaction_picker_level.cc Make max_subcompactions dynamically changeable (#7159) 2020-07-22 18:32:52 -07:00
compaction_picker_level.h Make max_subcompactions dynamically changeable (#7159) 2020-07-22 18:32:52 -07:00
compaction_picker_test.cc Add missing ComputeCompactionScore() for a new universal manual compaction (#7281) 2020-08-19 17:42:08 -07:00
compaction_picker_universal.cc Make sure assert(false) handles failures too (#7483) 2020-10-06 13:52:42 -07:00
compaction_picker_universal.h Make max_subcompactions dynamically changeable (#7159) 2020-07-22 18:32:52 -07:00
compaction_picker.cc Add missing ComputeCompactionScore() for a new universal manual compaction (#7281) 2020-08-19 17:42:08 -07:00
compaction_picker.h Make max_subcompactions dynamically changeable (#7159) 2020-07-22 18:32:52 -07:00
compaction.cc Allow compaction iterator to perform garbage collection (#7556) 2020-10-23 22:59:46 -07:00
compaction.h SST Partitioner interface that allows to split SST files (#6957) 2020-07-24 13:44:49 -07:00
sst_partitioner.cc SST Partitioner interface that allows to split SST files (#6957) 2020-07-24 13:44:49 -07:00