rocksdb/db/blob
Levi Tamasi 065bea1587 Add a class for measuring the amount of garbage generated during compaction (#8426)
Summary:
This is part of an alternative approach to https://github.com/facebook/rocksdb/issues/8316.
Unlike that approach, this one relies on key-values getting processed one by one
during compaction, and does not involve persistence.

Specifically, the patch adds a class `BlobGarbageMeter` that can track the number
and total size of blobs in a (sub)compaction's input and output on a per-blob file
basis. This information can then be used to compute the amount of additional
garbage generated by the compaction for any given blob file by subtracting the
"outflow" from the "inflow."

Note: this patch only adds `BlobGarbageMeter` and associated unit tests. I plan to
hook up this class to the input and output of `CompactionIterator` in a subsequent PR.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/8426

Test Plan: `make check`

Reviewed By: jay-zhuang

Differential Revision: D29242250

Pulled By: ltamasi

fbshipit-source-id: 597e50ad556540e413a50e804ba15bc044d809bb
2021-06-21 22:25:30 -07:00
..
blob_constants.h Move BlobDB related files under db/ to db/blob/ (#6519) 2020-03-12 11:00:56 -07:00
blob_fetcher.cc Support for Merge in Integrated BlobDB with base values (#8292) 2021-06-10 12:58:37 -07:00
blob_fetcher.h Support for Merge in Integrated BlobDB with base values (#8292) 2021-06-10 12:58:37 -07:00
blob_file_addition_test.cc Move BlobDB related files under db/ to db/blob/ (#6519) 2020-03-12 11:00:56 -07:00
blob_file_addition.cc Move BlobDB related files under db/ to db/blob/ (#6519) 2020-03-12 11:00:56 -07:00
blob_file_addition.h Move BlobDB related files under db/ to db/blob/ (#6519) 2020-03-12 11:00:56 -07:00
blob_file_builder_test.cc Rename ImmutableOptions variables (#8409) 2021-06-16 16:51:38 -07:00
blob_file_builder.cc Rename ImmutableOptions variables (#8409) 2021-06-16 16:51:38 -07:00
blob_file_builder.h Rename ImmutableOptions variables (#8409) 2021-06-16 16:51:38 -07:00
blob_file_cache_test.cc Rename ImmutableOptions variables (#8409) 2021-06-16 16:51:38 -07:00
blob_file_cache.cc Rename ImmutableOptions variables (#8409) 2021-06-16 16:51:38 -07:00
blob_file_cache.h Rename ImmutableOptions variables (#8409) 2021-06-16 16:51:38 -07:00
blob_file_completion_callback.h Fix for LITE mode failure on MacOS (#8189) 2021-04-15 09:45:02 -07:00
blob_file_garbage_test.cc Move BlobDB related files under db/ to db/blob/ (#6519) 2020-03-12 11:00:56 -07:00
blob_file_garbage.cc Move BlobDB related files under db/ to db/blob/ (#6519) 2020-03-12 11:00:56 -07:00
blob_file_garbage.h Move BlobDB related files under db/ to db/blob/ (#6519) 2020-03-12 11:00:56 -07:00
blob_file_meta.cc Maintain the set of linked SSTs in BlobFileMetaData (#6945) 2020-06-12 09:54:39 -07:00
blob_file_meta.h Maintain the set of linked SSTs in BlobFileMetaData (#6945) 2020-06-12 09:54:39 -07:00
blob_file_reader_test.cc Rename ImmutableOptions variables (#8409) 2021-06-16 16:51:38 -07:00
blob_file_reader.cc Rename ImmutableOptions variables (#8409) 2021-06-16 16:51:38 -07:00
blob_file_reader.h Make ImmutableOptions struct that inherits from ImmutableCFOptions and ImmutableDBOptions (#8262) 2021-05-05 14:00:17 -07:00
blob_garbage_meter_test.cc Add a class for measuring the amount of garbage generated during compaction (#8426) 2021-06-21 22:25:30 -07:00
blob_garbage_meter.cc Add a class for measuring the amount of garbage generated during compaction (#8426) 2021-06-21 22:25:30 -07:00
blob_garbage_meter.h Add a class for measuring the amount of garbage generated during compaction (#8426) 2021-06-21 22:25:30 -07:00
blob_index.h Introduce a blob file reader class (#7461) 2020-10-07 15:44:53 -07:00
blob_log_format.cc Introduce BlobFileCache and add support for blob files to Get() (#7540) 2020-10-15 13:04:47 -07:00
blob_log_format.h Add a class for measuring the amount of garbage generated during compaction (#8426) 2021-06-21 22:25:30 -07:00
blob_log_sequential_reader.cc Use SystemClock* instead of std::shared_ptr<SystemClock> in lower level routines (#8033) 2021-03-15 04:34:11 -07:00
blob_log_sequential_reader.h Use SystemClock* instead of std::shared_ptr<SystemClock> in lower level routines (#8033) 2021-03-15 04:34:11 -07:00
blob_log_writer.cc Use SystemClock* instead of std::shared_ptr<SystemClock> in lower level routines (#8033) 2021-03-15 04:34:11 -07:00
blob_log_writer.h Use SystemClock* instead of std::shared_ptr<SystemClock> in lower level routines (#8033) 2021-03-15 04:34:11 -07:00
db_blob_basic_test.cc Support for Merge in Integrated BlobDB with base values (#8292) 2021-06-10 12:58:37 -07:00
db_blob_compaction_test.cc Use SST file manager to track blob files as well (#8037) 2021-03-17 20:44:49 -07:00
db_blob_corruption_test.cc Extend VerifyFileChecksums API for blob files (#7979) 2021-02-22 22:09:22 -08:00
db_blob_index_test.cc Support for Merge in Integrated BlobDB with base values (#8292) 2021-06-10 12:58:37 -07:00