rocksdb/db/compaction
Levi Tamasi 5f025ea832 BlobDB GC: add SST <-> oldest blob file referenced mapping (#5903)
Summary:
This is groundwork for adding garbage collection support to BlobDB. The
patch adds logic that keeps track of the oldest blob file referred to by
each SST file. The oldest blob file is identified during flush/
compaction (similarly to how the range of keys covered by the SST is
identified), and persisted in the manifest as a custom field of the new
file edit record. Blob indexes with TTL are ignored for the purposes of
identifying the oldest blob file (since such blob files are cleaned up by the
TTL logic in BlobDB).
Pull Request resolved: https://github.com/facebook/rocksdb/pull/5903

Test Plan:
Added new unit tests; also ran db_bench in BlobDB mode, inspected the
manifest using ldb, and confirmed (by scanning the SST files using
sst_dump) that the value of the oldest blob file number field matches
the contents of the file for each SST.

Differential Revision: D17859997

Pulled By: ltamasi

fbshipit-source-id: 21662c137c6259a6af70446faaf3a9912c550e90
2019-10-14 15:21:01 -07:00
..
compaction_iteration_stats.h Organizing rocksdb/db directory 2019-05-31 11:57:01 -07:00
compaction_iterator_test.cc Organizing rocksdb/db directory 2019-05-31 11:57:01 -07:00
compaction_iterator.cc Apply formatter on recent 45 commits. (#5827) 2019-09-19 12:34:17 -07:00
compaction_iterator.h Apply formatter on recent 45 commits. (#5827) 2019-09-19 12:34:17 -07:00
compaction_job_stats_test.cc Apply formatter to recent 200+ commits. (#5830) 2019-09-20 12:04:26 -07:00
compaction_job_test.cc BlobDB GC: add SST <-> oldest blob file referenced mapping (#5903) 2019-10-14 15:21:01 -07:00
compaction_job.cc BlobDB GC: add SST <-> oldest blob file referenced mapping (#5903) 2019-10-14 15:21:01 -07:00
compaction_job.h Apply formatter on recent 45 commits. (#5827) 2019-09-19 12:34:17 -07:00
compaction_picker_fifo.cc Fix assertion failure in FIFO compaction with TTL (#5754) 2019-08-30 12:42:01 -07:00
compaction_picker_fifo.h Organizing rocksdb/db directory 2019-05-31 11:57:01 -07:00
compaction_picker_level.cc simplify include directive involving inttypes (#5402) 2019-06-06 13:56:07 -07:00
compaction_picker_level.h Organizing rocksdb/db directory 2019-05-31 11:57:01 -07:00
compaction_picker_test.cc BlobDB GC: add SST <-> oldest blob file referenced mapping (#5903) 2019-10-14 15:21:01 -07:00
compaction_picker_universal.cc Refactor UniversalCompactionPicker code a little bit (#5639) 2019-09-16 10:51:11 -07:00
compaction_picker_universal.h Refactor UniversalCompactionPicker code a little bit (#5639) 2019-09-16 10:51:11 -07:00
compaction_picker.cc simplify include directive involving inttypes (#5402) 2019-06-06 13:56:07 -07:00
compaction_picker.h Organizing rocksdb/db directory 2019-05-31 11:57:01 -07:00
compaction.cc simplify include directive involving inttypes (#5402) 2019-06-06 13:56:07 -07:00
compaction.h Organizing rocksdb/db directory 2019-05-31 11:57:01 -07:00