rocksdb/db/compaction
Yanqin Jin 6595267980 Allow compaction iterator to perform garbage collection (#7556)
Summary:
Add a threshold timestamp, full_history_ts_low_ of type `std::string*` to
`CompactionIterator`, so that RocksDB can also perform garbage collection during
compaction.
* If full_history_ts_low_ is nullptr, then compaction iterator does not perform
  GC, preserving all timestamp history for all keys. Compaction iterator will
treat user key with different timestamps as different user keys.
* If full_history_ts_low_ is not nullptr, then compaction iterator performs
  GC. GC will look at keys older than `*full_history_ts_low_` and determine their
  eligibility based on factors including snapshots.

Current rules of GC:
 * If an internal key is in the same snapshot as a previous counterpart
    with the same user key, and this key is eligible for GC, and the key is
    not single-delete or merge operand, then this key can be dropped. Note
    that the previous internal key cannot be a merge operand either.
 * If a tombstone is the most recent one in the earliest snapshot and it
    is eligible for GC, and keyNotExistsBeyondLevel() is true, then this
    tombstone can be dropped.
 * If a tombstone is the most recent one in a snapshot and it is eligible
    for GC, and the compaction is at bottommost level, then all other older
    internal keys of the same user key must also be eligible for GC, thus
    can be dropped
* Single-delete, delete-range and merge are not currently supported.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/7556

Test Plan: make check

Reviewed By: ltamasi

Differential Revision: D24507728

Pulled By: riversand963

fbshipit-source-id: 3c09c7301f41eed76dfcf4d1527e68cf6e0a8bb3
2020-10-23 22:59:46 -07:00
..
compaction_iteration_stats.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
compaction_iterator_test.cc Allow compaction iterator to perform garbage collection (#7556) 2020-10-23 22:59:46 -07:00
compaction_iterator.cc Allow compaction iterator to perform garbage collection (#7556) 2020-10-23 22:59:46 -07:00
compaction_iterator.h Allow compaction iterator to perform garbage collection (#7556) 2020-10-23 22:59:46 -07:00
compaction_job_stats_test.cc Add is_full_compaction to CompactionJobStats, cleanup (#7451) 2020-10-01 12:52:58 -07:00
compaction_job_test.cc Introduce options.check_flush_compaction_key_order (#7467) 2020-10-01 10:10:26 -07:00
compaction_job.cc Status check enforcement for error_handler_fs_test (#7342) 2020-10-02 16:41:13 -07:00
compaction_job.h Store FSWritableFilePtr object in WritableFileWriter (#7193) 2020-09-08 10:56:08 -07:00
compaction_picker_fifo.cc Make max_subcompactions dynamically changeable (#7159) 2020-07-22 18:32:52 -07:00
compaction_picker_fifo.h Make max_subcompactions dynamically changeable (#7159) 2020-07-22 18:32:52 -07:00
compaction_picker_level.cc Make max_subcompactions dynamically changeable (#7159) 2020-07-22 18:32:52 -07:00
compaction_picker_level.h Make max_subcompactions dynamically changeable (#7159) 2020-07-22 18:32:52 -07:00
compaction_picker_test.cc Add missing ComputeCompactionScore() for a new universal manual compaction (#7281) 2020-08-19 17:42:08 -07:00
compaction_picker_universal.cc Make sure assert(false) handles failures too (#7483) 2020-10-06 13:52:42 -07:00
compaction_picker_universal.h Make max_subcompactions dynamically changeable (#7159) 2020-07-22 18:32:52 -07:00
compaction_picker.cc Add missing ComputeCompactionScore() for a new universal manual compaction (#7281) 2020-08-19 17:42:08 -07:00
compaction_picker.h Make max_subcompactions dynamically changeable (#7159) 2020-07-22 18:32:52 -07:00
compaction.cc Allow compaction iterator to perform garbage collection (#7556) 2020-10-23 22:59:46 -07:00
compaction.h SST Partitioner interface that allows to split SST files (#6957) 2020-07-24 13:44:49 -07:00
sst_partitioner.cc SST Partitioner interface that allows to split SST files (#6957) 2020-07-24 13:44:49 -07:00