andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Add DeteleRange support for DBWithTTL (#8384)

Browse Source 
Summary:
This commit is for enabling `DBWithTTL` to use `DeteleRange` which it cannot before.
As (int32_t)Timestamp is suffixed to values in `DBWithTTL`, there is no reason that it
cannot use the common used api. I added `DeleteRangeCF` in `DBWithTTLImpl::Write`
so that we can use `DeteleRange` normally. When we run code like
`dbWithTtl->DeleteRange(start, end)`, it executes`WriteBatchInternal::DeleteRange`
internally. Intended to fix https://github.com/facebook/rocksdb/issues/7218

Pull Request resolved: https://github.com/facebook/rocksdb/pull/8384

Test Plan: added corresponded testing logic to existing unit test

Reviewed By: jay-zhuang

Differential Revision: D29176734

fbshipit-source-id: 6874ed979fc08e1d138149d03653e43a75f0e0e6
This commit is contained in:
聂佩轩 2021-06-17 15:59:53 -07:00 committed by Facebook GitHub Bot
parent 865a25101d
commit d53f7ff69a
3 changed files with 43 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
HISTORY.md
View File

@ -10,6 +10,9 @@
### New Features ### New Features
* Marked the Ribbon filter and optimize_filters_for_memory features as production-ready, each enabling memory savings for Bloom-like filters. Use `NewRibbonFilterPolicy` in place of `NewBloomFilterPolicy` to use Ribbon filters instead of Bloom, or `ribbonfilter` in place of `bloomfilter` in configuration string. * Marked the Ribbon filter and optimize_filters_for_memory features as production-ready, each enabling memory savings for Bloom-like filters. Use `NewRibbonFilterPolicy` in place of `NewBloomFilterPolicy` to use Ribbon filters instead of Bloom, or `ribbonfilter` in place of `bloomfilter` in configuration string.
### New Features
* Allow `DBWithTTL` to use `DeleteRange` api just like other DBs. `DeleteRangeCF()` which executes `WriteBatchInternal::DeleteRange()` has been added to the handler in `DBWithTTLImpl::Write()` to implement it.
## 6.21.0 (2021-05-21) ## 6.21.0 (2021-05-21)
### Bug Fixes ### Bug Fixes
* Fixed a bug in handling file rename error in distributed/network file systems when the server succeeds but client returns error. The bug can cause CURRENT file to point to non-existing MANIFEST file, thus DB cannot be opened. * Fixed a bug in handling file rename error in distributed/network file systems when the server succeeds but client returns error. The bug can cause CURRENT file to point to non-existing MANIFEST file, thus DB cannot be opened.

5
utilities/ttl/db_ttl_impl.cc
View File

@ -304,6 +304,11 @@ Status DBWithTTLImpl::Write(const WriteOptions& opts, WriteBatch* updates) {
Status DeleteCF(uint32_t column_family_id, const Slice& key) override { Status DeleteCF(uint32_t column_family_id, const Slice& key) override {
return WriteBatchInternal::Delete(&updates_ttl, column_family_id, key); return WriteBatchInternal::Delete(&updates_ttl, column_family_id, key);
} }
Status DeleteRangeCF(uint32_t column_family_id, const Slice& begin_key,
const Slice& end_key) override {
return WriteBatchInternal::DeleteRange(&updates_ttl, column_family_id,
begin_key, end_key);
}
void LogData(const Slice& blob) override { updates_ttl.PutLogData(blob); } void LogData(const Slice& blob) override { updates_ttl.PutLogData(blob); }
private: private:

35
utilities/ttl/ttl_test.cc
View File

@ -186,6 +186,18 @@ class TtlTest : public testing::Test {
} }
} }
// Runs a DeleteRange
void MakeDeleteRange(std::string start, std::string end,
ColumnFamilyHandle* cf = nullptr) {
ASSERT_TRUE(db_ttl_);
static WriteOptions wops;
WriteBatch wb;
ASSERT_OK(cf == nullptr
? wb.DeleteRange(db_ttl_->DefaultColumnFamily(), start, end)
: wb.DeleteRange(cf, start, end));
ASSERT_OK(db_ttl_->Write(wops, &wb));
}
// checks the whole kvmap_ to return correct values using KeyMayExist // checks the whole kvmap_ to return correct values using KeyMayExist
void SimpleKeyMayExistCheck() { void SimpleKeyMayExistCheck() {
static ReadOptions ropts; static ReadOptions ropts;
@ -684,6 +696,29 @@ TEST_F(TtlTest, ChangeTtlOnOpenDb) {
CloseTtl(); CloseTtl();
} }
// Test DeleteRange for DBWithTtl
TEST_F(TtlTest, DeleteRangeTest) {
OpenTtl();
ASSERT_OK(db_ttl_->Put(WriteOptions(), "a", "val"));
MakeDeleteRange("a", "b");
ASSERT_OK(db_ttl_->Put(WriteOptions(), "c", "val"));
MakeDeleteRange("b", "d");
ASSERT_OK(db_ttl_->Put(WriteOptions(), "e", "val"));
MakeDeleteRange("d", "e");
// first iteration verifies query correctness in memtable, second verifies
// query correctness for a single SST file
for (int i = 0; i < 2; i++) {
if (i > 0) {
ASSERT_OK(db_ttl_->Flush(FlushOptions()));
}
std::string value;
ASSERT_TRUE(db_ttl_->Get(ReadOptions(), "a", &value).IsNotFound());
ASSERT_TRUE(db_ttl_->Get(ReadOptions(), "c", &value).IsNotFound());
ASSERT_OK(db_ttl_->Get(ReadOptions(), "e", &value));
}
CloseTtl();
}
} // namespace ROCKSDB_NAMESPACE } // namespace ROCKSDB_NAMESPACE
// A black-box test for the ttl wrapper around rocksdb // A black-box test for the ttl wrapper around rocksdb