Timestamp and TTL Wrapper for rocksdb
Summary:
When opened with DBTimestamp::Open call, timestamps are prepended to and stripped from the value during subsequent Put and Get calls respectively. The Timestamp is used to discard values in Get and custom compaction filter which have exceeded their TTL which is specified during Open.
Have made a temporary change to Makefile to let us test with the temporary file TestTime.cc. Have also changed the private members of db_impl.h to protected to let them be inherited by the new class DBTimestamp
Test Plan: make db_timestamp; TestTime.cc(will not check it in) shows how to use the apis currently, but I will write unit-tests shortly
Reviewers: dhruba, vamsi, haobo, sheki, heyongqiang, vkrest
Reviewed By: vamsi
CC: zshao, xjin, vkrest, MarkCallaghan
Differential Revision: https://reviews.facebook.net/D10311
2013-04-15 22:33:13 +02:00
|
|
|
// Copyright (c) 2011 The LevelDB Authors. All rights reserved.
|
|
|
|
// Use of this source code is governed by a BSD-style license that can be
|
|
|
|
// found in the LICENSE file. See the AUTHORS file for names of contributors.
|
|
|
|
|
|
|
|
#include "utilities/ttl/db_ttl.h"
|
|
|
|
#include "db/filename.h"
|
|
|
|
#include "util/coding.h"
|
2013-08-23 17:38:13 +02:00
|
|
|
#include "include/rocksdb/env.h"
|
|
|
|
#include "include/rocksdb/iterator.h"
|
Timestamp and TTL Wrapper for rocksdb
Summary:
When opened with DBTimestamp::Open call, timestamps are prepended to and stripped from the value during subsequent Put and Get calls respectively. The Timestamp is used to discard values in Get and custom compaction filter which have exceeded their TTL which is specified during Open.
Have made a temporary change to Makefile to let us test with the temporary file TestTime.cc. Have also changed the private members of db_impl.h to protected to let them be inherited by the new class DBTimestamp
Test Plan: make db_timestamp; TestTime.cc(will not check it in) shows how to use the apis currently, but I will write unit-tests shortly
Reviewers: dhruba, vamsi, haobo, sheki, heyongqiang, vkrest
Reviewed By: vamsi
CC: zshao, xjin, vkrest, MarkCallaghan
Differential Revision: https://reviews.facebook.net/D10311
2013-04-15 22:33:13 +02:00
|
|
|
|
2013-10-04 06:49:15 +02:00
|
|
|
namespace rocksdb {
|
Timestamp and TTL Wrapper for rocksdb
Summary:
When opened with DBTimestamp::Open call, timestamps are prepended to and stripped from the value during subsequent Put and Get calls respectively. The Timestamp is used to discard values in Get and custom compaction filter which have exceeded their TTL which is specified during Open.
Have made a temporary change to Makefile to let us test with the temporary file TestTime.cc. Have also changed the private members of db_impl.h to protected to let them be inherited by the new class DBTimestamp
Test Plan: make db_timestamp; TestTime.cc(will not check it in) shows how to use the apis currently, but I will write unit-tests shortly
Reviewers: dhruba, vamsi, haobo, sheki, heyongqiang, vkrest
Reviewed By: vamsi
CC: zshao, xjin, vkrest, MarkCallaghan
Differential Revision: https://reviews.facebook.net/D10311
2013-04-15 22:33:13 +02:00
|
|
|
|
|
|
|
// Open the db inside DBWithTTL because options needs pointer to its ttl
|
|
|
|
DBWithTTL::DBWithTTL(const int32_t ttl,
|
|
|
|
const Options& options,
|
|
|
|
const std::string& dbname,
|
2013-05-10 23:19:47 +02:00
|
|
|
Status& st,
|
|
|
|
bool read_only)
|
2013-11-13 05:05:28 +01:00
|
|
|
: StackableDB(nullptr) {
|
Timestamp and TTL Wrapper for rocksdb
Summary:
When opened with DBTimestamp::Open call, timestamps are prepended to and stripped from the value during subsequent Put and Get calls respectively. The Timestamp is used to discard values in Get and custom compaction filter which have exceeded their TTL which is specified during Open.
Have made a temporary change to Makefile to let us test with the temporary file TestTime.cc. Have also changed the private members of db_impl.h to protected to let them be inherited by the new class DBTimestamp
Test Plan: make db_timestamp; TestTime.cc(will not check it in) shows how to use the apis currently, but I will write unit-tests shortly
Reviewers: dhruba, vamsi, haobo, sheki, heyongqiang, vkrest
Reviewed By: vamsi
CC: zshao, xjin, vkrest, MarkCallaghan
Differential Revision: https://reviews.facebook.net/D10311
2013-04-15 22:33:13 +02:00
|
|
|
Options options_to_open = options;
|
2013-08-03 09:40:22 +02:00
|
|
|
|
2013-08-13 19:56:20 +02:00
|
|
|
if (options.compaction_filter) {
|
|
|
|
ttl_comp_filter_.reset(
|
|
|
|
new TtlCompactionFilter(ttl, options.compaction_filter));
|
|
|
|
options_to_open.compaction_filter = ttl_comp_filter_.get();
|
|
|
|
} else {
|
|
|
|
options_to_open.compaction_filter_factory =
|
|
|
|
std::shared_ptr<CompactionFilterFactory>(
|
|
|
|
new TtlCompactionFilterFactory(
|
|
|
|
ttl, options.compaction_filter_factory));
|
|
|
|
}
|
2013-08-03 09:40:22 +02:00
|
|
|
|
2013-07-23 01:49:55 +02:00
|
|
|
if (options.merge_operator) {
|
2013-08-20 22:35:28 +02:00
|
|
|
options_to_open.merge_operator.reset(
|
|
|
|
new TtlMergeOperator(options.merge_operator));
|
2013-07-23 01:49:55 +02:00
|
|
|
}
|
2013-08-03 09:40:22 +02:00
|
|
|
|
2013-05-10 23:19:47 +02:00
|
|
|
if (read_only) {
|
|
|
|
st = DB::OpenForReadOnly(options_to_open, dbname, &db_);
|
|
|
|
} else {
|
|
|
|
st = DB::Open(options_to_open, dbname, &db_);
|
|
|
|
}
|
Timestamp and TTL Wrapper for rocksdb
Summary:
When opened with DBTimestamp::Open call, timestamps are prepended to and stripped from the value during subsequent Put and Get calls respectively. The Timestamp is used to discard values in Get and custom compaction filter which have exceeded their TTL which is specified during Open.
Have made a temporary change to Makefile to let us test with the temporary file TestTime.cc. Have also changed the private members of db_impl.h to protected to let them be inherited by the new class DBTimestamp
Test Plan: make db_timestamp; TestTime.cc(will not check it in) shows how to use the apis currently, but I will write unit-tests shortly
Reviewers: dhruba, vamsi, haobo, sheki, heyongqiang, vkrest
Reviewed By: vamsi
CC: zshao, xjin, vkrest, MarkCallaghan
Differential Revision: https://reviews.facebook.net/D10311
2013-04-15 22:33:13 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
DBWithTTL::~DBWithTTL() {
|
|
|
|
delete db_;
|
|
|
|
}
|
|
|
|
|
|
|
|
Status UtilityDB::OpenTtlDB(
|
|
|
|
const Options& options,
|
|
|
|
const std::string& dbname,
|
2013-08-06 02:55:44 +02:00
|
|
|
StackableDB** dbptr,
|
2013-05-10 23:19:47 +02:00
|
|
|
int32_t ttl,
|
|
|
|
bool read_only) {
|
Timestamp and TTL Wrapper for rocksdb
Summary:
When opened with DBTimestamp::Open call, timestamps are prepended to and stripped from the value during subsequent Put and Get calls respectively. The Timestamp is used to discard values in Get and custom compaction filter which have exceeded their TTL which is specified during Open.
Have made a temporary change to Makefile to let us test with the temporary file TestTime.cc. Have also changed the private members of db_impl.h to protected to let them be inherited by the new class DBTimestamp
Test Plan: make db_timestamp; TestTime.cc(will not check it in) shows how to use the apis currently, but I will write unit-tests shortly
Reviewers: dhruba, vamsi, haobo, sheki, heyongqiang, vkrest
Reviewed By: vamsi
CC: zshao, xjin, vkrest, MarkCallaghan
Differential Revision: https://reviews.facebook.net/D10311
2013-04-15 22:33:13 +02:00
|
|
|
Status st;
|
2013-05-10 23:19:47 +02:00
|
|
|
*dbptr = new DBWithTTL(ttl, options, dbname, st, read_only);
|
Timestamp and TTL Wrapper for rocksdb
Summary:
When opened with DBTimestamp::Open call, timestamps are prepended to and stripped from the value during subsequent Put and Get calls respectively. The Timestamp is used to discard values in Get and custom compaction filter which have exceeded their TTL which is specified during Open.
Have made a temporary change to Makefile to let us test with the temporary file TestTime.cc. Have also changed the private members of db_impl.h to protected to let them be inherited by the new class DBTimestamp
Test Plan: make db_timestamp; TestTime.cc(will not check it in) shows how to use the apis currently, but I will write unit-tests shortly
Reviewers: dhruba, vamsi, haobo, sheki, heyongqiang, vkrest
Reviewed By: vamsi
CC: zshao, xjin, vkrest, MarkCallaghan
Differential Revision: https://reviews.facebook.net/D10311
2013-04-15 22:33:13 +02:00
|
|
|
if (!st.ok()) {
|
2013-09-13 19:40:27 +02:00
|
|
|
delete *dbptr;
|
Timestamp and TTL Wrapper for rocksdb
Summary:
When opened with DBTimestamp::Open call, timestamps are prepended to and stripped from the value during subsequent Put and Get calls respectively. The Timestamp is used to discard values in Get and custom compaction filter which have exceeded their TTL which is specified during Open.
Have made a temporary change to Makefile to let us test with the temporary file TestTime.cc. Have also changed the private members of db_impl.h to protected to let them be inherited by the new class DBTimestamp
Test Plan: make db_timestamp; TestTime.cc(will not check it in) shows how to use the apis currently, but I will write unit-tests shortly
Reviewers: dhruba, vamsi, haobo, sheki, heyongqiang, vkrest
Reviewed By: vamsi
CC: zshao, xjin, vkrest, MarkCallaghan
Differential Revision: https://reviews.facebook.net/D10311
2013-04-15 22:33:13 +02:00
|
|
|
}
|
|
|
|
return st;
|
|
|
|
}
|
|
|
|
|
|
|
|
// Gives back the current time
|
2013-11-19 05:56:21 +01:00
|
|
|
Status DBWithTTL::GetCurrentTime(int64_t& curtime) {
|
|
|
|
return Env::Default()->GetCurrentTime(&curtime);
|
Timestamp and TTL Wrapper for rocksdb
Summary:
When opened with DBTimestamp::Open call, timestamps are prepended to and stripped from the value during subsequent Put and Get calls respectively. The Timestamp is used to discard values in Get and custom compaction filter which have exceeded their TTL which is specified during Open.
Have made a temporary change to Makefile to let us test with the temporary file TestTime.cc. Have also changed the private members of db_impl.h to protected to let them be inherited by the new class DBTimestamp
Test Plan: make db_timestamp; TestTime.cc(will not check it in) shows how to use the apis currently, but I will write unit-tests shortly
Reviewers: dhruba, vamsi, haobo, sheki, heyongqiang, vkrest
Reviewed By: vamsi
CC: zshao, xjin, vkrest, MarkCallaghan
Differential Revision: https://reviews.facebook.net/D10311
2013-04-15 22:33:13 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
// Appends the current timestamp to the string.
|
|
|
|
// Returns false if could not get the current_time, true if append succeeds
|
|
|
|
Status DBWithTTL::AppendTS(const Slice& val, std::string& val_with_ts) {
|
|
|
|
val_with_ts.reserve(kTSLength + val.size());
|
|
|
|
char ts_string[kTSLength];
|
2013-11-19 05:56:21 +01:00
|
|
|
int64_t curtime;
|
Timestamp and TTL Wrapper for rocksdb
Summary:
When opened with DBTimestamp::Open call, timestamps are prepended to and stripped from the value during subsequent Put and Get calls respectively. The Timestamp is used to discard values in Get and custom compaction filter which have exceeded their TTL which is specified during Open.
Have made a temporary change to Makefile to let us test with the temporary file TestTime.cc. Have also changed the private members of db_impl.h to protected to let them be inherited by the new class DBTimestamp
Test Plan: make db_timestamp; TestTime.cc(will not check it in) shows how to use the apis currently, but I will write unit-tests shortly
Reviewers: dhruba, vamsi, haobo, sheki, heyongqiang, vkrest
Reviewed By: vamsi
CC: zshao, xjin, vkrest, MarkCallaghan
Differential Revision: https://reviews.facebook.net/D10311
2013-04-15 22:33:13 +02:00
|
|
|
Status st = GetCurrentTime(curtime);
|
|
|
|
if (!st.ok()) {
|
|
|
|
return st;
|
|
|
|
}
|
2013-11-19 05:56:21 +01:00
|
|
|
EncodeFixed32(ts_string, (int32_t)curtime);
|
Timestamp and TTL Wrapper for rocksdb
Summary:
When opened with DBTimestamp::Open call, timestamps are prepended to and stripped from the value during subsequent Put and Get calls respectively. The Timestamp is used to discard values in Get and custom compaction filter which have exceeded their TTL which is specified during Open.
Have made a temporary change to Makefile to let us test with the temporary file TestTime.cc. Have also changed the private members of db_impl.h to protected to let them be inherited by the new class DBTimestamp
Test Plan: make db_timestamp; TestTime.cc(will not check it in) shows how to use the apis currently, but I will write unit-tests shortly
Reviewers: dhruba, vamsi, haobo, sheki, heyongqiang, vkrest
Reviewed By: vamsi
CC: zshao, xjin, vkrest, MarkCallaghan
Differential Revision: https://reviews.facebook.net/D10311
2013-04-15 22:33:13 +02:00
|
|
|
val_with_ts.append(val.data(), val.size());
|
|
|
|
val_with_ts.append(ts_string, kTSLength);
|
|
|
|
return st;
|
|
|
|
}
|
|
|
|
|
2013-05-10 02:33:27 +02:00
|
|
|
// Returns corruption if the length of the string is lesser than timestamp, or
|
|
|
|
// timestamp refers to a time lesser than ttl-feature release time
|
2013-06-20 20:50:33 +02:00
|
|
|
Status DBWithTTL::SanityCheckTimestamp(const Slice& str) {
|
2013-08-03 09:40:22 +02:00
|
|
|
if (str.size() < kTSLength) {
|
2013-05-10 02:33:27 +02:00
|
|
|
return Status::Corruption("Error: value's length less than timestamp's\n");
|
|
|
|
}
|
|
|
|
// Checks that TS is not lesser than kMinTimestamp
|
|
|
|
// Gaurds against corruption & normal database opened incorrectly in ttl mode
|
|
|
|
int32_t timestamp_value =
|
|
|
|
DecodeFixed32(str.data() + str.size() - kTSLength);
|
|
|
|
if (timestamp_value < kMinTimestamp){
|
|
|
|
return Status::Corruption("Error: Timestamp < ttl feature release time!\n");
|
|
|
|
}
|
|
|
|
return Status::OK();
|
|
|
|
}
|
|
|
|
|
Timestamp and TTL Wrapper for rocksdb
Summary:
When opened with DBTimestamp::Open call, timestamps are prepended to and stripped from the value during subsequent Put and Get calls respectively. The Timestamp is used to discard values in Get and custom compaction filter which have exceeded their TTL which is specified during Open.
Have made a temporary change to Makefile to let us test with the temporary file TestTime.cc. Have also changed the private members of db_impl.h to protected to let them be inherited by the new class DBTimestamp
Test Plan: make db_timestamp; TestTime.cc(will not check it in) shows how to use the apis currently, but I will write unit-tests shortly
Reviewers: dhruba, vamsi, haobo, sheki, heyongqiang, vkrest
Reviewed By: vamsi
CC: zshao, xjin, vkrest, MarkCallaghan
Differential Revision: https://reviews.facebook.net/D10311
2013-04-15 22:33:13 +02:00
|
|
|
// Checks if the string is stale or not according to TTl provided
|
|
|
|
bool DBWithTTL::IsStale(const Slice& value, int32_t ttl) {
|
|
|
|
if (ttl <= 0) { // Data is fresh if TTL is non-positive
|
|
|
|
return false;
|
|
|
|
}
|
2013-11-19 05:56:21 +01:00
|
|
|
int64_t curtime;
|
Timestamp and TTL Wrapper for rocksdb
Summary:
When opened with DBTimestamp::Open call, timestamps are prepended to and stripped from the value during subsequent Put and Get calls respectively. The Timestamp is used to discard values in Get and custom compaction filter which have exceeded their TTL which is specified during Open.
Have made a temporary change to Makefile to let us test with the temporary file TestTime.cc. Have also changed the private members of db_impl.h to protected to let them be inherited by the new class DBTimestamp
Test Plan: make db_timestamp; TestTime.cc(will not check it in) shows how to use the apis currently, but I will write unit-tests shortly
Reviewers: dhruba, vamsi, haobo, sheki, heyongqiang, vkrest
Reviewed By: vamsi
CC: zshao, xjin, vkrest, MarkCallaghan
Differential Revision: https://reviews.facebook.net/D10311
2013-04-15 22:33:13 +02:00
|
|
|
if (!GetCurrentTime(curtime).ok()) {
|
|
|
|
return false; // Treat the data as fresh if could not get current time
|
|
|
|
}
|
2013-08-03 09:40:22 +02:00
|
|
|
int32_t timestamp_value =
|
|
|
|
DecodeFixed32(value.data() + value.size() - kTSLength);
|
|
|
|
return (timestamp_value + ttl) < curtime;
|
Timestamp and TTL Wrapper for rocksdb
Summary:
When opened with DBTimestamp::Open call, timestamps are prepended to and stripped from the value during subsequent Put and Get calls respectively. The Timestamp is used to discard values in Get and custom compaction filter which have exceeded their TTL which is specified during Open.
Have made a temporary change to Makefile to let us test with the temporary file TestTime.cc. Have also changed the private members of db_impl.h to protected to let them be inherited by the new class DBTimestamp
Test Plan: make db_timestamp; TestTime.cc(will not check it in) shows how to use the apis currently, but I will write unit-tests shortly
Reviewers: dhruba, vamsi, haobo, sheki, heyongqiang, vkrest
Reviewed By: vamsi
CC: zshao, xjin, vkrest, MarkCallaghan
Differential Revision: https://reviews.facebook.net/D10311
2013-04-15 22:33:13 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
// Strips the TS from the end of the string
|
|
|
|
Status DBWithTTL::StripTS(std::string* str) {
|
|
|
|
Status st;
|
2013-08-03 09:40:22 +02:00
|
|
|
if (str->length() < kTSLength) {
|
|
|
|
return Status::Corruption("Bad timestamp in key-value");
|
|
|
|
}
|
Timestamp and TTL Wrapper for rocksdb
Summary:
When opened with DBTimestamp::Open call, timestamps are prepended to and stripped from the value during subsequent Put and Get calls respectively. The Timestamp is used to discard values in Get and custom compaction filter which have exceeded their TTL which is specified during Open.
Have made a temporary change to Makefile to let us test with the temporary file TestTime.cc. Have also changed the private members of db_impl.h to protected to let them be inherited by the new class DBTimestamp
Test Plan: make db_timestamp; TestTime.cc(will not check it in) shows how to use the apis currently, but I will write unit-tests shortly
Reviewers: dhruba, vamsi, haobo, sheki, heyongqiang, vkrest
Reviewed By: vamsi
CC: zshao, xjin, vkrest, MarkCallaghan
Differential Revision: https://reviews.facebook.net/D10311
2013-04-15 22:33:13 +02:00
|
|
|
// Erasing characters which hold the TS
|
|
|
|
str->erase(str->length() - kTSLength, kTSLength);
|
|
|
|
return st;
|
|
|
|
}
|
|
|
|
|
|
|
|
Status DBWithTTL::Put(
|
2013-07-23 01:49:55 +02:00
|
|
|
const WriteOptions& opt,
|
Timestamp and TTL Wrapper for rocksdb
Summary:
When opened with DBTimestamp::Open call, timestamps are prepended to and stripped from the value during subsequent Put and Get calls respectively. The Timestamp is used to discard values in Get and custom compaction filter which have exceeded their TTL which is specified during Open.
Have made a temporary change to Makefile to let us test with the temporary file TestTime.cc. Have also changed the private members of db_impl.h to protected to let them be inherited by the new class DBTimestamp
Test Plan: make db_timestamp; TestTime.cc(will not check it in) shows how to use the apis currently, but I will write unit-tests shortly
Reviewers: dhruba, vamsi, haobo, sheki, heyongqiang, vkrest
Reviewed By: vamsi
CC: zshao, xjin, vkrest, MarkCallaghan
Differential Revision: https://reviews.facebook.net/D10311
2013-04-15 22:33:13 +02:00
|
|
|
const Slice& key,
|
|
|
|
const Slice& val) {
|
2013-07-23 01:49:55 +02:00
|
|
|
WriteBatch batch;
|
|
|
|
batch.Put(key, val);
|
|
|
|
return Write(opt, &batch);
|
Timestamp and TTL Wrapper for rocksdb
Summary:
When opened with DBTimestamp::Open call, timestamps are prepended to and stripped from the value during subsequent Put and Get calls respectively. The Timestamp is used to discard values in Get and custom compaction filter which have exceeded their TTL which is specified during Open.
Have made a temporary change to Makefile to let us test with the temporary file TestTime.cc. Have also changed the private members of db_impl.h to protected to let them be inherited by the new class DBTimestamp
Test Plan: make db_timestamp; TestTime.cc(will not check it in) shows how to use the apis currently, but I will write unit-tests shortly
Reviewers: dhruba, vamsi, haobo, sheki, heyongqiang, vkrest
Reviewed By: vamsi
CC: zshao, xjin, vkrest, MarkCallaghan
Differential Revision: https://reviews.facebook.net/D10311
2013-04-15 22:33:13 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
Status DBWithTTL::Get(const ReadOptions& options,
|
|
|
|
const Slice& key,
|
|
|
|
std::string* value) {
|
|
|
|
Status st = db_->Get(options, key, value);
|
|
|
|
if (!st.ok()) {
|
|
|
|
return st;
|
|
|
|
}
|
2013-05-10 02:33:27 +02:00
|
|
|
st = SanityCheckTimestamp(*value);
|
|
|
|
if (!st.ok()) {
|
|
|
|
return st;
|
|
|
|
}
|
Timestamp and TTL Wrapper for rocksdb
Summary:
When opened with DBTimestamp::Open call, timestamps are prepended to and stripped from the value during subsequent Put and Get calls respectively. The Timestamp is used to discard values in Get and custom compaction filter which have exceeded their TTL which is specified during Open.
Have made a temporary change to Makefile to let us test with the temporary file TestTime.cc. Have also changed the private members of db_impl.h to protected to let them be inherited by the new class DBTimestamp
Test Plan: make db_timestamp; TestTime.cc(will not check it in) shows how to use the apis currently, but I will write unit-tests shortly
Reviewers: dhruba, vamsi, haobo, sheki, heyongqiang, vkrest
Reviewed By: vamsi
CC: zshao, xjin, vkrest, MarkCallaghan
Differential Revision: https://reviews.facebook.net/D10311
2013-04-15 22:33:13 +02:00
|
|
|
return StripTS(value);
|
|
|
|
}
|
|
|
|
|
2013-06-05 20:22:38 +02:00
|
|
|
std::vector<Status> DBWithTTL::MultiGet(const ReadOptions& options,
|
|
|
|
const std::vector<Slice>& keys,
|
|
|
|
std::vector<std::string>* values) {
|
|
|
|
return std::vector<Status>(keys.size(),
|
|
|
|
Status::NotSupported("MultiGet not\
|
|
|
|
supported with TTL"));
|
|
|
|
}
|
|
|
|
|
2013-08-19 20:42:47 +02:00
|
|
|
bool DBWithTTL::KeyMayExist(const ReadOptions& options,
|
2013-07-26 21:57:01 +02:00
|
|
|
const Slice& key,
|
|
|
|
std::string* value,
|
|
|
|
bool* value_found) {
|
2013-08-24 02:36:00 +02:00
|
|
|
bool ret = db_->KeyMayExist(options, key, value, value_found);
|
|
|
|
if (ret && value != nullptr && value_found != nullptr && *value_found) {
|
|
|
|
if (!SanityCheckTimestamp(*value).ok() || !StripTS(value).ok()) {
|
|
|
|
return false;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
return ret;
|
2013-07-06 03:49:18 +02:00
|
|
|
}
|
|
|
|
|
Timestamp and TTL Wrapper for rocksdb
Summary:
When opened with DBTimestamp::Open call, timestamps are prepended to and stripped from the value during subsequent Put and Get calls respectively. The Timestamp is used to discard values in Get and custom compaction filter which have exceeded their TTL which is specified during Open.
Have made a temporary change to Makefile to let us test with the temporary file TestTime.cc. Have also changed the private members of db_impl.h to protected to let them be inherited by the new class DBTimestamp
Test Plan: make db_timestamp; TestTime.cc(will not check it in) shows how to use the apis currently, but I will write unit-tests shortly
Reviewers: dhruba, vamsi, haobo, sheki, heyongqiang, vkrest
Reviewed By: vamsi
CC: zshao, xjin, vkrest, MarkCallaghan
Differential Revision: https://reviews.facebook.net/D10311
2013-04-15 22:33:13 +02:00
|
|
|
Status DBWithTTL::Delete(const WriteOptions& wopts, const Slice& key) {
|
|
|
|
return db_->Delete(wopts, key);
|
|
|
|
}
|
|
|
|
|
2013-07-23 01:49:55 +02:00
|
|
|
Status DBWithTTL::Merge(const WriteOptions& opt,
|
2013-03-21 23:59:47 +01:00
|
|
|
const Slice& key,
|
|
|
|
const Slice& value) {
|
2013-07-23 01:49:55 +02:00
|
|
|
WriteBatch batch;
|
|
|
|
batch.Merge(key, value);
|
|
|
|
return Write(opt, &batch);
|
2013-03-21 23:59:47 +01:00
|
|
|
}
|
|
|
|
|
Timestamp and TTL Wrapper for rocksdb
Summary:
When opened with DBTimestamp::Open call, timestamps are prepended to and stripped from the value during subsequent Put and Get calls respectively. The Timestamp is used to discard values in Get and custom compaction filter which have exceeded their TTL which is specified during Open.
Have made a temporary change to Makefile to let us test with the temporary file TestTime.cc. Have also changed the private members of db_impl.h to protected to let them be inherited by the new class DBTimestamp
Test Plan: make db_timestamp; TestTime.cc(will not check it in) shows how to use the apis currently, but I will write unit-tests shortly
Reviewers: dhruba, vamsi, haobo, sheki, heyongqiang, vkrest
Reviewed By: vamsi
CC: zshao, xjin, vkrest, MarkCallaghan
Differential Revision: https://reviews.facebook.net/D10311
2013-04-15 22:33:13 +02:00
|
|
|
Status DBWithTTL::Write(const WriteOptions& opts, WriteBatch* updates) {
|
2013-05-16 00:00:57 +02:00
|
|
|
class Handler : public WriteBatch::Handler {
|
|
|
|
public:
|
|
|
|
WriteBatch updates_ttl;
|
|
|
|
Status batch_rewrite_status;
|
|
|
|
virtual void Put(const Slice& key, const Slice& value) {
|
|
|
|
std::string value_with_ts;
|
|
|
|
Status st = AppendTS(value, value_with_ts);
|
|
|
|
if (!st.ok()) {
|
|
|
|
batch_rewrite_status = st;
|
|
|
|
} else {
|
|
|
|
updates_ttl.Put(key, value_with_ts);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
virtual void Merge(const Slice& key, const Slice& value) {
|
2013-07-23 01:49:55 +02:00
|
|
|
std::string value_with_ts;
|
|
|
|
Status st = AppendTS(value, value_with_ts);
|
|
|
|
if (!st.ok()) {
|
|
|
|
batch_rewrite_status = st;
|
|
|
|
} else {
|
|
|
|
updates_ttl.Merge(key, value_with_ts);
|
|
|
|
}
|
2013-05-16 00:00:57 +02:00
|
|
|
}
|
|
|
|
virtual void Delete(const Slice& key) {
|
|
|
|
updates_ttl.Delete(key);
|
|
|
|
}
|
2013-09-05 22:42:08 +02:00
|
|
|
virtual void LogData(const Slice& blob) {
|
|
|
|
updates_ttl.PutLogData(blob);
|
|
|
|
}
|
2013-05-16 00:00:57 +02:00
|
|
|
};
|
|
|
|
Handler handler;
|
|
|
|
updates->Iterate(&handler);
|
|
|
|
if (!handler.batch_rewrite_status.ok()) {
|
|
|
|
return handler.batch_rewrite_status;
|
|
|
|
} else {
|
|
|
|
return db_->Write(opts, &(handler.updates_ttl));
|
|
|
|
}
|
Timestamp and TTL Wrapper for rocksdb
Summary:
When opened with DBTimestamp::Open call, timestamps are prepended to and stripped from the value during subsequent Put and Get calls respectively. The Timestamp is used to discard values in Get and custom compaction filter which have exceeded their TTL which is specified during Open.
Have made a temporary change to Makefile to let us test with the temporary file TestTime.cc. Have also changed the private members of db_impl.h to protected to let them be inherited by the new class DBTimestamp
Test Plan: make db_timestamp; TestTime.cc(will not check it in) shows how to use the apis currently, but I will write unit-tests shortly
Reviewers: dhruba, vamsi, haobo, sheki, heyongqiang, vkrest
Reviewed By: vamsi
CC: zshao, xjin, vkrest, MarkCallaghan
Differential Revision: https://reviews.facebook.net/D10311
2013-04-15 22:33:13 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
Iterator* DBWithTTL::NewIterator(const ReadOptions& opts) {
|
2013-06-20 20:50:33 +02:00
|
|
|
return new TtlIterator(db_->NewIterator(opts));
|
Timestamp and TTL Wrapper for rocksdb
Summary:
When opened with DBTimestamp::Open call, timestamps are prepended to and stripped from the value during subsequent Put and Get calls respectively. The Timestamp is used to discard values in Get and custom compaction filter which have exceeded their TTL which is specified during Open.
Have made a temporary change to Makefile to let us test with the temporary file TestTime.cc. Have also changed the private members of db_impl.h to protected to let them be inherited by the new class DBTimestamp
Test Plan: make db_timestamp; TestTime.cc(will not check it in) shows how to use the apis currently, but I will write unit-tests shortly
Reviewers: dhruba, vamsi, haobo, sheki, heyongqiang, vkrest
Reviewed By: vamsi
CC: zshao, xjin, vkrest, MarkCallaghan
Differential Revision: https://reviews.facebook.net/D10311
2013-04-15 22:33:13 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
const Snapshot* DBWithTTL::GetSnapshot() {
|
|
|
|
return db_->GetSnapshot();
|
|
|
|
}
|
|
|
|
|
|
|
|
void DBWithTTL::ReleaseSnapshot(const Snapshot* snapshot) {
|
|
|
|
db_->ReleaseSnapshot(snapshot);
|
|
|
|
}
|
|
|
|
|
|
|
|
bool DBWithTTL::GetProperty(const Slice& property, std::string* value) {
|
|
|
|
return db_->GetProperty(property, value);
|
|
|
|
}
|
|
|
|
|
|
|
|
void DBWithTTL::GetApproximateSizes(const Range* r, int n, uint64_t* sizes) {
|
|
|
|
db_->GetApproximateSizes(r, n, sizes);
|
|
|
|
}
|
|
|
|
|
2013-06-30 08:21:36 +02:00
|
|
|
void DBWithTTL::CompactRange(const Slice* begin, const Slice* end,
|
2013-09-04 22:13:08 +02:00
|
|
|
bool reduce_level, int target_level) {
|
|
|
|
db_->CompactRange(begin, end, reduce_level, target_level);
|
Timestamp and TTL Wrapper for rocksdb
Summary:
When opened with DBTimestamp::Open call, timestamps are prepended to and stripped from the value during subsequent Put and Get calls respectively. The Timestamp is used to discard values in Get and custom compaction filter which have exceeded their TTL which is specified during Open.
Have made a temporary change to Makefile to let us test with the temporary file TestTime.cc. Have also changed the private members of db_impl.h to protected to let them be inherited by the new class DBTimestamp
Test Plan: make db_timestamp; TestTime.cc(will not check it in) shows how to use the apis currently, but I will write unit-tests shortly
Reviewers: dhruba, vamsi, haobo, sheki, heyongqiang, vkrest
Reviewed By: vamsi
CC: zshao, xjin, vkrest, MarkCallaghan
Differential Revision: https://reviews.facebook.net/D10311
2013-04-15 22:33:13 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
int DBWithTTL::NumberLevels() {
|
|
|
|
return db_->NumberLevels();
|
|
|
|
}
|
|
|
|
|
|
|
|
int DBWithTTL::MaxMemCompactionLevel() {
|
|
|
|
return db_->MaxMemCompactionLevel();
|
|
|
|
}
|
|
|
|
|
|
|
|
int DBWithTTL::Level0StopWriteTrigger() {
|
|
|
|
return db_->Level0StopWriteTrigger();
|
|
|
|
}
|
|
|
|
|
2013-11-25 21:39:23 +01:00
|
|
|
Env* DBWithTTL::GetEnv() const {
|
|
|
|
return db_->GetEnv();
|
|
|
|
}
|
|
|
|
|
2013-11-26 00:51:50 +01:00
|
|
|
const Options& DBWithTTL::GetOptions() const {
|
|
|
|
return db_->GetOptions();
|
|
|
|
}
|
|
|
|
|
Timestamp and TTL Wrapper for rocksdb
Summary:
When opened with DBTimestamp::Open call, timestamps are prepended to and stripped from the value during subsequent Put and Get calls respectively. The Timestamp is used to discard values in Get and custom compaction filter which have exceeded their TTL which is specified during Open.
Have made a temporary change to Makefile to let us test with the temporary file TestTime.cc. Have also changed the private members of db_impl.h to protected to let them be inherited by the new class DBTimestamp
Test Plan: make db_timestamp; TestTime.cc(will not check it in) shows how to use the apis currently, but I will write unit-tests shortly
Reviewers: dhruba, vamsi, haobo, sheki, heyongqiang, vkrest
Reviewed By: vamsi
CC: zshao, xjin, vkrest, MarkCallaghan
Differential Revision: https://reviews.facebook.net/D10311
2013-04-15 22:33:13 +02:00
|
|
|
Status DBWithTTL::Flush(const FlushOptions& fopts) {
|
|
|
|
return db_->Flush(fopts);
|
|
|
|
}
|
|
|
|
|
|
|
|
Status DBWithTTL::DisableFileDeletions() {
|
|
|
|
return db_->DisableFileDeletions();
|
|
|
|
}
|
|
|
|
|
|
|
|
Status DBWithTTL::EnableFileDeletions() {
|
|
|
|
return db_->EnableFileDeletions();
|
|
|
|
}
|
|
|
|
|
2013-10-03 23:38:32 +02:00
|
|
|
Status DBWithTTL::GetLiveFiles(std::vector<std::string>& vec, uint64_t* mfs,
|
|
|
|
bool flush_memtable) {
|
|
|
|
return db_->GetLiveFiles(vec, mfs, flush_memtable);
|
Timestamp and TTL Wrapper for rocksdb
Summary:
When opened with DBTimestamp::Open call, timestamps are prepended to and stripped from the value during subsequent Put and Get calls respectively. The Timestamp is used to discard values in Get and custom compaction filter which have exceeded their TTL which is specified during Open.
Have made a temporary change to Makefile to let us test with the temporary file TestTime.cc. Have also changed the private members of db_impl.h to protected to let them be inherited by the new class DBTimestamp
Test Plan: make db_timestamp; TestTime.cc(will not check it in) shows how to use the apis currently, but I will write unit-tests shortly
Reviewers: dhruba, vamsi, haobo, sheki, heyongqiang, vkrest
Reviewed By: vamsi
CC: zshao, xjin, vkrest, MarkCallaghan
Differential Revision: https://reviews.facebook.net/D10311
2013-04-15 22:33:13 +02:00
|
|
|
}
|
|
|
|
|
2013-10-25 04:09:02 +02:00
|
|
|
SequenceNumber DBWithTTL::GetLatestSequenceNumber() const {
|
Timestamp and TTL Wrapper for rocksdb
Summary:
When opened with DBTimestamp::Open call, timestamps are prepended to and stripped from the value during subsequent Put and Get calls respectively. The Timestamp is used to discard values in Get and custom compaction filter which have exceeded their TTL which is specified during Open.
Have made a temporary change to Makefile to let us test with the temporary file TestTime.cc. Have also changed the private members of db_impl.h to protected to let them be inherited by the new class DBTimestamp
Test Plan: make db_timestamp; TestTime.cc(will not check it in) shows how to use the apis currently, but I will write unit-tests shortly
Reviewers: dhruba, vamsi, haobo, sheki, heyongqiang, vkrest
Reviewed By: vamsi
CC: zshao, xjin, vkrest, MarkCallaghan
Differential Revision: https://reviews.facebook.net/D10311
2013-04-15 22:33:13 +02:00
|
|
|
return db_->GetLatestSequenceNumber();
|
|
|
|
}
|
|
|
|
|
2013-08-06 21:54:37 +02:00
|
|
|
Status DBWithTTL::GetSortedWalFiles(VectorLogPtr& files) {
|
|
|
|
return db_->GetSortedWalFiles(files);
|
|
|
|
}
|
|
|
|
|
2013-10-24 08:39:23 +02:00
|
|
|
Status DBWithTTL::DeleteFile(std::string name) {
|
|
|
|
return db_->DeleteFile(name);
|
2013-08-06 21:54:37 +02:00
|
|
|
}
|
|
|
|
|
2013-12-05 19:16:39 +01:00
|
|
|
Status DBWithTTL::GetDbIdentity(std::string& identity) {
|
|
|
|
return db_->GetDbIdentity(identity);
|
|
|
|
}
|
|
|
|
|
Timestamp and TTL Wrapper for rocksdb
Summary:
When opened with DBTimestamp::Open call, timestamps are prepended to and stripped from the value during subsequent Put and Get calls respectively. The Timestamp is used to discard values in Get and custom compaction filter which have exceeded their TTL which is specified during Open.
Have made a temporary change to Makefile to let us test with the temporary file TestTime.cc. Have also changed the private members of db_impl.h to protected to let them be inherited by the new class DBTimestamp
Test Plan: make db_timestamp; TestTime.cc(will not check it in) shows how to use the apis currently, but I will write unit-tests shortly
Reviewers: dhruba, vamsi, haobo, sheki, heyongqiang, vkrest
Reviewed By: vamsi
CC: zshao, xjin, vkrest, MarkCallaghan
Differential Revision: https://reviews.facebook.net/D10311
2013-04-15 22:33:13 +02:00
|
|
|
Status DBWithTTL::GetUpdatesSince(
|
|
|
|
SequenceNumber seq_number,
|
|
|
|
unique_ptr<TransactionLogIterator>* iter) {
|
|
|
|
return db_->GetUpdatesSince(seq_number, iter);
|
|
|
|
}
|
|
|
|
|
|
|
|
void DBWithTTL::TEST_Destroy_DBWithTtl() {
|
|
|
|
((DBImpl*) db_)->TEST_Destroy_DBImpl();
|
|
|
|
}
|
|
|
|
|
2013-10-04 06:49:15 +02:00
|
|
|
} // namespace rocksdb
|