Timestamp and TTL Wrapper for rocksdb
Summary:
When opened with DBTimestamp::Open call, timestamps are prepended to and stripped from the value during subsequent Put and Get calls respectively. The Timestamp is used to discard values in Get and custom compaction filter which have exceeded their TTL which is specified during Open.
Have made a temporary change to Makefile to let us test with the temporary file TestTime.cc. Have also changed the private members of db_impl.h to protected to let them be inherited by the new class DBTimestamp
Test Plan: make db_timestamp; TestTime.cc(will not check it in) shows how to use the apis currently, but I will write unit-tests shortly
Reviewers: dhruba, vamsi, haobo, sheki, heyongqiang, vkrest
Reviewed By: vamsi
CC: zshao, xjin, vkrest, MarkCallaghan
Differential Revision: https://reviews.facebook.net/D10311
2013-04-15 22:33:13 +02:00
|
|
|
// Copyright (c) 2011 The LevelDB Authors. All rights reserved.
|
|
|
|
// Use of this source code is governed by a BSD-style license that can be
|
|
|
|
// found in the LICENSE file. See the AUTHORS file for names of contributors.
|
|
|
|
|
2013-10-05 07:32:05 +02:00
|
|
|
#pragma once
|
2014-04-22 20:27:33 +02:00
|
|
|
#ifndef ROCKSDB_LITE
|
|
|
|
#include <vector>
|
|
|
|
#include <string>
|
|
|
|
|
|
|
|
#include "utilities/stackable_db.h"
|
|
|
|
#include "rocksdb/db.h"
|
Timestamp and TTL Wrapper for rocksdb
Summary:
When opened with DBTimestamp::Open call, timestamps are prepended to and stripped from the value during subsequent Put and Get calls respectively. The Timestamp is used to discard values in Get and custom compaction filter which have exceeded their TTL which is specified during Open.
Have made a temporary change to Makefile to let us test with the temporary file TestTime.cc. Have also changed the private members of db_impl.h to protected to let them be inherited by the new class DBTimestamp
Test Plan: make db_timestamp; TestTime.cc(will not check it in) shows how to use the apis currently, but I will write unit-tests shortly
Reviewers: dhruba, vamsi, haobo, sheki, heyongqiang, vkrest
Reviewed By: vamsi
CC: zshao, xjin, vkrest, MarkCallaghan
Differential Revision: https://reviews.facebook.net/D10311
2013-04-15 22:33:13 +02:00
|
|
|
|
2013-10-04 06:49:15 +02:00
|
|
|
namespace rocksdb {
|
Timestamp and TTL Wrapper for rocksdb
Summary:
When opened with DBTimestamp::Open call, timestamps are prepended to and stripped from the value during subsequent Put and Get calls respectively. The Timestamp is used to discard values in Get and custom compaction filter which have exceeded their TTL which is specified during Open.
Have made a temporary change to Makefile to let us test with the temporary file TestTime.cc. Have also changed the private members of db_impl.h to protected to let them be inherited by the new class DBTimestamp
Test Plan: make db_timestamp; TestTime.cc(will not check it in) shows how to use the apis currently, but I will write unit-tests shortly
Reviewers: dhruba, vamsi, haobo, sheki, heyongqiang, vkrest
Reviewed By: vamsi
CC: zshao, xjin, vkrest, MarkCallaghan
Differential Revision: https://reviews.facebook.net/D10311
2013-04-15 22:33:13 +02:00
|
|
|
|
2013-10-05 07:32:05 +02:00
|
|
|
// This class contains APIs to open rocksdb with specific support eg. TTL
|
Timestamp and TTL Wrapper for rocksdb
Summary:
When opened with DBTimestamp::Open call, timestamps are prepended to and stripped from the value during subsequent Put and Get calls respectively. The Timestamp is used to discard values in Get and custom compaction filter which have exceeded their TTL which is specified during Open.
Have made a temporary change to Makefile to let us test with the temporary file TestTime.cc. Have also changed the private members of db_impl.h to protected to let them be inherited by the new class DBTimestamp
Test Plan: make db_timestamp; TestTime.cc(will not check it in) shows how to use the apis currently, but I will write unit-tests shortly
Reviewers: dhruba, vamsi, haobo, sheki, heyongqiang, vkrest
Reviewed By: vamsi
CC: zshao, xjin, vkrest, MarkCallaghan
Differential Revision: https://reviews.facebook.net/D10311
2013-04-15 22:33:13 +02:00
|
|
|
class UtilityDB {
|
|
|
|
|
|
|
|
public:
|
|
|
|
// Open the database with TTL support.
|
|
|
|
//
|
|
|
|
// USE-CASES:
|
|
|
|
// This API should be used to open the db when key-values inserted are
|
|
|
|
// meant to be removed from the db in a non-strict 'ttl' amount of time
|
|
|
|
// Therefore, this guarantees that key-values inserted will remain in the
|
|
|
|
// db for >= ttl amount of time and the db will make efforts to remove the
|
|
|
|
// key-values as soon as possible after ttl seconds of their insertion.
|
|
|
|
//
|
|
|
|
// BEHAVIOUR:
|
|
|
|
// TTL is accepted in seconds
|
|
|
|
// (int32_t)Timestamp(creation) is suffixed to values in Put internally
|
|
|
|
// Expired TTL values deleted in compaction only:(Timestamp+ttl<time_now)
|
|
|
|
// Get/Iterator may return expired entries(compaction not run on them yet)
|
|
|
|
// Different TTL may be used during different Opens
|
|
|
|
// Example: Open1 at t=0 with ttl=4 and insert k1,k2, close at t=2
|
|
|
|
// Open2 at t=3 with ttl=5. Now k1,k2 should be deleted at t>=5
|
2013-05-10 23:19:47 +02:00
|
|
|
// read_only=true opens in the usual read-only mode. Compactions will not be
|
|
|
|
// triggered(neither manual nor automatic), so no expired entries removed
|
Timestamp and TTL Wrapper for rocksdb
Summary:
When opened with DBTimestamp::Open call, timestamps are prepended to and stripped from the value during subsequent Put and Get calls respectively. The Timestamp is used to discard values in Get and custom compaction filter which have exceeded their TTL which is specified during Open.
Have made a temporary change to Makefile to let us test with the temporary file TestTime.cc. Have also changed the private members of db_impl.h to protected to let them be inherited by the new class DBTimestamp
Test Plan: make db_timestamp; TestTime.cc(will not check it in) shows how to use the apis currently, but I will write unit-tests shortly
Reviewers: dhruba, vamsi, haobo, sheki, heyongqiang, vkrest
Reviewed By: vamsi
CC: zshao, xjin, vkrest, MarkCallaghan
Differential Revision: https://reviews.facebook.net/D10311
2013-04-15 22:33:13 +02:00
|
|
|
//
|
|
|
|
// CONSTRAINTS:
|
|
|
|
// Not specifying/passing or non-positive TTL behaves like TTL = infinity
|
|
|
|
//
|
|
|
|
// !!!WARNING!!!:
|
|
|
|
// Calling DB::Open directly to re-open a db created by this API will get
|
|
|
|
// corrupt values(timestamp suffixed) and no ttl effect will be there
|
|
|
|
// during the second Open, so use this API consistently to open the db
|
|
|
|
// Be careful when passing ttl with a small positive value because the
|
|
|
|
// whole database may be deleted in a small amount of time
|
|
|
|
static Status OpenTtlDB(const Options& options,
|
|
|
|
const std::string& name,
|
2013-08-06 02:55:44 +02:00
|
|
|
StackableDB** dbptr,
|
2013-05-10 23:19:47 +02:00
|
|
|
int32_t ttl = 0,
|
|
|
|
bool read_only = false);
|
2014-04-22 20:27:33 +02:00
|
|
|
|
|
|
|
// OpenTtlDB with column family support
|
|
|
|
static Status OpenTtlDB(
|
|
|
|
const DBOptions& db_options, const std::string& name,
|
|
|
|
const std::vector<ColumnFamilyDescriptor>& column_families,
|
|
|
|
std::vector<ColumnFamilyHandle*>* handles, StackableDB** dbptr,
|
|
|
|
std::vector<int32_t> ttls, bool read_only = false);
|
Timestamp and TTL Wrapper for rocksdb
Summary:
When opened with DBTimestamp::Open call, timestamps are prepended to and stripped from the value during subsequent Put and Get calls respectively. The Timestamp is used to discard values in Get and custom compaction filter which have exceeded their TTL which is specified during Open.
Have made a temporary change to Makefile to let us test with the temporary file TestTime.cc. Have also changed the private members of db_impl.h to protected to let them be inherited by the new class DBTimestamp
Test Plan: make db_timestamp; TestTime.cc(will not check it in) shows how to use the apis currently, but I will write unit-tests shortly
Reviewers: dhruba, vamsi, haobo, sheki, heyongqiang, vkrest
Reviewed By: vamsi
CC: zshao, xjin, vkrest, MarkCallaghan
Differential Revision: https://reviews.facebook.net/D10311
2013-04-15 22:33:13 +02:00
|
|
|
};
|
|
|
|
|
2013-10-04 06:49:15 +02:00
|
|
|
} // namespace rocksdb
|
2014-04-15 22:39:26 +02:00
|
|
|
#endif // ROCKSDB_LITE
|