andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Adding a method for memtable class for memtable getting flushed. (#4304)

Browse Source 
Summary:
Memtables are selected for flushing by the flush job. Currently we
have listener which is invoked when memtables for a column family are
flushed. That listener does not indicate which memtable was flushed in
the notification. If clients want to know if particular data in the
memtable was retired, there is no straight forward way to know this.
This method will help users who implement memtablerep factory and extend
interface for memtablerep, to know if the data in the memtable was
retired.
Another option that was tried, was to depend on memtable destructor to
be called after flush to mark that data was persisted. This works all
the time but sometimes there can huge delays between actual flush
happening and memtable getting destroyed. Hence, if anyone who is
waiting for data to persist will have to wait that longer.
It is expected that anyone who is implementing this method to have
return quickly as it blocks RocksDB.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/4304

Reviewed By: riversand963

Differential Revision: D9472312

Pulled By: gdrane

fbshipit-source-id: 8e693308dee749586af3a4c5d4fcf1fa5276ea4d
This commit is contained in:
Gauresh Rane 2018-08-23 17:02:06 -07:00 committed by Facebook Github Bot
parent da40d45267
commit ad789e4e0d
3 changed files with 17 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
db/memtable.h
View File

@ -336,6 +336,14 @@ class MemTable {
mem_tracker_.DoneAllocating(); mem_tracker_.DoneAllocating();
} }
// Notify the underlying storage that all data it contained has been
// persisted.
// REQUIRES: external synchronization to prevent simultaneous
// operations on the same MemTable.
void MarkFlushed() {
table_->MarkFlushed();
}
// return true if the current MemTableRep supports merge operator. // return true if the current MemTableRep supports merge operator.
bool IsMergeOperatorSupported() const { bool IsMergeOperatorSupported() const {
return table_->IsMergeOperatorSupported(); return table_->IsMergeOperatorSupported();

1
db/memtable_list.cc
View File

@ -248,6 +248,7 @@ void MemTableListVersion::Remove(MemTable* m,
assert(refs_ == 1); // only when refs_ == 1 is MemTableListVersion mutable assert(refs_ == 1); // only when refs_ == 1 is MemTableListVersion mutable
memlist_.remove(m); memlist_.remove(m);
m->MarkFlushed();
if (max_write_buffer_number_to_maintain_ > 0) { if (max_write_buffer_number_to_maintain_ > 0) {
memlist_history_.push_front(m); memlist_history_.push_front(m);
TrimHistory(to_delete); TrimHistory(to_delete);

8
include/rocksdb/memtablerep.h
View File

@ -144,6 +144,14 @@ class MemTableRep {
// or any writes done directly to entries accessed through the iterator.) // or any writes done directly to entries accessed through the iterator.)
virtual void MarkReadOnly() { } virtual void MarkReadOnly() { }
// Notify this table rep that it has been flushed to stable storage.
// By default, does nothing.
//
// Invariant: MarkReadOnly() is called, before MarkFlushed().
// Note that this method if overridden, should not run for an extended period
// of time. Otherwise, RocksDB may be blocked.
virtual void MarkFlushed() { }
// Look up key from the mem table, since the first key in the mem table whose // Look up key from the mem table, since the first key in the mem table whose
// user_key matches the one given k, call the function callback_func(), with // user_key matches the one given k, call the function callback_func(), with
// callback_args directly forwarded as the first parameter, and the mem table // callback_args directly forwarded as the first parameter, and the mem table