andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Implement missing Handler methods in ColumnFamilyCollector. (#8456)

Browse Source 
Summary:
When db is open as secondary, there are basically 2 step process:
1) Collect column families from wal log
2) Apply changes to Memtable
In case primary db is TransactionDB instance, wal log will contain some additional data, like noop, etc. ColumnFamilyCollector doesn't implement methods to handle these, so it fails to open a wal log written by TransactionDB. (Everything works fine with standard DB::Open).
Memtable recovery process knows how to handle such wal logs, so only missing piece seems to be ColumnFamilyCollector.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/8456

Reviewed By: ajkr

Differential Revision: D29455945

Pulled By: mrambacher

fbshipit-source-id: 5b29560fcbc008e17e95d0dc4b07558f3d63e26f
This commit is contained in:
Dmitry Vorobev 2021-07-12 09:04:40 -07:00 committed by Facebook GitHub Bot
parent bbdc4f2e9a
commit 0b75b22321
2 changed files with 44 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
db/db_impl/db_impl_secondary.h
View File

@ -278,6 +278,16 @@ class DBImplSecondary : public DBImpl {
return AddColumnFamilyId(column_family_id);
}
Status MarkBeginPrepare(bool) override { return Status::OK(); }
Status MarkEndPrepare(const Slice&) override { return Status::OK(); }
Status MarkRollback(const Slice&) override { return Status::OK(); }
Status MarkCommit(const Slice&) override { return Status::OK(); }
Status MarkNoop(bool) override { return Status::OK(); }
const std::unordered_set<uint32_t>& column_families() const {
return column_family_ids_;
}

34
db/db_secondary_test.cc
View File

@ -10,6 +10,7 @@
#include "db/db_impl/db_impl_secondary.h"
#include "db/db_test_util.h"
#include "port/stack_trace.h"
#include "rocksdb/utilities/transaction_db.h"
#include "test_util/sync_point.h"
#include "utilities/fault_injection_env.h"
@ -1115,6 +1116,39 @@ TEST_F(DBSecondaryTest, InconsistencyDuringCatchUp) {
Status s = db_secondary_->TryCatchUpWithPrimary();
ASSERT_TRUE(s.IsCorruption());
}
TEST_F(DBSecondaryTest, OpenWithTransactionDB) {
Options options = CurrentOptions();
options.create_if_missing = true;
// Destroy the DB to recreate as a TransactionDB.
Close();
Destroy(options, true);
// Create a TransactionDB.
TransactionDB* txn_db = nullptr;
TransactionDBOptions txn_db_opts;
ASSERT_OK(TransactionDB::Open(options, txn_db_opts, dbname_, &txn_db));
ASSERT_NE(txn_db, nullptr);
db_ = txn_db;
std::vector<std::string> cfs = {"new_CF"};
CreateColumnFamilies(cfs, options);
ASSERT_EQ(handles_.size(), 1);
WriteOptions wopts;
TransactionOptions txn_opts;
Transaction* txn1 = txn_db->BeginTransaction(wopts, txn_opts, nullptr);
ASSERT_NE(txn1, nullptr);
ASSERT_OK(txn1->Put(handles_[0], "k1", "v1"));
ASSERT_OK(txn1->Commit());
delete txn1;
options = CurrentOptions();
options.max_open_files = -1;
ASSERT_OK(TryOpenSecondary(options));
}
#endif //! ROCKSDB_LITE
} // namespace ROCKSDB_NAMESPACE