andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

db_stress: deep clean directory before checkpoint (#7039)

Browse Source 
Summary:
We see crash test occassionally fails with "A checkpoint operation failed with: Invalid argument: Directory exists". The suspicious is that the directory fails to be deleted because some trash files. Deep clean the directory after a DestroyDB() call.

Also add more debugging printf in case it fails.
Also, preserve the DB if verification fails.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/7039

Test Plan: Run db_stress with low --checkpoint_one_in value

Reviewed By: riversand963

Differential Revision: D22271694

fbshipit-source-id: 6a9b2abb664fc69a4dc666741df4f6b23703cd6d
This commit is contained in:
sdong 2020-06-30 12:00:32 -07:00 committed by Facebook GitHub Bot
parent 5be2cb6948
commit 2d1d51d385
1 changed files with 26 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

28
db_stress_tool/db_stress_test_base.cc
View File

@ -1335,10 +1335,34 @@ Status StressTest::TestCheckpoint(ThreadState* thread,
DestroyDB(checkpoint_dir, tmp_opts); DestroyDB(checkpoint_dir, tmp_opts);
if (db_stress_env->FileExists(checkpoint_dir).ok()) {
// If the directory might still exist, try to delete the files one by one.
// Likely a trash file is still there.
Status my_s = test::DestroyDir(db_stress_env, checkpoint_dir);
if (!my_s.ok()) {
fprintf(stderr, "Fail to destory directory before checkpoint: %s",
my_s.ToString().c_str());
}
}
Checkpoint* checkpoint = nullptr; Checkpoint* checkpoint = nullptr;
Status s = Checkpoint::Create(db_, &checkpoint); Status s = Checkpoint::Create(db_, &checkpoint);
if (s.ok()) { if (s.ok()) {
s = checkpoint->CreateCheckpoint(checkpoint_dir); s = checkpoint->CreateCheckpoint(checkpoint_dir);
if (!s.ok()) {
fprintf(stderr, "Fail to create checkpoint to %s\n",
checkpoint_dir.c_str());
std::vector<std::string> files;
Status my_s = db_stress_env->GetChildren(checkpoint_dir, &files);
if (my_s.ok()) {
for (const auto& f : files) {
fprintf(stderr, " %s\n", f.c_str());
}
} else {
fprintf(stderr, "Fail to get files under the directory to %s\n",
my_s.ToString().c_str());
}
}
} }
std::vector<ColumnFamilyHandle*> cf_handles; std::vector<ColumnFamilyHandle*> cf_handles;
DB* checkpoint_db = nullptr; DB* checkpoint_db = nullptr;
@ -1391,11 +1415,11 @@ Status StressTest::TestCheckpoint(ThreadState* thread,
checkpoint_db = nullptr; checkpoint_db = nullptr;
} }
DestroyDB(checkpoint_dir, tmp_opts);
if (!s.ok()) { if (!s.ok()) {
fprintf(stderr, "A checkpoint operation failed with: %s\n", fprintf(stderr, "A checkpoint operation failed with: %s\n",
s.ToString().c_str()); s.ToString().c_str());
} else {
DestroyDB(checkpoint_dir, tmp_opts);
} }
return s; return s;
} }