andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Crash test to make kill decision for every kill point

Browse Source 
Summary:
In crash test, when coming to each kill point, we start a random class using seed as current second. With this approach, for every second, the random number used is the same. However, in each second, there are multiple kill points with different frequency. It makes it hard to reason about chance of kill point to trigger. With this commit, we use thread local random seed to generate the random number, so that it will take different values per second, hoping it makes chances of killing much easier to reason about.

Also significantly reduce the kill odd to make sure time before kiling is similar as before.

Test Plan: Run white box crash test and see the killing happens as expected and the run time time before killing reasonable.

Reviewers: kradhakrishnan, IslamAbdelRahman, rven, yhchiang, andrewkr, anthony

Reviewed By: anthony

Subscribers: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D52971
This commit is contained in:
sdong 2016-01-19 14:40:54 -08:00
parent 39c3e94ff4
commit fdbff42391
2 changed files with 7 additions and 9 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
tools/db_crashtest.py
View File

@ -236,7 +236,7 @@ def whitebox_crash_main(args):
total_check_mode = 4
check_mode = 0
kill_random_test = 97
kill_random_test = 888887
kill_mode = 0
while time.time() < exit_time:
@ -255,13 +255,13 @@ def whitebox_crash_main(args):
})
elif kill_mode == 1:
additional_opts.update({
"kill_random_test": (kill_random_test / 2 + 1),
"kill_random_test": (kill_random_test / 10 + 1),
"kill_prefix_blacklist": "WritableFileWriter::Append,"
+ "WritableFileWriter::WriteBuffered",
})
elif kill_mode == 2:
additional_opts.update({
"kill_random_test": (kill_random_test / 4 + 1),
"kill_random_test": (kill_random_test / 5000 + 1),
"kill_prefix_blacklist": "WritableFileWriter::Append,"
"WritableFileWriter::WriteBuffered,"
"PosixMmapFile::Allocate,WritableFileWriter::Flush",

10
util/sync_point.cc
View File

@ -21,16 +21,14 @@ void TestKillRandom(std::string kill_point, int odds,
}
}
time_t curtime = time(nullptr);
Random r((uint32_t)curtime);
assert(odds > 0);
if (odds % 7 == 0) {
// class Rarndom uses multiplier 16807, which is 7^5. If odds are
// multiplier of 7, the first random value might have limited values.
// class Random uses multiplier 16807, which is 7^5. If odds are
// multiplier of 7, there might be limited values generated.
odds++;
}
bool crash = r.OneIn(odds);
auto* r = Random::GetTLSInstance();
bool crash = r->OneIn(odds);
if (crash) {
port::Crash(srcfile, srcline);
}