rocksdb/tools
Andrew Kryczka a4fb1f8c04 Add crash-recovery correctness check to db_stress
Summary:
Previously, our `db_stress` tool held the expected state of the DB in-memory, so after crash-recovery, there was no way to verify data correctness. This PR adds an option, `--expected_values_file`, which specifies a file holding the expected values.

In black-box testing, the `db_stress` process can be killed arbitrarily, so updates to the `--expected_values_file` must be atomic. We achieve this by `mmap`ing the file and relying on `std::atomic<uint32_t>` for atomicity. Actually this doesn't provide a total guarantee on what we want as `std::atomic<uint32_t>` could, in theory, be translated into multiple stores surrounded by a mutex. We can verify our assumption by looking at `std::atomic::is_always_lock_free`.

For the `mmap`'d file, we didn't have an existing way to expose its contents as a raw memory buffer. This PR adds it in the `Env::NewMemoryMappedFileBuffer` function, and `MemoryMappedFileBuffer` class.

`db_crashtest.py` is updated to use an expected values file for black-box testing. On the first iteration (when the DB is created), an empty file is provided as `db_stress` will populate it when it runs. On subsequent iterations, that same filename is provided so `db_stress` can check the data is as expected on startup.
Closes https://github.com/facebook/rocksdb/pull/3629

Differential Revision: D7463144

Pulled By: ajkr

fbshipit-source-id: c8f3e82c93e045a90055e2468316be155633bd8b
2018-04-24 15:58:22 -07:00
..
dump fix gflags namespace 2017-12-01 10:42:05 -08:00
rdb Fix /bin/bash shebangs 2017-08-03 15:56:46 -07:00
auto_sanity_test.sh Suppress lint in old files 2018-01-29 12:56:42 -08:00
benchmark_leveldb.sh Suppress lint in old files 2018-01-29 12:56:42 -08:00
benchmark.sh benchmark.sh to use --max_background_job 2018-03-20 18:57:55 -07:00
blob_dump.cc comment unused parameters to turn on -Wunused-parameter flag 2018-04-12 17:59:16 -07:00
check_format_compatible.sh Add 5.11 and 5.12 to tools/check_format_compatible.sh 2018-03-23 12:43:06 -07:00
CMakeLists.txt cmake support for linux and osx (#1358) 2016-09-28 11:53:15 -07:00
db_bench_tool_test.cc fix gflags namespace 2017-12-01 10:42:05 -08:00
db_bench_tool.cc Support lowering CPU priority of background threads 2018-04-24 08:41:51 -07:00
db_bench.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_crashtest.py Add crash-recovery correctness check to db_stress 2018-04-24 15:58:22 -07:00
db_repl_stress.cc comment unused parameters to turn on -Wunused-parameter flag 2018-04-12 17:59:16 -07:00
db_sanity_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_stress.cc Add crash-recovery correctness check to db_stress 2018-04-24 15:58:22 -07:00
dbench_monitor Fix /bin/bash shebangs 2017-08-03 15:56:46 -07:00
Dockerfile adding docker build script and dockerfile 2015-05-22 16:03:39 -07:00
generate_random_db.sh Fix /bin/bash shebangs 2017-08-03 15:56:46 -07:00
ldb_cmd_impl.h WritePrepared Txn: add write_committed option to dump_wal 2018-04-07 21:56:42 -07:00
ldb_cmd_test.cc fix memory leak in two_level_iterator 2018-04-15 17:26:26 -07:00
ldb_cmd.cc comment unused parameters to turn on -Wunused-parameter flag 2018-04-12 17:59:16 -07:00
ldb_test.py Suppress lint in old files 2018-01-29 12:56:42 -08:00
ldb_tool.cc Comment out unused variables 2018-03-05 13:13:41 -08:00
ldb.cc comment unused parameters to turn on -Wunused-parameter flag 2018-04-12 17:59:16 -07:00
pflag Fix /bin/bash shebangs 2017-08-03 15:56:46 -07:00
reduce_levels_test.cc fix memory leak in two_level_iterator 2018-04-15 17:26:26 -07:00
regression_test.sh Suppress lint in old files 2018-01-29 12:56:42 -08:00
report_lite_binary_size.sh Legocastle job to report lite build binary size to scuba 2018-02-15 17:27:24 -08:00
rocksdb_dump_test.sh Suppress lint in old files 2018-01-29 12:56:42 -08:00
run_flash_bench.sh Fix /bin/bash shebangs 2017-08-03 15:56:46 -07:00
run_leveldb.sh Fix /bin/bash shebangs 2017-08-03 15:56:46 -07:00
sample-dump.dmp First version of rocksdb_dump and rocksdb_undump. 2015-06-19 16:24:36 -07:00
sst_dump_test.cc fix memory leak in two_level_iterator 2018-04-15 17:26:26 -07:00
sst_dump_tool_imp.h make sst_dump compression size command consistent 2017-08-11 16:03:44 -07:00
sst_dump_tool.cc Comment out unused variables 2018-03-05 13:13:41 -08:00
sst_dump.cc comment unused parameters to turn on -Wunused-parameter flag 2018-04-12 17:59:16 -07:00
verify_random_db.sh Fix /bin/bash shebangs 2017-08-03 15:56:46 -07:00
write_stress_runner.py Suppress lint in old files 2018-01-29 12:56:42 -08:00
write_stress.cc Compilation fixes for powerpc build, -Wparentheses-equality error and missing header guards 2018-02-09 14:12:43 -08:00