andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
d6aa8c49f8
rocksdb/tools
History
Levi Tamasi 8df334342e Use the write amplification value calculated by RocksDB in benchmark.sh (#8915) 
Summary:
Currently, `benchmark.sh` computes write amplification itself; the patch
changes the script to use the value calculated by RocksDB (which is
printed as part of the periodic statistics). This also has the benefit
of being correct for BlobDB as well, since it also considers the amount
of data written to blob files.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/8915

Test Plan:
```
DB_DIR=/tmp/rocksdbtest/dbbench/ WAL_DIR=/tmp/rocksdbtest/dbbench/ NUM_KEYS=20000000 NUM_THREADS=32 tools/benchmark.sh overwrite --enable_blob_files=1 --enable_blob_garbage_collection=1

...

** Compaction Stats [default] **
Level    Files   Size     Score Read(GB)  Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB)
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  L0      7/5   43.93 MB   0.5      0.3     0.0      0.3       0.5      0.3       0.0   1.0      1.3     59.9    201.35            101.88       109    1.847     22M   499K       0.0      11.2
  L4      4/4   244.03 MB   0.0     11.4     0.3      1.6       1.6      0.0       0.0   1.1     50.6     49.3    231.10            288.84         7   33.014    156M    26M       9.5       9.5
  L5     36/0    3.28 GB   0.0      0.0     0.0      0.0       0.0      0.0       0.0   0.0      0.0      0.0      0.00              0.00         0    0.000       0      0       0.0       0.0
 Sum     47/9    3.56 GB   0.0     11.7     0.3      1.8       2.2      0.3       0.0   2.0     27.6     54.3    432.45            390.72       116    3.728    179M    26M       9.5      20.8
 Int      0/0    0.00 KB   0.0      3.5     0.1      0.5       0.6      0.1       0.0   2.2     31.2     55.6    115.01            109.53        29    3.966     51M  7353K       2.9       5.6

...

Completed overwrite (ID: ) in 289 seconds
ops/sec	mb/sec	Size-GB	L0_GB	Sum_GB	W-Amp	W-MB/s	usec/op	p50	p75	p99	p99.9	p99.99	Uptime	Stall-time	Stall%	Test	Date	Version	Job-ID
111784	44.8	0.0	0.5	2.2	2.0	9.2	285.9	215.3	264.4	1232	13299	23310	243	00:00:0.000	0.0	overwrite.t32.s0	2021-09-14T11:58:26.000-07:00	6.24
```

Reviewed By: zhichao-cao

Differential Revision: D30940352

Pulled By: ltamasi

fbshipit-source-id: ae7f5cd5440c8529788dda043266121fc2be0853
2021-09-15 12:16:59 -07:00
..
advisor Update branch as "main" in tools/advisor/README.md (#8744) 2021-09-01 20:26:28 -07:00
block_cache_analyzer Make TraceRecord and Replayer public (#8611) 2021-08-11 19:32:46 -07:00
dump Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
rdb Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
analyze_txn_stress_test.sh Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
auto_sanity_test.sh Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
backup_db.sh Revamp check_format_compatible.sh (#8012) 2021-03-02 11:42:27 -08:00
benchmark_leveldb.sh Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
benchmark.sh Use the write amplification value calculated by RocksDB in benchmark.sh (#8915) 2021-09-15 12:16:59 -07:00
blob_dump.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
check_all_python.py Allow missing "unversioned" python, as in CentOS 8 (#6883) 2020-05-29 11:29:23 -07:00
check_format_compatible.sh Add 6.24 to the format compatibility checker (#8716) 2021-08-26 16:35:58 -07:00
CMakeLists.txt Mark dependencies as PRIVATE and fix missing dependencies in tools. (#6790) 2020-05-12 21:07:55 -07:00
db_bench_tool_test.cc Add support for the integrated BlobDB to db_bench (#7956) 2021-02-17 11:10:18 -08:00
db_bench_tool.cc Make Statistics a Customizable Class (#8637) 2021-09-10 09:47:39 -07:00
db_bench.cc Add (& fix) some simple source code checks (#8821) 2021-09-07 21:19:27 -07:00
db_crashtest.py Temporarily disable block-based filter when stress testing timestamp (#8703) 2021-08-24 19:04:58 -07:00
db_repl_stress.cc Migrate away from Travis+Linux+amd64 (#7791) 2020-12-22 00:20:57 -08:00
db_sanity_test.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
dbench_monitor Fix /bin/bash shebangs 2017-08-03 15:56:46 -07:00
Dockerfile adding docker build script and dockerfile 2015-05-22 16:03:39 -07:00
generate_random_db.sh Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
ingest_external_sst.sh Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
io_tracer_parser_test.cc Fix cmake build failure with gflags (#8324) 2021-06-01 14:43:15 -07:00
io_tracer_parser_tool.cc Add request_id in IODebugContext. (#8045) 2021-04-01 13:14:51 -07:00
io_tracer_parser_tool.h Add IO Tracer Parser (#7333) 2020-09-23 15:50:26 -07:00
io_tracer_parser.cc Add IO Tracer Parser (#7333) 2020-09-23 15:50:26 -07:00
ldb_cmd_impl.h Some fixes and enhancements to ldb repair (#8544) 2021-07-28 16:44:14 -07:00
ldb_cmd_test.cc Fix flaky, dubious LdbCmdTest::*DumpFileChecksum* (#8898) 2021-09-13 17:07:21 -07:00
ldb_cmd.cc Fix flaky, dubious LdbCmdTest::*DumpFileChecksum* (#8898) 2021-09-13 17:07:21 -07:00
ldb_test.py Add list live files metadata (#8446) 2021-06-22 19:07:46 -07:00
ldb_tool.cc Add list live files metadata (#8446) 2021-06-22 19:07:46 -07:00
ldb.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
pflag Fix /bin/bash shebangs 2017-08-03 15:56:46 -07:00
reduce_levels_test.cc Remove some unneeded code (#8736) 2021-09-01 14:28:58 -07:00
regression_test.sh Fix regression test script (#8753) 2021-09-03 19:05:33 -07:00
restore_db.sh Revamp check_format_compatible.sh (#8012) 2021-03-02 11:42:27 -08:00
rocksdb_dump_test.sh Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
run_flash_bench.sh Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
run_leveldb.sh Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
sample-dump.dmp First version of rocksdb_dump and rocksdb_undump. 2015-06-19 16:24:36 -07:00
simulated_hybrid_file_system.cc Change the File System File Wrappers to std::unique_ptr (#8618) 2021-09-13 08:46:19 -07:00
simulated_hybrid_file_system.h Change the File System File Wrappers to std::unique_ptr (#8618) 2021-09-13 08:46:19 -07:00
sst_dump_test.cc Add CreateFrom methods to Env/FileSystem (#8174) 2021-06-15 03:43:48 -07:00
sst_dump_tool.cc Add CreateFrom methods to Env/FileSystem (#8174) 2021-06-15 03:43:48 -07:00
sst_dump.cc Implement a new subcommand "identify" for sst_dump (#6943) 2020-06-08 13:58:28 -07:00
trace_analyzer_test.cc Simplify TraceAnalyzer (#8697) 2021-08-24 18:18:36 -07:00
trace_analyzer_tool.cc Simplify TraceAnalyzer (#8697) 2021-08-24 18:18:36 -07:00
trace_analyzer_tool.h Simplify TraceAnalyzer (#8697) 2021-08-24 18:18:36 -07:00
trace_analyzer.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
verify_random_db.sh Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
write_external_sst.sh Revamp check_format_compatible.sh (#8012) 2021-03-02 11:42:27 -08:00
write_stress_runner.py Allow missing "unversioned" python, as in CentOS 8 (#6883) 2020-05-29 11:29:23 -07:00
write_stress.cc Add a SystemClock class to capture the time functions of an Env (#7858) 2021-01-25 22:09:11 -08:00