andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Use the write amplification value calculated by RocksDB in benchmark.sh (#8915)

Browse Source 
Summary:
Currently, `benchmark.sh` computes write amplification itself; the patch
changes the script to use the value calculated by RocksDB (which is
printed as part of the periodic statistics). This also has the benefit
of being correct for BlobDB as well, since it also considers the amount
of data written to blob files.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/8915

Test Plan:
```
DB_DIR=/tmp/rocksdbtest/dbbench/ WAL_DIR=/tmp/rocksdbtest/dbbench/ NUM_KEYS=20000000 NUM_THREADS=32 tools/benchmark.sh overwrite --enable_blob_files=1 --enable_blob_garbage_collection=1

...

** Compaction Stats [default] **
Level    Files   Size     Score Read(GB)  Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB)
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  L0      7/5   43.93 MB   0.5      0.3     0.0      0.3       0.5      0.3       0.0   1.0      1.3     59.9    201.35            101.88       109    1.847     22M   499K       0.0      11.2
  L4      4/4   244.03 MB   0.0     11.4     0.3      1.6       1.6      0.0       0.0   1.1     50.6     49.3    231.10            288.84         7   33.014    156M    26M       9.5       9.5
  L5     36/0    3.28 GB   0.0      0.0     0.0      0.0       0.0      0.0       0.0   0.0      0.0      0.0      0.00              0.00         0    0.000       0      0       0.0       0.0
 Sum     47/9    3.56 GB   0.0     11.7     0.3      1.8       2.2      0.3       0.0   2.0     27.6     54.3    432.45            390.72       116    3.728    179M    26M       9.5      20.8
 Int      0/0    0.00 KB   0.0      3.5     0.1      0.5       0.6      0.1       0.0   2.2     31.2     55.6    115.01            109.53        29    3.966     51M  7353K       2.9       5.6

...

Completed overwrite (ID: ) in 289 seconds
ops/sec	mb/sec	Size-GB	L0_GB	Sum_GB	W-Amp	W-MB/s	usec/op	p50	p75	p99	p99.9	p99.99	Uptime	Stall-time	Stall%	Test	Date	Version	Job-ID
111784	44.8	0.0	0.5	2.2	2.0	9.2	285.9	215.3	264.4	1232	13299	23310	243	00:00:0.000	0.0	overwrite.t32.s0	2021-09-14T11:58:26.000-07:00	6.24
```

Reviewed By: zhichao-cao

Differential Revision: D30940352

Pulled By: ltamasi

fbshipit-source-id: ae7f5cd5440c8529788dda043266121fc2be0853
This commit is contained in:
Levi Tamasi 2021-09-15 12:15:47 -07:00 committed by Facebook GitHub Bot
parent 12d798ac06
commit 8df334342e
1 changed files with 1 additions and 8 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

9
tools/benchmark.sh
View File

@ -246,14 +246,7 @@ function summarize_result {
l0_wgb=$( grep "^ L0" $test_out | tail -1 | awk '{ print $9 }' ) l0_wgb=$( grep "^ L0" $test_out | tail -1 | awk '{ print $9 }' )
sum_wgb=$( grep "^ Sum" $test_out | tail -1 | awk '{ print $9 }' ) sum_wgb=$( grep "^ Sum" $test_out | tail -1 | awk '{ print $9 }' )
sum_size=$( grep "^ Sum" $test_out | tail -1 | awk '{ printf "%.1f", $3 / 1024.0 }' ) sum_size=$( grep "^ Sum" $test_out | tail -1 | awk '{ printf "%.1f", $3 / 1024.0 }' )
if [[ "$l0_wgb" == "" ]]; then wamp=$( grep "^ Sum" $test_out | tail -1 | awk '{ printf "%.1f", $12 }' )
l0_wgb="0.0"
fi
if [[ "$l0_wgb" == "0.0" ]]; then
wamp="0.0"
else
wamp=$( echo "scale=1; $sum_wgb / $l0_wgb" | bc )
fi
if [[ "$sum_wgb" == "" ]]; then if [[ "$sum_wgb" == "" ]]; then
wmb_ps="" wmb_ps=""
else else