andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
rocksdb/tools
History
haoyuhuang bcfc53b436 Block cache tracing: Fix minor bugs with downsampling and some benchmark results. (#5473) 
Summary:
As the code changes for block cache tracing are almost complete, I did a benchmark to compare the performance when block cache tracing is enabled/disabled.

 With 1% downsampling ratio, the performance overhead of block cache tracing is negligible. When we trace all block accesses, the throughput drops by 6 folds with 16 threads issuing random reads and all reads are served in block cache.

Setup:
RocksDB:    version 6.2
Date:       Mon Jun 17 17:11:13 2019
CPU:        24 * Intel Core Processor (Skylake)
CPUCache:   16384 KB
Keys:       20 bytes each
Values:     100 bytes each (100 bytes after compression)
Entries:    10000000
Prefix:    20 bytes
Keys per prefix:    0
RawSize:    1144.4 MB (estimated)
FileSize:   1144.4 MB (estimated)
Write rate: 0 bytes/second
Read rate: 0 ops/second
Compression: NoCompression
Compression sampling rate: 0
Memtablerep: skip_list
Perf Level: 1

I ran the readrandom workload for 1 minute. Detailed throughput results:  (ops/second)
Sample rate 0: no block cache tracing.
Sample rate 1: trace all block accesses.
Sample rate 100: trace accesses 1% blocks.
1 thread |   |   |  -- | -- | -- | --
Sample rate | 0 | 1 | 100
1 MB block cache size | 13,094 | 13,166 | 13,341
10 GB block cache size | 202,243 | 188,677 | 229,182

16 threads |   |   |  -- | -- | -- | --
Sample rate | 0 | 1 | 100
1 MB block cache size | 208,761 | 178,700 | 201,872
10 GB block cache size | 2,645,996 | 426,295 | 2,587,605
Pull Request resolved: https://github.com/facebook/rocksdb/pull/5473

Differential Revision: D15869479

Pulled By: HaoyuHuang

fbshipit-source-id: 7ae802abe84811281a6af8649f489887cd7c4618
2019-06-17 17:59:02 -07:00
..
advisor
Rules Advisor: some fixes to support fetching stats from ODS (#4223)
2018-08-02 15:42:42 -07:00
dump
simplify include directive involving inttypes (#5402)
2019-06-06 13:56:07 -07:00
rdb
Add copyright headers per FB open-source checkup tool. (#5199)
2019-04-18 10:55:01 -07:00
analyze_txn_stress_test.sh
Add copyright headers per FB open-source checkup tool. (#5199)
2019-04-18 10:55:01 -07:00
auto_sanity_test.sh
Add copyright headers per FB open-source checkup tool. (#5199)
2019-04-18 10:55:01 -07:00
benchmark_leveldb.sh
Add copyright headers per FB open-source checkup tool. (#5199)
2019-04-18 10:55:01 -07:00
benchmark.sh
Add copyright headers per FB open-source checkup tool. (#5199)
2019-04-18 10:55:01 -07:00
blob_dump.cc
comment unused parameters to turn on -Wunused-parameter flag
2018-04-12 17:59:16 -07:00
block_cache_trace_analyzer_test.cc
Support computing miss ratio curves using sim_cache. (#5449)
2019-06-17 16:41:12 -07:00
block_cache_trace_analyzer_tool.cc
Support computing miss ratio curves using sim_cache. (#5449)
2019-06-17 16:41:12 -07:00
block_cache_trace_analyzer.cc
Block cache tracing: Fix minor bugs with downsampling and some benchmark results. (#5473)
2019-06-17 17:59:02 -07:00
block_cache_trace_analyzer.h
Support computing miss ratio curves using sim_cache. (#5449)
2019-06-17 16:41:12 -07:00
check_format_compatible.sh
Add copyright headers per FB open-source checkup tool. (#5199)
2019-04-18 10:55:01 -07:00
CMakeLists.txt
cmake support for linux and osx (#1358)
2016-09-28 11:53:15 -07:00
db_bench_tool_test.cc
Move some memory related files from util/ to memory/ (#5382)
2019-05-30 17:44:09 -07:00
db_bench_tool.cc
Persistent Stats: persist stats history to disk (#5046)
2019-06-17 15:21:50 -07:00
db_bench.cc
simplify include directive involving inttypes (#5402)
2019-06-06 13:56:07 -07:00
db_crashtest.py
Disable pipeline writes in stress test (#5445)
2019-06-12 11:12:36 -07:00
db_repl_stress.cc
Move some memory related files from util/ to memory/ (#5382)
2019-05-30 17:44:09 -07:00
db_sanity_test.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
db_stress.cc
simplify include directive involving inttypes (#5402)
2019-06-06 13:56:07 -07:00
dbench_monitor
Fix /bin/bash shebangs
2017-08-03 15:56:46 -07:00
Dockerfile
adding docker build script and dockerfile
2015-05-22 16:03:39 -07:00
generate_random_db.sh
Add copyright headers per FB open-source checkup tool. (#5199)
2019-04-18 10:55:01 -07:00
ingest_external_sst.sh
Add copyright headers per FB open-source checkup tool. (#5199)
2019-04-18 10:55:01 -07:00
ldb_cmd_impl.h
Add SST ingestion to ldb (#4205)
2018-08-09 14:29:11 -07:00
ldb_cmd_test.cc
Move test related files under util/ to test_util/ (#5377)
2019-05-30 11:25:51 -07:00
ldb_cmd.cc
Integrate block cache tracer into db_impl (#5433)
2019-06-13 15:43:10 -07:00
ldb_test.py
Add copyright headers per FB open-source checkup tool. (#5199)
2019-04-18 10:55:01 -07:00
ldb_tool.cc
Add SST ingestion to ldb (#4205)
2018-08-09 14:29:11 -07:00
ldb.cc
comment unused parameters to turn on -Wunused-parameter flag
2018-04-12 17:59:16 -07:00
pflag
Fix /bin/bash shebangs
2017-08-03 15:56:46 -07:00
reduce_levels_test.cc
Organizing rocksdb/db directory
2019-05-31 11:57:01 -07:00
regression_test.sh
Add copyright headers per FB open-source checkup tool. (#5199)
2019-04-18 10:55:01 -07:00
report_lite_binary_size.sh
Add copyright headers per FB open-source checkup tool. (#5199)
2019-04-18 10:55:01 -07:00
rocksdb_dump_test.sh
Add copyright headers per FB open-source checkup tool. (#5199)
2019-04-18 10:55:01 -07:00
run_flash_bench.sh
Add copyright headers per FB open-source checkup tool. (#5199)
2019-04-18 10:55:01 -07:00
run_leveldb.sh
Add copyright headers per FB open-source checkup tool. (#5199)
2019-04-18 10:55:01 -07:00
sample-dump.dmp
First version of rocksdb_dump and rocksdb_undump.
2015-06-19 16:24:36 -07:00
sst_dump_test.cc
Move some memory related files from util/ to memory/ (#5382)
2019-05-30 17:44:09 -07:00
sst_dump_tool_imp.h
tools: use provided options instead of the default (#4839)
2019-01-03 11:23:49 -08:00
sst_dump_tool.cc
simplify include directive involving inttypes (#5402)
2019-06-06 13:56:07 -07:00
sst_dump.cc
comment unused parameters to turn on -Wunused-parameter flag
2018-04-12 17:59:16 -07:00
trace_analyzer_test.cc
Move util/trace_replay.* to trace_replay/ (#5376)
2019-06-03 13:25:26 -07:00
trace_analyzer_tool.cc
simplify include directive involving inttypes (#5402)
2019-06-06 13:56:07 -07:00
trace_analyzer_tool.h
Move util/trace_replay.* to trace_replay/ (#5376)
2019-06-03 13:25:26 -07:00
trace_analyzer.cc
RocksDB Trace Analyzer (#4091)
2018-08-13 11:44:02 -07:00
verify_random_db.sh
Add copyright headers per FB open-source checkup tool. (#5199)
2019-04-18 10:55:01 -07:00
write_external_sst.sh
Add copyright headers per FB open-source checkup tool. (#5199)
2019-04-18 10:55:01 -07:00
write_stress_runner.py
Add copyright headers per FB open-source checkup tool. (#5199)
2019-04-18 10:55:01 -07:00
write_stress.cc
simplify include directive involving inttypes (#5402)
2019-06-06 13:56:07 -07:00