andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
7303d02bdf
rocksdb/cache
History
Zhichao Cao 7303d02bdf Use new Insert and Lookup APIs in table reader to support secondary cache (#8315) 
Summary:
Secondary cache is implemented to achieve the secondary cache tier for block cache. New Insert and Lookup APIs are introduced in https://github.com/facebook/rocksdb/issues/8271  . To support and use the secondary cache in block based table reader, this PR introduces the corresponding callback functions that will be used in secondary cache, and update the Insert and Lookup APIs accordingly.

benchmarking:
./db_bench --benchmarks="fillrandom" -num=1000000 -key_size=32 -value_size=256 -use_direct_io_for_flush_and_compaction=true -db=/tmp/rocks_t/db -partition_index_and_filters=true

./db_bench -db=/tmp/rocks_t/db -use_existing_db=true -benchmarks=readrandom -num=1000000 -key_size=32 -value_size=256 -use_direct_reads=true -cache_size=1073741824 -cache_numshardbits=5 -cache_index_and_filter_blocks=true -read_random_exp_range=17 -statistics -partition_index_and_filters=true -stats_dump_period_sec=30 -reads=50000000

master benchmarking results:
readrandom   :       3.923 micros/op 254881 ops/sec;   33.4 MB/s (23849796 of 50000000 found)
rocksdb.db.get.micros P50 : 2.820992 P95 : 5.636716 P99 : 16.450553 P100 : 8396.000000 COUNT : 50000000 SUM : 179947064

Current PR benchmarking results
readrandom   :       4.083 micros/op 244925 ops/sec;   32.1 MB/s (23849796 of 50000000 found)
rocksdb.db.get.micros P50 : 2.967687 P95 : 5.754916 P99 : 15.665912 P100 : 8213.000000 COUNT : 50000000 SUM : 187250053

About 3.8% throughput reduction.
P50: 5.2% increasing, P95, 2.09% increasing, P99 4.77% improvement

Pull Request resolved: https://github.com/facebook/rocksdb/pull/8315

Test Plan: added the testing case

Reviewed By: anand1976

Differential Revision: D28599774

Pulled By: zhichao-cao

fbshipit-source-id: 098c4df0d7327d3a546df7604b2f1602f13044ed
2021-05-21 18:29:12 -07:00
..
cache_bench_tool.cc Allow cache_bench/db_bench to use a custom secondary cache (#8312) 2021-05-19 15:26:18 -07:00
cache_bench.cc Allow cache_bench/db_bench to use a custom secondary cache (#8312) 2021-05-19 15:26:18 -07:00
cache_entry_roles.cc Use deleters to label cache entries and collect stats (#8297) 2021-05-19 16:51:13 -07:00
cache_entry_roles.h Use deleters to label cache entries and collect stats (#8297) 2021-05-19 16:51:13 -07:00
cache_entry_stats.h Use deleters to label cache entries and collect stats (#8297) 2021-05-19 16:51:13 -07:00
cache_helpers.h Use deleters to label cache entries and collect stats (#8297) 2021-05-19 16:51:13 -07:00
cache_test.cc Use deleters to label cache entries and collect stats (#8297) 2021-05-19 16:51:13 -07:00
cache.cc Refactor Option obj address from char* to void* (#8295) 2021-05-13 14:29:42 -07:00
clock_cache.cc Use deleters to label cache entries and collect stats (#8297) 2021-05-19 16:51:13 -07:00
clock_cache.h Change RocksDB License 2017-07-15 16:11:23 -07:00
lru_cache_test.cc Use new Insert and Lookup APIs in table reader to support secondary cache (#8315) 2021-05-21 18:29:12 -07:00
lru_cache.cc Use deleters to label cache entries and collect stats (#8297) 2021-05-19 16:51:13 -07:00
lru_cache.h Use deleters to label cache entries and collect stats (#8297) 2021-05-19 16:51:13 -07:00
sharded_cache.cc Use deleters to label cache entries and collect stats (#8297) 2021-05-19 16:51:13 -07:00
sharded_cache.h Use deleters to label cache entries and collect stats (#8297) 2021-05-19 16:51:13 -07:00