andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
f241d082b6
rocksdb/cache
History
gitbw95 f241d082b6 Prevent double caching in the compressed secondary cache (#9747) 
Summary:
###  **Summary:**
When both LRU Cache and CompressedSecondaryCache are configured together, there possibly are some data blocks double cached.

**Changes include:**
1. Update IS_PROMOTED to IS_IN_SECONDARY_CACHE to prevent confusions.
2. This PR updates SecondaryCacheResultHandle and use IsErasedFromSecondaryCache to determine whether the handle is erased in the secondary cache. Then, the caller can determine whether to SetIsInSecondaryCache().
3. Rename LRUSecondaryCache to CompressedSecondaryCache.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/9747

Test Plan:
**Test Scripts:**
1. Populate a DB. The on disk footprint is 482 MB. The data is set to be 50% compressible, so the total decompressed size is expected to be 964 MB.
./db_bench --benchmarks=fillrandom --num=10000000 -db=/db_bench_1

2. overwrite it to a stable state:
./db_bench --benchmarks=overwrite,stats --num=10000000 -use_existing_db -duration=10 --benchmark_write_rate_limit=2000000 -db=/db_bench_1

4. Run read tests with diffeernt cache setting:

T1:
./db_bench --benchmarks=seekrandom,stats --threads=16 --num=10000000 -use_existing_db -duration=120 --benchmark_write_rate_limit=52000000 -use_direct_reads --cache_size=520000000  --statistics -db=/db_bench_1

T2:
./db_bench --benchmarks=seekrandom,stats --threads=16 --num=10000000 -use_existing_db -duration=120 --benchmark_write_rate_limit=52000000 -use_direct_reads --cache_size=320000000 -compressed_secondary_cache_size=400000000 --statistics -use_compressed_secondary_cache -db=/db_bench_1

T3:
./db_bench --benchmarks=seekrandom,stats --threads=16 --num=10000000 -use_existing_db -duration=120 --benchmark_write_rate_limit=52000000 -use_direct_reads --cache_size=520000000 -compressed_secondary_cache_size=400000000 --statistics -use_compressed_secondary_cache -db=/db_bench_1

T4:
./db_bench --benchmarks=seekrandom,stats --threads=16 --num=10000000 -use_existing_db -duration=120 --benchmark_write_rate_limit=52000000 -use_direct_reads --cache_size=20000000 -compressed_secondary_cache_size=500000000 --statistics -use_compressed_secondary_cache -db=/db_bench_1

**Before this PR**
| Cache Size | Compressed Secondary Cache Size | Cache Hit Rate |
|------------|-------------------------------------|----------------|
|520 MB | 0 MB | 85.5% |
|320 MB | 400 MB | 96.2% |
|520 MB | 400 MB | 98.3% |
|20 MB | 500 MB | 98.8% |

**Before this PR**
| Cache Size | Compressed Secondary Cache Size | Cache Hit Rate |
|------------|-------------------------------------|----------------|
|520 MB | 0 MB | 85.5% |
|320 MB | 400 MB | 99.9% |
|520 MB | 400 MB | 99.9% |
|20 MB | 500 MB | 99.2% |

Reviewed By: anand1976

Differential Revision: D35117499

Pulled By: gitbw95

fbshipit-source-id: ea2657749fc13efebe91a8a1b56bc61d6a224a12
2022-04-11 13:28:33 -07:00
..
cache_bench_tool.cc Change type of cache buffer passed to Cache::CreateCallback() to const void* (#9595) 2022-02-17 21:09:56 -08:00
cache_bench.cc Add (& fix) some simple source code checks (#8821) 2021-09-07 21:19:27 -07:00
cache_entry_roles.cc Account memory of big memory users in BlockBasedTable in global memory limit (#9748) 2022-04-06 10:33:00 -07:00
cache_entry_roles.h Account memory of big memory users in BlockBasedTable in global memory limit (#9748) 2022-04-06 10:33:00 -07:00
cache_entry_stats.h New stable, fixed-length cache keys (#9126) 2021-12-16 17:15:13 -08:00
cache_helpers.h Use deleters to label cache entries and collect stats (#8297) 2021-05-19 16:51:13 -07:00
cache_key.cc Enhance new cache key testing & comments (#9329) 2022-02-04 14:15:58 -08:00
cache_key.h New stable, fixed-length cache keys (#9126) 2021-12-16 17:15:13 -08:00
cache_reservation_manager_test.cc Account memory of big memory users in BlockBasedTable in global memory limit (#9748) 2022-04-06 10:33:00 -07:00
cache_reservation_manager.cc Account memory of big memory users in BlockBasedTable in global memory limit (#9748) 2022-04-06 10:33:00 -07:00
cache_reservation_manager.h Account memory of big memory users in BlockBasedTable in global memory limit (#9748) 2022-04-06 10:33:00 -07:00
cache_test.cc Fix/improve 'must free heap allocations' code (#9209) 2021-11-29 10:53:52 -08:00
cache.cc Make SecondaryCache Customizable (#8480) 2021-07-06 09:18:08 -07:00
clock_cache.cc Update Cache::Release param from force_erase to erase_if_last_ref (#9728) 2022-03-22 10:22:18 -07:00
clock_cache.h Change RocksDB License 2017-07-15 16:11:23 -07:00
compressed_secondary_cache_test.cc Prevent double caching in the compressed secondary cache (#9747) 2022-04-11 13:28:33 -07:00
compressed_secondary_cache.cc Prevent double caching in the compressed secondary cache (#9747) 2022-04-11 13:28:33 -07:00
compressed_secondary_cache.h Prevent double caching in the compressed secondary cache (#9747) 2022-04-11 13:28:33 -07:00
lru_cache_test.cc Prevent double caching in the compressed secondary cache (#9747) 2022-04-11 13:28:33 -07:00
lru_cache.cc Prevent double caching in the compressed secondary cache (#9747) 2022-04-11 13:28:33 -07:00
lru_cache.h Prevent double caching in the compressed secondary cache (#9747) 2022-04-11 13:28:33 -07:00
sharded_cache.cc Update Cache::Release param from force_erase to erase_if_last_ref (#9728) 2022-03-22 10:22:18 -07:00
sharded_cache.h Update Cache::Release param from force_erase to erase_if_last_ref (#9728) 2022-03-22 10:22:18 -07:00