rocksdb/utilities
gitbw95 f241d082b6 Prevent double caching in the compressed secondary cache (#9747)
Summary:
###  **Summary:**
When both LRU Cache and CompressedSecondaryCache are configured together, there possibly are some data blocks double cached.

**Changes include:**
1. Update IS_PROMOTED to IS_IN_SECONDARY_CACHE to prevent confusions.
2. This PR updates SecondaryCacheResultHandle and use IsErasedFromSecondaryCache to determine whether the handle is erased in the secondary cache. Then, the caller can determine whether to SetIsInSecondaryCache().
3. Rename LRUSecondaryCache to CompressedSecondaryCache.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/9747

Test Plan:
**Test Scripts:**
1. Populate a DB. The on disk footprint is 482 MB. The data is set to be 50% compressible, so the total decompressed size is expected to be 964 MB.
./db_bench --benchmarks=fillrandom --num=10000000 -db=/db_bench_1

2. overwrite it to a stable state:
./db_bench --benchmarks=overwrite,stats --num=10000000 -use_existing_db -duration=10 --benchmark_write_rate_limit=2000000 -db=/db_bench_1

4. Run read tests with diffeernt cache setting:

T1:
./db_bench --benchmarks=seekrandom,stats --threads=16 --num=10000000 -use_existing_db -duration=120 --benchmark_write_rate_limit=52000000 -use_direct_reads --cache_size=520000000  --statistics -db=/db_bench_1

T2:
./db_bench --benchmarks=seekrandom,stats --threads=16 --num=10000000 -use_existing_db -duration=120 --benchmark_write_rate_limit=52000000 -use_direct_reads --cache_size=320000000 -compressed_secondary_cache_size=400000000 --statistics -use_compressed_secondary_cache -db=/db_bench_1

T3:
./db_bench --benchmarks=seekrandom,stats --threads=16 --num=10000000 -use_existing_db -duration=120 --benchmark_write_rate_limit=52000000 -use_direct_reads --cache_size=520000000 -compressed_secondary_cache_size=400000000 --statistics -use_compressed_secondary_cache -db=/db_bench_1

T4:
./db_bench --benchmarks=seekrandom,stats --threads=16 --num=10000000 -use_existing_db -duration=120 --benchmark_write_rate_limit=52000000 -use_direct_reads --cache_size=20000000 -compressed_secondary_cache_size=500000000 --statistics -use_compressed_secondary_cache -db=/db_bench_1

**Before this PR**
| Cache Size | Compressed Secondary Cache Size | Cache Hit Rate |
|------------|-------------------------------------|----------------|
|520 MB | 0 MB | 85.5% |
|320 MB | 400 MB | 96.2% |
|520 MB | 400 MB | 98.3% |
|20 MB | 500 MB | 98.8% |

**Before this PR**
| Cache Size | Compressed Secondary Cache Size | Cache Hit Rate |
|------------|-------------------------------------|----------------|
|520 MB | 0 MB | 85.5% |
|320 MB | 400 MB | 99.9% |
|520 MB | 400 MB | 99.9% |
|20 MB | 500 MB | 99.2% |

Reviewed By: anand1976

Differential Revision: D35117499

Pulled By: gitbw95

fbshipit-source-id: ea2657749fc13efebe91a8a1b56bc61d6a224a12
2022-04-11 13:28:33 -07:00
..
backup Fix remaining uses of "backupable" (#9792) 2022-04-05 09:52:33 -07:00
blob_db Fix FileStorageInfo fields from GetLiveFilesMetaData (#9769) 2022-03-29 14:36:35 -07:00
cassandra Restore Regex support for ObjectLibrary::Register, rename new APIs to allow old one to be deprecated in the future (#9362) 2022-01-11 06:33:48 -08:00
checkpoint Add Temperature info in NewSequentialFile() (#9499) 2022-02-18 18:23:07 -08:00
compaction_filters Make MergeOperator+CompactionFilter/Factory into Customizable Classes (#8481) 2021-08-06 08:27:25 -07:00
convenience Add a SystemClock class to capture the time functions of an Env (#7858) 2021-01-25 22:09:11 -08:00
leveldb_options Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
memory Make types of Immutable/Mutable Options fields match that of the underlying Option (#8176) 2021-04-22 20:43:54 -07:00
merge_operators Remove using namespace (#9369) 2022-01-12 09:31:12 -08:00
option_change_migration Fix some typos in comments and HISTORY.md (#9798) 2022-04-04 09:32:57 -07:00
options Allow WAL dir to change with db dir (#8582) 2021-07-30 12:16:44 -07:00
persistent_cache Add rate limiter priority to ReadOptions (#9424) 2022-02-16 23:18:14 -08:00
simulator_cache Update Cache::Release param from force_erase to erase_if_last_ref (#9728) 2022-03-22 10:22:18 -07:00
table_properties_collectors Restore Regex support for ObjectLibrary::Register, rename new APIs to allow old one to be deprecated in the future (#9362) 2022-01-11 06:33:48 -08:00
trace Add rate limiter priority to ReadOptions (#9424) 2022-02-16 23:18:14 -08:00
transactions Disallow commit-time-batch for write-prepared/write-unprepared txn conditionally (#9794) 2022-04-05 11:10:20 -07:00
ttl Work around some new clang-analyze failures (#9515) 2022-02-07 18:24:36 -08:00
write_batch_with_index Support WBWI for keys having timestamps (#9603) 2022-02-22 14:23:01 -08:00
cache_dump_load_impl.cc New stable, fixed-length cache keys (#9126) 2021-12-16 17:15:13 -08:00
cache_dump_load_impl.h Add rate limiter priority to ReadOptions (#9424) 2022-02-16 23:18:14 -08:00
cache_dump_load.cc Introduce a mechanism to dump out blocks from block cache and re-insert to secondary cache (#8912) 2021-10-07 11:42:31 -07:00
compaction_filters.cc Restore Regex support for ObjectLibrary::Register, rename new APIs to allow old one to be deprecated in the future (#9362) 2022-01-11 06:33:48 -08:00
counted_fs.cc Introduce a CountedFileSystem for counting file operations (#9283) 2022-02-03 15:01:23 -08:00
counted_fs.h Introduce a CountedFileSystem for counting file operations (#9283) 2022-02-03 15:01:23 -08:00
debug.cc In ParseInternalKey(), include corrupt key info in Status (#7515) 2020-10-28 10:12:58 -07:00
env_mirror_test.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
env_mirror.cc Fix clang13 build error (#9374) 2022-01-11 10:36:22 -08:00
env_timed_test.cc Make env*_test work with ASSERT_STATUS_CHECKED (#7176) 2020-07-28 22:59:48 -07:00
env_timed.cc Make FileSystem a Customizable Class (#8649) 2021-11-02 09:07:11 -07:00
env_timed.h Make FileSystem a Customizable Class (#8649) 2021-11-02 09:07:11 -07:00
fault_injection_env.cc Protect existing files in FaultInjectionTest{Env,FS}::ReopenWritableFile() (#8995) 2021-10-11 16:23:18 -07:00
fault_injection_env.h Make the Env class Customizable (#9293) 2022-01-04 16:45:49 -08:00
fault_injection_fs.cc Skip directory fsync for filesystem btrfs (#8903) 2021-11-03 12:21:27 -07:00
fault_injection_fs.h Fix a bug causing duplicate trailing entries in WritableFile (buffered IO) (#9236) 2021-12-13 09:00:36 -08:00
fault_injection_secondary_cache.cc Prevent double caching in the compressed secondary cache (#9747) 2022-04-11 13:28:33 -07:00
fault_injection_secondary_cache.h Prevent double caching in the compressed secondary cache (#9747) 2022-04-11 13:28:33 -07:00
memory_allocators.h Make MemoryAllocator into a Customizable class (#8980) 2021-12-17 04:20:47 -08:00
merge_operators.cc Restore Regex support for ObjectLibrary::Register, rename new APIs to allow old one to be deprecated in the future (#9362) 2022-01-11 06:33:48 -08:00
merge_operators.h Make MergeOperator+CompactionFilter/Factory into Customizable Classes (#8481) 2021-08-06 08:27:25 -07:00
object_registry_test.cc Add support for decimals to PatternEntry (#9577) 2022-02-16 11:15:19 -08:00
object_registry.cc Make FilterPolicy Customizable (#9590) 2022-02-18 13:22:31 -08:00
util_merge_operators_test.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
wal_filter.cc Make WalFilter, SstPartitionerFactory, FileChecksumGenFactory, and TableProperties Customizable (#8638) 2021-09-28 05:32:02 -07:00