5d660258e7
Summary: add class SimCache(base class with instrumentation api) and SimLRUCache(derived class with detailed implementation) which is used as an instrumented block cache that can predict hit rate for different cache size Test Plan: Add a test case in `db_block_cache_test.cc` called `SimCacheTest` to test basic logic of SimCache. Also add option `-simcache_size` in db_bench. if set with a value other than -1, then the benchmark will use this value as the size of the simulator cache and finally output the simulation result. ``` [gzh@dev9927.prn1 ~/local/rocksdb] ./db_bench -benchmarks "fillseq,readrandom" -cache_size 1000000 -simcache_size 1000000 RocksDB: version 4.8 Date: Tue May 17 16:56:16 2016 CPU: 32 * Intel(R) Xeon(R) CPU E5-2660 0 @ 2.20GHz CPUCache: 20480 KB Keys: 16 bytes each Values: 100 bytes each (50 bytes after compression) Entries: 1000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 110.6 MB (estimated) FileSize: 62.9 MB (estimated) Write rate: 0 bytes/second Compression: Snappy Memtablerep: skip_list Perf Level: 0 WARNING: Assertions are enabled; benchmarks unnecessarily slow ------------------------------------------------ DB path: [/tmp/rocksdbtest-112628/dbbench] fillseq : 6.809 micros/op 146874 ops/sec; 16.2 MB/s DB path: [/tmp/rocksdbtest-112628/dbbench] readrandom : 6.343 micros/op 157665 ops/sec; 17.4 MB/s (1000000 of 1000000 found) SIMULATOR CACHE STATISTICS: SimCache LOOKUPs: 986559 SimCache HITs: 264760 SimCache HITRATE: 26.84% [gzh@dev9927.prn1 ~/local/rocksdb] ./db_bench -benchmarks "fillseq,readrandom" -cache_size 1000000 -simcache_size 10000000 RocksDB: version 4.8 Date: Tue May 17 16:57:10 2016 CPU: 32 * Intel(R) Xeon(R) CPU E5-2660 0 @ 2.20GHz CPUCache: 20480 KB Keys: 16 bytes each Values: 100 bytes each (50 bytes after compression) Entries: 1000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 110.6 MB (estimated) FileSize: 62.9 MB (estimated) Write rate: 0 bytes/second Compression: Snappy Memtablerep: skip_list Perf Level: 0 WARNING: Assertions are enabled; benchmarks unnecessarily slow ------------------------------------------------ DB path: [/tmp/rocksdbtest-112628/dbbench] fillseq : 5.066 micros/op 197394 ops/sec; 21.8 MB/s DB path: [/tmp/rocksdbtest-112628/dbbench] readrandom : 6.457 micros/op 154870 ops/sec; 17.1 MB/s (1000000 of 1000000 found) SIMULATOR CACHE STATISTICS: SimCache LOOKUPs: 1059764 SimCache HITs: 374501 SimCache HITRATE: 35.34% [gzh@dev9927.prn1 ~/local/rocksdb] ./db_bench -benchmarks "fillseq,readrandom" -cache_size 1000000 -simcache_size 100000000 RocksDB: version 4.8 Date: Tue May 17 16:57:32 2016 CPU: 32 * Intel(R) Xeon(R) CPU E5-2660 0 @ 2.20GHz CPUCache: 20480 KB Keys: 16 bytes each Values: 100 bytes each (50 bytes after compression) Entries: 1000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 110.6 MB (estimated) FileSize: 62.9 MB (estimated) Write rate: 0 bytes/second Compression: Snappy Memtablerep: skip_list Perf Level: 0 WARNING: Assertions are enabled; benchmarks unnecessarily slow ------------------------------------------------ DB path: [/tmp/rocksdbtest-112628/dbbench] fillseq : 5.632 micros/op 177572 ops/sec; 19.6 MB/s DB path: [/tmp/rocksdbtest-112628/dbbench] readrandom : 6.892 micros/op 145094 ops/sec; 16.1 MB/s (1000000 of 1000000 found) SIMULATOR CACHE STATISTICS: SimCache LOOKUPs: 1150767 SimCache HITs: 1034535 SimCache HITRATE: 89.90% ``` Reviewers: IslamAbdelRahman, andrewkr, sdong Reviewed By: sdong Subscribers: MarkCallaghan, andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D57999
72 lines
2.7 KiB
C++
72 lines
2.7 KiB
C++
// Copyright (c) 2011-present, Facebook, Inc. All rights reserved.
|
|
// This source code is licensed under the BSD-style license found in the
|
|
// LICENSE file in the root directory of this source tree. An additional grant
|
|
// of patent rights can be found in the PATENTS file in the same directory.
|
|
//
|
|
// Copyright (c) 2011 The LevelDB Authors. All rights reserved.
|
|
// Use of this source code is governed by a BSD-style license that can be
|
|
// found in the LICENSE file. See the AUTHORS file for names of contributors.
|
|
|
|
#pragma once
|
|
|
|
#include <cassert>
|
|
#include <cstdint>
|
|
|
|
#include "port/port.h"
|
|
#include "rocksdb/slice.h"
|
|
|
|
namespace rocksdb {
|
|
// An entry is a variable length heap-allocated structure.
|
|
// Entries are referenced by cache and/or by any external entity.
|
|
// The cache keeps all its entries in table. Some elements
|
|
// are also stored on LRU list.
|
|
//
|
|
// LRUHandle can be in these states:
|
|
// 1. Referenced externally AND in hash table.
|
|
// In that case the entry is *not* in the LRU. (refs > 1 && in_cache == true)
|
|
// 2. Not referenced externally and in hash table. In that case the entry is
|
|
// in the LRU and can be freed. (refs == 1 && in_cache == true)
|
|
// 3. Referenced externally and not in hash table. In that case the entry is
|
|
// in not on LRU and not in table. (refs >= 1 && in_cache == false)
|
|
//
|
|
// All newly created LRUHandles are in state 1. If you call LRUCache::Release
|
|
// on entry in state 1, it will go into state 2. To move from state 1 to
|
|
// state 3, either call LRUCache::Erase or LRUCache::Insert with the same key.
|
|
// To move from state 2 to state 1, use LRUCache::Lookup.
|
|
// Before destruction, make sure that no handles are in state 1. This means
|
|
// that any successful LRUCache::Lookup/LRUCache::Insert have a matching
|
|
// RUCache::Release (to move into state 2) or LRUCache::Erase (for state 3)
|
|
|
|
struct LRUHandle {
|
|
void* value;
|
|
void (*deleter)(const Slice&, void* value);
|
|
LRUHandle* next_hash;
|
|
LRUHandle* next;
|
|
LRUHandle* prev;
|
|
size_t charge; // TODO(opt): Only allow uint32_t?
|
|
size_t key_length;
|
|
uint32_t refs; // a number of refs to this entry
|
|
// cache itself is counted as 1
|
|
bool in_cache; // true, if this entry is referenced by the hash table
|
|
uint32_t hash; // Hash of key(); used for fast sharding and comparisons
|
|
char key_data[1]; // Beginning of key
|
|
|
|
Slice key() const {
|
|
// For cheaper lookups, we allow a temporary Handle object
|
|
// to store a pointer to a key in "value".
|
|
if (next == this) {
|
|
return *(reinterpret_cast<Slice*>(value));
|
|
} else {
|
|
return Slice(key_data, key_length);
|
|
}
|
|
}
|
|
|
|
void Free() {
|
|
assert((refs == 1 && in_cache) || (refs == 0 && !in_cache));
|
|
(*deleter)(key(), value);
|
|
delete[] reinterpret_cast<char*>(this);
|
|
}
|
|
};
|
|
|
|
} // end namespace rocksdb
|