0050a73a4f
Summary: This change standardizes on a new 16-byte cache key format for block cache (incl compressed and secondary) and persistent cache (but not table cache and row cache). The goal is a really fast cache key with practically ideal stability and uniqueness properties without external dependencies (e.g. from FileSystem). A fixed key size of 16 bytes should enable future optimizations to the concurrent hash table for block cache, which is a heavy CPU user / bottleneck, but there appears to be measurable performance improvement even with no changes to LRUCache. This change replaces a lot of disjointed and ugly code handling cache keys with calls to a simple, clean new internal API (cache_key.h). (Preserving the old cache key logic under an option would be very ugly and likely negate the performance gain of the new approach. Complete replacement carries some inherent risk, but I think that's acceptable with sufficient analysis and testing.) The scheme for encoding new cache keys is complicated but explained in cache_key.cc. Also: EndianSwapValue is moved to math.h to be next to other bit operations. (Explains some new include "math.h".) ReverseBits operation added and unit tests added to hash_test for both. Fixes https://github.com/facebook/rocksdb/issues/7405 (presuming a root cause) Pull Request resolved: https://github.com/facebook/rocksdb/pull/9126 Test Plan: ### Basic correctness Several tests needed updates to work with the new functionality, mostly because we are no longer relying on filesystem for stable cache keys so table builders & readers need more context info to agree on cache keys. This functionality is so core, a huge number of existing tests exercise the cache key functionality. ### Performance Create db with `TEST_TMPDIR=/dev/shm ./db_bench -bloom_bits=10 -benchmarks=fillrandom -num=3000000 -partition_index_and_filters` And test performance with `TEST_TMPDIR=/dev/shm ./db_bench -readonly -use_existing_db -bloom_bits=10 -benchmarks=readrandom -num=3000000 -duration=30 -cache_index_and_filter_blocks -cache_size=250000 -threads=4` using DEBUG_LEVEL=0 and simultaneous before & after runs. Before ops/sec, avg over 100 runs: 121924 After ops/sec, avg over 100 runs: 125385 (+2.8%) ### Collision probability I have built a tool, ./cache_bench -stress_cache_key to broadly simulate host-wide cache activity over many months, by making some pessimistic simplifying assumptions: * Every generated file has a cache entry for every byte offset in the file (contiguous range of cache keys) * All of every file is cached for its entire lifetime We use a simple table with skewed address assignment and replacement on address collision to simulate files coming & going, with quite a variance (super-Poisson) in ages. Some output with `./cache_bench -stress_cache_key -sck_keep_bits=40`: ``` Total cache or DBs size: 32TiB Writing 925.926 MiB/s or 76.2939TiB/day Multiply by 9.22337e+18 to correct for simulation losses (but still assume whole file cached) ``` These come from default settings of 2.5M files per day of 32 MB each, and `-sck_keep_bits=40` means that to represent a single file, we are only keeping 40 bits of the 128-bit cache key. With file size of 2\*\*25 contiguous keys (pessimistic), our simulation is about 2\*\*(128-40-25) or about 9 billion billion times more prone to collision than reality. More default assumptions, relatively pessimistic: * 100 DBs in same process (doesn't matter much) * Re-open DB in same process (new session ID related to old session ID) on average every 100 files generated * Restart process (all new session IDs unrelated to old) 24 times per day After enough data, we get a result at the end: ``` (keep 40 bits) 17 collisions after 2 x 90 days, est 10.5882 days between (9.76592e+19 corrected) ``` If we believe the (pessimistic) simulation and the mathematical generalization, we would need to run a billion machines all for 97 billion days to expect a cache key collision. To help verify that our generalization ("corrected") is robust, we can make our simulation more precise with `-sck_keep_bits=41` and `42`, which takes more running time to get enough data: ``` (keep 41 bits) 16 collisions after 4 x 90 days, est 22.5 days between (1.03763e+20 corrected) (keep 42 bits) 19 collisions after 10 x 90 days, est 47.3684 days between (1.09224e+20 corrected) ``` The generalized prediction still holds. With the `-sck_randomize` option, we can see that we are beating "random" cache keys (except offsets still non-randomized) by a modest amount (roughly 20x less collision prone than random), which should make us reasonably comfortable even in "degenerate" cases: ``` 197 collisions after 1 x 90 days, est 0.456853 days between (4.21372e+18 corrected) ``` I've run other tests to validate other conditions behave as expected, never behaving "worse than random" unless we start chopping off structured data. Reviewed By: zhichao-cao Differential Revision: D33171746 Pulled By: pdillinger fbshipit-source-id: f16a57e369ed37be5e7e33525ace848d0537c88f
72 lines
2.8 KiB
C++
72 lines
2.8 KiB
C++
// Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved.
|
|
// This source code is licensed under both the GPLv2 (found in the
|
|
// COPYING file in the root directory) and Apache 2.0 License
|
|
// (found in the LICENSE.Apache file in the root directory).
|
|
|
|
// This file is for functions that generate unique identifiers by
|
|
// (at least in part) by extracting novel entropy or sources of uniqueness
|
|
// from the execution environment. (By contrast, random.h is for algorithmic
|
|
// pseudorandomness.)
|
|
//
|
|
// These functions could eventually migrate to public APIs, such as in Env.
|
|
|
|
#pragma once
|
|
|
|
#include <atomic>
|
|
#include <cstdint>
|
|
|
|
#include "rocksdb/rocksdb_namespace.h"
|
|
|
|
namespace ROCKSDB_NAMESPACE {
|
|
|
|
// Generates a new 128-bit identifier that is universally unique
|
|
// (with high probability) for each call. The result is split into
|
|
// two 64-bit pieces. This function has NOT been validated for use in
|
|
// cryptography.
|
|
//
|
|
// This is used in generating DB session IDs and by Env::GenerateUniqueId
|
|
// (used for DB IDENTITY) if the platform does not provide a generator of
|
|
// RFC 4122 UUIDs or fails somehow. (Set exclude_port_uuid=true if this
|
|
// function is used as a fallback for GenerateRfcUuid, because no need
|
|
// trying it again.)
|
|
void GenerateRawUniqueId(uint64_t* a, uint64_t* b,
|
|
bool exclude_port_uuid = false);
|
|
|
|
#ifndef NDEBUG
|
|
// A version of above with options for challenge testing
|
|
void TEST_GenerateRawUniqueId(uint64_t* a, uint64_t* b, bool exclude_port_uuid,
|
|
bool exclude_env_details,
|
|
bool exclude_random_device);
|
|
#endif
|
|
|
|
// Generates globally unique ids with lower probability of any collisions
|
|
// vs. each unique id being independently random (GenerateRawUniqueId).
|
|
// We call this "semi-structured" because between different
|
|
// SemiStructuredUniqueIdGen objects, the IDs are separated by random
|
|
// intervals (unstructured), but within a single SemiStructuredUniqueIdGen
|
|
// object, the generated IDs are trivially related (structured). See
|
|
// https://github.com/pdillinger/unique_id for how this improves probability
|
|
// of no collision. In short, if we have n SemiStructuredUniqueIdGen
|
|
// objects each generating m IDs, the first collision is expected at
|
|
// around n = sqrt(2^128 / m), equivalently n * sqrt(m) = 2^64,
|
|
// rather than n * m = 2^64 for fully random IDs.
|
|
class SemiStructuredUniqueIdGen {
|
|
public:
|
|
// Initializes with random starting state (from GenerateRawUniqueId)
|
|
SemiStructuredUniqueIdGen() { Reset(); }
|
|
// Re-initializes, but not thread safe
|
|
void Reset();
|
|
|
|
// Assuming no fork(), `lower` is guaranteed unique from one call
|
|
// to the next (thread safe).
|
|
void GenerateNext(uint64_t* upper, uint64_t* lower);
|
|
|
|
private:
|
|
uint64_t base_upper_;
|
|
uint64_t base_lower_;
|
|
std::atomic<uint64_t> counter_;
|
|
int64_t saved_process_id_;
|
|
};
|
|
|
|
} // namespace ROCKSDB_NAMESPACE
|