Summary: New experimental option BBTO::optimize_filters_for_memory builds filters that maximize their use of "usable size" from malloc_usable_size, which is also used to compute block cache charges. Rather than always "rounding up," we track state in the BloomFilterPolicy object to mix essentially "rounding down" and "rounding up" so that the average FP rate of all generated filters is the same as without the option. (YMMV as heavily accessed filters might be unluckily lower accuracy.) Thus, the option near-minimizes what the block cache considers as "memory used" for a given target Bloom filter false positive rate and Bloom filter implementation. There are no forward or backward compatibility issues with this change, though it only works on the format_version=5 Bloom filter. With Jemalloc, we see about 10% reduction in memory footprint (and block cache charge) for Bloom filters, but 1-2% increase in storage footprint, due to encoding efficiency losses (FP rate is non-linear with bits/key). Why not weighted random round up/down rather than state tracking? By only requiring malloc_usable_size, we don't actually know what the next larger and next smaller usable sizes for the allocator are. We pick a requested size, accept and use whatever usable size it has, and use the difference to inform our next choice. This allows us to narrow in on the right balance without tracking/predicting usable sizes. Why not weight history of generated filter false positive rates by number of keys? This could lead to excess skew in small filters after generating a large filter. Results from filter_bench with jemalloc (irrelevant details omitted): (normal keys/filter, but high variance) $ ./filter_bench -quick -impl=2 -average_keys_per_filter=30000 -vary_key_count_ratio=0.9 Build avg ns/key: 29.6278 Number of filters: 5516 Total size (MB): 200.046 Reported total allocated memory (MB): 220.597 Reported internal fragmentation: 10.2732% Bits/key stored: 10.0097 Average FP rate %: 0.965228 $ ./filter_bench -quick -impl=2 -average_keys_per_filter=30000 -vary_key_count_ratio=0.9 -optimize_filters_for_memory Build avg ns/key: 30.5104 Number of filters: 5464 Total size (MB): 200.015 Reported total allocated memory (MB): 200.322 Reported internal fragmentation: 0.153709% Bits/key stored: 10.1011 Average FP rate %: 0.966313 (very few keys / filter, optimization not as effective due to ~59 byte internal fragmentation in blocked Bloom filter representation) $ ./filter_bench -quick -impl=2 -average_keys_per_filter=1000 -vary_key_count_ratio=0.9 Build avg ns/key: 29.5649 Number of filters: 162950 Total size (MB): 200.001 Reported total allocated memory (MB): 224.624 Reported internal fragmentation: 12.3117% Bits/key stored: 10.2951 Average FP rate %: 0.821534 $ ./filter_bench -quick -impl=2 -average_keys_per_filter=1000 -vary_key_count_ratio=0.9 -optimize_filters_for_memory Build avg ns/key: 31.8057 Number of filters: 159849 Total size (MB): 200 Reported total allocated memory (MB): 208.846 Reported internal fragmentation: 4.42297% Bits/key stored: 10.4948 Average FP rate %: 0.811006 (high keys/filter) $ ./filter_bench -quick -impl=2 -average_keys_per_filter=1000000 -vary_key_count_ratio=0.9 Build avg ns/key: 29.7017 Number of filters: 164 Total size (MB): 200.352 Reported total allocated memory (MB): 221.5 Reported internal fragmentation: 10.5552% Bits/key stored: 10.0003 Average FP rate %: 0.969358 $ ./filter_bench -quick -impl=2 -average_keys_per_filter=1000000 -vary_key_count_ratio=0.9 -optimize_filters_for_memory Build avg ns/key: 30.7131 Number of filters: 160 Total size (MB): 200.928 Reported total allocated memory (MB): 200.938 Reported internal fragmentation: 0.00448054% Bits/key stored: 10.1852 Average FP rate %: 0.963387 And from db_bench (block cache) with jemalloc: $ ./db_bench -db=/dev/shm/dbbench.no_optimize -benchmarks=fillrandom -format_version=5 -value_size=90 -bloom_bits=10 -num=2000000 -threads=8 -compaction_style=2 -fifo_compaction_max_table_files_size_mb=10000 -fifo_compaction_allow_compaction=false $ ./db_bench -db=/dev/shm/dbbench -benchmarks=fillrandom -format_version=5 -value_size=90 -bloom_bits=10 -num=2000000 -threads=8 -optimize_filters_for_memory -compaction_style=2 -fifo_compaction_max_table_files_size_mb=10000 -fifo_compaction_allow_compaction=false $ (for FILE in /dev/shm/dbbench.no_optimize/*.sst; do ./sst_dump --file=$FILE --show_properties | grep 'filter block' ; done) | awk '{ t += $4; } END { print t; }' 17063835 $ (for FILE in /dev/shm/dbbench/*.sst; do ./sst_dump --file=$FILE --show_properties | grep 'filter block' ; done) | awk '{ t += $4; } END { print t; }' 17430747 $ #^ 2.1% additional filter storage $ ./db_bench -db=/dev/shm/dbbench.no_optimize -use_existing_db -benchmarks=readrandom,stats -statistics -bloom_bits=10 -num=2000000 -compaction_style=2 -fifo_compaction_max_table_files_size_mb=10000 -fifo_compaction_allow_compaction=false -duration=10 -cache_index_and_filter_blocks -cache_size=1000000000 rocksdb.block.cache.index.add COUNT : 33 rocksdb.block.cache.index.bytes.insert COUNT : 8440400 rocksdb.block.cache.filter.add COUNT : 33 rocksdb.block.cache.filter.bytes.insert COUNT : 21087528 rocksdb.bloom.filter.useful COUNT : 4963889 rocksdb.bloom.filter.full.positive COUNT : 1214081 rocksdb.bloom.filter.full.true.positive COUNT : 1161999 $ #^ 1.04 % observed FP rate $ ./db_bench -db=/dev/shm/dbbench -use_existing_db -benchmarks=readrandom,stats -statistics -bloom_bits=10 -num=2000000 -compaction_style=2 -fifo_compaction_max_table_files_size_mb=10000 -fifo_compaction_allow_compaction=false -optimize_filters_for_memory -duration=10 -cache_index_and_filter_blocks -cache_size=1000000000 rocksdb.block.cache.index.add COUNT : 33 rocksdb.block.cache.index.bytes.insert COUNT : 8448592 rocksdb.block.cache.filter.add COUNT : 33 rocksdb.block.cache.filter.bytes.insert COUNT : 18220328 rocksdb.bloom.filter.useful COUNT : 5360933 rocksdb.bloom.filter.full.positive COUNT : 1321315 rocksdb.bloom.filter.full.true.positive COUNT : 1262999 $ #^ 1.08 % observed FP rate, 13.6% less memory usage for filters (Due to specific key density, this example tends to generate filters that are "worse than average" for internal fragmentation. "Better than average" cases can show little or no improvement.) Pull Request resolved: https://github.com/facebook/rocksdb/pull/6427 Test Plan: unit test added, 'make check' with gcc, clang and valgrind Reviewed By: siying Differential Revision: D22124374 Pulled By: pdillinger fbshipit-source-id: f3e3aa152f9043ddf4fae25799e76341d0d8714e
153 lines
6.5 KiB
C++
153 lines
6.5 KiB
C++
// Copyright (c) 2011-present, Facebook, Inc. All rights reserved.
|
|
// This source code is licensed under both the GPLv2 (found in the
|
|
// COPYING file in the root directory) and Apache 2.0 License
|
|
// (found in the LICENSE.Apache file in the root directory).
|
|
// Copyright (c) 2012 The LevelDB Authors. All rights reserved.
|
|
// Use of this source code is governed by a BSD-style license that can be
|
|
// found in the LICENSE file. See the AUTHORS file for names of contributors.
|
|
|
|
#pragma once
|
|
|
|
#include <atomic>
|
|
#include <memory>
|
|
#include <string>
|
|
#include <vector>
|
|
|
|
#include "rocksdb/filter_policy.h"
|
|
#include "rocksdb/table.h"
|
|
|
|
namespace ROCKSDB_NAMESPACE {
|
|
|
|
class Slice;
|
|
|
|
// Exposes any extra information needed for testing built-in
|
|
// FilterBitsBuilders
|
|
class BuiltinFilterBitsBuilder : public FilterBitsBuilder {
|
|
public:
|
|
// Calculate number of bytes needed for a new filter, including
|
|
// metadata. Passing the result to CalculateNumEntry should
|
|
// return >= the num_entry passed in.
|
|
virtual uint32_t CalculateSpace(const int num_entry) = 0;
|
|
|
|
// Returns an estimate of the FP rate of the returned filter if
|
|
// `keys` keys are added and the filter returned by Finish is `bytes`
|
|
// bytes.
|
|
virtual double EstimatedFpRate(size_t keys, size_t bytes) = 0;
|
|
};
|
|
|
|
// RocksDB built-in filter policy for Bloom or Bloom-like filters.
|
|
// This class is considered internal API and subject to change.
|
|
// See NewBloomFilterPolicy.
|
|
class BloomFilterPolicy : public FilterPolicy {
|
|
public:
|
|
// An internal marker for operating modes of BloomFilterPolicy, in terms
|
|
// of selecting an implementation. This makes it easier for tests to track
|
|
// or to walk over the built-in set of Bloom filter implementations. The
|
|
// only variance in BloomFilterPolicy by mode/implementation is in
|
|
// GetFilterBitsBuilder(), so an enum is practical here vs. subclasses.
|
|
//
|
|
// This enum is essentially the union of all the different kinds of return
|
|
// value from GetFilterBitsBuilder, or "underlying implementation", and
|
|
// higher-level modes that choose an underlying implementation based on
|
|
// context information.
|
|
enum Mode {
|
|
// Legacy implementation of Bloom filter for full and partitioned filters.
|
|
// Set to 0 in case of value confusion with bool use_block_based_builder
|
|
// NOTE: TESTING ONLY as this mode does not use best compatible
|
|
// implementation
|
|
kLegacyBloom = 0,
|
|
// Deprecated block-based Bloom filter implementation.
|
|
// Set to 1 in case of value confusion with bool use_block_based_builder
|
|
// NOTE: DEPRECATED but user exposed
|
|
kDeprecatedBlock = 1,
|
|
// A fast, cache-local Bloom filter implementation. See description in
|
|
// FastLocalBloomImpl.
|
|
// NOTE: TESTING ONLY as this mode does not check format_version
|
|
kFastLocalBloom = 2,
|
|
// Automatically choose from the above (except kDeprecatedBlock) based on
|
|
// context at build time, including compatibility with format_version.
|
|
// NOTE: This is currently the only recommended mode that is user exposed.
|
|
kAuto = 100,
|
|
};
|
|
// All the different underlying implementations that a BloomFilterPolicy
|
|
// might use, as a mode that says "always use this implementation."
|
|
// Only appropriate for unit tests.
|
|
static const std::vector<Mode> kAllFixedImpls;
|
|
|
|
// All the different modes of BloomFilterPolicy that are exposed from
|
|
// user APIs. Only appropriate for higher-level unit tests. Integration
|
|
// tests should prefer using NewBloomFilterPolicy (user-exposed).
|
|
static const std::vector<Mode> kAllUserModes;
|
|
|
|
explicit BloomFilterPolicy(double bits_per_key, Mode mode);
|
|
|
|
~BloomFilterPolicy() override;
|
|
|
|
const char* Name() const override;
|
|
|
|
// Deprecated block-based filter only
|
|
void CreateFilter(const Slice* keys, int n, std::string* dst) const override;
|
|
|
|
// Deprecated block-based filter only
|
|
bool KeyMayMatch(const Slice& key, const Slice& bloom_filter) const override;
|
|
|
|
FilterBitsBuilder* GetFilterBitsBuilder() const override;
|
|
|
|
// To use this function, call GetBuilderFromContext().
|
|
//
|
|
// Neither the context nor any objects therein should be saved beyond
|
|
// the call to this function, unless it's shared_ptr.
|
|
FilterBitsBuilder* GetBuilderWithContext(
|
|
const FilterBuildingContext&) const override;
|
|
|
|
// Returns a new FilterBitsBuilder from the filter_policy in
|
|
// table_options of a context, or nullptr if not applicable.
|
|
// (An internal convenience function to save boilerplate.)
|
|
static FilterBitsBuilder* GetBuilderFromContext(const FilterBuildingContext&);
|
|
|
|
// Read metadata to determine what kind of FilterBitsReader is needed
|
|
// and return a new one. This must successfully process any filter data
|
|
// generated by a built-in FilterBitsBuilder, regardless of the impl
|
|
// chosen for this BloomFilterPolicy. Not compatible with CreateFilter.
|
|
FilterBitsReader* GetFilterBitsReader(const Slice& contents) const override;
|
|
|
|
// Essentially for testing only: configured millibits/key
|
|
int GetMillibitsPerKey() const { return millibits_per_key_; }
|
|
// Essentially for testing only: legacy whole bits/key
|
|
int GetWholeBitsPerKey() const { return whole_bits_per_key_; }
|
|
|
|
private:
|
|
// Newer filters support fractional bits per key. For predictable behavior
|
|
// of 0.001-precision values across floating point implementations, we
|
|
// round to thousandths of a bit (on average) per key.
|
|
int millibits_per_key_;
|
|
|
|
// Older filters round to whole number bits per key. (There *should* be no
|
|
// compatibility issue with fractional bits per key, but preserving old
|
|
// behavior with format_version < 5 just in case.)
|
|
int whole_bits_per_key_;
|
|
|
|
// Selected mode (a specific implementation or way of selecting an
|
|
// implementation) for building new SST filters.
|
|
Mode mode_;
|
|
|
|
// Whether relevant warnings have been logged already. (Remember so we
|
|
// only report once per BloomFilterPolicy instance, to keep the noise down.)
|
|
mutable std::atomic<bool> warned_;
|
|
|
|
// State for implementing optimize_filters_for_memory. Essentially, this
|
|
// tracks a surplus or deficit in total FP rate of filters generated by
|
|
// builders under this policy vs. what would have been generated without
|
|
// optimize_filters_for_memory.
|
|
//
|
|
// To avoid floating point weirdness, the actual value is
|
|
// Sum over all generated filters f:
|
|
// (predicted_fp_rate(f) - predicted_fp_rate(f|o_f_f_m=false)) * 2^32
|
|
mutable std::atomic<int64_t> aggregate_rounding_balance_;
|
|
|
|
// For newer Bloom filter implementation(s)
|
|
FilterBitsReader* GetBloomBitsReader(const Slice& contents) const;
|
|
};
|
|
|
|
} // namespace ROCKSDB_NAMESPACE
|