68a9c186d0
Summary: * Inefficient block-based filter is no longer customizable in the public API, though (for now) can still be enabled. * Removed deprecated FilterPolicy::CreateFilter() and FilterPolicy::KeyMayMatch() * Removed `rocksdb_filterpolicy_create()` from C API * Change meaning of nullptr return from GetBuilderWithContext() from "use block-based filter" to "generate no filter in this case." This is a cleaner solution to the proposal in https://github.com/facebook/rocksdb/issues/8250. * Also, when user specifies bits_per_key < 0.5, we now round this down to "no filter" because we expect a filter with >= 80% FP rate is unlikely to be worth the CPU cost of accessing it (esp with cache_index_and_filter_blocks=1 or partition_filters=1). * bits_per_key >= 0.5 and < 1.0 is still rounded up to 1.0 (for 62% FP rate) * This also gives us some support for configuring filters from OPTIONS file as currently saved: `filter_policy=rocksdb.BuiltinBloomFilter`. Opening from such an options file will enable reading filters (an improvement) but not writing new ones. (See Customizable follow-up below.) * Also removed deprecated functions * FilterBitsBuilder::CalculateNumEntry() * FilterPolicy::GetFilterBitsBuilder() * NewExperimentalRibbonFilterPolicy() * Remove default implementations of * FilterBitsBuilder::EstimateEntriesAdded() * FilterBitsBuilder::ApproximateNumEntries() * FilterPolicy::GetBuilderWithContext() * Remove support for "filter_policy=experimental_ribbon" configuration string. * Allow "filter_policy=bloomfilter:n" without bool to discourage use of block-based filter. Some pieces for https://github.com/facebook/rocksdb/issues/9389 Likely follow-up (later PRs): * Refactoring toward FilterPolicy Customizable, so that we can generate filters with same configuration as before when configuring from options file. * Remove support for user enabling block-based filter (ignore `bool use_block_based_builder`) * Some months after this change, we could even remove read support for block-based filter, because it is not critical to DB data preservation. * Make FilterBitsBuilder::FinishV2 to avoid `using FilterBitsBuilder::Finish` mess and add support for specifying a MemoryAllocator (for cache warming) Pull Request resolved: https://github.com/facebook/rocksdb/pull/9501 Test Plan: A number of obsolete tests deleted and new tests or test cases added or updated. Reviewed By: hx235 Differential Revision: D34008011 Pulled By: pdillinger fbshipit-source-id: a39a720457c354e00d5b59166b686f7f59e392aa
57 lines
1.9 KiB
C++
57 lines
1.9 KiB
C++
// Copyright (c) 2019-present, Facebook, Inc. All rights reserved.
|
|
// This source code is licensed under both the GPLv2 (found in the
|
|
// COPYING file in the root directory) and Apache 2.0 License
|
|
// (found in the LICENSE.Apache file in the root directory).
|
|
#pragma once
|
|
|
|
#include "rocksdb/filter_policy.h"
|
|
#include "table/block_based/block_based_filter_block.h"
|
|
#include "table/block_based/block_based_table_reader.h"
|
|
#include "table/block_based/filter_policy_internal.h"
|
|
|
|
namespace ROCKSDB_NAMESPACE {
|
|
namespace mock {
|
|
|
|
class MockBlockBasedTable : public BlockBasedTable {
|
|
public:
|
|
explicit MockBlockBasedTable(Rep* rep)
|
|
: BlockBasedTable(rep, nullptr /* block_cache_tracer */) {}
|
|
};
|
|
|
|
class MockBlockBasedTableTester {
|
|
static constexpr int kMockLevel = 0;
|
|
|
|
public:
|
|
Options options_;
|
|
ImmutableOptions ioptions_;
|
|
EnvOptions env_options_;
|
|
BlockBasedTableOptions table_options_;
|
|
InternalKeyComparator icomp_;
|
|
std::unique_ptr<BlockBasedTable> table_;
|
|
|
|
explicit MockBlockBasedTableTester(const FilterPolicy* filter_policy)
|
|
: ioptions_(options_),
|
|
env_options_(options_),
|
|
icomp_(options_.comparator) {
|
|
table_options_.filter_policy.reset(filter_policy);
|
|
|
|
constexpr bool skip_filters = false;
|
|
constexpr bool immortal_table = false;
|
|
table_.reset(new MockBlockBasedTable(new BlockBasedTable::Rep(
|
|
ioptions_, env_options_, table_options_, icomp_, skip_filters,
|
|
12345 /*file_size*/, kMockLevel, immortal_table)));
|
|
}
|
|
|
|
FilterBitsBuilder* GetBuilder() const {
|
|
FilterBuildingContext context(table_options_);
|
|
context.column_family_name = "mock_cf";
|
|
context.compaction_style = ioptions_.compaction_style;
|
|
context.level_at_creation = kMockLevel;
|
|
context.info_log = ioptions_.logger;
|
|
return BloomFilterPolicy::GetBuilderFromContext(context);
|
|
}
|
|
};
|
|
|
|
} // namespace mock
|
|
} // namespace ROCKSDB_NAMESPACE
|