andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

db_bench separate options for partition index and filters

Browse Source 
Summary:
Some workloads (like my current benchmarking) may want partitioned indexes without partitioned filters. Particularly, when `-optimize_filters_for_hits=true`, the total index size may be larger than the total filter size, so it can make sense to hold all filters in-memory but not all indexes.
Closes https://github.com/facebook/rocksdb/pull/3492

Differential Revision: D6970092

Pulled By: ajkr

fbshipit-source-id: b7fa1828e1d13829339aefb90fd56eb7c5337f61
This commit is contained in:
Andrew Kryczka 2018-02-12 14:54:50 -08:00 committed by Facebook Github Bot
parent 3f1bb07351
commit 0a0fad447b
1 changed files with 7 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
tools/db_bench_tool.cc
View File

@ -412,6 +412,8 @@ DEFINE_bool(cache_index_and_filter_blocks, false,
DEFINE_bool(partition_index_and_filters, false, DEFINE_bool(partition_index_and_filters, false,
"Partition index and filter blocks."); "Partition index and filter blocks.");
DEFINE_bool(partition_index, false, "Partition index blocks");
DEFINE_int64(metadata_block_size, DEFINE_int64(metadata_block_size,
rocksdb::BlockBasedTableOptions().metadata_block_size, rocksdb::BlockBasedTableOptions().metadata_block_size,
"Max partition size when partitioning index/filters"); "Max partition size when partitioning index/filters");
@ -3096,16 +3098,18 @@ void VerifyDBFromDB(std::string& truth_db_name) {
} else { } else {
block_based_options.index_type = BlockBasedTableOptions::kBinarySearch; block_based_options.index_type = BlockBasedTableOptions::kBinarySearch;
} }
if (FLAGS_partition_index_and_filters) { if (FLAGS_partition_index_and_filters || FLAGS_partition_index) {
if (FLAGS_use_hash_search) { if (FLAGS_use_hash_search) {
fprintf(stderr, fprintf(stderr,
"use_hash_search is incompatible with " "use_hash_search is incompatible with "
"partition_index_and_filters and is ignored"); "partition index and is ignored");
} }
block_based_options.index_type = block_based_options.index_type =
BlockBasedTableOptions::kTwoLevelIndexSearch; BlockBasedTableOptions::kTwoLevelIndexSearch;
block_based_options.partition_filters = true;
block_based_options.metadata_block_size = FLAGS_metadata_block_size; block_based_options.metadata_block_size = FLAGS_metadata_block_size;
if (FLAGS_partition_index_and_filters) {
block_based_options.partition_filters = true;
}
} }
if (cache_ == nullptr) { if (cache_ == nullptr) {
block_based_options.no_block_cache = true; block_based_options.no_block_cache = true;