andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
011fc8be2d
rocksdb/table
History
Maysam Yabandeh 5b66eee2cb Stats for false positive rate of full filtesr 
Summary:
Adds two stats to allow us measuring the false positive rate of full filters:
- The total count of positives: rocksdb.bloom.filter.full.positive
- The total count of true positives: rocksdb.bloom.filter.full.true.positive
Not the term "full" in the stat name to indicate that they are meaningful in full filters. block-based filters are to be deprecated soon and supporting it is not worth the the additional cost of if-then-else branches.

Closes #3680

Tested by:
$ ./db_bench -benchmarks=fillrandom  -db /dev/shm/rocksdb-tmpdb --num=1000000 -bloom_bits=10
$ ./db_bench -benchmarks="readwhilewriting"  -db /dev/shm/rocksdb-tmpdb --statistics -bloom_bits=10 --duration=60 --num=2000000 --use_existing_db 2>&1 > /tmp/full.log
$ grep filter.full /tmp/full.log
rocksdb.bloom.filter.full.positive COUNT : 3628593
rocksdb.bloom.filter.full.true.positive COUNT : 3536026
which gives the false positive rate of 2.5%
Closes https://github.com/facebook/rocksdb/pull/3681

Differential Revision: D7517570

Pulled By: maysamyabandeh

fbshipit-source-id: 630ab1a473afdce404916d297035b6318de4c052
2018-04-12 13:55:45 -07:00
..
adaptive_table_factory.cc Comment out unused variables 2018-03-05 13:13:41 -08:00
adaptive_table_factory.h Comment out unused variables 2018-03-05 13:13:41 -08:00
block_based_filter_block_test.cc Ignore empty filter block when data block is empty 2018-03-21 23:13:05 -07:00
block_based_filter_block.cc Fix a leak in FilterBlockBuilder when adding prefix 2018-03-27 15:13:56 -07:00
block_based_filter_block.h Ignore empty filter block when data block is empty 2018-03-21 23:13:05 -07:00
block_based_table_builder.cc Align SST file data blocks to avoid spanning multiple pages 2018-03-26 20:26:10 -07:00
block_based_table_builder.h Align SST file data blocks to avoid spanning multiple pages 2018-03-26 20:26:10 -07:00
block_based_table_factory.cc Align SST file data blocks to avoid spanning multiple pages 2018-03-26 20:26:10 -07:00
block_based_table_factory.h Align SST file data blocks to avoid spanning multiple pages 2018-03-26 20:26:10 -07:00
block_based_table_reader.cc Stats for false positive rate of full filtesr 2018-04-12 13:55:45 -07:00
block_based_table_reader.h Fix a block pinning regression introduced in b555ed30a4 2018-03-08 10:12:23 -08:00
block_builder.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
block_builder.h Change RocksDB License 2017-07-15 16:11:23 -07:00
block_fetcher.cc Imporve perf of random read and insert compare by suggesting inlining to the compiler 2018-03-23 13:26:55 -07:00
block_fetcher.h Fix BlockFetcher ASAN error 2017-12-12 12:12:38 -08:00
block_prefix_index.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
block_prefix_index.h Change RocksDB License 2017-07-15 16:11:23 -07:00
block_test.cc Comment out unused variables 2018-03-05 13:13:41 -08:00
block.cc Several small "fixes" 2018-02-15 16:57:37 -08:00
block.h BlockBasedTable::NewDataBlockIterator to always return BlockIter 2018-01-25 14:57:18 -08:00
bloom_block.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
bloom_block.h Change RocksDB License 2017-07-15 16:11:23 -07:00
cleanable_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
cuckoo_table_builder_test.cc Comment out unused variables 2018-03-05 13:13:41 -08:00
cuckoo_table_builder.cc Enable MSVC W4 with a few exceptions. Fix warnings and bugs 2017-10-19 10:57:12 -07:00
cuckoo_table_builder.h Change RocksDB License 2017-07-15 16:11:23 -07:00
cuckoo_table_factory.cc Comment out unused variables 2018-03-05 13:13:41 -08:00
cuckoo_table_factory.h Comment out unused variables 2018-03-05 13:13:41 -08:00
cuckoo_table_reader_test.cc Comment out unused variables 2018-03-05 13:13:41 -08:00
cuckoo_table_reader.cc Stats for false positive rate of full filtesr 2018-04-12 13:55:45 -07:00
cuckoo_table_reader.h Comment out unused variables 2018-03-05 13:13:41 -08:00
filter_block.h Ignore empty filter block when data block is empty 2018-03-21 23:13:05 -07:00
flush_block_policy.cc Align SST file data blocks to avoid spanning multiple pages 2018-03-26 20:26:10 -07:00
format.cc Several small "fixes" 2018-02-15 16:57:37 -08:00
format.h Refactor ReadBlockContents() 2017-12-11 15:27:32 -08:00
full_filter_bits_builder.h Change RocksDB License 2017-07-15 16:11:23 -07:00
full_filter_block_test.cc Ignore empty filter block when data block is empty 2018-03-21 23:13:05 -07:00
full_filter_block.cc Comment out unused variables 2018-03-05 13:13:41 -08:00
full_filter_block.h Ignore empty filter block when data block is empty 2018-03-21 23:13:05 -07:00
get_context.cc Stats for false positive rate of full filtesr 2018-04-12 13:55:45 -07:00
get_context.h Stats for false positive rate of full filtesr 2018-04-12 13:55:45 -07:00
index_builder.cc Enable MSVC W4 with a few exceptions. Fix warnings and bugs 2017-10-19 10:57:12 -07:00
index_builder.h Comment out unused variables 2018-03-05 13:13:41 -08:00
internal_iterator.h Comment out unused variables 2018-03-05 13:13:41 -08:00
iter_heap.h Make InternalKeyComparator final and directly use it in merging iterator 2017-09-11 12:04:21 -07:00
iterator_wrapper.h Change RocksDB License 2017-07-15 16:11:23 -07:00
iterator.cc Comment out unused variables 2018-03-05 13:13:41 -08:00
merger_test.cc Make InternalKeyComparator final and directly use it in merging iterator 2017-09-11 12:04:21 -07:00
merging_iterator.cc fix DBImpl::NewInternalIterator super-version leak on failure 2017-10-11 14:57:43 -07:00
merging_iterator.h fix DBImpl::NewInternalIterator super-version leak on failure 2017-10-11 14:57:43 -07:00
meta_blocks.cc Refactor ReadBlockContents() 2017-12-11 15:27:32 -08:00
meta_blocks.h Support prefetch last 512KB with direct I/O in block based file reader 2017-08-11 12:16:45 -07:00
mock_table.cc Stats for false positive rate of full filtesr 2018-04-12 13:55:45 -07:00
mock_table.h Comment out unused variables 2018-03-05 13:13:41 -08:00
partitioned_filter_block_test.cc Reduce heavy hitter for Get operation 2017-12-12 21:11:33 -08:00
partitioned_filter_block.cc Ignore empty filter block when data block is empty 2018-03-21 23:13:05 -07:00
partitioned_filter_block.h Ignore empty filter block when data block is empty 2018-03-21 23:13:05 -07:00
persistent_cache_helper.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
persistent_cache_helper.h Change RocksDB License 2017-07-15 16:11:23 -07:00
persistent_cache_options.h Change RocksDB License 2017-07-15 16:11:23 -07:00
plain_table_builder.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
plain_table_builder.h Change RocksDB License 2017-07-15 16:11:23 -07:00
plain_table_factory.cc Comment out unused variables 2018-03-05 13:13:41 -08:00
plain_table_factory.h Comment out unused variables 2018-03-05 13:13:41 -08:00
plain_table_index.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
plain_table_index.h table: Fix coverity issues 2017-12-07 11:57:36 -08:00
plain_table_key_coding.cc Comment out unused variables 2018-03-05 13:13:41 -08:00
plain_table_key_coding.h Change RocksDB License 2017-07-15 16:11:23 -07:00
plain_table_reader.cc Stats for false positive rate of full filtesr 2018-04-12 13:55:45 -07:00
plain_table_reader.h remove unnecessary internal_comparator param in newIterator 2017-07-27 14:30:42 -07:00
scoped_arena_iterator.h Change RocksDB License 2017-07-15 16:11:23 -07:00
sst_file_writer_collectors.h Comment out unused variables 2018-03-05 13:13:41 -08:00
sst_file_writer.cc Fix some typos in comments and docs. 2018-03-08 10:27:25 -08:00
table_builder.h TableProperty::oldest_key_time defaults to 0 2017-10-27 15:00:05 -07:00
table_properties_internal.h Eliminate some redundant block reads. 2018-01-10 17:11:58 -08:00
table_properties.cc Eliminate some redundant block reads. 2018-01-10 17:11:58 -08:00
table_reader_bench.cc fix gflags namespace 2017-12-01 10:42:05 -08:00
table_reader.h Comment out unused variables 2018-03-05 13:13:41 -08:00
table_test.cc Align SST file data blocks to avoid spanning multiple pages 2018-03-26 20:26:10 -07:00
two_level_iterator.cc Comment out unused variables 2018-03-05 13:13:41 -08:00
two_level_iterator.h Customized BlockBasedTableIterator and LevelIterator 2018-02-12 17:12:25 -08:00