andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
7711f8cbb4
rocksdb/table/block_based
History
Peter Dillinger ffe1e4b820 Make some FilterPolicy deprecations more clear (#9403) 
Summary:
The old block-based filter has been deprecated for years, but
this makes that more clear by marking the functions specific to it and
logging a warning when the feature is used.

It is deprecated because of performance. In that old design, you have to
binary search through the full SST index before a bloom filter query, which
is much more expensive than a bloom query itself.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/9403

Test Plan:
Used db_bench with and without -use_block_based_filter,
running at the same time

    TEST_TMPDIR=/dev/shm/rocksdb ./db_bench -benchmarks=fillrandom,readrandom -num=10000000 -duration=20 -disable_wal=1 -write_buffer_size=10000000 -bloom_bits=16 -compaction_style=2 -fifo_compaction_max_table_files_size_mb=10000 -fifo_compaction_allow_compaction=0

No significant difference in construction time but 3x slower readrandom
with -use_block_based_filter:
readrandom   :     100.517 micros/op 9948 ops/sec;    1.1 MB/s
vs.
readrandom   :      33.368 micros/op 29968 ops/sec;    3.3 MB/s

Also saw deprecation message (just once) in LOG only with
-use_block_based_filter

Reviewed By: ajkr

Differential Revision: D33673202

Pulled By: pdillinger

fbshipit-source-id: 99f6f0eff619408d9e5f7ef546954ed0be6c7a5b
2022-01-19 18:12:10 -08:00
..
binary_search_index_reader.cc Separate internal and user key comparators in BlockIter (#6944) 2020-07-07 17:26:16 -07:00
binary_search_index_reader.h Extend Get/MultiGet deadline support to table open (#6982) 2020-06-29 14:53:17 -07:00
block_based_filter_block_test.cc Add table properties for number of entries added to filters (#8323) 2021-05-21 17:11:32 -07:00
block_based_filter_block.cc Deallocate payload of BlockBasedTableBuilder::Rep::FilterBlockBuilder earlier for Full/PartitionedFilter (#9070) 2021-11-04 13:35:38 -07:00
block_based_filter_block.h Deallocate payload of BlockBasedTableBuilder::Rep::FilterBlockBuilder earlier for Full/PartitionedFilter (#9070) 2021-11-04 13:35:38 -07:00
block_based_table_builder.cc Enable db_test running in Centos 32 bit OS and Alpine 32 bit OS (#9294) 2022-01-14 11:58:18 -08:00
block_based_table_builder.h Fix segmentation fault in table_options.prepopulate_block_cache when used with partition_filters (#9263) 2021-12-08 12:44:38 -08:00
block_based_table_factory.cc Enable db_test running in Centos 32 bit OS and Alpine 32 bit OS (#9294) 2022-01-14 11:58:18 -08:00
block_based_table_factory.h Cleanup includes in dbformat.h (#8930) 2021-09-29 04:04:40 -07:00
block_based_table_iterator.cc Reuse internal auto readhead_size at each Level (expect L0) for Iterations (#9056) 2021-11-10 16:20:04 -08:00
block_based_table_iterator.h Fix bug in rocksdb internal automatic prefetching (#9234) 2021-11-30 22:53:10 -08:00
block_based_table_reader_impl.h New stable, fixed-length cache keys (#9126) 2021-12-16 17:15:13 -08:00
block_based_table_reader_test.cc Cleanup includes in dbformat.h (#8930) 2021-09-29 04:04:40 -07:00
block_based_table_reader.cc Add NewMetaDataIterator method (#8692) 2021-12-21 11:32:49 -08:00
block_based_table_reader.h New stable, fixed-length cache keys (#9126) 2021-12-16 17:15:13 -08:00
block_builder.cc Improve data block construction performance (#9040) 2021-10-19 12:36:21 -07:00
block_builder.h Improve data block construction performance (#9040) 2021-10-19 12:36:21 -07:00
block_like_traits.h fix lru caching test and fix reference binding to null pointer (#8326) 2021-05-24 08:37:00 -07:00
block_prefetcher.cc Improve / clean up meta block code & integrity (#9163) 2021-11-18 11:43:44 -08:00
block_prefetcher.h Fix bug in rocksdb internal automatic prefetching (#9234) 2021-11-30 22:53:10 -08:00
block_prefix_index.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
block_prefix_index.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
block_test.cc Improve / clean up meta block code & integrity (#9163) 2021-11-18 11:43:44 -08:00
block_type.h Fix and detect headers with missing dependencies (#8893) 2021-09-10 10:00:26 -07:00
block.cc Add NewMetaDataIterator method (#8692) 2021-12-21 11:32:49 -08:00
block.h Add NewMetaDataIterator method (#8692) 2021-12-21 11:32:49 -08:00
cachable_entry.h Parallelize secondary cache lookup in MultiGet (#8405) 2021-06-18 09:35:59 -07:00
data_block_footer.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
data_block_footer.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
data_block_hash_index_test.cc Make it possible to apply only a subrange of table property collectors (#8298) 2021-05-17 18:28:39 -07:00
data_block_hash_index.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
data_block_hash_index.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
filter_block_reader_common.cc Parallelize secondary cache lookup in MultiGet (#8405) 2021-06-18 09:35:59 -07:00
filter_block_reader_common.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
filter_block.h Account Bloom/Ribbon filter construction memory in global memory limit (#9073) 2021-11-18 09:42:20 -08:00
filter_policy_internal.h Add Bloom/Ribbon hybrid API support (#8679) 2021-08-20 18:00:16 -07:00
filter_policy.cc Make some FilterPolicy deprecations more clear (#9403) 2022-01-19 18:12:10 -08:00
flush_block_policy.cc Restore Regex support for ObjectLibrary::Register, rename new APIs to allow old one to be deprecated in the future (#9362) 2022-01-11 06:33:48 -08:00
flush_block_policy.h Make FlushBlockPolicyFactory into a Customizable class (#8432) 2021-07-12 09:04:59 -07:00
full_filter_block_test.cc Add table properties for number of entries added to filters (#8323) 2021-05-21 17:11:32 -07:00
full_filter_block.cc Deallocate payload of BlockBasedTableBuilder::Rep::FilterBlockBuilder earlier for Full/PartitionedFilter (#9070) 2021-11-04 13:35:38 -07:00
full_filter_block.h Account Bloom/Ribbon filter construction memory in global memory limit (#9073) 2021-11-18 09:42:20 -08:00
hash_index_reader.cc Make ImmutableOptions struct that inherits from ImmutableCFOptions and ImmutableDBOptions (#8262) 2021-05-05 14:00:17 -07:00
hash_index_reader.h Extend Get/MultiGet deadline support to table open (#6982) 2020-06-29 14:53:17 -07:00
index_builder.cc Add (& fix) some simple source code checks (#8821) 2021-09-07 21:19:27 -07:00
index_builder.h Make db_basic_test pass assert status checked (#7452) 2020-09-29 09:49:04 -07:00
index_reader_common.cc Parallelize secondary cache lookup in MultiGet (#8405) 2021-06-18 09:35:59 -07:00
index_reader_common.h Divide block_based_table_reader.cc (#6527) 2020-03-12 21:41:50 -07:00
mock_block_based_table.h Make ImmutableOptions struct that inherits from ImmutableCFOptions and ImmutableDBOptions (#8262) 2021-05-05 14:00:17 -07:00
parsed_full_filter_block.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
parsed_full_filter_block.h Use new Insert and Lookup APIs in table reader to support secondary cache (#8315) 2021-05-21 18:29:12 -07:00
partitioned_filter_block_test.cc More refactoring ahead of footer & meta changes (#9240) 2021-12-10 08:13:26 -08:00
partitioned_filter_block.cc Improve / clean up meta block code & integrity (#9163) 2021-11-18 11:43:44 -08:00
partitioned_filter_block.h Deallocate payload of BlockBasedTableBuilder::Rep::FilterBlockBuilder earlier for Full/PartitionedFilter (#9070) 2021-11-04 13:35:38 -07:00
partitioned_index_iterator.cc Reuse internal auto readhead_size at each Level (expect L0) for Iterations (#9056) 2021-11-10 16:20:04 -08:00
partitioned_index_iterator.h Fix bug in rocksdb internal automatic prefetching (#9234) 2021-11-30 22:53:10 -08:00
partitioned_index_reader.cc Improve / clean up meta block code & integrity (#9163) 2021-11-18 11:43:44 -08:00
partitioned_index_reader.h Clarify caching behavior for index and filter partitions (#9068) 2021-10-27 17:23:04 -07:00
reader_common.cc Improve / clean up meta block code & integrity (#9163) 2021-11-18 11:43:44 -08:00
reader_common.h Bring the Configurable options together (#5753) 2020-09-14 17:01:01 -07:00
uncompression_dict_reader.cc Cleanup includes in dbformat.h (#8930) 2021-09-29 04:04:40 -07:00
uncompression_dict_reader.h Extend Get/MultiGet deadline support to table open (#6982) 2020-06-29 14:53:17 -07:00