andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
b9f77ba12b
rocksdb/table
History
Andrew Kryczka e089db40f9 Skip bottom-level filter block caching when hit-optimized 
Summary:
When Get() or NewIterator() trigger file loads, skip caching the filter block if
(1) optimize_filters_for_hits is set and (2) the file is on the bottommost
level. Also skip checking filters under the same conditions, which means that
for a preloaded file or a file that was trivially-moved to the bottom level, its
filter block will eventually expire from the cache.

- added parameters/instance variables in various places in order to propagate the config ("skip_filters") from version_set to block_based_table_reader
- in BlockBasedTable::Rep, this optimization prevents filter from being loaded when the file is opened simply by setting filter_policy = nullptr
- in BlockBasedTable::Get/BlockBasedTable::NewIterator, this optimization prevents filter from being used (even if it was loaded already) by setting filter = nullptr

Test Plan:
updated unit test:

  $ ./db_test --gtest_filter=DBTest.OptimizeFiltersForHits

will also run 'make check'

Reviewers: sdong, igor, paultuckfield, anthony, rven, kradhakrishnan, IslamAbdelRahman, yhchiang

Reviewed By: yhchiang

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D51633
2015-12-23 10:15:07 -08:00
..
adaptive_table_factory.cc Pass column family ID to table property collector 2015-10-09 14:36:51 -07:00
adaptive_table_factory.h Pass column family ID to table property collector 2015-10-09 14:36:51 -07:00
block_based_filter_block_test.cc rocksdb: switch to gtest 2015-03-17 14:08:00 -07:00
block_based_filter_block.cc bloom hit/miss stats for SST and memtable 2015-10-07 11:23:20 -07:00
block_based_filter_block.h Remember whole key/prefix filtering on/off in SST file 2015-02-11 11:20:04 -08:00
block_based_table_builder.cc Introduce ReadOptions::pin_data (support zero copy for keys) 2015-12-16 12:08:30 -08:00
block_based_table_builder.h Move skip_table_builder_flush to BlockBasedTableOption 2015-10-30 18:33:01 -07:00
block_based_table_factory.cc Skip bottom-level filter block caching when hit-optimized 2015-12-23 10:15:07 -08:00
block_based_table_factory.h Merge pull request #805 from SherlockNoMad/OptionHelperFix 2015-11-17 16:24:52 -08:00
block_based_table_reader.cc Skip bottom-level filter block caching when hit-optimized 2015-12-23 10:15:07 -08:00
block_based_table_reader.h Skip bottom-level filter block caching when hit-optimized 2015-12-23 10:15:07 -08:00
block_builder.cc Introduce ReadOptions::pin_data (support zero copy for keys) 2015-12-16 12:08:30 -08:00
block_builder.h Introduce ReadOptions::pin_data (support zero copy for keys) 2015-12-16 12:08:30 -08:00
block_hash_index_test.cc Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00
block_hash_index.cc Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00
block_hash_index.h Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00
block_prefix_index.cc Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
block_prefix_index.h Windows Port from Microsoft 2015-07-01 16:13:56 -07:00
block_test.cc Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00
block.cc Introduce ReadOptions::pin_data (support zero copy for keys) 2015-12-16 12:08:30 -08:00
block.h Introduce ReadOptions::pin_data (support zero copy for keys) 2015-12-16 12:08:30 -08:00
bloom_block.cc table/bloom_block.*: pass func parameter by reference 2014-09-30 23:30:31 +02:00
bloom_block.h Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
cuckoo_table_builder_test.cc Block cuckoo table tests in ROCKSDB_LITE 2015-07-20 10:50:46 -07:00
cuckoo_table_builder.cc compaction assertion triggering test fix for sequence zeroing assertion trip 2015-12-18 16:08:31 -08:00
cuckoo_table_builder.h Move rate_limiter, write buffering, most perf context instrumentation and most random kill out of Env 2015-07-17 16:58:18 -07:00
cuckoo_table_factory.cc Pass column family ID to table property collector 2015-10-09 14:36:51 -07:00
cuckoo_table_factory.h Add OptionsUtil::LoadOptionsFromFile() API 2015-11-12 06:52:43 -08:00
cuckoo_table_reader_test.cc Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00
cuckoo_table_reader.cc Skip bottom-level filter block caching when hit-optimized 2015-12-23 10:15:07 -08:00
cuckoo_table_reader.h Skip bottom-level filter block caching when hit-optimized 2015-12-23 10:15:07 -08:00
filter_block.h Dump routine to BlockBasedTableReader 2014-12-23 13:24:07 -08:00
flush_block_policy.cc move block based table related options BlockBasedTableOptions 2014-08-25 14:22:05 -07:00
format.cc Add ZSTD (not final format) compression type 2015-08-28 11:01:13 -07:00
format.h Move rate_limiter, write buffering, most perf context instrumentation and most random kill out of Env 2015-07-17 16:58:18 -07:00
full_filter_block_test.cc rocksdb: switch to gtest 2015-03-17 14:08:00 -07:00
full_filter_block.cc bloom hit/miss stats for SST and memtable 2015-10-07 11:23:20 -07:00
full_filter_block.h Remember whole key/prefix filtering on/off in SST file 2015-02-11 11:20:04 -08:00
get_context.cc Use SST files for Transaction conflict detection 2015-12-11 12:34:11 -08:00
get_context.h Use SST files for Transaction conflict detection 2015-12-11 12:34:11 -08:00
internal_iterator.h Introduce ReadOptions::pin_data (support zero copy for keys) 2015-12-16 12:08:30 -08:00
iter_heap.h Replace std::priority_queue in MergingIterator with custom heap, take 2 2015-07-15 03:34:40 -07:00
iterator_wrapper.h Introduce ReadOptions::pin_data (support zero copy for keys) 2015-12-16 12:08:30 -08:00
iterator.cc Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00
merger_test.cc Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00
merger.cc Introduce ReadOptions::pin_data (support zero copy for keys) 2015-12-16 12:08:30 -08:00
merger.h Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00
meta_blocks.cc Revert "Adding new table properties" 2015-11-06 16:49:38 -08:00
meta_blocks.h Moving memtable related files from util to a new directory memtable 2015-10-16 14:10:33 -07:00
mock_table.cc Skip bottom-level filter block caching when hit-optimized 2015-12-23 10:15:07 -08:00
mock_table.h Skip bottom-level filter block caching when hit-optimized 2015-12-23 10:15:07 -08:00
plain_table_builder.cc Pass column family ID to table property collector 2015-10-09 14:36:51 -07:00
plain_table_builder.h Pass column family ID to table property collector 2015-10-09 14:36:51 -07:00
plain_table_factory.cc Merge pull request #805 from SherlockNoMad/OptionHelperFix 2015-11-17 16:24:52 -08:00
plain_table_factory.h Fix build broken by previous commit of "option helper refactor" 2015-11-17 16:52:54 -08:00
plain_table_index.cc "make format" against last 10 commits 2015-07-13 13:50:18 -07:00
plain_table_index.h Block plain_table_index.cc in ROCKSDB_LITE 2014-11-24 20:47:27 -08:00
plain_table_key_coding.cc Fix the bug of using freed memory introduced by recent plain table reader patch 2015-09-23 16:16:26 -07:00
plain_table_key_coding.h PlainTableReader to support non-mmap mode 2015-09-23 11:41:07 -07:00
plain_table_reader.cc Skip bottom-level filter block caching when hit-optimized 2015-12-23 10:15:07 -08:00
plain_table_reader.h Skip bottom-level filter block caching when hit-optimized 2015-12-23 10:15:07 -08:00
scoped_arena_iterator.h Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00
sst_file_writer.cc Move skip_table_builder_flush to BlockBasedTableOption 2015-10-30 18:33:01 -07:00
table_builder.h Skip bottom-level filter block caching when hit-optimized 2015-12-23 10:15:07 -08:00
table_properties_internal.h Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00
table_properties.cc Revert "Adding new table properties" 2015-11-06 16:49:38 -08:00
table_reader_bench.cc plain table reader: avoid re-read the same position for index and data in non-mmap mode 2015-11-18 16:49:08 -08:00
table_reader.h Skip bottom-level filter block caching when hit-optimized 2015-12-23 10:15:07 -08:00
table_test.cc Fix BlockBasedTableTest.BlockCacheLeak valgrind failure 2015-12-18 11:17:21 -08:00
two_level_iterator.cc Introduce ReadOptions::pin_data (support zero copy for keys) 2015-12-16 12:08:30 -08:00
two_level_iterator.h Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00