andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
1e35584251
rocksdb/table
History
Levi Tamasi 1e35584251 Move the index readers out of the block cache (#5298) 
Summary:
Currently, when the block cache is used for index blocks as well, it is
not really the index block that is stored in the cache but an
IndexReader object. Since this object is not pure data (it has, for
instance, pointers that might dangle), it's not really sharable. To
avoid the issues around this, the current code uses a dummy unique cache
key for each TableReader to store the IndexReader, and erases the
IndexReader entry when the TableReader is closed. Instead of doing this,
the new code moves the IndexReader out of the cache altogether. In
particular, instead of the TableReader owning, or caching/pinning the
IndexReader based on the customer's settings, the TableReader
unconditionally owns the IndexReader, which in turn owns/caches/pins
the index block (which is itself sharable and thus can be safely put in
the cache without any hacks).

Note: the change has two side effects:
1) Partitions of partitioned indexes no longer affect the read
amplification statistics.
2) Eviction statistics for index blocks are temporarily broken. We plan to fix
this in a separate phase.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/5298

Differential Revision: D15303203

Pulled By: ltamasi

fbshipit-source-id: 935a69ba59d87d5e44f42e2310619b790c366e47
2019-05-30 11:53:27 -07:00
..
adaptive_table_factory.cc Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
adaptive_table_factory.h Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
block_based_filter_block_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
block_based_filter_block.cc Apply automatic formatting to some files (#5114) 2019-03-27 16:24:45 -07:00
block_based_filter_block.h Apply automatic formatting to some files (#5114) 2019-03-27 16:24:45 -07:00
block_based_table_builder.cc Fix compilation errors for 32bits/LITE/ios build. (#5220) 2019-04-22 16:02:16 -07:00
block_based_table_builder.h Periodic Compactions (#5166) 2019-04-10 19:31:18 -07:00
block_based_table_factory.cc Unordered Writes (#5218) 2019-05-13 17:47:21 -07:00
block_based_table_factory.h Add BlockBasedTableOptions::index_shortening (#5174) 2019-04-22 08:20:35 -07:00
block_based_table_reader.cc Move the index readers out of the block cache (#5298) 2019-05-30 11:53:27 -07:00
block_based_table_reader.h Move the index readers out of the block cache (#5298) 2019-05-30 11:53:27 -07:00
block_builder.cc Apply automatic formatting to some files (#5114) 2019-03-27 16:24:45 -07:00
block_builder.h Apply automatic formatting to some files (#5114) 2019-03-27 16:24:45 -07:00
block_fetcher.cc Apply automatic formatting to some files (#5114) 2019-03-27 16:24:45 -07:00
block_fetcher.h Add class comment for BlockFetcher 2019-05-23 14:22:26 -07:00
block_prefix_index.cc Apply automatic formatting to some files (#5114) 2019-03-27 16:24:45 -07:00
block_prefix_index.h Apply automatic formatting to some files (#5114) 2019-03-27 16:24:45 -07:00
block_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
block.cc Reduce binary search when reseek into the same data block (#5256) 2019-05-01 14:26:30 -07:00
block.h Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
bloom_block.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
bloom_block.h Disallow customized hash function in DynamicBloom (#4915) 2019-01-24 10:34:30 -08:00
cachable_entry.h Turn CachableEntry into a proper resource handle (#5252) 2019-05-10 11:57:49 -07:00
cleanable_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
cuckoo_table_builder_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
cuckoo_table_builder.cc Promote rocksdb.{deleted.keys,merge.operands} to main table properties (#4594) 2018-10-30 15:34:27 -07:00
cuckoo_table_builder.h Change RocksDB License 2017-07-15 16:11:23 -07:00
cuckoo_table_factory.cc Update all unique/shared_ptr instances to be qualified with namespace std (#4638) 2018-11-09 11:19:58 -08:00
cuckoo_table_factory.h Update all unique/shared_ptr instances to be qualified with namespace std (#4638) 2018-11-09 11:19:58 -08:00
cuckoo_table_reader_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
cuckoo_table_reader.cc Apply modernize-use-override (2nd iteration) 2019-02-14 14:41:36 -08:00
cuckoo_table_reader.h Index value delta encoding (#3983) 2018-08-09 16:58:40 -07:00
data_block_footer.cc Add db_bench options of data block hash index (#4281) 2018-08-16 18:42:46 -07:00
data_block_footer.h Add db_bench options of data block hash index (#4281) 2018-08-16 18:42:46 -07:00
data_block_hash_index_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
data_block_hash_index.cc DataBlockHashIndex: Remove the division from EstimateSize() (#4293) 2018-08-20 23:13:50 -07:00
data_block_hash_index.h DataBlockHashIndex: Remove the division from EstimateSize() (#4293) 2018-08-20 23:13:50 -07:00
filter_block.h Remove a couple of non-public includes from public header file (#5219) 2019-04-19 11:10:33 -07:00
flush_block_policy.cc Apply modernize-use-override (2nd iteration) 2019-02-14 14:41:36 -08:00
flush_block_policy.h Avoid per-key upper bound check in BlockBasedTableIterator (#5142) 2019-04-16 11:37:47 -07:00
format.cc Apply automatic formatting to some files (#5114) 2019-03-27 16:24:45 -07:00
format.h Add some comments for BlockContents 2019-05-24 11:20:09 -07:00
full_filter_bits_builder.h Skip duplicate bloom keys when whole_key and prefix are mixed 2018-04-24 10:58:16 -07:00
full_filter_block_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
full_filter_block.cc Fix bugs detected by clang analyzer (#5185) 2019-04-12 10:45:56 -07:00
full_filter_block.h Introduce a new MultiGet batching implementation (#5011) 2019-04-11 14:28:26 -07:00
get_context.cc PlainTable should avoid copying Get() results from immortal source. (#4924) 2019-01-25 17:12:19 -08:00
get_context.h Add comments t get_context.h 2019-05-24 13:31:05 -07:00
index_builder.cc Add BlockBasedTableOptions::index_shortening (#5174) 2019-04-22 08:20:35 -07:00
index_builder.h Add BlockBasedTableOptions::index_shortening (#5174) 2019-04-22 08:20:35 -07:00
internal_iterator.h Reduce iterator key comparison for upper/lower bound check (#5111) 2019-05-17 10:28:31 -07:00
iter_heap.h Make InternalKeyComparator final and directly use it in merging iterator 2017-09-11 12:04:21 -07:00
iterator_wrapper.h Reduce iterator key comparison for upper/lower bound check (#5111) 2019-05-17 10:28:31 -07:00
iterator.cc Add back NewEmptyIterator (#5203) 2019-04-16 20:28:05 -07:00
merger_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
merging_iterator.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
merging_iterator.h Index value delta encoding (#3983) 2018-08-09 16:58:40 -07:00
meta_blocks.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
meta_blocks.h Feature for sampling and reporting compressibility (#4842) 2019-03-18 12:15:34 -07:00
mock_table.cc Refresh snapshot list during long compactions (2nd attempt) (#5278) 2019-05-03 17:30:22 -07:00
mock_table.h Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
multiget_context.h multiget: fix memory issues due to vector auto resizing (#5279) 2019-05-03 15:58:43 -07:00
partitioned_filter_block_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
partitioned_filter_block.cc Turn CachableEntry into a proper resource handle (#5252) 2019-05-10 11:57:49 -07:00
partitioned_filter_block.h Turn CachableEntry into a proper resource handle (#5252) 2019-05-10 11:57:49 -07:00
persistent_cache_helper.cc Remove two variables from BlockContents class and don't use class Block for compressed block (#4650) 2018-11-13 17:02:55 -08:00
persistent_cache_helper.h Change RocksDB License 2017-07-15 16:11:23 -07:00
persistent_cache_options.h Change RocksDB License 2017-07-15 16:11:23 -07:00
plain_table_builder.cc Revert "Remove PlainTable's feature store_index_in_file (#4914)" (#5034) 2019-03-01 15:45:45 -08:00
plain_table_builder.h Improve comments of classes for PlainTable (#5339) 2019-05-23 16:51:44 -07:00
plain_table_factory.cc Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
plain_table_factory.h Improve comments of classes for PlainTable (#5339) 2019-05-23 16:51:44 -07:00
plain_table_index.cc Fix many bugs in log statement arguments (#5089) 2019-04-04 12:12:11 -07:00
plain_table_index.h Improve comments of classes for PlainTable (#5339) 2019-05-23 16:51:44 -07:00
plain_table_key_coding.cc Comment out unused variables 2018-03-05 13:13:41 -08:00
plain_table_key_coding.h Improve comments of classes for PlainTable (#5339) 2019-05-23 16:51:44 -07:00
plain_table_reader.cc Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
plain_table_reader.h Improve comments of classes for PlainTable (#5339) 2019-05-23 16:51:44 -07:00
scoped_arena_iterator.h Change RocksDB License 2017-07-15 16:11:23 -07:00
sst_file_reader_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
sst_file_reader.cc Fix SstFileReader not able to open ingested file (#5097) 2019-03-26 10:25:18 -07:00
sst_file_writer_collectors.h Fix SstFileReader not able to open ingested file (#5097) 2019-03-26 10:25:18 -07:00
sst_file_writer.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
table_builder.h Periodic Compactions (#5166) 2019-04-10 19:31:18 -07:00
table_properties_internal.h Index value delta encoding (#3983) 2018-08-09 16:58:40 -07:00
table_properties.cc Periodic Compactions (#5166) 2019-04-10 19:31:18 -07:00
table_reader_bench.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
table_reader.h Comments for BlockBasedTable 2019-05-24 12:35:25 -07:00
table_test.cc Move the index readers out of the block cache (#5298) 2019-05-30 11:53:27 -07:00
two_level_iterator.cc Apply modernize-use-override (2nd iteration) 2019-02-14 14:41:36 -08:00
two_level_iterator.h Index value delta encoding (#3983) 2018-08-09 16:58:40 -07:00