andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
4f678b52e7
rocksdb/table/block_based
History
Peter Dillinger f8c685c4fc Check for and disallow shared key space in block caches (#9172) 
Summary:
We have three layers of block cache that often use the same key
but map to different physical data:
* BlockBasedTableOptions::block_cache
* BlockBasedTableOptions::block_cache_compressed
* BlockBasedTableOptions::persistent_cache

If any two of these happen to share an underlying implementation and key
space (insertion into one shows up in another), then memory safety is
broken. The simplest case is block_cache == block_cache_compressed.
(Credit mrambacher for asking about this case in a review.)

With this change, we explicitly check for overlap and preemptively and
safely fail with a Status code.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/9172

Test Plan: test added. Crashes without new check

Reviewed By: anand1976

Differential Revision: D32465659

Pulled By: pdillinger

fbshipit-source-id: 3876b45b6dce6167e5a7a642725ddc86b96f8e40
2021-11-16 11:16:05 -08:00
..
binary_search_index_reader.cc Separate internal and user key comparators in BlockIter (#6944) 2020-07-07 17:26:16 -07:00
binary_search_index_reader.h Extend Get/MultiGet deadline support to table open (#6982) 2020-06-29 14:53:17 -07:00
block_based_filter_block_test.cc Add table properties for number of entries added to filters (#8323) 2021-05-21 17:11:32 -07:00
block_based_filter_block.cc Deallocate payload of BlockBasedTableBuilder::Rep::FilterBlockBuilder earlier for Full/PartitionedFilter (#9070) 2021-11-04 13:35:38 -07:00
block_based_filter_block.h Deallocate payload of BlockBasedTableBuilder::Rep::FilterBlockBuilder earlier for Full/PartitionedFilter (#9070) 2021-11-04 13:35:38 -07:00
block_based_table_builder.cc Minor improvement to CacheReservationManager/WriteBufferManager/CompressionDictBuilding (#9139) 2021-11-05 16:13:47 -07:00
block_based_table_builder.h Some checksum code refactoring (#9113) 2021-11-04 09:09:34 -07:00
block_based_table_factory.cc Check for and disallow shared key space in block caches (#9172) 2021-11-16 11:16:05 -08:00
block_based_table_factory.h Cleanup includes in dbformat.h (#8930) 2021-09-29 04:04:40 -07:00
block_based_table_iterator.cc Reuse internal auto readhead_size at each Level (expect L0) for Iterations (#9056) 2021-11-10 16:20:04 -08:00
block_based_table_iterator.h Reuse internal auto readhead_size at each Level (expect L0) for Iterations (#9056) 2021-11-10 16:20:04 -08:00
block_based_table_reader_impl.h Parallelize secondary cache lookup in MultiGet (#8405) 2021-06-18 09:35:59 -07:00
block_based_table_reader_test.cc Cleanup includes in dbformat.h (#8930) 2021-09-29 04:04:40 -07:00
block_based_table_reader.cc Reuse internal auto readhead_size at each Level (expect L0) for Iterations (#9056) 2021-11-10 16:20:04 -08:00
block_based_table_reader.h Add lowest_used_cache_tier to ImmutableDBOptions to enable or disable Secondary Cache (#9050) 2021-10-19 15:54:23 -07:00
block_builder.cc Improve data block construction performance (#9040) 2021-10-19 12:36:21 -07:00
block_builder.h Improve data block construction performance (#9040) 2021-10-19 12:36:21 -07:00
block_like_traits.h fix lru caching test and fix reference binding to null pointer (#8326) 2021-05-24 08:37:00 -07:00
block_prefetcher.cc Reuse internal auto readhead_size at each Level (expect L0) for Iterations (#9056) 2021-11-10 16:20:04 -08:00
block_prefetcher.h Reuse internal auto readhead_size at each Level (expect L0) for Iterations (#9056) 2021-11-10 16:20:04 -08:00
block_prefix_index.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
block_prefix_index.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
block_test.cc More Makefile Cleanup (#7097) 2020-07-09 14:35:17 -07:00
block_type.h Fix and detect headers with missing dependencies (#8893) 2021-09-10 10:00:26 -07:00
block.cc Support timestamps in SstFileWriter (#8899) 2021-09-09 18:58:01 -07:00
block.h Cleanup includes in dbformat.h (#8930) 2021-09-29 04:04:40 -07:00
cachable_entry.h Parallelize secondary cache lookup in MultiGet (#8405) 2021-06-18 09:35:59 -07:00
data_block_footer.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
data_block_footer.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
data_block_hash_index_test.cc Make it possible to apply only a subrange of table property collectors (#8298) 2021-05-17 18:28:39 -07:00
data_block_hash_index.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
data_block_hash_index.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
filter_block_reader_common.cc Parallelize secondary cache lookup in MultiGet (#8405) 2021-06-18 09:35:59 -07:00
filter_block_reader_common.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
filter_block.h Deallocate payload of BlockBasedTableBuilder::Rep::FilterBlockBuilder earlier for Full/PartitionedFilter (#9070) 2021-11-04 13:35:38 -07:00
filter_policy_internal.h Add Bloom/Ribbon hybrid API support (#8679) 2021-08-20 18:00:16 -07:00
filter_policy.cc Cleanup includes in dbformat.h (#8930) 2021-09-29 04:04:40 -07:00
flush_block_policy.cc Make FlushBlockPolicyFactory into a Customizable class (#8432) 2021-07-12 09:04:59 -07:00
flush_block_policy.h Make FlushBlockPolicyFactory into a Customizable class (#8432) 2021-07-12 09:04:59 -07:00
full_filter_block_test.cc Add table properties for number of entries added to filters (#8323) 2021-05-21 17:11:32 -07:00
full_filter_block.cc Deallocate payload of BlockBasedTableBuilder::Rep::FilterBlockBuilder earlier for Full/PartitionedFilter (#9070) 2021-11-04 13:35:38 -07:00
full_filter_block.h Deallocate payload of BlockBasedTableBuilder::Rep::FilterBlockBuilder earlier for Full/PartitionedFilter (#9070) 2021-11-04 13:35:38 -07:00
hash_index_reader.cc Make ImmutableOptions struct that inherits from ImmutableCFOptions and ImmutableDBOptions (#8262) 2021-05-05 14:00:17 -07:00
hash_index_reader.h Extend Get/MultiGet deadline support to table open (#6982) 2020-06-29 14:53:17 -07:00
index_builder.cc Add (& fix) some simple source code checks (#8821) 2021-09-07 21:19:27 -07:00
index_builder.h Make db_basic_test pass assert status checked (#7452) 2020-09-29 09:49:04 -07:00
index_reader_common.cc Parallelize secondary cache lookup in MultiGet (#8405) 2021-06-18 09:35:59 -07:00
index_reader_common.h Divide block_based_table_reader.cc (#6527) 2020-03-12 21:41:50 -07:00
mock_block_based_table.h Make ImmutableOptions struct that inherits from ImmutableCFOptions and ImmutableDBOptions (#8262) 2021-05-05 14:00:17 -07:00
parsed_full_filter_block.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
parsed_full_filter_block.h Use new Insert and Lookup APIs in table reader to support secondary cache (#8315) 2021-05-21 18:29:12 -07:00
partitioned_filter_block_test.cc Deallocate payload of BlockBasedTableBuilder::Rep::FilterBlockBuilder earlier for Full/PartitionedFilter (#9070) 2021-11-04 13:35:38 -07:00
partitioned_filter_block.cc Deallocate payload of BlockBasedTableBuilder::Rep::FilterBlockBuilder earlier for Full/PartitionedFilter (#9070) 2021-11-04 13:35:38 -07:00
partitioned_filter_block.h Deallocate payload of BlockBasedTableBuilder::Rep::FilterBlockBuilder earlier for Full/PartitionedFilter (#9070) 2021-11-04 13:35:38 -07:00
partitioned_index_iterator.cc Reuse internal auto readhead_size at each Level (expect L0) for Iterations (#9056) 2021-11-10 16:20:04 -08:00
partitioned_index_iterator.h Reuse internal auto readhead_size at each Level (expect L0) for Iterations (#9056) 2021-11-10 16:20:04 -08:00
partitioned_index_reader.cc Reuse internal auto readhead_size at each Level (expect L0) for Iterations (#9056) 2021-11-10 16:20:04 -08:00
partitioned_index_reader.h Clarify caching behavior for index and filter partitions (#9068) 2021-10-27 17:23:04 -07:00
reader_common.cc Some checksum code refactoring (#9113) 2021-11-04 09:09:34 -07:00
reader_common.h Bring the Configurable options together (#5753) 2020-09-14 17:01:01 -07:00
uncompression_dict_reader.cc Cleanup includes in dbformat.h (#8930) 2021-09-29 04:04:40 -07:00
uncompression_dict_reader.h Extend Get/MultiGet deadline support to table open (#6982) 2020-06-29 14:53:17 -07:00