andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
5bf9a7d5ee
rocksdb/table
History
Peter Dillinger 5bf9a7d5ee Clarify caching behavior for index and filter partitions (#9068) 
Summary:
Somewhat confusingly, index and filter partition blocks are
never owned by table readers, even with
cache_index_and_filter_blocks=false. They still go into block cache
(possibly pinned by table reader) if there is a block cache. If no block
cache, they are only loaded transiently on demand.

This PR primarily clarifies the options APIs and some internal code
comments.

Also, this closes a hypothetical data corruption vulnerability where
some but not all index partitions are pinned. I haven't been able to
reproduce a case where it can happen (the failure seems to propagate
to abort table open) but it's worth patching nonetheless.

Fixes https://github.com/facebook/rocksdb/issues/8979

Pull Request resolved: https://github.com/facebook/rocksdb/pull/9068

Test Plan:
existing tests :-/  I could cover the new code using sync
points, but then I'd have to very carefully relax my `assert(false)`

Reviewed By: ajkr

Differential Revision: D31898284

Pulled By: pdillinger

fbshipit-source-id: f2511a7d3a36bc04b627935d8e6cfea6422f98be
2021-10-27 17:23:04 -07:00
..
adaptive Refactor: use TableBuilderOptions to reduce parameter lists (#8240) 2021-04-29 07:00:50 -07:00
block_based Clarify caching behavior for index and filter partitions (#9068) 2021-10-27 17:23:04 -07:00
cuckoo Cleanup includes in dbformat.h (#8930) 2021-09-29 04:04:40 -07:00
plain Cleanup includes in dbformat.h (#8930) 2021-09-29 04:04:40 -07:00
block_fetcher_test.cc Cleanup includes in dbformat.h (#8930) 2021-09-29 04:04:40 -07:00
block_fetcher.cc Rename variables in ImmutableCFOptions to avoid conflicts with ImmutableDBOptions (#8227) 2021-04-26 12:43:45 -07:00
block_fetcher.h Add db_test2 to to ASSERT_STATUS_CHECKED (#8640) 2021-08-16 08:10:32 -07:00
cleanable_test.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
format.cc More clear error message on uncompressing block (#8934) 2021-09-27 10:38:16 -07:00
format.h Make ImmutableOptions struct that inherits from ImmutableCFOptions and ImmutableDBOptions (#8262) 2021-05-05 14:00:17 -07:00
get_context.cc Cleanup includes in dbformat.h (#8930) 2021-09-29 04:04:40 -07:00
get_context.h Cleanup includes in dbformat.h (#8930) 2021-09-29 04:04:40 -07:00
internal_iterator.h Clean up InternalIterator upper bound logic a little bit (#7200) 2020-08-05 10:44:57 -07:00
iter_heap.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
iterator_wrapper.h Redesign block cache pinning API (#7520) 2020-10-11 14:58:24 -07:00
iterator.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
merger_test.cc Cleanup multiple implementations of VectorIterator (#8901) 2021-10-06 07:48:31 -07:00
merging_iterator.cc MergingIterator: rearrange fields to reduce paddings (#9024) 2021-10-14 12:01:56 -07:00
merging_iterator.h Cleanup includes in dbformat.h (#8930) 2021-09-29 04:04:40 -07:00
meta_blocks.cc Cleanup includes in dbformat.h (#8930) 2021-09-29 04:04:40 -07:00
meta_blocks.h Make ImmutableOptions struct that inherits from ImmutableCFOptions and ImmutableDBOptions (#8262) 2021-05-05 14:00:17 -07:00
mock_table.cc Introduce a mechanism to dump out blocks from block cache and re-insert to secondary cache (#8912) 2021-10-07 11:42:31 -07:00
mock_table.h Fix some minor issues in the Customizable infrastructure (#8566) 2021-08-19 10:10:47 -07:00
multiget_context.h Batch blob read IO for MultiGet (#8699) 2021-09-17 19:23:13 -07:00
persistent_cache_helper.cc Add more tests to ASSERT_STATUS_CHECKED (#7367) 2020-09-16 15:48:07 -07:00
persistent_cache_helper.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
persistent_cache_options.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
scoped_arena_iterator.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
sst_file_dumper.cc Make it possible to apply only a subrange of table property collectors (#8298) 2021-05-17 18:28:39 -07:00
sst_file_dumper.h Make ImmutableOptions struct that inherits from ImmutableCFOptions and ImmutableDBOptions (#8262) 2021-05-05 14:00:17 -07:00
sst_file_reader_test.cc Support timestamps in SstFileWriter (#8899) 2021-09-09 18:58:01 -07:00
sst_file_reader.cc Make ImmutableOptions struct that inherits from ImmutableCFOptions and ImmutableDBOptions (#8262) 2021-05-05 14:00:17 -07:00
sst_file_writer_collectors.h Cleanup includes in dbformat.h (#8930) 2021-09-29 04:04:40 -07:00
sst_file_writer.cc Support timestamps in SstFileWriter (#8899) 2021-09-09 18:58:01 -07:00
table_builder.h Partially revert the "apply subrange of table property collectors" change (#8465) 2021-07-06 10:14:32 -07:00
table_factory.cc Add customizable_util.h to the public API (#8301) 2021-06-29 09:08:57 -07:00
table_properties_internal.h Experimental support for SST unique IDs (#8990) 2021-10-18 23:32:01 -07:00
table_properties.cc Experimental support for SST unique IDs (#8990) 2021-10-18 23:32:01 -07:00
table_reader_bench.cc Make it possible to apply only a subrange of table property collectors (#8298) 2021-05-17 18:28:39 -07:00
table_reader_caller.h Fix and detect headers with missing dependencies (#8893) 2021-09-10 10:00:26 -07:00
table_reader.h dedup ReadOptions in iterator hierarchy (#7210) 2020-08-03 15:23:04 -07:00
table_test.cc Experimental support for SST unique IDs (#8990) 2021-10-18 23:32:01 -07:00
two_level_iterator.cc Clarify caching behavior for index and filter partitions (#9068) 2021-10-27 17:23:04 -07:00
two_level_iterator.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
unique_id_impl.h Experimental support for SST unique IDs (#8990) 2021-10-18 23:32:01 -07:00
unique_id.cc Experimental support for SST unique IDs (#8990) 2021-10-18 23:32:01 -07:00