andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Smaller tail readahead when not reading index/filters (#4159)

Browse Source 
Summary:
In all cases during `BlockBasedTable::Open`, we issue at least three read requests to the file's tail: (1) footer, (2) metaindex block, and (3) properties block. Depending on the config, we may also read other metablocks like filter and index.

This PR issues smaller readahead when we expect to do only the three necessary reads mentioned above. Then, 4KB should be enough (ignoring the case where there are lots of user-defined properties). We can keep doing 512KB readahead when additional reads are expected.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/4159

Differential Revision: D8924002

Pulled By: ajkr

fbshipit-source-id: cfc713275de4d05ce11f18571f1d72e27ccd3356
This commit is contained in:
Andrew Kryczka 2018-07-19 16:07:53 -07:00 committed by Facebook Github Bot
parent 78ab11cd71
commit b5613227a9
1 changed files with 11 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

15
table/block_based_table_reader.cc
View File

@ -738,8 +738,17 @@ Status BlockBasedTable::Open(const ImmutableCFOptions& ioptions,
std::unique_ptr<FilePrefetchBuffer> prefetch_buffer; std::unique_ptr<FilePrefetchBuffer> prefetch_buffer;
// Before read footer, readahead backwards to prefetch data // prefetch both index and filters, down to all partitions
const size_t kTailPrefetchSize = 512 * 1024; const bool prefetch_all = prefetch_index_and_filter_in_cache || level == 0;
const bool preload_all = !table_options.cache_index_and_filter_blocks;
// Before read footer, readahead backwards to prefetch data. Do more readahead
// if we're going to read index/filter.
// TODO: This may incorrectly select small readahead in case partitioned
// index/filter is enabled and top-level partition pinning is enabled. That's
// because we need to issue readahead before we read the properties, at which
// point we don't yet know the index type.
const size_t kTailPrefetchSize =
prefetch_all || preload_all ? 512 * 1024 : 4 * 1024;
size_t prefetch_off; size_t prefetch_off;
size_t prefetch_len; size_t prefetch_len;
if (file_size < kTailPrefetchSize) { if (file_size < kTailPrefetchSize) {
@ -945,8 +954,6 @@ Status BlockBasedTable::Open(const ImmutableCFOptions& ioptions,
bool need_upper_bound_check = bool need_upper_bound_check =
PrefixExtractorChanged(rep->table_properties.get(), prefix_extractor); PrefixExtractorChanged(rep->table_properties.get(), prefix_extractor);
// prefetch both index and filters, down to all partitions
const bool prefetch_all = prefetch_index_and_filter_in_cache || level == 0;
BlockBasedTableOptions::IndexType index_type = new_table->UpdateIndexType(); BlockBasedTableOptions::IndexType index_type = new_table->UpdateIndexType();
// prefetch the first level of index // prefetch the first level of index
const bool prefetch_index = const bool prefetch_index =