andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
91b7553293
rocksdb/table
History
Cheng Chang 91b7553293 Enable IO Uring in MultiGet in direct IO mode (#6815) 
Summary:
Currently, in direct IO mode, `MultiGet` retrieves the data blocks one by one instead of in parallel, see `BlockBasedTable::RetrieveMultipleBlocks`.

Since direct IO is supported in `RandomAccessFileReader::MultiRead` in https://github.com/facebook/rocksdb/pull/6446, this PR applies `MultiRead` to `MultiGet` so that the data blocks can be retrieved in parallel.

Also, in direct IO mode and when data blocks are compressed and need to uncompressed, this PR only allocates one continuous aligned buffer to hold the data blocks, and then directly uncompress the blocks to insert into block cache, there is no longer intermediate copies to scratch buffers.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/6815

Test Plan:
1. added a new unit test `BlockBasedTableReaderTest::MultiGet`.
2. existing unit tests and stress tests  contain tests against `MultiGet` in direct IO mode.

Reviewed By: anand1976

Differential Revision: D21426347

Pulled By: cheng-chang

fbshipit-source-id: b8446ae0e74152444ef9111e97f8e402ac31b24f
2020-05-14 23:26:26 -07:00
..
adaptive Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
block_based Enable IO Uring in MultiGet in direct IO mode (#6815) 2020-05-14 23:26:26 -07:00
cuckoo Pass a timeout to FileSystem for random reads (#6751) 2020-04-30 14:50:39 -07:00
plain Pass a timeout to FileSystem for random reads (#6751) 2020-04-30 14:50:39 -07:00
block_fetcher_test.cc Enable IO Uring in MultiGet in direct IO mode (#6815) 2020-05-14 23:26:26 -07:00
block_fetcher.cc Pass a timeout to FileSystem for random reads (#6751) 2020-04-30 14:50:39 -07:00
block_fetcher.h Reduce memory copies when fetching and uncompressing blocks from SST files (#6689) 2020-04-24 15:32:56 -07:00
cleanable_test.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
format.cc Add tests for compression failure in BlockBasedTableBuilder (#6709) 2020-05-12 09:27:35 -07:00
format.h Basic MultiGet support for partitioned filters (#6757) 2020-04-28 14:49:34 -07:00
get_context.cc Stats for redundant insertions into block cache (#6681) 2020-04-27 13:20:27 -07:00
get_context.h Stats for redundant insertions into block cache (#6681) 2020-04-27 13:20:27 -07:00
internal_iterator.h Properly report IO errors when IndexType::kBinarySearchWithFirstKey is used (#6621) 2020-04-15 17:40:44 -07:00
iter_heap.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
iterator_wrapper.h Properly report IO errors when IndexType::kBinarySearchWithFirstKey is used (#6621) 2020-04-15 17:40:44 -07:00
iterator.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
merger_test.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
merging_iterator.cc Properly report IO errors when IndexType::kBinarySearchWithFirstKey is used (#6621) 2020-04-15 17:40:44 -07:00
merging_iterator.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
meta_blocks.cc sst_dump to reduce number of file reads (#6836) 2020-05-12 18:23:33 -07:00
meta_blocks.h sst_dump to reduce number of file reads (#6836) 2020-05-12 18:23:33 -07:00
mock_table.cc Pass a timeout to FileSystem for random reads (#6751) 2020-04-30 14:50:39 -07:00
mock_table.h Properly report IO errors when IndexType::kBinarySearchWithFirstKey is used (#6621) 2020-04-15 17:40:44 -07:00
multiget_context.h Basic MultiGet support for partitioned filters (#6757) 2020-04-28 14:49:34 -07:00
persistent_cache_helper.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
persistent_cache_helper.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
persistent_cache_options.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
scoped_arena_iterator.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
sst_file_reader_test.cc Allow table/sst_file_reader_test.cc to use custom Env (#6536) 2020-03-17 11:02:13 -07:00
sst_file_reader.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
sst_file_writer_collectors.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
sst_file_writer.cc Make options.bottommost_compression, compression_opts and bottommost_compression_opts dynamically changeable. (#6615) 2020-03-31 12:11:42 -07:00
table_builder.h sst_dump to reduce number of file reads (#6836) 2020-05-12 18:23:33 -07:00
table_properties_internal.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
table_properties.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
table_reader_bench.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
table_reader_caller.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
table_reader.h Properly report IO errors when IndexType::kBinarySearchWithFirstKey is used (#6621) 2020-04-15 17:40:44 -07:00
table_test.cc sst_dump to reduce number of file reads (#6836) 2020-05-12 18:23:33 -07:00
two_level_iterator.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
two_level_iterator.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00