rocksdb/table
Zhichao Cao cddd637997 Merge adjacent file block reads in RocksDB MultiGet() and Add uncompressed block to cache (#6089)
Summary:
In the current MultiGet, if the KV-pairs do not belong to the data blocks in the block cache, multiple blocks are read from a SST. It will trigger one block read for each block request and read them in parallel. In some cases, if some data blocks are adjacent in the SST, the reads for these blocks can be combined to a single large read, which can reduce the system calls and reduce the read latency if possible.

Considering to fill the block cache, if multiple data blocks are in the same memory buffer, we need to copy them to the heap separately. Therefore, only in the case that 1) data block compression is enabled, and 2) compressed block cache is null, we can do combined read. Otherwise, extra memory copy is needed, which may cause extra overhead. In the current case, data blocks will be uncompressed to a new memory space.

Also, in the case that 1) data block compression is enabled, and 2) compressed block cache is null, it is possible the data block is actually not compressed. In the current logic, these data blocks will not be added to the uncompressed_cache. So if memory buffer is shared and the data block is not compressed, the data block are copied to the head and fill the cache.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/6089

Test Plan: Added test case to ParallelIO.MultiGet. Pass make asan_check

Differential Revision: D18734668

Pulled By: zhichao-cao

fbshipit-source-id: 67c5615ed373e51e42635fd74b36f8f3a66d5da4
2019-12-16 16:26:03 -08:00
..
adaptive Organizing rocksdb/table directory by format 2019-05-30 14:51:11 -07:00
block_based Merge adjacent file block reads in RocksDB MultiGet() and Add uncompressed block to cache (#6089) 2019-12-16 16:26:03 -08:00
cuckoo Introduce a new storage specific Env API (#5761) 2019-12-13 14:48:41 -08:00
plain Remove a few unnecessary includes 2019-11-19 08:20:42 -08:00
block_fetcher.cc Divide file_reader_writer.h and .cc (#5803) 2019-09-16 10:33:51 -07:00
block_fetcher.h Combine the read-ahead logic for user reads and compaction reads (#5431) 2019-06-19 14:10:46 -07:00
cleanable_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
format.cc Add new persistent 64-bit hash (#5984) 2019-10-31 16:36:35 -07:00
format.h New Bloom filter implementation for full and partitioned filters (#6007) 2019-11-13 16:44:01 -08:00
get_context.cc New API to get all merge operands for a Key (#5604) 2019-08-06 14:26:44 -07:00
get_context.h Fix local includes 2019-08-22 16:21:47 -07:00
internal_iterator.h Revert "Merging iterator to avoid child iterator reseek for some cases (#5286)" (#5871) 2019-10-01 11:22:41 -07:00
iter_heap.h Make InternalKeyComparator final and directly use it in merging iterator 2017-09-11 12:04:21 -07:00
iterator_wrapper.h Revert "Merging iterator to avoid child iterator reseek for some cases (#5286)" (#5871) 2019-10-01 11:22:41 -07:00
iterator.cc Add an option to put first key of each sst block in the index (#5289) 2019-06-24 20:54:04 -07:00
merger_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
merging_iterator.cc Fix a timer bug in MergingIterator::Seek() caused by #5871 (#5874) 2019-10-01 19:26:18 -07:00
merging_iterator.h Index value delta encoding (#3983) 2018-08-09 16:58:40 -07:00
meta_blocks.cc Divide file_reader_writer.h and .cc (#5803) 2019-09-16 10:33:51 -07:00
meta_blocks.h Fix comment of function NotifyCollectTableCollectorsOnFinish (#5738) 2019-08-29 10:57:01 -07:00
mock_table.cc Introduce a new storage specific Env API (#5761) 2019-12-13 14:48:41 -08:00
mock_table.h Apply formatter to recent 200+ commits. (#5830) 2019-09-20 12:04:26 -07:00
multiget_context.h Batched MultiGet API for multiple column families (#5816) 2019-11-12 13:52:55 -08:00
persistent_cache_helper.cc Organizing rocksdb/table directory by format 2019-05-30 14:51:11 -07:00
persistent_cache_helper.h Change RocksDB License 2017-07-15 16:11:23 -07:00
persistent_cache_options.h Change RocksDB License 2017-07-15 16:11:23 -07:00
scoped_arena_iterator.h Change RocksDB License 2017-07-15 16:11:23 -07:00
sst_file_reader_test.cc simplify include directive involving inttypes (#5402) 2019-06-06 13:56:07 -07:00
sst_file_reader.cc Introduce a new storage specific Env API (#5761) 2019-12-13 14:48:41 -08:00
sst_file_writer_collectors.h Fix SstFileReader not able to open ingested file (#5097) 2019-03-26 10:25:18 -07:00
sst_file_writer.cc Introduce a new storage specific Env API (#5761) 2019-12-13 14:48:41 -08:00
table_builder.h Divide file_reader_writer.h and .cc (#5803) 2019-09-16 10:33:51 -07:00
table_properties_internal.h Index value delta encoding (#3983) 2018-08-09 16:58:40 -07:00
table_properties.cc Organizing rocksdb/table directory by format 2019-05-30 14:51:11 -07:00
table_reader_bench.cc Introduce a new storage specific Env API (#5761) 2019-12-13 14:48:41 -08:00
table_reader_caller.h Add more callers for table reader. (#5454) 2019-06-20 14:31:48 -07:00
table_reader.h Apply formatter to recent 200+ commits. (#5830) 2019-09-20 12:04:26 -07:00
table_test.cc Introduce a new storage specific Env API (#5761) 2019-12-13 14:48:41 -08:00
two_level_iterator.cc Add an option to put first key of each sst block in the index (#5289) 2019-06-24 20:54:04 -07:00
two_level_iterator.h Add an option to put first key of each sst block in the index (#5289) 2019-06-24 20:54:04 -07:00