andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
8585059ae0
rocksdb/table
History
Zhongyi Xie 80bc35927c Should only decode restart points for uncompressed blocks (#3996) 
Summary:
The Block object assumes contents are uncompressed. Block's constructor tries to read the number of restarts, but does not get an accurate number when its contents are compressed, which is causing issues like https://github.com/facebook/rocksdb/issues/3843.
This PR address this issue by skipping reconstruction of restart points when blocks are known to be compressed. Somehow the restart points can be read directly when Snappy is used and some tests (for example https://github.com/facebook/rocksdb/blob/master/db/db_block_cache_test.cc#L196) expects blocks to be fully constructed even when Snappy compression is used, so here we keep the restart point logic for Snappy.
Closes https://github.com/facebook/rocksdb/pull/3996

Differential Revision: D8416186

Pulled By: miasantreble

fbshipit-source-id: 002c0b62b9e5d89fb7736563d354ce0023c8cb28
2018-06-15 19:26:58 -07:00
..
adaptive_table_factory.cc Comment out unused variables 2018-03-05 13:13:41 -08:00
adaptive_table_factory.h Comment out unused variables 2018-03-05 13:13:41 -08:00
block_based_filter_block_test.cc Move prefix_extractor to MutableCFOptions 2018-05-21 14:43:11 -07:00
block_based_filter_block.cc Move prefix_extractor to MutableCFOptions 2018-05-21 14:43:11 -07:00
block_based_filter_block.h Move prefix_extractor to MutableCFOptions 2018-05-21 14:43:11 -07:00
block_based_table_builder.cc run make format for PR 3838 (#3954) 2018-06-05 12:58:02 -07:00
block_based_table_builder.h run make format for PR 3838 (#3954) 2018-06-05 12:58:02 -07:00
block_based_table_factory.cc Move prefix_extractor to MutableCFOptions 2018-05-21 14:43:11 -07:00
block_based_table_factory.h Align SST file data blocks to avoid spanning multiple pages 2018-03-26 20:26:10 -07:00
block_based_table_reader.cc Should only decode restart points for uncompressed blocks (#3996) 2018-06-15 19:26:58 -07:00
block_based_table_reader.h Exclude seq from index keys 2018-05-25 18:42:43 -07:00
block_builder.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
block_builder.h Change RocksDB License 2017-07-15 16:11:23 -07:00
block_fetcher.cc run make format for PR 3838 (#3954) 2018-06-05 12:58:02 -07:00
block_fetcher.h Fix BlockFetcher ASAN error 2017-12-12 12:12:38 -08:00
block_prefix_index.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
block_prefix_index.h Change RocksDB License 2017-07-15 16:11:23 -07:00
block_test.cc Exclude seq from index keys 2018-05-25 18:42:43 -07:00
block.cc Should only decode restart points for uncompressed blocks (#3996) 2018-06-15 19:26:58 -07:00
block.h Fix performance regression in Get() for block-based tables (#3953) 2018-06-05 11:43:16 -07:00
bloom_block.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
bloom_block.h Change RocksDB License 2017-07-15 16:11:23 -07:00
cleanable_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
cuckoo_table_builder_test.cc Should only decode restart points for uncompressed blocks (#3996) 2018-06-15 19:26:58 -07:00
cuckoo_table_builder.cc Enable MSVC W4 with a few exceptions. Fix warnings and bugs 2017-10-19 10:57:12 -07:00
cuckoo_table_builder.h Change RocksDB License 2017-07-15 16:11:23 -07:00
cuckoo_table_factory.cc Comment out unused variables 2018-03-05 13:13:41 -08:00
cuckoo_table_factory.h comment unused parameters to turn on -Wunused-parameter flag 2018-04-12 17:59:16 -07:00
cuckoo_table_reader_test.cc Move prefix_extractor to MutableCFOptions 2018-05-21 14:43:11 -07:00
cuckoo_table_reader.cc Should only decode restart points for uncompressed blocks (#3996) 2018-06-15 19:26:58 -07:00
cuckoo_table_reader.h Move prefix_extractor to MutableCFOptions 2018-05-21 14:43:11 -07:00
filter_block.h Move prefix_extractor to MutableCFOptions 2018-05-21 14:43:11 -07:00
flush_block_policy.cc Align SST file data blocks to avoid spanning multiple pages 2018-03-26 20:26:10 -07:00
format.cc run make format for PR 3838 (#3954) 2018-06-05 12:58:02 -07:00
format.h run make format for PR 3838 (#3954) 2018-06-05 12:58:02 -07:00
full_filter_bits_builder.h Skip duplicate bloom keys when whole_key and prefix are mixed 2018-04-24 10:58:16 -07:00
full_filter_block_test.cc Move prefix_extractor to MutableCFOptions 2018-05-21 14:43:11 -07:00
full_filter_block.cc Move prefix_extractor to MutableCFOptions 2018-05-21 14:43:11 -07:00
full_filter_block.h Move prefix_extractor to MutableCFOptions 2018-05-21 14:43:11 -07:00
get_context.cc comment unused parameters to turn on -Wunused-parameter flag 2018-04-12 17:59:16 -07:00
get_context.h Stats for false positive rate of full filtesr 2018-04-05 15:58:48 -07:00
index_builder.cc Extend format 3 to partitioned index/filters (#3958) 2018-06-06 16:58:16 -07:00
index_builder.h Extend format 3 to partitioned index/filters (#3958) 2018-06-06 16:58:16 -07:00
internal_iterator.h Change and clarify the relationship between Valid(), status() and Seek*() for all iterators. Also fix some bugs 2018-05-17 02:56:56 -07:00
iter_heap.h Make InternalKeyComparator final and directly use it in merging iterator 2017-09-11 12:04:21 -07:00
iterator_wrapper.h Change and clarify the relationship between Valid(), status() and Seek*() for all iterators. Also fix some bugs 2018-05-17 02:56:56 -07:00
iterator.cc Comment out unused variables 2018-03-05 13:13:41 -08:00
merger_test.cc Make InternalKeyComparator final and directly use it in merging iterator 2017-09-11 12:04:21 -07:00
merging_iterator.cc Fix regression bug of Prev() with upper bound (#3989) 2018-06-12 16:57:36 -07:00
merging_iterator.h fix DBImpl::NewInternalIterator super-version leak on failure 2017-10-11 14:57:43 -07:00
meta_blocks.cc Should only decode restart points for uncompressed blocks (#3996) 2018-06-15 19:26:58 -07:00
meta_blocks.h Should only decode restart points for uncompressed blocks (#3996) 2018-06-15 19:26:58 -07:00
mock_table.cc Move prefix_extractor to MutableCFOptions 2018-05-21 14:43:11 -07:00
mock_table.h Move prefix_extractor to MutableCFOptions 2018-05-21 14:43:11 -07:00
partitioned_filter_block_test.cc Extend format 3 to partitioned index/filters (#3958) 2018-06-06 16:58:16 -07:00
partitioned_filter_block.cc Extend format 3 to partitioned index/filters (#3958) 2018-06-06 16:58:16 -07:00
partitioned_filter_block.h Extend format 3 to partitioned index/filters (#3958) 2018-06-06 16:58:16 -07:00
persistent_cache_helper.cc comment unused parameters to turn on -Wunused-parameter flag 2018-04-12 17:59:16 -07:00
persistent_cache_helper.h Change RocksDB License 2017-07-15 16:11:23 -07:00
persistent_cache_options.h Change RocksDB License 2017-07-15 16:11:23 -07:00
plain_table_builder.cc Move prefix_extractor to MutableCFOptions 2018-05-21 14:43:11 -07:00
plain_table_builder.h Move prefix_extractor to MutableCFOptions 2018-05-21 14:43:11 -07:00
plain_table_factory.cc Move prefix_extractor to MutableCFOptions 2018-05-21 14:43:11 -07:00
plain_table_factory.h Comment out unused variables 2018-03-05 13:13:41 -08:00
plain_table_index.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
plain_table_index.h Move prefix_extractor to MutableCFOptions 2018-05-21 14:43:11 -07:00
plain_table_key_coding.cc Comment out unused variables 2018-03-05 13:13:41 -08:00
plain_table_key_coding.h Change RocksDB License 2017-07-15 16:11:23 -07:00
plain_table_reader.cc Should only decode restart points for uncompressed blocks (#3996) 2018-06-15 19:26:58 -07:00
plain_table_reader.h Move prefix_extractor to MutableCFOptions 2018-05-21 14:43:11 -07:00
scoped_arena_iterator.h Change RocksDB License 2017-07-15 16:11:23 -07:00
sst_file_writer_collectors.h Comment out unused variables 2018-03-05 13:13:41 -08:00
sst_file_writer.cc Move prefix_extractor to MutableCFOptions 2018-05-21 14:43:11 -07:00
table_builder.h Move prefix_extractor to MutableCFOptions 2018-05-21 14:43:11 -07:00
table_properties_internal.h Eliminate some redundant block reads. 2018-01-10 17:11:58 -08:00
table_properties.cc Exclude seq from index keys 2018-05-25 18:42:43 -07:00
table_reader_bench.cc Move prefix_extractor to MutableCFOptions 2018-05-21 14:43:11 -07:00
table_reader.h Move prefix_extractor to MutableCFOptions 2018-05-21 14:43:11 -07:00
table_test.cc Should only decode restart points for uncompressed blocks (#3996) 2018-06-15 19:26:58 -07:00
two_level_iterator.cc Change and clarify the relationship between Valid(), status() and Seek*() for all iterators. Also fix some bugs 2018-05-17 02:56:56 -07:00
two_level_iterator.h fix memory leak in two_level_iterator 2018-04-15 17:26:26 -07:00