andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
ab01da5437
rocksdb/table
History
Islam AbdelRahman ab01da5437 Support SST files with Global sequence numbers 
Summary:
- Update SstFileWriter to include a property for a global sequence number in the SST file `rocksdb.external_sst_file.global_seqno`
- Update TableProperties to be aware of the offset of each property in the file
- Update BlockBasedTableReader and Block to be able to honor the sequence number in `rocksdb.external_sst_file.global_seqno` property and use it to overwrite all sequence number in the file

Something worth mentioning is that we don't update the seqno in the index block since and when doing a binary search, the reason for that is that it's guaranteed that SST files with global seqno will have only one user_key and each key will have seqno=0 encoded in it, This mean that this key is greater than any other key with seqno> 0. That mean that we can actually keep the current logic for these blocks

Test Plan: unit tests

Reviewers: andrewkr, yhchiang, yiwu, sdong

Reviewed By: sdong

Subscribers: hcz, andrewkr, dhruba

Differential Revision: https://reviews.facebook.net/D62523
2016-10-03 16:12:39 -07:00
..
adaptive_table_factory.cc solve the problem of table_factory_to_write_=nullptr (#1342) 2016-09-20 10:11:51 -07:00
adaptive_table_factory.h Only cache level 0 indexes and filter when opening table reader 2016-07-20 11:23:31 -07:00
block_based_filter_block_test.cc Add statistics field to show total size of index and filter blocks in block cache 2016-06-03 10:47:47 -07:00
block_based_filter_block.cc BlockBasedTable::FullFilterKeyMayMatch() Should skip prefix bloom if full key bloom exists 2016-06-10 16:27:56 -07:00
block_based_filter_block.h BlockBasedTable::FullFilterKeyMayMatch() Should skip prefix bloom if full key bloom exists 2016-06-10 16:27:56 -07:00
block_based_table_builder.cc Support SST files with Global sequence numbers 2016-10-03 16:12:39 -07:00
block_based_table_builder.h Experiments on column-aware encodings 2016-08-01 14:50:19 -07:00
block_based_table_factory.cc Only cache level 0 indexes and filter when opening table reader 2016-07-20 11:23:31 -07:00
block_based_table_factory.h Only cache level 0 indexes and filter when opening table reader 2016-07-20 11:23:31 -07:00
block_based_table_reader.cc Support SST files with Global sequence numbers 2016-10-03 16:12:39 -07:00
block_based_table_reader.h Remove ImmutableCFOptions from public API 2016-09-02 14:16:31 -07:00
block_builder.cc Miscellaneous performance improvements 2016-07-12 14:15:32 -07:00
block_builder.h TableBuilder / TableReader support for range deletion 2016-08-19 15:10:31 -07:00
block_prefix_index.cc Fix clang analyzer errors 2016-07-08 17:50:51 -07:00
block_prefix_index.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
block_test.cc Support SST files with Global sequence numbers 2016-10-03 16:12:39 -07:00
block.cc Support SST files with Global sequence numbers 2016-10-03 16:12:39 -07:00
block.h Support SST files with Global sequence numbers 2016-10-03 16:12:39 -07:00
bloom_block.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
bloom_block.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
cuckoo_table_builder_test.cc New Statistics to track Compression/Decompression (#1197) 2016-07-19 09:44:03 -07:00
cuckoo_table_builder.cc Embed column family name in SST file 2016-04-06 23:10:32 -07:00
cuckoo_table_builder.h Embed column family name in SST file 2016-04-06 23:10:32 -07:00
cuckoo_table_factory.cc Only cache level 0 indexes and filter when opening table reader 2016-07-20 11:23:31 -07:00
cuckoo_table_factory.h Only cache level 0 indexes and filter when opening table reader 2016-07-20 11:23:31 -07:00
cuckoo_table_reader_test.cc Embed column family name in SST file 2016-04-06 23:10:32 -07:00
cuckoo_table_reader.cc Add SeekForPrev() to Iterator 2016-09-27 18:20:57 -07:00
cuckoo_table_reader.h Remove ImmutableCFOptions from public API 2016-09-02 14:16:31 -07:00
filter_block.h BlockBasedTable::FullFilterKeyMayMatch() Should skip prefix bloom if full key bloom exists 2016-06-10 16:27:56 -07:00
flush_block_policy.cc Miscellaneous performance improvements 2016-07-12 14:15:32 -07:00
format.cc Support ZSTD with finalized format 2016-09-06 12:22:16 -07:00
format.h Remove ImmutableCFOptions from public API 2016-09-02 14:16:31 -07:00
full_filter_block_test.cc Add statistics field to show total size of index and filter blocks in block cache 2016-06-03 10:47:47 -07:00
full_filter_block.cc BlockBasedTable::FullFilterKeyMayMatch() Should skip prefix bloom if full key bloom exists 2016-06-10 16:27:56 -07:00
full_filter_block.h BlockBasedTable::FullFilterKeyMayMatch() Should skip prefix bloom if full key bloom exists 2016-06-10 16:27:56 -07:00
get_context.cc Introduce FullMergeV2 (eliminate memcpy from merge operators) 2016-07-20 09:49:03 -07:00
get_context.h Introduce FullMergeV2 (eliminate memcpy from merge operators) 2016-07-20 09:49:03 -07:00
internal_iterator.h Add SeekForPrev() to Iterator 2016-09-27 18:20:57 -07:00
iter_heap.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
iterator_wrapper.h Add SeekForPrev() to Iterator 2016-09-27 18:20:57 -07:00
iterator.cc Add SeekForPrev() to Iterator 2016-09-27 18:20:57 -07:00
merger_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
merger.cc Add SeekForPrev() to Iterator 2016-09-27 18:20:57 -07:00
merger.h Temporarily revert Prev() prefix support 2016-09-08 14:45:32 -07:00
meta_blocks.cc Support SST files with Global sequence numbers 2016-10-03 16:12:39 -07:00
meta_blocks.h New Statistics to track Compression/Decompression (#1197) 2016-07-19 09:44:03 -07:00
mock_table.cc Only cache level 0 indexes and filter when opening table reader 2016-07-20 11:23:31 -07:00
mock_table.h Add SeekForPrev() to Iterator 2016-09-27 18:20:57 -07:00
persistent_cache_helper.cc Added PersistentCache abstraction 2016-05-15 22:17:18 -07:00
persistent_cache_helper.h Added PersistentCache abstraction 2016-05-15 22:17:18 -07:00
plain_table_builder.cc store prefix_extractor_name in table 2016-08-26 11:46:32 -07:00
plain_table_builder.h Embed column family name in SST file 2016-04-06 23:10:32 -07:00
plain_table_factory.cc store prefix_extractor_name in table 2016-08-26 11:46:32 -07:00
plain_table_factory.h Only cache level 0 indexes and filter when opening table reader 2016-07-20 11:23:31 -07:00
plain_table_index.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
plain_table_index.h Remove ImmutableCFOptions from public API 2016-09-02 14:16:31 -07:00
plain_table_key_coding.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
plain_table_key_coding.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
plain_table_reader.cc Add SeekForPrev() to Iterator 2016-09-27 18:20:57 -07:00
plain_table_reader.h Skip bottom-level filter block caching when hit-optimized 2015-12-23 10:15:07 -08:00
scoped_arena_iterator.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
sst_file_writer_collectors.h Support SST files with Global sequence numbers 2016-10-03 16:12:39 -07:00
sst_file_writer.cc Support SST files with Global sequence numbers 2016-10-03 16:12:39 -07:00
table_builder.h Add TableBuilderOptions::level and relevant changes (#1335) 2016-09-17 22:30:43 -07:00
table_properties_internal.h TableBuilder / TableReader support for range deletion 2016-08-19 15:10:31 -07:00
table_properties.cc store prefix_extractor_name in table 2016-08-26 11:46:32 -07:00
table_reader_bench.cc Add TableBuilderOptions::level and relevant changes (#1335) 2016-09-17 22:30:43 -07:00
table_reader.h TableBuilder / TableReader support for range deletion 2016-08-19 15:10:31 -07:00
table_test.cc Support SST files with Global sequence numbers 2016-10-03 16:12:39 -07:00
two_level_iterator.cc Add SeekForPrev() to Iterator 2016-09-27 18:20:57 -07:00
two_level_iterator.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00