andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
5746767387
rocksdb/table
History
Andrew Kryczka af54c4092a fix SstFileWriter with dictionary compression (#7323) 
Summary:
In block-based table builder, the cut-over from buffered to unbuffered
mode involves sampling the buffered blocks and generating a dictionary.
There was a bug where `SstFileWriter` passed zero as the `target_file_size`
causing the cutover to happen immediately, so there were no samples
available for generating the dictionary.

This PR changes the meaning of `target_file_size == 0` to mean buffer
the whole file before cutting over. It also adds dictionary compression
support to `sst_dump --command=recompress` for easy evaluation.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/7323

Reviewed By: cheng-chang

Differential Revision: D23412158

Pulled By: ajkr

fbshipit-source-id: 3b232050e70ef3c2ee85a4b5f6fadb139c569873
2020-09-03 15:49:57 -07:00
..
adaptive Extend Get/MultiGet deadline support to table open (#6982) 2020-06-29 14:53:17 -07:00
block_based fix SstFileWriter with dictionary compression (#7323) 2020-09-03 15:49:57 -07:00
cuckoo Fix cmake build on MacOS (#7205) 2020-08-10 10:48:05 -07:00
plain Fixed Factory construct just for calling .Name() (#7080) 2020-07-08 11:54:00 -07:00
block_fetcher_test.cc More Makefile Cleanup (#7097) 2020-07-09 14:35:17 -07:00
block_fetcher.cc Skip unnecessary allocation for mmap reads under 5000 bytes (#7043) 2020-06-30 15:40:40 -07:00
block_fetcher.h Reduce memory copies when fetching and uncompressing blocks from SST files (#6689) 2020-04-24 15:32:56 -07:00
cleanable_test.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
format.cc Clean up CompressBlock/CompressBlockInternal a bit (#7249) 2020-08-12 18:25:48 -07:00
format.h Clean up CompressBlock/CompressBlockInternal a bit (#7249) 2020-08-12 18:25:48 -07:00
get_context.cc Add timestamp to delete (#6253) 2020-05-28 10:40:03 -07:00
get_context.h Stats for redundant insertions into block cache (#6681) 2020-04-27 13:20:27 -07:00
internal_iterator.h Clean up InternalIterator upper bound logic a little bit (#7200) 2020-08-05 10:44:57 -07:00
iter_heap.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
iterator_wrapper.h Clean up InternalIterator upper bound logic a little bit (#7200) 2020-08-05 10:44:57 -07:00
iterator.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
merger_test.cc More Makefile Cleanup (#7097) 2020-07-09 14:35:17 -07:00
merging_iterator.cc Add some simulator cache and block tracer tests to ASSERT_STATUS_CHECKED (#7305) 2020-08-24 16:43:31 -07:00
merging_iterator.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
meta_blocks.cc Separate internal and user key comparators in BlockIter (#6944) 2020-07-07 17:26:16 -07:00
meta_blocks.h Extend Get/MultiGet deadline support to table open (#6982) 2020-06-29 14:53:17 -07:00
mock_table.cc Add hash of key/value checks when paranoid_file_checks=true (#7134) 2020-07-22 11:04:40 -07:00
mock_table.h Add hash of key/value checks when paranoid_file_checks=true (#7134) 2020-07-22 11:04:40 -07:00
multiget_context.h Allow MultiGet users to limit cumulative value size (#6826) 2020-05-27 13:07:14 -07:00
persistent_cache_helper.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
persistent_cache_helper.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
persistent_cache_options.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
scoped_arena_iterator.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
sst_file_dumper.cc fix SstFileWriter with dictionary compression (#7323) 2020-09-03 15:49:57 -07:00
sst_file_dumper.h fix SstFileWriter with dictionary compression (#7323) 2020-09-03 15:49:57 -07:00
sst_file_reader_test.cc Allow table/sst_file_reader_test.cc to use custom Env (#6536) 2020-03-17 11:02:13 -07:00
sst_file_reader.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
sst_file_writer_collectors.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
sst_file_writer.cc Store DB identity and DB session ID in SST files (#6983) 2020-06-17 10:57:40 -07:00
table_builder.h Store DB identity and DB session ID in SST files (#6983) 2020-06-17 10:57:40 -07:00
table_properties_internal.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
table_properties.cc Store DB identity and DB session ID in SST files (#6983) 2020-06-17 10:57:40 -07:00
table_reader_bench.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
table_reader_caller.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
table_reader.h dedup ReadOptions in iterator hierarchy (#7210) 2020-08-03 15:23:04 -07:00
table_test.cc Clean up InternalIterator upper bound logic a little bit (#7200) 2020-08-05 10:44:57 -07:00
two_level_iterator.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
two_level_iterator.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00