andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
6.14.fb.laser
rocksdb/db
History
Andrew Kryczka 51c8841418 Expand effect of dictionary settings in ColumnFamilyOptions::compression_opts (#7619) 
Summary:
In dictionary compression's initial implementation, in order to save CPU overhead, we only enabled it
for bottom level under the assumption that the vast majority of data is
stored there. At that time, there was no
such thing as `ColumnFamilyOptions::bottommost_compression_opts`, so we just
hardcoded disabling dictionary compression in flush and compactions to
non-bottommost level. Now, we have users who generate all their files
through flush and are considering using dictionary compression.

To support such a use case, this PR expands the scope of `ColumnFamilyOptions::compression_opts` to
additionally include flushed files and files generated by compaction to
a non-bottommost level. Users can still get the old behavior by moving
their dictionary settings to `ColumnFamilyOptions::bottommost_compression_opts`
and explicitly enabling both that and `ColumnFamilyOptions::bottommost_compression`.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/7619

Reviewed By: ltamasi

Differential Revision: D24665610

Pulled By: ajkr

fbshipit-source-id: 656b90bce1033fe21c71e09af931ef5bde3e464c
2020-11-03 10:56:38 -08:00
..
blob Do not rely on the two-argument std::pair constructor being constexpr (#7519) 2020-10-08 10:49:40 -07:00
compaction Expand effect of dictionary settings in ColumnFamilyOptions::compression_opts (#7619) 2020-11-03 10:56:38 -08:00
db_impl Bug fix to remove function calling in assert statement (#7581) 2020-10-21 20:25:30 -07:00
arena_wrapped_db_iter.cc dedup ReadOptions in iterator hierarchy (#7210) 2020-08-03 15:23:04 -07:00
arena_wrapped_db_iter.h dedup ReadOptions in iterator hierarchy (#7210) 2020-08-03 15:23:04 -07:00
builder.cc Expand effect of dictionary settings in ColumnFamilyOptions::compression_opts (#7619) 2020-11-03 10:56:38 -08:00
builder.h Integrate blob file writing with the flush logic (#7345) 2020-09-14 21:11:43 -07:00
c_test.c Fix MSVC-related build issues (#7439) 2020-10-01 09:23:04 -07:00
c.cc RocksJava - Add errorIfLogFileExists parameter to RocksDB.openReadOnly (#7046) 2020-09-17 15:41:25 -07:00
column_family_test.cc Add status check enforcement for column_family_test.cc (#7484) 2020-10-02 13:35:15 -07:00
column_family.cc Change ParseInternalKey() to return Status instead of bool (#7457) 2020-09-30 19:16:47 -07:00
column_family.h Store FSRandomAccessPtr object in RandomAccessFileReader (#7192) 2020-08-27 11:21:52 -07:00
compact_files_test.cc Replace reinterpret_cast with static_cast_with_check (#7067) 2020-07-02 19:25:41 -07:00
compacted_db_impl.cc Periodically flush info log out of application buffer (#7488) 2020-10-01 19:14:14 -07:00
compacted_db_impl.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
comparator_db_test.cc More Makefile Cleanup (#7097) 2020-07-09 14:35:17 -07:00
convenience.cc sst_dump to reduce number of file reads (#6836) 2020-05-12 18:23:33 -07:00
corruption_test.cc Add missing regression test/release note for bug fix 2020-10-13 10:01:03 -07:00
cuckoo_table_db_test.cc Replace reinterpret_cast with static_cast_with_check (#7067) 2020-07-02 19:25:41 -07:00
db_basic_test.cc Add Stats for MultiGet (#7366) 2020-10-07 13:28:48 -07:00
db_block_cache_test.cc Expand effect of dictionary settings in ColumnFamilyOptions::compression_opts (#7619) 2020-11-03 10:56:38 -08:00
db_bloom_filter_test.cc Bring the Configurable options together (#5753) 2020-09-14 17:01:01 -07:00
db_compaction_filter_test.cc Change ParseInternalKey() to return Status instead of bool (#7457) 2020-09-30 19:16:47 -07:00
db_compaction_test.cc Disallow trivial move if BottommostLevelCompaction is kForce* (#7368) 2020-10-07 13:19:31 -07:00
db_dynamic_level_test.cc Whole DBTest to skip fsync (#7274) 2020-08-17 18:42:25 -07:00
db_encryption_test.cc Whole DBTest to skip fsync (#7274) 2020-08-17 18:42:25 -07:00
db_filesnapshot.cc Fix checkpoint file deletion race with avoid_unnecessary_blocking_io (#7369) 2020-09-10 22:35:25 -07:00
db_flush_test.cc Integrate blob file writing with the flush logic (#7345) 2020-09-14 21:11:43 -07:00
db_info_dumper.cc Add some simulator cache and block tracer tests to ASSERT_STATUS_CHECKED (#7305) 2020-08-24 16:43:31 -07:00
db_info_dumper.h Add a DB Session ID (#6959) 2020-06-15 10:47:02 -07:00
db_inplace_update_test.cc Whole DBTest to skip fsync (#7274) 2020-08-17 18:42:25 -07:00
db_io_failure_test.cc Whole DBTest to skip fsync (#7274) 2020-08-17 18:42:25 -07:00
db_iter_stress_test.cc Test CircleCI with CLANG-10 (#7025) 2020-06-24 16:22:49 -07:00
db_iter_test.cc Change ParseInternalKey() to return Status instead of bool (#7457) 2020-09-30 19:16:47 -07:00
db_iter.cc Make sure assert(false) handles failures too (#7483) 2020-10-06 13:52:42 -07:00
db_iter.h Make db_basic_test pass assert status checked (#7452) 2020-09-29 09:49:04 -07:00
db_iterator_test.cc Bring the Configurable options together (#5753) 2020-09-14 17:01:01 -07:00
db_log_iter_test.cc Whole DBTest to skip fsync (#7274) 2020-08-17 18:42:25 -07:00
db_logical_block_size_cache_test.cc Get block size only in direct IO mode (#6522) 2020-03-20 15:26:10 -07:00
db_memtable_test.cc Whole DBTest to skip fsync (#7274) 2020-08-17 18:42:25 -07:00
db_merge_operand_test.cc Whole DBTest to skip fsync (#7274) 2020-08-17 18:42:25 -07:00
db_merge_operator_test.cc Whole DBTest to skip fsync (#7274) 2020-08-17 18:42:25 -07:00
db_options_test.cc Add few unit test cases in ASSERT_STATUS_CHECKED build (#7427) 2020-09-24 21:48:57 -07:00
db_properties_test.cc Fix db_properties_test for ASSERT_STATUS_CHECKED (#7490) 2020-10-01 17:47:09 -07:00
db_range_del_test.cc Whole DBTest to skip fsync (#7274) 2020-08-17 18:42:25 -07:00
db_sst_test.cc add Status check assertions for repair_test (#7455) 2020-09-29 16:30:08 -07:00
db_statistics_test.cc Add a new stats level to exclude tickers (#7329) 2020-09-04 23:25:03 -07:00
db_table_properties_test.cc Whole DBTest to skip fsync (#7274) 2020-08-17 18:42:25 -07:00
db_tailing_iter_test.cc Whole DBTest to skip fsync (#7274) 2020-08-17 18:42:25 -07:00
db_test2.cc Expand effect of dictionary settings in ColumnFamilyOptions::compression_opts (#7619) 2020-11-03 10:56:38 -08:00
db_test_util.cc Enable ASSERT_STATUS_CHECKED for db_universal_compaction_test (#7460) 2020-10-06 14:42:12 -07:00
db_test_util.h add Status check assertions for repair_test (#7455) 2020-09-29 16:30:08 -07:00
db_test.cc Reduce the number of iterations in DBTest.FileCreationRandomFailure (#7481) 2020-10-01 10:42:58 -07:00
db_universal_compaction_test.cc Enable ASSERT_STATUS_CHECKED for db_universal_compaction_test (#7460) 2020-10-06 14:42:12 -07:00
db_wal_test.cc Integrate blob file writing with recovery (#7388) 2020-09-15 17:14:10 -07:00
db_with_timestamp_basic_test.cc Change ParseInternalKey() to return Status instead of bool (#7457) 2020-09-30 19:16:47 -07:00
db_with_timestamp_compaction_test.cc Whole DBTest to skip fsync (#7274) 2020-08-17 18:42:25 -07:00
db_write_test.cc Fix StallWrite crash with mixed of slowdown/no_slowdown writes (#7508) 2020-10-06 12:44:20 -07:00
dbformat_test.cc Change ParseInternalKey() to return Status instead of bool (#7457) 2020-09-30 19:16:47 -07:00
dbformat.cc Change ParseInternalKey() to return Status instead of bool (#7457) 2020-09-30 19:16:47 -07:00
dbformat.h Introduce options.check_flush_compaction_key_order (#7467) 2020-10-01 10:10:26 -07:00
deletefile_test.cc Whole DBTest to skip fsync (#7274) 2020-08-17 18:42:25 -07:00
error_handler_fs_test.cc Status check enforcement for error_handler_fs_test (#7342) 2020-10-02 16:41:13 -07:00
error_handler.cc Status check enforcement for timestamp_basic_test (#7454) 2020-09-29 18:23:27 -07:00
error_handler.h Map retryable IO error during Flush without WAL to soft error and no switch memtable during resume (#7310) 2020-09-17 20:25:45 -07:00
event_helpers.cc Status check enforcement for error_handler_fs_test (#7342) 2020-10-02 16:41:13 -07:00
event_helpers.h Pass SST file checksum information through OnTableFileCreated (#7108) 2020-08-25 10:46:11 -07:00
experimental.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
external_sst_file_basic_test.cc Whole DBTest to skip fsync (#7274) 2020-08-17 18:42:25 -07:00
external_sst_file_ingestion_job.cc Change ParseInternalKey() to return Status instead of bool (#7457) 2020-09-30 19:16:47 -07:00
external_sst_file_ingestion_job.h Store FSSequentialFilePtr object in SequenceFileReader (#7190) 2020-08-18 16:20:54 -07:00
external_sst_file_test.cc Disable fsync in some ExternalSSTFileTest tests (#7303) 2020-08-24 11:26:09 -07:00
fault_injection_test.cc More Makefile Cleanup (#7097) 2020-07-09 14:35:17 -07:00
file_indexer_test.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
file_indexer.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
file_indexer.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
filename_test.cc Add more tests to ASSERT_STATUS_CHECKED (#7211) 2020-08-06 17:19:41 -07:00
flush_job_test.cc Introduce options.check_flush_compaction_key_order (#7467) 2020-10-01 10:10:26 -07:00
flush_job.cc Add flush_job_test to the list of ASSERT_STATUS_CHECKED tests (#7445) 2020-09-28 14:59:02 -07:00
flush_job.h Store FSWritableFilePtr object in WritableFileWriter (#7193) 2020-09-08 10:56:08 -07:00
flush_scheduler.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
flush_scheduler.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
forward_iterator_bench.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
forward_iterator.cc Bug fix to remove function calling in assert statement (#7581) 2020-10-21 20:25:30 -07:00
forward_iterator.h Properly report IO errors when IndexType::kBinarySearchWithFirstKey is used (#6621) 2020-04-15 17:40:44 -07:00
import_column_family_job.cc Change ParseInternalKey() to return Status instead of bool (#7457) 2020-09-30 19:16:47 -07:00
import_column_family_job.h Store FSSequentialFilePtr object in SequenceFileReader (#7190) 2020-08-18 16:20:54 -07:00
import_column_family_test.cc Add few unit test cases in ASSERT_STATUS_CHECKED (#7500) 2020-10-08 11:22:44 -07:00
internal_stats.cc Introduce a blob file reader class (#7461) 2020-10-07 15:44:53 -07:00
internal_stats.h Introduce a blob file reader class (#7461) 2020-10-07 15:44:53 -07:00
job_context.h Expose the set of live blob files from Version/VersionSet (#6785) 2020-05-04 15:08:13 -07:00
listener_test.cc Bring the Configurable options together (#5753) 2020-09-14 17:01:01 -07:00
log_format.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
log_reader.cc Real fix for race in backup custom checksum checking (#7309) 2020-08-26 10:39:20 -07:00
log_reader.h Real fix for race in backup custom checksum checking (#7309) 2020-08-26 10:39:20 -07:00
log_test.cc Revert "Update googletest from 1.8.1 to 1.10.0 (#6808)" (#6923) 2020-06-03 15:55:03 -07:00
log_writer.cc Fail recovery when MANIFEST record checksum mismatch (#6996) 2020-06-18 10:09:12 -07:00
log_writer.h Pass IOStatus to write path and set retryable IO Error as hard error in BG jobs (#6487) 2020-03-27 16:04:43 -07:00
logs_with_prep_tracker.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
logs_with_prep_tracker.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
lookup_key.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
malloc_stats.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
malloc_stats.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
manual_compaction_test.cc Skip high levels with no key falling in the range in CompactRange (#6482) 2020-03-04 20:15:25 -08:00
memtable_list_test.cc Add few unit test cases in ASSERT_STATUS_CHECKED build (#7427) 2020-09-24 21:48:57 -07:00
memtable_list.cc Integrate blob file writing with the flush logic (#7345) 2020-09-14 21:11:43 -07:00
memtable_list.h Bug Fix for memtables not trimmed down. (#7296) 2020-08-21 13:29:05 -07:00
memtable.cc Introduce options.check_flush_compaction_key_order (#7467) 2020-10-01 10:10:26 -07:00
memtable.h Provide users with option to opt-in to get corrupt data in logs/messages (#7420) 2020-09-29 23:17:45 -07:00
merge_context.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
merge_helper_test.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
merge_helper.cc Change ParseInternalKey() to return Status instead of bool (#7457) 2020-09-30 19:16:47 -07:00
merge_helper.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
merge_operator.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
merge_test.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
obsolete_files_test.cc Whole DBTest to skip fsync (#7274) 2020-08-17 18:42:25 -07:00
options_file_test.cc Add more tests to ASSERT_STATUS_CHECKED (#7367) 2020-09-16 15:48:07 -07:00
output_validator.cc Introduce options.check_flush_compaction_key_order (#7467) 2020-10-01 10:10:26 -07:00
output_validator.h Introduce options.check_flush_compaction_key_order (#7467) 2020-10-01 10:10:26 -07:00
perf_context_test.cc Fix MSVC-related build issues (#7439) 2020-10-01 09:23:04 -07:00
periodic_work_scheduler_test.cc Periodically flush info log out of application buffer (#7488) 2020-10-01 19:14:14 -07:00
periodic_work_scheduler.cc Periodically flush info log out of application buffer (#7488) 2020-10-01 19:14:14 -07:00
periodic_work_scheduler.h Periodically flush info log out of application buffer (#7488) 2020-10-01 19:14:14 -07:00
pinned_iterators_manager.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
plain_table_db_test.cc More Makefile Cleanup (#7097) 2020-07-09 14:35:17 -07:00
pre_release_callback.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
prefix_test.cc Fix prefix_test for status check (#7495) 2020-10-02 17:01:15 -07:00
range_del_aggregator_bench.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
range_del_aggregator_test.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
range_del_aggregator.cc Change ParseInternalKey() to return Status instead of bool (#7457) 2020-09-30 19:16:47 -07:00
range_del_aggregator.h Change ParseInternalKey() to return Status instead of bool (#7457) 2020-09-30 19:16:47 -07:00
range_tombstone_fragmenter_test.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
range_tombstone_fragmenter.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
range_tombstone_fragmenter.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
read_callback.h Get() with timestamp should respect snapshot (#7227) 2020-08-14 19:20:58 -07:00
repair_test.cc add Status check assertions for repair_test (#7455) 2020-09-29 16:30:08 -07:00
repair.cc Change ParseInternalKey() to return Status instead of bool (#7457) 2020-09-30 19:16:47 -07:00
snapshot_checker.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
snapshot_impl.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
snapshot_impl.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
table_cache.cc Fix db_properties_test for ASSERT_STATUS_CHECKED (#7490) 2020-10-01 17:47:09 -07:00
table_cache.h Store FSRandomAccessPtr object in RandomAccessFileReader (#7192) 2020-08-27 11:21:52 -07:00
table_properties_collector_test.cc Bring the Configurable options together (#5753) 2020-09-14 17:01:01 -07:00
table_properties_collector.cc Change ParseInternalKey() to return Status instead of bool (#7457) 2020-09-30 19:16:47 -07:00
table_properties_collector.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
transaction_log_impl.cc Store FSSequentialFilePtr object in SequenceFileReader (#7190) 2020-08-18 16:20:54 -07:00
transaction_log_impl.h Store FileSystemPtr object that contains FileSystem ptr (#7180) 2020-08-12 17:31:23 -07:00
trim_history_scheduler.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
trim_history_scheduler.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
version_builder_test.cc Add version_builder_test to the list of ASSERT_STATUS_CHECKED tests (#7444) 2020-09-28 12:12:40 -07:00
version_builder.cc Enforce status check for corruption_test (#7453) 2020-10-02 22:11:00 -07:00
version_builder.h make L0 index/filter pinned memory usage predictable (#6911) 2020-06-09 16:51:23 -07:00
version_edit_handler.cc Store FSWritableFilePtr object in WritableFileWriter (#7193) 2020-09-08 10:56:08 -07:00
version_edit_handler.h Store FSWritableFilePtr object in WritableFileWriter (#7193) 2020-09-08 10:56:08 -07:00
version_edit_test.cc Track WAL in MANIFEST: update WalMetadata for WAL syncing (#7414) 2020-09-22 14:35:14 -07:00
version_edit.cc Integrate blob file writing with the flush logic (#7345) 2020-09-14 21:11:43 -07:00
version_edit.h Integrate blob file writing with the flush logic (#7345) 2020-09-14 21:11:43 -07:00
version_set_test.cc Store FileSystemPtr object that contains FileSystem ptr (#7180) 2020-08-12 17:31:23 -07:00
version_set.cc Add ldb_cmd_test to ASSERT_STATUS_CHECKED list (#7499) 2020-10-08 00:00:48 -07:00
version_set.h Fix TSAN build and re-enable the tests (#7386) 2020-09-25 14:46:28 -07:00
wal_edit_test.cc Track WAL in MANIFEST: add method to check WAL consistency (#7236) 2020-09-25 13:25:54 -07:00
wal_edit.cc Track WAL in MANIFEST: add method to check WAL consistency (#7236) 2020-09-25 13:25:54 -07:00
wal_edit.h Track WAL in MANIFEST: add method to check WAL consistency (#7236) 2020-09-25 13:25:54 -07:00
wal_manager_test.cc Store FileSystemPtr object that contains FileSystem ptr (#7180) 2020-08-12 17:31:23 -07:00
wal_manager.cc add Status check assertions for repair_test (#7455) 2020-09-29 16:30:08 -07:00
wal_manager.h Store FileSystemPtr object that contains FileSystem ptr (#7180) 2020-08-12 17:31:23 -07:00
write_batch_base.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
write_batch_internal.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
write_batch_test.cc Change ParseInternalKey() to return Status instead of bool (#7457) 2020-09-30 19:16:47 -07:00
write_batch.cc Add trace_analyzer_test to ASSERT_STATUS_CHECKED list (#7480) 2020-10-01 15:58:52 -07:00
write_callback_test.cc Divide WriteCallbackTest.WriteWithCallbackTest (#7037) 2020-06-30 12:31:30 -07:00
write_callback.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
write_controller_test.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
write_controller.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
write_controller.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
write_thread.cc Fix StallWrite crash with mixed of slowdown/no_slowdown writes (#7508) 2020-10-06 12:44:20 -07:00
write_thread.h Enforce status check for corruption_test (#7453) 2020-10-02 22:11:00 -07:00