rocksdb/db
Andrew Kryczka 1c5f13f2a5 Fail early when merge_operator not configured (#7667)
Summary:
An application may accidentally write merge operands without properly configuring `merge_operator`. We should alert them as early as possible that there's an API misuse. Previously RocksDB only notified them when a query or background operation needed to merge but couldn't. With this PR, RocksDB notifies them of the problem before applying the merge operand to the memtable (although it may already be in WAL, which seems it'd cause a crash loop until they enable `merge_operator`).

Pull Request resolved: https://github.com/facebook/rocksdb/pull/7667

Reviewed By: riversand963

Differential Revision: D24933360

Pulled By: ajkr

fbshipit-source-id: 3a4a2ceb0b7aed184113dd03b8efd735a8332f7f
2020-11-16 20:39:01 -08:00
..
blob Fix many tests to run with MEM_ENV and ENCRYPTED_ENV; Introduce a MemoryFileSystem class (#7566) 2020-10-27 10:33:09 -07:00
compaction Add full_history_ts_low_ to CompactionJob (#7657) 2020-11-12 11:43:24 -08:00
db_impl Add full_history_ts_low_ to FlushJob (#7655) 2020-11-12 18:44:34 -08:00
arena_wrapped_db_iter.cc dedup ReadOptions in iterator hierarchy (#7210) 2020-08-03 15:23:04 -07:00
arena_wrapped_db_iter.h dedup ReadOptions in iterator hierarchy (#7210) 2020-08-03 15:23:04 -07:00
builder.cc Add full_history_ts_low_ to FlushJob (#7655) 2020-11-12 18:44:34 -08:00
builder.h Add full_history_ts_low_ to FlushJob (#7655) 2020-11-12 18:44:34 -08:00
c_test.c Add getters to the C API for env, universal compaction options and fifo compaction options (#7501) 2020-10-16 11:04:01 -07:00
c.cc Remove unused includes (#7604) 2020-10-28 23:22:27 -07:00
column_family_test.cc ColumnFamilyTest often times out in internal test infra (#7638) 2020-11-06 10:25:20 -08:00
column_family.cc In ParseInternalKey(), include corrupt key info in Status (#7515) 2020-10-28 10:12:58 -07:00
column_family.h In ParseInternalKey(), include corrupt key info in Status (#7515) 2020-10-28 10:12:58 -07:00
compact_files_test.cc Replace reinterpret_cast with static_cast_with_check (#7067) 2020-07-02 19:25:41 -07:00
compacted_db_impl.cc Periodically flush info log out of application buffer (#7488) 2020-10-01 19:14:14 -07:00
compacted_db_impl.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
comparator_db_test.cc Remove unused includes (#7604) 2020-10-28 23:22:27 -07:00
convenience.cc sst_dump to reduce number of file reads (#6836) 2020-05-12 18:23:33 -07:00
corruption_test.cc Refactor with VersionEditHandler (#6581) 2020-11-11 08:00:14 -08:00
cuckoo_table_db_test.cc Fix a recovery corner case (#7621) 2020-11-07 22:23:27 -08:00
db_basic_test.cc Fix a recovery corner case (#7621) 2020-11-07 22:23:27 -08:00
db_block_cache_test.cc Expand effect of dictionary settings in ColumnFamilyOptions::compression_opts (#7619) 2020-11-02 19:21:11 -08:00
db_bloom_filter_test.cc Experimental (production candidate) SST schema for Ribbon filter (#7658) 2020-11-12 20:46:14 -08:00
db_compaction_filter_test.cc In ParseInternalKey(), include corrupt key info in Status (#7515) 2020-10-28 10:12:58 -07:00
db_compaction_test.cc Fix many tests to run with MEM_ENV and ENCRYPTED_ENV; Introduce a MemoryFileSystem class (#7566) 2020-10-27 10:33:09 -07:00
db_dynamic_level_test.cc Add a host location property to TableProperties (#7479) 2020-10-19 11:38:48 -07:00
db_encryption_test.cc Fix many tests to run with MEM_ENV and ENCRYPTED_ENV; Introduce a MemoryFileSystem class (#7566) 2020-10-27 10:33:09 -07:00
db_filesnapshot.cc Fix checkpoint file deletion race with avoid_unnecessary_blocking_io (#7369) 2020-09-10 22:35:25 -07:00
db_flush_test.cc Fix many tests to run with MEM_ENV and ENCRYPTED_ENV; Introduce a MemoryFileSystem class (#7566) 2020-10-27 10:33:09 -07:00
db_info_dumper.cc Make FileType Public and Replace kLogFile with kWalFile (#7580) 2020-10-22 17:06:20 -07:00
db_info_dumper.h Add a DB Session ID (#6959) 2020-06-15 10:47:02 -07:00
db_inplace_update_test.cc Whole DBTest to skip fsync (#7274) 2020-08-17 18:42:25 -07:00
db_io_failure_test.cc Whole DBTest to skip fsync (#7274) 2020-08-17 18:42:25 -07:00
db_iter_stress_test.cc Test CircleCI with CLANG-10 (#7025) 2020-06-24 16:22:49 -07:00
db_iter_test.cc In ParseInternalKey(), include corrupt key info in Status (#7515) 2020-10-28 10:12:58 -07:00
db_iter.cc Report if unpinnable value encountered during backward iteration (#7618) 2020-11-10 17:17:39 -08:00
db_iter.h Make db_basic_test pass assert status checked (#7452) 2020-09-29 09:49:04 -07:00
db_iterator_test.cc Report if unpinnable value encountered during backward iteration (#7618) 2020-11-10 17:17:39 -08:00
db_log_iter_test.cc Whole DBTest to skip fsync (#7274) 2020-08-17 18:42:25 -07:00
db_logical_block_size_cache_test.cc Get block size only in direct IO mode (#6522) 2020-03-20 15:26:10 -07:00
db_memtable_test.cc Fix many tests to run with MEM_ENV and ENCRYPTED_ENV; Introduce a MemoryFileSystem class (#7566) 2020-10-27 10:33:09 -07:00
db_merge_operand_test.cc Whole DBTest to skip fsync (#7274) 2020-08-17 18:42:25 -07:00
db_merge_operator_test.cc Disable fsync in DBMergeOperatorTest to save test time (#7640) 2020-11-06 15:24:17 -08:00
db_options_test.cc Always apply bottommost_compression_opts when enabled (#7633) 2020-11-11 20:32:28 -08:00
db_properties_test.cc Fix many tests to run with MEM_ENV and ENCRYPTED_ENV; Introduce a MemoryFileSystem class (#7566) 2020-10-27 10:33:09 -07:00
db_range_del_test.cc Fix a recovery corner case (#7621) 2020-11-07 22:23:27 -08:00
db_sst_test.cc Fix many tests to run with MEM_ENV and ENCRYPTED_ENV; Introduce a MemoryFileSystem class (#7566) 2020-10-27 10:33:09 -07:00
db_statistics_test.cc Add a new stats level to exclude tickers (#7329) 2020-09-04 23:25:03 -07:00
db_table_properties_test.cc DBTablePropertiesTest often times out in internal test infra (#7639) 2020-11-06 14:25:14 -08:00
db_tailing_iter_test.cc Whole DBTest to skip fsync (#7274) 2020-08-17 18:42:25 -07:00
db_test2.cc Avoid skipping a test in db_test2 (#7629) 2020-11-02 19:48:23 -08:00
db_test_util.cc Do not track WAL in MANIFEST when fsync is disabled in a test (#7669) 2020-11-13 13:37:14 -08:00
db_test_util.h Fix many tests to run with MEM_ENV and ENCRYPTED_ENV; Introduce a MemoryFileSystem class (#7566) 2020-10-27 10:33:09 -07:00
db_test.cc Fix many tests to run with MEM_ENV and ENCRYPTED_ENV; Introduce a MemoryFileSystem class (#7566) 2020-10-27 10:33:09 -07:00
db_universal_compaction_test.cc Fix many tests to run with MEM_ENV and ENCRYPTED_ENV; Introduce a MemoryFileSystem class (#7566) 2020-10-27 10:33:09 -07:00
db_wal_test.cc Avoid skipping a test in db_wal_test (#7628) 2020-11-03 09:48:16 -08:00
db_with_timestamp_basic_test.cc Fix a seek issue with prefix extractor and timestamp (#7644) 2020-11-10 14:53:13 -08:00
db_with_timestamp_compaction_test.cc Whole DBTest to skip fsync (#7274) 2020-08-17 18:42:25 -07:00
db_write_test.cc Fix many tests to run with MEM_ENV and ENCRYPTED_ENV; Introduce a MemoryFileSystem class (#7566) 2020-10-27 10:33:09 -07:00
dbformat_test.cc Remove unused includes (#7604) 2020-10-28 23:22:27 -07:00
dbformat.cc In ParseInternalKey(), include corrupt key info in Status (#7515) 2020-10-28 10:12:58 -07:00
dbformat.h In ParseInternalKey(), include corrupt key info in Status (#7515) 2020-10-28 10:12:58 -07:00
deletefile_test.cc Make FileType Public and Replace kLogFile with kWalFile (#7580) 2020-10-22 17:06:20 -07:00
error_handler_fs_test.cc Fix many tests to run with MEM_ENV and ENCRYPTED_ENV; Introduce a MemoryFileSystem class (#7566) 2020-10-27 10:33:09 -07:00
error_handler.cc Status check enforcement for timestamp_basic_test (#7454) 2020-09-29 18:23:27 -07:00
error_handler.h Map retryable IO error during Flush without WAL to soft error and no switch memtable during resume (#7310) 2020-09-17 20:25:45 -07:00
event_helpers.cc Status check enforcement for error_handler_fs_test (#7342) 2020-10-02 16:41:13 -07:00
event_helpers.h Pass SST file checksum information through OnTableFileCreated (#7108) 2020-08-25 10:46:11 -07:00
experimental.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
external_sst_file_basic_test.cc Fix many tests to run with MEM_ENV and ENCRYPTED_ENV; Introduce a MemoryFileSystem class (#7566) 2020-10-27 10:33:09 -07:00
external_sst_file_ingestion_job.cc Updated GenerateOneFileChecksum to use requested_checksum_func_name (#7586) 2020-10-28 16:47:12 -07:00
external_sst_file_ingestion_job.h Store FSSequentialFilePtr object in SequenceFileReader (#7190) 2020-08-18 16:20:54 -07:00
external_sst_file_test.cc Disable fsync in some ExternalSSTFileTest tests (#7303) 2020-08-24 11:26:09 -07:00
fault_injection_test.cc Remove unused includes (#7604) 2020-10-28 23:22:27 -07:00
file_indexer_test.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
file_indexer.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
file_indexer.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
filename_test.cc Remove unused includes (#7604) 2020-10-28 23:22:27 -07:00
flush_job_test.cc Add full_history_ts_low_ to FlushJob (#7655) 2020-11-12 18:44:34 -08:00
flush_job.cc Add full_history_ts_low_ to FlushJob (#7655) 2020-11-12 18:44:34 -08:00
flush_job.h Add full_history_ts_low_ to FlushJob (#7655) 2020-11-12 18:44:34 -08:00
flush_scheduler.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
flush_scheduler.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
forward_iterator_bench.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
forward_iterator.cc Bug fix to remove function calling in assert statement (#7581) 2020-10-21 20:18:06 -07:00
forward_iterator.h Properly report IO errors when IndexType::kBinarySearchWithFirstKey is used (#6621) 2020-04-15 17:40:44 -07:00
import_column_family_job.cc In ParseInternalKey(), include corrupt key info in Status (#7515) 2020-10-28 10:12:58 -07:00
import_column_family_job.h Store FSSequentialFilePtr object in SequenceFileReader (#7190) 2020-08-18 16:20:54 -07:00
import_column_family_test.cc Fix many tests to run with MEM_ENV and ENCRYPTED_ENV; Introduce a MemoryFileSystem class (#7566) 2020-10-27 10:33:09 -07:00
internal_stats.cc Fix InternalStats::DumpCFStats (#7666) 2020-11-12 17:33:04 -08:00
internal_stats.h Introduce BlobFileCache and add support for blob files to Get() (#7540) 2020-10-15 13:04:47 -07:00
job_context.h Expose the set of live blob files from Version/VersionSet (#6785) 2020-05-04 15:08:13 -07:00
listener_test.cc Remove unused includes (#7604) 2020-10-28 23:22:27 -07:00
log_format.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
log_reader.cc Real fix for race in backup custom checksum checking (#7309) 2020-08-26 10:39:20 -07:00
log_reader.h Real fix for race in backup custom checksum checking (#7309) 2020-08-26 10:39:20 -07:00
log_test.cc Revert "Update googletest from 1.8.1 to 1.10.0 (#6808)" (#6923) 2020-06-03 15:55:03 -07:00
log_writer.cc Fail recovery when MANIFEST record checksum mismatch (#6996) 2020-06-18 10:09:12 -07:00
log_writer.h Pass IOStatus to write path and set retryable IO Error as hard error in BG jobs (#6487) 2020-03-27 16:04:43 -07:00
logs_with_prep_tracker.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
logs_with_prep_tracker.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
lookup_key.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
malloc_stats.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
malloc_stats.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
manual_compaction_test.cc Skip high levels with no key falling in the range in CompactRange (#6482) 2020-03-04 20:15:25 -08:00
memtable_list_test.cc Add few unit test cases in ASSERT_STATUS_CHECKED build (#7427) 2020-09-24 21:48:57 -07:00
memtable_list.cc Track WAL in MANIFEST: LogAndApply WAL events to MANIFEST (#7601) 2020-11-06 17:22:36 -08:00
memtable_list.h Perform post-flush updates of memtable list in a callback (#6069) 2020-10-26 18:23:01 -07:00
memtable.cc Fix MultiGet unable to query timestamp data issue (#7589) 2020-11-03 09:45:41 -08:00
memtable.h Provide users with option to opt-in to get corrupt data in logs/messages (#7420) 2020-09-29 23:17:45 -07:00
merge_context.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
merge_helper_test.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
merge_helper.cc In ParseInternalKey(), include corrupt key info in Status (#7515) 2020-10-28 10:12:58 -07:00
merge_helper.h In ParseInternalKey(), include corrupt key info in Status (#7515) 2020-10-28 10:12:58 -07:00
merge_operator.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
merge_test.cc Perform post-flush updates of memtable list in a callback (#6069) 2020-10-26 18:23:01 -07:00
obsolete_files_test.cc Remove unused includes (#7604) 2020-10-28 23:22:27 -07:00
options_file_test.cc Add more tests to ASSERT_STATUS_CHECKED (#7367) 2020-09-16 15:48:07 -07:00
output_validator.cc Use NPHash64 in more places (#7632) 2020-11-10 23:42:13 -08:00
output_validator.h Use NPHash64 in more places (#7632) 2020-11-10 23:42:13 -08:00
perf_context_test.cc Fix many tests to run with MEM_ENV and ENCRYPTED_ENV; Introduce a MemoryFileSystem class (#7566) 2020-10-27 10:33:09 -07:00
periodic_work_scheduler_test.cc Fix many tests to run with MEM_ENV and ENCRYPTED_ENV; Introduce a MemoryFileSystem class (#7566) 2020-10-27 10:33:09 -07:00
periodic_work_scheduler.cc Remove unused includes (#7604) 2020-10-28 23:22:27 -07:00
periodic_work_scheduler.h Periodically flush info log out of application buffer (#7488) 2020-10-01 19:14:14 -07:00
pinned_iterators_manager.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
plain_table_db_test.cc Remove unused includes (#7604) 2020-10-28 23:22:27 -07:00
pre_release_callback.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
prefix_test.cc Fix prefix_test for status check (#7495) 2020-10-02 17:01:15 -07:00
range_del_aggregator_bench.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
range_del_aggregator_test.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
range_del_aggregator.cc In ParseInternalKey(), include corrupt key info in Status (#7515) 2020-10-28 10:12:58 -07:00
range_del_aggregator.h In ParseInternalKey(), include corrupt key info in Status (#7515) 2020-10-28 10:12:58 -07:00
range_tombstone_fragmenter_test.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
range_tombstone_fragmenter.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
range_tombstone_fragmenter.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
read_callback.h Get() with timestamp should respect snapshot (#7227) 2020-08-14 19:20:58 -07:00
repair_test.cc add Status check assertions for repair_test (#7455) 2020-09-29 16:30:08 -07:00
repair.cc Add full_history_ts_low_ to FlushJob (#7655) 2020-11-12 18:44:34 -08:00
snapshot_checker.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
snapshot_impl.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
snapshot_impl.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
table_cache.cc Fix MultiGet unable to query timestamp data issue (#7589) 2020-11-03 09:45:41 -08:00
table_cache.h Store FSRandomAccessPtr object in RandomAccessFileReader (#7192) 2020-08-27 11:21:52 -07:00
table_properties_collector_test.cc Bring the Configurable options together (#5753) 2020-09-14 17:01:01 -07:00
table_properties_collector.cc In ParseInternalKey(), include corrupt key info in Status (#7515) 2020-10-28 10:12:58 -07:00
table_properties_collector.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
transaction_log_impl.cc Store FSSequentialFilePtr object in SequenceFileReader (#7190) 2020-08-18 16:20:54 -07:00
transaction_log_impl.h Store FileSystemPtr object that contains FileSystem ptr (#7180) 2020-08-12 17:31:23 -07:00
trim_history_scheduler.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
trim_history_scheduler.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
version_builder_test.cc Remove unused includes (#7604) 2020-10-28 23:22:27 -07:00
version_builder.cc Enforce status check for corruption_test (#7453) 2020-10-02 22:11:00 -07:00
version_builder.h make L0 index/filter pinned memory usage predictable (#6911) 2020-06-09 16:51:23 -07:00
version_edit_handler.cc Refactor with VersionEditHandler (#6581) 2020-11-11 08:00:14 -08:00
version_edit_handler.h Refactor with VersionEditHandler (#6581) 2020-11-11 08:00:14 -08:00
version_edit_test.cc Track WAL in MANIFEST: LogAndApply WAL events to MANIFEST (#7601) 2020-11-06 17:22:36 -08:00
version_edit.cc Track WAL in MANIFEST: LogAndApply WAL events to MANIFEST (#7601) 2020-11-06 17:22:36 -08:00
version_edit.h Refactor with VersionEditHandler (#6581) 2020-11-11 08:00:14 -08:00
version_set_test.cc Refactor with VersionEditHandler (#6581) 2020-11-11 08:00:14 -08:00
version_set.cc Refactor with VersionEditHandler (#6581) 2020-11-11 08:00:14 -08:00
version_set.h Refactor with VersionEditHandler (#6581) 2020-11-11 08:00:14 -08:00
wal_edit_test.cc Track WAL in MANIFEST: LogAndApply WAL events to MANIFEST (#7601) 2020-11-06 17:22:36 -08:00
wal_edit.cc Track WAL in MANIFEST: LogAndApply WAL events to MANIFEST (#7601) 2020-11-06 17:22:36 -08:00
wal_edit.h Track WAL in MANIFEST: LogAndApply WAL events to MANIFEST (#7601) 2020-11-06 17:22:36 -08:00
wal_manager_test.cc Make FileType Public and Replace kLogFile with kWalFile (#7580) 2020-10-22 17:06:20 -07:00
wal_manager.cc Make FileType Public and Replace kLogFile with kWalFile (#7580) 2020-10-22 17:06:20 -07:00
wal_manager.h Store FileSystemPtr object that contains FileSystem ptr (#7180) 2020-08-12 17:31:23 -07:00
write_batch_base.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
write_batch_internal.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
write_batch_test.cc Fail early when merge_operator not configured (#7667) 2020-11-16 20:39:01 -08:00
write_batch.cc Fail early when merge_operator not configured (#7667) 2020-11-16 20:39:01 -08:00
write_callback_test.cc Divide WriteCallbackTest.WriteWithCallbackTest (#7037) 2020-06-30 12:31:30 -07:00
write_callback.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
write_controller_test.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
write_controller.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
write_controller.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
write_thread.cc Fix StallWrite crash with mixed of slowdown/no_slowdown writes (#7508) 2020-10-06 12:44:20 -07:00
write_thread.h Remove the status.PermitUncheckedError() from WriteGroup Destructor (#7555) 2020-10-14 10:47:58 -07:00