rocksdb/db
Cheng Chang 7169ca9c80 Do not track empty WALs (#7697)
Summary:
An empty WAL won't be backed up by the BackupEngine. So if we track the empty WALs in MANIFEST, then when restoring from a backup, it may report corruption that the empty WAL is missing, which is correct because the WAL is actually in the main DB but not in the backup DB, but missing an empty WAL does not logically break DB consistency.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/7697

Test Plan: watch existing tests to pass

Reviewed By: pdillinger

Differential Revision: D25077194

Pulled By: cheng-chang

fbshipit-source-id: 01917b57234b92b6063925f2ee9452c5732bdc03
2020-11-18 21:27:54 -08:00
..
blob Fix many tests to run with MEM_ENV and ENCRYPTED_ENV; Introduce a MemoryFileSystem class (#7566) 2020-10-27 10:33:09 -07:00
compaction Add full_history_ts_low_ to CompactionJob (#7657) 2020-11-12 11:43:24 -08:00
db_impl Do not track empty WALs (#7697) 2020-11-18 21:27:54 -08:00
arena_wrapped_db_iter.cc dedup ReadOptions in iterator hierarchy (#7210) 2020-08-03 15:23:04 -07:00
arena_wrapped_db_iter.h dedup ReadOptions in iterator hierarchy (#7210) 2020-08-03 15:23:04 -07:00
builder.cc Add full_history_ts_low_ to FlushJob (#7655) 2020-11-12 18:44:34 -08:00
builder.h Add full_history_ts_low_ to FlushJob (#7655) 2020-11-12 18:44:34 -08:00
c_test.c Add getters to the C API for env, universal compaction options and fifo compaction options (#7501) 2020-10-16 11:04:01 -07:00
c.cc Remove unused includes (#7604) 2020-10-28 23:22:27 -07:00
column_family_test.cc ColumnFamilyTest often times out in internal test infra (#7638) 2020-11-06 10:25:20 -08:00
column_family.cc In ParseInternalKey(), include corrupt key info in Status (#7515) 2020-10-28 10:12:58 -07:00
column_family.h In ParseInternalKey(), include corrupt key info in Status (#7515) 2020-10-28 10:12:58 -07:00
compact_files_test.cc Replace reinterpret_cast with static_cast_with_check (#7067) 2020-07-02 19:25:41 -07:00
compacted_db_impl.cc Periodically flush info log out of application buffer (#7488) 2020-10-01 19:14:14 -07:00
compacted_db_impl.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
comparator_db_test.cc Remove unused includes (#7604) 2020-10-28 23:22:27 -07:00
convenience.cc sst_dump to reduce number of file reads (#6836) 2020-05-12 18:23:33 -07:00
corruption_test.cc Refactor with VersionEditHandler (#6581) 2020-11-11 08:00:14 -08:00
cuckoo_table_db_test.cc Fix a recovery corner case (#7621) 2020-11-07 22:23:27 -08:00
db_basic_test.cc Clean up after two test failures in db_basic_test (#7682) 2020-11-16 22:09:01 -08:00
db_block_cache_test.cc Expand effect of dictionary settings in ColumnFamilyOptions::compression_opts (#7619) 2020-11-02 19:21:11 -08:00
db_bloom_filter_test.cc Experimental (production candidate) SST schema for Ribbon filter (#7658) 2020-11-12 20:46:14 -08:00
db_compaction_filter_test.cc In ParseInternalKey(), include corrupt key info in Status (#7515) 2020-10-28 10:12:58 -07:00
db_compaction_test.cc Fix many tests to run with MEM_ENV and ENCRYPTED_ENV; Introduce a MemoryFileSystem class (#7566) 2020-10-27 10:33:09 -07:00
db_dynamic_level_test.cc Add a host location property to TableProperties (#7479) 2020-10-19 11:38:48 -07:00
db_encryption_test.cc Fix many tests to run with MEM_ENV and ENCRYPTED_ENV; Introduce a MemoryFileSystem class (#7566) 2020-10-27 10:33:09 -07:00
db_filesnapshot.cc Fix checkpoint file deletion race with avoid_unnecessary_blocking_io (#7369) 2020-09-10 22:35:25 -07:00
db_flush_test.cc Track WAL in MANIFEST: Update logic for computing min_log_number_to_keep in atomic flush (#7660) 2020-11-17 15:55:55 -08:00
db_info_dumper.cc Make FileType Public and Replace kLogFile with kWalFile (#7580) 2020-10-22 17:06:20 -07:00
db_info_dumper.h Add a DB Session ID (#6959) 2020-06-15 10:47:02 -07:00
db_inplace_update_test.cc Whole DBTest to skip fsync (#7274) 2020-08-17 18:42:25 -07:00
db_io_failure_test.cc Whole DBTest to skip fsync (#7274) 2020-08-17 18:42:25 -07:00
db_iter_stress_test.cc Test CircleCI with CLANG-10 (#7025) 2020-06-24 16:22:49 -07:00
db_iter_test.cc In ParseInternalKey(), include corrupt key info in Status (#7515) 2020-10-28 10:12:58 -07:00
db_iter.cc Report if unpinnable value encountered during backward iteration (#7618) 2020-11-10 17:17:39 -08:00
db_iter.h Make db_basic_test pass assert status checked (#7452) 2020-09-29 09:49:04 -07:00
db_iterator_test.cc Report if unpinnable value encountered during backward iteration (#7618) 2020-11-10 17:17:39 -08:00
db_log_iter_test.cc Whole DBTest to skip fsync (#7274) 2020-08-17 18:42:25 -07:00
db_logical_block_size_cache_test.cc Get block size only in direct IO mode (#6522) 2020-03-20 15:26:10 -07:00
db_memtable_test.cc Fix many tests to run with MEM_ENV and ENCRYPTED_ENV; Introduce a MemoryFileSystem class (#7566) 2020-10-27 10:33:09 -07:00
db_merge_operand_test.cc Whole DBTest to skip fsync (#7274) 2020-08-17 18:42:25 -07:00
db_merge_operator_test.cc Disable fsync in DBMergeOperatorTest to save test time (#7640) 2020-11-06 15:24:17 -08:00
db_options_test.cc Always apply bottommost_compression_opts when enabled (#7633) 2020-11-11 20:32:28 -08:00
db_properties_test.cc Fix many tests to run with MEM_ENV and ENCRYPTED_ENV; Introduce a MemoryFileSystem class (#7566) 2020-10-27 10:33:09 -07:00
db_range_del_test.cc Fix a recovery corner case (#7621) 2020-11-07 22:23:27 -08:00
db_sst_test.cc Fix many tests to run with MEM_ENV and ENCRYPTED_ENV; Introduce a MemoryFileSystem class (#7566) 2020-10-27 10:33:09 -07:00
db_statistics_test.cc Add a new stats level to exclude tickers (#7329) 2020-09-04 23:25:03 -07:00
db_table_properties_test.cc DBTablePropertiesTest often times out in internal test infra (#7639) 2020-11-06 14:25:14 -08:00
db_tailing_iter_test.cc Whole DBTest to skip fsync (#7274) 2020-08-17 18:42:25 -07:00
db_test2.cc Avoid skipping a test in db_test2 (#7629) 2020-11-02 19:48:23 -08:00
db_test_util.cc Do not track WAL in MANIFEST when fsync is disabled in a test (#7669) 2020-11-13 13:37:14 -08:00
db_test_util.h Fix many tests to run with MEM_ENV and ENCRYPTED_ENV; Introduce a MemoryFileSystem class (#7566) 2020-10-27 10:33:09 -07:00
db_test.cc Fix many tests to run with MEM_ENV and ENCRYPTED_ENV; Introduce a MemoryFileSystem class (#7566) 2020-10-27 10:33:09 -07:00
db_universal_compaction_test.cc Fix many tests to run with MEM_ENV and ENCRYPTED_ENV; Introduce a MemoryFileSystem class (#7566) 2020-10-27 10:33:09 -07:00
db_wal_test.cc Avoid skipping a test in db_wal_test (#7628) 2020-11-03 09:48:16 -08:00
db_with_timestamp_basic_test.cc Fix a seek issue with prefix extractor and timestamp (#7644) 2020-11-10 14:53:13 -08:00
db_with_timestamp_compaction_test.cc Whole DBTest to skip fsync (#7274) 2020-08-17 18:42:25 -07:00
db_write_test.cc Fix many tests to run with MEM_ENV and ENCRYPTED_ENV; Introduce a MemoryFileSystem class (#7566) 2020-10-27 10:33:09 -07:00
dbformat_test.cc Remove unused includes (#7604) 2020-10-28 23:22:27 -07:00
dbformat.cc In ParseInternalKey(), include corrupt key info in Status (#7515) 2020-10-28 10:12:58 -07:00
dbformat.h In ParseInternalKey(), include corrupt key info in Status (#7515) 2020-10-28 10:12:58 -07:00
deletefile_test.cc Make FileType Public and Replace kLogFile with kWalFile (#7580) 2020-10-22 17:06:20 -07:00
error_handler_fs_test.cc Fix many tests to run with MEM_ENV and ENCRYPTED_ENV; Introduce a MemoryFileSystem class (#7566) 2020-10-27 10:33:09 -07:00
error_handler.cc Status check enforcement for timestamp_basic_test (#7454) 2020-09-29 18:23:27 -07:00
error_handler.h Map retryable IO error during Flush without WAL to soft error and no switch memtable during resume (#7310) 2020-09-17 20:25:45 -07:00
event_helpers.cc Status check enforcement for error_handler_fs_test (#7342) 2020-10-02 16:41:13 -07:00
event_helpers.h Pass SST file checksum information through OnTableFileCreated (#7108) 2020-08-25 10:46:11 -07:00
experimental.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
external_sst_file_basic_test.cc Fix many tests to run with MEM_ENV and ENCRYPTED_ENV; Introduce a MemoryFileSystem class (#7566) 2020-10-27 10:33:09 -07:00
external_sst_file_ingestion_job.cc Updated GenerateOneFileChecksum to use requested_checksum_func_name (#7586) 2020-10-28 16:47:12 -07:00
external_sst_file_ingestion_job.h Store FSSequentialFilePtr object in SequenceFileReader (#7190) 2020-08-18 16:20:54 -07:00
external_sst_file_test.cc Disable fsync in some ExternalSSTFileTest tests (#7303) 2020-08-24 11:26:09 -07:00
fault_injection_test.cc Remove unused includes (#7604) 2020-10-28 23:22:27 -07:00
file_indexer_test.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
file_indexer.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
file_indexer.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
filename_test.cc Remove unused includes (#7604) 2020-10-28 23:22:27 -07:00
flush_job_test.cc Add full_history_ts_low_ to FlushJob (#7655) 2020-11-12 18:44:34 -08:00
flush_job.cc Add full_history_ts_low_ to FlushJob (#7655) 2020-11-12 18:44:34 -08:00
flush_job.h Add full_history_ts_low_ to FlushJob (#7655) 2020-11-12 18:44:34 -08:00
flush_scheduler.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
flush_scheduler.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
forward_iterator_bench.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
forward_iterator.cc Bug fix to remove function calling in assert statement (#7581) 2020-10-21 20:18:06 -07:00
forward_iterator.h Properly report IO errors when IndexType::kBinarySearchWithFirstKey is used (#6621) 2020-04-15 17:40:44 -07:00
import_column_family_job.cc In ParseInternalKey(), include corrupt key info in Status (#7515) 2020-10-28 10:12:58 -07:00
import_column_family_job.h Store FSSequentialFilePtr object in SequenceFileReader (#7190) 2020-08-18 16:20:54 -07:00
import_column_family_test.cc Fix many tests to run with MEM_ENV and ENCRYPTED_ENV; Introduce a MemoryFileSystem class (#7566) 2020-10-27 10:33:09 -07:00
internal_stats.cc Fix InternalStats::DumpCFStats (#7666) 2020-11-12 17:33:04 -08:00
internal_stats.h Introduce BlobFileCache and add support for blob files to Get() (#7540) 2020-10-15 13:04:47 -07:00
job_context.h Expose the set of live blob files from Version/VersionSet (#6785) 2020-05-04 15:08:13 -07:00
listener_test.cc Remove unused includes (#7604) 2020-10-28 23:22:27 -07:00
log_format.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
log_reader.cc Real fix for race in backup custom checksum checking (#7309) 2020-08-26 10:39:20 -07:00
log_reader.h Real fix for race in backup custom checksum checking (#7309) 2020-08-26 10:39:20 -07:00
log_test.cc Revert "Update googletest from 1.8.1 to 1.10.0 (#6808)" (#6923) 2020-06-03 15:55:03 -07:00
log_writer.cc Fail recovery when MANIFEST record checksum mismatch (#6996) 2020-06-18 10:09:12 -07:00
log_writer.h Pass IOStatus to write path and set retryable IO Error as hard error in BG jobs (#6487) 2020-03-27 16:04:43 -07:00
logs_with_prep_tracker.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
logs_with_prep_tracker.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
lookup_key.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
malloc_stats.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
malloc_stats.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
manual_compaction_test.cc Skip high levels with no key falling in the range in CompactRange (#6482) 2020-03-04 20:15:25 -08:00
memtable_list_test.cc Add few unit test cases in ASSERT_STATUS_CHECKED build (#7427) 2020-09-24 21:48:57 -07:00
memtable_list.cc Track WAL in MANIFEST: Update logic for computing min_log_number_to_keep in atomic flush (#7660) 2020-11-17 15:55:55 -08:00
memtable_list.h Perform post-flush updates of memtable list in a callback (#6069) 2020-10-26 18:23:01 -07:00
memtable.cc Fix MultiGet unable to query timestamp data issue (#7589) 2020-11-03 09:45:41 -08:00
memtable.h Provide users with option to opt-in to get corrupt data in logs/messages (#7420) 2020-09-29 23:17:45 -07:00
merge_context.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
merge_helper_test.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
merge_helper.cc In ParseInternalKey(), include corrupt key info in Status (#7515) 2020-10-28 10:12:58 -07:00
merge_helper.h In ParseInternalKey(), include corrupt key info in Status (#7515) 2020-10-28 10:12:58 -07:00
merge_operator.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
merge_test.cc Perform post-flush updates of memtable list in a callback (#6069) 2020-10-26 18:23:01 -07:00
obsolete_files_test.cc Remove unused includes (#7604) 2020-10-28 23:22:27 -07:00
options_file_test.cc Add more tests to ASSERT_STATUS_CHECKED (#7367) 2020-09-16 15:48:07 -07:00
output_validator.cc Use NPHash64 in more places (#7632) 2020-11-10 23:42:13 -08:00
output_validator.h Use NPHash64 in more places (#7632) 2020-11-10 23:42:13 -08:00
perf_context_test.cc Fix many tests to run with MEM_ENV and ENCRYPTED_ENV; Introduce a MemoryFileSystem class (#7566) 2020-10-27 10:33:09 -07:00
periodic_work_scheduler_test.cc Fix many tests to run with MEM_ENV and ENCRYPTED_ENV; Introduce a MemoryFileSystem class (#7566) 2020-10-27 10:33:09 -07:00
periodic_work_scheduler.cc Remove unused includes (#7604) 2020-10-28 23:22:27 -07:00
periodic_work_scheduler.h Periodically flush info log out of application buffer (#7488) 2020-10-01 19:14:14 -07:00
pinned_iterators_manager.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
plain_table_db_test.cc Remove unused includes (#7604) 2020-10-28 23:22:27 -07:00
pre_release_callback.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
prefix_test.cc Fix prefix_test for status check (#7495) 2020-10-02 17:01:15 -07:00
range_del_aggregator_bench.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
range_del_aggregator_test.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
range_del_aggregator.cc In ParseInternalKey(), include corrupt key info in Status (#7515) 2020-10-28 10:12:58 -07:00
range_del_aggregator.h In ParseInternalKey(), include corrupt key info in Status (#7515) 2020-10-28 10:12:58 -07:00
range_tombstone_fragmenter_test.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
range_tombstone_fragmenter.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
range_tombstone_fragmenter.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
read_callback.h Get() with timestamp should respect snapshot (#7227) 2020-08-14 19:20:58 -07:00
repair_test.cc add Status check assertions for repair_test (#7455) 2020-09-29 16:30:08 -07:00
repair.cc Add full_history_ts_low_ to FlushJob (#7655) 2020-11-12 18:44:34 -08:00
snapshot_checker.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
snapshot_impl.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
snapshot_impl.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
table_cache.cc Fix MultiGet unable to query timestamp data issue (#7589) 2020-11-03 09:45:41 -08:00
table_cache.h Store FSRandomAccessPtr object in RandomAccessFileReader (#7192) 2020-08-27 11:21:52 -07:00
table_properties_collector_test.cc Bring the Configurable options together (#5753) 2020-09-14 17:01:01 -07:00
table_properties_collector.cc In ParseInternalKey(), include corrupt key info in Status (#7515) 2020-10-28 10:12:58 -07:00
table_properties_collector.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
transaction_log_impl.cc Store FSSequentialFilePtr object in SequenceFileReader (#7190) 2020-08-18 16:20:54 -07:00
transaction_log_impl.h Store FileSystemPtr object that contains FileSystem ptr (#7180) 2020-08-12 17:31:23 -07:00
trim_history_scheduler.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
trim_history_scheduler.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
version_builder_test.cc Remove unused includes (#7604) 2020-10-28 23:22:27 -07:00
version_builder.cc Enforce status check for corruption_test (#7453) 2020-10-02 22:11:00 -07:00
version_builder.h make L0 index/filter pinned memory usage predictable (#6911) 2020-06-09 16:51:23 -07:00
version_edit_handler.cc Refactor with VersionEditHandler (#6581) 2020-11-11 08:00:14 -08:00
version_edit_handler.h Refactor with VersionEditHandler (#6581) 2020-11-11 08:00:14 -08:00
version_edit_test.cc Track WAL in MANIFEST: LogAndApply WAL events to MANIFEST (#7601) 2020-11-06 17:22:36 -08:00
version_edit.cc Track WAL in MANIFEST: LogAndApply WAL events to MANIFEST (#7601) 2020-11-06 17:22:36 -08:00
version_edit.h Refactor with VersionEditHandler (#6581) 2020-11-11 08:00:14 -08:00
version_set_test.cc Refactor with VersionEditHandler (#6581) 2020-11-11 08:00:14 -08:00
version_set.cc Refactor with VersionEditHandler (#6581) 2020-11-11 08:00:14 -08:00
version_set.h Track WAL in MANIFEST: Update logic for computing min_log_number_to_keep in atomic flush (#7660) 2020-11-17 15:55:55 -08:00
wal_edit_test.cc Track WAL in MANIFEST: LogAndApply WAL events to MANIFEST (#7601) 2020-11-06 17:22:36 -08:00
wal_edit.cc Track WAL in MANIFEST: LogAndApply WAL events to MANIFEST (#7601) 2020-11-06 17:22:36 -08:00
wal_edit.h Track WAL in MANIFEST: LogAndApply WAL events to MANIFEST (#7601) 2020-11-06 17:22:36 -08:00
wal_manager_test.cc Make FileType Public and Replace kLogFile with kWalFile (#7580) 2020-10-22 17:06:20 -07:00
wal_manager.cc Make FileType Public and Replace kLogFile with kWalFile (#7580) 2020-10-22 17:06:20 -07:00
wal_manager.h Store FileSystemPtr object that contains FileSystem ptr (#7180) 2020-08-12 17:31:23 -07:00
write_batch_base.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
write_batch_internal.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
write_batch_test.cc Fail early when merge_operator not configured (#7667) 2020-11-16 20:39:01 -08:00
write_batch.cc Fail early when merge_operator not configured (#7667) 2020-11-16 20:39:01 -08:00
write_callback_test.cc Divide WriteCallbackTest.WriteWithCallbackTest (#7037) 2020-06-30 12:31:30 -07:00
write_callback.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
write_controller_test.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
write_controller.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
write_controller.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
write_thread.cc Fix StallWrite crash with mixed of slowdown/no_slowdown writes (#7508) 2020-10-06 12:44:20 -07:00
write_thread.h Remove the status.PermitUncheckedError() from WriteGroup Destructor (#7555) 2020-10-14 10:47:58 -07:00