Siying Dong d5afa73789 Revert "Skip deleted WALs during recovery" ...

Summary:
This reverts commit 73f21a7b2177aeb82b9f518222e2b9ea8fbb7c4f.

It breaks compatibility. When created a DB using a build with this new change, opening the DB and reading the data will fail with this error:

"Corruption: Can't access /000000.sst: IO error: while stat a file for size: /tmp/xxxx/000000.sst: No such file or directory"

This is because the dummy AddFile4 entry generated by the new code will be treated as a real entry by an older build. The older build will think there is a real file with number 0, but there isn't such a file.
Closes https://github.com/facebook/rocksdb/pull/3762

Differential Revision: D7730035

Pulled By: siying

fbshipit-source-id: f2051859eff20ef1837575ecb1e1bb96b3751e77

2018-04-23 12:01:26 -07:00

..

builder.cc

Support for Column family specific paths.

2018-04-05 19:58:20 -07:00

builder.h

Stream

2017-11-10 09:26:24 -08:00

c_test.c

comment unused parameters to turn on -Wunused-parameter flag

2018-04-12 17:59:16 -07:00

c.cc

Fix GitHub issue #3716: gcc-8 warnings

2018-04-20 13:42:47 -07:00

column_family_test.cc

fix memory leak in two_level_iterator

2018-04-15 17:26:26 -07:00

column_family.cc

comment unused parameters to turn on -Wunused-parameter flag

2018-04-12 17:59:16 -07:00

column_family.h

Support for Column family specific paths.

2018-04-05 19:58:20 -07:00

compact_files_test.cc

fix memory leak in two_level_iterator

2018-04-15 17:26:26 -07:00

compacted_db_impl.cc

Fix coverity issues column_family, compaction_db/iterator

2017-10-27 11:26:42 -07:00

compacted_db_impl.h

Comment out unused variables

2018-03-05 13:13:41 -08:00

compaction_iteration_stats.h

add counter for deletion dropping optimization

2017-08-19 14:10:08 -07:00

compaction_iterator_test.cc

Comment out unused variables

2018-03-05 13:13:41 -08:00

compaction_iterator.cc

Blob DB: Improve FIFO eviction

2018-03-06 11:57:42 -08:00

compaction_iterator.h

Comment out unused variables

2018-03-05 13:13:41 -08:00

compaction_job_stats_test.cc

fix memory leak in two_level_iterator

2018-04-15 17:26:26 -07:00

compaction_job_test.cc

fix memory leak in two_level_iterator

2018-04-15 17:26:26 -07:00

compaction_job.cc

comment unused parameters to turn on -Wunused-parameter flag

2018-04-12 17:59:16 -07:00

compaction_job.h

Make DBOption compaction_readahead_size dynamic

2017-11-16 17:57:25 -08:00

compaction_picker_test.cc

Support for Column family specific paths.

2018-04-05 19:58:20 -07:00

compaction_picker_universal.cc

Support for Column family specific paths.

2018-04-05 19:58:20 -07:00

compaction_picker_universal.h

Make Universal compaction options dynamic

2017-12-11 13:27:06 -08:00

compaction_picker.cc

comment unused parameters to turn on -Wunused-parameter flag

2018-04-12 17:59:16 -07:00

compaction_picker.h

Comment out unused variables

2018-03-05 13:13:41 -08:00

compaction.cc

Enable subcompactions in manual level-based compaction

2018-03-06 12:43:51 -08:00

compaction.h

Change RocksDB License

2017-07-15 16:11:23 -07:00

comparator_db_test.cc

Comment out unused variables

2018-03-05 13:13:41 -08:00

convenience.cc

Delete files in multiple ranges at once

2018-01-30 13:56:39 -08:00

corruption_test.cc

fix memory leak in two_level_iterator

2018-04-15 17:26:26 -07:00

cuckoo_table_db_test.cc

fix memory leak in two_level_iterator

2018-04-15 17:26:26 -07:00

db_basic_test.cc

Disallow compactions if there isn't enough free space

2018-03-06 16:27:54 -08:00

db_blob_index_test.cc

fix lite build

2017-10-17 08:57:09 -07:00

db_block_cache_test.cc

Comment out unused variables

2018-03-05 13:13:41 -08:00

db_bloom_filter_test.cc

comment unused parameters to turn on -Wunused-parameter flag

2018-04-12 17:59:16 -07:00

db_compaction_filter_test.cc

fix wrong length in snprintf

2018-03-16 13:27:55 -07:00

db_compaction_test.cc

fix memory leak in two_level_iterator

2018-04-15 17:26:26 -07:00

db_dynamic_level_test.cc

fix memory leak in two_level_iterator

2018-04-15 17:26:26 -07:00

db_encryption_test.cc

Change RocksDB License

2017-07-15 16:11:23 -07:00

db_filesnapshot.cc

fix behavior does not match name for "IsFileDeletionsEnabled"

2018-03-21 22:13:34 -07:00

db_flush_test.cc

Revert "Skip deleted WALs during recovery"

2018-04-23 12:01:26 -07:00

db_impl_compaction_flush.cc

check return status for Sync() and Append() calls to avoid corruption

2018-04-19 14:13:46 -07:00

db_impl_debug.cc

Disallow compactions if there isn't enough free space

2018-03-06 16:27:54 -08:00

db_impl_experimental.cc

Inform caller when rocksdb is stalling writes

2017-10-05 18:11:43 -07:00

db_impl_files.cc

Revert "Skip deleted WALs during recovery"

2018-04-23 12:01:26 -07:00

db_impl_open.cc

Revert "Skip deleted WALs during recovery"

2018-04-23 12:01:26 -07:00

db_impl_readonly.cc

comment unused parameters to turn on -Wunused-parameter flag

2018-04-12 17:59:16 -07:00

db_impl_readonly.h

allowing CompactFiles to return new file names

2018-03-15 11:58:12 -07:00

db_impl_write.cc

WritePrepared Txn: fix a bug in publishing recoverable state seq

2018-03-29 14:46:41 -07:00

db_impl.cc

Add a stat for MultiGet keys found, update memtable hit/miss stats

2018-04-20 15:28:19 -07:00

db_impl.h

WritePrepared Txn: add stats

2018-04-07 21:56:42 -07:00

db_info_dumper.cc

Change RocksDB License

2017-07-15 16:11:23 -07:00

db_info_dumper.h

Change RocksDB License

2017-07-15 16:11:23 -07:00

db_inplace_update_test.cc

Change RocksDB License

2017-07-15 16:11:23 -07:00

db_io_failure_test.cc

Fix LITE unit tests

2017-07-26 21:11:47 -07:00

db_iter_test.cc

comment unused parameters to turn on -Wunused-parameter flag

2018-04-12 17:59:16 -07:00

db_iter.cc

Add rocksdb.iterator.internal-key property

2018-02-20 19:12:09 -08:00

db_iter.h

WritePrepared Txn: Return NotSupported on iterator refresh

2017-12-18 22:29:30 -08:00

db_iterator_test.cc

Comment out unused variables

2018-03-05 13:13:41 -08:00

db_log_iter_test.cc

fix memory leak in two_level_iterator

2018-04-15 17:26:26 -07:00

db_memtable_test.cc

Comment out unused variables

2018-03-05 13:13:41 -08:00

db_merge_operator_test.cc

Fix TSAN timeout in MergeOperatorPinningTest.Randomized/x test

2018-03-02 16:27:21 -08:00

db_options_test.cc

comment unused parameters to turn on -Wunused-parameter flag

2018-04-12 17:59:16 -07:00

db_properties_test.cc

Add block cache related DB properties

2018-04-18 21:42:25 -07:00

db_range_del_test.cc

Fix wrong smallest key of delete range tombstones

2017-08-29 18:41:35 -07:00

db_sst_test.cc

comment unused parameters to turn on -Wunused-parameter flag

2018-04-12 17:59:16 -07:00

db_statistics_test.cc

Change RocksDB License

2017-07-15 16:11:23 -07:00

db_table_properties_test.cc

fix deletion-triggered compaction in table builder

2017-09-28 18:17:30 -07:00

db_tailing_iter_test.cc

fix memory leak in two_level_iterator

2018-04-15 17:26:26 -07:00

db_test2.cc

comment unused parameters to turn on -Wunused-parameter flag

2018-04-12 17:59:16 -07:00

db_test_util.cc

Support for Column family specific paths.

2018-04-05 19:58:20 -07:00

db_test_util.h

Revert "Skip deleted WALs during recovery"

2018-04-23 12:01:26 -07:00

db_test.cc

comment unused parameters to turn on -Wunused-parameter flag

2018-04-12 17:59:16 -07:00

db_universal_compaction_test.cc

fix memory leak in two_level_iterator

2018-04-15 17:26:26 -07:00

db_wal_test.cc

Revert "Skip deleted WALs during recovery"

2018-04-23 12:01:26 -07:00

db_write_test.cc

Rename function for handling WAL write error

2018-03-22 15:58:39 -07:00

dbformat_test.cc

Change RocksDB License

2017-07-15 16:11:23 -07:00

dbformat.cc

add kEntryRangeDeletion

2018-04-13 11:27:17 -07:00

dbformat.h

Imporve perf of random read and insert compare by suggesting inlining to the compiler

2018-03-23 13:26:55 -07:00

deletefile_test.cc

fix memory leak in two_level_iterator

2018-04-15 17:26:26 -07:00

event_helpers.cc

comment unused parameters to turn on -Wunused-parameter flag

2018-04-12 17:59:16 -07:00

event_helpers.h

Change RocksDB License

2017-07-15 16:11:23 -07:00

experimental.cc

comment unused parameters to turn on -Wunused-parameter flag

2018-04-12 17:59:16 -07:00

external_sst_file_basic_test.cc

optimize file ingestion checks for range deletion overlap

2017-11-28 11:27:02 -08:00

external_sst_file_ingestion_job.cc

Improve accuracy of I/O stats collection of external SST ingestion.

2018-04-13 10:58:42 -07:00

external_sst_file_ingestion_job.h

Initialize a boolean member variable of a struct.

2018-04-16 14:28:01 -07:00

external_sst_file_test.cc

Revert "Skip deleted WALs during recovery"

2018-04-23 12:01:26 -07:00

fault_injection_test.cc

comment unused parameters to turn on -Wunused-parameter flag

2018-04-12 17:59:16 -07:00

file_indexer_test.cc

Comment out unused variables

2018-03-05 13:13:41 -08:00

file_indexer.cc

Change RocksDB License

2017-07-15 16:11:23 -07:00

file_indexer.h

Change RocksDB License

2017-07-15 16:11:23 -07:00

filename_test.cc

Change RocksDB License

2017-07-15 16:11:23 -07:00

flush_job_test.cc

Add a histogram stat for memtable flush

2017-12-15 18:57:00 -08:00

flush_job.cc

Improve visibility into the reasons for compaction.

2018-04-11 10:58:44 -07:00

flush_job.h

WritePrepared Txn: Compaction/Flush

2017-10-06 10:41:53 -07:00

flush_scheduler.cc

Change RocksDB License

2017-07-15 16:11:23 -07:00

flush_scheduler.h

Change RocksDB License

2017-07-15 16:11:23 -07:00

forward_iterator_bench.cc

fix gflags namespace

2017-12-01 10:42:05 -08:00

forward_iterator.cc

Comment out unused variables

2018-03-05 13:13:41 -08:00

forward_iterator.h

Comment out unused variables

2018-03-05 13:13:41 -08:00

internal_stats.cc

Add block cache related DB properties

2018-04-18 21:42:25 -07:00

internal_stats.h

Add block cache related DB properties

2018-04-18 21:42:25 -07:00

job_context.h

comment unused parameters to turn on -Wunused-parameter flag

2018-04-12 17:59:16 -07:00

listener_test.cc

fix some text in comments.

2018-04-10 15:59:24 -07:00

log_format.h

Change RocksDB License

2017-07-15 16:11:23 -07:00

log_reader.cc

Change RocksDB License

2017-07-15 16:11:23 -07:00

log_reader.h

Suppress lint in old files

2018-01-29 12:56:42 -08:00

log_test.cc

Use nullptr instead of NULL / 0 more consistently.

2018-03-07 12:42:12 -08:00

log_writer.cc

check return status for Sync() and Append() calls to avoid corruption

2018-04-19 14:13:46 -07:00

log_writer.h

Clarification regarding record format

2018-03-08 10:27:25 -08:00

malloc_stats.cc

Disallow compactions if there isn't enough free space

2018-03-06 16:27:54 -08:00

malloc_stats.h

Change RocksDB License

2017-07-15 16:11:23 -07:00

managed_iterator.cc

Change RocksDB License

2017-07-15 16:11:23 -07:00

managed_iterator.h

Change RocksDB License

2017-07-15 16:11:23 -07:00

manual_compaction_test.cc

Comment out unused variables

2018-03-05 13:13:41 -08:00

memtable_list_test.cc

Change RocksDB License

2017-07-15 16:11:23 -07:00

memtable_list.cc

Comment out unused variables

2018-03-05 13:13:41 -08:00

memtable_list.h

Fix Flush() keep waiting after flush finish

2018-01-18 17:45:16 -08:00

memtable.cc

WritePrepared Txn: add stats

2018-04-07 21:56:42 -07:00

memtable.h

InlineSkiplist: don't decode keys unnecessarily during comparisons

2018-03-23 12:14:30 -07:00

merge_context.h

Change RocksDB License

2017-07-15 16:11:23 -07:00

merge_helper_test.cc

Change RocksDB License

2017-07-15 16:11:23 -07:00

merge_helper.cc

WritePrepared Txn: Support merge operator

2018-02-09 14:57:54 -08:00

merge_helper.h

WritePrepared Txn: Support merge operator

2018-02-09 14:57:54 -08:00

merge_operator.cc

Change RocksDB License

2017-07-15 16:11:23 -07:00

merge_test.cc

Comment out unused variables

2018-03-05 13:13:41 -08:00

obsolete_files_test.cc

fix memory leak in two_level_iterator

2018-04-15 17:26:26 -07:00

options_file_test.cc

fix memory leak in two_level_iterator

2018-04-15 17:26:26 -07:00

perf_context_test.cc

Change RocksDB License

2017-07-15 16:11:23 -07:00

pinned_iterators_manager.h

Change RocksDB License

2017-07-15 16:11:23 -07:00

plain_table_db_test.cc

fix memory leak in two_level_iterator

2018-04-15 17:26:26 -07:00

pre_release_callback.h

Fix pre_release callback argument list.

2018-04-05 11:12:16 -07:00

prefix_test.cc

fix memory leak in two_level_iterator

2018-04-15 17:26:26 -07:00

range_del_aggregator_test.cc

optimize file ingestion checks for range deletion overlap

2017-11-28 11:27:02 -08:00

range_del_aggregator.cc

Revert "Avoid adding tombstones of the same file to RangeDelAggregato…

2018-04-02 19:58:04 -07:00

range_del_aggregator.h

Revert "Avoid adding tombstones of the same file to RangeDelAggregato…

2018-04-02 19:58:04 -07:00

read_callback.h

write-prepared txn: call IsInSnapshot

2017-09-11 09:14:48 -07:00

repair_test.cc

fix memory leak in two_level_iterator

2018-04-15 17:26:26 -07:00

repair.cc

Support for Column family specific paths.

2018-04-05 19:58:20 -07:00

snapshot_checker.h

Comment out unused variables

2018-03-05 13:13:41 -08:00

snapshot_impl.cc

Change RocksDB License

2017-07-15 16:11:23 -07:00

snapshot_impl.h

WritePrepared Txn: smallest_prepare optimization

2018-04-02 20:27:41 -07:00

table_cache.cc

Support for Column family specific paths.

2018-04-05 19:58:20 -07:00

table_cache.h

Change RocksDB License

2017-07-15 16:11:23 -07:00

table_properties_collector_test.cc

Comment out unused variables

2018-03-05 13:13:41 -08:00

table_properties_collector.cc

Comment out unused variables

2018-03-05 13:13:41 -08:00

table_properties_collector.h

Comment out unused variables

2018-03-05 13:13:41 -08:00

transaction_log_impl.cc

Comment out unused variables

2018-03-05 13:13:41 -08:00

transaction_log_impl.h

WritePrepared Txn: Refactor conf params

2017-11-10 17:28:12 -08:00

version_builder_test.cc

Change RocksDB License

2017-07-15 16:11:23 -07:00

version_builder.cc

Fix some typos in comments and docs.

2018-03-08 10:27:25 -08:00

version_builder.h

Allow DB reopen with reduced options.num_levels

2017-08-24 16:10:54 -07:00

version_edit_test.cc

Revert "Skip deleted WALs during recovery"

2018-04-23 12:01:26 -07:00

version_edit.cc

Revert "Skip deleted WALs during recovery"

2018-04-23 12:01:26 -07:00

version_edit.h

Revert "Skip deleted WALs during recovery"

2018-04-23 12:01:26 -07:00

version_set_test.cc

Comment out unused variables

2018-03-05 13:13:41 -08:00

version_set.cc

Revert "Skip deleted WALs during recovery"

2018-04-23 12:01:26 -07:00

version_set.h

Revert "Skip deleted WALs during recovery"

2018-04-23 12:01:26 -07:00

wal_manager_test.cc

fix memory leak in two_level_iterator

2018-04-15 17:26:26 -07:00

wal_manager.cc

Fix memleak when DB::DeleteFile()

2018-01-11 18:57:33 -08:00

wal_manager.h

Fix memleak when DB::DeleteFile()

2018-01-11 18:57:33 -08:00

write_batch_base.cc

Change RocksDB License

2017-07-15 16:11:23 -07:00

write_batch_internal.h

Fix some typos in comments and docs.

2018-03-08 10:27:25 -08:00

write_batch_test.cc

Comment out unused variables

2018-03-05 13:13:41 -08:00

write_batch.cc

WritePrepared Txn: enable TryAgain for duplicates at the end of the batch

2018-04-20 15:28:19 -07:00

write_callback_test.cc

fix memory leak in two_level_iterator

2018-04-15 17:26:26 -07:00

write_callback.h

Change RocksDB License

2017-07-15 16:11:23 -07:00

write_controller_test.cc

Change RocksDB License

2017-07-15 16:11:23 -07:00

write_controller.cc

Change RocksDB License

2017-07-15 16:11:23 -07:00

write_controller.h

Change RocksDB License

2017-07-15 16:11:23 -07:00

write_thread.cc

Comment out unused variables

2018-03-05 13:13:41 -08:00

write_thread.h

WritePrepared Txn: Duplicate Keys, Txn Part

2018-02-05 18:43:24 -08:00