andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
73f21a7b21
rocksdb/db
History
Maysam Yabandeh 73f21a7b21 Skip deleted WALs during recovery 
Summary:
This patch record the deleted WAL numbers in the manifest to ignore them and any WAL older than them during recovery. This is to avoid scenarios when we have a gap between the WAL files are fed to the recovery procedure. The gap could happen by for example out-of-order WAL deletion. Such gap could cause problems in 2PC recovery where the prepared and commit entry are placed into two separate WAL and gap in the WALs could result into not processing the WAL with the commit entry and hence breaking the 2PC recovery logic.
Closes https://github.com/facebook/rocksdb/pull/3488

Differential Revision: D6967893

Pulled By: maysamyabandeh

fbshipit-source-id: 13119feb155a08ab6d4909f437c7a750480dc8a1
2018-03-30 11:28:05 -07:00
..
builder.cc Comment out unused variables 2018-03-05 13:13:41 -08:00
builder.h Stream 2017-11-10 09:26:24 -08:00
c_test.c Added save points for transactions C API 2017-09-14 14:18:59 -07:00
c.cc C API for PerfContext 2018-03-21 22:13:34 -07:00
column_family_test.cc Memory Problem Of Destorying ColumnFamilyHandle after deleting the CF 2018-03-20 17:13:12 -07:00
column_family.cc FlushReason improvement 2018-03-22 18:42:18 -07:00
column_family.h Memory Problem Of Destorying ColumnFamilyHandle after deleting the CF 2018-03-20 17:13:12 -07:00
compact_files_test.cc Comment out unused variables 2018-03-05 13:13:41 -08:00
compacted_db_impl.cc Fix coverity issues column_family, compaction_db/iterator 2017-10-27 11:26:42 -07:00
compacted_db_impl.h Comment out unused variables 2018-03-05 13:13:41 -08:00
compaction_iteration_stats.h add counter for deletion dropping optimization 2017-08-19 14:10:08 -07:00
compaction_iterator_test.cc Comment out unused variables 2018-03-05 13:13:41 -08:00
compaction_iterator.cc Blob DB: Improve FIFO eviction 2018-03-06 11:57:42 -08:00
compaction_iterator.h Comment out unused variables 2018-03-05 13:13:41 -08:00
compaction_job_stats_test.cc Comment out unused variables 2018-03-05 13:13:41 -08:00
compaction_job_test.cc WritePrepared Txn: PreReleaseCallback 2017-11-30 23:50:45 -08:00
compaction_job.cc uint64_t and size_t changes to compile for iOS 2018-03-06 12:43:51 -08:00
compaction_job.h Make DBOption compaction_readahead_size dynamic 2017-11-16 17:57:25 -08:00
compaction_picker_test.cc Comment out unused variables 2018-03-05 13:13:41 -08:00
compaction_picker_universal.cc Comment out unused variables 2018-03-05 13:13:41 -08:00
compaction_picker_universal.h Make Universal compaction options dynamic 2017-12-11 13:27:06 -08:00
compaction_picker.cc uint64_t and size_t changes to compile for iOS 2018-03-06 12:43:51 -08:00
compaction_picker.h Comment out unused variables 2018-03-05 13:13:41 -08:00
compaction.cc Enable subcompactions in manual level-based compaction 2018-03-06 12:43:51 -08:00
compaction.h Change RocksDB License 2017-07-15 16:11:23 -07:00
comparator_db_test.cc Comment out unused variables 2018-03-05 13:13:41 -08:00
convenience.cc Delete files in multiple ranges at once 2018-01-30 13:56:39 -08:00
corruption_test.cc fix corruption_test valgrind 2017-08-11 12:29:14 -07:00
cuckoo_table_db_test.cc Enable MSVC W4 with a few exceptions. Fix warnings and bugs 2017-10-19 10:57:12 -07:00
db_basic_test.cc Disallow compactions if there isn't enough free space 2018-03-06 16:27:54 -08:00
db_blob_index_test.cc fix lite build 2017-10-17 08:57:09 -07:00
db_block_cache_test.cc Comment out unused variables 2018-03-05 13:13:41 -08:00
db_bloom_filter_test.cc Revert "comment out unused parameters" 2017-07-21 18:26:26 -07:00
db_compaction_filter_test.cc fix wrong length in snprintf 2018-03-16 13:27:55 -07:00
db_compaction_test.cc fix db_compaction_test when compression disabled 2018-03-19 12:30:05 -07:00
db_dynamic_level_test.cc Revert "comment out unused parameters" 2017-07-21 18:26:26 -07:00
db_encryption_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_filesnapshot.cc fix behavior does not match name for "IsFileDeletionsEnabled" 2018-03-21 22:13:34 -07:00
db_flush_test.cc Skip deleted WALs during recovery 2018-03-30 11:28:05 -07:00
db_impl_compaction_flush.cc Fix race condition causing double deletion of ssts 2018-03-28 10:29:59 -07:00
db_impl_debug.cc Disallow compactions if there isn't enough free space 2018-03-06 16:27:54 -08:00
db_impl_experimental.cc Inform caller when rocksdb is stalling writes 2017-10-05 18:11:43 -07:00
db_impl_files.cc Skip deleted WALs during recovery 2018-03-30 11:28:05 -07:00
db_impl_open.cc Skip deleted WALs during recovery 2018-03-30 11:28:05 -07:00
db_impl_readonly.cc Comment out unused variables 2018-03-05 13:13:41 -08:00
db_impl_readonly.h allowing CompactFiles to return new file names 2018-03-15 11:58:12 -07:00
db_impl_write.cc WritePrepared Txn: fix a bug in publishing recoverable state seq 2018-03-29 14:46:41 -07:00
db_impl.cc Fix race condition causing double deletion of ssts 2018-03-28 10:29:59 -07:00
db_impl.h WritePrepared Txn: make recoverable state visible after flush 2018-03-28 12:12:08 -07:00
db_info_dumper.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_info_dumper.h Change RocksDB License 2017-07-15 16:11:23 -07:00
db_inplace_update_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_io_failure_test.cc Fix LITE unit tests 2017-07-26 21:11:47 -07:00
db_iter_test.cc fix Seek with lower_bound 2017-11-29 22:56:29 -08:00
db_iter.cc Add rocksdb.iterator.internal-key property 2018-02-20 19:12:09 -08:00
db_iter.h WritePrepared Txn: Return NotSupported on iterator refresh 2017-12-18 22:29:30 -08:00
db_iterator_test.cc Comment out unused variables 2018-03-05 13:13:41 -08:00
db_log_iter_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_memtable_test.cc Comment out unused variables 2018-03-05 13:13:41 -08:00
db_merge_operator_test.cc Fix TSAN timeout in MergeOperatorPinningTest.Randomized/x test 2018-03-02 16:27:21 -08:00
db_options_test.cc fix live WALs purged while file deletions disabled 2018-01-17 17:42:04 -08:00
db_properties_test.cc fix behavior does not match name for "IsFileDeletionsEnabled" 2018-03-21 22:13:34 -07:00
db_range_del_test.cc Fix wrong smallest key of delete range tombstones 2017-08-29 18:41:35 -07:00
db_sst_test.cc fix flaky DBSSTTest.DeleteSchedulerMultipleDBPaths 2018-03-13 14:57:26 -07:00
db_statistics_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_table_properties_test.cc fix deletion-triggered compaction in table builder 2017-09-28 18:17:30 -07:00
db_tailing_iter_test.cc Use nullptr instead of NULL / 0 more consistently. 2018-03-07 12:42:12 -08:00
db_test2.cc Windows cumulative patch 2018-03-06 11:57:43 -08:00
db_test_util.cc Comment out unused variables 2018-03-05 13:13:41 -08:00
db_test_util.h Skip deleted WALs during recovery 2018-03-30 11:28:05 -07:00
db_test.cc Fix race condition via concurrent FlushWAL 2018-03-26 16:29:56 -07:00
db_universal_compaction_test.cc Comment out unused variables 2018-03-05 13:13:41 -08:00
db_wal_test.cc Skip deleted WALs during recovery 2018-03-30 11:28:05 -07:00
db_write_test.cc Rename function for handling WAL write error 2018-03-22 15:58:39 -07:00
dbformat_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
dbformat.cc Imporve perf of random read and insert compare by suggesting inlining to the compiler 2018-03-23 13:26:55 -07:00
dbformat.h Imporve perf of random read and insert compare by suggesting inlining to the compiler 2018-03-23 13:26:55 -07:00
deletefile_test.cc Use nullptr instead of NULL / 0 more consistently. 2018-03-07 12:42:12 -08:00
event_helpers.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
event_helpers.h Change RocksDB License 2017-07-15 16:11:23 -07:00
experimental.cc Replace dynamic_cast<> 2017-07-28 16:27:16 -07:00
external_sst_file_basic_test.cc optimize file ingestion checks for range deletion overlap 2017-11-28 11:27:02 -08:00
external_sst_file_ingestion_job.cc Fsync after writing global seq number in ExternalSstFileIngestionJob 2018-03-22 17:42:56 -07:00
external_sst_file_ingestion_job.h Optimize overlap checking for external file ingestion 2018-03-16 10:43:17 -07:00
external_sst_file_test.cc Skip deleted WALs during recovery 2018-03-30 11:28:05 -07:00
fault_injection_test.cc Use nullptr instead of NULL / 0 more consistently. 2018-03-07 12:42:12 -08:00
file_indexer_test.cc Comment out unused variables 2018-03-05 13:13:41 -08:00
file_indexer.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
file_indexer.h Change RocksDB License 2017-07-15 16:11:23 -07:00
filename_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
flush_job_test.cc Add a histogram stat for memtable flush 2017-12-15 18:57:00 -08:00
flush_job.cc FlushReason improvement 2018-03-22 18:42:18 -07:00
flush_job.h WritePrepared Txn: Compaction/Flush 2017-10-06 10:41:53 -07:00
flush_scheduler.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
flush_scheduler.h Change RocksDB License 2017-07-15 16:11:23 -07:00
forward_iterator_bench.cc fix gflags namespace 2017-12-01 10:42:05 -08:00
forward_iterator.cc Comment out unused variables 2018-03-05 13:13:41 -08:00
forward_iterator.h Comment out unused variables 2018-03-05 13:13:41 -08:00
internal_stats.cc Comment out unused variables 2018-03-05 13:13:41 -08:00
internal_stats.h Add "rocksdb.live-sst-files-size" DB property 2018-03-01 18:01:10 -08:00
job_context.h Prevent unnecessary calls to PurgeObsoleteFiles 2018-01-12 13:27:08 -08:00
listener_test.cc Comment out unused variables 2018-03-05 13:13:41 -08:00
log_format.h Change RocksDB License 2017-07-15 16:11:23 -07:00
log_reader.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
log_reader.h Suppress lint in old files 2018-01-29 12:56:42 -08:00
log_test.cc Use nullptr instead of NULL / 0 more consistently. 2018-03-07 12:42:12 -08:00
log_writer.cc uint64_t and size_t changes to compile for iOS 2018-03-06 12:43:51 -08:00
log_writer.h Clarification regarding record format 2018-03-08 10:27:25 -08:00
malloc_stats.cc Disallow compactions if there isn't enough free space 2018-03-06 16:27:54 -08:00
malloc_stats.h Change RocksDB License 2017-07-15 16:11:23 -07:00
managed_iterator.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
managed_iterator.h Change RocksDB License 2017-07-15 16:11:23 -07:00
manual_compaction_test.cc Comment out unused variables 2018-03-05 13:13:41 -08:00
memtable_list_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
memtable_list.cc Comment out unused variables 2018-03-05 13:13:41 -08:00
memtable_list.h Fix Flush() keep waiting after flush finish 2018-01-18 17:45:16 -08:00
memtable.cc InlineSkiplist: don't decode keys unnecessarily during comparisons 2018-03-23 12:14:30 -07:00
memtable.h InlineSkiplist: don't decode keys unnecessarily during comparisons 2018-03-23 12:14:30 -07:00
merge_context.h Change RocksDB License 2017-07-15 16:11:23 -07:00
merge_helper_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
merge_helper.cc WritePrepared Txn: Support merge operator 2018-02-09 14:57:54 -08:00
merge_helper.h WritePrepared Txn: Support merge operator 2018-02-09 14:57:54 -08:00
merge_operator.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
merge_test.cc Comment out unused variables 2018-03-05 13:13:41 -08:00
obsolete_files_test.cc Fix race condition causing double deletion of ssts 2018-03-28 10:29:59 -07:00
options_file_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
perf_context_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
pinned_iterators_manager.h Change RocksDB License 2017-07-15 16:11:23 -07:00
plain_table_db_test.cc Comment out unused variables 2018-03-05 13:13:41 -08:00
pre_release_callback.h WritePrepared Txn: fix race condition on publishing seq 2018-03-22 14:43:36 -07:00
prefix_test.cc Comment out unused variables 2018-03-05 13:13:41 -08:00
range_del_aggregator_test.cc optimize file ingestion checks for range deletion overlap 2017-11-28 11:27:02 -08:00
range_del_aggregator.cc Avoid adding tombstones of the same file to RangeDelAggregator multiple times 2018-03-23 12:43:06 -07:00
range_del_aggregator.h Avoid adding tombstones of the same file to RangeDelAggregator multiple times 2018-03-23 12:43:06 -07:00
read_callback.h write-prepared txn: call IsInSnapshot 2017-09-11 09:14:48 -07:00
repair_test.cc fix file numbers after repair 2017-10-10 13:12:37 -07:00
repair.cc WritePrepared Txn: PreReleaseCallback 2017-11-30 23:50:45 -08:00
snapshot_checker.h Comment out unused variables 2018-03-05 13:13:41 -08:00
snapshot_impl.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
snapshot_impl.h Disable options_settable_test in UBSAN and fix UBSAN failure in blob_… 2018-02-07 14:42:26 -08:00
table_cache.cc Avoid adding tombstones of the same file to RangeDelAggregator multiple times 2018-03-23 12:43:06 -07:00
table_cache.h Change RocksDB License 2017-07-15 16:11:23 -07:00
table_properties_collector_test.cc Comment out unused variables 2018-03-05 13:13:41 -08:00
table_properties_collector.cc Comment out unused variables 2018-03-05 13:13:41 -08:00
table_properties_collector.h Comment out unused variables 2018-03-05 13:13:41 -08:00
transaction_log_impl.cc Comment out unused variables 2018-03-05 13:13:41 -08:00
transaction_log_impl.h WritePrepared Txn: Refactor conf params 2017-11-10 17:28:12 -08:00
version_builder_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
version_builder.cc Fix some typos in comments and docs. 2018-03-08 10:27:25 -08:00
version_builder.h Allow DB reopen with reduced options.num_levels 2017-08-24 16:10:54 -07:00
version_edit_test.cc Skip deleted WALs during recovery 2018-03-30 11:28:05 -07:00
version_edit.cc Skip deleted WALs during recovery 2018-03-30 11:28:05 -07:00
version_edit.h Skip deleted WALs during recovery 2018-03-30 11:28:05 -07:00
version_set_test.cc Comment out unused variables 2018-03-05 13:13:41 -08:00
version_set.cc Skip deleted WALs during recovery 2018-03-30 11:28:05 -07:00
version_set.h Skip deleted WALs during recovery 2018-03-30 11:28:05 -07:00
wal_manager_test.cc Comment out unused variables 2018-03-05 13:13:41 -08:00
wal_manager.cc Fix memleak when DB::DeleteFile() 2018-01-11 18:57:33 -08:00
wal_manager.h Fix memleak when DB::DeleteFile() 2018-01-11 18:57:33 -08:00
write_batch_base.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
write_batch_internal.h Fix some typos in comments and docs. 2018-03-08 10:27:25 -08:00
write_batch_test.cc Comment out unused variables 2018-03-05 13:13:41 -08:00
write_batch.cc Optionally create DuplicateDetector 2018-03-14 00:57:25 -07:00
write_callback_test.cc WritePrepared Txn: fix race condition on publishing seq 2018-03-22 14:43:36 -07:00
write_callback.h Change RocksDB License 2017-07-15 16:11:23 -07:00
write_controller_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
write_controller.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
write_controller.h Change RocksDB License 2017-07-15 16:11:23 -07:00
write_thread.cc Comment out unused variables 2018-03-05 13:13:41 -08:00
write_thread.h WritePrepared Txn: Duplicate Keys, Txn Part 2018-02-05 18:43:24 -08:00