andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
rocksdb/db
History
Maysam Yabandeh 73f21a7b21 Skip deleted WALs during recovery 
Summary:
This patch record the deleted WAL numbers in the manifest to ignore them and any WAL older than them during recovery. This is to avoid scenarios when we have a gap between the WAL files are fed to the recovery procedure. The gap could happen by for example out-of-order WAL deletion. Such gap could cause problems in 2PC recovery where the prepared and commit entry are placed into two separate WAL and gap in the WALs could result into not processing the WAL with the commit entry and hence breaking the 2PC recovery logic.
Closes https://github.com/facebook/rocksdb/pull/3488

Differential Revision: D6967893

Pulled By: maysamyabandeh

fbshipit-source-id: 13119feb155a08ab6d4909f437c7a750480dc8a1
2018-03-30 11:28:05 -07:00
..
builder.cc
Comment out unused variables
2018-03-05 13:13:41 -08:00
builder.h
Stream
2017-11-10 09:26:24 -08:00
c_test.c
Added save points for transactions C API
2017-09-14 14:18:59 -07:00
c.cc
C API for PerfContext
2018-03-21 22:13:34 -07:00
column_family_test.cc
Memory Problem Of Destorying ColumnFamilyHandle after deleting the CF
2018-03-20 17:13:12 -07:00
column_family.cc
FlushReason improvement
2018-03-22 18:42:18 -07:00
column_family.h
Memory Problem Of Destorying ColumnFamilyHandle after deleting the CF
2018-03-20 17:13:12 -07:00
compact_files_test.cc
Comment out unused variables
2018-03-05 13:13:41 -08:00
compacted_db_impl.cc
Fix coverity issues column_family, compaction_db/iterator
2017-10-27 11:26:42 -07:00
compacted_db_impl.h
Comment out unused variables
2018-03-05 13:13:41 -08:00
compaction_iteration_stats.h
add counter for deletion dropping optimization
2017-08-19 14:10:08 -07:00
compaction_iterator_test.cc
Comment out unused variables
2018-03-05 13:13:41 -08:00
compaction_iterator.cc
Blob DB: Improve FIFO eviction
2018-03-06 11:57:42 -08:00
compaction_iterator.h
Comment out unused variables
2018-03-05 13:13:41 -08:00
compaction_job_stats_test.cc
Comment out unused variables
2018-03-05 13:13:41 -08:00
compaction_job_test.cc
WritePrepared Txn: PreReleaseCallback
2017-11-30 23:50:45 -08:00
compaction_job.cc
uint64_t and size_t changes to compile for iOS
2018-03-06 12:43:51 -08:00
compaction_job.h
Make DBOption compaction_readahead_size dynamic
2017-11-16 17:57:25 -08:00
compaction_picker_test.cc
Comment out unused variables
2018-03-05 13:13:41 -08:00
compaction_picker_universal.cc
Comment out unused variables
2018-03-05 13:13:41 -08:00
compaction_picker_universal.h
Make Universal compaction options dynamic
2017-12-11 13:27:06 -08:00
compaction_picker.cc
uint64_t and size_t changes to compile for iOS
2018-03-06 12:43:51 -08:00
compaction_picker.h
Comment out unused variables
2018-03-05 13:13:41 -08:00
compaction.cc
Enable subcompactions in manual level-based compaction
2018-03-06 12:43:51 -08:00
compaction.h
Change RocksDB License
2017-07-15 16:11:23 -07:00
comparator_db_test.cc
Comment out unused variables
2018-03-05 13:13:41 -08:00
convenience.cc
Delete files in multiple ranges at once
2018-01-30 13:56:39 -08:00
corruption_test.cc
fix corruption_test valgrind
2017-08-11 12:29:14 -07:00
cuckoo_table_db_test.cc
Enable MSVC W4 with a few exceptions. Fix warnings and bugs
2017-10-19 10:57:12 -07:00
db_basic_test.cc
Disallow compactions if there isn't enough free space
2018-03-06 16:27:54 -08:00
db_blob_index_test.cc
fix lite build
2017-10-17 08:57:09 -07:00
db_block_cache_test.cc
Comment out unused variables
2018-03-05 13:13:41 -08:00
db_bloom_filter_test.cc
Revert "comment out unused parameters"
2017-07-21 18:26:26 -07:00
db_compaction_filter_test.cc
fix wrong length in snprintf
2018-03-16 13:27:55 -07:00
db_compaction_test.cc
fix db_compaction_test when compression disabled
2018-03-19 12:30:05 -07:00
db_dynamic_level_test.cc
Revert "comment out unused parameters"
2017-07-21 18:26:26 -07:00
db_encryption_test.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
db_filesnapshot.cc
fix behavior does not match name for "IsFileDeletionsEnabled"
2018-03-21 22:13:34 -07:00
db_flush_test.cc
Skip deleted WALs during recovery
2018-03-30 11:28:05 -07:00
db_impl_compaction_flush.cc
Fix race condition causing double deletion of ssts
2018-03-28 10:29:59 -07:00
db_impl_debug.cc
Disallow compactions if there isn't enough free space
2018-03-06 16:27:54 -08:00
db_impl_experimental.cc
Inform caller when rocksdb is stalling writes
2017-10-05 18:11:43 -07:00
db_impl_files.cc
Skip deleted WALs during recovery
2018-03-30 11:28:05 -07:00
db_impl_open.cc
Skip deleted WALs during recovery
2018-03-30 11:28:05 -07:00
db_impl_readonly.cc
Comment out unused variables
2018-03-05 13:13:41 -08:00
db_impl_readonly.h
allowing CompactFiles to return new file names
2018-03-15 11:58:12 -07:00
db_impl_write.cc
WritePrepared Txn: fix a bug in publishing recoverable state seq
2018-03-29 14:46:41 -07:00
db_impl.cc
Fix race condition causing double deletion of ssts
2018-03-28 10:29:59 -07:00
db_impl.h
WritePrepared Txn: make recoverable state visible after flush
2018-03-28 12:12:08 -07:00
db_info_dumper.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
db_info_dumper.h
Change RocksDB License
2017-07-15 16:11:23 -07:00
db_inplace_update_test.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
db_io_failure_test.cc
Fix LITE unit tests
2017-07-26 21:11:47 -07:00
db_iter_test.cc
fix Seek with lower_bound
2017-11-29 22:56:29 -08:00
db_iter.cc
Add rocksdb.iterator.internal-key property
2018-02-20 19:12:09 -08:00
db_iter.h
WritePrepared Txn: Return NotSupported on iterator refresh
2017-12-18 22:29:30 -08:00
db_iterator_test.cc
Comment out unused variables
2018-03-05 13:13:41 -08:00
db_log_iter_test.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
db_memtable_test.cc
Comment out unused variables
2018-03-05 13:13:41 -08:00
db_merge_operator_test.cc
Fix TSAN timeout in MergeOperatorPinningTest.Randomized/x test
2018-03-02 16:27:21 -08:00
db_options_test.cc
fix live WALs purged while file deletions disabled
2018-01-17 17:42:04 -08:00
db_properties_test.cc
fix behavior does not match name for "IsFileDeletionsEnabled"
2018-03-21 22:13:34 -07:00
db_range_del_test.cc
Fix wrong smallest key of delete range tombstones
2017-08-29 18:41:35 -07:00
db_sst_test.cc
fix flaky DBSSTTest.DeleteSchedulerMultipleDBPaths
2018-03-13 14:57:26 -07:00
db_statistics_test.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
db_table_properties_test.cc
fix deletion-triggered compaction in table builder
2017-09-28 18:17:30 -07:00
db_tailing_iter_test.cc
Use nullptr instead of NULL / 0 more consistently.
2018-03-07 12:42:12 -08:00
db_test2.cc
Windows cumulative patch
2018-03-06 11:57:43 -08:00
db_test_util.cc
Comment out unused variables
2018-03-05 13:13:41 -08:00
db_test_util.h
Skip deleted WALs during recovery
2018-03-30 11:28:05 -07:00
db_test.cc
Fix race condition via concurrent FlushWAL
2018-03-26 16:29:56 -07:00
db_universal_compaction_test.cc
Comment out unused variables
2018-03-05 13:13:41 -08:00
db_wal_test.cc
Skip deleted WALs during recovery
2018-03-30 11:28:05 -07:00
db_write_test.cc
Rename function for handling WAL write error
2018-03-22 15:58:39 -07:00
dbformat_test.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
dbformat.cc
Imporve perf of random read and insert compare by suggesting inlining to the compiler
2018-03-23 13:26:55 -07:00
dbformat.h
Imporve perf of random read and insert compare by suggesting inlining to the compiler
2018-03-23 13:26:55 -07:00
deletefile_test.cc
Use nullptr instead of NULL / 0 more consistently.
2018-03-07 12:42:12 -08:00
event_helpers.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
event_helpers.h
Change RocksDB License
2017-07-15 16:11:23 -07:00
experimental.cc
Replace dynamic_cast<>
2017-07-28 16:27:16 -07:00
external_sst_file_basic_test.cc
optimize file ingestion checks for range deletion overlap
2017-11-28 11:27:02 -08:00
external_sst_file_ingestion_job.cc
Fsync after writing global seq number in ExternalSstFileIngestionJob
2018-03-22 17:42:56 -07:00
external_sst_file_ingestion_job.h
Optimize overlap checking for external file ingestion
2018-03-16 10:43:17 -07:00
external_sst_file_test.cc
Skip deleted WALs during recovery
2018-03-30 11:28:05 -07:00
fault_injection_test.cc
Use nullptr instead of NULL / 0 more consistently.
2018-03-07 12:42:12 -08:00
file_indexer_test.cc
Comment out unused variables
2018-03-05 13:13:41 -08:00
file_indexer.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
file_indexer.h
Change RocksDB License
2017-07-15 16:11:23 -07:00
filename_test.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
flush_job_test.cc
Add a histogram stat for memtable flush
2017-12-15 18:57:00 -08:00
flush_job.cc
FlushReason improvement
2018-03-22 18:42:18 -07:00
flush_job.h
WritePrepared Txn: Compaction/Flush
2017-10-06 10:41:53 -07:00
flush_scheduler.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
flush_scheduler.h
Change RocksDB License
2017-07-15 16:11:23 -07:00
forward_iterator_bench.cc
fix gflags namespace
2017-12-01 10:42:05 -08:00
forward_iterator.cc
Comment out unused variables
2018-03-05 13:13:41 -08:00
forward_iterator.h
Comment out unused variables
2018-03-05 13:13:41 -08:00
internal_stats.cc
Comment out unused variables
2018-03-05 13:13:41 -08:00
internal_stats.h
Add "rocksdb.live-sst-files-size" DB property
2018-03-01 18:01:10 -08:00
job_context.h
Prevent unnecessary calls to PurgeObsoleteFiles
2018-01-12 13:27:08 -08:00
listener_test.cc
Comment out unused variables
2018-03-05 13:13:41 -08:00
log_format.h
Change RocksDB License
2017-07-15 16:11:23 -07:00
log_reader.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
log_reader.h
Suppress lint in old files
2018-01-29 12:56:42 -08:00
log_test.cc
Use nullptr instead of NULL / 0 more consistently.
2018-03-07 12:42:12 -08:00
log_writer.cc
uint64_t and size_t changes to compile for iOS
2018-03-06 12:43:51 -08:00
log_writer.h
Clarification regarding record format
2018-03-08 10:27:25 -08:00
malloc_stats.cc
Disallow compactions if there isn't enough free space
2018-03-06 16:27:54 -08:00
malloc_stats.h
Change RocksDB License
2017-07-15 16:11:23 -07:00
managed_iterator.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
managed_iterator.h
Change RocksDB License
2017-07-15 16:11:23 -07:00
manual_compaction_test.cc
Comment out unused variables
2018-03-05 13:13:41 -08:00
memtable_list_test.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
memtable_list.cc
Comment out unused variables
2018-03-05 13:13:41 -08:00
memtable_list.h
Fix Flush() keep waiting after flush finish
2018-01-18 17:45:16 -08:00
memtable.cc
InlineSkiplist: don't decode keys unnecessarily during comparisons
2018-03-23 12:14:30 -07:00
memtable.h
InlineSkiplist: don't decode keys unnecessarily during comparisons
2018-03-23 12:14:30 -07:00
merge_context.h
Change RocksDB License
2017-07-15 16:11:23 -07:00
merge_helper_test.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
merge_helper.cc
WritePrepared Txn: Support merge operator
2018-02-09 14:57:54 -08:00
merge_helper.h
WritePrepared Txn: Support merge operator
2018-02-09 14:57:54 -08:00
merge_operator.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
merge_test.cc
Comment out unused variables
2018-03-05 13:13:41 -08:00
obsolete_files_test.cc
Fix race condition causing double deletion of ssts
2018-03-28 10:29:59 -07:00
options_file_test.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
perf_context_test.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
pinned_iterators_manager.h
Change RocksDB License
2017-07-15 16:11:23 -07:00
plain_table_db_test.cc
Comment out unused variables
2018-03-05 13:13:41 -08:00
pre_release_callback.h
WritePrepared Txn: fix race condition on publishing seq
2018-03-22 14:43:36 -07:00
prefix_test.cc
Comment out unused variables
2018-03-05 13:13:41 -08:00
range_del_aggregator_test.cc
optimize file ingestion checks for range deletion overlap
2017-11-28 11:27:02 -08:00
range_del_aggregator.cc
Avoid adding tombstones of the same file to RangeDelAggregator multiple times
2018-03-23 12:43:06 -07:00
range_del_aggregator.h
Avoid adding tombstones of the same file to RangeDelAggregator multiple times
2018-03-23 12:43:06 -07:00
read_callback.h
write-prepared txn: call IsInSnapshot
2017-09-11 09:14:48 -07:00
repair_test.cc
fix file numbers after repair
2017-10-10 13:12:37 -07:00
repair.cc
WritePrepared Txn: PreReleaseCallback
2017-11-30 23:50:45 -08:00
snapshot_checker.h
Comment out unused variables
2018-03-05 13:13:41 -08:00
snapshot_impl.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
snapshot_impl.h
Disable options_settable_test in UBSAN and fix UBSAN failure in blob_…
2018-02-07 14:42:26 -08:00
table_cache.cc
Avoid adding tombstones of the same file to RangeDelAggregator multiple times
2018-03-23 12:43:06 -07:00
table_cache.h
Change RocksDB License
2017-07-15 16:11:23 -07:00
table_properties_collector_test.cc
Comment out unused variables
2018-03-05 13:13:41 -08:00
table_properties_collector.cc
Comment out unused variables
2018-03-05 13:13:41 -08:00
table_properties_collector.h
Comment out unused variables
2018-03-05 13:13:41 -08:00
transaction_log_impl.cc
Comment out unused variables
2018-03-05 13:13:41 -08:00
transaction_log_impl.h
WritePrepared Txn: Refactor conf params
2017-11-10 17:28:12 -08:00
version_builder_test.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
version_builder.cc
Fix some typos in comments and docs.
2018-03-08 10:27:25 -08:00
version_builder.h
Allow DB reopen with reduced options.num_levels
2017-08-24 16:10:54 -07:00
version_edit_test.cc
Skip deleted WALs during recovery
2018-03-30 11:28:05 -07:00
version_edit.cc
Skip deleted WALs during recovery
2018-03-30 11:28:05 -07:00
version_edit.h
Skip deleted WALs during recovery
2018-03-30 11:28:05 -07:00
version_set_test.cc
Comment out unused variables
2018-03-05 13:13:41 -08:00
version_set.cc
Skip deleted WALs during recovery
2018-03-30 11:28:05 -07:00
version_set.h
Skip deleted WALs during recovery
2018-03-30 11:28:05 -07:00
wal_manager_test.cc
Comment out unused variables
2018-03-05 13:13:41 -08:00
wal_manager.cc
Fix memleak when DB::DeleteFile()
2018-01-11 18:57:33 -08:00
wal_manager.h
Fix memleak when DB::DeleteFile()
2018-01-11 18:57:33 -08:00
write_batch_base.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
write_batch_internal.h
Fix some typos in comments and docs.
2018-03-08 10:27:25 -08:00
write_batch_test.cc
Comment out unused variables
2018-03-05 13:13:41 -08:00
write_batch.cc
Optionally create DuplicateDetector
2018-03-14 00:57:25 -07:00
write_callback_test.cc
WritePrepared Txn: fix race condition on publishing seq
2018-03-22 14:43:36 -07:00
write_callback.h
Change RocksDB License
2017-07-15 16:11:23 -07:00
write_controller_test.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
write_controller.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
write_controller.h
Change RocksDB License
2017-07-15 16:11:23 -07:00
write_thread.cc
Comment out unused variables
2018-03-05 13:13:41 -08:00
write_thread.h
WritePrepared Txn: Duplicate Keys, Txn Part
2018-02-05 18:43:24 -08:00