andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
92b1a68d87
rocksdb/db
History
Maysam Yabandeh 680864ae54 WritePrepared Txn: Fix bug with duplicate keys during recovery 
Summary:
Fix the following bugs:
- During recovery a duplicate key was inserted twice into the write batch of the recovery transaction,
once when the memtable returns false (because it was duplicates) and once for the 2nd attempt. This would result into different SubBatch count measured when the recovered transactions is committing.
- If a cf is flushed during recovery the memtable is not available to assist in detecting the duplicate key. This could result into not advancing the sequence number when iterating over duplicate keys of a flushed cf and hence inserting the next key with the wrong sequence number.
- SubBacthCounter would reset the comparator to default comparator after the first duplicate key. The 2nd duplicate key hence would have gone through a wrong comparator and not being detected.
Closes https://github.com/facebook/rocksdb/pull/3562

Differential Revision: D7149440

Pulled By: maysamyabandeh

fbshipit-source-id: 91ec317b165f363f5d11ff8b8c47c81cebb8ed77
2018-03-05 10:57:59 -08:00
..
builder.cc Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
builder.h Stream 2017-11-10 09:26:24 -08:00
c_test.c Added save points for transactions C API 2017-09-14 14:18:59 -07:00
c.cc Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
column_family_test.cc Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
column_family.cc Add "rocksdb.live-sst-files-size" DB property 2018-03-01 18:01:10 -08:00
column_family.h Add "rocksdb.live-sst-files-size" DB property 2018-03-01 18:01:10 -08:00
compact_files_test.cc Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
compacted_db_impl.cc Fix coverity issues column_family, compaction_db/iterator 2017-10-27 11:26:42 -07:00
compacted_db_impl.h Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
compaction_iteration_stats.h add counter for deletion dropping optimization 2017-08-19 14:10:08 -07:00
compaction_iterator_test.cc Blob DB: remove existing garbage collection implementation 2018-03-02 12:57:23 -08:00
compaction_iterator.cc Blob DB: remove existing garbage collection implementation 2018-03-02 12:57:23 -08:00
compaction_iterator.h Blob DB: remove existing garbage collection implementation 2018-03-02 12:57:23 -08:00
compaction_job_stats_test.cc Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
compaction_job_test.cc WritePrepared Txn: PreReleaseCallback 2017-11-30 23:50:45 -08:00
compaction_job.cc Blob DB: remove existing garbage collection implementation 2018-03-02 12:57:23 -08:00
compaction_job.h Make DBOption compaction_readahead_size dynamic 2017-11-16 17:57:25 -08:00
compaction_picker_test.cc Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
compaction_picker_universal.cc Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
compaction_picker_universal.h Make Universal compaction options dynamic 2017-12-11 13:27:06 -08:00
compaction_picker.cc Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
compaction_picker.h Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
compaction.cc Improve fallocate size in compaction output 2018-01-22 16:43:46 -08:00
compaction.h Change RocksDB License 2017-07-15 16:11:23 -07:00
comparator_db_test.cc Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
convenience.cc Delete files in multiple ranges at once 2018-01-30 13:56:39 -08:00
corruption_test.cc fix corruption_test valgrind 2017-08-11 12:29:14 -07:00
cuckoo_table_db_test.cc Enable MSVC W4 with a few exceptions. Fix warnings and bugs 2017-10-19 10:57:12 -07:00
db_basic_test.cc Fix the Logger::Close() and DBImpl::Close() design pattern 2018-02-23 13:57:26 -08:00
db_blob_index_test.cc fix lite build 2017-10-17 08:57:09 -07:00
db_block_cache_test.cc Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
db_bloom_filter_test.cc Revert "comment out unused parameters" 2017-07-21 18:26:26 -07:00
db_compaction_filter_test.cc Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
db_compaction_test.cc skip CompactRange flush based on memtable contents 2018-02-27 17:12:44 -08:00
db_dynamic_level_test.cc Revert "comment out unused parameters" 2017-07-21 18:26:26 -07:00
db_encryption_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_filesnapshot.cc log flush reason for better debugging experience 2018-02-09 12:12:43 -08:00
db_flush_test.cc Handle error return from WriteBuffer() 2018-02-05 13:59:34 -08:00
db_impl_compaction_flush.cc skip CompactRange flush based on memtable contents 2018-02-27 17:12:44 -08:00
db_impl_debug.cc Fix a leak in prepared_section_completed_ 2018-03-01 20:41:56 -08:00
db_impl_experimental.cc Inform caller when rocksdb is stalling writes 2017-10-05 18:11:43 -07:00
db_impl_files.cc Fix a leak in prepared_section_completed_ 2018-03-01 20:41:56 -08:00
db_impl_open.cc DB:Open should fail on tmpfs when use_direct_reads=true 2018-02-26 14:58:06 -08:00
db_impl_readonly.cc Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
db_impl_readonly.h Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
db_impl_write.cc Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
db_impl.cc skip CompactRange flush based on memtable contents 2018-02-27 17:12:44 -08:00
db_impl.h WritePrepared Txn: Fix bug with duplicate keys during recovery 2018-03-05 10:57:59 -08:00
db_info_dumper.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_info_dumper.h Change RocksDB License 2017-07-15 16:11:23 -07:00
db_inplace_update_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_io_failure_test.cc Fix LITE unit tests 2017-07-26 21:11:47 -07:00
db_iter_test.cc fix Seek with lower_bound 2017-11-29 22:56:29 -08:00
db_iter.cc Add rocksdb.iterator.internal-key property 2018-02-20 19:12:09 -08:00
db_iter.h WritePrepared Txn: Return NotSupported on iterator refresh 2017-12-18 22:29:30 -08:00
db_iterator_test.cc Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
db_log_iter_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_memtable_test.cc Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
db_merge_operator_test.cc Fix TSAN timeout in MergeOperatorPinningTest.Randomized/x test 2018-03-02 16:27:21 -08:00
db_options_test.cc fix live WALs purged while file deletions disabled 2018-01-17 17:42:04 -08:00
db_properties_test.cc Add "rocksdb.live-sst-files-size" DB property 2018-03-01 18:01:10 -08:00
db_range_del_test.cc Fix wrong smallest key of delete range tombstones 2017-08-29 18:41:35 -07:00
db_sst_test.cc Fix DeleteScheduler::MarkAsTrash() handling existing trash 2017-12-12 18:17:13 -08:00
db_statistics_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_table_properties_test.cc fix deletion-triggered compaction in table builder 2017-09-28 18:17:30 -07:00
db_tailing_iter_test.cc Revert "comment out unused parameters" 2017-07-21 18:26:26 -07:00
db_test2.cc Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
db_test_util.cc Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
db_test_util.h Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
db_test.cc Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
db_universal_compaction_test.cc Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
db_wal_test.cc WritePrepared Txn: PreReleaseCallback 2017-11-30 23:50:45 -08:00
db_write_test.cc BlobDB: Remove the need to get sequence number per write 2017-12-15 13:27:30 -08:00
dbformat_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
dbformat.cc WritePrepared Txn: Duplicate Keys, Memtable part 2018-01-31 18:57:07 -08:00
dbformat.h WritePrepared Txn: Duplicate Keys, Memtable part 2018-01-31 18:57:07 -08:00
deletefile_test.cc Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
event_helpers.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
event_helpers.h Change RocksDB License 2017-07-15 16:11:23 -07:00
experimental.cc Replace dynamic_cast<> 2017-07-28 16:27:16 -07:00
external_sst_file_basic_test.cc optimize file ingestion checks for range deletion overlap 2017-11-28 11:27:02 -08:00
external_sst_file_ingestion_job.cc skip CompactRange flush based on memtable contents 2018-02-27 17:12:44 -08:00
external_sst_file_ingestion_job.h skip CompactRange flush based on memtable contents 2018-02-27 17:12:44 -08:00
external_sst_file_test.cc Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
fault_injection_test.cc fix UBSAN errors in fault_injection_test 2018-02-13 14:12:40 -08:00
file_indexer_test.cc Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
file_indexer.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
file_indexer.h Change RocksDB License 2017-07-15 16:11:23 -07:00
filename_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
flush_job_test.cc Add a histogram stat for memtable flush 2017-12-15 18:57:00 -08:00
flush_job.cc log flush reason for better debugging experience 2018-02-09 12:12:43 -08:00
flush_job.h WritePrepared Txn: Compaction/Flush 2017-10-06 10:41:53 -07:00
flush_scheduler.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
flush_scheduler.h Change RocksDB License 2017-07-15 16:11:23 -07:00
forward_iterator_bench.cc fix gflags namespace 2017-12-01 10:42:05 -08:00
forward_iterator.cc Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
forward_iterator.h Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
internal_stats.cc Add "rocksdb.live-sst-files-size" DB property 2018-03-01 18:01:10 -08:00
internal_stats.h Add "rocksdb.live-sst-files-size" DB property 2018-03-01 18:01:10 -08:00
job_context.h Prevent unnecessary calls to PurgeObsoleteFiles 2018-01-12 13:27:08 -08:00
listener_test.cc Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
log_format.h Change RocksDB License 2017-07-15 16:11:23 -07:00
log_reader.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
log_reader.h Suppress lint in old files 2018-01-29 12:56:42 -08:00
log_test.cc Enable MSVC W4 with a few exceptions. Fix warnings and bugs 2017-10-19 10:57:12 -07:00
log_writer.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
log_writer.h Change RocksDB License 2017-07-15 16:11:23 -07:00
malloc_stats.cc Have a different function when ROCKSDB_JEMALLOC=0 2018-02-23 11:42:33 -08:00
malloc_stats.h Change RocksDB License 2017-07-15 16:11:23 -07:00
managed_iterator.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
managed_iterator.h Change RocksDB License 2017-07-15 16:11:23 -07:00
manual_compaction_test.cc Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
memtable_list_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
memtable_list.cc Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
memtable_list.h Fix Flush() keep waiting after flush finish 2018-01-18 17:45:16 -08:00
memtable.cc Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
memtable.h WritePrepared Txn: Duplicate Keys, Memtable part 2018-01-31 18:57:07 -08:00
merge_context.h Change RocksDB License 2017-07-15 16:11:23 -07:00
merge_helper_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
merge_helper.cc WritePrepared Txn: Support merge operator 2018-02-09 14:57:54 -08:00
merge_helper.h WritePrepared Txn: Support merge operator 2018-02-09 14:57:54 -08:00
merge_operator.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
merge_test.cc Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
options_file_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
perf_context_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
pinned_iterators_manager.h Change RocksDB License 2017-07-15 16:11:23 -07:00
plain_table_db_test.cc Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
pre_release_callback.h WritePrepared Txn: PreReleaseCallback 2017-11-30 23:50:45 -08:00
prefix_test.cc Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
range_del_aggregator_test.cc optimize file ingestion checks for range deletion overlap 2017-11-28 11:27:02 -08:00
range_del_aggregator.cc optimize file ingestion checks for range deletion overlap 2017-11-28 11:27:02 -08:00
range_del_aggregator.h optimize file ingestion checks for range deletion overlap 2017-11-28 11:27:02 -08:00
read_callback.h write-prepared txn: call IsInSnapshot 2017-09-11 09:14:48 -07:00
repair_test.cc fix file numbers after repair 2017-10-10 13:12:37 -07:00
repair.cc WritePrepared Txn: PreReleaseCallback 2017-11-30 23:50:45 -08:00
snapshot_checker.h Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
snapshot_impl.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
snapshot_impl.h Disable options_settable_test in UBSAN and fix UBSAN failure in blob_… 2018-02-07 14:42:26 -08:00
table_cache.cc Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
table_cache.h Change RocksDB License 2017-07-15 16:11:23 -07:00
table_properties_collector_test.cc Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
table_properties_collector.cc Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
table_properties_collector.h Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
transaction_log_impl.cc Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
transaction_log_impl.h WritePrepared Txn: Refactor conf params 2017-11-10 17:28:12 -08:00
version_builder_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
version_builder.cc Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
version_builder.h Allow DB reopen with reduced options.num_levels 2017-08-24 16:10:54 -07:00
version_edit_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
version_edit.cc Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
version_edit.h Fix coverity issues version, write_batch 2017-12-07 11:57:36 -08:00
version_set_test.cc Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
version_set.cc Add "rocksdb.live-sst-files-size" DB property 2018-03-01 18:01:10 -08:00
version_set.h Add "rocksdb.live-sst-files-size" DB property 2018-03-01 18:01:10 -08:00
wal_manager_test.cc Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
wal_manager.cc Fix memleak when DB::DeleteFile() 2018-01-11 18:57:33 -08:00
wal_manager.h Fix memleak when DB::DeleteFile() 2018-01-11 18:57:33 -08:00
write_batch_base.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
write_batch_internal.h Explictly fail writes if key or value is not smaller than 4GB 2018-02-09 14:57:54 -08:00
write_batch_test.cc Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
write_batch.cc WritePrepared Txn: Fix bug with duplicate keys during recovery 2018-03-05 10:57:59 -08:00
write_callback_test.cc Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
write_callback.h Change RocksDB License 2017-07-15 16:11:23 -07:00
write_controller_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
write_controller.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
write_controller.h Change RocksDB License 2017-07-15 16:11:23 -07:00
write_thread.cc Back out "[codemod] - comment out unused parameters" 2018-02-22 12:43:17 -08:00
write_thread.h WritePrepared Txn: Duplicate Keys, Txn Part 2018-02-05 18:43:24 -08:00