andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
rocksdb/db
History
Maysam Yabandeh 680864ae54 WritePrepared Txn: Fix bug with duplicate keys during recovery 
Summary:
Fix the following bugs:
- During recovery a duplicate key was inserted twice into the write batch of the recovery transaction,
once when the memtable returns false (because it was duplicates) and once for the 2nd attempt. This would result into different SubBatch count measured when the recovered transactions is committing.
- If a cf is flushed during recovery the memtable is not available to assist in detecting the duplicate key. This could result into not advancing the sequence number when iterating over duplicate keys of a flushed cf and hence inserting the next key with the wrong sequence number.
- SubBacthCounter would reset the comparator to default comparator after the first duplicate key. The 2nd duplicate key hence would have gone through a wrong comparator and not being detected.
Closes https://github.com/facebook/rocksdb/pull/3562

Differential Revision: D7149440

Pulled By: maysamyabandeh

fbshipit-source-id: 91ec317b165f363f5d11ff8b8c47c81cebb8ed77
2018-03-05 10:57:59 -08:00
..
builder.cc
Back out "[codemod] - comment out unused parameters"
2018-02-22 12:43:17 -08:00
builder.h
Stream
2017-11-10 09:26:24 -08:00
c_test.c
Added save points for transactions C API
2017-09-14 14:18:59 -07:00
c.cc
Back out "[codemod] - comment out unused parameters"
2018-02-22 12:43:17 -08:00
column_family_test.cc
Back out "[codemod] - comment out unused parameters"
2018-02-22 12:43:17 -08:00
column_family.cc
Add "rocksdb.live-sst-files-size" DB property
2018-03-01 18:01:10 -08:00
column_family.h
Add "rocksdb.live-sst-files-size" DB property
2018-03-01 18:01:10 -08:00
compact_files_test.cc
Back out "[codemod] - comment out unused parameters"
2018-02-22 12:43:17 -08:00
compacted_db_impl.cc
Fix coverity issues column_family, compaction_db/iterator
2017-10-27 11:26:42 -07:00
compacted_db_impl.h
Back out "[codemod] - comment out unused parameters"
2018-02-22 12:43:17 -08:00
compaction_iteration_stats.h
add counter for deletion dropping optimization
2017-08-19 14:10:08 -07:00
compaction_iterator_test.cc
Blob DB: remove existing garbage collection implementation
2018-03-02 12:57:23 -08:00
compaction_iterator.cc
Blob DB: remove existing garbage collection implementation
2018-03-02 12:57:23 -08:00
compaction_iterator.h
Blob DB: remove existing garbage collection implementation
2018-03-02 12:57:23 -08:00
compaction_job_stats_test.cc
Back out "[codemod] - comment out unused parameters"
2018-02-22 12:43:17 -08:00
compaction_job_test.cc
WritePrepared Txn: PreReleaseCallback
2017-11-30 23:50:45 -08:00
compaction_job.cc
Blob DB: remove existing garbage collection implementation
2018-03-02 12:57:23 -08:00
compaction_job.h
Make DBOption compaction_readahead_size dynamic
2017-11-16 17:57:25 -08:00
compaction_picker_test.cc
Back out "[codemod] - comment out unused parameters"
2018-02-22 12:43:17 -08:00
compaction_picker_universal.cc
Back out "[codemod] - comment out unused parameters"
2018-02-22 12:43:17 -08:00
compaction_picker_universal.h
Make Universal compaction options dynamic
2017-12-11 13:27:06 -08:00
compaction_picker.cc
Back out "[codemod] - comment out unused parameters"
2018-02-22 12:43:17 -08:00
compaction_picker.h
Back out "[codemod] - comment out unused parameters"
2018-02-22 12:43:17 -08:00
compaction.cc
Improve fallocate size in compaction output
2018-01-22 16:43:46 -08:00
compaction.h
Change RocksDB License
2017-07-15 16:11:23 -07:00
comparator_db_test.cc
Back out "[codemod] - comment out unused parameters"
2018-02-22 12:43:17 -08:00
convenience.cc
Delete files in multiple ranges at once
2018-01-30 13:56:39 -08:00
corruption_test.cc
fix corruption_test valgrind
2017-08-11 12:29:14 -07:00
cuckoo_table_db_test.cc
Enable MSVC W4 with a few exceptions. Fix warnings and bugs
2017-10-19 10:57:12 -07:00
db_basic_test.cc
Fix the Logger::Close() and DBImpl::Close() design pattern
2018-02-23 13:57:26 -08:00
db_blob_index_test.cc
fix lite build
2017-10-17 08:57:09 -07:00
db_block_cache_test.cc
Back out "[codemod] - comment out unused parameters"
2018-02-22 12:43:17 -08:00
db_bloom_filter_test.cc
Revert "comment out unused parameters"
2017-07-21 18:26:26 -07:00
db_compaction_filter_test.cc
Back out "[codemod] - comment out unused parameters"
2018-02-22 12:43:17 -08:00
db_compaction_test.cc
skip CompactRange flush based on memtable contents
2018-02-27 17:12:44 -08:00
db_dynamic_level_test.cc
Revert "comment out unused parameters"
2017-07-21 18:26:26 -07:00
db_encryption_test.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
db_filesnapshot.cc
log flush reason for better debugging experience
2018-02-09 12:12:43 -08:00
db_flush_test.cc
Handle error return from WriteBuffer()
2018-02-05 13:59:34 -08:00
db_impl_compaction_flush.cc
skip CompactRange flush based on memtable contents
2018-02-27 17:12:44 -08:00
db_impl_debug.cc
Fix a leak in prepared_section_completed_
2018-03-01 20:41:56 -08:00
db_impl_experimental.cc
Inform caller when rocksdb is stalling writes
2017-10-05 18:11:43 -07:00
db_impl_files.cc
Fix a leak in prepared_section_completed_
2018-03-01 20:41:56 -08:00
db_impl_open.cc
DB:Open should fail on tmpfs when use_direct_reads=true
2018-02-26 14:58:06 -08:00
db_impl_readonly.cc
Back out "[codemod] - comment out unused parameters"
2018-02-22 12:43:17 -08:00
db_impl_readonly.h
Back out "[codemod] - comment out unused parameters"
2018-02-22 12:43:17 -08:00
db_impl_write.cc
Back out "[codemod] - comment out unused parameters"
2018-02-22 12:43:17 -08:00
db_impl.cc
skip CompactRange flush based on memtable contents
2018-02-27 17:12:44 -08:00
db_impl.h
WritePrepared Txn: Fix bug with duplicate keys during recovery
2018-03-05 10:57:59 -08:00
db_info_dumper.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
db_info_dumper.h
Change RocksDB License
2017-07-15 16:11:23 -07:00
db_inplace_update_test.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
db_io_failure_test.cc
Fix LITE unit tests
2017-07-26 21:11:47 -07:00
db_iter_test.cc
fix Seek with lower_bound
2017-11-29 22:56:29 -08:00
db_iter.cc
Add rocksdb.iterator.internal-key property
2018-02-20 19:12:09 -08:00
db_iter.h
WritePrepared Txn: Return NotSupported on iterator refresh
2017-12-18 22:29:30 -08:00
db_iterator_test.cc
Back out "[codemod] - comment out unused parameters"
2018-02-22 12:43:17 -08:00
db_log_iter_test.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
db_memtable_test.cc
Back out "[codemod] - comment out unused parameters"
2018-02-22 12:43:17 -08:00
db_merge_operator_test.cc
Fix TSAN timeout in MergeOperatorPinningTest.Randomized/x test
2018-03-02 16:27:21 -08:00
db_options_test.cc
fix live WALs purged while file deletions disabled
2018-01-17 17:42:04 -08:00
db_properties_test.cc
Add "rocksdb.live-sst-files-size" DB property
2018-03-01 18:01:10 -08:00
db_range_del_test.cc
Fix wrong smallest key of delete range tombstones
2017-08-29 18:41:35 -07:00
db_sst_test.cc
Fix DeleteScheduler::MarkAsTrash() handling existing trash
2017-12-12 18:17:13 -08:00
db_statistics_test.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
db_table_properties_test.cc
fix deletion-triggered compaction in table builder
2017-09-28 18:17:30 -07:00
db_tailing_iter_test.cc
Revert "comment out unused parameters"
2017-07-21 18:26:26 -07:00
db_test2.cc
Back out "[codemod] - comment out unused parameters"
2018-02-22 12:43:17 -08:00
db_test_util.cc
Back out "[codemod] - comment out unused parameters"
2018-02-22 12:43:17 -08:00
db_test_util.h
Back out "[codemod] - comment out unused parameters"
2018-02-22 12:43:17 -08:00
db_test.cc
Back out "[codemod] - comment out unused parameters"
2018-02-22 12:43:17 -08:00
db_universal_compaction_test.cc
Back out "[codemod] - comment out unused parameters"
2018-02-22 12:43:17 -08:00
db_wal_test.cc
WritePrepared Txn: PreReleaseCallback
2017-11-30 23:50:45 -08:00
db_write_test.cc
BlobDB: Remove the need to get sequence number per write
2017-12-15 13:27:30 -08:00
dbformat_test.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
dbformat.cc
WritePrepared Txn: Duplicate Keys, Memtable part
2018-01-31 18:57:07 -08:00
dbformat.h
WritePrepared Txn: Duplicate Keys, Memtable part
2018-01-31 18:57:07 -08:00
deletefile_test.cc
Back out "[codemod] - comment out unused parameters"
2018-02-22 12:43:17 -08:00
event_helpers.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
event_helpers.h
Change RocksDB License
2017-07-15 16:11:23 -07:00
experimental.cc
Replace dynamic_cast<>
2017-07-28 16:27:16 -07:00
external_sst_file_basic_test.cc
optimize file ingestion checks for range deletion overlap
2017-11-28 11:27:02 -08:00
external_sst_file_ingestion_job.cc
skip CompactRange flush based on memtable contents
2018-02-27 17:12:44 -08:00
external_sst_file_ingestion_job.h
skip CompactRange flush based on memtable contents
2018-02-27 17:12:44 -08:00
external_sst_file_test.cc
Back out "[codemod] - comment out unused parameters"
2018-02-22 12:43:17 -08:00
fault_injection_test.cc
fix UBSAN errors in fault_injection_test
2018-02-13 14:12:40 -08:00
file_indexer_test.cc
Back out "[codemod] - comment out unused parameters"
2018-02-22 12:43:17 -08:00
file_indexer.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
file_indexer.h
Change RocksDB License
2017-07-15 16:11:23 -07:00
filename_test.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
flush_job_test.cc
Add a histogram stat for memtable flush
2017-12-15 18:57:00 -08:00
flush_job.cc
log flush reason for better debugging experience
2018-02-09 12:12:43 -08:00
flush_job.h
WritePrepared Txn: Compaction/Flush
2017-10-06 10:41:53 -07:00
flush_scheduler.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
flush_scheduler.h
Change RocksDB License
2017-07-15 16:11:23 -07:00
forward_iterator_bench.cc
fix gflags namespace
2017-12-01 10:42:05 -08:00
forward_iterator.cc
Back out "[codemod] - comment out unused parameters"
2018-02-22 12:43:17 -08:00
forward_iterator.h
Back out "[codemod] - comment out unused parameters"
2018-02-22 12:43:17 -08:00
internal_stats.cc
Add "rocksdb.live-sst-files-size" DB property
2018-03-01 18:01:10 -08:00
internal_stats.h
Add "rocksdb.live-sst-files-size" DB property
2018-03-01 18:01:10 -08:00
job_context.h
Prevent unnecessary calls to PurgeObsoleteFiles
2018-01-12 13:27:08 -08:00
listener_test.cc
Back out "[codemod] - comment out unused parameters"
2018-02-22 12:43:17 -08:00
log_format.h
Change RocksDB License
2017-07-15 16:11:23 -07:00
log_reader.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
log_reader.h
Suppress lint in old files
2018-01-29 12:56:42 -08:00
log_test.cc
Enable MSVC W4 with a few exceptions. Fix warnings and bugs
2017-10-19 10:57:12 -07:00
log_writer.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
log_writer.h
Change RocksDB License
2017-07-15 16:11:23 -07:00
malloc_stats.cc
Have a different function when ROCKSDB_JEMALLOC=0
2018-02-23 11:42:33 -08:00
malloc_stats.h
Change RocksDB License
2017-07-15 16:11:23 -07:00
managed_iterator.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
managed_iterator.h
Change RocksDB License
2017-07-15 16:11:23 -07:00
manual_compaction_test.cc
Back out "[codemod] - comment out unused parameters"
2018-02-22 12:43:17 -08:00
memtable_list_test.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
memtable_list.cc
Back out "[codemod] - comment out unused parameters"
2018-02-22 12:43:17 -08:00
memtable_list.h
Fix Flush() keep waiting after flush finish
2018-01-18 17:45:16 -08:00
memtable.cc
Back out "[codemod] - comment out unused parameters"
2018-02-22 12:43:17 -08:00
memtable.h
WritePrepared Txn: Duplicate Keys, Memtable part
2018-01-31 18:57:07 -08:00
merge_context.h
Change RocksDB License
2017-07-15 16:11:23 -07:00
merge_helper_test.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
merge_helper.cc
WritePrepared Txn: Support merge operator
2018-02-09 14:57:54 -08:00
merge_helper.h
WritePrepared Txn: Support merge operator
2018-02-09 14:57:54 -08:00
merge_operator.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
merge_test.cc
Back out "[codemod] - comment out unused parameters"
2018-02-22 12:43:17 -08:00
options_file_test.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
perf_context_test.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
pinned_iterators_manager.h
Change RocksDB License
2017-07-15 16:11:23 -07:00
plain_table_db_test.cc
Back out "[codemod] - comment out unused parameters"
2018-02-22 12:43:17 -08:00
pre_release_callback.h
WritePrepared Txn: PreReleaseCallback
2017-11-30 23:50:45 -08:00
prefix_test.cc
Back out "[codemod] - comment out unused parameters"
2018-02-22 12:43:17 -08:00
range_del_aggregator_test.cc
optimize file ingestion checks for range deletion overlap
2017-11-28 11:27:02 -08:00
range_del_aggregator.cc
optimize file ingestion checks for range deletion overlap
2017-11-28 11:27:02 -08:00
range_del_aggregator.h
optimize file ingestion checks for range deletion overlap
2017-11-28 11:27:02 -08:00
read_callback.h
write-prepared txn: call IsInSnapshot
2017-09-11 09:14:48 -07:00
repair_test.cc
fix file numbers after repair
2017-10-10 13:12:37 -07:00
repair.cc
WritePrepared Txn: PreReleaseCallback
2017-11-30 23:50:45 -08:00
snapshot_checker.h
Back out "[codemod] - comment out unused parameters"
2018-02-22 12:43:17 -08:00
snapshot_impl.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
snapshot_impl.h
Disable options_settable_test in UBSAN and fix UBSAN failure in blob_…
2018-02-07 14:42:26 -08:00
table_cache.cc
Back out "[codemod] - comment out unused parameters"
2018-02-22 12:43:17 -08:00
table_cache.h
Change RocksDB License
2017-07-15 16:11:23 -07:00
table_properties_collector_test.cc
Back out "[codemod] - comment out unused parameters"
2018-02-22 12:43:17 -08:00
table_properties_collector.cc
Back out "[codemod] - comment out unused parameters"
2018-02-22 12:43:17 -08:00
table_properties_collector.h
Back out "[codemod] - comment out unused parameters"
2018-02-22 12:43:17 -08:00
transaction_log_impl.cc
Back out "[codemod] - comment out unused parameters"
2018-02-22 12:43:17 -08:00
transaction_log_impl.h
WritePrepared Txn: Refactor conf params
2017-11-10 17:28:12 -08:00
version_builder_test.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
version_builder.cc
Back out "[codemod] - comment out unused parameters"
2018-02-22 12:43:17 -08:00
version_builder.h
Allow DB reopen with reduced options.num_levels
2017-08-24 16:10:54 -07:00
version_edit_test.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
version_edit.cc
Back out "[codemod] - comment out unused parameters"
2018-02-22 12:43:17 -08:00
version_edit.h
Fix coverity issues version, write_batch
2017-12-07 11:57:36 -08:00
version_set_test.cc
Back out "[codemod] - comment out unused parameters"
2018-02-22 12:43:17 -08:00
version_set.cc
Add "rocksdb.live-sst-files-size" DB property
2018-03-01 18:01:10 -08:00
version_set.h
Add "rocksdb.live-sst-files-size" DB property
2018-03-01 18:01:10 -08:00
wal_manager_test.cc
Back out "[codemod] - comment out unused parameters"
2018-02-22 12:43:17 -08:00
wal_manager.cc
Fix memleak when DB::DeleteFile()
2018-01-11 18:57:33 -08:00
wal_manager.h
Fix memleak when DB::DeleteFile()
2018-01-11 18:57:33 -08:00
write_batch_base.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
write_batch_internal.h
Explictly fail writes if key or value is not smaller than 4GB
2018-02-09 14:57:54 -08:00
write_batch_test.cc
Back out "[codemod] - comment out unused parameters"
2018-02-22 12:43:17 -08:00
write_batch.cc
WritePrepared Txn: Fix bug with duplicate keys during recovery
2018-03-05 10:57:59 -08:00
write_callback_test.cc
Back out "[codemod] - comment out unused parameters"
2018-02-22 12:43:17 -08:00
write_callback.h
Change RocksDB License
2017-07-15 16:11:23 -07:00
write_controller_test.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
write_controller.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
write_controller.h
Change RocksDB License
2017-07-15 16:11:23 -07:00
write_thread.cc
Back out "[codemod] - comment out unused parameters"
2018-02-22 12:43:17 -08:00
write_thread.h
WritePrepared Txn: Duplicate Keys, Txn Part
2018-02-05 18:43:24 -08:00