andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
6fbe42a6fd
rocksdb/db
History
Andrew Kryczka 6fbe42a6fd Fix seqno in ingested file boundary key metadata (#8209) 
Summary:
Fixes https://github.com/facebook/rocksdb/issues/6245.

Adapted from https://github.com/facebook/rocksdb/issues/8201 and https://github.com/facebook/rocksdb/issues/8205.

Previously we were writing the ingested file's smallest/largest internal keys
with sequence number zero, or `kMaxSequenceNumber` in case of range
tombstone. The former (sequence number zero) is incorrect and can lead
to files being incorrectly ordered. The fix in this PR is to overwrite
boundary keys that have sequence number zero with the ingested file's assigned
sequence number.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/8209

Test Plan: repro unit test

Reviewed By: riversand963

Differential Revision: D27885678

Pulled By: ajkr

fbshipit-source-id: 4a9f2c6efdfff81c3a9923e915ea88b250ee7b6a
2021-04-23 15:37:41 -07:00
..
blob Remove unused variable (#8067) 2021-03-19 12:13:59 -07:00
compaction Use SST file manager to track blob files as well (#8037) 2021-03-17 20:44:49 -07:00
db_impl Handle rename() failure in non-local FS (#8192) 2021-04-19 19:56:59 -07:00
arena_wrapped_db_iter.cc Add blob support to DBIter (#7731) 2020-12-04 21:29:38 -08:00
arena_wrapped_db_iter.h Add blob support to DBIter (#7731) 2020-12-04 21:29:38 -08:00
builder.cc Use SST file manager to track blob files as well (#8037) 2021-03-17 20:44:49 -07:00
builder.h Use SST file manager to track blob files as well (#8037) 2021-03-17 20:44:49 -07:00
c_test.c make:Fix c header prototypes (#7994) 2021-03-09 20:44:23 -08:00
c.cc Include C++ standard library headers instead of C compatibility headers (#8068) 2021-03-19 12:09:47 -07:00
column_family_test.cc Using emplace_back replace push_back (#7568) 2021-01-15 16:56:41 -08:00
column_family.cc Use SystemClock* instead of std::shared_ptr<SystemClock> in lower level routines (#8033) 2021-03-15 04:34:11 -07:00
column_family.h Fix checkpoint stuck (#7921) 2021-03-09 02:21:25 -08:00
compact_files_test.cc Add more tests for assert status checked (#7524) 2020-12-22 23:45:58 -08:00
compacted_db_impl.cc Periodically flush info log out of application buffer (#7488) 2020-10-01 19:14:14 -07:00
compacted_db_impl.h SyncWAL shouldn't be supported in compacted db (#7788) 2020-12-22 14:53:43 -08:00
comparator_db_test.cc Remove unused includes (#7604) 2020-10-28 23:22:27 -07:00
convenience.cc sst_dump to reduce number of file reads (#6836) 2020-05-12 18:23:33 -07:00
corruption_test.cc Extend VerifyFileChecksums API for blob files (#7979) 2021-02-22 22:09:22 -08:00
cuckoo_table_db_test.cc Revert "Turn on memtable bloom filter by default. (#6584)" (#7939) 2021-02-06 22:34:30 -08:00
db_basic_test.cc Create a CustomEnv class; Add WinFileSystem; Make LegacyFileSystemWrapper private (#7703) 2021-01-06 10:49:32 -08:00
db_block_cache_test.cc Add further tests to ASSERT_STATUS_CHECKED (2) (#7698) 2020-12-09 21:21:16 -08:00
db_bloom_filter_test.cc Fix a bug for SeekForPrev with partitioned filter and prefix (#8137) 2021-04-08 11:21:00 -07:00
db_compaction_filter_test.cc Add more tests to ASSERT_STATUS_CHECKED (3), API change (#7715) 2021-01-06 14:15:02 -08:00
db_compaction_test.cc Use SST file manager to track blob files as well (#8037) 2021-03-17 20:44:49 -07:00
db_dynamic_level_test.cc Add a SystemClock class to capture the time functions of an Env (#7858) 2021-01-25 22:09:11 -08:00
db_encryption_test.cc Improvements to Env::GetChildren (#7819) 2021-01-09 09:44:34 -08:00
db_filesnapshot.cc Fix checkpoint file deletion race with avoid_unnecessary_blocking_io (#7369) 2020-09-10 22:35:25 -07:00
db_flush_test.cc Separate handling of WAL Sync io error with SST flush io error (#8049) 2021-03-18 14:33:16 -07:00
db_info_dumper.cc Fix write-ahead log file size overflow (#7870) 2021-01-19 13:47:48 -08:00
db_info_dumper.h Add a DB Session ID (#6959) 2020-06-15 10:47:02 -07:00
db_inplace_update_test.cc Whole DBTest to skip fsync (#7274) 2020-08-17 18:42:25 -07:00
db_io_failure_test.cc Add more tests to ASSERT_STATUS_CHECKED (3), API change (#7715) 2021-01-06 14:15:02 -08:00
db_iter_stress_test.cc Add blob support to DBIter (#7731) 2020-12-04 21:29:38 -08:00
db_iter_test.cc Add more tests for assert status checked (#7524) 2020-12-22 23:45:58 -08:00
db_iter.cc Use SystemClock* instead of std::shared_ptr<SystemClock> in lower level routines (#8033) 2021-03-15 04:34:11 -07:00
db_iter.h Include C++ standard library headers instead of C compatibility headers (#8068) 2021-03-19 12:09:47 -07:00
db_iterator_test.cc Instantiate tests DBIteratorTestForPinnedData (#8051) 2021-03-12 12:31:29 -08:00
db_kv_checksum_test.cc Integrity protection for live updates to WriteBatch (#7748) 2021-01-29 12:18:58 -08:00
db_log_iter_test.cc Add a SystemClock class to capture the time functions of an Env (#7858) 2021-01-25 22:09:11 -08:00
db_logical_block_size_cache_test.cc Add further tests to ASSERT_STATUS_CHECKED (2) (#7698) 2020-12-09 21:21:16 -08:00
db_memtable_test.cc Integrity protection for live updates to WriteBatch (#7748) 2021-01-29 12:18:58 -08:00
db_merge_operand_test.cc Add further tests to ASSERT_STATUS_CHECKED (1) (#7679) 2020-12-08 15:55:04 -08:00
db_merge_operator_test.cc Add further tests to ASSERT_STATUS_CHECKED (1) (#7679) 2020-12-08 15:55:04 -08:00
db_options_test.cc Fix handling of Mutable options; Allow DB::SetOptions to update mutable TableFactory Options (#7936) 2021-02-19 10:29:02 -08:00
db_properties_test.cc Fix many tests to run with MEM_ENV and ENCRYPTED_ENV; Introduce a MemoryFileSystem class (#7566) 2020-10-27 10:33:09 -07:00
db_range_del_test.cc Fix compression dictionary sampling with dedicated range tombstone SSTs (#8141) 2021-04-01 10:00:27 -07:00
db_secondary_test.cc Move a test file to a better location (#8054) 2021-03-15 15:03:27 -07:00
db_sst_test.cc Use SST file manager to track blob files as well (#8037) 2021-03-17 20:44:49 -07:00
db_statistics_test.cc Add more tests for assert status checked (#7524) 2020-12-22 23:45:58 -08:00
db_table_properties_test.cc Add more tests for assert status checked (#7524) 2020-12-22 23:45:58 -08:00
db_tailing_iter_test.cc Add more tests to ASSERT_STATUS_CHECKED (3), API change (#7715) 2021-01-06 14:15:02 -08:00
db_test2.cc Handle rename() failure in non-local FS (#8192) 2021-04-19 19:56:59 -07:00
db_test_util.cc Handle rename() failure in non-local FS (#8192) 2021-04-19 19:56:59 -07:00
db_test_util.h Handle rename() failure in non-local FS (#8192) 2021-04-19 19:56:59 -07:00
db_test.cc Revamp WriteController (#8064) 2021-03-18 09:47:31 -07:00
db_universal_compaction_test.cc Revert "Turn on memtable bloom filter by default. (#6584)" (#7939) 2021-02-06 22:34:30 -08:00
db_wal_test.cc Break down the amount of data written during flushes/compactions per file type (#8013) 2021-03-02 09:48:00 -08:00
db_with_timestamp_basic_test.cc Fix a bug in key comparison when index type is kBinarySearchWithFirstKey (#8062) 2021-03-15 17:44:52 -07:00
db_with_timestamp_compaction_test.cc Whole DBTest to skip fsync (#7274) 2020-08-17 18:42:25 -07:00
db_write_test.cc Add more tests to ASSERT_STATUS_CHECKED (4) (#7718) 2020-12-22 15:09:39 -08:00
dbformat_test.cc Remove unused includes (#7604) 2020-10-28 23:22:27 -07:00
dbformat.cc Make CompactRange and GetApproximateSizes work with timestamp (#7684) 2020-12-02 13:00:53 -08:00
dbformat.h Enable backward iterator for keys with user-defined timestamp (#8035) 2021-03-10 11:15:46 -08:00
deletefile_test.cc Add more tests to ASSERT_STATUS_CHECKED (4) (#7718) 2020-12-22 15:09:39 -08:00
error_handler_fs_test.cc Remove disabled tests (#8123) 2021-03-28 23:06:08 -07:00
error_handler.cc Add the statistics and info log for Error handler (#8050) 2021-03-17 22:38:13 -07:00
error_handler.h Add the statistics and info log for Error handler (#8050) 2021-03-17 22:38:13 -07:00
event_helpers.cc Add more tests to ASSERT_STATUS_CHECKED (3), API change (#7715) 2021-01-06 14:15:02 -08:00
event_helpers.h Pass SST file checksum information through OnTableFileCreated (#7108) 2020-08-25 10:46:11 -07:00
experimental.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
external_sst_file_basic_test.cc Fix seqno in ingested file boundary key metadata (#8209) 2021-04-23 15:37:41 -07:00
external_sst_file_ingestion_job.cc Fix seqno in ingested file boundary key metadata (#8209) 2021-04-23 15:37:41 -07:00
external_sst_file_ingestion_job.h Use SystemClock* instead of std::shared_ptr<SystemClock> in lower level routines (#8033) 2021-03-15 04:34:11 -07:00
external_sst_file_test.cc Add more tests to ASSERT_STATUS_CHECKED (4) (#7718) 2020-12-22 15:09:39 -08:00
fault_injection_test.cc Add more tests for assert status checked (#7524) 2020-12-22 23:45:58 -08:00
file_indexer_test.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
file_indexer.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
file_indexer.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
filename_test.cc Remove unused includes (#7604) 2020-10-28 23:22:27 -07:00
flush_job_test.cc Fix testcase failures on windows (#7992) 2021-02-23 14:35:06 -08:00
flush_job.cc Use SST file manager to track blob files as well (#8037) 2021-03-17 20:44:49 -07:00
flush_job.h Use SST file manager to track blob files as well (#8037) 2021-03-17 20:44:49 -07:00
flush_scheduler.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
flush_scheduler.h Include C++ standard library headers instead of C compatibility headers (#8068) 2021-03-19 12:09:47 -07:00
forward_iterator_bench.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
forward_iterator.cc Fix compile error (#7908) 2021-03-09 20:53:33 -08:00
forward_iterator.h Properly report IO errors when IndexType::kBinarySearchWithFirstKey is used (#6621) 2020-04-15 17:40:44 -07:00
import_column_family_job.cc Add a SystemClock class to capture the time functions of an Env (#7858) 2021-01-25 22:09:11 -08:00
import_column_family_job.h Use SystemClock* instead of std::shared_ptr<SystemClock> in lower level routines (#8033) 2021-03-15 04:34:11 -07:00
import_column_family_test.cc No elide constructors (#7798) 2020-12-23 16:55:53 -08:00
internal_stats.cc Update compaction statistics to include the amount of data read from blob files (#8022) 2021-03-04 00:43:48 -08:00
internal_stats.h Use SystemClock* instead of std::shared_ptr<SystemClock> in lower level routines (#8033) 2021-03-15 04:34:11 -07:00
job_context.h Expose the set of live blob files from Version/VersionSet (#6785) 2020-05-04 15:08:13 -07:00
kv_checksum.h Integrity protection for live updates to WriteBatch (#7748) 2021-01-29 12:18:58 -08:00
listener_test.cc Add more tests for assert status checked (#7524) 2020-12-22 23:45:58 -08:00
log_format.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
log_reader.cc Fix kPointInTimeRecovery handling of truncated WAL (#7701) 2020-11-30 18:11:38 -08:00
log_reader.h Real fix for race in backup custom checksum checking (#7309) 2020-08-26 10:39:20 -07:00
log_test.cc Make StringEnv, StringSink, StringSource use FS classes (#7786) 2021-01-04 16:01:01 -08:00
log_writer.cc No elide constructors (#7798) 2020-12-23 16:55:53 -08:00
log_writer.h Include C++ standard library headers instead of C compatibility headers (#8068) 2021-03-19 12:09:47 -07:00
logs_with_prep_tracker.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
logs_with_prep_tracker.h Include C++ standard library headers instead of C compatibility headers (#8068) 2021-03-19 12:09:47 -07:00
lookup_key.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
malloc_stats.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
malloc_stats.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
manual_compaction_test.cc Add more tests for assert status checked (#7524) 2020-12-22 23:45:58 -08:00
memtable_list_test.cc Revert "Turn on memtable bloom filter by default. (#6584)" (#7939) 2021-02-06 22:34:30 -08:00
memtable_list.cc Add initial blob support to batched MultiGet (#7766) 2020-12-14 13:48:22 -08:00
memtable_list.h Add initial blob support to batched MultiGet (#7766) 2020-12-14 13:48:22 -08:00
memtable.cc Use SystemClock* instead of std::shared_ptr<SystemClock> in lower level routines (#8033) 2021-03-15 04:34:11 -07:00
memtable.h Use SystemClock* instead of std::shared_ptr<SystemClock> in lower level routines (#8033) 2021-03-15 04:34:11 -07:00
merge_context.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
merge_helper_test.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
merge_helper.cc Use SystemClock* instead of std::shared_ptr<SystemClock> in lower level routines (#8033) 2021-03-15 04:34:11 -07:00
merge_helper.h Use SystemClock* instead of std::shared_ptr<SystemClock> in lower level routines (#8033) 2021-03-15 04:34:11 -07:00
merge_operator.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
merge_test.cc MergeHelper::FilterMerge() calling ElapsedNanosSafe() upon exit even … (#7867) 2021-01-21 13:13:02 -08:00
obsolete_files_test.cc Fix a harmless data race affecting two test cases (#8055) 2021-03-12 16:44:35 -08:00
options_file_test.cc No elide constructors (#7798) 2020-12-23 16:55:53 -08:00
output_validator.cc Use NPHash64 in more places (#7632) 2020-11-10 23:42:13 -08:00
output_validator.h Use NPHash64 in more places (#7632) 2020-11-10 23:42:13 -08:00
perf_context_test.cc Use SystemClock* instead of std::shared_ptr<SystemClock> in lower level routines (#8033) 2021-03-15 04:34:11 -07:00
periodic_work_scheduler_test.cc Add a SystemClock class to capture the time functions of an Env (#7858) 2021-01-25 22:09:11 -08:00
periodic_work_scheduler.cc Use SystemClock* instead of std::shared_ptr<SystemClock> in lower level routines (#8033) 2021-03-15 04:34:11 -07:00
periodic_work_scheduler.h Add a SystemClock class to capture the time functions of an Env (#7858) 2021-01-25 22:09:11 -08:00
pinned_iterators_manager.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
plain_table_db_test.cc Make StringEnv, StringSink, StringSource use FS classes (#7786) 2021-01-04 16:01:01 -08:00
pre_release_callback.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
prefix_test.cc Use SystemClock* instead of std::shared_ptr<SystemClock> in lower level routines (#8033) 2021-03-15 04:34:11 -07:00
range_del_aggregator_bench.cc Use SystemClock* instead of std::shared_ptr<SystemClock> in lower level routines (#8033) 2021-03-15 04:34:11 -07:00
range_del_aggregator_test.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
range_del_aggregator.cc In ParseInternalKey(), include corrupt key info in Status (#7515) 2020-10-28 10:12:58 -07:00
range_del_aggregator.h In ParseInternalKey(), include corrupt key info in Status (#7515) 2020-10-28 10:12:58 -07:00
range_tombstone_fragmenter_test.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
range_tombstone_fragmenter.cc Include C++ standard library headers instead of C compatibility headers (#8068) 2021-03-19 12:09:47 -07:00
range_tombstone_fragmenter.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
read_callback.h Get() with timestamp should respect snapshot (#7227) 2020-08-14 19:20:58 -07:00
repair_test.cc Use SST file manager to track blob files as well (#8037) 2021-03-17 20:44:49 -07:00
repair.cc Use SystemClock* instead of std::shared_ptr<SystemClock> in lower level routines (#8033) 2021-03-15 04:34:11 -07:00
snapshot_checker.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
snapshot_impl.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
snapshot_impl.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
table_cache.cc Use SystemClock* instead of std::shared_ptr<SystemClock> in lower level routines (#8033) 2021-03-15 04:34:11 -07:00
table_cache.h Include C++ standard library headers instead of C compatibility headers (#8068) 2021-03-19 12:09:47 -07:00
table_properties_collector_test.cc Make StringEnv, StringSink, StringSource use FS classes (#7786) 2021-01-04 16:01:01 -08:00
table_properties_collector.cc In ParseInternalKey(), include corrupt key info in Status (#7515) 2020-10-28 10:12:58 -07:00
table_properties_collector.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
transaction_log_impl.cc Add more tests for assert status checked (#7524) 2020-12-22 23:45:58 -08:00
transaction_log_impl.h Store FileSystemPtr object that contains FileSystem ptr (#7180) 2020-08-12 17:31:23 -07:00
trim_history_scheduler.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
trim_history_scheduler.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
version_builder_test.cc Remove unused includes (#7604) 2020-10-28 23:22:27 -07:00
version_builder.cc Log sst number in Corruption status (#7767) 2020-12-14 14:07:52 -08:00
version_builder.h make L0 index/filter pinned memory usage predictable (#6911) 2020-06-09 16:51:23 -07:00
version_edit_handler.cc Make secondary instance use ManifestTailer (#7998) 2021-03-10 10:59:44 -08:00
version_edit_handler.h Make secondary instance use ManifestTailer (#7998) 2021-03-10 10:59:44 -08:00
version_edit_test.cc Make it able to ignore WAL related VersionEdits in older versions (#7873) 2021-01-19 19:27:53 -08:00
version_edit.cc Make blob related VersionEdit tags unignorable (#7886) 2021-01-20 20:29:04 -08:00
version_edit.h Accumulate blob file additions in VersionEdit during recovery (#7903) 2021-01-27 18:46:15 -08:00
version_set_test.cc Use SystemClock* instead of std::shared_ptr<SystemClock> in lower level routines (#8033) 2021-03-15 04:34:11 -07:00
version_set.cc Handle rename() failure in non-local FS (#8192) 2021-04-19 19:56:59 -07:00
version_set.h Use SystemClock* instead of std::shared_ptr<SystemClock> in lower level routines (#8033) 2021-03-15 04:34:11 -07:00
wal_edit_test.cc Always track WAL obsoletion (#7759) 2020-12-09 16:02:12 -08:00
wal_edit.cc Always track WAL obsoletion (#7759) 2020-12-09 16:02:12 -08:00
wal_edit.h Always track WAL obsoletion (#7759) 2020-12-09 16:02:12 -08:00
wal_manager_test.cc Use SystemClock* instead of std::shared_ptr<SystemClock> in lower level routines (#8033) 2021-03-15 04:34:11 -07:00
wal_manager.cc Use SystemClock* instead of std::shared_ptr<SystemClock> in lower level routines (#8033) 2021-03-15 04:34:11 -07:00
wal_manager.h Store FileSystemPtr object that contains FileSystem ptr (#7180) 2020-08-12 17:31:23 -07:00
write_batch_base.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
write_batch_internal.h Integrity protection for live updates to WriteBatch (#7748) 2021-01-29 12:18:58 -08:00
write_batch_test.cc Add further tests to ASSERT_STATUS_CHECKED (1) (#7679) 2020-12-08 15:55:04 -08:00
write_batch.cc Use SystemClock* instead of std::shared_ptr<SystemClock> in lower level routines (#8033) 2021-03-15 04:34:11 -07:00
write_callback_test.cc Add more tests for assert status checked (#7524) 2020-12-22 23:45:58 -08:00
write_callback.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
write_controller_test.cc Revamp WriteController (#8064) 2021-03-18 09:47:31 -07:00
write_controller.cc Revamp WriteController (#8064) 2021-03-18 09:47:31 -07:00
write_controller.h Revamp WriteController (#8064) 2021-03-18 09:47:31 -07:00
write_thread.cc Integrity protection for live updates to WriteBatch (#7748) 2021-01-29 12:18:58 -08:00
write_thread.h Include C++ standard library headers instead of C compatibility headers (#8068) 2021-03-19 12:09:47 -07:00