andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
rocksdb/db
History
Andrew Kryczka f8f6983f89 Skip range deletions at seqno zero when collapsing (#4216) 
Summary:
`CollapsedRangeDelMap` internally uses seqno zero as a sentinel value to
denote a gap between range tombstones or the end of range tombstones. It
therefore expects to never have consecutive sentinel tombstones.

However, since `DeleteRange` is now supported in `SstFileWriter`, an
ingested file may contain range tombstones, and that ingested file may
be assigned global seqno zero. When such tombstones are added to the
collapsed map, they resemble sentinel tombstones due to having seqno
zero. Then, the invariant mentioned above about never having consecutive
sentinel tombstones can be violated.

The symptom of this violation was dereferencing the `end()` iterator
(#4204). The fix in this PR is to not add range tombstones with seqno
zero to the collapsed map. They're not needed anyways since they can't
possibly cover anything (in case of a key and a range tombstone with the
same seqno, the key is visible).
Pull Request resolved: https://github.com/facebook/rocksdb/pull/4216

Differential Revision: D9121716

Pulled By: ajkr

fbshipit-source-id: f5b78a70bea9527354603ea7ac8542a7e2b6a210
2018-08-01 12:12:02 -07:00
..
builder.cc
Relax VersionStorageInfo::GetOverlappingInputs check (#4050)
2018-07-13 17:42:38 -07:00
builder.h
Move prefix_extractor to MutableCFOptions
2018-05-21 14:43:11 -07:00
c_test.c
comment unused parameters to turn on -Wunused-parameter flag
2018-04-12 17:59:16 -07:00
c.cc
Add bottommost_compression_opts to for bottommost_compression (#3985)
2018-06-27 17:42:38 -07:00
column_family_test.cc
Per-thread unique test db names (#4135)
2018-07-13 17:27:39 -07:00
column_family.cc
Skip deleted WALs during recovery
2018-05-03 15:43:09 -07:00
column_family.h
Add max_subcompactions as a compaction option
2018-04-27 11:57:39 -07:00
compact_files_test.cc
Per-thread unique test db names (#4135)
2018-07-13 17:27:39 -07:00
compacted_db_impl.cc
Move prefix_extractor to MutableCFOptions
2018-05-21 14:43:11 -07:00
compacted_db_impl.h
Comment out unused variables
2018-03-05 13:13:41 -08:00
compaction_iteration_stats.h
add counter for deletion dropping optimization
2017-08-19 14:10:08 -07:00
compaction_iterator_test.cc
option for timing measurement of non-blocking ops during compaction (#4029)
2018-06-21 21:28:05 -07:00
compaction_iterator.cc
Range deletion performance improvements + cleanup (#4014)
2018-07-12 14:42:39 -07:00
compaction_iterator.h
option for timing measurement of non-blocking ops during compaction (#4029)
2018-06-21 21:28:05 -07:00
compaction_job_stats_test.cc
Per-thread unique test db names (#4135)
2018-07-13 17:27:39 -07:00
compaction_job_test.cc
Per-thread unique test db names (#4135)
2018-07-13 17:27:39 -07:00
compaction_job.cc
Relax VersionStorageInfo::GetOverlappingInputs check (#4050)
2018-07-13 17:42:38 -07:00
compaction_job.h
Allow DB resume after background errors (#3997)
2018-06-28 12:34:40 -07:00
compaction_picker_test.cc
Remove random writes from SST file ingestion (#4172)
2018-07-27 16:12:23 -07:00
compaction_picker_universal.cc
Remove random writes from SST file ingestion (#4172)
2018-07-27 16:12:23 -07:00
compaction_picker_universal.h
Delete triggered compaction for universal style
2018-05-29 15:44:34 -07:00
compaction_picker.cc
compaction: fix max_subcompactions option for CompactRange (#4082)
2018-07-05 20:12:56 -07:00
compaction_picker.h
Add bottommost_compression_opts to for bottommost_compression (#3985)
2018-06-27 17:42:38 -07:00
compaction.cc
Add bottommost_compression_opts to for bottommost_compression (#3985)
2018-06-27 17:42:38 -07:00
compaction.h
Add bottommost_compression_opts to for bottommost_compression (#3985)
2018-06-27 17:42:38 -07:00
comparator_db_test.cc
Per-thread unique test db names (#4135)
2018-07-13 17:27:39 -07:00
convenience.cc
Pin mmap files in ReadOnlyDB (#4053)
2018-06-27 17:13:34 -07:00
corruption_test.cc
Per-thread unique test db names (#4135)
2018-07-13 17:27:39 -07:00
cuckoo_table_db_test.cc
Per-thread unique test db names (#4135)
2018-07-13 17:27:39 -07:00
db_basic_test.cc
Per-thread unique test db names (#4135)
2018-07-13 17:27:39 -07:00
db_blob_index_test.cc
fix lite build
2017-10-17 08:57:09 -07:00
db_block_cache_test.cc
LRUCache midpoint insertion
2018-05-24 15:57:33 -07:00
db_bloom_filter_test.cc
PrefixMayMatch: remove unnecessary check for prefix_extractor_ (#4067)
2018-06-27 20:42:43 -07:00
db_compaction_filter_test.cc
Remove tests from ROCKSDB_VALGRIND_RUN
2018-05-30 16:15:16 -07:00
db_compaction_test.cc
Allow ttl to be changed dynamically (#4133)
2018-07-16 14:27:53 -07:00
db_dynamic_level_test.cc
fix memory leak in two_level_iterator
2018-04-15 17:26:26 -07:00
db_encryption_test.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
db_filesnapshot.cc
fix behavior does not match name for "IsFileDeletionsEnabled"
2018-03-21 22:13:34 -07:00
db_flush_test.cc
Disable DBFlushTest.SyncFail and DBTest.GroupCommitTest on Travis (#4154)
2018-07-18 18:43:11 -07:00
db_impl_compaction_flush.cc
Remove random writes from SST file ingestion (#4172)
2018-07-27 16:12:23 -07:00
db_impl_debug.cc
Allow DB resume after background errors (#3997)
2018-06-28 12:34:40 -07:00
db_impl_experimental.cc
Remove random writes from SST file ingestion (#4172)
2018-07-27 16:12:23 -07:00
db_impl_files.cc
SetOptions Backup Race Condition (#4108)
2018-07-11 14:57:46 -07:00
db_impl_open.cc
Remove random writes from SST file ingestion (#4172)
2018-07-27 16:12:23 -07:00
db_impl_readonly.cc
Trace and Replay for RocksDB (#3837)
2018-08-01 00:27:08 -07:00
db_impl_readonly.h
allowing CompactFiles to return new file names
2018-03-15 11:58:12 -07:00
db_impl_write.cc
Trace and Replay for RocksDB (#3837)
2018-08-01 00:27:08 -07:00
db_impl.cc
Trace and Replay for RocksDB (#3837)
2018-08-01 00:27:08 -07:00
db_impl.h
Trace and Replay for RocksDB (#3837)
2018-08-01 00:27:08 -07:00
db_info_dumper.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
db_info_dumper.h
Change RocksDB License
2017-07-15 16:11:23 -07:00
db_inplace_update_test.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
db_io_failure_test.cc
Fix LITE unit tests
2017-07-26 21:11:47 -07:00
db_iter_stress_test.cc
Move prefix_extractor to MutableCFOptions
2018-05-21 14:43:11 -07:00
db_iter_test.cc
Fix regression bug of Prev() with upper bound (#3989)
2018-06-12 16:57:36 -07:00
db_iter.cc
Range deletion performance improvements + cleanup (#4014)
2018-07-12 14:42:39 -07:00
db_iter.h
Move prefix_extractor to MutableCFOptions
2018-05-21 14:43:11 -07:00
db_iterator_test.cc
Fix bug when seeking backward against an out-of-bound iterator (#4187)
2018-07-25 17:14:01 -07:00
db_log_iter_test.cc
fix memory leak in two_level_iterator
2018-04-15 17:26:26 -07:00
db_memtable_test.cc
Comment out unused variables
2018-03-05 13:13:41 -08:00
db_merge_operator_test.cc
WriteUnPrepared Txn: Disable seek to snapshot optimization (#3955)
2018-06-27 12:23:07 -07:00
db_options_test.cc
PersistRocksDBOptions() to use WritableFileWriter
2018-05-21 16:42:22 -07:00
db_properties_test.cc
Add table property tracking number of range deletions (#4016)
2018-06-26 20:27:35 -07:00
db_range_del_test.cc
Remove random writes from SST file ingestion (#4172)
2018-07-27 16:12:23 -07:00
db_sst_test.cc
DBSSTTest.DeleteSchedulerMultipleDBPaths data race (#4146)
2018-07-17 17:57:46 -07:00
db_statistics_test.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
db_table_properties_test.cc
fix deletion-triggered compaction in table builder
2017-09-28 18:17:30 -07:00
db_tailing_iter_test.cc
Remove managed iterator
2018-07-17 14:43:18 -07:00
db_test2.cc
Trace and Replay for RocksDB (#3837)
2018-08-01 00:27:08 -07:00
db_test_util.cc
Per-thread unique test db names (#4135)
2018-07-13 17:27:39 -07:00
db_test_util.h
Support range deletion tombstones in IngestExternalFile SSTs (#3778)
2018-07-13 22:43:09 -07:00
db_test.cc
Disable DBFlushTest.SyncFail and DBTest.GroupCommitTest on Travis (#4154)
2018-07-18 18:43:11 -07:00
db_universal_compaction_test.cc
Reformatting some recent changes (#4161)
2018-07-20 14:43:38 -07:00
db_wal_test.cc
Skip deleted WALs during recovery
2018-05-03 15:43:09 -07:00
db_write_test.cc
Suppress tsan lock-order-inversion on FlushWAL
2018-05-14 21:13:35 -07:00
dbformat_test.cc
Relax VersionStorageInfo::GetOverlappingInputs check (#4050)
2018-07-13 17:42:38 -07:00
dbformat.cc
add kEntryRangeDeletion
2018-04-13 11:27:17 -07:00
dbformat.h
Refactor IndexBlockIter (#4141)
2018-07-16 17:13:10 -07:00
deletefile_test.cc
Per-thread unique test db names (#4135)
2018-07-13 17:27:39 -07:00
error_handler_test.cc
Allow DB resume after background errors (#3997)
2018-06-28 12:34:40 -07:00
error_handler.cc
Allow DB resume after background errors (#3997)
2018-06-28 12:34:40 -07:00
error_handler.h
Allow DB resume after background errors (#3997)
2018-06-28 12:34:40 -07:00
event_helpers.cc
comment unused parameters to turn on -Wunused-parameter flag
2018-04-12 17:59:16 -07:00
event_helpers.h
Change RocksDB License
2017-07-15 16:11:23 -07:00
experimental.cc
comment unused parameters to turn on -Wunused-parameter flag
2018-04-12 17:59:16 -07:00
external_sst_file_basic_test.cc
Support range deletion tombstones in IngestExternalFile SSTs (#3778)
2018-07-13 22:43:09 -07:00
external_sst_file_ingestion_job.cc
Remove random writes from SST file ingestion (#4172)
2018-07-27 16:12:23 -07:00
external_sst_file_ingestion_job.h
Remove random writes from SST file ingestion (#4172)
2018-07-27 16:12:23 -07:00
external_sst_file_test.cc
Pending output file number should be released after bulkload failure (#4145)
2018-07-17 14:13:16 -07:00
fault_injection_test.cc
Per-thread unique test db names (#4135)
2018-07-13 17:27:39 -07:00
file_indexer_test.cc
Comment out unused variables
2018-03-05 13:13:41 -08:00
file_indexer.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
file_indexer.h
Change RocksDB License
2017-07-15 16:11:23 -07:00
filename_test.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
flush_job_test.cc
Remove random writes from SST file ingestion (#4172)
2018-07-27 16:12:23 -07:00
flush_job.cc
Remove random writes from SST file ingestion (#4172)
2018-07-27 16:12:23 -07:00
flush_job.h
Skip deleted WALs during recovery
2018-05-03 15:43:09 -07:00
flush_scheduler.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
flush_scheduler.h
Change RocksDB License
2017-07-15 16:11:23 -07:00
forward_iterator_bench.cc
Per-thread unique test db names (#4135)
2018-07-13 17:27:39 -07:00
forward_iterator.cc
Relax VersionStorageInfo::GetOverlappingInputs check (#4050)
2018-07-13 17:42:38 -07:00
forward_iterator.h
Comment out unused variables
2018-03-05 13:13:41 -08:00
internal_stats.cc
Add kOptionsStatistics to GetProperty() (#3966)
2018-06-15 17:28:01 -07:00
internal_stats.h
Add kOptionsStatistics to GetProperty() (#3966)
2018-06-15 17:28:01 -07:00
job_context.h
Introduce and use the option to disable stall notifications structures
2018-05-09 10:13:53 -07:00
listener_test.cc
fix some text in comments.
2018-04-10 15:59:24 -07:00
log_format.h
Change RocksDB License
2017-07-15 16:11:23 -07:00
log_reader.cc
Add GCC 8 to Travis (#3433)
2018-07-13 10:58:06 -07:00
log_reader.h
Suppress lint in old files
2018-01-29 12:56:42 -08:00
log_test.cc
Add file name info to SequentialFileReader. (#4026)
2018-06-21 08:42:24 -07:00
log_writer.cc
Pass manual_wal_flush also to the first wal file
2018-05-14 10:57:56 -07:00
log_writer.h
Pass manual_wal_flush also to the first wal file
2018-05-14 10:57:56 -07:00
logs_with_prep_tracker.cc
Skip deleted WALs during recovery
2018-05-03 15:43:09 -07:00
logs_with_prep_tracker.h
Skip deleted WALs during recovery
2018-05-03 15:43:09 -07:00
malloc_stats.cc
Disallow compactions if there isn't enough free space
2018-03-06 16:27:54 -08:00
malloc_stats.h
Change RocksDB License
2017-07-15 16:11:23 -07:00
manual_compaction_test.cc
Per-thread unique test db names (#4135)
2018-07-13 17:27:39 -07:00
memtable_list_test.cc
Per-thread unique test db names (#4135)
2018-07-13 17:27:39 -07:00
memtable_list.cc
Support group commits of version edits (#3944)
2018-06-28 12:34:39 -07:00
memtable_list.h
Skip deleted WALs during recovery
2018-05-03 15:43:09 -07:00
memtable.cc
Add GCC 8 to Travis (#3433)
2018-07-13 10:58:06 -07:00
memtable.h
InlineSkiplist: don't decode keys unnecessarily during comparisons
2018-03-23 12:14:30 -07:00
merge_context.h
Change RocksDB License
2017-07-15 16:11:23 -07:00
merge_helper_test.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
merge_helper.cc
Range deletion performance improvements + cleanup (#4014)
2018-07-12 14:42:39 -07:00
merge_helper.h
WritePrepared Txn: Support merge operator
2018-02-09 14:57:54 -08:00
merge_operator.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
merge_test.cc
Per-thread unique test db names (#4135)
2018-07-13 17:27:39 -07:00
obsolete_files_test.cc
Per-thread unique test db names (#4135)
2018-07-13 17:27:39 -07:00
options_file_test.cc
Per-thread unique test db names (#4135)
2018-07-13 17:27:39 -07:00
perf_context_test.cc
Per-thread unique test db names (#4135)
2018-07-13 17:27:39 -07:00
pinned_iterators_manager.h
Change RocksDB License
2017-07-15 16:11:23 -07:00
plain_table_db_test.cc
Per-thread unique test db names (#4135)
2018-07-13 17:27:39 -07:00
pre_release_callback.h
Fix pre_release callback argument list.
2018-04-05 11:12:16 -07:00
prefix_test.cc
Per-thread unique test db names (#4135)
2018-07-13 17:27:39 -07:00
range_del_aggregator_test.cc
Relax VersionStorageInfo::GetOverlappingInputs check (#4050)
2018-07-13 17:42:38 -07:00
range_del_aggregator.cc
Skip range deletions at seqno zero when collapsing (#4216)
2018-08-01 12:12:02 -07:00
range_del_aggregator.h
Support range deletion tombstones in IngestExternalFile SSTs (#3778)
2018-07-13 22:43:09 -07:00
read_callback.h
WriteUnPrepared Txn: Disable seek to snapshot optimization (#3955)
2018-06-27 12:23:07 -07:00
repair_test.cc
fix memory leak in two_level_iterator
2018-04-15 17:26:26 -07:00
repair.cc
Relax VersionStorageInfo::GetOverlappingInputs check (#4050)
2018-07-13 17:42:38 -07:00
snapshot_checker.h
Comment out unused variables
2018-03-05 13:13:41 -08:00
snapshot_impl.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
snapshot_impl.h
WritePrepared Txn: smallest_prepare optimization
2018-04-02 20:27:41 -07:00
table_cache.cc
Remove random writes from SST file ingestion (#4172)
2018-07-27 16:12:23 -07:00
table_cache.h
Relax VersionStorageInfo::GetOverlappingInputs check (#4050)
2018-07-13 17:42:38 -07:00
table_properties_collector_test.cc
Should only decode restart points for uncompressed blocks (#3996)
2018-06-15 19:26:58 -07:00
table_properties_collector.cc
Comment out unused variables
2018-03-05 13:13:41 -08:00
table_properties_collector.h
Comment out unused variables
2018-03-05 13:13:41 -08:00
transaction_log_impl.cc
WriteUnPrepared: Add support for recovering WriteUnprepared transactions (#4078)
2018-07-06 17:59:13 -07:00
transaction_log_impl.h
WritePrepared Txn: Refactor conf params
2017-11-10 17:28:12 -08:00
version_builder_test.cc
Remove random writes from SST file ingestion (#4172)
2018-07-27 16:12:23 -07:00
version_builder.cc
Remove random writes from SST file ingestion (#4172)
2018-07-27 16:12:23 -07:00
version_builder.h
Move prefix_extractor to MutableCFOptions
2018-05-21 14:43:11 -07:00
version_edit_test.cc
Skip deleted WALs during recovery
2018-05-03 15:43:09 -07:00
version_edit.cc
Remove random writes from SST file ingestion (#4172)
2018-07-27 16:12:23 -07:00
version_edit.h
Remove random writes from SST file ingestion (#4172)
2018-07-27 16:12:23 -07:00
version_set_test.cc
Relax VersionStorageInfo::GetOverlappingInputs check (#4050)
2018-07-13 17:42:38 -07:00
version_set.cc
Remove random writes from SST file ingestion (#4172)
2018-07-27 16:12:23 -07:00
version_set.h
Protect external file when ingesting (#4099)
2018-07-27 14:13:12 -07:00
wal_manager_test.cc
Per-thread unique test db names (#4135)
2018-07-13 17:27:39 -07:00
wal_manager.cc
Add file name info to SequentialFileReader. (#4026)
2018-06-21 08:42:24 -07:00
wal_manager.h
Fix memleak when DB::DeleteFile()
2018-01-11 18:57:33 -08:00
write_batch_base.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
write_batch_internal.h
WriteUnPrepared: Add new WAL marker kTypeBeginUnprepareXID (#4069)
2018-06-28 18:58:29 -07:00
write_batch_test.cc
WriteUnPrepared: Add support for recovering WriteUnprepared transactions (#4078)
2018-07-06 17:59:13 -07:00
write_batch.cc
WriteUnPrepared: Implement unprepared batches for transactions (#4104)
2018-07-24 00:13:18 -07:00
write_callback_test.cc
Per-thread unique test db names (#4135)
2018-07-13 17:27:39 -07:00
write_callback.h
Change RocksDB License
2017-07-15 16:11:23 -07:00
write_controller_test.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
write_controller.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
write_controller.h
Change RocksDB License
2017-07-15 16:11:23 -07:00
write_thread.cc
Fix write get stuck when pipelined write is enabled (#4143)
2018-07-17 17:27:51 -07:00
write_thread.h
Fix write get stuck when pipelined write is enabled (#4143)
2018-07-17 17:27:51 -07:00