andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
rocksdb/db
History
Yi Wu 4bd3bc5c4f BlobDB: Fix VisibleToActiveSnapshot() (#4236) 
Summary:
There are two issues with `VisibleToActiveSnapshot`:
1. If there are no snapshots, `oldest_snapshot` will be 0 and `VisibleToActiveSnapshot` will always return true. Since the method is used to decide whether it is safe to delete obsolete files, obsolete file won't be able to delete in this case.
2. The `auto` keyword of `auto snapshots = db_impl_->snapshots()` translate to a copy of `const SnapshotList` instead of a reference. Since copy constructor of `SnapshotList` is not defined, using the copy may yield unexpected result.

Issue 2 actually hide issue 1 from being catch by tests. During test `snapshots.empty()` can return false while it should actually be empty, and `snapshots.oldest()` return an invalid address, making `oldest_snapshot` being some random large number.

The issue was originally reported by BlobDB early adopter at Kuaishou.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/4236

Differential Revision: D9188706

Pulled By: yiwu-arbug

fbshipit-source-id: a0f2624b927cf9bf28c1bb534784fee5d106f5ea
2018-08-24 14:38:20 -07:00
..
builder.cc
Revert changes in PR #4003 (#4263)
2018-08-11 21:47:27 -07:00
builder.h
Move prefix_extractor to MutableCFOptions
2018-05-21 14:43:11 -07:00
c_test.c
comment unused parameters to turn on -Wunused-parameter flag
2018-04-12 17:59:16 -07:00
c.cc
Add bottommost_compression_opts to for bottommost_compression (#3985)
2018-06-27 17:42:38 -07:00
column_family_test.cc
Per-thread unique test db names (#4135)
2018-07-13 17:27:39 -07:00
column_family.cc
Skip deleted WALs during recovery
2018-05-03 15:43:09 -07:00
column_family.h
Add max_subcompactions as a compaction option
2018-04-27 11:57:39 -07:00
compact_files_test.cc
Per-thread unique test db names (#4135)
2018-07-13 17:27:39 -07:00
compacted_db_impl.cc
Move prefix_extractor to MutableCFOptions
2018-05-21 14:43:11 -07:00
compacted_db_impl.h
Comment out unused variables
2018-03-05 13:13:41 -08:00
compaction_iteration_stats.h
add counter for deletion dropping optimization
2017-08-19 14:10:08 -07:00
compaction_iterator_test.cc
option for timing measurement of non-blocking ops during compaction (#4029)
2018-06-21 21:28:05 -07:00
compaction_iterator.cc
Range deletion performance improvements + cleanup (#4014)
2018-07-12 14:42:39 -07:00
compaction_iterator.h
option for timing measurement of non-blocking ops during compaction (#4029)
2018-06-21 21:28:05 -07:00
compaction_job_stats_test.cc
Per-thread unique test db names (#4135)
2018-07-13 17:27:39 -07:00
compaction_job_test.cc
Per-thread unique test db names (#4135)
2018-07-13 17:27:39 -07:00
compaction_job.cc
Reduce empty SST creation/deletion during compaction (#4311)
2018-08-24 13:47:06 -07:00
compaction_job.h
Allow DB resume after background errors (#3997)
2018-06-28 12:34:40 -07:00
compaction_picker_test.cc
Delete triggered compaction for universal style
2018-05-29 15:44:34 -07:00
compaction_picker_universal.cc
Add bottommost_compression_opts to for bottommost_compression (#3985)
2018-06-27 17:42:38 -07:00
compaction_picker_universal.h
Delete triggered compaction for universal style
2018-05-29 15:44:34 -07:00
compaction_picker.cc
compaction: fix max_subcompactions option for CompactRange (#4082)
2018-07-05 20:12:56 -07:00
compaction_picker.h
Add bottommost_compression_opts to for bottommost_compression (#3985)
2018-06-27 17:42:38 -07:00
compaction.cc
Add bottommost_compression_opts to for bottommost_compression (#3985)
2018-06-27 17:42:38 -07:00
compaction.h
Add bottommost_compression_opts to for bottommost_compression (#3985)
2018-06-27 17:42:38 -07:00
comparator_db_test.cc
Per-thread unique test db names (#4135)
2018-07-13 17:27:39 -07:00
convenience.cc
Pin mmap files in ReadOnlyDB (#4053)
2018-06-27 17:13:34 -07:00
corruption_test.cc
Per-thread unique test db names (#4135)
2018-07-13 17:27:39 -07:00
cuckoo_table_db_test.cc
Per-thread unique test db names (#4135)
2018-07-13 17:27:39 -07:00
db_basic_test.cc
Per-thread unique test db names (#4135)
2018-07-13 17:27:39 -07:00
db_blob_index_test.cc
fix lite build
2017-10-17 08:57:09 -07:00
db_block_cache_test.cc
LRUCache midpoint insertion
2018-05-24 15:57:33 -07:00
db_bloom_filter_test.cc
PrefixMayMatch: remove unnecessary check for prefix_extractor_ (#4067)
2018-06-27 20:42:43 -07:00
db_compaction_filter_test.cc
Remove tests from ROCKSDB_VALGRIND_RUN
2018-05-30 16:15:16 -07:00
db_compaction_test.cc
Allow ttl to be changed dynamically (#4133)
2018-07-16 14:27:53 -07:00
db_dynamic_level_test.cc
fix memory leak in two_level_iterator
2018-04-15 17:26:26 -07:00
db_encryption_test.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
db_filesnapshot.cc
fix behavior does not match name for "IsFileDeletionsEnabled"
2018-03-21 22:13:34 -07:00
db_flush_test.cc
Skip deleted WALs during recovery
2018-05-03 15:43:09 -07:00
db_impl_compaction_flush.cc
Allow DB resume after background errors (#3997)
2018-06-28 12:34:40 -07:00
db_impl_debug.cc
Allow DB resume after background errors (#3997)
2018-06-28 12:34:40 -07:00
db_impl_experimental.cc
Inform caller when rocksdb is stalling writes
2017-10-05 18:11:43 -07:00
db_impl_files.cc
SetOptions Backup Race Condition (#4108)
2018-07-11 14:57:46 -07:00
db_impl_open.cc
WriteUnPrepared: Add new WAL marker kTypeBeginUnprepareXID (#4069)
2018-06-28 18:58:29 -07:00
db_impl_readonly.cc
Move prefix_extractor to MutableCFOptions
2018-05-21 14:43:11 -07:00
db_impl_readonly.h
allowing CompactFiles to return new file names
2018-03-15 11:58:12 -07:00
db_impl_write.cc
Allow DB resume after background errors (#3997)
2018-06-28 12:34:40 -07:00
db_impl.cc
VerifyChecksum() API should preserve options
2018-08-16 16:43:30 -07:00
db_impl.h
WriteUnPrepared: Add support for recovering WriteUnprepared transactions (#4078)
2018-07-06 17:59:13 -07:00
db_info_dumper.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
db_info_dumper.h
Change RocksDB License
2017-07-15 16:11:23 -07:00
db_inplace_update_test.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
db_io_failure_test.cc
Fix LITE unit tests
2017-07-26 21:11:47 -07:00
db_iter_stress_test.cc
Move prefix_extractor to MutableCFOptions
2018-05-21 14:43:11 -07:00
db_iter_test.cc
Fix regression bug of Prev() with upper bound (#3989)
2018-06-12 16:57:36 -07:00
db_iter.cc
Range deletion performance improvements + cleanup (#4014)
2018-07-12 14:42:39 -07:00
db_iter.h
Move prefix_extractor to MutableCFOptions
2018-05-21 14:43:11 -07:00
db_iterator_test.cc
Fix bug when seeking backward against an out-of-bound iterator (#4187)
2018-07-25 17:25:26 -07:00
db_log_iter_test.cc
fix memory leak in two_level_iterator
2018-04-15 17:26:26 -07:00
db_memtable_test.cc
Comment out unused variables
2018-03-05 13:13:41 -08:00
db_merge_operator_test.cc
WriteUnPrepared Txn: Disable seek to snapshot optimization (#3955)
2018-06-27 12:23:07 -07:00
db_options_test.cc
PersistRocksDBOptions() to use WritableFileWriter
2018-05-21 16:42:22 -07:00
db_properties_test.cc
Add table property tracking number of range deletions (#4016)
2018-06-26 20:27:35 -07:00
db_range_del_test.cc
Support range deletion tombstones in IngestExternalFile SSTs (#3778)
2018-07-13 22:43:09 -07:00
db_sst_test.cc
comment unused parameters to turn on -Wunused-parameter flag
2018-04-12 17:59:16 -07:00
db_statistics_test.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
db_table_properties_test.cc
fix deletion-triggered compaction in table builder
2017-09-28 18:17:30 -07:00
db_tailing_iter_test.cc
Remove managed iterator
2018-07-17 14:43:18 -07:00
db_test2.cc
Per-thread unique test db names (#4135)
2018-07-13 17:27:39 -07:00
db_test_util.cc
Per-thread unique test db names (#4135)
2018-07-13 17:27:39 -07:00
db_test_util.h
Support range deletion tombstones in IngestExternalFile SSTs (#3778)
2018-07-13 22:43:09 -07:00
db_test.cc
Per-thread unique test db names (#4135)
2018-07-13 17:27:39 -07:00
db_universal_compaction_test.cc
Return new operator for Status allocations for Windows (#4128)
2018-07-19 17:30:09 -07:00
db_wal_test.cc
Skip deleted WALs during recovery
2018-05-03 15:43:09 -07:00
db_write_test.cc
Suppress tsan lock-order-inversion on FlushWAL
2018-05-14 21:13:35 -07:00
dbformat_test.cc
Relax VersionStorageInfo::GetOverlappingInputs check (#4050)
2018-07-13 17:42:38 -07:00
dbformat.cc
add kEntryRangeDeletion
2018-04-13 11:27:17 -07:00
dbformat.h
Refactor IndexBlockIter (#4141)
2018-07-16 17:13:10 -07:00
deletefile_test.cc
Per-thread unique test db names (#4135)
2018-07-13 17:27:39 -07:00
error_handler_test.cc
Allow DB resume after background errors (#3997)
2018-06-28 12:34:40 -07:00
error_handler.cc
Allow DB resume after background errors (#3997)
2018-06-28 12:34:40 -07:00
error_handler.h
Allow DB resume after background errors (#3997)
2018-06-28 12:34:40 -07:00
event_helpers.cc
comment unused parameters to turn on -Wunused-parameter flag
2018-04-12 17:59:16 -07:00
event_helpers.h
Change RocksDB License
2017-07-15 16:11:23 -07:00
experimental.cc
comment unused parameters to turn on -Wunused-parameter flag
2018-04-12 17:59:16 -07:00
external_sst_file_basic_test.cc
Support range deletion tombstones in IngestExternalFile SSTs (#3778)
2018-07-13 22:43:09 -07:00
external_sst_file_ingestion_job.cc
Support range deletion tombstones in IngestExternalFile SSTs (#3778)
2018-07-13 22:43:09 -07:00
external_sst_file_ingestion_job.h
Support range deletion tombstones in IngestExternalFile SSTs (#3778)
2018-07-13 22:43:09 -07:00
external_sst_file_test.cc
Pending output file number should be released after bulkload failure (#4145)
2018-07-17 14:13:16 -07:00
fault_injection_test.cc
Per-thread unique test db names (#4135)
2018-07-13 17:27:39 -07:00
file_indexer_test.cc
Comment out unused variables
2018-03-05 13:13:41 -08:00
file_indexer.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
file_indexer.h
Change RocksDB License
2017-07-15 16:11:23 -07:00
filename_test.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
flush_job_test.cc
Per-thread unique test db names (#4135)
2018-07-13 17:27:39 -07:00
flush_job.cc
Allow DB resume after background errors (#3997)
2018-06-28 12:34:40 -07:00
flush_job.h
Skip deleted WALs during recovery
2018-05-03 15:43:09 -07:00
flush_scheduler.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
flush_scheduler.h
Change RocksDB License
2017-07-15 16:11:23 -07:00
forward_iterator_bench.cc
Per-thread unique test db names (#4135)
2018-07-13 17:27:39 -07:00
forward_iterator.cc
Relax VersionStorageInfo::GetOverlappingInputs check (#4050)
2018-07-13 17:42:38 -07:00
forward_iterator.h
Comment out unused variables
2018-03-05 13:13:41 -08:00
internal_stats.cc
Add kOptionsStatistics to GetProperty() (#3966)
2018-06-15 17:28:01 -07:00
internal_stats.h
Add kOptionsStatistics to GetProperty() (#3966)
2018-06-15 17:28:01 -07:00
job_context.h
Introduce and use the option to disable stall notifications structures
2018-05-09 10:13:53 -07:00
listener_test.cc
fix some text in comments.
2018-04-10 15:59:24 -07:00
log_format.h
Change RocksDB License
2017-07-15 16:11:23 -07:00
log_reader.cc
Add GCC 8 to Travis (#3433)
2018-07-13 10:58:06 -07:00
log_reader.h
Suppress lint in old files
2018-01-29 12:56:42 -08:00
log_test.cc
Add file name info to SequentialFileReader. (#4026)
2018-06-21 08:42:24 -07:00
log_writer.cc
Pass manual_wal_flush also to the first wal file
2018-05-14 10:57:56 -07:00
log_writer.h
Pass manual_wal_flush also to the first wal file
2018-05-14 10:57:56 -07:00
logs_with_prep_tracker.cc
Skip deleted WALs during recovery
2018-05-03 15:43:09 -07:00
logs_with_prep_tracker.h
Skip deleted WALs during recovery
2018-05-03 15:43:09 -07:00
malloc_stats.cc
Disallow compactions if there isn't enough free space
2018-03-06 16:27:54 -08:00
malloc_stats.h
Change RocksDB License
2017-07-15 16:11:23 -07:00
manual_compaction_test.cc
Per-thread unique test db names (#4135)
2018-07-13 17:27:39 -07:00
memtable_list_test.cc
Per-thread unique test db names (#4135)
2018-07-13 17:27:39 -07:00
memtable_list.cc
Support group commits of version edits (#3944)
2018-06-28 12:34:39 -07:00
memtable_list.h
Skip deleted WALs during recovery
2018-05-03 15:43:09 -07:00
memtable.cc
Summary:
2018-08-21 17:57:45 -07:00
memtable.h
InlineSkiplist: don't decode keys unnecessarily during comparisons
2018-03-23 12:14:30 -07:00
merge_context.h
Summary:
2018-08-21 17:57:45 -07:00
merge_helper_test.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
merge_helper.cc
Range deletion performance improvements + cleanup (#4014)
2018-07-12 14:42:39 -07:00
merge_helper.h
WritePrepared Txn: Support merge operator
2018-02-09 14:57:54 -08:00
merge_operator.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
merge_test.cc
Per-thread unique test db names (#4135)
2018-07-13 17:27:39 -07:00
obsolete_files_test.cc
Per-thread unique test db names (#4135)
2018-07-13 17:27:39 -07:00
options_file_test.cc
Per-thread unique test db names (#4135)
2018-07-13 17:27:39 -07:00
perf_context_test.cc
Per-thread unique test db names (#4135)
2018-07-13 17:27:39 -07:00
pinned_iterators_manager.h
Change RocksDB License
2017-07-15 16:11:23 -07:00
plain_table_db_test.cc
Per-thread unique test db names (#4135)
2018-07-13 17:27:39 -07:00
pre_release_callback.h
Fix pre_release callback argument list.
2018-04-05 11:12:16 -07:00
prefix_test.cc
Per-thread unique test db names (#4135)
2018-07-13 17:27:39 -07:00
range_del_aggregator_test.cc
Relax VersionStorageInfo::GetOverlappingInputs check (#4050)
2018-07-13 17:42:38 -07:00
range_del_aggregator.cc
Add SST ingestion to ldb (#4205)
2018-08-21 17:19:17 -07:00
range_del_aggregator.h
Support range deletion tombstones in IngestExternalFile SSTs (#3778)
2018-07-13 22:43:09 -07:00
read_callback.h
WriteUnPrepared Txn: Disable seek to snapshot optimization (#3955)
2018-06-27 12:23:07 -07:00
repair_test.cc
fix memory leak in two_level_iterator
2018-04-15 17:26:26 -07:00
repair.cc
Relax VersionStorageInfo::GetOverlappingInputs check (#4050)
2018-07-13 17:42:38 -07:00
snapshot_checker.h
Comment out unused variables
2018-03-05 13:13:41 -08:00
snapshot_impl.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
snapshot_impl.h
BlobDB: Fix VisibleToActiveSnapshot() (#4236)
2018-08-24 14:38:20 -07:00
table_cache.cc
Relax VersionStorageInfo::GetOverlappingInputs check (#4050)
2018-07-13 17:42:38 -07:00
table_cache.h
Relax VersionStorageInfo::GetOverlappingInputs check (#4050)
2018-07-13 17:42:38 -07:00
table_properties_collector_test.cc
Should only decode restart points for uncompressed blocks (#3996)
2018-06-15 19:26:58 -07:00
table_properties_collector.cc
Comment out unused variables
2018-03-05 13:13:41 -08:00
table_properties_collector.h
Comment out unused variables
2018-03-05 13:13:41 -08:00
transaction_log_impl.cc
WriteUnPrepared: Add support for recovering WriteUnprepared transactions (#4078)
2018-07-06 17:59:13 -07:00
transaction_log_impl.h
WritePrepared Txn: Refactor conf params
2017-11-10 17:28:12 -08:00
version_builder_test.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
version_builder.cc
Delay verify compaction output table (#3979)
2018-06-15 12:42:53 -07:00
version_builder.h
Move prefix_extractor to MutableCFOptions
2018-05-21 14:43:11 -07:00
version_edit_test.cc
Skip deleted WALs during recovery
2018-05-03 15:43:09 -07:00
version_edit.cc
Specify the underlying type of enums.
2018-05-23 16:12:59 -07:00
version_edit.h
Range deletion performance improvements + cleanup (#4014)
2018-07-12 14:42:39 -07:00
version_set_test.cc
Relax VersionStorageInfo::GetOverlappingInputs check (#4050)
2018-07-13 17:42:38 -07:00
version_set.cc
Fix a bug in MANIFEST group commit (#4157)
2018-07-24 09:44:14 -07:00
version_set.h
Allow ttl to be changed dynamically (#4133)
2018-07-16 14:27:53 -07:00
wal_manager_test.cc
Per-thread unique test db names (#4135)
2018-07-13 17:27:39 -07:00
wal_manager.cc
Add file name info to SequentialFileReader. (#4026)
2018-06-21 08:42:24 -07:00
wal_manager.h
Fix memleak when DB::DeleteFile()
2018-01-11 18:57:33 -08:00
write_batch_base.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
write_batch_internal.h
WriteUnPrepared: Add new WAL marker kTypeBeginUnprepareXID (#4069)
2018-06-28 18:58:29 -07:00
write_batch_test.cc
WriteUnPrepared: Add support for recovering WriteUnprepared transactions (#4078)
2018-07-06 17:59:13 -07:00
write_batch.cc
Add GCC 8 to Travis (#3433)
2018-07-13 10:58:06 -07:00
write_callback_test.cc
Per-thread unique test db names (#4135)
2018-07-13 17:27:39 -07:00
write_callback.h
Change RocksDB License
2017-07-15 16:11:23 -07:00
write_controller_test.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
write_controller.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
write_controller.h
Change RocksDB License
2017-07-15 16:11:23 -07:00
write_thread.cc
Fix write get stuck when pipelined write is enabled (#4143)
2018-07-18 10:18:50 -07:00
write_thread.h
Fix write get stuck when pipelined write is enabled (#4143)
2018-07-18 10:18:50 -07:00