rocksdb/utilities
Yi Wu b1ad6ebba8 WritePrepared: fix two versions in compaction see different status for released snapshots (#4890)
Summary:
Fix how CompactionIterator::findEarliestVisibleSnapshots handles released snapshot. It fixing the two scenarios:

Scenario 1:
key1 has two values v1 and v2. There're two snapshots s1 and s2 taken after v1 and v2 are committed. Right after compaction output v2, s1 is released. Now findEarliestVisibleSnapshot may see s1 being released, and return the next snapshot, which is s2. That's larger than v2's earliest visible snapshot, which was s1.
The fix: the only place we check against last snapshot and current key snapshot is when we decide whether to compact out a value if it is hidden by a later value. In the check if we see current snapshot is even larger than last snapshot, we know last snapshot is released, and we are safe to compact out current key.

Scenario 2:
key1 has two values v1 and v2. there are two snapshots s1 and s2 taken after v1 and v2 are committed. During compaction before we process the key, s1 is released. When compaction process v2, snapshot checker may return kSnapshotReleased, and the earliest visible snapshot for v2 become s2. When compaction process v1, snapshot checker may return kIsInSnapshot (for WritePrepared transaction, it could be because v1 is still in commit cache). The result will become inconsistent here.
The fix: remember the set of released snapshots ever reported by snapshot checker, and ignore them when finding result for findEarliestVisibleSnapshot.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/4890

Differential Revision: D13705538

Pulled By: maysamyabandeh

fbshipit-source-id: e577f0d9ee1ff5a6035f26859e56902ecc85a5a4
2019-01-18 17:24:06 -08:00
..
backupable Backup engine support for direct I/O reads (#4640) 2018-11-13 11:17:25 -08:00
blob_db Remove an unused option (#4888) 2019-01-16 11:48:43 -08:00
cassandra fix tombstone collectable test (#4755) 2018-12-07 10:13:54 -08:00
checkpoint Enable checkpoint of read-only db (#4681) 2018-12-07 17:06:02 -08:00
compaction_filters Comment out unused variables 2018-03-05 13:13:41 -08:00
convenience Change RocksDB License 2017-07-15 16:11:23 -07:00
leveldb_options Change RocksDB License 2017-07-15 16:11:23 -07:00
lua Per-thread unique test db names (#4135) 2018-07-13 17:27:39 -07:00
memory Per-thread unique test db names (#4135) 2018-07-13 17:27:39 -07:00
merge_operators Support pragma once in all header files and cleanup some warnings (#4339) 2018-09-05 18:13:31 -07:00
option_change_migration comment unused parameters to turn on -Wunused-parameter flag 2018-04-12 17:59:16 -07:00
options Update all unique/shared_ptr instances to be qualified with namespace std (#4638) 2018-11-09 11:19:58 -08:00
persistent_cache Update all unique/shared_ptr instances to be qualified with namespace std (#4638) 2018-11-09 11:19:58 -08:00
simulator_cache Add path to WritableFileWriter. (#4039) 2018-08-23 10:12:58 -07:00
table_properties_collectors Reduce runtime of compact_on_deletion_collector_test (#4779) 2018-12-13 14:47:08 -08:00
trace Add the max trace file size limitation option to Tracing (#4610) 2018-11-27 14:27:05 -08:00
transactions WritePrepared: fix two versions in compaction see different status for released snapshots (#4890) 2019-01-18 17:24:06 -08:00
ttl Update all unique/shared_ptr instances to be qualified with namespace std (#4638) 2018-11-09 11:19:58 -08:00
write_batch_with_index Revert "BaseDeltaIterator: always check valid() before accessing key(… (#4744) 2018-12-03 23:38:27 -08:00
debug.cc Remove v1 RangeDelAggregator (#4778) 2018-12-17 17:33:46 -08:00
env_librados_test.cc Update all unique/shared_ptr instances to be qualified with namespace std (#4638) 2018-11-09 11:19:58 -08:00
env_librados.cc Suppress lint in old files 2018-01-29 12:56:42 -08:00
env_librados.md Add EnvLibrados - RocksDB Env of RADOS (#1222) 2016-07-21 11:16:34 -07:00
env_mirror_test.cc Update all unique/shared_ptr instances to be qualified with namespace std (#4638) 2018-11-09 11:19:58 -08:00
env_mirror.cc Update all unique/shared_ptr instances to be qualified with namespace std (#4638) 2018-11-09 11:19:58 -08:00
env_timed_test.cc fix memory leak in two_level_iterator 2018-04-15 17:26:26 -07:00
env_timed.cc Update all unique/shared_ptr instances to be qualified with namespace std (#4638) 2018-11-09 11:19:58 -08:00
merge_operators.h Support StringAppendOperator(delimiter_char) constructor in java-api 2018-03-08 16:17:47 -08:00
object_registry_test.cc fix memory leak in two_level_iterator 2018-04-15 17:26:26 -07:00
util_merge_operators_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00