rocksdb/utilities
Yanqin Jin 2b367fa8cc Fix bug caused by releasing snapshot(s) during compaction (#8608)
Summary:
In debug mode, we are seeing assertion failure as follows

```
db/compaction/compaction_iterator.cc:980: void rocksdb::CompactionIterator::PrepareOutput(): \
Assertion `ikey_.type != kTypeDeletion && ikey_.type != kTypeSingleDeletion' failed.
```

It is caused by releasing earliest snapshot during compaction between the execution of
`NextFromInput()` and `PrepareOutput()`.

In one case, as demonstrated in unit test `WritePreparedTransaction.ReleaseEarliestSnapshotDuringCompaction_WithSD2`,
incorrect result may be returned by a following range scan if we disable assertion, as in opt compilation
level: the SingleDelete marker's sequence number is zeroed out, but the preceding PUT is also
outputted to the SST file after compaction. Due to the logic of DBIter, the PUT will not be
skipped and will be returned by iterator in range scan. https://github.com/facebook/rocksdb/issues/8661 illustrates what happened.

Fix by taking a more conservative approach: make compaction zero out sequence number only
if key is in the earliest snapshot when the compaction starts.

Another assertion failure is
```
Assertion `current_user_key_snapshot_ == last_snapshot' failed.
```

It's caused by releasing the snapshot between the PUT and SingleDelete during compaction.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/8608

Test Plan: make check

Reviewed By: jay-zhuang

Differential Revision: D30145645

Pulled By: riversand963

fbshipit-source-id: 699f58e66faf70732ad53810ccef43935d3bbe81
2021-08-17 22:14:20 -07:00
..
backupable Make backup restore atomic, with sync option (#8568) 2021-08-06 09:50:21 -07:00
blob_db Make EventListener into a Customizable Class (#8473) 2021-07-27 07:47:02 -07:00
cassandra Make MergeOperator+CompactionFilter/Factory into Customizable Classes (#8481) 2021-08-06 08:27:25 -07:00
checkpoint Revert checkpoint fix (#8607) 2021-08-02 18:29:35 -07:00
compaction_filters Make MergeOperator+CompactionFilter/Factory into Customizable Classes (#8481) 2021-08-06 08:27:25 -07:00
convenience Add a SystemClock class to capture the time functions of an Env (#7858) 2021-01-25 22:09:11 -08:00
leveldb_options Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
memory Make types of Immutable/Mutable Options fields match that of the underlying Option (#8176) 2021-04-22 20:43:54 -07:00
merge_operators Make MergeOperator+CompactionFilter/Factory into Customizable Classes (#8481) 2021-08-06 08:27:25 -07:00
option_change_migration Fix a minor issue with initializing the test path (#8555) 2021-07-23 08:38:45 -07:00
options Allow WAL dir to change with db dir (#8582) 2021-07-30 12:16:44 -07:00
persistent_cache Fix a minor issue with initializing the test path (#8555) 2021-07-23 08:38:45 -07:00
simulator_cache Make TraceRecord and Replayer public (#8611) 2021-08-11 19:32:46 -07:00
table_properties_collectors Add more tests for assert status checked (#7524) 2020-12-22 23:45:58 -08:00
trace Code cleanup for trace replayer (#8652) 2021-08-12 09:22:43 -07:00
transactions Fix bug caused by releasing snapshot(s) during compaction (#8608) 2021-08-17 22:14:20 -07:00
ttl Make MergeOperator+CompactionFilter/Factory into Customizable Classes (#8481) 2021-08-06 08:27:25 -07:00
write_batch_with_index Fix MultiGet with PinnableSlices and Merge for WBWI (#8299) 2021-05-18 14:35:47 -07:00
compaction_filters.cc Make MergeOperator+CompactionFilter/Factory into Customizable Classes (#8481) 2021-08-06 08:27:25 -07:00
debug.cc In ParseInternalKey(), include corrupt key info in Status (#7515) 2020-10-28 10:12:58 -07:00
env_librados_test.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
env_librados.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
env_librados.md Add EnvLibrados - RocksDB Env of RADOS (#1222) 2016-07-21 11:16:34 -07:00
env_mirror_test.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
env_mirror.cc Add new Append API with DataVerificationInfo to Env WritableFile (#8071) 2021-03-19 11:44:13 -07:00
env_timed_test.cc Make env*_test work with ASSERT_STATUS_CHECKED (#7176) 2020-07-28 22:59:48 -07:00
env_timed.cc Make ChRootEnv, EncryptedEnv, and TimedEnv into FileSystems (#7968) 2021-03-15 19:50:11 -07:00
fault_injection_env.cc No elide constructors (#7798) 2020-12-23 16:55:53 -08:00
fault_injection_env.h Fix incorrect Status::NoSpace() status check (#8504) 2021-07-20 18:09:51 -07:00
fault_injection_fs.cc Remove corruption error injection in FaultInjectionTestFS (#8616) 2021-08-04 15:48:54 -07:00
fault_injection_fs.h Fix incorrect Status::NoSpace() status check (#8504) 2021-07-20 18:09:51 -07:00
merge_operators.cc Make MergeOperator+CompactionFilter/Factory into Customizable Classes (#8481) 2021-08-06 08:27:25 -07:00
merge_operators.h Make MergeOperator+CompactionFilter/Factory into Customizable Classes (#8481) 2021-08-06 08:27:25 -07:00
object_registry_test.cc Add ObjectRegistry to ConfigOptions (#8166) 2021-05-11 06:47:22 -07:00
object_registry.cc Fix an race condition during multiple DB opening (#8574) 2021-07-22 13:43:06 -07:00
util_merge_operators_test.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00