rocksdb/utilities
Yanqin Jin 5237b39d2e Fix assertion error during compaction with write-prepared txn enabled (#9105)
Summary:
Pull Request resolved: https://github.com/facebook/rocksdb/pull/9105

The user contract of SingleDelete is that: a SingleDelete can only be issued to
a key that exists and has NOT been updated. For example, application can insert
one key `key`, and uses a SingleDelete to delete it in the future. The `key`
cannot be updated or removed using Delete.
In reality, especially when write-prepared transaction is being used, things
can get tricky. For example, a prepared transaction already writes `key` to the
memtable after a successful Prepare(). Afterwards, should the transaction
rollback, it will insert a Delete into the memtable to cancel out the prior
Put. Consider the following sequence of operations.

```
// operation sequence 1
Begin txn
Put(key)
Prepare()
Flush()

Rollback txn
Flush()
```

There will be two SSTs resulting from above. One of the contains a PUT, while
the second one contains a Delete. It is also known that releasing a snapshot
can lead to an L0 containing only a SD for a particular key. Consider the
following operations following the above block.

```
// operation sequence 2
db->Put(key)
db->SingleDelete(key)
Flush()
```

The operation sequence 2 can result in an L0 with only the SD.

Should there be a snapshot for conflict checking created before operation
sequence 1, then an attempt to compact the db may hit the assertion failure
below, because ikey_.type is Delete (from a rollback).

```
else if (clear_and_output_next_key_) {
  assert(ikey_.type == kTypeValue || ikey_.type == kTypeBlobIndex);
}
```

To fix the assertion failure, we can skip the SingleDelete if we detect an
earlier Delete in the same snapshot interval.

Reviewed By: ltamasi

Differential Revision: D32056848

fbshipit-source-id: 23620a91e28562d91c45cf7e95f414b54b729748
2021-11-05 15:29:18 -07:00
..
backupable Skip directory fsync for filesystem btrfs (#8903) 2021-11-03 12:21:27 -07:00
blob_db Skip directory fsync for filesystem btrfs (#8903) 2021-11-03 12:21:27 -07:00
cassandra Fix and detect headers with missing dependencies (#8893) 2021-09-10 10:00:26 -07:00
checkpoint Skip directory fsync for filesystem btrfs (#8903) 2021-11-03 12:21:27 -07:00
compaction_filters Make MergeOperator+CompactionFilter/Factory into Customizable Classes (#8481) 2021-08-06 08:27:25 -07:00
convenience Add a SystemClock class to capture the time functions of an Env (#7858) 2021-01-25 22:09:11 -08:00
leveldb_options Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
memory Make types of Immutable/Mutable Options fields match that of the underlying Option (#8176) 2021-04-22 20:43:54 -07:00
merge_operators Replace most typedef with using= (#8751) 2021-09-07 11:31:59 -07:00
option_change_migration Fix a minor issue with initializing the test path (#8555) 2021-07-23 08:38:45 -07:00
options Allow WAL dir to change with db dir (#8582) 2021-07-30 12:16:44 -07:00
persistent_cache Improve support for using regexes (#8740) 2021-09-07 13:05:23 -07:00
simulator_cache Make TraceRecord and Replayer public (#8611) 2021-08-11 19:32:46 -07:00
table_properties_collectors Make WalFilter, SstPartitionerFactory, FileChecksumGenFactory, and TableProperties Customizable (#8638) 2021-09-28 05:32:02 -07:00
trace Fix and detect headers with missing dependencies (#8893) 2021-09-10 10:00:26 -07:00
transactions Fix assertion error during compaction with write-prepared txn enabled (#9105) 2021-11-05 15:29:18 -07:00
ttl Cleanup includes in dbformat.h (#8930) 2021-09-29 04:04:40 -07:00
write_batch_with_index Some code cleanup (#9003) 2021-10-11 18:14:10 -07:00
cache_dump_load_impl.cc Introduce a mechanism to dump out blocks from block cache and re-insert to secondary cache (#8912) 2021-10-07 11:42:31 -07:00
cache_dump_load_impl.h Initialize cache dumper DumpUnit in constructor (#9014) 2021-10-11 13:05:35 -07:00
cache_dump_load.cc Introduce a mechanism to dump out blocks from block cache and re-insert to secondary cache (#8912) 2021-10-07 11:42:31 -07:00
compaction_filters.cc Make MergeOperator+CompactionFilter/Factory into Customizable Classes (#8481) 2021-08-06 08:27:25 -07:00
debug.cc In ParseInternalKey(), include corrupt key info in Status (#7515) 2020-10-28 10:12:58 -07:00
env_librados_test.cc Fix EnvLibrados and add to CI (#9088) 2021-10-29 08:19:03 -07:00
env_librados.cc Fix EnvLibrados and add to CI (#9088) 2021-10-29 08:19:03 -07:00
env_librados.md Update branch name to main in env_librados.md (#8738) 2021-09-01 14:28:58 -07:00
env_mirror_test.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
env_mirror.cc Add new Append API with DataVerificationInfo to Env WritableFile (#8071) 2021-03-19 11:44:13 -07:00
env_timed_test.cc Make env*_test work with ASSERT_STATUS_CHECKED (#7176) 2020-07-28 22:59:48 -07:00
env_timed.cc Make FileSystem a Customizable Class (#8649) 2021-11-02 09:07:11 -07:00
env_timed.h Make FileSystem a Customizable Class (#8649) 2021-11-02 09:07:11 -07:00
fault_injection_env.cc Protect existing files in FaultInjectionTest{Env,FS}::ReopenWritableFile() (#8995) 2021-10-11 16:23:18 -07:00
fault_injection_env.h Protect existing files in FaultInjectionTest{Env,FS}::ReopenWritableFile() (#8995) 2021-10-11 16:23:18 -07:00
fault_injection_fs.cc Skip directory fsync for filesystem btrfs (#8903) 2021-11-03 12:21:27 -07:00
fault_injection_fs.h Skip directory fsync for filesystem btrfs (#8903) 2021-11-03 12:21:27 -07:00
merge_operators.cc Make MergeOperator+CompactionFilter/Factory into Customizable Classes (#8481) 2021-08-06 08:27:25 -07:00
merge_operators.h Make MergeOperator+CompactionFilter/Factory into Customizable Classes (#8481) 2021-08-06 08:27:25 -07:00
object_registry_test.cc Add support to the ObjectRegistry for ManagedObjects (#8658) 2021-09-10 05:21:04 -07:00
object_registry.cc Add support to the ObjectRegistry for ManagedObjects (#8658) 2021-09-10 05:21:04 -07:00
util_merge_operators_test.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
wal_filter.cc Make WalFilter, SstPartitionerFactory, FileChecksumGenFactory, and TableProperties Customizable (#8638) 2021-09-28 05:32:02 -07:00