andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
440c7f6306
rocksdb/include/rocksdb
History
Yanqin Jin 06394ff4e7 Fix a bug of CompactionIterator/CompactionFilter using Delete (#9929) 
Summary:
When compaction filter determines that a key should be removed, it updates the internal key's type
to `Delete`. If this internal key is preserved in current compaction but seen by a later compaction
together with `SingleDelete`, it will cause compaction iterator to return Corruption.

To fix the issue, compaction filter should return more information in addition to the intention of removing
a key. Therefore, we add a new `kRemoveWithSingleDelete` to `CompactionFilter::Decision`. Seeing
`kRemoveWithSingleDelete`, compaction iterator will update the op type of the internal key to `kTypeSingleDelete`.

In addition, I updated db_stress_shared_state.[cc|h] so that `no_overwrite_ids_` becomes `const`. It is easier to
reason about thread-safety if accessed from multiple threads. This information is passed to `PrepareTxnDBOptions()`
when calling from `Open()` so that we can set up the rollback deletion type callback for transactions.

Finally, disable compaction filter for multiops_txn because the key removal logic of `DbStressCompactionFilter` does
not quite work with `MultiOpsTxnsStressTest`.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/9929

Test Plan:
make check
make crash_test
make crash_test_with_txn

Reviewed By: anand1976

Differential Revision: D36069678

Pulled By: riversand963

fbshipit-source-id: cedd2f1ba958af59ad3916f1ba6f424307955f92
2022-05-02 13:25:45 -07:00
..
utilities Add blob dump support to the dump command (#9881) 2022-04-21 20:37:07 -07:00
advanced_options.h compression_per_level should be used for flush and changeable (#9658) 2022-03-07 18:06:19 -08:00
c.h Add C API for setting strict_capacity_limit (#9855) 2022-04-19 09:34:02 -07:00
cache_bench_tool.h Allow cache_bench/db_bench to use a custom secondary cache (#8312) 2021-05-19 15:26:18 -07:00
cache.h Expose CacheEntryRole and map keys for block cache stat collections (#9838) 2022-04-14 09:38:55 -07:00
cleanable.h Eliminate unnecessary (slow) block cache Ref()ing in MultiGet (#9899) 2022-04-26 21:59:24 -07:00
compaction_filter.h Fix a bug of CompactionIterator/CompactionFilter using Delete (#9929) 2022-05-02 13:25:45 -07:00
compaction_job_stats.h Update compaction statistics to include the amount of data read from blob files (#8022) 2021-03-04 00:43:48 -08:00
comparator.h Mark destructors as override (#9404) 2022-01-20 08:44:27 -08:00
compression_type.h Move CompressionType to its own header file (#7162) 2020-08-03 15:49:31 -07:00
concurrent_task_limiter.h Some API clarifications (#9080) 2021-11-02 20:30:07 -07:00
configurable.h Improve performance of SliceTransform::AsString (#9401) 2022-01-27 10:05:33 -08:00
convenience.h Specify largest_seqno in VerifyChecksum (#9919) 2022-05-02 10:22:08 -07:00
customizable.h Mark destructors as override (#9404) 2022-01-20 08:44:27 -08:00
data_structure.h Add (Live)FileStorageInfo API (#8968) 2021-10-16 10:04:32 -07:00
db_bench_tool.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
db_dump_tool.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
db_stress_tool.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
db.h Mark GetLiveFilesStorageInfo ready for production use (#9868) 2022-04-20 16:09:34 -07:00
env_encryption.h Some API clarifications (#9080) 2021-11-02 20:30:07 -07:00
env.h Rate-limit automatic WAL flush after each user write (#9607) 2022-03-08 13:19:39 -08:00
experimental.h Add manifest fix-up utility for file temperatures (#9683) 2022-03-18 16:35:51 -07:00
file_checksum.h Mark destructors as override (#9404) 2022-01-20 08:44:27 -08:00
file_system.h Add experimental new FS API AbortIO to cancel read request (#9901) 2022-04-25 14:20:03 -07:00
filter_policy.h Fix a major performance bug in 7.0 re: filter compatibility (#9736) 2022-03-23 10:00:54 -07:00
flush_block_policy.h Some API clarifications (#9080) 2021-11-02 20:30:07 -07:00
functor_wrapper.h Fix and detect headers with missing dependencies (#8893) 2021-09-10 10:00:26 -07:00
io_status.h Combine data members of IOStatus with Status (#9549) 2022-02-22 11:23:01 -08:00
iostats_context.h Add file temperature related counter and bytes stats to and io_stats (#8710) 2021-10-07 14:58:41 -07:00
iterator.h Fix a few documentation errors including in public APIs (#9789) 2022-04-01 10:30:17 -07:00
ldb_tool.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
listener.h Add temperature information to the event listener callbacks (#9591) 2022-02-18 11:23:18 -08:00
memory_allocator.h Make MemoryAllocator into a Customizable class (#8980) 2021-12-17 04:20:47 -08:00
memtablerep.h Mark destructors as override (#9404) 2022-01-20 08:44:27 -08:00
merge_operator.h Fix compile warnings (#9199) 2021-11-24 11:19:06 -08:00
metadata.h Fix FileStorageInfo fields from GetLiveFilesMetaData (#9769) 2022-03-29 14:36:35 -07:00
options.h Add a fail_if_not_bottommost_level to IngestExternalFileOptions (#9849) 2022-04-15 18:12:06 -07:00
perf_context.h Add a PerfContext counter for secondary cache hits (#8685) 2021-08-20 15:17:30 -07:00
perf_level.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
persistent_cache.h Check for and disallow shared key space in block caches (#9172) 2021-11-16 11:16:05 -08:00
rate_limiter.h Remove deprecated option new_table_reader_for_compaction_inputs (#9443) 2022-02-08 19:31:28 -08:00
rocksdb_namespace.h Fix and detect headers with missing dependencies (#8893) 2021-09-10 10:00:26 -07:00
secondary_cache.h Prevent double caching in the compressed secondary cache (#9747) 2022-04-11 13:28:33 -07:00
slice_transform.h Some better API and other comments (#9533) 2022-02-17 18:51:08 -08:00
slice.h Require C++17 (#9481) 2022-02-04 17:13:10 -08:00
snapshot.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
sst_dump_tool.h Add --version and --help to ldb and sst_dump (#6951) 2020-06-09 10:04:01 -07:00
sst_file_manager.h Some API clarifications (#9080) 2021-11-02 20:30:07 -07:00
sst_file_reader.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
sst_file_writer.h Support timestamps in SstFileWriter (#8899) 2021-09-09 18:58:01 -07:00
sst_partitioner.h Mark destructors as override (#9404) 2022-01-20 08:44:27 -08:00
statistics.h Add stats related to async prefetching (#9845) 2022-04-25 21:58:22 -07:00
stats_history.h More refactoring ahead of footer & meta changes (#9240) 2021-12-10 08:13:26 -08:00
status.h Combine data members of IOStatus with Status (#9549) 2022-02-22 11:23:01 -08:00
system_clock.h Fix compile warnings (#9199) 2021-11-24 11:19:06 -08:00
table_properties.h Account memory of big memory users in BlockBasedTable in global memory limit (#9748) 2022-04-06 10:33:00 -07:00
table.h Make initial auto readahead_size configurable (#9836) 2022-04-15 17:28:09 -07:00
thread_status.h Fix and detect headers with missing dependencies (#8893) 2021-09-10 10:00:26 -07:00
threadpool.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
trace_reader_writer.h Update comments, fix typos. (#8721) 2021-08-27 13:16:32 -07:00
trace_record_result.h Add IteratorTraceExecutionResult for iterator related trace records. (#8687) 2021-08-20 15:35:56 -07:00
trace_record.h Add IteratorTraceExecutionResult for iterator related trace records. (#8687) 2021-08-20 15:35:56 -07:00
transaction_log.h Replace most typedef with using= (#8751) 2021-09-07 11:31:59 -07:00
types.h Expose blob file information through the EventListener interface (#8675) 2021-09-16 17:23:36 -07:00
unique_id.h Experimental support for SST unique IDs (#8990) 2021-10-18 23:32:01 -07:00
universal_compaction.h Incremental Space Amp Compactions in Universal Style (#8655) 2021-10-20 10:04:13 -07:00
version.h Update main version.h to NEXT release (7.3) (#9852) 2022-04-18 10:26:21 -07:00
wal_filter.h Fix compile warnings (#9199) 2021-11-24 11:19:06 -08:00
write_batch_base.h Revise APIs related to user-defined timestamp (#8946) 2022-02-01 22:19:01 -08:00
write_batch.h Update protection info on recovered logs data (#9875) 2022-04-28 14:42:00 -07:00
write_buffer_manager.h Account memory of big memory users in BlockBasedTable in global memory limit (#9748) 2022-04-06 10:33:00 -07:00