andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
rocksdb/db
History
Andrew Kryczka c6abe30ee3 Fix concurrent full purge and WAL recycling (#5900) 
Summary:
We were removing the file from `log_recycle_files_` before renaming it
with `ReuseWritableFile()`. Since `ReuseWritableFile()` occurs outside
the DB mutex, it was possible for a concurrent full purge to sneak in
and delete the file before it could be renamed. Consequently, `SwitchMemtable()`
would fail and the DB would enter read-only mode.

The fix is to hold the old file number in `log_recycle_files_` until
after the file has been renamed. Full purge uses that list to decide
which files to keep, so it can no longer delete a file pending recycling.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/5900

Test Plan: new unit test

Differential Revision: D19771719

Pulled By: ajkr

fbshipit-source-id: 094346349ca3fb499712e62de03905acc30b5ce8
2020-02-18 13:54:13 -08:00
..
compaction
Checksum for each SST file and stores in MANIFEST (#6216)
2020-02-10 15:52:52 -08:00
db_impl
Fix concurrent full purge and WAL recycling (#5900)
2020-02-18 13:54:13 -08:00
arena_wrapped_db_iter.cc
delete superversions in BackgroundCallPurge (#6146)
2019-12-17 13:22:57 -08:00
arena_wrapped_db_iter.h
Refactor ArenaWrappedDBIter into separate files (#5801)
2019-09-13 13:50:43 -07:00
blob_index.h
Mark BlobIndex::DebugString const
2019-12-11 17:19:43 -08:00
builder.cc
Checksum for each SST file and stores in MANIFEST (#6216)
2020-02-10 15:52:52 -08:00
builder.h
Introduce a new storage specific Env API (#5761)
2019-12-13 14:48:41 -08:00
c_test.c
Return NotSupported from WriteBatchWithIndex::DeleteRange (#5393)
2020-02-18 11:18:25 -08:00
c.cc
Add an option to prevent DB::Open() from querying sizes of all sst files (#6353)
2020-02-04 01:27:26 -08:00
column_family_test.cc
Add unit tests for concurrent CF iteration and drop (#6180)
2019-12-18 11:54:35 -08:00
column_family.cc
Avoid create directory for every column families (#6358)
2020-02-03 14:13:39 -08:00
column_family.h
Avoid create directory for every column families (#6358)
2020-02-03 14:13:39 -08:00
compact_files_test.cc
Use aggregate initialization for FlushJobInfo/CompactionJobInfo (#5997)
2019-11-01 11:46:19 -07:00
compacted_db_impl.cc
New API to get all merge operands for a Key (#5604)
2019-08-06 14:26:44 -07:00
compacted_db_impl.h
Use delete to disable automatic generated methods. (#5009)
2019-09-11 18:09:00 -07:00
comparator_db_test.cc
Checksum for each SST file and stores in MANIFEST (#6216)
2020-02-10 15:52:52 -08:00
convenience.cc
Introduce a new storage specific Env API (#5761)
2019-12-13 14:48:41 -08:00
corruption_test.cc
Fail fast in paranoid mode when LoadTableHandlers fail during recovering (#6368)
2020-02-14 08:17:10 -08:00
cuckoo_table_db_test.cc
Fail fast in paranoid mode when LoadTableHandlers fail during recovering (#6368)
2020-02-14 08:17:10 -08:00
db_basic_test.cc
Ensure all MultiGet IO errors are propagated to user (#6403)
2020-02-11 17:27:22 -08:00
db_blob_index_test.cc
Disable blob iterator test with max_sequential_skip_in_iterations==0 in LITE mode (#6052)
2019-11-19 15:02:41 -08:00
db_block_cache_test.cc
Small tidy and speed up of the travis build (#6181)
2019-12-17 13:56:45 -08:00
db_bloom_filter_test.cc
Fix db_bloom_filter_test clang LITE build (#6340)
2020-01-29 12:57:48 -08:00
db_compaction_filter_test.cc
Shorten certain test names to avoid infra failure (#6352)
2020-01-30 23:10:24 -08:00
db_compaction_test.cc
Make DBCompactionTest.SkipStatsUpdateTest more robust (#6306)
2020-01-21 12:55:55 -08:00
db_dynamic_level_test.cc
Fix flaky DBDynamicLevelTest.DynamicLevelMaxBytesBase2 (#4668)
2018-11-12 16:42:16 -08:00
db_encryption_test.cc
Fix EncryptedEnv assert (#5735)
2019-09-05 17:21:42 -07:00
db_filesnapshot.cc
Support concurrent CF iteration and drop (#6147)
2019-12-12 19:04:48 -08:00
db_flush_test.cc
Atomic flush rollback once on failure (#6385)
2020-02-07 10:52:10 -08:00
db_info_dumper.cc
Apply formatter to recent 200+ commits. (#5830)
2019-09-20 12:04:26 -07:00
db_info_dumper.h
Change RocksDB License
2017-07-15 16:11:23 -07:00
db_inplace_update_test.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
db_io_failure_test.cc
Disable DBIOFailureTest.NoSpaceCompactRange in LITE (#4596)
2018-10-29 14:36:31 -07:00
db_iter_stress_test.cc
Move some memory related files from util/ to memory/ (#5382)
2019-05-30 17:44:09 -07:00
db_iter_test.cc
Move some memory related files from util/ to memory/ (#5382)
2019-05-30 17:44:09 -07:00
db_iter.cc
Add ReadOptions.auto_prefix_mode (#6314)
2020-01-28 14:44:05 -08:00
db_iter.h
Add ReadOptions.auto_prefix_mode (#6314)
2020-01-28 14:44:05 -08:00
db_iterator_test.cc
Add unit tests for concurrent CF iteration and drop (#6180)
2019-12-18 11:54:35 -08:00
db_log_iter_test.cc
Apply modernize-use-override (2nd iteration)
2019-02-14 14:41:36 -08:00
db_memtable_test.cc
upgrade gtest 1.7.0 => 1.8.1 for json result writing
2019-09-09 11:24:11 -07:00
db_merge_operand_test.cc
New API to get all merge operands for a Key (#5604)
2019-08-06 14:26:44 -07:00
db_merge_operator_test.cc
unordered_write incompatible with max_successive_merges (#6284)
2020-01-10 16:53:19 -08:00
db_options_test.cc
Adjust thread pool sizes when setting max_background_jobs dynamically (#6300)
2020-01-16 14:35:10 -08:00
db_properties_test.cc
Refactor/consolidate legacy Bloom implementation details (#5784)
2019-09-16 16:17:09 -07:00
db_range_del_test.cc
Return NotSupported from WriteBatchWithIndex::DeleteRange (#5393)
2020-02-18 11:18:25 -08:00
db_sst_test.cc
Be able to read compatible leveldb sst files (#6370)
2020-02-06 10:15:44 -08:00
db_statistics_test.cc
Make statistics's stats_level change thread-safe (#5030)
2019-03-01 10:42:09 -08:00
db_table_properties_test.cc
upgrade gtest 1.7.0 => 1.8.1 for json result writing
2019-09-09 11:24:11 -07:00
db_tailing_iter_test.cc
Remove managed iterator
2018-07-17 14:43:18 -07:00
db_test2.cc
Fix DBTest2.ChangePrefixExtractor LITE build (#6356)
2020-01-31 15:44:14 -08:00
db_test_util.cc
Add oldest snapshot sequence property (#6228)
2020-01-07 08:36:44 -08:00
db_test_util.h
Add oldest snapshot sequence property (#6228)
2020-01-07 08:36:44 -08:00
db_test.cc
Avoid to get manifest file size when recovering from it. (#6369)
2020-02-04 11:39:23 -08:00
db_universal_compaction_test.cc
Shorten certain test names to avoid infra failure (#6352)
2020-01-30 23:10:24 -08:00
db_wal_test.cc
Fix concurrent full purge and WAL recycling (#5900)
2020-02-18 13:54:13 -08:00
db_write_test.cc
Fix queue manipulation in WriteThread::BeginWriteStall() (#6322)
2020-01-23 14:01:28 -08:00
dbformat_test.cc
Move some logging related files to logging/ (#5387)
2019-05-31 17:23:59 -07:00
dbformat.cc
Apply formatter to recent 200+ commits. (#5830)
2019-09-20 12:04:26 -07:00
dbformat.h
Use delete to disable automatic generated methods. (#5009)
2019-09-11 18:09:00 -07:00
deletefile_test.cc
Refactor deletefile_test.cc (#5822)
2019-09-18 16:58:21 -07:00
error_handler_test.cc
Fix a test failure in error_handler_test (#6367)
2020-02-03 18:16:52 -08:00
error_handler.cc
Force a new manifest file if append to current one fails (#6331)
2020-01-30 10:56:29 -08:00
error_handler.h
Fix typos in comments (#4456)
2018-10-04 20:46:50 -07:00
event_helpers.cc
BlobDB GC: add SST <-> oldest blob file referenced mapping (#5903)
2019-10-14 15:21:01 -07:00
event_helpers.h
BlobDB GC: add SST <-> oldest blob file referenced mapping (#5903)
2019-10-14 15:21:01 -07:00
experimental.cc
Organizing rocksdb/db directory
2019-05-31 11:57:01 -07:00
external_sst_file_basic_test.cc
Allow ingesting overlapping files (#5539)
2019-09-13 14:49:47 -07:00
external_sst_file_ingestion_job.cc
Checksum for each SST file and stores in MANIFEST (#6216)
2020-02-10 15:52:52 -08:00
external_sst_file_ingestion_job.h
Introduce a new storage specific Env API (#5761)
2019-12-13 14:48:41 -08:00
external_sst_file_test.cc
wait pending memtable writes on file ingestion or compact range (#6113)
2019-12-12 14:08:02 -08:00
fault_injection_test.cc
Move some logging related files to logging/ (#5387)
2019-05-31 17:23:59 -07:00
file_indexer_test.cc
Move some memory related files from util/ to memory/ (#5382)
2019-05-30 17:44:09 -07:00
file_indexer.cc
Bug when multiple files at one level contains the same smallest key (#6285)
2020-01-13 16:27:42 -08:00
file_indexer.h
Move some memory related files from util/ to memory/ (#5382)
2019-05-30 17:44:09 -07:00
filename_test.cc
Move some logging related files to logging/ (#5387)
2019-05-31 17:23:59 -07:00
flush_job_test.cc
Introduce a new storage specific Env API (#5761)
2019-12-13 14:48:41 -08:00
flush_job.cc
Checksum for each SST file and stores in MANIFEST (#6216)
2020-02-10 15:52:52 -08:00
flush_job.h
Introduce a new storage specific Env API (#5761)
2019-12-13 14:48:41 -08:00
flush_scheduler.cc
Support concurrent CF iteration and drop (#6147)
2019-12-12 19:04:48 -08:00
flush_scheduler.h
Refactor trimming logic for immutable memtables (#5022)
2019-08-23 13:55:34 -07:00
forward_iterator_bench.cc
simplify include directive involving inttypes (#5402)
2019-06-06 13:56:07 -07:00
forward_iterator.cc
delete superversions in BackgroundCallPurge (#6146)
2019-12-17 13:22:57 -08:00
forward_iterator.h
Move some memory related files from util/ to memory/ (#5382)
2019-05-30 17:44:09 -07:00
import_column_family_job.cc
Checksum for each SST file and stores in MANIFEST (#6216)
2020-02-10 15:52:52 -08:00
import_column_family_job.h
Introduce a new storage specific Env API (#5761)
2019-12-13 14:48:41 -08:00
import_column_family_test.cc
Apply formatter to recent 200+ commits. (#5830)
2019-09-20 12:04:26 -07:00
internal_stats.cc
Remove inaccurate code comment (#6274)
2020-01-08 17:51:42 -08:00
internal_stats.h
Add oldest snapshot sequence property (#6228)
2020-01-07 08:36:44 -08:00
job_context.h
WritePrepared: Fix visible key compacted out by compaction (#4883)
2019-01-15 21:34:38 -08:00
listener_test.cc
Propagate SST and blob file numbers through the EventListener interface (#5962)
2019-10-24 14:44:15 -07:00
log_format.h
Fix an inaccurate comment (#4315)
2018-08-24 18:13:20 -07:00
log_reader.cc
Divide file_reader_writer.h and .cc (#5803)
2019-09-16 10:33:51 -07:00
log_reader.h
Avoid to get manifest file size when recovering from it. (#6369)
2020-02-04 11:39:23 -08:00
log_test.cc
Introduce a new storage specific Env API (#5761)
2019-12-13 14:48:41 -08:00
log_writer.cc
Divide file_reader_writer.h and .cc (#5803)
2019-09-16 10:33:51 -07:00
log_writer.h
Use delete to disable automatic generated methods. (#5009)
2019-09-11 18:09:00 -07:00
logs_with_prep_tracker.cc
Skip deleted WALs during recovery
2018-05-03 15:43:09 -07:00
logs_with_prep_tracker.h
Skip deleted WALs during recovery
2018-05-03 15:43:09 -07:00
lookup_key.h
Avoid user key copying for Get/Put/Write with user-timestamp (#5502)
2019-07-25 15:27:39 -07:00
malloc_stats.cc
Support jemalloc compiled with --with-jemalloc-prefix (#5521)
2019-07-02 12:07:01 -07:00
malloc_stats.h
Change RocksDB License
2017-07-15 16:11:23 -07:00
manual_compaction_test.cc
Move some memory related files from util/ to memory/ (#5382)
2019-05-30 17:44:09 -07:00
memtable_list_test.cc
Introduce a new storage specific Env API (#5761)
2019-12-13 14:48:41 -08:00
memtable_list.cc
Fix a data race related to memtable trimming (#6187)
2019-12-16 13:16:31 -08:00
memtable_list.h
Fix a data race related to memtable trimming (#6187)
2019-12-16 13:16:31 -08:00
memtable.cc
Add ReadOptions.auto_prefix_mode (#6314)
2020-01-28 14:44:05 -08:00
memtable.h
Do not schedule memtable trimming if there is no history (#6177)
2019-12-13 19:11:19 -08:00
merge_context.h
Introduce a new MultiGet batching implementation (#5011)
2019-04-11 14:28:26 -07:00
merge_helper_test.cc
Move some memory related files from util/ to memory/ (#5382)
2019-05-30 17:44:09 -07:00
merge_helper.cc
Fix merging range tombstone covering put during flush/compaction (#5406)
2019-06-04 10:24:14 -07:00
merge_helper.h
Remove v1 RangeDelAggregator (#4778)
2018-12-17 17:33:46 -08:00
merge_operator.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
merge_test.cc
unordered_write incompatible with max_successive_merges (#6284)
2020-01-10 16:53:19 -08:00
obsolete_files_test.cc
Refactor ObsoleteFilesTest to inherit from DBTestBase (#5820)
2019-09-18 11:52:17 -07:00
options_file_test.cc
Organizing rocksdb/db directory
2019-05-31 11:57:01 -07:00
perf_context_test.cc
Move some memory related files from util/ to memory/ (#5382)
2019-05-30 17:44:09 -07:00
pinned_iterators_manager.h
Change RocksDB License
2017-07-15 16:11:23 -07:00
plain_table_db_test.cc
Fail fast in paranoid mode when LoadTableHandlers fail during recovering (#6368)
2020-02-14 08:17:10 -08:00
pre_release_callback.h
WritePrepared: reduce prepared_mutex_ overhead (#5420)
2019-06-10 11:53:31 -07:00
prefix_test.cc
Organizing rocksdb/db directory
2019-05-31 11:57:01 -07:00
range_del_aggregator_bench.cc
Move some memory related files from util/ to memory/ (#5382)
2019-05-30 17:44:09 -07:00
range_del_aggregator_test.cc
Move test related files under util/ to test_util/ (#5377)
2019-05-30 11:25:51 -07:00
range_del_aggregator.cc
Correct pragma once problem with Bazel on Windows (#6321)
2020-01-21 16:12:43 -08:00
range_del_aggregator.h
Correct pragma once problem with Bazel on Windows (#6321)
2020-01-21 16:12:43 -08:00
range_tombstone_fragmenter_test.cc
Move test related files under util/ to test_util/ (#5377)
2019-05-30 11:25:51 -07:00
range_tombstone_fragmenter.cc
Apply formatter to recent 200+ commits. (#5830)
2019-09-20 12:04:26 -07:00
range_tombstone_fragmenter.h
Initialized pinned_pos_ and pinned_seq_pos_ in FragmentedRangeTombstoneIterator (#5720)
2019-09-05 17:30:29 -07:00
read_callback.h
WriteUnPrepared: improve read your own write functionality (#5573)
2019-07-23 08:08:19 -07:00
repair_test.cc
Introduce a new storage specific Env API (#5761)
2019-12-13 14:48:41 -08:00
repair.cc
Checksum for each SST file and stores in MANIFEST (#6216)
2020-02-10 15:52:52 -08:00
snapshot_checker.h
WritePrepared: fix issue with snapshot released during compaction (#4858)
2019-01-16 09:55:32 -08:00
snapshot_impl.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
snapshot_impl.h
Add oldest snapshot sequence property (#6228)
2020-01-07 08:36:44 -08:00
table_cache.cc
Ensure all MultiGet IO errors are propagated to user (#6403)
2020-02-11 17:27:22 -08:00
table_cache.h
Introduce a new storage specific Env API (#5761)
2019-12-13 14:48:41 -08:00
table_properties_collector_test.cc
Introduce a new storage specific Env API (#5761)
2019-12-13 14:48:41 -08:00
table_properties_collector.cc
Feature for sampling and reporting compressibility (#4842)
2019-03-18 12:15:34 -07:00
table_properties_collector.h
Feature for sampling and reporting compressibility (#4842)
2019-03-18 12:15:34 -07:00
transaction_log_impl.cc
Introduce a new storage specific Env API (#5761)
2019-12-13 14:48:41 -08:00
transaction_log_impl.h
reuse scratch buffer in transaction_log_reader (#5702)
2019-08-26 11:26:29 -07:00
trim_history_scheduler.cc
Support concurrent CF iteration and drop (#6147)
2019-12-12 19:04:48 -08:00
trim_history_scheduler.h
Refactor trimming logic for immutable memtables (#5022)
2019-08-23 13:55:34 -07:00
version_builder_test.cc
Checksum for each SST file and stores in MANIFEST (#6216)
2020-02-10 15:52:52 -08:00
version_builder.cc
Clean up VersionEdit a bit (#6383)
2020-02-07 13:27:06 -08:00
version_builder.h
Introduce a new storage specific Env API (#5761)
2019-12-13 14:48:41 -08:00
version_edit_test.cc
Checksum for each SST file and stores in MANIFEST (#6216)
2020-02-10 15:52:52 -08:00
version_edit.cc
Checksum for each SST file and stores in MANIFEST (#6216)
2020-02-10 15:52:52 -08:00
version_edit.h
Checksum for each SST file and stores in MANIFEST (#6216)
2020-02-10 15:52:52 -08:00
version_set_test.cc
Checksum for each SST file and stores in MANIFEST (#6216)
2020-02-10 15:52:52 -08:00
version_set.cc
Fail fast in paranoid mode when LoadTableHandlers fail during recovering (#6368)
2020-02-14 08:17:10 -08:00
version_set.h
Checksum for each SST file and stores in MANIFEST (#6216)
2020-02-10 15:52:52 -08:00
wal_manager_test.cc
Introduce a new storage specific Env API (#5761)
2019-12-13 14:48:41 -08:00
wal_manager.cc
Introduce a new storage specific Env API (#5761)
2019-12-13 14:48:41 -08:00
wal_manager.h
Introduce a new storage specific Env API (#5761)
2019-12-13 14:48:41 -08:00
write_batch_base.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
write_batch_internal.h
Add insert hints for each writebatch (#5728)
2019-09-12 17:15:18 -07:00
write_batch_test.cc
Return NotSupported from WriteBatchWithIndex::DeleteRange (#5393)
2020-02-18 11:18:25 -08:00
write_batch.cc
WriteUnPrepared: Fix assertion during recovery (#6419)
2020-02-13 18:52:05 -08:00
write_callback_test.cc
WritePrepared: reduce prepared_mutex_ overhead (#5420)
2019-06-10 11:53:31 -07:00
write_callback.h
Change RocksDB License
2017-07-15 16:11:23 -07:00
write_controller_test.cc
Move test related files under util/ to test_util/ (#5377)
2019-05-30 11:25:51 -07:00
write_controller.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
write_controller.h
Change RocksDB License
2017-07-15 16:11:23 -07:00
write_thread.cc
db/write_thread.cc: Initialize state (#6275)
2020-02-07 15:20:38 -08:00
write_thread.h
Option to make write group size configurable (#5759)
2019-09-11 18:28:33 -07:00