rocksdb/db/db_impl
Cheng Chang 8c93b16f02 Track WAL in MANIFEST: Update logic for computing min_log_number_to_keep in atomic flush (#7660)
Summary:
The logic for computing min_log_number_to_keep in atomic flush was incorrect.

For example, when all column families are flushed, the min_log_number_to_keep should be the latest new log. But the incorrect logic calls `PrecomputeMinLogNumberToKeepNon2PC` for each column family, and returns the minimum of them. However, `PrecomputeMinLogNumberToKeepNon2PC(cf)` assumes column families other than `cf` are flushed, but in case all column families are flushed, this assumption is incorrect.

Without this fix, the WAL referenced by the computed min_log_number_to_keep may actually contain no unflushed data, so the WAL might have actually been deleted from disk on recovery, then an incorrect error `Corruption: missing WAL` will be reported.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/7660

Test Plan:
run `make crash_test_with_atomic_flush`  on devserver
added a unit test in `db_flush_test`

Reviewed By: riversand963

Differential Revision: D24906265

Pulled By: cheng-chang

fbshipit-source-id: 08deda62e71f67f59e3b7925cdd86dd09bd4f430
2020-11-17 15:55:55 -08:00
..
db_impl_compaction_flush.cc Track WAL in MANIFEST: LogAndApply WAL events to MANIFEST (#7601) 2020-11-06 17:22:36 -08:00
db_impl_debug.cc Periodically flush info log out of application buffer (#7488) 2020-10-01 19:14:14 -07:00
db_impl_experimental.cc Replace reinterpret_cast with static_cast_with_check (#7067) 2020-07-02 19:25:41 -07:00
db_impl_files.cc Track WAL in MANIFEST: Update logic for computing min_log_number_to_keep in atomic flush (#7660) 2020-11-17 15:55:55 -08:00
db_impl_open.cc Add full_history_ts_low_ to FlushJob (#7655) 2020-11-12 18:44:34 -08:00
db_impl_readonly.cc RocksJava - Add errorIfLogFileExists parameter to RocksDB.openReadOnly (#7046) 2020-09-17 15:41:25 -07:00
db_impl_readonly.h RocksJava - Add errorIfLogFileExists parameter to RocksDB.openReadOnly (#7046) 2020-09-17 15:41:25 -07:00
db_impl_secondary.cc Make FileType Public and Replace kLogFile with kWalFile (#7580) 2020-10-22 17:06:20 -07:00
db_impl_secondary.h RocksJava - Add errorIfLogFileExists parameter to RocksDB.openReadOnly (#7046) 2020-09-17 15:41:25 -07:00
db_impl_write.cc Track WAL in MANIFEST: LogAndApply WAL events to MANIFEST (#7601) 2020-11-06 17:22:36 -08:00
db_impl.cc Track WAL in MANIFEST: LogAndApply WAL events to MANIFEST (#7601) 2020-11-06 17:22:36 -08:00
db_impl.h Track WAL in MANIFEST: Update logic for computing min_log_number_to_keep in atomic flush (#7660) 2020-11-17 15:55:55 -08:00
db_secondary_test.cc Fix many tests to run with MEM_ENV and ENCRYPTED_ENV; Introduce a MemoryFileSystem class (#7566) 2020-10-27 10:33:09 -07:00