rocksdb/java
Akanksha Mahajan ae82d91492 Remove corrupted WAL files in kPointRecoveryMode with avoid_flush_duing_recovery set true (#9634)
Summary:
1) In case of non-TransactionDB and avoid_flush_during_recovery = true, RocksDB won't
flush the data from WAL to L0 for all column families if possible. As a
result, not all column families can increase their log_numbers, and
min_log_number_to_keep won't change.
2) For transaction DB (.allow_2pc), even with the flush, there may be old WAL files that it must not delete because they can contain data of uncommitted transactions and min_log_number_to_keep won't change.

If we persist a new MANIFEST with
advanced log_numbers for some column families, then during a second
crash after persisting the MANIFEST, RocksDB will see some column
families' log_numbers larger than the corrupted wal, and the "column family inconsistency" error will be hit, causing recovery to fail.

As a solution,
1. the corrupted WALs whose numbers are larger than the
corrupted wal and smaller than the new WAL will be moved to archive folder.
2. Currently, RocksDB DB::Open() may creates and writes to two new MANIFEST files even before recovery succeeds. This PR buffers the edits in a structure and writes to a new MANIFEST after recovery is successful

Pull Request resolved: https://github.com/facebook/rocksdb/pull/9634

Test Plan:
1. Added new unit tests
                2. make crast_test -j

Reviewed By: riversand963

Differential Revision: D34463666

Pulled By: akankshamahajan15

fbshipit-source-id: e233d3af0ed4e2028ca0cf051e5a334a0fdc9d19
2022-04-11 15:39:31 -07:00
..
benchmark/src/main/java/org/rocksdb/benchmark jni: expose memtable_whole_key_filtering option (#9394) 2022-02-04 16:01:16 -08:00
crossbuild Support C++17 Docker build environments for RocksJava (#9500) 2022-02-17 12:48:38 -08:00
jmh java / jni io_uring support (#9224) 2021-12-15 18:09:25 -08:00
rocksjni Update stats for Read and ReadAsync in random_access_file_reader for async prefetching (#9810) 2022-04-06 14:26:53 -07:00
samples/src/main/java fix java sample typo and replace deprecated code with latest (#7906) 2021-02-01 14:45:34 -08:00
src Remove corrupted WAL files in kPointRecoveryMode with avoid_flush_duing_recovery set true (#9634) 2022-04-11 15:39:31 -07:00
CMakeLists.txt Fix remaining uses of "backupable" (#9792) 2022-04-05 09:52:33 -07:00
HISTORY-JAVA.md Update JAVA-HISTORY.md for v3.13 2015-08-04 18:12:58 -07:00
jdb_bench.sh Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
Makefile Fb 9718 verify checksums is ignored (#9767) 2022-03-29 11:54:54 -07:00
pom.xml.template Update the versions of the test dependencies used for RocksJava (#7805) 2021-01-13 16:01:38 -08:00
RELEASE.md Add shared library for musl-libc (#3143) 2019-11-26 18:24:09 -08:00
understanding_options.md New-style blob option bindings, Java option getter and improve/fix option parsing (#8999) 2021-10-19 09:21:52 -07:00