rocksdb/java
Yanqin Jin 51ebe09f9a Handle rename() failure in non-local FS (#8192)
Summary:
In a distributed environment, a file `rename()` operation can succeed on server (remote)
side, but the client can somehow return non-ok status to RocksDB. Possible reasons include
network partition, connection issue, etc. This happens in `rocksdb::SetCurrentFile()`, which
can be called in `LogAndApply() -> ProcessManifestWrites()` if RocksDB tries to switch to a
new MANIFEST. We currently always delete the new MANIFEST if an error occurs.

This is problematic in distributed world. If the server-side successfully updates the CURRENT
file via renaming, then a subsequent `DB::Open()` will try to look for the new MANIFEST and fail.

As a fix, we can track the execution result of IO operations on the new MANIFEST.
- If IO operations on the new MANIFEST fail, then we know the CURRENT must point to the original
  MANIFEST. Therefore, it is safe to remove the new MANIFEST.
- If IO operations on the new MANIFEST all succeed, but somehow we end up in the clean up
  code block, then we do not know whether CURRENT points to the new or old MANIFEST. (For local
  POSIX-compliant FS, it should still point to old MANIFEST, but it does not matter if we keep the
  new MANIFEST.) Therefore, we keep the new MANIFEST.
    - Any future `LogAndApply()` will switch to a new MANIFEST and update CURRENT.
    - If process reopens the db immediately after the failure, then the CURRENT file can point
      to either the new MANIFEST or the old one, both of which exist. Therefore, recovery can
      succeed and ignore the other.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/8192

Test Plan: make check

Reviewed By: zhichao-cao

Differential Revision: D27804648

Pulled By: riversand963

fbshipit-source-id: 9c16f2a5ce41bc6aadf085e48449b19ede8423e4
2021-04-19 20:29:21 -07:00
..
benchmark/src/main/java/org/rocksdb/benchmark Apply formatter on recent 45 commits. (#5827) 2019-09-19 12:34:17 -07:00
crossbuild Automatically number the Maven artifacts (#7219) 2020-08-06 14:13:30 -07:00
jmh Improve RocksJava Comparator (#6252) 2020-02-03 12:30:13 -08:00
rocksjni Add sample_for_compression results to table properties (#8139) 2021-03-31 18:21:50 -07:00
samples/src/main/java fix java sample typo and replace deprecated code with latest (#7906) 2021-02-01 14:45:34 -08:00
src Handle rename() failure in non-local FS (#8192) 2021-04-19 20:29:21 -07:00
CMakeLists.txt support getUsage and getPinnedUsage in JavaAPI for Cache (#7925) 2021-03-17 09:30:33 -07:00
HISTORY-JAVA.md Update JAVA-HISTORY.md for v3.13 2015-08-04 18:12:58 -07:00
jdb_bench.sh Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
Makefile support getUsage and getPinnedUsage in JavaAPI for Cache (#7925) 2021-03-17 09:30:33 -07:00
pom.xml.template Update the versions of the test dependencies used for RocksJava (#7805) 2021-01-13 16:01:38 -08:00
RELEASE.md Add shared library for musl-libc (#3143) 2019-11-26 18:24:09 -08:00