9358178edc
Summary: This PR allows RocksDB to run in single-primary, multi-secondary process mode. The writer is a regular RocksDB (e.g. an `DBImpl`) instance playing the role of a primary. Multiple `DBImplSecondary` processes (secondaries) share the same set of SST files, MANIFEST, WAL files with the primary. Secondaries tail the MANIFEST of the primary and apply updates to their own in-memory state of the file system, e.g. `VersionStorageInfo`. This PR has several components: 1. (Originally in #4745). Add a `PathNotFound` subcode to `IOError` to denote the failure when a secondary tries to open a file which has been deleted by the primary. 2. (Similar to #4602). Add `FragmentBufferedReader` to handle partially-read, trailing record at the end of a log from where future read can continue. 3. (Originally in #4710 and #4820). Add implementation of the secondary, i.e. `DBImplSecondary`. 3.1 Tail the primary's MANIFEST during recovery. 3.2 Tail the primary's MANIFEST during normal processing by calling `ReadAndApply`. 3.3 Tailing WAL will be in a future PR. 4. Add an example in 'examples/multi_processes_example.cc' to demonstrate the usage of secondary RocksDB instance in a multi-process setting. Instructions to run the example can be found at the beginning of the source code. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4899 Differential Revision: D14510945 Pulled By: riversand963 fbshipit-source-id: 4ac1c5693e6012ad23f7b4b42d3c374fecbe8886 |
||
---|---|---|
.. | ||
advisor | ||
dump | ||
rdb | ||
analyze_txn_stress_test.sh | ||
auto_sanity_test.sh | ||
benchmark_leveldb.sh | ||
benchmark.sh | ||
blob_dump.cc | ||
check_format_compatible.sh | ||
CMakeLists.txt | ||
db_bench_tool_test.cc | ||
db_bench_tool.cc | ||
db_bench.cc | ||
db_crashtest.py | ||
db_repl_stress.cc | ||
db_sanity_test.cc | ||
db_stress.cc | ||
dbench_monitor | ||
Dockerfile | ||
generate_random_db.sh | ||
ingest_external_sst.sh | ||
ldb_cmd_impl.h | ||
ldb_cmd_test.cc | ||
ldb_cmd.cc | ||
ldb_test.py | ||
ldb_tool.cc | ||
ldb.cc | ||
pflag | ||
reduce_levels_test.cc | ||
regression_test.sh | ||
report_lite_binary_size.sh | ||
rocksdb_dump_test.sh | ||
run_flash_bench.sh | ||
run_leveldb.sh | ||
sample-dump.dmp | ||
sst_dump_test.cc | ||
sst_dump_tool_imp.h | ||
sst_dump_tool.cc | ||
sst_dump.cc | ||
trace_analyzer_test.cc | ||
trace_analyzer_tool.cc | ||
trace_analyzer_tool.h | ||
trace_analyzer.cc | ||
verify_random_db.sh | ||
write_external_sst.sh | ||
write_stress_runner.py | ||
write_stress.cc |