75741eb0ce
Summary: Hello and thanks for RocksDB, Here is a PR to add file deletes, renames and ```Flush()```, ```Sync()```, ```Fsync()``` and ```Close()``` to file ops report. The reason is to help tune RocksDB options when using an env/filesystem with high latencies for file level ("metadata") operations, typically seen during ```DB::Open``` (```db_bench -num 0``` also see https://github.com/facebook/rocksdb/pull/7203 where IOTracing does not trace ```DB::Open```). Before: ``` > db_bench -benchmarks updaterandom -num 0 -report_file_operations true ... Entries: 0 ... Num files opened: 12 Num Read(): 6 Num Append(): 8 Num bytes read: 6216 Num bytes written: 6289 ``` After: ``` > db_bench -benchmarks updaterandom -num 0 -report_file_operations true ... Entries: 0 ... Num files opened: 12 Num files deleted: 3 Num files renamed: 4 Num Flush(): 10 Num Sync(): 5 Num Fsync(): 1 Num Close(): 2 Num Read(): 6 Num Append(): 8 Num bytes read: 6216 Num bytes written: 6289 ``` Before: ``` > db_bench -benchmarks updaterandom -report_file_operations true ... Entries: 1000000 ... Num files opened: 18 Num Read(): 396339 Num Append(): 1000058 Num bytes read: 892030224 Num bytes written: 187569238 ``` After: ``` > db_bench -benchmarks updaterandom -report_file_operations true ... Entries: 1000000 ... Num files opened: 18 Num files deleted: 5 Num files renamed: 4 Num Flush(): 1000068 Num Sync(): 9 Num Fsync(): 1 Num Close(): 6 Num Read(): 396339 Num Append(): 1000058 Num bytes read: 892030224 Num bytes written: 187569238 ``` Another example showing how using ```DB::OpenForReadOnly``` reduces file operations compared to ```((Optimistic)Transaction)DB::Open```: ``` > db_bench -benchmarks updaterandom -num 1 > db_bench -benchmarks updaterandom -num 0 -use_existing_db true -readonly true -report_file_operations true ... Entries: 0 ... Num files opened: 8 Num files deleted: 0 Num files renamed: 0 Num Flush(): 0 Num Sync(): 0 Num Fsync(): 0 Num Close(): 0 Num Read(): 13 Num Append(): 0 Num bytes read: 374 Num bytes written: 0 ``` ``` > db_bench -benchmarks updaterandom -num 1 > db_bench -benchmarks updaterandom -num 0 -use_existing_db true -report_file_operations true ... Entries: 0 ... Num files opened: 14 Num files deleted: 3 Num files renamed: 4 Num Flush(): 14 Num Sync(): 5 Num Fsync(): 1 Num Close(): 3 Num Read(): 11 Num Append(): 10 Num bytes read: 7291 Num bytes written: 7357 ``` Pull Request resolved: https://github.com/facebook/rocksdb/pull/8448 Reviewed By: anand1976 Differential Revision: D29333818 Pulled By: zhichao-cao fbshipit-source-id: a06a8c87f799806462319115195b3e94faf5f542 |
||
---|---|---|
.. | ||
advisor | ||
block_cache_analyzer | ||
dump | ||
rdb | ||
analyze_txn_stress_test.sh | ||
auto_sanity_test.sh | ||
backup_db.sh | ||
benchmark_leveldb.sh | ||
benchmark.sh | ||
blob_dump.cc | ||
check_all_python.py | ||
check_format_compatible.sh | ||
CMakeLists.txt | ||
db_bench_tool_test.cc | ||
db_bench_tool.cc | ||
db_bench.cc | ||
db_crashtest.py | ||
db_repl_stress.cc | ||
db_sanity_test.cc | ||
dbench_monitor | ||
Dockerfile | ||
generate_random_db.sh | ||
ingest_external_sst.sh | ||
io_tracer_parser_test.cc | ||
io_tracer_parser_tool.cc | ||
io_tracer_parser_tool.h | ||
io_tracer_parser.cc | ||
ldb_cmd_impl.h | ||
ldb_cmd_test.cc | ||
ldb_cmd.cc | ||
ldb_test.py | ||
ldb_tool.cc | ||
ldb.cc | ||
pflag | ||
reduce_levels_test.cc | ||
regression_test.sh | ||
report_lite_binary_size.sh | ||
restore_db.sh | ||
rocksdb_dump_test.sh | ||
run_flash_bench.sh | ||
run_leveldb.sh | ||
sample-dump.dmp | ||
simulated_hybrid_file_system.cc | ||
simulated_hybrid_file_system.h | ||
sst_dump_test.cc | ||
sst_dump_tool.cc | ||
sst_dump.cc | ||
trace_analyzer_test.cc | ||
trace_analyzer_tool.cc | ||
trace_analyzer_tool.h | ||
trace_analyzer.cc | ||
verify_random_db.sh | ||
write_external_sst.sh | ||
write_stress_runner.py | ||
write_stress.cc |