892a156267
Summary: This pull request adds a README file and a blog post for the Advisor tool. It also adds the missing tests for some Optimizer modules. Some comments are added to the classes being tested for improved readability. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4201 Reviewed By: maysamyabandeh Differential Revision: D9125311 Pulled By: poojam23 fbshipit-source-id: aefcf2f06eaa05490cc2834ef5aa6e21f0d1dc55
2.8 KiB
| title | layout | author | category |
|---|---|---|---|
| Rocksdb Tuning Advisor | post | poojam23 | blog |
The performance of Rocksdb is contingent on its tuning. However, because of the complexity of its underlying technology and a large number of configurable parameters, a good configuration is sometimes hard to obtain. The aim of the python command-line tool, Rocksdb Advisor, is to automate the process of suggesting improvements in the configuration based on advice from Rocksdb experts.
Overview
Experts share their wisdom as rules comprising of conditions and suggestions in the INI format (refer rules.ini). Users provide the Rocksdb configuration that they want to improve upon (as the familiar Rocksdb OPTIONS file — example) and the path of the file which contains Rocksdb logs and statistics. The Advisor creates appropriate DataSource objects (for Rocksdb logs, options, statistics etc.) and provides them to the Rules Engine. The Rules uses rules from experts to parse data-sources and trigger appropriate rules. The Advisor's output gives information about which rules were triggered, why they were triggered and what each of them suggests. Each suggestion provided by a triggered rule advises some action on a Rocksdb configuration option, for example, increase CFOptions.write_buffer_size, set bloom_bits to 2 etc.
Usage
An example command to run the tool:
cd rocksdb/tools/advisor
python3 -m advisor.rule_parser_example --rules_spec=advisor/rules.ini --rocksdb_options=test/input_files/OPTIONS-000005 --log_files_path_prefix=test/input_files/LOG-0 --stats_dump_period_sec=20
Sample output where a Rocksdb log-based rule has been triggered :
Rule: stall-too-many-memtables
LogCondition: stall-too-many-memtables regex: Stopping writes because we have \d+ immutable memtables \(waiting for flush\), max_write_buffer_number is set to \d+
Suggestion: inc-bg-flush option : DBOptions.max_background_flushes action : increase suggested_values : ['2']
Suggestion: inc-write-buffer option : CFOptions.max_write_buffer_number action : increase
scope: col_fam:
{'default'}
Read more
For more information, refer to advisor.