andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
rocksdb/tools
History
Zhongyi Xie a703432808 MaxFileSizeForLevel: adjust max_file_size for dynamic level compaction 
Summary:
`MutableCFOptions::RefreshDerivedOptions` always assume base level is L1, which is not true when `level_compaction_dynamic_level_bytes=true` and Level based compaction is used.
This PR fixes this by recomputing `max_file_size` at query time (in `MaxFileSizeForLevel`)
Fixes https://github.com/facebook/rocksdb/issues/3229

In master:

```
Level Files Size(MB)
--------------------
  0       14      846
  1        0        0
  2        0        0
  3        0        0
  4        0        0
  5       15      366
  6       11      481
Cumulative compaction: 3.83 GB write, 2.27 GB read
```
In branch:
```
Level Files Size(MB)
--------------------
  0        9      544
  1        0        0
  2        0        0
  3        0        0
  4        0        0
  5        0        0
  6      445      935
Cumulative compaction: 2.91 GB write, 1.46 GB read
```

db_bench command used:
```
./db_bench --benchmarks="fillrandom,deleterandom,fillrandom,levelstats,stats" --statistics -deletes=5000 -db=tmp -compression_type=none --num=20000 -value_size=100000 -level_compaction_dynamic_level_bytes=true -target_file_size_base=2097152 -target_file_size_multiplier=2
```
Closes https://github.com/facebook/rocksdb/pull/3755

Differential Revision: D7721381

Pulled By: miasantreble

fbshipit-source-id: 39afb8503190bac3b466adf9bbf2a9b3655789f8
2018-05-03 16:42:13 -07:00
..
dump
fix gflags namespace
2017-12-01 10:42:05 -08:00
rdb
Fix /bin/bash shebangs
2017-08-03 15:56:46 -07:00
auto_sanity_test.sh
Suppress lint in old files
2018-01-29 12:56:42 -08:00
benchmark_leveldb.sh
Suppress lint in old files
2018-01-29 12:56:42 -08:00
benchmark.sh
benchmark.sh to use --max_background_job
2018-03-20 18:57:55 -07:00
blob_dump.cc
comment unused parameters to turn on -Wunused-parameter flag
2018-04-12 17:59:16 -07:00
check_format_compatible.sh
Add 5.11 and 5.12 to tools/check_format_compatible.sh
2018-03-23 12:43:06 -07:00
CMakeLists.txt
cmake support for linux and osx (#1358)
2016-09-28 11:53:15 -07:00
db_bench_tool_test.cc
fix gflags namespace
2017-12-01 10:42:05 -08:00
db_bench_tool.cc
MaxFileSizeForLevel: adjust max_file_size for dynamic level compaction
2018-05-03 16:42:13 -07:00
db_bench.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
db_crashtest.py
Second attempt at db_stress crash-recovery verification
2018-04-30 12:27:34 -07:00
db_repl_stress.cc
comment unused parameters to turn on -Wunused-parameter flag
2018-04-12 17:59:16 -07:00
db_sanity_test.cc
Change RocksDB License
2017-07-15 16:11:23 -07:00
db_stress.cc
Adjust pread/pwrite to return Status
2018-05-01 13:42:46 -07:00
dbench_monitor
Fix /bin/bash shebangs
2017-08-03 15:56:46 -07:00
Dockerfile
adding docker build script and dockerfile
2015-05-22 16:03:39 -07:00
generate_random_db.sh
Fix /bin/bash shebangs
2017-08-03 15:56:46 -07:00
ldb_cmd_impl.h
WritePrepared Txn: add write_committed option to dump_wal
2018-04-07 21:56:42 -07:00
ldb_cmd_test.cc
fix memory leak in two_level_iterator
2018-04-15 17:26:26 -07:00
ldb_cmd.cc
comment unused parameters to turn on -Wunused-parameter flag
2018-04-12 17:59:16 -07:00
ldb_test.py
Suppress lint in old files
2018-01-29 12:56:42 -08:00
ldb_tool.cc
Comment out unused variables
2018-03-05 13:13:41 -08:00
ldb.cc
comment unused parameters to turn on -Wunused-parameter flag
2018-04-12 17:59:16 -07:00
pflag
Fix /bin/bash shebangs
2017-08-03 15:56:46 -07:00
reduce_levels_test.cc
fix memory leak in two_level_iterator
2018-04-15 17:26:26 -07:00
regression_test.sh
Suppress lint in old files
2018-01-29 12:56:42 -08:00
report_lite_binary_size.sh
Legocastle job to report lite build binary size to scuba
2018-02-15 17:27:24 -08:00
rocksdb_dump_test.sh
Suppress lint in old files
2018-01-29 12:56:42 -08:00
run_flash_bench.sh
Fix /bin/bash shebangs
2017-08-03 15:56:46 -07:00
run_leveldb.sh
Fix /bin/bash shebangs
2017-08-03 15:56:46 -07:00
sample-dump.dmp
First version of rocksdb_dump and rocksdb_undump.
2015-06-19 16:24:36 -07:00
sst_dump_test.cc
fix memory leak in two_level_iterator
2018-04-15 17:26:26 -07:00
sst_dump_tool_imp.h
make sst_dump compression size command consistent
2017-08-11 16:03:44 -07:00
sst_dump_tool.cc
Comment out unused variables
2018-03-05 13:13:41 -08:00
sst_dump.cc
comment unused parameters to turn on -Wunused-parameter flag
2018-04-12 17:59:16 -07:00
verify_random_db.sh
Fix /bin/bash shebangs
2017-08-03 15:56:46 -07:00
write_stress_runner.py
Suppress lint in old files
2018-01-29 12:56:42 -08:00
write_stress.cc
Compilation fixes for powerpc build, -Wparentheses-equality error and missing header guards
2018-02-09 14:12:43 -08:00