rocksdb/utilities
Yi Wu b864bc9b5b Blob DB: Improve FIFO eviction
Summary:
Improving blob db FIFO eviction with the following changes,
* Change blob_dir_size to max_db_size. Take into account SST file size when computing DB size.
* FIFO now only take into account live sst files and live blob files. It is normal for disk usage to go over max_db_size because there are obsolete sst files and blob files pending deletion.
* FIFO eviction now also evict TTL blob files that's still open. It doesn't evict non-TTL blob files.
* If FIFO is triggered, it will pass an expiration and the current sequence number to compaction filter. Compaction filter will then filter inlined keys to evict those with an earlier expiration and smaller sequence number. So call LSM FIFO.
* Compaction filter also filter those blob indexes where corresponding blob file is gone.
* Add an event listener to listen compaction/flush event and update sst file size.
* Implement DB::Close() to make sure base db, as well as event listener and compaction filter, destruct before blob db.
* More blob db statistics around FIFO.
* Fix some locking issue when accessing a blob file.
Closes https://github.com/facebook/rocksdb/pull/3556

Differential Revision: D7139328

Pulled By: yiwu-arbug

fbshipit-source-id: ea5edb07b33dfceacb2682f4789bea61de28bbfa
2018-03-06 11:57:42 -08:00
..
backupable Comment out unused variables 2018-03-05 13:13:41 -08:00
blob_db Blob DB: Improve FIFO eviction 2018-03-06 11:57:42 -08:00
cassandra Comment out unused variables 2018-03-05 13:13:41 -08:00
checkpoint Comment out unused variables 2018-03-05 13:13:41 -08:00
compaction_filters Comment out unused variables 2018-03-05 13:13:41 -08:00
convenience Change RocksDB License 2017-07-15 16:11:23 -07:00
date_tiered WritePrepared Txn: Iterator 2017-10-09 17:15:28 -07:00
document Comment out unused variables 2018-03-05 13:13:41 -08:00
geodb Change RocksDB License 2017-07-15 16:11:23 -07:00
leveldb_options Change RocksDB License 2017-07-15 16:11:23 -07:00
lua Suppress unused warnings 2018-02-02 12:27:07 -08:00
memory Change RocksDB License 2017-07-15 16:11:23 -07:00
merge_operators Added bytes XOR merge operator 2018-03-06 10:27:36 -08:00
option_change_migration Change RocksDB License 2017-07-15 16:11:23 -07:00
options Comment out unused variables 2018-03-05 13:13:41 -08:00
persistent_cache Comment out unused variables 2018-03-05 13:13:41 -08:00
redis Comment out unused variables 2018-03-05 13:13:41 -08:00
simulator_cache Comment out unused variables 2018-03-05 13:13:41 -08:00
spatialdb Comment out unused variables 2018-03-05 13:13:41 -08:00
table_properties_collectors Comment out unused variables 2018-03-05 13:13:41 -08:00
transactions WritePrepared Txn: Move DuplicateDetector to util 2018-03-05 23:57:12 -08:00
ttl Comment out unused variables 2018-03-05 13:13:41 -08:00
write_batch_with_index Comment out unused variables 2018-03-05 13:13:41 -08:00
col_buf_decoder.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
col_buf_decoder.h Comment out unused variables 2018-03-05 13:13:41 -08:00
col_buf_encoder.cc fix shift UBSAN error in col_buf_encoder.cc 2018-02-20 16:44:00 -08:00
col_buf_encoder.h Change RocksDB License 2017-07-15 16:11:23 -07:00
column_aware_encoding_exp.cc fix gflags namespace 2017-12-01 10:42:05 -08:00
column_aware_encoding_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
column_aware_encoding_util.cc WritePrepared Txn: fix non-emptied PreparedHeap bug 2018-02-21 13:42:23 -08:00
column_aware_encoding_util.h Change RocksDB License 2017-07-15 16:11:23 -07:00
debug.cc Fix naming in InternalKey 2017-09-12 17:17:42 -07:00
env_librados_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
env_librados.cc Suppress lint in old files 2018-01-29 12:56:42 -08:00
env_librados.md Add EnvLibrados - RocksDB Env of RADOS (#1222) 2016-07-21 11:16:34 -07:00
env_mirror_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
env_mirror.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
env_timed_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
env_timed.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
merge_operators.h Added bytes XOR merge operator 2018-03-06 10:27:36 -08:00
object_registry_test.cc Comment out unused variables 2018-03-05 13:13:41 -08:00
util_merge_operators_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00