A library that provides an embeddable, persistent key-value store for fast storage.
Go to file
Anand Ananthabhotla a736255de8 Delete triggered compaction for universal style
Summary:
This is still WIP, but I'm hoping for early feedback on the overall approach.

This patch implements deletion triggered compaction, which till now only
worked for leveled, for universal style. SST files are marked for
compaction by the CompactOnDeletionCollertor table property. This is
expected to be used when free disk space is low and the user wants to
reclaim space by deleting a bunch of keys. The deletions are expected to
be dense. In such a situation, we want to avoid a full compaction due to
its space overhead.

The strategy used in this case is similar to leveled. We pick one file
from the set of files marked for compaction. We then expand the inputs
to a clean cut on the same level, and then pick overlapping files from
the next non-mepty level. Picking files from the next level can cause
the key range to expand, and we opportunistically expand inputs in the
source level to include files wholly in this key range.

The main side effect of this is that it breaks the property of no time
range overlap between levels. This shouldn't break any functionality.
Closes https://github.com/facebook/rocksdb/pull/3860

Differential Revision: D8124397

Pulled By: anand1976

fbshipit-source-id: bfa2a9dd6817930e991b35d3a8e7e61304ed3dcf
2018-05-29 15:44:34 -07:00
buckifier Update buckifier and TARGETS 2018-03-30 14:26:53 -07:00
build_tools Pass -latomic to linker when using clang 2018-04-25 12:13:41 -07:00
cache Fix LRUCache missing null check on destruct 2018-05-29 15:13:09 -07:00
cmake Search paths provided by intel's "tbbvars.sh". 2018-05-07 14:28:36 -07:00
coverage Suppress lint in old files 2018-01-29 12:56:42 -08:00
db Delete triggered compaction for universal style 2018-05-29 15:44:34 -07:00
docs Adding blog post for 5.10.2 release 2018-02-13 11:56:59 -08:00
env Fix Fadvise on closed file when reads use mmap 2018-05-25 10:57:57 -07:00
examples Pinnableslice examples and blog post 2017-08-24 12:26:07 -07:00
hdfs Comment out unused variables 2018-03-05 13:13:41 -08:00
include/rocksdb Exclude seq from index keys 2018-05-25 18:42:43 -07:00
java Fix an issue with unnecessary capture in lambda expressions 2018-05-25 15:12:44 -07:00
memtable Reorder field based on esan data 2018-05-17 17:57:48 -07:00
monitoring Print histogram count and sum in statistics string 2018-05-21 11:12:47 -07:00
options PersistRocksDBOptions() to use WritableFileWriter 2018-05-21 16:42:22 -07:00
port Catchup with posix features 2018-05-24 15:13:04 -07:00
table Check for rep_->table_properties being nullptr 2018-05-29 12:13:55 -07:00
third-party fix some text in comments. 2018-04-10 15:59:24 -07:00
tools LRUCache midpoint insertion 2018-05-24 15:57:33 -07:00
util Exclude seq from index keys 2018-05-25 18:42:43 -07:00
utilities Move prefix_extractor to MutableCFOptions 2018-05-21 14:43:11 -07:00
.clang-format A script that automatically reformat affected lines 2014-01-14 12:21:24 -08:00
.gitignore Remove leftover references to phutil_module_cache 2017-08-23 12:12:21 -07:00
.travis.yml add 4th test_group in travis 2018-03-13 18:57:29 -07:00
appveyor.yml Upgrade Appveyor to VS2017 2018-02-01 13:57:01 -08:00
AUTHORS Update RocksDB Authors File 2017-10-18 14:42:10 -07:00
CMakeLists.txt Change and clarify the relationship between Valid(), status() and Seek*() for all iterators. Also fix some bugs 2018-05-17 02:56:56 -07:00
CODE_OF_CONDUCT.md Add Code of Conduct 2017-12-05 18:42:35 -08:00
CONTRIBUTING.md Add Code of Conduct 2017-12-05 18:42:35 -08:00
COPYING Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
DEFAULT_OPTIONS_HISTORY.md options.delayed_write_rate use the rate of rate_limiter by default. 2017-05-24 09:58:24 -07:00
DUMP_FORMAT.md First version of rocksdb_dump and rocksdb_undump. 2015-06-19 16:24:36 -07:00
HISTORY.md Exclude seq from index keys 2018-05-25 18:42:43 -07:00
INSTALL.md Enable compilation on OpenBSD 2018-03-19 12:30:05 -07:00
issue_template.md Add a template for issues 2017-09-29 11:41:28 -07:00
LANGUAGE-BINDINGS.md Add Nim to the list of language bindings 2018-01-29 09:57:46 -08:00
LICENSE.Apache Change RocksDB License 2017-07-15 16:11:23 -07:00
LICENSE.leveldb Add back the LevelDB license file 2017-07-16 18:42:18 -07:00
Makefile Change and clarify the relationship between Valid(), status() and Seek*() for all iterators. Also fix some bugs 2018-05-17 02:56:56 -07:00
README.md Add dual-license info to README.md 2018-03-06 12:43:51 -08:00
ROCKSDB_LITE.md Fix some typos in comments and docs. 2018-03-08 10:27:25 -08:00
src.mk Add missing test files to src.mk 2018-05-21 09:43:29 -07:00
TARGETS Change and clarify the relationship between Valid(), status() and Seek*() for all iterators. Also fix some bugs 2018-05-17 02:56:56 -07:00
thirdparty.inc Make Windows dep switches compatible with other builds 2018-01-05 14:56:54 -08:00
USERS.md Added ProfaneDB 2017-11-19 10:11:44 -08:00
Vagrantfile Adding CentOS 7 Vagrantfile & build script 2018-02-26 15:27:17 -08:00
WINDOWS_PORT.md Commit both PR and internal code review changes 2015-07-07 16:58:20 -07:00

RocksDB: A Persistent Key-Value Store for Flash and RAM Storage

Linux/Mac Build Status Windows Build status PPC64le Build Status

RocksDB is developed and maintained by Facebook Database Engineering Team. It is built on earlier work on LevelDB by Sanjay Ghemawat (sanjay@google.com) and Jeff Dean (jeff@google.com)

This code is a library that forms the core building block for a fast key value server, especially suited for storing data on flash drives. It has a Log-Structured-Merge-Database (LSM) design with flexible tradeoffs between Write-Amplification-Factor (WAF), Read-Amplification-Factor (RAF) and Space-Amplification-Factor (SAF). It has multi-threaded compactions, making it specially suitable for storing multiple terabytes of data in a single database.

Start with example usage here: https://github.com/facebook/rocksdb/tree/master/examples

See the github wiki for more explanation.

The public interface is in include/. Callers should not include or rely on the details of any other header files in this package. Those internal APIs may be changed without warning.

Design discussions are conducted in https://www.facebook.com/groups/rocksdb.dev/

License

RocksDB is dual-licensed under both the GPLv2 (found in the COPYING file in the root directory) and Apache 2.0 License (found in the LICENSE.Apache file in the root directory). You may select, at your option, one of the above-listed licenses.