A library that provides an embeddable, persistent key-value store for fast storage.
Go to file
Maysam Yabandeh dccaf9f03c Turn compaction asserts to runtime check (#5935)
Summary:
Compaction iterator has many assert statements that are active only during test runs. Some rare bugs would show up only at runtime could violate the assert condition but go unnoticed since assert statements are not compiled in release mode. Turning the assert statements to runtime check sone pors and cons:
Pros:
- A bug that would result into incorrect data would be detected early before the incorrect data is written to the disk.

Cons:
- Runtime overhead: which should be negligible since compaction cpu is the minority in the overall cpu usage
- The assert statements might already being violated at runtime, and turning them to runtime failure might result into reliability issues.

The patch takes a conservative step in this direction by logging the assert violations at runtime. If we see any violation reported in logs, we investigate. Otherwise, we can go ahead turning them to runtime error.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/5935

Differential Revision: D18229697

Pulled By: maysamyabandeh

fbshipit-source-id: f1890eca80ccd7cca29737f1825badb9aa8038a8
2019-10-30 13:48:38 -07:00
buckifier Make buckifier python3 compatible (#5922) 2019-10-23 13:52:27 -07:00
build_tools Misc hashing updates / upgrades (#5909) 2019-10-24 17:16:46 -07:00
cache Misc hashing updates / upgrades (#5909) 2019-10-24 17:16:46 -07:00
cmake cmake: s/SNAPPY_LIBRARIES/snappy_LIBRARIES/ (#5687) 2019-08-16 15:49:23 -07:00
coverage Fix interpreter lines for files with python2-only syntax. 2019-07-09 10:51:37 -07:00
db Turn compaction asserts to runtime check (#5935) 2019-10-30 13:48:38 -07:00
docs Blog post for write_unprepared (#5711) 2019-08-15 14:41:13 -07:00
env Add Env::SanitizeEnvOptions (#5885) 2019-10-14 12:25:00 -07:00
examples Apply formatter to recent 200+ commits. (#5830) 2019-09-20 12:04:26 -07:00
file Apply formatter to recent 200+ commits. (#5830) 2019-09-20 12:04:26 -07:00
hdfs Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
include/rocksdb Auto enable Periodic Compactions if a Compaction Filter is used (#5865) 2019-10-29 15:05:51 -07:00
java Fix the rocksjava release Vagrant build on CentOS (#5901) 2019-10-10 17:21:18 -07:00
logging Apply formatter to recent 200+ commits. (#5830) 2019-09-20 12:04:26 -07:00
memory Charge block cache for cache internal usage (#5797) 2019-09-16 15:26:21 -07:00
memtable Misc hashing updates / upgrades (#5909) 2019-10-24 17:16:46 -07:00
monitoring Apply formatter to recent 200+ commits. (#5830) 2019-09-20 12:04:26 -07:00
options Apply formatter to recent 200+ commits. (#5830) 2019-09-20 12:04:26 -07:00
port Fix block cache ID uniqueness for Windows builds (#5844) 2019-10-11 18:19:31 -07:00
table Clean up some filter tests and comments (#5960) 2019-10-24 18:48:16 -07:00
test_util Apply formatter to recent 200+ commits. (#5830) 2019-09-20 12:04:26 -07:00
third-party Refactor/consolidate legacy Bloom implementation details (#5784) 2019-09-16 16:17:09 -07:00
tools crash_test: disable atomic flush with pipelined write (#5986) 2019-10-30 11:36:55 -07:00
trace_replay Misc hashing updates / upgrades (#5909) 2019-10-24 17:16:46 -07:00
util filter_bench not needed for ROCKSDB_LITE (#5978) 2019-10-28 14:12:36 -07:00
utilities Auto enable Periodic Compactions if a Compaction Filter is used (#5865) 2019-10-29 15:05:51 -07:00
.clang-format A script that automatically reformat affected lines 2014-01-14 12:21:24 -08:00
.gitignore Make buckifier python3 compatible (#5922) 2019-10-23 13:52:27 -07:00
.lgtm.yml Create lgtm.yml for LGTM.com C/C++ analysis (#4058) 2018-06-26 12:43:04 -07:00
.travis.yml Remove a webhook due to potential security concern (#5902) 2019-10-10 18:05:16 -07:00
.watchmanconfig Added .watchmanconfig file to rocksdb repo (#5593) 2019-07-19 15:00:33 -07:00
appveyor.yml New API to get all merge operands for a Key (#5604) 2019-08-06 14:26:44 -07:00
AUTHORS Update RocksDB Authors File 2017-10-18 14:42:10 -07:00
CMakeLists.txt FilterPolicy consolidation, part 2/2 (#5966) 2019-10-24 15:44:51 -07:00
CODE_OF_CONDUCT.md Adopt Contributor Covenant 2019-08-29 23:21:01 -07:00
CONTRIBUTING.md Add Code of Conduct 2017-12-05 18:42:35 -08:00
COPYING Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
DEFAULT_OPTIONS_HISTORY.md options.delayed_write_rate use the rate of rate_limiter by default. 2017-05-24 09:58:24 -07:00
defs.bzl Change buckifier to support parameterized dependencies (#5648) 2019-08-02 10:55:17 -07:00
DUMP_FORMAT.md First version of rocksdb_dump and rocksdb_undump. 2015-06-19 16:24:36 -07:00
HISTORY.md Auto enable Periodic Compactions if a Compaction Filter is used (#5865) 2019-10-29 15:05:51 -07:00
INSTALL.md Update the version of the dependencies used by the RocksJava static build (#4761) 2018-12-18 20:25:43 -08:00
issue_template.md Add a template for issues 2017-09-29 11:41:28 -07:00
LANGUAGE-BINDINGS.md LANGUAGE-BINDINGS.md: mention python-rocksdb 2019-03-20 11:10:48 -07:00
LICENSE.Apache Change RocksDB License 2017-07-15 16:11:23 -07:00
LICENSE.leveldb Add back the LevelDB license file 2017-07-16 18:42:18 -07:00
Makefile Include db_stress_tool in rocksdb tools lib (#5950) 2019-10-21 19:40:35 -07:00
README.md Replaced some words (#5877) 2019-10-07 12:28:09 -07:00
ROCKSDB_LITE.md Fix some typos in comments and docs. 2018-03-08 10:27:25 -08:00
src.mk FilterPolicy consolidation, part 2/2 (#5966) 2019-10-24 15:44:51 -07:00
TARGETS FilterPolicy consolidation, part 2/2 (#5966) 2019-10-24 15:44:51 -07:00
thirdparty.inc Fix build jemalloc api (#5470) 2019-06-24 17:40:32 -07:00
USERS.md Add avrio to USERS.md (#5748) 2019-09-15 21:29:09 -07:00
Vagrantfile Adding CentOS 7 Vagrantfile & build script 2018-02-26 15:27:17 -08:00
WINDOWS_PORT.md #5145 , rename port/dirent.h to port/port_dirent.h to avoid compile err when use port dir as header dir output (#5152) 2019-04-04 11:38:19 -07:00

RocksDB: A Persistent Key-Value Store for Flash and RAM Storage

Linux/Mac Build Status Windows Build status PPC64le Build Status

RocksDB is developed and maintained by Facebook Database Engineering Team. It is built on earlier work on LevelDB by Sanjay Ghemawat (sanjay@google.com) and Jeff Dean (jeff@google.com)

This code is a library that forms the core building block for a fast key-value server, especially suited for storing data on flash drives. It has a Log-Structured-Merge-Database (LSM) design with flexible tradeoffs between Write-Amplification-Factor (WAF), Read-Amplification-Factor (RAF) and Space-Amplification-Factor (SAF). It has multi-threaded compactions, making it especially suitable for storing multiple terabytes of data in a single database.

Start with example usage here: https://github.com/facebook/rocksdb/tree/master/examples

See the github wiki for more explanation.

The public interface is in include/. Callers should not include or rely on the details of any other header files in this package. Those internal APIs may be changed without warning.

Design discussions are conducted in https://www.facebook.com/groups/rocksdb.dev/

License

RocksDB is dual-licensed under both the GPLv2 (found in the COPYING file in the root directory) and Apache 2.0 License (found in the LICENSE.Apache file in the root directory). You may select, at your option, one of the above-listed licenses.