A library that provides an embeddable, persistent key-value store for fast storage.
Go to file
Igor Canadi 7a3577519f Don't artificially inflate L0 score
Summary:
This turns out to be pretty bad because if we prioritize L0->L1 then L1 can grow artificially large, which makes L0->L1 more and more expensive. For example:
256MB @ L0 + 256MB @ L1 --> 512MB @ L1
256MB @ L0 + 512MB @ L1 --> 768MB @ L1
256MB @ L0 + 768MB @ L1 --> 1GB @ L1

....

256MB @ L0 + 10GB @ L1 --> 10.2GB @ L1

At some point we need to start compacting L1->L2 to speed up L0->L1.

Test Plan:
The performance improvement is massive for heavy write workload. This is the benchmark I ran: https://phabricator.fb.com/P19842671. Before this change, the benchmark took 47 minutes to complete. After, the benchmark finished in 2minutes. You can see full results here: https://phabricator.fb.com/P19842674

Also, we ran this diff on MongoDB on RocksDB on one replicaset. Before the change, our initial sync was so slow that it couldn't keep up with primary writes. After the change, the import finished without any issues

Reviewers: dynamike, MarkCallaghan, rven, yhchiang, sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D38637
2015-05-21 11:40:48 -07:00
arcanist_util Integrate Jenkins with Phabricator 2015-04-07 11:56:29 -07:00
build_tools Build for CYGWIN 2015-04-23 21:33:44 -07:00
coverage Fix coverage script 2014-11-03 14:53:00 -08:00
db Don't artificially inflate L0 score 2015-05-21 11:40:48 -07:00
doc Remove seek compaction 2014-06-20 10:23:02 +02:00
examples Fix formatting 2015-02-09 09:53:30 -08:00
hdfs Remove unused parameter in CancelAllBackgroundWork 2015-03-16 21:07:54 -07:00
include Set stats_dump_period_sec to 600 by default 2015-05-21 14:22:16 -04:00
java Bugfix remove deprecated option use which was removed in previous commit 019ecd1932 2015-04-30 23:16:04 +01:00
port Build for CYGWIN 2015-04-23 21:33:44 -07:00
table Add more table properties to EventLogger 2015-05-12 15:53:55 -07:00
third-party Update COMMIT.md 2015-03-30 17:48:16 -07:00
tools Add Size-GB column to benchmark reports 2015-05-02 07:46:12 -07:00
util Set stats_dump_period_sec to 600 by default 2015-05-21 14:22:16 -04:00
utilities API to fetch from both a WriteBatchWithIndex and the db 2015-05-11 14:51:51 -07:00
.arcconfig Integrate Jenkins with Phabricator 2015-04-07 11:56:29 -07:00
.clang-format A script that automatically reformat affected lines 2014-01-14 12:21:24 -08:00
.gitignore run 'make check's rules (and even subtests) in parallel 2015-04-06 12:35:25 -07:00
.travis.yml Don't preinstall jemalloc in Travis 2015-04-24 18:43:07 -07:00
AUTHORS Add AUTHORS file. Fix #203 2014-09-29 10:52:18 -07:00
CONTRIBUTING.md facebook accounts are not required for CLA signers 2014-07-08 05:57:54 -04:00
HISTORY.md [Public API Change] Make DB::GetDbIdentity() be const function. 2015-05-21 11:01:48 -07:00
INSTALL.md Optimize default compile to compilation platform by default 2014-12-15 11:29:41 +01:00
LICENSE Fix copyright year 2014-03-12 12:06:58 -07:00
Makefile Add LDFLAGS to Java static library 2015-05-19 23:04:02 -04:00
PATENTS Update Patent Grant. 2015-04-13 10:33:43 +01:00
README.md Replaced "built on on earlier work" by "built on earlier work" in README.md 2014-09-17 01:16:17 -07:00
ROCKSDB_LITE.md RocksDBLite 2014-04-15 13:39:26 -07:00
src.mk Add more table properties to EventLogger 2015-05-12 15:53:55 -07:00
USERS.md Update USERS.md 2015-05-04 15:27:59 -07:00
Vagrantfile RocksDB on FreeBSD support 2015-02-26 15:19:17 -08:00

RocksDB: A Persistent Key-Value Store for Flash and RAM Storage

Build Status

RocksDB is developed and maintained by Facebook Database Engineering Team. It is built on earlier work on LevelDB by Sanjay Ghemawat (sanjay@google.com) and Jeff Dean (jeff@google.com)

This code is a library that forms the core building block for a fast key value server, especially suited for storing data on flash drives. It has a Log-Structured-Merge-Database (LSM) design with flexible tradeoffs between Write-Amplification-Factor (WAF), Read-Amplification-Factor (RAF) and Space-Amplification-Factor (SAF). It has multi-threaded compactions, making it specially suitable for storing multiple terabytes of data in a single database.

Start with example usage here: https://github.com/facebook/rocksdb/tree/master/examples

See the github wiki for more explanation.

The public interface is in include/. Callers should not include or rely on the details of any other header files in this package. Those internal APIs may be changed without warning.

Design discussions are conducted in https://www.facebook.com/groups/rocksdb.dev/