2013-11-12 19:50:46 +01:00
|
|
|
<!DOCTYPE html>
|
2013-11-15 02:32:29 +01:00
|
|
|
<html prefix="og: http://ogp.me/ns#">
|
2013-11-12 19:50:46 +01:00
|
|
|
<head>
|
|
|
|
|
<meta charset='utf-8'>
|
|
|
|
|
<title>RocksDB | A persistent key-value store for fast storage environments</title>
|
|
|
|
|
<link rel='stylesheet' href='static/rocksdb.css' type='text/css'/>
|
|
|
|
|
<link rel='shortcut icon' href='static/favicon.ico'>
|
|
|
|
|
<meta name='viewport' content='width=device-width'>
|
|
|
|
|
<meta property='og:type' content='website'>
|
|
|
|
|
<meta property='og:title' content='RocksDB | A persistent key-value store for fast storage environments'>
|
|
|
|
|
<meta property='og:description' content='A persistent key-value store for fast storage environments'>
|
2013-11-13 00:13:05 +01:00
|
|
|
<meta property='og:image' content='http://rocksdb.org/static/rocksdb.png'>
|
2013-11-12 19:50:46 +01:00
|
|
|
</head>
|
|
|
|
|
<body>
|
|
|
|
|
|
|
|
|
|
<header class='topbar'><nav class='width'>
|
|
|
|
|
<a href='index.html'><h1>RocksDB</h1></a>
|
|
|
|
|
<ul>
|
|
|
|
|
<li><a href='overview.html'>Overview</a></li>
|
|
|
|
|
<li><a href='https://github.com/facebook/rocksdb/wiki/_pages'>Wiki</a></li>
|
|
|
|
|
<li><a href='https://github.com/facebook/rocksdb'>GitHub</a>
|
|
|
|
|
</ul>
|
|
|
|
|
</nav></header>
|
|
|
|
|
|
|
|
|
|
<header class='hero'><div class='width'>
|
|
|
|
|
<hgroup>
|
|
|
|
|
<h1>
|
|
|
|
|
A persistent key-value store for fast storage environments
|
|
|
|
|
</h1>
|
|
|
|
|
<a class='button' href='overview.html'>Get Started</a>
|
|
|
|
|
</hgroup>
|
|
|
|
|
<aside>
|
2013-11-14 05:40:05 +01:00
|
|
|
<a href='overview.html'>
|
|
|
|
|
<img src='static/rocksdbhero.png' />
|
|
|
|
|
</a>
|
2013-11-12 19:50:46 +01:00
|
|
|
</aside>
|
|
|
|
|
</div></header>
|
|
|
|
|
|
|
|
|
|
<section class='content'><div class='width'>
|
|
|
|
|
|
|
|
|
|
<article>
|
2013-11-14 05:40:05 +01:00
|
|
|
|
|
|
|
|
<h2>What is RocksDB?</h2>
|
|
|
|
|
|
|
|
|
|
<p>
|
|
|
|
|
RocksDB is an embeddable persistent key-value store for fast storage. RocksDB
|
|
|
|
|
can also be the foundation for a client-server database but our current focus is
|
|
|
|
|
on embedded workloads.
|
|
|
|
|
</p>
|
|
|
|
|
|
|
|
|
|
<p>
|
|
|
|
|
RocksDB builds on <a href='https://code.google.com/p/leveldb/'>LevelDB</a> to be
|
|
|
|
|
scalable to run on servers with many CPU cores, to efficiently use fast storage,
|
|
|
|
|
to support IO-bound, in-memory and write-once workloads, and to be flexible to
|
|
|
|
|
allow for innovation.
|
|
|
|
|
</p>
|
|
|
|
|
|
2013-11-15 20:34:04 +01:00
|
|
|
<p>For more background on RocksDB, see <a href="intro.pdf">Dhruba Borthakur's
|
|
|
|
|
introductory talk</a> from the Data @ Scale 2013 conference.</p>
|
|
|
|
|
|
2013-11-14 05:40:05 +01:00
|
|
|
<h2>How does performance compare?</h2>
|
|
|
|
|
|
|
|
|
|
<p>
|
|
|
|
|
We benchmarked LevelDB and found that it was unsuitable for our server
|
|
|
|
|
workloads. The <a
|
|
|
|
|
href='http://leveldb.googlecode.com/svn/trunk/doc/benchmark.html'> benchmark
|
|
|
|
|
results</a> look awesome at first sight, but we quickly realized that those
|
|
|
|
|
results were for a database whose size was smaller than the size of RAM on the
|
|
|
|
|
test machine - where the entire database could fit in the OS page cache. When we
|
|
|
|
|
performed the same benchmarks on a database that was at least 5 times larger
|
|
|
|
|
than main memory, the performance results were dismal.
|
|
|
|
|
</p>
|
|
|
|
|
|
|
|
|
|
<p>
|
|
|
|
|
By contrast, we've published the <a
|
|
|
|
|
href='https://github.com/facebook/rocksdb/wiki/Performance-Benchmarks'> RocksDB
|
|
|
|
|
benchmark results</a> for server side workloads on Flash. We also measured the
|
|
|
|
|
performance of LevelDB on these server-workload benchmarks and found that
|
|
|
|
|
RocksDB solidly outperforms LevelDB for these IO bound workloads. We found that
|
|
|
|
|
LevelDB's single-threaded compaction process was insufficient to drive server
|
|
|
|
|
workloads. We saw frequent write-stalls with LevelDB that caused 99-percentile
|
|
|
|
|
latency to be tremendously large. We found that mmap-ing a file into the OS
|
|
|
|
|
cache introduced performance bottlenecks for reads. We could not make LevelDB
|
|
|
|
|
consume all the IOs offered by the underlying Flash storage.
|
|
|
|
|
</p>
|
|
|
|
|
|
|
|
|
|
<h2>What is RocksDB suitable for?</h2>
|
|
|
|
|
|
|
|
|
|
<p>
|
|
|
|
|
RocksDB can be used by applications that need low latency database accesses. A
|
|
|
|
|
user-facing application that stores the viewing history and state of users of a
|
|
|
|
|
website can potentially store this content on RocksDB. A spam detection
|
|
|
|
|
application that needs fast access to big data sets can use RocksDB. A
|
|
|
|
|
graph-search query that needs to scan a data set in realtime can use RocksDB.
|
|
|
|
|
RocksDB can be used to cache data from Hadoop, thereby allowing applications to
|
|
|
|
|
query Hadoop data in realtime. A message-queue that supports a high number of
|
|
|
|
|
inserts and deletes can use RocksDB.
|
|
|
|
|
</p>
|
|
|
|
|
|
|
|
|
|
<h2>Why is RocksDB open sourced?</h2>
|
|
|
|
|
|
|
|
|
|
<p>
|
|
|
|
|
We are open sourcing this project <a
|
|
|
|
|
href='http://github.com/facebook/rocksdb'>on GitHub</a> because we think it will
|
|
|
|
|
be useful beyond Facebook. We are hoping that software programmers and database
|
|
|
|
|
developers will use, enhance, and customize RocksDB for their use-cases. We
|
|
|
|
|
would also like to engage with the academic community on topics related to
|
|
|
|
|
efficiency for modern database algorithms.
|
|
|
|
|
</p>
|
2013-11-12 19:50:46 +01:00
|
|
|
|
|
|
|
|
</article>
|
|
|
|
|
|
|
|
|
|
<aside>
|
|
|
|
|
<h3>Quick links</h3>
|
|
|
|
|
<ul>
|
|
|
|
|
<li><a href='overview.html'>Overview</a></li>
|
|
|
|
|
<li><a href='https://github.com/facebook/rocksdb/wiki/_pages'>Wiki</a></li>
|
|
|
|
|
<li><a href='https://github.com/facebook/rocksdb'>GitHub</a>
|
|
|
|
|
</ul>
|
|
|
|
|
<h3>Community</h3>
|
|
|
|
|
<ul>
|
|
|
|
|
<li><a href='https://github.com/facebook/rocksdb/issues'>Report an issue</a></li>
|
2013-12-16 18:12:00 +01:00
|
|
|
<li><a href='https://www.facebook.com/groups/rocksdb.dev/'>Facebook Group</a></li>
|
2013-11-12 19:50:46 +01:00
|
|
|
<li><a href='https://www.facebook.com/rocksdb'>Facebook Page</a></li>
|
2014-03-19 00:35:43 +01:00
|
|
|
<li><a href='https://twitter.com/RocksDB'>Twitter</a></li>
|
2014-02-26 07:28:00 +01:00
|
|
|
<li><a href='meetup.html'>Local Meetups</a></li>
|
2013-11-12 19:50:46 +01:00
|
|
|
<li>
|
|
|
|
|
<div class='fb-like'
|
|
|
|
|
data-href='https://www.facebook.com/rocksdb'
|
|
|
|
|
data-layout='button_count' data-action='like'
|
|
|
|
|
data-show-faces='false' data-share='true'></div>
|
|
|
|
|
</li>
|
|
|
|
|
</ul>
|
|
|
|
|
<h3>Open source</h3>
|
|
|
|
|
<ul>
|
|
|
|
|
<li><a href='https://github.com/facebook/rocksdb/blob/master/LICENSE'>BSD license</a></li>
|
|
|
|
|
<li><a href='https://github.com/facebook/rocksdb/blob/master/CONTRIBUTING.md'>Contributing</a></li>
|
|
|
|
|
<li><a href='https://github.com/facebook/rocksdb/blob/master/PATENTS'>Patent grant</a></li>
|
|
|
|
|
</ul>
|
|
|
|
|
</aside>
|
|
|
|
|
|
|
|
|
|
</div></section>
|
|
|
|
|
|
|
|
|
|
<footer><div class='width'>
|
|
|
|
|
© Copyright 2013, Facebook
|
|
|
|
|
</div></footer>
|
|
|
|
|
|
|
|
|
|
<div id='fb-root'></div>
|
|
|
|
|
<script>(function(d, s, id) {
|
|
|
|
|
var js, fjs = d.getElementsByTagName(s)[0];
|
|
|
|
|
if (d.getElementById(id)) return;
|
|
|
|
|
js = d.createElement(s); js.id = id;
|
|
|
|
|
js.src = '//connect.facebook.net/en_US/all.js#xfbml=1';
|
|
|
|
|
fjs.parentNode.insertBefore(js, fjs);
|
|
|
|
|
}(document, 'script', 'facebook-jssdk'));</script>
|
|
|
|
|
|
|
|
|
|
</body>
|
|
|
|
|
</html>
|
