A library that provides an embeddable, persistent key-value store for fast storage.
Go to file
Venkatesh Radhakrishnan 030215bf01 Running manual compactions in parallel with other automatic or manual compactions in restricted cases
Summary:
This diff provides a framework for doing manual
compactions in parallel with other compactions. We now have a deque of manual compactions. We also pass manual compactions as an argument from RunManualCompactions down to
BackgroundCompactions, so that RunManualCompactions can be reentrant.
Parallelism is controlled by the two routines
ConflictingManualCompaction to allow/disallow new parallel/manual
compactions based on already existing ManualCompactions. In this diff, by default manual compactions still have to run exclusive of other compactions. However, by setting the compaction option, exclusive_manual_compaction to false, it is possible to run other compactions in parallel with a manual compaction. However, we are still restricted to one manual compaction per column family at a time. All of these restrictions will be relaxed in future diffs.
I will be adding more tests later.

Test Plan: Rocksdb regression + new tests + valgrind

Reviewers: igor, anthony, IslamAbdelRahman, kradhakrishnan, yhchiang, sdong

Reviewed By: sdong

Subscribers: yoshinorim, dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D47973
2015-12-14 11:20:34 -08:00
arcanist_util Don't spew warnings when flint doesn't exist 2015-10-19 18:47:59 -07:00
build_tools Improving parser 2015-12-11 11:06:42 -08:00
coverage Fix coverage script 2014-11-03 14:53:00 -08:00
db Running manual compactions in parallel with other automatic or manual compactions in restricted cases 2015-12-14 11:20:34 -08:00
doc Lint everything 2015-11-16 12:56:21 -08:00
examples Update examples/rocksdb_option_file_example.ini 2015-12-08 16:17:06 -08:00
hdfs Running manual compactions in parallel with other automatic or manual compactions in restricted cases 2015-12-14 11:20:34 -08:00
include/rocksdb Running manual compactions in parallel with other automatic or manual compactions in restricted cases 2015-12-14 11:20:34 -08:00
java fix typos in comments 2015-12-11 01:54:48 +09:00
memtable Enable C4305 'identifier' : truncation from 'type1' to 'type2' 2015-11-06 16:44:39 -08:00
port Running manual compactions in parallel with other automatic or manual compactions in restricted cases 2015-12-14 11:20:34 -08:00
table Use SST files for Transaction conflict detection 2015-12-11 12:34:11 -08:00
third-party Lint everything 2015-11-16 12:56:21 -08:00
tools fix typos in comments 2015-12-11 01:54:48 +09:00
util Running manual compactions in parallel with other automatic or manual compactions in restricted cases 2015-12-14 11:20:34 -08:00
utilities Running manual compactions in parallel with other automatic or manual compactions in restricted cases 2015-12-14 11:20:34 -08:00
.arcconfig Integrate Jenkins with Phabricator 2015-04-07 11:56:29 -07:00
.clang-format A script that automatically reformat affected lines 2014-01-14 12:21:24 -08:00
.gitignore New amalgamation target 2015-10-01 08:29:31 +13:00
.travis.yml Run ROCKSDB_LITE tests in travis 2015-10-16 10:47:37 -07:00
appveyor.yml Exclude DBTest.FileCreationRandomFailure as a long running test 2015-11-17 13:54:13 -08:00
AUTHORS Add AUTHORS file. Fix #203 2014-09-29 10:52:18 -07:00
CMakeLists.txt Merge pull request #862 from ceph/wip-env 2015-12-10 18:45:07 -08:00
CONTRIBUTING.md facebook accounts are not required for CLA signers 2014-07-08 05:57:54 -04:00
DUMP_FORMAT.md First version of rocksdb_dump and rocksdb_undump. 2015-06-19 16:24:36 -07:00
HISTORY.md fix typos in comments 2015-12-11 01:54:48 +09:00
INSTALL.md Update 4 is required for building with MS Visual Studio 13 2015-10-15 11:06:02 -07:00
LICENSE Fix copyright year 2014-03-12 12:06:58 -07:00
Makefile Merge pull request #862 from ceph/wip-env 2015-12-10 18:45:07 -08:00
PATENTS Update Patent Grant. 2015-04-13 10:33:43 +01:00
README.md Replaced "built on on earlier work" by "built on earlier work" in README.md 2014-09-17 01:16:17 -07:00
ROCKSDB_LITE.md Optimistic Transactions 2015-05-29 14:36:35 -07:00
src.mk Merge pull request #862 from ceph/wip-env 2015-12-10 18:45:07 -08:00
thirdparty.inc Enable override to 3rd party linkage 2015-11-24 11:51:37 -08:00
USERS.md Add Cloudera's blog post to USERS.md 2015-09-02 14:04:51 -07:00
Vagrantfile RocksDB on FreeBSD support 2015-02-26 15:19:17 -08:00
WINDOWS_PORT.md Commit both PR and internal code review changes 2015-07-07 16:58:20 -07:00

RocksDB: A Persistent Key-Value Store for Flash and RAM Storage

Build Status

RocksDB is developed and maintained by Facebook Database Engineering Team. It is built on earlier work on LevelDB by Sanjay Ghemawat (sanjay@google.com) and Jeff Dean (jeff@google.com)

This code is a library that forms the core building block for a fast key value server, especially suited for storing data on flash drives. It has a Log-Structured-Merge-Database (LSM) design with flexible tradeoffs between Write-Amplification-Factor (WAF), Read-Amplification-Factor (RAF) and Space-Amplification-Factor (SAF). It has multi-threaded compactions, making it specially suitable for storing multiple terabytes of data in a single database.

Start with example usage here: https://github.com/facebook/rocksdb/tree/master/examples

See the github wiki for more explanation.

The public interface is in include/. Callers should not include or rely on the details of any other header files in this package. Those internal APIs may be changed without warning.

Design discussions are conducted in https://www.facebook.com/groups/rocksdb.dev/