A library that provides an embeddable, persistent key-value store for fast storage.
Go to file
Reid Horuff 1b8a2e8fdd [rocksdb] Memtable Log Referencing and Prepared Batch Recovery
Summary:
This diff is built on top of WriteBatch modification: https://reviews.facebook.net/D54093 and adds the required functionality to rocksdb core necessary for rocksdb to support 2PC.

modfication of DBImpl::WriteImpl()
- added two arguments *uint64_t log_used = nullptr, uint64_t log_ref = 0;
- *log_used is an output argument which will return the log number which the incoming batch was inserted into, 0 if no WAL insert took place.
-  log_ref is a supplied log_number which all memtables inserted into will reference after the batch insert takes place. This number will reside in 'FindMinPrepLogReferencedByMemTable()' until all Memtables insertinto have flushed.

- Recovery/writepath is now aware of prepared batches and commit and rollback markers.

Test Plan: There is currently no test on this diff. All testing of this functionality takes place in the Transaction layer/diff but I will add some testing.

Reviewers: IslamAbdelRahman, sdong

Subscribers: leveldb, santoshb, andrewkr, vasilep, dhruba, hermanlee4

Differential Revision: https://reviews.facebook.net/D56919
2016-05-10 14:06:07 -07:00
arcanist_util Have sandcastle run lite_test for every diff 2016-05-06 14:51:20 -07:00
build_tools Have sandcastle run lite_test for every diff 2016-05-06 14:51:20 -07:00
coverage Fix coverage script 2014-11-03 14:53:00 -08:00
db [rocksdb] Memtable Log Referencing and Prepared Batch Recovery 2016-05-10 14:06:07 -07:00
doc Lint everything 2015-11-16 12:56:21 -08:00
examples Adding pin_l0_filter_and_index_blocks_in_cache feature and related fixes. 2016-04-01 10:42:39 -07:00
hdfs using java7 in runtime for hdfs env (#1072) 2016-04-12 00:08:26 -04:00
include/rocksdb [rocksdb] Memtable Log Referencing and Prepared Batch Recovery 2016-05-10 14:06:07 -07:00
java Java API - Add missing HEADER_LEVEL logging (#1104) 2016-05-06 15:06:12 -07:00
memtable Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
port Fix multiple issues with WinMmapFile fo sequential writing (#1108) 2016-04-29 16:43:13 -07:00
table BlockBasedTable::Get() not to use prefix bloom if read_options.total_order_seek = true 2016-05-06 10:16:11 -07:00
third-party Fix the build break on Ubuntu 15.10 when gcc 5.2.1 is used 2016-03-15 10:30:10 -07:00
tools Modification of WriteBatch to support two phase commit 2016-05-10 14:06:07 -07:00
util [rocksdb] Memtable Log Referencing and Prepared Batch Recovery 2016-05-10 14:06:07 -07:00
utilities Modification of WriteBatch to support two phase commit 2016-05-10 14:06:07 -07:00
.arcconfig Integrate Jenkins with Phabricator 2015-04-07 11:56:29 -07:00
.clang-format A script that automatically reformat affected lines 2014-01-14 12:21:24 -08:00
.gitignore Ignore db_test2 2016-03-07 15:56:16 -08:00
.travis.yml Split Travis unittests Job 2016-04-13 14:22:29 -07:00
appveyor.yml Exclude DBTest.FileCreationRandomFailure as a long running test 2015-11-17 13:54:13 -08:00
AUTHORS Add AUTHORS file. Fix #203 2014-09-29 10:52:18 -07:00
CMakeLists.txt Introduce chroot Env 2016-05-06 17:42:50 -07:00
CONTRIBUTING.md facebook accounts are not required for CLA signers 2014-07-08 05:57:54 -04:00
DEFAULT_OPTIONS_HISTORY.md Release RocksDB 4.8.0 2016-05-02 14:38:04 -07:00
DUMP_FORMAT.md First version of rocksdb_dump and rocksdb_undump. 2015-06-19 16:24:36 -07:00
HISTORY.md Add bottommost_compression option 2016-05-09 15:57:19 -07:00
INSTALL.md Simple changes to support builds for ppc64[le] consistent with X86 2016-01-19 09:08:19 -06:00
LANGUAGE-BINDINGS.md Merge pull request #1056 from facebook/igorcanadi-patch-1 2016-04-04 08:08:52 -07:00
LICENSE Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
Makefile Print memory allocation counters 2016-04-27 16:23:33 -07:00
PATENTS Update Patent Grant. 2015-04-13 10:33:43 +01:00
README.md Replaced "built on on earlier work" by "built on earlier work" in README.md 2014-09-17 01:16:17 -07:00
ROCKSDB_LITE.md Optimistic Transactions 2015-05-29 14:36:35 -07:00
src.mk Introduce chroot Env 2016-05-06 17:42:50 -07:00
thirdparty.inc Introduce XPRESS compresssion on Windows. (#1081) 2016-04-19 22:54:24 -07:00
USERS.md Update USERS.md with link to LinkedIn blog post (#1088) 2016-04-22 15:53:32 -07:00
Vagrantfile RocksDB on FreeBSD support 2015-02-26 15:19:17 -08:00
WINDOWS_PORT.md Commit both PR and internal code review changes 2015-07-07 16:58:20 -07:00

RocksDB: A Persistent Key-Value Store for Flash and RAM Storage

Build Status

RocksDB is developed and maintained by Facebook Database Engineering Team. It is built on earlier work on LevelDB by Sanjay Ghemawat (sanjay@google.com) and Jeff Dean (jeff@google.com)

This code is a library that forms the core building block for a fast key value server, especially suited for storing data on flash drives. It has a Log-Structured-Merge-Database (LSM) design with flexible tradeoffs between Write-Amplification-Factor (WAF), Read-Amplification-Factor (RAF) and Space-Amplification-Factor (SAF). It has multi-threaded compactions, making it specially suitable for storing multiple terabytes of data in a single database.

Start with example usage here: https://github.com/facebook/rocksdb/tree/master/examples

See the github wiki for more explanation.

The public interface is in include/. Callers should not include or rely on the details of any other header files in this package. Those internal APIs may be changed without warning.

Design discussions are conducted in https://www.facebook.com/groups/rocksdb.dev/