9da8748016
Summary: This changes loads to use vector memtable and disable the WAL. This also increases the chance we will see IO bottlenecks during loads which is good to stress test HW. But I also think it is a good way to load data quickly as this is a bulk operation and the WAL isn't needed. The two numbers below are the MB/sec rates for fillseq, bulkload using a skiplist or vector memtable and the WAL enabled or disabled. There is a big benefit from using the vector memtable and WAL disabled. Alas there is also a perf bug in the use of std::sort for ordered input when the vector is flushed. Task is open for that. 112, 66 - skiplist with wal 250, 116 - skiplist without wal 110, 108 - vector with wal 232, 370 - vector without wal Task ID: # Blame Rev: Test Plan: Revert Plan: Database Impact: Memcache Impact: Other Notes: EImportant: - begin *PUBLIC* platform impact section - Bugzilla: # - end platform impact - Reviewers: igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D36957
RocksDB: A Persistent Key-Value Store for Flash and RAM Storage
RocksDB is developed and maintained by Facebook Database Engineering Team. It is built on earlier work on LevelDB by Sanjay Ghemawat (sanjay@google.com) and Jeff Dean (jeff@google.com)
This code is a library that forms the core building block for a fast key value server, especially suited for storing data on flash drives. It has a Log-Structured-Merge-Database (LSM) design with flexible tradeoffs between Write-Amplification-Factor (WAF), Read-Amplification-Factor (RAF) and Space-Amplification-Factor (SAF). It has multi-threaded compactions, making it specially suitable for storing multiple terabytes of data in a single database.
Start with example usage here: https://github.com/facebook/rocksdb/tree/master/examples
See the github wiki for more explanation.
The public interface is in include/. Callers should not include or
rely on the details of any other header files in this package. Those
internal APIs may be changed without warning.
Design discussions are conducted in https://www.facebook.com/groups/rocksdb.dev/