andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
A library that provides an embeddable, persistent key-value store for fast storage.
2,148 Commits 152 Branches 236 Tags 271 MiB
C++ 82.1%
Java 10.3%
C 2.5%
Python 1.7%
Perl 1.1%
Other 2.1%
7e9f28cb23
Go to file
Lei Jin 7e9f28cb23 limit max bytes that can be read/written per pread/write syscall 
Summary:
BlockBasedTable sst file size can grow to a large size when universal
compaction is used. When index block exceeds 2G, pread seems to fail and
return truncated data and causes "trucated block" error. I tried to use
```
  #define _FILE_OFFSET_BITS 64
```
But the problem still persists. Splitting a big write/read into smaller
batches seems to solve the problem.

Test Plan:
successfully compacted a case with resulting sst file at ~90G (2.1G
index block size)

Reviewers: yhchiang, igor, sdong

Reviewed By: sdong

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D22569
2014-08-29 21:21:49 -07:00
build_tools Changes to support unity build: 2014-08-11 13:22:47 -04:00
coverage Disable the html-based coverage report by default 2014-02-06 12:58:13 -08:00
db ForwardIterator: reset incomplete iterators on Seek() 2014-08-29 16:21:29 -07:00
doc Remove seek compaction 2014-06-20 10:23:02 +02:00
examples Make it easier to start using RocksDB 2014-05-10 10:49:33 -07:00
hdfs hdfs cleanup and compile test against CDH 4.4. 2014-05-20 17:22:12 -04:00
helpers/memenv Expose in memory Env to the world 2014-04-14 12:28:15 -07:00
include Implementing a cache friendly version of Cuckoo Hash 2014-08-28 10:42:23 -07:00
java JNI changes corresponding to BlockBasedTableOptions migration 2014-08-25 14:22:55 -07:00
linters allow lambda function syntax in cpplint 2014-02-20 12:47:05 -08:00
port generic rate limiter 2014-07-08 11:41:57 -07:00
table Improve Cuckoo Table Reader performance. Inlined hash function and number of buckets a power of two. 2014-08-29 19:06:15 -07:00
third-party/rapidjson Fix a rapidjson compile error in mac. 2014-06-23 17:09:24 -06:00
tools move block based table related options BlockBasedTableOptions 2014-08-25 14:22:05 -07:00
util limit max bytes that can be read/written per pread/write syscall 2014-08-29 21:21:49 -07:00
utilities Compact SpatialDB as we go, not at the end 2014-08-28 11:32:49 -07:00
.arcconfig Improve/fix bugs for the cpp linter 2014-02-13 17:48:11 -08:00
.clang-format A script that automatically reformat affected lines 2014-01-14 12:21:24 -08:00
.gitignore Changes to support unity build: 2014-08-11 13:22:47 -04:00
.travis.yml Turn off travis notifications 2014-05-14 16:08:02 -07:00
CONTRIBUTING.md facebook accounts are not required for CLA signers 2014-07-08 05:57:54 -04:00
HISTORY.md improve OptimizeForPointLookup() 2014-08-26 14:15:00 -07:00
INSTALL.md specify the command to install build_tools/mac-install-gflags.sh file in doc 2014-06-17 17:03:21 -05:00
LICENSE Fix copyright year 2014-03-12 12:06:58 -07:00
Makefile Don't do memtable lookup in db_impl_readonly if memtables are empty while opening db. 2014-08-26 17:19:03 -07:00
PATENTS Fix the patent format 2013-10-16 15:37:32 -07:00
README.md Update README.md 2014-06-23 15:58:54 -07:00
ROCKSDB_LITE.md RocksDBLite 2014-04-15 13:39:26 -07:00

README.md

RocksDB: A Persistent Key-Value Store for Flash and RAM Storage

Build Status

RocksDB is developed and maintained by Facebook Database Engineering Team. It is built on on earlier work on LevelDB by Sanjay Ghemawat (sanjay@google.com) and Jeff Dean (jeff@google.com)

This code is a library that forms the core building block for a fast key value server, especially suited for storing data on flash drives. It has a Log-Structured-Merge-Database (LSM) design with flexible tradeoffs between Write-Amplification-Factor (WAF), Read-Amplification-Factor (RAF) and Space-Amplification-Factor (SAF). It has multi-threaded compactions, making it specially suitable for storing multiple terabytes of data in a single database.

Start with example usage here: https://github.com/facebook/rocksdb/tree/master/examples

See the github wiki for more explanation.

The public interface is in include/. Callers should not include or rely on the details of any other header files in this package. Those internal APIs may be changed without warning.

Design discussions are conducted in https://www.facebook.com/groups/rocksdb.dev/