andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
A library that provides an embeddable, persistent key-value store for fast storage.
3,185 Commits 152 Branches 236 Tags 271 MiB
C++ 82.1%
Java 10.3%
C 2.5%
Python 1.7%
Perl 1.1%
Other 2.1%
d9f4875e52
Go to file
krad d9f4875e52 Disable pre-fetching of index and filter blocks for sst_dump_tool. 
Summary:
BlockBasedTable pre-fetches the filter and index blocks on Open call.
This is an optimistic optimization targeted for runtime scenario. The
optimization is unnecessary for sst_dump_tool

- Added a provision to disable pre-fetching of index and filter blocks
  in BlockBasedTable
- Disabled pre-fetching for the sst_dump tool

Stack for reference :

#01  0x00000000005ed944 in snappy::InternalUncompress<snappy::SnappyArrayWriter> () from /home/engshare/third-party2/snappy/1.0.3/src/snappy-1.0.3/snappy.cc:148
#02  0x00000000005edeee in snappy::RawUncompress () from /home/engshare/third-party2/snappy/1.0.3/src/snappy-1.0.3/snappy.cc:947
#03  0x00000000004e0b4d in rocksdb::UncompressBlockContents () from /data/users/paultuckfield/rocksdb/./util/compression.h:69
#04  0x00000000004e145c in rocksdb::ReadBlockContents () from /data/users/paultuckfield/rocksdb/table/format.cc:334
#05  0x00000000004ca424 in rocksdb::(anonymous namespace)::ReadBlockFromFile () from /data/users/paultuckfield/rocksdb/table/block_based_table_reader.cc:70
#06  0x00000000004cccad in rocksdb::BlockBasedTable::CreateIndexReader () from /data/users/paultuckfield/rocksdb/table/block_based_table_reader.cc:173
#07  0x00000000004d17e5 in rocksdb::BlockBasedTable::Open () from /data/users/paultuckfield/rocksdb/table/block_based_table_reader.cc:553
#08  0x00000000004c8184 in rocksdb::BlockBasedTableFactory::NewTableReader () from /data/users/paultuckfield/rocksdb/table/block_based_table_factory.cc:51
#09  0x0000000000598463 in rocksdb::SstFileReader::NewTableReader () from /data/users/paultuckfield/rocksdb/util/sst_dump_tool.cc:69
#10  0x00000000005986c2 in rocksdb::SstFileReader::SstFileReader () from /data/users/paultuckfield/rocksdb/util/sst_dump_tool.cc:26
#11  0x0000000000599047 in rocksdb::SSTDumpTool::Run () from /data/users/paultuckfield/rocksdb/util/sst_dump_tool.cc:332
#12  0x0000000000409b06 in main () from /data/users/paultuckfield/rocksdb/tools/sst_dump.cc:12

Test Plan:
- Added a unit test to trigger the code.
- Also did some manual verification.
- Passed all unit tests

task #6296048

Reviewers: igor, rven, sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D34041
2015-02-25 16:34:26 -08:00
build_tools build: do not relink every single binary just for a timestamp 2015-02-19 13:11:10 -08:00
coverage Fix coverage script 2014-11-03 14:53:00 -08:00
db Limit key range to number of keys, not number of writes 2015-02-25 15:53:45 -08:00
doc Remove seek compaction 2014-06-20 10:23:02 +02:00
examples Fix formatting 2015-02-09 09:53:30 -08:00
hdfs Replace exception by abort() in dummy HdfsEnv implementation. 2014-12-05 13:30:57 -08:00
include maint: remove extraneous "const" attribute from return type 2015-02-20 11:07:07 -08:00
java [RocksJava] Updated TickerTypes and Histogram to 3.10 2015-02-25 23:34:10 +01:00
linters Revert "Fbson to Json" 2015-02-18 13:14:53 -08:00
port LevelDb include guard replaced with #pragma once 2015-02-24 09:05:06 +03:00
table Disable pre-fetching of index and filter blocks for sst_dump_tool. 2015-02-25 16:34:26 -08:00
third-party/rapidjson Revert "Fbson to Json" 2015-02-18 13:14:53 -08:00
tools rocksdb: Fixed 'Dead assignment' and 'Dead initialization' scan-build warnings 2015-02-23 14:10:09 -08:00
util Disable pre-fetching of index and filter blocks for sst_dump_tool. 2015-02-25 16:34:26 -08:00
utilities Revert "Fbson to Json" 2015-02-18 13:14:53 -08:00
.arcconfig build: abbreviate AR command, too 2015-02-19 11:48:09 -08:00
.clang-format A script that automatically reformat affected lines 2014-01-14 12:21:24 -08:00
.gitignore rocksdb: changes to support 'make analyze' in Jenkins 2015-02-23 17:45:25 -08:00
.travis.yml Upgrade compiler in Travis 2015-02-24 17:19:58 -08:00
AUTHORS Add AUTHORS file. Fix #203 2014-09-29 10:52:18 -07:00
CONTRIBUTING.md facebook accounts are not required for CLA signers 2014-07-08 05:57:54 -04:00
HISTORY.md Remember whole key/prefix filtering on/off in SST file 2015-02-11 11:20:04 -08:00
INSTALL.md Optimize default compile to compilation platform by default 2014-12-15 11:29:41 +01:00
LICENSE Fix copyright year 2014-03-12 12:06:58 -07:00
Makefile Introduce DISABLE_WARNING_AS_ERROR in Makefile 2015-02-24 09:52:28 -08:00
PATENTS Fix the patent format 2013-10-16 15:37:32 -07:00
README.md Replaced "built on on earlier work" by "built on earlier work" in README.md 2014-09-17 01:16:17 -07:00
ROCKSDB_LITE.md RocksDBLite 2014-04-15 13:39:26 -07:00
Vagrantfile Package generation for Ubuntu and CentOS 2014-09-29 16:09:46 -07:00

README.md

RocksDB: A Persistent Key-Value Store for Flash and RAM Storage

Build Status

RocksDB is developed and maintained by Facebook Database Engineering Team. It is built on earlier work on LevelDB by Sanjay Ghemawat (sanjay@google.com) and Jeff Dean (jeff@google.com)

This code is a library that forms the core building block for a fast key value server, especially suited for storing data on flash drives. It has a Log-Structured-Merge-Database (LSM) design with flexible tradeoffs between Write-Amplification-Factor (WAF), Read-Amplification-Factor (RAF) and Space-Amplification-Factor (SAF). It has multi-threaded compactions, making it specially suitable for storing multiple terabytes of data in a single database.

Start with example usage here: https://github.com/facebook/rocksdb/tree/master/examples

See the github wiki for more explanation.

The public interface is in include/. Callers should not include or rely on the details of any other header files in this package. Those internal APIs may be changed without warning.

Design discussions are conducted in https://www.facebook.com/groups/rocksdb.dev/