andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
A library that provides an embeddable, persistent key-value store for fast storage.
3,470 Commits 152 Branches 236 Tags 271 MiB
C++ 82.1%
Java 10.3%
C 2.5%
Python 1.7%
Perl 1.1%
Other 2.1%
77a5a543a5
Go to file
Yueh-Hsuan Chiang 77a5a543a5 Allow GetThreadList() to report basic compaction operation properties. 
Summary:
Now we're able to show more details about a compaction in
GetThreadList() :)

This patch allows GetThreadList() to report basic compaction
operation properties.  Basic compaction properties include:
    1. job id
    2. compaction input / output level
    3. compaction property flags (is_manual, is_deletion, .. etc)
    4. total input bytes
    5. the number of bytes has been read currently.
    6. the number of bytes has been written currently.

Flush operation properties will be done in a seperate diff.

Test Plan:
/db_bench --threads=30 --num=1000000 --benchmarks=fillrandom --thread_status_per_interval=1

Sample output of tracking same job:

          ThreadID ThreadType       cfName            Operation   ElapsedTime                                         Stage        State OperationProperties
   140664171987072    Low Pri      default           Compaction     31.357 ms     CompactionJob::FinishCompactionOutputFile              BaseInputLevel 1 | BytesRead 2264663 | BytesWritten 1934241 | IsDeletion 0 | IsManual 0 | IsTrivialMove 0 | JobID 277 | OutputLevel 2 | TotalInputBytes 3964158 |

          ThreadID ThreadType       cfName            Operation   ElapsedTime                                         Stage        State OperationProperties
   140664171987072    Low Pri      default           Compaction     59.440 ms     CompactionJob::FinishCompactionOutputFile              BaseInputLevel 1 | BytesRead 2264663 | BytesWritten 1934241 | IsDeletion 0 | IsManual 0 | IsTrivialMove 0 | JobID 277 | OutputLevel 2 | TotalInputBytes 3964158 |

          ThreadID ThreadType       cfName            Operation   ElapsedTime                                         Stage        State OperationProperties
   140664171987072    Low Pri      default           Compaction    226.375 ms                        CompactionJob::Install              BaseInputLevel 1 | BytesRead 3958013 | BytesWritten 3621940 | IsDeletion 0 | IsManual 0 | IsTrivialMove 0 | JobID 277 | OutputLevel 2 | TotalInputBytes 3964158 |

Reviewers: sdong, rven, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D37653
2015-05-06 22:51:06 -07:00
arcanist_util Integrate Jenkins with Phabricator 2015-04-07 11:56:29 -07:00
build_tools Build for CYGWIN 2015-04-23 21:33:44 -07:00
coverage Fix coverage script 2014-11-03 14:53:00 -08:00
db Allow GetThreadList() to report basic compaction operation properties. 2015-05-06 22:51:06 -07:00
doc Remove seek compaction 2014-06-20 10:23:02 +02:00
examples Fix formatting 2015-02-09 09:53:30 -08:00
hdfs Remove unused parameter in CancelAllBackgroundWork 2015-03-16 21:07:54 -07:00
include Allow GetThreadList() to report basic compaction operation properties. 2015-05-06 22:51:06 -07:00
java [RocksJava] Remove deprecated methods 2015-04-16 00:31:43 +02:00
port Build for CYGWIN 2015-04-23 21:33:44 -07:00
table fix typos 2015-04-25 18:14:27 +09:00
third-party Update COMMIT.md 2015-03-30 17:48:16 -07:00
tools Add Size-GB column to benchmark reports 2015-05-02 07:46:12 -07:00
util Allow GetThreadList() to report basic compaction operation properties. 2015-05-06 22:51:06 -07:00
utilities fix crashes in stats and compaction filter for db_ttl_impl 2015-05-05 16:54:47 -07:00
.arcconfig Integrate Jenkins with Phabricator 2015-04-07 11:56:29 -07:00
.clang-format A script that automatically reformat affected lines 2014-01-14 12:21:24 -08:00
.gitignore run 'make check's rules (and even subtests) in parallel 2015-04-06 12:35:25 -07:00
.travis.yml Don't preinstall jemalloc in Travis 2015-04-24 18:43:07 -07:00
AUTHORS Add AUTHORS file. Fix #203 2014-09-29 10:52:18 -07:00
CONTRIBUTING.md facebook accounts are not required for CLA signers 2014-07-08 05:57:54 -04:00
HISTORY.md Task 6532943: Rocksdb - SetCapacity() can dynamically change cache capacity if feasible 2015-04-24 14:12:58 -07:00
INSTALL.md Optimize default compile to compilation platform by default 2014-12-15 11:29:41 +01:00
LICENSE Fix copyright year 2014-03-12 12:06:58 -07:00
Makefile Don't treat warnings as error when building release 2015-05-05 11:54:13 -07:00
PATENTS Update Patent Grant. 2015-04-13 10:33:43 +01:00
README.md Replaced "built on on earlier work" by "built on earlier work" in README.md 2014-09-17 01:16:17 -07:00
ROCKSDB_LITE.md RocksDBLite 2014-04-15 13:39:26 -07:00
src.mk Add experimental API MarkForCompaction() 2015-04-17 16:44:45 -07:00
USERS.md Update USERS.md 2015-05-04 15:27:59 -07:00
Vagrantfile RocksDB on FreeBSD support 2015-02-26 15:19:17 -08:00

README.md

RocksDB: A Persistent Key-Value Store for Flash and RAM Storage

Build Status

RocksDB is developed and maintained by Facebook Database Engineering Team. It is built on earlier work on LevelDB by Sanjay Ghemawat (sanjay@google.com) and Jeff Dean (jeff@google.com)

This code is a library that forms the core building block for a fast key value server, especially suited for storing data on flash drives. It has a Log-Structured-Merge-Database (LSM) design with flexible tradeoffs between Write-Amplification-Factor (WAF), Read-Amplification-Factor (RAF) and Space-Amplification-Factor (SAF). It has multi-threaded compactions, making it specially suitable for storing multiple terabytes of data in a single database.

Start with example usage here: https://github.com/facebook/rocksdb/tree/master/examples

See the github wiki for more explanation.

The public interface is in include/. Callers should not include or rely on the details of any other header files in this package. Those internal APIs may be changed without warning.

Design discussions are conducted in https://www.facebook.com/groups/rocksdb.dev/