andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
9262061b0d
rocksdb/db
History
Dhruba Borthakur 5e9f3a9aa7 Better locking in vectorrep that increases throughput to match speed of storage. 
Summary:
There is a use-case where we want to insert data into rocksdb as
fast as possible. Vector rep is used for this purpose.

The background flush thread needs to flush the vectorrep to
storage. It acquires the dblock then sorts the vector, releases
the dblock and then writes the sorted vector to storage. This is
suboptimal because the lock is held during the sort, which
prevents new writes for occuring.

This patch moves the sorting of the vector rep to outside the
db mutex. Performance is now as fastas the underlying storage
system. If you are doing buffered writes to rocksdb files, then
you can observe throughput upwards of 200 MB/sec writes.

This is an early draft and not yet ready to be reviewed.

Test Plan:
make check

Task ID: #

Blame Rev:

Reviewers: haobo

Reviewed By: haobo

CC: leveldb, haobo

Differential Revision: https://reviews.facebook.net/D12987
2013-09-19 21:48:10 -07:00
..
.nfs00000000066c9ebb00000002 Enhance db_bench 2013-03-14 16:00:23 -07:00
builder.cc Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
builder.h Revert "Minor fixes found while trying to compile it using clang on Mac OS X" 2013-09-15 23:01:26 -07:00
c_test.c Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
c.cc Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
corruption_test.cc Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
db_bench.cc Better locking in vectorrep that increases throughput to match speed of storage. 2013-09-19 21:48:10 -07:00
db_filesnapshot.cc Return pathname relative to db dir in LogFile and cleanup AppendSortedWalsOfType 2013-09-04 13:44:43 -07:00
db_impl_readonly.cc Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
db_impl_readonly.h New ldb command to convert compaction style 2013-09-04 13:13:08 -07:00
db_impl.cc [RocksDB] Remove Log file immediately after memtable flush 2013-09-12 11:54:44 -07:00
db_impl.h [RocksDB] Remove Log file immediately after memtable flush 2013-09-12 11:54:44 -07:00
db_iter.cc An iterator may automatically invoke reseeks. 2013-09-06 11:50:53 -07:00
db_iter.h Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
db_statistics.h Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
db_stats_logger.cc Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
db_test.cc [RocksDB] Fix DBTest.UniversalCompactionSizeAmplification too 2013-09-17 21:29:33 -07:00
dbformat_test.cc Fix all warnings generated by -Wall option to the compiler. 2012-11-06 14:07:31 -08:00
dbformat.cc [RocksDB] Added nano second stopwatch and new perf counters to track block read cost 2013-09-07 21:14:54 -07:00
dbformat.h Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
deletefile_test.cc Fix build caused by DeleteFile not tolerating / at the beginning 2013-09-01 17:59:13 -07:00
filename_test.cc Added meta-database support. 2012-12-17 11:26:59 -08:00
filename.cc Return pathname relative to db dir in LogFile and cleanup AppendSortedWalsOfType 2013-09-04 13:44:43 -07:00
filename.h Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
log_format.h Fixed sign-comparison in rocksdb code-base and fixed Makefile 2013-03-19 14:35:23 -07:00
log_reader.cc Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
log_reader.h Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
log_test.cc Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
log_writer.cc Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
log_writer.h Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
memtable.cc [RocksDB] Remove Log file immediately after memtable flush 2013-09-12 11:54:44 -07:00
memtable.h [RocksDB] Remove Log file immediately after memtable flush 2013-09-12 11:54:44 -07:00
memtablelist.cc Flush was hanging because the configured options specified that more than 1 memtable need to be merged. 2013-09-06 16:28:33 -07:00
memtablelist.h Flush was hanging because the configured options specified that more than 1 memtable need to be merged. 2013-09-06 16:28:33 -07:00
merge_helper.cc Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
merge_helper.h Revert "Minor fixes found while trying to compile it using clang on Mac OS X" 2013-09-15 23:01:26 -07:00
merge_operator.cc Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
merge_test.cc Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
perf_context_test.cc [RocksDB] Unit test to show Seek key comparison number 2013-09-18 21:43:41 -07:00
prefix_filter_iterator.h Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
repair.cc Revert "Minor fixes found while trying to compile it using clang on Mac OS X" 2013-09-15 23:01:26 -07:00
skiplist_test.cc Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
skiplist.h Make arena block size configurable 2013-07-31 12:42:23 -07:00
snapshot.h Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
table_cache.cc Introduced a new flag non_blocking_io in ReadOptions. 2013-08-28 10:49:14 -07:00
table_cache.h Introduced a new flag non_blocking_io in ReadOptions. 2013-08-28 10:49:14 -07:00
transaction_log_impl.cc API for getting archived log files 2013-08-19 13:37:04 -07:00
transaction_log_impl.h Return pathname relative to db dir in LogFile and cleanup AppendSortedWalsOfType 2013-09-04 13:44:43 -07:00
version_edit_test.cc Reduce write amplification by merging files in L0 back into L0 2013-06-30 20:07:04 -07:00
version_edit.cc Merge remote-tracking branch 'origin' into performance 2013-08-12 09:58:50 -07:00
version_edit.h Merge remote-tracking branch 'origin' into performance 2013-08-12 09:58:50 -07:00
version_set_reduce_num_levels.cc Fix valgrind errors in rocksdb tests: auto_roll_logger_test, reduce_levels_test 2013-03-12 16:03:16 -07:00
version_set_test.cc Codemod NULL to nullptr 2013-02-28 18:04:58 -08:00
version_set.cc [RocksDB] Universal compaction trigger condition minor fix 2013-09-15 22:35:59 -07:00
version_set.h Added a parameter to limit the maximum space amplification for universal compaction. 2013-09-13 16:27:18 -07:00
write_batch_internal.h Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
write_batch_test.cc Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
write_batch.cc Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00