andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
9262061b0d
rocksdb/util
History
Dhruba Borthakur 5e9f3a9aa7 Better locking in vectorrep that increases throughput to match speed of storage. 
Summary:
There is a use-case where we want to insert data into rocksdb as
fast as possible. Vector rep is used for this purpose.

The background flush thread needs to flush the vectorrep to
storage. It acquires the dblock then sorts the vector, releases
the dblock and then writes the sorted vector to storage. This is
suboptimal because the lock is held during the sort, which
prevents new writes for occuring.

This patch moves the sorting of the vector rep to outside the
db mutex. Performance is now as fastas the underlying storage
system. If you are doing buffered writes to rocksdb files, then
you can observe throughput upwards of 200 MB/sec writes.

This is an early draft and not yet ready to be reviewed.

Test Plan:
make check

Task ID: #

Blame Rev:

Reviewers: haobo

Reviewed By: haobo

CC: leveldb, haobo

Differential Revision: https://reviews.facebook.net/D12987
2013-09-19 21:48:10 -07:00
..
arena_impl.cc Make arena block size configurable 2013-07-31 12:42:23 -07:00
arena_impl.h Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
arena_test.cc Make arena block size configurable 2013-07-31 12:42:23 -07:00
auto_roll_logger_test.cc Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
auto_roll_logger.cc [RocksDB] Fix PosixLogger and AutoRollLogger thread safety 2013-05-21 11:39:44 -07:00
auto_roll_logger.h [RocksDB] Fix PosixLogger and AutoRollLogger thread safety 2013-05-21 11:39:44 -07:00
bit_set.h [Rocksdb] Implement filluniquerandom 2013-06-14 16:17:56 -07:00
bloom_test.cc Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
bloom.cc Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
build_version.h Stop continually re-creating build_version.c 2013-01-24 17:51:39 -08:00
cache_test.cc Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
cache.cc Revert "Minor fixes found while trying to compile it using clang on Mac OS X" 2013-09-15 23:01:26 -07:00
coding_test.cc Codemod NULL to nullptr 2013-02-28 18:04:58 -08:00
coding.cc Add three new MemTableRep's 2013-08-22 23:10:02 -07:00
coding.h Add three new MemTableRep's 2013-08-22 23:10:02 -07:00
comparator.cc Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
crc32c_test.cc Fix all warnings generated by -Wall option to the compiler. 2012-11-06 14:07:31 -08:00
crc32c.cc Codemod NULL to nullptr 2013-02-28 18:04:58 -08:00
crc32c.h A number of fixes: 2011-10-31 17:22:06 +00:00
env_hdfs.cc Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
env_posix.cc Revert "Minor fixes found while trying to compile it using clang on Mac OS X" 2013-09-15 23:01:26 -07:00
env_test.cc [RocksDB] fix build env_test 2013-09-13 21:13:20 -07:00
env.cc Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
filelock_test.cc Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
filter_policy.cc Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
hash.cc A number of fixes: 2011-10-31 17:22:06 +00:00
hash.h reverting disastrous MOE commit, returning to r21 2011-04-19 23:11:15 +00:00
histogram_test.cc Introduce histogram in statistics.h 2013-02-20 10:43:32 -08:00
histogram.cc [Rocksdb] Remove unused double apis to record into histograms 2013-05-16 10:40:30 -07:00
histogram.h Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
ldb_cmd_execute_result.h Enhanced ldb to support data access commands 2013-01-28 11:38:26 -08:00
ldb_cmd.cc New ldb command to convert compaction style 2013-09-04 13:13:08 -07:00
ldb_cmd.h New ldb command to convert compaction style 2013-09-04 13:13:08 -07:00
ldb_tool.cc New ldb command to convert compaction style 2013-09-04 13:13:08 -07:00
logging.cc Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
logging.h A number of fixes: 2011-10-31 17:22:06 +00:00
manual_compaction_test.cc Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
murmurhash.cc Implement RowLocks for assoc schema 2012-10-03 23:19:01 -07:00
murmurhash.h Implement RowLocks for assoc schema 2012-10-03 23:19:01 -07:00
mutexlock.h Implement ReadWrite locks for leveldb 2012-10-01 22:37:39 -07:00
options.cc Added a parameter to limit the maximum space amplification for universal compaction. 2013-09-13 16:27:18 -07:00
perf_context_imp.h [RocksDB] Added nano second stopwatch and new perf counters to track block read cost 2013-09-07 21:14:54 -07:00
perf_context.cc [RocksDB] Added nano second stopwatch and new perf counters to track block read cost 2013-09-07 21:14:54 -07:00
posix_logger.h Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
random.h [RocksDB] Include 64bit random number generator 2013-06-04 13:52:27 -07:00
signal_test.cc [RocksDB] fix build 2013-04-20 10:26:51 -07:00
skiplistrep.cc Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
slice.cc Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
stack_trace.h [RocksDB] Add stacktrace signal handler 2013-04-20 10:26:50 -07:00
statistics.cc Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
stats_logger.h Clean up compiler warnings generated by -Wall option. 2012-08-29 14:24:51 -07:00
status.cc Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
stl_wrappers.h Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
stop_watch.h [RocksDB] Added nano second stopwatch and new perf counters to track block read cost 2013-09-07 21:14:54 -07:00
string_util.cc Ability to set different size fanout multipliers for every level. 2013-05-21 13:50:20 -07:00
string_util.h Ability to set different size fanout multipliers for every level. 2013-05-21 13:50:20 -07:00
testharness.cc Codemod NULL to nullptr 2013-02-28 18:04:58 -08:00
testharness.h Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
testutil.cc Fix all warnings generated by -Wall option to the compiler. 2012-11-06 14:07:31 -08:00
testutil.h Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
transformrep.cc Fix memory leak 2013-08-23 15:39:49 -07:00
vectorrep.cc Better locking in vectorrep that increases throughput to match speed of storage. 2013-09-19 21:48:10 -07:00