andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
e0aa19a94e
rocksdb/db
History
Haobo Xu e0aa19a94e [RocbsDB] Add an option to enable set based memtable for perf_context_test 
Summary:
as title.
Some result:

-- Sequential insertion of 1M key/value with stock skip list (all in on memtable)
time ./perf_context_test  --total_keys=1000000  --use_set_based_memetable=0
Inserting 1000000 key/value pairs
...
Put uesr key comparison:
Count: 1000000  Average: 8.0179  StdDev: 176.34
Min: 0.0000  Median: 2.5555  Max: 88933.0000
Percentiles: P50: 2.56 P75: 2.83 P99: 58.21 P99.9: 133.62 P99.99: 987.50
Get uesr key comparison:
Count: 1000000  Average: 43.4465  StdDev: 379.03
Min: 2.0000  Median: 36.0195  Max: 88939.0000
Percentiles: P50: 36.02 P75: 43.66 P99: 112.98 P99.9: 824.84 P99.99: 7615.38
real	0m21.345s
user	0m14.723s
sys	0m5.677s

-- Sequential insertion of 1M key/value with set based memtable (all in on memtable)
time ./perf_context_test  --total_keys=1000000  --use_set_based_memetable=1
Inserting 1000000 key/value pairs
...
Put uesr key comparison:
Count: 1000000  Average: 61.5022  StdDev: 6.49
Min: 0.0000  Median: 62.4295  Max: 71.0000
Percentiles: P50: 62.43 P75: 66.61 P99: 71.00 P99.9: 71.00 P99.99: 71.00
Get uesr key comparison:
Count: 1000000  Average: 29.3810  StdDev: 3.20
Min: 1.0000  Median: 29.1801  Max: 34.0000
Percentiles: P50: 29.18 P75: 32.06 P99: 34.00 P99.9: 34.00 P99.99: 34.00
real	0m28.875s
user	0m21.699s
sys	0m5.749s

Worst case comparison for a Put is 88933 (skiplist) vs 71 (set based memetable)

Of course, there's other in-efficiency in set based memtable implementation, which lead to the overall worst performance. However, P99 behavior advantage is very very obvious.

Test Plan: ./perf_context_test and viewstate shadow testing

Reviewers: dhruba

Reviewed By: dhruba

CC: leveldb

Differential Revision: https://reviews.facebook.net/D13095
2013-09-25 22:49:18 -07:00
..
.nfs00000000066c9ebb00000002 Enhance db_bench 2013-03-14 16:00:23 -07:00
builder.cc Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
builder.h Revert "Minor fixes found while trying to compile it using clang on Mac OS X" 2013-09-15 23:01:26 -07:00
c_test.c Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
c.cc Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
corruption_test.cc Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
db_bench.cc Better locking in vectorrep that increases throughput to match speed of storage. 2013-09-19 21:48:10 -07:00
db_filesnapshot.cc Return pathname relative to db dir in LogFile and cleanup AppendSortedWalsOfType 2013-09-04 13:44:43 -07:00
db_impl_readonly.cc Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
db_impl_readonly.h New ldb command to convert compaction style 2013-09-04 13:13:08 -07:00
db_impl.cc [RocksDB] Remove Log file immediately after memtable flush 2013-09-12 11:54:44 -07:00
db_impl.h [RocksDB] Remove Log file immediately after memtable flush 2013-09-12 11:54:44 -07:00
db_iter.cc An iterator may automatically invoke reseeks. 2013-09-06 11:50:53 -07:00
db_iter.h Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
db_statistics.h Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
db_stats_logger.cc Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
db_test.cc The vector rep implementation was segfaulting because of incorrect initialization of vector. 2013-09-25 11:33:52 -07:00
dbformat_test.cc Fix all warnings generated by -Wall option to the compiler. 2012-11-06 14:07:31 -08:00
dbformat.cc [RocksDB] Added nano second stopwatch and new perf counters to track block read cost 2013-09-07 21:14:54 -07:00
dbformat.h Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
deletefile_test.cc Fix build caused by DeleteFile not tolerating / at the beginning 2013-09-01 17:59:13 -07:00
filename_test.cc Added meta-database support. 2012-12-17 11:26:59 -08:00
filename.cc Return pathname relative to db dir in LogFile and cleanup AppendSortedWalsOfType 2013-09-04 13:44:43 -07:00
filename.h Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
log_format.h Fixed sign-comparison in rocksdb code-base and fixed Makefile 2013-03-19 14:35:23 -07:00
log_reader.cc Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
log_reader.h Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
log_test.cc Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
log_writer.cc Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
log_writer.h Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
memtable.cc [RocksDB] Remove Log file immediately after memtable flush 2013-09-12 11:54:44 -07:00
memtable.h [RocksDB] Remove Log file immediately after memtable flush 2013-09-12 11:54:44 -07:00
memtablelist.cc Flush was hanging because the configured options specified that more than 1 memtable need to be merged. 2013-09-06 16:28:33 -07:00
memtablelist.h Flush was hanging because the configured options specified that more than 1 memtable need to be merged. 2013-09-06 16:28:33 -07:00
merge_helper.cc Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
merge_helper.h Revert "Minor fixes found while trying to compile it using clang on Mac OS X" 2013-09-15 23:01:26 -07:00
merge_operator.cc Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
merge_test.cc Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
perf_context_test.cc [RocbsDB] Add an option to enable set based memtable for perf_context_test 2013-09-25 22:49:18 -07:00
prefix_filter_iterator.h Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
repair.cc Revert "Minor fixes found while trying to compile it using clang on Mac OS X" 2013-09-15 23:01:26 -07:00
skiplist_test.cc Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
skiplist.h Make arena block size configurable 2013-07-31 12:42:23 -07:00
snapshot.h Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
table_cache.cc Introduced a new flag non_blocking_io in ReadOptions. 2013-08-28 10:49:14 -07:00
table_cache.h Introduced a new flag non_blocking_io in ReadOptions. 2013-08-28 10:49:14 -07:00
transaction_log_impl.cc API for getting archived log files 2013-08-19 13:37:04 -07:00
transaction_log_impl.h Return pathname relative to db dir in LogFile and cleanup AppendSortedWalsOfType 2013-09-04 13:44:43 -07:00
version_edit_test.cc Reduce write amplification by merging files in L0 back into L0 2013-06-30 20:07:04 -07:00
version_edit.cc Merge remote-tracking branch 'origin' into performance 2013-08-12 09:58:50 -07:00
version_edit.h Merge remote-tracking branch 'origin' into performance 2013-08-12 09:58:50 -07:00
version_set_reduce_num_levels.cc Fix valgrind errors in rocksdb tests: auto_roll_logger_test, reduce_levels_test 2013-03-12 16:03:16 -07:00
version_set_test.cc Codemod NULL to nullptr 2013-02-28 18:04:58 -08:00
version_set.cc [RocksDB] Universal compaction trigger condition minor fix 2013-09-15 22:35:59 -07:00
version_set.h Added a parameter to limit the maximum space amplification for universal compaction. 2013-09-13 16:27:18 -07:00
write_batch_internal.h Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
write_batch_test.cc Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00
write_batch.cc Replace include/leveldb with include/rocksdb. 2013-08-23 10:51:00 -07:00