Haobo Xu e0aa19a94e [RocbsDB] Add an option to enable set based memtable for perf_context_test ...

Summary:
as title.
Some result:

-- Sequential insertion of 1M key/value with stock skip list (all in on memtable)
time ./perf_context_test  --total_keys=1000000  --use_set_based_memetable=0
Inserting 1000000 key/value pairs
...
Put uesr key comparison:
Count: 1000000  Average: 8.0179  StdDev: 176.34
Min: 0.0000  Median: 2.5555  Max: 88933.0000
Percentiles: P50: 2.56 P75: 2.83 P99: 58.21 P99.9: 133.62 P99.99: 987.50
Get uesr key comparison:
Count: 1000000  Average: 43.4465  StdDev: 379.03
Min: 2.0000  Median: 36.0195  Max: 88939.0000
Percentiles: P50: 36.02 P75: 43.66 P99: 112.98 P99.9: 824.84 P99.99: 7615.38
real	0m21.345s
user	0m14.723s
sys	0m5.677s

-- Sequential insertion of 1M key/value with set based memtable (all in on memtable)
time ./perf_context_test  --total_keys=1000000  --use_set_based_memetable=1
Inserting 1000000 key/value pairs
...
Put uesr key comparison:
Count: 1000000  Average: 61.5022  StdDev: 6.49
Min: 0.0000  Median: 62.4295  Max: 71.0000
Percentiles: P50: 62.43 P75: 66.61 P99: 71.00 P99.9: 71.00 P99.99: 71.00
Get uesr key comparison:
Count: 1000000  Average: 29.3810  StdDev: 3.20
Min: 1.0000  Median: 29.1801  Max: 34.0000
Percentiles: P50: 29.18 P75: 32.06 P99: 34.00 P99.9: 34.00 P99.99: 34.00
real	0m28.875s
user	0m21.699s
sys	0m5.749s

Worst case comparison for a Put is 88933 (skiplist) vs 71 (set based memetable)

Of course, there's other in-efficiency in set based memtable implementation, which lead to the overall worst performance. However, P99 behavior advantage is very very obvious.

Test Plan: ./perf_context_test and viewstate shadow testing

Reviewers: dhruba

Reviewed By: dhruba

CC: leveldb

Differential Revision: https://reviews.facebook.net/D13095

2013-09-25 22:49:18 -07:00

..

.nfs00000000066c9ebb00000002

Enhance db_bench

2013-03-14 16:00:23 -07:00

builder.cc

Replace include/leveldb with include/rocksdb.

2013-08-23 10:51:00 -07:00

builder.h

Revert "Minor fixes found while trying to compile it using clang on Mac OS X"

2013-09-15 23:01:26 -07:00

c_test.c

Replace include/leveldb with include/rocksdb.

2013-08-23 10:51:00 -07:00

c.cc

Replace include/leveldb with include/rocksdb.

2013-08-23 10:51:00 -07:00

corruption_test.cc

Replace include/leveldb with include/rocksdb.

2013-08-23 10:51:00 -07:00

db_bench.cc

Better locking in vectorrep that increases throughput to match speed of storage.

2013-09-19 21:48:10 -07:00

db_filesnapshot.cc

Return pathname relative to db dir in LogFile and cleanup AppendSortedWalsOfType

2013-09-04 13:44:43 -07:00

db_impl_readonly.cc

Replace include/leveldb with include/rocksdb.

2013-08-23 10:51:00 -07:00

db_impl_readonly.h

New ldb command to convert compaction style

2013-09-04 13:13:08 -07:00

db_impl.cc

[RocksDB] Remove Log file immediately after memtable flush

2013-09-12 11:54:44 -07:00

db_impl.h

[RocksDB] Remove Log file immediately after memtable flush

2013-09-12 11:54:44 -07:00

db_iter.cc

An iterator may automatically invoke reseeks.

2013-09-06 11:50:53 -07:00

db_iter.h

Replace include/leveldb with include/rocksdb.

2013-08-23 10:51:00 -07:00

db_statistics.h

Replace include/leveldb with include/rocksdb.

2013-08-23 10:51:00 -07:00

db_stats_logger.cc

Replace include/leveldb with include/rocksdb.

2013-08-23 10:51:00 -07:00

db_test.cc

The vector rep implementation was segfaulting because of incorrect initialization of vector.

2013-09-25 11:33:52 -07:00

dbformat_test.cc

Fix all warnings generated by -Wall option to the compiler.

2012-11-06 14:07:31 -08:00

dbformat.cc

[RocksDB] Added nano second stopwatch and new perf counters to track block read cost

2013-09-07 21:14:54 -07:00

dbformat.h

Replace include/leveldb with include/rocksdb.

2013-08-23 10:51:00 -07:00

deletefile_test.cc

Fix build caused by DeleteFile not tolerating / at the beginning

2013-09-01 17:59:13 -07:00

filename_test.cc

Added meta-database support.

2012-12-17 11:26:59 -08:00

filename.cc

Return pathname relative to db dir in LogFile and cleanup AppendSortedWalsOfType

2013-09-04 13:44:43 -07:00

filename.h

Replace include/leveldb with include/rocksdb.

2013-08-23 10:51:00 -07:00

log_format.h

Fixed sign-comparison in rocksdb code-base and fixed Makefile

2013-03-19 14:35:23 -07:00

log_reader.cc

Replace include/leveldb with include/rocksdb.

2013-08-23 10:51:00 -07:00

log_reader.h

Replace include/leveldb with include/rocksdb.

2013-08-23 10:51:00 -07:00

log_test.cc

Replace include/leveldb with include/rocksdb.

2013-08-23 10:51:00 -07:00

log_writer.cc

Replace include/leveldb with include/rocksdb.

2013-08-23 10:51:00 -07:00

log_writer.h

Replace include/leveldb with include/rocksdb.

2013-08-23 10:51:00 -07:00

memtable.cc

[RocksDB] Remove Log file immediately after memtable flush

2013-09-12 11:54:44 -07:00

memtable.h

[RocksDB] Remove Log file immediately after memtable flush

2013-09-12 11:54:44 -07:00

memtablelist.cc

Flush was hanging because the configured options specified that more than 1 memtable need to be merged.

2013-09-06 16:28:33 -07:00

memtablelist.h

Flush was hanging because the configured options specified that more than 1 memtable need to be merged.

2013-09-06 16:28:33 -07:00

merge_helper.cc

Replace include/leveldb with include/rocksdb.

2013-08-23 10:51:00 -07:00

merge_helper.h

Revert "Minor fixes found while trying to compile it using clang on Mac OS X"

2013-09-15 23:01:26 -07:00

merge_operator.cc

Replace include/leveldb with include/rocksdb.

2013-08-23 10:51:00 -07:00

merge_test.cc

Replace include/leveldb with include/rocksdb.

2013-08-23 10:51:00 -07:00

perf_context_test.cc

[RocbsDB] Add an option to enable set based memtable for perf_context_test

2013-09-25 22:49:18 -07:00

prefix_filter_iterator.h

Replace include/leveldb with include/rocksdb.

2013-08-23 10:51:00 -07:00

repair.cc

Revert "Minor fixes found while trying to compile it using clang on Mac OS X"

2013-09-15 23:01:26 -07:00

skiplist_test.cc

Replace include/leveldb with include/rocksdb.

2013-08-23 10:51:00 -07:00

skiplist.h

Make arena block size configurable

2013-07-31 12:42:23 -07:00

snapshot.h

Replace include/leveldb with include/rocksdb.

2013-08-23 10:51:00 -07:00

table_cache.cc

Introduced a new flag non_blocking_io in ReadOptions.

2013-08-28 10:49:14 -07:00

table_cache.h

Introduced a new flag non_blocking_io in ReadOptions.

2013-08-28 10:49:14 -07:00

transaction_log_impl.cc

API for getting archived log files

2013-08-19 13:37:04 -07:00

transaction_log_impl.h

Return pathname relative to db dir in LogFile and cleanup AppendSortedWalsOfType

2013-09-04 13:44:43 -07:00

version_edit_test.cc

Reduce write amplification by merging files in L0 back into L0

2013-06-30 20:07:04 -07:00

version_edit.cc

Merge remote-tracking branch 'origin' into performance

2013-08-12 09:58:50 -07:00

version_edit.h

Merge remote-tracking branch 'origin' into performance

2013-08-12 09:58:50 -07:00

version_set_reduce_num_levels.cc

Fix valgrind errors in rocksdb tests: auto_roll_logger_test, reduce_levels_test

2013-03-12 16:03:16 -07:00

version_set_test.cc

Codemod NULL to nullptr

2013-02-28 18:04:58 -08:00

version_set.cc

[RocksDB] Universal compaction trigger condition minor fix

2013-09-15 22:35:59 -07:00

version_set.h

Added a parameter to limit the maximum space amplification for universal compaction.

2013-09-13 16:27:18 -07:00

write_batch_internal.h

Replace include/leveldb with include/rocksdb.

2013-08-23 10:51:00 -07:00

write_batch_test.cc

Replace include/leveldb with include/rocksdb.

2013-08-23 10:51:00 -07:00

write_batch.cc

Replace include/leveldb with include/rocksdb.

2013-08-23 10:51:00 -07:00