andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
rocksdb/util
History
Lei Jin 4824014e3b speed up db_bench filluniquerandom mode 
Summary:
filluniquerandom is painfully slow due to the naive bitmap check to find
out if a key has been seen before. Majority of time is spent on searching
the last few keys. Split a giant BitSet to smaller ones so that we can
quickly check if a BitSet is full and thus can skip quickly.

It used to take over one hour to filluniquerandom for 100M keys, now it
takes about 10 mins.

Test Plan:
unit test
also verified correctness in db_bench and make sure all keys are
generated

Reviewers: igor, haobo, yhchiang

Reviewed By: igor

CC: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D17607
2014-04-09 11:25:21 -07:00
..
arena_test.cc
Clean up arena API
2014-01-30 22:10:10 -08:00
arena.cc
A heuristic way to check if a memtable is full
2014-03-12 16:40:14 -07:00
arena.h
Fix HashSkipList and HashLinkedList SIGSEGV
2014-03-14 10:02:04 -07:00
auto_roll_logger_test.cc
Merge branch 'master' into columnfamilies
2014-03-03 18:29:03 -08:00
auto_roll_logger.cc
Merge branch 'master' into columnfamilies
2014-04-02 10:48:05 -07:00
auto_roll_logger.h
Merge branch 'master' into columnfamilies
2014-03-05 16:59:22 -08:00
autovector_test.cc
Fix issue #57
2014-01-06 11:11:19 -08:00
autovector.h
Replace vector with autovector
2014-01-02 16:43:35 -08:00
blob_store_test.cc
Move the compiler back to 4.8.1 + more small fixes
2013-11-18 11:40:16 -08:00
blob_store.cc
IOError cleanup
2014-02-12 11:42:54 -08:00
blob_store.h
Implement max_size in BlobStore
2013-10-23 14:38:52 -07:00
bloom_test.cc
[RocksDB] [Performance Branch] Added dynamic bloom, to be used for memable non-existing key filtering
2013-12-11 00:15:14 -08:00
bloom.cc
Add appropriate LICENSE and Copyright message.
2013-10-16 17:48:41 -07:00
build_version.h
Fixed typos
2013-11-16 11:21:34 +00:00
cache_test.cc
[RocksDB] [Performance Branch] Trivia build fix
2013-12-13 14:21:59 -08:00
cache.cc
Merge branch 'master' into performance
2014-01-28 10:35:55 -08:00
coding_test.cc
Misc cleanup on performance branch
2014-01-17 14:26:29 -08:00
coding.cc
Merge branch 'master' into performance
2014-01-28 10:35:55 -08:00
coding.h
Merge branch 'master' into performance
2014-01-28 10:35:55 -08:00
comparator.cc
Add appropriate LICENSE and Copyright message.
2013-10-16 17:48:41 -07:00
crc32c_test.cc
Add appropriate LICENSE and Copyright message.
2013-10-16 17:48:41 -07:00
crc32c.cc
Dynamically choose SSE 4.2
2014-04-04 14:03:19 -07:00
crc32c.h
Add appropriate LICENSE and Copyright message.
2013-10-16 17:48:41 -07:00
dynamic_bloom_test.cc
Fix some other signed & unsigned comparisons
2014-04-03 20:51:27 -07:00
dynamic_bloom.cc
dynamicbloom fix: don't offset address when it is already aligned
2014-03-28 17:30:20 -07:00
dynamic_bloom.h
cache friendly blocked bloomfilter
2014-03-28 09:21:20 -07:00
env_hdfs.cc
Enable log info with different levels.
2014-02-26 14:41:28 -08:00
env_posix.cc
Fixed a compile error which tries to check whether a size_t < 0 in env_posix.cc
2014-04-01 11:09:06 -07:00
env_test.cc
Some small cleaning up to make some compiling environment happy
2014-03-26 18:11:41 -07:00
env.cc
Merge branch 'master' into columnfamilies
2014-03-18 13:24:27 -07:00
filelock_test.cc
Add appropriate LICENSE and Copyright message.
2013-10-16 17:48:41 -07:00
filter_policy.cc
Add appropriate LICENSE and Copyright message.
2013-10-16 17:48:41 -07:00
hash_linklist_rep.cc
make hash_link_list Node's key space consecutively followed at the end
2014-04-04 15:37:28 -07:00
hash_linklist_rep.h
Consolidate SliceTransform object ownership
2014-03-10 12:56:46 -07:00
hash_skiplist_rep.cc
make hash_link_list Node's key space consecutively followed at the end
2014-04-04 15:37:28 -07:00
hash_skiplist_rep.h
Consolidate SliceTransform object ownership
2014-03-10 12:56:46 -07:00
hash.cc
Add a hash-index component for block
2014-03-03 21:11:49 -08:00
hash.h
Add appropriate LICENSE and Copyright message.
2013-10-16 17:48:41 -07:00
histogram_test.cc
Add appropriate LICENSE and Copyright message.
2013-10-16 17:48:41 -07:00
histogram.cc
Statistics code cleanup
2014-01-17 12:46:06 -08:00
histogram.h
Statistics code cleanup
2014-01-17 12:46:06 -08:00
ldb_cmd_execute_result.h
Add appropriate LICENSE and Copyright message.
2013-10-16 17:48:41 -07:00
ldb_cmd.cc
Column family support for DB::OpenForReadOnly()
2014-04-09 09:56:17 -07:00
ldb_cmd.h
Merge branch 'master' into columnfamilies
2014-03-20 14:41:37 -07:00
ldb_tool.cc
Merge branch 'master' into columnfamilies
2014-03-20 14:41:37 -07:00
log_buffer.cc
Fix data corruption by LogBuffer
2014-03-21 15:32:48 -07:00
log_buffer.h
Fix data race against logging data structure because of LogBuffer
2014-03-11 16:09:53 -07:00
log_write_bench.cc
A simple benchmark to measure WAL append latency
2014-02-24 14:39:32 -08:00
logging.cc
Add appropriate LICENSE and Copyright message.
2013-10-16 17:48:41 -07:00
logging.h
Add appropriate LICENSE and Copyright message.
2013-10-16 17:48:41 -07:00
manual_compaction_test.cc
Fix CompactRange to apply filter to every key
2014-01-14 16:19:09 -08:00
murmurhash.cc
Add appropriate LICENSE and Copyright message.
2013-10-16 17:48:41 -07:00
murmurhash.h
Add a hash-index component for block
2014-03-03 21:11:49 -08:00
mutexlock.h
Add appropriate LICENSE and Copyright message.
2013-10-16 17:48:41 -07:00
options.cc
Initialize Options::bloom_locality
2014-04-07 16:33:03 -07:00
perf_context_imp.h
macros for perf_context
2014-04-08 10:58:07 -07:00
perf_context.cc
macros for perf_context
2014-04-08 10:58:07 -07:00
posix_logger.h
Enable log info with different levels.
2014-02-26 14:41:28 -08:00
random.h
Add appropriate LICENSE and Copyright message.
2013-10-16 17:48:41 -07:00
signal_test.cc
Add appropriate LICENSE and Copyright message.
2013-10-16 17:48:41 -07:00
skiplistrep.cc
make hash_link_list Node's key space consecutively followed at the end
2014-04-04 15:37:28 -07:00
slice.cc
Add appropriate LICENSE and Copyright message.
2013-10-16 17:48:41 -07:00
stack_trace.h
Print stack trace on assertion failure
2013-12-06 17:11:09 -08:00
statistics.cc
convert Tickers back to array with padding and alignment
2014-01-29 15:08:41 -08:00
statistics.h
cache SuperVersion in thread local storage to avoid mutex lock
2014-02-27 11:38:55 -08:00
stats_logger.h
Add appropriate LICENSE and Copyright message.
2013-10-16 17:48:41 -07:00
status.cc
IOError cleanup
2014-02-12 11:42:54 -08:00
stl_wrappers.h
Killing Transform Rep
2013-12-03 12:42:15 -08:00
stop_watch.h
Statistics code cleanup
2014-01-17 12:46:06 -08:00
string_util.cc
Add appropriate LICENSE and Copyright message.
2013-10-16 17:48:41 -07:00
string_util.h
Add appropriate LICENSE and Copyright message.
2013-10-16 17:48:41 -07:00
sync_point.cc
[RocksDB] Fix a race condition in GetSortedWalFiles
2014-04-02 22:12:29 -07:00
sync_point.h
[RocksDB] Fix a race condition in GetSortedWalFiles
2014-04-02 22:12:29 -07:00
testharness.cc
Add appropriate LICENSE and Copyright message.
2013-10-16 17:48:41 -07:00
testharness.h
Print stack trace on assertion failure
2013-12-06 17:11:09 -08:00
testutil.cc
Add appropriate LICENSE and Copyright message.
2013-10-16 17:48:41 -07:00
testutil.h
[Performance Branch] PlainTable to encode rows with seqID 0, value type using 1 internal byte.
2014-02-03 12:19:30 -08:00
thread_local_test.cc
reduce thread count in ThreadLocalTest.ConcurrentReadWriteTest
2014-04-01 16:26:07 -07:00
thread_local.cc
use CAS when returning SuperVersion to ThreadLocal
2014-03-07 14:43:22 -08:00
thread_local.h
Merge branch 'master' into columnfamilies
2014-03-07 16:59:47 -08:00
vectorrep.cc
make hash_link_list Node's key space consecutively followed at the end
2014-04-04 15:37:28 -07:00