andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
0f2d768191
rocksdb/db
History
Lei Jin 0f2d768191 hints for narrowing down FindFile range and avoiding checking unrelevant L0 files 
Summary:
The file tree structure in Version is prebuilt and the range of each file is known.
On the Get() code path, we do binary search in FindFile() by comparing
target key with each file's largest key and also check the range for each L0 file.
With some pre-calculated knowledge, each key comparision that has been done can serve
as a hint to narrow down further searches:
(1) If a key falls within a L0 file's range, we can safely skip the next
file if its range does not overlap with the current one.
(2) If a key falls within a file's range in level L0 - Ln-1, we should only
need to binary search in the next level for files that overlap with the current one.

(1) will be able to skip some files depending one the key distribution.
(2) can greatly reduce the range of binary search, especially for bottom
levels, given that one file most likely only overlaps with N files from
the level below (where N is max_bytes_for_level_multiplier). So on level
L, we will only look at ~N files instead of N^L files.

Some inital results: measured with 500M key DB, when write is light (10k/s = 1.2M/s), this
improves QPS ~7% on top of blocked bloom. When write is heavier (80k/s =
9.6M/s), it gives us ~13% improvement.

Test Plan: make all check

Reviewers: haobo, igor, dhruba, sdong, yhchiang

Reviewed By: haobo

CC: leveldb

Differential Revision: https://reviews.facebook.net/D17205
2014-04-21 09:10:12 -07:00
..
builder.cc Enhance partial merge to support multiple arguments 2014-03-24 17:57:13 -07:00
builder.h [Performance Branch] PlainTable to encode rows with seqID 0, value type using 1 internal byte. 2014-02-03 12:19:30 -08:00
c_test.c Turn on -Wmissing-prototypes 2014-04-09 21:17:14 -07:00
c.cc RocksDBLite 2014-04-15 13:39:26 -07:00
column_family_test.cc Close DB at the end of DontRollEmptyLogs test 2014-04-15 17:20:56 -07:00
column_family.cc Don't overflow size_t in mac 2014-04-16 15:15:22 -07:00
column_family.h thread local for tailing iterator 2014-04-14 10:48:01 -07:00
compaction_picker.cc Merge branch 'master' into columnfamilies 2014-04-07 11:31:11 -07:00
compaction_picker.h [CF] Code cleanup part 1 2014-03-12 09:56:53 -07:00
compaction.cc Merge branch 'master' into columnfamilies 2014-03-19 17:22:20 -07:00
compaction.h Merge branch 'master' into columnfamilies 2014-02-12 14:01:30 -08:00
corruption_test.cc make corruption_test:CompactionInputErrorParanoid deterministic 2014-04-11 12:48:38 -07:00
db_bench.cc Fix db_bench's multireadrandom 2014-04-14 15:43:34 -07:00
db_filesnapshot.cc RocksDBLite 2014-04-15 13:39:26 -07:00
db_impl_debug.cc Fix compile issues when doing make release 2014-04-15 16:00:10 -07:00
db_impl_readonly.cc Minimize accessing multiple objects in Version::Get() 2014-04-17 14:14:00 -07:00
db_impl_readonly.h [CF] Rethinking ColumnFamilyHandle and fix to dropping column families 2014-02-12 13:47:09 -08:00
db_impl.cc Minimize accessing multiple objects in Version::Get() 2014-04-17 14:14:00 -07:00
db_impl.h Fix compile issues when doing make release 2014-04-15 16:00:10 -07:00
db_iter.cc thread local for tailing iterator 2014-04-14 10:48:01 -07:00
db_iter.h thread local for tailing iterator 2014-04-14 10:48:01 -07:00
db_stats_logger.cc RocksDBLite 2014-04-15 13:39:26 -07:00
db_test.cc When creating a new DB, fail it when wal_dir contains existing log files 2014-04-15 14:01:57 -07:00
dbformat_test.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
dbformat.cc macros for perf_context 2014-04-08 10:58:07 -07:00
dbformat.h Enable hash index for block-based table 2014-04-10 14:19:43 -07:00
deletefile_test.cc Start DeleteFileTest with clean plate 2013-11-15 16:30:23 -08:00
file_indexer_test.cc hints for narrowing down FindFile range and avoiding checking unrelevant L0 files 2014-04-21 09:10:12 -07:00
file_indexer.cc hints for narrowing down FindFile range and avoiding checking unrelevant L0 files 2014-04-21 09:10:12 -07:00
file_indexer.h hints for narrowing down FindFile range and avoiding checking unrelevant L0 files 2014-04-21 09:10:12 -07:00
filename_test.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
filename.cc Remove tautological assert 2014-04-16 09:09:28 -07:00
filename.h Unify DeleteFile and DeleteWalFiles 2013-10-25 08:32:14 -07:00
internal_stats.cc Merge branch 'master' into columnfamilies 2014-03-31 13:39:24 -07:00
internal_stats.h Merge branch 'master' into columnfamilies 2014-03-31 13:39:24 -07:00
log_format.h Some minor refactoring on the code 2014-01-02 16:32:31 -08:00
log_reader.cc Make Log::Reader more robust 2014-02-28 13:19:47 -08:00
log_reader.h Fix UnmarkEOF for partial blocks 2014-01-27 14:49:10 -08:00
log_test.cc Make it compile on Debian/GCC 4.7 2014-03-14 22:44:35 +00:00
log_writer.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
log_writer.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
memtable_list.cc Merge branch 'master' into columnfamilies 2014-04-07 11:31:11 -07:00
memtable_list.h Merge branch 'master' into columnfamilies 2014-04-07 11:31:11 -07:00
memtable.cc macros for perf_context 2014-04-08 10:58:07 -07:00
memtable.h Merge branch 'master' into columnfamilies 2014-03-13 13:21:20 -07:00
merge_context.h Enhance partial merge to support multiple arguments 2014-03-24 17:57:13 -07:00
merge_helper.cc Fix the bug in MergeUtil which causes mixing values of different keys. 2014-03-27 16:15:25 -07:00
merge_helper.h [rocksdb] new CompactionFilterV2 API 2014-03-24 20:47:53 -07:00
merge_operator.cc Some small cleaning up to make some compiling environment happy 2014-03-26 18:11:41 -07:00
merge_test.cc Fix merge_test failure due to incorrect assert behavior in the release mode. 2014-04-14 12:06:49 -07:00
perf_context_test.cc Missing includes 2014-03-14 13:02:20 -07:00
plain_table_db_test.cc Turn on -Wmissing-prototypes 2014-04-09 21:17:14 -07:00
prefix_filter_iterator.h Moving Some includes from options.h to forward declaration 2014-01-24 17:16:22 -08:00
prefix_test.cc Turn on -Wmissing-prototypes 2014-04-09 21:17:14 -07:00
repair.cc RocksDBLite 2014-04-15 13:39:26 -07:00
simple_table_db_test.cc Make table properties shareable 2014-02-07 19:26:49 -08:00
skiplist_test.cc Clean up arena API 2014-01-30 22:10:10 -08:00
skiplist.h Consolidate SliceTransform object ownership 2014-03-10 12:56:46 -07:00
snapshot.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
table_cache.cc Fix bugs introduced by D17961 2014-04-17 17:25:28 -07:00
table_cache.h Merge branch 'master' into columnfamilies 2014-04-07 11:31:11 -07:00
table_properties_collector_test.cc Turn on -Wmissing-prototypes 2014-04-09 21:17:14 -07:00
table_properties_collector.cc Extract metaindex block from block-based table 2013-12-05 16:34:16 -08:00
table_properties_collector.h Extract metaindex block from block-based table 2013-12-05 16:34:16 -08:00
tailing_iter.cc RocksDBLite 2014-04-15 13:39:26 -07:00
tailing_iter.h RocksDBLite 2014-04-15 13:39:26 -07:00
transaction_log_impl.cc RocksDBLite 2014-04-15 13:39:26 -07:00
transaction_log_impl.h RocksDBLite 2014-04-15 13:39:26 -07:00
version_edit_test.cc [CF] Dont reuse dropped column family IDs 2014-03-05 12:13:44 -08:00
version_edit.cc Merge branch 'master' into columnfamilies 2014-03-31 13:39:24 -07:00
version_edit.h Minimize accessing multiple objects in Version::Get() 2014-04-17 14:14:00 -07:00
version_set_test.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
version_set.cc hints for narrowing down FindFile range and avoiding checking unrelevant L0 files 2014-04-21 09:10:12 -07:00
version_set.h hints for narrowing down FindFile range and avoiding checking unrelevant L0 files 2014-04-21 09:10:12 -07:00
write_batch_internal.h [CF] Code cleanup part 1 2014-03-12 09:56:53 -07:00
write_batch_test.cc Change WriteBatch interface 2014-03-14 13:40:06 -07:00
write_batch.cc [CF] WriteBatch to take in ColumnFamilyHandle 2014-03-14 11:30:14 -07:00