andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
c11d604ab3
rocksdb/db
History
Feng Zhu c11d604ab3 store file_indexer info in sequential memory 
Summary:
  use arena to allocate space for next_level_index_ and level_rb_
  Thus increasing data locality and make Version::Get faster.

Benchmark detail
Base version: commit d2a727c182

command used:
./db_bench --db=/mnt/db/rocksdb --num_levels=6 --key_size=20 --prefix_size=20 --keys_per_prefix=0 --value_size=100 --block_size=4096 --cache_size=17179869184 --cache_numshardbits=6 --compression_type=none --compression_ratio=1 --min_level_to_compress=-1 --disable_seek_compaction=1 --hard_rate_limit=2 --write_buffer_size=134217728 --max_write_buffer_number=2 --level0_file_num_compaction_trigger=8 --target_file_size_base=2097152 --max_bytes_for_level_base=1073741824 --disable_wal=0 --sync=0 --disable_data_sync=1 --verify_checksum=1 --delete_obsolete_files_period_micros=314572800 --max_grandparent_overlap_factor=10 --max_background_compactions=4 --max_background_flushes=0 --level0_slowdown_writes_trigger=16 --level0_stop_writes_trigger=24 --statistics=0 --stats_per_interval=0 --stats_interval=1048576 --histogram=0 --use_plain_table=1 --open_files=-1 --mmap_read=1 --mmap_write=0 --memtablerep=prefix_hash --bloom_bits=10 --bloom_locality=1 --perf_level=0 --benchmarks=fillseq, readrandom,readrandom,readrandom --use_existing_db=0 --num=52428800 --threads=1

Result:
cpu running percentage:
Version::Get, improved from 7.98% to 7.42%
FileIndexer::GetNextLevelIndex, improved from 1.18% to 0.68%.

Test Plan:
  make all check

Reviewers: ljin, haobo, yhchiang, sdong

Reviewed By: sdong

Subscribers: dhruba, igor

Differential Revision: https://reviews.facebook.net/D19845
2014-07-16 11:21:30 -07:00
..
builder.cc integrate rate limiter into rocksdb 2014-07-08 12:31:49 -07:00
builder.h integrate rate limiter into rocksdb 2014-07-08 12:31:49 -07:00
c_test.c C API: create missing cf's, cleanup 2014-07-10 12:55:53 -07:00
c.cc C API: create missing cf's, cleanup 2014-07-10 12:55:53 -07:00
column_family_test.cc Remove seek compaction 2014-06-20 10:23:02 +02:00
column_family.cc Support Multiple DB paths (without having an interface to expose to users) 2014-07-02 21:14:44 -07:00
column_family.h Cache some conditions for DBImpl::MakeRoomForWrite 2014-06-26 16:45:27 -07:00
compaction_picker.cc Support multiple DB directories in universal compaction style 2014-07-15 12:06:28 -07:00
compaction_picker.h Support multiple DB directories in universal compaction style 2014-07-15 12:06:28 -07:00
compaction.cc use FileLevel in LevelFileNumIterator 2014-07-11 12:52:41 -07:00
compaction.h use FileLevel in LevelFileNumIterator 2014-07-11 12:52:41 -07:00
corruption_test.cc Fix corruption test 2014-04-24 14:56:41 -04:00
db_bench.cc Adding NUMA support to db_bench tests 2014-07-07 10:53:31 -07:00
db_filesnapshot.cc Support Multiple DB paths (without having an interface to expose to users) 2014-07-02 21:14:44 -07:00
db_impl_debug.cc FIFO compaction style 2014-05-21 11:43:35 -07:00
db_impl_readonly.cc Remove seek compaction 2014-06-20 10:23:02 +02:00
db_impl_readonly.h Remove stats logger 2014-07-15 09:16:32 -04:00
db_impl.cc Support multiple DB directories in universal compaction style 2014-07-15 12:06:28 -07:00
db_impl.h Remove stats logger 2014-07-15 09:16:32 -04:00
db_iter.cc Add Prev() for merge operator 2014-07-15 16:10:18 -07:00
db_iter.h In DB::NewIterator(), try to allocate the whole iterator tree in an arena 2014-06-02 17:44:57 -07:00
db_test.cc Add Prev() for merge operator 2014-07-15 16:10:18 -07:00
dbformat_test.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
dbformat.cc macros for perf_context 2014-04-08 10:58:07 -07:00
dbformat.h HashLinkList memtable switches a bucket to a skip list to reduce performance outliers 2014-07-01 17:14:15 -07:00
deletefile_test.cc Start DeleteFileTest with clean plate 2013-11-15 16:30:23 -08:00
file_indexer_test.cc store file_indexer info in sequential memory 2014-07-16 11:21:30 -07:00
file_indexer.cc store file_indexer info in sequential memory 2014-07-16 11:21:30 -07:00
file_indexer.h store file_indexer info in sequential memory 2014-07-16 11:21:30 -07:00
filename_test.cc Support multiple DB directories in universal compaction style 2014-07-15 12:06:28 -07:00
filename.cc Support multiple DB directories in universal compaction style 2014-07-15 12:06:28 -07:00
filename.h Support multiple DB directories in universal compaction style 2014-07-15 12:06:28 -07:00
forward_iterator.cc ForwardIterator seek bugfix 2014-07-10 16:46:13 -07:00
forward_iterator.h Fix compile errors on Mac 2014-06-03 12:28:58 -07:00
internal_stats.cc BG -> GB 2014-07-14 09:06:38 -07:00
internal_stats.h improve InternalStats output 2014-07-11 15:03:30 -07:00
log_and_apply_bench.cc Support Multiple DB paths (without having an interface to expose to users) 2014-07-02 21:14:44 -07:00
log_format.h Some minor refactoring on the code 2014-01-02 16:32:31 -08:00
log_reader.cc Make Log::Reader more robust 2014-02-28 13:19:47 -08:00
log_reader.h Fix UnmarkEOF for partial blocks 2014-01-27 14:49:10 -08:00
log_test.cc Make it compile on Debian/GCC 4.7 2014-03-14 22:44:35 +00:00
log_writer.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
log_writer.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
memtable_list.cc Support Multiple DB paths (without having an interface to expose to users) 2014-07-02 21:14:44 -07:00
memtable_list.h Support Multiple DB paths (without having an interface to expose to users) 2014-07-02 21:14:44 -07:00
memtable.cc use arena to allocate memtable's bloomfilter and hashskiplist's buckets_ 2014-06-30 15:54:31 -07:00
memtable.h In DB::NewIterator(), try to allocate the whole iterator tree in an arena 2014-06-02 17:44:57 -07:00
merge_context.h Enhance partial merge to support multiple arguments 2014-03-24 17:57:13 -07:00
merge_helper.cc Fix the bug in MergeUtil which causes mixing values of different keys. 2014-03-27 16:15:25 -07:00
merge_helper.h [rocksdb] new CompactionFilterV2 API 2014-03-24 20:47:53 -07:00
merge_operator.cc Some small cleaning up to make some compiling environment happy 2014-03-26 18:11:41 -07:00
merge_test.cc Use new DBWithTTL API in tests 2014-04-28 23:46:24 -04:00
perf_context_test.cc Missing includes 2014-03-14 13:02:20 -07:00
plain_table_db_test.cc Removing NewTotalOrderPlainTableFactory 2014-07-10 11:32:04 -07:00
prefix_test.cc HashLinkList memtable switches a bucket to a skip list to reduce performance outliers 2014-07-01 17:14:15 -07:00
repair.cc Support multiple DB directories in universal compaction style 2014-07-15 12:06:28 -07:00
simple_table_db_test.cc Remove seek compaction 2014-06-20 10:23:02 +02:00
skiplist_test.cc Clean up arena API 2014-01-30 22:10:10 -08:00
skiplist.h Consolidate SliceTransform object ownership 2014-03-10 12:56:46 -07:00
snapshot.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
table_cache.cc Support Multiple DB paths (without having an interface to expose to users) 2014-07-02 21:14:44 -07:00
table_cache.h Support multiple DB directories in universal compaction style 2014-07-15 12:06:28 -07:00
table_properties_collector_test.cc TablePropertiesCollectorFactory 2014-05-13 12:30:55 -07:00
table_properties_collector.cc Extract metaindex block from block-based table 2013-12-05 16:34:16 -08:00
table_properties_collector.h TablePropertiesCollectorFactory 2014-05-13 12:30:55 -07:00
tailing_iter.cc kill ReadOptions.prefix and .prefix_seek 2014-04-25 12:21:34 -07:00
tailing_iter.h kill ReadOptions.prefix and .prefix_seek 2014-04-25 12:21:34 -07:00
transaction_log_impl.cc Fixed a file-not-found issue when a log file is moved to archive. 2014-05-12 17:50:21 -07:00
transaction_log_impl.h RocksDBLite 2014-04-15 13:39:26 -07:00
version_edit_test.cc Support Multiple DB paths (without having an interface to expose to users) 2014-07-02 21:14:44 -07:00
version_edit.cc Support Multiple DB paths (without having an interface to expose to users) 2014-07-02 21:14:44 -07:00
version_edit.h use FileLevel in LevelFileNumIterator 2014-07-11 12:52:41 -07:00
version_set_test.cc use FileLevel in LevelFileNumIterator 2014-07-11 12:52:41 -07:00
version_set.cc store file_indexer info in sequential memory 2014-07-16 11:21:30 -07:00
version_set.h store file_indexer info in sequential memory 2014-07-16 11:21:30 -07:00
write_batch_internal.h JSON (Document) API sketch 2014-07-10 09:31:42 -07:00
write_batch_test.cc kill ReadOptions.prefix and .prefix_seek 2014-04-25 12:21:34 -07:00
write_batch.cc JSON (Document) API sketch 2014-07-10 09:31:42 -07:00