andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
rocksdb/db
History
Islam AbdelRahman 8c71eb5afc Optimize DBIter::Prev() by reducing stack overhead 
Summary:
It looks like we are spending significant amount of time creating std::deque<std::string> every time we do Iterator::Prev()

{F921567}

By using merge_operands_ as a DBIter data member w create it once and reduce this overhead and see ~30% performance improvement when using Iterator::Prev() on hot data

Orignal performance

```
DEBUG_LEVEL=0 make db_bench -j64 && ./db_bench --benchmarks="readreverse" --db="/dev/shm/bench_prev_opt/" --use_existing_db --disable_auto_compactions
readreverse  :       0.713 micros/op 1402219 ops/sec;  155.1 MB/s
readreverse  :       0.609 micros/op 1641386 ops/sec;  181.6 MB/s
readreverse  :       0.684 micros/op 1461150 ops/sec;  161.6 MB/s
readreverse  :       0.629 micros/op 1589842 ops/sec;  175.9 MB/s
readreverse  :       0.647 micros/op 1544530 ops/sec;  170.9 MB/s
```

After optimization

```
DEBUG_LEVEL=0 make db_bench -j64 && ./db_bench --benchmarks="readreverse" --db="/dev/shm/bench_prev_opt/" --use_existing_db --disable_auto_compactions
readreverse  :       0.488 micros/op 2051189 ops/sec;  226.9 MB/s
readreverse  :       0.505 micros/op 1980892 ops/sec;  219.1 MB/s
readreverse  :       0.541 micros/op 1846971 ops/sec;  204.3 MB/s
readreverse  :       0.497 micros/op 2013612 ops/sec;  222.8 MB/s
readreverse  :       0.480 micros/op 2082665 ops/sec;  230.4 MB/s
```

Test Plan: make check -j64

Reviewers: sdong, anthony, rven, igor, yhchiang

Reviewed By: yhchiang

Subscribers: jkedgar, dhruba

Differential Revision: https://reviews.facebook.net/D52563
2016-01-07 07:59:14 -08:00
..
builder.cc
Use SST files for Transaction conflict detection
2015-12-11 12:34:11 -08:00
builder.h
Use SST files for Transaction conflict detection
2015-12-11 12:34:11 -08:00
c_test.c
Deprecate CompactionFilterV2
2015-07-17 18:59:11 +02:00
c.cc
Merge pull request #916 from warrenfalk/capi_huge_page_option
2016-01-04 09:24:30 -08:00
column_family_test.cc
Add ColumnFamilyHandle::GetDescriptor()
2016-01-06 18:14:01 -08:00
column_family.cc
Add ColumnFamilyHandle::GetDescriptor()
2016-01-06 18:14:01 -08:00
column_family.h
Add ColumnFamilyHandle::GetDescriptor()
2016-01-06 18:14:01 -08:00
compact_files_test.cc
Make sure that CompactFiles does not run two parallel Level 0 compactions
2015-11-13 12:01:00 -08:00
compacted_db_impl.cc
Remove db_impl_readonly dependency on utilities
2015-07-14 11:32:54 -07:00
compacted_db_impl.h
Remove db_impl_readonly dependency on utilities
2015-07-14 11:32:54 -07:00
compaction_iterator_test.cc
Support marking snapshots for write-conflict checking - Take 2
2015-12-08 16:47:31 -08:00
compaction_iterator.cc
compaction assertion triggering test fix for sequence zeroing assertion trip
2015-12-18 16:08:31 -08:00
compaction_iterator.h
Change SingleDelete to support conflict checking
2015-12-10 11:35:38 -08:00
compaction_job_stats_test.cc
Fix rebase issues and new code warnings.
2015-12-11 16:56:24 -08:00
compaction_job_test.cc
compaction assertion triggering test fix for sequence zeroing assertion trip
2015-12-18 16:08:31 -08:00
compaction_job.cc
Merge pull request #846 from yuslepukhin/enble_c4244_lossofdata
2015-12-23 22:59:42 -08:00
compaction_job.h
fix typos in comments
2015-12-11 01:54:48 +09:00
compaction_picker_test.cc
Fix condition for bottommost level
2015-10-05 17:40:18 -07:00
compaction_picker.cc
Report compaction reason in CompactionListener
2015-12-22 11:37:19 -08:00
compaction_picker.h
Fix minor bugs in delete operator, snprintf, and size_t usage
2015-12-15 15:26:20 -08:00
compaction.cc
Merge pull request #846 from yuslepukhin/enble_c4244_lossofdata
2015-12-23 22:59:42 -08:00
compaction.h
Report compaction reason in CompactionListener
2015-12-22 11:37:19 -08:00
comparator_db_test.cc
Moving memtable related files from util to a new directory memtable
2015-10-16 14:10:33 -07:00
convenience.cc
Delete files in given key range
2015-12-29 13:22:13 -08:00
corruption_test.cc
Merge pull request #846 from yuslepukhin/enble_c4244_lossofdata
2015-12-23 22:59:42 -08:00
cuckoo_table_db_test.cc
Block cuckoo table tests in ROCKSDB_LITE
2015-07-20 10:50:46 -07:00
db_bench.cc
Enhance db_bench write rate limit
2016-01-04 12:01:27 -08:00
db_compaction_filter_test.cc
compaction assertion triggering test fix for sequence zeroing assertion trip
2015-12-18 16:08:31 -08:00
db_compaction_test.cc
DeleteFilesInRange: Clean job context if no files deleted
2016-01-04 10:55:31 -08:00
db_dynamic_level_test.cc
No need to #ifdef test only code on windows
2015-10-22 15:15:37 -07:00
db_filesnapshot.cc
Add wal files to Checkpoint for multiple column families.
2015-06-19 16:08:31 -07:00
db_impl_debug.cc
Running manual compactions in parallel with other automatic or manual compactions in restricted cases
2015-12-14 11:20:34 -08:00
db_impl_experimental.cc
Clean up InstallSuperVersion
2015-06-17 12:37:59 -07:00
db_impl_readonly.cc
Remove db_impl_readonly dependency on utilities
2015-07-14 11:32:54 -07:00
db_impl_readonly.h
Override DBImplReadOnly::SyncWAL() to return NotSupported. Previously, calling it caused program abort.
2015-09-25 21:25:30 -07:00
db_impl.cc
Optimize GetLatestSequenceForKey
2016-01-06 13:43:22 -08:00
db_impl.h
Delete files in given key range
2015-12-29 13:22:13 -08:00
db_inplace_update_test.cc
Clean up dependency: Move db_test_util.* to db directory
2015-10-12 13:05:42 -07:00
db_iter_test.cc
No need to #ifdef test only code on windows
2015-10-22 15:15:37 -07:00
db_iter.cc
Optimize DBIter::Prev() by reducing stack overhead
2016-01-07 07:59:14 -08:00
db_iter.h
Introduce ReadOptions::pin_data (support zero copy for keys)
2015-12-16 12:08:30 -08:00
db_log_iter_test.cc
Fix clang warnings regarding unnecessary std::move
2015-12-24 04:10:00 +08:00
db_table_properties_test.cc
Avoid empty ranges vector with subsequent zero element access
2015-12-02 14:50:33 -08:00
db_tailing_iter_test.cc
Fix minor bugs in delete operator, snprintf, and size_t usage
2015-12-15 15:26:20 -08:00
db_test_util.cc
Merge pull request #853 from Vaisman/enable_C4267_warning
2015-12-08 17:59:24 -08:00
db_test_util.h
Merge pull request #853 from Vaisman/enable_C4267_warning
2015-12-08 17:59:24 -08:00
db_test.cc
fix potential test SleepingTask race condition
2016-01-06 11:51:09 -08:00
db_universal_compaction_test.cc
Running manual compactions in parallel with other automatic or manual compactions in restricted cases
2015-12-14 11:20:34 -08:00
db_wal_test.cc
No need to #ifdef test only code on windows
2015-10-22 15:15:37 -07:00
dbformat_test.cc
Avoid manipulating const char* arrays
2015-07-14 00:21:41 -07:00
dbformat.cc
Support for SingleDelete()
2015-09-17 11:42:56 -07:00
dbformat.h
Introduce ReadOptions::pin_data (support zero copy for keys)
2015-12-16 12:08:30 -08:00
deletefile_test.cc
Fix minor bugs in delete operator, snprintf, and size_t usage
2015-12-15 15:26:20 -08:00
event_helpers.cc
Enable MS compiler warning c4244.
2015-12-11 16:47:34 -08:00
event_helpers.h
Add EventListener::OnTableFileDeletion()
2015-06-03 19:57:01 -07:00
experimental.cc
Implement DB::PromoteL0 method
2015-04-23 12:10:36 -07:00
fault_injection_test.cc
fix sporadic failure in fault_injection_test
2016-01-06 11:51:53 -08:00
file_indexer_test.cc
Fix possible SIGSEGV in CompactRange (github issue #596)
2015-04-29 10:52:31 -07:00
file_indexer.cc
Fix possible SIGSEGV in CompactRange (github issue #596)
2015-04-29 10:52:31 -07:00
file_indexer.h
fix typos in comments
2015-12-11 01:54:48 +09:00
filename_test.cc
rocksdb: switch to gtest
2015-03-17 14:08:00 -07:00
filename.cc
Enable RocksDB to persist Options file.
2015-11-10 22:58:01 -08:00
filename.h
Enable RocksDB to persist Options file.
2015-11-10 22:58:01 -08:00
flush_job_test.cc
Use SST files for Transaction conflict detection
2015-12-11 12:34:11 -08:00
flush_job.cc
Use SST files for Transaction conflict detection
2015-12-11 12:34:11 -08:00
flush_job.h
Use SST files for Transaction conflict detection
2015-12-11 12:34:11 -08:00
flush_scheduler.cc
support for concurrent adds to memtable
2015-12-25 11:03:40 -08:00
flush_scheduler.h
support for concurrent adds to memtable
2015-12-25 11:03:40 -08:00
forward_iterator_bench.cc
Block forward_iterator_bench under MAC and Windows
2015-11-17 11:51:37 -08:00
forward_iterator.cc
Fix minor bugs in delete operator, snprintf, and size_t usage
2015-12-15 15:26:20 -08:00
forward_iterator.h
Reuse file iterators in tailing iterator when memtable is flushed
2015-11-13 15:50:59 -08:00
inlineskiplist_test.cc
support for concurrent adds to memtable
2015-12-25 11:03:40 -08:00
inlineskiplist.h
support for concurrent adds to memtable
2015-12-25 11:03:40 -08:00
internal_stats.cc
support for concurrent adds to memtable
2015-12-25 11:03:40 -08:00
internal_stats.h
support for concurrent adds to memtable
2015-12-25 11:03:40 -08:00
job_context.h
fixed leaking log::Writers
2015-07-07 12:10:10 -07:00
listener_test.cc
Report compaction reason in CompactionListener
2015-12-22 11:37:19 -08:00
log_format.h
log_{reader,write}: recyclable record format
2015-10-19 17:24:05 -04:00
log_reader.cc
log_{reader,write}: recyclable record format
2015-10-19 17:24:05 -04:00
log_reader.h
log_{reader,write}: recyclable record format
2015-10-19 17:24:05 -04:00
log_test.cc
log_{reader,write}: recyclable record format
2015-10-19 17:24:05 -04:00
log_writer.cc
Enable C4267 warning
2015-11-24 16:33:09 +03:00
log_writer.h
Enable C4267 warning
2015-11-24 16:33:09 +03:00
managed_iterator.cc
Windows Port from Microsoft
2015-07-01 16:13:56 -07:00
managed_iterator.h
Fixed xfunc related compile errors in ROCKSDB_LITE
2015-04-09 21:05:18 -07:00
manual_compaction_test.cc
Move manual_compaction_test.cc from util to db
2015-10-14 11:06:27 -07:00
memtable_allocator.cc
support for concurrent adds to memtable
2015-12-25 11:03:40 -08:00
memtable_allocator.h
support for concurrent adds to memtable
2015-12-25 11:03:40 -08:00
memtable_list_test.cc
Removing duplicate code
2015-08-05 07:33:27 -07:00
memtable_list.cc
Seperate InternalIterator from Iterator
2015-10-13 15:32:13 -07:00
memtable_list.h
Seperate InternalIterator from Iterator
2015-10-13 15:32:13 -07:00
memtable.cc
Optimize GetLatestSequenceForKey
2016-01-06 13:43:22 -08:00
memtable.h
support for concurrent adds to memtable
2015-12-25 11:03:40 -08:00
memtablerep_bench.cc
Merge pull request #811 from OverlordQ/unused-variable-warning
2015-11-02 12:44:27 -08:00
merge_context.h
API to fetch from both a WriteBatchWithIndex and the db
2015-05-11 14:51:51 -07:00
merge_helper_test.cc
Compaction filter on merge operands
2015-10-07 09:30:03 -07:00
merge_helper.cc
Seperate InternalIterator from Iterator
2015-10-13 15:32:13 -07:00
merge_helper.h
Seperate InternalIterator from Iterator
2015-10-13 15:32:13 -07:00
merge_operator.cc
Call merge operators with empty values
2015-06-26 11:35:46 -07:00
merge_test.cc
Fix minor bugs in delete operator, snprintf, and size_t usage
2015-12-15 15:26:20 -08:00
options_file_test.cc
Fixed build failure of RocksDBLite test on options_file_test.cc
2015-11-10 23:23:36 -08:00
perf_context_test.cc
Make perf_context.db_mutex_lock_nanos and db_condition_wait_nanos only measures DB Mutex
2015-10-13 10:41:48 -07:00
plain_table_db_test.cc
PlainTableReader to support non-mmap mode
2015-09-23 11:41:07 -07:00
prefix_test.cc
Prefix-based iterating only shows keys in prefix
2015-11-05 13:24:05 -08:00
repair.cc
Disable Visual Studio Warning C4351
2015-12-28 15:06:34 -08:00
skiplist_test.cc
rocksdb: switch to gtest
2015-03-17 14:08:00 -07:00
skiplist.h
Switch to thread-local random for skiplist
2015-11-09 19:25:22 -08:00
slice.cc
Create an abstract interface for write batches
2015-03-17 19:23:08 -07:00
snapshot_impl.cc
Support marking snapshots for write-conflict checking - Take 2
2015-12-08 16:47:31 -08:00
snapshot_impl.h
Use SST files for Transaction conflict detection
2015-12-11 12:34:11 -08:00
table_cache.cc
Skip bottom-level filter block caching when hit-optimized
2015-12-23 10:15:07 -08:00
table_cache.h
Skip bottom-level filter block caching when hit-optimized
2015-12-23 10:15:07 -08:00
table_properties_collector_test.cc
Pass column family ID to table property collector
2015-10-09 14:36:51 -07:00
table_properties_collector.cc
Support for SingleDelete()
2015-09-17 11:42:56 -07:00
table_properties_collector.h
Pass column family ID to table property collector
2015-10-09 14:36:51 -07:00
transaction_log_impl.cc
log_reader: pass log_number and optional info_log to ctor
2015-10-18 21:24:32 -04:00
transaction_log_impl.h
Move rate_limiter, write buffering, most perf context instrumentation and most random kill out of Env
2015-07-17 16:58:18 -07:00
version_builder_test.cc
Add a mode to always pick the oldest file to compact for each level
2015-09-21 17:21:59 -07:00
version_builder.cc
Fix minor bugs in delete operator, snprintf, and size_t usage
2015-12-15 15:26:20 -08:00
version_builder.h
Log more information for the add file with overlapping range failure
2015-10-19 17:31:13 -07:00
version_edit_test.cc
New Manifest format to allow customized fields in NewFile.
2015-10-08 15:51:45 -07:00
version_edit.cc
New Manifest format to allow customized fields in NewFile.
2015-10-08 15:51:45 -07:00
version_edit.h
Enable MS compiler warning c4244.
2015-12-11 16:47:34 -08:00
version_set_test.cc
Fix minor bugs in delete operator, snprintf, and size_t usage
2015-12-15 15:26:20 -08:00
version_set.cc
Merge pull request #846 from yuslepukhin/enble_c4244_lossofdata
2015-12-23 22:59:42 -08:00
version_set.h
When slowdown is triggered, reduce the write rate
2015-12-23 11:33:15 -08:00
wal_manager_test.cc
Fix clang warnings regarding unnecessary std::move
2015-12-24 04:10:00 +08:00
wal_manager.cc
Fix clang warnings regarding unnecessary std::move
2015-12-24 04:10:00 +08:00
wal_manager.h
Fix -Wnon-virtual-dtor errors
2014-11-10 17:39:38 -05:00
write_batch_base.cc
Support for SingleDelete()
2015-09-17 11:42:56 -07:00
write_batch_internal.h
support for concurrent adds to memtable
2015-12-25 11:03:40 -08:00
write_batch_test.cc
support for concurrent adds to memtable
2015-12-25 11:03:40 -08:00
write_batch.cc
support for concurrent adds to memtable
2015-12-25 11:03:40 -08:00
write_callback_test.cc
Fix compile for write_callback_test in ROCKSDB_LITE
2015-07-20 10:54:15 -07:00
write_callback.h
Optimistic Transactions
2015-05-29 14:36:35 -07:00
write_controller_test.cc
When slowdown is triggered, reduce the write rate
2015-12-23 11:33:15 -08:00
write_controller.cc
When slowdown is triggered, reduce the write rate
2015-12-23 11:33:15 -08:00
write_controller.h
When slowdown is triggered, reduce the write rate
2015-12-23 11:33:15 -08:00
write_thread.cc
use -Werror=missing-field-initializers, to closer match MyRocks build
2015-12-30 14:56:18 -08:00
write_thread.h
use -Werror=missing-field-initializers, to closer match MyRocks build
2015-12-30 14:56:18 -08:00
writebuffer.h
support for concurrent adds to memtable
2015-12-25 11:03:40 -08:00