andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
54d94e9c2c
rocksdb/db
History
Maysam Yabandeh 54d94e9c2c PinnableSlice 
Summary:
Currently the point lookup values are copied to a string provided by the user.
This incures an extra memcpy cost. This patch allows doing point lookup
via a PinnableSlice which pins the source memory location (instead of
copying their content) and releases them after the content is consumed
by the user. The old API of Get(string) is translated to the new API
underneath.

 Here is the summary for improvements:
 1. value 100 byte: 1.8%  regular, 1.2% merge values
 2. value 1k   byte: 11.5% regular, 7.5% merge values
 3. value 10k byte: 26% regular,    29.9% merge values

 The improvement for merge could be more if we extend this approach to
 pin the merge output and delay the full merge operation until the user
 actually needs it. We have put that for future work.

PS:
Sometimes we observe a small decrease in performance when switching from
t5452014 to this patch but with the old Get(string) API. The difference
is a little and could be noise. More importantly it is safely
cancelled
Closes https://github.com/facebook/rocksdb/pull/1732

Differential Revision: D4374613

Pulled By: maysamyabandeh

fbshipit-source-id: a077f1a
2017-01-08 13:54:13 -08:00
..
auto_roll_logger_test.cc Cleanup auto-roll logger flush-while-rolling test 2016-07-07 11:35:40 -07:00
auto_roll_logger.cc Eliminate use of 'using namespace std'. Also remove a number of ADL references to std functions. 2016-05-20 07:42:18 -07:00
auto_roll_logger.h Redo SyncPoints for flush while rolling test 2016-02-22 21:32:19 -08:00
builder.cc Range deletion microoptimizations 2016-11-21 12:24:13 -08:00
builder.h Remove Arena in RangeDelAggregator 2016-11-19 14:24:12 -08:00
c_test.c C API: support total_order_mode 2017-01-03 18:39:14 -08:00
c.cc C API: support total_order_mode 2017-01-03 18:39:14 -08:00
column_family_test.cc Improve Write Stalling System 2016-11-23 09:24:15 -08:00
column_family.cc Fix implicit conversion between int64_t to int 2016-11-29 10:54:15 -08:00
column_family.h Improve Write Stalling System 2016-11-23 09:24:15 -08:00
compact_files_test.cc fix data race in compact_files_test 2016-10-19 13:37:51 -07:00
compacted_db_impl.cc PinnableSlice 2017-01-08 13:54:13 -08:00
compacted_db_impl.h PinnableSlice 2017-01-08 13:54:13 -08:00
compaction_iteration_stats.h Support for range skips in compaction filter 2016-12-01 07:09:15 -08:00
compaction_iterator_test.cc Fixed CompactionFilter::Decision::kRemoveAndSkipUntil 2016-12-05 15:24:11 -08:00
compaction_iterator.cc Maintain position in range deletions map 2017-01-05 10:39:12 -08:00
compaction_iterator.h Support for range skips in compaction filter 2016-12-01 07:09:15 -08:00
compaction_job_stats_test.cc UniversalCompaction should ignore sorted runs being compacted (when compacting for file num) 2016-08-10 12:37:43 -07:00
compaction_job_test.cc Split DBOptions into ImmutableDBOptions and MutableDBOptions 2016-09-23 16:34:04 -07:00
compaction_job.cc Remove sst_file_manager option from LITE 2016-12-21 17:54:21 -08:00
compaction_job.h DeleteRange compaction statistics 2016-11-28 11:54:12 -08:00
compaction_picker_test.cc Compaction::IsTrivialMove relaxing 2016-12-07 11:54:11 -08:00
compaction_picker.cc Change max_bytes_for_level_multiplier to double 2016-11-01 21:09:23 -07:00
compaction_picker.h Compaction::IsTrivialMove relaxing 2016-12-07 11:54:11 -08:00
compaction.cc Compaction::IsTrivialMove relaxing 2016-12-07 11:54:11 -08:00
compaction.h Compaction::IsTrivialMove relaxing 2016-12-07 11:54:11 -08:00
comparator_db_test.cc Add SeekForPrev() to Iterator 2016-09-27 18:20:57 -07:00
convenience.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
corruption_test.cc Change several option defaults 2016-04-28 17:50:58 -07:00
cuckoo_table_db_test.cc Enable allow_concurrent_memtable_write and enable_write_thread_adaptive_yield by default 2016-11-16 09:39:09 -08:00
db_block_cache_test.cc Not insert into block cache if cache is full and not holding handle 2016-08-23 13:53:49 -07:00
db_bloom_filter_test.cc Enable allow_concurrent_memtable_write and enable_write_thread_adaptive_yield by default 2016-11-16 09:39:09 -08:00
db_compaction_filter_test.cc Support for range skips in compaction filter 2016-12-01 07:09:15 -08:00
db_compaction_test.cc Eliminate redundant cache lookup with range deletion 2016-11-21 21:24:11 -08:00
db_dynamic_level_test.cc Disable test to unblock travis build 2016-12-13 11:54:14 -08:00
db_filesnapshot.cc Remove unnecessary header include 2016-12-09 11:54:21 -08:00
db_flush_test.cc Fix unit test which breaks lite build 2016-07-21 14:32:12 -07:00
db_impl_debug.cc Make max_background_compactions and base_background_compactions dynamic changeable 2016-10-14 12:25:39 -07:00
db_impl_experimental.cc Split DBOptions into ImmutableDBOptions and MutableDBOptions 2016-09-23 16:34:04 -07:00
db_impl_readonly.cc PinnableSlice 2017-01-08 13:54:13 -08:00
db_impl_readonly.h PinnableSlice 2017-01-08 13:54:13 -08:00
db_impl.cc PinnableSlice 2017-01-08 13:54:13 -08:00
db_impl.h PinnableSlice 2017-01-08 13:54:13 -08:00
db_info_dumper.cc Split DBOptions into ImmutableDBOptions and MutableDBOptions 2016-09-23 16:34:04 -07:00
db_info_dumper.h Split DBOptions into ImmutableDBOptions and MutableDBOptions 2016-09-23 16:34:04 -07:00
db_inplace_update_test.cc Enable allow_concurrent_memtable_write and enable_write_thread_adaptive_yield by default 2016-11-16 09:39:09 -08:00
db_io_failure_test.cc Add WriteOptions.no_slowdown 2016-11-21 18:09:13 -08:00
db_iter_test.cc fix db_stress assertion failure 2016-10-18 16:07:10 -07:00
db_iter.cc Maintain position in range deletions map 2017-01-05 10:39:12 -08:00
db_iter.h DeleteRange user iterator support 2016-11-04 12:09:22 -07:00
db_iterator_test.cc gcc-7 requires include <functional> for std::function 2016-12-16 11:24:18 -08:00
db_log_iter_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
db_memtable_test.cc Enable allow_concurrent_memtable_write and enable_write_thread_adaptive_yield by default 2016-11-16 09:39:09 -08:00
db_merge_operator_test.cc Iterator should be in corrupted status if merge operator return false 2016-12-16 11:09:16 -08:00
db_options_test.cc Made delete_obsolete_files_period_micros option dynamic 2016-12-05 14:24:16 -08:00
db_properties_test.cc Unified InlineSkipList::Insert algorithm with hinting 2016-11-22 14:09:13 -08:00
db_range_del_test.cc Collapse range deletions 2016-12-19 16:54:12 -08:00
db_sst_test.cc Remove sst_file_manager option from LITE 2016-12-21 17:54:21 -08:00
db_table_properties_test.cc DBTablePropertiesTest.GetPropertiesOfTablesInRange: Fix Flaky 2016-07-19 15:46:20 -07:00
db_tailing_iter_test.cc Enable allow_concurrent_memtable_write and enable_write_thread_adaptive_yield by default 2016-11-16 09:39:09 -08:00
db_test2.cc Dump persistent cache options 2016-12-19 14:09:12 -08:00
db_test_util.cc direct io write support 2016-12-22 13:09:19 -08:00
db_test_util.h Fix tests under GCC_481 2017-01-03 17:54:12 -08:00
db_test.cc PinnableSlice 2017-01-08 13:54:13 -08:00
db_universal_compaction_test.cc Restrict running condition of UniversalCompactionTrivialMoveTest2 2016-10-20 15:43:00 -07:00
db_wal_test.cc Fix 2PC Recovery SeqId Miscount 2016-11-10 11:09:22 -08:00
dbformat_test.cc Improve BytewiseComparatorImpl::FindShortestSeparator 2016-04-25 23:02:14 -07:00
dbformat.cc Support for range skips in compaction filter 2016-12-01 07:09:15 -08:00
dbformat.h Collapse range deletions 2016-12-19 16:54:12 -08:00
deletefile_test.cc Fix a bug that accesses invalid address in iterator cleanup function 2016-07-05 11:57:14 -07:00
event_helpers.cc Added EventListener::OnTableFileCreationStarted() callback 2016-04-29 11:35:00 -07:00
event_helpers.h Added EventListener::OnTableFileCreationStarted() callback 2016-04-29 11:35:00 -07:00
experimental.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
external_sst_file_ingestion_job.cc Always fsync the file after file copying 2016-12-28 19:09:16 -08:00
external_sst_file_ingestion_job.h Add EventListener::OnExternalFileIngested() event 2016-12-06 14:09:17 -08:00
external_sst_file_test.cc Make ExternalSSTFileTest::CompactionDeadlock more deterministic 2017-01-03 18:09:20 -08:00
fault_injection_test.cc Fix travis (compile for clang < 3.9) 2016-12-01 10:09:22 -08:00
file_indexer_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
file_indexer.cc gcc-7 requires include <functional> for std::function 2016-12-16 11:24:18 -08:00
file_indexer.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
filename_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
filename.cc Split DBOptions into ImmutableDBOptions and MutableDBOptions 2016-09-23 16:34:04 -07:00
filename.h Split DBOptions into ImmutableDBOptions and MutableDBOptions 2016-09-23 16:34:04 -07:00
flush_job_test.cc Make range deletion inclusive-exclusive 2016-11-14 17:39:13 -08:00
flush_job.cc Fixed a crash in debug build in flush_job.cc 2016-12-09 10:39:12 -08:00
flush_job.h Split DBOptions into ImmutableDBOptions and MutableDBOptions 2016-09-23 16:34:04 -07:00
flush_scheduler.cc Fix clang analyzer errors 2016-07-08 17:50:51 -07:00
flush_scheduler.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
forward_iterator_bench.cc direct io write support 2016-12-22 13:09:19 -08:00
forward_iterator.cc Decouple data iterator and range deletion iterator in TableCache 2016-11-15 17:24:28 -08:00
forward_iterator.h Add SeekForPrev() to Iterator 2016-09-27 18:20:57 -07:00
inlineskiplist_test.cc Unified InlineSkipList::Insert algorithm with hinting 2016-11-22 14:09:13 -08:00
inlineskiplist.h Unified InlineSkipList::Insert algorithm with hinting 2016-11-22 14:09:13 -08:00
internal_stats.cc Fix bug of Checkpoint loses recent transactions with 2PC 2016-12-28 12:24:16 -08:00
internal_stats.h Fix bug of Checkpoint loses recent transactions with 2PC 2016-12-28 12:24:16 -08:00
job_context.h fixup commit 2016-10-13 08:48:40 -07:00
listener_test.cc Gcc 7 error expansion to defined 2016-12-13 18:39:14 -08:00
log_format.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
log_reader.cc Fix clang analyzer errors 2016-07-08 17:50:51 -07:00
log_reader.h db/log_reader: treat bad record length or checksum as EOF 2016-05-22 22:00:15 -07:00
log_test.cc db/log_reader: treat bad record length or checksum as EOF 2016-05-22 22:00:15 -07:00
log_writer.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
log_writer.h Need to make sure log file synced before flushing memtable of one column family 2016-07-21 16:29:06 -07:00
managed_iterator.cc Add SeekForPrev() to Iterator 2016-09-27 18:20:57 -07:00
managed_iterator.h Add SeekForPrev() to Iterator 2016-09-27 18:20:57 -07:00
manual_compaction_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
memtable_allocator.cc Avoid updating memtable allocated bytes if write_buffer_size is not set 2016-07-13 19:33:57 -07:00
memtable_allocator.h Add options.write_buffer_manager: control total memtable size across DB instances 2016-07-05 18:11:25 -07:00
memtable_list_test.cc DeleteRange Get support 2016-11-03 18:54:20 -07:00
memtable_list.cc PinnableSlice 2017-01-08 13:54:13 -08:00
memtable_list.h PinnableSlice 2017-01-08 13:54:13 -08:00
memtable.cc PinnableSlice 2017-01-08 13:54:13 -08:00
memtable.h PinnableSlice 2017-01-08 13:54:13 -08:00
memtablerep_bench.cc Add options.write_buffer_manager: control total memtable size across DB instances 2016-07-05 18:11:25 -07:00
merge_context.h Fix MergeContext::copied_operands_ strings moving 2016-07-25 15:31:41 -07:00
merge_helper_test.cc Compaction Support for Range Deletion 2016-10-18 12:04:56 -07:00
merge_helper.cc Maintain position in range deletions map 2017-01-05 10:39:12 -08:00
merge_helper.h Support for range skips in compaction filter 2016-12-01 07:09:15 -08:00
merge_operator.cc Introduce FullMergeV2 (eliminate memcpy from merge operators) 2016-07-20 09:49:03 -07:00
merge_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
options_file_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
perf_context_test.cc CompactedDB should not be used if there is outstanding WAL files 2016-04-26 14:22:07 -07:00
pinned_iterators_manager.h Delegate Cleanables 2016-12-29 15:54:19 -08:00
plain_table_db_test.cc Allow plain table to store index on file with bloom filter disabled 2016-11-17 11:09:13 -08:00
prefix_test.cc Enable allow_concurrent_memtable_write and enable_write_thread_adaptive_yield by default 2016-11-16 09:39:09 -08:00
range_del_aggregator_test.cc Maintain position in range deletions map 2017-01-05 10:39:12 -08:00
range_del_aggregator.cc Maintain position in range deletions map 2017-01-05 10:39:12 -08:00
range_del_aggregator.h Maintain position in range deletions map 2017-01-05 10:39:12 -08:00
repair_test.cc ColumnFamilyOptions API [CF + RepairDB part 3/3] 2016-06-24 16:29:43 -07:00
repair.cc Remove Arena in RangeDelAggregator 2016-11-19 14:24:12 -08:00
skiplist_test.cc Add SeekForPrev() to Iterator 2016-09-27 18:20:57 -07:00
skiplist.h Add SeekForPrev() to Iterator 2016-09-27 18:20:57 -07:00
snapshot_impl.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
snapshot_impl.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
table_cache.cc Fix range deletion covering key in same SST file 2016-11-28 22:54:13 -08:00
table_cache.h Eliminate redundant cache lookup with range deletion 2016-11-21 21:24:11 -08:00
table_properties_collector_test.cc Change ioptions to store user_comparator, fix bug 2016-10-21 11:31:42 -07:00
table_properties_collector.cc Added "number of merge operands" to statistics in ssts. 2016-05-19 14:24:48 -07:00
table_properties_collector.h Added "number of merge operands" to statistics in ssts. 2016-05-19 14:24:48 -07:00
transaction_log_impl.cc Split DBOptions into ImmutableDBOptions and MutableDBOptions 2016-09-23 16:34:04 -07:00
transaction_log_impl.h Split DBOptions into ImmutableDBOptions and MutableDBOptions 2016-09-23 16:34:04 -07:00
version_builder_test.cc Support running consistency checks in release mode 2016-10-07 17:21:45 -07:00
version_builder.cc gcc-7 requires include <functional> for std::function 2016-12-16 11:24:18 -08:00
version_builder.h Only cache level 0 indexes and filter when opening table reader 2016-07-20 11:23:31 -07:00
version_edit_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
version_edit.cc Miscellaneous performance improvements 2016-07-12 14:15:32 -07:00
version_edit.h Avoid FileMetaData copy 2016-07-13 15:36:22 -07:00
version_set_test.cc Support running consistency checks in release mode 2016-10-07 17:21:45 -07:00
version_set.cc PinnableSlice 2017-01-08 13:54:13 -08:00
version_set.h PinnableSlice 2017-01-08 13:54:13 -08:00
wal_manager_test.cc Split DBOptions into ImmutableDBOptions and MutableDBOptions 2016-09-23 16:34:04 -07:00
wal_manager.cc Split DBOptions into ImmutableDBOptions and MutableDBOptions 2016-09-23 16:34:04 -07:00
wal_manager.h Split DBOptions into ImmutableDBOptions and MutableDBOptions 2016-09-23 16:34:04 -07:00
write_batch_base.cc WriteBatch support for range deletion 2016-08-16 08:16:04 -07:00
write_batch_internal.h Add facility to write only a portion of WriteBatch to WAL 2016-10-07 11:32:10 -07:00
write_batch_test.cc Range deletion microoptimizations 2016-11-21 12:24:13 -08:00
write_batch.cc DeleteRange unsupported in non-block-based tables 2016-11-15 15:24:16 -08:00
write_callback_test.cc gcc-7 requires include <functional> for std::function 2016-12-16 11:24:18 -08:00
write_callback.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
write_controller_test.cc Improve Write Stalling System 2016-11-23 09:24:15 -08:00
write_controller.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
write_controller.h Improve Write Stalling System 2016-11-23 09:24:15 -08:00
write_thread.cc Bug: paralle_group status updated in WriteThread::CompleteParallelWorker 2016-12-01 09:54:11 -08:00
write_thread.h Add WriteOptions.no_slowdown 2016-11-21 18:09:13 -08:00
xfunc_test_points.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
xfunc_test_points.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00