andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
11526252cc
rocksdb/db
History
Maysam Yabandeh 11526252cc Pinnableslice (2nd attempt) 
Summary:
PinnableSlice

    Summary:
    Currently the point lookup values are copied to a string provided by the
    user. This incures an extra memcpy cost. This patch allows doing point lookup
    via a PinnableSlice which pins the source memory location (instead of
    copying their content) and releases them after the content is consumed
    by the user. The old API of Get(string) is translated to the new API
    underneath.

    Here is the summary for improvements:

    value 100 byte: 1.8% regular, 1.2% merge values
    value 1k byte: 11.5% regular, 7.5% merge values
    value 10k byte: 26% regular, 29.9% merge values
    The improvement for merge could be more if we extend this approach to
    pin the merge output and delay the full merge operation until the user
    actually needs it. We have put that for future work.

    PS:
    Sometimes we observe a small decrease in performance when switching from
    t5452014 to this patch but with the old Get(string) API. The d
Closes https://github.com/facebook/rocksdb/pull/1756

Differential Revision: D4391738

Pulled By: maysamyabandeh

fbshipit-source-id: 6f3edd3
2017-03-13 11:54:10 -07:00
..
auto_roll_logger_test.cc Remove bulk loading and auto_roll_logger in rocksdb_lite 2017-02-28 11:09:11 -08:00
auto_roll_logger.cc Remove bulk loading and auto_roll_logger in rocksdb_lite 2017-02-28 11:09:11 -08:00
auto_roll_logger.h Remove bulk loading and auto_roll_logger in rocksdb_lite 2017-02-28 11:09:11 -08:00
builder.cc Statistic for how often rate limiter is drained 2017-03-02 17:54:15 -08:00
builder.h Remove Arena in RangeDelAggregator 2016-11-19 14:24:12 -08:00
c_test.c c: allow set savepoint to writebatch 2017-01-20 13:24:13 -08:00
c.cc Add a new SstFileWriter constructor without explicit comparator 2017-03-13 11:39:13 -07:00
column_family_test.cc Remove disableDataSync option 2017-02-13 11:09:13 -08:00
column_family.cc Remove XFunc tests 2017-02-23 12:09:11 -08:00
column_family.h Fix for 2PC causing WAL to grow too large 2017-01-19 15:39:12 -08:00
compact_files_test.cc Windows thread 2017-02-06 14:54:18 -08:00
compacted_db_impl.cc Pinnableslice (2nd attempt) 2017-03-13 11:54:10 -07:00
compacted_db_impl.h Pinnableslice (2nd attempt) 2017-03-13 11:54:10 -07:00
compaction_iteration_stats.h Support for range skips in compaction filter 2016-12-01 07:09:15 -08:00
compaction_iterator_test.cc [rocksdb][PR] Remove option min_partial_merge_operands and verify_checksums_in_comp… 2017-02-23 15:09:12 -08:00
compaction_iterator.cc Abort compactions more reliably when closing DB 2017-01-11 15:09:21 -08:00
compaction_iterator.h Abort compactions more reliably when closing DB 2017-01-11 15:09:21 -08:00
compaction_job_stats_test.cc Remove XFunc tests 2017-02-23 12:09:11 -08:00
compaction_job_test.cc Split DBOptions into ImmutableDBOptions and MutableDBOptions 2016-09-23 16:34:04 -07:00
compaction_job.cc Statistic for how often rate limiter is drained 2017-03-02 17:54:15 -08:00
compaction_job.h Abort compactions more reliably when closing DB 2017-01-11 15:09:21 -08:00
compaction_picker_test.cc level compaction expansion 2017-02-21 10:24:17 -08:00
compaction_picker.cc level compaction expansion 2017-02-21 10:24:17 -08:00
compaction_picker.h Compaction::IsTrivialMove relaxing 2016-12-07 11:54:11 -08:00
compaction.cc Compaction::IsTrivialMove relaxing 2016-12-07 11:54:11 -08:00
compaction.h Compaction::IsTrivialMove relaxing 2016-12-07 11:54:11 -08:00
comparator_db_test.cc Add SeekForPrev() to Iterator 2016-09-27 18:20:57 -07:00
convenience.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
corruption_test.cc Change corruption_test to use 4 bits. 2017-01-27 11:24:16 -08:00
cuckoo_table_db_test.cc Enable allow_concurrent_memtable_write and enable_write_thread_adaptive_yield by default 2016-11-16 09:39:09 -08:00
db_basic_test.cc Separate small subset tests in DBTest 2017-02-27 12:24:11 -08:00
db_block_cache_test.cc Not insert into block cache if cache is full and not holding handle 2016-08-23 13:53:49 -07:00
db_bloom_filter_test.cc Fix some bugs in MockEnv 2017-03-13 09:54:11 -07:00
db_compaction_filter_test.cc Support for range skips in compaction filter 2016-12-01 07:09:15 -08:00
db_compaction_test.cc Windows thread 2017-02-06 14:54:18 -08:00
db_dynamic_level_test.cc Fix some bugs in MockEnv 2017-03-13 09:54:11 -07:00
db_filesnapshot.cc Remove unnecessary header include 2016-12-09 11:54:21 -08:00
db_flush_test.cc Fix some bugs in MockEnv 2017-03-13 09:54:11 -07:00
db_impl_debug.cc Fix for 2PC causing WAL to grow too large 2017-01-19 15:39:12 -08:00
db_impl_experimental.cc Split DBOptions into ImmutableDBOptions and MutableDBOptions 2016-09-23 16:34:04 -07:00
db_impl_readonly.cc Pinnableslice (2nd attempt) 2017-03-13 11:54:10 -07:00
db_impl_readonly.h Pinnableslice (2nd attempt) 2017-03-13 11:54:10 -07:00
db_impl.cc Pinnableslice (2nd attempt) 2017-03-13 11:54:10 -07:00
db_impl.h Pinnableslice (2nd attempt) 2017-03-13 11:54:10 -07:00
db_info_dumper.cc Split DBOptions into ImmutableDBOptions and MutableDBOptions 2016-09-23 16:34:04 -07:00
db_info_dumper.h Split DBOptions into ImmutableDBOptions and MutableDBOptions 2016-09-23 16:34:04 -07:00
db_inplace_update_test.cc Enable allow_concurrent_memtable_write and enable_write_thread_adaptive_yield by default 2016-11-16 09:39:09 -08:00
db_io_failure_test.cc Add WriteOptions.no_slowdown 2016-11-21 18:09:13 -08:00
db_iter_test.cc Reset DBIter::saved_key_ with proper user key anywhere before pass to DBIter::FindNextUserEntry 2017-03-08 17:24:11 -08:00
db_iter.cc Reset DBIter::saved_key_ with proper user key anywhere before pass to DBIter::FindNextUserEntry 2017-03-08 17:24:11 -08:00
db_iter.h DeleteRange user iterator support 2016-11-04 12:09:22 -07:00
db_iterator_test.cc Fix some bugs in MockEnv 2017-03-13 09:54:11 -07:00
db_log_iter_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
db_memtable_test.cc Fix some bugs in MockEnv 2017-03-13 09:54:11 -07:00
db_merge_operator_test.cc Fix some bugs in MockEnv 2017-03-13 09:54:11 -07:00
db_options_test.cc Fix some bugs in MockEnv 2017-03-13 09:54:11 -07:00
db_properties_test.cc Fix some bugs in MockEnv 2017-03-13 09:54:11 -07:00
db_range_del_test.cc Fix DeleteRange including sentinels in output files 2017-01-25 11:09:12 -08:00
db_sst_test.cc fix db_sst_test flakiness 2017-03-07 11:24:13 -08:00
db_table_properties_test.cc Fix some bugs in MockEnv 2017-03-13 09:54:11 -07:00
db_tailing_iter_test.cc Fix some bugs in MockEnv 2017-03-13 09:54:11 -07:00
db_test_util.cc Fix some bugs in MockEnv 2017-03-13 09:54:11 -07:00
db_test_util.h Fix some bugs in MockEnv 2017-03-13 09:54:11 -07:00
db_test.cc Pinnableslice (2nd attempt) 2017-03-13 11:54:10 -07:00
db_test2.cc Add a memtable-only iterator 2017-03-07 11:54:10 -08:00
db_universal_compaction_test.cc Fix flaky DBTestUniversalCompaction.UniversalCompactionTrivialMoveTest2 2017-02-23 18:39:13 -08:00
db_wal_test.cc Make db_wal_test slightly faster 2017-02-28 17:39:10 -08:00
dbformat_test.cc Improve BytewiseComparatorImpl::FindShortestSeparator 2016-04-25 23:02:14 -07:00
dbformat.cc Add ability to search for key prefix in sst_dump tool 2017-03-13 10:39:12 -07:00
dbformat.h Collapse range deletions 2016-12-19 16:54:12 -08:00
deletefile_test.cc Fix a bug that accesses invalid address in iterator cleanup function 2016-07-05 11:57:14 -07:00
event_helpers.cc Performance: Iterate vector by reference 2017-01-11 10:54:37 -08:00
event_helpers.h Added EventListener::OnTableFileCreationStarted() callback 2016-04-29 11:35:00 -07:00
experimental.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
external_sst_file_basic_test.cc Add a new SstFileWriter constructor without explicit comparator 2017-03-13 11:39:13 -07:00
external_sst_file_ingestion_job.cc Remove bulk loading and auto_roll_logger in rocksdb_lite 2017-02-28 11:09:11 -08:00
external_sst_file_ingestion_job.h Add EventListener::OnExternalFileIngested() event 2016-12-06 14:09:17 -08:00
external_sst_file_test.cc Add a new SstFileWriter constructor without explicit comparator 2017-03-13 11:39:13 -07:00
fault_injection_test.cc Fix travis (compile for clang < 3.9) 2016-12-01 10:09:22 -08:00
file_indexer_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
file_indexer.cc gcc-7 requires include <functional> for std::function 2016-12-16 11:24:18 -08:00
file_indexer.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
filename_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
filename.cc Remove disableDataSync option 2017-02-13 11:09:13 -08:00
filename.h Split DBOptions into ImmutableDBOptions and MutableDBOptions 2016-09-23 16:34:04 -07:00
flush_job_test.cc Make range deletion inclusive-exclusive 2016-11-14 17:39:13 -08:00
flush_job.cc Remove disableDataSync option 2017-02-13 11:09:13 -08:00
flush_job.h Flush job should release reference current version if sync log failed 2017-01-19 23:09:15 -08:00
flush_scheduler.cc Fix clang analyzer errors 2016-07-08 17:50:51 -07:00
flush_scheduler.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
forward_iterator_bench.cc Windows thread 2017-02-06 14:54:18 -08:00
forward_iterator.cc Rename merger.h -> merging_iterator.h 2017-02-02 16:54:19 -08:00
forward_iterator.h Add SeekForPrev() to Iterator 2016-09-27 18:20:57 -07:00
inlineskiplist_test.cc Unified InlineSkipList::Insert algorithm with hinting 2016-11-22 14:09:13 -08:00
inlineskiplist.h Unified InlineSkipList::Insert algorithm with hinting 2016-11-22 14:09:13 -08:00
internal_stats.cc Print the missed last layer in cfstats 2017-02-08 10:39:15 -08:00
internal_stats.h Print the missed last layer in cfstats 2017-02-08 10:39:15 -08:00
job_context.h fixup commit 2016-10-13 08:48:40 -07:00
listener_test.cc Gcc 7 error expansion to defined 2016-12-13 18:39:14 -08:00
log_format.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
log_reader.cc Fix clang analyzer errors 2016-07-08 17:50:51 -07:00
log_reader.h db/log_reader: treat bad record length or checksum as EOF 2016-05-22 22:00:15 -07:00
log_test.cc db/log_reader: treat bad record length or checksum as EOF 2016-05-22 22:00:15 -07:00
log_writer.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
log_writer.h Need to make sure log file synced before flushing memtable of one column family 2016-07-21 16:29:06 -07:00
managed_iterator.cc Remove XFunc tests 2017-02-23 12:09:11 -08:00
managed_iterator.h Add SeekForPrev() to Iterator 2016-09-27 18:20:57 -07:00
manual_compaction_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
memtable_allocator.cc Avoid updating memtable allocated bytes if write_buffer_size is not set 2016-07-13 19:33:57 -07:00
memtable_allocator.h Add options.write_buffer_manager: control total memtable size across DB instances 2016-07-05 18:11:25 -07:00
memtable_list_test.cc DeleteRange Get support 2016-11-03 18:54:20 -07:00
memtable_list.cc Adding GetApproximateMemTableStats method 2017-02-06 14:54:16 -08:00
memtable_list.h Adding GetApproximateMemTableStats method 2017-02-06 14:54:16 -08:00
memtable.cc Adding GetApproximateMemTableStats method 2017-02-06 14:54:16 -08:00
memtable.h Adding GetApproximateMemTableStats method 2017-02-06 14:54:16 -08:00
memtablerep_bench.cc Windows thread 2017-02-06 14:54:18 -08:00
merge_context.h Fix MergeContext::copied_operands_ strings moving 2016-07-25 15:31:41 -07:00
merge_helper_test.cc [rocksdb][PR] Remove option min_partial_merge_operands and verify_checksums_in_comp… 2017-02-23 15:09:12 -08:00
merge_helper.cc [rocksdb][PR] Remove option min_partial_merge_operands and verify_checksums_in_comp… 2017-02-23 15:09:12 -08:00
merge_helper.h [rocksdb][PR] Remove option min_partial_merge_operands and verify_checksums_in_comp… 2017-02-23 15:09:12 -08:00
merge_operator.cc Introduce FullMergeV2 (eliminate memcpy from merge operators) 2016-07-20 09:49:03 -07:00
merge_test.cc [rocksdb][PR] Remove option min_partial_merge_operands and verify_checksums_in_comp… 2017-02-23 15:09:12 -08:00
options_file_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
perf_context_test.cc Windows thread 2017-02-06 14:54:18 -08:00
pinned_iterators_manager.h Delegate Cleanables 2016-12-29 15:54:19 -08:00
plain_table_db_test.cc Allow plain table to store index on file with bloom filter disabled 2016-11-17 11:09:13 -08:00
prefix_test.cc Enable allow_concurrent_memtable_write and enable_write_thread_adaptive_yield by default 2016-11-16 09:39:09 -08:00
range_del_aggregator_test.cc Maintain position in range deletions map 2017-01-05 10:39:12 -08:00
range_del_aggregator.cc Fix DeleteRange including sentinels in output files 2017-01-25 11:09:12 -08:00
range_del_aggregator.h Maintain position in range deletions map 2017-01-05 10:39:12 -08:00
repair_test.cc avoid direct io in rocksdb_lite 2017-02-16 10:39:13 -08:00
repair.cc Fix repair issues 2017-02-10 10:54:12 -08:00
skiplist_test.cc Add SeekForPrev() to Iterator 2016-09-27 18:20:57 -07:00
skiplist.h Add SeekForPrev() to Iterator 2016-09-27 18:20:57 -07:00
snapshot_impl.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
snapshot_impl.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
table_cache.cc Fix range deletion covering key in same SST file 2016-11-28 22:54:13 -08:00
table_cache.h Eliminate redundant cache lookup with range deletion 2016-11-21 21:24:11 -08:00
table_properties_collector_test.cc Change ioptions to store user_comparator, fix bug 2016-10-21 11:31:42 -07:00
table_properties_collector.cc Added "number of merge operands" to statistics in ssts. 2016-05-19 14:24:48 -07:00
table_properties_collector.h Added "number of merge operands" to statistics in ssts. 2016-05-19 14:24:48 -07:00
transaction_log_impl.cc Split DBOptions into ImmutableDBOptions and MutableDBOptions 2016-09-23 16:34:04 -07:00
transaction_log_impl.h Split DBOptions into ImmutableDBOptions and MutableDBOptions 2016-09-23 16:34:04 -07:00
version_builder_test.cc Support running consistency checks in release mode 2016-10-07 17:21:45 -07:00
version_builder.cc Windows thread 2017-02-06 14:54:18 -08:00
version_builder.h Only cache level 0 indexes and filter when opening table reader 2016-07-20 11:23:31 -07:00
version_edit_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
version_edit.cc Miscellaneous performance improvements 2016-07-12 14:15:32 -07:00
version_edit.h Avoid FileMetaData copy 2016-07-13 15:36:22 -07:00
version_set_test.cc Support running consistency checks in release mode 2016-10-07 17:21:45 -07:00
version_set.cc Pinnableslice (2nd attempt) 2017-03-13 11:54:10 -07:00
version_set.h Pinnableslice (2nd attempt) 2017-03-13 11:54:10 -07:00
wal_manager_test.cc Split DBOptions into ImmutableDBOptions and MutableDBOptions 2016-09-23 16:34:04 -07:00
wal_manager.cc Split DBOptions into ImmutableDBOptions and MutableDBOptions 2016-09-23 16:34:04 -07:00
wal_manager.h Split DBOptions into ImmutableDBOptions and MutableDBOptions 2016-09-23 16:34:04 -07:00
write_batch_base.cc WriteBatch support for range deletion 2016-08-16 08:16:04 -07:00
write_batch_internal.h Add facility to write only a portion of WriteBatch to WAL 2016-10-07 11:32:10 -07:00
write_batch_test.cc Range deletion microoptimizations 2016-11-21 12:24:13 -08:00
write_batch.cc DeleteRange unsupported in non-block-based tables 2016-11-15 15:24:16 -08:00
write_callback_test.cc Windows thread 2017-02-06 14:54:18 -08:00
write_callback.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
write_controller_test.cc Use monotonic time points in write_controller.cc and rate_limiter.cc 2017-02-14 18:24:24 -08:00
write_controller.cc detect logical sector size 2017-02-23 11:25:36 -08:00
write_controller.h Use monotonic time points in write_controller.cc and rate_limiter.cc 2017-02-14 18:24:24 -08:00
write_thread.cc Fix 2PC with concurrent memtable insert 2017-01-17 11:24:28 -08:00
write_thread.h Add WriteOptions.no_slowdown 2016-11-21 18:09:13 -08:00