andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
rocksdb/db
History
Andrew Kryczka 3f62215210 Lazily initialize RangeDelAggregator's map and pinning manager 
Summary:
Since a RangeDelAggregator is created for each read request, these heap-allocating member variables were consuming significant CPU (~3% total) which slowed down request throughput. The map and pinning manager are only necessary when range deletions exist, so we can defer their initialization until the first range deletion is encountered. Currently lazy initialization is done for reads only since reads pass us a single snapshot, which is easier to store on the stack for later insertion into the map than the vector passed to us by flush or compaction.

Note the Arena member variable is still expensive, I will figure out what to do with it in a subsequent diff. It cannot be lazily initialized because we currently use this arena even to allocate empty iterators, which is necessary even when no range deletions exist.
Closes https://github.com/facebook/rocksdb/pull/1539

Differential Revision: D4203488

Pulled By: ajkr

fbshipit-source-id: 3b36279
2016-11-18 17:09:11 -08:00
..
auto_roll_logger_test.cc
Cleanup auto-roll logger flush-while-rolling test
2016-07-07 11:35:40 -07:00
auto_roll_logger.cc
Eliminate use of 'using namespace std'. Also remove a number of ADL references to std functions.
2016-05-20 07:42:18 -07:00
auto_roll_logger.h
Redo SyncPoints for flush while rolling test
2016-02-22 21:32:19 -08:00
builder.cc
Decouple data iterator and range deletion iterator in TableCache
2016-11-15 17:24:28 -08:00
builder.h
DeleteRange flush support
2016-10-31 20:54:18 -07:00
c_test.c
Enable allow_concurrent_memtable_write and enable_write_thread_adaptive_yield by default
2016-11-16 09:39:09 -08:00
c.cc
Enable allow_concurrent_memtable_write and enable_write_thread_adaptive_yield by default
2016-11-16 09:39:09 -08:00
column_family_test.cc
Enable allow_concurrent_memtable_write and enable_write_thread_adaptive_yield by default
2016-11-16 09:39:09 -08:00
column_family.cc
DeleteRange unsupported in non-block-based tables
2016-11-15 15:24:16 -08:00
column_family.h
DeleteRange unsupported in non-block-based tables
2016-11-15 15:24:16 -08:00
compact_files_test.cc
fix data race in compact_files_test
2016-10-19 13:37:51 -07:00
compacted_db_impl.cc
DeleteRange Get support
2016-11-03 18:54:20 -07:00
compacted_db_impl.h
Updated all copyright headers to the new format.
2016-02-09 15:12:00 -08:00
compaction_iterator_test.cc
Fix uninitialized variable gcc error for MyRocks
2016-10-19 10:59:46 -07:00
compaction_iterator.cc
DeleteRange user iterator support
2016-11-04 12:09:22 -07:00
compaction_iterator.h
Compaction Support for Range Deletion
2016-10-18 12:04:56 -07:00
compaction_job_stats_test.cc
UniversalCompaction should ignore sorted runs being compacted (when compacting for file num)
2016-08-10 12:37:43 -07:00
compaction_job_test.cc
Split DBOptions into ImmutableDBOptions and MutableDBOptions
2016-09-23 16:34:04 -07:00
compaction_job.cc
Decouple data iterator and range deletion iterator in TableCache
2016-11-15 17:24:28 -08:00
compaction_job.h
Compaction Support for Range Deletion
2016-10-18 12:04:56 -07:00
compaction_picker_test.cc
Fix a bug that may cause a deleted row to appear again
2016-10-24 09:49:07 -07:00
compaction_picker.cc
Change max_bytes_for_level_multiplier to double
2016-11-01 21:09:23 -07:00
compaction_picker.h
Fix compaction conflict with running compaction
2016-10-13 10:49:06 -07:00
compaction.cc
Refactor MutableCFOptions
2016-09-13 21:11:59 -07:00
compaction.h
Fix compaction conflict with running compaction
2016-10-13 10:49:06 -07:00
comparator_db_test.cc
Add SeekForPrev() to Iterator
2016-09-27 18:20:57 -07:00
convenience.cc
Updated all copyright headers to the new format.
2016-02-09 15:12:00 -08:00
corruption_test.cc
Change several option defaults
2016-04-28 17:50:58 -07:00
cuckoo_table_db_test.cc
Enable allow_concurrent_memtable_write and enable_write_thread_adaptive_yield by default
2016-11-16 09:39:09 -08:00
db_block_cache_test.cc
Not insert into block cache if cache is full and not holding handle
2016-08-23 13:53:49 -07:00
db_bloom_filter_test.cc
Enable allow_concurrent_memtable_write and enable_write_thread_adaptive_yield by default
2016-11-16 09:39:09 -08:00
db_compaction_filter_test.cc
DeleteRange user iterator support
2016-11-04 12:09:22 -07:00
db_compaction_test.cc
Decouple data iterator and range deletion iterator in TableCache
2016-11-15 17:24:28 -08:00
db_dynamic_level_test.cc
Merge options source_compaction_factor, max_grandparent_overlap_bytes and expanded_compaction_factor into max_compaction_bytes
2016-09-01 14:33:24 -07:00
db_filesnapshot.cc
Split DBOptions into ImmutableDBOptions and MutableDBOptions
2016-09-23 16:34:04 -07:00
db_flush_test.cc
Fix unit test which breaks lite build
2016-07-21 14:32:12 -07:00
db_impl_debug.cc
Make max_background_compactions and base_background_compactions dynamic changeable
2016-10-14 12:25:39 -07:00
db_impl_experimental.cc
Split DBOptions into ImmutableDBOptions and MutableDBOptions
2016-09-23 16:34:04 -07:00
db_impl_readonly.cc
Lazily initialize RangeDelAggregator's map and pinning manager
2016-11-18 17:09:11 -08:00
db_impl_readonly.h
Updated all copyright headers to the new format.
2016-02-09 15:12:00 -08:00
db_impl.cc
Lazily initialize RangeDelAggregator's map and pinning manager
2016-11-18 17:09:11 -08:00
db_impl.h
Dynamic max_total_wal_size option
2016-11-14 22:54:17 -08:00
db_info_dumper.cc
Split DBOptions into ImmutableDBOptions and MutableDBOptions
2016-09-23 16:34:04 -07:00
db_info_dumper.h
Split DBOptions into ImmutableDBOptions and MutableDBOptions
2016-09-23 16:34:04 -07:00
db_inplace_update_test.cc
Enable allow_concurrent_memtable_write and enable_write_thread_adaptive_yield by default
2016-11-16 09:39:09 -08:00
db_io_failure_test.cc
Add NoSpace subcode to IOError (#1320)
2016-09-07 12:37:45 -07:00
db_iter_test.cc
fix db_stress assertion failure
2016-10-18 16:07:10 -07:00
db_iter.cc
Lazily initialize RangeDelAggregator's map and pinning manager
2016-11-18 17:09:11 -08:00
db_iter.h
DeleteRange user iterator support
2016-11-04 12:09:22 -07:00
db_iterator_test.cc
Fix MSVC compile error in 32 bit compilation
2016-10-31 17:24:18 -07:00
db_log_iter_test.cc
Updated all copyright headers to the new format.
2016-02-09 15:12:00 -08:00
db_memtable_test.cc
Enable allow_concurrent_memtable_write and enable_write_thread_adaptive_yield by default
2016-11-16 09:39:09 -08:00
db_options_test.cc
Dynamic max_total_wal_size option
2016-11-14 22:54:17 -08:00
db_properties_test.cc
Enable allow_concurrent_memtable_write and enable_write_thread_adaptive_yield by default
2016-11-16 09:39:09 -08:00
db_sst_test.cc
DBSSTTest.RateLimitedDelete: not to use real clock
2016-10-24 10:35:00 -07:00
db_table_properties_test.cc
DBTablePropertiesTest.GetPropertiesOfTablesInRange: Fix Flaky
2016-07-19 15:46:20 -07:00
db_tailing_iter_test.cc
Enable allow_concurrent_memtable_write and enable_write_thread_adaptive_yield by default
2016-11-16 09:39:09 -08:00
db_test2.cc
Enable allow_concurrent_memtable_write and enable_write_thread_adaptive_yield by default
2016-11-16 09:39:09 -08:00
db_test_util.cc
Enable allow_concurrent_memtable_write and enable_write_thread_adaptive_yield by default
2016-11-16 09:39:09 -08:00
db_test_util.h
Enable allow_concurrent_memtable_write and enable_write_thread_adaptive_yield by default
2016-11-16 09:39:09 -08:00
db_test.cc
Enable allow_concurrent_memtable_write and enable_write_thread_adaptive_yield by default
2016-11-16 09:39:09 -08:00
db_universal_compaction_test.cc
Restrict running condition of UniversalCompactionTrivialMoveTest2
2016-10-20 15:43:00 -07:00
db_wal_test.cc
Fix 2PC Recovery SeqId Miscount
2016-11-10 11:09:22 -08:00
dbformat_test.cc
Improve BytewiseComparatorImpl::FindShortestSeparator
2016-04-25 23:02:14 -07:00
dbformat.cc
Add SeekForPrev() to Iterator
2016-09-27 18:20:57 -07:00
dbformat.h
Fix uninitialized variable gcc error for MyRocks
2016-10-19 10:59:46 -07:00
deletefile_test.cc
Fix a bug that accesses invalid address in iterator cleanup function
2016-07-05 11:57:14 -07:00
event_helpers.cc
Added EventListener::OnTableFileCreationStarted() callback
2016-04-29 11:35:00 -07:00
event_helpers.h
Added EventListener::OnTableFileCreationStarted() callback
2016-04-29 11:35:00 -07:00
experimental.cc
Updated all copyright headers to the new format.
2016-02-09 15:12:00 -08:00
external_sst_file_ingestion_job.cc
DeleteRange user iterator support
2016-11-04 12:09:22 -07:00
external_sst_file_ingestion_job.h
Support IngestExternalFile (remove AddFile restrictions)
2016-10-20 17:05:32 -07:00
external_sst_file_test.cc
Fix SstFileWriter destructor
2016-11-12 20:11:19 -08:00
fault_injection_test.cc
Add facility to write only a portion of WriteBatch to WAL
2016-10-07 11:32:10 -07:00
file_indexer_test.cc
Updated all copyright headers to the new format.
2016-02-09 15:12:00 -08:00
file_indexer.cc
Updated all copyright headers to the new format.
2016-02-09 15:12:00 -08:00
file_indexer.h
Updated all copyright headers to the new format.
2016-02-09 15:12:00 -08:00
filename_test.cc
Updated all copyright headers to the new format.
2016-02-09 15:12:00 -08:00
filename.cc
Split DBOptions into ImmutableDBOptions and MutableDBOptions
2016-09-23 16:34:04 -07:00
filename.h
Split DBOptions into ImmutableDBOptions and MutableDBOptions
2016-09-23 16:34:04 -07:00
flush_job_test.cc
Make range deletion inclusive-exclusive
2016-11-14 17:39:13 -08:00
flush_job.cc
DeleteRange flush support
2016-10-31 20:54:18 -07:00
flush_job.h
Split DBOptions into ImmutableDBOptions and MutableDBOptions
2016-09-23 16:34:04 -07:00
flush_scheduler.cc
Fix clang analyzer errors
2016-07-08 17:50:51 -07:00
flush_scheduler.h
Updated all copyright headers to the new format.
2016-02-09 15:12:00 -08:00
forward_iterator_bench.cc
Updated all copyright headers to the new format.
2016-02-09 15:12:00 -08:00
forward_iterator.cc
Decouple data iterator and range deletion iterator in TableCache
2016-11-15 17:24:28 -08:00
forward_iterator.h
Add SeekForPrev() to Iterator
2016-09-27 18:20:57 -07:00
inlineskiplist_test.cc
Optimize sequential insert into memtable - Part 2: Implementation
2016-11-13 13:09:16 -08:00
inlineskiplist.h
Optimize sequential insert into memtable - Part 2: Implementation
2016-11-13 13:09:16 -08:00
internal_stats.cc
Exporting compaction stats in the form of a map
2016-11-11 20:54:14 -08:00
internal_stats.h
Exporting compaction stats in the form of a map
2016-11-11 20:54:14 -08:00
job_context.h
fixup commit
2016-10-13 08:48:40 -07:00
listener_test.cc
add cfh deletion started listener
2016-09-22 11:56:18 -07:00
log_format.h
Updated all copyright headers to the new format.
2016-02-09 15:12:00 -08:00
log_reader.cc
Fix clang analyzer errors
2016-07-08 17:50:51 -07:00
log_reader.h
db/log_reader: treat bad record length or checksum as EOF
2016-05-22 22:00:15 -07:00
log_test.cc
db/log_reader: treat bad record length or checksum as EOF
2016-05-22 22:00:15 -07:00
log_writer.cc
Updated all copyright headers to the new format.
2016-02-09 15:12:00 -08:00
log_writer.h
Need to make sure log file synced before flushing memtable of one column family
2016-07-21 16:29:06 -07:00
managed_iterator.cc
Add SeekForPrev() to Iterator
2016-09-27 18:20:57 -07:00
managed_iterator.h
Add SeekForPrev() to Iterator
2016-09-27 18:20:57 -07:00
manual_compaction_test.cc
Updated all copyright headers to the new format.
2016-02-09 15:12:00 -08:00
memtable_allocator.cc
Avoid updating memtable allocated bytes if write_buffer_size is not set
2016-07-13 19:33:57 -07:00
memtable_allocator.h
Add options.write_buffer_manager: control total memtable size across DB instances
2016-07-05 18:11:25 -07:00
memtable_list_test.cc
DeleteRange Get support
2016-11-03 18:54:20 -07:00
memtable_list.cc
DeleteRange user iterator support
2016-11-04 12:09:22 -07:00
memtable_list.h
DeleteRange user iterator support
2016-11-04 12:09:22 -07:00
memtable.cc
Report memory usage by memtable insert hints map.
2016-11-15 20:24:13 -08:00
memtable.h
Optimize sequential insert into memtable - Part 1: Interface
2016-11-13 19:09:18 -08:00
memtablerep_bench.cc
Add options.write_buffer_manager: control total memtable size across DB instances
2016-07-05 18:11:25 -07:00
merge_context.h
Fix MergeContext::copied_operands_ strings moving
2016-07-25 15:31:41 -07:00
merge_helper_test.cc
Compaction Support for Range Deletion
2016-10-18 12:04:56 -07:00
merge_helper.cc
Compaction Support for Range Deletion
2016-10-18 12:04:56 -07:00
merge_helper.h
Compaction Support for Range Deletion
2016-10-18 12:04:56 -07:00
merge_operator.cc
Introduce FullMergeV2 (eliminate memcpy from merge operators)
2016-07-20 09:49:03 -07:00
merge_test.cc
Updated all copyright headers to the new format.
2016-02-09 15:12:00 -08:00
options_file_test.cc
Updated all copyright headers to the new format.
2016-02-09 15:12:00 -08:00
perf_context_test.cc
CompactedDB should not be used if there is outstanding WAL files
2016-04-26 14:22:07 -07:00
pinned_iterators_manager.h
Minor PinnedIteratorsManager Refactoring
2016-08-11 11:54:17 -07:00
plain_table_db_test.cc
Allow plain table to store index on file with bloom filter disabled
2016-11-17 11:09:13 -08:00
prefix_test.cc
Enable allow_concurrent_memtable_write and enable_write_thread_adaptive_yield by default
2016-11-16 09:39:09 -08:00
range_del_aggregator.cc
Lazily initialize RangeDelAggregator's map and pinning manager
2016-11-18 17:09:11 -08:00
range_del_aggregator.h
Lazily initialize RangeDelAggregator's map and pinning manager
2016-11-18 17:09:11 -08:00
repair_test.cc
ColumnFamilyOptions API [CF + RepairDB part 3/3]
2016-06-24 16:29:43 -07:00
repair.cc
Decouple data iterator and range deletion iterator in TableCache
2016-11-15 17:24:28 -08:00
skiplist_test.cc
Add SeekForPrev() to Iterator
2016-09-27 18:20:57 -07:00
skiplist.h
Add SeekForPrev() to Iterator
2016-09-27 18:20:57 -07:00
snapshot_impl.cc
Updated all copyright headers to the new format.
2016-02-09 15:12:00 -08:00
snapshot_impl.h
Updated all copyright headers to the new format.
2016-02-09 15:12:00 -08:00
table_cache.cc
Lazily initialize RangeDelAggregator's map and pinning manager
2016-11-18 17:09:11 -08:00
table_cache.h
Decouple data iterator and range deletion iterator in TableCache
2016-11-15 17:24:28 -08:00
table_properties_collector_test.cc
Change ioptions to store user_comparator, fix bug
2016-10-21 11:31:42 -07:00
table_properties_collector.cc
Added "number of merge operands" to statistics in ssts.
2016-05-19 14:24:48 -07:00
table_properties_collector.h
Added "number of merge operands" to statistics in ssts.
2016-05-19 14:24:48 -07:00
transaction_log_impl.cc
Split DBOptions into ImmutableDBOptions and MutableDBOptions
2016-09-23 16:34:04 -07:00
transaction_log_impl.h
Split DBOptions into ImmutableDBOptions and MutableDBOptions
2016-09-23 16:34:04 -07:00
version_builder_test.cc
Support running consistency checks in release mode
2016-10-07 17:21:45 -07:00
version_builder.cc
Support IngestExternalFile (remove AddFile restrictions)
2016-10-20 17:05:32 -07:00
version_builder.h
Only cache level 0 indexes and filter when opening table reader
2016-07-20 11:23:31 -07:00
version_edit_test.cc
Updated all copyright headers to the new format.
2016-02-09 15:12:00 -08:00
version_edit.cc
Miscellaneous performance improvements
2016-07-12 14:15:32 -07:00
version_edit.h
Avoid FileMetaData copy
2016-07-13 15:36:22 -07:00
version_set_test.cc
Support running consistency checks in release mode
2016-10-07 17:21:45 -07:00
version_set.cc
Decouple data iterator and range deletion iterator in TableCache
2016-11-15 17:24:28 -08:00
version_set.h
DeleteRange user iterator support
2016-11-04 12:09:22 -07:00
wal_manager_test.cc
Split DBOptions into ImmutableDBOptions and MutableDBOptions
2016-09-23 16:34:04 -07:00
wal_manager.cc
Split DBOptions into ImmutableDBOptions and MutableDBOptions
2016-09-23 16:34:04 -07:00
wal_manager.h
Split DBOptions into ImmutableDBOptions and MutableDBOptions
2016-09-23 16:34:04 -07:00
write_batch_base.cc
WriteBatch support for range deletion
2016-08-16 08:16:04 -07:00
write_batch_internal.h
Add facility to write only a portion of WriteBatch to WAL
2016-10-07 11:32:10 -07:00
write_batch_test.cc
Add facility to write only a portion of WriteBatch to WAL
2016-10-07 11:32:10 -07:00
write_batch.cc
DeleteRange unsupported in non-block-based tables
2016-11-15 15:24:16 -08:00
write_callback_test.cc
write_callback_test: clean test directory before running tests
2016-04-18 10:18:41 -07:00
write_callback.h
Updated all copyright headers to the new format.
2016-02-09 15:12:00 -08:00
write_controller_test.cc
Updated all copyright headers to the new format.
2016-02-09 15:12:00 -08:00
write_controller.cc
Updated all copyright headers to the new format.
2016-02-09 15:12:00 -08:00
write_controller.h
Allowed delayed_write_rate option to be dynamically set.
2016-11-12 15:54:11 -08:00
write_thread.cc
WriteBatch support for range deletion
2016-08-16 08:16:04 -07:00
write_thread.h
fix simple typos (#1183)
2016-06-25 08:29:40 +01:00
xfunc_test_points.cc
Updated all copyright headers to the new format.
2016-02-09 15:12:00 -08:00
xfunc_test_points.h
Updated all copyright headers to the new format.
2016-02-09 15:12:00 -08:00