andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
6f71d3b68b
rocksdb/db
History
reid horuff 6f71d3b68b Improve perf of Pessimistic Transaction expirations (and optimistic transactions) 
Summary:
copy from task 8196669:

1) Optimistic transactions do not support batching writes from different threads.
2) Pessimistic transactions do not support batching writes if an expiration time is set.

In these 2 cases, we currently do not do any write batching in DBImpl::WriteImpl() because there is a WriteCallback that could decide at the last minute to abort the write.  But we could support batching write operations with callbacks if we make sure to process the callbacks correctly.

To do this, we would first need to modify write_thread.cc to stop preventing writes with callbacks from being batched together.  Then we would need to change DBImpl::WriteImpl() to call all WriteCallback's in a batch, only write the batches that succeed, and correctly set the state of each batch's WriteThread::Writer.

Test Plan: Added test WriteWithCallbackTest to write_callback_test.cc which creates multiple client threads and verifies that writes are batched and executed properly.

Reviewers: hermanlee4, anthony, ngbronson

Subscribers: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D52863
2016-02-05 10:44:13 -08:00
..
auto_roll_logger_test.cc [directory includes cleanup] Finish removing util->db dependencies 2016-01-26 10:49:24 -08:00
auto_roll_logger.cc RollLogFile tries to find non conflicting file until there is no conflict. 2016-02-02 10:33:49 +01:00
auto_roll_logger.h [directory includes cleanup] Finish removing util->db dependencies 2016-01-26 10:49:24 -08:00
builder.cc Use SST files for Transaction conflict detection 2015-12-11 12:34:11 -08:00
builder.h Use SST files for Transaction conflict detection 2015-12-11 12:34:11 -08:00
c_test.c Deprecate CompactionFilterV2 2015-07-17 18:59:11 +02:00
c.cc Merge pull request #916 from warrenfalk/capi_huge_page_option 2016-01-04 09:24:30 -08:00
column_family_test.cc Fixed the asan error on column_family_test 2016-02-01 12:45:45 -08:00
column_family.cc Add options.base_background_compactions as a number of compaction threads for low compaction debt 2016-01-29 16:15:53 -08:00
column_family.h Add ColumnFamilyHandle::GetDescriptor() 2016-01-06 18:14:01 -08:00
compact_files_test.cc Fix an ASAN error in compact_files_test 2016-01-26 11:30:30 -08:00
compacted_db_impl.cc Remove db_impl_readonly dependency on utilities 2015-07-14 11:32:54 -07:00
compacted_db_impl.h Remove db_impl_readonly dependency on utilities 2015-07-14 11:32:54 -07:00
compaction_iterator_test.cc Support marking snapshots for write-conflict checking - Take 2 2015-12-08 16:47:31 -08:00
compaction_iterator.cc compaction assertion triggering test fix for sequence zeroing assertion trip 2015-12-18 16:08:31 -08:00
compaction_iterator.h Change SingleDelete to support conflict checking 2015-12-10 11:35:38 -08:00
compaction_job_stats_test.cc Fix rebase issues and new code warnings. 2015-12-11 16:56:24 -08:00
compaction_job_test.cc compaction assertion triggering test fix for sequence zeroing assertion trip 2015-12-18 16:08:31 -08:00
compaction_job.cc Add SstFileManager (component tracking all SST file in DBs and control the deletion rate) 2016-01-28 18:35:01 -08:00
compaction_job.h fix typos in comments 2015-12-11 01:54:48 +09:00
compaction_picker_test.cc Not scheduling more L1->L2 compaction if L0->L1 is pending with higher priority 2016-01-08 13:56:57 -08:00
compaction_picker.cc Not scheduling more L1->L2 compaction if L0->L1 is pending with higher priority 2016-01-08 13:56:57 -08:00
compaction_picker.h Fix minor bugs in delete operator, snprintf, and size_t usage 2015-12-15 15:26:20 -08:00
compaction.cc Merge pull request #846 from yuslepukhin/enble_c4244_lossofdata 2015-12-23 22:59:42 -08:00
compaction.h Report compaction reason in CompactionListener 2015-12-22 11:37:19 -08:00
comparator_db_test.cc Moving memtable related files from util to a new directory memtable 2015-10-16 14:10:33 -07:00
convenience.cc Delete files in given key range 2015-12-29 13:22:13 -08:00
corruption_test.cc Merge pull request #846 from yuslepukhin/enble_c4244_lossofdata 2015-12-23 22:59:42 -08:00
cuckoo_table_db_test.cc Block cuckoo table tests in ROCKSDB_LITE 2015-07-20 10:50:46 -07:00
db_bench.cc db_bench: explicitly clear buffer in compress benchmark 2016-01-19 18:11:46 -08:00
db_compaction_filter_test.cc compaction assertion triggering test fix for sequence zeroing assertion trip 2015-12-18 16:08:31 -08:00
db_compaction_test.cc Add options.base_background_compactions as a number of compaction threads for low compaction debt 2016-01-29 16:15:53 -08:00
db_dynamic_level_test.cc No need to #ifdef test only code on windows 2015-10-22 15:15:37 -07:00
db_filesnapshot.cc Add wal files to Checkpoint for multiple column families. 2015-06-19 16:08:31 -07:00
db_impl_debug.cc Running manual compactions in parallel with other automatic or manual compactions in restricted cases 2015-12-14 11:20:34 -08:00
db_impl_experimental.cc Clean up InstallSuperVersion 2015-06-17 12:37:59 -07:00
db_impl_readonly.cc Remove db_impl_readonly dependency on utilities 2015-07-14 11:32:54 -07:00
db_impl_readonly.h Override DBImplReadOnly::SyncWAL() to return NotSupported. Previously, calling it caused program abort. 2015-09-25 21:25:30 -07:00
db_impl.cc Improve perf of Pessimistic Transaction expirations (and optimistic transactions) 2016-02-05 10:44:13 -08:00
db_impl.h Eliminate duplicated property constants 2016-02-02 19:14:56 -08:00
db_info_dumper.cc [directory includes cleanup] Finish removing util->db dependencies 2016-01-26 10:49:24 -08:00
db_info_dumper.h [directory includes cleanup] Finish removing util->db dependencies 2016-01-26 10:49:24 -08:00
db_inplace_update_test.cc Clean up dependency: Move db_test_util.* to db directory 2015-10-12 13:05:42 -07:00
db_iter_test.cc No need to #ifdef test only code on windows 2015-10-22 15:15:37 -07:00
db_iter.cc Optimize DBIter::Prev() by reducing stack overhead 2016-01-07 07:59:14 -08:00
db_iter.h Introduce ReadOptions::pin_data (support zero copy for keys) 2015-12-16 12:08:30 -08:00
db_log_iter_test.cc Fix clang warnings regarding unnecessary std::move 2015-12-24 04:10:00 +08:00
db_properties_test.cc Eliminate duplicated property constants 2016-02-02 19:14:56 -08:00
db_table_properties_test.cc Avoid empty ranges vector with subsequent zero element access 2015-12-02 14:50:33 -08:00
db_tailing_iter_test.cc Fix minor bugs in delete operator, snprintf, and size_t usage 2015-12-15 15:26:20 -08:00
db_test_util.cc Add BlockBasedTableOptions::index_block_restart_interval 2016-02-05 10:22:37 -08:00
db_test_util.h Add BlockBasedTableOptions::index_block_restart_interval 2016-02-05 10:22:37 -08:00
db_test.cc Improve perf of Pessimistic Transaction expirations (and optimistic transactions) 2016-02-05 10:44:13 -08:00
db_universal_compaction_test.cc Skip filters for last L0 file if hit-optimized 2016-02-01 14:58:46 -08:00
db_wal_test.cc No need to #ifdef test only code on windows 2015-10-22 15:15:37 -07:00
dbformat_test.cc Avoid manipulating const char* arrays 2015-07-14 00:21:41 -07:00
dbformat.cc Support for SingleDelete() 2015-09-17 11:42:56 -07:00
dbformat.h Introduce ReadOptions::pin_data (support zero copy for keys) 2015-12-16 12:08:30 -08:00
deletefile_test.cc Fix minor bugs in delete operator, snprintf, and size_t usage 2015-12-15 15:26:20 -08:00
event_helpers.cc Enable MS compiler warning c4244. 2015-12-11 16:47:34 -08:00
event_helpers.h Add EventListener::OnTableFileDeletion() 2015-06-03 19:57:01 -07:00
experimental.cc Implement DB::PromoteL0 method 2015-04-23 12:10:36 -07:00
fault_injection_test.cc fix sporadic failure in fault_injection_test 2016-01-06 11:51:53 -08:00
file_indexer_test.cc Fix possible SIGSEGV in CompactRange (github issue #596) 2015-04-29 10:52:31 -07:00
file_indexer.cc Fix possible SIGSEGV in CompactRange (github issue #596) 2015-04-29 10:52:31 -07:00
file_indexer.h fix typos in comments 2015-12-11 01:54:48 +09:00
filename_test.cc rocksdb: switch to gtest 2015-03-17 14:08:00 -07:00
filename.cc Enable RocksDB to persist Options file. 2015-11-10 22:58:01 -08:00
filename.h Enable RocksDB to persist Options file. 2015-11-10 22:58:01 -08:00
flush_job_test.cc Use SST files for Transaction conflict detection 2015-12-11 12:34:11 -08:00
flush_job.cc [directory includes cleanup] Remove util->db dependency for ThreadStatusUtil 2016-01-26 10:49:16 -08:00
flush_job.h Use SST files for Transaction conflict detection 2015-12-11 12:34:11 -08:00
flush_scheduler.cc support for concurrent adds to memtable 2015-12-25 11:03:40 -08:00
flush_scheduler.h support for concurrent adds to memtable 2015-12-25 11:03:40 -08:00
forward_iterator_bench.cc Block forward_iterator_bench under MAC and Windows 2015-11-17 11:51:37 -08:00
forward_iterator.cc Fix minor bugs in delete operator, snprintf, and size_t usage 2015-12-15 15:26:20 -08:00
forward_iterator.h Reuse file iterators in tailing iterator when memtable is flushed 2015-11-13 15:50:59 -08:00
inlineskiplist_test.cc support for concurrent adds to memtable 2015-12-25 11:03:40 -08:00
inlineskiplist.h always invalidate sequential-insertion cache for concurrent skiplist adds 2016-02-03 11:08:16 -08:00
internal_stats.cc Eliminate duplicated property constants 2016-02-02 19:14:56 -08:00
internal_stats.h Eliminate duplicated property constants 2016-02-02 19:14:56 -08:00
job_context.h fixed leaking log::Writers 2015-07-07 12:10:10 -07:00
listener_test.cc Report compaction reason in CompactionListener 2015-12-22 11:37:19 -08:00
log_format.h log_{reader,write}: recyclable record format 2015-10-19 17:24:05 -04:00
log_reader.cc log_{reader,write}: recyclable record format 2015-10-19 17:24:05 -04:00
log_reader.h log_{reader,write}: recyclable record format 2015-10-19 17:24:05 -04:00
log_test.cc log_{reader,write}: recyclable record format 2015-10-19 17:24:05 -04:00
log_writer.cc Enable C4267 warning 2015-11-24 16:33:09 +03:00
log_writer.h Enable C4267 warning 2015-11-24 16:33:09 +03:00
managed_iterator.cc [directory includes cleanup] Move cross-function test points 2016-01-26 10:49:05 -08:00
managed_iterator.h Fixed xfunc related compile errors in ROCKSDB_LITE 2015-04-09 21:05:18 -07:00
manual_compaction_test.cc Move manual_compaction_test.cc from util to db 2015-10-14 11:06:27 -07:00
memtable_allocator.cc support for concurrent adds to memtable 2015-12-25 11:03:40 -08:00
memtable_allocator.h support for concurrent adds to memtable 2015-12-25 11:03:40 -08:00
memtable_list_test.cc Removing duplicate code 2015-08-05 07:33:27 -07:00
memtable_list.cc Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00
memtable_list.h Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00
memtable.cc Optimize GetLatestSequenceForKey 2016-01-06 13:43:22 -08:00
memtable.h support for concurrent adds to memtable 2015-12-25 11:03:40 -08:00
memtablerep_bench.cc Merge pull request #811 from OverlordQ/unused-variable-warning 2015-11-02 12:44:27 -08:00
merge_context.h API to fetch from both a WriteBatchWithIndex and the db 2015-05-11 14:51:51 -07:00
merge_helper_test.cc Compaction filter on merge operands 2015-10-07 09:30:03 -07:00
merge_helper.cc Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00
merge_helper.h Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00
merge_operator.cc Call merge operators with empty values 2015-06-26 11:35:46 -07:00
merge_test.cc Fix minor bugs in delete operator, snprintf, and size_t usage 2015-12-15 15:26:20 -08:00
options_file_test.cc Fixed build failure of RocksDBLite test on options_file_test.cc 2015-11-10 23:23:36 -08:00
perf_context_test.cc PerfContext::ToString() add option to exclude zero counters 2016-02-01 13:41:13 -08:00
plain_table_db_test.cc Fix compile error. 2016-01-11 16:10:48 -08:00
prefix_test.cc Prefix-based iterating only shows keys in prefix 2015-11-05 13:24:05 -08:00
repair.cc Disable Visual Studio Warning C4351 2015-12-28 15:06:34 -08:00
skiplist_test.cc rocksdb: switch to gtest 2015-03-17 14:08:00 -07:00
skiplist.h Switch to thread-local random for skiplist 2015-11-09 19:25:22 -08:00
slice.cc Create an abstract interface for write batches 2015-03-17 19:23:08 -07:00
snapshot_impl.cc Support marking snapshots for write-conflict checking - Take 2 2015-12-08 16:47:31 -08:00
snapshot_impl.h Use SST files for Transaction conflict detection 2015-12-11 12:34:11 -08:00
table_cache.cc Skip bottom-level filter block caching when hit-optimized 2015-12-23 10:15:07 -08:00
table_cache.h Skip bottom-level filter block caching when hit-optimized 2015-12-23 10:15:07 -08:00
table_properties_collector_test.cc Pass column family ID to table property collector 2015-10-09 14:36:51 -07:00
table_properties_collector.cc Support for SingleDelete() 2015-09-17 11:42:56 -07:00
table_properties_collector.h Pass column family ID to table property collector 2015-10-09 14:36:51 -07:00
transaction_log_impl.cc log_reader: pass log_number and optional info_log to ctor 2015-10-18 21:24:32 -04:00
transaction_log_impl.h Move rate_limiter, write buffering, most perf context instrumentation and most random kill out of Env 2015-07-17 16:58:18 -07:00
version_builder_test.cc Add a mode to always pick the oldest file to compact for each level 2015-09-21 17:21:59 -07:00
version_builder.cc Fix minor bugs in delete operator, snprintf, and size_t usage 2015-12-15 15:26:20 -08:00
version_builder.h Log more information for the add file with overlapping range failure 2015-10-19 17:31:13 -07:00
version_edit_test.cc New Manifest format to allow customized fields in NewFile. 2015-10-08 15:51:45 -07:00
version_edit.cc New Manifest format to allow customized fields in NewFile. 2015-10-08 15:51:45 -07:00
version_edit.h Enable MS compiler warning c4244. 2015-12-11 16:47:34 -08:00
version_set_test.cc Fix minor bugs in delete operator, snprintf, and size_t usage 2015-12-15 15:26:20 -08:00
version_set.cc Skip filters for last L0 file if hit-optimized 2016-02-01 14:58:46 -08:00
version_set.h Skip filters for last L0 file if hit-optimized 2016-02-01 14:58:46 -08:00
wal_manager_test.cc Fix clang warnings regarding unnecessary std::move 2015-12-24 04:10:00 +08:00
wal_manager.cc Fix clang warnings regarding unnecessary std::move 2015-12-24 04:10:00 +08:00
wal_manager.h Fix -Wnon-virtual-dtor errors 2014-11-10 17:39:38 -05:00
write_batch_base.cc Support for SingleDelete() 2015-09-17 11:42:56 -07:00
write_batch_internal.h Improve perf of Pessimistic Transaction expirations (and optimistic transactions) 2016-02-05 10:44:13 -08:00
write_batch_test.cc Fix WriteBatchTest.ManyUpdates, WriteBatchTest.LargeKeyValue under clang 2016-02-01 16:07:53 -08:00
write_batch.cc Improve perf of Pessimistic Transaction expirations (and optimistic transactions) 2016-02-05 10:44:13 -08:00
write_callback_test.cc Improve perf of Pessimistic Transaction expirations (and optimistic transactions) 2016-02-05 10:44:13 -08:00
write_callback.h Improve perf of Pessimistic Transaction expirations (and optimistic transactions) 2016-02-05 10:44:13 -08:00
write_controller_test.cc When slowdown is triggered, reduce the write rate 2015-12-23 11:33:15 -08:00
write_controller.cc Add options.base_background_compactions as a number of compaction threads for low compaction debt 2016-01-29 16:15:53 -08:00
write_controller.h Add options.base_background_compactions as a number of compaction threads for low compaction debt 2016-01-29 16:15:53 -08:00
write_thread.cc Improve perf of Pessimistic Transaction expirations (and optimistic transactions) 2016-02-05 10:44:13 -08:00
write_thread.h Improve perf of Pessimistic Transaction expirations (and optimistic transactions) 2016-02-05 10:44:13 -08:00
writebuffer.h support for concurrent adds to memtable 2015-12-25 11:03:40 -08:00
xfunc_test_points.cc [directory includes cleanup] Move cross-function test points 2016-01-26 10:49:05 -08:00
xfunc_test_points.h [directory includes cleanup] Move cross-function test points 2016-01-26 10:49:05 -08:00