andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
07bdcb91fe
rocksdb/db
History
Yi Wu 07bdcb91fe New WriteImpl to pipeline WAL/memtable write 
Summary:
PipelineWriteImpl is an alternative approach to WriteImpl. In WriteImpl, only one thread is allow to write at the same time. This thread will do both WAL and memtable writes for all write threads in the write group. Pending writers wait in queue until the current writer finishes. In the pipeline write approach, two queue is maintained: one WAL writer queue and one memtable writer queue. All writers (regardless of whether they need to write WAL) will still need to first join the WAL writer queue, and after the house keeping work and WAL writing, they will need to join memtable writer queue if needed. The benefit of this approach is that
1. Writers without memtable writes (e.g. the prepare phase of two phase commit) can exit write thread once WAL write is finish. They don't need to wait for memtable writes in case of group commit.
2. Pending writers only need to wait for previous WAL writer finish to be able to join the write thread, instead of wait also for previous memtable writes.

Merging #2056 and #2058 into this PR.
Closes https://github.com/facebook/rocksdb/pull/2286

Differential Revision: D5054606

Pulled By: yiwu-arbug

fbshipit-source-id: ee5b11efd19d3e39d6b7210937b11cefdd4d1c8d
2017-05-19 14:26:42 -07:00
..
builder.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
builder.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
c_test.c add Transactions and Checkpoint to C API 2017-05-16 22:59:43 -07:00
c.cc add Transactions and Checkpoint to C API 2017-05-16 22:59:43 -07:00
column_family_test.cc column_family_test: EnvCounter::num_new_writable_file_ to be atomic 2017-05-18 13:56:12 -07:00
column_family.cc Support ingest_behind for IngestExternalFile 2017-05-17 11:42:42 -07:00
column_family.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
compact_files_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
compacted_db_impl.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
compacted_db_impl.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
compaction_iteration_stats.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
compaction_iterator_test.cc Support ingest_behind for IngestExternalFile 2017-05-17 11:42:42 -07:00
compaction_iterator.cc Support ingest_behind for IngestExternalFile 2017-05-17 11:42:42 -07:00
compaction_iterator.h Support ingest_behind for IngestExternalFile 2017-05-17 11:42:42 -07:00
compaction_job_stats_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
compaction_job_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
compaction_job.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
compaction_job.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
compaction_picker_test.cc Support ingest_behind for IngestExternalFile 2017-05-17 11:42:42 -07:00
compaction_picker_universal.cc Support ingest_behind for IngestExternalFile 2017-05-17 11:42:42 -07:00
compaction_picker_universal.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
compaction_picker.cc Support ingest_behind for IngestExternalFile 2017-05-17 11:42:42 -07:00
compaction_picker.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
compaction.cc Avoid calling fallocate with UINT64_MAX 2017-05-04 17:43:22 -07:00
compaction.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
comparator_db_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
convenience.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
corruption_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
cuckoo_table_db_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
db_basic_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
db_block_cache_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
db_bloom_filter_test.cc dont skip IO for filter blocks 2017-05-09 09:52:02 -07:00
db_compaction_filter_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
db_compaction_test.cc Roundup read bytes in ReadaheadRandomAccessFile 2017-05-05 12:14:14 -07:00
db_dynamic_level_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
db_filesnapshot.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
db_flush_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
db_impl_compaction_flush.cc Fixed some spelling mistakes 2017-05-17 23:12:36 -07:00
db_impl_debug.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
db_impl_experimental.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
db_impl_files.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
db_impl_open.cc Add bulk create/drop column family API 2017-05-07 23:20:46 -07:00
db_impl_readonly.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
db_impl_readonly.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
db_impl_write.cc New WriteImpl to pipeline WAL/memtable write 2017-05-19 14:26:42 -07:00
db_impl.cc New WriteImpl to pipeline WAL/memtable write 2017-05-19 14:26:42 -07:00
db_impl.h New WriteImpl to pipeline WAL/memtable write 2017-05-19 14:26:42 -07:00
db_info_dumper.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
db_info_dumper.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
db_inplace_update_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
db_io_failure_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
db_iter_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
db_iter.cc Suppress clang-analyzer false positive 2017-05-19 10:56:28 -07:00
db_iter.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
db_iterator_test.cc do not read next datablock if upperbound is reached 2017-05-05 23:20:01 -07:00
db_log_iter_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
db_memtable_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
db_merge_operator_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
db_options_test.cc Max open files mutable 2017-05-03 21:13:14 -07:00
db_properties_test.cc Add DB:ResetStats() 2017-04-18 16:56:48 -07:00
db_range_del_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
db_sst_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
db_statistics_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
db_table_properties_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
db_tailing_iter_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
db_test2.cc fix readampbitmap tests 2017-05-10 12:29:23 -07:00
db_test_util.cc Configure index partition size 2017-03-28 12:09:12 -07:00
db_test_util.h Avoid calling fallocate with UINT64_MAX 2017-05-04 17:43:22 -07:00
db_test.cc Allow IntraL0 compaction in FIFO Compaction 2017-05-04 18:16:13 -07:00
db_universal_compaction_test.cc Fixed some spelling mistakes 2017-05-17 23:12:36 -07:00
db_wal_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
dbformat_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
dbformat.cc do not read next datablock if upperbound is reached 2017-05-05 23:20:01 -07:00
dbformat.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
deletefile_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
event_helpers.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
event_helpers.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
experimental.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
external_sst_file_basic_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
external_sst_file_ingestion_job.cc Support ingest_behind for IngestExternalFile 2017-05-17 11:42:42 -07:00
external_sst_file_ingestion_job.h Support ingest_behind for IngestExternalFile 2017-05-17 11:42:42 -07:00
external_sst_file_test.cc Support ingest_behind for IngestExternalFile 2017-05-17 11:42:42 -07:00
fault_injection_test.cc Suppress clang-analyzer false positive 2017-05-19 10:56:28 -07:00
file_indexer_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
file_indexer.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
file_indexer.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
filename_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
flush_job_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
flush_job.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
flush_job.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
flush_scheduler.cc New WriteImpl to pipeline WAL/memtable write 2017-05-19 14:26:42 -07:00
flush_scheduler.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
forward_iterator_bench.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
forward_iterator.cc do not read next datablock if upperbound is reached 2017-05-05 23:20:01 -07:00
forward_iterator.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
internal_stats.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
internal_stats.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
job_context.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
listener_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
log_format.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
log_reader.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
log_reader.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
log_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
log_writer.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
log_writer.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
managed_iterator.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
managed_iterator.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
manual_compaction_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
memtable_list_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
memtable_list.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
memtable_list.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
memtable.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
memtable.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
merge_context.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
merge_helper_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
merge_helper.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
merge_helper.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
merge_operator.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
merge_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
options_file_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
perf_context_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
pinned_iterators_manager.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
plain_table_db_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
prefix_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
range_del_aggregator_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
range_del_aggregator.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
range_del_aggregator.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
repair_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
repair.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
snapshot_impl.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
snapshot_impl.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
table_cache.cc do not read next datablock if upperbound is reached 2017-05-05 23:20:01 -07:00
table_cache.h max_open_files dynamic set, follow up 2017-05-04 10:42:45 -07:00
table_properties_collector_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
table_properties_collector.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
table_properties_collector.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
transaction_log_impl.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
transaction_log_impl.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
version_builder_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
version_builder.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
version_builder.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
version_edit_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
version_edit.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
version_edit.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
version_set_test.cc Set lower-bound on dynamic level sizes 2017-05-04 18:16:12 -07:00
version_set.cc do not read next datablock if upperbound is reached 2017-05-05 23:20:01 -07:00
version_set.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
wal_manager_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
wal_manager.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
wal_manager.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
write_batch_base.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
write_batch_internal.h New WriteImpl to pipeline WAL/memtable write 2017-05-19 14:26:42 -07:00
write_batch_test.cc support PopSavePoint for WriteBatch 2017-05-03 10:57:45 -07:00
write_batch.cc New WriteImpl to pipeline WAL/memtable write 2017-05-19 14:26:42 -07:00
write_callback_test.cc New WriteImpl to pipeline WAL/memtable write 2017-05-19 14:26:42 -07:00
write_callback.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
write_controller_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
write_controller.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
write_controller.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
write_thread.cc New WriteImpl to pipeline WAL/memtable write 2017-05-19 14:26:42 -07:00
write_thread.h New WriteImpl to pipeline WAL/memtable write 2017-05-19 14:26:42 -07:00