andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
95b0e89b5d
rocksdb/db
History
Siying Dong 95b0e89b5d Improve write buffer manager (and allow the size to be tracked in block cache) 
Summary:
Improve write buffer manager in several ways:
1. Size is tracked when arena block is allocated, rather than every allocation, so that it can better track actual memory usage and the tracking overhead is slightly lower.
2. We start to trigger memtable flush when 7/8 of the memory cap hits, instead of 100%, and make 100% much harder to hit.
3. Allow a cache object to be passed into buffer manager and the size allocated by memtable can be costed there. This can help users have one single memory cap across block cache and memtable.
Closes https://github.com/facebook/rocksdb/pull/2350

Differential Revision: D5110648

Pulled By: siying

fbshipit-source-id: b4238113094bf22574001e446b5d88523ba00017
2017-06-02 14:26:56 -07:00
..
builder.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
builder.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
c_test.c support merge and delete in file ingestion 2017-05-26 12:11:21 -07:00
c.cc support merge and delete in file ingestion 2017-05-26 12:11:21 -07:00
column_family_test.cc fix column_family_test asan 2017-05-24 16:41:51 -07:00
column_family.cc Pass CF ID to MemTableRepFactory 2017-06-02 12:12:06 -07:00
column_family.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
compact_files_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
compacted_db_impl.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
compacted_db_impl.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
compaction_iteration_stats.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
compaction_iterator_test.cc Support ingest_behind for IngestExternalFile 2017-05-17 11:42:42 -07:00
compaction_iterator.cc Support ingest_behind for IngestExternalFile 2017-05-17 11:42:42 -07:00
compaction_iterator.h Support ingest_behind for IngestExternalFile 2017-05-17 11:42:42 -07:00
compaction_job_stats_test.cc New API for background work in single thread pool 2017-05-23 11:12:27 -07:00
compaction_job_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
compaction_job.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
compaction_job.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
compaction_picker_test.cc account for L0 size in estimated compaction bytes 2017-06-01 17:56:59 -07:00
compaction_picker_universal.cc Support ingest_behind for IngestExternalFile 2017-05-17 11:42:42 -07:00
compaction_picker_universal.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
compaction_picker.cc Support ingest_behind for IngestExternalFile 2017-05-17 11:42:42 -07:00
compaction_picker.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
compaction.cc Avoid calling fallocate with UINT64_MAX 2017-05-04 17:43:22 -07:00
compaction.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
comparator_db_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
convenience.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
corruption_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
cuckoo_table_db_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
db_basic_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
db_block_cache_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
db_bloom_filter_test.cc dont skip IO for filter blocks 2017-05-09 09:52:02 -07:00
db_compaction_filter_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
db_compaction_test.cc Introduce max_background_jobs mutable option 2017-05-24 11:29:08 -07:00
db_dynamic_level_test.cc Improve write buffer manager (and allow the size to be tracked in block cache) 2017-06-02 14:26:56 -07:00
db_filesnapshot.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
db_flush_test.cc New API for background work in single thread pool 2017-05-23 11:12:27 -07:00
db_impl_compaction_flush.cc Introduce max_background_jobs mutable option 2017-05-24 11:29:08 -07:00
db_impl_debug.cc Introduce max_background_jobs mutable option 2017-05-24 11:29:08 -07:00
db_impl_experimental.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
db_impl_files.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
db_impl_open.cc Introduce max_background_jobs mutable option 2017-05-24 11:29:08 -07:00
db_impl_readonly.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
db_impl_readonly.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
db_impl_write.cc Fixing blob db sequence number handling 2017-05-31 10:56:45 -07:00
db_impl.cc db: avoid #includeing malloc and jemalloc simultaneously 2017-05-31 22:43:02 -07:00
db_impl.h Introduce max_background_jobs mutable option 2017-05-24 11:29:08 -07:00
db_info_dumper.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
db_info_dumper.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
db_inplace_update_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
db_io_failure_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
db_iter_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
db_iter.cc Histogram of number of merge operands 2017-05-31 07:41:44 -07:00
db_iter.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
db_iterator_test.cc disable direct reads for log and manifest and add direct io to tests 2017-05-22 18:41:28 -07:00
db_log_iter_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
db_memtable_test.cc Improve write buffer manager (and allow the size to be tracked in block cache) 2017-06-02 14:26:56 -07:00
db_merge_operator_test.cc support merge and delete in file ingestion 2017-05-26 12:11:21 -07:00
db_options_test.cc Introduce max_background_jobs mutable option 2017-05-24 11:29:08 -07:00
db_properties_test.cc account for L0 size in estimated compaction bytes 2017-06-01 17:56:59 -07:00
db_range_del_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
db_sst_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
db_statistics_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
db_table_properties_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
db_tailing_iter_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
db_test2.cc Improve write buffer manager (and allow the size to be tracked in block cache) 2017-06-02 14:26:56 -07:00
db_test_util.cc Fixing blob db sequence number handling 2017-05-31 10:56:45 -07:00
db_test_util.h Improve write buffer manager (and allow the size to be tracked in block cache) 2017-06-02 14:26:56 -07:00
db_test.cc Improve write buffer manager (and allow the size to be tracked in block cache) 2017-06-02 14:26:56 -07:00
db_universal_compaction_test.cc Fixed some spelling mistakes 2017-05-17 23:12:36 -07:00
db_wal_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
db_write_test.cc Fix DBWriteTest::ReturnSequenceNumberMultiThreaded data race 2017-06-02 11:42:11 -07:00
dbformat_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
dbformat.cc do not read next datablock if upperbound is reached 2017-05-05 23:20:01 -07:00
dbformat.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
deletefile_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
event_helpers.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
event_helpers.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
experimental.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
external_sst_file_basic_test.cc Support ingest file when range deletions exist 2017-05-31 13:57:19 -07:00
external_sst_file_ingestion_job.cc Fix TSAN: avoid arena mode with range deletions 2017-06-01 22:26:49 -07:00
external_sst_file_ingestion_job.h Support ingest file when range deletions exist 2017-05-31 13:57:19 -07:00
external_sst_file_test.cc support merge and delete in file ingestion 2017-05-26 12:11:21 -07:00
fault_injection_test.cc Avoid unsupported attributes when not building with UBSAN 2017-05-30 11:13:01 -07:00
file_indexer_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
file_indexer.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
file_indexer.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
filename_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
flush_job_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
flush_job.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
flush_job.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
flush_scheduler.cc New WriteImpl to pipeline WAL/memtable write 2017-05-19 14:26:42 -07:00
flush_scheduler.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
forward_iterator_bench.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
forward_iterator.cc do not read next datablock if upperbound is reached 2017-05-05 23:20:01 -07:00
forward_iterator.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
internal_stats.cc Fix column family seconds_up accounting 2017-05-23 17:14:04 -07:00
internal_stats.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
job_context.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
listener_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
log_format.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
log_reader.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
log_reader.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
log_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
log_writer.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
log_writer.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
malloc_stats.cc db: avoid #includeing malloc and jemalloc simultaneously 2017-05-31 22:43:02 -07:00
malloc_stats.h db: avoid #includeing malloc and jemalloc simultaneously 2017-05-31 22:43:02 -07:00
managed_iterator.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
managed_iterator.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
manual_compaction_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
memtable_list_test.cc Pass CF ID to MemTableRepFactory 2017-06-02 12:12:06 -07:00
memtable_list.cc Fix TSAN: avoid arena mode with range deletions 2017-06-01 22:26:49 -07:00
memtable_list.h Fix TSAN: avoid arena mode with range deletions 2017-06-01 22:26:49 -07:00
memtable.cc Improve write buffer manager (and allow the size to be tracked in block cache) 2017-06-02 14:26:56 -07:00
memtable.h Improve write buffer manager (and allow the size to be tracked in block cache) 2017-06-02 14:26:56 -07:00
merge_context.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
merge_helper_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
merge_helper.cc Histogram of number of merge operands 2017-05-31 07:41:44 -07:00
merge_helper.h Histogram of number of merge operands 2017-05-31 07:41:44 -07:00
merge_operator.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
merge_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
options_file_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
perf_context_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
pinned_iterators_manager.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
plain_table_db_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
prefix_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
range_del_aggregator_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
range_del_aggregator.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
range_del_aggregator.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
repair_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
repair.cc disable direct reads for log and manifest and add direct io to tests 2017-05-22 18:41:28 -07:00
snapshot_impl.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
snapshot_impl.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
table_cache.cc Support ingest file when range deletions exist 2017-05-31 13:57:19 -07:00
table_cache.h Support ingest file when range deletions exist 2017-05-31 13:57:19 -07:00
table_properties_collector_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
table_properties_collector.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
table_properties_collector.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
transaction_log_impl.cc disable direct reads for log and manifest and add direct io to tests 2017-05-22 18:41:28 -07:00
transaction_log_impl.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
version_builder_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
version_builder.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
version_builder.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
version_edit_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
version_edit.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
version_edit.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
version_set_test.cc Set lower-bound on dynamic level sizes 2017-05-04 18:16:12 -07:00
version_set.cc Fix TSAN: avoid arena mode with range deletions 2017-06-01 22:26:49 -07:00
version_set.h Fix TSAN: avoid arena mode with range deletions 2017-06-01 22:26:49 -07:00
wal_manager_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
wal_manager.cc disable direct reads for log and manifest and add direct io to tests 2017-05-22 18:41:28 -07:00
wal_manager.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
write_batch_base.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
write_batch_internal.h Fixing blob db sequence number handling 2017-05-31 10:56:45 -07:00
write_batch_test.cc Pass CF ID to MemTableRepFactory 2017-06-02 12:12:06 -07:00
write_batch.cc Fixing blob db sequence number handling 2017-05-31 10:56:45 -07:00
write_callback_test.cc Fix data races caught by tsan 2017-05-22 10:27:23 -07:00
write_callback.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
write_controller_test.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
write_controller.cc Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
write_controller.h Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
write_thread.cc New WriteImpl to pipeline WAL/memtable write 2017-05-19 14:26:42 -07:00
write_thread.h New WriteImpl to pipeline WAL/memtable write 2017-05-19 14:26:42 -07:00