andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
rocksdb/db
History
sdong b9f77ba12b When slowdown is triggered, reduce the write rate 
Summary: It's usually hard for users to set a value of options.delayed_write_rate. With this diff, after slowdown condition triggers, we greedily reduce write rate if estimated pending compaction bytes increase. If estimated compaction pending bytes drop, we increase the write rate.

Test Plan:
Add a unit test
Test with db_bench setting:
TEST_TMPDIR=/dev/shm/ ./db_bench --benchmarks=fillrandom -num=10000000 --soft_pending_compaction_bytes_limit=1000000000 --hard_pending_compaction_bytes_limit=3000000000 --delayed_write_rate=100000000

and make sure without the commit, write stop will happen, but with the commit, it will not happen.

Reviewers: igor, anthony, rven, yhchiang, kradhakrishnan, IslamAbdelRahman

Reviewed By: IslamAbdelRahman

Subscribers: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D52131
2015-12-23 11:33:15 -08:00
..
builder.cc
Use SST files for Transaction conflict detection
2015-12-11 12:34:11 -08:00
builder.h
Use SST files for Transaction conflict detection
2015-12-11 12:34:11 -08:00
c_test.c
Deprecate CompactionFilterV2
2015-07-17 18:59:11 +02:00
c.cc
Move skip_table_builder_flush to BlockBasedTableOption
2015-10-30 18:33:01 -07:00
column_family_test.cc
When slowdown is triggered, reduce the write rate
2015-12-23 11:33:15 -08:00
column_family.cc
When slowdown is triggered, reduce the write rate
2015-12-23 11:33:15 -08:00
column_family.h
When slowdown is triggered, reduce the write rate
2015-12-23 11:33:15 -08:00
compact_files_test.cc
Make sure that CompactFiles does not run two parallel Level 0 compactions
2015-11-13 12:01:00 -08:00
compacted_db_impl.cc
Remove db_impl_readonly dependency on utilities
2015-07-14 11:32:54 -07:00
compacted_db_impl.h
Remove db_impl_readonly dependency on utilities
2015-07-14 11:32:54 -07:00
compaction_iterator_test.cc
Support marking snapshots for write-conflict checking - Take 2
2015-12-08 16:47:31 -08:00
compaction_iterator.cc
compaction assertion triggering test fix for sequence zeroing assertion trip
2015-12-18 16:08:31 -08:00
compaction_iterator.h
Change SingleDelete to support conflict checking
2015-12-10 11:35:38 -08:00
compaction_job_stats_test.cc
No need to #ifdef test only code on windows
2015-10-22 15:15:37 -07:00
compaction_job_test.cc
compaction assertion triggering test fix for sequence zeroing assertion trip
2015-12-18 16:08:31 -08:00
compaction_job.cc
Sorting std::vector instead of using std::set
2015-12-22 14:34:57 +08:00
compaction_job.h
fix typos in comments
2015-12-11 01:54:48 +09:00
compaction_picker_test.cc
Fix condition for bottommost level
2015-10-05 17:40:18 -07:00
compaction_picker.cc
Report compaction reason in CompactionListener
2015-12-22 11:37:19 -08:00
compaction_picker.h
Fix minor bugs in delete operator, snprintf, and size_t usage
2015-12-15 15:26:20 -08:00
compaction.cc
Report compaction reason in CompactionListener
2015-12-22 11:37:19 -08:00
compaction.h
Report compaction reason in CompactionListener
2015-12-22 11:37:19 -08:00
comparator_db_test.cc
Moving memtable related files from util to a new directory memtable
2015-10-16 14:10:33 -07:00
convenience.cc
move convenience.h out of utilities
2015-07-15 14:51:51 -07:00
corruption_test.cc
Fix minor bugs in delete operator, snprintf, and size_t usage
2015-12-15 15:26:20 -08:00
cuckoo_table_db_test.cc
Block cuckoo table tests in ROCKSDB_LITE
2015-07-20 10:50:46 -07:00
db_bench.cc
Fix use-after free in db_bench
2015-12-18 06:42:57 -08:00
db_compaction_filter_test.cc
compaction assertion triggering test fix for sequence zeroing assertion trip
2015-12-18 16:08:31 -08:00
db_compaction_test.cc
Fix clang warning regarding implicit conversion
2015-12-21 23:57:55 +08:00
db_dynamic_level_test.cc
No need to #ifdef test only code on windows
2015-10-22 15:15:37 -07:00
db_filesnapshot.cc
Add wal files to Checkpoint for multiple column families.
2015-06-19 16:08:31 -07:00
db_impl_debug.cc
Running manual compactions in parallel with other automatic or manual compactions in restricted cases
2015-12-14 11:20:34 -08:00
db_impl_experimental.cc
Clean up InstallSuperVersion
2015-06-17 12:37:59 -07:00
db_impl_readonly.cc
Remove db_impl_readonly dependency on utilities
2015-07-14 11:32:54 -07:00
db_impl_readonly.h
Override DBImplReadOnly::SyncWAL() to return NotSupported. Previously, calling it caused program abort.
2015-09-25 21:25:30 -07:00
db_impl.cc
Report compaction reason in CompactionListener
2015-12-22 11:37:19 -08:00
db_impl.h
When slowdown is triggered, reduce the write rate
2015-12-23 11:33:15 -08:00
db_inplace_update_test.cc
Clean up dependency: Move db_test_util.* to db directory
2015-10-12 13:05:42 -07:00
db_iter_test.cc
No need to #ifdef test only code on windows
2015-10-22 15:15:37 -07:00
db_iter.cc
Introduce ReadOptions::pin_data (support zero copy for keys)
2015-12-16 12:08:30 -08:00
db_iter.h
Introduce ReadOptions::pin_data (support zero copy for keys)
2015-12-16 12:08:30 -08:00
db_log_iter_test.cc
No need to #ifdef test only code on windows
2015-10-22 15:15:37 -07:00
db_table_properties_test.cc
Avoid empty ranges vector with subsequent zero element access
2015-12-02 14:50:33 -08:00
db_tailing_iter_test.cc
Fix minor bugs in delete operator, snprintf, and size_t usage
2015-12-15 15:26:20 -08:00
db_test_util.cc
Merge pull request #853 from Vaisman/enable_C4267_warning
2015-12-08 17:59:24 -08:00
db_test_util.h
Merge pull request #853 from Vaisman/enable_C4267_warning
2015-12-08 17:59:24 -08:00
db_test.cc
When slowdown is triggered, reduce the write rate
2015-12-23 11:33:15 -08:00
db_universal_compaction_test.cc
Running manual compactions in parallel with other automatic or manual compactions in restricted cases
2015-12-14 11:20:34 -08:00
db_wal_test.cc
No need to #ifdef test only code on windows
2015-10-22 15:15:37 -07:00
dbformat_test.cc
Avoid manipulating const char* arrays
2015-07-14 00:21:41 -07:00
dbformat.cc
Support for SingleDelete()
2015-09-17 11:42:56 -07:00
dbformat.h
Introduce ReadOptions::pin_data (support zero copy for keys)
2015-12-16 12:08:30 -08:00
deletefile_test.cc
Fix minor bugs in delete operator, snprintf, and size_t usage
2015-12-15 15:26:20 -08:00
event_helpers.cc
Passing table properties to compaction callback
2015-10-09 18:10:55 -07:00
event_helpers.h
Add EventListener::OnTableFileDeletion()
2015-06-03 19:57:01 -07:00
experimental.cc
Implement DB::PromoteL0 method
2015-04-23 12:10:36 -07:00
fault_injection_test.cc
Fix minor bugs in delete operator, snprintf, and size_t usage
2015-12-15 15:26:20 -08:00
file_indexer_test.cc
Fix possible SIGSEGV in CompactRange (github issue #596)
2015-04-29 10:52:31 -07:00
file_indexer.cc
Fix possible SIGSEGV in CompactRange (github issue #596)
2015-04-29 10:52:31 -07:00
file_indexer.h
fix typos in comments
2015-12-11 01:54:48 +09:00
filename_test.cc
rocksdb: switch to gtest
2015-03-17 14:08:00 -07:00
filename.cc
Enable RocksDB to persist Options file.
2015-11-10 22:58:01 -08:00
filename.h
Enable RocksDB to persist Options file.
2015-11-10 22:58:01 -08:00
flush_job_test.cc
Use SST files for Transaction conflict detection
2015-12-11 12:34:11 -08:00
flush_job.cc
Use SST files for Transaction conflict detection
2015-12-11 12:34:11 -08:00
flush_job.h
Use SST files for Transaction conflict detection
2015-12-11 12:34:11 -08:00
flush_scheduler.cc
Don't return (or dereference) dangling pointer
2014-10-02 14:33:16 -07:00
flush_scheduler.h
Fix data race #1
2015-01-26 11:48:07 -08:00
forward_iterator_bench.cc
Block forward_iterator_bench under MAC and Windows
2015-11-17 11:51:37 -08:00
forward_iterator.cc
Fix minor bugs in delete operator, snprintf, and size_t usage
2015-12-15 15:26:20 -08:00
forward_iterator.h
Reuse file iterators in tailing iterator when memtable is flushed
2015-11-13 15:50:59 -08:00
inlineskiplist_test.cc
InlineSkipList - part 2/3
2015-11-24 14:30:56 -08:00
inlineskiplist.h
InlineSkipList part 3/3 - new skiplist type that colocates key and node
2015-11-24 15:16:02 -08:00
internal_stats.cc
Slowdown when writing to the last write buffer
2015-12-17 10:49:08 -08:00
internal_stats.h
Slowdown when writing to the last write buffer
2015-12-17 10:49:08 -08:00
job_context.h
fixed leaking log::Writers
2015-07-07 12:10:10 -07:00
listener_test.cc
Report compaction reason in CompactionListener
2015-12-22 11:37:19 -08:00
log_format.h
log_{reader,write}: recyclable record format
2015-10-19 17:24:05 -04:00
log_reader.cc
log_{reader,write}: recyclable record format
2015-10-19 17:24:05 -04:00
log_reader.h
log_{reader,write}: recyclable record format
2015-10-19 17:24:05 -04:00
log_test.cc
log_{reader,write}: recyclable record format
2015-10-19 17:24:05 -04:00
log_writer.cc
Enable C4267 warning
2015-11-24 16:33:09 +03:00
log_writer.h
Enable C4267 warning
2015-11-24 16:33:09 +03:00
managed_iterator.cc
Windows Port from Microsoft
2015-07-01 16:13:56 -07:00
managed_iterator.h
Fixed xfunc related compile errors in ROCKSDB_LITE
2015-04-09 21:05:18 -07:00
manual_compaction_test.cc
Move manual_compaction_test.cc from util to db
2015-10-14 11:06:27 -07:00
memtable_allocator.cc
Enforce write buffer memory limit across column families
2014-12-02 12:09:20 -08:00
memtable_allocator.h
Enforce write buffer memory limit across column families
2014-12-02 12:09:20 -08:00
memtable_list_test.cc
Removing duplicate code
2015-08-05 07:33:27 -07:00
memtable_list.cc
Seperate InternalIterator from Iterator
2015-10-13 15:32:13 -07:00
memtable_list.h
Seperate InternalIterator from Iterator
2015-10-13 15:32:13 -07:00
memtable.cc
Introduce ReadOptions::pin_data (support zero copy for keys)
2015-12-16 12:08:30 -08:00
memtable.h
Fix typo
2015-12-15 23:47:47 +08:00
memtablerep_bench.cc
Merge pull request #811 from OverlordQ/unused-variable-warning
2015-11-02 12:44:27 -08:00
merge_context.h
API to fetch from both a WriteBatchWithIndex and the db
2015-05-11 14:51:51 -07:00
merge_helper_test.cc
Compaction filter on merge operands
2015-10-07 09:30:03 -07:00
merge_helper.cc
Seperate InternalIterator from Iterator
2015-10-13 15:32:13 -07:00
merge_helper.h
Seperate InternalIterator from Iterator
2015-10-13 15:32:13 -07:00
merge_operator.cc
Call merge operators with empty values
2015-06-26 11:35:46 -07:00
merge_test.cc
Fix minor bugs in delete operator, snprintf, and size_t usage
2015-12-15 15:26:20 -08:00
options_file_test.cc
Fixed build failure of RocksDBLite test on options_file_test.cc
2015-11-10 23:23:36 -08:00
perf_context_test.cc
Make perf_context.db_mutex_lock_nanos and db_condition_wait_nanos only measures DB Mutex
2015-10-13 10:41:48 -07:00
plain_table_db_test.cc
PlainTableReader to support non-mmap mode
2015-09-23 11:41:07 -07:00
prefix_test.cc
Prefix-based iterating only shows keys in prefix
2015-11-05 13:24:05 -08:00
repair.cc
Fix minor bugs in delete operator, snprintf, and size_t usage
2015-12-15 15:26:20 -08:00
skiplist_test.cc
rocksdb: switch to gtest
2015-03-17 14:08:00 -07:00
skiplist.h
Switch to thread-local random for skiplist
2015-11-09 19:25:22 -08:00
slice.cc
Create an abstract interface for write batches
2015-03-17 19:23:08 -07:00
snapshot_impl.cc
Support marking snapshots for write-conflict checking - Take 2
2015-12-08 16:47:31 -08:00
snapshot_impl.h
Use SST files for Transaction conflict detection
2015-12-11 12:34:11 -08:00
table_cache.cc
Skip bottom-level filter block caching when hit-optimized
2015-12-23 10:15:07 -08:00
table_cache.h
Skip bottom-level filter block caching when hit-optimized
2015-12-23 10:15:07 -08:00
table_properties_collector_test.cc
Pass column family ID to table property collector
2015-10-09 14:36:51 -07:00
table_properties_collector.cc
Support for SingleDelete()
2015-09-17 11:42:56 -07:00
table_properties_collector.h
Pass column family ID to table property collector
2015-10-09 14:36:51 -07:00
transaction_log_impl.cc
log_reader: pass log_number and optional info_log to ctor
2015-10-18 21:24:32 -04:00
transaction_log_impl.h
Move rate_limiter, write buffering, most perf context instrumentation and most random kill out of Env
2015-07-17 16:58:18 -07:00
version_builder_test.cc
Add a mode to always pick the oldest file to compact for each level
2015-09-21 17:21:59 -07:00
version_builder.cc
Fix minor bugs in delete operator, snprintf, and size_t usage
2015-12-15 15:26:20 -08:00
version_builder.h
Log more information for the add file with overlapping range failure
2015-10-19 17:31:13 -07:00
version_edit_test.cc
New Manifest format to allow customized fields in NewFile.
2015-10-08 15:51:45 -07:00
version_edit.cc
New Manifest format to allow customized fields in NewFile.
2015-10-08 15:51:45 -07:00
version_edit.h
New Manifest format to allow customized fields in NewFile.
2015-10-08 15:51:45 -07:00
version_set_test.cc
Fix minor bugs in delete operator, snprintf, and size_t usage
2015-12-15 15:26:20 -08:00
version_set.cc
When slowdown is triggered, reduce the write rate
2015-12-23 11:33:15 -08:00
version_set.h
When slowdown is triggered, reduce the write rate
2015-12-23 11:33:15 -08:00
wal_manager_test.cc
log_writer: pass log number and whether recycling is enabled to ctor
2015-10-18 21:24:32 -04:00
wal_manager.cc
log_reader: pass log_number and optional info_log to ctor
2015-10-18 21:24:32 -04:00
wal_manager.h
Fix -Wnon-virtual-dtor errors
2014-11-10 17:39:38 -05:00
write_batch_base.cc
Support for SingleDelete()
2015-09-17 11:42:56 -07:00
write_batch_internal.h
Don't merge WriteBatch-es if WAL is disabled
2015-11-12 10:50:38 -08:00
write_batch_test.cc
track WriteBatch contents
2015-11-10 16:56:06 -08:00
write_batch.cc
Fix minor bugs in delete operator, snprintf, and size_t usage
2015-12-15 15:26:20 -08:00
write_callback_test.cc
Fix compile for write_callback_test in ROCKSDB_LITE
2015-07-20 10:54:15 -07:00
write_callback.h
Optimistic Transactions
2015-05-29 14:36:35 -07:00
write_controller_test.cc
When slowdown is triggered, reduce the write rate
2015-12-23 11:33:15 -08:00
write_controller.cc
When slowdown is triggered, reduce the write rate
2015-12-23 11:33:15 -08:00
write_controller.h
When slowdown is triggered, reduce the write rate
2015-12-23 11:33:15 -08:00
write_thread.cc
Resubmit the fix for a race condition in persisting options
2015-12-08 17:01:02 -08:00
write_thread.h
reduce db mutex contention for write batch groups
2015-08-14 10:55:43 -07:00
writebuffer.h
Enforce write buffer memory limit across column families
2014-12-02 12:09:20 -08:00