andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
cd7c4143d7
rocksdb/util
History
Siying Dong cd7c4143d7 Improve Write Stalling System 
Summary:
Current write stalling system has the problem of lacking of positive feedback if the restricted rate is already too low. Users sometimes stack in very low slowdown value. With the diff, we add a positive feedback (increasing the slowdown value) if we recover from slowdown state back to normal. To avoid the positive feedback to keep the slowdown value to be to high, we add issue a negative feedback every time we are close to the stop condition. Experiments show it is easier to reach a relative balance than before.

Also increase level0_stop_writes_trigger default from 24 to 32. Since level0_slowdown_writes_trigger default is 20, stop trigger 24 only gives four files as the buffer time to slowdown writes. In order to avoid stop in four files while 20 files have been accumulated, the slowdown value must be very low, which is amost the same as stop. It also doesn't give enough time for the slowdown value to converge. Increase it to 32 will smooth out the system.
Closes https://github.com/facebook/rocksdb/pull/1562

Differential Revision: D4218519

Pulled By: siying

fbshipit-source-id: 95e4088
2016-11-23 09:24:15 -08:00
..
aligned_buffer.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
allocator.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
arena_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
arena.cc FreeBSD: malloc_usable_size is in <malloc_np.h> (#1428) 2016-10-28 10:44:52 -07:00
arena.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
autovector_test.cc Eliminate use of 'using namespace std'. Also remove a number of ADL references to std functions. 2016-05-20 07:42:18 -07:00
autovector.h Report memory usage by memtable insert hints map. 2016-11-15 20:24:13 -08:00
bloom_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
bloom.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
build_version.cc.in Makefile: generate util/build_version.cc from .in file (#1384) 2016-10-25 11:31:39 -07:00
build_version.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
cache_bench.cc Introduce ClockCache 2016-08-19 12:28:19 -07:00
cache_test.cc Fix cache_test valgrind_check failure 2016-08-29 10:40:00 -07:00
cf_options.cc Range deletion microoptimizations 2016-11-21 12:24:13 -08:00
cf_options.h Range deletion microoptimizations 2016-11-21 12:24:13 -08:00
channel.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
clock_cache.cc Fix ClockCache memory leak 2016-08-31 08:56:34 -07:00
clock_cache.h Introduce ClockCache 2016-08-19 12:28:19 -07:00
coding_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
coding.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
coding.h TableBuilder / TableReader support for range deletion 2016-08-19 15:10:31 -07:00
compaction_job_stats_impl.cc Single Delete Mismatch and Fallthrough statistics 2016-08-16 08:21:43 -07:00
comparator.cc Improve BytewiseComparatorImpl::FindShortestSeparator 2016-04-25 23:02:14 -07:00
compression.h Remove use of deprecated LZ4 function 2016-11-21 12:24:14 -08:00
concurrent_arena.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
concurrent_arena.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
crc32c_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
crc32c.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
crc32c.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
db_options.cc Dynamic max_total_wal_size option 2016-11-14 22:54:17 -08:00
db_options.h Dynamic max_total_wal_size option 2016-11-14 22:54:17 -08:00
delete_scheduler_test.cc Support SST files with Global sequence numbers [reland] 2016-10-18 16:59:37 -07:00
delete_scheduler.cc DBSSTTest.RateLimitedDelete: not to use real clock 2016-10-24 10:35:00 -07:00
delete_scheduler.h DBSSTTest.RateLimitedDelete: not to use real clock 2016-10-24 10:35:00 -07:00
dynamic_bloom_test.cc Fix clang analyzer errors 2016-07-08 17:50:51 -07:00
dynamic_bloom.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
dynamic_bloom.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
env_basic_test.cc TableBuilder / TableReader support for range deletion 2016-08-19 15:10:31 -07:00
env_chroot.cc Support POSIX RandomRWFile 2016-09-13 12:08:22 -07:00
env_chroot.h Introduce chroot Env 2016-05-06 17:42:50 -07:00
env_hdfs.cc Add NoSpace subcode to IOError (#1320) 2016-09-07 12:37:45 -07:00
env_posix.cc Fix fd leak when using direct IOs 2016-11-21 12:24:13 -08:00
env_test.cc Implement PositionedAppend for PosixWritableFile 2016-11-18 17:24:13 -08:00
env.cc Add handy option to turn on direct I/O in db_bench (#1424) 2016-10-28 10:36:05 -07:00
event_logger_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
event_logger.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
event_logger.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
fault_injection_test_env.cc [rocksdb] Recovery path sequence miscount fix 2016-05-10 14:06:07 -07:00
fault_injection_test_env.h [rocksdb] Recovery path sequence miscount fix 2016-05-10 14:06:07 -07:00
file_reader_writer_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
file_reader_writer.cc Cleanup unused variable pending_fsync_. 2016-08-05 10:31:41 -07:00
file_reader_writer.h Cleanup unused variable pending_fsync_. 2016-08-05 10:31:41 -07:00
file_util.cc Split DBOptions into ImmutableDBOptions and MutableDBOptions 2016-09-23 16:34:04 -07:00
file_util.h Split DBOptions into ImmutableDBOptions and MutableDBOptions 2016-09-23 16:34:04 -07:00
filelock_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
filter_policy.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
hash_map.h Use more efficient hash map for deadlock detection 2016-11-19 11:39:15 -08:00
hash.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
hash.h Optimize sequential insert into memtable - Part 1: Interface 2016-11-13 19:09:18 -08:00
heap_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
heap.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
histogram_test.cc Fix Build Error 2016-03-11 22:56:25 -08:00
histogram_windowing.cc Fix in HistogramWindowingImpl 2016-03-17 14:28:41 -07:00
histogram_windowing.h Histogram Concurrency Improvement and Time-Windowing Support 2016-03-11 16:54:25 -08:00
histogram.cc Fix FB internal CI build failure 2016-03-15 11:38:15 -07:00
histogram.h Histogram Concurrency Improvement and Time-Windowing Support 2016-03-11 16:54:25 -08:00
instrumented_mutex.cc DBSSTTest.RateLimitedDelete: not to use real clock 2016-10-24 10:35:00 -07:00
instrumented_mutex.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
io_posix.cc Direct I/O Reads Handle the last sector correctly. 2016-11-18 19:24:13 -08:00
io_posix.h Implement PositionedAppend for PosixWritableFile 2016-11-18 17:24:13 -08:00
iostats_context_imp.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
iostats_context_test.cc IOStatsContext::ToString() add option to exclude zero counters 2016-02-23 10:26:24 -08:00
iostats_context.cc IOStatsContext::ToString() add option to exclude zero counters 2016-02-23 10:26:24 -08:00
kv_map.h Store internal keys in TombstoneMap 2016-11-09 15:09:18 -08:00
log_buffer.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
log_buffer.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
log_write_bench.cc Fix log_write_bench -bytes_per_sync option. (#1375) 2016-10-11 16:45:51 -07:00
logging.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
logging.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
lru_cache_test.cc LRU cache mid-point insertion 2016-08-19 16:43:31 -07:00
lru_cache.cc Not insert into block cache if cache is full and not holding handle 2016-08-23 13:53:49 -07:00
lru_cache.h Option to cache index/filter blocks with priority 2016-08-23 13:44:13 -07:00
memenv.cc In-memory environment read beyond EOF 2016-05-27 12:10:26 -07:00
memory_usage.h Report memory usage by memtable insert hints map. 2016-11-15 20:24:13 -08:00
mock_env_test.cc Create env_basic_test [pluggable Env part 2] 2016-06-03 15:13:03 -07:00
mock_env.cc In-memory environment read beyond EOF 2016-05-27 12:10:26 -07:00
mock_env.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
murmurhash.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
murmurhash.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
mutexlock.h Persistent Read Cache (6) Persistent cache tier implentation - File layout 2016-07-19 12:01:46 -07:00
options_helper.cc Dynamic max_total_wal_size option 2016-11-14 22:54:17 -08:00
options_helper.h Dynamic max_total_wal_size option 2016-11-14 22:54:17 -08:00
options_parser.cc Remove function local statics that interfere with memory pooling (#1392) 2016-10-14 13:09:18 -07:00
options_parser.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
options_sanity_check.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
options_sanity_check.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
options_settable_test.cc Optimize sequential insert into memtable - Part 1: Interface 2016-11-13 19:09:18 -08:00
options_test.cc Change max_bytes_for_level_multiplier to double 2016-11-01 21:09:23 -07:00
options.cc Improve Write Stalling System 2016-11-23 09:24:15 -08:00
perf_context_imp.h fix ios build error 2016-02-17 20:22:40 +08:00
perf_context.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
perf_level_imp.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
perf_level.cc Assert boundary checks for SetPerfLevel() 2016-06-01 09:07:09 -07:00
perf_step_timer.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
posix_logger.h Show More DB Stats in info logs 2016-10-29 16:09:18 -07:00
random.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
random.h Fixes warnings and ensure correct int behavior on 32-bit platforms. 2016-03-16 22:57:57 +01:00
rate_limiter_test.cc Relax the acceptable bias RateLimiterTest::Rate test be 25% 2016-10-13 14:26:12 -07:00
rate_limiter.cc fix rate limiter to avoid starvation 2016-07-01 00:16:29 -07:00
rate_limiter.h Handle overflow case of rate limiter's paramters 2016-05-27 16:15:28 -07:00
sharded_cache.cc add stats to Cache::LookUp() 2016-09-01 13:50:39 -07:00
sharded_cache.h add stats to Cache::LookUp() 2016-09-01 13:50:39 -07:00
slice_transform_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
slice.cc Merge db/slice.cc into util/slice.cc 2016-06-10 16:37:36 -07:00
sst_file_manager_impl.cc Fix SstFileManager uninitialized data member 2016-02-18 11:25:19 -08:00
sst_file_manager_impl.h Introduce SstFileManager::SetMaxAllowedSpaceUsage() to cap disk space usage 2016-02-17 15:20:23 -08:00
statistics_test.cc Fix Statistics TickersNameMap miss match with Tickers enum 2016-07-25 16:05:50 -07:00
statistics.cc Fix deadlock when calling getMergedHistogram 2016-11-20 18:24:12 -08:00
statistics.h Add Statistics::getAndResetTickerCount(). 2016-10-11 10:54:11 -07:00
status_message.cc Add NoSpace subcode to IOError (#1320) 2016-09-07 12:37:45 -07:00
status.cc Add NoSpace subcode to IOError (#1320) 2016-09-07 12:37:45 -07:00
stderr_logger.h Stderr info logger 2016-04-01 11:06:06 -07:00
stop_watch.h Use StopWatch to do statistic job in db_impl_add_file.cc 2016-08-02 14:53:29 -07:00
string_util.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
string_util.h FreeBSD does not have std::to_string (#1190) 2016-06-29 07:35:17 -07:00
sync_point.cc Make SyncPoint return immediately when disabled 2016-08-16 06:19:46 -07:00
sync_point.h Add a new feature to enforce a sync point only active on a thread 2016-07-07 11:29:14 -07:00
testharness.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
testharness.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
testutil.cc Add avoid_flush_during_shutdown DB option 2016-11-02 15:39:18 -07:00
testutil.h Support SST files with Global sequence numbers [reland] 2016-10-18 16:59:37 -07:00
thread_list_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
thread_local_test.cc Fix lambda expression for clang/windows 2016-08-23 13:34:56 -07:00
thread_local.cc Update documentation to point at gcc 4.8 2016-10-29 12:09:17 -07:00
thread_local.h Fold function for thread-local data 2016-08-22 15:37:39 -07:00
thread_operation.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
thread_status_impl.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
thread_status_updater_debug.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
thread_status_updater.cc Use pure if-then check instead of assert in EraseColumnFamilyInfo 2016-03-04 16:03:31 -08:00
thread_status_updater.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
thread_status_util_debug.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
thread_status_util.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
thread_status_util.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
threadpool_imp.cc Fix Windows Build 2016-09-02 17:10:28 -07:00
threadpool_imp.h Expose ThreadPool under include/rocksdb/threadpool.h 2016-08-26 10:41:35 -07:00
transaction_test_util.cc Add multithreaded transaction test 2016-03-11 15:16:52 -08:00
transaction_test_util.h Fix AppVeyor build error 2016-03-15 10:57:33 -07:00
xfunc.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
xfunc.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
xxhash.cc Enable MS Warning C4804 : unsafe use of type 'bool' in operation 2015-11-18 16:23:19 -08:00
xxhash.h Prevent xxhash symbols from polluting global namespace 2015-03-12 12:07:10 -07:00