andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
011241bb99
rocksdb/util
History
Igor Canadi a2bb7c3c33 Push- instead of pull-model for managing Write stalls 
Summary:
Introducing WriteController, which is a source of truth about per-DB write delays. Let's define an DB epoch as a period where there are no flushes and compactions (i.e. new epoch is started when flush or compaction finishes). Each epoch can either:
* proceed with all writes without delay
* delay all writes by fixed time
* stop all writes

The three modes are recomputed at each epoch change (flush, compaction), rather than on every write (which is currently the case).

When we have a lot of column families, our current pull behavior adds a big overhead, since we need to loop over every column family for every write. With new push model, overhead on Write code-path is minimal.

This is just the start. Next step is to also take care of stalls introduced by slow memtable flushes. The final goal is to eliminate function MakeRoomForWrite(), which currently needs to be called for every column family by every write.

Test Plan: make check for now. I'll add some unit tests later. Also, perf test.

Reviewers: dhruba, yhchiang, MarkCallaghan, sdong, ljin

Reviewed By: ljin

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D22791
2014-09-08 11:20:25 -07:00
..
arena_test.cc Arena to inline 2KB of data in it. 2014-05-14 11:49:01 -07:00
arena.cc Fix typo huage => huge 2014-08-14 17:01:20 -07:00
arena.h In DB::NewIterator(), try to allocate the whole iterator tree in an arena 2014-06-02 17:44:57 -07:00
auto_roll_logger_test.cc attempt to fix auto_roll_logger_test 2014-08-18 10:23:18 -07:00
auto_roll_logger.cc Fix a bug causing LOG is not created when max_log_file_size is set. 2014-06-16 10:27:42 -07:00
auto_roll_logger.h Renamed InfoLogLevel::DEBUG to InfoLogLevel::DEBUG_LEVEL 2014-04-10 15:27:42 -07:00
autovector_test.cc autovector::resize 2014-05-08 13:50:49 -07:00
autovector.h Fix autovector iterator increment/decrement comments 2014-08-14 14:56:11 -07:00
benchharness_test.cc Fix unsigned and signed integer compare 2014-04-29 12:29:22 -07:00
benchharness.cc Fix more signed/unsigned comparsions 2014-04-29 12:40:18 -07:00
benchharness.h log_and_apply_bench on a new benchmark framework 2014-05-05 11:11:48 -07:00
blob_store_test.cc Move the compiler back to 4.8.1 + more small fixes 2013-11-18 11:40:16 -08:00
blob_store.cc RocksDBLite 2014-04-15 13:39:26 -07:00
blob_store.h RocksDBLite 2014-04-15 13:39:26 -07:00
bloom_test.cc Implement full filter for block based table. 2014-09-08 10:37:05 -07:00
bloom.cc Implement full filter for block based table. 2014-09-08 10:37:05 -07:00
build_version.h Dont build version if compiling for IOS 2014-04-11 10:54:47 -07:00
cache_bench.cc add cache bench 2014-09-05 15:55:43 -07:00
cache_test.cc ApplyToAllCacheEntries 2014-05-02 16:24:04 -04:00
cache.cc ApplyToAllCacheEntries 2014-05-02 16:24:04 -04:00
coding_test.cc Remove BitStream* tests 2014-08-19 09:52:54 -04:00
coding.cc Removing BitStream* functions 2014-08-19 06:48:21 -07:00
coding.h Removing BitStream* functions 2014-08-19 06:48:21 -07:00
comparator.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
crc32c_test.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
crc32c.cc Dynamically choose SSE 4.2 2014-04-04 14:03:19 -07:00
crc32c.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
db_info_dummper.cc fix more compile warnings 2014-09-05 14:14:37 +08:00
dynamic_bloom_test.cc fix more compile warnings 2014-09-05 14:14:37 +08:00
dynamic_bloom.cc Changes to support unity build: 2014-08-11 13:22:47 -04:00
dynamic_bloom.h Adding option to save PlainTable index and bloom filter in SST file. 2014-07-18 16:58:13 -07:00
env_hdfs.cc cleanup exception text 2014-05-21 07:54:22 -04:00
env_posix.cc limit max bytes that can be read/written per pread/write syscall 2014-08-29 21:21:49 -07:00
env_test.cc Fix travis builds 2014-09-04 10:23:45 -07:00
env.cc integrate rate limiter into rocksdb 2014-07-08 12:31:49 -07:00
filelock_test.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
filter_policy.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
hash_cuckoo_rep.cc sizeof(int*) where sizeof(int) was intended 2014-09-05 20:50:29 -07:00
hash_cuckoo_rep.h Pass logger to memtable rep and TLB page allocation error logged to info logs 2014-05-05 16:43:37 -07:00
hash_linklist_rep.cc HashLinkList memtable switches a bucket to a skip list to reduce performance outliers 2014-07-01 17:14:15 -07:00
hash_linklist_rep.h HashLinkList memtable switches a bucket to a skip list to reduce performance outliers 2014-07-01 17:14:15 -07:00
hash_skiplist_rep.cc use arena to allocate memtable's bloomfilter and hashskiplist's buckets_ 2014-06-30 15:54:31 -07:00
hash_skiplist_rep.h Pass logger to memtable rep and TLB page allocation error logged to info logs 2014-05-05 16:43:37 -07:00
hash.cc Add a hash-index component for block 2014-03-03 21:11:49 -08:00
hash.h Changes to support unity build: 2014-08-11 13:22:47 -04:00
histogram_test.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
histogram.cc fix a few compile warnings 2014-09-04 23:06:23 +08:00
histogram.h fix a few compile warnings 2014-09-04 23:06:23 +08:00
iostats_context_imp.h Fix ios compile 2014-08-28 12:46:05 -04:00
iostats_context.cc Fix ios compile 2014-08-28 12:46:05 -04:00
ldb_cmd_execute_result.h Revert "Don't compile ldb tool into static library" 2014-04-15 11:29:02 -07:00
ldb_cmd.cc Push- instead of pull-model for managing Write stalls 2014-09-08 11:20:25 -07:00
ldb_cmd.h Move include/utilities/*.h to include/rocksdb/utilities/*.h 2014-07-23 10:21:38 -04:00
ldb_tool.cc [RocksDB] allow LDB tool to have customized key formatter 2014-06-23 15:35:40 -07:00
log_buffer.cc Fix data corruption by LogBuffer 2014-03-21 15:32:48 -07:00
log_buffer.h Fix errors while building with clang 2014-05-15 12:34:53 +08:00
log_write_bench.cc Fix more gflag namespace issues 2014-05-09 08:41:02 -07:00
logging.cc fix more compile warnings 2014-09-05 14:14:37 +08:00
logging.h Clean up compaction logging 2014-05-14 12:13:50 -07:00
manual_compaction_test.cc Fix CompactRange to apply filter to every key 2014-01-14 16:19:09 -08:00
murmurhash.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
murmurhash.h Add a hash-index component for block 2014-03-03 21:11:49 -08:00
mutexlock.h Add separate Read/WriteUnlock methods in MutexRW. 2014-06-16 15:41:46 -07:00
options_builder.cc Changes to support unity build: 2014-08-11 13:22:47 -04:00
options_test.cc fix more compile warnings 2014-09-05 14:14:37 +08:00
options.cc fix more compile warnings 2014-09-05 14:14:37 +08:00
perf_context_imp.h Refactor PerfStepTimer to stop on destruct 2014-09-02 12:04:22 -07:00
perf_context.cc Add a function to return current perf level 2014-07-10 11:35:48 -07:00
posix_logger.h Renamed InfoLogLevel::DEBUG to InfoLogLevel::DEBUG_LEVEL 2014-04-10 15:27:42 -07:00
random.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
rate_limiter_test.cc fix more compile warnings 2014-09-05 14:14:37 +08:00
rate_limiter.cc expose RateLimiter definition 2014-07-25 15:17:06 -07:00
rate_limiter.h expose RateLimiter definition 2014-07-25 15:17:06 -07:00
scoped_arena_iterator.h Remove path with arena==nullptr from NewInternalIterator 2014-09-04 17:40:41 -07:00
signal_test.cc Print out stack trace in mac, too 2014-04-23 09:11:35 -04:00
skiplistrep.cc Remove MemTableRep::GetIterator(const Slice& slice) 2014-06-25 14:09:29 -07:00
slice.cc FixedPrefixTransform to include prefix length in its name 2014-05-13 16:08:21 -07:00
statistics.cc fix more compile warnings 2014-09-05 14:14:37 +08:00
statistics.h make statistics forward-able 2014-07-28 12:10:49 -07:00
status.cc Add timeout_hint_us to WriteOptions and introduce Status::TimeOut. 2014-07-03 15:47:02 -07:00
stl_wrappers.h Killing Transform Rep 2013-12-03 12:42:15 -08:00
stop_watch.h Change StopWatch interface 2014-07-28 12:22:37 -07:00
string_util.cc Turn on -Wmissing-prototypes 2014-04-09 21:17:14 -07:00
string_util.h Turn on -Wmissing-prototypes 2014-04-09 21:17:14 -07:00
sync_point.cc Don't compile sync_point if NDEBUG 2014-04-17 10:49:58 -07:00
sync_point.h Fix ifdef NDEBUG 2014-04-17 14:29:28 -07:00
testharness.cc Better stack trace in MAC 2014-04-25 09:50:51 -04:00
testharness.h Print out stack trace in mac, too 2014-04-23 09:11:35 -04:00
testutil.cc Adding a user comparator for comparing Uint64 slices. 2014-08-27 10:39:31 -07:00
testutil.h Adding a user comparator for comparing Uint64 slices. 2014-08-27 10:39:31 -07:00
thread_local_test.cc simplify ThreadLocalPtr a little bit 2014-04-22 21:13:34 -07:00
thread_local.cc simplify ThreadLocalPtr a little bit 2014-04-22 21:13:34 -07:00
thread_local.h simplify ThreadLocalPtr a little bit 2014-04-22 21:13:34 -07:00
vectorrep.cc Remove assert in vector rep 2014-08-27 11:05:41 -07:00
xxhash.cc xxHash 2014-05-01 14:09:32 -04:00
xxhash.h xxHash 2014-05-01 14:09:32 -04:00