andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
bd45633b71
rocksdb/util
History
sdong bd45633b71 Fix data race against logging data structure because of LogBuffer 
Summary:
@igor pointed out that there is a potential data race because of the way we use the newly introduced LogBuffer. After "bg_compaction_scheduled_--" or "bg_flush_scheduled_--", they can both become 0. As soon as the lock is released after that, DBImpl's deconstructor can go ahead and deconstruct all the states inside DB, including the info_log object hold in a shared pointer of the options object it keeps. At that point it is not safe anymore to continue using the info logger to write the delayed logs.

With the patch, lock is released temporarily for log buffer to be flushed before "bg_compaction_scheduled_--" or "bg_flush_scheduled_--". In order to make sure we don't miss any pending flush or compaction, a new flag bg_schedule_needed_ is added, which is set to be true if there is a pending flush or compaction but not scheduled because of the max thread limit. If the flag is set to be true, the scheduling function will be called before compaction or flush thread finishes.

Thanks @igor for this finding!

Test Plan: make all check

Reviewers: haobo, igor

Reviewed By: haobo

CC: dhruba, ljin, yhchiang, igor, leveldb

Differential Revision: https://reviews.facebook.net/D16767
2014-03-11 16:09:53 -07:00
..
arena_test.cc Clean up arena API 2014-01-30 22:10:10 -08:00
arena.cc Clean up arena API 2014-01-30 22:10:10 -08:00
arena.h Add a hash-index component for block 2014-03-03 21:11:49 -08:00
auto_roll_logger_test.cc Fix some compilation bugs in different platforms 2014-02-27 22:20:17 -08:00
auto_roll_logger.cc Allow user to specify log level for info_log 2014-03-05 14:54:31 -08:00
auto_roll_logger.h Allow user to specify log level for info_log 2014-03-05 14:54:31 -08:00
autovector_test.cc Fix issue #57 2014-01-06 11:11:19 -08:00
autovector.h Replace vector with autovector 2014-01-02 16:43:35 -08:00
bit_set.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
blob_store_test.cc Move the compiler back to 4.8.1 + more small fixes 2013-11-18 11:40:16 -08:00
blob_store.cc IOError cleanup 2014-02-12 11:42:54 -08:00
blob_store.h Implement max_size in BlobStore 2013-10-23 14:38:52 -07:00
bloom_test.cc [RocksDB] [Performance Branch] Added dynamic bloom, to be used for memable non-existing key filtering 2013-12-11 00:15:14 -08:00
bloom.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
build_version.h Fixed typos 2013-11-16 11:21:34 +00:00
cache_test.cc [RocksDB] [Performance Branch] Trivia build fix 2013-12-13 14:21:59 -08:00
cache.cc Merge branch 'master' into performance 2014-01-28 10:35:55 -08:00
coding_test.cc Misc cleanup on performance branch 2014-01-17 14:26:29 -08:00
coding.cc Merge branch 'master' into performance 2014-01-28 10:35:55 -08:00
coding.h Merge branch 'master' into performance 2014-01-28 10:35:55 -08:00
comparator.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
crc32c_test.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
crc32c.cc crc32c: choose function in static initialization 2014-02-04 19:13:57 -08:00
crc32c.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
dynamic_bloom_test.cc [RocksDB] [Performance Branch] Added dynamic bloom, to be used for memable non-existing key filtering 2013-12-11 00:15:14 -08:00
dynamic_bloom.cc Optimize MayContainHash() 2014-01-14 22:03:57 -08:00
dynamic_bloom.h Misc cleanup on performance branch 2014-01-17 14:26:29 -08:00
env_hdfs.cc Enable log info with different levels. 2014-02-26 14:41:28 -08:00
env_posix.cc Env to add a function to allow users to query waiting queue length 2014-03-11 10:19:02 -07:00
env_test.cc Env to add a function to allow users to query waiting queue length 2014-03-11 10:19:02 -07:00
env.cc [RocksDB] LogBuffer Cleanup 2014-03-10 11:05:44 -07:00
filelock_test.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
filter_policy.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
hash_linklist_rep.cc Consolidate SliceTransform object ownership 2014-03-10 12:56:46 -07:00
hash_linklist_rep.h Consolidate SliceTransform object ownership 2014-03-10 12:56:46 -07:00
hash_skiplist_rep.cc Consolidate SliceTransform object ownership 2014-03-10 12:56:46 -07:00
hash_skiplist_rep.h Consolidate SliceTransform object ownership 2014-03-10 12:56:46 -07:00
hash.cc Add a hash-index component for block 2014-03-03 21:11:49 -08:00
hash.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
histogram_test.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
histogram.cc Statistics code cleanup 2014-01-17 12:46:06 -08:00
histogram.h Statistics code cleanup 2014-01-17 12:46:06 -08:00
ldb_cmd_execute_result.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
ldb_cmd.cc Support for LZ4 compression. 2014-02-08 14:15:51 -08:00
ldb_cmd.h Task #3071144 Enhance ldb (db dump tool for leveldb) to report row counters for each row type 2013-11-01 13:59:14 -07:00
ldb_tool.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
log_buffer.cc [RocksDB] LogBuffer Cleanup 2014-03-10 11:05:44 -07:00
log_buffer.h Fix data race against logging data structure because of LogBuffer 2014-03-11 16:09:53 -07:00
log_write_bench.cc A simple benchmark to measure WAL append latency 2014-02-24 14:39:32 -08:00
logging.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
logging.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
manual_compaction_test.cc Fix CompactRange to apply filter to every key 2014-01-14 16:19:09 -08:00
murmurhash.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
murmurhash.h Add a hash-index component for block 2014-03-03 21:11:49 -08:00
mutexlock.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
options.cc Add option verify_checksums_in_compaction 2014-03-10 10:06:34 -07:00
perf_context_imp.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
perf_context.cc output perf_context in db_bench readrandom 2014-03-05 10:32:54 -08:00
posix_logger.h Enable log info with different levels. 2014-02-26 14:41:28 -08:00
random.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
signal_test.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
skiplistrep.cc Consolidate SliceTransform object ownership 2014-03-10 12:56:46 -07:00
slice.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
stack_trace.h Print stack trace on assertion failure 2013-12-06 17:11:09 -08:00
statistics.cc convert Tickers back to array with padding and alignment 2014-01-29 15:08:41 -08:00
statistics.h cache SuperVersion in thread local storage to avoid mutex lock 2014-02-27 11:38:55 -08:00
stats_logger.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
status.cc IOError cleanup 2014-02-12 11:42:54 -08:00
stl_wrappers.h Killing Transform Rep 2013-12-03 12:42:15 -08:00
stop_watch.h Statistics code cleanup 2014-01-17 12:46:06 -08:00
string_util.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
string_util.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
testharness.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
testharness.h Print stack trace on assertion failure 2013-12-06 17:11:09 -08:00
testutil.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
testutil.h [Performance Branch] PlainTable to encode rows with seqID 0, value type using 1 internal byte. 2014-02-03 12:19:30 -08:00
thread_local_test.cc use CAS when returning SuperVersion to ThreadLocal 2014-03-07 14:43:22 -08:00
thread_local.cc use CAS when returning SuperVersion to ThreadLocal 2014-03-07 14:43:22 -08:00
thread_local.h use CAS when returning SuperVersion to ThreadLocal 2014-03-07 14:43:22 -08:00
vectorrep.cc Consolidate SliceTransform object ownership 2014-03-10 12:56:46 -07:00