andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

add call to install superversion and schedule work in enableautocompactions

Browse Source 
Summary:
This patch fixes https://github.com/facebook/mysql-5.6/issues/121

There is a recent change in rocksdb to disable auto compactions on startup: https://reviews.facebook.net/D51147. However, there is a small timing window where a column family needs to be compacted and schedules a compaction, but the scheduled compaction fails when it checks the disable_auto_compactions setting. The expectation is once the application is ready, it will call EnableAutoCompactions() to allow new compactions to go through. However, if the Column family is stalled because L0 is full, and no writes can go through, it is possible the column family may never have a new compaction request get scheduled. EnableAutoCompaction() should probably schedule an new flush and compaction event when it resets disable_auto_compaction.

Using InstallSuperVersionAndScheduleWork, we call SchedulePendingFlush,
SchedulePendingCompaction, as well as MaybeScheduleFlushOrcompaction on all the
column families to avoid the situation above.

This is still a first pass for feedback.
Could also just call SchedePendingFlush and SchedulePendingCompaction directly.

Test Plan:
Run on Asan build
cd _build-5.6-ASan/ && ./mysql-test/mtr --mem --big --testcase-timeout=36000 --suite-timeout=12000 --parallel=16 --suite=rocksdb,rocksdb_rpl,rocksdb_sys_vars --mysqld=--default-storage-engine=rocksdb --mysqld=--skip-innodb --mysqld=--default-tmp-storage-engine=MyISAM --mysqld=--rocksdb rocksdb_rpl.rpl_rocksdb_stress_crash --repeat=1000

Ensure that it no longer hangs during the test.

Reviewers: hermanlee4, yhchiang, anthony

Reviewed By: anthony

Subscribers: leveldb, yhchiang, dhruba

Differential Revision: https://reviews.facebook.net/D51747
This commit is contained in:
Alex Yang 2015-12-09 00:29:35 -08:00
parent 22c6b50ee8
commit 33e09c0e19
2 changed files with 18 additions and 10 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

18
db/db_impl.cc
View File

@ -2393,15 +2393,17 @@ Status DBImpl::EnableAutoCompaction(
const std::vector<ColumnFamilyHandle*>& column_family_handles) {
Status s;
for (auto cf_ptr : column_family_handles) {
// check options here, enable only if didn't initially disable
if (s.ok()) {
s = this->SetOptions(cf_ptr, {{"disable_auto_compactions", "false"}});
}
}
if (s.ok()) {
Status status =
this->SetOptions(cf_ptr, {{"disable_auto_compactions", "false"}});
if (status.ok()) {
ColumnFamilyData* cfd =
reinterpret_cast<ColumnFamilyHandleImpl*>(cf_ptr)->cfd();
InstrumentedMutexLock guard_lock(&mutex_);
MaybeScheduleFlushOrCompaction();
delete this->InstallSuperVersionAndScheduleWork(
cfd, nullptr, *cfd->GetLatestMutableCFOptions());
} else {
s = status;
}
}
return s;

10
include/rocksdb/db.h
View File

@ -559,8 +559,14 @@ class DB {
virtual Status ContinueBackgroundWork() = 0;
// This function will enable automatic compactions for the given column
// families if they were previously disabled via the disable_auto_compactions
// option.
// families if they were previously disabled. The function will first set the
// disable_auto_compactions option for each column family to 'false', after
// which it will schedule a flush/compaction.
//
// NOTE: Setting disable_auto_compactions to 'false' through SetOptions() API
// does NOT schedule a flush/compaction afterwards, and only changes the
// parameter itself within the column family option.
//
virtual Status EnableAutoCompaction(
const std::vector<ColumnFamilyHandle*>& column_family_handles) = 0;