andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Dedicate cacheline for DB mutex (#9637)

Browse Source 
Summary:
We found a case of cacheline bouncing due to writers locking/unlocking `mutex_` and readers accessing `block_cache_tracer_`. We discovered it only after the issue was fixed by https://github.com/facebook/rocksdb/issues/9462 shifting the `DBImpl` members such that `mutex_` and `block_cache_tracer_` were naturally placed in separate cachelines in our regression testing setup. This PR forces the cacheline alignment of `mutex_` so we don't accidentally reintroduce the problem.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/9637

Reviewed By: riversand963

Differential Revision: D34502233

Pulled By: ajkr

fbshipit-source-id: 46aa313b7fe83e80c3de254e332b6fb242434c07
This commit is contained in:
Andrew Kryczka 2022-02-27 11:36:54 -08:00 committed by Facebook GitHub Bot
parent 9ed96703d1
commit 9983eecdfb
2 changed files with 12 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
db/db_impl/db_impl.h
View File

@ -1187,7 +1187,10 @@ class DBImpl : public DB {
// WriteToWAL need different synchronization: log_empty_, alive_log_files_, // WriteToWAL need different synchronization: log_empty_, alive_log_files_,
// logs_, logfile_number_. Refer to the definition of each variable below for // logs_, logfile_number_. Refer to the definition of each variable below for
// more description. // more description.
mutable InstrumentedMutex mutex_; //
// `mutex_` can be a hot lock in some workloads, so it deserves dedicated
// cachelines.
mutable CacheAlignedInstrumentedMutex mutex_;
ColumnFamilyHandleImpl* default_cf_handle_; ColumnFamilyHandleImpl* default_cf_handle_;
InternalStats* default_cf_internal_stats_; InternalStats* default_cf_internal_stats_;

8
monitoring/instrumented_mutex.h
View File

@ -51,6 +51,14 @@ class InstrumentedMutex {
int stats_code_; int stats_code_;
}; };
class ALIGN_AS(CACHE_LINE_SIZE) CacheAlignedInstrumentedMutex
: public InstrumentedMutex {
using InstrumentedMutex::InstrumentedMutex;
char padding[(CACHE_LINE_SIZE - sizeof(InstrumentedMutex) % CACHE_LINE_SIZE) %
CACHE_LINE_SIZE] ROCKSDB_FIELD_UNUSED;
};
static_assert(sizeof(CacheAlignedInstrumentedMutex) % CACHE_LINE_SIZE == 0);
// RAII wrapper for InstrumentedMutex // RAII wrapper for InstrumentedMutex
class InstrumentedMutexLock { class InstrumentedMutexLock {
public: public: