andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Disable onboard cache for compaction output

Browse Source 
Summary:
FILE_FLAG_WRITE_THROUGH is for disabling device on-board cache in windows API, which should be disabled if user doesn't need system cache.
There was a perf issue related with this, we found during memtable flush, the high percentile latency jumps significantly. During profiling, we found those high latency (P99.9) read requests got queue-jumped by write requests from memtable flush and takes 80ms or even more time to wait, even when SSD overall IO throughput is relatively low.

After enabling FILE_FLAG_WRITE_THROUGH, we rerun the test found high percentile latency drops a lot without observable impact on writes.

Scenario 1: 40MB/s + 40MB/s  R/W compaction throughput

 Original | FILE_FLAG_WRITE_THROUGH | Percentage reduction
---------------------------------------------------------------
P99.9 | 56.897 ms | 35.593 ms | -37.4%
P99 | 3.905 ms | 3.896 ms | -2.8%

Scenario 2:  14MB/s + 14MB/s R/W compaction throughput, cohosted with 100+ other rocksdb instances have manually triggered memtable flush operations (memtable is tiny), creating a lot of randomized the small file writes operations during test.

Original | FILE_FLAG_WRITE_THROUGH | Percentage reduction
---------------------------------------------------------------
P99.9 | 86.227   ms | 50.436 ms | -41.5%
P99 | 8.415   ms | 3.356 ms | -60.1%
Closes https://github.com/facebook/rocksdb/pull/3225

Differential Revision: D6624174

Pulled By: miasantreble

fbshipit-source-id: 321b86aee9d74470840c70e5d0d4fa9880660a91
This commit is contained in:
burtonli 2017-12-21 18:37:27 -08:00 committed by Facebook Github Bot
parent f00e176c5b
commit b5c99cc908
3 changed files with 9 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
port/win/env_win.cc
View File

@ -253,7 +253,7 @@ Status WinEnvIO::OpenWritableFile(const std::string& fname,
DWORD fileFlags = FILE_ATTRIBUTE_NORMAL;
if (local_options.use_direct_writes && !local_options.use_mmap_writes) {
fileFlags = FILE_FLAG_NO_BUFFERING;
fileFlags = FILE_FLAG_NO_BUFFERING | FILE_FLAG_WRITE_THROUGH;
}
// Desired access. We are want to write only here but if we want to memory

8
port/win/io_win.cc
View File

@ -880,7 +880,7 @@ inline
Status WinWritableImpl::SyncImpl() {
Status s;
// Calls flush buffers
if (fsync(file_data_->GetFileHandle()) < 0) {
if (!file_data_->use_direct_io() && fsync(file_data_->GetFileHandle()) < 0) {
auto lastError = GetLastError();
s = IOErrorFromWindowsError(
"fsync failed at Sync() for: " + file_data_->GetName(), lastError);
@ -961,7 +961,11 @@ Status WinWritableFile::Sync() {
return SyncImpl();
}
Status WinWritableFile::Fsync() { return SyncImpl(); }
Status WinWritableFile::Fsync() {
return SyncImpl();
}
bool WinWritableFile::IsSyncThreadSafe() const { return true; }
uint64_t WinWritableFile::GetFileSize() {
return GetFileNextWriteOffset();

2
port/win/io_win.h
View File

@ -368,6 +368,8 @@ class WinWritableFile : private WinFileData,
virtual Status Fsync() override;
virtual bool IsSyncThreadSafe() const override;
// Indicates if the class makes use of direct I/O
// Use PositionedAppend
virtual bool use_direct_io() const override;