b6655a679d
Summary: While profiling compaction in our service I noticed a lot of CPU (~15% of compaction) being spent in MergingIterator and key comparison. Looking at the code I found MergingIterator was (understandably) using std::priority_queue for the multiway merge. Keys in our dataset include sequence numbers that increase with time. Adjacent keys in an L0 file are very likely to be adjacent in the full database. Consequently, compaction will often pick a chunk of rows from the same L0 file before switching to another one. It would be great to avoid the O(log K) operation per row while compacting. This diff replaces std::priority_queue with a custom binary heap implementation. It has a "replace top" operation that is cheap when the new top is the same as the old one (i.e. the priority of the top entry is decreased but it still stays on top). Test Plan: make check To test the effect on performance, I generated databases with data patterns that mimic what I describe in the summary (rows have a mostly increasing sequence number). I see a 10-15% CPU decrease for compaction (and a matching throughput improvement on tmpfs). The exact improvement depends on the number of L0 files and the amount of locality. Performance on randomly distributed keys seems on par with the old code. Reviewers: kailiu, sdong, igor Reviewed By: igor Subscribers: yoshinorim, dhruba, tnovak Differential Revision: https://reviews.facebook.net/D29133
43 lines
1.2 KiB
C++
43 lines
1.2 KiB
C++
// Copyright (c) 2013, Facebook, Inc. All rights reserved.
|
|
// This source code is licensed under the BSD-style license found in the
|
|
// LICENSE file in the root directory of this source tree. An additional grant
|
|
// of patent rights can be found in the PATENTS file in the same directory.
|
|
//
|
|
|
|
#pragma once
|
|
|
|
#include "rocksdb/comparator.h"
|
|
#include "table/iterator_wrapper.h"
|
|
|
|
namespace rocksdb {
|
|
|
|
// When used with std::priority_queue, this comparison functor puts the
|
|
// iterator with the max/largest key on top.
|
|
class MaxIteratorComparator {
|
|
public:
|
|
MaxIteratorComparator(const Comparator* comparator) :
|
|
comparator_(comparator) {}
|
|
|
|
bool operator()(IteratorWrapper* a, IteratorWrapper* b) const {
|
|
return comparator_->Compare(a->key(), b->key()) < 0;
|
|
}
|
|
private:
|
|
const Comparator* comparator_;
|
|
};
|
|
|
|
// When used with std::priority_queue, this comparison functor puts the
|
|
// iterator with the min/smallest key on top.
|
|
class MinIteratorComparator {
|
|
public:
|
|
MinIteratorComparator(const Comparator* comparator) :
|
|
comparator_(comparator) {}
|
|
|
|
bool operator()(IteratorWrapper* a, IteratorWrapper* b) const {
|
|
return comparator_->Compare(a->key(), b->key()) > 0;
|
|
}
|
|
private:
|
|
const Comparator* comparator_;
|
|
};
|
|
|
|
} // namespace rocksdb
|