e817bc9628
Summary: **Summary**: 2 new statistics counters are added to RocksDB: `MEMTABLE_PAYLOAD_BYTES_AT_FLUSH` and `MEMTABLE_GARBAGE_BYTES_AT_FLUSH`. The former tracks how many raw bytes of useful data are present on the memtable at flush time, whereas the latter is tracks how many of these raw bytes are considered garbage, meaning that they ended up not being imported on the SSTables resulting from the flush operations. **Unit test**: run `make db_flush_test -j$(nproc); ./db_flush_test` to run the unit test. This executable includes 3 tests, that test support and correct stat calculations for workloads with inserts, deletes, and DeleteRanges. The parameters are set such that the workloads are performed on a single memtable, and a single SSTable is created as a result of the flush operation. The flush operation is manually called in the test file. The tests verify that the values of these 2 statistics counters introduced in this PR can be exactly predicted, showing that we have a full understanding of the underlying operations. **Performance testing**: `./db_bench -statistics -benchmarks=fillrandom -num=10000000` repeated 10 times. Timing done using "date" function in a bash script. _Results_: Original Rocksdb fork: mean 66.6 sec, std 1.18 sec. This feature branch: mean 67.4 sec, std 1.35 sec. Pull Request resolved: https://github.com/facebook/rocksdb/pull/8411 Reviewed By: akankshamahajan15 Differential Revision: D29150629 Pulled By: bjlemaire fbshipit-source-id: 7b3c2e86d50c6aa34fa50fd134282eacb543a5b1
74 lines
3.0 KiB
C++
74 lines
3.0 KiB
C++
// Copyright (c) 2011-present, Facebook, Inc. All rights reserved.
|
|
// This source code is licensed under both the GPLv2 (found in the
|
|
// COPYING file in the root directory) and Apache 2.0 License
|
|
// (found in the LICENSE.Apache file in the root directory).
|
|
// Copyright (c) 2011 The LevelDB Authors. All rights reserved.
|
|
// Use of this source code is governed by a BSD-style license that can be
|
|
// found in the LICENSE file. See the AUTHORS file for names of contributors.
|
|
#pragma once
|
|
#include <string>
|
|
#include <utility>
|
|
#include <vector>
|
|
#include "db/range_tombstone_fragmenter.h"
|
|
#include "db/table_properties_collector.h"
|
|
#include "logging/event_logger.h"
|
|
#include "options/cf_options.h"
|
|
#include "rocksdb/comparator.h"
|
|
#include "rocksdb/env.h"
|
|
#include "rocksdb/listener.h"
|
|
#include "rocksdb/options.h"
|
|
#include "rocksdb/status.h"
|
|
#include "rocksdb/table_properties.h"
|
|
#include "rocksdb/types.h"
|
|
#include "table/scoped_arena_iterator.h"
|
|
|
|
namespace ROCKSDB_NAMESPACE {
|
|
|
|
struct FileMetaData;
|
|
|
|
class VersionSet;
|
|
class BlobFileAddition;
|
|
class SnapshotChecker;
|
|
class TableCache;
|
|
class TableBuilder;
|
|
class WritableFileWriter;
|
|
class InternalStats;
|
|
class BlobFileCompletionCallback;
|
|
|
|
// Convenience function for NewTableBuilder on the embedded table_factory.
|
|
TableBuilder* NewTableBuilder(const TableBuilderOptions& tboptions,
|
|
WritableFileWriter* file);
|
|
|
|
// Build a Table file from the contents of *iter. The generated file
|
|
// will be named according to number specified in meta. On success, the rest of
|
|
// *meta will be filled with metadata about the generated table.
|
|
// If no data is present in *iter, meta->file_size will be set to
|
|
// zero, and no Table file will be produced.
|
|
//
|
|
// @param column_family_name Name of the column family that is also identified
|
|
// by column_family_id, or empty string if unknown.
|
|
extern Status BuildTable(
|
|
const std::string& dbname, VersionSet* versions,
|
|
const ImmutableDBOptions& db_options, const TableBuilderOptions& tboptions,
|
|
const FileOptions& file_options, TableCache* table_cache,
|
|
InternalIterator* iter,
|
|
std::vector<std::unique_ptr<FragmentedRangeTombstoneIterator>>
|
|
range_del_iters,
|
|
FileMetaData* meta, std::vector<BlobFileAddition>* blob_file_additions,
|
|
std::vector<SequenceNumber> snapshots,
|
|
SequenceNumber earliest_write_conflict_snapshot,
|
|
SnapshotChecker* snapshot_checker, bool paranoid_file_checks,
|
|
InternalStats* internal_stats, IOStatus* io_status,
|
|
const std::shared_ptr<IOTracer>& io_tracer,
|
|
EventLogger* event_logger = nullptr, int job_id = 0,
|
|
const Env::IOPriority io_priority = Env::IO_HIGH,
|
|
TableProperties* table_properties = nullptr,
|
|
Env::WriteLifeTimeHint write_hint = Env::WLTH_NOT_SET,
|
|
const std::string* full_history_ts_low = nullptr,
|
|
BlobFileCompletionCallback* blob_callback = nullptr,
|
|
uint64_t* num_input_entries = nullptr,
|
|
uint64_t* memtable_payload_bytes = nullptr,
|
|
uint64_t* memtable_garbage_bytes = nullptr);
|
|
|
|
} // namespace ROCKSDB_NAMESPACE
|