a8a422e962
Summary: The goal of this change is to allow changes to the "current" (in FileSystem) file temperatures to feed back into DB metadata, so that they can inform decisions and stats reporting. In part because of modular code factoring, it doesn't seem easy to do this automagically, where opening an SST file and observing current Temperature different from expected would trigger a change in metadata and DB manifest write (essentially giving the deep read path access to the write path). It is also difficult to do this while the DB is open because of the limitations of LogAndApply. This change allows updating file temperature metadata on a closed DB using an experimental utility function UpdateManifestForFilesState() or `ldb update_manifest --update_temperatures`. This should suffice for "migration" scenarios where outside tooling has placed or re-arranged DB files into a (different) tiered configuration without going through RocksDB itself (currently, only compaction can change temperature metadata). Some details: * Refactored and added unit test for `ldb unsafe_remove_sst_file` because of shared functionality * Pulled in autovector.h changes from https://github.com/facebook/rocksdb/issues/9546 to fix SuperVersionContext move constructor (related to an older draft of this change) Possible follow-up work: * Support updating manifest with file checksums, such as when a new checksum function is used and want existing DB metadata updated for it. * It's possible that for some repair scenarios, lighter weight than full repair, we might want to support UpdateManifestForFilesState() to modify critical file details like size or checksum using same algorithm. But let's make sure these are differentiated from modifying file details in ways that don't suspect corruption (or require extreme trust). Pull Request resolved: https://github.com/facebook/rocksdb/pull/9683 Test Plan: unit tests added Reviewed By: jay-zhuang Differential Revision: D34798828 Pulled By: pdillinger fbshipit-source-id: cfd83e8fb10761d8c9e7f9c020d68c9106a95554
57 lines
2.4 KiB
C++
57 lines
2.4 KiB
C++
// Copyright (c) 2011-present, Facebook, Inc. All rights reserved.
|
|
// This source code is licensed under both the GPLv2 (found in the
|
|
// COPYING file in the root directory) and Apache 2.0 License
|
|
// (found in the LICENSE.Apache file in the root directory).
|
|
|
|
#pragma once
|
|
|
|
#include "rocksdb/db.h"
|
|
#include "rocksdb/status.h"
|
|
|
|
namespace ROCKSDB_NAMESPACE {
|
|
namespace experimental {
|
|
|
|
// Supported only for Leveled compaction
|
|
Status SuggestCompactRange(DB* db, ColumnFamilyHandle* column_family,
|
|
const Slice* begin, const Slice* end);
|
|
Status SuggestCompactRange(DB* db, const Slice* begin, const Slice* end);
|
|
|
|
// Move all L0 files to target_level skipping compaction.
|
|
// This operation succeeds only if the files in L0 have disjoint ranges; this
|
|
// is guaranteed to happen, for instance, if keys are inserted in sorted
|
|
// order. Furthermore, all levels between 1 and target_level must be empty.
|
|
// If any of the above condition is violated, InvalidArgument will be
|
|
// returned.
|
|
Status PromoteL0(DB* db, ColumnFamilyHandle* column_family,
|
|
int target_level = 1);
|
|
|
|
struct UpdateManifestForFilesStateOptions {
|
|
// When true, read current file temperatures from FileSystem and update in
|
|
// DB manifest when a temperature other than Unknown is reported and
|
|
// inconsistent with manifest.
|
|
bool update_temperatures = true;
|
|
|
|
// TODO: new_checksums: to update files to latest file checksum algorithm
|
|
};
|
|
|
|
// Utility for updating manifest of DB directory (not open) for current state
|
|
// of files on filesystem. See UpdateManifestForFilesStateOptions.
|
|
//
|
|
// To minimize interference with ongoing DB operations, only the following
|
|
// guarantee is provided, assuming no IO error encountered:
|
|
// * Only files live in DB at start AND end of call to
|
|
// UpdateManifestForFilesState() are guaranteed to be updated (as needed) in
|
|
// manifest.
|
|
// * For example, new files after start of call to
|
|
// UpdateManifestForFilesState() might not be updated, but that is not
|
|
// typically required to achieve goal of manifest consistency/completeness
|
|
// (because current DB configuration would ensure new files get the desired
|
|
// consistent metadata).
|
|
Status UpdateManifestForFilesState(
|
|
const DBOptions& db_opts, const std::string& db_name,
|
|
const std::vector<ColumnFamilyDescriptor>& column_families,
|
|
const UpdateManifestForFilesStateOptions& opts = {});
|
|
|
|
} // namespace experimental
|
|
} // namespace ROCKSDB_NAMESPACE
|