3ffb3baa0b
Summary: New classes FileStorageInfo and LiveFileStorageInfo and 'experimental' function DB::GetLiveFilesStorageInfo, which is intended to largely replace several fragmented DB functions needed to create checkpoints and backups. This function is now used to create checkpoints and backups, because it fixes many (probably not all) of the prior complexities of checkpoint not having atomic access to DB metadata. This also ensures strong functional test coverage of the new API. Specifically, much of the old CheckpointImpl::CreateCustomCheckpoint has been migrated to and updated in DBImpl::GetLiveFilesStorageInfo, with the former now calling the latter. Also, the class FileStorageInfo in metadata.h compatibly replaces BackupFileInfo and serves as a new base class for SstFileMetaData. Some old fields of SstFileMetaData are still provided (for now) but deprecated. Although FileStorageInfo::directory is accurate when using db_paths and/or cf_paths, these have never been supported by Checkpoint nor BackupEngine and still are not. This change does now detect these cases and return NotSupported when appropriate. (More work needed for support.) Somehow this change broke ProgressCallbackDuringBackup, but the progress_callback logic was dubious to begin with because it would call the callback based on copy buffer size, not size actually copied. Logic and test updated to track size actually copied per-thread. Pull Request resolved: https://github.com/facebook/rocksdb/pull/8968 Test Plan: tests updated. DB::GetLiveFilesStorageInfo mostly tested by use in CheckpointImpl. DBTest.SnapshotFiles updated to also test GetLiveFilesStorageInfo, including reading the data after DB close. Added CheckpointTest.CheckpointWithDbPath (NotSupported). Reviewed By: siying Differential Revision: D31242045 Pulled By: pdillinger fbshipit-source-id: b183d1ce9799e220daaefd6b3b5365d98de676c0
66 lines
2.3 KiB
C++
66 lines
2.3 KiB
C++
// Copyright (c) 2017-present, Facebook, Inc. All rights reserved.
|
|
// This source code is licensed under both the GPLv2 (found in the
|
|
// COPYING file in the root directory) and Apache 2.0 License
|
|
// (found in the LICENSE.Apache file in the root directory).
|
|
|
|
#pragma once
|
|
#ifndef ROCKSDB_LITE
|
|
|
|
#include "rocksdb/utilities/checkpoint.h"
|
|
|
|
#include <string>
|
|
#include "file/filename.h"
|
|
#include "rocksdb/db.h"
|
|
|
|
namespace ROCKSDB_NAMESPACE {
|
|
|
|
class CheckpointImpl : public Checkpoint {
|
|
public:
|
|
explicit CheckpointImpl(DB* db) : db_(db) {}
|
|
|
|
Status CreateCheckpoint(const std::string& checkpoint_dir,
|
|
uint64_t log_size_for_flush,
|
|
uint64_t* sequence_number_ptr) override;
|
|
|
|
Status ExportColumnFamily(ColumnFamilyHandle* handle,
|
|
const std::string& export_dir,
|
|
ExportImportFilesMetaData** metadata) override;
|
|
|
|
// Checkpoint logic can be customized by providing callbacks for link, copy,
|
|
// or create.
|
|
Status CreateCustomCheckpoint(
|
|
std::function<Status(const std::string& src_dirname,
|
|
const std::string& fname, FileType type)>
|
|
link_file_cb,
|
|
std::function<Status(const std::string& src_dirname,
|
|
const std::string& fname, uint64_t size_limit_bytes,
|
|
FileType type, const std::string& checksum_func_name,
|
|
const std::string& checksum_val)>
|
|
copy_file_cb,
|
|
std::function<Status(const std::string& fname,
|
|
const std::string& contents, FileType type)>
|
|
create_file_cb,
|
|
uint64_t* sequence_number, uint64_t log_size_for_flush,
|
|
bool get_live_table_checksum = false);
|
|
|
|
private:
|
|
void CleanStagingDirectory(const std::string& path, Logger* info_log);
|
|
|
|
// Export logic customization by providing callbacks for link or copy.
|
|
Status ExportFilesInMetaData(
|
|
const DBOptions& db_options, const ColumnFamilyMetaData& metadata,
|
|
std::function<Status(const std::string& src_dirname,
|
|
const std::string& fname)>
|
|
link_file_cb,
|
|
std::function<Status(const std::string& src_dirname,
|
|
const std::string& fname)>
|
|
copy_file_cb);
|
|
|
|
private:
|
|
DB* db_;
|
|
};
|
|
|
|
} // namespace ROCKSDB_NAMESPACE
|
|
|
|
#endif // ROCKSDB_LITE
|