rocksdb/db/db_impl
Zitan Chen 373d5ac485 BackupEngine verifies table file checksums on creating new backups (#7015)
Summary:
When table file checksums are enabled and stored in the DB manifest by using the RocksDB default crc32c checksum function, BackupEngine will calculate the crc32c checksum of the file to be copied and compare the calculated result with the one stored in the DB manifest before copying the file to the backup directory.

After copying to the backup directory, BackupEngine will verify the checksum of the copied file with the one calculated before copying. This helps detect some rare corruption events such as bit-flips during the copying process.

No verification with checksums in DB manifest will be performed if the table file checksum function is not the RocksDB default crc32c checksum function.

In addition, If `share_table_files` and `share_files_with_checksum` are true, BackupEngine will compare the checksums computed before and after copying of the table files.

Corresponding tests are added.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/7015

Test Plan: Passed make check

Reviewed By: pdillinger

Differential Revision: D22165732

Pulled By: gg814

fbshipit-source-id: ee0e8cc397c455eba64545c29380b9d9853588ec
2020-07-02 18:15:12 -07:00
..
db_impl_compaction_flush.cc Fix data race to VersionSet::io_status_ (#7034) 2020-06-27 08:57:31 -07:00
db_impl_debug.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
db_impl_experimental.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
db_impl_files.cc First step towards handling MANIFEST write error (#6949) 2020-06-24 19:07:08 -07:00
db_impl_open.cc Store DB identity and DB session ID in SST files (#6983) 2020-06-17 10:57:40 -07:00
db_impl_readonly.cc API change: DB::OpenForReadOnly will not write to the file system unless create_if_missing is true (#6900) 2020-06-03 18:57:49 -07:00
db_impl_readonly.h API change: DB::OpenForReadOnly will not write to the file system unless create_if_missing is true (#6900) 2020-06-03 18:57:49 -07:00
db_impl_secondary.cc Properly report IO errors when IndexType::kBinarySearchWithFirstKey is used (#6621) 2020-04-15 17:40:44 -07:00
db_impl_secondary.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
db_impl_write.cc Add timestamp to delete (#6253) 2020-05-28 10:40:03 -07:00
db_impl.cc BackupEngine verifies table file checksums on creating new backups (#7015) 2020-07-02 18:15:12 -07:00
db_impl.h BackupEngine verifies table file checksums on creating new backups (#7015) 2020-07-02 18:15:12 -07:00
db_secondary_test.cc Revert "Whole DBTest to skip fsync (#7049)" (#7070) 2020-07-02 10:22:43 -07:00