andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
f19612970d
rocksdb/db
History
Akanksha Mahajan f19612970d Support retrieving checksums for blob files from the MANIFEST when checkpointing (#8003) 
Summary:
The checkpointing logic supports passing file level checksums
to the copy_file_cb callback function which is used by the backup code
for detecting corruption during file copies.
However, this is currently implemented only for table files.

This PR extends the checksum retrieval to blob files as well.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/8003

Test Plan: Add new test units

Reviewed By: ltamasi

Differential Revision: D26680701

Pulled By: akankshamahajan15

fbshipit-source-id: 1bd1e2464df6e9aa31091d35b8c72786d94cd1c5
2021-03-01 20:07:07 -08:00
..
blob Compaction filter support for (new) BlobDB (#7974) 2021-02-25 16:32:35 -08:00
compaction Compaction filter support for (new) BlobDB (#7974) 2021-02-25 16:32:35 -08:00
db_impl Compaction filter support for (new) BlobDB (#7974) 2021-02-25 16:32:35 -08:00
arena_wrapped_db_iter.cc Add blob support to DBIter (#7731) 2020-12-04 21:29:38 -08:00
arena_wrapped_db_iter.h Add blob support to DBIter (#7731) 2020-12-04 21:29:38 -08:00
builder.cc Add support for IOTracing in blob files (#7958) 2021-02-16 09:49:10 -08:00
builder.h Add full_history_ts_low_ to FlushJob (#7655) 2020-11-12 18:44:34 -08:00
c_test.c Add get/set deadline and io_timeout C functions (read options) (#7914) 2021-02-04 17:00:58 -08:00
c.cc Limit buffering for collecting samples for compression dictionary (#7970) 2021-02-19 14:09:54 -08:00
column_family_test.cc Using emplace_back replace push_back (#7568) 2021-01-15 16:56:41 -08:00
column_family.cc Fix handling of Mutable options; Allow DB::SetOptions to update mutable TableFactory Options (#7936) 2021-02-19 10:29:02 -08:00
column_family.h Track WAL obsoletion when updating empty CF's log number (#7781) 2020-12-18 21:34:36 -08:00
compact_files_test.cc Add more tests for assert status checked (#7524) 2020-12-22 23:45:58 -08:00
compacted_db_impl.cc Periodically flush info log out of application buffer (#7488) 2020-10-01 19:14:14 -07:00
compacted_db_impl.h SyncWAL shouldn't be supported in compacted db (#7788) 2020-12-22 14:53:43 -08:00
comparator_db_test.cc Remove unused includes (#7604) 2020-10-28 23:22:27 -07:00
convenience.cc sst_dump to reduce number of file reads (#6836) 2020-05-12 18:23:33 -07:00
corruption_test.cc Extend VerifyFileChecksums API for blob files (#7979) 2021-02-22 22:09:22 -08:00
cuckoo_table_db_test.cc Revert "Turn on memtable bloom filter by default. (#6584)" (#7939) 2021-02-06 22:34:30 -08:00
db_basic_test.cc Create a CustomEnv class; Add WinFileSystem; Make LegacyFileSystemWrapper private (#7703) 2021-01-06 10:49:32 -08:00
db_block_cache_test.cc Add further tests to ASSERT_STATUS_CHECKED (2) (#7698) 2020-12-09 21:21:16 -08:00
db_bloom_filter_test.cc Revert "Turn on memtable bloom filter by default. (#6584)" (#7939) 2021-02-06 22:34:30 -08:00
db_compaction_filter_test.cc Add more tests to ASSERT_STATUS_CHECKED (3), API change (#7715) 2021-01-06 14:15:02 -08:00
db_compaction_test.cc Handoff checksum Implementation (#7523) 2021-02-10 22:20:32 -08:00
db_dynamic_level_test.cc Add a SystemClock class to capture the time functions of an Env (#7858) 2021-01-25 22:09:11 -08:00
db_encryption_test.cc Improvements to Env::GetChildren (#7819) 2021-01-09 09:44:34 -08:00
db_filesnapshot.cc Fix checkpoint file deletion race with avoid_unnecessary_blocking_io (#7369) 2020-09-10 22:35:25 -07:00
db_flush_test.cc Handoff checksum Implementation (#7523) 2021-02-10 22:20:32 -08:00
db_info_dumper.cc Fix write-ahead log file size overflow (#7870) 2021-01-19 13:47:48 -08:00
db_info_dumper.h Add a DB Session ID (#6959) 2020-06-15 10:47:02 -07:00
db_inplace_update_test.cc Whole DBTest to skip fsync (#7274) 2020-08-17 18:42:25 -07:00
db_io_failure_test.cc Add more tests to ASSERT_STATUS_CHECKED (3), API change (#7715) 2021-01-06 14:15:02 -08:00
db_iter_stress_test.cc Add blob support to DBIter (#7731) 2020-12-04 21:29:38 -08:00
db_iter_test.cc Add more tests for assert status checked (#7524) 2020-12-22 23:45:58 -08:00
db_iter.cc Introduce a new trace file format (v 0.2) for better extension (#7977) 2021-02-18 23:05:35 -08:00
db_iter.h Remove unused/incorrect fwd declaration (#8002) 2021-02-25 23:07:31 -08:00
db_iterator_test.cc Add further tests to ASSERT_STATUS_CHECKED (1) (#7679) 2020-12-08 15:55:04 -08:00
db_kv_checksum_test.cc Integrity protection for live updates to WriteBatch (#7748) 2021-01-29 12:18:58 -08:00
db_log_iter_test.cc Add a SystemClock class to capture the time functions of an Env (#7858) 2021-01-25 22:09:11 -08:00
db_logical_block_size_cache_test.cc Add further tests to ASSERT_STATUS_CHECKED (2) (#7698) 2020-12-09 21:21:16 -08:00
db_memtable_test.cc Integrity protection for live updates to WriteBatch (#7748) 2021-01-29 12:18:58 -08:00
db_merge_operand_test.cc Add further tests to ASSERT_STATUS_CHECKED (1) (#7679) 2020-12-08 15:55:04 -08:00
db_merge_operator_test.cc Add further tests to ASSERT_STATUS_CHECKED (1) (#7679) 2020-12-08 15:55:04 -08:00
db_options_test.cc Fix handling of Mutable options; Allow DB::SetOptions to update mutable TableFactory Options (#7936) 2021-02-19 10:29:02 -08:00
db_properties_test.cc Fix many tests to run with MEM_ENV and ENCRYPTED_ENV; Introduce a MemoryFileSystem class (#7566) 2020-10-27 10:33:09 -07:00
db_range_del_test.cc Add more tests to ASSERT_STATUS_CHECKED (4) (#7718) 2020-12-22 15:09:39 -08:00
db_sst_test.cc Disable PeriodicWorkScheduler during RateLimited test (#7810) 2021-01-11 15:01:52 -08:00
db_statistics_test.cc Add more tests for assert status checked (#7524) 2020-12-22 23:45:58 -08:00
db_table_properties_test.cc Add more tests for assert status checked (#7524) 2020-12-22 23:45:58 -08:00
db_tailing_iter_test.cc Add more tests to ASSERT_STATUS_CHECKED (3), API change (#7715) 2021-01-06 14:15:02 -08:00
db_test2.cc Limit buffering for collecting samples for compression dictionary (#7970) 2021-02-19 14:09:54 -08:00
db_test_util.cc Revert "Turn on memtable bloom filter by default. (#6584)" (#7939) 2021-02-06 22:34:30 -08:00
db_test_util.h Add a SystemClock class to capture the time functions of an Env (#7858) 2021-01-25 22:09:11 -08:00
db_test.cc Revert "Turn on memtable bloom filter by default. (#6584)" (#7939) 2021-02-06 22:34:30 -08:00
db_universal_compaction_test.cc Revert "Turn on memtable bloom filter by default. (#6584)" (#7939) 2021-02-06 22:34:30 -08:00
db_wal_test.cc Handoff checksum Implementation (#7523) 2021-02-10 22:20:32 -08:00
db_with_timestamp_basic_test.cc Add full_history_ts_low option to compaction (#7884) 2021-02-08 13:45:48 -08:00
db_with_timestamp_compaction_test.cc Whole DBTest to skip fsync (#7274) 2020-08-17 18:42:25 -07:00
db_write_test.cc Add more tests to ASSERT_STATUS_CHECKED (4) (#7718) 2020-12-22 15:09:39 -08:00
dbformat_test.cc Remove unused includes (#7604) 2020-10-28 23:22:27 -07:00
dbformat.cc Make CompactRange and GetApproximateSizes work with timestamp (#7684) 2020-12-02 13:00:53 -08:00
dbformat.h Integrity protection for live updates to WriteBatch (#7748) 2021-01-29 12:18:58 -08:00
deletefile_test.cc Add more tests to ASSERT_STATUS_CHECKED (4) (#7718) 2020-12-22 15:09:39 -08:00
error_handler_fs_test.cc Disable flaky error_handler_fs_test that could hang (#7964) 2021-02-15 09:45:23 -08:00
error_handler.cc Do not set bg error for compaction in retryable IO Error case (#7899) 2021-01-27 17:58:12 -08:00
error_handler.h Change ErrorHandler methods to return const Status& (#7539) 2020-12-07 20:11:35 -08:00
event_helpers.cc Add more tests to ASSERT_STATUS_CHECKED (3), API change (#7715) 2021-01-06 14:15:02 -08:00
event_helpers.h Pass SST file checksum information through OnTableFileCreated (#7108) 2020-08-25 10:46:11 -07:00
experimental.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
external_sst_file_basic_test.cc Add further tests to ASSERT_STATUS_CHECKED (2) (#7698) 2020-12-09 21:21:16 -08:00
external_sst_file_ingestion_job.cc Add a SystemClock class to capture the time functions of an Env (#7858) 2021-01-25 22:09:11 -08:00
external_sst_file_ingestion_job.h Add a SystemClock class to capture the time functions of an Env (#7858) 2021-01-25 22:09:11 -08:00
external_sst_file_test.cc Add more tests to ASSERT_STATUS_CHECKED (4) (#7718) 2020-12-22 15:09:39 -08:00
fault_injection_test.cc Add more tests for assert status checked (#7524) 2020-12-22 23:45:58 -08:00
file_indexer_test.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
file_indexer.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
file_indexer.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
filename_test.cc Remove unused includes (#7604) 2020-10-28 23:22:27 -07:00
flush_job_test.cc Fix testcase failures on windows (#7992) 2021-02-23 14:35:06 -08:00
flush_job.cc Add a SystemClock class to capture the time functions of an Env (#7858) 2021-01-25 22:09:11 -08:00
flush_job.h Add a SystemClock class to capture the time functions of an Env (#7858) 2021-01-25 22:09:11 -08:00
flush_scheduler.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
flush_scheduler.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
forward_iterator_bench.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
forward_iterator.cc Add more tests for assert status checked (#7524) 2020-12-22 23:45:58 -08:00
forward_iterator.h Properly report IO errors when IndexType::kBinarySearchWithFirstKey is used (#6621) 2020-04-15 17:40:44 -07:00
import_column_family_job.cc Add a SystemClock class to capture the time functions of an Env (#7858) 2021-01-25 22:09:11 -08:00
import_column_family_job.h Add a SystemClock class to capture the time functions of an Env (#7858) 2021-01-25 22:09:11 -08:00
import_column_family_test.cc No elide constructors (#7798) 2020-12-23 16:55:53 -08:00
internal_stats.cc Add a SystemClock class to capture the time functions of an Env (#7858) 2021-01-25 22:09:11 -08:00
internal_stats.h Add a SystemClock class to capture the time functions of an Env (#7858) 2021-01-25 22:09:11 -08:00
job_context.h Expose the set of live blob files from Version/VersionSet (#6785) 2020-05-04 15:08:13 -07:00
kv_checksum.h Integrity protection for live updates to WriteBatch (#7748) 2021-01-29 12:18:58 -08:00
listener_test.cc Add more tests for assert status checked (#7524) 2020-12-22 23:45:58 -08:00
log_format.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
log_reader.cc Fix kPointInTimeRecovery handling of truncated WAL (#7701) 2020-11-30 18:11:38 -08:00
log_reader.h Real fix for race in backup custom checksum checking (#7309) 2020-08-26 10:39:20 -07:00
log_test.cc Make StringEnv, StringSink, StringSource use FS classes (#7786) 2021-01-04 16:01:01 -08:00
log_writer.cc No elide constructors (#7798) 2020-12-23 16:55:53 -08:00
log_writer.h Pass IOStatus to write path and set retryable IO Error as hard error in BG jobs (#6487) 2020-03-27 16:04:43 -07:00
logs_with_prep_tracker.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
logs_with_prep_tracker.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
lookup_key.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
malloc_stats.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
malloc_stats.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
manual_compaction_test.cc Add more tests for assert status checked (#7524) 2020-12-22 23:45:58 -08:00
memtable_list_test.cc Revert "Turn on memtable bloom filter by default. (#6584)" (#7939) 2021-02-06 22:34:30 -08:00
memtable_list.cc Add initial blob support to batched MultiGet (#7766) 2020-12-14 13:48:22 -08:00
memtable_list.h Add initial blob support to batched MultiGet (#7766) 2020-12-14 13:48:22 -08:00
memtable.cc Integrity protection for live updates to WriteBatch (#7748) 2021-01-29 12:18:58 -08:00
memtable.h Integrity protection for live updates to WriteBatch (#7748) 2021-01-29 12:18:58 -08:00
merge_context.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
merge_helper_test.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
merge_helper.cc Add a SystemClock class to capture the time functions of an Env (#7858) 2021-01-25 22:09:11 -08:00
merge_helper.h Add a SystemClock class to capture the time functions of an Env (#7858) 2021-01-25 22:09:11 -08:00
merge_operator.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
merge_test.cc MergeHelper::FilterMerge() calling ElapsedNanosSafe() upon exit even … (#7867) 2021-01-21 13:13:02 -08:00
obsolete_files_test.cc Add more tests for assert status checked (#7524) 2020-12-22 23:45:58 -08:00
options_file_test.cc No elide constructors (#7798) 2020-12-23 16:55:53 -08:00
output_validator.cc Use NPHash64 in more places (#7632) 2020-11-10 23:42:13 -08:00
output_validator.h Use NPHash64 in more places (#7632) 2020-11-10 23:42:13 -08:00
perf_context_test.cc Add a SystemClock class to capture the time functions of an Env (#7858) 2021-01-25 22:09:11 -08:00
periodic_work_scheduler_test.cc Add a SystemClock class to capture the time functions of an Env (#7858) 2021-01-25 22:09:11 -08:00
periodic_work_scheduler.cc Add a SystemClock class to capture the time functions of an Env (#7858) 2021-01-25 22:09:11 -08:00
periodic_work_scheduler.h Add a SystemClock class to capture the time functions of an Env (#7858) 2021-01-25 22:09:11 -08:00
pinned_iterators_manager.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
plain_table_db_test.cc Make StringEnv, StringSink, StringSource use FS classes (#7786) 2021-01-04 16:01:01 -08:00
pre_release_callback.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
prefix_test.cc Add a SystemClock class to capture the time functions of an Env (#7858) 2021-01-25 22:09:11 -08:00
range_del_aggregator_bench.cc Add a SystemClock class to capture the time functions of an Env (#7858) 2021-01-25 22:09:11 -08:00
range_del_aggregator_test.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
range_del_aggregator.cc In ParseInternalKey(), include corrupt key info in Status (#7515) 2020-10-28 10:12:58 -07:00
range_del_aggregator.h In ParseInternalKey(), include corrupt key info in Status (#7515) 2020-10-28 10:12:58 -07:00
range_tombstone_fragmenter_test.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
range_tombstone_fragmenter.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
range_tombstone_fragmenter.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
read_callback.h Get() with timestamp should respect snapshot (#7227) 2020-08-14 19:20:58 -07:00
repair_test.cc Create a CustomEnv class; Add WinFileSystem; Make LegacyFileSystemWrapper private (#7703) 2021-01-06 10:49:32 -08:00
repair.cc Remove Legacy and Custom FileWrapper classes from header files (#7851) 2021-01-28 22:10:32 -08:00
snapshot_checker.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
snapshot_impl.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
snapshot_impl.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
table_cache.cc Integrity protection for live updates to WriteBatch (#7748) 2021-01-29 12:18:58 -08:00
table_cache.h Store FSRandomAccessPtr object in RandomAccessFileReader (#7192) 2020-08-27 11:21:52 -07:00
table_properties_collector_test.cc Make StringEnv, StringSink, StringSource use FS classes (#7786) 2021-01-04 16:01:01 -08:00
table_properties_collector.cc In ParseInternalKey(), include corrupt key info in Status (#7515) 2020-10-28 10:12:58 -07:00
table_properties_collector.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
transaction_log_impl.cc Add more tests for assert status checked (#7524) 2020-12-22 23:45:58 -08:00
transaction_log_impl.h Store FileSystemPtr object that contains FileSystem ptr (#7180) 2020-08-12 17:31:23 -07:00
trim_history_scheduler.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
trim_history_scheduler.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
version_builder_test.cc Remove unused includes (#7604) 2020-10-28 23:22:27 -07:00
version_builder.cc Log sst number in Corruption status (#7767) 2020-12-14 14:07:52 -08:00
version_builder.h make L0 index/filter pinned memory usage predictable (#6911) 2020-06-09 16:51:23 -07:00
version_edit_handler.cc Support retrieving checksums for blob files from the MANIFEST when checkpointing (#8003) 2021-03-01 20:07:07 -08:00
version_edit_handler.h Refactor with VersionEditHandler (#6581) 2020-11-11 08:00:14 -08:00
version_edit_test.cc Make it able to ignore WAL related VersionEdits in older versions (#7873) 2021-01-19 19:27:53 -08:00
version_edit.cc Make blob related VersionEdit tags unignorable (#7886) 2021-01-20 20:29:04 -08:00
version_edit.h Accumulate blob file additions in VersionEdit during recovery (#7903) 2021-01-27 18:46:15 -08:00
version_set_test.cc Remove Legacy and Custom FileWrapper classes from header files (#7851) 2021-01-28 22:10:32 -08:00
version_set.cc Handoff checksum Implementation (#7523) 2021-02-10 22:20:32 -08:00
version_set.h Add a SystemClock class to capture the time functions of an Env (#7858) 2021-01-25 22:09:11 -08:00
wal_edit_test.cc Always track WAL obsoletion (#7759) 2020-12-09 16:02:12 -08:00
wal_edit.cc Always track WAL obsoletion (#7759) 2020-12-09 16:02:12 -08:00
wal_edit.h Always track WAL obsoletion (#7759) 2020-12-09 16:02:12 -08:00
wal_manager_test.cc Remove Legacy and Custom FileWrapper classes from header files (#7851) 2021-01-28 22:10:32 -08:00
wal_manager.cc Add more tests for assert status checked (#7524) 2020-12-22 23:45:58 -08:00
wal_manager.h Store FileSystemPtr object that contains FileSystem ptr (#7180) 2020-08-12 17:31:23 -07:00
write_batch_base.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
write_batch_internal.h Integrity protection for live updates to WriteBatch (#7748) 2021-01-29 12:18:58 -08:00
write_batch_test.cc Add further tests to ASSERT_STATUS_CHECKED (1) (#7679) 2020-12-08 15:55:04 -08:00
write_batch.cc Integrity protection for live updates to WriteBatch (#7748) 2021-01-29 12:18:58 -08:00
write_callback_test.cc Add more tests for assert status checked (#7524) 2020-12-22 23:45:58 -08:00
write_callback.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
write_controller_test.cc Add a SystemClock class to capture the time functions of an Env (#7858) 2021-01-25 22:09:11 -08:00
write_controller.cc Add a SystemClock class to capture the time functions of an Env (#7858) 2021-01-25 22:09:11 -08:00
write_controller.h Add a SystemClock class to capture the time functions of an Env (#7858) 2021-01-25 22:09:11 -08:00
write_thread.cc Integrity protection for live updates to WriteBatch (#7748) 2021-01-29 12:18:58 -08:00
write_thread.h Integrity protection for live updates to WriteBatch (#7748) 2021-01-29 12:18:58 -08:00