rocksdb/table/plain
Zhichao Cao 4369f2c7bb Checksum for each SST file and stores in MANIFEST (#6216)
Summary:
In the current code base, RocksDB generate the checksum for each block and verify the checksum at usage. Current PR enable SST file checksum. After a SST file is generated by Flush or Compaction, RocksDB generate the SST file checksum and store the checksum value and checksum method name in the vs_info and MANIFEST as part for the FileMetadata.

Added the enable_sst_file_checksum to Options to enable or disable file checksum. Added sst_file_checksum to Options such that user can plugin their own SST file checksum calculate method via overriding the SstFileChecksum class. The checksum information inlcuding uint32_t checksum value and a checksum name (string).  A new tool is added to LDB such that user can dump out a list of file checksum information from MANIFEST. If user enables the file checksum but does not provide the sst_file_checksum instance, RocksDB will use the default crc32checksum implemented in table/sst_file_checksum_crc32c.h
Pull Request resolved: https://github.com/facebook/rocksdb/pull/6216

Test Plan: Added the testing case in table_test and ldb_cmd_test to verify checksum is correct in different level. Pass make asan_check.

Differential Revision: D19171461

Pulled By: zhichao-cao

fbshipit-source-id: b2e53479eefc5bb0437189eaa1941670e5ba8b87
2020-02-10 15:52:52 -08:00
..
plain_table_bloom.cc Apply formatter to recent 200+ commits. (#5830) 2019-09-20 12:04:26 -07:00
plain_table_bloom.h Apply formatter to recent 200+ commits. (#5830) 2019-09-20 12:04:26 -07:00
plain_table_builder.cc Checksum for each SST file and stores in MANIFEST (#6216) 2020-02-10 15:52:52 -08:00
plain_table_builder.h Checksum for each SST file and stores in MANIFEST (#6216) 2020-02-10 15:52:52 -08:00
plain_table_factory.cc Organizing rocksdb/table directory by format 2019-05-30 14:51:11 -07:00
plain_table_factory.h Organizing rocksdb/table directory by format 2019-05-30 14:51:11 -07:00
plain_table_index.cc simplify include directive involving inttypes (#5402) 2019-06-06 13:56:07 -07:00
plain_table_index.h Remove a few unnecessary includes 2019-11-19 08:20:42 -08:00
plain_table_key_coding.cc Divide file_reader_writer.h and .cc (#5803) 2019-09-16 10:33:51 -07:00
plain_table_key_coding.h Move some memory related files from util/ to memory/ (#5382) 2019-05-30 17:44:09 -07:00
plain_table_reader.cc Add ReadOptions.auto_prefix_mode (#6314) 2020-01-28 14:44:05 -08:00
plain_table_reader.h Fix PlainTableReader not to crash sst_dump (#5940) 2019-10-18 14:44:42 -07:00