rocksdb/utilities
Peter Dillinger 8a72bb14bd More fixes to auto-GarbageCollect in BackupEngine (#6023)
Summary:
Production:
* Fixes GarbageCollect (and auto-GC triggered by PurgeOldBackups, DeleteBackup, or CreateNewBackup) to clean up backup directory independent of current settings (except max_valid_backups_to_open; see issue https://github.com/facebook/rocksdb/issues/4997) and prior settings used with same backup directory.
* Fixes GarbageCollect (and auto-GC) not to attempt to remove "." and ".." entries from directories.
* Clarifies contract with users in modifying BackupEngine operations. In short, leftovers from any incomplete operation are cleaned up by any subsequent call to that same kind of operation (PurgeOldBackups and DeleteBackup considered the same kind of operation). GarbageCollect is available to clean up after all kinds. (NB: right now PurgeOldBackups and DeleteBackup will clean up after incomplete CreateNewBackup, but we aren't promising to continue that behavior.)
Pull Request resolved: https://github.com/facebook/rocksdb/pull/6023

Test Plan:
* Refactors open parameters to use an option enum, for readability, etc. (Also fixes an unused parameter bug in the redundant OpenDBAndBackupEngineShareWithChecksum.)
* Fixes an apparent bug in ShareTableFilesWithChecksumsTransition in which old backup data was destroyed in the transition to be tested. That test is now augmented to ensure GarbageCollect (or auto-GC) does not remove shared files when BackupEngine is opened with share_table_files=false.
* Augments DeleteTmpFiles test to ensure that CreateNewBackup does auto-GC when an incompletely created backup is detected.

Differential Revision: D18453559

Pulled By: pdillinger

fbshipit-source-id: 5e54e7b08d711b161bc9c656181012b69a8feac4
2019-11-15 12:15:12 -08:00
..
backupable More fixes to auto-GarbageCollect in BackupEngine (#6023) 2019-11-15 12:15:12 -08:00
blob_db Use delete to disable automatic generated methods. (#5009) 2019-09-11 18:09:00 -07:00
cassandra Organizing rocksdb/db directory 2019-05-31 11:57:01 -07:00
checkpoint Export Import sst files (#5495) 2019-07-17 12:27:14 -07:00
compaction_filters Comment out unused variables 2018-03-05 13:13:41 -08:00
convenience Auto roll logger to enforce options.keep_log_file_num immediately after a new file is created (#5370) 2019-05-31 10:50:19 -07:00
leveldb_options Change RocksDB License 2017-07-15 16:11:23 -07:00
memory Organizing rocksdb/db directory 2019-05-31 11:57:01 -07:00
merge_operators New API to get all merge operands for a Key (#5604) 2019-08-06 14:26:44 -07:00
option_change_migration Avoid double-compacting data in bottom level in manual compactions (#5138) 2019-04-16 23:32:20 -07:00
options simplify include directive involving inttypes (#5402) 2019-06-06 13:56:07 -07:00
persistent_cache Fix MyRocks compile warnings-treated-as-errors on Fedora 30, gcc 9.1.1 (#5553) 2019-07-12 17:30:51 -07:00
simulator_cache Pysim more algorithms (#5644) 2019-08-06 18:50:59 -07:00
table_properties_collectors Reduce the number of random iterations in compact_on_deletion_collector_test (#5635) 2019-07-26 15:53:34 -07:00
trace Move util/trace_replay.* to trace_replay/ (#5376) 2019-06-03 13:25:26 -07:00
transactions Use delete to disable automatic generated methods. (#5009) 2019-09-11 18:09:00 -07:00
ttl Fix segfalut in ~DBWithTTLImpl() when called after Close() (#5485) 2019-06-20 13:08:17 -07:00
write_batch_with_index upgrade gtest 1.7.0 => 1.8.1 for json result writing 2019-09-09 11:24:11 -07:00
debug.cc Support GetAllKeyVersions() for non-default cf (#5544) 2019-07-07 22:43:52 -07:00
env_librados_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
env_librados.cc Optionally wait on bytes_per_sync to smooth I/O (#5183) 2019-04-22 11:51:39 -07:00
env_librados.md Add EnvLibrados - RocksDB Env of RADOS (#1222) 2016-07-21 11:16:34 -07:00
env_mirror_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
env_mirror.cc Optionally wait on bytes_per_sync to smooth I/O (#5183) 2019-04-22 11:51:39 -07:00
env_timed_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
env_timed.cc Apply modernize-use-override (2nd iteration) 2019-02-14 14:41:36 -08:00
merge_operators.h New API to get all merge operands for a Key (#5604) 2019-08-06 14:26:44 -07:00
object_registry_test.cc The ObjectRegistry class replaces the Registrar and NewCustomObjects.… (#5293) 2019-07-23 17:13:05 -07:00
object_registry.cc The ObjectRegistry class replaces the Registrar and NewCustomObjects.… (#5293) 2019-07-23 17:13:05 -07:00
util_merge_operators_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00