rocksdb/include/rocksdb
Merlin Mao 55f7ded80d Checkpoint dir options fix (#8572)
Summary:
Originally the 2 options `db_log_dir` and `wal_dir` will be reused in a snapshot db since the options files are just copied. By default, if `wal_dir` was not set when a db was created, it is set to the db's dir. Therefore, the snapshot db will use the same WAL dir. If both the original db and the snapshot db write to or delete from the WAL dir, one may modify or delete files which belong to the other. The same applies to `db_log_dir` as well, but as info log files are not copied or linked, it is simpler for this option.

2 arguments are added to `Checkpoint::CreateCheckpoint()`, allowing to override these 2 options.

`wal_dir`:  If the function argument `wal_dir` is empty, or set to the original db location, or the checkpoint location, the snapshot's `wal_dir` option will be updated to the checkpoint location. Otherwise, the absolute path specified in the argument will be used. During checkpointing, live WAL files will be copied or linked the new location, instead of the current WAL dir specified in the original db.

`db_log_dir`: Same as `wal_dir`, but no files will be copied or linked.

A new unit test was added: `CheckpointTest.CheckpointWithOptionsDirsTest`.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/8572

Test Plan:
New unit test
```
checkpoint_test --gtest_filter="CheckpointTest.CheckpointWithOptionsDirsTest"
```

Output
```
Note: Google Test filter = CheckpointTest.CheckpointWithOptionsDirsTest
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from CheckpointTest
[ RUN      ] CheckpointTest.CheckpointWithOptionsDirsTest
[       OK ] CheckpointTest.CheckpointWithOptionsDirsTest (11712 ms)
[----------] 1 test from CheckpointTest (11712 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test case ran. (11713 ms total)
[  PASSED  ] 1 test.
```
This test will fail without this patch. Just modify the code to remove the 2 arguments introduced in this patch in `CreateCheckpoint()`.

Reviewed By: zhichao-cao

Differential Revision: D29832761

Pulled By: autopear

fbshipit-source-id: e6a639b4d674380df82998c0839e79cab695fe29
2021-07-23 11:13:01 -07:00
..
utilities Checkpoint dir options fix (#8572) 2021-07-23 11:13:01 -07:00
advanced_options.h The formal parameter types of CompressionOptions constructor should b… (#8510) 2021-07-15 18:06:50 -07:00
c.h Add ribbon filter to C API (#8486) 2021-07-09 16:22:48 -07:00
cache_bench_tool.h Allow cache_bench/db_bench to use a custom secondary cache (#8312) 2021-05-19 15:26:18 -07:00
cache.h Parallelize secondary cache lookup in MultiGet (#8405) 2021-06-18 09:35:59 -07:00
cleanable.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
compaction_filter.h Allow applying CompactionFilter outside of compaction (#8243) 2021-05-07 16:01:40 -07:00
compaction_job_stats.h Update compaction statistics to include the amount of data read from blob files (#8022) 2021-03-04 00:43:48 -08:00
comparator.h Make Comparator into a Customizable Object (#8336) 2021-06-11 06:22:59 -07:00
compression_type.h Move CompressionType to its own header file (#7162) 2020-08-03 15:49:31 -07:00
concurrent_task_limiter.h Fix spelling in comments in include/rocksdb/ (#8120) 2021-03-29 05:05:06 -07:00
configurable.h Make FlushBlockPolicyFactory into a Customizable class (#8432) 2021-07-12 09:04:59 -07:00
convenience.h Add ObjectRegistry to ConfigOptions (#8166) 2021-05-11 06:47:22 -07:00
customizable.h Fix PrepareOptions for Customizable Classes (#8468) 2021-06-30 14:09:36 -07:00
data_structure.h Handoff checksum Implementation (#7523) 2021-02-10 22:20:32 -08:00
db_bench_tool.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
db_dump_tool.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
db_stress_tool.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
db.h Add BlobMetaData retrieval methods (#8273) 2021-06-28 08:13:29 -07:00
env_encryption.h Make EncryptionProvider and BlockCipher into Customizable objects (#8354) 2021-07-16 07:58:51 -07:00
env.h Add CreateFrom methods to Env/FileSystem (#8174) 2021-06-15 03:43:48 -07:00
experimental.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
file_checksum.h Fix spelling in comments in include/rocksdb/ (#8120) 2021-03-29 05:05:06 -07:00
file_system.h Add CreateFrom methods to Env/FileSystem (#8174) 2021-06-15 03:43:48 -07:00
filter_policy.h Mark Ribbon filter and optimize_filters_for_memory as production (#8408) 2021-06-17 12:29:16 -07:00
flush_block_policy.h Make FlushBlockPolicyFactory into a Customizable class (#8432) 2021-07-12 09:04:59 -07:00
functor_wrapper.h Add StartThread type checking wrapper (#8303) 2021-05-19 16:51:13 -07:00
io_status.h Add a new IOStatus subcode to indicate that writes are fenced off (#7374) 2020-09-14 16:04:47 -07:00
iostats_context.h Disable IOStatsContext/PerfContext if no thread local (#8117) 2021-04-13 07:56:59 -07:00
iterator.h Enable backward iterator for keys with user-defined timestamp (#8035) 2021-03-10 11:15:46 -08:00
ldb_tool.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
listener.h Some API clarification for manual compaction and listeners (#8330) 2021-05-26 08:14:38 -07:00
memory_allocator.h slightly improve jemalloc allocator API header (#7592) 2020-10-28 13:47:12 -07:00
memtablerep.h Fix comment spelling (#7960) 2021-03-29 10:37:24 -07:00
merge_operator.h Fix spelling in comments in include/rocksdb/ (#8120) 2021-03-29 05:05:06 -07:00
metadata.h Add BlobMetaData retrieval methods (#8273) 2021-06-28 08:13:29 -07:00
options.h Remove extra double quote in options.h (#8550) 2021-07-16 17:05:25 -07:00
perf_context.h Disable IOStatsContext/PerfContext if no thread local (#8117) 2021-04-13 07:56:59 -07:00
perf_level.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
persistent_cache.h Fix persistent cache on windows (#6932) 2020-06-13 13:28:31 -07:00
rate_limiter.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
rocksdb_namespace.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
secondary_cache.h Make SecondaryCache Customizable (#8480) 2021-07-06 09:18:08 -07:00
slice_transform.h Fix spelling in comments in include/rocksdb/ (#8120) 2021-03-29 05:05:06 -07:00
slice.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
snapshot.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
sst_dump_tool.h Add --version and --help to ldb and sst_dump (#6951) 2020-06-09 10:04:01 -07:00
sst_file_manager.h Use SST file manager to track blob files as well (#8037) 2021-03-17 20:44:49 -07:00
sst_file_reader.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
sst_file_writer.h Add more LSM info to FilterBuildingContext (#8246) 2021-04-30 13:50:13 -07:00
sst_partitioner.h Fix spelling in comments in include/rocksdb/ (#8120) 2021-03-29 05:05:06 -07:00
statistics.h Don't hold DB mutex for block cache entry stat scans (#8538) 2021-07-16 14:13:08 -07:00
stats_history.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
status.h Fix comments in Status (#8429) 2021-06-21 13:41:08 -07:00
system_clock.h Add a SystemClock class to capture the time functions of an Env (#7858) 2021-01-25 22:09:11 -08:00
table_properties.h Add table properties for number of entries added to filters (#8323) 2021-05-21 17:11:32 -07:00
table.h Cache warming data blocks during flush (#8242) 2021-06-17 21:56:47 -07:00
thread_status.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
threadpool.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
trace_reader_writer.h Fix spelling in comments in include/rocksdb/ (#8120) 2021-03-29 05:05:06 -07:00
transaction_log.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
types.h Add more LSM info to FilterBuildingContext (#8246) 2021-04-30 13:50:13 -07:00
universal_compaction.h Fix spelling in comments in include/rocksdb/ (#8120) 2021-03-29 05:05:06 -07:00
version.h Update HISTORY.md and version.h 6.23 release (#8552) 2021-07-16 17:52:14 -07:00
wal_filter.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
write_batch_base.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
write_batch.h Fix spelling in comments in include/rocksdb/ (#8120) 2021-03-29 05:05:06 -07:00
write_buffer_manager.h Stall writes in WriteBufferManager when memory_usage exceeds buffer_size (#7898) 2021-04-21 13:54:02 -07:00