rocksdb/db
haoyuhuang 74a334a2eb Provide an option so that SST ingestion won't fall back to copy after hard linking fails (#5333)
Summary:
RocksDB always tries to perform a hard link operation on the external SST file to ingest. This operation can fail if the external SST resides on a different device/FS, or the underlying FS does not support hard link. Currently RocksDB assumes that if the link fails, the user is willing to perform file copy, which is not true according to the post. This commit provides an option named  'failed_move_fall_back_to_copy' for users to choose which behavior they want.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/5333

Differential Revision: D15457597

Pulled By: HaoyuHuang

fbshipit-source-id: f3626e13f845db4f7ed970a53ec8a2b1f0d62214
2019-05-23 21:58:52 -07:00
..
builder.cc Periodic Compactions (#5166) 2019-04-10 19:31:18 -07:00
builder.h Periodic Compactions (#5166) 2019-04-10 19:31:18 -07:00
c_test.c add missing rocksdb_flush_cf in c (#5243) 2019-04-25 11:25:43 -07:00
c.cc Unordered Writes (#5218) 2019-05-13 17:47:21 -07:00
column_family_test.cc Apply modernize-use-override (2nd iteration) 2019-02-14 14:41:36 -08:00
column_family.cc Don't call FindObsoleteFiles() in ~ColumnFamilyHandleImpl() if CF is not dropped (#5238) 2019-04-24 17:11:36 -07:00
column_family.h Avoid double-compacting data in bottom level in manual compactions (#5138) 2019-04-16 23:32:20 -07:00
compact_files_test.cc Apply modernize-use-override (2nd iteration) 2019-02-14 14:41:36 -08:00
compacted_db_impl.cc move dump stats to a separate thread (#4382) 2018-10-08 22:54:43 -07:00
compacted_db_impl.h Enable checkpoint of read-only db (#4681) 2018-12-07 17:06:02 -08:00
compaction_iteration_stats.h add counter for deletion dropping optimization 2017-08-19 14:10:08 -07:00
compaction_iterator_test.cc Apply modernize-use-override (2nd iteration) 2019-02-14 14:41:36 -08:00
compaction_iterator.cc WritePrepared: Fix deadlock in WriteRecoverableState (#5306) 2019-05-15 13:53:54 -07:00
compaction_iterator.h Refresh snapshot list during long compactions (2nd attempt) (#5278) 2019-05-03 17:30:22 -07:00
compaction_job_stats_test.cc Apply modernize-use-override (2nd iteration) 2019-02-14 14:41:36 -08:00
compaction_job_test.cc Refresh snapshot list during long compactions (2nd attempt) (#5278) 2019-05-03 17:30:22 -07:00
compaction_job.cc improve comments for CompactionJob (#5341) 2019-05-23 16:57:46 -07:00
compaction_job.h improve comments for CompactionJob (#5341) 2019-05-23 16:57:46 -07:00
compaction_picker_fifo.cc Restrict L0->L0 compaction according to max_compaction_bytes option (#5329) 2019-05-22 23:40:57 -07:00
compaction_picker_fifo.h Avoid double-compacting data in bottom level in manual compactions (#5138) 2019-04-16 23:32:20 -07:00
compaction_picker_test.cc Restrict L0->L0 compaction according to max_compaction_bytes option (#5329) 2019-05-22 23:40:57 -07:00
compaction_picker_universal.cc Add two more StatsLevel (#5027) 2019-02-28 10:27:59 -08:00
compaction_picker_universal.h Delete triggered compaction for universal style 2018-05-29 15:44:34 -07:00
compaction_picker.cc Restrict L0->L0 compaction according to max_compaction_bytes option (#5329) 2019-05-22 23:40:57 -07:00
compaction_picker.h Restrict L0->L0 compaction according to max_compaction_bytes option (#5329) 2019-05-22 23:40:57 -07:00
compaction.cc Dictionary compression for files written by SstFileWriter (#4978) 2019-02-14 11:23:55 -08:00
compaction.h Truncate range tombstones by leveraging InternalKeys (#4432) 2018-10-09 15:19:38 -07:00
comparator_db_test.cc Apply modernize-use-override (2nd iteration) 2019-02-14 14:41:36 -08:00
convenience.cc Update all unique/shared_ptr instances to be qualified with namespace std (#4638) 2018-11-09 11:19:58 -08:00
corruption_test.cc Apply modernize-use-override (2nd iteration) 2019-02-14 14:41:36 -08:00
cuckoo_table_db_test.cc Apply modernize-use-override (2nd iteration) 2019-02-14 14:41:36 -08:00
db_basic_test.cc Fix MultiGet ASSERT bug when passing unsorted result (#5195) 2019-04-15 11:35:21 -07:00
db_blob_index_test.cc fix lite build 2017-10-17 08:57:09 -07:00
db_block_cache_test.cc Add BlockBasedTableOptions::index_shortening (#5174) 2019-04-22 08:20:35 -07:00
db_bloom_filter_test.cc Unordered Writes (#5218) 2019-05-13 17:47:21 -07:00
db_compaction_filter_test.cc Apply modernize-use-override (2nd iteration) 2019-02-14 14:41:36 -08:00
db_compaction_test.cc Use separate status code for column family drop and db shutdown in progress (#5275) 2019-05-20 10:47:32 -07:00
db_dynamic_level_test.cc Fix flaky DBDynamicLevelTest.DynamicLevelMaxBytesBase2 (#4668) 2018-11-12 16:42:16 -08:00
db_encryption_test.cc Update all unique/shared_ptr instances to be qualified with namespace std (#4638) 2018-11-09 11:19:58 -08:00
db_filesnapshot.cc Remove redundant member var and set options (#4631) 2018-11-12 12:24:26 -08:00
db_flush_test.cc Use separate status code for column family drop and db shutdown in progress (#5275) 2019-05-20 10:47:32 -07:00
db_impl_compaction_flush.cc Use separate status code for column family drop and db shutdown in progress (#5275) 2019-05-20 10:47:32 -07:00
db_impl_debug.cc Avoid double-compacting data in bottom level in manual compactions (#5138) 2019-04-16 23:32:20 -07:00
db_impl_experimental.cc Update JobContext. (#3949) 2018-08-03 17:42:34 -07:00
db_impl_files.cc Close WAL files before deletion (#5233) 2019-04-25 10:11:41 -07:00
db_impl_open.cc Unordered Writes (#5218) 2019-05-13 17:47:21 -07:00
db_impl_readonly.cc Add two more StatsLevel (#5027) 2019-02-28 10:27:59 -08:00
db_impl_readonly.h Apply automatic formatting to some files (#5114) 2019-03-27 16:24:45 -07:00
db_impl_secondary.cc Log replay integration for secondary instance (#5305) 2019-05-17 19:19:51 -07:00
db_impl_secondary.h Log replay integration for secondary instance (#5305) 2019-05-17 19:19:51 -07:00
db_impl_write.cc WritePrepared: Clarify the need for two_write_queues in unordered_write (#5313) 2019-05-20 07:49:20 -07:00
db_impl.cc DB::Close() to fail when there are unreleased snapshots (#5272) 2019-05-01 10:17:30 -07:00
db_impl.h Log replay integration for secondary instance (#5305) 2019-05-17 19:19:51 -07:00
db_info_dumper.cc avoid copying when iterating using range-based for (#4459) 2018-10-09 17:15:51 -07:00
db_info_dumper.h Change RocksDB License 2017-07-15 16:11:23 -07:00
db_inplace_update_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_io_failure_test.cc Disable DBIOFailureTest.NoSpaceCompactRange in LITE (#4596) 2018-10-29 14:36:31 -07:00
db_iter_stress_test.cc Move prefix_extractor to MutableCFOptions 2018-05-21 14:43:11 -07:00
db_iter_test.cc Apply modernize-use-override (2nd iteration) 2019-02-14 14:41:36 -08:00
db_iter.cc Reduce iterator key comparison for upper/lower bound check (#5111) 2019-05-17 10:28:31 -07:00
db_iter.h Add comments to db/db_iter.h (#5340) 2019-05-23 16:11:38 -07:00
db_iterator_test.cc Merging iterator to avoid child iterator reseek for some cases (#5286) 2019-05-09 14:20:04 -07:00
db_log_iter_test.cc Apply modernize-use-override (2nd iteration) 2019-02-14 14:41:36 -08:00
db_memtable_test.cc Fix tsan complaint in ConcurrentMergeWrite test (#5308) 2019-05-15 11:21:48 -07:00
db_merge_operator_test.cc WriteUnPrepared: less virtual in iterator callback (#5049) 2019-04-02 14:47:16 -07:00
db_options_test.cc Update RepeatableThreadTest with MockTimeEnv (#5107) 2019-03-29 10:08:50 -07:00
db_properties_test.cc Deprecate ttl option from CompactionOptionsFIFO (#4965) 2019-02-15 09:51:41 -08:00
db_range_del_test.cc Avoid double-compacting data in bottom level in manual compactions (#5138) 2019-04-16 23:32:20 -07:00
db_secondary_test.cc Log replay integration for secondary instance (#5305) 2019-05-17 19:19:51 -07:00
db_sst_test.cc Avoid double-compacting data in bottom level in manual compactions (#5138) 2019-04-16 23:32:20 -07:00
db_statistics_test.cc Make statistics's stats_level change thread-safe (#5030) 2019-03-01 10:42:09 -08:00
db_table_properties_test.cc Improve DBTablePropertiesTest.GetPropertiesOfTablesInRange (#5302) 2019-05-20 13:50:53 -07:00
db_tailing_iter_test.cc Remove managed iterator 2018-07-17 14:43:18 -07:00
db_test2.cc DB::Close() to fail when there are unreleased snapshots (#5272) 2019-05-01 10:17:30 -07:00
db_test_util.cc Unordered Writes (#5218) 2019-05-13 17:47:21 -07:00
db_test_util.h Unordered Writes (#5218) 2019-05-13 17:47:21 -07:00
db_test.cc Introduce a new MultiGet batching implementation (#5011) 2019-04-11 14:28:26 -07:00
db_universal_compaction_test.cc Apply modernize-use-override (2nd iteration) 2019-02-14 14:41:36 -08:00
db_wal_test.cc Preload some files even if options.max_open_files (#3340) 2018-12-28 18:02:28 -08:00
db_write_test.cc Expose DB methods to lock and unlock the WAL (#5146) 2019-04-06 06:40:36 -07:00
dbformat_test.cc Relax VersionStorageInfo::GetOverlappingInputs check (#4050) 2018-07-13 17:42:38 -07:00
dbformat.cc Apply automatic formatting to some files (#5114) 2019-03-27 16:24:45 -07:00
dbformat.h Add comments in db/dbformat.h 2019-05-23 16:44:20 -07:00
deletefile_test.cc Fix a flaky test with test sync point (#5310) 2019-05-15 15:17:55 -07:00
error_handler_test.cc Fix regression test failures introduced by PR #4164 (#4375) 2018-09-17 13:14:07 -07:00
error_handler.cc Fix typos in comments (#4456) 2018-10-04 20:46:50 -07:00
error_handler.h Fix typos in comments (#4456) 2018-10-04 20:46:50 -07:00
event_helpers.cc Log file_creation_time table property (#5232) 2019-04-22 15:30:07 -07:00
event_helpers.h Auto recovery from out of space errors (#4164) 2018-09-15 13:43:04 -07:00
experimental.cc comment unused parameters to turn on -Wunused-parameter flag 2018-04-12 17:59:16 -07:00
external_sst_file_basic_test.cc Apply modernize-use-override (2nd iteration) 2019-02-14 14:41:36 -08:00
external_sst_file_ingestion_job.cc Provide an option so that SST ingestion won't fall back to copy after hard linking fails (#5333) 2019-05-23 21:58:52 -07:00
external_sst_file_ingestion_job.h Atomic ingest (#4895) 2019-02-12 19:16:17 -08:00
external_sst_file_test.cc Provide an option so that SST ingestion won't fall back to copy after hard linking fails (#5333) 2019-05-23 21:58:52 -07:00
fault_injection_test.cc Apply modernize-use-override (2nd iteration) 2019-02-14 14:41:36 -08:00
file_indexer_test.cc Apply modernize-use-override (2nd iteration) 2019-02-14 14:41:36 -08:00
file_indexer.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
file_indexer.h Change RocksDB License 2017-07-15 16:11:23 -07:00
filename_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
flush_job_test.cc Collect compaction stats by priority and dump to info LOG (#5050) 2019-03-19 17:28:19 -07:00
flush_job.cc Use separate status code for column family drop and db shutdown in progress (#5275) 2019-05-20 10:47:32 -07:00
flush_job.h Collect compaction stats by priority and dump to info LOG (#5050) 2019-03-19 17:28:19 -07:00
flush_scheduler.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
flush_scheduler.h Unordered Writes (#5218) 2019-05-13 17:47:21 -07:00
forward_iterator_bench.cc Per-thread unique test db names (#4135) 2018-07-13 17:27:39 -07:00
forward_iterator.cc Avoid per-key upper bound check in BlockBasedTableIterator (#5142) 2019-04-16 11:37:47 -07:00
forward_iterator.h Comment out unused variables 2018-03-05 13:13:41 -08:00
in_memory_stats_history.cc add GetStatsHistory to retrieve stats snapshots (#4748) 2019-02-20 15:52:54 -08:00
in_memory_stats_history.h add GetStatsHistory to retrieve stats snapshots (#4748) 2019-02-20 15:52:54 -08:00
internal_stats.cc Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
internal_stats.h Collect compaction stats by priority and dump to info LOG (#5050) 2019-03-19 17:28:19 -07:00
job_context.h WritePrepared: Fix visible key compacted out by compaction (#4883) 2019-01-15 21:34:38 -08:00
listener_test.cc Avoid double-compacting data in bottom level in manual compactions (#5138) 2019-04-16 23:32:20 -07:00
log_format.h Fix an inaccurate comment (#4315) 2018-08-24 18:13:20 -07:00
log_reader.cc Support for single-primary, multi-secondary instances (#4899) 2019-03-26 16:45:31 -07:00
log_reader.h secondary instance: add support for WAL tailing on OpenAsSecondary 2019-04-24 12:08:44 -07:00
log_test.cc Support for single-primary, multi-secondary instances (#4899) 2019-03-26 16:45:31 -07:00
log_writer.cc LogWriter to only flush after finish generating whole record (#5328) 2019-05-21 12:33:17 -07:00
log_writer.h Close WAL files before deletion (#5233) 2019-04-25 10:11:41 -07:00
logs_with_prep_tracker.cc Skip deleted WALs during recovery 2018-05-03 15:43:09 -07:00
logs_with_prep_tracker.h Skip deleted WALs during recovery 2018-05-03 15:43:09 -07:00
lookup_key.h Introduce a new MultiGet batching implementation (#5011) 2019-04-11 14:28:26 -07:00
malloc_stats.cc Detect if Jemalloc is linked with the binary (#4844) 2019-01-03 16:30:12 -08:00
malloc_stats.h Change RocksDB License 2017-07-15 16:11:23 -07:00
manual_compaction_test.cc Apply modernize-use-override (2nd iteration) 2019-02-14 14:41:36 -08:00
memtable_list_test.cc Apply modernize-use-override (2nd iteration) 2019-02-14 14:41:36 -08:00
memtable_list.cc Use separate status code for column family drop and db shutdown in progress (#5275) 2019-05-20 10:47:32 -07:00
memtable_list.h Log replay integration for secondary instance (#5305) 2019-05-17 19:19:51 -07:00
memtable.cc rename variable to avoid shadowing (#5204) 2019-04-17 10:15:05 -07:00
memtable.h add whole key bloom filter support in memtables (#4985) 2019-02-19 12:15:39 -08:00
merge_context.h Introduce a new MultiGet batching implementation (#5011) 2019-04-11 14:28:26 -07:00
merge_helper_test.cc Apply modernize-use-override (2nd iteration) 2019-02-14 14:41:36 -08:00
merge_helper.cc Add two more StatsLevel (#5027) 2019-02-28 10:27:59 -08:00
merge_helper.h Remove v1 RangeDelAggregator (#4778) 2018-12-17 17:33:46 -08:00
merge_operator.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
merge_test.cc Apply modernize-use-override (2nd iteration) 2019-02-14 14:41:36 -08:00
obsolete_files_test.cc Improve obsolete_files_test (#5125) 2019-03-28 13:16:02 -07:00
options_file_test.cc Per-thread unique test db names (#4135) 2018-07-13 17:27:39 -07:00
perf_context_test.cc Fix some variable naming in db/transaction_log_impl.* (#5112) 2019-03-27 12:27:54 -07:00
pinned_iterators_manager.h Change RocksDB License 2017-07-15 16:11:23 -07:00
plain_table_db_test.cc Unordered Writes (#5218) 2019-05-13 17:47:21 -07:00
pre_release_callback.h Mark logs with prepare in PreReleaseCallback (#5121) 2019-04-02 15:17:47 -07:00
prefix_test.cc Use pre-increment instead of post-increment for iterators (#5296) 2019-05-15 13:19:15 -07:00
range_del_aggregator_bench.cc Fix Windows broken build error due to non-const override (#4798) 2018-12-19 13:29:51 -08:00
range_del_aggregator_test.cc Remove v1 RangeDelAggregator (#4778) 2018-12-17 17:33:46 -08:00
range_del_aggregator.cc Make ReadRangeDelAggregator::ShouldDelete() more inline friendly (#5202) 2019-04-18 12:27:25 -07:00
range_del_aggregator.h RangeDelAggregator::StripeRep::Invalidate() to be skipped if empty (#5312) 2019-05-16 15:24:28 -07:00
range_tombstone_fragmenter_test.cc Prepare FragmentedRangeTombstoneIterator for use in compaction (#4740) 2018-12-11 12:10:48 -08:00
range_tombstone_fragmenter.cc Avoid per-key upper bound check in BlockBasedTableIterator (#5142) 2019-04-16 11:37:47 -07:00
range_tombstone_fragmenter.h Add compaction logic to RangeDelAggregatorV2 (#4758) 2018-12-17 13:20:51 -08:00
read_callback.h WritePrepared: fix race condition in reading batch with duplicate keys (#5147) 2019-04-12 14:40:41 -07:00
repair_test.cc Acquire lock on DB LOCK file before starting repair. (#4435) 2018-10-12 10:41:54 -07:00
repair.cc Force read existing data during db repair (#5209) 2019-04-19 11:55:13 -07:00
snapshot_checker.h WritePrepared: fix issue with snapshot released during compaction (#4858) 2019-01-16 09:55:32 -08:00
snapshot_impl.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
snapshot_impl.h Refresh snapshot list during long compactions (2nd attempt) (#5278) 2019-05-03 17:30:22 -07:00
table_cache.cc Improve explicit user readahead performance (#5246) 2019-04-26 21:24:10 -07:00
table_cache.h Introduce a new MultiGet batching implementation (#5011) 2019-04-11 14:28:26 -07:00
table_properties_collector_test.cc Feature for sampling and reporting compressibility (#4842) 2019-03-18 12:15:34 -07:00
table_properties_collector.cc Feature for sampling and reporting compressibility (#4842) 2019-03-18 12:15:34 -07:00
table_properties_collector.h Feature for sampling and reporting compressibility (#4842) 2019-03-18 12:15:34 -07:00
transaction_log_impl.cc Fix some variable naming in db/transaction_log_impl.* (#5112) 2019-03-27 12:27:54 -07:00
transaction_log_impl.h Fix some variable naming in db/transaction_log_impl.* (#5112) 2019-03-27 12:27:54 -07:00
version_builder_test.cc Apply modernize-use-override (3) 2019-02-19 13:39:49 -08:00
version_builder.cc Support for single-primary, multi-secondary instances (#4899) 2019-03-26 16:45:31 -07:00
version_builder.h Support for single-primary, multi-secondary instances (#4899) 2019-03-26 16:45:31 -07:00
version_edit_test.cc Add a unit test to Ignorable manfiest record (#4964) 2019-02-11 11:20:24 -08:00
version_edit.cc Add a placeholder in manifest indicating ignorable record (#4960) 2019-02-08 11:33:11 -08:00
version_edit.h Support for single-primary, multi-secondary instances (#4899) 2019-03-26 16:45:31 -07:00
version_set_test.cc Apply modernize-use-override (3) 2019-02-19 13:39:49 -08:00
version_set.cc Use GetCurrentManifestPath to locate current MANIFEST file (#5331) 2019-05-22 09:21:56 -07:00
version_set.h Use GetCurrentManifestPath to locate current MANIFEST file (#5331) 2019-05-22 09:21:56 -07:00
wal_manager_test.cc Update all unique/shared_ptr instances to be qualified with namespace std (#4638) 2018-11-09 11:19:58 -08:00
wal_manager.cc Smooth the deletion of WAL files (#5116) 2019-03-28 15:17:13 -07:00
wal_manager.h Fix memleak when DB::DeleteFile() 2018-01-11 18:57:33 -08:00
write_batch_base.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
write_batch_internal.h WriteUnPrepared: Add new WAL marker kTypeBeginUnprepareXID (#4069) 2018-06-28 18:58:29 -07:00
write_batch_test.cc Apply modernize-use-override (3) 2019-02-19 13:39:49 -08:00
write_batch.cc Unordered Writes (#5218) 2019-05-13 17:47:21 -07:00
write_callback_test.cc Unordered Writes (#5218) 2019-05-13 17:47:21 -07:00
write_callback.h Change RocksDB License 2017-07-15 16:11:23 -07:00
write_controller_test.cc Apply modernize-use-override (3) 2019-02-19 13:39:49 -08:00
write_controller.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
write_controller.h Change RocksDB License 2017-07-15 16:11:23 -07:00
write_thread.cc Add more sync point to fix flaky test GroupCommitTest 2018-11-07 14:07:53 -08:00
write_thread.h Fix skip WAL for whole write_group when leader's callback fail (#4838) 2019-01-03 12:40:42 -08:00