andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
ccb761364d
rocksdb/utilities
History
Andrew Kryczka 0cdaa1a804 Fix WAL corruption from checkpoint/backup race condition 
Summary:
`Writer::WriteBuffer` was always called at the beginning of checkpoint/backup. But that log writer has no internal synchronization, which meant the same buffer could be flushed twice in a race condition case, causing a WAL entry to be duplicated. Then subsequent WAL entries would be at unexpected offsets, causing the 32KB block boundaries to be overlapped and manifesting as a corruption.

This PR fixes the behavior to only use `WriteBuffer` (via `FlushWAL`) in checkpoint/backup when manual WAL flush is enabled. In that case, users are responsible for providing synchronization between WAL flushes. We can also consider removing the call entirely.
Closes https://github.com/facebook/rocksdb/pull/3603

Differential Revision: D7277447

Pulled By: ajkr

fbshipit-source-id: 1b15bd7fd930511222b075418c10de0aaa70a35a
2018-03-14 16:12:50 -07:00
..
backupable Disallow compactions if there isn't enough free space 2018-03-06 16:27:54 -08:00
blob_db Blob DB: remove unreacheable code 2018-03-14 14:27:28 -07:00
cassandra Comment out unused variables 2018-03-05 13:13:41 -08:00
checkpoint Fix WAL corruption from checkpoint/backup race condition 2018-03-14 16:12:50 -07:00
compaction_filters Comment out unused variables 2018-03-05 13:13:41 -08:00
convenience Change RocksDB License 2017-07-15 16:11:23 -07:00
date_tiered WritePrepared Txn: Iterator 2017-10-09 17:15:28 -07:00
document Comment out unused variables 2018-03-05 13:13:41 -08:00
geodb Change RocksDB License 2017-07-15 16:11:23 -07:00
leveldb_options Change RocksDB License 2017-07-15 16:11:23 -07:00
lua Suppress unused warnings 2018-02-02 12:27:07 -08:00
memory Change RocksDB License 2017-07-15 16:11:23 -07:00
merge_operators Support StringAppendOperator(delimiter_char) constructor in java-api 2018-03-08 16:17:47 -08:00
option_change_migration Change RocksDB License 2017-07-15 16:11:23 -07:00
options Comment out unused variables 2018-03-05 13:13:41 -08:00
persistent_cache Comment out unused variables 2018-03-05 13:13:41 -08:00
redis Use nullptr instead of NULL / 0 more consistently. 2018-03-07 12:42:12 -08:00
simulator_cache Comment out unused variables 2018-03-05 13:13:41 -08:00
spatialdb Comment out unused variables 2018-03-05 13:13:41 -08:00
table_properties_collectors Comment out unused variables 2018-03-05 13:13:41 -08:00
transactions Fix some typos in comments and docs. 2018-03-08 10:27:25 -08:00
ttl Comment out unused variables 2018-03-05 13:13:41 -08:00
write_batch_with_index Comment out unused variables 2018-03-05 13:13:41 -08:00
col_buf_decoder.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
col_buf_decoder.h Comment out unused variables 2018-03-05 13:13:41 -08:00
col_buf_encoder.cc fix shift UBSAN error in col_buf_encoder.cc 2018-02-20 16:44:00 -08:00
col_buf_encoder.h Change RocksDB License 2017-07-15 16:11:23 -07:00
column_aware_encoding_exp.cc fix gflags namespace 2017-12-01 10:42:05 -08:00
column_aware_encoding_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
column_aware_encoding_util.cc WritePrepared Txn: fix non-emptied PreparedHeap bug 2018-02-21 13:42:23 -08:00
column_aware_encoding_util.h Change RocksDB License 2017-07-15 16:11:23 -07:00
debug.cc Fix naming in InternalKey 2017-09-12 17:17:42 -07:00
env_librados_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
env_librados.cc Suppress lint in old files 2018-01-29 12:56:42 -08:00
env_librados.md Add EnvLibrados - RocksDB Env of RADOS (#1222) 2016-07-21 11:16:34 -07:00
env_mirror_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
env_mirror.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
env_timed_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
env_timed.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
merge_operators.h Support StringAppendOperator(delimiter_char) constructor in java-api 2018-03-08 16:17:47 -08:00
object_registry_test.cc Comment out unused variables 2018-03-05 13:13:41 -08:00
util_merge_operators_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00