andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
58ca641d53
rocksdb/util
History
Igor Canadi 58ca641d53 Make Log::Reader more robust 
Summary:
This diff does two things:
(1) Log::Reader does not report a corruption when the last record in a log or manifest file is truncated (meaning that log writer died in the middle of the write). Inherited the code from LevelDB: https://code.google.com/p/leveldb/source/detail?r=269fc6ca9416129248db5ca57050cd5d39d177c8#
(2) Turn off mmap writes for all writes to log and manifest files

(2) is necessary because if we use mmap writes, the last record is not truncated, but is actually filled with zeros, making checksum fail. It is hard to recover from checksum failing.

Test Plan:
Added unit tests from LevelDB
Actually recovered a "corrupted" MANIFEST file.

Reviewers: dhruba, haobo

Reviewed By: haobo

CC: leveldb

Differential Revision: https://reviews.facebook.net/D16119
2014-02-28 13:19:47 -08:00
..
arena_test.cc Clean up arena API 2014-01-30 22:10:10 -08:00
arena.cc Clean up arena API 2014-01-30 22:10:10 -08:00
arena.h Clean up arena API 2014-01-30 22:10:10 -08:00
auto_roll_logger_test.cc Fix some compilation bugs in different platforms 2014-02-27 22:20:17 -08:00
auto_roll_logger.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
auto_roll_logger.h Enable log info with different levels. 2014-02-26 14:41:28 -08:00
autovector_test.cc Fix issue #57 2014-01-06 11:11:19 -08:00
autovector.h Replace vector with autovector 2014-01-02 16:43:35 -08:00
bit_set.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
blob_store_test.cc Move the compiler back to 4.8.1 + more small fixes 2013-11-18 11:40:16 -08:00
blob_store.cc IOError cleanup 2014-02-12 11:42:54 -08:00
blob_store.h Implement max_size in BlobStore 2013-10-23 14:38:52 -07:00
bloom_test.cc [RocksDB] [Performance Branch] Added dynamic bloom, to be used for memable non-existing key filtering 2013-12-11 00:15:14 -08:00
bloom.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
build_version.h Fixed typos 2013-11-16 11:21:34 +00:00
cache_test.cc [RocksDB] [Performance Branch] Trivia build fix 2013-12-13 14:21:59 -08:00
cache.cc Merge branch 'master' into performance 2014-01-28 10:35:55 -08:00
coding_test.cc Misc cleanup on performance branch 2014-01-17 14:26:29 -08:00
coding.cc Merge branch 'master' into performance 2014-01-28 10:35:55 -08:00
coding.h Merge branch 'master' into performance 2014-01-28 10:35:55 -08:00
comparator.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
crc32c_test.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
crc32c.cc crc32c: choose function in static initialization 2014-02-04 19:13:57 -08:00
crc32c.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
dynamic_bloom_test.cc [RocksDB] [Performance Branch] Added dynamic bloom, to be used for memable non-existing key filtering 2013-12-11 00:15:14 -08:00
dynamic_bloom.cc Optimize MayContainHash() 2014-01-14 22:03:57 -08:00
dynamic_bloom.h Misc cleanup on performance branch 2014-01-17 14:26:29 -08:00
env_hdfs.cc Enable log info with different levels. 2014-02-26 14:41:28 -08:00
env_posix.cc thread local pointer storage 2014-02-25 17:47:37 -08:00
env_test.cc make util/env_posix.cc work under mac 2013-11-16 23:44:39 -08:00
env.cc Make Log::Reader more robust 2014-02-28 13:19:47 -08:00
filelock_test.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
filter_policy.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
hash_linklist_rep.cc Reduce malloc of iterators in Get() code paths 2014-02-11 10:32:51 -08:00
hash_linklist_rep.h Merge branch 'master' into performance 2014-01-15 23:12:31 -08:00
hash_skiplist_rep.cc Reduce malloc of iterators in Get() code paths 2014-02-11 10:32:51 -08:00
hash_skiplist_rep.h Merge branch 'master' into performance 2014-01-15 23:12:31 -08:00
hash.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
hash.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
histogram_test.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
histogram.cc Statistics code cleanup 2014-01-17 12:46:06 -08:00
histogram.h Statistics code cleanup 2014-01-17 12:46:06 -08:00
ldb_cmd_execute_result.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
ldb_cmd.cc Support for LZ4 compression. 2014-02-08 14:15:51 -08:00
ldb_cmd.h Task #3071144 Enhance ldb (db dump tool for leveldb) to report row counters for each row type 2013-11-01 13:59:14 -07:00
ldb_tool.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
log_write_bench.cc A simple benchmark to measure WAL append latency 2014-02-24 14:39:32 -08:00
logging.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
logging.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
manual_compaction_test.cc Fix CompactRange to apply filter to every key 2014-01-14 16:19:09 -08:00
murmurhash.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
murmurhash.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
mutexlock.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
options.cc cache SuperVersion in thread local storage to avoid mutex lock 2014-02-27 11:38:55 -08:00
perf_context_imp.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
perf_context.cc Revert "Allow users to profile a query and see bottleneck of the query" 2013-11-21 17:40:39 -08:00
posix_logger.h Enable log info with different levels. 2014-02-26 14:41:28 -08:00
random.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
signal_test.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
skiplistrep.cc Reduce malloc of iterators in Get() code paths 2014-02-11 10:32:51 -08:00
slice.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
stack_trace.h Print stack trace on assertion failure 2013-12-06 17:11:09 -08:00
statistics.cc convert Tickers back to array with padding and alignment 2014-01-29 15:08:41 -08:00
statistics.h cache SuperVersion in thread local storage to avoid mutex lock 2014-02-27 11:38:55 -08:00
stats_logger.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
status.cc IOError cleanup 2014-02-12 11:42:54 -08:00
stl_wrappers.h Killing Transform Rep 2013-12-03 12:42:15 -08:00
stop_watch.h Statistics code cleanup 2014-01-17 12:46:06 -08:00
string_util.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
string_util.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
testharness.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
testharness.h Print stack trace on assertion failure 2013-12-06 17:11:09 -08:00
testutil.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
testutil.h [Performance Branch] PlainTable to encode rows with seqID 0, value type using 1 internal byte. 2014-02-03 12:19:30 -08:00
thread_local_test.cc Fix some compilation bugs in different platforms 2014-02-27 22:20:17 -08:00
thread_local.cc cache SuperVersion in thread local storage to avoid mutex lock 2014-02-27 11:38:55 -08:00
thread_local.h thread local pointer storage 2014-02-25 17:47:37 -08:00
vectorrep.cc Reduce malloc of iterators in Get() code paths 2014-02-11 10:32:51 -08:00