andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
557fbc9b3b
rocksdb/db
History
Igor Canadi 768d424dd9 [fix] SIGSEGV when VersionEdit in MANIFEST is corrupted 
Summary:
This was reported by our customers in task #4295529.

Cause:
* MANIFEST file contains a VersionEdit, which contains file entries whose 'smallest' and 'largest' internal keys are empty. String with zero characters. Root cause of corruption was not investigated. We should report corruption when this happens. However, we currently SIGSEGV.

Here's what happens:
* VersionEdit encodes zero-strings happily and stores them in smallest and largest InternalKeys. InternalKey::Encode() does assert when `rep_.empty()`, but we don't assert in production environemnts. Also, we should never assert as a result of DB corruption.
* As part of our ConsistencyCheck, we call GetLiveFilesMetaData()
* GetLiveFilesMetadata() calls `file->largest.user_key().ToString()`
* user_key() function does: 1. assert(size > 8) (ooops, no assert), 2. returns `Slice(internal_key.data(), internal_key.size() - 8)`
* since `internal_key.size()` is unsigned int, this call translates to `Slice(whatever, 1298471928561892576182756)`. Bazinga.

Fix:
* VersionEdit checks if InternalKey is valid in `VersionEdit::GetInternalKey()`. If it's invalid, returns corruption.

Lessons learned:
* Always keep in mind that even if you `assert()`, production code will continue execution even if assert fails.
* Never `assert` based on DB corruption. Assert only if the code should guarantee that assert can't fail.

Test Plan: dumped offending manifest. Before: assert. Now: corruption

Reviewers: dhruba, haobo, sdong

Reviewed By: dhruba

CC: leveldb

Differential Revision: https://reviews.facebook.net/D18507
2014-05-07 16:52:12 -07:00
..
builder.cc Enhance partial merge to support multiple arguments 2014-03-24 17:57:13 -07:00
builder.h [Performance Branch] PlainTable to encode rows with seqID 0, value type using 1 internal byte. 2014-02-03 12:19:30 -08:00
c_test.c kill ReadOptions.prefix and .prefix_seek 2014-04-25 12:21:34 -07:00
c.cc Add rocksdb_open_for_read_only to C API 2014-04-27 20:57:10 +01:00
column_family_test.cc Flush stale column families 2014-04-30 14:33:40 -04:00
column_family.cc Don't overflow size_t in mac 2014-04-16 15:15:22 -07:00
column_family.h thread local for tailing iterator 2014-04-14 10:48:01 -07:00
compaction_picker.cc Column family logging 2014-04-25 09:51:16 -04:00
compaction_picker.h [CF] Code cleanup part 1 2014-03-12 09:56:53 -07:00
compaction.cc Merge branch 'master' into columnfamilies 2014-03-19 17:22:20 -07:00
compaction.h Merge branch 'master' into columnfamilies 2014-02-12 14:01:30 -08:00
corruption_test.cc Fix corruption test 2014-04-24 14:56:41 -04:00
db_bench.cc xxHash 2014-05-01 14:09:32 -04:00
db_filesnapshot.cc RocksDBLite 2014-04-15 13:39:26 -07:00
db_impl_debug.cc Cache result of ReadFirstRecord() 2014-04-29 13:27:58 -04:00
db_impl_readonly.cc Minimize accessing multiple objects in Version::Get() 2014-04-17 14:14:00 -07:00
db_impl_readonly.h [CF] Rethinking ColumnFamilyHandle and fix to dropping column families 2014-02-12 13:47:09 -08:00
db_impl.cc fsync directory after creating current file in NewDB() 2014-05-06 17:51:33 -07:00
db_impl.h Flush stale column families 2014-04-30 14:33:40 -04:00
db_iter.cc thread local for tailing iterator 2014-04-14 10:48:01 -07:00
db_iter.h thread local for tailing iterator 2014-04-14 10:48:01 -07:00
db_stats_logger.cc RocksDBLite 2014-04-15 13:39:26 -07:00
db_test.cc log_and_apply_bench on a new benchmark framework 2014-05-05 11:11:48 -07:00
dbformat_test.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
dbformat.cc macros for perf_context 2014-04-08 10:58:07 -07:00
dbformat.h [fix] SIGSEGV when VersionEdit in MANIFEST is corrupted 2014-05-07 16:52:12 -07:00
deletefile_test.cc Start DeleteFileTest with clean plate 2013-11-15 16:30:23 -08:00
file_indexer_test.cc Some fixes as preparation for release 2014-04-30 09:03:24 -07:00
file_indexer.cc hints for narrowing down FindFile range and avoiding checking unrelevant L0 files 2014-04-21 09:10:12 -07:00
file_indexer.h Remove TransactionLogIteratorRace when -DNDEBUG 2014-04-21 11:08:30 -07:00
filename_test.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
filename.cc fsync directory after creating current file in NewDB() 2014-05-06 17:51:33 -07:00
filename.h fsync directory after creating current file in NewDB() 2014-05-06 17:51:33 -07:00
internal_stats.cc Expose number of entries in mem tables to users 2014-04-22 22:13:21 -07:00
internal_stats.h Expose number of entries in mem tables to users 2014-04-22 22:13:21 -07:00
log_and_apply_bench.cc log_and_apply_bench on a new benchmark framework 2014-05-05 11:11:48 -07:00
log_format.h Some minor refactoring on the code 2014-01-02 16:32:31 -08:00
log_reader.cc Make Log::Reader more robust 2014-02-28 13:19:47 -08:00
log_reader.h Fix UnmarkEOF for partial blocks 2014-01-27 14:49:10 -08:00
log_test.cc Make it compile on Debian/GCC 4.7 2014-03-14 22:44:35 +00:00
log_writer.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
log_writer.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
memtable_list.cc Column family logging 2014-04-25 09:51:16 -04:00
memtable_list.h Expose number of entries in mem tables to users 2014-04-22 22:13:21 -07:00
memtable.cc Pass logger to memtable rep and TLB page allocation error logged to info logs 2014-05-05 16:43:37 -07:00
memtable.h Add a new mem-table representation based on cuckoo hash. 2014-04-29 17:13:46 -07:00
merge_context.h Enhance partial merge to support multiple arguments 2014-03-24 17:57:13 -07:00
merge_helper.cc Fix the bug in MergeUtil which causes mixing values of different keys. 2014-03-27 16:15:25 -07:00
merge_helper.h [rocksdb] new CompactionFilterV2 API 2014-03-24 20:47:53 -07:00
merge_operator.cc Some small cleaning up to make some compiling environment happy 2014-03-26 18:11:41 -07:00
merge_test.cc Use new DBWithTTL API in tests 2014-04-28 23:46:24 -04:00
perf_context_test.cc Missing includes 2014-03-14 13:02:20 -07:00
plain_table_db_test.cc Revert "Revert "Allow allocating dynamic bloom, plain table indexes and hash linked list from huge page TLB"" 2014-05-04 13:56:29 -07:00
prefix_test.cc Revert "Revert "Allow allocating dynamic bloom, plain table indexes and hash linked list from huge page TLB"" 2014-05-04 13:56:29 -07:00
repair.cc fsync directory after creating current file in NewDB() 2014-05-06 17:51:33 -07:00
simple_table_db_test.cc kill ReadOptions.prefix and .prefix_seek 2014-04-25 12:21:34 -07:00
skiplist_test.cc Clean up arena API 2014-01-30 22:10:10 -08:00
skiplist.h Consolidate SliceTransform object ownership 2014-03-10 12:56:46 -07:00
snapshot.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
table_cache.cc avoid calling FindFile twice in TwoLevelIterator for PlainTable 2014-04-25 12:23:07 -07:00
table_cache.h avoid calling FindFile twice in TwoLevelIterator for PlainTable 2014-04-25 12:23:07 -07:00
table_properties_collector_test.cc Turn on -Wmissing-prototypes 2014-04-09 21:17:14 -07:00
table_properties_collector.cc Extract metaindex block from block-based table 2013-12-05 16:34:16 -08:00
table_properties_collector.h Extract metaindex block from block-based table 2013-12-05 16:34:16 -08:00
tailing_iter.cc kill ReadOptions.prefix and .prefix_seek 2014-04-25 12:21:34 -07:00
tailing_iter.h kill ReadOptions.prefix and .prefix_seek 2014-04-25 12:21:34 -07:00
transaction_log_impl.cc RocksDBLite 2014-04-15 13:39:26 -07:00
transaction_log_impl.h RocksDBLite 2014-04-15 13:39:26 -07:00
version_edit_test.cc [CF] Dont reuse dropped column family IDs 2014-03-05 12:13:44 -08:00
version_edit.cc [fix] SIGSEGV when VersionEdit in MANIFEST is corrupted 2014-05-07 16:52:12 -07:00
version_edit.h Minimize accessing multiple objects in Version::Get() 2014-04-17 14:14:00 -07:00
version_set_test.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
version_set.cc fsync directory after creating current file in NewDB() 2014-05-06 17:51:33 -07:00
version_set.h Fix OSX compile 2014-04-26 17:15:43 -04:00
write_batch_internal.h Support for column families in TTL DB 2014-04-22 11:27:33 -07:00
write_batch_test.cc kill ReadOptions.prefix and .prefix_seek 2014-04-25 12:21:34 -07:00
write_batch.cc Support for column families in TTL DB 2014-04-22 11:27:33 -07:00