andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
dbbffbd772
rocksdb/db
History
Igor Canadi dbbffbd772 Mark the log_number file number used 
Summary:
VersionSet::next_file_number_ is always assumed to be strictly greater than VersionSet::log_number_. In our new recovery code, we artificially set log_number_  to be (log_number + 1), so that once we flush, we don't recover from the same log file again (this is important because of merge operator non-idempotence)

When we set VersionSet::log_number_ to (log_number + 1), we also have to mark that file number used, such that next_file_number_ is increased to a legal level. Otherwise, VersionSet might assert.

This has not be a problem so far because here's what happens:
1. assume next_file_number is 5, we're recovering log_number 10
2. in DBImpl::Recover() we call MarkFileNumberUsed with 10. This will set VersionSet::next_file_number_ to 11.
3. If there are some updates, we will call WriteTable0ForRecovery(), which will use file number 11 as a new table file and advance VersionSet::next_file_number_ to 12.
4. When we LogAndApply() with log_number 11, assertion is true: assert(11 <= 12);

However, this was a lucky occurrence. Even though this diff doesn't cause a bug, I think the issue is important to fix.

Test Plan: In column families I have different recovery logic and this code path asserted. When adding MarkFileNumberUsed(log_number + 1) assert is gone.

Reviewers: dhruba, kailiu

Reviewed By: kailiu

CC: leveldb

Differential Revision: https://reviews.facebook.net/D15783
2014-01-31 14:43:16 -08:00
..
builder.cc Don't always compress L0 files written by memtable flush 2014-01-07 21:50:26 -08:00
builder.h Don't always compress L0 files written by memtable flush 2014-01-07 21:50:26 -08:00
c_test.c Rename leveldb to rocksdb in C api 2013-12-10 10:48:35 +02:00
c.cc C API: add rocksdb_env_set_high_priority_background_threads 2013-12-31 15:14:18 -08:00
compaction_picker.cc Moving Some includes from options.h to forward declaration 2014-01-24 17:16:22 -08:00
compaction_picker.h Make VersionSet::ReduceNumberOfLevels() static 2014-01-24 14:57:04 -08:00
compaction.cc Unfriending classes 2014-01-22 10:55:16 -08:00
compaction.h Unfriending classes 2014-01-22 10:55:16 -08:00
corruption_test.cc Make "Table" pluggable 2013-10-28 17:54:09 -07:00
db_bench.cc Moving Some includes from options.h to forward declaration 2014-01-24 17:16:22 -08:00
db_filesnapshot.cc Move functions from VersionSet to Version 2014-01-15 16:18:04 -08:00
db_impl_readonly.cc Refactor Recover() code 2014-01-22 10:45:26 -08:00
db_impl_readonly.h CompactRange() to return status 2014-01-23 16:41:46 -08:00
db_impl.cc Mark the log_number file number used 2014-01-31 14:43:16 -08:00
db_impl.h InternalStatistics 2014-01-29 20:40:41 -08:00
db_iter.cc Get rid of some shared_ptrs 2013-12-03 11:17:58 -08:00
db_iter.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
db_stats_logger.cc Move functions from VersionSet to Version 2014-01-15 16:18:04 -08:00
db_test.cc Moving Some includes from options.h to forward declaration 2014-01-24 17:16:22 -08:00
dbformat_test.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
dbformat.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
dbformat.h Make "Table" pluggable 2013-10-28 17:54:09 -07:00
deletefile_test.cc Start DeleteFileTest with clean plate 2013-11-15 16:30:23 -08:00
filename_test.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
filename.cc Unify DeleteFile and DeleteWalFiles 2013-10-25 08:32:14 -07:00
filename.h Unify DeleteFile and DeleteWalFiles 2013-10-25 08:32:14 -07:00
internal_stats.cc InternalStatistics 2014-01-29 20:40:41 -08:00
internal_stats.h InternalStatistics 2014-01-29 20:40:41 -08:00
log_format.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
log_reader.cc Fix UnmarkEOF for partial blocks 2014-01-27 14:49:10 -08:00
log_reader.h Fix UnmarkEOF for partial blocks 2014-01-27 14:49:10 -08:00
log_test.cc Fix UnmarkEOF for partial blocks 2014-01-27 14:49:10 -08:00
log_writer.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
log_writer.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
memtable.cc Statistics code cleanup 2014-01-17 12:46:06 -08:00
memtable.h Remove the unnecessary use of shared_ptr 2014-01-15 18:22:01 -08:00
memtablelist.cc Fsync directory after we create a new file 2014-01-27 11:02:21 -08:00
memtablelist.h Fsync directory after we create a new file 2014-01-27 11:02:21 -08:00
merge_context.h Introduce MergeContext to Lazily Initialize merge operand list 2013-12-11 11:37:28 -08:00
merge_helper.cc Statistics code cleanup 2014-01-17 12:46:06 -08:00
merge_helper.h [RocksDB] Use raw pointer instead of shared pointer when passing Statistics object internally 2013-11-25 10:38:15 -08:00
merge_operator.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
merge_test.cc Use ASSERT_EQ() instead of assert() in merge_test 2014-01-11 09:25:47 -08:00
perf_context_test.cc Killing Transform Rep 2013-12-03 12:42:15 -08:00
prefix_filter_iterator.h Moving Some includes from options.h to forward declaration 2014-01-24 17:16:22 -08:00
prefix_test.cc Killing Transform Rep 2013-12-03 12:42:15 -08:00
repair.cc VersionEdit not to take NumLevels() 2014-01-14 15:27:09 -08:00
simple_table_db_test.cc Statistics code cleanup 2014-01-17 12:46:06 -08:00
skiplist_test.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
skiplist.h Moving Some includes from options.h to forward declaration 2014-01-24 17:16:22 -08:00
snapshot.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
table_cache.cc Hotfix the bug in table cache's GetSliceForFileNumber 2014-01-02 10:30:42 -08:00
table_cache.h Follow-up Cleaning-up After D13521 2013-10-30 10:52:33 -07:00
table_properties_collector_test.cc Move flush_block_policy from Options to TableFactory 2013-11-19 22:00:48 -08:00
table_properties_collector.cc Improve the readability of the TableProperties::ToString() 2013-11-21 17:54:23 -08:00
table_properties_collector.h Improve the "table stats" 2013-11-19 16:29:42 -08:00
tailing_iter.cc Moving Some includes from options.h to forward declaration 2014-01-24 17:16:22 -08:00
tailing_iter.h Tailing iterator 2014-01-23 16:26:08 -08:00
transaction_log_impl.cc Fix the string format issue 2013-11-12 21:05:39 -08:00
transaction_log_impl.h Making the transaction log iterator more robust 2013-11-04 20:49:03 -08:00
version_edit_test.cc Remove compaction pointers 2014-01-16 14:06:53 -08:00
version_edit.cc Remove compaction pointers 2014-01-16 14:06:53 -08:00
version_edit.h Remove compaction pointers 2014-01-16 14:06:53 -08:00
version_set_test.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
version_set.cc The default settings enable checksum verification on every read. 2014-01-30 19:14:03 -08:00
version_set.h add include <atomic> to version_set.h 2014-01-29 08:17:43 -08:00
write_batch_internal.h In-place updates for equal keys and similar sized values 2013-10-31 11:27:12 -07:00
write_batch_test.cc Fix memtable construction in tests 2014-01-14 15:36:12 -08:00
write_batch.cc Statistics code cleanup 2014-01-17 12:46:06 -08:00