rocksdb/db
Igor Canadi 6fe9b57748 Refactor Recover() code
Summary:
This diff does two things:
* Rethinks how we call Recover() with read_only option. Before, we call it with pointer to memtable where we'd like to apply those changes to. This memtable is set in db_impl_readonly.cc and it's actually DBImpl::mem_. Why don't we just apply updates to mem_ right away? It seems more intuitive.
* Changes when we apply updates to manifest. Before, the process is to recover all the logs, flush it to sst files and then do one giant commit that atomically adds all recovered sst files and sets the next log number. This works good enough, but causes some small troubles for my column family approach, since I can't have one VersionEdit apply to more than single column family[1]. The change here is to commit the files recovered from logs right away. Here is the state of the world before the change:
1. Recover log 5, add new sst files to edit
2. Recover log 7, add new sst files to edit
3. Recover log 8, add new sst files to edit
4. Commit all added sst files to manifest and mark log files 5, 7 and 8 as recoverd (via SetLogNumber(9) function)
After the change, we'll do:
1. Recover log 5, commit the new sst files and set log 5 as recovered
2. Recover log 7, commit the new sst files and set log 7 as recovered
3. Recover log 8, commit the new sst files and set log 8 as recovered

The added (small) benefit is that if we fail after (2), the new recovery will only have to recover log 8. In previous case, we'll have to restart the recovery from the beginning. The bigger benefit will be to enable easier integration of multiple column families in Recovery code path.

[1] I'm happy to dicuss this decison, but I believe this is the cleanest way to go. It also makes backward compatibility much easier. We don't have a requirement of adding multiple column families atomically.

Test Plan: make check

Reviewers: dhruba, haobo, kailiu, sdong

Reviewed By: kailiu

CC: leveldb

Differential Revision: https://reviews.facebook.net/D15237
2014-01-22 10:45:26 -08:00
..
builder.cc Don't always compress L0 files written by memtable flush 2014-01-07 21:50:26 -08:00
builder.h Don't always compress L0 files written by memtable flush 2014-01-07 21:50:26 -08:00
c_test.c Rename leveldb to rocksdb in C api 2013-12-10 10:48:35 +02:00
c.cc C API: add rocksdb_env_set_high_priority_background_threads 2013-12-31 15:14:18 -08:00
compaction_picker.cc Statistics code cleanup 2014-01-17 12:46:06 -08:00
compaction_picker.h Fix SIGSEGV in compaction picker 2014-01-17 12:02:03 -08:00
compaction.cc Moving Compaction class to separate header file 2014-01-15 16:22:34 -08:00
compaction.h CompactionPicker 2014-01-16 13:03:52 -08:00
corruption_test.cc Make "Table" pluggable 2013-10-28 17:54:09 -07:00
db_bench.cc Statistics code cleanup 2014-01-17 12:46:06 -08:00
db_filesnapshot.cc Move functions from VersionSet to Version 2014-01-15 16:18:04 -08:00
db_impl_readonly.cc Refactor Recover() code 2014-01-22 10:45:26 -08:00
db_impl_readonly.h Support multi-threaded DisableFileDeletions() and EnableFileDeletions() 2014-01-02 03:33:42 -08:00
db_impl.cc Refactor Recover() code 2014-01-22 10:45:26 -08:00
db_impl.h Refactor Recover() code 2014-01-22 10:45:26 -08:00
db_iter.cc Get rid of some shared_ptrs 2013-12-03 11:17:58 -08:00
db_iter.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
db_stats_logger.cc Move functions from VersionSet to Version 2014-01-15 16:18:04 -08:00
db_test.cc Refactor Recover() code 2014-01-22 10:45:26 -08:00
dbformat_test.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
dbformat.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
dbformat.h Make "Table" pluggable 2013-10-28 17:54:09 -07:00
deletefile_test.cc Start DeleteFileTest with clean plate 2013-11-15 16:30:23 -08:00
filename_test.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
filename.cc Unify DeleteFile and DeleteWalFiles 2013-10-25 08:32:14 -07:00
filename.h Unify DeleteFile and DeleteWalFiles 2013-10-25 08:32:14 -07:00
log_format.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
log_reader.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
log_reader.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
log_test.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
log_writer.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
log_writer.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
memtable.cc Statistics code cleanup 2014-01-17 12:46:06 -08:00
memtable.h Remove the unnecessary use of shared_ptr 2014-01-15 18:22:01 -08:00
memtablelist.cc Introduce MergeContext to Lazily Initialize merge operand list 2013-12-11 11:37:28 -08:00
memtablelist.h Introduce MergeContext to Lazily Initialize merge operand list 2013-12-11 11:37:28 -08:00
merge_context.h Introduce MergeContext to Lazily Initialize merge operand list 2013-12-11 11:37:28 -08:00
merge_helper.cc Statistics code cleanup 2014-01-17 12:46:06 -08:00
merge_helper.h [RocksDB] Use raw pointer instead of shared pointer when passing Statistics object internally 2013-11-25 10:38:15 -08:00
merge_operator.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
merge_test.cc Use ASSERT_EQ() instead of assert() in merge_test 2014-01-11 09:25:47 -08:00
perf_context_test.cc Killing Transform Rep 2013-12-03 12:42:15 -08:00
prefix_filter_iterator.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
prefix_test.cc Killing Transform Rep 2013-12-03 12:42:15 -08:00
repair.cc VersionEdit not to take NumLevels() 2014-01-14 15:27:09 -08:00
simple_table_db_test.cc Statistics code cleanup 2014-01-17 12:46:06 -08:00
skiplist_test.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
skiplist.h Implementing DynamicIterator for TransformRepNoLock 2013-11-08 00:31:09 -08:00
snapshot.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
table_cache.cc Hotfix the bug in table cache's GetSliceForFileNumber 2014-01-02 10:30:42 -08:00
table_cache.h Follow-up Cleaning-up After D13521 2013-10-30 10:52:33 -07:00
table_properties_collector_test.cc Move flush_block_policy from Options to TableFactory 2013-11-19 22:00:48 -08:00
table_properties_collector.cc Improve the readability of the TableProperties::ToString() 2013-11-21 17:54:23 -08:00
table_properties_collector.h Improve the "table stats" 2013-11-19 16:29:42 -08:00
transaction_log_impl.cc Fix the string format issue 2013-11-12 21:05:39 -08:00
transaction_log_impl.h Making the transaction log iterator more robust 2013-11-04 20:49:03 -08:00
version_edit_test.cc Remove compaction pointers 2014-01-16 14:06:53 -08:00
version_edit.cc Remove compaction pointers 2014-01-16 14:06:53 -08:00
version_edit.h Remove compaction pointers 2014-01-16 14:06:53 -08:00
version_set_reduce_num_levels.cc Remove compaction pointers 2014-01-16 14:06:53 -08:00
version_set_test.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
version_set.cc Remove compaction pointers 2014-01-16 14:06:53 -08:00
version_set.h Remove compaction pointers 2014-01-16 14:06:53 -08:00
write_batch_internal.h In-place updates for equal keys and similar sized values 2013-10-31 11:27:12 -07:00
write_batch_test.cc Fix memtable construction in tests 2014-01-14 15:36:12 -08:00
write_batch.cc Statistics code cleanup 2014-01-17 12:46:06 -08:00