andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
ae8e0770b4
rocksdb/db
History
Mayank Agarwal ae8e0770b4 Disallow transaction log iterator to skip sequences 
Summary:
This is expected to solve the "gaps in transaction log iterator" problem.
* After a lot of observations on the gaps on the sigmafio machines I found that it is due to a race between log reader and writer always.
* So when we drop the wormhole subscription and refresh the iterator, the gaps are not there.
* It is NOT due to some boundary or corner case left unattended in the iterator logic because I checked many instances of the gaps against their log files with ldb. The log files are NOT corrupted also.
* The solution is to not allow the iterator to read incompletely written sequences and detect gaps inside itself and invalidate it which will cause the application to refresh the iterator normally and seek to the required sequence properly.
* Thus, the iterator can at least guarantee that it will not give any gaps.

Test Plan:
* db_test based log iterator tests
* db_repl_stress
* testing on sigmafio setup to see gaps go away

Reviewers: dhruba, haobo

Reviewed By: dhruba

CC: leveldb

Differential Revision: https://reviews.facebook.net/D13593
2013-10-22 11:45:35 -07:00
..
builder.cc Universal Compaction to Have a Size Percentage Threshold To Decide Whether to Compress 2013-10-17 13:33:39 -07:00
builder.h Universal Compaction to Have a Size Percentage Threshold To Decide Whether to Compress 2013-10-17 13:33:39 -07:00
c_test.c Migrate names of properties from 'leveldb' prefix to 'rocksdb' prefix. 2013-10-06 00:14:26 -07:00
c.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
corruption_test.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
db_bench.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
db_filesnapshot.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
db_impl_readonly.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
db_impl_readonly.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
db_impl.cc Universal Compaction to Have a Size Percentage Threshold To Decide Whether to Compress 2013-10-17 13:33:39 -07:00
db_impl.h Universal Compaction to Have a Size Percentage Threshold To Decide Whether to Compress 2013-10-17 13:33:39 -07:00
db_iter.cc Fix Bug: iterator.Prev() or iterator.SeekToLast() might return the first element instead of the correct one 2013-10-17 18:33:18 -07:00
db_iter.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
db_statistics.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
db_stats_logger.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
db_test.cc Fix Bug: iterator.Prev() or iterator.SeekToLast() might return the first element instead of the correct one 2013-10-17 18:33:18 -07:00
dbformat_test.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
dbformat.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
dbformat.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
deletefile_test.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
filename_test.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
filename.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
filename.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
log_format.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
log_reader.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
log_reader.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
log_test.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
log_writer.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
log_writer.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
memtable.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
memtable.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
memtablelist.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
memtablelist.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
merge_helper.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
merge_helper.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
merge_operator.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
merge_test.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
perf_context_test.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
prefix_filter_iterator.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
repair.cc Universal Compaction to Have a Size Percentage Threshold To Decide Whether to Compress 2013-10-17 13:33:39 -07:00
skiplist_test.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
skiplist.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
snapshot.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
table_cache.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
table_cache.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
transaction_log_impl.cc Disallow transaction log iterator to skip sequences 2013-10-22 11:45:35 -07:00
transaction_log_impl.h Disallow transaction log iterator to skip sequences 2013-10-22 11:45:35 -07:00
version_edit_test.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
version_edit.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
version_edit.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
version_set_reduce_num_levels.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
version_set_test.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
version_set.cc Universal Compaction to Have a Size Percentage Threshold To Decide Whether to Compress 2013-10-17 13:33:39 -07:00
version_set.h Universal Compaction to Have a Size Percentage Threshold To Decide Whether to Compress 2013-10-17 13:33:39 -07:00
write_batch_internal.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
write_batch_test.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
write_batch.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00