andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
e673d5d26d
rocksdb/db
History
Dhruba Borthakur e673d5d26d Do not submit multiple simultaneous seek-compaction requests. 
Summary:
The code was such that if multi-threaded-compactions as well
as seek compaction are enabled then it submits multiple
compaction request for the same range of keys. This causes
extraneous sst-files to accumulate at various levels.

Test Plan:
I am not able to write a very good unit test for this one
but can easily reproduce this bug with 'dbstress' with the
following options.

batch=1;maxk=100000000;ops=100000000;ro=0;fm=2;bpl=10485760;of=500000; wbn=3; mbc=20; mb=2097152; wbs=4194304; dds=1; sync=0;  t=32; bs=16384; cs=1048576; of=500000; ./db_stress --disable_seek_compaction=0 --mmap_read=0 --threads=$t --block_size=$bs --cache_size=$cs --open_files=$of --verify_checksum=1 --db=/data/mysql/leveldb/dbstress.dir --sync=$sync --disable_wal=1 --disable_data_sync=$dds --write_buffer_size=$wbs --target_file_size_base=$mb --target_file_size_multiplier=$fm --max_write_buffer_number=$wbn --max_background_compactions=$mbc --max_bytes_for_level_base=$bpl --reopen=$ro --ops_per_thread=$ops --max_key=$maxk --test_batches_snapshots=$batch

Reviewers: leveldb, emayanke

Reviewed By: emayanke

Differential Revision: https://reviews.facebook.net/D11055
2013-06-10 15:49:19 -07:00
..
.nfs00000000066c9ebb00000002 Enhance db_bench 2013-03-14 16:00:23 -07:00
builder.cc [Rocksdb] Measure all FSYNC/SYNC times 2013-06-05 11:06:21 -07:00
builder.h Ability to configure bufferedio-reads, filesystem-readaheads and mmap-read-write per database. 2013-03-20 23:14:03 -07:00
c_test.c Fix poor error on num_levels mismatch and few other minor improvements 2013-01-25 15:37:26 -08:00
c.cc Fix poor error on num_levels mismatch and few other minor improvements 2013-01-25 15:37:26 -08:00
corruption_test.cc [RocksDB] Fix CorruptionTest 2013-05-28 12:36:42 -07:00
db_bench.cc [Rocksdb] measure table open io in a histogram 2013-06-07 10:02:28 -07:00
db_filesnapshot.cc [Rocksdb] Log on disable/enable file deletions 2013-06-05 10:48:24 -07:00
db_impl_readonly.cc [Rocksdb] Support Merge operation in rocksdb 2013-05-03 16:59:02 -07:00
db_impl_readonly.h [Rocksdb] Support Merge operation in rocksdb 2013-05-03 16:59:02 -07:00
db_impl.cc Max_mem_compaction_level can have maximum value of num_levels-1 2013-06-09 10:38:55 -07:00
db_impl.h Very basic Multiget and simple test cases. 2013-06-05 11:22:38 -07:00
db_iter.cc Record the number of open db iterators. 2013-05-29 08:47:08 -07:00
db_iter.h [Rocksdb] Support Merge operation in rocksdb 2013-05-03 16:59:02 -07:00
db_statistics.h [RocksDB] Expose DBStatistics 2013-05-23 11:49:38 -07:00
db_stats_logger.cc remove boost 2012-09-16 19:33:43 -07:00
db_test.cc Fixed valgrind errors 2013-06-05 17:25:16 -07:00
dbformat_test.cc Fix all warnings generated by -Wall option to the compiler. 2012-11-06 14:07:31 -08:00
dbformat.cc Fix refering freed memory in earlier commit. 2013-06-10 15:08:13 -07:00
dbformat.h Fix refering freed memory in earlier commit. 2013-06-10 15:08:13 -07:00
filename_test.cc Added meta-database support. 2012-12-17 11:26:59 -08:00
filename.cc Allow the logs to be purged by TTL. 2013-02-04 19:42:40 -08:00
filename.h Added meta-database support. 2012-12-17 11:26:59 -08:00
log_file.h GetUpdatesSince API to enable replication. 2012-12-07 11:42:13 -08:00
log_format.h Fixed sign-comparison in rocksdb code-base and fixed Makefile 2013-03-19 14:35:23 -07:00
log_reader.cc Codemod NULL to nullptr 2013-02-28 18:04:58 -08:00
log_reader.h TransactionLogIter should stall at the last record. Currently it errors out 2013-03-21 15:12:35 -07:00
log_test.cc Fix more signed-unsigned comparisons 2013-03-19 17:21:36 -07:00
log_writer.cc Fix a number of object lifetime/ownership issues 2013-01-23 16:54:11 -08:00
log_writer.h Fix a number of object lifetime/ownership issues 2013-01-23 16:54:11 -08:00
memtable.cc [RocksDB] [Performance Bug] MemTable::Get Slow 2013-05-21 13:40:38 -07:00
memtable.h [Rocksdb] Support Merge operation in rocksdb 2013-05-03 16:59:02 -07:00
memtablelist.cc [Rocksdb] Support Merge operation in rocksdb 2013-05-03 16:59:02 -07:00
memtablelist.h [Rocksdb] Support Merge operation in rocksdb 2013-05-03 16:59:02 -07:00
merge_helper.cc [Rocksdb] Support Merge operation in rocksdb 2013-05-03 16:59:02 -07:00
merge_helper.h [Rocksdb] Support Merge operation in rocksdb 2013-05-03 16:59:02 -07:00
merge_test.cc [Rocksdb] Support Merge operation in rocksdb 2013-05-03 16:59:02 -07:00
repair.cc Ability to configure bufferedio-reads, filesystem-readaheads and mmap-read-write per database. 2013-03-20 23:14:03 -07:00
skiplist_test.cc Codemod NULL to nullptr 2013-02-28 18:04:58 -08:00
skiplist.h Codemod NULL to nullptr 2013-02-28 18:04:58 -08:00
snapshot.h [RocksDB] fix compaction filter trigger condition 2013-05-13 12:33:02 -07:00
table_cache.cc [Rocksdb] measure table open io in a histogram 2013-06-07 10:02:28 -07:00
table_cache.h [RocksDB] [Performance] Allow different posix advice to be applied to the same table file 2013-05-30 19:08:44 -07:00
transaction_log_iterator_impl.cc Allocate the LogReporter from heap. Summary: 2013-04-29 13:19:24 -07:00
transaction_log_iterator_impl.h Allocate the LogReporter from heap. Summary: 2013-04-29 13:19:24 -07:00
version_edit_test.cc Make some variables configurable for each db instance 2012-06-27 14:36:31 -07:00
version_edit.cc Codemod NULL to nullptr 2013-02-28 18:04:58 -08:00
version_edit.h Fix poor error on num_levels mismatch and few other minor improvements 2013-01-25 15:37:26 -08:00
version_set_reduce_num_levels.cc Fix valgrind errors in rocksdb tests: auto_roll_logger_test, reduce_levels_test 2013-03-12 16:03:16 -07:00
version_set_test.cc Codemod NULL to nullptr 2013-02-28 18:04:58 -08:00
version_set.cc Do not submit multiple simultaneous seek-compaction requests. 2013-06-10 15:49:19 -07:00
version_set.h [RocksDB] Add score column to leveldb.stats 2013-06-03 17:38:27 -07:00
write_batch_internal.h GetUpdatesSince API to enable replication. 2012-12-07 11:42:13 -08:00
write_batch_test.cc [Rocksdb] Support Merge operation in rocksdb 2013-05-03 16:59:02 -07:00
write_batch.cc [Rocksdb] Support Merge operation in rocksdb 2013-05-03 16:59:02 -07:00