andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
3ce3bb3da2
rocksdb/db
History
Islam AbdelRahman 3ce3bb3da2 Allowing L0 -> L1 trivial move on sorted data 
Summary:
This diff updates the logic of how we do trivial move, now trivial move can run on any number of files in input level as long as they are not overlapping

The conditions for trivial move have been updated

Introduced conditions:
  - Trivial move cannot happen if we have a compaction filter (except if the compaction is not manual)
  - Input level files cannot be overlapping

Removed conditions:
  - Trivial move only run when the compaction is not manual
  - Input level should can contain only 1 file

More context on what tests failed because of Trivial move
```
DBTest.CompactionsGenerateMultipleFiles
This test is expecting compaction on a file in L0 to generate multiple files in L1, this test will fail with trivial move because we end up with one file in L1
```

```
DBTest.NoSpaceCompactRange
This test expect compaction to fail when we force environment to report running out of space, of course this is not valid in trivial move situation
because trivial move does not need any extra space, and did not check for that
```

```
DBTest.DropWrites
Similar to DBTest.NoSpaceCompactRange
```

```
DBTest.DeleteObsoleteFilesPendingOutputs
This test expect that a file in L2 is deleted after it's moved to L3, this is not valid with trivial move because although the file was moved it is now used by L3
```

```
CuckooTableDBTest.CompactionIntoMultipleFiles
Same as DBTest.CompactionsGenerateMultipleFiles
```

This diff is based on a work by @sdong https://reviews.facebook.net/D34149

Test Plan: make -j64 check

Reviewers: rven, sdong, igor

Reviewed By: igor

Subscribers: yhchiang, ott, march, dhruba, sdong

Differential Revision: https://reviews.facebook.net/D34797
2015-06-04 16:51:25 -07:00
..
builder.cc Allow GetThreadList to report Flush properties. 2015-05-15 23:22:22 -07:00
builder.h Add more table properties to EventLogger 2015-05-12 15:53:55 -07:00
c_test.c Revert "Merge pull request #621 from rdallman/c-slice-parts-support" 2015-06-03 13:34:07 -04:00
c.cc Revert "Merge pull request #621 from rdallman/c-slice-parts-support" 2015-06-03 13:34:07 -04:00
column_family_test.cc Support saving history in memtable_list 2015-05-28 16:34:24 -07:00
column_family.cc Allow EventListener::OnCompactionCompleted to return CompactionJobStats. 2015-06-02 17:07:16 -07:00
column_family.h Allow EventListener::OnCompactionCompleted to return CompactionJobStats. 2015-06-02 17:07:16 -07:00
compact_files_test.cc rocksdb: switch to gtest 2015-03-17 14:08:00 -07:00
compaction_job_stats_test.cc Allowing L0 -> L1 trivial move on sorted data 2015-06-04 16:51:25 -07:00
compaction_job_test.cc Allow EventListener::OnCompactionCompleted to return CompactionJobStats. 2015-06-02 17:07:16 -07:00
compaction_job.cc Allowing L0 -> L1 trivial move on sorted data 2015-06-04 16:51:25 -07:00
compaction_job.h Allow EventListener::OnCompactionCompleted to return CompactionJobStats. 2015-06-02 17:07:16 -07:00
compaction_picker_test.cc Allowing L0 -> L1 trivial move on sorted data 2015-06-04 16:51:25 -07:00
compaction_picker.cc CompactRange skips levels 1 to base_level -1 for dynamic level base size 2015-05-18 10:54:11 -07:00
compaction_picker.h Optimize GetRange Function 2015-05-05 09:57:47 -07:00
compaction.cc Allowing L0 -> L1 trivial move on sorted data 2015-06-04 16:51:25 -07:00
compaction.h Allowing L0 -> L1 trivial move on sorted data 2015-06-04 16:51:25 -07:00
comparator_db_test.cc rocksdb: Remove #include "util/string_util.h" from util/testharness.h 2015-03-19 17:29:37 -07:00
corruption_test.cc fix typos 2015-04-25 18:14:27 +09:00
cuckoo_table_db_test.cc Allowing L0 -> L1 trivial move on sorted data 2015-06-04 16:51:25 -07:00
db_bench.cc Fix compile on darwin 2015-05-30 12:25:45 -04:00
db_filesnapshot.cc Don't delete files when column family is dropped 2015-03-19 17:04:29 -07:00
db_impl_debug.cc Allowing L0 -> L1 trivial move on sorted data 2015-06-04 16:51:25 -07:00
db_impl_experimental.cc Don't compact bottommost level in SuggestCompactRange 2015-04-29 13:35:48 -07:00
db_impl_readonly.cc Move GetThreadList() feature under Env. 2014-12-22 12:20:17 -08:00
db_impl_readonly.h Block ReadOnlyDB in ROCKSDB_LITE 2014-11-26 11:37:59 -08:00
db_impl.cc Allowing L0 -> L1 trivial move on sorted data 2015-06-04 16:51:25 -07:00
db_impl.h Allowing L0 -> L1 trivial move on sorted data 2015-06-04 16:51:25 -07:00
db_iter_test.cc rocksdb: Remove #include "util/string_util.h" from util/testharness.h 2015-03-19 17:29:37 -07:00
db_iter.cc fix typos 2015-04-25 18:14:27 +09:00
db_iter.h reduce references to cfd->options() in DBImpl 2014-09-08 15:04:34 -07:00
db_test.cc Allowing L0 -> L1 trivial move on sorted data 2015-06-04 16:51:25 -07:00
dbformat_test.cc rocksdb: switch to gtest 2015-03-17 14:08:00 -07:00
dbformat.cc Optimistic Transactions 2015-05-29 14:36:35 -07:00
dbformat.h Optimistic Transactions 2015-05-29 14:36:35 -07:00
deletefile_test.cc rocksdb: Remove #include "util/string_util.h" from util/testharness.h 2015-03-19 17:29:37 -07:00
event_helpers.cc Add EventListener::OnTableFileDeletion() 2015-06-03 19:57:01 -07:00
event_helpers.h Add EventListener::OnTableFileDeletion() 2015-06-03 19:57:01 -07:00
experimental.cc Implement DB::PromoteL0 method 2015-04-23 12:10:36 -07:00
fault_injection_test.cc fault_injection_test: add a test case to cover log syncing after a log roll 2015-04-09 16:15:42 -07:00
file_indexer_test.cc Fix possible SIGSEGV in CompactRange (github issue #596) 2015-04-29 10:52:31 -07:00
file_indexer.cc Fix possible SIGSEGV in CompactRange (github issue #596) 2015-04-29 10:52:31 -07:00
file_indexer.h Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
filename_test.cc rocksdb: switch to gtest 2015-03-17 14:08:00 -07:00
filename.cc Sync manifest file when initializing it 2015-01-22 14:32:03 -08:00
filename.h Sync manifest file when initializing it 2015-01-22 14:32:03 -08:00
flush_job_test.cc Optimistic Transactions 2015-05-29 14:36:35 -07:00
flush_job.cc Add EventListener::OnTableFileCreated() 2015-06-02 14:12:23 -07:00
flush_job.h Allow GetThreadList to report Flush properties. 2015-05-15 23:22:22 -07:00
flush_scheduler.cc Don't return (or dereference) dangling pointer 2014-10-02 14:33:16 -07:00
flush_scheduler.h Fix data race #1 2015-01-26 11:48:07 -08:00
forward_iterator.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
forward_iterator.h rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
internal_stats.cc Support saving history in memtable_list 2015-05-28 16:34:24 -07:00
internal_stats.h Support saving history in memtable_list 2015-05-28 16:34:24 -07:00
job_context.h Clean up old log files in background threads 2015-03-30 15:04:10 -04:00
listener_test.cc Fix a compile warning in listener_test.cc 2015-06-02 14:20:27 -07:00
log_format.h Some minor refactoring on the code 2014-01-02 16:32:31 -08:00
log_reader.cc rocksdb: Fixed 'Dead assignment' and 'Dead initialization' scan-build warnings 2015-02-23 14:10:09 -08:00
log_reader.h Log writer record format doc. 2015-04-07 16:25:56 -07:00
log_test.cc rocksdb: switch to gtest 2015-03-17 14:08:00 -07:00
log_writer.cc Fix comparison between signed and usigned integers 2015-05-19 10:59:30 -07:00
log_writer.h Log writer record format doc. 2015-04-07 16:25:56 -07:00
managed_iterator.cc Fix compile error on MacOS. 2015-02-24 16:24:53 -08:00
managed_iterator.h Fixed xfunc related compile errors in ROCKSDB_LITE 2015-04-09 21:05:18 -07:00
memtable_allocator.cc Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
memtable_allocator.h Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
memtable_list_test.cc Optimistic Transactions 2015-05-29 14:36:35 -07:00
memtable_list.cc Optimistic Transactions 2015-05-29 14:36:35 -07:00
memtable_list.h Optimistic Transactions 2015-05-29 14:36:35 -07:00
memtable.cc Optimistic Transactions 2015-05-29 14:36:35 -07:00
memtable.h Optimistic Transactions 2015-05-29 14:36:35 -07:00
memtablerep_bench.cc build: avoid unused-variable warning 2015-05-02 13:19:10 -07:00
merge_context.h API to fetch from both a WriteBatchWithIndex and the db 2015-05-11 14:51:51 -07:00
merge_helper.cc Allow EventListener::OnCompactionCompleted to return CompactionJobStats. 2015-06-02 17:07:16 -07:00
merge_helper.h Helper function to time Merges 2015-04-27 20:23:50 -07:00
merge_operator.cc Some small cleaning up to make some compiling environment happy 2014-03-26 18:11:41 -07:00
merge_test.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
perf_context_test.cc Makefile minor cleanup 2015-03-30 16:05:35 -04:00
plain_table_db_test.cc rocksdb: Remove #include "util/string_util.h" from util/testharness.h 2015-03-19 17:29:37 -07:00
prefix_test.cc rocksdb: Remove #include "util/string_util.h" from util/testharness.h 2015-03-19 17:29:37 -07:00
repair.cc Optimistic Transactions 2015-05-29 14:36:35 -07:00
skiplist_test.cc rocksdb: switch to gtest 2015-03-17 14:08:00 -07:00
skiplist.h Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
slice.cc Create an abstract interface for write batches 2015-03-17 19:23:08 -07:00
snapshot.h Cleanup CompactionJob 2015-05-05 19:01:12 -07:00
table_cache.cc TableMock + framework for mock classes 2014-10-28 17:52:32 -07:00
table_cache.h use GetContext to replace callback function pointer 2014-09-29 11:09:09 -07:00
table_properties_collector_test.cc A new call back to TablePropertiesCollector to allow users know the entry is add, delete or merge 2015-04-06 10:27:21 -07:00
table_properties_collector.cc A new call back to TablePropertiesCollector to allow users know the entry is add, delete or merge 2015-04-06 10:27:21 -07:00
table_properties_collector.h A new call back to TablePropertiesCollector to allow users know the entry is add, delete or merge 2015-04-06 10:27:21 -07:00
transaction_log_impl.cc Turn -Wshadow back on 2014-11-06 11:14:28 -08:00
transaction_log_impl.h rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
version_builder_test.cc Allowing L0 -> L1 trivial move on sorted data 2015-06-04 16:51:25 -07:00
version_builder.cc Fix deleting obsolete files 2015-02-06 08:44:30 -08:00
version_builder.h Move VersionBuilder logic to a separate .cc file 2014-10-31 16:34:38 -07:00
version_edit_test.cc rocksdb: switch to gtest 2015-03-17 14:08:00 -07:00
version_edit.cc Turn on -Wshadow 2014-10-31 11:59:54 -07:00
version_edit.h Add experimental API MarkForCompaction() 2015-04-17 16:44:45 -07:00
version_set_test.cc Fix level size overflow for options_.level_compaction_dynamic_level_bytes=true 2015-04-03 09:04:35 -07:00
version_set.cc Allowing L0 -> L1 trivial move on sorted data 2015-06-04 16:51:25 -07:00
version_set.h Allowing L0 -> L1 trivial move on sorted data 2015-06-04 16:51:25 -07:00
wal_manager_test.cc Fix flakiness of WalManagerTest 2015-04-13 16:15:05 -07:00
wal_manager.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
wal_manager.h Fix -Wnon-virtual-dtor errors 2014-11-10 17:39:38 -05:00
write_batch_base.cc WriteBatch.Merge w/ SliceParts support 2015-05-29 04:30:03 -07:00
write_batch_internal.h WriteBatch.Merge w/ SliceParts support 2015-05-29 04:30:03 -07:00
write_batch_test.cc Optimistic Transactions 2015-05-29 14:36:35 -07:00
write_batch.cc WriteBatch.Merge w/ SliceParts support 2015-05-29 04:30:03 -07:00
write_callback_test.cc Optimistic Transactions 2015-05-29 14:36:35 -07:00
write_callback.h Optimistic Transactions 2015-05-29 14:36:35 -07:00
write_controller_test.cc maint: use ASSERT_TRUE, not ASSERT_EQ(true; same for false 2015-04-17 14:54:17 -07:00
write_controller.cc Push- instead of pull-model for managing Write stalls 2014-09-08 11:20:25 -07:00
write_controller.h Fix #284 2014-09-13 14:14:10 -07:00
write_thread.cc Optimistic Transactions 2015-05-29 14:36:35 -07:00
write_thread.h Optimistic Transactions 2015-05-29 14:36:35 -07:00
writebuffer.h Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00