andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
b23bbaa82a
rocksdb/db
History
sdong b23bbaa82a Universal Compactions with Small Files 
Summary:
With this change, we use L1 and up to store compaction outputs in universal compaction.
The compaction pick logic stays the same. Outputs are stored in the largest "level" as possible.

If options.num_levels=1, it behaves all the same as now.

Test Plan:
1) convert most of existing unit tests for universal comapaction to include the option of one level and multiple levels.
2) add a unit test to cover parallel compaction in universal compaction and run it in one level and multiple levels
3) add unit test to migrate from multiple level setting back to one level setting
4) add a unit test to insert keys to trigger multiple rounds of compactions and verify results.

Reviewers: rven, kradhakrishnan, yhchiang, igor

Reviewed By: igor

Subscribers: meyering, leveldb, MarkCallaghan, dhruba

Differential Revision: https://reviews.facebook.net/D34539
2015-03-30 15:12:02 -07:00
..
builder.cc Add columnfamily option optimize_filters_for_hits to optimize for key hits only 2015-02-26 16:25:56 -08:00
builder.h Add columnfamily option optimize_filters_for_hits to optimize for key hits only 2015-02-26 16:25:56 -08:00
c_test.c rocksdb: Fixed 'Dead assignment' and 'Dead initialization' scan-build warnings 2015-02-23 14:10:09 -08:00
c.cc Deprecate removeScanCountLimit in NewLRUCache 2015-03-17 15:04:37 -07:00
column_family_test.cc Universal Compactions with Small Files 2015-03-30 15:12:02 -07:00
column_family.cc Universal Compactions with Small Files 2015-03-30 15:12:02 -07:00
column_family.h Don't delete files when column family is dropped 2015-03-19 17:04:29 -07:00
compact_files_test.cc rocksdb: switch to gtest 2015-03-17 14:08:00 -07:00
compaction_job_test.cc rocksdb: Remove #include "util/string_util.h" from util/testharness.h 2015-03-19 17:29:37 -07:00
compaction_job.cc Adding stats for the merge and filter operation 2015-03-24 14:42:04 -07:00
compaction_job.h Adding stats for the merge and filter operation 2015-03-24 14:42:04 -07:00
compaction_picker_test.cc rocksdb: Remove #include "util/string_util.h" from util/testharness.h 2015-03-19 17:29:37 -07:00
compaction_picker.cc Universal Compactions with Small Files 2015-03-30 15:12:02 -07:00
compaction_picker.h Universal Compactions with Small Files 2015-03-30 15:12:02 -07:00
compaction.cc options.level_compaction_dynamic_level_bytes to allow RocksDB to pick size bases of levels dynamically. 2015-03-02 22:40:41 -08:00
compaction.h options.level_compaction_dynamic_level_bytes to allow RocksDB to pick size bases of levels dynamically. 2015-03-02 22:40:41 -08:00
comparator_db_test.cc rocksdb: Remove #include "util/string_util.h" from util/testharness.h 2015-03-19 17:29:37 -07:00
corruption_test.cc rocksdb: switch to gtest 2015-03-17 14:08:00 -07:00
cuckoo_table_db_test.cc rocksdb: switch to gtest 2015-03-17 14:08:00 -07:00
db_bench.cc Add --stats_interval_seconds to db_bench 2015-03-30 12:58:32 -07:00
db_filesnapshot.cc Don't delete files when column family is dropped 2015-03-19 17:04:29 -07:00
db_impl_debug.cc Clean up old log files in background threads 2015-03-30 15:04:10 -04:00
db_impl_readonly.cc Move GetThreadList() feature under Env. 2014-12-22 12:20:17 -08:00
db_impl_readonly.h Block ReadOnlyDB in ROCKSDB_LITE 2014-11-26 11:37:59 -08:00
db_impl.cc Universal Compactions with Small Files 2015-03-30 15:12:02 -07:00
db_impl.h Clean up old log files in background threads 2015-03-30 15:04:10 -04:00
db_iter_test.cc rocksdb: Remove #include "util/string_util.h" from util/testharness.h 2015-03-19 17:29:37 -07:00
db_iter.cc Adding stats for the merge and filter operation 2015-03-24 14:42:04 -07:00
db_iter.h reduce references to cfd->options() in DBImpl 2014-09-08 15:04:34 -07:00
db_test.cc Universal Compactions with Small Files 2015-03-30 15:12:02 -07:00
dbformat_test.cc rocksdb: switch to gtest 2015-03-17 14:08:00 -07:00
dbformat.cc Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
dbformat.h rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
deletefile_test.cc rocksdb: Remove #include "util/string_util.h" from util/testharness.h 2015-03-19 17:29:37 -07:00
fault_injection_test.cc rocksdb: switch to gtest 2015-03-17 14:08:00 -07:00
file_indexer_test.cc rocksdb: switch to gtest 2015-03-17 14:08:00 -07:00
file_indexer.cc Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
file_indexer.h Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
filename_test.cc rocksdb: switch to gtest 2015-03-17 14:08:00 -07:00
filename.cc Sync manifest file when initializing it 2015-01-22 14:32:03 -08:00
filename.h Sync manifest file when initializing it 2015-01-22 14:32:03 -08:00
flush_job_test.cc rocksdb: Remove #include "util/string_util.h" from util/testharness.h 2015-03-19 17:29:37 -07:00
flush_job.cc Allow GetThreadList() to report operation stage. 2015-03-13 10:45:40 -07:00
flush_job.h Allow GetThreadList() to report operation stage. 2015-03-13 10:45:40 -07:00
flush_scheduler.cc Don't return (or dereference) dangling pointer 2014-10-02 14:33:16 -07:00
flush_scheduler.h Fix data race #1 2015-01-26 11:48:07 -08:00
forward_iterator.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
forward_iterator.h rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
internal_stats.cc Make the benchmark scripts configurable and add tests 2015-03-30 11:28:25 -07:00
internal_stats.h Add a DB Property For Number of Deletions in Memtables 2015-03-18 17:03:59 -07:00
job_context.h Clean up old log files in background threads 2015-03-30 15:04:10 -04:00
listener_test.cc rocksdb: Remove #include "util/string_util.h" from util/testharness.h 2015-03-19 17:29:37 -07:00
log_and_apply_bench.cc Add a counter for collecting the wait time on db mutex. 2015-02-04 21:39:45 -08:00
log_format.h Some minor refactoring on the code 2014-01-02 16:32:31 -08:00
log_reader.cc rocksdb: Fixed 'Dead assignment' and 'Dead initialization' scan-build warnings 2015-02-23 14:10:09 -08:00
log_reader.h Fix UnmarkEOF for partial blocks 2014-01-27 14:49:10 -08:00
log_test.cc rocksdb: switch to gtest 2015-03-17 14:08:00 -07:00
log_writer.cc fix erroneous assert: cast kBlockSize (of type unsigned int) to "int" 2015-02-20 11:07:17 -08:00
log_writer.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
managed_iterator.cc Fix compile error on MacOS. 2015-02-24 16:24:53 -08:00
managed_iterator.h Managed iterator 2015-02-18 11:49:31 -08:00
memtable_allocator.cc Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
memtable_allocator.h Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
memtable_list.cc Add a DB Property For Number of Deletions in Memtables 2015-03-18 17:03:59 -07:00
memtable_list.h Add a DB Property For Number of Deletions in Memtables 2015-03-18 17:03:59 -07:00
memtable.cc Adding stats for the merge and filter operation 2015-03-24 14:42:04 -07:00
memtable.h Adding stats for the merge and filter operation 2015-03-24 14:42:04 -07:00
memtablerep_bench.cc Fix clang build 2015-01-13 12:27:28 -08:00
merge_context.h Enhance partial merge to support multiple arguments 2014-03-24 17:57:13 -07:00
merge_helper.cc Adding stats for the merge and filter operation 2015-03-24 14:42:04 -07:00
merge_helper.h Adding stats for the merge and filter operation 2015-03-24 14:42:04 -07:00
merge_operator.cc Some small cleaning up to make some compiling environment happy 2014-03-26 18:11:41 -07:00
merge_test.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
perf_context_test.cc Makefile minor cleanup 2015-03-30 16:05:35 -04:00
plain_table_db_test.cc rocksdb: Remove #include "util/string_util.h" from util/testharness.h 2015-03-19 17:29:37 -07:00
prefix_test.cc rocksdb: Remove #include "util/string_util.h" from util/testharness.h 2015-03-19 17:29:37 -07:00
repair.cc Deprecate removeScanCountLimit in NewLRUCache 2015-03-17 15:04:37 -07:00
skiplist_test.cc rocksdb: switch to gtest 2015-03-17 14:08:00 -07:00
skiplist.h Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
slice.cc Create an abstract interface for write batches 2015-03-17 19:23:08 -07:00
snapshot.h rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
table_cache.cc TableMock + framework for mock classes 2014-10-28 17:52:32 -07:00
table_cache.h use GetContext to replace callback function pointer 2014-09-29 11:09:09 -07:00
table_properties_collector_test.cc rocksdb: switch to gtest 2015-03-17 14:08:00 -07:00
table_properties_collector.cc Add rocksdb::ToString() to address cases where std::to_string is not available. 2014-11-24 20:44:49 -08:00
table_properties_collector.h rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
transaction_log_impl.cc Turn -Wshadow back on 2014-11-06 11:14:28 -08:00
transaction_log_impl.h rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
version_builder_test.cc rocksdb: Remove #include "util/string_util.h" from util/testharness.h 2015-03-19 17:29:37 -07:00
version_builder.cc Fix deleting obsolete files 2015-02-06 08:44:30 -08:00
version_builder.h Move VersionBuilder logic to a separate .cc file 2014-10-31 16:34:38 -07:00
version_edit_test.cc rocksdb: switch to gtest 2015-03-17 14:08:00 -07:00
version_edit.cc Turn on -Wshadow 2014-10-31 11:59:54 -07:00
version_edit.h Fix deleting obsolete files 2015-02-06 08:44:30 -08:00
version_set_test.cc rocksdb: switch to gtest 2015-03-17 14:08:00 -07:00
version_set.cc Universal Compactions with Small Files 2015-03-30 15:12:02 -07:00
version_set.h Universal Compactions with Small Files 2015-03-30 15:12:02 -07:00
wal_manager_test.cc rocksdb: Remove #include "util/string_util.h" from util/testharness.h 2015-03-19 17:29:37 -07:00
wal_manager.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
wal_manager.h Fix -Wnon-virtual-dtor errors 2014-11-10 17:39:38 -05:00
write_batch_base.cc Create an abstract interface for write batches 2015-03-17 19:23:08 -07:00
write_batch_internal.h remove all remaining references to cfd->options() 2014-11-18 10:20:10 -08:00
write_batch_test.cc rocksdb: Remove #include "util/string_util.h" from util/testharness.h 2015-03-19 17:29:37 -07:00
write_batch.cc Adding stats for the merge and filter operation 2015-03-24 14:42:04 -07:00
write_controller_test.cc rocksdb: switch to gtest 2015-03-17 14:08:00 -07:00
write_controller.cc Push- instead of pull-model for managing Write stalls 2014-09-08 11:20:25 -07:00
write_controller.h Fix #284 2014-09-13 14:14:10 -07:00
write_thread.cc WriteThread 2014-09-12 16:23:58 -07:00
write_thread.h Add a counter for collecting the wait time on db mutex. 2015-02-04 21:39:45 -08:00
writebuffer.h Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00