andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
3b2a1ddd2e
rocksdb/db
History
Venkatesh Radhakrishnan 3b2a1ddd2e Add options.base_background_compactions as a number of compaction threads for low compaction debt 
Summary:
If options.base_background_compactions is given, we try to schedule number of compactions not existing this number, only when L0 files increase to certain number, or pending compaction bytes more than certain threshold, we schedule compactions based on options.max_background_compactions.

The watermarks are calculated based on slowdown thresholds.

Test Plan:
Add new test cases in column_family_test.
Adding more unit tests.

Reviewers: IslamAbdelRahman, yhchiang, kradhakrishnan, rven, anthony

Reviewed By: anthony

Subscribers: leveldb, dhruba, yoshinorim

Differential Revision: https://reviews.facebook.net/D53409
2016-01-29 16:15:53 -08:00
..
auto_roll_logger_test.cc [directory includes cleanup] Finish removing util->db dependencies 2016-01-26 10:49:24 -08:00
auto_roll_logger.cc [directory includes cleanup] Finish removing util->db dependencies 2016-01-26 10:49:24 -08:00
auto_roll_logger.h [directory includes cleanup] Finish removing util->db dependencies 2016-01-26 10:49:24 -08:00
builder.cc Use SST files for Transaction conflict detection 2015-12-11 12:34:11 -08:00
builder.h Use SST files for Transaction conflict detection 2015-12-11 12:34:11 -08:00
c_test.c Deprecate CompactionFilterV2 2015-07-17 18:59:11 +02:00
c.cc Merge pull request #916 from warrenfalk/capi_huge_page_option 2016-01-04 09:24:30 -08:00
column_family_test.cc Add options.base_background_compactions as a number of compaction threads for low compaction debt 2016-01-29 16:15:53 -08:00
column_family.cc Add options.base_background_compactions as a number of compaction threads for low compaction debt 2016-01-29 16:15:53 -08:00
column_family.h Add ColumnFamilyHandle::GetDescriptor() 2016-01-06 18:14:01 -08:00
compact_files_test.cc Fix an ASAN error in compact_files_test 2016-01-26 11:30:30 -08:00
compacted_db_impl.cc Remove db_impl_readonly dependency on utilities 2015-07-14 11:32:54 -07:00
compacted_db_impl.h Remove db_impl_readonly dependency on utilities 2015-07-14 11:32:54 -07:00
compaction_iterator_test.cc Support marking snapshots for write-conflict checking - Take 2 2015-12-08 16:47:31 -08:00
compaction_iterator.cc compaction assertion triggering test fix for sequence zeroing assertion trip 2015-12-18 16:08:31 -08:00
compaction_iterator.h Change SingleDelete to support conflict checking 2015-12-10 11:35:38 -08:00
compaction_job_stats_test.cc Fix rebase issues and new code warnings. 2015-12-11 16:56:24 -08:00
compaction_job_test.cc compaction assertion triggering test fix for sequence zeroing assertion trip 2015-12-18 16:08:31 -08:00
compaction_job.cc Add SstFileManager (component tracking all SST file in DBs and control the deletion rate) 2016-01-28 18:35:01 -08:00
compaction_job.h fix typos in comments 2015-12-11 01:54:48 +09:00
compaction_picker_test.cc Not scheduling more L1->L2 compaction if L0->L1 is pending with higher priority 2016-01-08 13:56:57 -08:00
compaction_picker.cc Not scheduling more L1->L2 compaction if L0->L1 is pending with higher priority 2016-01-08 13:56:57 -08:00
compaction_picker.h Fix minor bugs in delete operator, snprintf, and size_t usage 2015-12-15 15:26:20 -08:00
compaction.cc Merge pull request #846 from yuslepukhin/enble_c4244_lossofdata 2015-12-23 22:59:42 -08:00
compaction.h Report compaction reason in CompactionListener 2015-12-22 11:37:19 -08:00
comparator_db_test.cc Moving memtable related files from util to a new directory memtable 2015-10-16 14:10:33 -07:00
convenience.cc Delete files in given key range 2015-12-29 13:22:13 -08:00
corruption_test.cc Merge pull request #846 from yuslepukhin/enble_c4244_lossofdata 2015-12-23 22:59:42 -08:00
cuckoo_table_db_test.cc Block cuckoo table tests in ROCKSDB_LITE 2015-07-20 10:50:46 -07:00
db_bench.cc db_bench: explicitly clear buffer in compress benchmark 2016-01-19 18:11:46 -08:00
db_compaction_filter_test.cc compaction assertion triggering test fix for sequence zeroing assertion trip 2015-12-18 16:08:31 -08:00
db_compaction_test.cc Add options.base_background_compactions as a number of compaction threads for low compaction debt 2016-01-29 16:15:53 -08:00
db_dynamic_level_test.cc No need to #ifdef test only code on windows 2015-10-22 15:15:37 -07:00
db_filesnapshot.cc Add wal files to Checkpoint for multiple column families. 2015-06-19 16:08:31 -07:00
db_impl_debug.cc Running manual compactions in parallel with other automatic or manual compactions in restricted cases 2015-12-14 11:20:34 -08:00
db_impl_experimental.cc Clean up InstallSuperVersion 2015-06-17 12:37:59 -07:00
db_impl_readonly.cc Remove db_impl_readonly dependency on utilities 2015-07-14 11:32:54 -07:00
db_impl_readonly.h Override DBImplReadOnly::SyncWAL() to return NotSupported. Previously, calling it caused program abort. 2015-09-25 21:25:30 -07:00
db_impl.cc Add options.base_background_compactions as a number of compaction threads for low compaction debt 2016-01-29 16:15:53 -08:00
db_impl.h Add options.base_background_compactions as a number of compaction threads for low compaction debt 2016-01-29 16:15:53 -08:00
db_info_dumper.cc [directory includes cleanup] Finish removing util->db dependencies 2016-01-26 10:49:24 -08:00
db_info_dumper.h [directory includes cleanup] Finish removing util->db dependencies 2016-01-26 10:49:24 -08:00
db_inplace_update_test.cc Clean up dependency: Move db_test_util.* to db directory 2015-10-12 13:05:42 -07:00
db_iter_test.cc No need to #ifdef test only code on windows 2015-10-22 15:15:37 -07:00
db_iter.cc Optimize DBIter::Prev() by reducing stack overhead 2016-01-07 07:59:14 -08:00
db_iter.h Introduce ReadOptions::pin_data (support zero copy for keys) 2015-12-16 12:08:30 -08:00
db_log_iter_test.cc Fix clang warnings regarding unnecessary std::move 2015-12-24 04:10:00 +08:00
db_properties_test.cc Add named constants for remaining properties 2016-01-21 10:59:36 -08:00
db_table_properties_test.cc Avoid empty ranges vector with subsequent zero element access 2015-12-02 14:50:33 -08:00
db_tailing_iter_test.cc Fix minor bugs in delete operator, snprintf, and size_t usage 2015-12-15 15:26:20 -08:00
db_test_util.cc Add SstFileManager (component tracking all SST file in DBs and control the deletion rate) 2016-01-28 18:35:01 -08:00
db_test_util.h Add SstFileManager (component tracking all SST file in DBs and control the deletion rate) 2016-01-28 18:35:01 -08:00
db_test.cc Add SstFileManager (component tracking all SST file in DBs and control the deletion rate) 2016-01-28 18:35:01 -08:00
db_universal_compaction_test.cc Should not skip bloom filter for L0 during the query. 2016-01-27 16:16:39 -08:00
db_wal_test.cc No need to #ifdef test only code on windows 2015-10-22 15:15:37 -07:00
dbformat_test.cc Avoid manipulating const char* arrays 2015-07-14 00:21:41 -07:00
dbformat.cc Support for SingleDelete() 2015-09-17 11:42:56 -07:00
dbformat.h Introduce ReadOptions::pin_data (support zero copy for keys) 2015-12-16 12:08:30 -08:00
deletefile_test.cc Fix minor bugs in delete operator, snprintf, and size_t usage 2015-12-15 15:26:20 -08:00
event_helpers.cc Enable MS compiler warning c4244. 2015-12-11 16:47:34 -08:00
event_helpers.h Add EventListener::OnTableFileDeletion() 2015-06-03 19:57:01 -07:00
experimental.cc Implement DB::PromoteL0 method 2015-04-23 12:10:36 -07:00
fault_injection_test.cc fix sporadic failure in fault_injection_test 2016-01-06 11:51:53 -08:00
file_indexer_test.cc Fix possible SIGSEGV in CompactRange (github issue #596) 2015-04-29 10:52:31 -07:00
file_indexer.cc Fix possible SIGSEGV in CompactRange (github issue #596) 2015-04-29 10:52:31 -07:00
file_indexer.h fix typos in comments 2015-12-11 01:54:48 +09:00
filename_test.cc rocksdb: switch to gtest 2015-03-17 14:08:00 -07:00
filename.cc Enable RocksDB to persist Options file. 2015-11-10 22:58:01 -08:00
filename.h Enable RocksDB to persist Options file. 2015-11-10 22:58:01 -08:00
flush_job_test.cc Use SST files for Transaction conflict detection 2015-12-11 12:34:11 -08:00
flush_job.cc [directory includes cleanup] Remove util->db dependency for ThreadStatusUtil 2016-01-26 10:49:16 -08:00
flush_job.h Use SST files for Transaction conflict detection 2015-12-11 12:34:11 -08:00
flush_scheduler.cc support for concurrent adds to memtable 2015-12-25 11:03:40 -08:00
flush_scheduler.h support for concurrent adds to memtable 2015-12-25 11:03:40 -08:00
forward_iterator_bench.cc Block forward_iterator_bench under MAC and Windows 2015-11-17 11:51:37 -08:00
forward_iterator.cc Fix minor bugs in delete operator, snprintf, and size_t usage 2015-12-15 15:26:20 -08:00
forward_iterator.h Reuse file iterators in tailing iterator when memtable is flushed 2015-11-13 15:50:59 -08:00
inlineskiplist_test.cc support for concurrent adds to memtable 2015-12-25 11:03:40 -08:00
inlineskiplist.h support for concurrent adds to memtable 2015-12-25 11:03:40 -08:00
internal_stats.cc Updated GetProperty documentation 2016-01-25 14:04:55 -08:00
internal_stats.h support for concurrent adds to memtable 2015-12-25 11:03:40 -08:00
job_context.h fixed leaking log::Writers 2015-07-07 12:10:10 -07:00
listener_test.cc Report compaction reason in CompactionListener 2015-12-22 11:37:19 -08:00
log_format.h log_{reader,write}: recyclable record format 2015-10-19 17:24:05 -04:00
log_reader.cc log_{reader,write}: recyclable record format 2015-10-19 17:24:05 -04:00
log_reader.h log_{reader,write}: recyclable record format 2015-10-19 17:24:05 -04:00
log_test.cc log_{reader,write}: recyclable record format 2015-10-19 17:24:05 -04:00
log_writer.cc Enable C4267 warning 2015-11-24 16:33:09 +03:00
log_writer.h Enable C4267 warning 2015-11-24 16:33:09 +03:00
managed_iterator.cc [directory includes cleanup] Move cross-function test points 2016-01-26 10:49:05 -08:00
managed_iterator.h Fixed xfunc related compile errors in ROCKSDB_LITE 2015-04-09 21:05:18 -07:00
manual_compaction_test.cc Move manual_compaction_test.cc from util to db 2015-10-14 11:06:27 -07:00
memtable_allocator.cc support for concurrent adds to memtable 2015-12-25 11:03:40 -08:00
memtable_allocator.h support for concurrent adds to memtable 2015-12-25 11:03:40 -08:00
memtable_list_test.cc Removing duplicate code 2015-08-05 07:33:27 -07:00
memtable_list.cc Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00
memtable_list.h Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00
memtable.cc Optimize GetLatestSequenceForKey 2016-01-06 13:43:22 -08:00
memtable.h support for concurrent adds to memtable 2015-12-25 11:03:40 -08:00
memtablerep_bench.cc Merge pull request #811 from OverlordQ/unused-variable-warning 2015-11-02 12:44:27 -08:00
merge_context.h API to fetch from both a WriteBatchWithIndex and the db 2015-05-11 14:51:51 -07:00
merge_helper_test.cc Compaction filter on merge operands 2015-10-07 09:30:03 -07:00
merge_helper.cc Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00
merge_helper.h Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00
merge_operator.cc Call merge operators with empty values 2015-06-26 11:35:46 -07:00
merge_test.cc Fix minor bugs in delete operator, snprintf, and size_t usage 2015-12-15 15:26:20 -08:00
options_file_test.cc Fixed build failure of RocksDBLite test on options_file_test.cc 2015-11-10 23:23:36 -08:00
perf_context_test.cc [directory includes cleanup] Remove util->db dependency for ThreadStatusUtil 2016-01-26 10:49:16 -08:00
plain_table_db_test.cc Fix compile error. 2016-01-11 16:10:48 -08:00
prefix_test.cc Prefix-based iterating only shows keys in prefix 2015-11-05 13:24:05 -08:00
repair.cc Disable Visual Studio Warning C4351 2015-12-28 15:06:34 -08:00
skiplist_test.cc rocksdb: switch to gtest 2015-03-17 14:08:00 -07:00
skiplist.h Switch to thread-local random for skiplist 2015-11-09 19:25:22 -08:00
slice.cc Create an abstract interface for write batches 2015-03-17 19:23:08 -07:00
snapshot_impl.cc Support marking snapshots for write-conflict checking - Take 2 2015-12-08 16:47:31 -08:00
snapshot_impl.h Use SST files for Transaction conflict detection 2015-12-11 12:34:11 -08:00
table_cache.cc Skip bottom-level filter block caching when hit-optimized 2015-12-23 10:15:07 -08:00
table_cache.h Skip bottom-level filter block caching when hit-optimized 2015-12-23 10:15:07 -08:00
table_properties_collector_test.cc Pass column family ID to table property collector 2015-10-09 14:36:51 -07:00
table_properties_collector.cc Support for SingleDelete() 2015-09-17 11:42:56 -07:00
table_properties_collector.h Pass column family ID to table property collector 2015-10-09 14:36:51 -07:00
transaction_log_impl.cc log_reader: pass log_number and optional info_log to ctor 2015-10-18 21:24:32 -04:00
transaction_log_impl.h Move rate_limiter, write buffering, most perf context instrumentation and most random kill out of Env 2015-07-17 16:58:18 -07:00
version_builder_test.cc Add a mode to always pick the oldest file to compact for each level 2015-09-21 17:21:59 -07:00
version_builder.cc Fix minor bugs in delete operator, snprintf, and size_t usage 2015-12-15 15:26:20 -08:00
version_builder.h Log more information for the add file with overlapping range failure 2015-10-19 17:31:13 -07:00
version_edit_test.cc New Manifest format to allow customized fields in NewFile. 2015-10-08 15:51:45 -07:00
version_edit.cc New Manifest format to allow customized fields in NewFile. 2015-10-08 15:51:45 -07:00
version_edit.h Enable MS compiler warning c4244. 2015-12-11 16:47:34 -08:00
version_set_test.cc Fix minor bugs in delete operator, snprintf, and size_t usage 2015-12-15 15:26:20 -08:00
version_set.cc Should not skip bloom filter for L0 during the query. 2016-01-27 16:16:39 -08:00
version_set.h Revert D7809 2016-01-21 17:14:43 -08:00
wal_manager_test.cc Fix clang warnings regarding unnecessary std::move 2015-12-24 04:10:00 +08:00
wal_manager.cc Fix clang warnings regarding unnecessary std::move 2015-12-24 04:10:00 +08:00
wal_manager.h Fix -Wnon-virtual-dtor errors 2014-11-10 17:39:38 -05:00
write_batch_base.cc Support for SingleDelete() 2015-09-17 11:42:56 -07:00
write_batch_internal.h support for concurrent adds to memtable 2015-12-25 11:03:40 -08:00
write_batch_test.cc support for concurrent adds to memtable 2015-12-25 11:03:40 -08:00
write_batch.cc support for concurrent adds to memtable 2015-12-25 11:03:40 -08:00
write_callback_test.cc Fix compile for write_callback_test in ROCKSDB_LITE 2015-07-20 10:54:15 -07:00
write_callback.h Optimistic Transactions 2015-05-29 14:36:35 -07:00
write_controller_test.cc When slowdown is triggered, reduce the write rate 2015-12-23 11:33:15 -08:00
write_controller.cc Add options.base_background_compactions as a number of compaction threads for low compaction debt 2016-01-29 16:15:53 -08:00
write_controller.h Add options.base_background_compactions as a number of compaction threads for low compaction debt 2016-01-29 16:15:53 -08:00
write_thread.cc use -Werror=missing-field-initializers, to closer match MyRocks build 2015-12-30 14:56:18 -08:00
write_thread.h use -Werror=missing-field-initializers, to closer match MyRocks build 2015-12-30 14:56:18 -08:00
writebuffer.h support for concurrent adds to memtable 2015-12-25 11:03:40 -08:00
xfunc_test_points.cc [directory includes cleanup] Move cross-function test points 2016-01-26 10:49:05 -08:00
xfunc_test_points.h [directory includes cleanup] Move cross-function test points 2016-01-26 10:49:05 -08:00