andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
e813f5b6d9
rocksdb/db
History
Yueh-Hsuan Chiang e813f5b6d9 Allow compaction to reclaim storage more effectively. 
Summary:
This diff allows compaction to reclaim storage more effectively.
In the current design, compactions are mainly triggered based on
the file sizes.  However, since deletion entries does not have
value, files which have many deletion entries are less likely
to be compacted.  As a result, it may took a while to make
deletion entries to be compacted.

This diff address issue by compensating the size of deletion
entries during compaction process: the size of each deletion
entry in the compaction process is augmented by 2x average
value size.  The diff applies to both leveled and universal
compacitons.

Test Plan:
develop CompactionDeletionTrigger
make db_test
./db_test

Reviewers: haobo, igor, ljin, sdong

Reviewed By: sdong

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D19029
2014-06-24 16:37:06 -06:00
..
builder.cc Refactor: group metadata needed to open an SST file to a separate copyable struct 2014-06-16 16:10:52 -07:00
builder.h Refactor: group metadata needed to open an SST file to a separate copyable struct 2014-06-16 16:10:52 -07:00
c_test.c Add a test for using compaction filters via the C API 2014-06-19 21:46:58 +12:00
c.cc Add a way to set compaction filter in the C API 2014-06-19 16:31:24 +12:00
column_family_test.cc Remove seek compaction 2014-06-20 10:23:02 +02:00
column_family.cc Include max_write_buffer_number >= 2 to SanitizeOptions. 2014-06-16 16:26:46 -07:00
column_family.h Flush stale column families less aggressively 2014-06-02 15:33:54 -07:00
compaction_picker.cc Allow compaction to reclaim storage more effectively. 2014-06-24 16:37:06 -06:00
compaction_picker.h FIFO compaction style 2014-05-21 11:43:35 -07:00
compaction.cc Refactor: group metadata needed to open an SST file to a separate copyable struct 2014-06-16 16:10:52 -07:00
compaction.h Correctly preallocate files in universal compaction 2014-06-05 13:19:35 -07:00
corruption_test.cc Fix corruption test 2014-04-24 14:56:41 -04:00
db_bench.cc Improve an assertion in RandomGenerator::Generate() in db_bench. 2014-06-24 15:29:28 -06:00
db_filesnapshot.cc Call EnableFileDeletions with false as argument 2014-05-20 14:28:51 -07:00
db_impl_debug.cc FIFO compaction style 2014-05-21 11:43:35 -07:00
db_impl_readonly.cc Remove seek compaction 2014-06-20 10:23:02 +02:00
db_impl_readonly.h [CF] Rethinking ColumnFamilyHandle and fix to dropping column families 2014-02-12 13:47:09 -08:00
db_impl.cc Remove seek compaction 2014-06-20 10:23:02 +02:00
db_impl.h bug fix: iteration over ColumnFamilySet needs to be under mutex 2014-06-19 09:31:14 -07:00
db_iter.cc Fix compile errors on Mac 2014-06-03 12:28:58 -07:00
db_iter.h In DB::NewIterator(), try to allocate the whole iterator tree in an arena 2014-06-02 17:44:57 -07:00
db_stats_logger.cc RocksDBLite 2014-04-15 13:39:26 -07:00
db_test.cc Allow compaction to reclaim storage more effectively. 2014-06-24 16:37:06 -06:00
dbformat_test.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
dbformat.cc macros for perf_context 2014-04-08 10:58:07 -07:00
dbformat.h PlainTable to encode to avoid to rewrite prefix when it is the same as the previous key 2014-06-18 20:41:52 -07:00
deletefile_test.cc Start DeleteFileTest with clean plate 2013-11-15 16:30:23 -08:00
file_indexer_test.cc Some fixes as preparation for release 2014-04-30 09:03:24 -07:00
file_indexer.cc hints for narrowing down FindFile range and avoiding checking unrelevant L0 files 2014-04-21 09:10:12 -07:00
file_indexer.h Remove TransactionLogIteratorRace when -DNDEBUG 2014-04-21 11:08:30 -07:00
filename_test.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
filename.cc fsync directory after creating current file in NewDB() 2014-05-06 17:51:33 -07:00
filename.h fsync directory after creating current file in NewDB() 2014-05-06 17:51:33 -07:00
forward_iterator.cc Refactor: group metadata needed to open an SST file to a separate copyable struct 2014-06-16 16:10:52 -07:00
forward_iterator.h Fix compile errors on Mac 2014-06-03 12:28:58 -07:00
internal_stats.cc Expose number of entries in mem tables to users 2014-04-22 22:13:21 -07:00
internal_stats.h Expose number of entries in mem tables to users 2014-04-22 22:13:21 -07:00
log_and_apply_bench.cc log_and_apply_bench on a new benchmark framework 2014-05-05 11:11:48 -07:00
log_format.h Some minor refactoring on the code 2014-01-02 16:32:31 -08:00
log_reader.cc Make Log::Reader more robust 2014-02-28 13:19:47 -08:00
log_reader.h Fix UnmarkEOF for partial blocks 2014-01-27 14:49:10 -08:00
log_test.cc Make it compile on Debian/GCC 4.7 2014-03-14 22:44:35 +00:00
log_writer.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
log_writer.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
memtable_list.cc In DB::NewIterator(), try to allocate the whole iterator tree in an arena 2014-06-02 17:44:57 -07:00
memtable_list.h In DB::NewIterator(), try to allocate the whole iterator tree in an arena 2014-06-02 17:44:57 -07:00
memtable.cc Add separate Read/WriteUnlock methods in MutexRW. 2014-06-16 15:41:46 -07:00
memtable.h In DB::NewIterator(), try to allocate the whole iterator tree in an arena 2014-06-02 17:44:57 -07:00
merge_context.h Enhance partial merge to support multiple arguments 2014-03-24 17:57:13 -07:00
merge_helper.cc Fix the bug in MergeUtil which causes mixing values of different keys. 2014-03-27 16:15:25 -07:00
merge_helper.h [rocksdb] new CompactionFilterV2 API 2014-03-24 20:47:53 -07:00
merge_operator.cc Some small cleaning up to make some compiling environment happy 2014-03-26 18:11:41 -07:00
merge_test.cc Use new DBWithTTL API in tests 2014-04-28 23:46:24 -04:00
perf_context_test.cc Missing includes 2014-03-14 13:02:20 -07:00
plain_table_db_test.cc PlainTable to encode to avoid to rewrite prefix when it is the same as the previous key 2014-06-18 20:41:52 -07:00
prefix_test.cc Make RocksDB work with newer gflags 2014-05-08 17:25:13 -07:00
repair.cc Refactor: group metadata needed to open an SST file to a separate copyable struct 2014-06-16 16:10:52 -07:00
simple_table_db_test.cc Remove seek compaction 2014-06-20 10:23:02 +02:00
skiplist_test.cc Clean up arena API 2014-01-30 22:10:10 -08:00
skiplist.h Consolidate SliceTransform object ownership 2014-03-10 12:56:46 -07:00
snapshot.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
table_cache.cc Remove seek compaction 2014-06-20 10:23:02 +02:00
table_cache.h Remove seek compaction 2014-06-20 10:23:02 +02:00
table_properties_collector_test.cc TablePropertiesCollectorFactory 2014-05-13 12:30:55 -07:00
table_properties_collector.cc Extract metaindex block from block-based table 2013-12-05 16:34:16 -08:00
table_properties_collector.h TablePropertiesCollectorFactory 2014-05-13 12:30:55 -07:00
tailing_iter.cc kill ReadOptions.prefix and .prefix_seek 2014-04-25 12:21:34 -07:00
tailing_iter.h kill ReadOptions.prefix and .prefix_seek 2014-04-25 12:21:34 -07:00
transaction_log_impl.cc Fixed a file-not-found issue when a log file is moved to archive. 2014-05-12 17:50:21 -07:00
transaction_log_impl.h RocksDBLite 2014-04-15 13:39:26 -07:00
version_edit_test.cc [CF] Dont reuse dropped column family IDs 2014-03-05 12:13:44 -08:00
version_edit.cc Refactor: group metadata needed to open an SST file to a separate copyable struct 2014-06-16 16:10:52 -07:00
version_edit.h Allow compaction to reclaim storage more effectively. 2014-06-24 16:37:06 -06:00
version_set_test.cc Refactor: group metadata needed to open an SST file to a separate copyable struct 2014-06-16 16:10:52 -07:00
version_set.cc Allow compaction to reclaim storage more effectively. 2014-06-24 16:37:06 -06:00
version_set.h Allow compaction to reclaim storage more effectively. 2014-06-24 16:37:06 -06:00
write_batch_internal.h Support for column families in TTL DB 2014-04-22 11:27:33 -07:00
write_batch_test.cc kill ReadOptions.prefix and .prefix_seek 2014-04-25 12:21:34 -07:00
write_batch.cc Support for column families in TTL DB 2014-04-22 11:27:33 -07:00