andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
e7c434c364
rocksdb/db
History
Sameet Agarwal e7c434c364 Add columnfamily option optimize_filters_for_hits to optimize for key hits only 
Summary:
    Summary:
    Added a new option to ColumnFamllyOptions  - optimize_filters_for_hits. This option can be used in the case where most
    accesses to the store are key hits and we dont need to optimize performance for key misses.
    This is useful when you have a very large database and most of your lookups succeed.  The option allows the store to
     not store and use filters in the last level (the largest level which contains data). These filters can take a large amount of
     space for large databases (in memory and on-disk). For the last level, these filters are only useful for key misses and not
     for key hits. If we are not optimizing for key misses, we can choose to not store these filters for that level.

    This option is only provided for BlockBasedTable. We skip the filters when we are compacting

Test Plan:
1. Modified db_test toalso run tests with an additonal option (skip_filters_on_last_level)
 2. Added another unit test to db_test which specifically tests that filters are being skipped

Reviewers: rven, igor, sdong

Reviewed By: sdong

Subscribers: lgalanis, yoshinorim, MarkCallaghan, rven, dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D33717
2015-02-26 16:25:56 -08:00
..
builder.cc Add columnfamily option optimize_filters_for_hits to optimize for key hits only 2015-02-26 16:25:56 -08:00
builder.h Add columnfamily option optimize_filters_for_hits to optimize for key hits only 2015-02-26 16:25:56 -08:00
c_test.c rocksdb: Fixed 'Dead assignment' and 'Dead initialization' scan-build warnings 2015-02-23 14:10:09 -08:00
c.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
column_family_test.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
column_family.cc catch config errors with L0 file count triggers 2015-02-23 16:08:27 -08:00
column_family.h rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
compaction_job_test.cc Introduce job_id for flush and compaction 2015-02-12 09:54:48 -08:00
compaction_job.cc Add columnfamily option optimize_filters_for_hits to optimize for key hits only 2015-02-26 16:25:56 -08:00
compaction_job.h Introduce job_id for flush and compaction 2015-02-12 09:54:48 -08:00
compaction_picker_test.cc Switch logv with loglevel to virtual 2015-02-09 20:59:29 +01:00
compaction_picker.cc Fix data race #3 2015-02-04 16:04:51 -08:00
compaction_picker.h rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
compaction.cc Remove Compaction::ReleaseInputs(). 2015-01-15 12:44:19 -08:00
compaction.h inputs: restore "const" attribute removed by D33759 2015-02-20 11:52:20 -08:00
comparator_db_test.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
corruption_test.cc Fail DB::Open() on WAL corruption 2015-01-05 10:26:34 -08:00
cuckoo_table_db_test.cc remove unreliable test in db/cuckoo_table_db_test.cc 2014-11-24 15:18:09 -08:00
db_bench.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
db_filesnapshot.cc Introduce job_id for flush and compaction 2015-02-12 09:54:48 -08:00
db_impl_debug.cc Add a counter for collecting the wait time on db mutex. 2015-02-04 21:39:45 -08:00
db_impl_readonly.cc Move GetThreadList() feature under Env. 2014-12-22 12:20:17 -08:00
db_impl_readonly.h Block ReadOnlyDB in ROCKSDB_LITE 2014-11-26 11:37:59 -08:00
db_impl.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
db_impl.h rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
db_iter_test.cc Divide test DBIteratorTest.DBIterator to smaller tests 2015-02-03 09:48:03 -08:00
db_iter.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
db_iter.h reduce references to cfd->options() in DBImpl 2014-09-08 15:04:34 -07:00
db_test.cc Add columnfamily option optimize_filters_for_hits to optimize for key hits only 2015-02-26 16:25:56 -08:00
dbformat_test.cc Use IterKey instead of string in Block::Iter to reduce malloc 2014-07-23 12:31:11 -07:00
dbformat.cc Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
dbformat.h rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
deletefile_test.cc Add rocksdb::ToString() to address cases where std::to_string is not available. 2014-11-24 20:44:49 -08:00
fault_injection_test.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
file_indexer_test.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
file_indexer.cc Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
file_indexer.h Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
filename_test.cc Support purging logs from separate log directory 2014-08-14 13:22:50 -07:00
filename.cc Sync manifest file when initializing it 2015-01-22 14:32:03 -08:00
filename.h Sync manifest file when initializing it 2015-01-22 14:32:03 -08:00
flush_job_test.cc Introduce job_id for flush and compaction 2015-02-12 09:54:48 -08:00
flush_job.cc Allow GetThreadList to reflect flush activity. 2015-02-17 10:13:52 -08:00
flush_job.h Add a counter for collecting the wait time on db mutex. 2015-02-04 21:39:45 -08:00
flush_scheduler.cc Don't return (or dereference) dangling pointer 2014-10-02 14:33:16 -07:00
flush_scheduler.h Fix data race #1 2015-01-26 11:48:07 -08:00
forward_iterator.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
forward_iterator.h rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
internal_stats.cc Add rocksdb.num-live-versions: number of live versions 2015-02-19 13:10:37 -08:00
internal_stats.h Add rocksdb.num-live-versions: number of live versions 2015-02-19 13:10:37 -08:00
job_context.h Introduce job_id for flush and compaction 2015-02-12 09:54:48 -08:00
listener_test.cc Fixed a compile warning in clang in db/listener_test.cc 2015-01-27 15:01:04 -08:00
log_and_apply_bench.cc Add a counter for collecting the wait time on db mutex. 2015-02-04 21:39:45 -08:00
log_format.h Some minor refactoring on the code 2014-01-02 16:32:31 -08:00
log_reader.cc rocksdb: Fixed 'Dead assignment' and 'Dead initialization' scan-build warnings 2015-02-23 14:10:09 -08:00
log_reader.h Fix UnmarkEOF for partial blocks 2014-01-27 14:49:10 -08:00
log_test.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
log_writer.cc fix erroneous assert: cast kBlockSize (of type unsigned int) to "int" 2015-02-20 11:07:17 -08:00
log_writer.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
managed_iterator.cc Fix compile error on MacOS. 2015-02-24 16:24:53 -08:00
managed_iterator.h Managed iterator 2015-02-18 11:49:31 -08:00
memtable_allocator.cc Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
memtable_allocator.h Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
memtable_list.cc Add a counter for collecting the wait time on db mutex. 2015-02-04 21:39:45 -08:00
memtable_list.h Add a counter for collecting the wait time on db mutex. 2015-02-04 21:39:45 -08:00
memtable.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
memtable.h rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
memtablerep_bench.cc Fix clang build 2015-01-13 12:27:28 -08:00
merge_context.h Enhance partial merge to support multiple arguments 2014-03-24 17:57:13 -07:00
merge_helper.cc Turn -Wshadow back on 2014-11-06 11:14:28 -08:00
merge_helper.h Fixed the crash when merge_operator is not properly set after reopen. 2014-07-30 17:24:36 -07:00
merge_operator.cc Some small cleaning up to make some compiling environment happy 2014-03-26 18:11:41 -07:00
merge_test.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
perf_context_test.cc Perf Context to report DB mutex waiting time 2015-02-09 17:55:12 -08:00
plain_table_db_test.cc Add rocksdb::ToString() to address cases where std::to_string is not available. 2014-11-24 20:44:49 -08:00
prefix_test.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
repair.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
skiplist_test.cc Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
skiplist.h Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
snapshot.h rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
table_cache.cc TableMock + framework for mock classes 2014-10-28 17:52:32 -07:00
table_cache.h use GetContext to replace callback function pointer 2014-09-29 11:09:09 -07:00
table_properties_collector_test.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
table_properties_collector.cc Add rocksdb::ToString() to address cases where std::to_string is not available. 2014-11-24 20:44:49 -08:00
table_properties_collector.h rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
transaction_log_impl.cc Turn -Wshadow back on 2014-11-06 11:14:28 -08:00
transaction_log_impl.h rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
version_builder_test.cc Fix data race #3 2015-02-04 16:04:51 -08:00
version_builder.cc Fix deleting obsolete files 2015-02-06 08:44:30 -08:00
version_builder.h Move VersionBuilder logic to a separate .cc file 2014-10-31 16:34:38 -07:00
version_edit_test.cc Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
version_edit.cc Turn on -Wshadow 2014-10-31 11:59:54 -07:00
version_edit.h Fix deleting obsolete files 2015-02-06 08:44:30 -08:00
version_set_test.cc rename FileLevel to LevelFilesBrief / unfriend CompactedDBImpl 2014-10-28 10:03:13 -07:00
version_set.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
version_set.h rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
wal_manager_test.cc Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
wal_manager.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
wal_manager.h Fix -Wnon-virtual-dtor errors 2014-11-10 17:39:38 -05:00
write_batch_internal.h remove all remaining references to cfd->options() 2014-11-18 10:20:10 -08:00
write_batch_test.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
write_batch.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
write_controller_test.cc Push- instead of pull-model for managing Write stalls 2014-09-08 11:20:25 -07:00
write_controller.cc Push- instead of pull-model for managing Write stalls 2014-09-08 11:20:25 -07:00
write_controller.h Fix #284 2014-09-13 14:14:10 -07:00
write_thread.cc WriteThread 2014-09-12 16:23:58 -07:00
write_thread.h Add a counter for collecting the wait time on db mutex. 2015-02-04 21:39:45 -08:00
writebuffer.h Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00