andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
afa8156af9
rocksdb/util
History
Sameet Agarwal e7c434c364 Add columnfamily option optimize_filters_for_hits to optimize for key hits only 
Summary:
    Summary:
    Added a new option to ColumnFamllyOptions  - optimize_filters_for_hits. This option can be used in the case where most
    accesses to the store are key hits and we dont need to optimize performance for key misses.
    This is useful when you have a very large database and most of your lookups succeed.  The option allows the store to
     not store and use filters in the last level (the largest level which contains data). These filters can take a large amount of
     space for large databases (in memory and on-disk). For the last level, these filters are only useful for key misses and not
     for key hits. If we are not optimizing for key misses, we can choose to not store these filters for that level.

    This option is only provided for BlockBasedTable. We skip the filters when we are compacting

Test Plan:
1. Modified db_test toalso run tests with an additonal option (skip_filters_on_last_level)
 2. Added another unit test to db_test which specifically tests that filters are being skipped

Reviewers: rven, igor, sdong

Reviewed By: sdong

Subscribers: lgalanis, yoshinorim, MarkCallaghan, rven, dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D33717
2015-02-26 16:25:56 -08:00
..
allocator.h Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
arena_test.cc Make arena use hugepage if possible 2014-11-21 14:11:40 -08:00
arena.cc Make arena use hugepage if possible 2014-11-21 14:11:40 -08:00
arena.h Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
auto_roll_logger_test.cc Fixed a bug in the test case 2015-02-23 11:59:39 -08:00
auto_roll_logger.cc Add Header to logging to capture application level information 2015-02-06 10:37:45 -08:00
auto_roll_logger.h rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
autovector_test.cc Add rocksdb::ToString() to address cases where std::to_string is not available. 2014-11-24 20:44:49 -08:00
autovector.h Replace exception by assertion in autovector 2014-12-04 11:41:56 -08:00
benchharness_test.cc Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
benchharness.cc Add rocksdb::ToString() to address cases where std::to_string is not available. 2014-11-24 20:44:49 -08:00
benchharness.h mark as unused some variables with cpp-derived names 2015-02-20 11:07:21 -08:00
bloom_test.cc Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
bloom.cc Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
build_version.h build: do not relink every single binary just for a timestamp 2015-02-19 13:11:10 -08:00
cache_bench.cc Fix -Wshadow for tools 2014-11-07 15:04:30 -08:00
cache_test.cc Fix Mac compile errors on util/cache_test.cc 2014-12-11 14:15:13 -08:00
cache.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
coding_test.cc Remove BitStream* tests 2014-08-19 09:52:54 -04:00
coding.cc Removing BitStream* functions 2014-08-19 06:48:21 -07:00
coding.h Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
comparator.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
compression.h New BlockBasedTable version -- better compressed block format 2015-01-14 16:24:24 -08:00
crc32c_test.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
crc32c.cc Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
crc32c.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
db_info_dumper.cc Fix iOS compile with -Wshorten-64-to-32 2014-11-13 14:39:30 -05:00
db_info_dumper.h Fix iOS compile with -Wshorten-64-to-32 2014-11-13 14:39:30 -05:00
dynamic_bloom_test.cc rocksdb: Fix 'Division by zero' scan-build warning 2015-02-17 17:54:02 -08:00
dynamic_bloom.cc Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
dynamic_bloom.h Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
env_hdfs.cc Add rocksdb::ToString() to address cases where std::to_string is not available. 2014-11-24 20:44:49 -08:00
env_posix.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
env_test.cc Switch logv with loglevel to virtual 2015-02-09 20:59:29 +01:00
env.cc Add Header to logging to capture application level information 2015-02-06 10:37:45 -08:00
file_util.cc Provide openable snapshots 2014-11-14 11:38:26 -08:00
file_util.h Provide openable snapshots 2014-11-14 11:38:26 -08:00
filelock_test.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
filter_policy.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
hash_cuckoo_rep.cc Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
hash_cuckoo_rep.h Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
hash_linklist_rep.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
hash_linklist_rep.h Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
hash_skiplist_rep.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
hash_skiplist_rep.h Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
hash.cc Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
hash.h Introduce GetThreadList API 2014-11-20 10:49:32 -08:00
histogram_test.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
histogram.cc Fix iOS compile with -Wshorten-64-to-32 2014-11-13 14:39:30 -05:00
histogram.h Fix iOS compile with -Wshorten-64-to-32 2014-11-13 14:39:30 -05:00
instrumented_mutex.cc Perf Context to report DB mutex waiting time 2015-02-09 17:55:12 -08:00
instrumented_mutex.h Add a counter for collecting the wait time on db mutex. 2015-02-04 21:39:45 -08:00
iostats_context_imp.h Fix ios compile 2014-08-28 12:46:05 -04:00
iostats_context.cc Fix ios compile 2014-08-28 12:46:05 -04:00
ldb_cmd_execute_result.h Revert "Fix lint errors and coding style of ldb related codes." 2014-10-31 19:22:49 -07:00
ldb_cmd.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
ldb_cmd.h rocksdb: Fix uninitialized use error 2015-02-26 14:19:51 -08:00
ldb_tool.cc Added 'dump_live_files' command to ldb tool. 2014-12-12 17:50:36 -08:00
log_buffer.cc Enlarge log size cap when printing file summary 2014-09-23 16:56:34 -07:00
log_buffer.h RocksDB on FreeBSD support 2015-02-26 15:19:17 -08:00
log_write_bench.cc Fix more gflag namespace issues 2014-05-09 08:41:02 -07:00
logging.cc remove cast, replace %llu with % PRIu64 2014-09-23 01:10:46 +08:00
logging.h remove unused variable 2014-09-21 22:20:00 +08:00
manual_compaction_test.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
memenv_test.cc Compile MemEnv with standard RocksDB library 2015-01-29 16:33:11 -08:00
memenv.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
mock_env_test.cc Add fsync / corrupt simulation to env_mem 2014-10-31 15:16:31 -07:00
mock_env.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
mock_env.h rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
murmurhash.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
murmurhash.h Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
mutable_cf_options.cc remove all remaining references to cfd->options() 2014-11-18 10:20:10 -08:00
mutable_cf_options.h remove all remaining references to cfd->options() 2014-11-18 10:20:10 -08:00
mutexlock.h Add separate Read/WriteUnlock methods in MutexRW. 2014-06-16 15:41:46 -07:00
options_builder.cc Remove the compability check on log2 OS_ANDROID as it's already blocked by ROCKSDB_LITE 2014-12-04 13:56:14 -08:00
options_helper.cc Add columnfamily option optimize_filters_for_hits to optimize for key hits only 2015-02-26 16:25:56 -08:00
options_helper.h Missing header in build on CentOS 2014-11-18 22:21:02 +01:00
options_test.cc Add columnfamily option optimize_filters_for_hits to optimize for key hits only 2015-02-26 16:25:56 -08:00
options.cc Add columnfamily option optimize_filters_for_hits to optimize for key hits only 2015-02-26 16:25:56 -08:00
perf_context_imp.h Refactor PerfStepTimer to stop on destruct 2014-09-02 12:04:22 -07:00
perf_context.cc Perf Context to report DB mutex waiting time 2015-02-09 17:55:12 -08:00
posix_logger.h rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
random.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
rate_limiter_test.cc Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
rate_limiter.cc Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
rate_limiter.h expose RateLimiter definition 2014-07-25 15:17:06 -07:00
scoped_arena_iterator.h Remove path with arena==nullptr from NewInternalIterator 2014-09-04 17:40:41 -07:00
signal_test.cc rocksdb: Fix scan-build 'Called C++ object pointer is null' and 'Dereference of null pointer' bugs 2015-02-13 15:10:47 -08:00
skiplistrep.cc Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
slice_transform_test.cc CappedFixTransform: return fixed length prefix, or full key if key is shorter than the fixed length 2015-01-30 16:04:30 -08:00
slice.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
sst_dump_test.cc Disable pre-fetching of index and filter blocks for sst_dump_tool. 2015-02-25 16:34:26 -08:00
sst_dump_tool_imp.h Disable pre-fetching of index and filter blocks for sst_dump_tool. 2015-02-25 16:34:26 -08:00
sst_dump_tool.cc Disable pre-fetching of index and filter blocks for sst_dump_tool. 2015-02-25 16:34:26 -08:00
statistics.cc Fix assert in histogramData 2015-01-23 18:10:52 -08:00
statistics.h make statistics forward-able 2014-07-28 12:10:49 -07:00
status.cc Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
stl_wrappers.h Killing Transform Rep 2013-12-03 12:42:15 -08:00
stop_watch.h Change StopWatch interface 2014-07-28 12:22:37 -07:00
string_util.cc Clean up StringSplit 2014-11-21 11:05:28 -05:00
string_util.h Add rocksdb::ToString() to address cases where std::to_string is not available. 2014-11-24 20:44:49 -08:00
sync_point.cc Fix race in sync point. 2015-02-26 15:11:50 -08:00
sync_point.h Fix ifdef NDEBUG 2014-04-17 14:29:28 -07:00
testharness.cc introduce TestMemEnv and use it in db_test 2014-10-31 15:08:10 -07:00
testharness.h mark as unused some variables with cpp-derived names 2015-02-20 11:07:21 -08:00
testutil.cc Merger test 2014-09-08 22:24:40 -07:00
testutil.h rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
thread_list_test.cc Add utility functions for interpreting ThreadStatus 2015-02-04 01:47:32 -08:00
thread_local_test.cc Turn on -Wshadow 2014-10-31 11:59:54 -07:00
thread_local.cc Use ustricter consistency in thread local operations 2015-01-27 13:56:03 -08:00
thread_local.h Improve the comment of util/thread_local.h 2014-10-21 17:28:31 -07:00
thread_operation.h Add a missing field for STATE_MUTEX_WAIT to global_state_table 2015-02-06 02:38:14 -08:00
thread_status_impl.cc Add utility functions for interpreting ThreadStatus 2015-02-04 01:47:32 -08:00
thread_status_updater_debug.cc Allow GetThreadList() to indicate a thread is doing Compaction. 2015-01-13 00:04:08 -08:00
thread_status_updater.cc Allow GetThreadList() to indicate a thread is doing Compaction. 2015-01-13 00:04:08 -08:00
thread_status_updater.h Allow GetThreadList() to indicate a thread is doing Compaction. 2015-01-13 00:04:08 -08:00
thread_status_util_debug.cc Add thread_status_util_debug.cc back 2015-02-17 11:29:36 -08:00
thread_status_util.cc Allow GetThreadList() to indicate a thread is doing Compaction. 2015-01-13 00:04:08 -08:00
thread_status_util.h Add thread_status_util_debug.cc back 2015-02-17 11:29:36 -08:00
vectorrep.cc Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
xfunc.cc Managed iterator 2015-02-18 11:49:31 -08:00
xfunc.h RocksDB on FreeBSD support 2015-02-26 15:19:17 -08:00
xxhash.cc xxHash 2014-05-01 14:09:32 -04:00
xxhash.h xxHash 2014-05-01 14:09:32 -04:00