andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
e6de02103a
rocksdb/util
History
sdong e6de02103a Add a utility function to guess optimized options based on constraints 
Summary:
Add a function GetOptions(), where based on four parameters users give: read/write amplification threshold, memory budget for mem tables and target DB size, it picks up a compaction style and parameters for them. Background threads are not touched yet.

One limit of this algorithm: since compression rate and key/value size are hard to predict, it's hard to predict level 0 file size from write buffer size. Simply make 1:1 ratio here.

Sample results: https://reviews.facebook.net/P477

Test Plan: Will add some a unit test where some sample scenarios are given and see they pick the results that make sense

Reviewers: yhchiang, dhruba, haobo, igor, ljin

Reviewed By: ljin

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D18741
2014-07-22 15:24:21 -07:00
..
arena_test.cc Arena to inline 2KB of data in it. 2014-05-14 11:49:01 -07:00
arena.cc declare kInline size in arena.cc 2014-05-14 12:40:49 -07:00
arena.h In DB::NewIterator(), try to allocate the whole iterator tree in an arena 2014-06-02 17:44:57 -07:00
auto_roll_logger_test.cc Fix a bug causing LOG is not created when max_log_file_size is set. 2014-06-16 10:27:42 -07:00
auto_roll_logger.cc Fix a bug causing LOG is not created when max_log_file_size is set. 2014-06-16 10:27:42 -07:00
auto_roll_logger.h Renamed InfoLogLevel::DEBUG to InfoLogLevel::DEBUG_LEVEL 2014-04-10 15:27:42 -07:00
autovector_test.cc autovector::resize 2014-05-08 13:50:49 -07:00
autovector.h autovector::resize 2014-05-08 13:50:49 -07:00
benchharness_test.cc Fix unsigned and signed integer compare 2014-04-29 12:29:22 -07:00
benchharness.cc Fix more signed/unsigned comparsions 2014-04-29 12:40:18 -07:00
benchharness.h log_and_apply_bench on a new benchmark framework 2014-05-05 11:11:48 -07:00
blob_store_test.cc Move the compiler back to 4.8.1 + more small fixes 2013-11-18 11:40:16 -08:00
blob_store.cc RocksDBLite 2014-04-15 13:39:26 -07:00
blob_store.h RocksDBLite 2014-04-15 13:39:26 -07:00
bloom_test.cc Fix more gflag namespace issues 2014-05-09 08:41:02 -07:00
bloom.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
build_version.h Dont build version if compiling for IOS 2014-04-11 10:54:47 -07:00
cache_test.cc ApplyToAllCacheEntries 2014-05-02 16:24:04 -04:00
cache.cc ApplyToAllCacheEntries 2014-05-02 16:24:04 -04:00
coding_test.cc Misc cleanup on performance branch 2014-01-17 14:26:29 -08:00
coding.cc Merge branch 'master' into performance 2014-01-28 10:35:55 -08:00
coding.h JSONDocument 2014-06-20 11:14:14 +02:00
comparator.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
crc32c_test.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
crc32c.cc Dynamically choose SSE 4.2 2014-04-04 14:03:19 -07:00
crc32c.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
dynamic_bloom_test.cc use arena to allocate memtable's bloomfilter and hashskiplist's buckets_ 2014-06-30 15:54:31 -07:00
dynamic_bloom.cc Adding option to save PlainTable index and bloom filter in SST file. 2014-07-18 16:58:13 -07:00
dynamic_bloom.h Adding option to save PlainTable index and bloom filter in SST file. 2014-07-18 16:58:13 -07:00
env_hdfs.cc cleanup exception text 2014-05-21 07:54:22 -04:00
env_posix.cc integrate rate limiter into rocksdb 2014-07-08 12:31:49 -07:00
env_test.cc - hdfs cleanup; fix to NewDirectory to comply with definition in env.h 2014-05-21 07:50:37 -04:00
env.cc integrate rate limiter into rocksdb 2014-07-08 12:31:49 -07:00
filelock_test.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
filter_policy.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
hash_cuckoo_rep.cc Remove MemTableRep::GetIterator(const Slice& slice) 2014-06-25 14:09:29 -07:00
hash_cuckoo_rep.h Pass logger to memtable rep and TLB page allocation error logged to info logs 2014-05-05 16:43:37 -07:00
hash_linklist_rep.cc HashLinkList memtable switches a bucket to a skip list to reduce performance outliers 2014-07-01 17:14:15 -07:00
hash_linklist_rep.h HashLinkList memtable switches a bucket to a skip list to reduce performance outliers 2014-07-01 17:14:15 -07:00
hash_skiplist_rep.cc use arena to allocate memtable's bloomfilter and hashskiplist's buckets_ 2014-06-30 15:54:31 -07:00
hash_skiplist_rep.h Pass logger to memtable rep and TLB page allocation error logged to info logs 2014-05-05 16:43:37 -07:00
hash.cc Add a hash-index component for block 2014-03-03 21:11:49 -08:00
hash.h Adding option to save PlainTable index and bloom filter in SST file. 2014-07-18 16:58:13 -07:00
histogram_test.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
histogram.cc Statistics code cleanup 2014-01-17 12:46:06 -08:00
histogram.h Statistics code cleanup 2014-01-17 12:46:06 -08:00
iostats_context_imp.h Finer report I/O stats about Flush and Compaction. 2014-07-03 16:28:03 -07:00
iostats_context.cc Finer report I/O stats about Flush and Compaction. 2014-07-03 16:28:03 -07:00
ldb_cmd_execute_result.h Revert "Don't compile ldb tool into static library" 2014-04-15 11:29:02 -07:00
ldb_cmd.cc Support multiple DB directories in universal compaction style 2014-07-15 12:06:28 -07:00
ldb_cmd.h Revert the default setting of InitFromCmdLineArgs(). 2014-06-23 18:51:00 -06:00
ldb_tool.cc [RocksDB] allow LDB tool to have customized key formatter 2014-06-23 15:35:40 -07:00
log_buffer.cc Fix data corruption by LogBuffer 2014-03-21 15:32:48 -07:00
log_buffer.h Fix errors while building with clang 2014-05-15 12:34:53 +08:00
log_write_bench.cc Fix more gflag namespace issues 2014-05-09 08:41:02 -07:00
logging.cc Clean up compaction logging 2014-05-14 12:13:50 -07:00
logging.h Clean up compaction logging 2014-05-14 12:13:50 -07:00
manual_compaction_test.cc Fix CompactRange to apply filter to every key 2014-01-14 16:19:09 -08:00
murmurhash.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
murmurhash.h Add a hash-index component for block 2014-03-03 21:11:49 -08:00
mutexlock.h Add separate Read/WriteUnlock methods in MutexRW. 2014-06-16 15:41:46 -07:00
options_builder.cc Add a utility function to guess optimized options based on constraints 2014-07-22 15:24:21 -07:00
options_test.cc Add a utility function to guess optimized options based on constraints 2014-07-22 15:24:21 -07:00
options.cc Remove stats logger 2014-07-15 09:16:32 -04:00
perf_context_imp.h macros for perf_context 2014-04-08 10:58:07 -07:00
perf_context.cc Add a function to return current perf level 2014-07-10 11:35:48 -07:00
posix_logger.h Renamed InfoLogLevel::DEBUG to InfoLogLevel::DEBUG_LEVEL 2014-04-10 15:27:42 -07:00
random.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
rate_limiter_test.cc Fix mac compile 2014-07-10 13:19:47 -07:00
rate_limiter.cc generic rate limiter 2014-07-08 11:41:57 -07:00
rate_limiter.h generic rate limiter 2014-07-08 11:41:57 -07:00
signal_test.cc Print out stack trace in mac, too 2014-04-23 09:11:35 -04:00
skiplistrep.cc Remove MemTableRep::GetIterator(const Slice& slice) 2014-06-25 14:09:29 -07:00
slice.cc FixedPrefixTransform to include prefix length in its name 2014-05-13 16:08:21 -07:00
statistics.cc convert Tickers back to array with padding and alignment 2014-01-29 15:08:41 -08:00
statistics.h cache SuperVersion in thread local storage to avoid mutex lock 2014-02-27 11:38:55 -08:00
status.cc Add timeout_hint_us to WriteOptions and introduce Status::TimeOut. 2014-07-03 15:47:02 -07:00
stl_wrappers.h Killing Transform Rep 2013-12-03 12:42:15 -08:00
stop_watch.h Add timeout_hint_us to WriteOptions and introduce Status::TimeOut. 2014-07-03 15:47:02 -07:00
string_util.cc Turn on -Wmissing-prototypes 2014-04-09 21:17:14 -07:00
string_util.h Turn on -Wmissing-prototypes 2014-04-09 21:17:14 -07:00
sync_point.cc Don't compile sync_point if NDEBUG 2014-04-17 10:49:58 -07:00
sync_point.h Fix ifdef NDEBUG 2014-04-17 14:29:28 -07:00
testharness.cc Better stack trace in MAC 2014-04-25 09:50:51 -04:00
testharness.h Print out stack trace in mac, too 2014-04-23 09:11:35 -04:00
testutil.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
testutil.h [Performance Branch] PlainTable to encode rows with seqID 0, value type using 1 internal byte. 2014-02-03 12:19:30 -08:00
thread_local_test.cc simplify ThreadLocalPtr a little bit 2014-04-22 21:13:34 -07:00
thread_local.cc simplify ThreadLocalPtr a little bit 2014-04-22 21:13:34 -07:00
thread_local.h simplify ThreadLocalPtr a little bit 2014-04-22 21:13:34 -07:00
vectorrep.cc Remove MemTableRep::GetIterator(const Slice& slice) 2014-06-25 14:09:29 -07:00
xxhash.cc xxHash 2014-05-01 14:09:32 -04:00
xxhash.h xxHash 2014-05-01 14:09:32 -04:00