andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
1b1d264342
rocksdb/include/rocksdb
History
Maysam Yabandeh caf0f53a74 Index value delta encoding (#3983) 
Summary:
Given that index value is a BlockHandle, which is basically an <offset, size> pair we can apply delta encoding on the values. The first value at each index restart interval encoded the full BlockHandle but the rest encode only the size. Refer to IndexBlockIter::DecodeCurrentValue for the detail of the encoding. This reduces the index size which helps using the  block cache more efficiently. The feature is enabled with using format_version 4.

The feature comes with a bit of cpu overhead which should be paid back by the higher cache hits due to smaller index block size.
Results with sysbench read-only using 4k blocks and using 16 index restart interval:
Format 2:
19585   rocksdb read-only range=100
Format 3:
19569   rocksdb read-only range=100
Format 4:
19352   rocksdb read-only range=100
Pull Request resolved: https://github.com/facebook/rocksdb/pull/3983

Differential Revision: D8361343

Pulled By: maysamyabandeh

fbshipit-source-id: f882ee082322acac32b0072e2bdbb0b5f854e651
2018-08-09 16:58:40 -07:00
..
utilities Add SST ingestion to ldb (#4205) 2018-08-09 14:29:11 -07:00
advanced_options.h Allow ttl to be changed dynamically (#4133) 2018-07-16 14:27:53 -07:00
c.h Remove managed iterator 2018-07-17 14:43:18 -07:00
cache.h LRUCache midpoint insertion 2018-05-24 15:57:33 -07:00
cleanable.h Fix some typos in comments and docs. 2018-03-08 10:27:25 -08:00
compaction_filter.h Comment out unused variables 2018-03-05 13:13:41 -08:00
compaction_job_stats.h Fix some typos in comments and docs. 2018-03-08 10:27:25 -08:00
comparator.h use user_key and iterate_upper_bound to determine compatibility of bloom filters (#3899) 2018-06-26 15:57:26 -07:00
convenience.h Delete files in multiple ranges at once 2018-01-30 13:56:39 -08:00
db_bench_tool.h Change RocksDB License 2017-07-15 16:11:23 -07:00
db_dump_tool.h Fix some typos in comments and docs. 2018-03-08 10:27:25 -08:00
db.h Trace and Replay for RocksDB (#3837) 2018-08-01 00:27:08 -07:00
env_encryption.h Fix some typos in comments and docs. 2018-03-08 10:27:25 -08:00
env.h In delete scheduler, before ftruncate file for slow delete, check whether there is other hard links (#4093) 2018-07-09 15:28:12 -07:00
experimental.h Change RocksDB License 2017-07-15 16:11:23 -07:00
filter_policy.h Comment out unused variables 2018-03-05 13:13:41 -08:00
flush_block_policy.h Change RocksDB License 2017-07-15 16:11:23 -07:00
iostats_context.h Change RocksDB License 2017-07-15 16:11:23 -07:00
iterator.h Change and clarify the relationship between Valid(), status() and Seek*() for all iterators. Also fix some bugs 2018-05-17 02:56:56 -07:00
ldb_tool.h Change RocksDB License 2017-07-15 16:11:23 -07:00
listener.h Improve visibility into the reasons for compaction. 2018-04-11 10:58:44 -07:00
memtablerep.h InlineSkiplist: don't decode keys unnecessarily during comparisons 2018-03-23 12:14:30 -07:00
merge_operator.h Update comments about MergeOperator::AllowSingleOperand 2018-03-27 17:13:53 -07:00
metadata.h Fix coverity issues in include/rocksdb 2017-11-02 17:56:48 -07:00
options.h Trace and Replay for RocksDB (#3837) 2018-08-01 00:27:08 -07:00
perf_context.h Improve write time breakdown stats 2018-04-23 17:58:54 -07:00
perf_level.h Change RocksDB License 2017-07-15 16:11:23 -07:00
persistent_cache.h Change RocksDB License 2017-07-15 16:11:23 -07:00
rate_limiter.h Comment out unused variables 2018-03-05 13:13:41 -08:00
slice_transform.h use user_key and iterate_upper_bound to determine compatibility of bloom filters (#3899) 2018-06-26 15:57:26 -07:00
slice.h Make rocksdb::Slice more interoperable with std::string_view (#4242) 2018-08-09 14:43:34 -07:00
snapshot.h Change RocksDB License 2017-07-15 16:11:23 -07:00
sst_dump_tool.h Change RocksDB License 2017-07-15 16:11:23 -07:00
sst_file_manager.h Expose GetTotalTrashSize in SstFileManager interface (#4206) 2018-08-04 17:57:48 -07:00
sst_file_writer.h Support range deletion tombstones in IngestExternalFile SSTs (#3778) 2018-07-13 22:43:09 -07:00
statistics.h Print histogram count and sum in statistics string 2018-05-21 11:12:47 -07:00
status.h move static msgs out of Status class (#4144) 2018-07-23 15:44:16 -07:00
table_properties.h Index value delta encoding (#3983) 2018-08-09 16:58:40 -07:00
table.h DataBlockHashIndex: Specify that DataBlockHashIndex is not yet implemented in the comment 2018-07-31 11:43:08 -07:00
thread_status.h fix ThreadStatus for bottom-pri compaction threads 2017-12-14 14:57:49 -08:00
threadpool.h Change RocksDB License 2017-07-15 16:11:23 -07:00
trace_reader_writer.h Trace and Replay for RocksDB (#3837) 2018-08-01 00:27:08 -07:00
transaction_log.h Change RocksDB License 2017-07-15 16:11:23 -07:00
types.h types: add kEntryBlobIndex for TablePropertiesCollector (#4233) 2018-08-06 18:27:44 -07:00
universal_compaction.h Change RocksDB License 2017-07-15 16:11:23 -07:00
version.h Increase version number to 5.16 (#4176) 2018-07-24 13:43:33 -07:00
wal_filter.h Comment out unused variables 2018-03-05 13:13:41 -08:00
write_batch_base.h Fix some typos in comments and docs. 2018-03-08 10:27:25 -08:00
write_batch.h WriteUnPrepared: Add support for recovering WriteUnprepared transactions (#4078) 2018-07-06 17:59:13 -07:00
write_buffer_manager.h Change RocksDB License 2017-07-15 16:11:23 -07:00