rocksdb/table
Islam AbdelRahman aececc209e Introduce ReadOptions::pin_data (support zero copy for keys)
Summary:
This patch update the Iterator API to introduce new functions that allow users to keep the Slices returned by key() valid as long as the Iterator is not deleted

ReadOptions::pin_data : If true keep loaded blocks in memory as long as the iterator is not deleted
Iterator::IsKeyPinned() : If true, this mean that the Slice returned by key() is valid as long as the iterator is not deleted

Also add a new option BlockBasedTableOptions::use_delta_encoding to allow users to disable delta_encoding if needed.

Benchmark results (using https://phabricator.fb.com/P20083553)

```
// $ du -h /home/tec/local/normal.4K.Snappy/db10077
// 6.1G    /home/tec/local/normal.4K.Snappy/db10077

// $ du -h /home/tec/local/zero.8K.LZ4/db10077
// 6.4G    /home/tec/local/zero.8K.LZ4/db10077

// Benchmarks for shard db10077
// _build/opt/rocks/benchmark/rocks_copy_benchmark \
//      --normal_db_path="/home/tec/local/normal.4K.Snappy/db10077" \
//      --zero_db_path="/home/tec/local/zero.8K.LZ4/db10077"

// First run
// ============================================================================
// rocks/benchmark/RocksCopyBenchmark.cpp          relative  time/iter  iters/s
// ============================================================================
// BM_StringCopy                                                 1.73s  576.97m
// BM_StringPiece                                   103.74%      1.67s  598.55m
// ============================================================================
// Match rate : 1000000 / 1000000

// Second run
// ============================================================================
// rocks/benchmark/RocksCopyBenchmark.cpp          relative  time/iter  iters/s
// ============================================================================
// BM_StringCopy                                              611.99ms     1.63
// BM_StringPiece                                   203.76%   300.35ms     3.33
// ============================================================================
// Match rate : 1000000 / 1000000
```

Test Plan: Unit tests

Reviewers: sdong, igor, anthony, yhchiang, rven

Reviewed By: rven

Subscribers: dhruba, lovro, adsharma

Differential Revision: https://reviews.facebook.net/D48999
2015-12-16 12:08:30 -08:00
..
adaptive_table_factory.cc Pass column family ID to table property collector 2015-10-09 14:36:51 -07:00
adaptive_table_factory.h Pass column family ID to table property collector 2015-10-09 14:36:51 -07:00
block_based_filter_block_test.cc rocksdb: switch to gtest 2015-03-17 14:08:00 -07:00
block_based_filter_block.cc bloom hit/miss stats for SST and memtable 2015-10-07 11:23:20 -07:00
block_based_filter_block.h Remember whole key/prefix filtering on/off in SST file 2015-02-11 11:20:04 -08:00
block_based_table_builder.cc Introduce ReadOptions::pin_data (support zero copy for keys) 2015-12-16 12:08:30 -08:00
block_based_table_builder.h Move skip_table_builder_flush to BlockBasedTableOption 2015-10-30 18:33:01 -07:00
block_based_table_factory.cc Fix build broken by previous commit of "option helper refactor" 2015-11-17 16:52:54 -08:00
block_based_table_factory.h Merge pull request #805 from SherlockNoMad/OptionHelperFix 2015-11-17 16:24:52 -08:00
block_based_table_reader.cc Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00
block_based_table_reader.h Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00
block_builder.cc Introduce ReadOptions::pin_data (support zero copy for keys) 2015-12-16 12:08:30 -08:00
block_builder.h Introduce ReadOptions::pin_data (support zero copy for keys) 2015-12-16 12:08:30 -08:00
block_hash_index_test.cc Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00
block_hash_index.cc Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00
block_hash_index.h Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00
block_prefix_index.cc Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
block_prefix_index.h Windows Port from Microsoft 2015-07-01 16:13:56 -07:00
block_test.cc Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00
block.cc Introduce ReadOptions::pin_data (support zero copy for keys) 2015-12-16 12:08:30 -08:00
block.h Introduce ReadOptions::pin_data (support zero copy for keys) 2015-12-16 12:08:30 -08:00
bloom_block.cc table/bloom_block.*: pass func parameter by reference 2014-09-30 23:30:31 +02:00
bloom_block.h Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
cuckoo_table_builder_test.cc Block cuckoo table tests in ROCKSDB_LITE 2015-07-20 10:50:46 -07:00
cuckoo_table_builder.cc Move rate_limiter, write buffering, most perf context instrumentation and most random kill out of Env 2015-07-17 16:58:18 -07:00
cuckoo_table_builder.h Move rate_limiter, write buffering, most perf context instrumentation and most random kill out of Env 2015-07-17 16:58:18 -07:00
cuckoo_table_factory.cc Pass column family ID to table property collector 2015-10-09 14:36:51 -07:00
cuckoo_table_factory.h Add OptionsUtil::LoadOptionsFromFile() API 2015-11-12 06:52:43 -08:00
cuckoo_table_reader_test.cc Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00
cuckoo_table_reader.cc Use SST files for Transaction conflict detection 2015-12-11 12:34:11 -08:00
cuckoo_table_reader.h Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00
filter_block.h Dump routine to BlockBasedTableReader 2014-12-23 13:24:07 -08:00
flush_block_policy.cc move block based table related options BlockBasedTableOptions 2014-08-25 14:22:05 -07:00
format.cc Add ZSTD (not final format) compression type 2015-08-28 11:01:13 -07:00
format.h Move rate_limiter, write buffering, most perf context instrumentation and most random kill out of Env 2015-07-17 16:58:18 -07:00
full_filter_block_test.cc rocksdb: switch to gtest 2015-03-17 14:08:00 -07:00
full_filter_block.cc bloom hit/miss stats for SST and memtable 2015-10-07 11:23:20 -07:00
full_filter_block.h Remember whole key/prefix filtering on/off in SST file 2015-02-11 11:20:04 -08:00
get_context.cc Use SST files for Transaction conflict detection 2015-12-11 12:34:11 -08:00
get_context.h Use SST files for Transaction conflict detection 2015-12-11 12:34:11 -08:00
internal_iterator.h Introduce ReadOptions::pin_data (support zero copy for keys) 2015-12-16 12:08:30 -08:00
iter_heap.h Replace std::priority_queue in MergingIterator with custom heap, take 2 2015-07-15 03:34:40 -07:00
iterator_wrapper.h Introduce ReadOptions::pin_data (support zero copy for keys) 2015-12-16 12:08:30 -08:00
iterator.cc Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00
merger_test.cc Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00
merger.cc Introduce ReadOptions::pin_data (support zero copy for keys) 2015-12-16 12:08:30 -08:00
merger.h Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00
meta_blocks.cc Revert "Adding new table properties" 2015-11-06 16:49:38 -08:00
meta_blocks.h Moving memtable related files from util to a new directory memtable 2015-10-16 14:10:33 -07:00
mock_table.cc Fix MockTable ID storage 2015-10-28 10:53:14 -07:00
mock_table.h Fix MockTable ID storage 2015-10-28 10:53:14 -07:00
plain_table_builder.cc Pass column family ID to table property collector 2015-10-09 14:36:51 -07:00
plain_table_builder.h Pass column family ID to table property collector 2015-10-09 14:36:51 -07:00
plain_table_factory.cc Merge pull request #805 from SherlockNoMad/OptionHelperFix 2015-11-17 16:24:52 -08:00
plain_table_factory.h Fix build broken by previous commit of "option helper refactor" 2015-11-17 16:52:54 -08:00
plain_table_index.cc "make format" against last 10 commits 2015-07-13 13:50:18 -07:00
plain_table_index.h Block plain_table_index.cc in ROCKSDB_LITE 2014-11-24 20:47:27 -08:00
plain_table_key_coding.cc Fix the bug of using freed memory introduced by recent plain table reader patch 2015-09-23 16:16:26 -07:00
plain_table_key_coding.h PlainTableReader to support non-mmap mode 2015-09-23 11:41:07 -07:00
plain_table_reader.cc plain table reader: avoid re-read the same position for index and data in non-mmap mode 2015-11-18 16:49:08 -08:00
plain_table_reader.h plain table reader: avoid re-read the same position for index and data in non-mmap mode 2015-11-18 16:49:08 -08:00
scoped_arena_iterator.h Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00
sst_file_writer.cc Move skip_table_builder_flush to BlockBasedTableOption 2015-10-30 18:33:01 -07:00
table_builder.h Move skip_table_builder_flush to BlockBasedTableOption 2015-10-30 18:33:01 -07:00
table_properties_internal.h Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00
table_properties.cc Revert "Adding new table properties" 2015-11-06 16:49:38 -08:00
table_reader_bench.cc plain table reader: avoid re-read the same position for index and data in non-mmap mode 2015-11-18 16:49:08 -08:00
table_reader.h Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00
table_test.cc Introduce ReadOptions::pin_data (support zero copy for keys) 2015-12-16 12:08:30 -08:00
two_level_iterator.cc Introduce ReadOptions::pin_data (support zero copy for keys) 2015-12-16 12:08:30 -08:00
two_level_iterator.h Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00