Commit Graph

12 Commits

Author SHA1 Message Date
Marton Trencseni
9b51987521 Adding pin_l0_filter_and_index_blocks_in_cache feature and related fixes.
Summary:
When a block based table file is opened, if prefetch_index_and_filter is true, it will prefetch the index and filter blocks, putting them into the block cache.
What this feature adds: when a L0 block based table file is opened, if pin_l0_filter_and_index_blocks_in_cache is true in the options (and prefetch_index_and_filter is true), then the filter and index blocks aren't released back to the block cache at the end of BlockBasedTableReader::Open(). Instead the table reader takes ownership of them, hence pinning them, ie. the LRU cache will never push them out. Meanwhile in the table reader, further accesses will not hit the block cache, thus avoiding lock contention.

Test Plan:
'export TEST_TMPDIR=/dev/shm/ && DISABLE_JEMALLOC=1 OPT=-g make all valgrind_check -j32' is OK.
I didn't run the Java tests, I don't have Java set up on my devserver.

Reviewers: sdong

Reviewed By: sdong

Subscribers: andrewkr, dhruba

Differential Revision: https://reviews.facebook.net/D56133
2016-04-01 10:42:39 -07:00
sdong
b1fafcaca6 Revert "Adding pin_l0_filter_and_index_blocks_in_cache feature."
This reverts commit 522de4f59e.

It has bug of index block cleaning up.
2016-03-21 11:50:42 -07:00
Marton Trencseni
522de4f59e Adding pin_l0_filter_and_index_blocks_in_cache feature.
Summary:
When a block based table file is opened, if prefetch_index_and_filter is true, it will prefetch the index and filter blocks, putting them into the block cache.
What this feature adds: when a L0 block based table file is opened, if pin_l0_filter_and_index_blocks_in_cache is true in the options (and prefetch_index_and_filter is true), then the filter and index blocks aren't released back to the block cache at the end of BlockBasedTableReader::Open(). Instead the table reader takes ownership of them, hence pinning them, ie. the LRU cache will never push them out. Meanwhile in the table reader, further accesses will not hit the block cache, thus avoiding lock contention.
When the table reader is destroyed, it releases the pinned blocks (if there were any). This has to happen before the cache is destroyed, so I had to introduce a TableReader::Close(), to guarantee the order of destruction.

Test Plan:
Added two unit tests for this. Existing unit tests run fine (default is pin_l0_filter_and_index_blocks_in_cache=false).

DISABLE_JEMALLOC=1 OPT=-g make all valgrind_check -j32
  Mac: OK.
  Linux: with D55287 patched in it's OK.

Reviewers: sdong

Reviewed By: sdong

Subscribers: andrewkr, leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D54801
2016-03-17 22:40:01 +00:00
Baraa Hamodi
21e95811d1 Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
fyrz
908258a4f2 [RocksJava] BlockBasedTableConfig 3.10
- Added support for format version in BlockBasedTableConfig
2015-01-22 23:46:38 +01:00
fyrz
2c1bd8846f BlockBasedTableConfig & PlainTableConfig enhancements
Summary:
BlockBasedTableConfig
- ported Checksum
- ported IndexType

PlainTableConfig
- added missing options
- added EncodingType

Test Plan:
make rocksdbjava
make jtest

Differential Revision: https://reviews.facebook.net/D26595
2014-10-30 20:54:35 +01:00
fyrz
bafbc23baa Filters getting disposed by System.gc before EOL
Previous to this commit Filters passed as parameters to the
BlockTableConfig are disposed before they should be disposed.

Further Smart pointer usage was corrected.

Java holds now the smart pointer to the FilterPolicy correctly
and cares about freeing underlying c++ structures.
2014-10-15 20:59:25 +02:00
fyrz
d410b39d59 BlockBasedTableConfig Filter policy support RocksJava
As proposed by yhchiang the filter can now be set in Java for
a BlockBasedTableConfig instance.
2014-10-02 21:17:48 +02:00
Ankit Gupta
1dfb7bb980 Add block based table config options 2014-09-24 11:43:35 -07:00
Lei Jin
66f62e5c78 JNI changes corresponding to BlockBasedTableOptions migration
Summary: as title

Test Plan:
tested on my mac
make rocksdbjava
make jtest

Reviewers: sdong, igor, yhchiang

Reviewed By: yhchiang

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D21963
2014-08-25 14:22:55 -07:00
Yueh-Hsuan Chiang
80a94d0517 [Java] Fixed the Java compilation error caused by PlainTableFactory API change.
Summary: Fixed the Java compilation error caused by PlainTableFactory API change.

Test Plan:
make rocksdbjava
make jdb_bench
cd java;./jdb_bench.sh

Reviewers: sdong, ljin, ankgup87, swapnilghike, zzbennett, rsumbaly, igor

Reviewed By: igor

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D20295
2014-07-21 09:19:28 -07:00
Yueh-Hsuan Chiang
ef8b8a8ef6 [Java] Add Java bindings for memtables and sst format.
Summary:
Add Java bindings for memtables and sst format.  Specifically,
add two abstract Java classses --- MemTableConfig and SstFormatConfig.
Each MemTable / SST implementation should has its own config class
extends MemTableConfig / SstFormatConfig respectively and pass it
to Options via setMemTableConfig / setSstConfig.

Test Plan:
make rocksdbjava
make jdb_test
make jdb_bench
java/jdb_bench.sh \
  --benchmarks=fillseq,readrandom,readwhilewriting \
  --memtablerep=hash_skiplist \
  --use_plain_table=1 \
  --key_size=20 \
  --prefix_size=12 \
  --value_size=100 \
  --cache_size=17179869184 \
  --disable_wal=0 \
  --sync=0 \

Reviewers: haobo, ankgup87, sdong

Reviewed By: haobo

CC: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D17997
2014-04-21 15:40:46 -07:00