andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
bde7d1055f
rocksdb/table
History
Aaron Gao e532877940 Add statistics field to show total size of index and filter blocks in block cache 
Summary: With `table_options.cache_index_and_filter_blocks = true`, index and filter blocks are stored in block cache. Then people are curious how much of the block cache total size is used by indexes and bloom filters. It will be nice we have a way to report that. It can help people tune performance and plan for optimized hardware setting. We add several enum values for db Statistics. BLOCK_CACHE_INDEX/FILTER_BYTES_INSERT - BLOCK_CACHE_INDEX/FILTER_BYTES_ERASE = current INDEX/FILTER total block size in bytes.

Test Plan:
write a test case called `DBBlockCacheTest.IndexAndFilterBlocksStats`. The result is:
```
[gzh@dev9927.prn1 ~/local/rocksdb]  make db_block_cache_test -j64 && ./db_block_cache_test --gtest_filter=DBBlockCacheTest.IndexAndFilterBlocksStats
Makefile:101: Warning: Compiling in debug mode. Don't use the resulting binary in production
  GEN      util/build_version.cc
  make: `db_block_cache_test' is up to date.
  Note: Google Test filter = DBBlockCacheTest.IndexAndFilterBlocksStats
  [==========] Running 1 test from 1 test case.
  [----------] Global test environment set-up.
  [----------] 1 test from DBBlockCacheTest
  [ RUN      ] DBBlockCacheTest.IndexAndFilterBlocksStats
  [       OK ] DBBlockCacheTest.IndexAndFilterBlocksStats (689 ms)
  [----------] 1 test from DBBlockCacheTest (689 ms total)

  [----------] Global test environment tear-down
  [==========] 1 test from 1 test case ran. (689 ms total)
  [  PASSED  ] 1 test.
```

Reviewers: IslamAbdelRahman, andrewkr, sdong

Reviewed By: sdong

Subscribers: andrewkr, dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D58677
2016-06-03 10:47:47 -07:00
..
adaptive_table_factory.cc Fix null-pointer-dereference detected by Infer (https://github.com/facebook/infer) 2016-04-25 20:09:36 +01:00
adaptive_table_factory.h Pass column family ID to table property collector 2015-10-09 14:36:51 -07:00
block_based_filter_block_test.cc Add statistics field to show total size of index and filter blocks in block cache 2016-06-03 10:47:47 -07:00
block_based_filter_block.cc Add statistics field to show total size of index and filter blocks in block cache 2016-06-03 10:47:47 -07:00
block_based_filter_block.h Add statistics field to show total size of index and filter blocks in block cache 2016-06-03 10:47:47 -07:00
block_based_table_builder.cc Store SST file compression algorithm as a TableProperty 2016-05-12 09:47:16 -07:00
block_based_table_builder.h Shared dictionary compression using reference block 2016-04-27 17:36:03 -07:00
block_based_table_factory.cc Shared dictionary compression using reference block 2016-04-27 17:36:03 -07:00
block_based_table_factory.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
block_based_table_reader.cc Add statistics field to show total size of index and filter blocks in block cache 2016-06-03 10:47:47 -07:00
block_based_table_reader.h Add statistics field to show total size of index and filter blocks in block cache 2016-06-03 10:47:47 -07:00
block_builder.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
block_builder.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
block_prefix_index.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
block_prefix_index.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
block_test.cc Deprecate BlockBasedTableOptions.hash_index_allow_collision=false. 2016-05-20 17:52:27 -07:00
block.cc Deprecate BlockBasedTableOptions.hash_index_allow_collision=false. 2016-05-20 17:52:27 -07:00
block.h Deprecate BlockBasedTableOptions.hash_index_allow_collision=false. 2016-05-20 17:52:27 -07:00
bloom_block.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
bloom_block.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
cuckoo_table_builder_test.cc Embed column family name in SST file 2016-04-06 23:10:32 -07:00
cuckoo_table_builder.cc Embed column family name in SST file 2016-04-06 23:10:32 -07:00
cuckoo_table_builder.h Embed column family name in SST file 2016-04-06 23:10:32 -07:00
cuckoo_table_factory.cc Embed column family name in SST file 2016-04-06 23:10:32 -07:00
cuckoo_table_factory.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
cuckoo_table_reader_test.cc Embed column family name in SST file 2016-04-06 23:10:32 -07:00
cuckoo_table_reader.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
cuckoo_table_reader.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
filter_block.h Add statistics field to show total size of index and filter blocks in block cache 2016-06-03 10:47:47 -07:00
flush_block_policy.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
format.cc Added PersistentCache abstraction 2016-05-15 22:17:18 -07:00
format.h Added PersistentCache abstraction 2016-05-15 22:17:18 -07:00
full_filter_block_test.cc Add statistics field to show total size of index and filter blocks in block cache 2016-06-03 10:47:47 -07:00
full_filter_block.cc Add statistics field to show total size of index and filter blocks in block cache 2016-06-03 10:47:47 -07:00
full_filter_block.h Add statistics field to show total size of index and filter blocks in block cache 2016-06-03 10:47:47 -07:00
get_context.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
get_context.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
internal_iterator.h Introduce PinnedIteratorsManager (Reduce PinData() overhead / Refactor PinData) 2016-04-26 12:41:07 -07:00
iter_heap.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
iterator_wrapper.h Introduce PinnedIteratorsManager (Reduce PinData() overhead / Refactor PinData) 2016-04-26 12:41:07 -07:00
iterator.cc Rename iterator property "rocksdb.iterator.is.key.pinned" => "rocksdb.iterator.is-key-pinned" 2016-03-01 13:47:12 -08:00
merger_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
merger.cc Introduce PinnedIteratorsManager (Reduce PinData() overhead / Refactor PinData) 2016-04-26 12:41:07 -07:00
merger.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
meta_blocks.cc Added PersistentCache abstraction 2016-05-15 22:17:18 -07:00
meta_blocks.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
mock_table.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
mock_table.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
persistent_cache_helper.cc Added PersistentCache abstraction 2016-05-15 22:17:18 -07:00
persistent_cache_helper.h Added PersistentCache abstraction 2016-05-15 22:17:18 -07:00
plain_table_builder.cc Embed column family name in SST file 2016-04-06 23:10:32 -07:00
plain_table_builder.h Embed column family name in SST file 2016-04-06 23:10:32 -07:00
plain_table_factory.cc Embed column family name in SST file 2016-04-06 23:10:32 -07:00
plain_table_factory.h Fix build broken by previous commit of "option helper refactor" 2015-11-17 16:52:54 -08:00
plain_table_index.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
plain_table_index.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
plain_table_key_coding.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
plain_table_key_coding.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
plain_table_reader.cc Skip bottom-level filter block caching when hit-optimized 2015-12-23 10:15:07 -08:00
plain_table_reader.h Skip bottom-level filter block caching when hit-optimized 2015-12-23 10:15:07 -08:00
scoped_arena_iterator.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
sst_file_writer.cc [rocksdb] make more options dynamic 2016-05-17 13:11:56 -07:00
table_builder.h Shared dictionary compression using reference block 2016-04-27 17:36:03 -07:00
table_properties_internal.h Shared dictionary compression using reference block 2016-04-27 17:36:03 -07:00
table_properties.cc Store SST file compression algorithm as a TableProperty 2016-05-12 09:47:16 -07:00
table_reader_bench.cc Shared dictionary compression using reference block 2016-04-27 17:36:03 -07:00
table_reader.h Adding pin_l0_filter_and_index_blocks_in_cache feature and related fixes. 2016-04-01 10:42:39 -07:00
table_test.cc Store SST file compression algorithm as a TableProperty 2016-05-12 09:47:16 -07:00
two_level_iterator.cc Introduce PinnedIteratorsManager (Reduce PinData() overhead / Refactor PinData) 2016-04-26 12:41:07 -07:00
two_level_iterator.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00