andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
54db29b8f0
rocksdb/db
History
Aaron Gao e532877940 Add statistics field to show total size of index and filter blocks in block cache 
Summary: With `table_options.cache_index_and_filter_blocks = true`, index and filter blocks are stored in block cache. Then people are curious how much of the block cache total size is used by indexes and bloom filters. It will be nice we have a way to report that. It can help people tune performance and plan for optimized hardware setting. We add several enum values for db Statistics. BLOCK_CACHE_INDEX/FILTER_BYTES_INSERT - BLOCK_CACHE_INDEX/FILTER_BYTES_ERASE = current INDEX/FILTER total block size in bytes.

Test Plan:
write a test case called `DBBlockCacheTest.IndexAndFilterBlocksStats`. The result is:
```
[gzh@dev9927.prn1 ~/local/rocksdb]  make db_block_cache_test -j64 && ./db_block_cache_test --gtest_filter=DBBlockCacheTest.IndexAndFilterBlocksStats
Makefile:101: Warning: Compiling in debug mode. Don't use the resulting binary in production
  GEN      util/build_version.cc
  make: `db_block_cache_test' is up to date.
  Note: Google Test filter = DBBlockCacheTest.IndexAndFilterBlocksStats
  [==========] Running 1 test from 1 test case.
  [----------] Global test environment set-up.
  [----------] 1 test from DBBlockCacheTest
  [ RUN      ] DBBlockCacheTest.IndexAndFilterBlocksStats
  [       OK ] DBBlockCacheTest.IndexAndFilterBlocksStats (689 ms)
  [----------] 1 test from DBBlockCacheTest (689 ms total)

  [----------] Global test environment tear-down
  [==========] 1 test from 1 test case ran. (689 ms total)
  [  PASSED  ] 1 test.
```

Reviewers: IslamAbdelRahman, andrewkr, sdong

Reviewed By: sdong

Subscribers: andrewkr, dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D58677
2016-06-03 10:47:47 -07:00
..
auto_roll_logger_test.cc Eliminate use of 'using namespace std'. Also remove a number of ADL references to std functions. 2016-05-20 07:42:18 -07:00
auto_roll_logger.cc Eliminate use of 'using namespace std'. Also remove a number of ADL references to std functions. 2016-05-20 07:42:18 -07:00
auto_roll_logger.h Redo SyncPoints for flush while rolling test 2016-02-22 21:32:19 -08:00
builder.cc [rocksdb] make more options dynamic 2016-05-17 13:11:56 -07:00
builder.h [rocksdb] make more options dynamic 2016-05-17 13:11:56 -07:00
c_test.c Shared dictionary compression using reference block 2016-04-27 17:36:03 -07:00
c.cc allow updating block cache capacity from C (#1149) 2016-06-03 14:04:51 +01:00
column_family_test.cc Eliminate use of 'using namespace std'. Also remove a number of ADL references to std functions. 2016-05-20 07:42:18 -07:00
column_family.cc Fix #1110, 32-bit build failure on Mac OSX (#1112) 2016-05-02 10:04:37 -07:00
column_family.h Fix calling GetCurrentMutableCFOptions in CompactionJob::ProcessKeyValueCompaction() 2016-04-29 17:00:50 -07:00
compact_files_test.cc Fixed CompactFiles() spuriously failing or corrupting DB 2016-02-22 13:54:58 -08:00
compacted_db_impl.cc CompactedDB should not be used if there is outstanding WAL files 2016-04-26 14:22:07 -07:00
compacted_db_impl.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
compaction_iterator_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
compaction_iterator.cc Fixing snapshot 0 assertion 2016-04-16 01:47:15 -07:00
compaction_iterator.h Fix data race issue when sub-compaction is used in CompactionJob 2016-03-24 19:36:39 -07:00
compaction_job_stats_test.cc Introduce XPRESS compresssion on Windows. (#1081) 2016-04-19 22:54:24 -07:00
compaction_job_test.cc Introduce SstFileManager::SetMaxAllowedSpaceUsage() to cap disk space usage 2016-02-17 15:20:23 -08:00
compaction_job.cc [rocksdb] make more options dynamic 2016-05-17 13:11:56 -07:00
compaction_job.h [rocksdb] make more options dynamic 2016-05-17 13:11:56 -07:00
compaction_picker_test.cc Clean up the ComputeCompactionScore() API 2016-05-23 15:55:29 -07:00
compaction_picker.cc Clean up the ComputeCompactionScore() API 2016-05-23 15:55:29 -07:00
compaction_picker.h [rocksdb] make more options dynamic 2016-05-17 13:11:56 -07:00
compaction.cc [rocksdb] make more options dynamic 2016-05-17 13:11:56 -07:00
compaction.h Merge pull request #1101 from flyd1005/wip-fix-typo 2016-04-28 02:30:44 -07:00
comparator_db_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
convenience.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
corruption_test.cc Change several option defaults 2016-04-28 17:50:58 -07:00
cuckoo_table_db_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
db_block_cache_test.cc Add statistics field to show total size of index and filter blocks in block cache 2016-06-03 10:47:47 -07:00
db_bloom_filter_test.cc BlockBasedTable::Get() not to use prefix bloom if read_options.total_order_seek = true 2016-05-06 10:16:11 -07:00
db_compaction_filter_test.cc Change some RocksDB default options 2016-03-31 17:12:18 -07:00
db_compaction_test.cc Change several option defaults 2016-04-28 17:50:58 -07:00
db_dynamic_level_test.cc Fix flaky DBTestDynamicLevel.DynamicLevelMaxBytesBase2 2016-05-26 10:13:24 -07:00
db_filesnapshot.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
db_impl_debug.cc [rocksdb] make more options dynamic 2016-05-17 13:11:56 -07:00
db_impl_experimental.cc Clean up the ComputeCompactionScore() API 2016-05-23 15:55:29 -07:00
db_impl_readonly.cc Add Iterator Property rocksdb.iterator.version_number 2016-03-02 16:23:59 -08:00
db_impl_readonly.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
db_impl.cc Fix race condition in SwitchMemtable 2016-06-02 17:11:45 -07:00
db_impl.h Add a callback for when memtable is moved to immutable (#1137) 2016-06-02 11:57:31 -07:00
db_info_dumper.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
db_info_dumper.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
db_inplace_update_test.cc Change some RocksDB default options 2016-03-31 17:12:18 -07:00
db_io_failure_test.cc Move IO failure test to separate file 2016-05-18 17:09:20 -07:00
db_iter_test.cc Add Iterator Property rocksdb.iterator.version_number 2016-03-02 16:23:59 -08:00
db_iter.cc Fix Iterator::Prev memory pinning bug 2016-05-03 16:50:01 -07:00
db_iter.h Introduce PinnedIteratorsManager (Reduce PinData() overhead / Refactor PinData) 2016-04-26 12:41:07 -07:00
db_iterator_test.cc Fix win build 2016-05-09 11:52:28 -07:00
db_log_iter_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
db_properties_test.cc Improve BytewiseComparatorImpl::FindShortestSeparator 2016-04-25 23:02:14 -07:00
db_sst_test.cc [rocksdb] make more options dynamic 2016-05-17 13:11:56 -07:00
db_table_properties_test.cc Embed column family name in SST file 2016-04-06 23:10:32 -07:00
db_tailing_iter_test.cc Change Property name from "rocksdb.current_version_number" to "rocksdb.current-super-version-number" 2016-03-04 18:15:29 -08:00
db_test2.cc Fix mutex unlock issue between scheduled compaction and ReleaseCompactionFiles() 2016-05-18 14:56:30 -07:00
db_test_util.cc Change several option defaults 2016-04-28 17:50:58 -07:00
db_test_util.h Fix win build 2016-05-09 11:52:28 -07:00
db_test.cc Eliminate use of 'using namespace std'. Also remove a number of ADL references to std functions. 2016-05-20 07:42:18 -07:00
db_universal_compaction_test.cc Alpine Linux Build (#990) 2016-04-22 16:49:12 -07:00
db_wal_test.cc Introduce XPRESS compresssion on Windows. (#1081) 2016-04-19 22:54:24 -07:00
dbformat_test.cc Improve BytewiseComparatorImpl::FindShortestSeparator 2016-04-25 23:02:14 -07:00
dbformat.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
dbformat.h Modification of WriteBatch to support two phase commit 2016-05-10 14:06:07 -07:00
deletefile_test.cc Handle concurrent manifest update and backup creation 2016-02-29 12:56:55 -08:00
event_helpers.cc Added EventListener::OnTableFileCreationStarted() callback 2016-04-29 11:35:00 -07:00
event_helpers.h Added EventListener::OnTableFileCreationStarted() callback 2016-04-29 11:35:00 -07:00
experimental.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
fault_injection_test.cc [rocksdb] Recovery path sequence miscount fix 2016-05-10 14:06:07 -07:00
file_indexer_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
file_indexer.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
file_indexer.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
filename_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
filename.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
filename.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
flush_job_test.cc Rename options.compaction_measure_io_stats to options.report_bg_io_stats and include flush too. 2016-04-15 10:22:18 -07:00
flush_job.cc Small tweaks to logging to track the number of immutable memtables 2016-06-01 11:11:33 -07:00
flush_job.h Rename options.compaction_measure_io_stats to options.report_bg_io_stats and include flush too. 2016-04-15 10:22:18 -07:00
flush_scheduler.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
flush_scheduler.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
forward_iterator_bench.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
forward_iterator.cc Change Property name from "rocksdb.current_version_number" to "rocksdb.current-super-version-number" 2016-03-04 18:15:29 -08:00
forward_iterator.h Add Iterator Property rocksdb.iterator.version_number 2016-03-02 16:23:59 -08:00
inlineskiplist_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
inlineskiplist.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
internal_stats.cc Shouldn't report default column family's compaction stats as DB compaction stats 2016-04-25 13:56:59 -07:00
internal_stats.h Shouldn't report default column family's compaction stats as DB compaction stats 2016-04-25 13:56:59 -07:00
job_context.h Cleanup stale manifests outside of full purge 2016-03-10 18:16:21 -08:00
listener_test.cc Add a callback for when memtable is moved to immutable (#1137) 2016-06-02 11:57:31 -07:00
log_format.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
log_reader.cc db/log_reader: combine kBadRecord{Len,Checksum} for readability 2016-05-22 22:00:15 -07:00
log_reader.h db/log_reader: treat bad record length or checksum as EOF 2016-05-22 22:00:15 -07:00
log_test.cc db/log_reader: treat bad record length or checksum as EOF 2016-05-22 22:00:15 -07:00
log_writer.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
log_writer.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
managed_iterator.cc Fixes warnings and ensure correct int behavior on 32-bit platforms. 2016-03-16 22:57:57 +01:00
managed_iterator.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
manual_compaction_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
memtable_allocator.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
memtable_allocator.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
memtable_list_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
memtable_list.cc [rocksdb] Memtable Log Referencing and Prepared Batch Recovery 2016-05-10 14:06:07 -07:00
memtable_list.h [rocksdb] Memtable Log Referencing and Prepared Batch Recovery 2016-05-10 14:06:07 -07:00
memtable.cc [rocksdb] Memtable Log Referencing and Prepared Batch Recovery 2016-05-10 14:06:07 -07:00
memtable.h [rocksdb] Memtable Log Referencing and Prepared Batch Recovery 2016-05-10 14:06:07 -07:00
memtablerep_bench.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
merge_context.h Eliminate std::deque initialization while iterating over merge operands 2016-04-01 15:48:55 -07:00
merge_helper_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
merge_helper.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
merge_helper.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
merge_operator.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
merge_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
options_file_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
perf_context_test.cc CompactedDB should not be used if there is outstanding WAL files 2016-04-26 14:22:07 -07:00
pinned_iterators_manager.h Introduce PinnedIteratorsManager (Reduce PinData() overhead / Refactor PinData) 2016-04-26 12:41:07 -07:00
plain_table_db_test.cc Embed column family name in SST file 2016-04-06 23:10:32 -07:00
prefix_test.cc Use prefix_same_as_start to avoid iteration in FindNextUserEntryInternal. (#1102) 2016-04-28 16:48:03 -07:00
repair_test.cc Add unit tests for RepairDB 2016-03-18 15:18:42 -07:00
repair.cc [rocksdb] make more options dynamic 2016-05-17 13:11:56 -07:00
skiplist_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
skiplist.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
slice.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
snapshot_impl.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
snapshot_impl.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
table_cache.cc Enable configurable readahead for iterators 2016-05-04 15:25:58 -07:00
table_cache.h Enable configurable readahead for iterators 2016-05-04 15:25:58 -07:00
table_properties_collector_test.cc Added "number of merge operands" to statistics in ssts. 2016-05-19 14:24:48 -07:00
table_properties_collector.cc Added "number of merge operands" to statistics in ssts. 2016-05-19 14:24:48 -07:00
table_properties_collector.h Added "number of merge operands" to statistics in ssts. 2016-05-19 14:24:48 -07:00
transaction_log_impl.cc Eliminate use of 'using namespace std'. Also remove a number of ADL references to std functions. 2016-05-20 07:42:18 -07:00
transaction_log_impl.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
version_builder_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
version_builder.cc Adding pin_l0_filter_and_index_blocks_in_cache feature and related fixes. 2016-04-01 10:42:39 -07:00
version_builder.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
version_edit_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
version_edit.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
version_edit.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
version_set_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
version_set.cc Clean up the ComputeCompactionScore() API 2016-05-23 15:55:29 -07:00
version_set.h Clean up the ComputeCompactionScore() API 2016-05-23 15:55:29 -07:00
wal_manager_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
wal_manager.cc In all the places where log records are read, there was a check that 2016-03-30 23:05:22 +05:30
wal_manager.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
write_batch_base.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
write_batch_internal.h [rocksdb] Recovery path sequence miscount fix 2016-05-10 14:06:07 -07:00
write_batch_test.cc Modification of WriteBatch to support two phase commit 2016-05-10 14:06:07 -07:00
write_batch.cc [rocksdb] 2PC double recovery bug fix 2016-05-10 14:06:07 -07:00
write_callback_test.cc write_callback_test: clean test directory before running tests 2016-04-18 10:18:41 -07:00
write_callback.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
write_controller_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
write_controller.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
write_controller.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
write_thread.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
write_thread.h [rocksdb] Memtable Log Referencing and Prepared Batch Recovery 2016-05-10 14:06:07 -07:00
writebuffer.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
xfunc_test_points.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
xfunc_test_points.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00