andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
1e35584251
rocksdb/db
History
Levi Tamasi 1e35584251 Move the index readers out of the block cache (#5298) 
Summary:
Currently, when the block cache is used for index blocks as well, it is
not really the index block that is stored in the cache but an
IndexReader object. Since this object is not pure data (it has, for
instance, pointers that might dangle), it's not really sharable. To
avoid the issues around this, the current code uses a dummy unique cache
key for each TableReader to store the IndexReader, and erases the
IndexReader entry when the TableReader is closed. Instead of doing this,
the new code moves the IndexReader out of the cache altogether. In
particular, instead of the TableReader owning, or caching/pinning the
IndexReader based on the customer's settings, the TableReader
unconditionally owns the IndexReader, which in turn owns/caches/pins
the index block (which is itself sharable and thus can be safely put in
the cache without any hacks).

Note: the change has two side effects:
1) Partitions of partitioned indexes no longer affect the read
amplification statistics.
2) Eviction statistics for index blocks are temporarily broken. We plan to fix
this in a separate phase.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/5298

Differential Revision: D15303203

Pulled By: ltamasi

fbshipit-source-id: 935a69ba59d87d5e44f42e2310619b790c366e47
2019-05-30 11:53:27 -07:00
..
builder.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
builder.h Periodic Compactions (#5166) 2019-04-10 19:31:18 -07:00
c_test.c add missing rocksdb_flush_cf in c (#5243) 2019-04-25 11:25:43 -07:00
c.cc Unordered Writes (#5218) 2019-05-13 17:47:21 -07:00
column_family_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
column_family.cc Move some file related files outside util/ (#5375) 2019-05-29 20:47:06 -07:00
column_family.h Update comments in column_family.h (#5347) 2019-05-24 12:07:15 -07:00
compact_files_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
compacted_db_impl.cc move dump stats to a separate thread (#4382) 2018-10-08 22:54:43 -07:00
compacted_db_impl.h Enable checkpoint of read-only db (#4681) 2018-12-07 17:06:02 -08:00
compaction_iteration_stats.h add counter for deletion dropping optimization 2017-08-19 14:10:08 -07:00
compaction_iterator_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
compaction_iterator.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
compaction_iterator.h Refresh snapshot list during long compactions (2nd attempt) (#5278) 2019-05-03 17:30:22 -07:00
compaction_job_stats_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
compaction_job_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
compaction_job.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
compaction_job.h improve comments for CompactionJob (#5341) 2019-05-23 16:57:46 -07:00
compaction_picker_fifo.cc Restrict L0->L0 compaction according to max_compaction_bytes option (#5329) 2019-05-22 23:40:57 -07:00
compaction_picker_fifo.h Avoid double-compacting data in bottom level in manual compactions (#5138) 2019-04-16 23:32:20 -07:00
compaction_picker_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
compaction_picker_universal.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
compaction_picker_universal.h Delete triggered compaction for universal style 2018-05-29 15:44:34 -07:00
compaction_picker.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
compaction_picker.h Add comments in compaction_picker.h 2019-05-28 12:24:38 -07:00
compaction.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
compaction.h Improve comments in compaction.h 2019-05-24 12:24:28 -07:00
comparator_db_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
convenience.cc Update all unique/shared_ptr instances to be qualified with namespace std (#4638) 2018-11-09 11:19:58 -08:00
corruption_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
cuckoo_table_db_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
db_basic_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
db_blob_index_test.cc fix lite build 2017-10-17 08:57:09 -07:00
db_block_cache_test.cc Move the index readers out of the block cache (#5298) 2019-05-30 11:53:27 -07:00
db_bloom_filter_test.cc Unordered Writes (#5218) 2019-05-13 17:47:21 -07:00
db_compaction_filter_test.cc Apply modernize-use-override (2nd iteration) 2019-02-14 14:41:36 -08:00
db_compaction_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
db_dynamic_level_test.cc Fix flaky DBDynamicLevelTest.DynamicLevelMaxBytesBase2 (#4668) 2018-11-12 16:42:16 -08:00
db_encryption_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
db_filesnapshot.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
db_flush_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
db_impl_compaction_flush.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
db_impl_debug.cc Avoid double-compacting data in bottom level in manual compactions (#5138) 2019-04-16 23:32:20 -07:00
db_impl_experimental.cc Update JobContext. (#3949) 2018-08-03 17:42:34 -07:00
db_impl_files.cc Move some file related files outside util/ (#5375) 2019-05-29 20:47:06 -07:00
db_impl_open.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
db_impl_readonly.cc Add two more StatsLevel (#5027) 2019-02-28 10:27:59 -08:00
db_impl_readonly.h Apply automatic formatting to some files (#5114) 2019-03-27 16:24:45 -07:00
db_impl_secondary.cc improve comments in db_impl_secondary 2019-05-24 15:32:03 -07:00
db_impl_secondary.h improve comments in db_impl_secondary 2019-05-24 15:32:03 -07:00
db_impl_write.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
db_impl.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
db_impl.h Improve comment in db_impl.h (#5338) 2019-05-24 13:09:55 -07:00
db_info_dumper.cc Move some file related files outside util/ (#5375) 2019-05-29 20:47:06 -07:00
db_info_dumper.h Change RocksDB License 2017-07-15 16:11:23 -07:00
db_inplace_update_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_io_failure_test.cc Disable DBIOFailureTest.NoSpaceCompactRange in LITE (#4596) 2018-10-29 14:36:31 -07:00
db_iter_stress_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
db_iter_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
db_iter.cc Move some file related files outside util/ (#5375) 2019-05-29 20:47:06 -07:00
db_iter.h Add comments to db/db_iter.h (#5340) 2019-05-23 16:11:38 -07:00
db_iterator_test.cc Merging iterator to avoid child iterator reseek for some cases (#5286) 2019-05-09 14:20:04 -07:00
db_log_iter_test.cc Apply modernize-use-override (2nd iteration) 2019-02-14 14:41:36 -08:00
db_memtable_test.cc Fix tsan complaint in ConcurrentMergeWrite test (#5308) 2019-05-15 11:21:48 -07:00
db_merge_operator_test.cc WriteUnPrepared: less virtual in iterator callback (#5049) 2019-04-02 14:47:16 -07:00
db_options_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
db_properties_test.cc Deprecate ttl option from CompactionOptionsFIFO (#4965) 2019-02-15 09:51:41 -08:00
db_range_del_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
db_secondary_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
db_sst_test.cc Increase Trash/DB size ratio in DBSSTTest.RateLimitedWALDelete (#5366) 2019-05-30 11:12:59 -07:00
db_statistics_test.cc Make statistics's stats_level change thread-safe (#5030) 2019-03-01 10:42:09 -08:00
db_table_properties_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
db_tailing_iter_test.cc Remove managed iterator 2018-07-17 14:43:18 -07:00
db_test2.cc DB::Close() to fail when there are unreleased snapshots (#5272) 2019-05-01 10:17:30 -07:00
db_test_util.cc Unordered Writes (#5218) 2019-05-13 17:47:21 -07:00
db_test_util.h Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
db_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
db_universal_compaction_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
db_wal_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
db_write_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
dbformat_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
dbformat.cc Apply automatic formatting to some files (#5114) 2019-03-27 16:24:45 -07:00
dbformat.h Add comments in db/dbformat.h 2019-05-23 16:44:20 -07:00
deletefile_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
error_handler_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
error_handler.cc Move some file related files outside util/ (#5375) 2019-05-29 20:47:06 -07:00
error_handler.h Fix typos in comments (#4456) 2018-10-04 20:46:50 -07:00
event_helpers.cc Log file_creation_time table property (#5232) 2019-04-22 15:30:07 -07:00
event_helpers.h Auto recovery from out of space errors (#4164) 2018-09-15 13:43:04 -07:00
experimental.cc comment unused parameters to turn on -Wunused-parameter flag 2018-04-12 17:59:16 -07:00
external_sst_file_basic_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
external_sst_file_ingestion_job.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
external_sst_file_ingestion_job.h Atomic ingest (#4895) 2019-02-12 19:16:17 -08:00
external_sst_file_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
fault_injection_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
file_indexer_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
file_indexer.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
file_indexer.h Change RocksDB License 2017-07-15 16:11:23 -07:00
filename_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
flush_job_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
flush_job.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
flush_job.h Collect compaction stats by priority and dump to info LOG (#5050) 2019-03-19 17:28:19 -07:00
flush_scheduler.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
flush_scheduler.h Unordered Writes (#5218) 2019-05-13 17:47:21 -07:00
forward_iterator_bench.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
forward_iterator.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
forward_iterator.h Comment out unused variables 2018-03-05 13:13:41 -08:00
in_memory_stats_history.cc Improve comments for StatsHistoryIterator and InMemoryStatsHistoryIterator 2019-05-24 11:40:05 -07:00
in_memory_stats_history.h Improve comments for StatsHistoryIterator and InMemoryStatsHistoryIterator 2019-05-24 11:40:05 -07:00
internal_stats.cc Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
internal_stats.h Collect compaction stats by priority and dump to info LOG (#5050) 2019-03-19 17:28:19 -07:00
job_context.h WritePrepared: Fix visible key compacted out by compaction (#4883) 2019-01-15 21:34:38 -08:00
listener_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
log_format.h Fix an inaccurate comment (#4315) 2018-08-24 18:13:20 -07:00
log_reader.cc Support for single-primary, multi-secondary instances (#4899) 2019-03-26 16:45:31 -07:00
log_reader.h secondary instance: add support for WAL tailing on OpenAsSecondary 2019-04-24 12:08:44 -07:00
log_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
log_writer.cc LogWriter to only flush after finish generating whole record (#5328) 2019-05-21 12:33:17 -07:00
log_writer.h Close WAL files before deletion (#5233) 2019-04-25 10:11:41 -07:00
logs_with_prep_tracker.cc Skip deleted WALs during recovery 2018-05-03 15:43:09 -07:00
logs_with_prep_tracker.h Skip deleted WALs during recovery 2018-05-03 15:43:09 -07:00
lookup_key.h Introduce a new MultiGet batching implementation (#5011) 2019-04-11 14:28:26 -07:00
malloc_stats.cc Detect if Jemalloc is linked with the binary (#4844) 2019-01-03 16:30:12 -08:00
malloc_stats.h Change RocksDB License 2017-07-15 16:11:23 -07:00
manual_compaction_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
memtable_list_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
memtable_list.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
memtable_list.h Move some file related files outside util/ (#5375) 2019-05-29 20:47:06 -07:00
memtable.cc rename variable to avoid shadowing (#5204) 2019-04-17 10:15:05 -07:00
memtable.h add whole key bloom filter support in memtables (#4985) 2019-02-19 12:15:39 -08:00
merge_context.h Introduce a new MultiGet batching implementation (#5011) 2019-04-11 14:28:26 -07:00
merge_helper_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
merge_helper.cc Add two more StatsLevel (#5027) 2019-02-28 10:27:59 -08:00
merge_helper.h Remove v1 RangeDelAggregator (#4778) 2018-12-17 17:33:46 -08:00
merge_operator.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
merge_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
obsolete_files_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
options_file_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
perf_context_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
pinned_iterators_manager.h Change RocksDB License 2017-07-15 16:11:23 -07:00
plain_table_db_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
pre_release_callback.h Mark logs with prepare in PreReleaseCallback (#5121) 2019-04-02 15:17:47 -07:00
prefix_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
range_del_aggregator_bench.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
range_del_aggregator_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
range_del_aggregator.cc Make ReadRangeDelAggregator::ShouldDelete() more inline friendly (#5202) 2019-04-18 12:27:25 -07:00
range_del_aggregator.h RangeDelAggregator::StripeRep::Invalidate() to be skipped if empty (#5312) 2019-05-16 15:24:28 -07:00
range_tombstone_fragmenter_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
range_tombstone_fragmenter.cc Avoid per-key upper bound check in BlockBasedTableIterator (#5142) 2019-04-16 11:37:47 -07:00
range_tombstone_fragmenter.h Add compaction logic to RangeDelAggregatorV2 (#4758) 2018-12-17 13:20:51 -08:00
read_callback.h WritePrepared: fix race condition in reading batch with duplicate keys (#5147) 2019-04-12 14:40:41 -07:00
repair_test.cc Move some file related files outside util/ (#5375) 2019-05-29 20:47:06 -07:00
repair.cc Move some file related files outside util/ (#5375) 2019-05-29 20:47:06 -07:00
snapshot_checker.h WritePrepared: fix issue with snapshot released during compaction (#4858) 2019-01-16 09:55:32 -08:00
snapshot_impl.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
snapshot_impl.h Refresh snapshot list during long compactions (2nd attempt) (#5278) 2019-05-03 17:30:22 -07:00
table_cache.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
table_cache.h Add some comments in table_cache.h 2019-05-24 14:26:43 -07:00
table_properties_collector_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
table_properties_collector.cc Feature for sampling and reporting compressibility (#4842) 2019-03-18 12:15:34 -07:00
table_properties_collector.h Feature for sampling and reporting compressibility (#4842) 2019-03-18 12:15:34 -07:00
transaction_log_impl.cc Fix some variable naming in db/transaction_log_impl.* (#5112) 2019-03-27 12:27:54 -07:00
transaction_log_impl.h Move some file related files outside util/ (#5375) 2019-05-29 20:47:06 -07:00
version_builder_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
version_builder.cc Support for single-primary, multi-secondary instances (#4899) 2019-03-26 16:45:31 -07:00
version_builder.h Support for single-primary, multi-secondary instances (#4899) 2019-03-26 16:45:31 -07:00
version_edit_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
version_edit.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
version_edit.h Support for single-primary, multi-secondary instances (#4899) 2019-03-26 16:45:31 -07:00
version_set_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
version_set.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
version_set.h Use GetCurrentManifestPath to locate current MANIFEST file (#5331) 2019-05-22 09:21:56 -07:00
wal_manager_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
wal_manager.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
wal_manager.h improve comment for WalManager (#5350) 2019-05-24 10:40:30 -07:00
write_batch_base.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
write_batch_internal.h WriteUnPrepared: Add new WAL marker kTypeBeginUnprepareXID (#4069) 2018-06-28 18:58:29 -07:00
write_batch_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
write_batch.cc Unordered Writes (#5218) 2019-05-13 17:47:21 -07:00
write_callback_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
write_callback.h Change RocksDB License 2017-07-15 16:11:23 -07:00
write_controller_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
write_controller.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
write_controller.h Change RocksDB License 2017-07-15 16:11:23 -07:00
write_thread.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
write_thread.h Fix skip WAL for whole write_group when leader's callback fail (#4838) 2019-01-03 12:40:42 -08:00