andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
e8cb32ed67
rocksdb/table
History
Levi Tamasi e8cb32ed67 Introduce BlobFileCache and add support for blob files to Get() (#7540) 
Summary:
The patch adds blob file support to the `Get` API by extending `Version` so that
whenever a blob reference is read from a file, the blob is retrieved from the corresponding
blob file and passed back to the caller. (This is assuming the blob reference is valid
and the blob file is actually part of the given `Version`.) It also introduces a cache
of `BlobFileReader`s called `BlobFileCache` that enables sharing `BlobFileReader`s
between callers. `BlobFileCache` uses the same backing cache as `TableCache`, so
`max_open_files` (if specified) limits the total number of open (table + blob) files.

TODO: proactively open/cache blob files and pin the cache handles of the readers in the
metadata objects similarly to what `VersionBuilder::LoadTableHandlers` does for
table files.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/7540

Test Plan: `make check`

Reviewed By: riversand963

Differential Revision: D24260219

Pulled By: ltamasi

fbshipit-source-id: a8a2a4f11d3d04d6082201b52184bc4d7b0857ba
2020-10-15 13:04:47 -07:00
..
adaptive Bring the Configurable options together (#5753) 2020-09-14 17:01:01 -07:00
block_based Return error if Get/Multi() fails in Prefetching Filter blocks (#7543) 2020-10-14 10:45:36 -07:00
cuckoo Change ParseInternalKey() to return Status instead of bool (#7457) 2020-09-30 19:16:47 -07:00
plain Add plain_table_db_test to ASSERT_STATUS_CHECKED list (#7482) 2020-10-13 12:00:09 -07:00
block_fetcher_test.cc Add Status check enforcement for unit tests (#7464) 2020-09-30 12:48:05 -07:00
block_fetcher.cc Skip unnecessary allocation for mmap reads under 5000 bytes (#7043) 2020-06-30 15:40:40 -07:00
block_fetcher.h Reduce memory copies when fetching and uncompressing blocks from SST files (#6689) 2020-04-24 15:32:56 -07:00
cleanable_test.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
format.cc Introduce a helper method UncompressData (#7434) 2020-09-25 09:01:45 -07:00
format.h Clean up CompressBlock/CompressBlockInternal a bit (#7249) 2020-08-12 18:25:48 -07:00
get_context.cc Introduce BlobFileCache and add support for blob files to Get() (#7540) 2020-10-15 13:04:47 -07:00
get_context.h Introduce BlobFileCache and add support for blob files to Get() (#7540) 2020-10-15 13:04:47 -07:00
internal_iterator.h Clean up InternalIterator upper bound logic a little bit (#7200) 2020-08-05 10:44:57 -07:00
iter_heap.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
iterator_wrapper.h Redesign block cache pinning API (#7520) 2020-10-11 14:58:24 -07:00
iterator.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
merger_test.cc More Makefile Cleanup (#7097) 2020-07-09 14:35:17 -07:00
merging_iterator.cc Add some simulator cache and block tracer tests to ASSERT_STATUS_CHECKED (#7305) 2020-08-24 16:43:31 -07:00
merging_iterator.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
meta_blocks.cc Restore file size in backup table file names (and other cleanup) (#7400) 2020-09-17 10:24:22 -07:00
meta_blocks.h Extend Get/MultiGet deadline support to table open (#6982) 2020-06-29 14:53:17 -07:00
mock_table.cc Introduce options.check_flush_compaction_key_order (#7467) 2020-10-01 10:10:26 -07:00
mock_table.h Introduce options.check_flush_compaction_key_order (#7467) 2020-10-01 10:10:26 -07:00
multiget_context.h Status check enforcement for timestamp_basic_test (#7454) 2020-09-29 18:23:27 -07:00
persistent_cache_helper.cc Add more tests to ASSERT_STATUS_CHECKED (#7367) 2020-09-16 15:48:07 -07:00
persistent_cache_helper.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
persistent_cache_options.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
scoped_arena_iterator.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
sst_file_dumper.cc Change ParseInternalKey() to return Status instead of bool (#7457) 2020-09-30 19:16:47 -07:00
sst_file_dumper.h Add sst_file_dumper status check (#7315) 2020-09-04 19:26:42 -07:00
sst_file_reader_test.cc Fix scope of ReadOptions in SstFileReader (#7432) 2020-09-23 15:24:03 -07:00
sst_file_reader.cc Fix scope of ReadOptions in SstFileReader (#7432) 2020-09-23 15:24:03 -07:00
sst_file_writer_collectors.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
sst_file_writer.cc enable Status check assertions for sst_file_reader_test (#7448) 2020-09-28 15:13:15 -07:00
table_builder.h Store DB identity and DB session ID in SST files (#6983) 2020-06-17 10:57:40 -07:00
table_factory.cc Add plain_table_db_test to ASSERT_STATUS_CHECKED list (#7482) 2020-10-13 12:00:09 -07:00
table_properties_internal.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
table_properties.cc Store DB identity and DB session ID in SST files (#6983) 2020-06-17 10:57:40 -07:00
table_reader_bench.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
table_reader_caller.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
table_reader.h dedup ReadOptions in iterator hierarchy (#7210) 2020-08-03 15:23:04 -07:00
table_test.cc Fix for clang_analyzer build failure in table_test (#7553) 2020-10-14 12:44:59 -07:00
two_level_iterator.cc Redesign block cache pinning API (#7520) 2020-10-11 14:58:24 -07:00
two_level_iterator.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00