andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
0f0076ed5a
rocksdb/table
History
Haobo Xu 0f0076ed5a [RocksDB] Reduce memory footprint of the blockbased table hash index. 
Summary:
Currently, the in-memory hash index of blockbased table uses a precise hash map to track the prefix to block range mapping. In some use cases, especially when prefix itself is big, the memory overhead becomes a problem. This diff introduces a fixed hash bucket array that does not store the prefix and allows prefix collision, which is similar to the plaintable hash index, in order to reduce the memory consumption.
Just a quick draft, still testing and refining.

Test Plan: unit test and shadow testing

Reviewers: dhruba, kailiu, sdong

Reviewed By: sdong

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D19047
2014-06-18 18:16:07 -07:00
..
adaptive_table_factory.cc Change order of parameters in adaptive table factory 2014-06-18 07:04:37 +02:00
adaptive_table_factory.h Change order of parameters in adaptive table factory 2014-06-18 07:04:37 +02:00
block_based_table_builder.cc [RocksDB] Reduce memory footprint of the blockbased table hash index. 2014-06-18 18:16:07 -07:00
block_based_table_builder.h Don't compress block bigger than 2GB 2014-06-09 12:26:09 -07:00
block_based_table_factory.cc Materialize the hash index 2014-05-15 14:09:03 -07:00
block_based_table_factory.h Materialize the hash index 2014-05-15 14:09:03 -07:00
block_based_table_reader.cc [RocksDB] Reduce memory footprint of the blockbased table hash index. 2014-06-18 18:16:07 -07:00
block_based_table_reader.h In DB::NewIterator(), try to allocate the whole iterator tree in an arena 2014-06-02 17:44:57 -07:00
block_builder.cc [Performance Branch] PlainTable to encode rows with seqID 0, value type using 1 internal byte. 2014-02-03 12:19:30 -08:00
block_builder.h [Performance Branch] PlainTable to encode rows with seqID 0, value type using 1 internal byte. 2014-02-03 12:19:30 -08:00
block_hash_index_test.cc Materialize the hash index 2014-05-15 14:09:03 -07:00
block_hash_index.cc Materialize the hash index 2014-05-15 14:09:03 -07:00
block_hash_index.h Materialize the hash index 2014-05-15 14:09:03 -07:00
block_prefix_index.cc [RocksDB] Reduce memory footprint of the blockbased table hash index. 2014-06-18 18:16:07 -07:00
block_prefix_index.h [RocksDB] Reduce memory footprint of the blockbased table hash index. 2014-06-18 18:16:07 -07:00
block_test.cc Materialize the hash index 2014-05-15 14:09:03 -07:00
block.cc [RocksDB] Reduce memory footprint of the blockbased table hash index. 2014-06-18 18:16:07 -07:00
block.h [RocksDB] Reduce memory footprint of the blockbased table hash index. 2014-06-18 18:16:07 -07:00
filter_block_test.cc [Performance Branch] PlainTable to encode rows with seqID 0, value type using 1 internal byte. 2014-02-03 12:19:30 -08:00
filter_block.cc Consolidate SliceTransform object ownership 2014-03-10 12:56:46 -07:00
filter_block.h [CF] Rethinking ColumnFamilyHandle and fix to dropping column families 2014-02-12 13:47:09 -08:00
flush_block_policy.cc Remove the terrible hack in for flush_block_policy_factory 2014-02-28 16:39:27 -08:00
format.cc Fix iOS compile 2014-05-07 17:48:31 -07:00
format.h xxHash 2014-05-01 14:09:32 -04:00
iter_heap.h Turn on -Wmissing-prototypes 2014-04-09 21:17:14 -07:00
iterator_wrapper.h In DB::NewIterator(), try to allocate the whole iterator tree in an arena 2014-06-02 17:44:57 -07:00
iterator.cc In DB::NewIterator(), try to allocate the whole iterator tree in an arena 2014-06-02 17:44:57 -07:00
merger.cc In DB::NewIterator(), try to allocate the whole iterator tree in an arena 2014-06-02 17:44:57 -07:00
merger.h In DB::NewIterator(), try to allocate the whole iterator tree in an arena 2014-06-02 17:44:57 -07:00
meta_blocks.cc Materialize the hash index 2014-05-15 14:09:03 -07:00
meta_blocks.h Materialize the hash index 2014-05-15 14:09:03 -07:00
plain_table_builder.cc TablePropertiesCollectorFactory 2014-05-13 12:30:55 -07:00
plain_table_builder.h TablePropertiesCollectorFactory 2014-05-13 12:30:55 -07:00
plain_table_factory.cc Revert "Revert "Allow allocating dynamic bloom, plain table indexes and hash linked list from huge page TLB"" 2014-05-04 13:56:29 -07:00
plain_table_factory.h PlainTableFactory::PlainTableFactory() to have huge TLB turned off by default 2014-05-05 11:05:54 -07:00
plain_table_reader.cc prefetch bloom filter data block for L0 files 2014-06-12 10:06:18 -07:00
plain_table_reader.h prefetch bloom filter data block for L0 files 2014-06-12 10:06:18 -07:00
table_builder.h First phase API clean up 2014-02-03 00:30:43 -08:00
table_properties.cc Fix SIGFAULT when running sst_dump on v2.6 db 2014-04-21 17:49:47 -07:00
table_reader_bench.cc Fix more gflag namespace issues 2014-05-09 08:41:02 -07:00
table_reader.h prefetch bloom filter data block for L0 files 2014-06-12 10:06:18 -07:00
table_test.cc [RocksDB] Reduce memory footprint of the blockbased table hash index. 2014-06-18 18:16:07 -07:00
two_level_iterator.cc In DB::NewIterator(), try to allocate the whole iterator tree in an arena 2014-06-02 17:44:57 -07:00
two_level_iterator.h In DB::NewIterator(), try to allocate the whole iterator tree in an arena 2014-06-02 17:44:57 -07:00