andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
rocksdb/table
History
Lei Jin 94997eab5e reduce memory usage of cuckoo table builder 
Summary:
builder currently buffers all key value pairs as a vector of
pair<string, string>. That is too much due to std::string
overhead. It wasn't able to fit 1B key/values (12bytes total) in 100GB
of ram. Switch to use a plain string to store the key/value sequence and
use only 12GB of ram as a result.

Test Plan: db_bench

Reviewers: igor, sdong, yhchiang

Reviewed By: sdong

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D23763
2014-09-25 16:34:24 -07:00
..
adaptive_table_factory.cc
introduce ImmutableOptions
2014-09-04 16:18:36 -07:00
adaptive_table_factory.h
introduce ImmutableOptions
2014-09-04 16:18:36 -07:00
block_based_filter_block_test.cc
Remove some unnecessary constructors
2014-09-17 16:45:58 -07:00
block_based_filter_block.cc
Remove some unnecessary constructors
2014-09-17 16:45:58 -07:00
block_based_filter_block.h
Remove some unnecessary constructors
2014-09-17 16:45:58 -07:00
block_based_table_builder.cc
Run make format on PR #249
2014-09-17 15:08:50 -07:00
block_based_table_builder.h
introduce ImmutableOptions
2014-09-04 16:18:36 -07:00
block_based_table_factory.cc
introduce ImmutableOptions
2014-09-04 16:18:36 -07:00
block_based_table_factory.h
introduce ImmutableOptions
2014-09-04 16:18:36 -07:00
block_based_table_reader.cc
re-enable BlockBasedTable::SetupForCompaction()
2014-09-23 14:18:57 -07:00
block_based_table_reader.h
Implement full filter for block based table.
2014-09-08 10:37:05 -07:00
block_builder.cc
Fix compile
2014-09-02 11:49:38 -07:00
block_builder.h
delete unused Comparator
2014-09-04 09:10:13 +08:00
block_hash_index_test.cc
Materialize the hash index
2014-05-15 14:09:03 -07:00
block_hash_index.cc
Materialize the hash index
2014-05-15 14:09:03 -07:00
block_hash_index.h
Materialize the hash index
2014-05-15 14:09:03 -07:00
block_prefix_index.cc
fix a few compile warnings
2014-09-04 23:06:23 +08:00
block_prefix_index.h
fix a few compile warnings
2014-09-04 23:06:23 +08:00
block_test.cc
Remove some unnecessary constructors
2014-09-17 16:45:58 -07:00
block.cc
Remove some unnecessary constructors
2014-09-17 16:45:58 -07:00
block.h
Remove some unnecessary constructors
2014-09-17 16:45:58 -07:00
bloom_block.cc
Adding option to save PlainTable index and bloom filter in SST file.
2014-07-18 16:58:13 -07:00
bloom_block.h
Adding option to save PlainTable index and bloom filter in SST file.
2014-07-18 16:58:13 -07:00
cuckoo_table_builder_test.cc
option to choose module when calculating CuckooTable hash
2014-09-25 13:53:27 -07:00
cuckoo_table_builder.cc
reduce memory usage of cuckoo table builder
2014-09-25 16:34:24 -07:00
cuckoo_table_builder.h
reduce memory usage of cuckoo table builder
2014-09-25 16:34:24 -07:00
cuckoo_table_factory.cc
option to choose module when calculating CuckooTable hash
2014-09-25 13:53:27 -07:00
cuckoo_table_factory.h
option to choose module when calculating CuckooTable hash
2014-09-25 13:53:27 -07:00
cuckoo_table_reader_test.cc
option to choose module when calculating CuckooTable hash
2014-09-25 13:53:27 -07:00
cuckoo_table_reader.cc
improve memory efficiency of cuckoo reader
2014-09-25 16:15:23 -07:00
cuckoo_table_reader.h
improve memory efficiency of cuckoo reader
2014-09-25 16:15:23 -07:00
filter_block.h
Replace naked calls to operator new and delete (Fixes #222)
2014-09-17 13:50:07 -07:00
flush_block_policy.cc
move block based table related options BlockBasedTableOptions
2014-08-25 14:22:05 -07:00
format.cc
Run make format on PR #249
2014-09-17 15:08:50 -07:00
format.h
Run make format on PR #249
2014-09-17 15:08:50 -07:00
full_filter_block_test.cc
Fix compile
2014-09-22 15:19:20 -07:00
full_filter_block.cc
Run make format on PR #249
2014-09-17 15:08:50 -07:00
full_filter_block.h
Replace naked calls to operator new and delete (Fixes #222)
2014-09-17 13:50:07 -07:00
iter_heap.h
Turn on -Wmissing-prototypes
2014-04-09 21:17:14 -07:00
iterator_wrapper.h
In DB::NewIterator(), try to allocate the whole iterator tree in an arena
2014-06-02 17:44:57 -07:00
iterator.cc
In DB::NewIterator(), try to allocate the whole iterator tree in an arena
2014-06-02 17:44:57 -07:00
merger_test.cc
Merger test
2014-09-08 22:24:40 -07:00
merger.cc
Refactor PerfStepTimer to stop on destruct
2014-09-02 12:04:22 -07:00
merger.h
In DB::NewIterator(), try to allocate the whole iterator tree in an arena
2014-06-02 17:44:57 -07:00
meta_blocks.cc
Run make format on PR #249
2014-09-17 15:08:50 -07:00
meta_blocks.h
Adding option to save PlainTable index and bloom filter in SST file.
2014-07-18 16:58:13 -07:00
plain_table_builder.cc
Implement full filter for block based table.
2014-09-08 10:37:05 -07:00
plain_table_builder.h
introduce ImmutableOptions
2014-09-04 16:18:36 -07:00
plain_table_factory.cc
introduce ImmutableOptions
2014-09-04 16:18:36 -07:00
plain_table_factory.h
fix comments
2014-09-08 16:34:04 +08:00
plain_table_index.cc
introduce ImmutableOptions
2014-09-04 16:18:36 -07:00
plain_table_index.h
introduce ImmutableOptions
2014-09-04 16:18:36 -07:00
plain_table_key_coding.cc
replace hard-coded number with named variable
2014-09-08 11:10:17 +08:00
plain_table_key_coding.h
typo improvement
2014-09-06 23:21:26 +08:00
plain_table_reader.cc
Replace naked calls to operator new and delete (Fixes #222)
2014-09-17 13:50:07 -07:00
plain_table_reader.h
introduce ImmutableOptions
2014-09-04 16:18:36 -07:00
table_builder.h
First phase API clean up
2014-02-03 00:30:43 -08:00
table_properties.cc
Fix SIGFAULT when running sst_dump on v2.6 db
2014-04-21 17:49:47 -07:00
table_reader_bench.cc
CuckooTable: add one option to allow identity function for the first hash function
2014-09-18 11:00:48 -07:00
table_reader.h
Add DB property "rocksdb.estimate-table-readers-mem"
2014-08-06 11:39:46 -07:00
table_test.cc
Merge pull request #249 from tdfischer/decompression-refactoring
2014-09-17 15:01:40 -07:00
two_level_iterator.cc
ForwardIterator: reset incomplete iterators on Seek()
2014-08-29 16:21:29 -07:00
two_level_iterator.h
In DB::NewIterator(), try to allocate the whole iterator tree in an arena
2014-06-02 17:44:57 -07:00