andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
e6e505a4d9
rocksdb/table
History
agiardullo 3bfd3d39a3 Use SST files for Transaction conflict detection 
Summary:
Currently, transactions can fail even if there is no actual write conflict.  This is due to relying on only the memtables to check for write-conflicts.  Users have to tune memtable settings to try to avoid this, but it's hard to figure out exactly how to tune these settings.

With this diff, TransactionDB will use both memtables and SST files to determine if there are any write conflicts.  This relies on the fact that BlockBasedTable stores sequence numbers for all writes that happen after any open snapshot.  Also, D50295 is needed to prevent SingleDelete from disappearing writes (the TODOs in this test code will be fixed once the other diff is approved and merged).

Note that Optimistic transactions will still rely on tuning memtable settings as we do not want to read from SST while on the write thread.  Also, memtable settings can still be used to reduce how often TransactionDB needs to read SST files.

Test Plan: unit tests, db bench

Reviewers: rven, yhchiang, kradhakrishnan, IslamAbdelRahman, sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb, yoshinorim

Differential Revision: https://reviews.facebook.net/D50475
2015-12-11 12:34:11 -08:00
..
adaptive_table_factory.cc Pass column family ID to table property collector 2015-10-09 14:36:51 -07:00
adaptive_table_factory.h Pass column family ID to table property collector 2015-10-09 14:36:51 -07:00
block_based_filter_block_test.cc rocksdb: switch to gtest 2015-03-17 14:08:00 -07:00
block_based_filter_block.cc bloom hit/miss stats for SST and memtable 2015-10-07 11:23:20 -07:00
block_based_filter_block.h Remember whole key/prefix filtering on/off in SST file 2015-02-11 11:20:04 -08:00
block_based_table_builder.cc Revert "Adding new table properties" 2015-11-06 16:49:38 -08:00
block_based_table_builder.h Move skip_table_builder_flush to BlockBasedTableOption 2015-10-30 18:33:01 -07:00
block_based_table_factory.cc Fix build broken by previous commit of "option helper refactor" 2015-11-17 16:52:54 -08:00
block_based_table_factory.h Merge pull request #805 from SherlockNoMad/OptionHelperFix 2015-11-17 16:24:52 -08:00
block_based_table_reader.cc Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00
block_based_table_reader.h Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00
block_builder.cc Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
block_builder.h delete unused Comparator 2014-09-04 09:10:13 +08:00
block_hash_index_test.cc Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00
block_hash_index.cc Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00
block_hash_index.h Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00
block_prefix_index.cc Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
block_prefix_index.h Windows Port from Microsoft 2015-07-01 16:13:56 -07:00
block_test.cc Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00
block.cc Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00
block.h Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00
bloom_block.cc table/bloom_block.*: pass func parameter by reference 2014-09-30 23:30:31 +02:00
bloom_block.h Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
cuckoo_table_builder_test.cc Block cuckoo table tests in ROCKSDB_LITE 2015-07-20 10:50:46 -07:00
cuckoo_table_builder.cc Move rate_limiter, write buffering, most perf context instrumentation and most random kill out of Env 2015-07-17 16:58:18 -07:00
cuckoo_table_builder.h Move rate_limiter, write buffering, most perf context instrumentation and most random kill out of Env 2015-07-17 16:58:18 -07:00
cuckoo_table_factory.cc Pass column family ID to table property collector 2015-10-09 14:36:51 -07:00
cuckoo_table_factory.h Add OptionsUtil::LoadOptionsFromFile() API 2015-11-12 06:52:43 -08:00
cuckoo_table_reader_test.cc Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00
cuckoo_table_reader.cc Use SST files for Transaction conflict detection 2015-12-11 12:34:11 -08:00
cuckoo_table_reader.h Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00
filter_block.h Dump routine to BlockBasedTableReader 2014-12-23 13:24:07 -08:00
flush_block_policy.cc move block based table related options BlockBasedTableOptions 2014-08-25 14:22:05 -07:00
format.cc Add ZSTD (not final format) compression type 2015-08-28 11:01:13 -07:00
format.h Move rate_limiter, write buffering, most perf context instrumentation and most random kill out of Env 2015-07-17 16:58:18 -07:00
full_filter_block_test.cc rocksdb: switch to gtest 2015-03-17 14:08:00 -07:00
full_filter_block.cc bloom hit/miss stats for SST and memtable 2015-10-07 11:23:20 -07:00
full_filter_block.h Remember whole key/prefix filtering on/off in SST file 2015-02-11 11:20:04 -08:00
get_context.cc Use SST files for Transaction conflict detection 2015-12-11 12:34:11 -08:00
get_context.h Use SST files for Transaction conflict detection 2015-12-11 12:34:11 -08:00
internal_iterator.h Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00
iter_heap.h Replace std::priority_queue in MergingIterator with custom heap, take 2 2015-07-15 03:34:40 -07:00
iterator_wrapper.h Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00
iterator.cc Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00
merger_test.cc Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00
merger.cc Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00
merger.h Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00
meta_blocks.cc Revert "Adding new table properties" 2015-11-06 16:49:38 -08:00
meta_blocks.h Moving memtable related files from util to a new directory memtable 2015-10-16 14:10:33 -07:00
mock_table.cc Fix MockTable ID storage 2015-10-28 10:53:14 -07:00
mock_table.h Fix MockTable ID storage 2015-10-28 10:53:14 -07:00
plain_table_builder.cc Pass column family ID to table property collector 2015-10-09 14:36:51 -07:00
plain_table_builder.h Pass column family ID to table property collector 2015-10-09 14:36:51 -07:00
plain_table_factory.cc Merge pull request #805 from SherlockNoMad/OptionHelperFix 2015-11-17 16:24:52 -08:00
plain_table_factory.h Fix build broken by previous commit of "option helper refactor" 2015-11-17 16:52:54 -08:00
plain_table_index.cc "make format" against last 10 commits 2015-07-13 13:50:18 -07:00
plain_table_index.h Block plain_table_index.cc in ROCKSDB_LITE 2014-11-24 20:47:27 -08:00
plain_table_key_coding.cc Fix the bug of using freed memory introduced by recent plain table reader patch 2015-09-23 16:16:26 -07:00
plain_table_key_coding.h PlainTableReader to support non-mmap mode 2015-09-23 11:41:07 -07:00
plain_table_reader.cc plain table reader: avoid re-read the same position for index and data in non-mmap mode 2015-11-18 16:49:08 -08:00
plain_table_reader.h plain table reader: avoid re-read the same position for index and data in non-mmap mode 2015-11-18 16:49:08 -08:00
scoped_arena_iterator.h Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00
sst_file_writer.cc Move skip_table_builder_flush to BlockBasedTableOption 2015-10-30 18:33:01 -07:00
table_builder.h Move skip_table_builder_flush to BlockBasedTableOption 2015-10-30 18:33:01 -07:00
table_properties_internal.h Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00
table_properties.cc Revert "Adding new table properties" 2015-11-06 16:49:38 -08:00
table_reader_bench.cc plain table reader: avoid re-read the same position for index and data in non-mmap mode 2015-11-18 16:49:08 -08:00
table_reader.h Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00
table_test.cc Revert "Adding new table properties" 2015-11-06 16:49:38 -08:00
two_level_iterator.cc Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00
two_level_iterator.h Seperate InternalIterator from Iterator 2015-10-13 15:32:13 -07:00