andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Run a subset of TSAN tests in CIrcleCI (#7122)

Browse Source 
Summary:
It is helpful to add some TSAN coverage before a pull request is committed. This diff adds some of them.
Some slow tests are excluded for the running speed. Some are blacklisted because they show warnings. Will investigate these warnings and see whether we can fix or suppress them.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/7122

Test Plan: Watch CIrcleCI runs

Reviewed By: riversand963

Differential Revision: D22532133

fbshipit-source-id: 81ddd02d9df19c513a12811979e8ddabae911354
This commit is contained in:
sdong 2020-07-14 15:31:57 -07:00 committed by Facebook GitHub Bot
parent bdf4de6cb9
commit e930ba1d18
2 changed files with 18 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

16
.circleci/config.yml
View File

@ -97,6 +97,19 @@ jobs:
- run: sudo apt-get update -y && sudo apt-get install -y clang-10 libgflags-dev
- run: SKIP_FORMAT_BUCK_CHECKS=1 COMPILE_WITH_ASAN=1 CC=clang-10 CXX=clang++-10 ROCKSDB_DISABLE_ALIGNED_NEW=1 USE_CLANG=1 PRINT_PARALLEL_OUTPUTS=1 make V=1 -j32 check | .circleci/cat_ignore_eagain # aligned new doesn't work for reason we haven't figured out
build-linux-clang10-mini-tsan:
machine:
image: ubuntu-1604:201903-01
resource_class: 2xlarge
steps:
- checkout # check out the code in the project directory
- run: pyenv global 3.5.2
- run: echo "deb http://apt.llvm.org/xenial/ llvm-toolchain-xenial-10 main" | sudo tee -a /etc/apt/sources.list
- run: echo "deb-src http://apt.llvm.org/xenial/ llvm-toolchain-xenial-10 main" | sudo tee -a /etc/apt/sources.list
- run: sudo apt-get update -y && sudo apt-get install -y clang-10 libgflags-dev
- run: SKIP_FORMAT_BUCK_CHECKS=1 COMPILE_WITH_TSAN=1 CC=clang-10 CXX=clang++-10 ROCKSDB_DISABLE_ALIGNED_NEW=1 USE_CLANG=1 PRINT_PARALLEL_OUTPUTS=1 EXCLUDE_TESTS_REGEX="DBTestCompactionFilterWithCompactParam.CompactionFilterWithValueChang|DBTestRandomized/DBTestRandomized.Randomized|DBTestWithParam.MergeCompactionTimeTest|DBTestUniversalCompaction.UniversalCompactionTrivialMoveTest1|DBTestUniversalCompactionMultiLevels.UniversalCompactionMultiLevels|ExternalSSTFileTest.IngestFileWithGlobalSeqnoRandomized|ReadaheadSequentialFileTest|ReadExceedsReadaheadSize/ReadaheadSequentialFileTest|TransactionStressTest|MySQLStyleTransactionTest|SnapshotConcurrentAccessTest|eqAdvanceConcurrentTest|WritePreparedTransactionTest|WriteUnpreparedTransactionTest|WriteUnpreparedStressTest|MultiThreadedDBTest|LogMarkLeakTest" make V=1 -j32 check | .circleci/cat_ignore_eagain # aligned new doesn't work for reason we haven't figured out. Exclude some tests to speed up. MultiThreadedDBTest, TransactionStressTest, TransactionStressTest, TransactionStressTest and TransactionStressTest are showing TSAN warning so excluding them for now. Will investigate later.
build-linux-clang10-ubsan:
machine:
image: ubuntu-1604:201903-01
@ -242,6 +255,9 @@ workflows:
build-linux-clang10-asan:
jobs:
- build-linux-clang10-asan
build-linux-clang10-mini-tsan:
jobs:
- build-linux-clang10-mini-tsan
build-linux-clang10-ubsan:
jobs:
- build-linux-clang10-ubsan

2
Makefile
View File

@ -826,6 +826,7 @@ J ?= 100%
# Use this regexp to select the subset of tests whose names match.
tests-regexp = .
EXCLUDE_TESTS_REGEX ?= "^$"
ifeq ($(PRINT_PARALLEL_OUTPUTS), 1)
parallel_com = '{}'
@ -846,6 +847,7 @@ check_0:
} \
| $(prioritize_long_running_tests) \
| grep -E '$(tests-regexp)' \
| grep -E -v '$(EXCLUDE_TESTS_REGEX)' \
| build_tools/gnu_parallel -j$(J) --plain --joblog=LOG $$eta --gnu $(parallel_com) ; \
parallel_retcode=$$? ; \
awk '{ if ($$7 != 0 || $$8 != 0) { if ($$7 == "Exitval") { h = $$0; } else { if (!f) print h; print; f = 1 } } } END { if(f) exit 1; }' < LOG ; \