Motivation:
Currently the PR verification takes a lot of time. We should try to only run builds for the affected modules in most cases to speed up the build.
Modifications:
Adjust docker files and workflow to only run build for affected modules for a lot of jobs while still run full builds for windows + java11 with BoringSSL
Result:
Hopefully quicker feedback loop when verify PRs
Motivation:
Native image compatibility is fragile and breaks easily, so we need a PR build to tell us when this happens.
Modification:
Add a graalvm-based build to the PR build matrix.
Result:
Every PR is now also tested on Graal.
Motivation:
When Maven does not run in batch mode, it will continuously print its progress as it downloads dependencies.
This can produce a very large amount of log output, that makes it harder to debug build failures.
Modification:
Make all Maven builds run in batch mode by adding the `-B` command line flag, and have transfer progress suppressed with the `-ntp` flag.
Some builds were already running batch mode but had the flag in a different location – these have had their `-B` flag moved so all builds are consistent.
Result:
Much less output in our build logs where Maven is just downloading stuff.
Motivation:
Let's also build on windows during PR validation
Modifications:
Add build on windows during PR
Result:
Validate that all also pass on windows
Motivation:
Just use MAVEN_OPTS to setup all the timeouts etc for dependency downloads. This way we at least can be sure these are applied.
Modifications:
- Use MAVEN_OPTS
- Remove ci profile
- Remove unused settings.xml file
- Always use ./mvnw
Result:
Build stability improvements
Motivation:
We should setup the caching so it will be able to use different restore keys and so almost never need to start from scratch
Modifications:
Adjust caching config to make use of different restore keys for maven caching but also docker caching
Result:
Better cache usage
Motivation:
We should use the same maven cache for all builds so we can re-use as much of the downloaded maven dependencies as possible
Modifications:
- Just use the same cache for all
Result:
Hopefully be able to re-use most of the dependencies
Motivation:
0f25213918 introduced some properties that were used to make builds more stable on the ci. All of these properties were duplicated everywhere, this made it hard to maintain
Modifications:
- Add profile which sets the properties.
- Just use the profile when build on the ci
Result:
Easier to maintain custom properties for the ci build
Motivation:
It seems like it is a known issue that maven frequently sees connection reset / connection timeout during CI builds. We should workaround these issues like others did:
- https://github.com/kiegroup/kie-wb-common/pull/3416
Modifications:
Add extra maven options during build to reduce the likelyness of timeouts / resets
Result:
More stable builds
Motivation:
The last non-LTS release is JDK16 now.
Modifications:
Update from JDK15 to JDK16 for building as this is the last non-LTS release atm
Result:
Build with latest non-LTS release as well
Motivation:
In the past we did see problems sometime when run-on-arch-action was used. We are multiple releases behind, lets update and so maybe fix the problems.
Modifications:
Update to latest release
Result:
Use latest run-on-arch-action release
Motivation:
It turned out we didnt run the openssl tests on the CI when we used the non-static version of netty-tcnative.
Modifications:
- Upgrade netty-tcnative to fix segfault when using shared openssl
- Adjust tests to only run session cache tests when openssl supports it
- Fix some more tests to only depend on KeyManager if the underlying openssl version supports it
Result:
Run all openssl test on the CI even when shared library is used
Motivation:
Newer versions of dawidd6/action-download-artifact changed the default
workflow_conclusion from "completed" to "completed, success" which can
result in download failures if the associated job fails, which is
expected when tests fail.
Modifications:
- Explicitly set the workflow_conclusion to "completed"
Result:
Test failures which result in build failures will still download test
data and generate reports after updating
dawidd6/action-download-artifact.
Motivation:
Doing releases manually is error-prone, it would be better if we could do it via a workflow
Modification:
- Add workflow to cut releases
- Add related scripts
Result:
Be able to easily cut a release via a workflow
Motivation:
To make it easier to understand why a build failed let us publish the rest results
Modifications:
Use a new workflow to be able to publish the test reports
Result:
Easier to understand why a PR did fail
Motivation:
It turns out we can't use the action to check for build failures as it can't be used when a PR is done from a fork. Let's just use our simple script.
Modifications:
- Replace action with custom script
Result:
Builds for PRs that are done via forks work again.
Motivation:
To make it easier to understand why the build fails lets use an action that will report which unit test failed
Modifications:
- Replace custom script with action-surefire-report
Result:
Easier to understand test failures
Motivation:
As shown in the past we need to verify we actually can load the native as otherwise we may introduce regressions.
Modifications:
- Add new maven module which tests loading of native modules
- Add job that will also test loading on aarch64
Result:
Less likely to introduce regressions related to loading native code in the future
Motivation:
Let us just only use one build config when building the 4.1 branch.
Modifications:
As we already do a full validation when doing the PR builds we can just only use one build config for pushes to the "main" branches
Result:
Faster build times
Motivation:
We need to take special care when deploying snapshots as we need to generate the jars in multiple steps
Modifications:
- Use the nexus staging pluging to stage jars locally in multiple steps
- Add extra job that will merge these staged jars and deploy these
Result:
Fixes https://github.com/netty/netty/issues/10887
Motivation:
When validating PRs we should also at least run one job that uses boringssl
Modifications:
- Add job that uses boringssl
- Cleanup docker compose files
- Fix buffer leak in test
Result:
Also run with boringssl when PRs are validated
Motivation:
To prevent failures to problems while downloading dependencies we shoud cache these
Modifications:
Add maven cache
Result:
No more failures due problems while downloading dependencies
Motivation:
We need to find a way to deploy SNAPSHOTS for different arch with the same timestamp. Otherwise it will cause problems.
See https://github.com/netty/netty/issues/10887
Modification:
Skip all other deploys then x86_64
Result:
Users are able to use SNAPSHOTS for x86_6
Motivation:
We should also deploy snapshots for our cross compiled native jars.
Modifications:
- Add job and docker files for deploying cross compiled native jars
- Ensure we map the maven cache into our docker containers
Result:
Deploy aarch64 jars and re-use cache
Motivation:
We should just use GitHub Actions for the CI
Modifications:
- Adjust docker / docker compose files
- Add different workflows and jobs to deploy and build the project
Result:
Don't depend on external CI services
Motivation:
Printing download progress in the build log makes it harder to see what's wrong when the build fails.
Modification:
Change the maven command to not show transfer progress, also enable batch mode so Maven don't print in colors that we can't see anyway.
Result:
More concise code analysis build logs.
Motivation:
Github now allows to run CodeQL during pull request verification. This allows to detect errors / security problems early.
Modification:
Add config
Result:
Fixes https://github.com/netty/netty/issues/10669
Co-authored-by: Artem Smotrakov <artem.smotrakov@sap.com>
