Commit Graph

1324 Commits

Author SHA1 Message Date
Norman Maurer
7b3e28f96f
Disable two tests which currently fail in master (#11344)
Motivation:

We have currently two test-failures in master. Let's disable these and then open a PR with a fix once we know why. This way we can make progress in master

Modifications:

Disable the two failing tests

Result:

Master builds again
2021-06-01 13:41:20 +02:00
skyguard1
cfd64f7ced Add default block in IdleStateHandler (#11341)
Motivation:

We should have a default case in every switch block.

Modification:

Add default block in IdleStateHandler

Result:

Cleanup

Signed-off-by: xingrufei <xingrufei@sogou-inc.com>
2021-06-01 09:22:02 +02:00
Norman Maurer
f0c919768b Add license header to our scripts and workflows (#11282)
Motivation:

We should have license header whenever possible.

Modifications:

Add header to scripts and workflow config

Result:

More clear licensing
2021-05-19 14:07:00 +02:00
Norman Maurer
c56e5e6e4f Fail the build if we can't load the OpenSSL library (#11269)
Motivation:

We should better fail the build if we can't load the OpenSSL library to ensure we not introduce a regression at some point related to native library loading

Modifications:

Remove usages of assumeTrue and let the tests fail if we cant load the native lib

Result:

Ensure we not regress
2021-05-19 11:03:01 +02:00
Aayush Atharva
0d93c24547 Change asterisk to 'x' in FQDN of SelfSignedCertificate (#11245)
Motivation:

`SelfSignedCertificate` creates a certificate and private key files and store them in a temporary directory. However, if the certificate uses a wildcard hostname that uses asterisk *, e.g. `*.shieldblaze.com`, it'll throw an error because * is not a valid character in the file system.

Modification:
Replace the asterisk with 'x'

Result:
Fixes #11240
2021-05-12 19:33:05 +02:00
Norman Maurer
1602b96a25 Use tasks by default when using openssl based SSL provider (#11242)
Motivation:

We introduced the ability to offload certain operations to an executor that may take some time to complete. At the moment this is not enabled by default when using the openssl based SSL provider. Let's enable it by default as we have this support for some while now and didnt see any issues yet. This will also make things less confusing and more consistent with the JDK based provider.

Modifications:

Use true as default value for io.netty.handler.ssl.openssl.useTasks.

Result:

Offloading works with openssl based SSL provider as well by default
2021-05-12 15:01:22 +02:00
Norman Maurer
f546718df6 Disable TLSv1 and TLSv1.1 by default (#11237)
Motivation:

TLSv1 and TLSv1.1 is considered insecure. Let's follow the JDK and disable these by default

Modifications:

- Disable TLSv1 and TLSv1.1 by default when using OpenSSL.
- Add unit tests

Result:

Use only strong TLS versions by default when using OpenSSL
2021-05-11 10:43:55 +02:00
Norman Maurer
e54aeea1da Update conscrypt and add workaround for test failure (#11238)
Motivation:

Conscrypt not correctly filters out non support TLS versions which may lead to test failures.

Related to https://github.com/google/conscrypt/issues/1013

Modifications:

- Bump up to latest patch release
- Add workaround

Result:

No more test failures caused by conscrypt
2021-05-11 10:41:17 +02:00
Idel Pivnitskiy
b9685a63de Use PlatformDependent#normalizedOs() instead of reading os.name prop (#11239)
Motivation:

`PlatformDependent#normalizedOs()` already caches normalized variant of
the value of `os.name` system property. Instead of inconsistently
normalizing it in every case, use the utility method.

Modifications:

- `PlatformDependent`: `isWindows0()` and `isOsx0()` use `NORMALIZED_OS`;
- `PlatformDependent#normalizeOs(String)` define `darwin` as `osx`;
- `OpenSsl#loadTcNative()` does not require `equalsIgnoreCase` bcz `os`
is already normalized;
- Epoll and KQueue: `Native#loadNativeLibrary()` use `normalizedOs()`;
- Use consistent `Locale.US` for lower case conversion of `os.name`;
- `MacOSDnsServerAddressStreamProvider#loadNativeLibrary()` uses
`PlatformDependent.isOsx()`;

Result:

Consistent approach for `os.name` parsing.
2021-05-11 08:53:47 +02:00
skyguard1
e10c1af314 Add explicit null checks in OpenSslX509KeyManagerFactory (#11230)
Motivation:

We should add explicit null checks so its easier for people to understand why it throws.

Modification:

Add explicit checkNotNull(...)

Result:

Easier to understand for users why it fails.

Signed-off-by: xingrufei <xingrufei@sogou-inc.com>

Co-authored-by: xingrufei <xingrufei@sogou-inc.com>
2021-05-07 11:22:47 +02:00
Chris Vest
584d674acd Bump initial timeouts in SSLEngineTest (#11221)
Motivation:
We've seen (very rare) flaky test failures due to timeouts.
They are too rare to analyse properly, but a theory is that on overloaded, small cloud CI instances, it can sometimes take a surprising amount of time to start a thread.
It could be that the event loop thread is getting an unlucky schedule, and takes seconds to start, causing the timeouts to elapse.

Modification:
Increase the initial timeouts in the SSLEngineTest, that could end up waiting for the event loop thread to start.
Also fix a few simple warnings from Intellij.

Result:
Hopefully we will not see these tests be flaky again.
2021-05-05 15:15:17 +02:00
Scott Mitchell
382885538f ReferenceCountedOpenSslEngine unwrap handshake complete status fix (#11210)
Motivation:
ReferenceCountedOpenSslEngine may unwrap data and complete the handshake
in a single unwrap() call. However it may return HanshakeStatus of
HandshakeStatus of NEED_UNWRAP instead of FINISHED. This may result in
the SslHandler sending the unwrapped data up the pipeline before
notifying that the handshake has completed, and result in out-of-order
events.

Modifications:
- if ReferenceCountedOpenSslEngine handshake status is NEED_UNWRAP and
  produced data, or NEED_WRAP and consumed some data, we should call
  handshake() to get the current state.

Result:
ReferenceCountedOpenSslEngine correctly indicates when the handshake has
finished if at the same time data was produced or consumed.
2021-04-29 10:08:11 +02:00
Norman Maurer
c919b385e2 Re-enable running openssl (shared) tests on CI (#11197)
Motivation:

It turned out we didnt run the openssl tests on the CI when we used the non-static version of netty-tcnative.

Modifications:

- Upgrade netty-tcnative to fix segfault when using shared openssl
- Adjust tests to only run session cache tests when openssl supports it
- Fix some more tests to only depend on KeyManager if the underlying openssl version supports it

Result:

Run all openssl test on the CI even when shared library is used
2021-04-27 13:49:06 +02:00
skyguard1
f221e4d706 Enable Tlsv1.3 when using BouncyCastle ALPN support (#11193)
Motivation:

In the latest version of BouncyCastle, BCJSSE:'TLSv1.3' is now a supported protocol for both client and server. So should consider enabling TLSv1.3 when TLSv1.3 is available

Modification:

This pr is to enable TLSv1.3 when using BouncyCastle ALPN support, please review this pr,thanks

Result:

Enable TLSv1.3 when using BouncyCastle ALPN support

Signed-off-by: xingrufei <xingrufei@sogou-inc.com>

Co-authored-by: xingrufei <xingrufei@sogou-inc.com>
2021-04-26 10:05:56 +02:00
Boris Unckel
83297ed2ba Utilize i.n.u.internal.ObjectUtil to assert Preconditions (handler) (#11170) (#11180)
Motivation:

NullChecks resulting in a NullPointerException or IllegalArgumentException, numeric ranges (>0, >=0) checks, not empty strings/arrays checks must never be anonymous but with the parameter or variable name which is checked. They must be specific and should not be done with an "OR-Logic" (if a == null || b == null) throw new NullPointerEx.

Modifications:

* import static relevant checks
* Replace manual checks with ObjectUtil methods

Result:

All checks needed are done with ObjectUtil, some exception texts are improved.

Fixes #11170
2021-04-22 13:00:33 +02:00
Karsten Ohme
d4b9001d1f BouncyCastle ALPN support (#11157)
Motivation:

Under Android it was not possible to load a specific web page. It might be related to the (missing?) ALPN of the internal TLS implementation. BouncyCastle as a replacement works but this was not supported so far by Netty.
BouncyCastle also has the benefit to be a pure Java solution, all the other providers (OpenSSL, Conscrypt) require native libraries which are not available under Android at least.

Modification:

BouncyCastleAlpnSslEngine.java and support classes have been added. It is relying on the JDK code, hence some support classes had to be opened to prevent code duplication.

Result:

BouncyCastle can be used as TLS provider.

Co-authored-by: Norman Maurer <norman_maurer@apple.com>
2021-04-22 08:04:13 +02:00
Chris Vest
d1b896b701 Log fewer stack traces from initialisation code (#11164)
Motivation:
We are increasingly running in environments where Unsafe, setAccessible, etc. are not available.
When debug logging is enabled, we log a complete stack trace every time one of these initialisations fail.
Seeing these stack traces can cause people unnecessary concern.
For instance, people might have alerts that are triggered by a stack trace showing up in logs, regardless of its log level.

Modification:
We continue to print debug log messages on the result of our initialisations, but now we only include the full stack trace is _trace_ logging (or FINEST, or equivalent in whatever logging framework is configured) is enabled.

Result:
We now only log these initialisation stack traces when the lowest possible log level is enabled.

Fixes #7817
2021-04-19 09:17:32 +02:00
Scott Mitchell
59867fa0fd SslHandler LocalChannel read/unwrap reentry fix (#11156)
Motivation:
SslHandler invokes channel.read() during the handshake process. For some
channel implementations (e.g. LocalChannel) this may result in re-entry
conditions into unwrap. Unwrap currently defers updating the input
buffer indexes until the unwrap method returns to avoid intermediate
updates if not necessary, but this may result in unwrapping the same
contents multiple times which leads to handshake failures [1][2].

[1] ssl3_get_record:decryption failed or bad record mac
[2] ssl3_read_bytes:sslv3 alert bad record mac

Modifications:
- SslHandler#unwrap updates buffer indexes on each iteration so that if
  reentry scenario happens the correct indexes will be visible.

Result:
Fixes https://github.com/netty/netty/issues/11146
2021-04-16 08:33:13 -07:00
Scott Mitchell
3049eacc45 SslHandler consolidate state to save memory (#11160)
Motivation:
SslHandler has many independent boolean member variables. They can be
collapsed into a single variable to save memory.

Modifications:
- SslHandler boolean state consolidated into a single short variable.

Result:
Savings of 8 bytes per SslHandler (which is per connection) observed on
OpenJDK.
2021-04-15 09:49:40 -07:00
Idel Pivnitskiy
01768f0a65 Fire SslHandshakeCompletionEvent after the last decoded data chunk (#11148)
Motivation:

`SslHandler#unwrap` may produce `SslHandshakeCompletionEvent` if it
receives `close_notify` alert. This alert indicates that the engine is
closed and no more data are expected in the pipeline. However, it fires
the event before the last data chunk. As the result, further handlers
may loose data if they handle `SslHandshakeCompletionEvent`.
This issue was not visible before #11133 because we did not write
`close_notify` alert reliably.

Modifications:

- Add tests to reproduce described behavior;
- Move `notifyClosePromise` after fire of the last `decodeOut`;

Result:

`SslHandshakeCompletionEvent` correctly indicates that the engine is
closed and no more data are expected on the pipeline.
2021-04-15 14:10:19 +02:00
Chris Vest
6d35db57bd Less blocking in ChunkedStream (#11150)
Motivation:
We should avoid blocking in the event loop as much as possible.
The InputStream.read() is a blocking method, and we don't need to call it if available() returns a positive number.

Modification:
Bypass calling InputStream.read() if available() returns a positive number.

Result:
Fewer blocking calls in the event loop, in general, when ChunkedStream is used.
2021-04-12 13:42:15 +02:00
Scott Mitchell
ad7372b112 SslHandler wrap reentry bug fix (#11133)
Motivation:
SslHandler's wrap method notifies the handshakeFuture and sends a
SslHandshakeCompletionEvent user event down the pipeline before writing
the plaintext that has just been wrapped. It is possible the application
may write as a result of these events and re-enter into wrap to write
more data. This will result in out of sequence data and result in alerts
such as SSLV3_ALERT_BAD_RECORD_MAC.

Modifications:
- SslHandler wrap should write any pending data before notifying
  promises, generating user events, or anything else that may create a
  re-entry scenario.

Result:
SslHandler will wrap/write data in the same order.
2021-04-01 11:00:54 +02:00
Norman Maurer
98b9158f66 Don't bind to a specify port during SSLEngine tests as the port may also be used (#11102)
Motivation:

We should always bind to port 0 to ensure we not try to bind to a port that is already in use during our tests.
As we missed to do this in one test we did see test failures sometimes.

This showed up here:
https://pipelines.actions.githubusercontent.com/obCqqsCMwwGme5y2aRyYOiZvWeJK4O0EY5AYRUDMSELILdqEjV/_apis/pipelines/1/runs/1963/signedlogcontent/18?urlExpires=2021-03-19T12%3A41%3A21.4370902Z&urlSigningMethod=HMACV1&urlSignature=zL6O0msEkghT%2B0hOAL1lqLK66SR0Mp99QIjiau1yPe4%3D

Modifications:

- Use new InetSocketAddress(0)

Result:

Fixes possible test failures.
2021-03-20 12:46:08 +01:00
Scott Mitchell
e5ff6216ff SslHandler flushing with TCP Fast Open fix (#11077)
Motivation:
SslHandler owns the responsibility to flush non-application data
(e.g. handshake, renegotiation, etc.) to the socket. However when
TCP Fast Open is supported but the client_hello cannot be written
in the SYN the client_hello may not always be flushed. SslHandler
may not wrap/flush previously written/flushed data in the event
it was not able to be wrapped due to NEED_UNWRAP state being
encountered in wrap (e.g. peer initiated renegotiation).

Modifications:
- SslHandler to flush in channelActive() if TFO is enabled and
  the client_hello cannot be written in the SYN.
- SslHandler to wrap application data after non-application data
  wrap and handshake status is FINISHED.
- SocketSslEchoTest only flushes when writes are done, and waits
  for the handshake to complete before writing.

Result:
SslHandler flushes handshake data for TFO, and previously flushed
application data after peer initiated renegotiation finishes.
2021-03-14 14:27:10 +01:00
Norman Maurer
8a9320161c Support session cache for client and server when using native SSLEngine implementation (#10994)
Motivation:

At the moment we don't support session caching on the client side at all when using the native SSL implementation. We should at least allow to enable it.

Modification:

Allow to enable session cache for client side but disable ti by default due a JDK bug atm.

Result:

Be able to cache sessions on the client side when using native SSL implementation .
2021-03-07 19:35:31 +01:00
Norman Maurer
d209eb0e18 Move OpenSsl*X509Certificate to util package and rename it (#10955)
Motivation:

Creating certificates from a byte[] while lazy parse it is general useful and is also needed by https://github.com/netty/netty-incubator-codec-quic/pull/141

Modifications:

Move classes, rename these and make them public

Result:

Be able to reuse code
2021-03-07 19:30:17 +01:00
Norman Maurer
b1a8de0d7a Introduce SslContextOption which can be used for "optional" features … (#10981)
Motivation:

Some of the features we want to support can only be supported by some of the SslContext implementations. We should allow to configure these in a consistent way the same way as we do it with Channel / ChannelOption

Modifications:

- Add SslContextOption and add builder methods that take these
- Add OpenSslContextOption and define two options there which are specific to openssl

Result:

More flexible configuration and implementation of SslContext
2021-03-07 19:25:22 +01:00
Norman Maurer
2ce03e0a08 Fix NPE that can happen in the WriteTimeoutHandler when multiple executors are used (#11056)
Motivation:

In WriteTimeoutHandler we did make the assumption that the executor which is used to schedule the timeout is the same that is backing the write promise. This may not be true which will cause concurrency issues

Modifications:

Ensure we are on the right thread when try to modify the doubly-linked-list and if not schedule it on the right thread.

Result:

Fixes https://github.com/netty/netty/issues/11053
2021-03-04 15:27:02 +01:00
Norman Maurer
bcb165bccd Ensure removal from queue happens before writeAndFlush(...) is called (#11049)
Motivation:

We need to ensure that we call queue.remove() before we cal writeAndFlush() as this operation may cause an event that also touches the queue and remove from it. If we miss to do so we may see NoSuchElementExceptions.

Modifications:

- Call queue.remove() before calling writeAndFlush(...)
- Add unit test

Result:

Fixes https://github.com/netty/netty/issues/11046
2021-03-02 22:02:27 +01:00
Chris Vest
d8ae9dfea3
Bring forward build automation changes (#11052)
This brings forward the build and release automation changes from 4.1 (#10879, #10883, #10884, #10886, #10888, #10889, #10893, #10900, #10933, #10945, #10966, #10968, #11002, and #11019) to 5.0.

Details are as follows:

* Use Github workflows for CI (#10879)

Motivation:

We should just use GitHub Actions for the CI

Modifications:

- Adjust docker / docker compose files
- Add different workflows and jobs to deploy and build the project

Result:

Don't depend on external CI services

* Fix non leak build condition

* Only use build and deploy workflows for 4.1 for now

* Add deploy job for cross compiled aarch64 (#10883)

Motivation:

We should also deploy snapshots for our cross compiled native jars.

Modifications:

- Add job and docker files for deploying cross compiled native jars
- Ensure we map the maven cache into our docker containers

Result:

Deploy aarch64 jars and re-use cache

* Use correct docker-compose file to deploy cross compiled artifacts

* Use correct docker-compose task to deploy for cross compiled artifacts

* Split pr and normal build (#10884)

Motivation:

We should better use seperate workflows for PR and normal builds

Modifications:

- Split workflows
- Better cache reuse

Result:

Cleanup

* Only deploy snapshots for one arch

Motivation:

We need to find a way to deploy SNAPSHOTS for different arch with the same timestamp. Otherwise it will cause problems.

See https://github.com/netty/netty/issues/10887

Modification:

Skip all other deploys then x86_64

Result:

Users are able to use SNAPSHOTS for x86_6

* Use maven cachen when running analyze job (#10888)

Motivation:

To prevent failures to problems while downloading dependencies we shoud cache these

Modifications:

Add maven cache

Result:

No more failures due problems while downloading dependencies

* Also include one PR job that uses boringssl (#10886)

Motivation:

When validating PRs we should also at least run one job that uses boringssl

Modifications:

- Add job that uses boringssl
- Cleanup docker compose files
- Fix buffer leak in test

Result:

Also run with boringssl when PRs are validated

* Use matrix for job configurations (#10889)

Motivation:

We can use the matrix feature to define our jobs. This reduces a lot of config

Modification:

Use job matrix

Result:

Easier to maintain

* Correctly deploy artifacts that are build on different archs (#10893)

Motivation:

We need to take special care when deploying snapshots as we need to generate the jars in multiple steps

Modifications:

- Use the nexus staging pluging to stage jars locally in multiple steps
- Add extra job that will merge these staged jars and deploy these

Result:

Fixes https://github.com/netty/netty/issues/10887

* Dont use cron for PRs

Motivation:

It doesnt make sense to use cron for PRs

Modifications:

Remove cron config

Result:

Cleanup

* We run all combinations when validate the PR, let's just use one type for normal push

Motivation:

Let us just only use one build config when building the 4.1 branch.

Modifications:

As we already do a full validation when doing the PR builds we can just only use one build config for pushes to the "main" branches

Result:

Faster build times

* Update action-docker-layer-caching (#10900)

Motivation:

We are three releases behind.

Modifications:

Update to latest version

Result:

Use up-to-date action-docker-layer-caching version

* Verify we can load native modules and add job that verifies on aarch64 as well (#10933)

Motivation:

As shown in the past we need to verify we actually can load the native as otherwise we may introduce regressions.

Modifications:

- Add new maven module which tests loading of native modules
- Add job that will also test loading on aarch64

Result:

Less likely to introduce regressions related to loading native code in the future

* Let script fail if one command fail (#10945)

Motivation:

We should use `set -e` to ensure we fail the script if one command fails.

Modifications:

Add set -e to script

Result:

Fail fast

* Use action to report unit test errors (#10966)

Motivation:

To make it easier to understand why the build fails lets use an action that will report which unit test failed

Modifications:

- Replace custom script with action-surefire-report

Result:

Easier to understand test failures

* Use custom script to check for build failures (#10968)

Motivation:

It turns out we can't use the action to check for build failures as it can't be used when a PR is done from a fork. Let's just use our simple script.

Modifications:

- Replace action with custom script

Result:

Builds for PRs that are done via forks work again.

* Publish test results after PR run (#11002)

Motivation:

To make it easier to understand why a build failed let us publish the rest results

Modifications:

Use a new workflow to be able to publish the test reports

Result:

Easier to understand why a PR did fail

* Fix test reports name

* Add workflow to cut releases (#11019)

Motivation:

Doing releases manually is error-prone, it would be better if we could do it via a workflow

Modification:

- Add workflow to cut releases
- Add related scripts

Result:

Be able to easily cut a release via a workflow

* Update build for master branch

Motivation:
The build changes were brought forward from 4.1, and contain many things specific to 4.1.

Modification:
Changed baseline Java version from 8 to 11, and changed branch references from "4.1" to "master".

Result:
Builds should now work for the master branch.

Co-authored-by: Norman Maurer <norman_maurer@apple.com>
2021-03-02 17:44:03 +01:00
Chris Vest
ec18aa8731
Introduce ByteBufConvertible interface (#11036)
Motivation:
To make it possible to experiment with alternative buffer implementations, we need a way to abstract away the concrete buffers used throughout most of the Netty pipelines, while still having a common currency for doing IO in the end.

Modification:
- Introduce an ByteBufConvertible interface, that allow arbitrary objects to convert themselves into ByteBuf objects.
- Every place in the code, where we did an instanceof check for ByteBuf, we now do an instanceof check for ByteBufConvertible.
- ByteBuf itself implements ByteBufConvertible, and returns itself from the asByteBuf method.

Result:
It is now possible to use Netty with alternative buffer implementations, as long as they can be converted to ByteBuf.
This has been verified elsewhere, with an alternative buffer implementation.
2021-02-26 15:03:58 +01:00
Idel Pivnitskiy
b29df0f87a Simplify flushAtEnd flag computation in SslHandler#handlerAdded (#11025)
Motivation:

The `!fastOpen` part of `active || !fastOpen` is always false.

Modification:

- Remove `!fastOpen` and keep only `active` as a `flushAtEnd` flag for
`startHandshakeProcessing`;
- Update comment;

Result:

Simplified `flushAtEnd` flag computation in `SslHandler#handlerAdded`.
2021-02-16 14:05:26 +01:00
Chris Vest
d60b1651fc TCP Fast Open for clients (#11006)
Support TCP Fast Open for clients and make SslHandler take advantage

Motivation:
- TCP Fast Open allow us to send a small amount of data along side the initial SYN packet when establishing a TCP connection.
- The TLS Client Hello packet is small enough to fit in there, and is also idempotent (another requirement for using TCP Fast Open), so if we can save a round-trip when establishing TLS connections when using TFO.

Modification:
- Add support for client-side TCP Fast Open for Epoll, and also lowers the Linux kernel version requirements to 3.6.
- When adding the SslHandler to a pipeline, if TCP Fast Open is enabled for the channel (and the channel is not already active) then start the handshake early by writing it to the outbound buffer.
- An important detail to note here, is that the outbound buffer is not flushed at this point, like it would for normal handshakes. The flushing happens later as part of establishing the TCP connection.

Result:
- It is now possible for clients (on epoll) to open connections with TCP Fast Open.
- The SslHandler automatically detects when this is the case, and now send its Client Hello message as part of the initial data in the TCP Fast Open flow when available, saving a round-trip when establishing TLS connections.

Co-authored-by: Colin Godsey <crgodsey@gmail.com>
2021-02-15 14:29:03 +01:00
Norman Maurer
b43b67553e Disable flaky test (#11017)
Motivation:

The testGlobalWriteThrottle is flaky and failed our build multiple times now. Lets disable it for now until we had time to investigate

Modifications:

Disable flaky test

Result:

Less failures during build
2021-02-11 13:27:38 +01:00
Norman Maurer
42a2c9c831 Enable stateless resumption for TLSv1.3 by default when using OpenSSL / BoringSSL (#10997)
Motivation:

At the moment we always set SSL_OP_NO_TICKET when building our context. The problem with this is that this also disables resumption for TLSv1.3 in BoringSSL as it only supports stateless resumption for TLSv1.3 which uses tickets.
We should better clear this option when TLSv1.3 is enabled to be able to resume sessions. This is also inline with the OpenJDK which enables this for TLSv1.3 by default as well.

Modifications:

Check for enabled protocols and if TLSv1.3 is set clear SSL_OP_NO_TICKET.

Result:

Be able to resume sessions for TLSv1.3 when using BoringSSL.
2021-02-08 20:56:42 +01:00
Norman Maurer
9c2de76add Use Files.createTempFile(...) to ensure the file is created with proper permissions
Motivation:

File.createTempFile(String, String)` will create a temporary file in the system temporary directory if the 'java.io.tmpdir'. The permissions on that file utilize the umask. In a majority of cases, this means that the file that java creates has the permissions: `-rw-r--r--`, thus, any other local user on that system can read the contents of that file.
This can be a security concern if any sensitive data is stored in this file.

This was reported by Jonathan Leitschuh <jonathan.leitschuh@gmail.com> as a security problem.

Modifications:

Use Files.createTempFile(...) which will use safe-defaults when running on java 7 and later. If running on java 6 there isnt much we can do, which is fair enough as java 6 shouldnt be considered "safe" anyway.

Result:

Create temporary files with sane permissions by default.
2021-02-08 18:17:31 +01:00
Norman Maurer
f25b12077a Clarify who is responsible closing the InputStream once SslContext w… (#10983)
Motivation:

It was not 100% clear who is responsible calling close() on the InputStream.

Modifications:

Clarify javadocs.

Result:

Related to https://github.com/netty/netty/issues/10974

Co-authored-by: Chris Vest <christianvest_hansen@apple.com>
2021-02-01 11:50:55 +01:00
Norman Maurer
980ea9c351 Revert "Enable SSL_MODE_ENABLE_FALSE_START if jdkCompatibilityMode is false (#10407)" (#10980)
Motivation:

TLS_FALSE_START slightly changes the "flow" during handshake which may cause suprises for the end-user. We should better disable it by default again and later add a way to enable it for the user.

Modification:

This reverts commit 514d349e1f.

Result:

Restore "old flow" during TLS handshakes.
2021-01-29 19:56:12 +01:00
Norman Maurer
99fc0e486d Correctly filter out TLSv1.3 ciphers if TLSv1.3 is not enabled (#10919)
Motivation:

We didnt correctly filter out TLSv1.3 ciphers when TLSv1.3 is not enabled.

Modifications:

- Filter out ciphers that are not supported due the selected TLS version
- Add unit test

Result:

Fixes https://github.com/netty/netty/issues/10911

Co-authored-by: Bryce Anderson <banderson@twitter.com>
2021-01-28 11:01:57 +01:00
Carl Mastrangelo
c68af5c8f4 Don't use Fixed ports for UDP test (#10961)
Motivation:
If the given port is already bound, the PcapWriteHandlerTest will sometimes fail.

Modification:
Use a dynamic port using `0`, which is more reliable

Result:
Less Flaky
2021-01-26 08:31:22 +01:00
Norman Maurer
56d2635733 Override ALPN methods on ReferenceCountedOpenSslEngine (#10954)
Motivation:

We should override the get*ApplicationProtocol() methods in ReferenceCountedOpenSslEngine to make it easier for users to obtain the selected application protocol

Modifications:

Add missing overrides

Result:

Easier for the user to get the selected application protocol (if any)
2021-01-21 14:07:44 +01:00
Norman Maurer
6ae8cd6e44 Mark some methods as protected to make it easier to write own SslContext implementations (#10953)
Motivation:

We should expose some methods as protected to make it easier to write custom SslContext implementations.
This will be reused by the code for https://github.com/netty/netty-incubator-codec-quic/issues/97

Modifications:

- Add protected to some static methods which are useful for sub-classes
- Remove some unused methods
- Move *Wrapper classes to util package and make these public

Result:

Easier to write custom SslContext implementations
2021-01-21 14:07:28 +01:00
Norman Maurer
dc632e378f We need to ensure we always drain the error stack when a callback throws (#10920)
Motivation:

We need to ensure we always drain the error stack when a callback throws as otherwise we may pick up the error on a different SSL instance which uses the same thread.

Modifications:

- Correctly drain the error stack if native method throws
- Add a unit test which failed before the change

Result:

Always drain the error stack
2021-01-11 20:57:16 +01:00
Norman Maurer
4f7e6d4841 Workaround possible JDK bug in SSLEngineImpl when using TLSv1.3 that lead to multiple notifications (#10860)
Motivation:

When using the JDKs SSLEngineImpl with TLSv1.3 it sometimes returns HandshakeResult.FINISHED multiple times. This can lead to have SslHandshakeCompletionEvents to be fired multiple times.

Modifications:

- Keep track of if we notified before and if so not do so again if we use TLSv1.3
- Add unit test

Result:

Consistent usage of events
2020-12-15 08:07:19 +01:00
terrarier2111
38e06ef7ae Removed redundant assignment (#10853)
Motivation:

Found a redundant assignment.

Modification:

Removed the redundant assignment.

Result:

Minor performance improvement.
2020-12-10 10:29:49 +01:00
Norman Maurer
6c446d14fd OpenSsl.memoryAddress(...) should use internalNioBuffer(...) if it can't access the memoryAddress (#10818)
Motivation:

We can make use of internalNioBuffer(...) if we cant access the memoryAddress. This at least will reduce the object creations.

Modifications:

Use internalNioBuffer(...) and so reduce the GC

Result:

Less object creation if we can't access the memory address.
2020-11-25 10:32:43 +01:00
Norman Maurer
eeece4cfa5 Use http in xmlns URIs to make maven release plugin happy again (#10788)
Motivation:

https in xmlns URIs does not work and will let the maven release plugin fail:

```
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  1.779 s
[INFO] Finished at: 2020-11-10T07:45:21Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-release-plugin:2.5.3:prepare (default-cli) on project netty-parent: Execution default-cli of goal org.apache.maven.plugins:maven-release-plugin:2.5.3:prepare failed: The namespace xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" could not be added as a namespace to "project": The namespace prefix "xsi" collides with an additional namespace declared by the element -> [Help 1]
[ERROR]
```

See also https://issues.apache.org/jira/browse/HBASE-24014.

Modifications:

Use http for xmlns

Result:

Be able to use maven release plugin
2020-11-10 10:51:05 +01:00
Norman Maurer
70cbe74367 Use special exception when failing because the SSLEngine was closed (#10783)
Motivation:

Sometimes it would be helpful to easily detect if an operation failed due the SSLEngine already be closed.

Modifications:

Add special exception that is used when the engine was closed

Result:

Easier to detect a failure caused by a closed exception
2020-11-09 15:33:34 +01:00
Artem Smotrakov
51db4c9a9f Better hash algorithm in FingerprintTrustManagerFactory (#10683)
Motivation:

FingerprintTrustManagerFactory can only use SHA-1 that is considered
insecure.

Modifications:

- Updated FingerprintTrustManagerFactory to accept a stronger hash algorithm.
- Remove the constructors that still use SHA-1.
- Added a test for FingerprintTrustManagerFactory.

Result:

A user can now configure FingerprintTrustManagerFactory to use a
stronger hash algorithm.

Co-authored-by: Norman Maurer <norman_maurer@apple.com>
2020-10-26 14:37:33 +01:00
Artem Smotrakov
b8ae2a2af4 Enable nohttp check during the build (#10708)
Motivation:

HTTP is a plaintext protocol which means that someone may be able
to eavesdrop the data. To prevent this, HTTPS should be used whenever
possible. However, maintaining using https:// in all URLs may be
difficult. The nohttp tool can help here. The tool scans all the files
in a repository and reports where http:// is used.

Modifications:

- Added nohttp (via checkstyle) into the build process.
- Suppressed findings for the websites
  that don't support HTTPS or that are not reachable

Result:

- Prevent using HTTP in the future.
- Encourage users to use HTTPS when they follow the links they found in
  the code.
2020-10-23 15:26:25 +02:00
Norman Maurer
c061bd1798
Ensure SniCompletionEvent is not lost after onLookupComplete(...) (#10709)
Motivation:

In the master branch we fail fire* operations on the ChannelHandlerContext once the handler was removed. This is by design as it is "unspecified" what the semantics could be after the handler was removed and may lead to very hard to debug problems. Because of this we need to select the right ChannelHandlerContext for firing the event.

Modifications:

Choose a valid ChannelHandlerContext based on the state of the context of the handler

Result:

No more test failures
2020-10-20 09:01:15 +02:00
Norman Maurer
3f2c5ccd46 Replace deprecated Assert.assertThat(...) with MatcherAssert.assertThat(...) (#10699)
Motivation:

junit deprecated Assert.assertThat(...)

Modifications:

Use MatcherAssert.assertThat(...) as replacement for deprecated method

Result:

Less deprecation warnings
2020-10-18 14:55:21 +02:00
Aayush Atharva
87719f4fd9 Add null rule check in rules array of RuleBasedIpFilter (#10527)
Motivation:

We can filter out `null` rules while initializing the instance of `RuleBasedIpFilter` so we don't have to keep checking for `null` rules while iterating through `rules` array in `for loop` which is just a waste of CPU cycles.

Modification:
Added `null` rule check inside the constructor.

Result:
No more wasting CPU cycles on check the `null` rule each time in `for loop` and makes the overall operation more faster.
2020-10-17 20:23:52 +02:00
Artem Smotrakov
f0448d6a8a Fix or suppress LGTM findings (#10689)
Motivation:

LGTM reports multiple issues. They need to be triaged,
and real ones should be fixed.

Modifications:
- Fixed multiple issues reported by LGTM, such as redundant conditions,
  resource leaks, typos, possible integer overflows.
- Suppressed false-positives.
- Added a few testcases.

Result:

Fixed several possible issues, get rid of false alarms in the LGTM report.
2020-10-17 09:57:52 +02:00
Norman Maurer
647dbe0244 Fire SniCompletionEvent after onLookupComplete(...) was called (#10688)
Motivation:

Users may want to do special actions when onComplete(...) was called and depend on these once they receive the SniCompletionEvent

Modifications:

Switch order and so call onLookupComplete(...) before we fire the event

Result:

Fixes https://github.com/netty/netty/issues/10655
2020-10-15 21:01:18 +02:00
Chris Vest
6201c6d80f
Add a build profile for JDK 16 (#10675)
Motivation:
Java 16 will come around eventually anyway, and this makes it easier for people to experiment with Early Access builds.

Modification:
- Added Maven profiles for JDK 16 to relevant pom files.
- Removed the `--add-exports java.base/sun.security.x509=ALL-UNNAMED` argument when running tests; we've not needed it since the Java11-as-baseline PR landed.

Result:
Netty now builds on JDK 16 pre-releases (provided they've not broken compatibility in some way).
2020-10-12 16:42:40 +02:00
Chris Vest
a179db8066
Raise the Netty 5 minimum required Java version to Java 11. (#10650)
Raise the Netty 5 minimum required Java version to Java 11.

Motivation:
Java 11 has been out for some time, and Netty 5 is still some ways out.
There are also many good features in Java 11 that we wish to use, such as VarHandles, var-keyword, and the module system.
There is no reason for Netty 5 to not require Java 11, since Netty 4.x will still be supported for the time being.

Modification:
Remove everything in the pom files related to Java versions older than Java 11.
Remove the animal-sniffer plug-in and rely on the `--release` compiler flag instead.
Remove docker files related to Java versions older than Java 11.
Remove the copied SCTP APIs -- we should test this commit independently on Windows.
Remove the OpenJdkSelfSignedCertGenerator.java file and just always use Bouncy Castle for generating self-signed certificates for testing.
Make netty-testsuite tests pass by including Bouncy Castle as a test dependency, so we're able to generate our self-signed certificate.

Result:
Java 11 is now the minimum required Java version.
2020-10-12 14:13:01 +02:00
Aayush Atharva
09dc15aaa0 Add Close method for closing OutputStream and PcapWriteHandler (#10638)
Motivation:
We should implement the Closeable method to properly close `OutputStream` and `PcapWriteHandler`. So whenever `handlerRemoved(ChannelHandlerContext)` is called or the user wants to stop the Pcap writes into `OutputStream`, we have a proper method to close it otherwise writing data to close `OutputStream` will result in `IOException`.

Modification:
Implemented `Closeable` in `PcapWriteHandler` which calls `PcapWriter#close` and closes `OutputStream` and stops Pcap writes.

Result:
Better handling of Pcap writes.
2020-10-08 11:53:13 +02:00
Chris Vest
86c8f24d9a
Replace UNSAFE.throwException with alternatives supported on Java 8 (#10629)
Motivation:
 We wish to use Unsafe as little as possible, and Java 8 allows us
 to take some short-cuts or play some tricks with generics,
 for the purpose of working around having to declare all checked
 exceptions. Ideally all checked exceptions would be declared, but
 the code base is not ready for that yet.

Modification:
 The call to UNSAFE.throwException has been removed, so when we need
 that feature, we instead use the generic exception trick.
 In may cases, Java 8 allows us to throw Throwable directly. This
 happens in cases where no exception is declared to be thrown in a
 scope.
 Finally, some warnings have also been fixed, and some imports have
 been reorganised and cleaned up while I was modifying the files
 anyway.

Result:
 We no longer use Unsafe for throwing any exceptions.
2020-10-02 08:29:07 +02:00
Norman Maurer
ba43065482 Respect the Provider when detecting if TLSv1.3 is used by default / supported (#10621)
Motivation:

We need to take the Provider into account as well when trying to detect if TLSv1.3 is used by default / supported

Modifications:

- Change utility method to respect provider as well
- Change testcode

Result:

Less error-prone tests
2020-09-29 20:49:11 +02:00
Norman Maurer
d77edcb6e2 Use SelfSignedCertificate to fix test-failure related to small key size (#10620)
Motivation:

Some JDKs dissallow the usage of keysizes < 2048, so we should not use such small keysizes in tests.

This showed up on fedora 32:

```
Caused by: java.security.cert.CertPathValidatorException: Algorithm constraints check failed on keysize limits. RSA 1024bit key used with certificate: CN=tlsclient.  Usage was tls client
        at sun.security.util.DisabledAlgorithmConstraints$KeySizeConstraint.permits(DisabledAlgorithmConstraints.java:817)
        at sun.security.util.DisabledAlgorithmConstraints$Constraints.permits(DisabledAlgorithmConstraints.java:419)
        at sun.security.util.DisabledAlgorithmConstraints.permits(DisabledAlgorithmConstraints.java:167)
        at sun.security.provider.certpath.AlgorithmChecker.check(AlgorithmChecker.java:326)
        at sun.security.provider.certpath.PKIXMasterCertPathValidator.validate(PKIXMasterCertPathValidator.java:125)
        ... 23 more
```

Modifications:

Replace hardcoded keys / certs with SelfSignedCertificate

Result:

No test-failures related to small key sizes anymore.
2020-09-29 13:06:54 +02:00
Norman Maurer
fc656f605f Only set the keymaterial once and correctly handle errors during keymaterial setting on the client-side as well (#10613)
Motivation:

We should stop as soon as we were able to set the key material on the server side as otherwise we may select keymaterial that "belongs" to a less prefered cipher. Beside this it also is just useless work.
We also need to propagate the exception when it happens during key material selection on the client side so openssl will produce the right alert.

Modifications:

- Stop once we were able to select a key material on the server side
- Ensure we not call choose*Alias more often then needed
- Propagate exceptions during selection of the keymaterial on the client side.

Result:

Less overhead and more correct behaviour
2020-09-29 09:28:25 +02:00
Norman Maurer
ced5faa440 Correctly report back when we fail to select the key material and ens… (#10610)
Motivation:

We need to let openssl know that we failed to find the key material so it will produce an alert for the remote peer to consume. Beside this we also need to ensure we wrap(...) until we produced everything as otherwise the remote peer may see partial data when an alert is produced in multiple steps.

Modifications:

- Correctly throw if we could not find the keymaterial
- wrap until we produced everything
- Add test

Result:

Correctly handle the case when key material could not be found
2020-09-29 09:21:35 +02:00
Norman Maurer
d92d92a923 Filter out duplicates before trying to find the alias to use (#10612)
Motivation:

Calling chooseServerAlias(...) may be expensive so we should ensure we not call it multiple times for the same auth methods.

Modifications:

Remove duplicated from authMethods before trying to call chooseServerAlias(...)

Result:

Less performance overhead during key material selection
2020-09-25 19:13:33 +02:00
Yosfik Alqadri
c9e42106d1 Fix Javadoc Typo (#10603)
Motivation:

Following Javadoc standard

Modification:

Change from `@param KeyManager` to `@param keyManager`

Result:

The `@param` matches the actual parameter variable name
2020-09-24 15:28:30 +02:00
Aayush Atharva
379be5085f Add PcapWriteHandler Support (#10498)
Motivation:
Write TCP and UDP packets into Pcap `OutputStream` which helps a lot in debugging.

Modification:
Added TCP and UDP Pcap writer.

Result:
New handler can write packets into an `OutputStream`, e.g. a file that can be opened with Wireshark.

Fixes #10385.
2020-09-11 16:15:39 +02:00
Norman Maurer
338b7ce314 Revert "Support session cache for client and server when using native SSLEngine implementation (#10331)"
Motivation:

This reverts commit 7bf1ffb2d4 as it turns out it introduced a big performance regression.

Modifications:

Revert 7bf1ffb2d4

Result:

Performance of TLS is back to normal
2020-09-03 08:33:22 +02:00
Aayush Atharva
a49afaef35 Binary search based IpSubnetFilter (#10492)
Motivation:

`IpSubnetFilter` uses Binary Search for IP Address search which is fast if we have large set of IP addresses to filter.

Modification:

Added `IpSubnetFilter` which takes `IpSubnetFilterRule` for filtering.

Result:
Faster IP address filter.
2020-09-01 11:15:19 +02:00
Aayush Atharva
890c261759 Fix JavaDoc of RuleBasedIpFilter (#10521)
Motivation:

`RuleBasedIpFilter` had JavaDoc `{@link #channelRejected(ChannelHandlerContext, SocketAddress)}` instead of `{@link AbstractRemoteAddressFilter#channelRejected(ChannelHandlerContext, SocketAddress)}`.

Modification:
Added `AbstractRemoteAddressFilter` reference.

Result:
Fixed JavaDoc error and made documentation more clear.
2020-09-01 11:02:52 +02:00
Michael Nitschinger
571e61ab36 Allow to customize how SslMasterKeyHandler is enabled (#10513)
Motivation:

Right now after a SslMasterKeyHandler is added to the pipeline,
it also needs to be enabled via a system property explicitly. In
some environments where the handler is conditionally added to
the pipeline this is redundant and a bit confusing.

Modifications:

This changeset keeps the default behavior, but allows child
implementations to tweak the way on how it detects that it
should actually handle the event when it is being raised.

Also, removed a private static that is not used in the wireshark
handler.

Result:

Child implementations can be more flexible in deciding when
and how the handler should perform its work (without changing
any of the defaults).
2020-08-31 10:35:06 +02:00
Norman Maurer
514d349e1f Enable SSL_MODE_ENABLE_FALSE_START if jdkCompatibilityMode is false (#10407)
Motivation:

To reduce latency and RTTs we should use TLS False Start when jdkCompatibilityMode is not required and its supported

Modifications:

Use SSL_MODE_ENABLE_FALSE_START when jdkCompatibilityMode is false

Result:

Less RTTs and so lower latency when TLS False Start is supported
2020-08-18 19:01:09 +02:00
Norman Maurer
ccca21015b Cleanup Conscrypt initialization (#10466)
Motivation:

How we init our static fields in Conscrypt was kind of error-prone and may even lead to NPE later on when methods were invoked out of order.

Modifications:

- Move all the init code to a static block
- Remove static field which is not needed anymore

Result:

Cleanup and also fixes https://github.com/netty/netty/issues/10413
2020-08-11 21:08:17 +02:00
Norman Maurer
6aa30b2b6e
Fix invokeExact(...) usage in JdkAlpnSslUtils (#10471)
Motivation:

fd0d06e introduced the usage of MethodHandles and so also introduced some usages of invokeExact(...). Unfortunally when calling this method we missed to also cast the return value in the static init block of JdkAlpnSslUtils which lead to an exception to be thrown as the JVM did assume the return value is void which is not true.

Modifications:

Correctly cast the return value of invokeExact(...)

Result:

ALPN can be used in master with JDK again
2020-08-11 16:04:15 +02:00
Norman Maurer
1208f27070 Revert #10326 due regression in FlowControlHandler
Motivation:

This reverts commit b559711f3e due regression introduced by it.

Modification:

Revert commit

Result:

Fixes https://github.com/netty/netty/issues/10464
2020-08-11 09:08:13 +02:00
Norman Maurer
43ae49ed78 Enable TLS1.3 by default of JDK SSLEngine implementation does by default (#10451)
Motiviation:

When TLSv1.3 was introduced almost 2 years ago, it was decided to disable it by default, even when it's supported by the underlying TLS implementation.

TLSv13 is pretty stable now in Java (out of the box in Java 11, OpenJSSE for Java 8, BoringSSL and OpenSSL) and may be enabled by default.

Modifications:

Ensure TLSv13 is enabled by default when the underyling JDK SSLEngine implementation enables it as well

Result:

TLSv1.3 is now enabled by default, so users don't have to explicitly enable it.

Co-authored-by: Stephane Landelle <slandelle@gatling.io>
2020-08-10 14:04:29 +02:00
Norman Maurer
a2ebd65322 Fix compilation error on JDK 15 (#10462)
Motivation:

AlgorithmId.sha256WithRSAEncryption_oid was removed in JDK15 and later so we should not depend on it as otherwise we will see compilation errors

Modifications:

Replace AlgorithmId.sha256WithRSAEncryption_oid usage with specify the OID directly

Result:

Compiles on JDK15+
2020-08-10 14:03:43 +02:00
skyguard1
9c5dbfd1b6 Replace Class.getClassLoader with io.netty.util.internal.PlatformDependent.getClassLoader in Openssl (#10454)
Motivation:

Replace Class.getClassLoader with io.netty.util.internal.PlatformDependent.getClassLoader in Openssl so it also works when a SecurityManager is in place

Modification:

Replace Class.getClassLoader with io.netty.util.internal.PlatformDependent.getClassLoader in Openssl

Result:

No issues when a SecurityManager is in place
2020-08-06 09:03:01 +02:00
Norman Maurer
220995f155 Make the TLSv1.3 check more robust and not depend on the Java version… (#10409)
Motivation:

TLSv1.3 is not strictly limited to Java11+ anymore as different vendors backported TLSv1.3 to Java8 as well. We should ensure we make the detection of if TLSv1.3 is supported not depend on the Java version that is used.

Modifications:

- Add SslProvider.isTlsv13Supported(...) and use it in tests to detect if we should run tests against TLSv1.3 as well
- Adjust testcase to work on latest JDK 8 release as well

Result:

Correct detection of TLSv1.3 support even if Java version < 11.
2020-07-17 07:17:56 +02:00
Norman Maurer
7bf1ffb2d4 Support session cache for client and server when using native SSLEngine implementation (#10331)
Motivation:

At the moment we don't support session caching for client side when using native SSLEngine implementation and our implementation of SSLSessionContext is incomplete.

Modification:

- Consume netty-tcnative changes to be able to cache session in an external cache
- Add and adjust unit tests to test session caching
- Add an in memory session cache that is hooked into native SSLEngine

Result:

Support session caching on the client and server side
2020-07-14 15:20:44 +02:00
Norman Maurer
47bbc590ee jdk.tls.client.enableSessionTicketExtension must be respected by OPENSSL and OPENSSL_REFCNT SslProviders (#10401)
Motivation:

jdk.tls.client.enableSessionTicketExtension property must be respect by OPENSSL and OPENSSL_REFCNT SslProvider to ensure a consistent behavior. Due a bug this was not the case and it only worked for OPENSSL_REFCNT but not for OPENSSL.

Modifications:

Move the property check into static method that is used by both

Result:

Correctly respect jdk.tls.client.enableSessionTicketExtension
2020-07-13 16:17:41 +02:00
Norman Maurer
165ade5d9f Correctly return NEED_WRAP if we produced some data even if we could not consume any during SSLEngine.wrap(...) (#10396)
Motivation:

At the moment we may report BUFFER_OVERFLOW when wrap(...) fails to consume data but still prodce some. This is not correct and we should better report NEED_WRAP as we already have produced some data (for example tickets). This way the user will try again without increasing the buffer size which is more correct and may reduce the number of allocations

Modifications:

Return NEED_WRAP when we produced some data but not consumed any.

Result:

Fix ReferenceCountedOpenSslEngine.wrap(...) state machine
2020-07-09 08:52:30 +02:00
Daniel Zou
fca62510f3 Modify OpenSSL native library loading to accommodate GraalVM (#10395)
**Motivation:**

We are interested in building Netty libraries with Ahead-of-time compilation with GraalVM. We saw there was [prior work done on this](https://github.com/netty/netty/search?q=graalvm&unscoped_q=graalvm). We want to introduce a change which will unblock GraalVM support for applications using netty and `netty-tcnative`.

This solves the error [that some others encounter](https://github.com/oracle/graal/search?q=UnsatisfiedLinkError+sslOpCipherServerPreference&type=Issues):

```
Exception in thread "main" java.lang.UnsatisfiedLinkError: io.grpc.netty.shaded.io.netty.internal.tcnative.NativeStaticallyReferencedJniMethods.sslOpCipherServerPreference()I [symbol: Java_io_grpc_netty_shaded_io_netty_internal_tcnative_NativeStaticallyReferencedJniMethods_sslOpCipherServerPreference or Java_io_grpc_netty_shaded_io_netty_internal_tcnative_NativeStaticallyReferencedJniMethods_sslOpCipherServerPreference__]
```

**Modification:**

The root cause of the issue is that in the tcnative libraries, the [`SSL.java` class](783a8b6b69/openssl-dynamic/src/main/java/io/netty/internal/tcnative/SSL.java (L67)) references a native call in the static initialization of the class - the method `sslOpCipherServerPreference()` on line 67 is used to initialize the static variable. But you see that `OpenSsl` also uses[ `SSL.class` to check if `netty-tcnative` is on the classpath before loading the native library](cbe238a95b/handler/src/main/java/io/netty/handler/ssl/OpenSsl.java (L123)). 

So this is the problem because in ahead-of-time compilation, when it references the SSL class, it will try to load those static initializers and make the native library call, but it cannot do so because the native library has not been loaded yet since the `SSL` class is being referenced to check if the library should be loaded in the first place.

**Solution:** So the proposed solution here is to just choose a different class in the `tcnative` package which does not make a native library call during static initialization. I just chose `SSLContext` if this is OK.

This should have no negative effects other than unblocking the GraalVM use-case.

**Result:**

It fixes the unsatisfied link error. It will fix error for future users; it is a common error people encounter.
2020-07-08 10:47:10 +02:00
Norman Maurer
0c79863db5 Update to netty-tcnative 2.0.31.Final and make SslErrorTest more robust (#10392)
Motivation:

There was a new netty-tcnative release which we should use. Beside this the SSLErrorTest was quite fragile and so should be adjusted.

Modifications:

Update netty-tcnative and adjust test

Result:

Use latest netty-tcnative release
2020-07-07 10:56:42 +02:00
Norman Maurer
8950144567 Correctly include TLS1.3 ciphers in the enabled ciphersuites when using BoringSSL (#10388)
Motivation:

BoringSSL behaves differently then OpenSSL and not include any TLS1.3 ciphers in the returned array when calling SSL_get_ciphers(...). This is due the fact that it also not allow to explicit configure which are supported and which not for TLS1.3. To mimic the behaviour expected by the SSLEngine API we should workaround this.

Modifications:

- Add a unit test that verifies enabled protocols / ciphers
- Add special handling for BoringSSL and tls1.3

Result:

Make behaviour consistent
2020-07-02 21:37:04 +02:00
Norman Maurer
91c75a4a8f Ensure we preserve the original cause of the SSLException in all case… (#10372)
Motivation:

We did not correctly preserve the original cause of the SSLException when all the following are true:

 * SSL_read failed
 * handshake was finished
 * some outbound data was produced durigin SSL_read (for example ssl alert) that needs to be picked up by wrap(...)

Modifications:

Ensure we correctly preserve the original cause of the SSLException and rethrow it once we produced all data via wrap(...)

Result:

Be able to understand why we see an error in all cases
2020-06-25 22:08:57 +02:00
Norman Maurer
b2726c4919 X509TrustManager with OPENSSL provider is not wrapped with hostname verification if Conscrypt is inserted in the first place (#10375)
Motivation:

Modifications:

Directly specify the provider which is used to create the SSLContext

Result:

Fixes https://github.com/netty/netty/issues/10374
2020-06-25 20:40:49 +02:00
Norman Maurer
163c2fc220 Ensure we feed all data to the SSLEngine during handshaking in our tests (#10373)
Motivation:

Due a bug in our test we may dropped data on the floor which are generated during handshaking (or slightly after). This could lead to corrupt state in the engine itself and so fail tests. This is especially true for TLS1.3 which generates the sessions on the server after the "actual handshake" is done.

Modifications:

Contine with wrap / unwrap until all data was consumed

Result:

Correctly feed all data to the engine during testing
2020-06-25 14:56:54 +02:00
Norman Maurer
f051b0c297 Ensure ApplicationProtocolNegotiationHandler does handle handshake fa… (#10363)
Motivation:

When ApplicationProtocolNegotiationHandler is in the pipeline we should expect that its handshakeFailure(...) method will be able to completly handle the handshake error. At the moment this is not the case as it only handled SslHandshakeCompletionEvent but not the exceptionCaught(...) that is also triggered in this case

Modifications:

- Call handshakeFailure(...) in exceptionCaught and so fix double notification.
- Add testcases

Result:

Fixes https://github.com/netty/netty/issues/10342
2020-06-24 08:47:31 +02:00
Norman Maurer
e76fc0a577 Fix compilation error in test due bad cherry-pick
Motivation:

I did not correctly adjust the code before cherry-pick, causing a compilation error in the test

Modifications:

Use ChannelHandler

Result:

No more compilation error
2020-06-12 11:21:47 +02:00
Kareem Ali
121daab927 Motivation: (#10326)
The current FLowControlHandler keeps a flag to track whether a read() call is pending.
This could lead to a scenario where you call read multiple times when the queue is empty,
and when the FlowControlHandler Queue starts getting messages, channelRead will be fired only once,
when we should've fired x many times, once for each time the handlers downstream called read().

Modifications:

Minor change to replace the boolean flag with a counter and adding a unit test for this scenario.

Result:

I used TDD, so I wrote the test, made sure it's failing, then updated the code and re-ran the test
to make sure it's successful after the changes.

Co-authored-by: Kareem Ali <kali@localhost.localdomain>
2020-06-04 19:18:23 +02:00
Norman Maurer
feeb3ee920 Update test to directly check for SslHandshakeTimeoutException (#10339)
Motivation:

9b7e091 added a special SSLHandshakeException sub-class to signal handshake timeouts but we missed to update a testcase to directly assert the type of the exception.

Modifications:

Assert directly that SslHandshakeTimeoutException is used

Result:

Test cleanup
2020-06-04 18:30:25 +02:00
feijermu
8bbd89d72a Fix a test case problem: testSwallowedReadComplete(...) may fail with an AssertionError sometimes. (#10313)
Motivation:

It seems that `testSwallowedReadComplete(...)` may fail with an AssertionError sometimes after my tests. The relevant stack trace is as follows:

```
java.lang.AssertionError: expected:<IdleStateEvent(READER_IDLE, first)> but was:<null>
	at org.junit.Assert.fail(Assert.java:88)
	at org.junit.Assert.failNotEquals(Assert.java:834)
	at org.junit.Assert.assertEquals(Assert.java:118)
	at org.junit.Assert.assertEquals(Assert.java:144)
	at io.netty.handler.flow.FlowControlHandlerTest.testSwallowedReadComplete(FlowControlHandlerTest.java:478)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:89)
	at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:41)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:542)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:770)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:464)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:210)

```
Obviously the `readerIdleTime` of `IdleStateHandler` and the thread sleep time before `EmbeddedChannel.runPendingTasks` are both 100ms. And if `userEvents.poll()` happened before `userEvents.add(...)` or no `IdleStateEvent` fired at all, this test case would fail.

Modification:

Sleep for a little more time before running all pending tasks in the `EmbeddedChannel`.

Result:

Fix the problem of small probability of failure.
2020-06-03 09:19:54 +02:00
Scott Mitchell
32a77394a3 SslHandler#wrap to preserve exception if SSLEngine is closed (#10327)
Motivation:
SslHandler currently throws a general SSLException if a wrap attempt
fails due to the SSLEngine being closed. If writes are queued the
failure rational typically requires more investigation to track down the
original failure from a previous event. We may have more informative
rational for the failure and so we should use it.

Modifications:
- SslHandler#wrap to use failure information from the handshake or prior
transport closure if available

Result:
More informative exceptions from SslHandler#wrap if the SSLEngine has
been previously closed.
2020-06-02 09:51:43 +02:00
feijermu
9fe5f6f958 Dequeue all cached messages and destroy the queue instance after removing the FlowControlHandler from channel pipeline. (#10304)
Motivation:

The `FlowControlHandler` may cache the received messages in a queue in order to do the flow control. However, if this handler is manually removed from pipeline during runtime, those cached messages might not be passed to the next channel handler forever.

Modification:

Dequeue all these cached messages and call `ChannelHandlerContext.fireChannelRead(...)` in `handlerRemoved(...)` method.

Result:
Avoid losing the received messages.
2020-05-19 10:05:54 +02:00
Idel Pivnitskiy
1fe92e3077 Do not require BoringSSL for testSessionTicketsWithTLSv12AndNoKey (#10301)
Motivation:

`SslHandlerTest.testSessionTicketsWithTLSv12AndNoKey` does not require
BoringSSL and works with OpenSSL as well.

Modifications:

- Remove assume statement that expected BoringSSL;

Result:

Test works for any implementation of `OPENSSL` provider.
2020-05-18 14:25:40 +02:00
Norman Maurer
51805e3248 Check if SSL pointer was freed before using it in RefereceCountedOpenSslEngine in all cases (#10299)
Motivation:

To ensure we not crash in all cases we should better check that the SSL pointer was not freed before using it.

Modifications:

Add missing `isDestroyed()` checks

Result:

Ensure we not crash due usage of freed pointer.
2020-05-18 09:41:10 +02:00
sanjaypujare
f29b187159 Make ReferenceCountedOpenSslContext.setUseTasks public (#10289)
Motivation:
make the existing setter `ReferenceCountedOpenSslContext.setUseTasks` public

Modification:

Added the `public` qualified and removed the comment "for tests only"

Result:

Fixes #10288
2020-05-15 10:04:26 +02:00
Norman Maurer
fc84c9cc47 Respect jdk.tls.client.enableSessionTicketExtension and jdk.tls.server.enableSessionTicketExtension when using native SSL impl (#10296)
Motivation:

We should respect jdk.tls.client.enableSessionTicketExtension and jdk.tls.server.enableSessionTicketExtension when using the native SSL implementation as well to make the usage of it easier and more consistent. These properties were introduced by JDK13:

https://seanjmullan.org/blog/2019/08/05/jdk13

Modifications:

Check if the properties are set to true and if so enable tickets

Result:

Easier to enable tickets and be more consistent
2020-05-15 10:03:30 +02:00
Norman Maurer
6b0655aecb Allow to have the session tickets automatically managed by the native library
Motivation:

BoringSSL supports to automatically manage the session tickets to be used and so also rotate them etc. This is often prefered by users as it removed some complexity. We should support to make use of this.

Modifications:

- Allow to have setSessionTickets() called without an argument or an empty array
- Add tests

Result:

Easier usage of session tickets
2020-05-14 12:09:49 +02:00
Norman Maurer
11b63f8744 OpenSslSession.getLocalCertificates() and getLocalPrincipal() must r… (#10275)
Motivation:

OpenSslSession.getLocalCertificates() and  getLocalPrincipal() must return null on client side if mTLS is not used as stated in the API documentation. At the moment this is not always the case

Modifications:

- Ensure we only return non-null if mTLS is used
- Add unit tests

Result:

Follow SSLSession API contract
2020-05-13 07:16:56 +02:00