Commit Graph

1324 Commits

Author SHA1 Message Date
Norman Maurer
7c57c4be17
Introduce OpenSslAsyncPrivateKeyMethod which allows to asynchronously sign / decrypt the private key (#11390)
Motivation:

At the moment we only support signing / decrypting the private key in a synchronous fashion. This is quite limited as we may want to do a network call to do so on a remote system for example.

Modifications:

- Update to latest netty-tcnative which supports running tasks in an asynchronous fashion.
- Add OpenSslAsyncPrivateKeyMethod interface
- Adjust SslHandler to be able to handle asynchronous task execution
- Adjust unit tests to test that asynchronous task execution works in all cases

Result:

Be able to asynchronous do key signing operations
2021-07-06 08:25:31 +02:00
Norman Maurer
ae7944459d Fix test failure introduced by 7cc31b8653 2021-07-05 08:37:41 +02:00
Norman Maurer
7cc31b8653
Use Junit5 for handler module (#11444)
Motivation:

We should aim to use junit5 everywhere

Modifications:

Migrate most of the handler module to use junit5

Result:

Part of #10757
2021-07-02 15:06:06 +02:00
Aayush Atharva
750d23583c
Add ALPN Buffering to support HTTP/2 Prior Knowledge (#11407)
Motivation:
Currently, Netty cannot handle HTTP/2 Preface messages if the client used the Prior knowledge technique. In Prior knowledge, the client sends an HTTP/2 preface message immediately after finishing TLS Handshake. But in Netty, when TLS Handshake is finished, ALPNHandler is triggered to configure the pipeline. And between these 2 operations, if an HTTP/2 preface message arrives, it gets dropped.

Modification:

Buffer messages until we are done with the ALPN handling.

Result:
Fixes #11403.

Co-authored-by: Norman Maurer <norman_maurer@apple.com>
2021-07-01 14:10:52 +02:00
wujimin
3226e77485
Add support for GMSSL (#11406) (#11410)
__Motivation__

Add support for GMSSL protocol to SslUtils.

__Modification__

Modify `SslUtils.getEncryptedPacketLength(ByteBuf buffer, int offset)` to get packet length when protocol is GMSSL.
Modify `SslUtils.getEncryptedPacketLength(ByteBuffer buffer)` to get packet length when protocol is GMSSL.

__Result__

`SslUtils.getEncryptedPacketLength` now supports GMSSL protocol. Fixes https://github.com/netty/netty/issues/11406
2021-07-01 08:17:45 +02:00
Unev
d8ad931488 ByteBufFormat constructor for LoggingHandler (#11420)
__Motivation__

`LoggingHandler` misses a constructor variant that only takes `ByteBufFormat`

__Modification__

Added the missing constructor variant.

__Result__

`LoggingHandler` can be constructed with `ByteBufFormat` only.

Co-authored-by: Nitesh Kant <nitesh_kant@apple.com>
2021-06-29 10:28:27 -07:00
Norman Maurer
a71ec15fc4
Correctly use HandshakeStatus.NEED_WRAP when a handshake failed and a alert was produced (#11412)
Motivation:

We need to ensure we always "consumed" all alerts etc via SSLEngine.wrap(...) before we teardown the engine. Failing to do so may lead to a situation where the remote peer will not be able to see the actual cause of the handshake failure but just see the connection being closed.

Modifications:

Correctly return HandshakeStatus.NEED_WRAP when we need to wrap some data first before we shutdown the engine because of a handshake failure.

Result:

Fixes https://github.com/netty/netty/issues/11388
2021-06-24 10:05:47 +02:00
Norman Maurer
0c9a86db81
Log if the user tries to explicit set TLSv1.3 ciphers and using BoringSSL (#11392)
Motivation:

At the moment BoringSSL doesnt support explicit set the TLSv1.3 ciphers that should be used. If TLSv1.3 should be used it just enables all ciphers. We should better log if the user tries to explicit set a specific ciphers and using BoringSSL to inform the user that what is tried doesnt really work.

Modifications:

Log if the user tries to not use all TLSv1.3 ciphers and use BoringSSL

Result:

Easier for the user to understand why always all TLSv1.3 ciphers are enabled when using BoringSSL

Co-authored-by: Trustin Lee <trustin@gmail.com>
2021-06-21 08:54:58 +02:00
skyguard1
dbf56c527d
Fix IpSubnetFilterRule with IPv6 Default Route does not accept all IPv6 addresses (#11351)
Motivation:

In this issue(https://github.com/netty/netty/issues/11349 ),IpSubnetFilterRule needs to support ipv6 reserved addresses, such as 8000::, but the current implementation does not support

Modification:

Added support for default rule

Result:

Fixes https://github.com/netty/netty/issues/11349

Signed-off-by: xingrufei <xingrufei@sogou-inc.com>
2021-06-07 08:06:52 +02:00
skyguard1
d334146ccd
Add default block in IdleStateHandler (#11341)
Motivation:

We should have a default case in every switch block.

Modification:

Add default block in IdleStateHandler

Result:

Cleanup

Signed-off-by: xingrufei <xingrufei@sogou-inc.com>
2021-06-01 08:01:32 +02:00
Netty Project Bot
02142a2510 [maven-release-plugin] prepare for next development iteration 2021-05-19 12:09:18 +00:00
Netty Project Bot
9e06241ddb [maven-release-plugin] prepare release netty-4.1.65.Final 2021-05-19 12:09:14 +00:00
Norman Maurer
eca0e160ed
Add license header to our scripts and workflows (#11282)
Motivation:

We should have license header whenever possible.

Modifications:

Add header to scripts and workflow config

Result:

More clear licensing
2021-05-19 14:06:42 +02:00
Norman Maurer
08dbd72758
Fail the build if we can't load the OpenSSL library (#11269)
Motivation:

We should better fail the build if we can't load the OpenSSL library to ensure we not introduce a regression at some point related to native library loading

Modifications:

Remove usages of assumeTrue and let the tests fail if we cant load the native lib

Result:

Ensure we not regress
2021-05-19 08:19:15 +02:00
Netty Project Bot
d5c81d9a3e [maven-release-plugin] prepare for next development iteration 2021-05-17 07:04:14 +00:00
Netty Project Bot
67bedf0efe [maven-release-plugin] prepare release netty-4.1.64.Final 2021-05-17 07:04:08 +00:00
Aayush Atharva
11e6a77fba
Change asterisk to 'x' in FQDN of SelfSignedCertificate (#11245)
Motivation:

`SelfSignedCertificate` creates a certificate and private key files and store them in a temporary directory. However, if the certificate uses a wildcard hostname that uses asterisk *, e.g. `*.shieldblaze.com`, it'll throw an error because * is not a valid character in the file system.

Modification:
Replace the asterisk with 'x'

Result:
Fixes #11240
2021-05-12 19:32:31 +02:00
Norman Maurer
d0b7f76b9b
Use tasks by default when using openssl based SSL provider (#11242)
Motivation:

We introduced the ability to offload certain operations to an executor that may take some time to complete. At the moment this is not enabled by default when using the openssl based SSL provider. Let's enable it by default as we have this support for some while now and didnt see any issues yet. This will also make things less confusing and more consistent with the JDK based provider.

Modifications:

Use true as default value for io.netty.handler.ssl.openssl.useTasks.

Result:

Offloading works with openssl based SSL provider as well by default
2021-05-12 15:01:02 +02:00
Norman Maurer
00fd4ea264
Disable TLSv1 and TLSv1.1 by default (#11237)
Motivation:

TLSv1 and TLSv1.1 is considered insecure. Let's follow the JDK and disable these by default

Modifications:

- Disable TLSv1 and TLSv1.1 by default when using OpenSSL.
- Add unit tests

Result:

Use only strong TLS versions by default when using OpenSSL
2021-05-11 10:43:40 +02:00
Norman Maurer
57a70c5d83
Update conscrypt and add workaround for test failure (#11238)
Motivation:

Conscrypt not correctly filters out non support TLS versions which may lead to test failures.

Related to https://github.com/google/conscrypt/issues/1013

Modifications:

- Bump up to latest patch release
- Add workaround

Result:

No more test failures caused by conscrypt
2021-05-11 10:40:54 +02:00
Idel Pivnitskiy
3ecb32357c
Use PlatformDependent#normalizedOs() instead of reading os.name prop (#11239)
Motivation:

`PlatformDependent#normalizedOs()` already caches normalized variant of
the value of `os.name` system property. Instead of inconsistently
normalizing it in every case, use the utility method.

Modifications:

- `PlatformDependent`: `isWindows0()` and `isOsx0()` use `NORMALIZED_OS`;
- `PlatformDependent#normalizeOs(String)` define `darwin` as `osx`;
- `OpenSsl#loadTcNative()` does not require `equalsIgnoreCase` bcz `os`
is already normalized;
- Epoll and KQueue: `Native#loadNativeLibrary()` use `normalizedOs()`;
- Use consistent `Locale.US` for lower case conversion of `os.name`;
- `MacOSDnsServerAddressStreamProvider#loadNativeLibrary()` uses
`PlatformDependent.isOsx()`;

Result:

Consistent approach for `os.name` parsing.
2021-05-11 08:52:29 +02:00
skyguard1
bb7b05be01
Add explicit null checks in OpenSslX509KeyManagerFactory (#11230)
Motivation:

We should add explicit null checks so its easier for people to understand why it throws.

Modification:

Add explicit checkNotNull(...)

Result:

Easier to understand for users why it fails.

Signed-off-by: xingrufei <xingrufei@sogou-inc.com>

Co-authored-by: xingrufei <xingrufei@sogou-inc.com>
2021-05-07 11:22:27 +02:00
Chris Vest
03c2644c19
Bump initial timeouts in SSLEngineTest (#11221)
Motivation:
We've seen (very rare) flaky test failures due to timeouts.
They are too rare to analyse properly, but a theory is that on overloaded, small cloud CI instances, it can sometimes take a surprising amount of time to start a thread.
It could be that the event loop thread is getting an unlucky schedule, and takes seconds to start, causing the timeouts to elapse.

Modification:
Increase the initial timeouts in the SSLEngineTest, that could end up waiting for the event loop thread to start.
Also fix a few simple warnings from Intellij.

Result:
Hopefully we will not see these tests be flaky again.
2021-05-05 15:14:23 +02:00
Scott Mitchell
d0fea48604
ReferenceCountedOpenSslEngine unwrap handshake complete status fix (#11210)
Motivation:
ReferenceCountedOpenSslEngine may unwrap data and complete the handshake
in a single unwrap() call. However it may return HanshakeStatus of
HandshakeStatus of NEED_UNWRAP instead of FINISHED. This may result in
the SslHandler sending the unwrapped data up the pipeline before
notifying that the handshake has completed, and result in out-of-order
events.

Modifications:
- if ReferenceCountedOpenSslEngine handshake status is NEED_UNWRAP and
  produced data, or NEED_WRAP and consumed some data, we should call
  handshake() to get the current state.

Result:
ReferenceCountedOpenSslEngine correctly indicates when the handshake has
finished if at the same time data was produced or consumed.
2021-04-29 09:56:01 +02:00
Norman Maurer
216cd38ff2
Re-enable running openssl (shared) tests on CI (#11197)
Motivation:

It turned out we didnt run the openssl tests on the CI when we used the non-static version of netty-tcnative.

Modifications:

- Upgrade netty-tcnative to fix segfault when using shared openssl
- Adjust tests to only run session cache tests when openssl supports it
- Fix some more tests to only depend on KeyManager if the underlying openssl version supports it

Result:

Run all openssl test on the CI even when shared library is used
2021-04-27 13:38:08 +02:00
skyguard1
a98c60283b
Enable Tlsv1.3 when using BouncyCastle ALPN support (#11193)
Motivation:

In the latest version of BouncyCastle, BCJSSE:'TLSv1.3' is now a supported protocol for both client and server. So should consider enabling TLSv1.3 when TLSv1.3 is available

Modification:

This pr is to enable TLSv1.3 when using BouncyCastle ALPN support, please review this pr,thanks

Result:

Enable TLSv1.3 when using BouncyCastle ALPN support

Signed-off-by: xingrufei <xingrufei@sogou-inc.com>

Co-authored-by: xingrufei <xingrufei@sogou-inc.com>
2021-04-26 10:01:16 +02:00
Boris Unckel
2f4beae8ec
Utilize i.n.u.internal.ObjectUtil to assert Preconditions (handler) (#11170) (#11180)
Motivation:

NullChecks resulting in a NullPointerException or IllegalArgumentException, numeric ranges (>0, >=0) checks, not empty strings/arrays checks must never be anonymous but with the parameter or variable name which is checked. They must be specific and should not be done with an "OR-Logic" (if a == null || b == null) throw new NullPointerEx.

Modifications:

* import static relevant checks
* Replace manual checks with ObjectUtil methods

Result:

All checks needed are done with ObjectUtil, some exception texts are improved.

Fixes #11170
2021-04-22 12:51:23 +02:00
Karsten Ohme
56ef371676
BouncyCastle ALPN support (#11157)
Motivation:

Under Android it was not possible to load a specific web page. It might be related to the (missing?) ALPN of the internal TLS implementation. BouncyCastle as a replacement works but this was not supported so far by Netty.
BouncyCastle also has the benefit to be a pure Java solution, all the other providers (OpenSSL, Conscrypt) require native libraries which are not available under Android at least.

Modification:

BouncyCastleAlpnSslEngine.java and support classes have been added. It is relying on the JDK code, hence some support classes had to be opened to prevent code duplication.

Result:

BouncyCastle can be used as TLS provider.

Co-authored-by: Norman Maurer <norman_maurer@apple.com>
2021-04-21 16:18:34 +02:00
Chris Vest
1a5ebfb999
Log fewer stack traces from initialisation code (#11164)
Motivation:
We are increasingly running in environments where Unsafe, setAccessible, etc. are not available.
When debug logging is enabled, we log a complete stack trace every time one of these initialisations fail.
Seeing these stack traces can cause people unnecessary concern.
For instance, people might have alerts that are triggered by a stack trace showing up in logs, regardless of its log level.

Modification:
We continue to print debug log messages on the result of our initialisations, but now we only include the full stack trace is _trace_ logging (or FINEST, or equivalent in whatever logging framework is configured) is enabled.

Result:
We now only log these initialisation stack traces when the lowest possible log level is enabled.

Fixes #7817
2021-04-19 08:53:08 +02:00
Scott Mitchell
0ef2f05373
SslHandler LocalChannel read/unwrap reentry fix (#11156)
Motivation:
SslHandler invokes channel.read() during the handshake process. For some
channel implementations (e.g. LocalChannel) this may result in re-entry
conditions into unwrap. Unwrap currently defers updating the input
buffer indexes until the unwrap method returns to avoid intermediate
updates if not necessary, but this may result in unwrapping the same
contents multiple times which leads to handshake failures [1][2].

[1] ssl3_get_record:decryption failed or bad record mac
[2] ssl3_read_bytes:sslv3 alert bad record mac

Modifications:
- SslHandler#unwrap updates buffer indexes on each iteration so that if
  reentry scenario happens the correct indexes will be visible.

Result:
Fixes https://github.com/netty/netty/issues/11146
2021-04-16 08:21:40 -07:00
Scott Mitchell
16b40d8a37
SslHandler consolidate state to save memory (#11160)
Motivation:
SslHandler has many independent boolean member variables. They can be
collapsed into a single variable to save memory.

Modifications:
- SslHandler boolean state consolidated into a single short variable.

Result:
Savings of 8 bytes per SslHandler (which is per connection) observed on
OpenJDK.
2021-04-15 08:18:54 -07:00
Idel Pivnitskiy
c93419ba57
Fire SslHandshakeCompletionEvent after the last decoded data chunk (#11148)
Motivation:

`SslHandler#unwrap` may produce `SslHandshakeCompletionEvent` if it
receives `close_notify` alert. This alert indicates that the engine is
closed and no more data are expected in the pipeline. However, it fires
the event before the last data chunk. As the result, further handlers
may loose data if they handle `SslHandshakeCompletionEvent`.
This issue was not visible before #11133 because we did not write
`close_notify` alert reliably.

Modifications:

- Add tests to reproduce described behavior;
- Move `notifyClosePromise` after fire of the last `decodeOut`;

Result:

`SslHandshakeCompletionEvent` correctly indicates that the engine is
closed and no more data are expected on the pipeline.
2021-04-15 14:06:42 +02:00
Chris Vest
df53de5b68
Less blocking in ChunkedStream (#11150)
Motivation:
We should avoid blocking in the event loop as much as possible.
The InputStream.read() is a blocking method, and we don't need to call it if available() returns a positive number.

Modification:
Bypass calling InputStream.read() if available() returns a positive number.

Result:
Fewer blocking calls in the event loop, in general, when ChunkedStream is used.
2021-04-12 13:40:14 +02:00
Netty Project Bot
6724786dcc [maven-release-plugin] prepare for next development iteration 2021-04-01 10:50:23 +00:00
Netty Project Bot
b78d8f2abd [maven-release-plugin] prepare release netty-4.1.63.Final 2021-04-01 10:50:18 +00:00
Scott Mitchell
6b48e690fd
SslHandler wrap reentry bug fix (#11133)
Motivation:
SslHandler's wrap method notifies the handshakeFuture and sends a
SslHandshakeCompletionEvent user event down the pipeline before writing
the plaintext that has just been wrapped. It is possible the application
may write as a result of these events and re-enter into wrap to write
more data. This will result in out of sequence data and result in alerts
such as SSLV3_ALERT_BAD_RECORD_MAC.

Modifications:
- SslHandler wrap should write any pending data before notifying
  promises, generating user events, or anything else that may create a
  re-entry scenario.

Result:
SslHandler will wrap/write data in the same order.
2021-04-01 10:00:18 +02:00
Netty Project Bot
e4dd6ee532 [maven-release-plugin] prepare for next development iteration 2021-03-30 17:22:25 +00:00
Netty Project Bot
573d1f4bf5 [maven-release-plugin] prepare release netty-4.1.62.Final 2021-03-30 17:22:18 +00:00
Netty Project Bot
7192ab0f0a [maven-release-plugin] prepare for next development iteration 2021-03-30 07:42:06 +00:00
Netty Project Bot
ccb3ff388f [maven-release-plugin] prepare release netty-4.1.61.Final 2021-03-30 07:42:01 +00:00
Norman Maurer
36cacc201e
Don't bind to a specify port during SSLEngine tests as the port may also be used (#11102)
Motivation:

We should always bind to port 0 to ensure we not try to bind to a port that is already in use during our tests.
As we missed to do this in one test we did see test failures sometimes.

This showed up here:
https://pipelines.actions.githubusercontent.com/obCqqsCMwwGme5y2aRyYOiZvWeJK4O0EY5AYRUDMSELILdqEjV/_apis/pipelines/1/runs/1963/signedlogcontent/18?urlExpires=2021-03-19T12%3A41%3A21.4370902Z&urlSigningMethod=HMACV1&urlSignature=zL6O0msEkghT%2B0hOAL1lqLK66SR0Mp99QIjiau1yPe4%3D

Modifications:

- Use new InetSocketAddress(0)

Result:

Fixes possible test failures.
2021-03-19 21:09:46 +01:00
Scott Mitchell
0b0c234bed
SslHandler flushing with TCP Fast Open fix (#11077)
Motivation:
SslHandler owns the responsibility to flush non-application data
(e.g. handshake, renegotiation, etc.) to the socket. However when
TCP Fast Open is supported but the client_hello cannot be written
in the SYN the client_hello may not always be flushed. SslHandler
may not wrap/flush previously written/flushed data in the event
it was not able to be wrapped due to NEED_UNWRAP state being
encountered in wrap (e.g. peer initiated renegotiation).

Modifications:
- SslHandler to flush in channelActive() if TFO is enabled and
  the client_hello cannot be written in the SYN.
- SslHandler to wrap application data after non-application data
  wrap and handshake status is FINISHED.
- SocketSslEchoTest only flushes when writes are done, and waits
  for the handshake to complete before writing.

Result:
SslHandler flushes handshake data for TFO, and previously flushed
application data after peer initiated renegotiation finishes.
2021-03-14 14:18:27 +01:00
Netty Project Bot
e83132fcf2 [maven-release-plugin] prepare for next development iteration 2021-03-09 08:18:31 +00:00
Netty Project Bot
eef26e8bb5 [maven-release-plugin] prepare release netty-4.1.60.Final 2021-03-09 08:18:26 +00:00
Netty Project Bot
b8d843757b [maven-release-plugin] rollback the release of netty-4.1.60.Final 2021-03-09 07:39:13 +00:00
Netty Project Bot
4f099dd11f [maven-release-plugin] prepare for next development iteration 2021-03-09 07:21:55 +00:00
Netty Project Bot
44e392e851 [maven-release-plugin] prepare release netty-4.1.60.Final 2021-03-09 07:21:51 +00:00
Norman Maurer
7d4aaa268b
Support session cache for client and server when using native SSLEngine implementation (#10994)
Motivation:

At the moment we don't support session caching on the client side at all when using the native SSL implementation. We should at least allow to enable it.

Modification:

Allow to enable session cache for client side but disable ti by default due a JDK bug atm.

Result:

Be able to cache sessions on the client side when using native SSL implementation .
2021-03-07 19:15:39 +01:00
Norman Maurer
5d41611b79
Fix NPE that can happen in the WriteTimeoutHandler when multiple executors are used (#11056)
Motivation:

In WriteTimeoutHandler we did make the assumption that the executor which is used to schedule the timeout is the same that is backing the write promise. This may not be true which will cause concurrency issues

Modifications:

Ensure we are on the right thread when try to modify the doubly-linked-list and if not schedule it on the right thread.

Result:

Fixes https://github.com/netty/netty/issues/11053
2021-03-04 15:18:20 +01:00
Norman Maurer
18c66a9d70
Ensure removal from queue happens before writeAndFlush(...) is called (#11049)
Motivation:

We need to ensure that we call queue.remove() before we cal writeAndFlush() as this operation may cause an event that also touches the queue and remove from it. If we miss to do so we may see NoSuchElementExceptions.

Modifications:

- Call queue.remove() before calling writeAndFlush(...)
- Add unit test

Result:

Fixes https://github.com/netty/netty/issues/11046
2021-03-02 21:56:59 +01:00