Commit Graph

965 Commits

Author SHA1 Message Date
Norman Maurer
a137291ad1
Add OpenSslX509KeyManagerFactory which makes it even easier for peopl… (#8084)
* Add OpenSslX509KeyManagerFactory which makes it even easier for people to get the maximum performance when using OpenSSL / LibreSSL / BoringSSL  with netty.

Motivation:

To make it even easier for people to get the maximum performance when using native SSL we should provide our own KeyManagerFactory implementation that people can just use to configure their key material.

Modifications:

- Add OpenSslX509KeyManagerFactory which users can use for maximum performance with native SSL
- Refactor some internal code to re-use logic and not duplicate it.

Result:

Easier to get the max performance out of native SSL implementation.
2018-07-10 00:42:37 -04:00
Norman Maurer
83710cb2e1
Replace toArray(new T[size]) with toArray(new T[0]) to eliminate zero-out and allow the VM to optimize. (#8075)
Motivation:

Using toArray(new T[0]) is usually the faster aproach these days. We should use it.

See also https://shipilev.net/blog/2016/arrays-wisdom-ancients/#_conclusion.

Modifications:

Replace toArray(new T[size]) with toArray(new T[0]).

Result:

Faster code.
2018-06-29 07:56:04 +02:00
Norman Maurer
ecc238bea5
Only try to call SSL.setHostnameValidation(...) if needed. (#8074)
Motivation:

As the used OpenSSL version may not support hostname validation we should only really call SSL.setHostNameValidation(...) if we detect that its needed.

Modifications:

Only call SSL.setHostNameValidation if it was disabled before and now it needs to be enabled or if it was enabled before and it should be disabled now.

Result:

Less risk of an exception when using an OpenSSL version that does not support hostname validation.
2018-06-28 11:07:13 +02:00
Norman Maurer
2818730092
OpenSSL (and so netty-tcnative) should allow to use custom engine. (#8050)
Motivation:

OpenSSL allows to use a custom engine for its cryptographic operations. We should allow the user to make use of it if needed.

See also: https://www.openssl.org/docs/man1.0.2/crypto/engine.html.

Modifications:

Add new system property which can be used to specify the engine to use (null is the default and will use the build in default impl).

Result:

More flexible way of using OpenSSL.
2018-06-28 08:13:52 +02:00
Norman Maurer
0337ecdcc8 Allow to cache keymaterial when using OpenSSL
Motiviation:

During profiling it showed that a lot of time during the handshake is spent by parsing the key / chain over and over again. We should cache these parsed structures if possible to reduce the overhead during handshake.

Modification:

- Use new APIs provided by https://github.com/netty/netty-tcnative/pull/360.
- Introduce OpensslStaticX509KeyManagerFactory which allows to wrap another KeyManagerFactory and caches the key material provided by it.

Result:

In benchmarks handshake times have improved by 30 %.
2018-06-24 07:36:27 +02:00
Norman Maurer
3fb1b992ef
Remove some cipher protocol combos for tests due removal in more recent versions of OpenSSL (#8033)
Motivation:

Some of the cipher protocol combos that were used are no longer included in more recent OpenSSL releases.

Modifications:

Remove some combos that were used for testing.

Result:

Tests also pass in more recent OpenSSL versions (1.1.0+).
2018-06-19 08:12:02 +02:00
Roger
3e3e5155b9 Check if Log level is enabled before creating log statements (#8022)
Motivation

There is a cost to concatenating strings and calling methods that will be wasted if the Logger's level is not enabled.

Modifications

Check if Log level is enabled before producing log statement. These are just a few cases found by RegEx'ing in the code.

Result

Tiny bit more efficient code.
2018-06-13 23:21:53 -07:00
Norman Maurer
a4393831f0
Fix race in SslHandlerTest that could lead to NPE. (#7989)
Motivation:

SslHandlerTest tried to get access to the SslHandler in the pipeline via pipeline.get(...) which may return null if the channel was already closed and so the pipeline was teared down.

This showed up in a test run as:

```
-------------------------------------------------------------------------------
Test set: io.netty.handler.ssl.SslHandlerTest
-------------------------------------------------------------------------------
Tests run: 17, Failures: 0, Errors: 1, Skipped: 1, Time elapsed: 0.802 sec <<< FAILURE! - in io.netty.handler.ssl.SslHandlerTest
testCloseOnHandshakeFailure(io.netty.handler.ssl.SslHandlerTest)  Time elapsed: 0.188 sec  <<< ERROR!
java.lang.NullPointerException
        at io.netty.handler.ssl.SslHandlerTest.testCloseOnHandshakeFailure(SslHandlerTest.java:640)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
        at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
        at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
        at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
        at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
        at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.lang.Thread.run(Thread.java:844)
```

Modifications:

Use an AtomicReference to propagate the SslHandler instance to the outer scope.

Result:

No more NPE.
2018-05-30 22:07:42 +02:00
Norman Maurer
987c443888
Use ByteBufAllocator used by the ReferenceCountedOpenSslEngine when build key-material. (#7952)
Motivation:

When we build the key-material we should use the ByteBufAllocator used by the ReferenceCountedOpenSslEngine when possible.

Modifications:

Whenever we have access to the ReferenceCountedOpenSslEngine we use its allocator.

Result:

Use correct allocator
2018-05-18 19:36:57 +02:00
Norman Maurer
7727649b2c
Add tests for the Conscrypt based SSLEngine. (#7950)
Motivation:

We currently have only interopt tests for Conscrypt, we should also have non-interopt tests.

Modifications:

Add ConscryptSslEngineTest

Result:

More tests
2018-05-18 19:36:40 +02:00
Norman Maurer
546ddd2c28
Correctly calculate and respect if we can correctly fullfil wrap for alerts. (#7945)
Motivation:

We previously did not correctly take into account when we could not wrap (and so produce) the full SSL record with an alert when the SSLEngine was closed.

There are two problems here:

- If we call wrap(...) with an empty dst buffer after closeOutbound() was called we will not notify the user if we could not store the whole SSLRecord into the dst buffer and so we may produce incomplete SSLRecords

Modifications:

Add unit test which failed before.

Result:

Correctly handle the case when the dst buffer is not big enough and and alert needs to be produced.
2018-05-16 20:08:05 +02:00
Norman Maurer
47985c11c1 Add missing parameter when delegate to SSLEngine.
Motivation:

https://github.com/netty/netty/pull/7943 had a bug which caused to not have the argument passed to the delegating method.

Modifications:

Add argument to release call.

Result:

Correctly delegate method.
2018-05-16 20:01:58 +02:00
Norman Maurer
0bce0450c0
Adjust tests to also pass when using BoringSSL (#7946)
Motivation:

Some of the tests failed when using BoringSSL as some protocol / cipher combinations are not supported and it uses a different alert when the cert is not valid yet.

Modification:

- Remove protocol / cipher combos that are not supported by BoringSSL
- Test for different alert when using BoringSSL

Result:

Not test failures when using BoringSSL.
2018-05-16 18:58:27 +02:00
Norman Maurer
932d77b83e
Verify error stack is empty after each operation when using ReferenceCountedOpenSslEngine. (#7943)
Motivation:

https://github.com/netty/netty/pull/7941 proved that its easy to not correctly clear the error stack sometimes. We should do carefully test this.

Modifications:

Add a new SSLEngine wrapper that is used during tests, which verifies that the error stack is empty after each method call.

Result:

Better testing.
2018-05-16 13:50:37 +02:00
Norman Maurer
69c644bb98
Correctly detect if protocol is enabled when using netty-tcnative (#7940)
Motivation:

We sometimes did not correctly detect when a protocol is not enabled when using netty-tcnative as we did not take into account when the option flag is 0 (as for example in BoringSSL for SSLv2).

Modifications:

- Correctly take an option flag with 0 into account.
- Add unit tests.

Result:

Fixes https://github.com/netty/netty/issues/7935.
2018-05-16 07:23:55 +02:00
Norman Maurer
ed54ab034d
Correctly clear the error stack in all cases when using ReferenceCountedOpenSslEngine. (#7941)
Motivation:

We missed to correctly clear the error stack in one case when using the ReferenceCountedOpenSslEngine. Because of this it was possible to pick up an error on an unrelated operation.

Modifications:

- Correctly clear the stack
- Add verification of empty error stack to tests.

Result:

Not possible to observe unrelated errors.
2018-05-15 19:44:02 +02:00
Norman Maurer
64bb279f47 [maven-release-plugin] prepare for next development iteration 2018-05-14 11:11:45 +00:00
Norman Maurer
c67a3b0507 [maven-release-plugin] prepare release netty-4.1.25.Final 2018-05-14 11:11:24 +00:00
Norman Maurer
54c3de0f8a
Add test to validate SSLEngine does not zero out src buffer on wrap. (#7914)
Motivation:

We had a bug-report that claimed the src buffer used by OpenSslEngine will be zero out.

Modifications:

Add testcase to ensure correct behaviour

Result:

Testcase for https://github.com/netty/netty/issues/7753
2018-05-07 20:10:06 +02:00
Norman Maurer
358249e5c9
Allow to disable native transport and native ssl support via system property. (#7903)
Motivation:

Sometimes it's useful to disable native transports / native ssl to debug a problem. We should allow to do so with a system property so people not need to adjust code for this.

Modifications:

Add system properties which allow to disable native transport and native ssl.

Result:

Easier to disable native code usage without code changes.
2018-05-04 14:44:44 +02:00
Norman Maurer
b75f44db9a [maven-release-plugin] prepare for next development iteration 2018-04-19 11:56:07 +00:00
Norman Maurer
04fac00c8c [maven-release-plugin] prepare release netty-4.1.24.Final 2018-04-19 11:55:47 +00:00
Nikolay Fedorovskikh
401b196623 Extract common parts from if statements (#7831)
Motivation:
Some `if` statements contains common parts that can be extracted.

Modifications:
Extract common parts from `if` statements.

Result:
Less code and bytecode. The code is simpler and more clear.
2018-04-11 14:36:56 +02:00
root
0a61f055f5 [maven-release-plugin] prepare for next development iteration 2018-04-04 10:44:46 +00:00
root
8c549bad38 [maven-release-plugin] prepare release netty-4.1.23.Final 2018-04-04 10:44:15 +00:00
Nicolae Mihalache
8d78893a76 Avoid writing two times the same message in case channelWritabilityChanged event is called during a write.
Motivation:

ChunkedWriteHandler.doFlush is called twice from the same write if the channelWritabilityChanged event is invoked during the write. The buffer is already written so no extra data is sent on the socket but it causes the "promise already done" exception to be thrown.
This error happens only when the message is not ChunkedInput.

Modification:
Clear out the currentWrite reference before the ctx.write call, such that next time when the method is invoked the same object is not used twice.

Result:

Fixes #7819
2018-03-30 19:32:08 +02:00
Norman Maurer
2e92a2f5cd Ensure we not schedule multiple timeouts for close notify
Motivation:

We should only schedule one timeout to wait for the close notify to be done.

Modifications:

Keep track of if we already scheduled a timeout for close notify and if so not schedule another one.

Result:

No duplicated timeouts.
2018-03-27 09:43:46 +02:00
Norman Maurer
40af10b782 Skip NPN tests when libressl 2.6.1+ is used.
Motivation:

LibreSSL removed support for NPN in its 2.6.1+ releases.

Modifications:

Skip NPN tests in libressl 2.6.1+

Result:

Be able to run netty tests against libressl 2.6.1+ as well.
2018-03-22 08:30:49 +01:00
Norman Maurer
352e36a179 Remove code duplication in ChunkedWriteHandler
Motivation:

We had some code duplication in ChunkedWriteHandler.

Modifications:

Factor out duplicated code into private methods and reuse it.

Result:

Less code duplication.
2018-03-19 09:14:07 +01:00
Norman Maurer
d1055e0665 Add testcase for c11b23bbc1
Motivation:

c11b23bbc1 added a fix for closing the SSLEngine otbound but no test was provided.

Modifications:

Add testcase.

Result:

More tests.
2018-03-06 14:33:54 +09:00
Carl Mastrangelo
c11b23bbc1 Close SSLEngine when connection fails.
Motivation:
When using the JdkSslEngine, the ALPN class is used keep a reference
to the engine.   In the event that the TCP connection fails, the
SSLEngine is not removed from the map, creating a memory leak.

Modification:
Always close the SSLEngine regardless of if the channel became
active.  Also, record the SSLEngine was closed in all places.

Result:
Fixes: https://github.com/grpc/grpc-java/issues/3080
2018-03-04 06:55:51 -08:00
Norman Maurer
bf8cac4939 Workaround SSLEngine.unwrap(...) bug in Android 5.0
Motivation:

Android 5.0 sometimes not correctly update the bytesConsumed of the SSLEngineResult when consuming data from the input ByteBuffer. This will lead to handshake failures.

Modifications:

Add a workaround for Android 5.0

Result:

Be able to use netty on Android 5.0 by fixing https://github.com/netty/netty/issues/7758 .
2018-03-03 15:01:39 -08:00
Norman Maurer
69582c0b6c [maven-release-plugin] prepare for next development iteration 2018-02-21 12:52:33 +00:00
Norman Maurer
786f35c6c9 [maven-release-plugin] prepare release netty-4.1.22.Final 2018-02-21 12:52:19 +00:00
Norman Maurer
268b901844 SSL connection not closed properly after handshake failure
Motivation:

When SSL handshake fails, the connection should be closed. This is not true anymore after 978a46c.

Modifications:

- Ensure we always flush and close the channel on handshake failure.
- Add testcase.

Result:

Fixes [#7724].
2018-02-16 08:25:47 -08:00
Carl Mastrangelo
e00f24961a Get memory address from Unsafe for OpenSSL
Motivation:
Profiling tcnative SSL code showed a non trivial percentage (1%)
of time spent in JNI code for InstaceOf.   This turned out to be
from `Buffer.address` which makes a JNI call, which safely checks
on each call that The ByteBuffer is direct.

Modification:
Prefer using the address field of the pojo rather than looking it
up with JNI.  This is the same approach taken by the `OpenSsl`
class.

Result:
Less JNI overhead
2018-02-16 07:57:43 +01:00
Eric Anderson
8b273983f0 Load Conscrypt method via reflection only once
Motivation:

The code did reflection every method call which made the code slower and
harder to read with additional cases to consider.

Modifications:

Instead of loading the method and then throwing it away, save the Method
reference instead of the Class reference. Then also use more precise
exception handling for the method invocation.

Result:

Simpler, speedier code.
2018-02-09 21:13:42 +01:00
Stephane Landelle
40ae4fefc7 Introduce an alternative IdentityCipherSuiteFilter that defaults to supportedCiphers, close #7655
Motivation:

Sometimes, it would be convenient to be able to easily enable all
supported cipher suites, regardless of security.

Currently, the only way it to retrieve all supported ciphers and pass
them explicitly.

Modification:

Introduce a new IdentityCipherSuiteFilter singleton that defaults to
supportedCiphers instead of defaultCiphers when ciphers are null.

Result:

Convenient way to enabled all supported cipher suites.
2018-02-07 13:37:25 +01:00
Norman Maurer
e71fa1e7b6 [maven-release-plugin] prepare for next development iteration 2018-02-05 12:02:35 +00:00
Norman Maurer
41ebb5fcca [maven-release-plugin] prepare release netty-4.1.21.Final 2018-02-05 12:02:19 +00:00
Eric Anderson
2923f33530 Adapt to API changes in Conscrypt 1.0.0.RC11
Motivation:

In google/conscrypt#313 the Conscrypt.Engines class was removed in favor
of methods directly on Conscrypt and overloading. The Conscrypt-using
code in Netty used reflection to access the old API, that doesn't exist
anymore. And thus recent versions of Conscrypt fail to enable things
like ALPN with Netty.

Modifications:

Instead of calling Conscrypt.Engines.isConscrypt, call
Conscrypt.isConscrypt.

Result:

Conscrypt detected properly at runtime.
2018-02-02 07:25:59 +01:00
Scott Mitchell
3f3d309a28
JdkSslContext supported cipher suites incorrect
Motivation:
JdkSslContext builds the list of supported cipher suites, but assumes that ciphers prefixed with SSL_ and TLS_ will be interchangeable. However this is not the case and only applies to a small subset of ciphers. This results in the JdkSslContext attempting to use unsupported ciphers.

Modifications:
- When building the list of ciphers in JdkSslContext we should first check if the engine supports the TLS_ prefix cipher.

Result:
Fixes https://github.com/netty/netty/issues/7673
2018-02-01 09:34:44 -08:00
Scott Mitchell
978a46cc0a SslHandler unwrap out of order promise/event notificaiton
Motivation:
SslHandler#decode methods catch any exceptions and attempt to wrap
before shutting down the engine. The intention is to write any alerts
which the engine may have pending. However the wrap process may also
attempt to write user data, and may also complete the associated
promises. If this is the case, and a promise listener closes the channel
then SslHandler may later propagate a SslHandshakeCompletionEvent user
event through the pipeline. Since the channel has already been closed
the user may no longer be paying attention to user events.

Modifications:
- Sslhandler#decode should first fail the associated handshake promise
and propagate the SslHandshakeCompletionEvent before attempting to wrap

Result:
Fixes https://github.com/netty/netty/issues/7639
2018-02-01 09:25:40 -08:00
Norman Maurer
ea58dc7ac7 [maven-release-plugin] prepare for next development iteration 2018-01-21 12:53:51 +00:00
Norman Maurer
96c7132dee [maven-release-plugin] prepare release netty-4.1.20.Final 2018-01-21 12:53:34 +00:00
Dmitriy Dumanskiy
e6c9ac968d Cleanup: replaced deprecated ctx.attr() and ctx.hasAttr() methods usage with ch.attr() and ch.hasAttr().
Motivation:

Will allow easy removal of deprecated methods in future.

Modification:

Replaced ctx.attr(), ctx.hasAttr() with ctx.channel().attr(), ctx.channel().hasAttr().

Result:

No deprecated ctx.attr(), ctx.hasAttr() methods usage.
2018-01-18 15:00:41 +00:00
Scott Mitchell
062d926912 Remove remote initiated renegotiation support
Motivation:
We recently removed support for renegotiation, but there are still some hooks to attempt to allow remote initiated renegotiation to succeed. The remote initated renegotiation can be even more problematic from a security stand point and should also be removed.

Modifications:
- Remove state related to remote iniated renegotiation from OpenSslEngine

Result:
More renegotiation code removed from the OpenSslEngine code path.
2018-01-15 10:16:08 +01:00
Norman Maurer
ab9f0a0fda Remove direct usage of JKS and SunX509
Motivation:

When using netty on android or with for example a IBM JVM it may not be able to build a SslContext as we hardcoded the use of JKS and SunX509 (which both may not be present).

Modifications:

- Use the default algorithm / type which can be override via a System property
- Remove System property check as its redundant with KeyManagerFactory.getDefaultAlgorithm()

Result:

More portable code. Fixes [#7546].
2018-01-03 18:32:18 -08:00
Chris West
e24e06bfcb OpenSslEngine: Remove renegotiation support
Motivation:

SSL.setState() has gone from openssl 1.1. Calling it is, and probably
always has been, incorrect. Doing renogitation in this manner is
potentially insecure. There have been at least two insecure
renegotiation vulnerabilities in users of the OpenSSL library.

Renegotiation is not necessary for correct operation of the TLS protocol.

BoringSSL has already eliminated this functionality, and the tests
(now deleted) were not running on BoringSSL.

Modifications:

If the connection setup has completed, always return that
negotiation is not supported. Previously this was done only if we were
the client.

Remove the tests for this functionality.

Fixes #6320.
2018-01-02 13:00:11 -05:00
Norman Maurer
264a5daa41 [maven-release-plugin] prepare for next development iteration 2017-12-15 13:10:54 +00:00