Commit Graph

1006 Commits

Author SHA1 Message Date
Norman Maurer
ce39773e04
Add support for boringssl and TLSv1.3 (#8412)
Motivation:

0ddc62cec0 added support for TLSv1.3 when using openssl 1.1.1. Now that BoringSSL chromium-stable branch supports it as well we can also support it with netty-tcnative-boringssl-static.
During this some unit tests failed with BoringSSL which was caused by not correctly handling flush() while the handshake is still in progress.

Modification:

- Upgrade netty-tcnative version which also supports TLSv1.3 when using BoringSSL
- Correctly handle flush() when done while the handshake is still in progress in all cases.

Result:

Easier for people to enable TLSv1.3 when using native SSL impl.
Ensure flush() while handshake is in progress will always be honored.
2018-10-26 15:29:49 -07:00
Norman Maurer
9e762e8816
Correctly detect if KeyManagerFactory is supported by OpenSSL even when sun.security.x509.* can not be accessed and bouncycastle is not on the classpath. (#8415)
Motivation:

OpenSsl used SelfSignedCertificate in its static init block to detect if KeyManagerFactory is supported. Unfortunally this only works when either sun.security.x509.* can be accessed or bouncycastle is on the classpath.
We should not depend on either of it.

This came up in https://github.com/netty/netty-tcnative/issues/404#issuecomment-431551890.

Modifications:

Just directly use the bytes to generate the X509Certificate and so not depend on sun.security.x509.* / bouncycastle.

Result:

Correctly be able to detect if KeyManagerFactory can be supported in all cases.
2018-10-23 17:08:23 +02:00
Norman Maurer
91201fb338
Remove workaround in tests for TLSv1.3 bug in Java11 as it was fixed in 11.0.1 (#8409)
Motivation:

We had put some workaround in our tests due a bug in the Java11 implementation of TLSv1.3. This was now fixes as part of 11.0.1.

See https://bugs.openjdk.java.net/browse/JDK-8211067.

Modifications:

Remove workaround in SSL tests.

Result:

Run all tests with supported TLS version.
2018-10-19 17:21:04 +02:00
Norman Maurer
201e984cb3
Allow to use TLSv1.3 with netty-tcnative withe java versions prior to 11. (#8394)
Motivation:

At the moment it's only possible to use TLSv1.3 with netty-tcnative if Java 11 is used. It should be possible to do so even with Java 8, 9 and 10.

Modification:

Add a workaround to be able to use TLSv1.3 also when using Java version prior to Java 11 and the default X509ExtendedTrustManager is used.

Result:

Be able to use TLSv1.3 also with past versions of Java.
2018-10-18 13:50:12 +02:00
Norman Maurer
3543e17967
Ensure OpenSslEngine will not try to call SSL_free multiple times even when constructor throws. (#8399)
Motivation:

When the constructor of OpenSslEngine threw we could end up to self call SSL_free by ourself and then have the finalizer do the same which may lead to double free-ing and so SIGSEV.

Modifications:

Just call shutdown() when the constructor throws and so ensure SSL_free is guarded correctly in the finalizer.

Result:

No more SIGSEV possible.
2018-10-18 07:38:03 +02:00
Norman Maurer
0ddc62cec0
Add support for TLSv1.3 (#8293)
Motivation:

TLSv1.3 support is included in java11 and is also supported by OpenSSL 1.1.1, so we should support when possible.

Modifications:
- Add support for TLSv1.3 using either the JDK implementation or the native implementation provided by netty-tcnative when compiled against openssl 1.1.1
- Adjust unit tests for semantics provided by TLSv1.3
- Correctly handle custom Provider implementations that not support TLSv1.3

Result:

Be able to use TLSv1.3 with netty.
2018-10-17 08:35:35 +02:00
Stephane Landelle
9eebe7ed74 Add full JdkSslContext public constructor, close #8384 (#8389)
Motivation:

JdkSslContext provides public constructors to wrap an existing `javax.net.ssl.SSLContext`.

Sadly, some options combinations are not possible with the existing constructors, eg:
*  protocols is not exposed and always forced to null, so default protocols are always enforced
* startTls is not exposed and always forced to false

Modification:

Add full constructor that take protocols and startTls parameters.

Result:

It's possible to create a JdkSslContext from an existing SSLContext and still have control over protocols and startTls
2018-10-17 08:28:39 +02:00
Norman Maurer
aae7cdca96
Prevent NPE when attempting to set client key material with no alias (#8378)
Motivation:

It is possible that a client is unable to locate a certificate alias given the list of issuers and key types. In this case the X509KeyManager
will return a null which when past to the OpenSslKeyMaterialProvider implementation may produce a NPE. If no matching alias could be found we should not
call OpenSslKeyMaterialProvider at all which is also consistent what OpenJDK does.

Modifications:

- Add null check before calling OpenSslKeyMaterialProvider
- Add unit test.

Result:

No more NPE caused by passing null as client alias.
2018-10-12 09:27:46 +02:00
Norman Maurer
59973e93dd
Ensure X509KeyManager methods are called on the correct time when using server-side and support more methods of ExtendedSSLSession. (#8283)
Motivation:

Before when on server-side we just called the X509KeyManager methods when handshake() was called the first time which is not quite correct as we may not have received the full SSL hello / handshake and so could not extra for example the SNI hostname that was requested.
OpenSSL exposes the SSL_CTX_set_cert_cb function which allows to set a callback which is executed at the correct moment, so we should use it. This also allows us to support more methods of ExtendedSSLSession easily.

Modifications:

- Make use of new methods exposed by netty-tcnative since https://github.com/netty/netty-tcnative/pull/388 to ensure we select the key material at the correct time.
- Implement more methods of ExtendedOpenSslSession
- Add unit tests to ensure we are able to retrieve various things on server-side in the X509KeyManager and so verify it is called at the correct time.
- Simplify code by using new netty-tcnative methods.

Result:

More correct implementation for server-side usage and more complete implemented of ExtendedSSLSession.
2018-09-28 11:34:38 +02:00
Norman Maurer
73acac13f4
Check if hostname validation is supported before trying to use in test. (#8333)
Motivation:

a208f6dc7c added a testcase which uses hostname validation which may not be supported by OpenSSL depending on the version that is used. We should check first before we try to use it.

Modifications:

Add assumeTrue(...) check to ensure hostname validation is supported before trying to run the test.

Result:

No more test-failures on OpenSSL versions < 1.0.2.
2018-09-28 10:54:05 +02:00
Norman Maurer
a208f6dc7c
Do the same extended checks as the JDK when a X509TrustManager is used with the OpenSSL provider. (#8307)
Motivation:

When a X509TrustManager is used while configure the SslContext the JDK automatically does some extra checks during validation of provided certs by the remote peer. We should do the same when our native implementation is used.

Modification:

- Automatically wrap a X509TrustManager and so do the same validations as the JDK does.
- Add unit tests.

Result:

More consistent behaviour. Fixes https://github.com/netty/netty/issues/6664.
2018-09-28 09:19:58 +02:00
root
2d7cb47edd [maven-release-plugin] prepare for next development iteration 2018-09-27 19:00:45 +00:00
root
3a9ac829d5 [maven-release-plugin] prepare release netty-4.1.30.Final 2018-09-27 18:56:12 +00:00
Norman Maurer
652650e127
Fix leak in SniClientJava8TestUtil (#8326)
Motivation:

4d1458604a did fix some leaks in SniClientTest but missed the ones in SniClientJava8TestUtil.

Modifications:

Correctly release SslContext.

Result:

No more leaks in SNI tests.
2018-09-27 09:32:41 +02:00
Norman Maurer
4d1458604a
Fix leak in SniClientTest. (#8324)
Motivation:

We need to release the ReferenceCountedSslContext to eliminate resource leaks.

Reported in https://garage.netty.io/teamcity/viewLog.html?buildId=33353&buildTypeId=netty_build_oraclejdk8&tab=buildLog#_focus=157264.

Modifications:

Call release on the SslContext instances.

Result:

No more leaks in tests.
2018-09-26 19:59:00 +02:00
Norman Maurer
a80c49828f
Cleanup SSL test. (#8301)
Motivation:

I noticed that we had some errors showing up in a test (which did not fail it tho) because we tried to full-fill the promise multiples times.

Modifications:

Use trySuccess(...) as we may produce multiple exceptions.

Result:

Less errors during test-run.
2018-09-21 08:04:24 -07:00
Norman Maurer
01db30a163
Correctly implement ExtendedSSLSession.getStatusResponses() for ReferenceCountedOpenSslEngine (#8297)
Motivation:

Java9 added getStatusResponses() to ExtendedSSLSession which we should correctly support when possible.

Modifications:

Implement the method correctly.

Result:

More complete and correct implementation.
2018-09-19 17:13:44 -07:00
Norman Maurer
34d52fcbfe
Implemented ExtendedOpenSslSession.getStatusResponses() so it not throws an UnsupportedOperationException. (#8290)
Motivation:

6ed7c6c75d added support for ExtendedOpenSslSession but we did not override getStatusResponses(). This lead to test failures on java9.

Modifications:

Implement ExtendedOpenSslSession.getStatusResponses() so it just returns an empty list.

Result:

Test pass again on Java9.
2018-09-14 20:33:09 +02:00
Norman Maurer
2b1514ec5a
Only use KeyManagerFactory in SniClientTest when supported by OpenSSL version. (#8289)
Motivation:

6ed7c6c75d added a test which blindly assumed we can use a KeyManagerFactory all the time. This is only true if have OpenSSL 1.0.2 or later, which may not be the case.

Modifications:

Only use KeyManagerFactory in test if the OpenSSL version does support it.

Result:

More robust tests.
2018-09-14 19:01:55 +02:00
Norman Maurer
6ed7c6c75d
Return an ExtendSSLSession whenever possible to allow more strict checking when using OpenSSL (#8281)
Motivation:

When an ExtendedSSLSession is used its possible to do more strict checking of the keys during handshake. We should do this whenever possible.

Modification:

- Return an ExtendedSSLSession when using client-mode and Java7+
- Add unit test
- Simplify unit tests

Result:

More consistent behaviour.
2018-09-14 14:33:11 +02:00
Norman Maurer
5ff6b57940
PemPrivateKey.toPem(...) should throw IllegalArgumentException when P… (#8253)
* PemPrivateKey.toPem(...) should throw IllegalArgumentException when PrivateKey which does not support encoding is used.

Motivation:

At the moment when a PrivateKey is used that does not support encoding we throw a NPE when trying to convert the key. We should better throw an IllegalArgumentException with the details about what key we tried to encode.

Modifications:

- Check if PrivateKey.getEncoded() returns null and if so throw an IllegalArgumentException
- Add unit test.

Result:

Better handling of non-supported PrivateKey implementations.
2018-09-05 20:33:40 +02:00
Norman Maurer
f4bafd4fe0
Correctly check if cipher is supported for each SslProvider before trying to run test and fix buffer leaks in test. (#8247)
Motivation:

5aaa16b24c introduced a testcase for specific ciphersuites and checked if these are supported by our native implementation before running it. Unfortunally this is not good enough as even on the JDK it may not be supported on various JDK versions (like Java7). Beside this the test leaked buffers.

Modifications:

- Correctly check if ciphersuite is supported on each SslProvider before trying to run test.
- Fix buffer leaks.

Result:

Testsuite pass again on Java7 and others when -Pleak is used.
2018-09-01 08:11:13 +02:00
Norman Maurer
a644563625
Add more debug informations when log SSL errors. (#8241)
Motivation:

ea626ef8c3 added more debug logging but we can even include a bit more.

Modifications:

Always log the error number as well.

Result:

More informations for debugging SSL errors.
2018-08-30 20:44:47 +02:00
Norman Maurer
ea626ef8c3
Log more details when shutdown SSL because of an error. (#8236)
Motivation:

We should log a bit more details about why we shutdown the SSL.

Modifications:

Add the return value of SSL_get_error(...) as well in debug mode.

Result:

More logging to make it easier to understand why an SSL error happened.
2018-08-29 21:52:26 +02:00
Roger
5aaa16b24c An unit test to ensure that cipher suites don't break/disappear between releases. (#8225)
Motivation

Ensure classes of cipher suites continue working between releases. Adding just a DHE check for now as it caused #8165 but this test can be expaned to other suites.

Modifications

Adding an unit test that checks for the presence of a cipher suite.

Result

Prevent #8165 from happening in the future.
2018-08-29 14:14:26 +02:00
root
a580dc7585 [maven-release-plugin] prepare for next development iteration 2018-08-24 06:36:33 +00:00
root
3fc789e83f [maven-release-plugin] prepare release netty-4.1.29.Final 2018-08-24 06:36:06 +00:00
Norman Maurer
df00539fa2
Allow to load PrivateKey via OpenSSL Engine (#8200)
Motivation:

OpenSSL itself has an abstraction which allows you to customize some things. For example it is possible to load the PrivateKey from the engine. We should support this.

Modifications:

Add two new static methods to OpenSslX509KeyManagerFactory which allow to create an OpenSslX509KeyManagerFactory that loads its PrivateKey via the OpenSSL Engine directly.

Result:

More flexible usage of OpenSSL possible
2018-08-18 07:20:44 +02:00
Norman Maurer
bbe2e4d224
We should try to load netty-tcnative before using it in OpenSslCertificateException. (#8202)
Motivation:

In OpenSslCertificateException we should ensure we try to load netty-tcnative before trying to use any class from it as otherwise it may throw an error due missing linking of the native libs.

Modifications:

- Ensure we call OpenSsl.isAvailable() before we try to use netty-tcnative for validation
- Add testcase.

Result:

No more errors causing by not loading native libs before trying to use these.
2018-08-18 06:26:45 +02:00
Norman Maurer
8255f85f24
Rename SslHandler.close(...) to closeOutbound(...) as it is still useful and delegate to the methods. (#8193)
* Rename SslHandler.close(...) to closeOutbound(...) as it is still useful and delegate to the methods.

Motivation:

Sometimes the user may want to send a close_notify without closing the underlying Channel. For this we offered the SslHandler.close(...) methods which were marked as deeprecated. We should offer an way to still do this without the user calling deprecated methods.

See https://stackoverflow.com/questions/51710231/using-nettys-sslhandlerclosechannelhandlercontext-channelpromise/51753742#comment90555949_51753742 .

Modifications:

- Remove deprecation of the SslHandler.close(...) method that exactly allows this and rename these to closeOutbound(...) as this is more clear.
- Add close(...) methods that delegate to these and mark these as deprecated.

Result:

Be able to send close_notify without closing the Channel.
2018-08-15 20:07:56 +02:00
Lai Jiang
dbc9ec1ab2 Set SNI servernames in OpenSSL engine when created in client mode (#8178)
Motivation:

When using the JDK SSL provider in client mode, the SNI host names (called serverNames in SslEngineImpl) is set to the peerHost (if available) that is used to initialize the SSL Engine:

http://hg.openjdk.java.net/jdk8/jdk8/jdk/file/687fd7c7986d/src/share/classes/sun/security/ssl/SSLEngineImpl.java#l377

This allows one to call SslEngine.getSSLParameters() and inspect what is the SNI name to be sent. The same should be done in the OpenSSL provider as well. Currently even though the the SNI name is sent by the OpenSSL provider during handshake when the peerHost is specified, it is missing from the parameters.

Modification:

Set the sniHostNames field when SNI is to be used. Also verifies the peer is actually a hostname before setting it as the SNI name, which is consistent with JDK SSL provider's behavior.

Result:

SslEngine using the OpenSSL provider created in client mode with peerHost will initialize sniHostNames with the peerHost.

Calling SslEngine.getSSLParameters().getServerNames() will return a list that contains that name.
2018-08-09 21:30:02 +02:00
Norman Maurer
44d3753c48
Fix NPE exception when using invalid cipher during building SslContext. (#8171)
Motivation:

We missed to do a null check before trying to destroy the OpenSslSessionContext, which could lead to a NPE.

Modifications:

Add null check and tests.

Result:

Fix https://github.com/netty/netty/issues/8170.
2018-08-02 21:42:21 +02:00
Norman Maurer
fe14bad69c
Adjust SSL related tests to be more correct and so pass in the next EA release of java11. (#8162)
Motivation:

In some of our tests we not correctly init the SSLEngine before trying to perform a handshake which can cause an IllegalStateException. While this not happened in previous java releases it does now on Java11 (which is "ok" as its even mentioned in the api docs). Beside this how we selected the ciphersuite to test renegotation was not 100 % safe.

Modifications:

- Correctly init SSLEngine before using it
- Correctly select ciphersuite before testing for renegotation.

Result:

More correct tests and also pass on next java11 EA release.
2018-08-01 06:37:53 +02:00
root
fcb19cb589 [maven-release-plugin] prepare for next development iteration 2018-07-27 04:59:28 +00:00
root
ff785fbe39 [maven-release-plugin] prepare release netty-4.1.28.Final 2018-07-27 04:59:06 +00:00
Norman Maurer
620dad0c26
Allow to validate sni hostname with underscore (#8150)
Motivation:

We should allow to also validate sni hostname which contains for example underscore when using our native SSL impl. The JDK implementation does this as well.

Modifications:

- Construct the SNIHostName via byte[] and not String.
- Add unit test

Result:

Fixes https://github.com/netty/netty/issues/8144.
2018-07-27 01:56:32 +08:00
root
b4dbdc2036 [maven-release-plugin] prepare for next development iteration 2018-07-11 15:37:40 +00:00
root
1c16519ac8 [maven-release-plugin] prepare release netty-4.1.27.Final 2018-07-11 15:37:21 +00:00
Norman Maurer
df08467d7c
Fix possible NPE introduced by a137291ad1 when using SslProvider.OPENSSL and init via files or OpenSslX509KeyManagerFactory (#8126)
Motivation:

a137291ad1 introduced a way to get the most speed out of OpenSSL by not only caching keymaterial but pre-compute these. The problem was we missed to check for null before doing an instanceof check and then a cast which could lead to a NPE as we tried to cast null to Exception and throw it.

Modifications:

Add null check and unit test.

Result:

No more NPE when keymaterial was not found for requested alias.
2018-07-11 15:19:37 +01:00
root
7bb9e7eafe [maven-release-plugin] prepare for next development iteration 2018-07-10 05:21:24 +00:00
root
8ca5421bd2 [maven-release-plugin] prepare release netty-4.1.26.Final 2018-07-10 05:18:13 +00:00
Norman Maurer
a137291ad1
Add OpenSslX509KeyManagerFactory which makes it even easier for peopl… (#8084)
* Add OpenSslX509KeyManagerFactory which makes it even easier for people to get the maximum performance when using OpenSSL / LibreSSL / BoringSSL  with netty.

Motivation:

To make it even easier for people to get the maximum performance when using native SSL we should provide our own KeyManagerFactory implementation that people can just use to configure their key material.

Modifications:

- Add OpenSslX509KeyManagerFactory which users can use for maximum performance with native SSL
- Refactor some internal code to re-use logic and not duplicate it.

Result:

Easier to get the max performance out of native SSL implementation.
2018-07-10 00:42:37 -04:00
Norman Maurer
83710cb2e1
Replace toArray(new T[size]) with toArray(new T[0]) to eliminate zero-out and allow the VM to optimize. (#8075)
Motivation:

Using toArray(new T[0]) is usually the faster aproach these days. We should use it.

See also https://shipilev.net/blog/2016/arrays-wisdom-ancients/#_conclusion.

Modifications:

Replace toArray(new T[size]) with toArray(new T[0]).

Result:

Faster code.
2018-06-29 07:56:04 +02:00
Norman Maurer
ecc238bea5
Only try to call SSL.setHostnameValidation(...) if needed. (#8074)
Motivation:

As the used OpenSSL version may not support hostname validation we should only really call SSL.setHostNameValidation(...) if we detect that its needed.

Modifications:

Only call SSL.setHostNameValidation if it was disabled before and now it needs to be enabled or if it was enabled before and it should be disabled now.

Result:

Less risk of an exception when using an OpenSSL version that does not support hostname validation.
2018-06-28 11:07:13 +02:00
Norman Maurer
2818730092
OpenSSL (and so netty-tcnative) should allow to use custom engine. (#8050)
Motivation:

OpenSSL allows to use a custom engine for its cryptographic operations. We should allow the user to make use of it if needed.

See also: https://www.openssl.org/docs/man1.0.2/crypto/engine.html.

Modifications:

Add new system property which can be used to specify the engine to use (null is the default and will use the build in default impl).

Result:

More flexible way of using OpenSSL.
2018-06-28 08:13:52 +02:00
Norman Maurer
0337ecdcc8 Allow to cache keymaterial when using OpenSSL
Motiviation:

During profiling it showed that a lot of time during the handshake is spent by parsing the key / chain over and over again. We should cache these parsed structures if possible to reduce the overhead during handshake.

Modification:

- Use new APIs provided by https://github.com/netty/netty-tcnative/pull/360.
- Introduce OpensslStaticX509KeyManagerFactory which allows to wrap another KeyManagerFactory and caches the key material provided by it.

Result:

In benchmarks handshake times have improved by 30 %.
2018-06-24 07:36:27 +02:00
Norman Maurer
3fb1b992ef
Remove some cipher protocol combos for tests due removal in more recent versions of OpenSSL (#8033)
Motivation:

Some of the cipher protocol combos that were used are no longer included in more recent OpenSSL releases.

Modifications:

Remove some combos that were used for testing.

Result:

Tests also pass in more recent OpenSSL versions (1.1.0+).
2018-06-19 08:12:02 +02:00
Roger
3e3e5155b9 Check if Log level is enabled before creating log statements (#8022)
Motivation

There is a cost to concatenating strings and calling methods that will be wasted if the Logger's level is not enabled.

Modifications

Check if Log level is enabled before producing log statement. These are just a few cases found by RegEx'ing in the code.

Result

Tiny bit more efficient code.
2018-06-13 23:21:53 -07:00
Norman Maurer
a4393831f0
Fix race in SslHandlerTest that could lead to NPE. (#7989)
Motivation:

SslHandlerTest tried to get access to the SslHandler in the pipeline via pipeline.get(...) which may return null if the channel was already closed and so the pipeline was teared down.

This showed up in a test run as:

```
-------------------------------------------------------------------------------
Test set: io.netty.handler.ssl.SslHandlerTest
-------------------------------------------------------------------------------
Tests run: 17, Failures: 0, Errors: 1, Skipped: 1, Time elapsed: 0.802 sec <<< FAILURE! - in io.netty.handler.ssl.SslHandlerTest
testCloseOnHandshakeFailure(io.netty.handler.ssl.SslHandlerTest)  Time elapsed: 0.188 sec  <<< ERROR!
java.lang.NullPointerException
        at io.netty.handler.ssl.SslHandlerTest.testCloseOnHandshakeFailure(SslHandlerTest.java:640)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
        at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
        at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
        at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
        at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
        at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.lang.Thread.run(Thread.java:844)
```

Modifications:

Use an AtomicReference to propagate the SslHandler instance to the outer scope.

Result:

No more NPE.
2018-05-30 22:07:42 +02:00
Norman Maurer
987c443888
Use ByteBufAllocator used by the ReferenceCountedOpenSslEngine when build key-material. (#7952)
Motivation:

When we build the key-material we should use the ByteBufAllocator used by the ReferenceCountedOpenSslEngine when possible.

Modifications:

Whenever we have access to the ReferenceCountedOpenSslEngine we use its allocator.

Result:

Use correct allocator
2018-05-18 19:36:57 +02:00