Commit Graph

1137 Commits

Author SHA1 Message Date
Norman Maurer
1fe931b6e2
Make it possible to use a wrapped EventLoop with a Channel (#8677)
Motiviation:

Because of how we implemented the registration / deregistration of an EventLoop it was not possible to wrap an EventLoop implementation and use it with a Channel.

Modification:

- Introduce EventLoop.Unsafe which is responsible for the actual registration.
- Move validation of EventLoop / Channel combo to the EventLoop
- Add unit test that verifies that wrapping works

Result:

Be able to wrap an EventLoop and so add some extra functionality.
2019-01-17 09:17:51 +01:00
Oleksii Kachaiev
f004b72662 Correctly propagate write failures from ChunkedWriteHandler (#8716)
Motivation:

ChunkedWriteHandler should report write operation as failed
in case *any* chunked was not written. Right now this is not
true for the last chunk.

Modifications:

* Check if the appropriate write operation was succesfull when
  reporting the last chunk

* Skip writing chunks if the write operation was already marked
  as "done"

* Test cases to cover write failures when dealing with chunked input

Result:

Fix https://github.com/netty/netty/issues/8700
2019-01-16 11:08:11 +01:00
Norman Maurer
c10ccc5dec
Tighten contract between Channel and EventLoop by require the EventLoop on Channel construction. (#8587)
Motivation:

At the moment it’s possible to have a Channel in Netty that is not registered / assigned to an EventLoop until register(...) is called. This is suboptimal as if the Channel is not registered it is also not possible to do anything useful with a ChannelFuture that belongs to the Channel. We should think about if we should have the EventLoop as a constructor argument of a Channel and have the register / deregister method only have the effect of add a Channel to KQueue/Epoll/... It is also currently possible to deregister a Channel from one EventLoop and register it with another EventLoop. This operation defeats the threading model assumptions that are wide spread in Netty, and requires careful user level coordination to pull off without any concurrency issues. It is not a commonly used feature in practice, may be better handled by other means (e.g. client side load balancing), and therefore we propose removing this feature.

Modifications:

- Change all Channel implementations to require an EventLoop for construction ( + an EventLoopGroup for all ServerChannel implementations)
- Remove all register(...) methods from EventLoopGroup
- Add ChannelOutboundInvoker.register(...) which now basically means we want to register on the EventLoop for IO.
- Change ChannelUnsafe.register(...) to not take an EventLoop as parameter (as the EventLoop is supplied on custruction).
- Change ChannelFactory to take an EventLoop to create new Channels and introduce ServerChannelFactory which takes an EventLoop and one EventLoopGroup to create new ServerChannel instances.
- Add ServerChannel.childEventLoopGroup()
- Ensure all operations on the accepted Channel is done in the EventLoop of the Channel in ServerBootstrap
- Change unit tests for new behaviour

Result:

A Channel always has an EventLoop assigned which will never change during its life-time. This ensures we are always be able to call any operation on the Channel once constructed (unit the EventLoop is shutdown). This also simplifies the logic in DefaultChannelPipeline a lot as we can always call handlerAdded / handlerRemoved directly without the need to wait for register() to happen.

Also note that its still possible to deregister a Channel and register it again. It's just not possible anymore to move from one EventLoop to another (which was not really safe anyway).

Fixes https://github.com/netty/netty/issues/8513.
2019-01-14 20:11:13 +01:00
kashike
c0aa1ea5c7 Fix minor spelling issues in javadocs (#8701)
Motivation:

Javadocs contained some spelling errors, we should fix these.

Modification:

Fix spelling

Result:

Javadoc cleanup.
2019-01-14 07:25:13 +01:00
Jon Chambers
400b3081b0 Publicize default explicitFlushAfterFlushes count. (#8683)
Motivation:

Users who want to construct a `FlushConsolidationHandler` with a default `explicitFlushAfterFlushes` but non-default `consolidateWhenNoReadInProgress` may benefit from having an easy way to get the default "flush after flushes" count.

Modifications:

- Moved default `explicitFlushAfterFlushes` value to a public constant.
- Adjusted Javadoc accordingly.

Result:

Default `explicitFlushAfterFlushes` is accessible to callers.
2018-12-25 22:36:23 +01:00
Norman Maurer
26e1411897 Add test for correctly handling SSLSessionBindingEvent when acting on th… (#8649)
Motivation:

During some other work I noticed we do not have any tests to ensure we correctly use SSLSessionBindingEvent. We should add some testing.

Modifications:

- Added unit test to verify we correctly implement it.
- Ignore the test when using Conscrypt as it not correctly implements it.

Result:

More tests for custom SSL impl.
2018-12-19 12:56:14 +01:00
Norman Maurer
7f20adb5fc Skip tests that use KeyManagerFactory if not supported by OpenSSL version / flavor (#8662)
Motivation:

We missed to skip a few tests that depend on the KeyManagerFactory if the used OpenSSL version / flavor not support it.

Modifications:

Add missing overrides.

Result:

Testsuite also passes for example when using LibreSSL.
2018-12-14 21:34:05 +01:00
Feri73
563793688f Adding support for whitespace in resource path in tests (#8606)
Motivation:

In windows if the project is in a path that contains whitespace,
resources cannot be accessed and tests fail.

Modifications:

Adds ResourcesUtil.java in netty-common. Tests use ResourcesUtil.java to access a resource.

Result:

Being able to build netty in a path containing whitespace
2018-12-12 10:29:19 +01:00
Norman Maurer
68f4c82d5a SSLSession.putValue / getValue / removeValue / getValueNames must be thread-safe. (#8648)
Motivation:

SSLSession.putValue / getValue / removeValue / getValueNames must be thread-safe as it may be called from multiple threads. This is also the case in the OpenJDK implementation.

Modifications:

Guard with synchronized (this) blocks to keep the memory overhead low as we do not expect to have these called frequently.

Result:

SSLSession implementation is thread-safe.
2018-12-12 07:41:31 +01:00
Norman Maurer
d9a6cf341c
Remove support for marking reader and writerIndex in ByteBuf to reduce overhead and complexity. (#8636)
Motivation:

ByteBuf supports “marker indexes”. The intended use case for these is if a speculative operation (e.g. decode) is in process the user can “mark” and interface and refer to it later if the operation isn’t successful (e.g. not enough data). However this is rarely used in practice,
requires extra memory to maintain, and introduces complexity in the state management for derived/pooled buffer initialization, resizing, and other operations which may modify reader/writer indexes.

Modifications:

Remove support for marking and adjust testcases / code.

Result:

Fixes https://github.com/netty/netty/issues/8535.
2018-12-11 14:00:49 +01:00
Paul Verest
9c594e5068 ReadTimeoutHandler - missing ) within JavaDoc example (#8645)
Motivation:

improve docs

Modification:

ReadTimeoutHandler - missing ) within JavaDoc example

No logic/unit tests affected
2018-12-10 20:50:43 +01:00
多巴胺
0288570b06 Fix concurrency problem in UniqueIpFilter (#8635)
Motivation:

If two requests from the same IP are reached at the same time, `connected.contains(remoteIp)` may return false in both threads.

Modifications:

Check if there is already a connection with the same IP using return values.

Result:

Become thread safe.
2018-12-07 13:50:44 +01:00
Feri73
5618229203 Correcting Maven Dependencies (#8622)
Motivation:

Most of the maven modules do not explicitly declare their
dependencies and rely on transitivity, which is not always correct.

Modifications:

For all maven modules, add all of their dependencies to pom.xml

Result:

All of the (essentially non-transitive) depepdencies of the modules are explicitly declared in pom.xml
2018-12-06 09:02:00 +01:00
Norman Maurer
2c78dde749 Update version number to start working on Netty 5 2018-11-20 15:49:57 +01:00
Norman Maurer
38524ec3e2
Fix test that assumed detection of peer supported algs is not supported in BoringSSL. (#8573)
Motivation:

0d2e38d5d6 added supported for detection of peer supported algorithms but we missed to fix the testcase.

Modifications:

Fix test-case.

Result:

No more failing tests with BoringSSL.
2018-11-19 12:13:05 +01:00
JStroom
ce02d5a184 Update SslHandler.java (#8564)
Swallow SSL Exception "closing inbound before receiving peer's close_notify" when running on Java 11 (#8463)

Motivation:

When closing a inbound SSL connection before the remote peer has send a close notify, the Java JDK is trigger happy to throw an exception. This exception can be ignored since the connection is about to be closed.
The exception wasn't printed in Java 8, based on filtering on the exception message. In Java 11 the exception message has been changed.

Modifications:
Update the if statement to also filter/swallow the message on Java 11.

Result:
On Java 11 the exception isn't printed with log levels set to debug. The old behaviour is maintained.
2018-11-16 10:32:34 +01:00
Norman Maurer
8d4d76d216
ReferenceCountedOpenSslEngine SSLSession.getLocalCertificates() / getLocalPrincipial() did not work when KeyManagerFactory was used. (#8560)
Motivation:

The SSLSession.getLocalCertificates() / getLocalPrincipial() methods did not correctly return the local configured certificate / principal if a KeyManagerFactory was used when configure the SslContext.

Modifications:

- Correctly update the local certificates / principial when the key material is selected.
- Add test case that verifies the SSLSession after the handshake to ensure we correctly return all values.

Result:

SSLSession returns correct values also when KeyManagerFactory is used with the OpenSSL provider.
2018-11-16 07:38:32 +01:00
Norman Maurer
20d4fda55e
Return the correct pointer from ReferenceCountedOpenSslContext.context() and sslCtxPointer() (#8562)
Motivation:

We did not return the pointer to SSL_CTX put to the internal datastructure of tcnative.

Modifications:

Return the correct pointer.

Result:

Methods work as documented in the javadocs.
2018-11-16 07:37:57 +01:00
Norman Maurer
0d2e38d5d6
Correctly convert supported signature algorithms when using BoringSSL (#8481)
* Correctly convert supported signature algorithms when using BoringSSL

Motivation:

BoringSSL uses different naming schemes for the signature algorithms so we need to adjust the regex to also handle these.

Modifications:

- Adjust SignatureAlgorithmConverter to handle BoringSSL naming scheme
- Ensure we do not include duplicates
- Add unit tests.

Result:

Correctly convert boringssl signature algorithm names.
2018-11-14 19:23:11 +01:00
Norman Maurer
d1654484c1
Correctly convert between openssl / boringssl and java cipher names when using TLSv1.3 (#8485)
Motivation:

We did not correctly convert between openssl / boringssl and java ciphers when using TLV1.3 which had different effects when either using openssl or boringssl.
 - When using openssl and TLSv1.3 we always returned SSL_NULL_WITH_NULL_NULL as cipher name
 - When using boringssl with TLSv1.3 we always returned an incorrect constructed cipher name which does not match what is defined by Java.

Modifications:

 - Add correct mappings in CipherSuiteConverter for both openssl and boringssl
 - Add unit tests for CipherSuiteConvert
 - Add unit in SSLEngine which checks that we do not return SSL_NULL_WITH_NULL_NULL ever and that server and client returns the same cipher name.

Result:

Fixes https://github.com/netty/netty/issues/8477.
2018-11-14 08:49:13 +01:00
Tim Brooks
11ec7d892e Cleanup SslHandler handshake/renegotiation (#8555)
Motivation:

The code for initiating a TLS handshake or renegotiation process is
currently difficult to reason about.

Modifications:

This commit introduces to clear paths for starting a handshake. The
first path is a normal handshake. The handshake is started and a timeout
is scheduled.

The second path is renegotiation. If the first handshake is incomplete,
the renegotiation promise is added as a listener to the handshake
promise. Otherwise, the renegotiation promise replaces the original
promsie. At that point the handshake is started again and a timeout is
scheduled.

Result:

Cleaner and easier to understand code.
2018-11-14 08:19:06 +01:00
Norman Maurer
4c73d24ea8
Handshake timeout may never be scheduled if handshake starts via a flush or starttls is used. (#8494)
Motivation:

We did not correctly schedule the handshake timeout if the handshake was either started by a flush(...) or if starttls was used.

Modifications:

- Correctly setup timeout in all cases
- Add unit tests.

Result:

Fixes https://github.com/netty/netty/issues/8493.
2018-11-13 19:22:38 +01:00
Norman Maurer
c0dfb568a2
SSLHandler may throw AssertionError if writes occur before channelAct… (#8486)
Motivation:

If you attempt to write to a channel with an SslHandler prior to channelActive being called you can hit an assertion. In particular - if you write to a channel it forces some handshaking (through flush calls) to occur.

The AssertionError only happens on Java11+.

Modifications:

- Replace assert by an "early return" in case of the handshake be done already.
- Add unit test that verifies we do not hit the AssertionError anymore and that the future is correctly failed.

Result:

Fixes https://github.com/netty/netty/issues/8479.
2018-11-11 07:23:08 +01:00
Norman Maurer
8a24df88a4
Ensure we correctly call wrapEngine(...) during tests. (#8473)
Motivation:

We should call wrapEngine(...) in our SSLEngineTest to correctly detect all errors in case of the OpenSSLEngine.

Modifications:

Add missing wrapEngine(...) calls.

Result:

More correct tests
2018-11-08 15:22:33 +01:00
Norman Maurer
fd57d971d1
Override and so delegate all methods in OpenSslX509Certificate (#8472)
Motivation:

We did not override all methods in OpenSslX509Certificate and delegate to the internal 509Certificate.

Modifications:

Add missing overrides.

Result:

More correct implementation
2018-11-07 12:16:04 +01:00
Norman Maurer
4760dc5c2d
Don't double release ByteBuf when parsing of the X509Certificate fails (#8457)
Motivation:

Due a bug in our implementation we tried to release the same ByteBuf two times when we failed to parse the X509Certificate as closing the ByteBufInputStream already closed it.

Modifications:

- Don't close the ByteBuf when closing the ByteBufInputStream
- Explicit release all ByteBufs after we are done parsing in a finally block.
- Add testcase.

Result:

Do not produce an IllegalReferenceCountException and throw the correct CertificateException.
2018-11-02 17:08:53 +01:00
Daniel Gartmann
9c70dc8ba5 Replaced obsolete cryptographic primitive with a modern/secure one. (#8450)
Motivation:

SHA1 is a broken hash function and shouldn't be used anymore (see: https://shattered.io/).
Security scanning tools will raise this as an issue and it will reflect badly on netty and I, therefore, recommend to use a SHA2 hash function which is secure and won't be flagged by such tools.

Modifications:

Replaced insecure SHA1 based signing scheme with SHA2.

Result:

Modern and thus secure cryptographic primitives will be in use and won't be flagged by security scanning tools.
2018-11-02 07:20:54 +01:00
Norman Maurer
52699bd6dd
Add test to verify that invalid ciphers are handled in all SSLEngine implementations correctly. (#8443)
Motivation:

https://github.com/netty/netty/issues/8442 reported that we fail to build a SslContext when an invalid cipher is used with netty-tcnative-boringssl-static, while it worked before. This test verifies that this is now consistent with all other SSLEngine implementations.

Modifications:

Add test-case to verify consistent behaviour

Result:

More tests to assert consistent behaviour across SSLEngine implementations
2018-10-30 17:58:08 +01:00
Norman Maurer
46460de243
Correctly init X509Certificate array when testing if we need to wrap the KeyManager due of TLSv1.3 (#8435)
Motivation:

201e984cb3 added support to use native TLSv1.3 support even with Java versions prior to 11. For this we try to detect if we need to wrap the used KeyManager or not. This testing code did create an X509Certificate[1] but does not correctly also set the certficiate on index 0. While this should be harmless we should better do the right thing and set it.

Modifications:

Correctly init the array.

Result:

Cleaner and more correct code.
2018-10-30 08:17:31 +01:00
root
3e7ddb36c7 [maven-release-plugin] prepare for next development iteration 2018-10-29 15:38:51 +00:00
root
9e50739601 [maven-release-plugin] prepare release netty-4.1.31.Final 2018-10-29 15:37:47 +00:00
Norman Maurer
ce39773e04
Add support for boringssl and TLSv1.3 (#8412)
Motivation:

0ddc62cec0 added support for TLSv1.3 when using openssl 1.1.1. Now that BoringSSL chromium-stable branch supports it as well we can also support it with netty-tcnative-boringssl-static.
During this some unit tests failed with BoringSSL which was caused by not correctly handling flush() while the handshake is still in progress.

Modification:

- Upgrade netty-tcnative version which also supports TLSv1.3 when using BoringSSL
- Correctly handle flush() when done while the handshake is still in progress in all cases.

Result:

Easier for people to enable TLSv1.3 when using native SSL impl.
Ensure flush() while handshake is in progress will always be honored.
2018-10-26 15:29:49 -07:00
Norman Maurer
9e762e8816
Correctly detect if KeyManagerFactory is supported by OpenSSL even when sun.security.x509.* can not be accessed and bouncycastle is not on the classpath. (#8415)
Motivation:

OpenSsl used SelfSignedCertificate in its static init block to detect if KeyManagerFactory is supported. Unfortunally this only works when either sun.security.x509.* can be accessed or bouncycastle is on the classpath.
We should not depend on either of it.

This came up in https://github.com/netty/netty-tcnative/issues/404#issuecomment-431551890.

Modifications:

Just directly use the bytes to generate the X509Certificate and so not depend on sun.security.x509.* / bouncycastle.

Result:

Correctly be able to detect if KeyManagerFactory can be supported in all cases.
2018-10-23 17:08:23 +02:00
Norman Maurer
91201fb338
Remove workaround in tests for TLSv1.3 bug in Java11 as it was fixed in 11.0.1 (#8409)
Motivation:

We had put some workaround in our tests due a bug in the Java11 implementation of TLSv1.3. This was now fixes as part of 11.0.1.

See https://bugs.openjdk.java.net/browse/JDK-8211067.

Modifications:

Remove workaround in SSL tests.

Result:

Run all tests with supported TLS version.
2018-10-19 17:21:04 +02:00
Norman Maurer
201e984cb3
Allow to use TLSv1.3 with netty-tcnative withe java versions prior to 11. (#8394)
Motivation:

At the moment it's only possible to use TLSv1.3 with netty-tcnative if Java 11 is used. It should be possible to do so even with Java 8, 9 and 10.

Modification:

Add a workaround to be able to use TLSv1.3 also when using Java version prior to Java 11 and the default X509ExtendedTrustManager is used.

Result:

Be able to use TLSv1.3 also with past versions of Java.
2018-10-18 13:50:12 +02:00
Norman Maurer
3543e17967
Ensure OpenSslEngine will not try to call SSL_free multiple times even when constructor throws. (#8399)
Motivation:

When the constructor of OpenSslEngine threw we could end up to self call SSL_free by ourself and then have the finalizer do the same which may lead to double free-ing and so SIGSEV.

Modifications:

Just call shutdown() when the constructor throws and so ensure SSL_free is guarded correctly in the finalizer.

Result:

No more SIGSEV possible.
2018-10-18 07:38:03 +02:00
Norman Maurer
0ddc62cec0
Add support for TLSv1.3 (#8293)
Motivation:

TLSv1.3 support is included in java11 and is also supported by OpenSSL 1.1.1, so we should support when possible.

Modifications:
- Add support for TLSv1.3 using either the JDK implementation or the native implementation provided by netty-tcnative when compiled against openssl 1.1.1
- Adjust unit tests for semantics provided by TLSv1.3
- Correctly handle custom Provider implementations that not support TLSv1.3

Result:

Be able to use TLSv1.3 with netty.
2018-10-17 08:35:35 +02:00
Stephane Landelle
9eebe7ed74 Add full JdkSslContext public constructor, close #8384 (#8389)
Motivation:

JdkSslContext provides public constructors to wrap an existing `javax.net.ssl.SSLContext`.

Sadly, some options combinations are not possible with the existing constructors, eg:
*  protocols is not exposed and always forced to null, so default protocols are always enforced
* startTls is not exposed and always forced to false

Modification:

Add full constructor that take protocols and startTls parameters.

Result:

It's possible to create a JdkSslContext from an existing SSLContext and still have control over protocols and startTls
2018-10-17 08:28:39 +02:00
Norman Maurer
aae7cdca96
Prevent NPE when attempting to set client key material with no alias (#8378)
Motivation:

It is possible that a client is unable to locate a certificate alias given the list of issuers and key types. In this case the X509KeyManager
will return a null which when past to the OpenSslKeyMaterialProvider implementation may produce a NPE. If no matching alias could be found we should not
call OpenSslKeyMaterialProvider at all which is also consistent what OpenJDK does.

Modifications:

- Add null check before calling OpenSslKeyMaterialProvider
- Add unit test.

Result:

No more NPE caused by passing null as client alias.
2018-10-12 09:27:46 +02:00
Norman Maurer
59973e93dd
Ensure X509KeyManager methods are called on the correct time when using server-side and support more methods of ExtendedSSLSession. (#8283)
Motivation:

Before when on server-side we just called the X509KeyManager methods when handshake() was called the first time which is not quite correct as we may not have received the full SSL hello / handshake and so could not extra for example the SNI hostname that was requested.
OpenSSL exposes the SSL_CTX_set_cert_cb function which allows to set a callback which is executed at the correct moment, so we should use it. This also allows us to support more methods of ExtendedSSLSession easily.

Modifications:

- Make use of new methods exposed by netty-tcnative since https://github.com/netty/netty-tcnative/pull/388 to ensure we select the key material at the correct time.
- Implement more methods of ExtendedOpenSslSession
- Add unit tests to ensure we are able to retrieve various things on server-side in the X509KeyManager and so verify it is called at the correct time.
- Simplify code by using new netty-tcnative methods.

Result:

More correct implementation for server-side usage and more complete implemented of ExtendedSSLSession.
2018-09-28 11:34:38 +02:00
Norman Maurer
73acac13f4
Check if hostname validation is supported before trying to use in test. (#8333)
Motivation:

a208f6dc7c added a testcase which uses hostname validation which may not be supported by OpenSSL depending on the version that is used. We should check first before we try to use it.

Modifications:

Add assumeTrue(...) check to ensure hostname validation is supported before trying to run the test.

Result:

No more test-failures on OpenSSL versions < 1.0.2.
2018-09-28 10:54:05 +02:00
Norman Maurer
a208f6dc7c
Do the same extended checks as the JDK when a X509TrustManager is used with the OpenSSL provider. (#8307)
Motivation:

When a X509TrustManager is used while configure the SslContext the JDK automatically does some extra checks during validation of provided certs by the remote peer. We should do the same when our native implementation is used.

Modification:

- Automatically wrap a X509TrustManager and so do the same validations as the JDK does.
- Add unit tests.

Result:

More consistent behaviour. Fixes https://github.com/netty/netty/issues/6664.
2018-09-28 09:19:58 +02:00
root
2d7cb47edd [maven-release-plugin] prepare for next development iteration 2018-09-27 19:00:45 +00:00
root
3a9ac829d5 [maven-release-plugin] prepare release netty-4.1.30.Final 2018-09-27 18:56:12 +00:00
Norman Maurer
652650e127
Fix leak in SniClientJava8TestUtil (#8326)
Motivation:

4d1458604a did fix some leaks in SniClientTest but missed the ones in SniClientJava8TestUtil.

Modifications:

Correctly release SslContext.

Result:

No more leaks in SNI tests.
2018-09-27 09:32:41 +02:00
Norman Maurer
4d1458604a
Fix leak in SniClientTest. (#8324)
Motivation:

We need to release the ReferenceCountedSslContext to eliminate resource leaks.

Reported in https://garage.netty.io/teamcity/viewLog.html?buildId=33353&buildTypeId=netty_build_oraclejdk8&tab=buildLog#_focus=157264.

Modifications:

Call release on the SslContext instances.

Result:

No more leaks in tests.
2018-09-26 19:59:00 +02:00
Norman Maurer
a80c49828f
Cleanup SSL test. (#8301)
Motivation:

I noticed that we had some errors showing up in a test (which did not fail it tho) because we tried to full-fill the promise multiples times.

Modifications:

Use trySuccess(...) as we may produce multiple exceptions.

Result:

Less errors during test-run.
2018-09-21 08:04:24 -07:00
Norman Maurer
01db30a163
Correctly implement ExtendedSSLSession.getStatusResponses() for ReferenceCountedOpenSslEngine (#8297)
Motivation:

Java9 added getStatusResponses() to ExtendedSSLSession which we should correctly support when possible.

Modifications:

Implement the method correctly.

Result:

More complete and correct implementation.
2018-09-19 17:13:44 -07:00
Norman Maurer
34d52fcbfe
Implemented ExtendedOpenSslSession.getStatusResponses() so it not throws an UnsupportedOperationException. (#8290)
Motivation:

6ed7c6c75d added support for ExtendedOpenSslSession but we did not override getStatusResponses(). This lead to test failures on java9.

Modifications:

Implement ExtendedOpenSslSession.getStatusResponses() so it just returns an empty list.

Result:

Test pass again on Java9.
2018-09-14 20:33:09 +02:00
Norman Maurer
2b1514ec5a
Only use KeyManagerFactory in SniClientTest when supported by OpenSSL version. (#8289)
Motivation:

6ed7c6c75d added a test which blindly assumed we can use a KeyManagerFactory all the time. This is only true if have OpenSSL 1.0.2 or later, which may not be the case.

Modifications:

Only use KeyManagerFactory in test if the OpenSSL version does support it.

Result:

More robust tests.
2018-09-14 19:01:55 +02:00
Norman Maurer
6ed7c6c75d
Return an ExtendSSLSession whenever possible to allow more strict checking when using OpenSSL (#8281)
Motivation:

When an ExtendedSSLSession is used its possible to do more strict checking of the keys during handshake. We should do this whenever possible.

Modification:

- Return an ExtendedSSLSession when using client-mode and Java7+
- Add unit test
- Simplify unit tests

Result:

More consistent behaviour.
2018-09-14 14:33:11 +02:00
Norman Maurer
5ff6b57940
PemPrivateKey.toPem(...) should throw IllegalArgumentException when P… (#8253)
* PemPrivateKey.toPem(...) should throw IllegalArgumentException when PrivateKey which does not support encoding is used.

Motivation:

At the moment when a PrivateKey is used that does not support encoding we throw a NPE when trying to convert the key. We should better throw an IllegalArgumentException with the details about what key we tried to encode.

Modifications:

- Check if PrivateKey.getEncoded() returns null and if so throw an IllegalArgumentException
- Add unit test.

Result:

Better handling of non-supported PrivateKey implementations.
2018-09-05 20:33:40 +02:00
Norman Maurer
f4bafd4fe0
Correctly check if cipher is supported for each SslProvider before trying to run test and fix buffer leaks in test. (#8247)
Motivation:

5aaa16b24c introduced a testcase for specific ciphersuites and checked if these are supported by our native implementation before running it. Unfortunally this is not good enough as even on the JDK it may not be supported on various JDK versions (like Java7). Beside this the test leaked buffers.

Modifications:

- Correctly check if ciphersuite is supported on each SslProvider before trying to run test.
- Fix buffer leaks.

Result:

Testsuite pass again on Java7 and others when -Pleak is used.
2018-09-01 08:11:13 +02:00
Norman Maurer
a644563625
Add more debug informations when log SSL errors. (#8241)
Motivation:

ea626ef8c3 added more debug logging but we can even include a bit more.

Modifications:

Always log the error number as well.

Result:

More informations for debugging SSL errors.
2018-08-30 20:44:47 +02:00
Norman Maurer
ea626ef8c3
Log more details when shutdown SSL because of an error. (#8236)
Motivation:

We should log a bit more details about why we shutdown the SSL.

Modifications:

Add the return value of SSL_get_error(...) as well in debug mode.

Result:

More logging to make it easier to understand why an SSL error happened.
2018-08-29 21:52:26 +02:00
Roger
5aaa16b24c An unit test to ensure that cipher suites don't break/disappear between releases. (#8225)
Motivation

Ensure classes of cipher suites continue working between releases. Adding just a DHE check for now as it caused #8165 but this test can be expaned to other suites.

Modifications

Adding an unit test that checks for the presence of a cipher suite.

Result

Prevent #8165 from happening in the future.
2018-08-29 14:14:26 +02:00
root
a580dc7585 [maven-release-plugin] prepare for next development iteration 2018-08-24 06:36:33 +00:00
root
3fc789e83f [maven-release-plugin] prepare release netty-4.1.29.Final 2018-08-24 06:36:06 +00:00
Norman Maurer
df00539fa2
Allow to load PrivateKey via OpenSSL Engine (#8200)
Motivation:

OpenSSL itself has an abstraction which allows you to customize some things. For example it is possible to load the PrivateKey from the engine. We should support this.

Modifications:

Add two new static methods to OpenSslX509KeyManagerFactory which allow to create an OpenSslX509KeyManagerFactory that loads its PrivateKey via the OpenSSL Engine directly.

Result:

More flexible usage of OpenSSL possible
2018-08-18 07:20:44 +02:00
Norman Maurer
bbe2e4d224
We should try to load netty-tcnative before using it in OpenSslCertificateException. (#8202)
Motivation:

In OpenSslCertificateException we should ensure we try to load netty-tcnative before trying to use any class from it as otherwise it may throw an error due missing linking of the native libs.

Modifications:

- Ensure we call OpenSsl.isAvailable() before we try to use netty-tcnative for validation
- Add testcase.

Result:

No more errors causing by not loading native libs before trying to use these.
2018-08-18 06:26:45 +02:00
Norman Maurer
8255f85f24
Rename SslHandler.close(...) to closeOutbound(...) as it is still useful and delegate to the methods. (#8193)
* Rename SslHandler.close(...) to closeOutbound(...) as it is still useful and delegate to the methods.

Motivation:

Sometimes the user may want to send a close_notify without closing the underlying Channel. For this we offered the SslHandler.close(...) methods which were marked as deeprecated. We should offer an way to still do this without the user calling deprecated methods.

See https://stackoverflow.com/questions/51710231/using-nettys-sslhandlerclosechannelhandlercontext-channelpromise/51753742#comment90555949_51753742 .

Modifications:

- Remove deprecation of the SslHandler.close(...) method that exactly allows this and rename these to closeOutbound(...) as this is more clear.
- Add close(...) methods that delegate to these and mark these as deprecated.

Result:

Be able to send close_notify without closing the Channel.
2018-08-15 20:07:56 +02:00
Lai Jiang
dbc9ec1ab2 Set SNI servernames in OpenSSL engine when created in client mode (#8178)
Motivation:

When using the JDK SSL provider in client mode, the SNI host names (called serverNames in SslEngineImpl) is set to the peerHost (if available) that is used to initialize the SSL Engine:

http://hg.openjdk.java.net/jdk8/jdk8/jdk/file/687fd7c7986d/src/share/classes/sun/security/ssl/SSLEngineImpl.java#l377

This allows one to call SslEngine.getSSLParameters() and inspect what is the SNI name to be sent. The same should be done in the OpenSSL provider as well. Currently even though the the SNI name is sent by the OpenSSL provider during handshake when the peerHost is specified, it is missing from the parameters.

Modification:

Set the sniHostNames field when SNI is to be used. Also verifies the peer is actually a hostname before setting it as the SNI name, which is consistent with JDK SSL provider's behavior.

Result:

SslEngine using the OpenSSL provider created in client mode with peerHost will initialize sniHostNames with the peerHost.

Calling SslEngine.getSSLParameters().getServerNames() will return a list that contains that name.
2018-08-09 21:30:02 +02:00
Norman Maurer
44d3753c48
Fix NPE exception when using invalid cipher during building SslContext. (#8171)
Motivation:

We missed to do a null check before trying to destroy the OpenSslSessionContext, which could lead to a NPE.

Modifications:

Add null check and tests.

Result:

Fix https://github.com/netty/netty/issues/8170.
2018-08-02 21:42:21 +02:00
Norman Maurer
fe14bad69c
Adjust SSL related tests to be more correct and so pass in the next EA release of java11. (#8162)
Motivation:

In some of our tests we not correctly init the SSLEngine before trying to perform a handshake which can cause an IllegalStateException. While this not happened in previous java releases it does now on Java11 (which is "ok" as its even mentioned in the api docs). Beside this how we selected the ciphersuite to test renegotation was not 100 % safe.

Modifications:

- Correctly init SSLEngine before using it
- Correctly select ciphersuite before testing for renegotation.

Result:

More correct tests and also pass on next java11 EA release.
2018-08-01 06:37:53 +02:00
root
fcb19cb589 [maven-release-plugin] prepare for next development iteration 2018-07-27 04:59:28 +00:00
root
ff785fbe39 [maven-release-plugin] prepare release netty-4.1.28.Final 2018-07-27 04:59:06 +00:00
Norman Maurer
620dad0c26
Allow to validate sni hostname with underscore (#8150)
Motivation:

We should allow to also validate sni hostname which contains for example underscore when using our native SSL impl. The JDK implementation does this as well.

Modifications:

- Construct the SNIHostName via byte[] and not String.
- Add unit test

Result:

Fixes https://github.com/netty/netty/issues/8144.
2018-07-27 01:56:32 +08:00
root
b4dbdc2036 [maven-release-plugin] prepare for next development iteration 2018-07-11 15:37:40 +00:00
root
1c16519ac8 [maven-release-plugin] prepare release netty-4.1.27.Final 2018-07-11 15:37:21 +00:00
Norman Maurer
df08467d7c
Fix possible NPE introduced by a137291ad1 when using SslProvider.OPENSSL and init via files or OpenSslX509KeyManagerFactory (#8126)
Motivation:

a137291ad1 introduced a way to get the most speed out of OpenSSL by not only caching keymaterial but pre-compute these. The problem was we missed to check for null before doing an instanceof check and then a cast which could lead to a NPE as we tried to cast null to Exception and throw it.

Modifications:

Add null check and unit test.

Result:

No more NPE when keymaterial was not found for requested alias.
2018-07-11 15:19:37 +01:00
root
7bb9e7eafe [maven-release-plugin] prepare for next development iteration 2018-07-10 05:21:24 +00:00
root
8ca5421bd2 [maven-release-plugin] prepare release netty-4.1.26.Final 2018-07-10 05:18:13 +00:00
Norman Maurer
a137291ad1
Add OpenSslX509KeyManagerFactory which makes it even easier for peopl… (#8084)
* Add OpenSslX509KeyManagerFactory which makes it even easier for people to get the maximum performance when using OpenSSL / LibreSSL / BoringSSL  with netty.

Motivation:

To make it even easier for people to get the maximum performance when using native SSL we should provide our own KeyManagerFactory implementation that people can just use to configure their key material.

Modifications:

- Add OpenSslX509KeyManagerFactory which users can use for maximum performance with native SSL
- Refactor some internal code to re-use logic and not duplicate it.

Result:

Easier to get the max performance out of native SSL implementation.
2018-07-10 00:42:37 -04:00
Norman Maurer
83710cb2e1
Replace toArray(new T[size]) with toArray(new T[0]) to eliminate zero-out and allow the VM to optimize. (#8075)
Motivation:

Using toArray(new T[0]) is usually the faster aproach these days. We should use it.

See also https://shipilev.net/blog/2016/arrays-wisdom-ancients/#_conclusion.

Modifications:

Replace toArray(new T[size]) with toArray(new T[0]).

Result:

Faster code.
2018-06-29 07:56:04 +02:00
Norman Maurer
ecc238bea5
Only try to call SSL.setHostnameValidation(...) if needed. (#8074)
Motivation:

As the used OpenSSL version may not support hostname validation we should only really call SSL.setHostNameValidation(...) if we detect that its needed.

Modifications:

Only call SSL.setHostNameValidation if it was disabled before and now it needs to be enabled or if it was enabled before and it should be disabled now.

Result:

Less risk of an exception when using an OpenSSL version that does not support hostname validation.
2018-06-28 11:07:13 +02:00
Norman Maurer
2818730092
OpenSSL (and so netty-tcnative) should allow to use custom engine. (#8050)
Motivation:

OpenSSL allows to use a custom engine for its cryptographic operations. We should allow the user to make use of it if needed.

See also: https://www.openssl.org/docs/man1.0.2/crypto/engine.html.

Modifications:

Add new system property which can be used to specify the engine to use (null is the default and will use the build in default impl).

Result:

More flexible way of using OpenSSL.
2018-06-28 08:13:52 +02:00
Norman Maurer
0337ecdcc8 Allow to cache keymaterial when using OpenSSL
Motiviation:

During profiling it showed that a lot of time during the handshake is spent by parsing the key / chain over and over again. We should cache these parsed structures if possible to reduce the overhead during handshake.

Modification:

- Use new APIs provided by https://github.com/netty/netty-tcnative/pull/360.
- Introduce OpensslStaticX509KeyManagerFactory which allows to wrap another KeyManagerFactory and caches the key material provided by it.

Result:

In benchmarks handshake times have improved by 30 %.
2018-06-24 07:36:27 +02:00
Norman Maurer
3fb1b992ef
Remove some cipher protocol combos for tests due removal in more recent versions of OpenSSL (#8033)
Motivation:

Some of the cipher protocol combos that were used are no longer included in more recent OpenSSL releases.

Modifications:

Remove some combos that were used for testing.

Result:

Tests also pass in more recent OpenSSL versions (1.1.0+).
2018-06-19 08:12:02 +02:00
Roger
3e3e5155b9 Check if Log level is enabled before creating log statements (#8022)
Motivation

There is a cost to concatenating strings and calling methods that will be wasted if the Logger's level is not enabled.

Modifications

Check if Log level is enabled before producing log statement. These are just a few cases found by RegEx'ing in the code.

Result

Tiny bit more efficient code.
2018-06-13 23:21:53 -07:00
Norman Maurer
a4393831f0
Fix race in SslHandlerTest that could lead to NPE. (#7989)
Motivation:

SslHandlerTest tried to get access to the SslHandler in the pipeline via pipeline.get(...) which may return null if the channel was already closed and so the pipeline was teared down.

This showed up in a test run as:

```
-------------------------------------------------------------------------------
Test set: io.netty.handler.ssl.SslHandlerTest
-------------------------------------------------------------------------------
Tests run: 17, Failures: 0, Errors: 1, Skipped: 1, Time elapsed: 0.802 sec <<< FAILURE! - in io.netty.handler.ssl.SslHandlerTest
testCloseOnHandshakeFailure(io.netty.handler.ssl.SslHandlerTest)  Time elapsed: 0.188 sec  <<< ERROR!
java.lang.NullPointerException
        at io.netty.handler.ssl.SslHandlerTest.testCloseOnHandshakeFailure(SslHandlerTest.java:640)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
        at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
        at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
        at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
        at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
        at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.lang.Thread.run(Thread.java:844)
```

Modifications:

Use an AtomicReference to propagate the SslHandler instance to the outer scope.

Result:

No more NPE.
2018-05-30 22:07:42 +02:00
Norman Maurer
987c443888
Use ByteBufAllocator used by the ReferenceCountedOpenSslEngine when build key-material. (#7952)
Motivation:

When we build the key-material we should use the ByteBufAllocator used by the ReferenceCountedOpenSslEngine when possible.

Modifications:

Whenever we have access to the ReferenceCountedOpenSslEngine we use its allocator.

Result:

Use correct allocator
2018-05-18 19:36:57 +02:00
Norman Maurer
7727649b2c
Add tests for the Conscrypt based SSLEngine. (#7950)
Motivation:

We currently have only interopt tests for Conscrypt, we should also have non-interopt tests.

Modifications:

Add ConscryptSslEngineTest

Result:

More tests
2018-05-18 19:36:40 +02:00
Norman Maurer
546ddd2c28
Correctly calculate and respect if we can correctly fullfil wrap for alerts. (#7945)
Motivation:

We previously did not correctly take into account when we could not wrap (and so produce) the full SSL record with an alert when the SSLEngine was closed.

There are two problems here:

- If we call wrap(...) with an empty dst buffer after closeOutbound() was called we will not notify the user if we could not store the whole SSLRecord into the dst buffer and so we may produce incomplete SSLRecords

Modifications:

Add unit test which failed before.

Result:

Correctly handle the case when the dst buffer is not big enough and and alert needs to be produced.
2018-05-16 20:08:05 +02:00
Norman Maurer
47985c11c1 Add missing parameter when delegate to SSLEngine.
Motivation:

https://github.com/netty/netty/pull/7943 had a bug which caused to not have the argument passed to the delegating method.

Modifications:

Add argument to release call.

Result:

Correctly delegate method.
2018-05-16 20:01:58 +02:00
Norman Maurer
0bce0450c0
Adjust tests to also pass when using BoringSSL (#7946)
Motivation:

Some of the tests failed when using BoringSSL as some protocol / cipher combinations are not supported and it uses a different alert when the cert is not valid yet.

Modification:

- Remove protocol / cipher combos that are not supported by BoringSSL
- Test for different alert when using BoringSSL

Result:

Not test failures when using BoringSSL.
2018-05-16 18:58:27 +02:00
Norman Maurer
932d77b83e
Verify error stack is empty after each operation when using ReferenceCountedOpenSslEngine. (#7943)
Motivation:

https://github.com/netty/netty/pull/7941 proved that its easy to not correctly clear the error stack sometimes. We should do carefully test this.

Modifications:

Add a new SSLEngine wrapper that is used during tests, which verifies that the error stack is empty after each method call.

Result:

Better testing.
2018-05-16 13:50:37 +02:00
Norman Maurer
69c644bb98
Correctly detect if protocol is enabled when using netty-tcnative (#7940)
Motivation:

We sometimes did not correctly detect when a protocol is not enabled when using netty-tcnative as we did not take into account when the option flag is 0 (as for example in BoringSSL for SSLv2).

Modifications:

- Correctly take an option flag with 0 into account.
- Add unit tests.

Result:

Fixes https://github.com/netty/netty/issues/7935.
2018-05-16 07:23:55 +02:00
Norman Maurer
ed54ab034d
Correctly clear the error stack in all cases when using ReferenceCountedOpenSslEngine. (#7941)
Motivation:

We missed to correctly clear the error stack in one case when using the ReferenceCountedOpenSslEngine. Because of this it was possible to pick up an error on an unrelated operation.

Modifications:

- Correctly clear the stack
- Add verification of empty error stack to tests.

Result:

Not possible to observe unrelated errors.
2018-05-15 19:44:02 +02:00
Norman Maurer
64bb279f47 [maven-release-plugin] prepare for next development iteration 2018-05-14 11:11:45 +00:00
Norman Maurer
c67a3b0507 [maven-release-plugin] prepare release netty-4.1.25.Final 2018-05-14 11:11:24 +00:00
Norman Maurer
54c3de0f8a
Add test to validate SSLEngine does not zero out src buffer on wrap. (#7914)
Motivation:

We had a bug-report that claimed the src buffer used by OpenSslEngine will be zero out.

Modifications:

Add testcase to ensure correct behaviour

Result:

Testcase for https://github.com/netty/netty/issues/7753
2018-05-07 20:10:06 +02:00
Norman Maurer
358249e5c9
Allow to disable native transport and native ssl support via system property. (#7903)
Motivation:

Sometimes it's useful to disable native transports / native ssl to debug a problem. We should allow to do so with a system property so people not need to adjust code for this.

Modifications:

Add system properties which allow to disable native transport and native ssl.

Result:

Easier to disable native code usage without code changes.
2018-05-04 14:44:44 +02:00
Norman Maurer
b75f44db9a [maven-release-plugin] prepare for next development iteration 2018-04-19 11:56:07 +00:00
Norman Maurer
04fac00c8c [maven-release-plugin] prepare release netty-4.1.24.Final 2018-04-19 11:55:47 +00:00
Nikolay Fedorovskikh
401b196623 Extract common parts from if statements (#7831)
Motivation:
Some `if` statements contains common parts that can be extracted.

Modifications:
Extract common parts from `if` statements.

Result:
Less code and bytecode. The code is simpler and more clear.
2018-04-11 14:36:56 +02:00
root
0a61f055f5 [maven-release-plugin] prepare for next development iteration 2018-04-04 10:44:46 +00:00
root
8c549bad38 [maven-release-plugin] prepare release netty-4.1.23.Final 2018-04-04 10:44:15 +00:00
Nicolae Mihalache
8d78893a76 Avoid writing two times the same message in case channelWritabilityChanged event is called during a write.
Motivation:

ChunkedWriteHandler.doFlush is called twice from the same write if the channelWritabilityChanged event is invoked during the write. The buffer is already written so no extra data is sent on the socket but it causes the "promise already done" exception to be thrown.
This error happens only when the message is not ChunkedInput.

Modification:
Clear out the currentWrite reference before the ctx.write call, such that next time when the method is invoked the same object is not used twice.

Result:

Fixes #7819
2018-03-30 19:32:08 +02:00
Norman Maurer
2e92a2f5cd Ensure we not schedule multiple timeouts for close notify
Motivation:

We should only schedule one timeout to wait for the close notify to be done.

Modifications:

Keep track of if we already scheduled a timeout for close notify and if so not schedule another one.

Result:

No duplicated timeouts.
2018-03-27 09:43:46 +02:00
Norman Maurer
40af10b782 Skip NPN tests when libressl 2.6.1+ is used.
Motivation:

LibreSSL removed support for NPN in its 2.6.1+ releases.

Modifications:

Skip NPN tests in libressl 2.6.1+

Result:

Be able to run netty tests against libressl 2.6.1+ as well.
2018-03-22 08:30:49 +01:00
Norman Maurer
352e36a179 Remove code duplication in ChunkedWriteHandler
Motivation:

We had some code duplication in ChunkedWriteHandler.

Modifications:

Factor out duplicated code into private methods and reuse it.

Result:

Less code duplication.
2018-03-19 09:14:07 +01:00
Norman Maurer
d1055e0665 Add testcase for c11b23bbc1
Motivation:

c11b23bbc1 added a fix for closing the SSLEngine otbound but no test was provided.

Modifications:

Add testcase.

Result:

More tests.
2018-03-06 14:33:54 +09:00
Carl Mastrangelo
c11b23bbc1 Close SSLEngine when connection fails.
Motivation:
When using the JdkSslEngine, the ALPN class is used keep a reference
to the engine.   In the event that the TCP connection fails, the
SSLEngine is not removed from the map, creating a memory leak.

Modification:
Always close the SSLEngine regardless of if the channel became
active.  Also, record the SSLEngine was closed in all places.

Result:
Fixes: https://github.com/grpc/grpc-java/issues/3080
2018-03-04 06:55:51 -08:00
Norman Maurer
bf8cac4939 Workaround SSLEngine.unwrap(...) bug in Android 5.0
Motivation:

Android 5.0 sometimes not correctly update the bytesConsumed of the SSLEngineResult when consuming data from the input ByteBuffer. This will lead to handshake failures.

Modifications:

Add a workaround for Android 5.0

Result:

Be able to use netty on Android 5.0 by fixing https://github.com/netty/netty/issues/7758 .
2018-03-03 15:01:39 -08:00
Norman Maurer
69582c0b6c [maven-release-plugin] prepare for next development iteration 2018-02-21 12:52:33 +00:00
Norman Maurer
786f35c6c9 [maven-release-plugin] prepare release netty-4.1.22.Final 2018-02-21 12:52:19 +00:00
Norman Maurer
268b901844 SSL connection not closed properly after handshake failure
Motivation:

When SSL handshake fails, the connection should be closed. This is not true anymore after 978a46c.

Modifications:

- Ensure we always flush and close the channel on handshake failure.
- Add testcase.

Result:

Fixes [#7724].
2018-02-16 08:25:47 -08:00
Carl Mastrangelo
e00f24961a Get memory address from Unsafe for OpenSSL
Motivation:
Profiling tcnative SSL code showed a non trivial percentage (1%)
of time spent in JNI code for InstaceOf.   This turned out to be
from `Buffer.address` which makes a JNI call, which safely checks
on each call that The ByteBuffer is direct.

Modification:
Prefer using the address field of the pojo rather than looking it
up with JNI.  This is the same approach taken by the `OpenSsl`
class.

Result:
Less JNI overhead
2018-02-16 07:57:43 +01:00
Eric Anderson
8b273983f0 Load Conscrypt method via reflection only once
Motivation:

The code did reflection every method call which made the code slower and
harder to read with additional cases to consider.

Modifications:

Instead of loading the method and then throwing it away, save the Method
reference instead of the Class reference. Then also use more precise
exception handling for the method invocation.

Result:

Simpler, speedier code.
2018-02-09 21:13:42 +01:00
Stephane Landelle
40ae4fefc7 Introduce an alternative IdentityCipherSuiteFilter that defaults to supportedCiphers, close #7655
Motivation:

Sometimes, it would be convenient to be able to easily enable all
supported cipher suites, regardless of security.

Currently, the only way it to retrieve all supported ciphers and pass
them explicitly.

Modification:

Introduce a new IdentityCipherSuiteFilter singleton that defaults to
supportedCiphers instead of defaultCiphers when ciphers are null.

Result:

Convenient way to enabled all supported cipher suites.
2018-02-07 13:37:25 +01:00
Norman Maurer
e71fa1e7b6 [maven-release-plugin] prepare for next development iteration 2018-02-05 12:02:35 +00:00
Norman Maurer
41ebb5fcca [maven-release-plugin] prepare release netty-4.1.21.Final 2018-02-05 12:02:19 +00:00
Eric Anderson
2923f33530 Adapt to API changes in Conscrypt 1.0.0.RC11
Motivation:

In google/conscrypt#313 the Conscrypt.Engines class was removed in favor
of methods directly on Conscrypt and overloading. The Conscrypt-using
code in Netty used reflection to access the old API, that doesn't exist
anymore. And thus recent versions of Conscrypt fail to enable things
like ALPN with Netty.

Modifications:

Instead of calling Conscrypt.Engines.isConscrypt, call
Conscrypt.isConscrypt.

Result:

Conscrypt detected properly at runtime.
2018-02-02 07:25:59 +01:00
Scott Mitchell
3f3d309a28
JdkSslContext supported cipher suites incorrect
Motivation:
JdkSslContext builds the list of supported cipher suites, but assumes that ciphers prefixed with SSL_ and TLS_ will be interchangeable. However this is not the case and only applies to a small subset of ciphers. This results in the JdkSslContext attempting to use unsupported ciphers.

Modifications:
- When building the list of ciphers in JdkSslContext we should first check if the engine supports the TLS_ prefix cipher.

Result:
Fixes https://github.com/netty/netty/issues/7673
2018-02-01 09:34:44 -08:00
Scott Mitchell
978a46cc0a SslHandler unwrap out of order promise/event notificaiton
Motivation:
SslHandler#decode methods catch any exceptions and attempt to wrap
before shutting down the engine. The intention is to write any alerts
which the engine may have pending. However the wrap process may also
attempt to write user data, and may also complete the associated
promises. If this is the case, and a promise listener closes the channel
then SslHandler may later propagate a SslHandshakeCompletionEvent user
event through the pipeline. Since the channel has already been closed
the user may no longer be paying attention to user events.

Modifications:
- Sslhandler#decode should first fail the associated handshake promise
and propagate the SslHandshakeCompletionEvent before attempting to wrap

Result:
Fixes https://github.com/netty/netty/issues/7639
2018-02-01 09:25:40 -08:00
Norman Maurer
ea58dc7ac7 [maven-release-plugin] prepare for next development iteration 2018-01-21 12:53:51 +00:00
Norman Maurer
96c7132dee [maven-release-plugin] prepare release netty-4.1.20.Final 2018-01-21 12:53:34 +00:00
Dmitriy Dumanskiy
e6c9ac968d Cleanup: replaced deprecated ctx.attr() and ctx.hasAttr() methods usage with ch.attr() and ch.hasAttr().
Motivation:

Will allow easy removal of deprecated methods in future.

Modification:

Replaced ctx.attr(), ctx.hasAttr() with ctx.channel().attr(), ctx.channel().hasAttr().

Result:

No deprecated ctx.attr(), ctx.hasAttr() methods usage.
2018-01-18 15:00:41 +00:00
Scott Mitchell
062d926912 Remove remote initiated renegotiation support
Motivation:
We recently removed support for renegotiation, but there are still some hooks to attempt to allow remote initiated renegotiation to succeed. The remote initated renegotiation can be even more problematic from a security stand point and should also be removed.

Modifications:
- Remove state related to remote iniated renegotiation from OpenSslEngine

Result:
More renegotiation code removed from the OpenSslEngine code path.
2018-01-15 10:16:08 +01:00
Norman Maurer
ab9f0a0fda Remove direct usage of JKS and SunX509
Motivation:

When using netty on android or with for example a IBM JVM it may not be able to build a SslContext as we hardcoded the use of JKS and SunX509 (which both may not be present).

Modifications:

- Use the default algorithm / type which can be override via a System property
- Remove System property check as its redundant with KeyManagerFactory.getDefaultAlgorithm()

Result:

More portable code. Fixes [#7546].
2018-01-03 18:32:18 -08:00
Chris West
e24e06bfcb OpenSslEngine: Remove renegotiation support
Motivation:

SSL.setState() has gone from openssl 1.1. Calling it is, and probably
always has been, incorrect. Doing renogitation in this manner is
potentially insecure. There have been at least two insecure
renegotiation vulnerabilities in users of the OpenSSL library.

Renegotiation is not necessary for correct operation of the TLS protocol.

BoringSSL has already eliminated this functionality, and the tests
(now deleted) were not running on BoringSSL.

Modifications:

If the connection setup has completed, always return that
negotiation is not supported. Previously this was done only if we were
the client.

Remove the tests for this functionality.

Fixes #6320.
2018-01-02 13:00:11 -05:00
Norman Maurer
264a5daa41 [maven-release-plugin] prepare for next development iteration 2017-12-15 13:10:54 +00:00
Norman Maurer
0786c4c8d9 [maven-release-plugin] prepare release netty-4.1.19.Final 2017-12-15 13:09:30 +00:00
Norman Maurer
b2bc6407ab [maven-release-plugin] prepare for next development iteration 2017-12-08 09:26:15 +00:00
Norman Maurer
96732f47d8 [maven-release-plugin] prepare release netty-4.1.18.Final 2017-12-08 09:25:56 +00:00
Norman Maurer
1453f8d18b Ensure we not try to call select when the AbstractSniHandler was already removed from the pipeline.
Motivation:

We tried to call `select` after we closed the channel (and so removed all the handlers from the pipeline) when we detected a non SSL record. This would cause an exception like this:

```
Caused by: java.util.NoSuchElementException: io.netty.handler.ssl.SniHandler
	at io.netty.channel.DefaultChannelPipeline.getContextOrDie(DefaultChannelPipeline.java:1098)
	at io.netty.channel.DefaultChannelPipeline.replace(DefaultChannelPipeline.java:506)
	at io.netty.handler.ssl.SniHandler.replaceHandler(SniHandler.java:133)
	at io.netty.handler.ssl.SniHandler.onLookupComplete(SniHandler.java:113)
	at io.netty.handler.ssl.AbstractSniHandler.select(AbstractSniHandler.java:225)
	at io.netty.handler.ssl.AbstractSniHandler.decode(AbstractSniHandler.java:218)
	at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:489)
	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:428)
	... 40 more
```

Modifications:

- Ensure we rethrow the NotSslRecordException when detecting it (and closing the channel). This will also ensure we not call `select(...)`
- Not catch `Throwable` but only `Exception`
- Add test case.

Result:

Correctly handle the case of an non SSL record.
2017-12-08 07:42:15 +01:00
louxiu
805ac002e6 FIX: force a read operation for peer instead of self (#7454)
* FIX: force a read operation for peer instead of self

Motivation:
When A is in `writeInProgress` and call self close, A should
`finishPeerRead` for B(A' peer).

Modifications:
Call `finishPeerRead` with peer in `LocalChannel#doClose`

Result:
Clear confuse of code logic

* FIX: preserves order of close after write in same event loop

Motivation:
If client and server(client's peer channel) are in same event loop, client writes data to
server in `ChannelActive`. Server receives the data and write it
back. The client's read can't be triggered becasue client's
`ChannelActive` is not finished at this point and its `readInProgress`
is false. Then server closes itself, it will also close the client's
channel. And client has no chance to receive the data.

Modifications:
1. Add a test case to demonstrate the problem
2. When `doClose` peer, we always call
`peer.eventLoop().execute()` and `registerInProgress` is not needed.
3. Remove test case
`testClosePeerInWritePromiseCompleteSameEventLoopPreservesOrder`. This
test case can't pass becasue of this commit. IMHO, I think it is OK,
becasue it is reasonable that the client flushes the data to socket,
then server close the channel without received the data.
4. For mismatch test in SniClientTest, the client should receive server's alert before closed(caused by server's close)

Result:
The problem is gone.
2017-12-07 17:05:57 -08:00
Norman Maurer
aabb73a9d2 Add SniCompletionEvent which allows to easily retrieve the hostname that was used to select the SslContext.
Motivation:

At the moment its a bit "hacky" to retrieve the hostname that was used during SNI as you need to hold a reference to SniHandler and then call hostname() once the selection is done. It would be better to fire an event to let the user know we did the selection.

Modifications:

Add a SniCompletionEvent that can be used to get the hostname that was used to do the selection and was included in the SNI extension.

Result:

Easier usage of SNI.
2017-12-06 14:09:11 +01:00
Norman Maurer
ca1e1fcddf Only try to match SSLException message when debug logging is enabled.
Motivation:

We only want to log for the particular case when debug logging is enabled so we not need to try to match the message if this is not the case.

Modifications:

Guard with logger.isDebugEnabled()

Result:

Less overhead when debug logging is not enabled.
2017-12-05 20:57:08 +01:00
Scott Mitchell
d7c977dd71 SslHandler aggregation prefer copy over CompositeByteBuf
Motivation:
SslHandler will do aggregation of writes by default in an attempt to improve goodput and reduce the number of discrete buffers which must be accumulated. However if aggregation is not possible then a CompositeByteBuf is used to accumulate multiple buffers. Using a CompositeByteBuf doesn't provide any of the benefits of better goodput and in the case of small + large writes (e.g. http/2 frame header + data) this can reduce the amount of data that can be passed to writev by about half. This has the impact of increasing latency as well as reducing goodput.

Modifications:
- SslHandler should prefer copying instead of using a CompositeByteBuf

Result:
Better goodput (and potentially improved latency) at the cost of copy operations.
2017-12-04 11:02:33 -08:00
Silvio Giebl
3c8f4b81d7 Fixed default OpenSsl cipher suites
Motivation:

The default enabled cipher suites of the OpenSsl engine are not set to
SslUtils#DEFAULT_CIPHER_SUITES. Instead all available cipher suites are
enabled. This should happen only as a fallback.

Modifications:

Moved the line in the static initializer in OpenSsl which adds the
SslUtils#DEFAULT_CIPHER_SUITES to the default enabled cipher suites up
before the fallback.

Result:

The default enabled cipher suites of the OpenSsl engine are set to the
available ones of the SslUtils#DEFAULT_CIPHER_SUITES.
The default enabled cipher suites of the OpenSsl engine are only set to
all available cipher suites if no one of the
SslUtils#DEFAULT_CIPHER_SUITES is supported.
2017-11-29 12:12:43 +01:00
Tomasz Jędrzejewski
e8540c2b7a Adding stable JDK9 module names that follow reverse-DNS style
Automatic-Module-Name entry provides a stable JDK9 module name, when Netty is used in a modular JDK9 applications. More info: http://blog.joda.org/2017/05/java-se-9-jpms-automatic-modules.html

When Netty migrates to JDK9 in the future, the entry can be replaced by actual module-info descriptor.

Modification:

The POM-s are configured to put the correct module names to the manifest.

Result:

Fixes #7218.
2017-11-29 11:50:24 +01:00
Norman Maurer
c921742a42 Dont fire an SslHandshakeEvent if the handshake was not started at all.
Motivation:

We should not fire a SslHandshakeEvent if the channel is closed but the handshake was not started.

Modifications:

- Add a variable to SslHandler which tracks if an handshake was started yet or not and depending on this fire the event.
- Add a unit test

Result:

Fixes [#7262].
2017-11-16 19:57:04 +01:00
Scott Mitchell
3648ab0355 Add comments for ApplicationProtocolConfig
Motivation:
The behavior for SelectorFailureBehavior and SelectedListenerFailureBehavior enum values are not clear. Additional comments would clarify the expected behavior.

Modifications:
- Add comments for each value in SelectedListenerFailureBehavior and SelectorFailureBehavior which clarify the expected behavior

Result:
The behavior of SelectedListenerFailureBehavior and SelectorFailureBehavior are more clearly communicated.
2017-11-15 08:01:47 +01:00
Norman Maurer
2adb8bd80f Use Parameterized to run SslHandler tests with different SslProviders.
Motivation:

At the moment use loops to run SslHandler tests with different SslProviders which is error-prone and also make it difficult to understand with which provider these failed.

Modifications:

- Move unit tests that should run with multiple SslProviders to extra class.
- Use junit Parameterized to run with different SslProvider combinations

Result:

Easier to understand which SslProvider produced test failures
2017-11-10 07:20:35 -08:00
Norman Maurer
188ea59c9d [maven-release-plugin] prepare for next development iteration 2017-11-08 22:36:53 +00:00
Norman Maurer
812354cf1f [maven-release-plugin] prepare release netty-4.1.17.Final 2017-11-08 22:36:33 +00:00
Scott Mitchell
8c5eeb581e SslHandler promise completion incorrect if write doesn't immediately
complete

Motivation:
SslHandler removes a Buffer/Promise pair from
AbstractCoalescingBufferQueue when wrapping data. However it is possible
the SSLEngine will not consume the entire buffer. In this case
SslHandler adds the Buffer back to the queue, but doesn't add the
Promise back to the queue. This may result in the promise completing
immediately in finishFlush, and generally not correlating to the
completion of writing the corresponding Buffer

Modifications:
- AbstractCoalescingBufferQueue#addFirst should also support adding the
ChannelPromise
- In the event of a handshake timeout we should immediately fail pending
writes immediately to get a more accurate exception

Result:
Fixes https://github.com/netty/netty/issues/7378.
2017-11-07 09:24:40 -08:00
Scott Mitchell
570d96d8c2 SslHandler leak
Motivation:
SslHandler only supports ByteBuf objects, but will not release objects of other types. SslHandler will also not release objects if its internal state is not correctly setup.

Modifications:
- Release non-ByteBuf objects in write
- Release all objects if the SslHandler queue is not setup

Result:
Less leaks in SslHandler.
2017-11-06 15:41:42 -08:00
Scott Mitchell
35b0cd58fb HTTP/2 write of released buffer should not write and should fail the promise
Motivation:
HTTP/2 allows writes of 0 length data frames. However in some cases EMPTY_BUFFER is used instead of the actual buffer that was written. This may mask writes of released buffers or otherwise invalid buffer objects. It is also possible that if the buffer is invalid AbstractCoalescingBufferQueue will not release the aggregated buffer nor fail the associated promise.

Modifications:
- DefaultHttp2FrameCodec should take care to fail the promise, even if releasing the data throws
- AbstractCoalescingBufferQueue should release any aggregated data and fail the associated promise if something goes wrong during aggregation

Result:
More correct handling of invalid buffers in HTTP/2 code.
2017-11-06 14:38:58 -08:00
Scott Mitchell
fbe0e3506e OpenSslEngine support unwrap plaintext greater than 2^14 and avoid
infinite loop

Motivation:
If SslHandler sets jdkCompatibilityMode to false and ReferenceCountedOpenSslEngine sets jdkCompatibilityMode to true there is a chance we will get stuck in an infinite loop if the peer sends a TLS packet with length greater than 2^14 (the maximum length allowed in the TLS 1.2 RFC [1]). However there are legacy implementations which actually send larger TLS payloads than 2^14 (e.g. OpenJDK's SSLSessionImpl [2]) and in this case ReferenceCountedOpenSslEngine will return BUFFER_OVERFLOW in an attempt to notify that a larger buffer is to be used, but if the buffer is already at max size this process will repeat indefinitely.

[1] https://tools.ietf.org/html/rfc5246#section-6.2.1
[2] http://hg.openjdk.java.net/jdk8u/jdk8u/jdk/file/d5a00b1e8f78/src/share/classes/sun/security/ssl/SSLSessionImpl.java#l793

Modifications:
- Support TLS payload sizes greater than 2^14 in ReferenceCountedOpenSslEngine
- ReferenceCountedOpenSslEngine should throw an exception if a
BUFFER_OVERFLOW is impossible to rectify

Result:
No more infinite loop in ReferenceCountedOpenSslEngine due to
BUFFER_OVERFLOW and large TLS payload lengths.
2017-11-02 11:42:38 -07:00
Norman Maurer
7321418eb5 Fix possible NPE in ReferenceCountedOpenSslEngine.rejectRemoteInitiatedRenegotiation()
Motivation:

ReferenceCountedOpenSslEngine.rejectRemoteInitiatedRenegotiation() is called in a finally block to ensure we always check for renegotiation. The problem here is that sometimes we will already shutdown the engine before we call the method which will lead to an NPE in this case as the ssl pointer was already destroyed.

Modifications:

Check that the engine is not destroyed yet before calling SSL.getHandshakeCount(...)

Result:

Fixes [#7353].
2017-11-02 14:28:59 +01:00
Scott Mitchell
fa584c146f SslHandler dervies jdkCompatibilityMode from SSLEngine
Motivation:
Some SSLEngine implementations (e.g. ReferenceCountedOpenSslContext) support unwrapping/wrapping multiple packets at a time. The SslHandler behaves differently if the SSLEngine supports this feature, but currently requires that the constructor argument between the SSLEngine creation and SslHandler are coordinated. This can be difficult, or require package private access, if extending the SslHandler.

Modifications:
- The SslHandler should inspect the SSLEngine to see if it supports jdkCompatibilityMode instead of relying on getting an extra constructor argument which maybe out of synch with the SSLEngine

Result:
Easier to override SslHandler and have consistent jdkCompatibilityMode between SSLEngine and SslHandler.
2017-11-01 12:00:51 -07:00
Norman Maurer
ad1f0d46b3 Take the architecture into account when loading netty-tcnative
Motivation:

We should ensure we only try to load the netty-tcnative version that was compiled for the architecture we are using.

Modifications:

Include architecture into native lib name.

Result:

Only load native lib if the architecture is supported.
2017-11-01 08:52:09 +01:00
Norman Maurer
92b786e2f3 Fix possible leak in SslHandler if wrap(...) throws.
Motivation:

We can end up with a buffer leak if SSLEngine.wrap(...) throws.

Modifications:

Correctly release the ByteBuf if SSLEngine.wrap(...) throws.

Result:

Fixes [#7337].
2017-10-24 18:52:20 +02:00
Norman Maurer
55b501d0d4 Correctly update Channel writability when queueing data in SslHandler.
Motivation:

A regression was introduced in 86e653e which had the effect that the writability was not updated for a Channel while queueing data in the SslHandler.

Modifications:

- Factor out code that will increment / decrement pending bytes and use it in AbstractCoalescingBufferQueue and PendingWriteQueue
- Add test-case

Result:

Channel writability changes are triggered again.
2017-10-24 09:13:15 +02:00
Nikolay Fedorovskikh
dcc39e5b21 Fixes a LoggingHandler#format method with two arguments
Motivation:
Bug in capacity calculation: occurs auto convert to string instead of sum up.

Modifications:
Use `eventName.length()` in sum.

Result:
Less trash in logs.
2017-10-24 06:56:54 +02:00
Norman Maurer
521e87984d SslHandler.setHandshakeTimeout*(...) should also been enforced on the server side.
Motivation:

We should also enforce the handshake timeout on the server-side to allow closing connections which will not finish the handshake in an expected amount of time.

Modifications:

- Enforce the timeout on the server and client side
- Add unit test.

Result:

Fixes [#7230].
2017-10-23 19:33:07 +02:00
Idel Pivnitskiy
50a067a8f7 Make methods 'static' where it possible
Motivation:

Even if it's a super micro-optimization (most JVM could optimize such
 cases in runtime), in theory (and according to some perf tests) it
 may help a bit. It also makes a code more clear and allows you to
 access such methods in the test scope directly, without instance of
 the class.

Modifications:

Add 'static' modifier for all methods, where it possible. Mostly in
test scope.

Result:

Cleaner code with proper 'static' modifiers.
2017-10-21 14:59:26 +02:00
Idel Pivnitskiy
558097449c Add missed 'serialVersionUID' field for Serializable classes
Motivation:

Without a 'serialVersionUID' field, any change to a class will make
previously serialized versions unreadable.

Modifications:

Add missed 'serialVersionUID' field for all Serializable
classes.

Result:

Proper deserialization of previously serialized objects.
2017-10-21 14:41:18 +02:00