Commit Graph

842 Commits

Author SHA1 Message Date
Carl Mastrangelo
83de77fbe5 Make Native loading work better with Java 8
Motivation:
Enable static linking for Java 8.  These commits are the same as those introduced to netty tcnative.  The goal is to allow lots of JNI libraries to be statically linked together without having conflict `JNI_OnLoad` methods.

Modification:
* add JNI_OnLoad suffixes to enable static linking
* Add static names to the list of libraries that try to be loaded
* Enable compiling with JNI 1.8
* Sort includes

Result:
Enable statically linked JNI code.
2017-06-23 19:42:13 +02:00
Nikolay Fedorovskikh
01eb428b39 Move methods for decode hex dump into StringUtil
Motivation:

PR #6811 introduced a public utility methods to decode hex dump and its parts, but they are not visible from netty-common.

Modifications:

1. Move the `decodeHexByte`, `decodeHexDump` and `decodeHexNibble` methods into `StringUtils`.
2. Apply these methods where applicable.
3. Remove similar methods from other locations (e.g. `HpackHex` test class).

Result:

Less code duplication.
2017-06-23 18:52:42 +02:00
Carl Mastrangelo
cbde38f6e9 Add cause to thrown exception in SelfSignedCert
Motivation:
Exceptions with causes are easier to debug

Modification:
Add the cause when generating a SelfSignedCert

Results:
More debugging context
2017-06-23 07:24:11 +02:00
Scott Mitchell
6d029ad3ac OpenSSL CHACHA20 CipherSuiteConverter updates
Motivation:
For historical reasons OpenSSL's internal naming convention for CHACHA20 based cipher suites does not include the HMAC algorithm in the cipher name. This will prevent the CHACHA20 cipher suites from being used if the RFC cipher names are specified.

Modifications:
- Add a special case for CHACHA20 cipher name conversions in CipherSuiteConverter
- Update OPENSSL_CIPHERSUITE_PATTERN to accommodate the new naming scheme for CHACHA20 cipher suites

Result:
CipherSuiteConverter now works with CHACHA20 cipher suites.
2017-06-21 06:50:43 +02:00
Roger Kapsi
7460d90a67 Add listener to returned Future rather than passed in Promise
Motivation

It's cleaner to add listeners to returned Futures rather than provided Promises because the latter can have strange side effects in terms of listeners firing and called methods returning. Adding listeners preemtively may yield also to more OPS than necessary when there's an Exception in the to be called method.

Modifications

Add listener to returned ChannelFuture rather than given ChannelPromise

Result

Cleaner completion and exception handling
2017-06-16 13:21:19 +01:00
Norman Maurer
e597756a56 Remove synchronized (ReferenceCountedOpenSslContext.class) blocks
Motivation:

We had some useless synchronized (ReferenceCountedOpenSslContext.class) blocks in our code which could slow down concurrent collecting and creating of ReferenceCountedOpenSslContext instances. Beside this we missed a few guards.

Modifications:

Use ReadWriteLock to correctly guard. A ReadWriteLock was choosen as SSL.newSSL(...) will be called from multiple threads all the time so using synchronized would be worse and there would be no way for the JIT to optimize it away

Result:

Faster concurrent creating and collecting of ReferenceCountedOpenSslContext instances and correctly guard in all cases.
2017-06-15 06:29:14 +02:00
Norman Maurer
fd67a2354d [maven-release-plugin] prepare for next development iteration 2017-06-08 21:06:24 +02:00
Norman Maurer
3acd5c68ea [maven-release-plugin] prepare release netty-4.1.12.Final 2017-06-08 21:06:01 +02:00
Scott Mitchell
24f801c7d1 OpenSslEngine return NEED_WRAP if the destination buffered filled
Motivation:
If the destination buffer is completely filled during a call to OpenSslEngine#wrap(..) we may return NEED_UNWRAP because there is no data pending in the SSL buffers. However during a handshake if the SSL buffers were just drained, and filled up the destination buffer it is possible OpenSSL may produce more data on the next call to SSL_write. This means we should keep trying to call SSL_write as long as the destination buffer is filled and only return NEED_UNWRAP when the destination buffer is not full and there is no data pending in OpenSSL's buffers.

Modifications:
- If the handshake produces data in OpenSslEngine#wrap(..) we should return NEED_WRAP if the destination buffer is completely filled

Result:
OpenSslEngine returns the correct handshake status from wrap().
Fixes https://github.com/netty/netty/issues/6796.
2017-06-02 07:47:35 -07:00
Roger Kapsi
2db4f2557d The SNI extension value is ASCII encoded but Netty uses UTF-8.
Motivation

RFC 6066 (https://tools.ietf.org/html/rfc6066#page-6) says that the hostname in the SNI extension is ASCII encoded but Netty decodes it using UTF-8.

Modifications

Use ASCII instead of UTF-8

Result

Fixes #6717
2017-05-19 15:12:06 -07:00
Roger Kapsi
915bf5f5b7 SslHandler#handlerRemoved0() shouldn't care about the SSLEngine being a specific type but only if it's ReferenceCounted
Motivation

SslHandler should release any type of SSLEngine if it implements the ReferenceCounted interface

Modifications

Change condition to check for ReferenceCounted interface

Result

Better use of interfaces
2017-05-19 19:31:51 +02:00
Norman Maurer
732948b3c4 Ensure SslUtils and so SslHandler works when using with Little-Endian buffers.
Motivation:

We not correctly handle LE buffers when try to read the packet length out of the buffer and just assume it always is a BE buffer.

Modifications:

Correctly account for the endianess of the buffer when reading the packet lenght.

Result:

Fixes [#6709].
2017-05-18 07:08:37 +02:00
Scott Mitchell
f20063d26b SslHandler#wrapNonAppData return early
Motivation:

SslHandler#wrapNonAppData may be able to return early if it is called from a unwrap method and the status is NEED_UNWRAP. This has been observed to occur while using the OpenSslEngine and can avoid allocation of an extra ByteBuf of size 2048.

Modifications:
- Return early from SslHandler#wrapNonAppData if NEED_UNWRAP and we are called from an unwrap method

Result:
Less buffer allocations and early return from SslHandler#wrapNonAppData.
2017-05-17 13:53:42 -07:00
Norman Maurer
0db2901f4d [maven-release-plugin] prepare for next development iteration 2017-05-11 16:00:55 +02:00
Norman Maurer
f7a19d330c [maven-release-plugin] prepare release netty-4.1.11.Final 2017-05-11 16:00:16 +02:00
Norman Maurer
dd837fe803 Remove some dead-code and cleanup 2017-05-10 20:48:47 +02:00
Norman Maurer
c053c5144d Correctly detect if Ocsp is supported
Motivation:

We only used the openssl version to detect if Ocsp is supported or not which is not good enough as even the version is correct it may be compiled without support for OCSP (like for example on ubuntu).

Modifications:

Try to enable OCSP while static init OpenSsl and based on if this works return true or false when calling OpenSsl.isOcspSupported().

Result:

Correctly detect if OSCP is supported.
2017-05-10 08:42:28 +02:00
Norman Maurer
ec935c5a7b Correctly delete SelfSignedCertificate once done with it.
Motivation:

In OpenSsl init code we create a SelfSignedCertificate which we not explicitly delete. This can lead to have the deletion delayed.

Modifications:

Delete the SelfSignedCertificate once done with it.

Result:

Fixes [#6716]
2017-05-09 11:28:20 +02:00
Scott Mitchell
141089998f OpenSslEngine wrap may generate bad data if multiple src buffers
Motivation:
SSL_write requires a fixed amount of bytes for overhead related to the encryption process for each call. OpenSslEngine#wrap(..) will attempt to encrypt multiple input buffers until MAX_PLAINTEXT_LENGTH are consumed, but the size estimation provided by calculateOutNetBufSize may not leave enough room for each call to SSL_write. If SSL_write is not able to completely write results to the destination buffer it will keep state and attempt to write it later. Netty doesn't account for SSL_write keeping state and assumes all writes will complete synchronously (by attempting to allocate enough space to account for the overhead) and feeds the same data to SSL_write again later which results in corrupted data being generated.

Modifications:
- OpenSslEngine#wrap should only produce a single TLS packet according to the SSLEngine API specificaiton [1].
[1] https://docs.oracle.com/javase/8/docs/api/javax/net/ssl/SSLEngine.html#wrap-java.nio.ByteBuffer:A-int-int-java.nio.ByteBuffer-
- OpenSslEngine#wrap should only consider a single buffer when determining if there is enough space to write, because only a single buffer will ever be consumed.

Result:
OpenSslEngine#wrap will no longer produce corrupted data due to incorrect accounting of space required in the destination buffers.
2017-05-08 14:43:36 -07:00
Norman Maurer
aab89b058e Ensure Netty is usable on Java7
Motivation:

When adding SNIMatcher support we missed to use static delegating methods and so may try to load classes that not exists in Java7. Which will lead to errors.

Modifications:

- Correctly only try to load classes when running on java8+
- Ensure Java8+ related tests only run when using java8+

Result:

Fixes [#6700]
2017-05-04 14:10:53 -07:00
Norman Maurer
6915ec3bb9 [maven-release-plugin] prepare for next development iteration 2017-04-29 14:10:00 +02:00
Norman Maurer
f30f242fee [maven-release-plugin] prepare release netty-4.1.10.Final 2017-04-29 14:09:32 +02:00
Norman Maurer
3ba4da8a79 Mark conscrypt as optional
Motivation:

Conscrypt is not needed when using the handler module, so it should be marked as optional

Modifications:

Mark conscrypt as optional

Result:

Be able to use handler module without conscrypt
2017-04-28 17:36:31 +02:00
Roger Kapsi
57d3393527 Ability to extend SniHandler and configure it with arbitrary runtime data
Motivation

SniHandler is "hardcoded" to use hostname -> SslContext mappings but there are use-cases where it's desireable and necessary to return more information than a SslContext. The only option so far has been to use a delegation pattern

Modifications

Extract parts of the existing SniHandler into an abstract base class and extend SniHandler from it. Users can do the same by extending the new abstract base class and implement custom behavior that is possibly very different from the common/default SniHandler.

Touches

- f97866dbc6
- b604a22395

Result

Fixes #6603
2017-04-26 19:05:16 -07:00
Norman Maurer
7f3b75a509 Revert "SslHandler avoid calling wrap/unwrap when unnecessary"
This reverts commit 6353c229fd to "fix" [#6578].
2017-04-23 21:08:03 +02:00
Dmitriy Dumanskiy
e78ccd6d52 #6657 do not throw ClassCastException when rule subnet version doesn't match remote IP version 2017-04-23 08:30:02 +02:00
Norman Maurer
7214740c06 Add support for SNIMatcher when using SslProvider.OPENSSL* and Java8+
Motivation:

Java8 adds support for SNIMatcher to reject SNI when the hostname not matches what is expected. We not supported doing this when using SslProvider.OPENSSL*.

Modifications:

- Add support for SNIMatcher when using SslProvider.OPENSSL*
- Add unit tests

Result:

SNIMatcher now support with our own SSLEngine as well.
2017-04-21 10:23:47 +02:00
Norman Maurer
4dd6c14ba2 Only use test SslProviders that are supported in SslHandlerTest.testCompositeBufSizeEstimationGuaranteesSynchronousWrite().
Motivation:

We need to ensure we only try to to test with the SslProviders that are supported when running the SslHandlerTest.testCompositeBufSizeEstimationGuaranteesSynchronousWrite test.

Modifications:

Skip SslProvider.OPENSSL* if not supported.

Result:

No more test-failures if openssl is not installed on the system.
2017-04-20 19:29:52 +02:00
Nikolay Fedorovskikh
0692bf1b6a fix the typos 2017-04-20 04:56:09 +02:00
Norman Maurer
1dfd852dff Revert "Add support for SNIMatcher when using SslProvider.OPENSSL* and Java8+"
This reverts commit cc5d1d0a7e.
2017-04-18 13:43:03 +02:00
Norman Maurer
cc5d1d0a7e Add support for SNIMatcher when using SslProvider.OPENSSL* and Java8+
Motivation:

Java8 adds support for SNIMatcher to reject SNI when the hostname not matches what is expected. We not supported doing this when using SslProvider.OPENSSL*.

Modifications:

- Add support for SNIMatcher when using SslProvider.OPENSSL*
- Add unit tests

Result:

SNIMatcher now support with our own SSLEngine as well.
2017-04-18 08:16:33 +02:00
Roger Kapsi
077a1988b9 OCSP stapling support for Netty using netty-tcnative.
https://github.com/netty/netty-tcnative/pull/215

Motivation

OCSP stapling (formally known as TLS Certificate Status Request extension) is alternative approach for checking the revocation status of X.509 Certificates. Servers can preemptively fetch the OCSP response from the CA's responder, cache it for some period of time, and pass it along during (a.k.a. staple) the TLS handshake. The client no longer has to reach out on its own to the CA to check the validity of a cetitficate. Some of the key benefits are:

1) Speed. The client doesn't have to crosscheck the certificate.
2) Efficiency. The Internet is no longer DDoS'ing the CA's OCSP responder servers.
3) Safety. Less operational dependence on the CA. Certificate owners can sustain short CA outages.
4) Privacy. The CA can lo longer track the users of a certificate.

https://en.wikipedia.org/wiki/OCSP_stapling
https://letsencrypt.org/2016/10/24/squarespace-ocsp-impl.html

Modifications

https://www.openssl.org/docs/man1.0.2/ssl/SSL_set_tlsext_status_type.html

Result

High-level API to enable OCSP stapling
2017-04-03 11:56:53 -07:00
Kevin Oliver
34e0007f07 LoggingHandler does not override channelReadComplete or channelWritabilityChanged
Motivation:

`io.netty.handler.logging.LoggingHandler` does not log when these
events happen.

Modifiations:

Add overrides with logging to these methods.

Result:

Logging now happens for these two events.
2017-04-03 11:46:01 -07:00
Norman Maurer
4bcfa07a7d Fix OpenSslCertificateException error code validation
Motivation:

In OpenSslCertificateException we tried to validate the supplied error code but did not correctly account for all different valid error codes and so threw an IllegalArgumentException.

Modifications:

- Fix validation by updating to latest netty-tcnative and use CertificateVerifier.isValid
- Add unit tests

Result:

Validation of error code works as expected.
2017-04-03 11:12:15 -07:00
Norman Maurer
5163869439 Only try to load conscrypt class in tests when supported.
Motivation:

1419f5b601 added support for conscrypt but the CI started to fail when running tests with java7 as conscrypt is compiled with java8. This was partly fixed in c4832cd9d9 but we also need to ensure we not try to even load the classes.

Modifications:

Only try to load conscrypt classes when on java8+-

Result:

CI not fails anymore.
2017-04-01 22:51:51 +02:00
Norman Maurer
c4832cd9d9 Only support using Conscrypt on Java8+
Motivation:

1419f5b601 added support for conscrypt but the CI started to fail when running tests with java7 as conscrypt is compiled with java8.

Modifications:

Only support conscrypt on Java8+

Result:

CI not fails anymore.
2017-04-01 20:38:33 +02:00
Nathan Mittler
1419f5b601 Adding support for Conscrypt (#6271)
Motivation:

Conscrypt is a Java Security provider that wraps OpenSSL (specifically BoringSSL). It's a possible alternative to Netty-tcnative that we should explore. So this commit is just to enable us to further investigate its use.

Modifications:

Modifying the SslContext creation path to support the Conscrypt provider.

Result:

Netty will support OpenSSL with conscrypt.
2017-03-31 13:55:59 -07:00
Norman Maurer
ed1071d327 Limit the maximum size of the allocated outbound buffer to MAX_ENCRYPTED_PACKET_LENGTH
Motivation:

We should limit the size of the allocated outbound buffer to MAX_ENCRYPTED_PACKET_LENGTH to ensure we not cause an OOME when the user tries to encrypt a very big buffer.

Modifications:

Limit the size of the allocated outbound buffer to MAX_ENCRYPTED_PACKET_LENGTH

Result:

Fixes [#6564]
2017-03-31 07:53:50 +02:00
R Kaja Mohideen
13cd69c5ec Recognizing TLS Extension "HeartBeat" as Valid TLS Packet
Motivation:

The widely used SSL Implementation, OpenSSL, already supports Heartbeat Extension; both sending and responding to Heartbeat Messages. But, since Netty is not recognizing that extension as valid packet, peers won't be able to use this extension.

Modification:

Update SslUtils.java to recognize Heartbeat Extension as valid tls packet.

Result:

With this change, softwares using Netty + OpenSSL will be able to respond for TLS Heartbeat requests (actually taken care by OpenSSL - no need of any extra implementation from Clients)
2017-03-28 11:43:26 +02:00
Michael O'Brien
61f53c4d07 ChunkedWriteHandler flushes too often
Motivation:

ChunkedWriteHandler queues written messages and actually writes them
when flush is called. In its doFlush method, it needs to flush after
each chunk is written to preserve memory. However, non-chunked messages
(those that aren't of type ChunkedInput) are treated in the same way,
which means that flush is called after each message is written.

Modifications:

Moved the call to flush() inside the if block that tests if the message
is an instance of ChunkedInput. To ensure flush is called at least once,
the existing boolean flushed is checked at the end of doFlush. This
check was previously in ChunkedWriteHandler.flush(), but wasn't checked in
other invocations of doFlush, e.g. in channelInactive.

Result:

When this handler is present in a pipeline, writing a series
of non-chunked messages will be flushed as the developer intended.
2017-03-18 08:05:59 -07:00
Norman Maurer
2b8c8e0805 [maven-release-plugin] prepare for next development iteration 2017-03-10 07:46:17 +01:00
Norman Maurer
1db58ea980 [maven-release-plugin] prepare release netty-4.1.9.Final 2017-03-10 07:45:28 +01:00
Johno Crawford
cfebaa36c0 Support for handling SSL and non-SSL in pipeline
Motivation:

Some pipelines require support for both SSL and non-SSL messaging.

Modifications:

Add utility decoder to support both SSL and non-SSL handlers based on the initial message.

Result:

Less boilerplate code to write for developers.
2017-03-09 01:00:43 -08:00
Scott Mitchell
10d9f82f14 Remove duplicate call to SSLContext.setVerify from ReferenceCountedOpenSslServerContext
Motivation:
5e64985089 introduced support for the KeyManagerFactory while using OpenSSL. This same commit also introduced 2 calls to SSLContext.setVerify when 1 should be sufficient.

Modifications:
- Remove the duplicate call to SSLContext.setVerify

Result:
Less duplicate code in ReferenceCountedOpenSslServerContext.
2017-03-08 13:56:12 -08:00
Scott Mitchell
a2304287a1 SslContext to support TLS/SSL protocols
Motivation:
SslContext and SslContextBuilder do not support a way to specify the desired TLS protocols. This currently requires that the user extracts the SSLEngine once a context is built and manually call SSLEngine#setEnabledProtocols(String[]). Something this critical should be supported at the SslContext level.

Modifications:
- SslContextBuilder should accept a list of protocols to configure for each SslEngine

Result:
SslContext consistently sets the supported TLS/SSL protocols.
2017-03-08 09:24:59 -08:00
Scott Mitchell
6bb661302f OpenSsl tests incomplete check for supporting key manager
Motivaiton:
It is possible that if the OpenSSL library supports the interfaces required to use the KeyManagerFactory, but we fail to get the io.netty.handler.ssl.openssl.useKeyManagerFactory system property (or this property is set to false) that SSLEngineTest based unit tests which use a KeyManagerFactory will fail.

Modifications:
- We should check if the OpenSSL library supports the KeyManagerFactory interfaces and if the system property allows them to be used in OpenSslEngineTests

Result:
Unit tests which use OpenSSL and KeyManagerFactory will be skipped instead of failing.
2017-03-07 08:24:06 +01:00
Scott Mitchell
53fc693901 SslHandler and OpenSslEngine miscalculation of wrap destination buffer size
Motivation:
When we do a wrap operation we calculate the maximum size of the destination buffer ahead of time, and return a BUFFER_OVERFLOW exception if the destination buffer is not big enough. However if there is a CompositeByteBuf the wrap operation may consist of multiple ByteBuffers and each incurs its own overhead during the encryption. We currently don't account for the overhead required for encryption if there are multiple ByteBuffers and we assume the overhead will only apply once to the entire input size. If there is not enough room to write an entire encrypted packed into the BIO SSL_write will return -1 despite having actually written content to the BIO. We then attempt to retry the write with a bigger buffer, but because SSL_write is stateful the remaining bytes from the previous operation are put into the BIO. This results in sending the second half of the encrypted data being sent to the peer which is not of proper format and the peer will be confused and ultimately not get the expected data (which may result in a fatal error). In this case because SSL_write returns -1 we have no way to know how many bytes were actually consumed and so the best we can do is ensure that we always allocate a destination buffer with enough space so we are guaranteed to complete the write operation synchronously.

Modifications:
- SslHandler#allocateNetBuf should take into account how many ByteBuffers will be wrapped and apply the encryption overhead for each
- Include the TLS header length in the overhead computation

Result:
Fixes https://github.com/netty/netty/issues/6481
2017-03-06 08:15:13 -08:00
Scott Mitchell
2cff918044 Correct usages of internalNioBuffer
Motivation:
There are numerous usages of internalNioBuffer which hard code 0 for the index when the intention was to use the readerIndex().

Modifications:
- Remove hard coded 0 for the index and use readerIndex()

Result:
We are less susceptible to using the wrong index, and don't make assumptions about the ByteBufAllocator.
2017-03-02 12:51:22 -08:00
Scott Mitchell
1f6782894a OpenSslEngine wrap with heap buffer bug
Motivation:
ReferenceCountedOpenSslEngine#wrap must have a direct buffer for a destination to interact with JNI. If the user doesn't supply a direct buffer we internally allocate one to write the results of wrap into. After this operation completes we copy the contents of the direct buffer into the heap buffer and use internalNioBuffer to get the content. However we pass in the end index but the internalNioBuffer expects a length.

Modifications:
- pass the length instead of end index to internalNioBuffer

Result:
ReferenceCountedOpenSslEngine#wrap will copy the correct amount of data into the destination buffer when heap buffers are wrapped.
2017-03-02 12:49:35 -08:00
Norman Maurer
7e7e10fb1e Correct SslContextBuilder javadocs
Motivation:

SslContextBuilder sill state the KeyManagerFactory and TrustManagerFactory are only supported when SslProvider.JDK is used. This is not correct anymore.

Modifications:

Fix javadocs.

Result:

Correct javadocs.
2017-03-02 09:36:45 +01:00