Commit Graph

1249 Commits

Author SHA1 Message Date
Norman Maurer
a2ebd65322 Fix compilation error on JDK 15 (#10462)
Motivation:

AlgorithmId.sha256WithRSAEncryption_oid was removed in JDK15 and later so we should not depend on it as otherwise we will see compilation errors

Modifications:

Replace AlgorithmId.sha256WithRSAEncryption_oid usage with specify the OID directly

Result:

Compiles on JDK15+
2020-08-10 14:03:43 +02:00
skyguard1
9c5dbfd1b6 Replace Class.getClassLoader with io.netty.util.internal.PlatformDependent.getClassLoader in Openssl (#10454)
Motivation:

Replace Class.getClassLoader with io.netty.util.internal.PlatformDependent.getClassLoader in Openssl so it also works when a SecurityManager is in place

Modification:

Replace Class.getClassLoader with io.netty.util.internal.PlatformDependent.getClassLoader in Openssl

Result:

No issues when a SecurityManager is in place
2020-08-06 09:03:01 +02:00
Norman Maurer
220995f155 Make the TLSv1.3 check more robust and not depend on the Java version… (#10409)
Motivation:

TLSv1.3 is not strictly limited to Java11+ anymore as different vendors backported TLSv1.3 to Java8 as well. We should ensure we make the detection of if TLSv1.3 is supported not depend on the Java version that is used.

Modifications:

- Add SslProvider.isTlsv13Supported(...) and use it in tests to detect if we should run tests against TLSv1.3 as well
- Adjust testcase to work on latest JDK 8 release as well

Result:

Correct detection of TLSv1.3 support even if Java version < 11.
2020-07-17 07:17:56 +02:00
Norman Maurer
7bf1ffb2d4 Support session cache for client and server when using native SSLEngine implementation (#10331)
Motivation:

At the moment we don't support session caching for client side when using native SSLEngine implementation and our implementation of SSLSessionContext is incomplete.

Modification:

- Consume netty-tcnative changes to be able to cache session in an external cache
- Add and adjust unit tests to test session caching
- Add an in memory session cache that is hooked into native SSLEngine

Result:

Support session caching on the client and server side
2020-07-14 15:20:44 +02:00
Norman Maurer
47bbc590ee jdk.tls.client.enableSessionTicketExtension must be respected by OPENSSL and OPENSSL_REFCNT SslProviders (#10401)
Motivation:

jdk.tls.client.enableSessionTicketExtension property must be respect by OPENSSL and OPENSSL_REFCNT SslProvider to ensure a consistent behavior. Due a bug this was not the case and it only worked for OPENSSL_REFCNT but not for OPENSSL.

Modifications:

Move the property check into static method that is used by both

Result:

Correctly respect jdk.tls.client.enableSessionTicketExtension
2020-07-13 16:17:41 +02:00
Norman Maurer
165ade5d9f Correctly return NEED_WRAP if we produced some data even if we could not consume any during SSLEngine.wrap(...) (#10396)
Motivation:

At the moment we may report BUFFER_OVERFLOW when wrap(...) fails to consume data but still prodce some. This is not correct and we should better report NEED_WRAP as we already have produced some data (for example tickets). This way the user will try again without increasing the buffer size which is more correct and may reduce the number of allocations

Modifications:

Return NEED_WRAP when we produced some data but not consumed any.

Result:

Fix ReferenceCountedOpenSslEngine.wrap(...) state machine
2020-07-09 08:52:30 +02:00
Daniel Zou
fca62510f3 Modify OpenSSL native library loading to accommodate GraalVM (#10395)
**Motivation:**

We are interested in building Netty libraries with Ahead-of-time compilation with GraalVM. We saw there was [prior work done on this](https://github.com/netty/netty/search?q=graalvm&unscoped_q=graalvm). We want to introduce a change which will unblock GraalVM support for applications using netty and `netty-tcnative`.

This solves the error [that some others encounter](https://github.com/oracle/graal/search?q=UnsatisfiedLinkError+sslOpCipherServerPreference&type=Issues):

```
Exception in thread "main" java.lang.UnsatisfiedLinkError: io.grpc.netty.shaded.io.netty.internal.tcnative.NativeStaticallyReferencedJniMethods.sslOpCipherServerPreference()I [symbol: Java_io_grpc_netty_shaded_io_netty_internal_tcnative_NativeStaticallyReferencedJniMethods_sslOpCipherServerPreference or Java_io_grpc_netty_shaded_io_netty_internal_tcnative_NativeStaticallyReferencedJniMethods_sslOpCipherServerPreference__]
```

**Modification:**

The root cause of the issue is that in the tcnative libraries, the [`SSL.java` class](783a8b6b69/openssl-dynamic/src/main/java/io/netty/internal/tcnative/SSL.java (L67)) references a native call in the static initialization of the class - the method `sslOpCipherServerPreference()` on line 67 is used to initialize the static variable. But you see that `OpenSsl` also uses[ `SSL.class` to check if `netty-tcnative` is on the classpath before loading the native library](cbe238a95b/handler/src/main/java/io/netty/handler/ssl/OpenSsl.java (L123)). 

So this is the problem because in ahead-of-time compilation, when it references the SSL class, it will try to load those static initializers and make the native library call, but it cannot do so because the native library has not been loaded yet since the `SSL` class is being referenced to check if the library should be loaded in the first place.

**Solution:** So the proposed solution here is to just choose a different class in the `tcnative` package which does not make a native library call during static initialization. I just chose `SSLContext` if this is OK.

This should have no negative effects other than unblocking the GraalVM use-case.

**Result:**

It fixes the unsatisfied link error. It will fix error for future users; it is a common error people encounter.
2020-07-08 10:47:10 +02:00
Norman Maurer
0c79863db5 Update to netty-tcnative 2.0.31.Final and make SslErrorTest more robust (#10392)
Motivation:

There was a new netty-tcnative release which we should use. Beside this the SSLErrorTest was quite fragile and so should be adjusted.

Modifications:

Update netty-tcnative and adjust test

Result:

Use latest netty-tcnative release
2020-07-07 10:56:42 +02:00
Norman Maurer
8950144567 Correctly include TLS1.3 ciphers in the enabled ciphersuites when using BoringSSL (#10388)
Motivation:

BoringSSL behaves differently then OpenSSL and not include any TLS1.3 ciphers in the returned array when calling SSL_get_ciphers(...). This is due the fact that it also not allow to explicit configure which are supported and which not for TLS1.3. To mimic the behaviour expected by the SSLEngine API we should workaround this.

Modifications:

- Add a unit test that verifies enabled protocols / ciphers
- Add special handling for BoringSSL and tls1.3

Result:

Make behaviour consistent
2020-07-02 21:37:04 +02:00
Norman Maurer
91c75a4a8f Ensure we preserve the original cause of the SSLException in all case… (#10372)
Motivation:

We did not correctly preserve the original cause of the SSLException when all the following are true:

 * SSL_read failed
 * handshake was finished
 * some outbound data was produced durigin SSL_read (for example ssl alert) that needs to be picked up by wrap(...)

Modifications:

Ensure we correctly preserve the original cause of the SSLException and rethrow it once we produced all data via wrap(...)

Result:

Be able to understand why we see an error in all cases
2020-06-25 22:08:57 +02:00
Norman Maurer
b2726c4919 X509TrustManager with OPENSSL provider is not wrapped with hostname verification if Conscrypt is inserted in the first place (#10375)
Motivation:

Modifications:

Directly specify the provider which is used to create the SSLContext

Result:

Fixes https://github.com/netty/netty/issues/10374
2020-06-25 20:40:49 +02:00
Norman Maurer
163c2fc220 Ensure we feed all data to the SSLEngine during handshaking in our tests (#10373)
Motivation:

Due a bug in our test we may dropped data on the floor which are generated during handshaking (or slightly after). This could lead to corrupt state in the engine itself and so fail tests. This is especially true for TLS1.3 which generates the sessions on the server after the "actual handshake" is done.

Modifications:

Contine with wrap / unwrap until all data was consumed

Result:

Correctly feed all data to the engine during testing
2020-06-25 14:56:54 +02:00
Norman Maurer
f051b0c297 Ensure ApplicationProtocolNegotiationHandler does handle handshake fa… (#10363)
Motivation:

When ApplicationProtocolNegotiationHandler is in the pipeline we should expect that its handshakeFailure(...) method will be able to completly handle the handshake error. At the moment this is not the case as it only handled SslHandshakeCompletionEvent but not the exceptionCaught(...) that is also triggered in this case

Modifications:

- Call handshakeFailure(...) in exceptionCaught and so fix double notification.
- Add testcases

Result:

Fixes https://github.com/netty/netty/issues/10342
2020-06-24 08:47:31 +02:00
Norman Maurer
e76fc0a577 Fix compilation error in test due bad cherry-pick
Motivation:

I did not correctly adjust the code before cherry-pick, causing a compilation error in the test

Modifications:

Use ChannelHandler

Result:

No more compilation error
2020-06-12 11:21:47 +02:00
Kareem Ali
121daab927 Motivation: (#10326)
The current FLowControlHandler keeps a flag to track whether a read() call is pending.
This could lead to a scenario where you call read multiple times when the queue is empty,
and when the FlowControlHandler Queue starts getting messages, channelRead will be fired only once,
when we should've fired x many times, once for each time the handlers downstream called read().

Modifications:

Minor change to replace the boolean flag with a counter and adding a unit test for this scenario.

Result:

I used TDD, so I wrote the test, made sure it's failing, then updated the code and re-ran the test
to make sure it's successful after the changes.

Co-authored-by: Kareem Ali <kali@localhost.localdomain>
2020-06-04 19:18:23 +02:00
Norman Maurer
feeb3ee920 Update test to directly check for SslHandshakeTimeoutException (#10339)
Motivation:

9b7e091 added a special SSLHandshakeException sub-class to signal handshake timeouts but we missed to update a testcase to directly assert the type of the exception.

Modifications:

Assert directly that SslHandshakeTimeoutException is used

Result:

Test cleanup
2020-06-04 18:30:25 +02:00
feijermu
8bbd89d72a Fix a test case problem: testSwallowedReadComplete(...) may fail with an AssertionError sometimes. (#10313)
Motivation:

It seems that `testSwallowedReadComplete(...)` may fail with an AssertionError sometimes after my tests. The relevant stack trace is as follows:

```
java.lang.AssertionError: expected:<IdleStateEvent(READER_IDLE, first)> but was:<null>
	at org.junit.Assert.fail(Assert.java:88)
	at org.junit.Assert.failNotEquals(Assert.java:834)
	at org.junit.Assert.assertEquals(Assert.java:118)
	at org.junit.Assert.assertEquals(Assert.java:144)
	at io.netty.handler.flow.FlowControlHandlerTest.testSwallowedReadComplete(FlowControlHandlerTest.java:478)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:89)
	at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:41)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:542)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:770)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:464)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:210)

```
Obviously the `readerIdleTime` of `IdleStateHandler` and the thread sleep time before `EmbeddedChannel.runPendingTasks` are both 100ms. And if `userEvents.poll()` happened before `userEvents.add(...)` or no `IdleStateEvent` fired at all, this test case would fail.

Modification:

Sleep for a little more time before running all pending tasks in the `EmbeddedChannel`.

Result:

Fix the problem of small probability of failure.
2020-06-03 09:19:54 +02:00
Scott Mitchell
32a77394a3 SslHandler#wrap to preserve exception if SSLEngine is closed (#10327)
Motivation:
SslHandler currently throws a general SSLException if a wrap attempt
fails due to the SSLEngine being closed. If writes are queued the
failure rational typically requires more investigation to track down the
original failure from a previous event. We may have more informative
rational for the failure and so we should use it.

Modifications:
- SslHandler#wrap to use failure information from the handshake or prior
transport closure if available

Result:
More informative exceptions from SslHandler#wrap if the SSLEngine has
been previously closed.
2020-06-02 09:51:43 +02:00
feijermu
9fe5f6f958 Dequeue all cached messages and destroy the queue instance after removing the FlowControlHandler from channel pipeline. (#10304)
Motivation:

The `FlowControlHandler` may cache the received messages in a queue in order to do the flow control. However, if this handler is manually removed from pipeline during runtime, those cached messages might not be passed to the next channel handler forever.

Modification:

Dequeue all these cached messages and call `ChannelHandlerContext.fireChannelRead(...)` in `handlerRemoved(...)` method.

Result:
Avoid losing the received messages.
2020-05-19 10:05:54 +02:00
Idel Pivnitskiy
1fe92e3077 Do not require BoringSSL for testSessionTicketsWithTLSv12AndNoKey (#10301)
Motivation:

`SslHandlerTest.testSessionTicketsWithTLSv12AndNoKey` does not require
BoringSSL and works with OpenSSL as well.

Modifications:

- Remove assume statement that expected BoringSSL;

Result:

Test works for any implementation of `OPENSSL` provider.
2020-05-18 14:25:40 +02:00
Norman Maurer
51805e3248 Check if SSL pointer was freed before using it in RefereceCountedOpenSslEngine in all cases (#10299)
Motivation:

To ensure we not crash in all cases we should better check that the SSL pointer was not freed before using it.

Modifications:

Add missing `isDestroyed()` checks

Result:

Ensure we not crash due usage of freed pointer.
2020-05-18 09:41:10 +02:00
sanjaypujare
f29b187159 Make ReferenceCountedOpenSslContext.setUseTasks public (#10289)
Motivation:
make the existing setter `ReferenceCountedOpenSslContext.setUseTasks` public

Modification:

Added the `public` qualified and removed the comment "for tests only"

Result:

Fixes #10288
2020-05-15 10:04:26 +02:00
Norman Maurer
fc84c9cc47 Respect jdk.tls.client.enableSessionTicketExtension and jdk.tls.server.enableSessionTicketExtension when using native SSL impl (#10296)
Motivation:

We should respect jdk.tls.client.enableSessionTicketExtension and jdk.tls.server.enableSessionTicketExtension when using the native SSL implementation as well to make the usage of it easier and more consistent. These properties were introduced by JDK13:

https://seanjmullan.org/blog/2019/08/05/jdk13

Modifications:

Check if the properties are set to true and if so enable tickets

Result:

Easier to enable tickets and be more consistent
2020-05-15 10:03:30 +02:00
Norman Maurer
6b0655aecb Allow to have the session tickets automatically managed by the native library
Motivation:

BoringSSL supports to automatically manage the session tickets to be used and so also rotate them etc. This is often prefered by users as it removed some complexity. We should support to make use of this.

Modifications:

- Allow to have setSessionTickets() called without an argument or an empty array
- Add tests

Result:

Easier usage of session tickets
2020-05-14 12:09:49 +02:00
Norman Maurer
11b63f8744 OpenSslSession.getLocalCertificates() and getLocalPrincipal() must r… (#10275)
Motivation:

OpenSslSession.getLocalCertificates() and  getLocalPrincipal() must return null on client side if mTLS is not used as stated in the API documentation. At the moment this is not always the case

Modifications:

- Ensure we only return non-null if mTLS is used
- Add unit tests

Result:

Follow SSLSession API contract
2020-05-13 07:16:56 +02:00
Norman Maurer
f23c33822c Rename testmethods to make these more clear (#10231)
Motivation:

The currently used method names don't make a lot of sense.

Modifications:

Rename to cleanup

Result:

Cleanup
2020-04-29 17:57:59 +02:00
Aayush Atharva
f22993a530 Specify algorithm for key pair in self signed certificate to generate EC or RSA based certificate. (#10223)
Motivation:

EC is better than RSA because of the small key size, efficient and secure which makes it perfect for testing purposes.

Modification:

Added support to specify an algorithm (EC or RSA) in constructors for key pair generation. The default key size is 256-bits as promulgated by NSA Suite B.

Result:
Able to generate a self-signed certificate of EC or RSA.
2020-04-29 16:53:31 +02:00
Norman Maurer
ae95e9c3d6 Update to latest Conscrypt release and add workarounds for bugs (#10211)
Motivation:

We are far behind with the version of Conscrypt we are using during testing. We should ensure we use the latest.

Modifications:

- Update conscrypt dependency
- Ensure we use conscrypt provider in tests
- Add workarounds for conscrypt bugs in testsuite

Result:

Use latest Conscrypt release
2020-04-28 09:50:22 +02:00
Saranya Krishnakumar
391cdcdd77 Add check for DefaultFileRegion to calculate size of msg in AbstractTrafficShapingHandler (#10215)
Motivation:

Currently calculateSize method in AbstractTrafficShapingHandler calculates size for object of type ByteBuf or ByteBufHolder. Adding a check for FileRegion, makes it possible to do traffic shaping for FileRegion objects as well

Modification:

Check if object to be sent is of type FileRegion, if yes calculate the size using its count() method.

Co-authored-by: Dinesh Joshi <dinesh.joshi@apple.com>
2020-04-27 14:50:09 +02:00
Norman Maurer
411ad9d5b6 Update testsuite / pom.xml to be able to build with Java15 (#10210)
Motivation:

We need to make some slightly changes to be able to build on Java15 as some previous deprecated methods now throw UnsupportedOperationException

Modifications:

- Add code to handle UnsupportedOperationException
- Revert previous applied workaround for bug that was fixed in Java15
- Add maven profile

Result:

Be able to build with latest Java15 EA release
2020-04-27 06:33:54 +02:00
Norman Maurer
ffaf9cab65 Clarify exception message when ALPN is not supported (#10155)
Motivation:

We should provide more informations when ALPN is not supported and a user tries to use it.

Modifications:

- Use UnsupportedOperationException

Result:

Easier to debug ALPN problems
2020-04-22 08:07:47 +02:00
Norman Maurer
2f4fa3444e Ensure we support ALPN when using java 8u251 (#10196)
Motivation:

ALPN support was backported to java 8 lately. Ensure we support it if the user uses the latest java 8 release

Modifications:

- Update logic to be able to detect if ALPN is supported out of the box when using Java8
- Update jetty alpn version

Result:

Be able to use ALPN out of the box when using java 8u251
2020-04-21 15:23:28 +02:00
feijermu
fc61dbe188 Release the channel attribute--REOPEN_TASK after removing the TrafficShapingHandler from channel pipeline. (#10177)
Motivation:

The `AbstractTrafficShapingHandler` caches the `ReopenReadTimerTask` instance in the channel attribute. However, if this handler is removed from the channel pipeline, this `ReopenReadTimerTask` instance may not be released.

Modification:

Release the channel attribute `REOPEN_TASK` in `handlerRemoved` method.

Result:

Avoid a channel attribute leak.
2020-04-15 09:07:54 +02:00
Stephane Landelle
43c139c2fa Log protocol version during handshake (#10178)
Motivation:

Only cipher suite is logged during handshake. Picked protocol is interesting too.

Modification:

Log protocol as well.

Result:

More interesting information when debugging TLS handshakes.
2020-04-14 10:49:04 +02:00
Norman Maurer
0368da9e3c SslHandler should fail handshake / close promise and notify pipeline on removal (#10161)
Motivation:

If the SslHandler is removed from the pipeline we also need to ensure we fail the handshake / close promise if it was not notified before as otherwise we may never do so.

Modifications:

- Correctly fail promise and notify pipeline if handshake was not done yet when the SslHandler is removed
- Add unit test

Result:

Fix https://github.com/netty/netty/issues/10158
2020-04-03 09:04:56 +02:00
Norman Maurer
492ecae4cf Update link for JDK14 regression to point to the actual bugreport 2020-04-02 16:01:26 +02:00
Norman Maurer
ff9667b1f9 Add profile to build on JDK 14 (#10148)
Motivation:

JDK 14 was released and need some special settings to be able to build with. Also there seems to be one regression that we need to workaround for now.

Modifications:

- Add maven profile for JDK 14
- Update blockhound version to be able to work on JDK 14
- Add workaround for possible JDK 14 regression

Result:

Be able to build on JDK 14
2020-03-31 16:03:10 +02:00
Norman Maurer
00b4cf9fc9 Don't produce multiple calls to exceptionCaught(...) on SSL handshake failure (#10134)
Motivation:

Before release 4.1.23, there was only ONE call to exceptionCaught method when an ssl handshake failure occurs, now we have two when using the JDK provider.

Modifications:

- Ensure we only propagate one exception fi we already failed the handshake and channelInactive(...) produce an exception again
- Add unit test

Result:

Fixes https://github.com/netty/netty/issues/10119
2020-03-26 09:25:15 +01:00
Norman Maurer
fd0d06ee39
Replace reflection usage with MethodHandles when performance matters (#10097)
Motivation:

As we have java8 as a minimum target we can use MethodHandles. We should do so when we expect to have a method called multiple times.

Modifications:

- Replace usage of reflection with MethodHandles where it makes sense
- Remove some code which was there to support java < 8

Result:

Faster code
2020-03-11 21:04:40 +01:00
Norman Maurer
118e1c66dc Add log level check simply before logging. (#10080)
Motivation:

In general, we will close the debug log in a product environment. However, logging without external level check may still affect performance as varargs will need to allocate an array.

Modification:

Add log level check simply before logging.

Result:

Improve performance slightly in a product environment.
2020-03-05 14:46:22 +01:00
Norman Maurer
ae0fbb45e4
Ensure the DefaultChannelHandlerContext is unlinked once removed (#9970)
Motivation:

At the moment the next / prev references are not set to "null" in the DefaultChannelHandlerContext once the ChannelHandler is removed. This is bad as it basically let users still use the ChannelHandlerContext of a ChannelHandler after it is removed and may produce very suprising behaviour.

Modifications:

- Fail if someone tries to use the ChannelHandlerContext once the ChannelHandler was removed (for outbound operations fail the promise, for inbound fire the error through the ChannelPipeline)
- Fix some handlers to ensure we not use the ChannelHandlerContext after the handler was removed
- Adjust DefaultChannelPipeline / DefaultChannelHandlerContext to fixes races with removal / replacement of handlers

Result:

Cleanup behaviour and make it more predictable for pipeline modifications
2020-03-01 08:13:33 +01:00
Norman Maurer
a1c5eb938c Add SslHandshakeTimeoutException and use it for handshake timeouts (#10062)
Motivation:

Often it is useful to be able to detect different sorts of SSL errors that cause the handshake to fail. To make this easier we should throw and explicit exception type for handshake timeouts.

Modifications:

- Add SslHandshakeTimeoutException (which extends SSLHandshakeException) and use it for handshake timeouts
- Adjust testcases

Result:

Easier to detect that handshake failed because of a timeout
2020-02-27 09:01:06 +01:00
Norman Maurer
1c28cf3a14 Correctly calculate the produced bytes in all cases when calling Refe… (#10063)
Motivation:

We did not correctly account for produced bytes when SSL_write(...) returns -1 in all cases. This could lead to lost data and so a corrupt SSL connection.

Modifications:

- Always ensure we calculate the produced bytes correctly
- Add unit tests

Result:

Fixes https://github.com/netty/netty/issues/10041
2020-02-27 08:55:25 +01:00
Norman Maurer
b2dbd4cedf Don't depend on implementation details of SSLEngine in SniHandlerTest (#10037)
Motivation:

In SniHandlerTest we depended on implementation details of the SSLEngine. We should better not doing this

Modifications:

Just release all outbound data

Result:

Dont depend on implementation details
2020-02-18 15:05:23 +01:00
Norman Maurer
8a343a24e9 Correctly handle lifecycle of clientHello ByteBuf in SslClientHelloHandler (#10030)
Motivation:

Due incorrectly handling of reference count of the clientHello ByteBuf we may overrelease the buffer. This did show up in the log of a test:

11:55:16.595 [main] DEBUG i.n.h.ssl.SslClientHelloHandler - Unexpected client hello packet: 16030100bd010000b90303a74225676d1814ba57faff3b3663656ed05ee9dbb2a4dbb1bb1c32d2ea5fc39e0000000100008c0000001700150000164348415434e380824c45414e434c4f5544e38082434e000b000403000102000a00340032000e000d0019000b000c00180009000a00160017000800060007001400150004000500120013000100020003000f0010001100230000000d0020001e060106020603050105020503040104020403030103020303020102020203000f00010133740000
io.netty.util.IllegalReferenceCountException: refCnt: 0, decrement: 1
	at io.netty.util.internal.ReferenceCountUpdater.toLiveRealRefCnt(ReferenceCountUpdater.java:74)
	at io.netty.util.internal.ReferenceCountUpdater.release(ReferenceCountUpdater.java:138)
	at io.netty.buffer.AbstractReferenceCountedByteBuf.release(AbstractReferenceCountedByteBuf.java:100)
	at io.netty.handler.ssl.SslClientHelloHandler.releaseIfNotNull(SslClientHelloHandler.java:181)
	at io.netty.handler.ssl.SslClientHelloHandler.select(SslClientHelloHandler.java:225)
	at io.netty.handler.ssl.SslClientHelloHandler.decode(SslClientHelloHandler.java:149)
	at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:498)
	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:437)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
	at io.netty.channel.embedded.EmbeddedChannel.writeInbound(EmbeddedChannel.java:343)
	at io.netty.handler.ssl.SniHandlerTest.testNonAsciiServerNameParsing(SniHandlerTest.java:297)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.ExpectException.evaluate(ExpectException.java:19)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.junit.runners.Suite.runChild(Suite.java:128)
	at org.junit.runners.Suite.runChild(Suite.java:27)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
	at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)

Modifications:

Correctly transfer lifetime of buffer and so not over-release it.

Result:

Correctly handle buffer lifecycle and so not swallow the original exception
2020-02-15 18:28:01 +01:00
Norman Maurer
8304069a30 Add SslClientHelloHandler which allows to do something based on the S… (#9827)
Motivation:

Sometimes it is useful to do something depending on the Ssl ClientHello (like for example select a SslContext to use). At the moment we only allow to hook into the SNI extension but this is not enough.

Modifications:

Add SslClientHelloHandler which allows to hook into ClientHello messages. This class is now also the super class of AbstractSniHandler

Result:

More flexible processing of SSL handshakes
2020-02-05 14:57:05 +01:00
Johno Crawford
7413372c01 SSL / BlockHound works out of the box with the default SSL provider (#9969)
Motivation:

JDK is the default SSL provider and internally uses blocking IO operations.

Modifications:

Add allowBlockingCallsInside configuration for SslHandler runAllDelegate function.

Result:

When BlockHound is installed, SSL works out of the box with the default SSL provider.

Co-authored-by: violetagg <milesg78@gmail.com>
2020-01-30 11:50:15 +01:00
Norman Maurer
6a43807843
Use lambdas whenever possible (#9979)
Motivation:

We should update our code to use lamdas whenever possible

Modifications:

Use lambdas when possible

Result:

Cleanup code for Java8
2020-01-30 09:28:24 +01:00
Norman Maurer
a2bf0bfd7a SslHandler.wrap(...) must ensure it not loose the original exception when finishWrap(...) fails (#9974)
Motivation:

When SslHandler.finishWrap throws an exception, ensure that the promise and buf is not reused to avoid throwing IllegalArgumentException or IllegalReferenceCountException which causes the original exception to be lost.

Modification:

The change ensures that the values for the promise and bytebuf are nulled before calling finishWrap so that it will not be called again with the same arguments.

Result:

Fixes #9971 .

Co-authored-by: Norman Maurer <norman_maurer@apple.com>

Co-authored-by: Antony T Curtis <atcurtis@gmail.com>
2020-01-29 08:47:04 +01:00
Bennett Lynch
4950a2fb43 Add ByteBufFormat option to LoggingHandler (#9915)
Motivation

LoggingHandler is a very useful tool for debugging and for tracking the
sequence of events in a pipeline. LoggingHandler also includes the
functionality to log a hex dump of all written and received ByteBufs.
This can be useful for small messages, but for large messages, this can
potentially result in extremely large logs. E.g., a 1 MB payload will
result in over a 1 MB log message being recorded. While LoggingHandler
may only be intended for debugging, this can still be too excessive in
some debugging scenarios.

Modifications

* Create a new ByteBufFormat enum that allows users to specify "SIMPLE"
or "HEX_DUMP" logging for ByteBufs.
* For all constructors that currently accept a LogLevel parameter,
create new overloaded constructors that also accept this enum as a
parameter.
* Continue to record hex dumps by default.

Result

Users will be able to opt out of full hex dump recording, if they wish
to.
2020-01-23 16:58:35 -08:00
Norman Maurer
70ea670ca5 Reduce allocations in ChunkedWriteHandler when processing the queued … (#9960)
Motivation:

At the moment we create a new ChannelFutureListener per chunk when trying to write these to the underlying transport. This can be optimized by replacing the seperate write and flush call with writeAndFlush and only allocate the listener if the future is not complete yet.

Modifications:

- Replace seperate write and flush calls with writeAndFlush
- Only create listener if needed, otherwise execute directly

Result:

Less allocations
2020-01-21 15:46:23 -08:00
Norman Maurer
dce7157be9 Remove extra field from ChunkedWriteHandler to make it less error-prone (#9958)
Motivation:

At the moment we use an extra field in ChunedWriteHandler to hold the current write. This is not needed and makes sense even more error-prone. We can just peek in the queue.

Modifications:

Use Queue.peek() to keep track of current write

Result:

Less error-prone code
2020-01-21 16:44:37 +01:00
Norman Maurer
9e29c39daa
Cleanup usage of Channel*Handler (#9959)
Motivation:

In next major version of netty users should use ChannelHandler everywhere. We should ensure we do the same

Modifications:

Replace usage of deprecated classes / interfaces with ChannelHandler

Result:

Use non-deprecated code
2020-01-20 17:47:17 -08:00
Norman Maurer
43cfe26b47 Add ResolveAddressHandler which can be used to resolve addresses on the fly (#9947)
Motivation:

At the moment resolving addresses during connect is done via setting an AddressResolverGroup on the Bootstrap. While this works most of the times as expected sometimes the user want to trigger the connect() from the Channel itself and not via the Bootstrap. For this cases we should provide a ChannelHandler that the user can use that will do the resolution.

Modifications:

Add ResolveAddressHandler and tests

Result:

Be able to resolve addresses without Bootstrap
2020-01-20 19:34:09 +01:00
Norman Maurer
9b23d3a79a Fix SniHandlerTest when jdkCompatibilityMode is false (#9934)
Motivation:

41c47b4 introduced a change in an existing testcase which let the build fail when jdkCompatibilityMode is false.

Modifications:

Fix unit tests

Result:

Build passes when jdkCompatibilityMode is false as well
2020-01-10 10:15:19 +01:00
Nick Hill
b06878d40a Fix event loop hang in SniHandler (#9933)
Motivation

A bug was introduced in #9806 which looks likely to be the cause of
#9919. SniHandler will enter an infinite loop if an SSL record is
received with SSL major version byte != 3 (i.e. something other than TLS
or SSL3.0)

Modifications

- Follow default path as intended  for majorVersion != 3 in
AbstractSniHandler#decode(...)
- Add unit test to reproduce the hang

Result

Fixes #9919
2020-01-10 08:49:33 +01:00
时无两丶
044b6b0661 FlushConsolidationHandler may suppress flushes by mistake (#9931)
Motivation:

When `consolidatedWhenNoReadInProgress` is true, `channel.writeAndFlush (data) .addListener (f-> channel.writeAndFlush (data2))` Will cause data2 to never be flushed.

Because the flush operation will synchronously execute the `channel.writeAndFlush (data2))` in the `listener`, and at this time, since the current execution thread is still an `eventloop`(`executor.inEventLoop()` was true), all handlers will be executed synchronously. At this time, since `nextScheduledFlush` is still not null, the `flush` operation of `data2` will be ignored in `FlushConsolidationHandler#scheduleFlush`.

Modification:

 - reset `nextScheduledFlush` before `ctx.flush`
 - use `ObjectUtil` to polish code

Result:

Fixes https://github.com/netty/netty/issues/9923
2020-01-09 15:14:04 +01:00
Norman Maurer
0e4c073bcf
Remove the intermediate List from ByteToMessageDecoder (and sub-class… (#8626)
Motivation:

ByteToMessageDecoder requires using an intermediate List to put results into. This intermediate list adds overhead (memory/CPU) which grows as the number of objects increases. This overhead can be avoided by directly propagating events through the ChannelPipeline via ctx.fireChannelRead(...). This also makes the semantics more clear and allows us to keep track if we need to call ctx.read() in all cases.

Modifications:

- Remove List from the method signature of ByteToMessageDecoder.decode(...) and decodeLast(...)
- Adjust all sub-classes
- Adjust unit tests
- Fix javadocs.

Result:

Adjust ByteToMessageDecoder as noted in https://github.com/netty/netty/issues/8525.
2019-12-16 21:00:32 +01:00
Norman Maurer
ff8846f1b5
Replace ObjectUtil.checkNonNull(...) with Objects.requireNonNull(...) (#9864)
Motivation:

We should use Objects.requireNonNull(...) as we require java8

Modifications:

Replace ObjectUtil.checkNonNull(...) with Objects.requireNonNull(...)

Result:

Code cleanup
2019-12-10 11:27:32 +01:00
Norman Maurer
29c471ec52 Correctly handle fragmented Handshake message when trying to detect SNI (#9806)
Motivation:

At the moment our AbstractSniHandler makes the assemption that Handshake messages are not fragmented. This is incorrect as it is completely valid to split these across multiple TLSPlaintext records.

Thanks to @sskrobotov for bringing this to my attentation and to @Lukasa for the help.

Modifications:

- Adjust logic in AbstractSniHandler to handle fragmentation
- Add unit tests

Result:

Correctly handle fragmented Handshake message in AbstractSniHandler (and so SniHandler).
2019-11-29 09:53:29 +01:00
Norman Maurer
b7ba807b30 Fix compile error introduced by 2c3d263e23 2019-11-27 09:08:20 +01:00
ZhenLian
2c3d263e23 Support Passing KeyManager and TrustManager into SslContextBuilder (#9805) (#9786)
Motivation:

This is a PR to solve the problem described here: https://github.com/netty/netty/issues/9767
Basically this PR is to add two more APIs in SslContextBuilder, for users to directly specify
the KeyManager or TrustManager they want to use when building SslContext. This is very helpful
when users want to pass in some customized implementation of KeyManager or TrustManager.

Modification:

This PR takes the first approach in here:
https://github.com/netty/netty/issues/9767#issuecomment-551927994 (comment)
which is to immediately convert the managers into factories and let factories continue to pass
through Netty.

1. Add in SslContextBuilder the two APIs mentioned above
2. Create a KeyManagerFactoryWrapper and a TrustManagerFactoryWrapper, which take a KeyManager
and a TrustManager respectively. These are two simple wrappers that do the conversion from
XXXManager class to XXXManagerFactory class
3.Create a SimpleKeyManagerFactory class(and internally X509KeyManagerWrapper for compatibility),
which hides the unnecessary details such as KeyManagerFactorySpi. This serves the similar
functionalities with SimpleTrustManagerFactory, which was already inside Netty.

Result:

Easier usage.
2019-11-26 14:26:04 +01:00
stroller
2e449a6769 Improve java doc for MINIMAL_WAIT (#9779)
Motivation:

MINIMAL_WAIT is the key constant. Thus, When we see the constant, we must read more code logic to see if it is ms or ns. So improving java doc will be better.

Modifications:
Improve java doc by add "10ms" such as DEFAULT_CHECK_INTERVAL with "1s".

Result:

Easy to know it is ms and keep same java doc style with other constants such as DEFAULT_CHECK_INTERVAL.
2019-11-17 17:42:10 +01:00
Norman Maurer
f8b05b1c84 Don't cache key material if sun.security.ssl.X509KeyManagerImpl is used (#9762)
Motivation:

sun.security.ssl.X509KeyManagerImpl will not use "stable" aliases and so aliases may be changed during invocations. This means caching is useless. Because of this we should disable the cache if its used.

Modifications:

- Disable caching if sun.security.ssl.X509KeyManagerImpl is used
- Add tests

Result:

More protection against https://github.com/netty/netty/issues/9747.
2019-11-07 15:28:00 +01:00
Norman Maurer
38dd3b6bd1 At the moment the cache provided by OpenSslCachingKeyMaterialProvider… (#9759)
Motivation:

At the moment te cache is not bound and so lead to huge memory consumpation. We should ensure its bound by default.

Modifications:

Ensure cache is bound

Result:

Fixes https://github.com/netty/netty/issues/9747.
2019-11-07 15:27:42 +01:00
Nick Hill
7df012884f Rename SimpleChannelInboundHandler.channelRead0() to messageReceived() (#8819)
Motivation

Per javadoc in 4.1.x SimpleChannelInboundHandler:

"Please keep in mind that channelRead0(ChannelHandlerContext, I) will be
renamed to messageReceived(ChannelHandlerContext, I) in 5.0."

Modifications

Rename aforementioned method and all references/overrides.

Result

Method is renamed.
2019-11-01 07:23:07 +01:00
Johno Crawford
6ad59d14d7 Complete todo in SelfSignedCertificate (#9720)
Motivation:

Easier to debug SelfSignedCertificate failures.

Modifications:

Add first throwable as suppressed to thrown exception.

Result:

Less technical debt.
2019-10-28 14:52:14 +01:00
Norman Maurer
4be554a21f Hide Recycler implemention to allow experimenting with different implementions of an Object pool (#9715)
Motivation:

At the moment we directly extend the Recycler base class in our code which makes it hard to experiment with different Object pool implementation. It would be nice to be able to switch from one to another by using a system property in the future. This would also allow to more easily test things like https://github.com/netty/netty/pull/8052.

Modifications:

- Introduce ObjectPool class with static method that we now use internally to obtain an ObjectPool implementation.
- Wrap the Recycler into an ObjectPool and return it for now

Result:

Preparation for different ObjectPool implementations
2019-10-26 09:43:21 +02:00
Anuraag Agrawal
e4d400fa4a Accept Iterable as argument to SslContextBuilder methods. (#9711)
Motivation:

It is common, especially in frameworks, for the parameters to `SslContextBuilder` methods to be built up as a `List` or similar `Iterable`. It is currently difficult to use `SslContextBuilder` in this case because it requires a conversion to array.

Modification:

Add overloads for methods that accept varargs to also accept `Iterable`, delegating by copying into an array.

Result:

Fixes #9293
2019-10-25 15:02:43 +02:00
Zhao Yang
b002f1ffc1 Changed Netty JDK SSL to use default protocols instead of hardcoded supported (#9707)
Motivation:

Netty should respect JVM flags to control SSL protocols, eg. `-Djdk.tls.client.protocols`

Modification:

Changed `JdkSslContext` to use `SSLContext.getDefaultSSLParameters().getProtocols()` instead of `engine.getSupportedProtocols()` which is hardcoded as `SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2`.

Result:

Without `-Djdk.tls.client.protocols`, `SSLContext.getDefaultSSLParameters().getProtocols()` returns `TLSv1, TLSv1.1, TLSv1.2`.

With `-Djdk.tls.client.protocols=TLSv1.2`, `SSLContext.getDefaultSSLParameters().getProtocols()` returns `TLSv1.2`.

Fixes #9706
2019-10-25 08:43:39 +02:00
Carl Mastrangelo
05480c190f Make only default IdleStateEvents cached string representation (#9705)
Motivation:

In PR https://github.com/netty/netty/pull/9695   IdleStateEvents
were made to cache their string representation.   The reason for this
was to avoid creating garbage as these values would be used frequently.
However, these objects may be used on multiple event loops and this
may cause an unexpected race to occur.

Modification:
Only make the events that Netty creates cache their toString representation.

Result:
No races.
2019-10-24 08:35:59 +02:00
Norman Maurer
7150b42a56 Refactor SslHandler internals to always use heap buffers for JDK SSLE… (#9696)
Motivation:

We should aim to always use heap buffers when using the JDK SSLEngine for now as it wants to operate on byte[] and so will do internal memory copies if a non heap buffer is used. Beside this it will always return BUFFER_OVERFLOW when a smaller buffer then 16kb is used when calling wrap(...) (even if a very small amount of bytes should be encrypted). This can lead to excercive direct memory usage and pressure for no good reason.

Modifications:

Refactor internals of SslHandler to ensure we use heap buffers for the JDK SSLEngine impelementation

Result:

Less direct memory usage when JDK SSLEngine implementation is used
2019-10-23 14:29:01 +02:00
ursa
67e3ddb568 Bugfix #9667: FlowControllerHandler swallows read-complete event when auto-read is disabled (#9691)
FlowControllerHandler currently may swell read-complete events in some situations.

* Fire read-complete event from flow controller, when it previously was swallowed
* New unit test to cover this case

Fixes #9667: FlowControllerHandler swallows read-complete event when auto-read is disabled
2019-10-23 11:02:05 +02:00
ursa
aa78d49de1 Add 'toString' method into IdleStateEvent (#9695)
IdleStateEvent is very convenient and frequently used type of events. However both in runtime (logs) and in debug you need some manual steps to see their actual content. Default implementation generates worthless trash like this:

    io.netty.handler.timeout.IdleStateEvent@27f674d

There are examples already, where event has convenient and useful toString implementation:

* io.netty.handler.proxy.ProxyConnectionEvent
* io.netty.handler.ssl.SslCompletionEvent

* Implement 'IdleStateEvent.toString' method.
* Unit test.

More useful String representation of IdleStateEvent
2019-10-23 10:48:25 +02:00
Idel Pivnitskiy
c9c290019e Add a utility that checks if the a SslProvider supports ALPN (#9693)
Motivation:

We have a public utility `OpenSsl.isAlpnSupported()` that helps users to
check if ALPN is available for `SslProvider.OPENSSL`. However, we do not
provide a similar utility for `SslProvider.JDK`. Therefore, users who
configured ALPN with `SslProvider.JDK` will get a runtime exception at
the time when a new connection will be created.

Modifications:

- Add public `SslProvider.isAlpnSupported(SslProvider)` utility method
that returns `true` if the `SslProvider` supports ALPN;
- Deprecate `OpenSsl.isAlpnSupported()`;

Result:

Users can verify if their environment supports ALPN with
`SslProvider` upfront (at bootstrap), instead of failing with
runtime exception when a new connection will be created.
2019-10-23 08:58:32 +02:00
Norman Maurer
7d767d08f5 Add ability to set attributes on a SslContext (#9654)
Motivation:

Sometimes it is useful to be able to set attributes on a SslContext.

Modifications:

Add new method that will return a AttributeMap that is tied to a SslContext instance

Result:

Fixes https://github.com/netty/netty/issues/6542.
2019-10-22 15:40:03 +02:00
Norman Maurer
69575cf5a6 Remove usage of AtomicIntegerFieldUpdater in ReferenceCountedOpenSslE… (#9653)
Motivation:

There is not need to use a CAS as everything is synchronized anyway. We can simplify the code a bit by not using it.

Modifications:

- Just remove the CAS operation
- Change from int to boolean

Result:

Code cleanup
2019-10-13 11:03:08 +02:00
Tim Brooks
678983f2a7 Do not mandate direct bytes in SslHandler queue (#9656)
Motivation:

Currently when the SslHandler coalesces outbound bytes it always
allocates a direct byte buffer. This does not make sense if the JDK
engine is being used as the bytes will have to be copied back to heap
bytes for the engine to operate on them.

Modifications:

Inspect engine type when coalescing outbound bytes and allocate heap
buffer if heap bytes are preferred by the engine.

Result:

Improved performance for JDK engine. Better performance in environments
without direct buffer pooling.
2019-10-12 20:12:37 +02:00
Norman Maurer
d794365411 Fix SSL tests that use SslProvider.OPENSSL_REFCNT (#9649)
Motivation:

031c2e2e88 introduced some change to reduce the risk of have the `ReferenceCountedOpenSslContext` be destroyed while the `ReferenceCountedSslEngine` is still in us. Unfortunaly it missed to adjust a few tests which make assumptions about the refCnt of the context.

Modifications:

Adjust tests to take new semenatics into acount.

Result:

No more tests failures
2019-10-10 08:59:33 +02:00
康智冬
1c69448e2e Fix typos in javadocs (#9527)
Motivation:

We should have correct docs without typos

Modification:

Fix typos and spelling

Result:

More correct docs
2019-10-09 15:25:41 +02:00
Bryce Anderson
3f7a7949db Reference-counted SslEngines retain a reference to their parent SslContext (#9626)
Motivation:
With the Netty ref-counted OpenSSL implementation the parent SslContext
maintains state necessary for the SslEngine's it produces. However, it's
possible for the parent context to be closed and release those resources
before the child engines are finished which causes problems.

Modification:
Spawned ReferenceCountedOpenSslEngine's retain a reference to their
parent ReferenceCountedOpenSslContext.

Result:
The lifetime of the shared data is extended to include the lifetime of
the dependents.
2019-10-07 08:13:05 +02:00
Norman Maurer
14a820d5fa
Always notify FutureListener via the EventExecutor (#9489)
Motiviation:

A lot of reentrancy bugs and cycles can happen because the DefaultPromise will notify the FutureListener directly when completely in the calling Thread if the Thread is the EventExecutor Thread. To reduce the risk of this we should always notify the listeners via the EventExecutor which basically means that we will put a task into the taskqueue of the EventExecutor and pick it up for execution after the setSuccess / setFailure methods complete the promise.

Modifications:

- Always notify via the EventExecutor
- Adjust test to ensure we correctly account for this
- Adjust tests that use the EmbeddedChannel to ensure we execute the scheduled work.

Result:

Reentrancy bugs related to the FutureListeners cant happen anymore.
2019-09-24 09:10:59 +02:00
Francesco Nigro
e5eb94668c ChunkedNioFile can use absolute FileChannel::read to read chunks (#9592)
Motivation:

Users can reuse the same FileChannel for different ChunkedNioFile
instances without being worried that FileChannel::position will be
changed concurrently by them.
In addition, FileChannel::read with absolute position allows to
use on *nix pread that is more efficient then fread.

Modifications:

Always use absolute FileChannel::read ops

Result:

Faster and more flexible uses of FileChannel for ChunkedNioFile
2019-09-24 07:17:26 +02:00
Norman Maurer
790c29ee21 Fix *SslEngineTest to not throw ClassCastException and pass in all cases (#9588)
Motivation:

Due some bug we did endup with ClassCastExceptions in some cases. Beside this we also did not correctly handle the case when ReferenceCountedOpenSslEngineTest did produce tasks to run in on test.

Modifications:

- Correctly unwrap the engine before to fix ClassCastExceptions
- Run delegated tasks when needed.

Result:

All tests pass with different OpenSSL implementations (OpenSSL, BoringSSL etc)
2019-09-21 14:59:10 +02:00
Norman Maurer
fafde4aeec No need to explicit use the AccessController when SystemPropertyUtil is used (#9577)
Motivation:

SystemPropertyUtil already uses the AccessController internally so not need to wrap its usage with AccessController as well.

Modifications:

Remove explicit AccessController usage when SystemPropertyUtil is used.

Result:

Code cleanup
2019-09-19 08:50:36 +02:00
Norman Maurer
2ba99b4996 Correctly handle task offloading when using BoringSSL / OpenSSL (#9575)
Motivation:

We did not correctly handle taskoffloading when using BoringSSL / OpenSSL. This could lead to the situation that we did not write the SSL alert out for the remote peer before closing the connection.

Modifications:

- Correctly handle exceptions when we resume processing on the EventLoop after the task was offloadded
- Ensure we call SSL.doHandshake(...) to flush the alert out to the outboundbuffer when an handshake exception was detected
- Correctly signal back the need to call WRAP again when a handshake exception is pending. This will ensure we flush out the alert in all cases.

Result:

No more failures when task offloading is used.
2019-09-19 08:17:45 +02:00
Norman Maurer
cda8ee95b3 Correctly synchronize before trying to set key material to fix possible native crash (#9566)
Motivation:

When using io.netty.handler.ssl.openssl.useTasks=true we may call ReferenceCountedOpenSslEngine.setKeyMaterial(...) from another thread and so need to synchronize and also check if the engine was destroyed in the meantime to eliminate of the possibility of a native crash.
The same is try when trying to access the authentication methods.

Modification:

- Add synchronized and isDestroyed() checks where missing
- Add null checks for the case when a callback is executed by another thread after the engine was destroyed already
- Move code for master key extraction to ReferenceCountedOpenSslEngine to ensure there can be no races.

Result:

No native crash possible anymore when using io.netty.handler.ssl.openssl.useTasks=true
2019-09-16 11:15:06 +02:00
Norman Maurer
027aec23b8 Allow to build on powerpc
Motivation:

At the moment it is not possible to build netty on a power 8 systems.

Modifications:

- Improve detection of the possibility of using Conscrypt
- Skip testsuite-shading when not on x86_64 as this is the only platform for which we build tcnative atm
- Only include classifier if on x86_64 for tcnative as dependency as this is the only platform for which we build tcnative atm
- Better detect if UDT test can be run

Result:

Fixes https://github.com/netty/netty/issues/9479
2019-09-13 22:21:36 +02:00
stroller
38e5983463 Fix WriteTimeoutException java doc description (#9554)
Motivation:

The java doc doesn't match the real case: The exception only happen when a write operation
 cannot finish in a certain period of time instead of write idle happen.

Modification:

Correct java doc

Result:
java doc matched the real case
2019-09-09 13:59:06 +02:00
Norman Maurer
3099bbcc13
Change semantics of EmbeddedChannel to match other transports more closely. (#9529)
Motiviation:

EmbeddedChannel currently is quite differently in terms of semantics to other Channel implementations. We should better change it to be more closely aligned and so have the testing code be more robust.

Modifications:

- Change EmbeddedEventLoop.inEventLoop() to only return true if we currenlty run pending / scheduled tasks
- Change EmbeddedEventLoop.execute(...) to automatically process pending tasks if not already doing so
- Adjust a few tests for the new semantics (which is closer to other Channel implementations)

Result:

EmbeddedChannel works more like other Channel implementations
2019-09-04 12:00:06 +02:00
Xiaoqin Fu
88aa12cc1a Remove extra checks to fix #9456 (#9523)
Motivation:

There are some extra log level checks (logger.isWarnEnabled()).

Modification:

Remove log level checks (logger.isWarnEnabled()) from io.netty.channel.epoll.AbstractEpollStreamChannel, io.netty.channel.DefaultFileRegion, io.netty.channel.nio.AbstractNioChannel, io.netty.util.HashedWheelTimer, io.netty.handler.stream.ChunkedWriteHandler and io.netty.channel.udt.nio.NioUdtMessageConnectorChannel

Result:

Fixes #9456
2019-08-30 10:40:04 +02:00
Codrut Stancu
de126fdf65 Update GraalVM Native Image configuration. (#9515)
Motivation:

The Netty classes are initialized at build time by default for GraalVM Native Image compilation. This is configured via the `--initialize-at-build-time=io.netty` option. While this reduces start-up time it can lead to some problems:

 - The class initializer of `io.netty.buffer.PooledByteBufAllocator` looks at the maximum memory size to compute the size of internal buffers. If the class initializer runs during image generation, then the buffers are sized according to the very large heap size that the image generator uses, and Netty allocates several arrays that are 16 MByte. The fix is to initialize the following 3 classes at run time: `io.netty.buffer.PooledByteBufAllocator,io.netty.buffer.ByteBufAllocator,io.netty.buffer.ByteBufUtil`. This fix was dependent on a GraalVM Native Image fix that was included in 19.2.0.

 - The class initializer of `io.netty.handler.ssl.util.ThreadLocalInsecureRandom` needs to be initialized at runtime to ensure that the generated values are trully random and not fixed for each generated image.

 - The class initializers of `io.netty.buffer.AbstractReferenceCountedByteBuf` and `io.netty.util.AbstractReferenceCounted` compute field offsets. While the field offset recomputation is necessary for correct execution as a native image these initializers also have logic that depends on the presence/absence of `sun.misc.Unsafe`, e.g., via the `-Dio.netty.noUnsafe=true` flag. The fix is to push these initializers to runtime so that the field offset lookups (and the logic depending on them) run at run time. This way no manual substitutions are necessary either.
 
Modifications:

Add `META-INF/native-image` configuration files that correctly trigger the inialization of the above classes at run time via `--initialize-at-run-time=...` flags.
 
Result:

Fixes the initialisation issues described above for Netty executables built with GraalVM.
2019-08-30 09:21:33 +02:00
Norman Maurer
e6839aa228 Use same JDK SSL test workaround when using ACCP as when just using the JDK SSL implementation (#9490)
Motivation:

14607979f6 added tests for using ACCP but did miss to use the same unwrapping technique of exceptions as JdkSslEngineTest. This can lead to test-failures on specific JDK versions

Modifications:

Add the same unwrapping code

Result:

No more test failures
2019-08-21 20:28:16 +02:00
Norman Maurer
642c9166f4 Add tests for using Amazon Corretto Crypto Provider with Netty (#9480)
Motivation:

Amazon lately released Amazon Corretto Crypto Provider, so we should include it in our testsuite

Modifications:

Add tests related to Amazon Corretto Crypto Provider

Result:

Test netty with Amazon Corretto Crypto Provider
2019-08-20 14:56:03 +02:00
Norman Maurer
6f616bb3cf Avoid creating FileInputStream and FileOutputStream for obtaining Fil… (#8110)
Motivation:

If all we need is the FileChannel we should better use RandomAccessFile as FileInputStream and FileOutputStream use a finalizer.

Modifications:

Replace FileInputStream and FileOutputStream with RandomAccessFile when possible.

Result:

Fixes https://github.com/netty/netty/issues/8078.
2019-08-17 09:52:16 +02:00
Norman Maurer
16e290796d Always wrap X509ExtendedTrustManager when using OpenSSL and JDK < 11 (#9443)
Motivation:

When using OpenSSL and JDK < 11 is used we need to wrap the user provided X509ExtendedTrustManager to be able to support TLS1.3. We had a check in place that first tried to see if wrapping is needed at all which could lead to missleading calls of the user provided trustmanager. We should remove these calls and just always wrap if needed.

Modifications:

Always wrap if OpenSSL + JDK < 11 and TLS1.3 is supported

Result:

Less missleading calls to user provided trustmanager
2019-08-13 10:26:56 +02:00
Nico Kruber
d285623925 Try to load native linux libraries with matching classifier first (#9411)
Motivation:

Users' runtime systems may have incompatible dynamic libraries to the ones our
tcnative wrappers link to. Unfortunately, we cannot determine and catch these
scenarios (in which the JVM crashes) but we can make a more educated guess on
what library to load and try to find one that works better before crashing.

Modifications:

1) Build dynamically linked openSSL builds for more OSs (netty-tcnative)
2) Load native linux libraries with matching classifier (first)

Result:

More developers / users can use the dynamically-linked native libraries.
2019-08-12 08:48:58 +02:00
Farid Zakaria
794bb6c7b6 Introduce SslMasterKeyHandler (#8653)
Motivation

Debugging SSL/TLS connections through wireshark is a pain -- if the cipher used involves Diffie-Hellman then it is essentially impossible unless you can have the client dump out the master key [1]

This is a work-in-progress change (tests & comments to come!) that introduces a new handler you can set on the SslContext to receive the master key & session id. I'm hoping to get feedback if a change in this vein would be welcomed.

An implementation that conforms to Wireshark's NSS key log[2] file is also included.

Depending on feedback on the PR going forward I am planning to "clean it up" by adding documentation, example server & tests. Implementation will need to be finished as well for retrieving the master key from the OpenSSL context.

[1] https://jimshaver.net/2015/02/11/decrypting-tls-browser-traffic-with-wireshark-the-easy-way/
[2] https://developer.mozilla.org/en-US/docs/Mozilla/Projects/NSS/Key_Log_Format

Modification

- Added SslMasterKeyHandler
- An implementation of the handler that conforms to Wireshark's key log format is included.

Result:

Be able to debug SSL / TLS connections more easily.

Signed-off-by: Farid Zakaria <farid.m.zakaria@gmail.com>
2019-07-10 12:16:56 +02:00
jingene
af614e4d6e Change the netty.io homepage scheme(http -> https) (#9344)
Motivation:

Netty homepage(netty.io) serves both "http" and "https".
It's recommended to use https than http.
Modification:

I changed from "http://netty.io" to "https://netty.io"
Result:

No effects.
2019-07-09 21:10:14 +02:00
Norman Maurer
d951140c56 Fix compile error introduced by bad cherry-pick of f7e8603d60 2019-07-04 09:02:23 +02:00