Commit Graph

10479 Commits

Author SHA1 Message Date
Norman Maurer
f0ffb6ac2e Fix compile error introduced by bad cherry-pick of 3c5b6c3e66 2021-08-20 19:02:09 +02:00
Norman Maurer
3c5b6c3e66 Allow server initiated renegotiate when using OpenSSL / BoringSSL based SSLEngine (#11601)
Motivation:

We should allow server initiated renegotiation when OpenSSL / BoringSSL bases SSLEngine is used as it might be used for client auth.

Modifications:

- Upgrade netty-tcnative version to be able to allow renegotiate once
- Adjust code

Result
Fixes https://github.com/netty/netty/issues/11529
2021-08-20 18:52:43 +02:00
Chen Liu
ea5bc27c83 Always release the sslEngine inside SslHandler's handlerRemoved0 (#11605)
Motivation:
Make SslHandler's handlerRemoved0 method release the sslEngine
even if it fails in the middle.
See details in https://github.com/netty/netty/issues/11595.

Modifications:
Wrap the release of sslEngine into a finally block.

Result:
The sslEngine would be released eventually.

Co-authored-by: Chen Liu <cliu@splunk.com>
2021-08-20 18:51:24 +02:00
Norman Maurer
fb1face924 Only suppert TLSv1.3 when JDK does support it as well (#11604)
Motivation:

We tightly integrate the TrustManger and KeyManager into our native SSL implementation which means that both of them need to support TLSv1.3 as protocol. This is not always the case and so can produce runtime exceptions.

As TLSv1.3 support was backported to Java8 quite some time now we should be a bit more conservative and only enable TLSv1.3 for our native implementation if the JDK implementation supports it as well. This also allows us to remove some hacks we had in place to be able to support it before in Java8.

Modifications:

- Only enable TLSv1.3 support for our native SSL implementation when the JDK supports it as well
- Remove OpenSslTlsv13X509ExtendedTrustManager as its not needed anymore

Result:

Fixes https://github.com/netty/netty/issues/11589
2021-08-20 11:42:29 +02:00
Chris Vest
7971a252a5
Clean up Future/Promises API (#11575)
Motivation:
The generics for the existing futures, promises, and listeners are too complicated.
This complication comes from the existence of `ChannelPromise` and `ChannelFuture`, which forces listeners to care about the particular _type_ of future being listened on.

Modification:
* Add a `FutureContextListener` which can take a context object as an additional argument. This allows our listeners to have the channel piped through to them, so they don't need to rely on the `ChannelFuture.channel()` method.
* Make the `FutureListener`, along with the `FutureContextListener` sibling, the default listener API, retiring the `GenericFutureListener` since we no longer need to abstract over the type of the future.
* Change all uses of `ChannelPromise` to `Promise<Void>`.
* Change all uses of `ChannelFuture` to `Future<Void>`.
* Change all uses of `GenericFutureListener` to either `FutureListener` or `FutureContextListener` as needed.
* Remove `ChannelFutureListener` and `GenericFutureListener`.
* Introduce a `ChannelFutureListeners` enum to house the constants that previously lived in `ChannelFutureListener`. These constants now implement `FutureContextListener` and take the `Channel` as a context.
* Remove `ChannelPromise` and `ChannelFuture` — all usages now rely on the plain `Future` and `Promise` APIs.
* Add static factory methods to `DefaultPromise` that allow us to create promises that are initialised as successful or failed.
* Remove `CompleteFuture`, `SucceededFuture`, `FailedFuture`, `CompleteChannelFuture`, `SucceededChannelFuture`, and `FailedChannelFuture`.
* Remove `ChannelPromiseNotifier`.

Result:
Cleaner generics and more straight forward code.
2021-08-20 09:55:16 +02:00
Norman Maurer
afd812c3a4 Call OpenSsl.ensureAvailability() before trying to construct OpenSsl*Context (#11602)
Motivation:

At the moment why you try to build a SslContext via SslProvider.OPENSSL* and netty-tcnative* is not on the classpath it will fail with:

```
Exception in thread "main" java.lang.NoClassDefFoundError: io/netty/internal/tcnative/SSLPrivateKeyMethod
	at io.netty.handler.ssl.SslContext.newClientContextInternal(SslContext.java:830)
	at io.netty.handler.ssl.SslContextBuilder.build(SslContextBuilder.java:611)
	at io.netty.handler.codec.http.Main.main(Main.java:34)
Caused by: java.lang.ClassNotFoundException: io.netty.internal.tcnative.SSLPrivateKeyMethod
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
	at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
	... 3 more
```

We should do better here.

Modifications:

Call `OpenSsl.ensureAvailability()` before trying to construct OpenSsl*Context instances

Result:

More clear error message like:

```
java.lang.UnsatisfiedLinkError: failed to load the required native library

	at io.netty.handler.ssl.OpenSsl.ensureAvailability(OpenSsl.java:540)
	at io.netty.handler.ssl.SslContext.newClientContextInternal(SslContext.java:830)
	at io.netty.handler.ssl.SslContextBuilder.build(SslContextBuilder.java:611)
        ...
Caused by: java.lang.ClassNotFoundException: io.netty.internal.tcnative.SSLContext
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
	at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
	at java.base/java.lang.Class.forName0(Native Method)
	at java.base/java.lang.Class.forName(Class.java:398)
	at io.netty.handler.ssl.OpenSsl.<clinit>(OpenSsl.java:126)
	... 68 more
```
2021-08-19 20:56:17 +02:00
Chris Vest
d27a2b3df9 Backport some fixes and cleanups for DefaultPromiseTest (#11600)
Motivation:
These cleanups were done in another PR but were not directly related to that PR.
This extracts those changes and backports them to 4.1.

Modification:
* Remove the use of mocking in DefaultPromiseTest.
* Fix a few warnings.
* Make `testStackOverFlowChainedFuturesB` test with the right listener chain.

Result:
Cleaner code.
2021-08-19 13:30:17 +02:00
skyguard1
aa69d5b5cf Add Zstd.isAvailable() check in ZstdOptions (#11597)
Motivation:

At present, the verification methods of `ZstdOptions` and `BrotliOptions` are not consistent, and the processing methods of `ZstdOptions` and `BrotliOptions` in `HttpContentCompressor` are also inconsistent.
The http2 module does not add zstd-jni dependency, so `ClassNotFoundException` may be thrown

Modification:

Added `Zstd.isAvailable()` check in `ZstdOptions` to be consistent, and added zstd-jni dependency in http2 module

Result:

The verification methods of `ZstdOptions` and `BrotliOptions` are consistent, and `ClassNotFoundException` will not be thrown


Signed-off-by: xingrufei <xingrufei@sogou-inc.com>
2021-08-19 11:10:40 +02:00
Jeremy Kuhn
009497f5f9 Fix support for optional encoders errors in HttpContentCompressor (#11582)
Motivation:

- Fix `HttpContentCompressor` errors due to missing optional compressor libraries such as Brotli and Zstd at runtime.
- Improve support for optional encoders by only considering the `CompressionOptions` provided to the constructor and ignoring those for which the encoder is unavailable.

Modification:

The `HttpContentCompressor` constructor now only creates encoder factories for the CompressionOptions passed to the constructor when the encoder is available which must be checked for Brotli and Zstd. In case of Brotli, it is not possible to create BrotliOptions if brotly4j is not available so there's actually nothing to check. In case of Zstd, I had to create class `io.netty.handler.codec.compression.Zstd` similar to `io.netty.handler.codec.compression.Brotli` which is used to check that zstd-jni is availabie at runtime.

The `determineEncoding()` method had to change as well in order to ignore encodings for which there's no `CompressionEncoderFactory` instance.

When the HttpContentCompressor is created using deprecated constructor (ie. with no CompressionOptions), we consider all available encoders.

Result:

Fixes #11581.
2021-08-19 08:44:08 +02:00
skyguard1
b0e28e3740 Refactor HttpContentCompressor using CompressionEncoderFactory (#11480)
Motivation:

The `HttpContentCompressor.beginEncode()` method has too many if else, so consider refactoring

Modification:

Create the corresponding `CompressionEncoderFactory` according to the compression algorithm, remove the if else

Result:

The code of `HttpContentCompressor` is cleaner than the previous implementation

Signed-off-by: xingrufei <xingrufei@sogou-inc.com>
Co-authored-by: Norman Maurer <norman_maurer@apple.com>
2021-08-19 08:44:04 +02:00
Chris Vest
4a7fa3777e Remove dead code and fix warnings in the http2 module (#11593)
Motivation:
Opportunities for clean up found while working on a different PR.

Modification:
* Dead code has been removed.
* Unnecessary parenthesis, qualifiers, etc. removed.
* Unused imports removed.
* Override annotations added where missing.

Result:
Cleaner code
2021-08-18 20:55:44 +02:00
Chris Vest
9eb4f0ee85 Fix a problem with IP protocol version confusion on MacOS when TCP FastOpen is enabled (#11588)
Motivation:
This fixes a bug that would result in an `io.netty.channel.unix.Errors$NativeIoException: connectx(..) failed: Address family not supported by protocol family` error.
This happens when the connecting socket is configured to use IPv6 but the address being connected to is IPv4.
This can occur because, for instance, Netty and `InetAddress.getLoopbackAddress()` have different preferences for IPv6 vs. IPv4.

Modification:
Pass the correct ipv6 or ipv4 flags to connectx, depending on whether the socket was created for AF_INET or AF_INET6, rather than relying on the IP version of the destination address.

Result:
No more issue with TCP FastOpen on MacOS when using addresses of the "wrong" IP version.
2021-08-18 20:47:56 +02:00
Norman Maurer
0e2f4adc9d Fix JDK17 profile
Motivation:

89866da252 did introduce a JDK17 profile but did not adjust it for the master branch which needs java11 at least

Modifications:

Fix config

Result:

Be able to compile with JDK17
2021-08-18 16:22:26 +02:00
Norman Maurer
766fefa0f9 Use StandardSocketOptions#IP_MULTICAST_IF as default source when joing multicast groups (#11585)
Motivation:

We should use StandardSocketOptions#IP_MULTICAST_IF as default source when joing multicast groups and only try to use the localAddress if this returns null.

Modifications:

First check if StandardSocketOptions#IP_MULTICAST_IF was set and if so use the network interface when joining mulicast groups

Result:

Fixes https://github.com/netty/netty/issues/11541
2021-08-18 16:18:57 +02:00
Norman Maurer
8e4465d14c Fix IndexOutOfBoundsException caused by consuming the buffer multiple times in DatagramDnsQueryDecoder (#11592)
Motivation:

ccef8feedd introduced some changes to share code but did introduce a regression when decoding queries.

Modifications:

- Correctly only decode one time.
- Adjust unit test

Result:

Fixes https://github.com/netty/netty/issues/11591
2021-08-18 15:05:45 +02:00
Norman Maurer
89866da252 Add CI job for JDK17 (#11584)
Motivation:

As JDK17 is really close to be released we should add a CI job for it to ensure netty works correctly when using it.

Modifications:

Add docker config and workflow config to run CI job for JDK17

Result:

Ensure netty works on JDK17 as well
2021-08-18 10:13:20 +02:00
Norman Maurer
8549d7dc87 Add cast to help compiler to build (#11587)
Motivation:

Sometimes intellij fails to build because the compiler cant really figure out what to do. We can workaround this by adding some explicit casts.

Modifications:

Add explicits casts

Result:

No more problems with intellij
2021-08-18 09:43:03 +02:00
Norman Maurer
22e71e4efd Add some docs for io.netty.leakDetection.acquireAndReleaseOnly.
Motivation:

We should add some docs / comment to explain what io.netty.leakDetection.acquireAndReleaseOnly is all about.

Modifications:

Add some comments

Result:

Fixes https://github.com/netty/netty/issues/11576.
2021-08-17 09:10:16 +02:00
Norman Maurer
8083818b45 Ensure we only log message on BoringSSL when the ciphers really are not the default (#11583)
Motivation:

0c9a86db81 added a change to log a message if someone tried to change the TLSv1.3 ciphers when using BoringSSL. Unfortunally the code had some error and so even if the user did not change these we logged something.

Modifications:

- Ensure there are no duplicates in the ciphers
- Correctly take TLSv1.3 extra ciphers into account when using BoringSSL

Result:

Correctly log or not log
2021-08-16 22:26:55 +02:00
Chris Vest
3e2e36eac5
Remove the deprecated ThreadDeathWatcher (#11574)
Motivation:
The deprecated ThreadDeathWatcher produces more garbage and can delay resource release, when compared to manual resource management.

Modification:
Remove the ThreadDeathWatcher and other deprecated APIs that rely on it.

Result:
Less deprecated code.
2021-08-16 14:33:58 +02:00
DD
85b0eb2d3e server h2c upgrade fail when request doesn't have connection header (#11569)
__Motivation__
Since request.headers().getAll() will never return null. And the check null condition will not work as expected.

__Modification__

Add isEmpty() checking as well.

__Result__

Fixes #11568
2021-08-12 09:05:59 -07:00
Chris Vest
25699e44e9 Add support for client-side TCP FastOpen to KQueue MacOS (#11560)
Motivation:
The MacOS-specific `connectx(2)` system call make it possible to establish client-side connections with TCP FastOpen.

Modification:
Add support for TCP FastOpen to the KQueue transport, and add the `connectx(2)` system call to `BsdSocket`.

Result:
It's now possible to use TCP FastOpen when initiating connections on MacOS.
2021-08-12 13:50:34 +02:00
Chris Vest
bcdc07fe13 Improve test failure reporting of EpollSocketChannelConfigTest (#11570)
Motivation:
This test is inherently flaky due to file descriptor reuse.
Even though we have taken steps to make it less flaky, it still fails sometimes.
When it does, the error message is not very helpful.

Modification:
Make use of assertThrows and assertThat to get more descriptive error messages when the tests fail.

Result:
More meaningful messages on test failures, which may help us make the tests more resilient in the future
2021-08-12 11:39:19 +02:00
Chris Vest
a3d5617d45
Remove deprecated stuff around ResourceLeakDetector (#11572)
Motivation:
A number of classes and APIs around the ResourceLeakDetector have been deprecated for removal in Netty 5.x, because better alternatives exist.

Modification:
Remove everything in and around ResourceLeakDetector that is deprecated, and fix the few usages that were found.

Result:
Less deprecated code.
2021-08-11 21:41:49 +02:00
Chris Vest
11fcfe1f73
Remove PromiseAggregator (#11571)
Motivation:
This class was deprecated, since a better alternative exists in `PromiseCombiner`.

Modification:
Remove `PromiseAggregator`, its Channel companion, and its test.

Result:
Less deprecated code.
2021-08-11 21:41:16 +02:00
Aayush Atharva
99bd5895dc Inline variables to make code more readable (#11565)
Motivation:
There are lots of redundant variable declarations which should be inlined to make good look better.

Modification:
Made variables inlined.

Result:
Less redundant variable and more readable code.
2021-08-11 17:07:32 +02:00
skyguard1
00d1dfb686 Some small improvements (#11564)
Signed-off-by: xingrufei <xingrufei@sogou-inc.com>
2021-08-11 13:42:24 +02:00
dependabot[bot]
b9ac4b62bf Bump commons-compress from 1.19 to 1.21 (#11539)
Bumps commons-compress from 1.19 to 1.21.

---
updated-dependencies:
- dependency-name: org.apache.commons:commons-compress
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-08-10 10:54:17 -07:00
dependabot[bot]
f921e08ea2 Bump ant from 1.10.9 to 1.10.11 (#11538)
Bumps ant from 1.10.9 to 1.10.11.

---
updated-dependencies:
- dependency-name: org.apache.ant:ant
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-08-10 10:52:12 -07:00
Chris Vest
ce947cc781 Make the TCP_FASTOPEN channel option transport agnostic (#11559)
Motivation:
TCP FastOpen is a pure optimisation, that is opportunistically applied.
There is no reason to make it specific to the epoll transport, and in the future we could add support to other transports.
Besides, the client-side equivalent, TCP_FASTOPEN_CONNECT, is already transport agnostic.

Modification:
Move the TCP_FASTOPEN channel option from EpollChannelOption to ChannelOption.
Mark the field in EpollChannelOption as deprecated.

Result:
All channel options related to TCP FastOpen are now transport agnostic.
However, we still only actually support TFO on the epoll transport.
2021-08-10 09:07:43 +02:00
Aayush Atharva
9e6cd2822f Disable flaky Http2MultiplexTransportTest test on Windows (#11554)
* Disable flaky tests on Windows
2021-08-10 09:07:18 +02:00
Ikko Ashimine
c95f99c81c Fix typo in AbstractSearchProcessorFactory.java (#11562)
Motivation:

Fixed typo.

occurences -> occurrences
2021-08-10 09:07:09 +02:00
Chris Vest
6a92a3354e Use the standard japicmp.skip instead of the custom skipJapicmp (#11558) 2021-08-10 09:07:04 +02:00
Scott Mitchell
c41478da0f Remove io.netty.http2.validateContentLength SystemProperty (#11561)
Motivation:
io.netty.http2.validateContentLength SystemProperty was added as a way
to opt-out for compabitility for libraries/applications that violated
the RFC's content-length matching requirements [1] but have not yet been
fixed. This SystemProperty has been around for a few months now and it
is assumed these issues have now been addressed in 3rd party code.

[1] https://tools.ietf.org/html/rfc7540#section-8.1.2.6

Modifications:
- Remove the io.netty.http2.validateContentLength SystemProperty,
  preventing folks from opting out of RFC's required content-length
  matching.

Result:
No more escape hatch in H2 for content-length matching enforcement.
2021-08-10 09:04:56 +02:00
Aayush Atharva
25a0a6d425 Make variables final (#11548)
Motivation:
We should make variables `final` which are not reinstated again in code to match the code style and makes the code look better.

Modification:
Made couples of variables as `final`.

Result:
Variables marked as `final`.
2021-08-06 09:28:12 +02:00
Aayush Atharva
ea8f6774d5 Simplify Bitwise operations (#11547)
Motivation:
We should keep bitwise operations simple and easy to understand.

Modification:
Simplify few Bitwise operations.

Result:
Less complicated bitwise operation code
2021-08-06 09:17:02 +02:00
Chris Vest
ef203fa6cb
Fix a number of javadoc issues (#11544)
Motivation:
Let's have fewer warnings about broken, missing, or abuse of javadoc comments.

Modification:
Added descriptions to throws clauses that were missing them.
Remove link clauses from throws clauses - these are implied.
Turned some javadoc comments into block comments because they were not applied to APIs.
Use code clauses instead of code tags.

Result:
Fewer javadoc crimes.
2021-08-06 09:14:04 +02:00
Chris Vest
6c2813bdb0 Make EpollSocketChannelConfigTest tollerant of fd reuse race (#11552)
Motivation:
We cannot control when "the system" reuses file descriptors.
This makes any test that assert on the behaviour of closed file descriptors inherently racy.

Modification:
Allow the EpollSocketChannelConfigTest socketoption tests a few tries to get the correct assertion on the behaviour of closed socket file descriptors.

Result:
The EpollSocketChannelConfigTest should now be much less flaky.
2021-08-06 08:46:38 +02:00
Aayush Atharva
6d6d2060af Remove unnecessary toString calls (#11550)
Motivation:
We should get rid of the unnecessary toString calls because they're redundant in nature.

Modification:
Removed unnecessary toString calls.

Result:
Better code
2021-08-05 14:17:23 +02:00
Aayush Atharva
1c574e0d22 Remove unnecessary semicolons (#11549)
Motivation:
We should get rid of unnecessary semicolons because they don't do anything in code.

Modification:
Removed unnecessary semicolons.

Result:
Better code
2021-08-05 14:10:40 +02:00
Aayush Atharva
b700793951 Remove Unused Imports (#11546)
Motivation:
There are lots of imports which are unused. We should get rid of them to make the code look better,

Modification:
Removed unused imports.

Result:
No unused imports.
2021-08-05 14:08:07 +02:00
Chris Vest
91e48902bb
Small cleanups (#11545)
Motivation:
Code that isn't used, or has better alternatives in the JDK should be removed or replaced, respectively.

Modification:
Remove dead code in DefaultPromise.
Replace the synchronised-based reachability fence in ResourceLeakDetector, with the JDK Reference.reachabilityFence.

Result:
Cleaner code
2021-08-05 10:37:52 +02:00
Chris Vest
6b11f7fbc2
All *Bootstrap methods that used to return ChannelFuture now return Future<Channel> (#11517)
Bootstrap methods now return Future<Channel> instead of ChannelFuture

Motivation:
In #8516 it was proposed to at some point remove the specialised ChannelFuture and ChannelPromise.
Or at least make them not extend Future and Promise, respectively.
One pain point encountered in this discussion is the need to get access to the channel object after it has been initialised, but without waiting for the channel registration to propagate through the pipeline.

Modification:
Add a Bootstrap.createUnregistered method, which will return a Channel directly.
All other Bootstrap methods that previously returned ChannelFuture now return Future<Channel>

Result:
It's now possible to obtain an initialised but unregistered channel from a bootstrap, without blocking.
And the other bootstrap methods now only release their channels through the result of their futures, preventing racy access to the channels.
2021-08-03 19:43:38 +02:00
Chris Vest
4f0f889dbf Fix a bug with delegate/async SSL (#11537)
Motivation:
This bug could occasionally cause SSL handshakes to time out, because the server-side handshake would fail to resume its event loop.

Modification:
Async delegate SSL tasks now lower their NEED_TASK status after they have executed, but before they run their completion callback.
This is important because the completion callback could be querying the handshake status.
This could cause the task delegator thread and the event look to race.
If the event look queries the handshake status first, it might think that it still needs to delegate another task.
If this happens, the delegator find a null task, and then fail to resume the event loop, causing the handshake to stall.

Result:
This data race no longer causes handshake timeouts.
2021-08-03 10:14:41 +02:00
Norman Maurer
202aee34c4 Ensure we always wrap if there is something left to be send to the remote peer (#11535)
Motivation:

We need to ensure we call wrap as long as there is something left to be send to the remote peer in cases of non-application data (like for example alerts).

Modifications:

Check the pending data and based on it return NEED_WRAP even when the handshake was done.

Result:

Always produce alerts etc
2021-08-02 10:14:11 +02:00
Aayush Atharva
7d1de8c970 Add Increment and Decrement count positive check in ReferenceCountUtil (#11523)
Motivation:
We should have guards in place to check increment or decrement count in `ReferenceCountUtil`. Increment and Decrement counts must be a positive integer.

Modification:
Added `ObjectUtil#checkPositive` checks.

Result:
Prevent release due to invalid count.
2021-07-29 20:51:39 +02:00
Norman Maurer
7f87b09a96 Ensure we include hs_err* files when build failed (#11525)
Motivation:

We the build failed we should ensure we also include hs_err* files as it may have failed due a native crash

Modifications:

Adjust path matching to include hs_err as well

Result:

Easier to debug native crashes
2021-07-29 08:48:31 +02:00
Norman Maurer
1d39dd5195 Add @UnstableApi to TcpDnsResponseEncoder (#11526)
Motivation:

We should mark TcpDnsResponseEncoder as unstable

Modifications:

Add annotation

Result:

Mark the encoder as unstable
2021-07-28 19:53:30 +02:00
Norman Maurer
4c78c93a62 Fix false-positive report of japicmp-maven-plugin introduced by OCccef8feedd726743d0355b34799e4915536d72ab 2021-07-28 12:06:24 +02:00
Norman Maurer
7f3512d42a Add @UnstableApi annotation to TcpDnsQueryDecoder introduced by ccef8feedd 2021-07-28 11:38:00 +02:00