Commit Graph

10567 Commits

Author SHA1 Message Date
Norman Maurer e038fb7b0e Deprecate UnaryPromiseNotifier
Motivation:

Users should use PromiseNotifier.cascade(...) methods and so the UnaryPromiseNotifier becomes useless.

Modifications:

- Mark UnaryPromiseNotifier as deprecated
- Replaces usages with PromiseNotifier.cascade(...)

Result:

Cleanup
2021-09-03 08:42:23 +02:00
Violeta Georgieva 47bababfc7
Ensure HttpData#addContent/setContent releases the buffer before throwing IOException (#11621)
Motivation:
When the ByteBuf size exceeds the max limit/defined size, IOException is thrown.
HttpData#addContent/setContent should release the buffer in such cases otherwise memory leak will happen.

Modification:
- Release the provided ByteBuf before throwing IOException
- Add unit tests

Result:
Fixes #11618
2021-09-02 13:59:44 -07:00
skyguard1 d58d8a1df8
Remove useless imports and redundant type cast (#11639)
Motivation:

There are some redundant imports and unnecessary type cast

Modification:

Remove useless imports and unnecessary type cast

Result:

The code is cleaner than original

Signed-off-by: xingrufei <xingrufei@sogou-inc.com>
2021-09-02 19:14:00 +02:00
Idel Pivnitskiy 5ebdee8537
Always log Http2UnknownStreamError at `DEBUG` level (#11643)
Motivation:

Exception logged by `onHttp2UnknownStreamError` is propagated to the
upper layers anyway. Receiver of the exception is responsible for
correct handling. Inside netty, it's enough to log at `DEBUG` level.

Modifications:

- `Http2FrameCodec#onHttp2UnknownStreamError` always logs at `DEBUG`
level;

Result:

Less noise in logs when `Http2UnknownStreamError` is properly handled by
upper layers.

Co-authored-by: Norman Maurer <norman_maurer@apple.com>
Co-authored-by: Chris Vest <mr.chrisvest@gmail.com>
2021-09-02 19:11:46 +02:00
Francesco Nigro c8c45cfa4c
O(1) buffer next capacity computation (#11641)
Motivation:

Enlarging buffers approaching 4 MiB size requires n iterations

Modification:

Use a single instruction to compute the next buffer capacity

Result:

Faster/Simpler calculateNewCapacity
2021-09-01 17:48:26 +02:00
Norman Maurer 77cf43cfe9
Fix release problems caused by the testsuite-shading module (#11640)
Motivation:

Last time when we tried to do a release it failed due the fact that no javadoc jar was generated. Beside this how we did configure the shading module did introduce a lot of duplication

Modifications:

- Generate an empty javadoc jar
- Share config for shading plugin

Result:

No more problems during release process
2021-09-01 14:39:46 +02:00
Chris Vest b3d3269b8c
Avoid overspecifying ScheduledFuture when Future will do (#11636)
Motivation:
Overspecifying interfaces can increase coupling and make refactoring harder down the line.

Modification:
Places that were specifying fields, variables, and return types as ScheduledFuture, but did not use any features specific to ScheduledFuture, have been changed to specify Future instead.

Result:
Cleaner code.
2021-08-31 16:05:21 +02:00
Chun-Han, Hsiao d027ba7320
[HTTP2] Fix memory leak while writing empty data frame with padding (#11633)
Motivation:

There is a memory leak while writing empty data frame with padding.

The empty data frame was occurred because we are running a proxy server
built by netty, and found that google services always sent data frames
followed by an empty data frame.

Modifications:

Calls the ctx.write even the payload is empty.

Result:

Fix memory leak.

Co-authored-by: Norman Maurer <norman_maurer@apple.com>
2021-08-31 12:41:49 +02:00
Violeta Georgieva d2aceea3b0
Ensure DiskFileUpload#toString returns a string that uses the correct deleteOnExit (#11628)
Motivation:
DiskFileUpload#toString should use the value provided when constructing DiskFileUpload
and not the default one

Modification:
- DiskFileUpload#toString is modified to use the correct deleteOnExit value

Result:
DiskFileUpload#toString returns a string that uses the correct deleteOnExit
2021-08-31 08:21:56 +02:00
Norman Maurer 5c842dd89f
Add missing annotations to test overrides (#11630)
Motivation:

When we override a test with params we also need to ensure we add the correct annotations to the override

Modifications:

Add the correct annotations so the tests are actually run in intellij

Result:

Each test can be run
2021-08-31 08:07:56 +02:00
Norman Maurer 6f00c5ebf6
Fix leak in TcpDnsTest (#11632)
Motivation:

We did get a report for a leak on the CI.

Modifications:

Fix leak in test by correctly releasing the query

Result:

No more leaks
2021-08-31 08:06:15 +02:00
kushalagrawal 7b8050ae53
Added "RSASSA-PSS" algorithm in allowed algorithm list. (#11626)
Motivation:
While OpenSSK is provided support for the "RSASSA-PSS" algorithm this was still not valid from netty. Which was causing issue in validating certificates which was signed using this algorithm.

Modification:
Added "RSASSA-PSS" in LOCAL_SUPPORTED_SIGNATURE_ALGORITHMS.
validation:
Validated and tested with CA and User cert singed with RSASSA-PSS algorithm.

Result:
Fixes #11360

Co-authored-by: Norman Maurer <norman_maurer@apple.com>
2021-08-30 17:15:57 -07:00
Norman Maurer 95e1a21836 Replace master with main branch (#11629)
Motivation:

We should not use master as branch name, better to switch to main

Modifications:

Replace master branch name with main

Result:

Use main as replacement for master
2021-08-30 15:39:46 +02:00
skyguard1 88fcb5124b
Fix QueryStringEncoder encodes tilde (#11590)
Motivation:

In this issue(#11578 ) discussed that tilde should not be encoded in QueryStringEncoder, this pr is to fix this problem

Modification:

Modified QueryStringEncoder so that it does not encode tilde, and added a test case

Result:

Fixes #11578
2021-08-26 12:53:07 -07:00
Chris Vest 6a7cccded0
Try to make the PR Reports workflow more robust (#11620)
Motiviation:
The workflow from the default branch is the one that gets to run for all PRs, regardless of what branch they are targeting.
This means the 4.1 PR Reports workflow needs to tollerate master branch PRs, where there is no Java 8 builds.

Modification:
Make the PR Reports tolerate that the Java 8 matrix fails if they are missing.

Result:
Hopefully we'll get working PR Reports for master branch PRs now.
2021-08-26 11:19:11 +02:00
Norman Maurer 6b7c58a2f9
Don't throw if null is given as ByteBuf when adding components. (#11613)
Motivation:

232c669fa4 did add some overflow protection but did not handle null elements in the array the same as before.

Modifications:

- Break the loop if a null element was found
- Add unit test

Result:

Fixes https://github.com/netty/netty/issues/11612
2021-08-26 08:20:17 +02:00
Norman Maurer 60f8ed01cd
Correctly respect array offset when check for overflow (#11614)
Motivation:

232c669fa4 did add overflow protection but did miss to take the array offset into account and so could report false-positives

Modifications:

- Correctly take offset into account when check for overflow.
- Add unit tests

Result:

Correctly take offset into account when overflow check is performed
2021-08-26 08:03:35 +02:00
Chris Vest fc8311cd18
Try to get the PR Reports workflow running again (#11610)
Motivation:
This workflow has been broken for a couple of months now.
It would be nice if we could get junit test reports back in out PR checks.

Modificatoin:
Fix a test matrix typo.
Update workflow action versions.

Result:
Hopefully the PR Reports workflow starts working again.

Draft until it actually does work. There are probably several things wrong that will take time to work through.
2021-08-25 14:15:13 +02:00
Norman Maurer 5e7eb3748d
Add version for netty-tcnative* to bom (#11609)
Motivation:

Keeping the version of netty-tcnative correct can sometimes be hard.

Modifications:

Add entries for netty-tcnative* to the bom

Result:

Fixes https://github.com/netty/netty/issues/11567
2021-08-25 08:54:04 +02:00
Diego Lovison 2110755af0
SimpleChannelPool::notifyConnect should tryFailure when an exception occurs (#11566)
Motivation:

During the channelAcquired handler invocation if an exception occurred the application will hang

Modifications:

Call tryFailure of the promise if an exception occurred

Result:

After this change, the promise will fail and the system won't hang.

Co-authored-by: NiteshKant <nitesh_kant@apple.com>
2021-08-24 15:36:05 -07:00
Norman Maurer 9d25bc8a4e
Fix leak in TcpDnsTest (#11606)
Motivation:

In another PR we did observe a leak report for TcpDnsTest

Modifications:

Correctly release the query.

Result:

No more leaks
2021-08-20 19:10:54 +02:00
Norman Maurer 33b63c325f
Allow server initiated renegotiate when using OpenSSL / BoringSSL based SSLEngine (#11601)
Motivation:

We should allow server initiated renegotiation when OpenSSL / BoringSSL bases SSLEngine is used as it might be used for client auth.

Modifications:

- Upgrade netty-tcnative version to be able to allow renegotiate once
- Adjust code

Result
Fixes https://github.com/netty/netty/issues/11529
2021-08-20 18:52:22 +02:00
Chen Liu 36eb399b4b
Always release the sslEngine inside SslHandler's handlerRemoved0 (#11605)
Motivation:
Make SslHandler's handlerRemoved0 method release the sslEngine
even if it fails in the middle.
See details in https://github.com/netty/netty/issues/11595.

Modifications:
Wrap the release of sslEngine into a finally block.

Result:
The sslEngine would be released eventually.

Co-authored-by: Chen Liu <cliu@splunk.com>
2021-08-20 18:50:19 +02:00
Norman Maurer fd47554669
Only suppert TLSv1.3 when JDK does support it as well (#11604)
Motivation:

We tightly integrate the TrustManger and KeyManager into our native SSL implementation which means that both of them need to support TLSv1.3 as protocol. This is not always the case and so can produce runtime exceptions.

As TLSv1.3 support was backported to Java8 quite some time now we should be a bit more conservative and only enable TLSv1.3 for our native implementation if the JDK implementation supports it as well. This also allows us to remove some hacks we had in place to be able to support it before in Java8.

Modifications:

- Only enable TLSv1.3 support for our native SSL implementation when the JDK supports it as well
- Remove OpenSslTlsv13X509ExtendedTrustManager as its not needed anymore

Result:

Fixes https://github.com/netty/netty/issues/11589
2021-08-20 11:41:52 +02:00
Norman Maurer 22a188c8cc
Call OpenSsl.ensureAvailability() before trying to construct OpenSsl*Context (#11602)
Motivation:

At the moment why you try to build a SslContext via SslProvider.OPENSSL* and netty-tcnative* is not on the classpath it will fail with:

```
Exception in thread "main" java.lang.NoClassDefFoundError: io/netty/internal/tcnative/SSLPrivateKeyMethod
	at io.netty.handler.ssl.SslContext.newClientContextInternal(SslContext.java:830)
	at io.netty.handler.ssl.SslContextBuilder.build(SslContextBuilder.java:611)
	at io.netty.handler.codec.http.Main.main(Main.java:34)
Caused by: java.lang.ClassNotFoundException: io.netty.internal.tcnative.SSLPrivateKeyMethod
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
	at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
	... 3 more
```

We should do better here.

Modifications:

Call `OpenSsl.ensureAvailability()` before trying to construct OpenSsl*Context instances

Result:

More clear error message like:

```
java.lang.UnsatisfiedLinkError: failed to load the required native library

	at io.netty.handler.ssl.OpenSsl.ensureAvailability(OpenSsl.java:540)
	at io.netty.handler.ssl.SslContext.newClientContextInternal(SslContext.java:830)
	at io.netty.handler.ssl.SslContextBuilder.build(SslContextBuilder.java:611)
        ...
Caused by: java.lang.ClassNotFoundException: io.netty.internal.tcnative.SSLContext
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
	at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
	at java.base/java.lang.Class.forName0(Native Method)
	at java.base/java.lang.Class.forName(Class.java:398)
	at io.netty.handler.ssl.OpenSsl.<clinit>(OpenSsl.java:126)
	... 68 more
```
2021-08-19 20:55:53 +02:00
Chris Vest e9893ad021
Backport some fixes and cleanups for DefaultPromiseTest (#11600)
Motivation:
These cleanups were done in another PR but were not directly related to that PR.
This extracts those changes and backports them to 4.1.

Modification:
* Remove the use of mocking in DefaultPromiseTest.
* Fix a few warnings.
* Make `testStackOverFlowChainedFuturesB` test with the right listener chain.

Result:
Cleaner code.
2021-08-19 13:28:32 +02:00
skyguard1 9f4e155e36
Add Zstd.isAvailable() check in ZstdOptions (#11597)
Motivation:

At present, the verification methods of `ZstdOptions` and `BrotliOptions` are not consistent, and the processing methods of `ZstdOptions` and `BrotliOptions` in `HttpContentCompressor` are also inconsistent.
The http2 module does not add zstd-jni dependency, so `ClassNotFoundException` may be thrown

Modification:

Added `Zstd.isAvailable()` check in `ZstdOptions` to be consistent, and added zstd-jni dependency in http2 module

Result:

The verification methods of `ZstdOptions` and `BrotliOptions` are consistent, and `ClassNotFoundException` will not be thrown


Signed-off-by: xingrufei <xingrufei@sogou-inc.com>
2021-08-19 11:10:09 +02:00
Jeremy Kuhn 93484071d6
Fix support for optional encoders errors in HttpContentCompressor (#11582)
Motivation:

- Fix `HttpContentCompressor` errors due to missing optional compressor libraries such as Brotli and Zstd at runtime.
- Improve support for optional encoders by only considering the `CompressionOptions` provided to the constructor and ignoring those for which the encoder is unavailable.

Modification:

The `HttpContentCompressor` constructor now only creates encoder factories for the CompressionOptions passed to the constructor when the encoder is available which must be checked for Brotli and Zstd. In case of Brotli, it is not possible to create BrotliOptions if brotly4j is not available so there's actually nothing to check. In case of Zstd, I had to create class `io.netty.handler.codec.compression.Zstd` similar to `io.netty.handler.codec.compression.Brotli` which is used to check that zstd-jni is availabie at runtime.

The `determineEncoding()` method had to change as well in order to ignore encodings for which there's no `CompressionEncoderFactory` instance.

When the HttpContentCompressor is created using deprecated constructor (ie. with no CompressionOptions), we consider all available encoders.

Result:

Fixes #11581.
2021-08-19 08:40:25 +02:00
Chris Vest 201f6b6bb2
Remove dead code and fix warnings in the http2 module (#11593)
Motivation:
Opportunities for clean up found while working on a different PR.

Modification:
* Dead code has been removed.
* Unnecessary parenthesis, qualifiers, etc. removed.
* Unused imports removed.
* Override annotations added where missing.

Result:
Cleaner code
2021-08-18 20:43:57 +02:00
Chris Vest 322f75c5a2
Fix a problem with IP protocol version confusion on MacOS when TCP FastOpen is enabled (#11588)
Motivation:
This fixes a bug that would result in an `io.netty.channel.unix.Errors$NativeIoException: connectx(..) failed: Address family not supported by protocol family` error.
This happens when the connecting socket is configured to use IPv6 but the address being connected to is IPv4.
This can occur because, for instance, Netty and `InetAddress.getLoopbackAddress()` have different preferences for IPv6 vs. IPv4.

Modification:
Pass the correct ipv6 or ipv4 flags to connectx, depending on whether the socket was created for AF_INET or AF_INET6, rather than relying on the IP version of the destination address.

Result:
No more issue with TCP FastOpen on MacOS when using addresses of the "wrong" IP version.
2021-08-18 20:43:01 +02:00
Norman Maurer b82cb34c64
Fix IndexOutOfBoundsException caused by consuming the buffer multiple times in DatagramDnsQueryDecoder (#11592)
Motivation:

ccef8feedd introduced some changes to share code but did introduce a regression when decoding queries.

Modifications:

- Correctly only decode one time.
- Adjust unit test

Result:

Fixes https://github.com/netty/netty/issues/11591
2021-08-18 14:57:57 +02:00
Norman Maurer 228a01e268
Use StandardSocketOptions#IP_MULTICAST_IF as default source when joing multicast groups (#11585)
Motivation:

We should use StandardSocketOptions#IP_MULTICAST_IF as default source when joing multicast groups and only try to use the localAddress if this returns null.

Modifications:

First check if StandardSocketOptions#IP_MULTICAST_IF was set and if so use the network interface when joining mulicast groups

Result:

Fixes https://github.com/netty/netty/issues/11541
2021-08-18 12:11:48 +02:00
Norman Maurer cafc669436
Add CI job for JDK17 (#11584)
Motivation:

As JDK17 is really close to be released we should add a CI job for it to ensure netty works correctly when using it.

Modifications:

Add docker config and workflow config to run CI job for JDK17

Result:

Ensure netty works on JDK17 as well
2021-08-18 10:08:49 +02:00
Norman Maurer 69f0de223f
Add cast to help compiler to build (#11587)
Motivation:

Sometimes intellij fails to build because the compiler cant really figure out what to do. We can workaround this by adding some explicit casts.

Modifications:

Add explicits casts

Result:

No more problems with intellij
2021-08-18 09:21:19 +02:00
Norman Maurer 794c45fd59
Add some docs for io.netty.leakDetection.acquireAndReleaseOnly. (#11580)
Motivation:

We should add some docs / comment to explain what io.netty.leakDetection.acquireAndReleaseOnly is all about.

Modifications:

Add some comments

Result:

Fixes https://github.com/netty/netty/issues/11576.
2021-08-17 09:09:50 +02:00
Norman Maurer ea932dd706
Ensure we only log message on BoringSSL when the ciphers really are not the default (#11583)
Motivation:

0c9a86db81 added a change to log a message if someone tried to change the TLSv1.3 ciphers when using BoringSSL. Unfortunally the code had some error and so even if the user did not change these we logged something.

Modifications:

- Ensure there are no duplicates in the ciphers
- Correctly take TLSv1.3 extra ciphers into account when using BoringSSL

Result:

Correctly log or not log
2021-08-16 22:23:25 +02:00
Netty Project Bot f8f17f676d [maven-release-plugin] prepare for next development iteration 2021-08-16 08:23:23 +00:00
Netty Project Bot b785fa9eed [maven-release-plugin] prepare release netty-4.1.67.Final 2021-08-16 08:23:20 +00:00
Norman Maurer 7903a44a14 Revert "[maven-release-plugin] prepare release netty-4.1.67.Final"
This reverts commit f773733af6.
2021-08-16 10:14:09 +02:00
Norman Maurer ec23189e72 Revert "[maven-release-plugin] prepare for next development iteration"
This reverts commit 76c55caeb9.
2021-08-16 10:13:59 +02:00
Netty Project Bot 76c55caeb9 [maven-release-plugin] prepare for next development iteration 2021-08-12 17:12:05 +00:00
Netty Project Bot f773733af6 [maven-release-plugin] prepare release netty-4.1.67.Final 2021-08-12 17:12:01 +00:00
DD 056eba4db4
server h2c upgrade fail when request doesn't have connection header (#11569)
__Motivation__
Since request.headers().getAll() will never return null. And the check null condition will not work as expected.

__Modification__

Add isEmpty() checking as well.

__Result__

Fixes #11568
2021-08-12 09:04:18 -07:00
Chris Vest f750e2eb6a
Add support for client-side TCP FastOpen to KQueue MacOS (#11560)
Motivation:
The MacOS-specific `connectx(2)` system call make it possible to establish client-side connections with TCP FastOpen.

Modification:
Add support for TCP FastOpen to the KQueue transport, and add the `connectx(2)` system call to `BsdSocket`.

Result:
It's now possible to use TCP FastOpen when initiating connections on MacOS.
2021-08-12 13:38:46 +02:00
Chris Vest 699549dcbc
Improve test failure reporting of EpollSocketChannelConfigTest (#11570)
Motivation:
This test is inherently flaky due to file descriptor reuse.
Even though we have taken steps to make it less flaky, it still fails sometimes.
When it does, the error message is not very helpful.

Modification:
Make use of assertThrows and assertThat to get more descriptive error messages when the tests fail.

Result:
More meaningful messages on test failures, which may help us make the tests more resilient in the future
2021-08-12 11:37:56 +02:00
Aayush Atharva beb4588b4a
Inline variables to make code more readable (#11565)
Motivation:
There are lots of redundant variable declarations which should be inlined to make good look better.

Modification:
Made variables inlined.

Result:
Less redundant variable and more readable code.
2021-08-11 17:05:48 +02:00
skyguard1 3c27e008b6
Some small improvements (#11564)
Signed-off-by: xingrufei <xingrufei@sogou-inc.com>
2021-08-11 13:40:21 +02:00
dependabot[bot] 3f8dab5ad0
Bump commons-compress from 1.19 to 1.21 (#11539)
Bumps commons-compress from 1.19 to 1.21.

---
updated-dependencies:
- dependency-name: org.apache.commons:commons-compress
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-08-10 10:53:15 -07:00
dependabot[bot] ea45158c97
Bump ant from 1.10.9 to 1.10.11 (#11538)
Bumps ant from 1.10.9 to 1.10.11.

---
updated-dependencies:
- dependency-name: org.apache.ant:ant
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-08-10 10:49:36 -07:00
Chris Vest 32523c5337
Make the TCP_FASTOPEN channel option transport agnostic (#11559)
Motivation:
TCP FastOpen is a pure optimisation, that is opportunistically applied.
There is no reason to make it specific to the epoll transport, and in the future we could add support to other transports.
Besides, the client-side equivalent, TCP_FASTOPEN_CONNECT, is already transport agnostic.

Modification:
Move the TCP_FASTOPEN channel option from EpollChannelOption to ChannelOption.
Mark the field in EpollChannelOption as deprecated.

Result:
All channel options related to TCP FastOpen are now transport agnostic.
However, we still only actually support TFO on the epoll transport.
2021-08-10 09:03:05 +02:00