Motivation:
We can remove some classes and duplication if we add default methods
Modifications:
- Add default methods to EventExecutor / EventExecutorGroup / EventLoop / EventLoopGroup
- Remove code duplication
- Remove AbstractEventExecutorGroup as it is not needed anymore
Result:
Cleanup and removal of code-duplication. Also makes it easier for people to implement their custom executors / groups
__Motivation__
As we start to migrate codecs to use the new `Buffer` API, we need a way for them to get a handle of `BufferAllocator`.
__Modification__
Added `bufferAllocator()` method to `ChannelConfig`, `Channel` and `ChannelHandlerContext`
__Result__
Codecs can allocate `Buffer` instances
Motivation:
The way how an EventLoopGroup is created for a specific transport has changed. We should remove all the old deprecated implementations and change all our code to use the new way how to init groups
Modifications:
- Remove LocalEventLoopGroup, NioEventLoopGroup, EpollEventLoopGroup and KQueueEventLoopGroup
- Adjust code to use the new way how to setup EventLoopGroups
Result:
Remove deprecate classes and usages
Motivation:
When users receive "Maximum active streams violated for this endpoint"
exception, it's useful to know what is the current max streams limit on
HTTP/2 connection.
Modifications:
- Include current number of maximum active streams in exception message;
Result:
Easier debugging of HTTP/2 connections.
Motivation:
When the ByteBuf size exceeds the max limit/defined size, IOException is thrown.
HttpData#addContent/setContent should release the buffer in such cases otherwise memory leak will happen.
Modification:
- Release the provided ByteBuf before throwing IOException
- Add unit tests
Result:
Fixes#11618
Motivation:
There are some redundant imports and unnecessary type cast
Modification:
Remove useless imports and unnecessary type cast
Result:
The code is cleaner than original
Signed-off-by: xingrufei <xingrufei@sogou-inc.com>
Motivation:
Exception logged by `onHttp2UnknownStreamError` is propagated to the
upper layers anyway. Receiver of the exception is responsible for
correct handling. Inside netty, it's enough to log at `DEBUG` level.
Modifications:
- `Http2FrameCodec#onHttp2UnknownStreamError` always logs at `DEBUG`
level;
Result:
Less noise in logs when `Http2UnknownStreamError` is properly handled by
upper layers.
Co-authored-by: Norman Maurer <norman_maurer@apple.com>
Co-authored-by: Chris Vest <mr.chrisvest@gmail.com>
Motivation:
While we use DefaultPromise as our implementation of Promise and Future users should not really use it directly. Users should always use the EventExecutor / EventLoop to create a Promise / Future.
Modifications:
- Change static Promise methods to be package-private
- Add default implementations for Promise and Future creation to EventExecutor
- Change public constructor to protected
- Remove usage of DefaultPromise in our tests
Result:
Less likely users will depend on the actual Promise implementation
Motivation:
We wish to separate these two into clearer write/read interfaces.
In particular, we don't want to be able to add listeners to promises, because it makes it easy to add them out of order.
We can't prevent it entirely, because any promise can be freely converted to a future where listeners can be added.
We can, however, discourage this in the API.
Modification:
The Promise interface no longer extends the Future interface.
Numerous changes to make the project compile and its tests run.
Result:
Clearer separation of concerns in the code.
Motivation:
Enlarging buffers approaching 4 MiB size requires n iterations
Modification:
Use a single instruction to compute the next buffer capacity
Result:
Faster/Simpler calculateNewCapacity
Motivation:
Last time when we tried to do a release it failed due the fact that no javadoc jar was generated. Beside this how we did configure the shading module did introduce a lot of duplication
Modifications:
- Generate an empty javadoc jar
- Share config for shading plugin
Result:
No more problems during release process
Motivation:
If we don't need the scheduled future, then it will be one less complication when we change Netty Future to no longer extend JDK Future.
It would also result in fewer elements in our API.
Modification:
There was only one real use of ScheduledFuture in our code, in Cache.
This has been changed to wrap an ordinary future with a deadline for implementing the Delayed interface.
All other places were effectively overspecifying by relying on ScheduledFuture.
A few places were also specifying JDK Futures - these have been changed to specify Netty Futures.
Result:
Reduced dependency on the ScheduledFuture interfaces.
Motivation:
Usually the outbound operation should start at the "current" ChanneöHandlercontext which was often not the case
Modifications:
Use the ChannelHandlerContext for closing the connection
Result:
Start the operation on the right position of the pipeline
Motivation:
There is a memory leak while writing empty data frame with padding.
The empty data frame was occurred because we are running a proxy server
built by netty, and found that google services always sent data frames
followed by an empty data frame.
Modifications:
Calls the ctx.write even the payload is empty.
Result:
Fix memory leak.
Co-authored-by: Norman Maurer <norman_maurer@apple.com>
Motivation:
DiskFileUpload#toString should use the value provided when constructing DiskFileUpload
and not the default one
Modification:
- DiskFileUpload#toString is modified to use the correct deleteOnExit value
Result:
DiskFileUpload#toString returns a string that uses the correct deleteOnExit
Motivation:
When we override a test with params we also need to ensure we add the correct annotations to the override
Modifications:
Add the correct annotations so the tests are actually run in intellij
Result:
Each test can be run
Motivation:
While OpenSSK is provided support for the "RSASSA-PSS" algorithm this was still not valid from netty. Which was causing issue in validating certificates which was signed using this algorithm.
Modification:
Added "RSASSA-PSS" in LOCAL_SUPPORTED_SIGNATURE_ALGORITHMS.
validation:
Validated and tested with CA and User cert singed with RSASSA-PSS algorithm.
Result:
Fixes#11360
Co-authored-by: Norman Maurer <norman_maurer@apple.com>
Motivation:
We should not use master as branch name, better to switch to main
Modifications:
Replace master branch name with main
Result:
Use main as replacement for master
Motivation:
We did recently change the Channel / ChannelHandler API to always act on the Future only. We should do the same for our handlers.
Modifications:
- Adjust http2 API
- Adjust other handlers API
Result:
Easier to use API and more consistent
Motivation:
Our current ChannelFutureListeners are too restrictive in terms of the type of the context. We should lift this a bit
Modifications:
Refactor ChannelFutureListeners to not be an enum and so allow most of its instances to be used with any ChannelOutboundInvoker.
Result:
More flexible usage
Motivation:
The need of cascade from a Future to a Promise exists. We should add some default implementation for it.
Modifications:
- Merge PromiseNotifier into Futures
- Add default cascadeTo(...) methods to Future
- Add tests to FuturesTest
- Replace usage of PromiseNotifier with Future.cascadeTo
- Use combination of map(...) and cascadeTo(...) in *Bootstrap to reduce code duplication
Result:
Provide default implementation of cascadeTo.
Motivation:
In this issue(#11578 ) discussed that tilde should not be encoded in QueryStringEncoder, this pr is to fix this problem
Modification:
Modified QueryStringEncoder so that it does not encode tilde, and added a test case
Result:
Fixes#11578
Motivation:
Making futures easier to compose, combine, and extend is useful to have as part of the API, since implementing this correctly and efficiently can be tricky.
Modification:
Add `Future.map(Function<V,R>) -> Future<R>` and `Future.flatMap(Function<V,Future<R>>) -> Future<R>` default methods to the `Future` interface.
These methods return new Future instance, that will be completed when the original future completes, and the result will be processed through the given mapping function.
These two methods take care to propagate cancellation and exceptions correctly:
Cancellation propagates both ways between the new and original future.
Failures only propagate from the original future to the returned new Future instance.
Result:
A few convenient methods for modifying and composing futures.
This PR fixes#8523, and perhaps also #2105
Motiviation:
The workflow from the default branch is the one that gets to run for all PRs, regardless of what branch they are targeting.
This means the 4.1 PR Reports workflow needs to tollerate master branch PRs, where there is no Java 8 builds.
Modification:
Make the PR Reports tolerate that the Java 8 matrix fails if they are missing.
Result:
Hopefully we'll get working PR Reports for master branch PRs now.
Motivation:
The expression "not is success" can mean that either the future failed, or it has not yet completed.
However, many places where such an expression is used is expecting the future to have completed.
Specifically, they are expecting to be able to call `cause()` on the future.
It is both more correct, and semantically clearer, to call `isFailed()` instead of `!isSuccess()`.
Modification:
Change all places that used `!isSuccess()` to mean that the future had failed, to use `isFailed()`.
A few places are relying on `isSuccess()` returning `false` for _incomplete_ futures, and these places have been left unchanged.
Result:
Clearer code, with potentially fewer latent bugs.
Motivation:
We did miss to update some code that setup the EventLoop mock and so did see test-failures due NPE's
Modifications:
Correctly mock the newPromise() method.
Result:
No more test-failures
Motivation:
232c669fa4 did add some overflow protection but did not handle null elements in the array the same as before.
Modifications:
- Break the loop if a null element was found
- Add unit test
Result:
Fixes https://github.com/netty/netty/issues/11612
Motivation:
232c669fa4 did add overflow protection but did miss to take the array offset into account and so could report false-positives
Modifications:
- Correctly take offset into account when check for overflow.
- Add unit tests
Result:
Correctly take offset into account when overflow check is performed
* Remove deprecated Channel*Handler* classes
Motivation:
There is no need to keep the older adapter and duplex classes around.
Modifications:
- Remove old adapter and duplex classes
- Adjust javadocs
Result:
Cleanup
* Address nit
Motivation:
We should just add `executor()` to the `ChannelOutboundInvoker` interface and override this method in `Channel` to return `EventLoop`.
Modifications:
- Add `executor()` method to `ChannelOutboundInvoker`
- Let `Channel` override this method and return `EventLoop`.
- Adjust all usages of `eventLoop()`
- Add some default implementations
Result:
API cleanup
Motivation:
This workflow has been broken for a couple of months now.
It would be nice if we could get junit test reports back in out PR checks.
Modificatoin:
Fix a test matrix typo.
Update workflow action versions.
Result:
Hopefully the PR Reports workflow starts working again.
Draft until it actually does work. There are probably several things wrong that will take time to work through.
Motivation:
At the moment the outbound operations of ChannelHandler take a Promise as argument. This Promise needs to be carried forward to the next handler in the pipeline until it hits the transport. This is API choice has a few quirks which we should aim to remove:
- There is a difference between if you add a FutureListener to the Promise or the Future that is returned by the outbound method in terms of the ordering of execution of the listeners. Sometimes we add the listener to the promise while in reality we usually always want to add it to the future to ensure the listerns are executed in the "correct order".
- It is quite easy to "loose" a promise by forgetting to use the right method which also takes a promise
- We have no idea what EventExecutor is used for the passed in Promise which may invalid our assumption of threading.
While changing the method signature of the outbound operations of the ChannelHandler is a good step forward we should also take care of just remove all the methods from ChannelOutboundInvoker (and its sub-types) that take a Promise and just always use the methods that return a Future only.
Modifications:
- Change the signature of the methods that took a Promise to not take one anymore and just return a Future
- Remove all operations for ChannelOutboundInvoker that take a Promise.
- Adjust all code to cope with the API changes
Result:
Cleaner API which is easier to reason about and easier to use.
Motivation:
Keeping the version of netty-tcnative correct can sometimes be hard.
Modifications:
Add entries for netty-tcnative* to the bom
Result:
Fixes https://github.com/netty/netty/issues/11567
Motivation:
Since most futures in Netty are of the `Void` type, methods like `getNow()` and `cause()` cannot distinguish if the future has finished or not.
This can cause data race bugs which, in the case of `Void` futures, can be silent.
Modification:
The methods `getNow()` and `cause()` now throw an `IllegalStateException` if the future has not yet completed.
Most use of these methods are inside listeners, and so are not impacted.
One place in `AbstractBootstrap` was doing a racy read and has been adjusted.
Result:
Data race bugs around `getNow()` and `cause()` are no longer silent.
Motivation:
We can make things easier for implementations by providing some default methods
Modifications:
- Add default methods to Channel
- Remove code from AbstractChannel
Result:
Easier to implement custom Channel
Motivation:
We should allow server initiated renegotiation when OpenSSL / BoringSSL bases SSLEngine is used as it might be used for client auth.
Modifications:
- Upgrade netty-tcnative version to be able to allow renegotiate once
- Adjust code
Result
Fixes https://github.com/netty/netty/issues/11529
Motivation:
Make SslHandler's handlerRemoved0 method release the sslEngine
even if it fails in the middle.
See details in https://github.com/netty/netty/issues/11595.
Modifications:
Wrap the release of sslEngine into a finally block.
Result:
The sslEngine would be released eventually.
Co-authored-by: Chen Liu <cliu@splunk.com>
Motivation:
We tightly integrate the TrustManger and KeyManager into our native SSL implementation which means that both of them need to support TLSv1.3 as protocol. This is not always the case and so can produce runtime exceptions.
As TLSv1.3 support was backported to Java8 quite some time now we should be a bit more conservative and only enable TLSv1.3 for our native implementation if the JDK implementation supports it as well. This also allows us to remove some hacks we had in place to be able to support it before in Java8.
Modifications:
- Only enable TLSv1.3 support for our native SSL implementation when the JDK supports it as well
- Remove OpenSslTlsv13X509ExtendedTrustManager as its not needed anymore
Result:
Fixes https://github.com/netty/netty/issues/11589
Motivation:
The generics for the existing futures, promises, and listeners are too complicated.
This complication comes from the existence of `ChannelPromise` and `ChannelFuture`, which forces listeners to care about the particular _type_ of future being listened on.
Modification:
* Add a `FutureContextListener` which can take a context object as an additional argument. This allows our listeners to have the channel piped through to them, so they don't need to rely on the `ChannelFuture.channel()` method.
* Make the `FutureListener`, along with the `FutureContextListener` sibling, the default listener API, retiring the `GenericFutureListener` since we no longer need to abstract over the type of the future.
* Change all uses of `ChannelPromise` to `Promise<Void>`.
* Change all uses of `ChannelFuture` to `Future<Void>`.
* Change all uses of `GenericFutureListener` to either `FutureListener` or `FutureContextListener` as needed.
* Remove `ChannelFutureListener` and `GenericFutureListener`.
* Introduce a `ChannelFutureListeners` enum to house the constants that previously lived in `ChannelFutureListener`. These constants now implement `FutureContextListener` and take the `Channel` as a context.
* Remove `ChannelPromise` and `ChannelFuture` — all usages now rely on the plain `Future` and `Promise` APIs.
* Add static factory methods to `DefaultPromise` that allow us to create promises that are initialised as successful or failed.
* Remove `CompleteFuture`, `SucceededFuture`, `FailedFuture`, `CompleteChannelFuture`, `SucceededChannelFuture`, and `FailedChannelFuture`.
* Remove `ChannelPromiseNotifier`.
Result:
Cleaner generics and more straight forward code.
Motivation:
At the moment why you try to build a SslContext via SslProvider.OPENSSL* and netty-tcnative* is not on the classpath it will fail with:
```
Exception in thread "main" java.lang.NoClassDefFoundError: io/netty/internal/tcnative/SSLPrivateKeyMethod
at io.netty.handler.ssl.SslContext.newClientContextInternal(SslContext.java:830)
at io.netty.handler.ssl.SslContextBuilder.build(SslContextBuilder.java:611)
at io.netty.handler.codec.http.Main.main(Main.java:34)
Caused by: java.lang.ClassNotFoundException: io.netty.internal.tcnative.SSLPrivateKeyMethod
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
... 3 more
```
We should do better here.
Modifications:
Call `OpenSsl.ensureAvailability()` before trying to construct OpenSsl*Context instances
Result:
More clear error message like:
```
java.lang.UnsatisfiedLinkError: failed to load the required native library
at io.netty.handler.ssl.OpenSsl.ensureAvailability(OpenSsl.java:540)
at io.netty.handler.ssl.SslContext.newClientContextInternal(SslContext.java:830)
at io.netty.handler.ssl.SslContextBuilder.build(SslContextBuilder.java:611)
...
Caused by: java.lang.ClassNotFoundException: io.netty.internal.tcnative.SSLContext
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
at java.base/java.lang.Class.forName0(Native Method)
at java.base/java.lang.Class.forName(Class.java:398)
at io.netty.handler.ssl.OpenSsl.<clinit>(OpenSsl.java:126)
... 68 more
```
Motivation:
These cleanups were done in another PR but were not directly related to that PR.
This extracts those changes and backports them to 4.1.
Modification:
* Remove the use of mocking in DefaultPromiseTest.
* Fix a few warnings.
* Make `testStackOverFlowChainedFuturesB` test with the right listener chain.
Result:
Cleaner code.
Motivation:
At present, the verification methods of `ZstdOptions` and `BrotliOptions` are not consistent, and the processing methods of `ZstdOptions` and `BrotliOptions` in `HttpContentCompressor` are also inconsistent.
The http2 module does not add zstd-jni dependency, so `ClassNotFoundException` may be thrown
Modification:
Added `Zstd.isAvailable()` check in `ZstdOptions` to be consistent, and added zstd-jni dependency in http2 module
Result:
The verification methods of `ZstdOptions` and `BrotliOptions` are consistent, and `ClassNotFoundException` will not be thrown
Signed-off-by: xingrufei <xingrufei@sogou-inc.com>
Motivation:
- Fix `HttpContentCompressor` errors due to missing optional compressor libraries such as Brotli and Zstd at runtime.
- Improve support for optional encoders by only considering the `CompressionOptions` provided to the constructor and ignoring those for which the encoder is unavailable.
Modification:
The `HttpContentCompressor` constructor now only creates encoder factories for the CompressionOptions passed to the constructor when the encoder is available which must be checked for Brotli and Zstd. In case of Brotli, it is not possible to create BrotliOptions if brotly4j is not available so there's actually nothing to check. In case of Zstd, I had to create class `io.netty.handler.codec.compression.Zstd` similar to `io.netty.handler.codec.compression.Brotli` which is used to check that zstd-jni is availabie at runtime.
The `determineEncoding()` method had to change as well in order to ignore encodings for which there's no `CompressionEncoderFactory` instance.
When the HttpContentCompressor is created using deprecated constructor (ie. with no CompressionOptions), we consider all available encoders.
Result:
Fixes#11581.
Motivation:
The `HttpContentCompressor.beginEncode()` method has too many if else, so consider refactoring
Modification:
Create the corresponding `CompressionEncoderFactory` according to the compression algorithm, remove the if else
Result:
The code of `HttpContentCompressor` is cleaner than the previous implementation
Signed-off-by: xingrufei <xingrufei@sogou-inc.com>
Co-authored-by: Norman Maurer <norman_maurer@apple.com>
Motivation:
Opportunities for clean up found while working on a different PR.
Modification:
* Dead code has been removed.
* Unnecessary parenthesis, qualifiers, etc. removed.
* Unused imports removed.
* Override annotations added where missing.
Result:
Cleaner code
Motivation:
This fixes a bug that would result in an `io.netty.channel.unix.Errors$NativeIoException: connectx(..) failed: Address family not supported by protocol family` error.
This happens when the connecting socket is configured to use IPv6 but the address being connected to is IPv4.
This can occur because, for instance, Netty and `InetAddress.getLoopbackAddress()` have different preferences for IPv6 vs. IPv4.
Modification:
Pass the correct ipv6 or ipv4 flags to connectx, depending on whether the socket was created for AF_INET or AF_INET6, rather than relying on the IP version of the destination address.
Result:
No more issue with TCP FastOpen on MacOS when using addresses of the "wrong" IP version.
