Motivation:
HttpServerUpgradeHandler takes a list of protocols from an incoming
request and uses them for building a response.
Although the class does some validation while parsing the list,
it then disables HTTP header validation when it builds a responst.
The disabled validation may potentially allow
HTTP response splitting attacks.
Modifications:
- Enabled HTTP header validation in HttpServerUpgradeHandler
as a defense-in-depth measure to prevent possible
HTTP response splitting attacks.
- Added a new constructor that allows disabling the validation.
Result:
HttpServerUpgradeHandler validates incoming protocols
before including them into a response.
That should prevent possible HTTP response splitting attacks.
Motivation:
Subscription ID property of the PUBLISH message may be repeated multiple times, which wasn't taken into account when developing `MqttProperties` API.
Modification:
Store Subscription ID properties separately from others - in `MqttProperties.subscriptionIds`.
Add `MqttProperties.getProperties` method to retrieve properties that may be repeated.
Change internal representation of User Properties for uniformity with Subscription ID - now they're stored in `MqttProperties.userProperties` rather than the common hash map.
Result:
Multiple Subscription ID properties can be set or retrieved.
Motivation:
a63faa4fa1 missed to update the macos specific c files for the resolver.
Modifications:
Fix up c files so it compiles again on macos
Result:
Compiles on macos again
Motivation:
We had a lot of duplication in our jni code which was mostly due macros but also related to how we support shading. By using netty-jni-util we can share all the code between netty and netty-tcnative ( and possible other jni based netty projects in the future).
Modifications:
- Use netty-jni-util and re-use its macros / functions
- Remove duplicated code
- Adjust build files
Result:
Less code duplication for JNI
Motivation:
`addHttp2ToHttpHeaders(int streamId, Http2Headers sourceHeaders, FullHttpMessage destinationMessage, boolean addToTrailer)`
should match
`addHttp2ToHttpHeaders(int streamId, Http2Headers inputHeaders, HttpHeaders outputHeaders, HttpVersion httpVersion, boolean isTrailer, boolean isRequest)`.
However, the `Http2Headers` variable name is different.
Modification:
Changed `sourceHeaders` to `inputHeaders`.
Result:
Variable and JavaDoc naming now correct.
Motivation:
Http2ToHttpHeaderTranslator is a private class but translateHeaders(Iterable<Entry<CharSequence, CharSequence>>) is public but it should be package-private.
Modification:
Removed public.
Result:
Correct access modifer.
Motivation:
Some buffers implement ByteBuf#order(order) by wrapping themselves in a SwappedByteBuf.
The SwappedByteBuf is then responsible for swapping the byte order on accesses.
The explicitly little-endian accessor methods, however, should not be swapped to big-endian, but instead remain explicitly little-endian.
Modification:
The SwappedByteBuf was passing through calls to e.g. writeIntLE, to the big-endian equivalent, e.g. writeInt.
This has been changed so that these calls delegate to their explicitly little-endian counterpart.
Result:
This makes all buffers that make use of SwappedByteBuf for their endian-ness configuration, consistent with all the buffers that use other implementation strategies.
In the end, all buffers now behave exactly the same, when using their explicitly little-endian accessor methods.
Motivation:
ddebc1027d missed to make Errors.throwConnectException(...) public
Modifications:
Make method public
Result:
Be able to use Errors.throwConnectException(...) from other module
Motivation:
03aafb9cff did ensure we fail while loading a natibe library which is not compatible. While this is great it is still sometimes hard for people to understand what NoSuchMethodError means in this context.
Modifications:
If possible rethrow the NoSuchMethodError and provide some more hints about multiple versions of the shared library
Result:
Easier to understand for people why loading fails
Motivation:
ddebc10 added a few adjustments that are needed by io_uring that we will add as an incubator repository. Unfortunally we missed the changed in PlatformDependent*.
Modifications:
Add missing methods
Result:
Be able to compile io_uring code against core netty
Motivation:
During the last few month we did develop an io_uring based transport which shows very promising performance numbers. To give it more time to bake we will develop it outside of netty in an "incubator" module which will make it clear to users what to expect and also allow us to seperate its release cycle. While the implementation of it is very self contained there are few small adjustments that need to be made in netty itself to allow us to reuse code.
Modifications:
- AbstractChannel: Add method which can be used when a write fails and remove final from one method
- IovArray: Allow to create an IovArray from a ByteBuf instance
- FileDescriptor: Allow to reuse mark close logic via sub-class
Result:
Be able to reuse netty core classes in io_uring incubator repository
Motivation:
03aafb9cff did ensure we unregister all native methods when we unload / or fail during load of the native library. Unfortunally it missed to unregister in one case for kqueue.
Modifications:
Add unregister calls to the unload function
Result:
Correctly unregister in all cases
Motivation:
Conscrypt 2.5.1 is available and it's a good idea to upgrade to the latest version.
Modification:
Upgraded Conscrypt 2.4.0 to 2.5.1
Result:
Newer Conscrypt version.
Motivation:
FingerprintTrustManagerFactory can only use SHA-1 that is considered
insecure.
Modifications:
- Updated FingerprintTrustManagerFactory to accept a stronger hash algorithm.
- Deprecated the constructors that still use SHA-1.
- Added a test for FingerprintTrustManagerFactory.
Result:
A user can now configure FingerprintTrustManagerFactory to use a
stronger hash algorithm.
The compiler shows a warning if the code still uses one of the
unsafe constructors.
Co-authored-by: Norman Maurer <norman_maurer@apple.com>
Motivation:
It's important to unload all previous registered native methods when there is a failure during loading the native lib. Failing to do so may lead to an "invalid state" and so may segfault the JVM when trying to call a native method that was previous loaded.
This was observed when two versions of netty-tcnative were on the classpath which had different requirements in terms of linking.
Something like this was reported in he hs log:
```
Instructions: (pc=0x0000000116413bf0)
0x0000000116413bd0:
[error occurred during error reporting (printing registers, top of stack, instructions near pc), id 0xb]
Register to memory mapping:
RAX=0x0000000116413bf0 is an unknown value
RBX={method} {0x000000011422e708} 'aprMajorVersion' '()I' in 'io/netty/internal/tcnative/Library'
RCX=0x000000000000000a is an unknown value
RDX=0x000000000000000a is an unknown value
```
Modifications:
- Unregister previous registered native methods on failure
- Unregister previous registered native methods on on unload of the native lib
Result:
No more segfault caused by invalid state when loading of the native lib fails in between. In this case the user will receive an error now like:
Motivation:
SLF4J 1.7.30 is the latest version in 1.7.x and we should upgrade to it from 1.7.21.
Modification:
Changed 1.7.21 to 1.7.30
Result:
Newer version of SLF4J
Motivation:
We should preferable always release the message before we notify the promise. Thhis has a few advantages:
- Release memory as soon as possible
- Listeners observe the "more correct" reference count
Modifications:
Release message before fail the promises
Result:
Faster releasing of resources. This came up in https://github.com/netty/netty/issues/10723
Motivation:
Since GraalVM version 19.3.0, instances of java.net.InetAddress (and its subclasses Inet4Address and Inet6Address) are not allowed in native image heap (that is, they cannot be stored in static fields of classes initialized at build time or be reachable through static fields of such classes). When building a native image, it makes sense to initialize at build time as many classes as possible.
But some fields of some classes in Netty (for example, NetUtil.LOCALHOST4) contain InetAddress instances. If a program is using code path that makes it possible to reach such fields at build time initialization, it becomes impossible to build a native image initializing core Netty classes initialized at runtime. An example of such a program is a client that uses netty-dns.
Modifications:
- Add netty-testsuite-native-image-client Maven module to test that such an example program can be built after the corresponding fixes
- Add native-image.properties to resolver-dns module to move initialization of some classes to runtime (some of them are parsing configuration during initialization, so it makes no sense to initialize them at build time; for others, it's needed to avoid InetAddress reachability at build time)
- Add substitutions for NetUtil.LOCALHOST4, NetUtil.LOCALHOST6 and NetUtil.LOCALHOST to overcome the InetAddress-related prohibition
- Extract some initialization code from NetUtil to NetUtilInitializations to allow it to be used by the substitutions
Result:
A client program using netty-dns with --initialize-at-build-time=io.netty builds successfully
Motivation:
Thread.stop() works by producing a ThreadDeath error in the target thread. EventLoops swallow all Throwables, which makes them effectively unkillable. This is effectively a memory leak, for our application. Beside this we should also just regrow all `Error` as there is almost no way to recover.
Modification:
Edit the EventLoops that swallow Throwables to instead rethrow Error.
Result:
`EventLoop` can crash if `Error` is thrown
Motivation:
At the moment we have only one base `WebSocketHandshakeException` for handling WebSocket upgrade issues.
Unfortunately, this message contains only a string message about the cause of the failure, which is inconvenient in handling.
Modification:
Provide new `WebSocketClientHandshakeException` with `HttpResponse` field and `WebSocketServerHandshakeException` with `HttpRequest` field both of them without content for avoid reference counting
problems.
Result:
More information for more flexible handling.
Fixes#10277#4528#10639.
Motivation:
Http2Headers has JavaDoc error which says Sets the {@link PseudoHeaderName#AUTHORITY} header or {@code null} if there is no such header however it should be Sets the {@link PseudoHeaderName#AUTHORITY} header in Http2Headers#authority(CharSequence) methods because it only sets CharSequence.
This is true for all setters in Http2Headers.
Modification:
Fixed all JavaDoc errors.
Result:
Better JavaDoc.
Motivation:
HTTP is a plaintext protocol which means that someone may be able
to eavesdrop the data. To prevent this, HTTPS should be used whenever
possible. However, maintaining using https:// in all URLs may be
difficult. The nohttp tool can help here. The tool scans all the files
in a repository and reports where http:// is used.
Modifications:
- Added nohttp (via checkstyle) into the build process.
- Suppressed findings for the websites
that don't support HTTPS or that are not reachable
Result:
- Prevent using HTTP in the future.
- Encourage users to use HTTPS when they follow the links they found in
the code.
Motivation:
In some situations we could have end up calling some functions with NULL parameters which in this case could lead to undefined behavior. All of this would have happened during loading of the native lib.
Modifications:
Add NULL check as guards and return early
Result:
Fix some possible undefined behavior
Motivation:
I was collecting stats for failed promises with a FixedChannelPool and I was bucketing by stats using cause.getSimpleName(). After #9152 was released, the introduction of the anonymous classes make getSimpleName() return "" causing confusion.
Modification:
Use named classes in the ChannelPool implementations. I made them private, but I can change that if you think otherwise.
Result:
The SimpleChannelPool fails the promises with a ChannelPoolFullException. The FixedChannelPool fails the promises with an AcquireTimeoutException. Also AcquireTimeoutException is more specific than just a plain TimeoutException, which is also useful for troubleshooting. If you want different class names, please advise.
Motivation:
DatagramDnsResponseDecoder should rethrow as CorruptedFrameException if an IndexOutOfBoundsException happens.
Modifications:
- Catch IndexOutOfBoundsException and rethrow as CorruptedFrameException
- Add a testcase
Result:
Less noise in the logs
Motivation:
DuplexChannel allow for half-closure, we should have a special config interface for it as well.
Modifications:
Add DuplexChannelConfig which allows to configure half-closure.
Result:
More consistent types
Motivation:
I noticed WebSocketServerExtensionHandler taking up a non-trivial
amount of CPU time for a non-websocket based menchmark. This attempts
to speed it up.
Modifications:
- It is faster to check for a 101 response than to look at headers,
so an initial response code check is done
- Move all the actual upgrade code into its own method to increase
chance of this method being inlined
- Add an extra contains() check for the upgrade header, to avoid
allocating an iterator if there is no upgrade header
Result:
A small but noticable performance increase.
Signed-off-by: Stuart Douglas <stuart.w.douglas@gmail.com>
Motivation:
junit deprecated Assert.assertThat(...)
Modifications:
Use MatcherAssert.assertThat(...) as replacement for deprecated method
Result:
Less deprecation warnings
Motivation:
Github now allows to run CodeQL during pull request verification. This allows to detect errors / security problems early.
Modification:
Add config
Result:
Fixes https://github.com/netty/netty/issues/10669
Co-authored-by: Artem Smotrakov <artem.smotrakov@sap.com>
Motivation:
We need to ensure we not leak in tests. We did see some leaks reported related to HaProxyMessageEncoderTest on our CI.
Modifications:
- Use readSlice(...) and so not create new ByteBuf instances that need to be released
Result:
No more leaks
Motivation:
We should use the latest patch releases when building via docker
Modifications:
Update all java versions to the latest patch release
Result:
Use latest releases
Motivation:
We can filter out `null` rules while initializing the instance of `RuleBasedIpFilter` so we don't have to keep checking for `null` rules while iterating through `rules` array in `for loop` which is just a waste of CPU cycles.
Modification:
Added `null` rule check inside the constructor.
Result:
No more wasting CPU cycles on check the `null` rule each time in `for loop` and makes the overall operation more faster.
Motivation:
LGTM reports multiple issues. They need to be triaged,
and real ones should be fixed.
Modifications:
- Fixed multiple issues reported by LGTM, such as redundant conditions,
resource leaks, typos, possible integer overflows.
- Suppressed false-positives.
- Added a few testcases.
Result:
Fixed several possible issues, get rid of false alarms in the LGTM report.
Motivation:
We need to also add -XX:+AllowRedefinitionToAddDeleteMethods for JDK15 and 16 as otherwise blockhound will not work
Modifications:
Add profiles for JDK15 and JDK16
Result:
Blockhound tests pass again
Motivation:
We had two unit tests that sometimes failed due timeouts. After insepecting these I noticed these can be improved to run faster while still do the right validation
Modifications:
- Only submit one task for execution per execute
- Cleanup
Result:
No test failures due timeout
Motivation:
OpenJDK15 was released, we should compile with it on the CI
Modifications:
Add docker-compose files to be able to compile with OpenJDK15
Result:
Compile with latest major JDK version
Motivation:
As the PooledByteBufAllocator is a critical part of netty we should ensure it works as expected.
Modifications:
- Add a few more asserts to ensure we not see any corrupted state
- Null out slot in the subpage array once the subpage was freed and removed from the pool
- Merge methods into constructor as it was only called from the constructor anyway.
Result:
Code cleanup
Motivation:
Users may want to do special actions when onComplete(...) was called and depend on these once they receive the SniCompletionEvent
Modifications:
Switch order and so call onLookupComplete(...) before we fire the event
Result:
Fixes https://github.com/netty/netty/issues/10655
Motivation:
We have a few classes in which we store and reuse static instances of various exceptions. When doing so it is important to also override fillInStacktrace() and so prevent the leak of the ClassLoader in the internal backtrace field.
Modifications:
- Add overrides of fillInStracktrace when needed
- Move ThrowableUtil usage in the static methods
Result:
Fixes https://github.com/netty/netty/pull/10686
Motivation:
The process of reporting security issues should be documented
and easy to find.
Modification:
Added a SECURITY.md file that describes how to report a security issue.
Result:
It's a bit easier to find the docs that describe how security issues
should be reported. Also, when someone creates an issue the repository,
they will see a link to the security policy.
Motivation:
Fix Broken Link of OWASP HttpOnly Cookie in Cookie class.
Modification:
Updated the broken link.
Result:
Broken Link Fix for better Documentation.
Motivation:
We should use ObjectUtil for checking if Compression parameters are in range. This will reduce LOC and make code more readable.
Modification:
Used ObjectUtil
Result:
More readable code
Motivation:
All scheduled executors should behave in accordance to their API.
The bug here is that scheduled tasks were not run more than once because we executed the runnables directly, instead of through the provided runnable future.
Modification:
We now run tasks through the provided future, so that when each run completes, the internal state of the task is reset and the ScheduledThreadPoolExecutor is informed of the completion.
This allows the executor to prepare the next run.
Result:
The UnorderedThreadPoolEventExecutor is now able to run scheduled tasks more than once.
Which is what one would expect from the API.
Motivation:
Java 16 will come around eventually anyway, and this makes it easier for people to experiment with Early Access builds.
Modification:
Added Maven profiles for JDK 16 to relevant pom files.
Result:
Netty now builds on JDK 16 pre-releases (provided they've not broken compatibility in some way).
Motivation:
We check lots of numbers if it lies in a range. So it's better to add a method in `ObjectUtil` to check if a number lies inside a range.
Modification:
Added Range check method.
Result:
A faster and better way to check if a number lies inside a range.
