Commit Graph

10035 Commits

Author SHA1 Message Date
Chris Vest
ff2e790e89 Revert "ByteProcessor shouldn't throw checked exception"
This reverts commit b70d0fa6e3.
2020-11-03 16:12:54 +01:00
Chris Vest
b70d0fa6e3 ByteProcessor shouldn't throw checked exception
Motivation:
There is no need for ByteProcessor to throw a checked exception.
The declared checked exception causes unnecessary code complications just to propagate it.
This can be cleaned up.

Modification:
ByteProcessor.process no longer declares to throw a checked exception, and all the places that were trying to cope with the checked exception have been simplified.

Result:
Simpler code.
2020-11-03 16:12:13 +01:00
Aayush Atharva
2a1e031fa3 HttpConversionUtil#toHttpResponse should use false in isRequest parameter (#10760)
Motivation:
`HttpConversionUtil#toHttpResponse` translates `Http2Headers` to `HttpResponse`. It uses `#addHttp2ToHttpHeaders(..., boolean isRequest)` to do so. However, `isRequest` field is set to `true` instead of `false`. It should be set to `false` because we're doing conversion of Response not Request.

Modification:
Changed `true` to `false`.

Result:
Correctly translates `Http2Headers` to `HttpResponse`.
2020-11-03 09:39:03 +01:00
Aayush Atharva
e4baa6737f Add toString method in DefaultHttp2WindowUpdateFrame (#10763)
Motivation:
We should have the `toString` method in `DefaultHttp2WindowUpdateFrame` because it makes debugging a little easy.

Modification:
Added `toString` method.

Result:
`toString` method to help in debugging.


Co-authored-by: Norman Maurer <norman_maurer@apple.com>
2020-11-03 09:37:42 +01:00
Norman Maurer
d16c23d3f5 Revert "Add support for heartbeat in STOMP decoder/encoder. (#10695)"
This reverts commit 78d5ab4027.
2020-11-03 09:36:40 +01:00
Aayush Atharva
3dce4de50f Remove extra empty line (#10754)
Motivation:
`Http2Frame` has extra empty line after `String name();`. However, it should not be there.

Modification:
Removed extra empty line.

Result:
Empty-line code style now matching with other classes.
2020-11-02 15:08:04 +01:00
Aayush Atharva
7d9b009cdf Fix typo in Http2FrameCodec#write(...) comment
Motivation:
`Http2FrameCodec#write(...)` has typo in comment.
`// In the event of manual SETTINGS ACK is is assumed the encoder will apply the earliest received but not`.
The typo is `is is`. However, it should be `it is`.

Modification:
Changed `is is` to `it is`.

Result:
Correct comment without typos.
2020-11-02 08:49:35 +01:00
Aayush Atharva
3aef41922b Fix typo in Http2DataFrame javadocs (#10755)
Motivation:
`Http2DataFrame#isEndStream()` JavaDoc says `Returns {@code true} if the END_STREAM flag ist set.`. The typo is `ist` word. However, it should be `is`.

Modification:
Changed `ist` to `is`.

Result:
Better JavaDoc by fixing the typo.
2020-10-30 15:59:46 +01:00
Aayush Atharva
1910528b30 Fix typo in Http2HeadersFrame javadocs (#10756)
Motivation:
`Http2HeadersFrame#isEndStream()` JavaDoc says `Returns {@code true} if the END_STREAM flag ist set.`. The typo is `ist` word. However, it should be `is`.

Modification:
Changed `ist` to `is`.

Result:
Better JavaDoc by fixing the typo.
2020-10-30 15:58:59 +01:00
Dave Syer
1a90fd33e0 Add reflection config for JMX runtime (#10753)
Motivation:

`DefaultChannelId` uses reflection to access the JMX runtime. This needs some extra config for GraalVM.

Modification:

Add config for GraalVM

Result:

Works when using GraalVM native image
2020-10-30 15:27:42 +01:00
Benjamin Roux
78d5ab4027 Add support for heartbeat in STOMP decoder/encoder. (#10695)
Motivation:

Heart-beat is a functionality of STOMP enabling clients and servers to know the healthiness of the connection. The current decoder didn't allow for heart-beat messages to be forwarded to the decoder and were simply swallowed as part of the frame decoding.

Modifications:

Adding support for heartbeat message parsing by introducing a new HEARTBEAT command (not a real STOMP command).
Heartbeat received on the channel will trigger a StompFrame with the command set to HEARTBEAT.
Sending heartbeat on the channel is achieved by creating a StompFrame with the command set to HEARTBEAT.

Result:

Heartbeat can now be received/sent and acted upon to determine the healthiness of the connection and terminate it if needed.
2020-10-30 15:27:29 +01:00
Artem Smotrakov
3e41a7f231 Enable header valication in HttpServerUpgradeHandler (#10643)
Motivation:

HttpServerUpgradeHandler takes a list of protocols from an incoming
request and uses them for building a response.
Although the class does some validation while parsing the list,
it then disables HTTP header validation when it builds a responst.
The disabled validation may potentially allow
HTTP response splitting attacks.

Modifications:

- Enabled HTTP header validation in HttpServerUpgradeHandler
  as a defense-in-depth measure to prevent possible
  HTTP response splitting attacks.
- Added a new constructor that allows disabling the validation.

Result:

HttpServerUpgradeHandler validates incoming protocols
before including them into a response.
That should prevent possible HTTP response splitting attacks.
2020-10-30 11:29:45 +01:00
Paul Lysak
30e9b6846d MQTT5: support multiple Subscription ID properties (#10734)
Motivation:

Subscription ID property of the PUBLISH message may be repeated multiple times, which wasn't taken into account when developing `MqttProperties` API.

Modification:

Store Subscription ID properties separately from others - in `MqttProperties.subscriptionIds`. 
Add `MqttProperties.getProperties` method to retrieve properties that may be repeated.
Change internal representation of User Properties for uniformity with Subscription ID - now they're stored in `MqttProperties.userProperties` rather than the common hash map.

Result:

Multiple Subscription ID properties can be set or retrieved.
2020-10-30 11:19:21 +01:00
Norman Maurer
7579054a8f Fix compile error introduced by a63faa4fa1 (#10750)
Motivation:

a63faa4fa1 missed to update the macos specific c files for the resolver.

Modifications:

Fix up c files so it compiles again on macos

Result:

Compiles on macos again
2020-10-30 11:08:09 +01:00
Norman Maurer
475f20aa0f Use netty-jni-util and so remove a lot of duplication (#10735)
Motivation:

We had a lot of duplication in our jni code which was mostly due macros but also related to how we support shading. By using netty-jni-util we can share all the code between netty and netty-tcnative ( and possible other jni based netty projects in the future).

Modifications:

- Use netty-jni-util and re-use its macros / functions
- Remove duplicated code
- Adjust build files

Result:

Less code duplication for JNI
2020-10-29 16:46:33 +01:00
Aayush Atharva
180f0b6ced Change variable name of Http2Headers (#10743)
Motivation:
`addHttp2ToHttpHeaders(int streamId, Http2Headers sourceHeaders, FullHttpMessage destinationMessage, boolean addToTrailer)` 
should match 
`addHttp2ToHttpHeaders(int streamId, Http2Headers inputHeaders, HttpHeaders outputHeaders, HttpVersion httpVersion, boolean isTrailer, boolean isRequest)`.

However, the `Http2Headers` variable name is different.

Modification:
Changed `sourceHeaders` to `inputHeaders`.

Result:
Variable and JavaDoc naming now correct.
2020-10-29 10:41:21 +01:00
Aayush Atharva
71f0e23bcb Make Http2ToHttpHeaderTranslator method package-private. (#10736)
Motivation:
Http2ToHttpHeaderTranslator is a private class but translateHeaders(Iterable<Entry<CharSequence, CharSequence>>) is public but it should be package-private.

Modification:
Removed public.

Result:
Correct access modifer.
2020-10-29 10:39:17 +01:00
Chris Vest
57cb7a8a91 Fix explicitly little-endian accessors in SwappedByteBuf (#10747)
Motivation:
Some buffers implement ByteBuf#order(order) by wrapping themselves in a SwappedByteBuf.
The SwappedByteBuf is then responsible for swapping the byte order on accesses.
The explicitly little-endian accessor methods, however, should not be swapped to big-endian, but instead remain explicitly little-endian.

Modification:
The SwappedByteBuf was passing through calls to e.g. writeIntLE, to the big-endian equivalent, e.g. writeInt.
This has been changed so that these calls delegate to their explicitly little-endian counterpart.

Result:
This makes all buffers that make use of SwappedByteBuf for their endian-ness configuration, consistent with all the buffers that use other implementation strategies.
In the end, all buffers now behave exactly the same, when using their explicitly little-endian accessor methods.
2020-10-29 10:38:06 +01:00
Norman Maurer
ceb9dd53aa Errors.throwConnectException(...) should be public (#10745)
Motivation:

ddebc1027d missed to make Errors.throwConnectException(...) public

Modifications:

Make method public

Result:

Be able to use Errors.throwConnectException(...) from other module
2020-10-29 10:36:51 +01:00
Chris Vest
d804e34cf0
Remove CleanerJava6 (#10746)
Motivation:
Netty 5 will require Java 11 at a minimum, so this pre-Java 9 specific code will never be used.

Modification:
Removed the dead code.

Result:
Less code to maintain.
2020-10-29 08:19:37 +01:00
Norman Maurer
38adfd8316 Rethrow NoSuchMethodError with more hints about incompatible native library versions (#10740)
Motivation:

03aafb9cff did ensure we fail while loading a natibe library which is not compatible. While this is great it is still sometimes hard for people to understand what NoSuchMethodError means in this context.

Modifications:

If possible rethrow the NoSuchMethodError and provide some more hints about multiple versions of the shared library

Result:

Easier to understand for people why loading fails
2020-10-28 20:37:47 +01:00
Norman Maurer
3d9d473685 Add PlatformDependent* methods that are needed by io_uring (#10744)
Motivation:

ddebc10 added a few adjustments that are needed by io_uring that we will add as an incubator repository. Unfortunally we missed the changed in PlatformDependent*.

Modifications:

Add missing methods

Result:

Be able to compile io_uring code against core netty
2020-10-28 19:21:11 +01:00
Norman Maurer
78c0b0e51a Make changes to prepare for io_uring incubator repository (#10741)
Motivation:

During the last few month we did develop an io_uring based transport which shows very promising performance numbers. To give it more time to bake we will develop it outside of netty in an "incubator" module which will make it clear to users what to expect and also allow us to seperate its release cycle. While the implementation of it is very self contained there are few small adjustments that need to be made in netty itself to allow us to reuse code.

Modifications:

- AbstractChannel: Add method which can be used when a write fails and remove final from one method
- IovArray: Allow to create an IovArray from a ByteBuf instance
- FileDescriptor: Allow to reuse mark close logic via sub-class

Result:

Be able to reuse netty core classes in io_uring incubator repository
2020-10-28 15:31:27 +01:00
Norman Maurer
436e47f12b Unregister all native methods on unload for kqueue (#10738)
Motivation:

03aafb9cff did ensure we unregister all native methods when we unload / or fail during load of the native library. Unfortunally it missed to unregister in one case for kqueue.

Modifications:

Add unregister calls to the unload function

Result:

Correctly unregister in all cases
2020-10-28 13:08:53 +01:00
Aayush Atharva
3b89206153 Upgrade Conscrypt to 2.5.1 (#10732)
Motivation:
Conscrypt 2.5.1 is available and it's a good idea to upgrade to the latest version.

Modification:
Upgraded Conscrypt 2.4.0 to 2.5.1

Result:
Newer Conscrypt version.
2020-10-26 19:41:15 +01:00
Aayush Atharva
1bcfd7e39e Fix possible NPEs and IndexOutOfBoundsExceptions in HTTP/2 Codec (#10640)
Motivation:

There are possible NPEs and IndexOutOfBoundsExceptions in HTTP/2 code. 

Modification:
Fixed possible NPEs and IOOBEs

Result:
Better code
2020-10-26 14:42:14 +01:00
Artem Smotrakov
51db4c9a9f Better hash algorithm in FingerprintTrustManagerFactory (#10683)
Motivation:

FingerprintTrustManagerFactory can only use SHA-1 that is considered
insecure.

Modifications:

- Updated FingerprintTrustManagerFactory to accept a stronger hash algorithm.
- Remove the constructors that still use SHA-1.
- Added a test for FingerprintTrustManagerFactory.

Result:

A user can now configure FingerprintTrustManagerFactory to use a
stronger hash algorithm.

Co-authored-by: Norman Maurer <norman_maurer@apple.com>
2020-10-26 14:37:33 +01:00
Norman Maurer
31ffe11e47 Unregister all previous registered native methods if loading of native code fails… (#10719)
Motivation:

It's important to unload all previous registered native methods when there is a failure during loading the native lib. Failing to do so may lead to an "invalid state" and so may segfault the JVM when trying to call a native method that was previous loaded.

This was observed when two versions of netty-tcnative were on the classpath which had different requirements in terms of linking.

Something like this was reported in he hs log:

```
Instructions: (pc=0x0000000116413bf0)
0x0000000116413bd0:
[error occurred during error reporting (printing registers, top of stack, instructions near pc), id 0xb]

Register to memory mapping:

RAX=0x0000000116413bf0 is an unknown value
RBX={method} {0x000000011422e708} 'aprMajorVersion' '()I' in 'io/netty/internal/tcnative/Library'
RCX=0x000000000000000a is an unknown value
RDX=0x000000000000000a is an unknown value
```

Modifications:

- Unregister previous registered native methods on failure
- Unregister previous registered native methods on on unload of the native lib

Result:

No more segfault caused by invalid state when loading of the native lib fails in between. In this case the user will receive an error now like:
2020-10-26 14:15:22 +01:00
Aayush Atharva
56c47e4361 Upgrade Slf4j to 1.7.30 (#10712)
Motivation:
SLF4J 1.7.30 is the latest version in 1.7.x and we should upgrade to it from 1.7.21.

Modification:
Changed 1.7.21 to 1.7.30

Result:
Newer version of SLF4J
2020-10-26 13:03:40 +01:00
Norman Maurer
cf43b26321 Release message before notify promise (#10726)
Motivation:

We should preferable always release the message before we notify the promise. Thhis has a few advantages:

 - Release memory as soon as possible
 - Listeners observe the "more correct" reference count

Modifications:

Release message before fail the promises

Result:

Faster releasing of resources. This came up in https://github.com/netty/netty/issues/10723
2020-10-26 13:03:34 +01:00
Roman Puchkovskiy
dba46aa3da Fix native image build on modern GraalVM versions for the cases when the program uses netty-dns (#10630)
Motivation:

Since GraalVM version 19.3.0, instances of java.net.InetAddress (and its subclasses Inet4Address and Inet6Address) are not allowed in native image heap (that is, they cannot be stored in static fields of classes initialized at build time or be reachable through static fields of such classes). When building a native image, it makes sense to initialize at build time as many classes as possible.
But some fields of some classes in Netty (for example, NetUtil.LOCALHOST4) contain InetAddress instances. If a program is using code path that makes it possible to reach such fields at build time initialization, it becomes impossible to build a native image initializing core Netty classes initialized at runtime. An example of such a program is a client that uses netty-dns.

Modifications:

- Add netty-testsuite-native-image-client Maven module to test that such an example program can be built after the corresponding fixes
- Add native-image.properties to resolver-dns module to move initialization of some classes to runtime (some of them are parsing configuration during initialization, so it makes no sense to initialize them at build time; for others, it's needed to avoid InetAddress reachability at build time)
- Add substitutions for NetUtil.LOCALHOST4, NetUtil.LOCALHOST6 and NetUtil.LOCALHOST to overcome the InetAddress-related prohibition
- Extract some initialization code from NetUtil to NetUtilInitializations to allow it to be used by the substitutions

Result:

A client program using netty-dns with --initialize-at-build-time=io.netty builds successfully
2020-10-26 08:49:31 +01:00
greenjustin
5ad80c5887 Allow EventLoops to rethrow Error (#10694)
Motivation:

Thread.stop() works by producing a ThreadDeath error in the target thread. EventLoops swallow all Throwables, which makes them effectively unkillable. This is effectively a memory leak, for our application. Beside this we should also just regrow all `Error` as there is almost no way to recover.

Modification:

Edit the EventLoops that swallow Throwables to instead rethrow Error.

Result:

`EventLoop` can crash if `Error` is thrown
2020-10-24 15:09:29 +02:00
Norman Maurer
930aec7f0a Fix checkstyle errors introduced by 33de96f448 2020-10-24 14:50:52 +02:00
Andrey Mizurov
634a6b70d7 Provide new client and server websocket handshake exceptions (#10646)
Motivation:

At the moment we have only one base `WebSocketHandshakeException` for handling WebSocket upgrade issues.
Unfortunately, this message contains only a string message about the cause of the failure, which is inconvenient in handling.

Modification:

Provide new `WebSocketClientHandshakeException` with `HttpResponse` field  and `WebSocketServerHandshakeException` with `HttpRequest` field both of them without content for avoid reference counting
problems.

Result:

More information for more flexible handling.

Fixes #10277 #4528 #10639.
2020-10-24 14:44:24 +02:00
Aayush Atharva
9f8590e194 Fix JavaDoc of Http2Headers (#10711)
Motivation:
Http2Headers has JavaDoc error which says Sets the {@link PseudoHeaderName#AUTHORITY} header or {@code null} if there is no such header however it should be Sets the {@link PseudoHeaderName#AUTHORITY} header in Http2Headers#authority(CharSequence) methods because it only sets CharSequence.

This is true for all setters in Http2Headers.

Modification:
Fixed all JavaDoc errors.

Result:
Better JavaDoc.
2020-10-23 15:36:16 +02:00
Artem Smotrakov
b8ae2a2af4 Enable nohttp check during the build (#10708)
Motivation:

HTTP is a plaintext protocol which means that someone may be able
to eavesdrop the data. To prevent this, HTTPS should be used whenever
possible. However, maintaining using https:// in all URLs may be
difficult. The nohttp tool can help here. The tool scans all the files
in a repository and reports where http:// is used.

Modifications:

- Added nohttp (via checkstyle) into the build process.
- Suppressed findings for the websites
  that don't support HTTPS or that are not reachable

Result:

- Prevent using HTTP in the future.
- Encourage users to use HTTPS when they follow the links they found in
  the code.
2020-10-23 15:26:25 +02:00
Norman Maurer
5e1c660416 Add NULL checks to fix possible undefined behavior (#10718)
Motivation:

In some situations we could have end up calling some functions with NULL parameters which in this case could lead to undefined behavior. All of this would have happened during loading of the native lib.

Modifications:

Add NULL check as guards and return early

Result:

Fix some possible undefined behavior
2020-10-23 14:39:54 +02:00
Norman Maurer
0f8e6a30ef DatagramDnsResponseDecoder should rethrow as CorruptedFrameException (#10714)
Motivation:

DatagramDnsResponseDecoder should rethrow as CorruptedFrameException if an IndexOutOfBoundsException happens.

Modifications:

- Catch IndexOutOfBoundsException and rethrow as CorruptedFrameException
- Add a testcase

Result:

Less noise in the logs
2020-10-22 09:17:58 +02:00
Norman Maurer
97a9772fa2
We should have a special config that allows to configure half closure for DuplexChannel (#10701) (#10716)
Motivation:

DuplexChannel allow for half-closure, we should have a special config interface for it as well.

Modifications:

Add DuplexChannelConfig which allows to configure half-closure.

Result:

More consistent types
2020-10-22 09:05:27 +02:00
Norman Maurer
ef1f24fb7f Fix compiler settings when building on JDK15
Motivation:

26f3cd89ef did introduce a profile for compiling on JDK15 but did miss to set the compiler settings correctly for the requirements of this branch

Modifications:

Correct compiler settings.

Result:

Be able to build with JDK15
2020-10-22 09:04:09 +02:00
Norman Maurer
26f3cd89ef Add docker-compose files to compile with OpenJDK15 (#10697)
Motivation:

OpenJDK15 was released, we should compile with it on the CI

Modifications:

Add docker-compose files to be able to compile with OpenJDK15

Result:

Compile with latest major JDK version
2020-10-22 08:51:13 +02:00
Stuart Douglas
7d971a78a0 Minor performance improvement in websocket upgrade (#10710)
Motivation:

I noticed WebSocketServerExtensionHandler taking up a non-trivial
amount of CPU time for a non-websocket based menchmark. This attempts
to speed it up.

Modifications:

- It is faster to check for a 101 response than to look at headers,
so an initial response code check is done
- Move all the actual upgrade code into its own method to increase
chance of this method being inlined
- Add an extra contains() check for the upgrade header, to avoid
allocating an iterator if there is no upgrade header

Result:

A small but noticable performance increase.

Signed-off-by: Stuart Douglas <stuart.w.douglas@gmail.com>
2020-10-21 13:00:08 +02:00
Norman Maurer
c061bd1798
Ensure SniCompletionEvent is not lost after onLookupComplete(...) (#10709)
Motivation:

In the master branch we fail fire* operations on the ChannelHandlerContext once the handler was removed. This is by design as it is "unspecified" what the semantics could be after the handler was removed and may lead to very hard to debug problems. Because of this we need to select the right ChannelHandlerContext for firing the event.

Modifications:

Choose a valid ChannelHandlerContext based on the state of the context of the handler

Result:

No more test failures
2020-10-20 09:01:15 +02:00
Norman Maurer
d2e0f2a02c
Explicit specify jdk11 for codeql (#10706)
Motivation:

The master branch requires jdk11

Modifications:

Specifiy jdk11 to use

Result:

no more failure during executing the action
2020-10-20 09:00:10 +02:00
Norman Maurer
3f2c5ccd46 Replace deprecated Assert.assertThat(...) with MatcherAssert.assertThat(...) (#10699)
Motivation:

junit deprecated Assert.assertThat(...)

Modifications:

Use MatcherAssert.assertThat(...) as replacement for deprecated method

Result:

Less deprecation warnings
2020-10-18 14:55:21 +02:00
Norman Maurer
0a70135be4 Create codeql-analysis.yml (#10696)
Motivation:

Github now allows to run CodeQL during pull request verification. This allows to detect errors / security problems early.

Modification:

Add config

Result:

Fixes https://github.com/netty/netty/issues/10669


Co-authored-by: Artem Smotrakov <artem.smotrakov@sap.com>
2020-10-18 14:26:08 +02:00
Norman Maurer
e7e19b9917 Fix ByteBuf leaks in HaProxyMessageEncoderTest (#10704)
Motivation:

We need to ensure we not leak in tests. We did see some leaks reported related to HaProxyMessageEncoderTest on our CI.

Modifications:

- Use readSlice(...) and so not create new ByteBuf instances that need to be released

Result:

No more leaks
2020-10-18 14:10:58 +02:00
Norman Maurer
8dffd0914f Update java patch versions (#10703)
Motivation:

We should use the latest patch releases when building via docker

Modifications:

Update all java versions to the latest patch release

Result:

Use latest releases
2020-10-17 20:24:54 +02:00
Aayush Atharva
87719f4fd9 Add null rule check in rules array of RuleBasedIpFilter (#10527)
Motivation:

We can filter out `null` rules while initializing the instance of `RuleBasedIpFilter` so we don't have to keep checking for `null` rules while iterating through `rules` array in `for loop` which is just a waste of CPU cycles.

Modification:
Added `null` rule check inside the constructor.

Result:
No more wasting CPU cycles on check the `null` rule each time in `for loop` and makes the overall operation more faster.
2020-10-17 20:23:52 +02:00
Artem Smotrakov
f0448d6a8a Fix or suppress LGTM findings (#10689)
Motivation:

LGTM reports multiple issues. They need to be triaged,
and real ones should be fixed.

Modifications:
- Fixed multiple issues reported by LGTM, such as redundant conditions,
  resource leaks, typos, possible integer overflows.
- Suppressed false-positives.
- Added a few testcases.

Result:

Fixed several possible issues, get rid of false alarms in the LGTM report.
2020-10-17 09:57:52 +02:00