Commit Graph

10045 Commits

Author SHA1 Message Date
Norman Maurer
26f3cd89ef Add docker-compose files to compile with OpenJDK15 (#10697)
Motivation:

OpenJDK15 was released, we should compile with it on the CI

Modifications:

Add docker-compose files to be able to compile with OpenJDK15

Result:

Compile with latest major JDK version
2020-10-22 08:51:13 +02:00
Stuart Douglas
7d971a78a0 Minor performance improvement in websocket upgrade (#10710)
Motivation:

I noticed WebSocketServerExtensionHandler taking up a non-trivial
amount of CPU time for a non-websocket based menchmark. This attempts
to speed it up.

Modifications:

- It is faster to check for a 101 response than to look at headers,
so an initial response code check is done
- Move all the actual upgrade code into its own method to increase
chance of this method being inlined
- Add an extra contains() check for the upgrade header, to avoid
allocating an iterator if there is no upgrade header

Result:

A small but noticable performance increase.

Signed-off-by: Stuart Douglas <stuart.w.douglas@gmail.com>
2020-10-21 13:00:08 +02:00
Norman Maurer
c061bd1798
Ensure SniCompletionEvent is not lost after onLookupComplete(...) (#10709)
Motivation:

In the master branch we fail fire* operations on the ChannelHandlerContext once the handler was removed. This is by design as it is "unspecified" what the semantics could be after the handler was removed and may lead to very hard to debug problems. Because of this we need to select the right ChannelHandlerContext for firing the event.

Modifications:

Choose a valid ChannelHandlerContext based on the state of the context of the handler

Result:

No more test failures
2020-10-20 09:01:15 +02:00
Norman Maurer
d2e0f2a02c
Explicit specify jdk11 for codeql (#10706)
Motivation:

The master branch requires jdk11

Modifications:

Specifiy jdk11 to use

Result:

no more failure during executing the action
2020-10-20 09:00:10 +02:00
Norman Maurer
3f2c5ccd46 Replace deprecated Assert.assertThat(...) with MatcherAssert.assertThat(...) (#10699)
Motivation:

junit deprecated Assert.assertThat(...)

Modifications:

Use MatcherAssert.assertThat(...) as replacement for deprecated method

Result:

Less deprecation warnings
2020-10-18 14:55:21 +02:00
Norman Maurer
0a70135be4 Create codeql-analysis.yml (#10696)
Motivation:

Github now allows to run CodeQL during pull request verification. This allows to detect errors / security problems early.

Modification:

Add config

Result:

Fixes https://github.com/netty/netty/issues/10669


Co-authored-by: Artem Smotrakov <artem.smotrakov@sap.com>
2020-10-18 14:26:08 +02:00
Norman Maurer
e7e19b9917 Fix ByteBuf leaks in HaProxyMessageEncoderTest (#10704)
Motivation:

We need to ensure we not leak in tests. We did see some leaks reported related to HaProxyMessageEncoderTest on our CI.

Modifications:

- Use readSlice(...) and so not create new ByteBuf instances that need to be released

Result:

No more leaks
2020-10-18 14:10:58 +02:00
Norman Maurer
8dffd0914f Update java patch versions (#10703)
Motivation:

We should use the latest patch releases when building via docker

Modifications:

Update all java versions to the latest patch release

Result:

Use latest releases
2020-10-17 20:24:54 +02:00
Aayush Atharva
87719f4fd9 Add null rule check in rules array of RuleBasedIpFilter (#10527)
Motivation:

We can filter out `null` rules while initializing the instance of `RuleBasedIpFilter` so we don't have to keep checking for `null` rules while iterating through `rules` array in `for loop` which is just a waste of CPU cycles.

Modification:
Added `null` rule check inside the constructor.

Result:
No more wasting CPU cycles on check the `null` rule each time in `for loop` and makes the overall operation more faster.
2020-10-17 20:23:52 +02:00
Artem Smotrakov
f0448d6a8a Fix or suppress LGTM findings (#10689)
Motivation:

LGTM reports multiple issues. They need to be triaged,
and real ones should be fixed.

Modifications:
- Fixed multiple issues reported by LGTM, such as redundant conditions,
  resource leaks, typos, possible integer overflows.
- Suppressed false-positives.
- Added a few testcases.

Result:

Fixed several possible issues, get rid of false alarms in the LGTM report.
2020-10-17 09:57:52 +02:00
Norman Maurer
dbe13b41e4 Fix unit tests that sometimes failed due timeouts (#10698)
Motivation:

We had two unit tests that sometimes failed due timeouts. After insepecting these I noticed these can be improved to run faster while still do the right validation

Modifications:

- Only submit one task for execution per execute
- Cleanup

Result:

No test failures due timeout
2020-10-16 21:47:15 +02:00
Norman Maurer
de15b18087 Cleanup PoolChunk / PoolSubpage and add a few more asserts (#10690)
Motivation:

As the PooledByteBufAllocator is a critical part of netty we should ensure it works as expected.

Modifications:

- Add a few more asserts to ensure we not see any corrupted state
- Null out slot in the subpage array once the subpage was freed and removed from the pool
- Merge methods into constructor as it was only called from the constructor anyway.

Result:

Code cleanup
2020-10-15 21:02:11 +02:00
Norman Maurer
647dbe0244 Fire SniCompletionEvent after onLookupComplete(...) was called (#10688)
Motivation:

Users may want to do special actions when onComplete(...) was called and depend on these once they receive the SniCompletionEvent

Modifications:

Switch order and so call onLookupComplete(...) before we fire the event

Result:

Fixes https://github.com/netty/netty/issues/10655
2020-10-15 21:01:18 +02:00
Norman Maurer
5b00058fa7 Ensure we don't leak the ClassLoader in the backtrace (#10691)
Motivation:

We have a few classes in which we store and reuse static instances of various exceptions. When doing so it is important to also override fillInStacktrace() and so prevent the leak of the ClassLoader in the internal backtrace field.

Modifications:

- Add overrides of fillInStracktrace when needed
- Move ThrowableUtil usage in the static methods

Result:

Fixes https://github.com/netty/netty/pull/10686
2020-10-15 20:50:01 +02:00
Artem Smotrakov
e3b3cf27da Added a security policy (#10692)
Motivation:

The process of reporting security issues should be documented
and easy to find.

Modification:

Added a SECURITY.md file that describes how to report a security issue.

Result:

It's a bit easier to find the docs that describe how security issues
should be reported. Also, when someone creates an issue the repository,
they will see a link to the security policy.
2020-10-15 20:40:05 +02:00
Aayush Atharva
da8412a054 Update OWASP Links in Cookie class (#10677)
Motivation:
Fix Broken Link of OWASP HttpOnly Cookie in Cookie class.

Modification:
Updated the broken link.

Result:
Broken Link Fix for better Documentation.
2020-10-15 14:08:56 +02:00
Aayush Atharva
473471bac6 Use ObjectUtil for multiple operations (#10679)
Motivation:
We should use ObjectUtil for checking if Compression parameters are in range. This will reduce LOC and make code more readable.

Modification:
Used ObjectUtil

Result:
More readable code
2020-10-14 12:13:10 +02:00
dependabot[bot]
f39d2162f5 Bump junit from 4.12 to 4.13.1 (#10681)
Bumps [junit](https://github.com/junit-team/junit4) from 4.12 to 4.13.1.
- [Release notes](https://github.com/junit-team/junit4/releases)
- [Changelog](https://github.com/junit-team/junit4/blob/main/doc/ReleaseNotes4.12.md)
- [Commits](https://github.com/junit-team/junit4/compare/r4.12...r4.13.1)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2020-10-14 12:13:01 +02:00
Chris Vest
0ca76c42a5 Fix #10614 by making UnorderedTPEExecutor.scheduleAtFixedRate run tasks more than once (#10659)
Motivation:
All scheduled executors should behave in accordance to their API.
The bug here is that scheduled tasks were not run more than once because we executed the runnables directly, instead of through the provided runnable future.

Modification:
We now run tasks through the provided future, so that when each run completes, the internal state of the task is reset and the ScheduledThreadPoolExecutor is informed of the completion.
This allows the executor to prepare the next run.

Result:
The UnorderedThreadPoolEventExecutor is now able to run scheduled tasks more than once.
Which is what one would expect from the API.
2020-10-14 11:33:56 +02:00
Aayush Atharva
3cbbef687e Add checkInRange in ObjectUtil (#10668)
Motivation:
We check lots of numbers if it lies in a range. So it's better to add a method in `ObjectUtil` to check if a number lies inside a range.

Modification:
Added Range check method.

Result:
A faster and better way to check if a number lies inside a range.
2020-10-12 18:26:16 +02:00
Chris Vest
6201c6d80f
Add a build profile for JDK 16 (#10675)
Motivation:
Java 16 will come around eventually anyway, and this makes it easier for people to experiment with Early Access builds.

Modification:
- Added Maven profiles for JDK 16 to relevant pom files.
- Removed the `--add-exports java.base/sun.security.x509=ALL-UNNAMED` argument when running tests; we've not needed it since the Java11-as-baseline PR landed.

Result:
Netty now builds on JDK 16 pre-releases (provided they've not broken compatibility in some way).
2020-10-12 16:42:40 +02:00
Chris Vest
a179db8066
Raise the Netty 5 minimum required Java version to Java 11. (#10650)
Raise the Netty 5 minimum required Java version to Java 11.

Motivation:
Java 11 has been out for some time, and Netty 5 is still some ways out.
There are also many good features in Java 11 that we wish to use, such as VarHandles, var-keyword, and the module system.
There is no reason for Netty 5 to not require Java 11, since Netty 4.x will still be supported for the time being.

Modification:
Remove everything in the pom files related to Java versions older than Java 11.
Remove the animal-sniffer plug-in and rely on the `--release` compiler flag instead.
Remove docker files related to Java versions older than Java 11.
Remove the copied SCTP APIs -- we should test this commit independently on Windows.
Remove the OpenJdkSelfSignedCertGenerator.java file and just always use Bouncy Castle for generating self-signed certificates for testing.
Make netty-testsuite tests pass by including Bouncy Castle as a test dependency, so we're able to generate our self-signed certificate.

Result:
Java 11 is now the minimum required Java version.
2020-10-12 14:13:01 +02:00
Aayush Atharva
2464bee521 Simplify and Remove useless Bit operations from HpackHuffmanDecoder (#10656)
Motivation:
We should simplify and remove useless bit operations to make code more efficient, faster, and easier to understand.

Modification:
Simplified and Removed Useless Bit Operations

Result:
Simpler Code.
2020-10-12 11:45:08 +02:00
Artem Smotrakov
97c7c00dd5 Avoid casting numbers to narrower types (#10645)
Motivation:

Avoid implicit conversions to narrower types in
AbstractMemoryHttpData and Bzip2HuffmanStageEncoder classes
reported by LGTM.

Modifications:

Updated the classes to avoid implicit casting to narrower types.
It doesn't look like that an integer overflow is possible there,
therefore no checks for overflows were added.

Result:

No warnings about implicit conversions to narrower types.
2020-10-12 09:34:06 +02:00
Artem Smotrakov
327071c7b3 Suppress warnings about weak hash algorithms (#10647)
Motivation:

LGTM reported that WebSocketUtil uses MD5 and SHA-1
that are considered weak. Although those algorithms
are insecure, they are required by draft-ietf-hybi-thewebsocketprotocol-00
specification that is implemented in the corresponding WebSocket
handshakers. Once the handshakers are removed, WebSocketUtil can be
updated to stop using those weak hash functions.

Modifications:

Added SuppressWarnings annotations.

Result:

Suppressed warnings.
2020-10-12 09:24:29 +02:00
Matthew Kavanagh
9707ce183a Avoid integer overflow in ByteBuf.ensureWritable (#10648)
Motivation:

- To make ensureWritable throw IOOBE when maxCapacity is exceeded, even if
the requested new capacity would overflow Integer.MAX_VALUE

Modification:

- AbstractByteBuf.ensureWritable0 is modified to detect when
targetCapacity has wrapped around
- Test added for correct behaviour in AbstractByteBufTest

Result:

- Calls to ensureWritable will always throw IOOBE when maxCapacity is
exceeded (and bounds checking is enabled)
2020-10-12 09:23:43 +02:00
Aayush Atharva
423943e895 Remove extra line from end (#10673)
Motivation:
`Http2MultiplexHandler` have to 2 empty lines at the end instead of 1.

Modification:
Removed 1 extra line.

Result:
Little better code style.
2020-10-12 09:22:47 +02:00
Doyun Geum
2233d1ca17 Add validation check about websocket path (#10583)
Add validation check about websocket path

Motivation:

I add websocket handler in custom server with netty.
I first add WebSocketServerProtocolHandler in my channel pipeline.
It does work! but I found that it can pass "/websocketabc". (websocketPath is "/websocket")

Modification:
`isWebSocketPath()` method of `WebSocketServerProtocolHandshakeHandler` now checks that "startsWith" applies to the first URL path component, rather than the URL as a string.

Result:
Requests to "/websocketabc" are no longer passed to handlers for requests that starts-with "/websocket".
2020-10-08 12:07:10 +02:00
Aayush Atharva
09dc15aaa0 Add Close method for closing OutputStream and PcapWriteHandler (#10638)
Motivation:
We should implement the Closeable method to properly close `OutputStream` and `PcapWriteHandler`. So whenever `handlerRemoved(ChannelHandlerContext)` is called or the user wants to stop the Pcap writes into `OutputStream`, we have a proper method to close it otherwise writing data to close `OutputStream` will result in `IOException`.

Modification:
Implemented `Closeable` in `PcapWriteHandler` which calls `PcapWriter#close` and closes `OutputStream` and stops Pcap writes.

Result:
Better handling of Pcap writes.
2020-10-08 11:53:13 +02:00
Aayush Atharva
bb7a79452e Use ObjectUtil#checkPositive and Add missing JavaDoc In InboundHttp2ToHttpAdapter (#10637)
Motivation:
We can use ObjectUtil#checkPositive instead of manually checking maxContentLength. Also, there was a missing JavaDoc for Http2Exception,

Modification:
Used ObjectUtil#checkPositive for checking maxContentLength.
Added missing JavaDoc.

Result:
More readable code.
2020-10-07 11:16:02 +02:00
Francesco Nigro
4624b6309d Reduce DefaultAttributeMap lookup cost (#10530)
Motivation:

DefaultAttributeMap::attr has a blocking behaviour on lookup of an existing attribute:
it can be made non-blocking.

Modification:

Replace the existing fixed bucket table using a locked intrusive linked list
with an hand-rolled copy-on-write ordered single array

Result:
Non blocking behaviour for the lookup happy path
2020-10-02 21:19:03 +02:00
Chris Vest
86c8f24d9a
Replace UNSAFE.throwException with alternatives supported on Java 8 (#10629)
Motivation:
 We wish to use Unsafe as little as possible, and Java 8 allows us
 to take some short-cuts or play some tricks with generics,
 for the purpose of working around having to declare all checked
 exceptions. Ideally all checked exceptions would be declared, but
 the code base is not ready for that yet.

Modification:
 The call to UNSAFE.throwException has been removed, so when we need
 that feature, we instead use the generic exception trick.
 In may cases, Java 8 allows us to throw Throwable directly. This
 happens in cases where no exception is declared to be thrown in a
 scope.
 Finally, some warnings have also been fixed, and some imports have
 been reorganised and cleaned up while I was modifying the files
 anyway.

Result:
 We no longer use Unsafe for throwing any exceptions.
2020-10-02 08:29:07 +02:00
Norman Maurer
ba43065482 Respect the Provider when detecting if TLSv1.3 is used by default / supported (#10621)
Motivation:

We need to take the Provider into account as well when trying to detect if TLSv1.3 is used by default / supported

Modifications:

- Change utility method to respect provider as well
- Change testcode

Result:

Less error-prone tests
2020-09-29 20:49:11 +02:00
Norman Maurer
d77edcb6e2 Use SelfSignedCertificate to fix test-failure related to small key size (#10620)
Motivation:

Some JDKs dissallow the usage of keysizes < 2048, so we should not use such small keysizes in tests.

This showed up on fedora 32:

```
Caused by: java.security.cert.CertPathValidatorException: Algorithm constraints check failed on keysize limits. RSA 1024bit key used with certificate: CN=tlsclient.  Usage was tls client
        at sun.security.util.DisabledAlgorithmConstraints$KeySizeConstraint.permits(DisabledAlgorithmConstraints.java:817)
        at sun.security.util.DisabledAlgorithmConstraints$Constraints.permits(DisabledAlgorithmConstraints.java:419)
        at sun.security.util.DisabledAlgorithmConstraints.permits(DisabledAlgorithmConstraints.java:167)
        at sun.security.provider.certpath.AlgorithmChecker.check(AlgorithmChecker.java:326)
        at sun.security.provider.certpath.PKIXMasterCertPathValidator.validate(PKIXMasterCertPathValidator.java:125)
        ... 23 more
```

Modifications:

Replace hardcoded keys / certs with SelfSignedCertificate

Result:

No test-failures related to small key sizes anymore.
2020-09-29 13:06:54 +02:00
Norman Maurer
fc656f605f Only set the keymaterial once and correctly handle errors during keymaterial setting on the client-side as well (#10613)
Motivation:

We should stop as soon as we were able to set the key material on the server side as otherwise we may select keymaterial that "belongs" to a less prefered cipher. Beside this it also is just useless work.
We also need to propagate the exception when it happens during key material selection on the client side so openssl will produce the right alert.

Modifications:

- Stop once we were able to select a key material on the server side
- Ensure we not call choose*Alias more often then needed
- Propagate exceptions during selection of the keymaterial on the client side.

Result:

Less overhead and more correct behaviour
2020-09-29 09:28:25 +02:00
Norman Maurer
ced5faa440 Correctly report back when we fail to select the key material and ens… (#10610)
Motivation:

We need to let openssl know that we failed to find the key material so it will produce an alert for the remote peer to consume. Beside this we also need to ensure we wrap(...) until we produced everything as otherwise the remote peer may see partial data when an alert is produced in multiple steps.

Modifications:

- Correctly throw if we could not find the keymaterial
- wrap until we produced everything
- Add test

Result:

Correctly handle the case when key material could not be found
2020-09-29 09:21:35 +02:00
Norman Maurer
d92d92a923 Filter out duplicates before trying to find the alias to use (#10612)
Motivation:

Calling chooseServerAlias(...) may be expensive so we should ensure we not call it multiple times for the same auth methods.

Modifications:

Remove duplicated from authMethods before trying to call chooseServerAlias(...)

Result:

Less performance overhead during key material selection
2020-09-25 19:13:33 +02:00
Yosfik Alqadri
c9e42106d1 Fix Javadoc Typo (#10603)
Motivation:

Following Javadoc standard

Modification:

Change from `@param KeyManager` to `@param keyManager`

Result:

The `@param` matches the actual parameter variable name
2020-09-24 15:28:30 +02:00
Norman Maurer
1663eb5965 Increase initial buffer size in AdaptiveRecvByteBufAllocator (#10600)
Motivation:

We should use an initial buffer size with is >= 1500 (which is a common setting for MTU) to reduce the need for memory copies when a new connection is established. This is especially interesting when SSL / TLS comes into the mix.

This was ported from swiftnio:

https://github.com/apple/swift-nio/pull/1641

Modifications:

Increase the initial size from 1024 to 2048.

Result:

Possible less memory copies on new connections
2020-09-22 17:27:45 +02:00
Paul Lysak
2a87b00984 Not truncate MQTT SUBACK reason codes (#10584)
Motivation:

Reason code for MQTT SUBACK messages is truncated, thus not allowing to get new codes supported by MQTT v.5

Modification:

Changed `MqttCodecTest.testUnsubAckMessageForMqtt5` to catch it then, removed truncation in `MqttDecoder.decodeSubackPayload` to make it pass.

Result:

Codec users can get all reason codes supported by MQTT5 now.
2020-09-22 09:04:45 +02:00
Norman Maurer
5bb535e78a Only create ConnectTimeoutException if really needed (#10595)
Motivation:

Creating exceptions is expensive so we should only do so if really needed.

Modifications:

Only create the ConnectTimeoutException if we really need it.

Result:

Less overhead
2020-09-21 21:40:25 +02:00
Divij Vaidya
58911c1482 Clear scheduled timeout if channel is closed with incomplete WebSocket handshake (#10510)
Motivation:

Consider a scenario when the client iniitiates a WebSocket handshake but before the handshake is complete,
the channel is closed due to some reason. In such scenario, the handshake timeout scheduled on the executor
is not cleared. The reason it is not cleared is because in such cases the handshakePromise is not completed.

Modifications:

This change completes the handshakePromise exceptinoally on channelInactive callback, if it has not been
completed so far. This triggers the callback on completion of the promise which clears the timeout scheduled
on the executor.

This PR also adds a test case which reproduces the scenario described above. The test case fails before the
fix is added and succeeds when the fix is applied.

Result:

After this change, the timeout scheduled on the executor will be cleared, thus freeing up thread resources.
2020-09-16 10:10:50 +02:00
Aayush Atharva
7d985d26b4 Call long as l instead of i (#10578)
Motivation:

Long should be called `l` instead of `i` because `i` is generally used for `int`.

Modification:

Changed `i` to `l`.

Result:
Better naming
2020-09-16 07:58:23 +02:00
Paul Lysak
2fca020be5 Make MQTT property value publicly accessible (#10577)
Motivation:

There was no way to read MQTT properties outside of `io.netty.handler.codec.mqtt` package

Modification:

Expose `MqttProperties.MqttProperty` fields `value` and `propertyId` via public methods

Result:

It's possible to read MQTT properties now
2020-09-16 07:56:31 +02:00
Kevin Wu
eef67e2f0e Fix DeleteOnExitHook cause memory leak (#10560)
Motivation:

If DeleteOnExitHook is in the open state and the program runs for a long time, the DeleteOnExitHook file keeps increasing.
This results in a memory leak

Modification:

I re-customized a DeleteOnExitHook hook. If DeleteOnExitHook is turned on, this hook will be added when creating a temporary file. After the request ends and the corresponding resources are released, the current file will be removed from this hook, so that it will not increase all the time.

Result:

Fixes https://github.com/netty/netty/issues/10351
2020-09-15 16:45:03 +02:00
dependabot[bot]
cc3a686357 Bump ant from 1.9.7 to 1.9.15 (#10573)
Bumps ant from 1.9.7 to 1.9.15.

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2020-09-15 15:31:15 +02:00
Paul Lysak
575c2d16de Fix NPE with MqttUnsubAckMessage - regression of MQTT5 support (#10557)
Recent changes for MQTT5 may cause NPE if UNSUBACK message is created with `MqttMessageFactory`
and client code isn't up-to-date.

Modifications:

Added default body in `MqttUnsubAckMessage` constructor if null body is passed,
added null checks in `encodeUnsubAckMessage`

Result:

`MqttUnsubAckMessage` created with `MqttMessageFactory` doesn't cause NPE
even if null body is supplied.
2020-09-15 15:14:45 +02:00
Aayush Atharva
379be5085f Add PcapWriteHandler Support (#10498)
Motivation:
Write TCP and UDP packets into Pcap `OutputStream` which helps a lot in debugging.

Modification:
Added TCP and UDP Pcap writer.

Result:
New handler can write packets into an `OutputStream`, e.g. a file that can be opened with Wireshark.

Fixes #10385.
2020-09-11 16:15:39 +02:00
Paul Lysak
dc66e8d67f Make RetainedHandlingPolicy enum public (#10565)
Motivation:

Impossible to specify retained messages handling policy because the corresponding enum
isn't public (https://github.com/netty/netty/issues/10562)

Modification:

Made `RetainedHandlingPolicy` public

Result:

Fixes #10562.
2020-09-11 08:33:39 +02:00
Chris Vest
7f423772ba Disable Maven build HTTP connection pooling (#10558)
Motivation:
 The builds often fail when downloading dependencies.
 This might be caused by the build taking a long time, and cause pooled connections to be closed
 by the remote end, if they are idle for too long.

Modification:
 Disable connection pooling. This should force Maven to reestablish the connection for each download,
 thus reducing the likelihood of the remote end closing connections we wish to use.

Result:
 I'll leave it up the statistis of our CI to confirm, but we should see more stable builds.
2020-09-09 17:08:03 +02:00