Commit Graph

10326 Commits

Author SHA1 Message Date
Norman Maurer
71d034593f
Only create ConnectTimeoutException if really needed (#10595)
Motivation:

Creating exceptions is expensive so we should only do so if really needed.

Modifications:

Only create the ConnectTimeoutException if we really need it.

Result:

Less overhead
2020-09-21 21:32:20 +02:00
Divij Vaidya
79a7c157a3
Clear scheduled timeout if channel is closed with incomplete WebSocket handshake (#10510)
Motivation:

Consider a scenario when the client iniitiates a WebSocket handshake but before the handshake is complete,
the channel is closed due to some reason. In such scenario, the handshake timeout scheduled on the executor
is not cleared. The reason it is not cleared is because in such cases the handshakePromise is not completed.

Modifications:

This change completes the handshakePromise exceptinoally on channelInactive callback, if it has not been
completed so far. This triggers the callback on completion of the promise which clears the timeout scheduled
on the executor.

This PR also adds a test case which reproduces the scenario described above. The test case fails before the
fix is added and succeeds when the fix is applied.

Result:

After this change, the timeout scheduled on the executor will be cleared, thus freeing up thread resources.
2020-09-16 09:40:42 +02:00
Aayush Atharva
836bb74051
Call long as l instead of i (#10578)
Motivation:

Long should be called `l` instead of `i` because `i` is generally used for `int`.

Modification:

Changed `i` to `l`.

Result:
Better naming
2020-09-16 07:58:03 +02:00
Paul Lysak
aa85ea9947
Make MQTT property value publicly accessible (#10577)
Motivation:

There was no way to read MQTT properties outside of `io.netty.handler.codec.mqtt` package

Modification:

Expose `MqttProperties.MqttProperty` fields `value` and `propertyId` via public methods

Result:

It's possible to read MQTT properties now
2020-09-16 07:56:17 +02:00
Kevin Wu
95ce1b95ea
Fix DeleteOnExitHook cause memory leak (#10560)
Motivation:

If DeleteOnExitHook is in the open state and the program runs for a long time, the DeleteOnExitHook file keeps increasing.
This results in a memory leak


Modification:

I re-customized a DeleteOnExitHook hook. If DeleteOnExitHook is turned on, this hook will be added when creating a temporary file. After the request ends and the corresponding resources are released, the current file will be removed from this hook, so that it will not increase all the time.

Result:

Fixes https://github.com/netty/netty/issues/10351
2020-09-15 16:38:42 +02:00
dependabot[bot]
e8aaf7f586
Bump ant from 1.9.7 to 1.9.15 (#10573)
Bumps ant from 1.9.7 to 1.9.15.

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2020-09-15 15:12:10 +02:00
Paul Lysak
3b47427cf7
Fix NPE with MqttUnsubAckMessage - regression of MQTT5 support (#10557)
Recent changes for MQTT5 may cause NPE if UNSUBACK message is created with `MqttMessageFactory`
and client code isn't up-to-date.

Modifications:

Added default body in `MqttUnsubAckMessage` constructor if null body is passed,
added null checks in `encodeUnsubAckMessage`

Result:

`MqttUnsubAckMessage` created with `MqttMessageFactory` doesn't cause NPE
even if null body is supplied.
2020-09-14 12:04:55 +02:00
Aayush Atharva
d889397c1e
Add PcapWriteHandler Support (#10498)
Motivation:
Write TCP and UDP packets into Pcap `OutputStream` which helps a lot in debugging.

Modification:
Added TCP and UDP Pcap writer.

Result:
New handler can write packets into an `OutputStream`, e.g. a file that can be opened with Wireshark.

Fixes #10385.
2020-09-11 16:06:16 +02:00
Paul Lysak
8c1db6ccb8
Make RetainedHandlingPolicy enum public (#10565)
Motivation:

Impossible to specify retained messages handling policy because the corresponding enum
isn't public (https://github.com/netty/netty/issues/10562)

Modification:

Made `RetainedHandlingPolicy` public

Result:

Fixes #10562.
2020-09-11 08:32:16 +02:00
Chris Vest
1dda2be8aa
Disable Maven build HTTP connection pooling (#10558)
Motivation:
 The builds often fail when downloading dependencies.
 This might be caused by the build taking a long time, and cause pooled connections to be closed
 by the remote end, if they are idle for too long.

Modification:
 Disable connection pooling. This should force Maven to reestablish the connection for each download,
 thus reducing the likelihood of the remote end closing connections we wish to use.

Result:
 I'll leave it up the statistis of our CI to confirm, but we should see more stable builds.
2020-09-09 17:07:26 +02:00
Francesco Nigro
162e59848a
Improve predictability of writeUtf8/writeAscii performance (#10368)
Motivation:

writeUtf8 can suffer from inlining issues and/or megamorphic call-sites on the hot path due to ByteBuf hierarchy

Modifications:

Duplicate and specialize the code paths to reduce the need of polymorphic calls

Result:

Performance are more stable in user code
2020-09-09 16:10:26 +02:00
Norman Maurer
5631f1b2b7
Make kernel version detection code in EpollReuseAddrTest more robust (#10556)
Motivation:

When we try to parse the kernel version we need to be careful what to
expect. Especially when a custom kernel is used we may get extra chars
in the version numbers. For example I had this one fail because of my
custom kernel that I built for io_uring:

5.8.7ioring-fixes+

Modifications:

- Try to be a bit more lenient when parsing
- If we cant parse the kernel version just use 0.0.0

Result:

Tests are more robust
2020-09-09 15:51:37 +02:00
Norman Maurer
b43ce7ae1d
Fix regression when trying to bind an EpollDatagramChannel with port (#10552)
only

Motivation:

4b7dba1 introduced a change which was not 100 % complete and so
introduce a regression when a user specified to use
InetProtocolFamily.IPv4 and trying to bind to a port (without specify
the ip).

Modifications:

- Fix regression by respect the InetProtocolFamily
- Add unit test

Result:

Fix regression when binding to port explicit
2020-09-09 10:44:46 +02:00
root
957ef746d8 [maven-release-plugin] prepare for next development iteration 2020-09-08 05:26:25 +00:00
root
ada9c38c0a [maven-release-plugin] prepare release netty-4.1.52.Final 2020-09-08 05:26:05 +00:00
Francesco Nigro
38f01e0840
Reduce garbage on MQTT (#10509)
Reduce garbage on MQTT encoding

Motivation:

MQTT encoding and decoding is doing unnecessary object allocation in a number of places:
- MqttEncoder create many byte[] to encode Strings into UTF-8 bytes
- MqttProperties uses Integer keys instead of int
- Some enums valueOf create unnecessary arrays on the hot paths
- MqttDecoder was using unecessary Result<T>

Modification:

- ByteBufUtil::utf8Bytes and ByteBufUtil::reserveAndWriteUtf8 allows to perform the same operation GC-free
- MqttProperties uses a primitive key map
- Implemented GC free const table lookup/switch valueOf
- Use some bit-tricks to pack 2 ints into a single primitive long to store both result and numberOfBytesConsumed and use byte[].length to compute numberOfByteConsumed on fly. These changes allowed to save creating Result<T>.

Result:
Significantly less garbage produced in MQTT encoding/decoding
2020-09-04 18:27:22 +02:00
Graham Edgecombe
a99a8f2a1f
Fix ByteBufUtil.getBytes() incorrectly sharing the array in some cases (#10529)
Motivation:

If ByteBufUtil.getBytes() is called with copy=false, it does not
correctly check that the underlying array can be shared in some cases.

In particular:

* It does not check that the arrayOffset() is zero. This causes it to
  incorrectly return the underlying array if the other conditions are
  met. The returned array will be longer than requested, with additional
  unwanted bytes at its start.

* It assumes that the capacity() of the ByteBuf is equal to the backing
  array length. This is not true for some types of ByteBuf, such as
  PooledHeapByteBuf. This causes it to incorrectly return the underlying
  array if the other conditions are met. The returned array will be
  longer than requested, with additional unwanted bytes at its end.

Modifications:

This commit fixes the two bugs by:

* Checking that the arrayOffset() is zero before returning the
  underlying array.

* Comparing the requested length to the underlying array's length,
  rather than the ByteBuf's capacity, before returning the underlying
  array.

This commit also adds a series of test cases for ByteBufUtil.getBytes().

Result:

ByteBufUtil.getBytes() now correctly checks whether the underlying array
can be shared or not.

The test cases will ensure the bug is not reintroduced in the future.
2020-09-04 13:15:35 +02:00
Norman Maurer
5157d3b8e0
Revert "Support session cache for client and server when using native SSLEngine implementation (#10331)" (#10528)
Motivation:

This reverts commit 825916c7f0 as it turns out it introduced a big performance regression.

Modifications:

Revert 825916c7f0

Result:

Performance of TLS is back to normal
2020-09-03 08:31:57 +02:00
Maksym Ostroverkhov
fb39fc450e
Http2ConnectionHandlerBuilder: make initialSettings() public (#10526)
Motivation:
be able to modify Http2Settings of Http2ConnectionHandlerBuilder.

Modifications:
make initialSettings() of Http2ConnectionHandlerBuilder public.

Result:
public initialSettings() of Http2ConnectionHandlerBuilder allow Http2Settings modification.
2020-09-02 21:12:35 +02:00
Norman Maurer
4ececfb70d
Create a stackless ClosedChannelException to reduce overhead when the… (#10523)
Motivation:

In some benchmarks closing the Channel attributes to a lot of overhead due the call of fillInStackTrace(). We should reduce this overhead.

Modifications:

- Create a StacklessClosedChannelException and use it to reduce overhead.
- Only call ChannelOutboundBuffer.failFlushed(...) when there was a flushed message at all.

Result:

Less performance overhead when closing the Channel
2020-09-01 15:25:41 +02:00
Aayush Atharva
5d27a3d0bb
Binary search based IpSubnetFilter (#10492)
Motivation:

`IpSubnetFilter` uses Binary Search for IP Address search which is fast if we have large set of IP addresses to filter.

Modification:

Added `IpSubnetFilter` which takes `IpSubnetFilterRule` for filtering.

Result:
Faster IP address filter.
2020-09-01 11:07:51 +02:00
Aayush Atharva
39c840d711
Fix JavaDoc of RuleBasedIpFilter (#10521)
Motivation:

`RuleBasedIpFilter` had JavaDoc `{@link #channelRejected(ChannelHandlerContext, SocketAddress)}` instead of `{@link AbstractRemoteAddressFilter#channelRejected(ChannelHandlerContext, SocketAddress)}`.

Modification:
Added `AbstractRemoteAddressFilter` reference.

Result:
Fixed JavaDoc error and made documentation more clear.
2020-09-01 11:02:10 +02:00
Michael Nitschinger
d487086a2c
Allow to customize how SslMasterKeyHandler is enabled (#10513)
Motivation:

Right now after a SslMasterKeyHandler is added to the pipeline,
it also needs to be enabled via a system property explicitly. In
some environments where the handler is conditionally added to
the pipeline this is redundant and a bit confusing.

Modifications:

This changeset keeps the default behavior, but allows child
implementations to tweak the way on how it detects that it
should actually handle the event when it is being raised.

Also, removed a private static that is not used in the wireshark
handler.

Result:

Child implementations can be more flexible in deciding when
and how the handler should perform its work (without changing
any of the defaults).
2020-08-31 10:34:44 +02:00
Francesco Nigro
d2c03c9a29
Improve MqttMessageType::valueOf cost (#10400)
Motivation:

MqttMessageType::valueOf has O(N) cost

Modifications:

MqttMessageType::valueOf uses a const lookup table

Result:

MqttMessageType::valueOf has O(1) cost
2020-08-31 10:32:33 +02:00
Paul Lysak
be2cd68443
MQTT5 support for netty-codec-mqtt (#10483)
Motivation:

 MQTT Specification version 5 was released over a year ago,
 netty-codec-mqtt should be changed to support it.

Modifications:

  Added more message and header types in `io.netty.handler.codec.mqtt`
  package in `netty-coded-mqtt` subproject,
  changed `MqttEncoder` and `MqttDecoder` to handle them properly,
  added attribute `NETTY_CODEC_MQTT_VERSION` to track protocol version

Result:

  `netty-codec-mqtt` supports both MQTT5 and MQTT3 now.
2020-08-31 09:16:40 +02:00
Aayush Atharva
fde6bc8885
Add ipv4AddressToInt(Inet4Address) in NetUtils (#10500)
Motivation:

We're converting `Inet4Address` to `Integer` quite frequently so it's a good idea to keep that code in `NetUtils`.

Modification:

Added ipv4AddressToInt(Inet4Address) in NetUtils

Result:
Easy conversion of  `Inet4Address` to `Integer`.
2020-08-31 09:13:14 +02:00
Norman Maurer
6c1840749d
Use all configured nameservers when using DnsNameResolver in all cases (#10503)
Motivation:

Due a change introduced in 68105b257d we incorrectly skipped the usage of nameservers in some cases.

Modifications:

Only fetch a new stream of nameserver if the hostname not matches the original hostname in the query.

Result:

Use all configured nameservers. Fixes https://github.com/netty/netty/issues/10499
2020-08-31 09:12:12 +02:00
Norman Maurer
bc344617b4
Adjust testsuite to pass on JDK15+ (#10511)
Motivation:

JDK15 is about to be released as GA, we should ensure netty works and builds on it. SSLSession#getPeerCertificateChain() throws UnsupportedOperationException in JDK15 and later as it was deprecated before and people should use SSLSession#getPeerCertificates(). We need to account for that in our tests

Modifications:

- Catch UnsupportedOperationException in our testsuite and ignore it when on JDK15+ while rethrowing it otherwise.

Result:

Testsuite passes on JDK15+
2020-08-31 09:05:02 +02:00
Chris Vest
82fa2d780a Remove javassist dependency (#10514)
Motivation:
 Avoid keeping unused dependencies around.

Modification:
 Remove all references to javassist dependency, since it does not appear to be used by anything.

Result:
 One less dependency to worry about.
2020-08-31 09:02:50 +02:00
Aayush Atharva
ee77fad3f8
Minor typo (#10518)
Motivation:
I was working on the transport part in Netty (ofc, solving a major issue) and I found this typo so thought to fix it.

Modification:
Fixed Typo

Result:
No more confusion between `us` and `use`.
2020-08-31 08:59:27 +02:00
Nick Hill
b908c82b7b
Lazily construct contained DataOutputStream in ByteBufOutputStream (#10507)
Motivation

This is used solely for the DataOutput#writeUTF8() method, which may
often not be used.

Modifications

Lazily construct the contained DataOutputStream in ByteBufOutputStream.

Result

Saves an allocation in some common cases
2020-08-28 09:22:58 +02:00
Nick Hill
8e8f01e01d
Use ByteBuf#isAccessible() in more places (#10506)
Motivation

ByteBuf has an isAccessible method which was introduced as part of ref
counting optimizations but there are some places still doing
accessibility checks by accessing the volatile refCnt() directly.

Modifications

- Have PooledNonRetained(Duplicate|Sliced)ByteBuf#isAccessible() use
their refcount delegate's isAccessible() method
- Add static isAccessible(buf) and ensureAccessible(buf) methods to
ByteBufUtil
(since ByteBuf#isAccessible() is package-private)
- Adjust DefaultByteBufHolder and similar classes to use these methods
rather than access refCnt() directly

Result

- More efficient accessibility checks in more places
2020-08-28 09:18:13 +02:00
Norman Maurer
98c98d53d7
Don't try to remove the task when the underlying executor fails the execution of self (#10505)
Motivation:

It makes no sense to remove the task when the underlying executor fails as we may be able to pick it up later. Beside this the used Queue doesnt support remove(...) and so will throw.

Modifications:

Remove the queue.remove(...) call

Result:

Fixes https://github.com/netty/netty/issues/10501.
2020-08-27 08:22:41 +02:00
Violeta Georgieva
4be9387151
Add validation when constructing Http2FrameLogger (#10495)
Motivation:

There should be a validation for the input arguments when constructing Http2FrameLogger

Modification:

Check that the provided arguments are not null

Result:

Proper validation when constructing Http2FrameLogger
2020-08-21 14:26:08 +02:00
Norman Maurer
b2e52e98a3
Update netty-tcnative to 2.0.34.Final (#10494)
Motivation:

A new netty-tcative was released

Modifications:

Update to latest version

Result:

Use latest version
2020-08-21 14:12:00 +02:00
Andrey Mizurov
f51ae686c4
Small fix that takes into account the remainder when assigning the size (see #10453) (#10491)
Motivation:

This is small fixes for #10453 PR according @njhill and @normanmaurer conversation.

Modification:

Simple refactor and takes into account remainder when calculate size.

Result:

Behavior is correct
2020-08-21 09:18:48 +02:00
Chris Vest
0bbe4ce9fd
Expose a LoggingDnsQueryLifeCycleObserverFactory (#10490)
Expose a LoggingDnsQueryLifeCycleObserverFactory

Motivation:
 There is a use case for having logging in the DnsNameResolver, similar to the LoggingHandler.
 Previously, one could set `traceEnabled` on the DnsNameResolverBuilder, but this is not very configurable.
 Specifically, the log level and the logger context cannot be changed.

Modification:
 Expose a LoggingDnsQueryLifeCycleObserverFactory, that permit changing the log-level
 and logger context.

Result:
 It is now possible to get logging in the DnsNameResolver at a custom log level and logger,
 without very much effort.

Fixes #10485
2020-08-19 20:22:38 +02:00
Norman Maurer
830fc0d660
Remove incorrect javadocs that were not updated for new versions of netty (#10487)
Motivation:

At some point a ChannelHandlerContext did have its own AttributeMap which is not true since 4.1.x was released. Unfortunally we missed to update the javadocs and so these don't reflect reality

Modifications:

Update javadocs

Result:

Fixes https://github.com/netty/netty/issues/10477
2020-08-19 17:13:01 +02:00
Norman Maurer
32178fac7f
Enable SSL_MODE_ENABLE_FALSE_START if jdkCompatibilityMode is false (#10407)
Motivation:

To reduce latency and RTTs we should use TLS False Start when jdkCompatibilityMode is not required and its supported

Modifications:

Use SSL_MODE_ENABLE_FALSE_START when jdkCompatibilityMode is false

Result:

Less RTTs and so lower latency when TLS False Start is supported
2020-08-18 19:00:08 +02:00
Norman Maurer
46cb4015ff
Correctly limit queries done to resolve unresolved nameservers (#10478)
Motivation:

We need limit the number of queries done to resolve unresolved nameserver as otherwise we may never fail the resolve when there is a missconfigured dns server that lead to a "resolve loop".

Modifications:

- When resolve unresolved nameservers ensure we use the allowedQueries as max upper limit for queries so it will eventually fail
- Add unit tests

Result:

No more possibility to fail in a loop while resolve. This is related to https://github.com/netty/netty/issues/10420
2020-08-14 16:02:53 +02:00
Violeta Georgieva
b27914e302
AbstractDiskHttpData#getChunk closes fileChannel only if everything w… (#10481)
Motivation:
AbstractDiskHttpData#getChunk opens and closes fileChannel every time when it is invoked,
as a result the uploaded file is corrupted. This is a regression caused by #10270.

Modifications:

- Close the fileChannel only if everything was read or an exception is thrown
- Add unit test

Result:
AbstractDiskHttpData#getChunk closes fileChannel only if everything was read or an exception is thrown
2020-08-14 10:54:43 +02:00
Norman Maurer
fdd4706783
Include TLSv1.3 ciphers as recommented ciphers for HTTP2 (#10480)
Motivation:

We should include TLSv1.3 ciphers as well as recommented ciphers these days for HTTP/2. That is especially true as Java supports TLSv1.3 these days out of the box

Modifications:

- Add TLSv1.3 ciphers that are recommended by mozilla as was for HTTP/2
- Add unit test

Result:

Include TLSv1.3 ciphers as well
2020-08-13 20:33:07 +02:00
Norman Maurer
92fff79c5f
DnsAddressResolverGroup should respect configured EventLoop (#10479)
Motivation:

DnsAddressResolverGroup allows to be constructed with a DnsNameResolverBuilder and so should respect its configured EventLoop.

Modifications:

- Correctly respect the configured EventLoop
- Ensure there are no thread-issues by calling copy()
- Add unit tests

Result:

Fixes https://github.com/netty/netty/issues/10460
2020-08-13 20:30:14 +02:00
Kevin Wu
54bfd21e52
Fix #10434 OutOfDirectMemoryError causes cpu load too high and socket is full (#10457)
Motivation:

When we were using the netty http protocol, OOM occurred, this problem has been in 4.1.51.Final Fix [# 10424](https://github.com/netty/netty/issues/10424), even if OOM is up, the service will still receive new connection events, will occur again OOM and eventually cause the connection not to be released. 

code `byteBuf = allocHandle.allocate(allocator);`

Modification:

I fail to create buffer when I try to receive new data, i determine if it is OOM then the close read event releases the connection.
```java
        if (close || cause instanceof OutOfMemoryError || cause instanceof IOException) {
            closeOnRead(pipeline);
        }
```

Result:

Fixes # [10434](https://github.com/netty/netty/issues/10434).
2020-08-13 10:14:19 +02:00
Eric Anderson
61b0fa97d7
Permit h2 PRIORITY frames with a dependency on the connection (#10473)
Motivation:

Setting a dependency on the connection is normal and permitted; streams
actually default to depending on the connection. Using a PRIORITY frame
with a dependency on the connection could reset a previous PRIORITY,
change the relative weight, or make all streams dependent on one stream.

The previous code was disallowing these usages as it considered
depending on the connection to be a validation failure.

Modifications:

Loosen validation check to also allow depending on the connection. Fix
error message when the validation check fails.

Result:

Setting a dependency on connection would be permitted. Fixes #10416
2020-08-11 13:50:56 -07:00
Norman Maurer
564e817cfb
Cleanup Conscrypt initialization (#10466)
Motivation:

How we init our static fields in Conscrypt was kind of error-prone and may even lead to NPE later on when methods were invoked out of order.

Modifications:

- Move all the init code to a static block
- Remove static field which is not needed anymore

Result:

Cleanup and also fixes https://github.com/netty/netty/issues/10413
2020-08-11 21:00:51 +02:00
Chris Vest
cfeda0fef2
Fix overflow bug in GenericEventExecutorChooser (#10468)
Motivation:
 The executor chooser should pluck executors in round-robin, but at the 32-bit
 overflow boundary, the round-robin sequence was disrupted when the number of
 executors are not a power of 2.

Modification:
 Changed the index counter from a 32-bit to a 64-bit long. The overflow bug is
 still technically there, but it now takes so long to reach that it will never
 happen in practice. For example, 2^63 nanoseconds is almost 300 years.

Result:
 The round-robin behaviour for all EventExecutorChoosers is now preserved in
 practice.
 This fixes #10423.
2020-08-11 20:53:48 +02:00
Chris Vest
3d7ec896ac
Avoid name-clash with future java.lang.Record (#10470)
Motivation:
 Recent Intellij versions are starting to anticipate
 future versions of Java that include a
 `java.lang.Record` class, and the Intellij compiler
 gets confused by the `Record` class in our
 `ResorceLeakDetector`.

Modification:
 Rename our `Record` class to `TracerRecord`.
 This matches what the class is doing, while avoiding
 any future name clashes.

Result:
 Intellij can now compile the project again, even when
 configured to use a future (snapshot or early access)
 version of Java.
2020-08-11 20:51:39 +02:00
Chris Vest
f58223982c
Fix netty_unix_errors.c compilation on MacOS (#10469)
Motivation:
 The feature test macros did not work as expected on MacOS, so we ended up compiling for the GNU
 variant which resulted in compilation errors.
Modification:
 Add `__APPLE__` as another indicator to use the XSI variant of `strerror_r`.
Result:
 The project now once again compiles on MacOS.
2020-08-11 14:00:08 +02:00
Chris Vest
8007f907dd
Use strerror_r for JNI error messages. (#10463)
Motivation:
We previously relied on `strerror`, but this function is unfortunately not thread-safe.

Modification:
The use of `strerror` has been changed to `strerror_r`, which is thread-safe.
This function has a more complicated API, and has portability concerns that needs to be handled.
This accounts for the relatively large increase in lines of code.

Result:
Error messages from JNI are now always generated in a thread-safe way.
2020-08-11 09:58:56 +02:00