Commit Graph

625 Commits

Author SHA1 Message Date
skyguard1
d58d8a1df8
Remove useless imports and redundant type cast (#11639)
Motivation:

There are some redundant imports and unnecessary type cast

Modification:

Remove useless imports and unnecessary type cast

Result:

The code is cleaner than original

Signed-off-by: xingrufei <xingrufei@sogou-inc.com>
2021-09-02 19:14:00 +02:00
Chris Vest
b3d3269b8c
Avoid overspecifying ScheduledFuture when Future will do (#11636)
Motivation:
Overspecifying interfaces can increase coupling and make refactoring harder down the line.

Modification:
Places that were specifying fields, variables, and return types as ScheduledFuture, but did not use any features specific to ScheduledFuture, have been changed to specify Future instead.

Result:
Cleaner code.
2021-08-31 16:05:21 +02:00
Chris Vest
322f75c5a2
Fix a problem with IP protocol version confusion on MacOS when TCP FastOpen is enabled (#11588)
Motivation:
This fixes a bug that would result in an `io.netty.channel.unix.Errors$NativeIoException: connectx(..) failed: Address family not supported by protocol family` error.
This happens when the connecting socket is configured to use IPv6 but the address being connected to is IPv4.
This can occur because, for instance, Netty and `InetAddress.getLoopbackAddress()` have different preferences for IPv6 vs. IPv4.

Modification:
Pass the correct ipv6 or ipv4 flags to connectx, depending on whether the socket was created for AF_INET or AF_INET6, rather than relying on the IP version of the destination address.

Result:
No more issue with TCP FastOpen on MacOS when using addresses of the "wrong" IP version.
2021-08-18 20:43:01 +02:00
Norman Maurer
228a01e268
Use StandardSocketOptions#IP_MULTICAST_IF as default source when joing multicast groups (#11585)
Motivation:

We should use StandardSocketOptions#IP_MULTICAST_IF as default source when joing multicast groups and only try to use the localAddress if this returns null.

Modifications:

First check if StandardSocketOptions#IP_MULTICAST_IF was set and if so use the network interface when joining mulicast groups

Result:

Fixes https://github.com/netty/netty/issues/11541
2021-08-18 12:11:48 +02:00
Netty Project Bot
f8f17f676d [maven-release-plugin] prepare for next development iteration 2021-08-16 08:23:23 +00:00
Netty Project Bot
b785fa9eed [maven-release-plugin] prepare release netty-4.1.67.Final 2021-08-16 08:23:20 +00:00
Norman Maurer
7903a44a14 Revert "[maven-release-plugin] prepare release netty-4.1.67.Final"
This reverts commit f773733af6.
2021-08-16 10:14:09 +02:00
Norman Maurer
ec23189e72 Revert "[maven-release-plugin] prepare for next development iteration"
This reverts commit 76c55caeb9.
2021-08-16 10:13:59 +02:00
Netty Project Bot
76c55caeb9 [maven-release-plugin] prepare for next development iteration 2021-08-12 17:12:05 +00:00
Netty Project Bot
f773733af6 [maven-release-plugin] prepare release netty-4.1.67.Final 2021-08-12 17:12:01 +00:00
Chris Vest
f750e2eb6a
Add support for client-side TCP FastOpen to KQueue MacOS (#11560)
Motivation:
The MacOS-specific `connectx(2)` system call make it possible to establish client-side connections with TCP FastOpen.

Modification:
Add support for TCP FastOpen to the KQueue transport, and add the `connectx(2)` system call to `BsdSocket`.

Result:
It's now possible to use TCP FastOpen when initiating connections on MacOS.
2021-08-12 13:38:46 +02:00
Chris Vest
699549dcbc
Improve test failure reporting of EpollSocketChannelConfigTest (#11570)
Motivation:
This test is inherently flaky due to file descriptor reuse.
Even though we have taken steps to make it less flaky, it still fails sometimes.
When it does, the error message is not very helpful.

Modification:
Make use of assertThrows and assertThat to get more descriptive error messages when the tests fail.

Result:
More meaningful messages on test failures, which may help us make the tests more resilient in the future
2021-08-12 11:37:56 +02:00
Aayush Atharva
beb4588b4a
Inline variables to make code more readable (#11565)
Motivation:
There are lots of redundant variable declarations which should be inlined to make good look better.

Modification:
Made variables inlined.

Result:
Less redundant variable and more readable code.
2021-08-11 17:05:48 +02:00
Chris Vest
32523c5337
Make the TCP_FASTOPEN channel option transport agnostic (#11559)
Motivation:
TCP FastOpen is a pure optimisation, that is opportunistically applied.
There is no reason to make it specific to the epoll transport, and in the future we could add support to other transports.
Besides, the client-side equivalent, TCP_FASTOPEN_CONNECT, is already transport agnostic.

Modification:
Move the TCP_FASTOPEN channel option from EpollChannelOption to ChannelOption.
Mark the field in EpollChannelOption as deprecated.

Result:
All channel options related to TCP FastOpen are now transport agnostic.
However, we still only actually support TFO on the epoll transport.
2021-08-10 09:03:05 +02:00
Chris Vest
45c97fbdfd
Make EpollSocketChannelConfigTest tollerant of fd reuse race (#11552)
Motivation:
We cannot control when "the system" reuses file descriptors.
This makes any test that assert on the behaviour of closed file descriptors inherently racy.

Modification:
Allow the EpollSocketChannelConfigTest socketoption tests a few tries to get the correct assertion on the behaviour of closed socket file descriptors.

Result:
The EpollSocketChannelConfigTest should now be much less flaky.
2021-08-06 08:45:24 +02:00
Aayush Atharva
1ce28e80d1
Remove Unused Imports (#11546)
Motivation:
There are lots of imports which are unused. We should get rid of them to make the code look better,

Modification:
Removed unused imports.

Result:
No unused imports.
2021-08-05 13:54:48 +02:00
Netty Project Bot
2a184ab932 [maven-release-plugin] prepare for next development iteration 2021-07-16 06:23:23 +00:00
Netty Project Bot
e43d0d99f1 [maven-release-plugin] prepare release netty-4.1.66.Final 2021-07-16 06:23:17 +00:00
Violeta Georgieva
3f9a5f50be
Add support for Unix domain datagram sockets when using native epoll/kqueue transport (#11423)
Motivation:

There are use cases when Unix domain datagram sockets are needed for communication. This PR adds such support for Epoll/KQueue.

Modification:

- Expose Channel, Config and Packet interfaces/classes for Unix domain datagram sockets. All interfaces/classes are in `transport-native-unix-common` module in order to be available for KQueue and Epoll implementations
- Add JNI code for Unix domain datagram sockets
- Refactor `DatagramUnicastTest` so that it can be used for testing also Unix domain datagram sockets
- Add Unix domain datagram sockets implementation for KQueue transport
- Add Unix domain datagram sockets implementation for Epoll transport

Result:

Fixes #6737
2021-07-09 09:45:43 +02:00
Norman Maurer
7c6be99da0
Migrate rest of epoll module to junit5 (#11442)
Motivation:

We should update to use junit5 in all modules.

Modifications:

Adjust missing epoll tests to use junit5

Result:

Part of https://github.com/netty/netty/issues/10757
2021-07-02 08:04:48 +02:00
Maksym Ostroverkhov
2abe20a6b5
Tail tasks queue: configure separately from tasks queue (#11400)
Motivation:

IO transports (primarily epoll, but also applies to kqueue, nio) cant be configured with separate tail tasks queue factory -
instead single queue factory is used for both normal tasks and tail tasks.

Modifications:

Add constructor accepting tail EventLoopTaskQueueFactory to aforementioned transports

Result:

IO transports can be configured with separate tail tasks
2021-07-01 10:51:10 +02:00
Riley Park
bc7434aedb Migrate testsuite, transport-native-epoll, transport-native-kqueue, and transport-native-unix-common-tests tests to JUnit 5 (#11320)
Motivation:

JUnit 5 is more expressive, extensible, and composable in many ways, and it's better able to run tests in parallel.

Modifications:

Use JUnit5 in tests

Result:

Related to https://github.com/netty/netty/issues/10757
2021-05-27 15:55:58 +02:00
Netty Project Bot
02142a2510 [maven-release-plugin] prepare for next development iteration 2021-05-19 12:09:18 +00:00
Netty Project Bot
9e06241ddb [maven-release-plugin] prepare release netty-4.1.65.Final 2021-05-19 12:09:14 +00:00
Norman Maurer
6e866317d2
Ensure we can load the native library or fail the build (#11262)
Motivation:

We used assumeTrue(...) in some places before to detect if we could load the native library but this could lead to the sitation that we not notice if we break native loading.

Modifications:

Always fail if we cant load the native library

Result:

Ensure we not cause any regression in the native loading code in the future
2021-05-18 08:14:32 +02:00
Netty Project Bot
d5c81d9a3e [maven-release-plugin] prepare for next development iteration 2021-05-17 07:04:14 +00:00
Netty Project Bot
67bedf0efe [maven-release-plugin] prepare release netty-4.1.64.Final 2021-05-17 07:04:08 +00:00
Idel Pivnitskiy
3ecb32357c
Use PlatformDependent#normalizedOs() instead of reading os.name prop (#11239)
Motivation:

`PlatformDependent#normalizedOs()` already caches normalized variant of
the value of `os.name` system property. Instead of inconsistently
normalizing it in every case, use the utility method.

Modifications:

- `PlatformDependent`: `isWindows0()` and `isOsx0()` use `NORMALIZED_OS`;
- `PlatformDependent#normalizeOs(String)` define `darwin` as `osx`;
- `OpenSsl#loadTcNative()` does not require `equalsIgnoreCase` bcz `os`
is already normalized;
- Epoll and KQueue: `Native#loadNativeLibrary()` use `normalizedOs()`;
- Use consistent `Locale.US` for lower case conversion of `os.name`;
- `MacOSDnsServerAddressStreamProvider#loadNativeLibrary()` uses
`PlatformDependent.isOsx()`;

Result:

Consistent approach for `os.name` parsing.
2021-05-11 08:52:29 +02:00
Norman Maurer
6b3ec62e6e
Preload classes before calling native OnLoad function to prevent clas… (#11215)
Motivation:

It turns out it is quite easy to cause a classloader deadlock in more recent java updates if you cause classloading while you are in native code. Because of this we should just workaround this issue by pre-load all the classes that needs to be accessed in the OnLoad function.

Modifications:

- Preload all classes that would otherwise be loaded by native OnLoad functions.

Result:

Workaround for https://github.com/netty/netty/issues/11209 and https://bugs.openjdk.java.net/browse/JDK-8266310
2021-05-03 10:22:22 +02:00
Boris Unckel
c3416d8ad2
Utilize i.n.u.internal.ObjectUtil to assert Preconditions (transport*) (#11170) (#11181)
Motivation:

NullChecks resulting in a NullPointerException or IllegalArgumentException, numeric ranges (>0, >=0) checks, not empty strings/arrays checks must never be anonymous but with the parameter or variable name which is checked. They must be specific and should not be done with an "OR-Logic" (if a == null || b == null) throw new NullPointerEx.

Modifications:

* import static relevant checks
* Replace manual checks with ObjectUtil methods

Result:

All checks needed are done with ObjectUtil, some exception texts are improved.

Fixes #11170
2021-04-22 14:13:14 +02:00
Norman Maurer
e27831d767
Fix support for IP_RECVORIGDSTADDR when using native epoll transport (#11173)
Motivation:

While adding support for GRO (b05fdf3ff8) we broke support for IP_RECVORIGDSTADDR when using the native transport. Beside this we also didnt correctly handle IP_RECVORIGDSTADDR when recvmmsg was used.

Modifications:

- Fix support for IP_RECVORIGDSTADDR when using the native epoll transport for normal reads (recvmsg) but also for scattering reads (recvmmsg)
- Remove code from unix code-base as the support is linux specific and we not need the code there anymore

Result:

Fixes https://github.com/netty/netty/issues/11141
2021-04-21 16:04:12 +02:00
Chris Vest
df53de5b68
Less blocking in ChunkedStream (#11150)
Motivation:
We should avoid blocking in the event loop as much as possible.
The InputStream.read() is a blocking method, and we don't need to call it if available() returns a positive number.

Modification:
Bypass calling InputStream.read() if available() returns a positive number.

Result:
Fewer blocking calls in the event loop, in general, when ChunkedStream is used.
2021-04-12 13:40:14 +02:00
Netty Project Bot
6724786dcc [maven-release-plugin] prepare for next development iteration 2021-04-01 10:50:23 +00:00
Netty Project Bot
b78d8f2abd [maven-release-plugin] prepare release netty-4.1.63.Final 2021-04-01 10:50:18 +00:00
Netty Project Bot
e4dd6ee532 [maven-release-plugin] prepare for next development iteration 2021-03-30 17:22:25 +00:00
Netty Project Bot
573d1f4bf5 [maven-release-plugin] prepare release netty-4.1.62.Final 2021-03-30 17:22:18 +00:00
Netty Project Bot
7192ab0f0a [maven-release-plugin] prepare for next development iteration 2021-03-30 07:42:06 +00:00
Netty Project Bot
ccb3ff388f [maven-release-plugin] prepare release netty-4.1.61.Final 2021-03-30 07:42:01 +00:00
Norman Maurer
6c741932aa
Move SegmentedDatagramPacket to transport-native-unix-common (#11121)
Motivation:

As we can supported SegmentedDatagramPacket in multiple native
transports (like in epoll and io_uring) we should just move it to
unix-common so we can share code.

Modification:

- Move SegmentedDatagrampPacket to transport-native-unixu
- Mark the SegmentedDatagramPacket in epoll as deprecated
- Update code to use updated package.

Result:

Possibility of code re-use
2021-03-29 13:55:43 +02:00
Norman Maurer
b05fdf3ff8
Add support for UDP_GRO (#11120)
Motivation:

UDP_GRO can improve performance when reading UDP datagrams. This patch adds support for it.

See https://lwn.net/Articles/768995/

Modifications:

- Add recvmsg(...)
- Add support for UDP_GRO in recvmsg(...) and recvmmsg(...)
- Remove usage of recvfrom(...) and just always use recvmsg(...) or recvmmsg(...) to simplify things
- Refactor some code for sharing
- Add EpollChannelOption.UDP_GRO and the getter / setter in EpollDatagramConfig

Result:

UDP_GRO is supported when the underlying system supports it.
2021-03-29 08:51:08 +02:00
Norman Maurer
7ed840e62a
Use netty-jni-util 0.0.3.Final (#11115)
netty-jni-util 0.0.2.Final is incompatible with static linking. Before
the netty-jni-util dependency was introduced netty-tcnative supported
static linking via NETTY_BUILD_STATIC. netty-jni-util 0.0.3.Final adds
static linking compatibility.

Modifications:

Bump netty-jni-util to version 0.0.3.Final and update to its new API
which requires the caller to manage packagePrefix.

Result:

Using latest version of netty-jni-util and restored static linking
compatibility.
2021-03-25 11:40:22 +01:00
Norman Maurer
72d7ec9d8c
Let's use gcc10 when cross-compiling for LSE support (#11112)
Motivation:

LSE (https://mysqlonarm.github.io/ARM-LSE-and-MySQL/) can have a huge performance difference. Let's ensure we use a compiler that can support it.

Modifications:

Update to gc10 when cross-compiling as it supports LSE and enables it by default

Result:

More optimized builds for aarch64
2021-03-25 09:32:57 +01:00
Norman Maurer
a82af64047
Continue reading when the number of bytes is less then the configured… (#11089)
... number of bytes when using DatagramChannels

Motivation:

In our FixedRecvByteBufAllocator we dont continue to read if the number of bytes is less then what was configured. This is correct when using it for TCP but not when using it for UDP. When using UDP the number of bytes is the maximum of what we want to support but we often end up processing smaller datagrams in general. Because of this we should use contineReading(UncheckedBooleanSupplier) to determite if we should continue reading

Modifications:

- use contineReading(UncheckedBooleanSupplier) for DatagramChannels

Result:

Read more then once in the general case for DatagramChannels with the default config
2021-03-17 13:10:57 +01:00
Norman Maurer
95652fef12
Allow to configure the maximum number of message to write per eventloop (#11086)
Motivation:

Allow to configure the maximum number of messages to write per eventloop run. This can be useful to ensure we read data in a timely manner and not let writes dominate the CPU time. This is especially useful in protocols like QUIC where you need to read "fast enough" as otherwise you may not read the ACKs fast enough.

Modifications:

- Add new ChannelOption / config that allows to limit the number of messages to write per eventloop run.
- Respect this setting for DatagramChannels

Result:

Reduce the risk of having WRITES block the processing of other events in a timely manner

Co-authored-by: terrarier2111 <58695553+terrarier2111@users.noreply.github.com>
2021-03-17 12:58:06 +01:00
Scott Mitchell
0b0c234bed
SslHandler flushing with TCP Fast Open fix (#11077)
Motivation:
SslHandler owns the responsibility to flush non-application data
(e.g. handshake, renegotiation, etc.) to the socket. However when
TCP Fast Open is supported but the client_hello cannot be written
in the SYN the client_hello may not always be flushed. SslHandler
may not wrap/flush previously written/flushed data in the event
it was not able to be wrapped due to NEED_UNWRAP state being
encountered in wrap (e.g. peer initiated renegotiation).

Modifications:
- SslHandler to flush in channelActive() if TFO is enabled and
  the client_hello cannot be written in the SYN.
- SslHandler to wrap application data after non-application data
  wrap and handshake status is FINISHED.
- SocketSslEchoTest only flushes when writes are done, and waits
  for the handshake to complete before writing.

Result:
SslHandler flushes handshake data for TFO, and previously flushed
application data after peer initiated renegotiation finishes.
2021-03-14 14:18:27 +01:00
Chris Vest
3f9750173c
Make EpollSocketConnectTest correctly configure TFO (#11083)
Motivation:
The EpollSocketConnectTest was not correctly configuring TCP Fast Open on the server socket.
It's an option, not a child option.

Modification:
EpollSocketConnectTest now correctly enables TCP Fast Open on the server side, when available, for the test that needs it.

Result:
Test covers what it was intended to.
2021-03-14 14:17:06 +01:00
Norman Maurer
773ecb81fe
Also support CompositeByteBuf with SegmentedDatagramPacket (#11081)
Motivation:

c22c6b845d introduced support for
UDP_SEGMENT but did restrict it to continous buffers. This is not needed
as it is also fine to use CompositeByteBuf

Modifications:

- Allow to use CompositeByteBuf as well
- Add unit test

Result:

More flexible usage of segmented datagrams possible
2021-03-12 15:37:46 +01:00
Netty Project Bot
e83132fcf2 [maven-release-plugin] prepare for next development iteration 2021-03-09 08:18:31 +00:00
Netty Project Bot
eef26e8bb5 [maven-release-plugin] prepare release netty-4.1.60.Final 2021-03-09 08:18:26 +00:00
Netty Project Bot
b8d843757b [maven-release-plugin] rollback the release of netty-4.1.60.Final 2021-03-09 07:39:13 +00:00