Commit Graph

525 Commits

Author SHA1 Message Date
Steve Buzzard
70731bfa7e Added UDP multicast (with caveats: getInterface, getNetworkInterface, block or loopback-mode-disabled operations).
Motivation:

Provide epoll/native multicast to support high load multicast users (we are using it for a high load telecomm app at my day job).

Modification:

Added support for source specific and any source multicast for epoll transport. Some caveats: no support for disabling loop back mode, retrieval of interface and block operation, all of which tend to be less frequently used.

Result:

Provides epoll transport multicast for common use cases.

Co-authored-by: Norman Maurer <norman_maurer@apple.com>
2019-05-25 08:00:16 +02:00
Julien Viet
e348bd9217 KQueueEventLoop | EpollEventLoop may incorrectly update registration when FD is reused.
Motivation:

The current KQueueEventLoop implementation does not process concurrent domain socket channel registration/unregistration in the order they actual
happen since unregistration are delated by an event loop task scheduling. When a domain socket is closed, it's file descriptor might be reused
quickly and therefore trigger a new channel registration using the same descriptor.

Consequently the KQueueEventLoop#add(AbstractKQueueChannel) method will overwrite the current inactive channels having the same descriptor
and the delayed KQueueEventLoop#remove(AbstractKQueueChannel) will remove the active channel that replaced the inactive one.

As active channels are registered, events for this file descriptor won't be processed anymore and the channels will never be closed.

The same problem can also happen in EpollEventLoop. Beside this we also may never remove the AbstractEpollChannel from the internal map
when it is unregistered which will prevent it from be GC'ed

Modifications:

- Change logic of native KQueue and Epoll implementations to ensure we correctly handle the case of FD reuse
- Only try to update kevent / epoll if the Channel is still open (as otherwise it will be handled by kqueue / epoll itself)
- Correctly remove AbstractEpollChannel from internal map in all cases
- Make implementation of closeAll() consistent for Epoll and KQueueEventLoop

Result:

KQueue and Epoll native transports correctly handle FD reuse

Co-authored-by: Norman Maurer <norman_maurer@apple.com>
2019-05-22 09:23:09 +02:00
root
ba06eafa1c [maven-release-plugin] prepare for next development iteration 2019-04-30 16:42:29 +00:00
root
49a451101c [maven-release-plugin] prepare release netty-4.1.36.Final 2019-04-30 16:41:28 +00:00
root
baab215f66 [maven-release-plugin] prepare for next development iteration 2019-04-17 07:26:24 +00:00
root
dfe657e2d4 [maven-release-plugin] prepare release netty-4.1.35.Final 2019-04-17 07:25:40 +00:00
Norman Maurer
741bcd485d
Make Multicast tests more robust (#9053)
Motivation:

86dd388637 reverted the usage of IPv6 Multicast test. This commit makes the whole multicast testing a lot more robust by selecting the correct interface in any case and also reverts the `@Ignore`

Modifications:

- More robust multicast testing by selecting the right NetworkInterface
- Remove the `@Ignore` again for the IPv6 test

Result:

More robust multicast testing
2019-04-15 21:39:31 +02:00
Norman Maurer
9e491dda14 Ignore ipv6 multicast test that was added in 778ff2057e for now
Motivation:

The multicast ipv6 test fails on some systems. As I just added it let me ignore it for now while investigating.

Modifications:

Add @ignore

Result:

Stable testsuite while investigate
2019-04-12 16:56:44 +02:00
Norman Maurer
778ff2057e
Add IPv6 multicast test to testsuite (#9037)
Motivation:

We currently only cover ipv4 multicast in the testsuite but we should also have tests for ipv6.

Modifications:

- Add test for ipv6
- Ensure we only try to run multicast test for ipv4 / ipv6 if the loopback interface supports it.

Result:

Better test coverage
2019-04-12 12:29:08 +02:00
Vladimir Kostyukov
0a0da67f43 Introduce SingleThreadEventLoop.registeredChannels (#8428)
Motivation:

Systems depending on Netty may benefit (telemetry, alternative even loop scheduling algorithms) from knowing the number of channels assigned to each EventLoop.

Modification:

Expose the number of channels registered in the EventLoop via SingleThreadEventLoop.registeredChannels.

Result:

Fixes #8276.
2019-03-28 11:33:12 +00:00
Lunfu Zhong
e7b3195570 Support ALLOW_HALF_CLOSURE channel option on Unix domain socket. (#8932)
Motivation:

Since DomainSocketChannel is a DuplexChannel,  which be able to shutdown input or output individually on demands, but ALLOW_HALF_CLOSURE channel option has not been supported yet.

I thought this could be a missing feature of Unix domain socket, so here the PR for it.

Modifications:

1. Added allHalfClosure property both in  EpollDomainSocketChannelConfig and KQueueDomainSocketChannelConfig,
2. Enabled isAllowHalfClosure method of native channel to support domain channel config,
3. Created EpollDomainSocketShutdownOutputByPeerTest and KQueueDomainSocketShutdownOutputByPeerTest to verify the change.

Result:

ALLOW_HALF_CLOSURE channel option can be set with DomainSocketChannel, and no more warning of Unknown channel option 'ALLOW_HALF_CLOSURE'.
2019-03-19 11:24:07 +01:00
root
92b19cfedd [maven-release-plugin] prepare for next development iteration 2019-03-08 08:55:45 +00:00
root
ff7a9fa091 [maven-release-plugin] prepare release netty-4.1.34.Final 2019-03-08 08:51:34 +00:00
Norman Maurer
14ef469f31
Use maven plugin to prevent API/ABI breakage as part of build process (#8904)
Motivation:

Netty is very widely used which can lead to a lot of pain when we break API / ABI. We should make use japicmp-maven-plugin during the build to verify we do not introduce breakage by mistake.

Modifications:

- Add japicmp-maven-plugin to the build process
- Fix a method signature change in HttpProxyHandler that was flagged as a possible problem.

Result:

Ensure no API/ABI breakage accour between releases.
2019-03-01 19:42:29 +01:00
Norman Maurer
90ea3ec9f6
Adjust tests to be able to build / test when using IBM J9 / OpenJ9 (#8900)
Motivation:

We should run a CI job using J9 to ensure netty also works when using different JVMs.

Modifications:

- Adjust PooledByteBufAllocatorTest to be able to complete faster when using a JVM which takes longer when joining Threads (this seems to be the case with J9).
- Skip UDT tests on J9 as UDT is not supported there.

Result:

Be able to run CI against J9.
2019-03-01 06:47:56 +01:00
Norman Maurer
81e43d5088
DefaultFileRegion.transferTo with invalid count may cause busy-spin (#8885)
Motivation:

`DefaultFileRegion.transferTo` will return 0 all the time when we request more data then the actual file size. This may result in a busy spin while processing the fileregion during writes.

Modifications:

- If we wrote 0 bytes check if the underlying file size is smaller then the requested count and if so throw an IOException
- Add DefaultFileRegionTest
- Add a test to the testsuite

Result:

Fixes https://github.com/netty/netty/issues/8868.
2019-02-26 11:08:09 +01:00
Norman Maurer
fa6a8cb09c
Support using an Executor to offload blocking / long-running tasks wh… (#8847)
Motivation:

The SSLEngine does provide a way to signal to the caller that it may need to execute a blocking / long-running task which then can be offloaded to an Executor to ensure the I/O thread is not blocked. Currently how we handle this in SslHandler is not really optimal as while we offload to the Executor we still block the I/O Thread.

Modifications:

- Correctly support offloading the task to the Executor while suspending processing of SSL in the I/O Thread
- Add new methods to SslContext to specify the Executor when creating a SslHandler
- Remove @deprecated annotations from SslHandler constructor that takes an Executor
- Adjust tests to also run with the Executor to ensure all works as expected.

Result:

Be able to offload long running tasks to an Executor when using SslHandler. Partly fixes https://github.com/netty/netty/issues/7862 and https://github.com/netty/netty/issues/7020.
2019-02-11 09:47:44 +01:00
Norman Maurer
7bba4f49cf
Reduce GC produced by native DatagramChannel implementations when in connected mode. (#8806)
Motivation:

In the native code EpollDatagramChannel / KQueueDatagramChannel creates a DatagramSocketAddress object for each received UDP datagram even when in connected mode as it uses the recvfrom(...) / recvmsg(...)  method. Creating these is quite heavy in terms of allocations as internally, char[], String, Inet4Address, InetAddressHolder, InetSocketAddressHolder, InetAddress[], byte[] objects are getting generated when constructing the object. When in connected mode we can just use regular read(...) calls which do not need to allocate all of these.

Modifications:

- When in connected mode use read(...) and NOT recvfrom(..) / readmsg(...) to reduce allocations when possible.
- Adjust tests to ensure read works as expected when in connected mode.

Result:

Less allocations and GC when using native datagram channels in connected mode. Fixes https://github.com/netty/netty/issues/8770.
2019-02-01 10:29:36 +01:00
root
cf03ed0478 [maven-release-plugin] prepare for next development iteration 2019-01-21 12:26:44 +00:00
root
37484635cb [maven-release-plugin] prepare release netty-4.1.33.Final 2019-01-21 12:26:12 +00:00
Norman Maurer
51a650979f
Skip test on windows as the semantics we expect are only true on Linux / Unix / BSD / MacOS (#8629)
Motivation:

In the test we assume some semantics on how RST is done that are not true for Windows so we should skip it.

Modifications:

Skip test when on windows.

Result:

Be able to run testsuite on windows. Fixes https://github.com/netty/netty/issues/8571.
2018-12-06 20:43:40 +01:00
Feri73
5df235c083 Correcting Maven Dependencies (#8622)
Motivation:

Most of the maven modules do not explicitly declare their
dependencies and rely on transitivity, which is not always correct.

Modifications:

For all maven modules, add all of their dependencies to pom.xml

Result:

All of the (essentially non-transitive) depepdencies of the modules are explicitly declared in pom.xml
2018-12-06 09:01:14 +01:00
root
8eb313072e [maven-release-plugin] prepare for next development iteration 2018-11-29 11:15:09 +00:00
root
afcb4a37d3 [maven-release-plugin] prepare release netty-4.1.32.Final 2018-11-29 11:14:20 +00:00
root
3e7ddb36c7 [maven-release-plugin] prepare for next development iteration 2018-10-29 15:38:51 +00:00
root
9e50739601 [maven-release-plugin] prepare release netty-4.1.31.Final 2018-10-29 15:37:47 +00:00
Norman Maurer
0ddc62cec0
Add support for TLSv1.3 (#8293)
Motivation:

TLSv1.3 support is included in java11 and is also supported by OpenSSL 1.1.1, so we should support when possible.

Modifications:
- Add support for TLSv1.3 using either the JDK implementation or the native implementation provided by netty-tcnative when compiled against openssl 1.1.1
- Adjust unit tests for semantics provided by TLSv1.3
- Correctly handle custom Provider implementations that not support TLSv1.3

Result:

Be able to use TLSv1.3 with netty.
2018-10-17 08:35:35 +02:00
root
2d7cb47edd [maven-release-plugin] prepare for next development iteration 2018-09-27 19:00:45 +00:00
root
3a9ac829d5 [maven-release-plugin] prepare release netty-4.1.30.Final 2018-09-27 18:56:12 +00:00
root
a580dc7585 [maven-release-plugin] prepare for next development iteration 2018-08-24 06:36:33 +00:00
root
3fc789e83f [maven-release-plugin] prepare release netty-4.1.29.Final 2018-08-24 06:36:06 +00:00
Norman Maurer
fe14bad69c
Adjust SSL related tests to be more correct and so pass in the next EA release of java11. (#8162)
Motivation:

In some of our tests we not correctly init the SSLEngine before trying to perform a handshake which can cause an IllegalStateException. While this not happened in previous java releases it does now on Java11 (which is "ok" as its even mentioned in the api docs). Beside this how we selected the ciphersuite to test renegotation was not 100 % safe.

Modifications:

- Correctly init SSLEngine before using it
- Correctly select ciphersuite before testing for renegotation.

Result:

More correct tests and also pass on next java11 EA release.
2018-08-01 06:37:53 +02:00
root
fcb19cb589 [maven-release-plugin] prepare for next development iteration 2018-07-27 04:59:28 +00:00
root
ff785fbe39 [maven-release-plugin] prepare release netty-4.1.28.Final 2018-07-27 04:59:06 +00:00
root
b4dbdc2036 [maven-release-plugin] prepare for next development iteration 2018-07-11 15:37:40 +00:00
root
1c16519ac8 [maven-release-plugin] prepare release netty-4.1.27.Final 2018-07-11 15:37:21 +00:00
root
7bb9e7eafe [maven-release-plugin] prepare for next development iteration 2018-07-10 05:21:24 +00:00
root
8ca5421bd2 [maven-release-plugin] prepare release netty-4.1.26.Final 2018-07-10 05:18:13 +00:00
Norman Maurer
370de7c446
Revert "Ignore some test-flakiness when using Java11+ due outstanding Java11 bug. (#7984)" (#8035)
Motivation:

This reverts commit 4b728cd5bc as it was fixes in Java 11 ea+17.

Modification:

Revert previous added workaround as this is fixed in Java 11 now.

Result:

No more workaround for test included.
2018-06-19 08:11:10 +02:00
Scott Mitchell
12f6500a4f Epoll and Kqueue shouldn't read by default (#8024)
Motivation:
Epoll and Kqueue channels have internal state which forces
a single read operation after channel construction. This
violates the Channel#read() interface which indicates that
data shouldn't be delivered until this method is called.
The behavior is also inconsistent with the NIO transport.

Modifications:
- Epoll and Kqueue shouldn't unconditionally read upon
initialization, and instead should rely upon Channel#read()
or auto_read.

Result:
Epoll and Kqueue are more consistent with NIO.
2018-06-15 10:28:50 +02:00
Norman Maurer
4b728cd5bc
Ignore some test-flakiness when using Java11+ due outstanding Java11 bug. (#7984)
Motivation:

Java11 disallow draining any remaining bytes from the socket if a write causes a connection reset. This should be completely safe to do. At the moment if a write is causing a connection-reset you basically loose all the pending bytes that are sitting on the socket and are waiting to be read.

This happens because SocketOutputStream.write(…) may call AbstractPlainSocketImpl.setConnectionReset(…). Once this method is called any read(…) call will just throw a SocketException without even attempt to read any remaining data.

This is related:
 - https://bugs.openjdk.java.net/browse/JDK-8199329
 - http://hg.openjdk.java.net/jdk/jdk/rev/92cca24c8807
 - http://mail.openjdk.java.net/pipermail/net-dev/2018-May/011511.html

Modifications:

Tolarate if remaining bytes could not be read when using OIO.

Result:

Be able to build Netty and run testsuite while using Java11
2018-05-29 19:48:40 +02:00
Norman Maurer
8ae126aaa8
Revert workaround in test for Java 11 as it produces a connection-reset as expected now. (#7951)
Motivation:

We added a workaround for Java 11 as it not produced a connect-reset when SO_LINGER with 0 was set and NIO was used. This was fixed in the latest ea release of Java 11:

 - http://hg.openjdk.java.net/jdk/jdk/rev/ea54197f4fe4
 - https://bugs.openjdk.java.net/browse/JDK-8203059

Modifications:

Revert workaround.

Result:

Test that Java 11 behave the same way as earlier Java versions again.
2018-05-18 12:27:56 +02:00
Norman Maurer
64bb279f47 [maven-release-plugin] prepare for next development iteration 2018-05-14 11:11:45 +00:00
Norman Maurer
c67a3b0507 [maven-release-plugin] prepare release netty-4.1.25.Final 2018-05-14 11:11:24 +00:00
Norman Maurer
dc0cf3e099
Ensure Netty can be build with Java 11 (#7926)
Motivation:

Java 11 will be out soon, so we should be able to build (and run tests) netty.

Modifications:

- Add dependency that is needed till Java 11
- Adjust tests so these also pass on Java 11 (SocketChannelImpl.close() behavious a bit differently now).

Result:

Build also works (and tests pass) on Java 11.
2018-05-12 20:25:11 +02:00
Norman Maurer
cff87de44c
Make build pass on Java 10 (#7922)
Motivation:

Java 10 is out so we should be able to build netty with it (and run the tests).

Modifications:

- Update Mockito and JBoss Marshalling to support Java 10
- Fix unit test to not depend on specific cipher which is not present in Java 10 anymore

Result:

Netty builds (and runs all tests) when using Java 10
2018-05-10 08:36:17 +02:00
Devrim Şahin
b818852cdb Update DatagramPacket.recipient() to return the actual destination IP (#7879)
Motivation:

DatagramPacket.recipient() doesn't return the actual destination IP, but the IP the app is bound to.

Modification:

- IP_RECVORIGDSTADDR option is enabled for UDP sockets, which allows retrieval of ancillary information containing the original recipient.
- _recvFrom(...) function from transport-native-unix-common/src/main/c/netty_unix_socket.c is modified such that if IP_RECVORIGDSTADDR is set, recvmsg is used instead of recvfrom; enabling the retrieval of the original recipient.
- DatagramSocketAddress also contains a 'local' address, representing the recipient.
- EpollDatagramChannel is updated to return the retrieved recipient address instead of the address the channel is bound to.

Result:

Fixes #4950.
2018-04-26 08:00:36 +02:00
Norman Maurer
b75f44db9a [maven-release-plugin] prepare for next development iteration 2018-04-19 11:56:07 +00:00
Norman Maurer
04fac00c8c [maven-release-plugin] prepare release netty-4.1.24.Final 2018-04-19 11:55:47 +00:00
root
0a61f055f5 [maven-release-plugin] prepare for next development iteration 2018-04-04 10:44:46 +00:00