Commit Graph

307 Commits

Author SHA1 Message Date
Scott Mitchell
3cc4052963 New native transport for kqueue
Motivation:
We currently don't have a native transport which supports kqueue https://www.freebsd.org/cgi/man.cgi?query=kqueue&sektion=2. This can be useful for BSD systems such as MacOS to take advantage of native features, and provide feature parity with the Linux native transport.

Modifications:
- Make a new transport-native-unix-common module with all the java classes and JNI code for generic unix items. This module will build a static library for each unix platform, and included in the dynamic libraries used for JNI (e.g. transport-native-epoll, and eventually kqueue).
- Make a new transport-native-unix-common-tests module where the tests for the transport-native-unix-common module will live. This is so each unix platform can inherit from these test and ensure they pass.
- Add a new transport-native-kqueue module which uses JNI to directly interact with kqueue

Result:
JNI support for kqueue.
Fixes https://github.com/netty/netty/issues/2448
Fixes https://github.com/netty/netty/issues/4231
2017-05-03 09:53:22 -07:00
Norman Maurer
6915ec3bb9 [maven-release-plugin] prepare for next development iteration 2017-04-29 14:10:00 +02:00
Norman Maurer
f30f242fee [maven-release-plugin] prepare release netty-4.1.10.Final 2017-04-29 14:09:32 +02:00
Norman Maurer
2b8c8e0805 [maven-release-plugin] prepare for next development iteration 2017-03-10 07:46:17 +01:00
Norman Maurer
1db58ea980 [maven-release-plugin] prepare release netty-4.1.9.Final 2017-03-10 07:45:28 +01:00
Jason Brown
3861b7de2b Motivation:
Make the FileRegion comments about which transports are supported more accurate.
Also, eleminate any outstanding references to FileRegion.transfered as the method was renamed for spelling.

Modifications:

Class-level comment on FileRegion, can call renamed method.

Result:

More accurate documentation and less calls to deprecated methods.
2017-03-06 14:43:52 -08:00
Scott Mitchell
93f5f62a20 Cleanup EPOLL native exceptions
Motivation:
Exceptions generated from transport-native-epoll may include duplicate error string description or inconsistent usage of the method name in the string description.

Modifications:
- Ensure the method name from static exceptions and dynamic exceptions is of the same format
- Remove duplicate string rational from the exception messages

Result:
More consistent error messages with no duplicate error description.
2017-03-01 05:42:48 -08:00
Norman Maurer
c5324ea48c Make netty build work on Java9
Motivation:

We missed some stuff in 5728e0eb2c and so the build failed on java9

Modifications:

- Add extra cmdline args when needed
- skip the autobahntestsuite as jython not works with java9
- skip the osgi testsuite as the maven plugin not works with java9

Result:

Build finally passed on java9
2017-02-16 20:26:30 +01:00
Scott Mitchell
0befcd8431 EpollDatagramChannel avoid getOption
Motivation:
EpollDatagramChannel uses getOption in the isActive method. getOption is backed by a relatively large conditional if/else if block and this conditional checking can be avoided in the epoll transport.

Modifications:
- Add EpollDatagramChannelConfig#getActiveOnOpen and use this in EpollDatagramChannel

Result:
Conditional checking due to getOption is removed from EpollDatagramChannel.
2017-02-16 11:04:37 -08:00
Scott Mitchell
f3dd410142 EPOLLRDHUP prematurely marking input as shutdown
Motivation:
When the EPOLLRDHUP event is received we assume that the read side of the FD is no longer functional and force the input state to be shutdown. However if the channel is still active we should rely upon EPOLLIN and read to indicate there is no more data before we update the shutdown state. If we do not do this we may not read all pending data in the FD if the RecvByteBufAllocator doesn't want to consume it all in a single read operation.

Modifications:
- AbstractEpollChannel#epollRdHupReady() shouldn't force shutdown the input if the channel is active

Result:
All data can be read even if the RecvByteBufAllocator doesn't read it all in the current read loop.
Fixes https://github.com/netty/netty/issues/6303
2017-02-16 08:40:32 -08:00
Norman Maurer
5728e0eb2c Use the correct arguments when run with jdk9
Motivation:

We need to pass special arguments to run with jdk9 as otherwise some tests will not be able to run.

Modifications:

Allow to define extra arguments when running with jdk9

Result:

Tests pass with jdk9
2017-02-15 10:15:00 +01:00
Scott Mitchell
413d6eba53 EPOLL include error description and cause in exceptions
Motivation:
EPOLL annotates some exceptions to provide the remote address, but the original exception is not preserved. This may make determining a root cause more difficult. The static EPOLL exceptions references the native method that failed, but does not provide a description of the actual error number. Without the description users have to know intimate details about the native calls and how they may fail to debug issues.

Modifications:
- annotated exceptions should preserve the original exception
- static exceptions should include the string description of the expected errno

Result:
EPOLL exceptions provide more context and are more useful to end users.
2017-02-13 18:37:16 -08:00
Scott Mitchell
a1b5b5dcca EpollRecvByteAllocatorHandle doesn't inform delegate of more data
Motivation:
EpollRecvByteAllocatorHandle intends to override the meaning of "maybe more data to read" which is a concept also used in all existing implementations of RecvByteBufAllocator$Handle but the interface doesn't support overriding. Because the interfaces lack the ability to propagate this computation EpollRecvByteAllocatorHandle attempts to implement a heuristic on top of the delegate which may lead to reading when we shouldn't or not reading data.

Modifications:
- Create a new interface ExtendedRecvByteBufAllocator and ExtendedHandle which allows the "maybe more data to read" between interfaces
- Deprecate RecvByteBufAllocator and change all existing implementations to extend ExtendedRecvByteBufAllocator
- transport-native-epoll should require ExtendedRecvByteBufAllocator so the "maybe more data to read" can be propagated to the ExtendedHandle

Result:
Fixes https://github.com/netty/netty/issues/6303.
2017-02-13 17:42:24 -08:00
Dmitriy Dumanskiy
b9abd3c9fc Cleanup : for loops for arrays to make code easier to read and removed unnecessary toLowerCase() 2017-02-06 07:47:59 +01:00
Norman Maurer
735d6dd636 [maven-release-plugin] prepare for next development iteration 2017-01-30 15:14:02 +01:00
Norman Maurer
76e22e63f3 [maven-release-plugin] prepare release netty-4.1.8.Final 2017-01-30 15:12:36 +01:00
Norman Maurer
81735c2535 Check if Epoll is avaible when construct EpollEventLoopGroup
Motivation:

We should call Epoll.ensureAvailability() when init EpollEventLoopGroup to fail fast and with a proper exception.

Modifications:

Call Epoll.ensureAvailability() during EpollEventLoopGroup init.

Result:

Fail fast if epoll is not availability (for whatever reason).
2017-01-27 08:19:45 +01:00
Norman Maurer
7f01da8d0f [maven-release-plugin] prepare for next development iteration 2017-01-12 11:36:51 +01:00
Norman Maurer
7a21eb1178 [maven-release-plugin] prepare release netty-4.1.7.Final 2017-01-12 11:35:58 +01:00
Scott Mitchell
631077c793 EPOLL RDHUP processing
Motivation:
EpollRecvByteAllocatorHandle will read unconditionally if EPOLLRDHUP has been received. However we can just treat this the same was we do as data maybe pending in ET mode, and let LT mode notify us if we haven't read all data.

Modifications:
- EpollRecvByteAllocatorHandle should not always force a read just because EPOLLRDHUP has been received, but just treated as an indicator that there maybe more data to read in ET mode

Result:
Fixes https://github.com/netty/netty/issues/6173.
2017-01-10 13:11:18 -08:00
Norman Maurer
89e93968ac Remove usage of own Atomic*FieldUpdater in favor of JDKs
Motivation:

In later Java8 versions our Atomic*FieldUpdater are slower then the JDK implementations so we should not use ours anymore. Even worse the JDK implementations provide for example an optimized version of addAndGet(...) using intrinsics which makes it a lot faster for this use-case.

Modifications:

- Remove methods that return our own Atomic*FieldUpdaters.
- Use the JDK implementations everywhere.

Result:

Faster code.
2016-12-15 08:09:06 +00:00
Norman Maurer
5fec897ef7 Fail build when unused c code is detected
Motivation:

To keep our code clean we should fail the build when unused c code is detected.

Modifications:

- Add '-Wunused-variable' to build flags

Result:

Cleaner code.
2016-12-15 08:04:25 +00:00
T Jake Luciani
8d664fa0fd Include frame-pointer to improve epoll flamegraph tracing
Motivation:

When attempting to flamegraph netty w/ epoll it was noticed the stacks are lost going from
java to epoll lib.

Modifications:

added the -fno-omit-framepointer flag to compiler flags to ensure the fp is kept intact

Result:

Flamegraphs will now show native code in the same stack as java code using perf-java-flames
2016-12-07 08:06:26 +01:00
Norman Maurer
2de644224c Correctly mark EpollServerDomainSocketChannel.isActive() as true after bind is complete.]
Motivation:

We missed to set active = true in EpollServerDomainSocketChannel.doBind(...) which also means that channelActive(...) was never triggered.

Modifications:

Correct set active = true in doBind(...)

Result:

EpollServerDomainSocketChannel is correctly set to active when bound.
2016-12-01 21:26:11 +01:00
Demetrius
b3fa976028 Added support to fetch the UID, GID, and PID of the connected unix domain socket (EG: SO_PEERCREDS)
Motivation:

I had a need to know the user credentials of a connected unix domain socket.

Modifications:

Added a class to encapsulate user credentials (UID, GID, and the PID).
Augemented the Socket class to provide the JNI native interface to return this new class
Augemented the c code to call getSockOpts passing <a href=http://man7.org/linux/man-pages/man7/socket.7.html>SO_PEERCRED</a>
Then surfaced the ability to get user credentials in the EpollDomainSocketChannel

Result:

The EpollDomainSocketChannel now has a the following function signature:
public PeerCredentials peerCredentials() throws IOException allowing a caller to get the UID, GID, and PID of the linux process
connected to the unix domain socket.
2016-11-04 07:27:30 +01:00
Christoph Ingwersen
b6099bc193 Fix unbalanced quotes in Bundle-NativeCode manifest header
Motivation:

The previously generated manifest causes a parse exception when loaded into an Apache Felix OSGI container.

Modifications:

Fix parameter delimiter and unbalanced quotes in manifest entry. Suffixed with asterisk so the bundle is resolved on other architectures as well even if native libs won't be loaded.

Result:

Bundle will load properly in OSGI containers.
2016-11-01 10:24:42 +01:00
Norman Maurer
5f533b7358 [maven-release-plugin] prepare for next development iteration 2016-10-14 13:20:41 +02:00
Norman Maurer
35fb0babe2 [maven-release-plugin] prepare release netty-4.1.6.Final 2016-10-14 12:47:19 +02:00
Norman Maurer
a09e56850e [#5882] Ensure we even process tasks if processing of ready channels throws an Exception in event loop.
Motivation:

If an exception is thrown while processing the ready channels in the EventLoop we should still run all tasks as this may allow to recover. For example a OutOfMemoryError may be thrown and runAllTasks() will free up memory again. Beside this we should also ensure we always allow to shutdown even if an exception was thrown.

Modifications:

- Call runAllTasks() in a finally block
- Ensure shutdown is always handles.

Result:

More robust EventLoop implementations for NIO and Epoll.
2016-10-10 07:49:57 +02:00
Norman Maurer
2c4a7a2539 [#5800] Support any FileRegion implementation when using epoll transport
Motivation:

At the moment only DefaultFileRegion is supported when using the native epoll transport.

Modification:

- Add support for any FileRegion implementation
- Add test case

Result:

Also custom FileRegion implementation are supported when using the epoll transport.
2016-09-15 23:03:37 -07:00
Norman Maurer
54b1a100f4 [maven-release-plugin] prepare for next development iteration 2016-08-26 10:06:32 +02:00
Norman Maurer
eb450d8b2f Correct throw ClosedChannelException when attempt to call shutdown*(...) on closed EpollSocketChannel.
Motivition:

NIO throws ClosedChannelException when a user tries to call shutdown*() on a closed Channel. We should do the same for the EPOLL transport

Modification:

Throw ClosedChannelException when a user tries to call shutdown*() on a closed channel.

Result:

Consistent behavior.
2016-09-01 08:16:02 +02:00
Norman Maurer
1208b90f57 [maven-release-plugin] prepare release netty-4.1.5.Final 2016-08-26 04:59:35 +02:00
Norman Maurer
d0dc02d826 Correctly guard against NotYetConnectedExceptions when handling RDHUP.
Motivation:

Commit 2c1f17faa2 introduced a regression which could cause NotYetConnectedExceptions when handling RDHUP events.

Modifications:

Correct ignore NotYetConnectedException when handling RDHUP events.

Result:

No more regression.
2016-08-29 12:43:11 +02:00
Norman Maurer
d3cb95ef00 Make NIO and EPOLL transport connect errors more consistent with the JDK
Motivation:

The NIO transport used an IllegalStateException if a user tried to issue another connect(...) while the connect was still in process. For this case the JDK specified a ConnectPendingException which we should use. The same issues exists in the EPOLL transport. Beside this the EPOLL transport also does not throw the right exceptions for ENETUNREACH and EISCONN errno codes.

Modifications:

- Replace IllegalStateException with ConnectPendingException in NIO and EPOLL transport
- throw correct exceptions for ENETUNREACH and EISCONN in EPOLL transport
- Add test case

Result:

More correct error handling for connect attempts when using NIO and EPOLL transport
2016-08-27 20:57:36 +02:00
Norman Maurer
5e148d5670 [#5639] Ensure fireChannelActive() is also called if Channel is closed in connect promise.
Motivation:

We need to ensure we also call fireChannelActive() if the Channel is directly closed in a ChannelFutureListener that is belongs to the promise for the connect. Otherwise we will see missing active events.

Modifications:

Ensure we always call fireChannelActive() if the Channel was active.

Result:

No missing events.
2016-08-24 08:47:49 +02:00
Norman Maurer
2c1f17faa2 Throw correct NotYetConnectedException when ENOTCONN errno is set
Motivation:

We should throw a NotYetConnectedException when ENOTCONN errno is set. This is also consistent with NIO.

Modification:

Throw correct exception and add test case

Result:

More correct and consistent behavior.
2016-08-24 07:41:24 +02:00
Dmitriy Dumanskiy
3bc97962f8 removed unnecessary static holders 2016-08-03 06:17:31 -07:00
Dmitriy Dumanskiy
a80ea46b8e Removed custom split method as it is not effective anymore. 2016-08-01 21:49:33 +02:00
Norman Maurer
cb7cf4491c [maven-release-plugin] prepare for next development iteration 2016-07-27 13:29:56 +02:00
Norman Maurer
9466b32d05 [maven-release-plugin] prepare release netty-4.1.4.Final 2016-07-27 13:16:59 +02:00
Scott Mitchell
3d7ae97359 Make Epoll ChannelMetadata more consistent with NIO
Motivation:
In 4.0 AbstractNioByteChannel has a default of 16 max messages per read. However in 4.1 that constraint was applied at the NioSocketChannel which is not equivalent. In 4.1 AbstractEpollStreamChannel also did not have the default of 16 max messages per read applied.

Modifications:
- Make Nio consistent with 4.0
- Make Epoll consistent with Nio

Result:
Nio and Epoll both have consistent ChannelMetadata and are consistent with 4.0.
2016-07-18 13:26:05 +02:00
Norman Maurer
047f6aed28 [maven-release-plugin] prepare for next development iteration 2016-07-15 09:09:13 +02:00
Norman Maurer
b2adea87a0 [maven-release-plugin] prepare release netty-4.1.3.Final 2016-07-15 09:08:53 +02:00
Norman Maurer
25c37cf44a Include Bundle-NativeCode manifest entry in transport-native-epoll jar.
Motivation:

The build generates a OSGi bundle with missing Bundle-NativeCode manifest entry.

Modifications:

Add missing manifest entry.

Result:

Be able to use transport-native-epoll in osgi container.
2016-07-14 09:11:09 +02:00
Scott Mitchell
df033e150c EPOLL Cached ECONNREFUSED Exception
Motivation:
ECONNREFUSED can be a common type of exception when attempting to finish the connection process. Generating a new exception each time can be costly and quickly bloat memory usage.

Modifications:
- Expose ECONNREFUSED from JNI and cache this exception in Socket.finishConnect

Result:
ECONNREFUSED during finish connect doesn't create a new exception each time.
2016-07-06 12:02:42 -07:00
Norman Maurer
4676a2271c [maven-release-plugin] prepare for next development iteration 2016-07-01 10:33:32 +02:00
Norman Maurer
ad270c02b9 [maven-release-plugin] prepare release netty-4.1.2.Final 2016-07-01 09:07:40 +02:00
Norman Maurer
731f52fdf7 Allow to inject RejectedExecutionHandler for different EventLoops and EventExecutors
Motiviation:

Sometimes it is useful to allow to specify a custom strategy to handle rejected tasks. For example if someone tries to add tasks from outside the eventloop it may make sense to try to backoff and retries and so give the executor time to recover.

Modification:

Add RejectedEventExecutor interface and implementations and allow to inject it.

Result:

More flexible handling of executor overload.
2016-06-24 17:08:30 +02:00
Norman Maurer
a0d4fb16fc Allow to set max capacity for task queue for EventExecutors and EventLoops
Motivation:

To restrict the memory usage of a system it is sometimes needed to adjust the number of max pending tasks in the tasks queue.

Modifications:

- Add new constructors to modify the number of allowed pending tasks.
- Add system properties to configure the default values.

Result:

More flexible configuration.
2016-06-24 14:00:50 +02:00