Commit Graph

198 Commits

Author SHA1 Message Date
Scott Mitchell
f60698a538 EpollEventLoopGroup support Executor
Motivation:
NioEventLoopGroup supports constructors which take an executor but EpollEventLoopGroup does not. EPOLL should be consistent with NIO where ever possible.

Modifications:
- Add constructors to EpollEventLoopGroup which accept an Executor as a parameter

Result:
EpollEventLoopGroup is more consistent with NioEventLoopGroup
Fixes https://github.com/netty/netty/issues/5161
2016-04-21 08:33:49 -07:00
nmittler
de2515ddd8 Support preloading of native epoll lib
Motivation:

Some applications may use alternative methods of loading the epoll JNI symbols. We should support this use case.

Modifications:

Attempt to use a side effect free JNI method. If that fails, load the library.

Result:

Fixes #5122
2016-04-12 11:09:06 -07:00
Norman Maurer
eb14e5b2f5 Ensure EpollSocketChannel.localAddress() returns correct address after connect(...) call.
Motivation:

We missed to correctly retrieve the localAddress() after we called Socket.connect(..) and so the user would always see an incorrect address when calling EpollSocketChannel.localAddress().

Modifications:

- Ensure we always retrieve the localAddress() after we called Socket.connect(...) as only after this we will be able to receive the correct address.
- Add unit test

Result:

Correct and consistent behaviour across different transports (NIO/OIO/EPOLL).
2016-04-09 19:04:45 +02:00
Scott Mitchell
9a63aafe22 EPOLL ET Set ReadFlag and Limit epollInReadyRunnable
Motivation:
441aa4c575 conditionally set the readFlag based upon if maybeMoreDataToRead is set. It is possible that the read flag will not be set, and nothing will be read by executeEpollInReadyRunnable and no actual data will be read even though the user requested it.

Modifications:
- Always set the readFlag in doBeginRead
- Make it so only a single epollInReadyRunnable can execute for a channel at a time

Result:
Less chance of missing read events in EPOLL transport.
2016-04-08 11:03:52 +02:00
Scott Mitchell
5b48fc284e Make OIO/NIO/EPOLL autoReadClear consistent
Motivation:
OIO/NIO use a volatile variable to track if a read is pending. EPOLL does not use a volatile an executes a Runnable on the event loop thread to set readPending to false. These mechansims should be consistent, and not using a volatile variable is preferable because the variable is written to frequently in the event loop thread.
OIO also does not set readPending to false before each fireChannelRead operation and may result in reading more data than the user desires.

Modifications:
- OIO/NIO should not use a volatile variable for readPending
- OIO should set readPending to false before each fireChannelRead

Result:
OIO/NIO/EPOLL are more consistent w.r.t. readPending and volatile variable operations are reduced
Fixes https://github.com/netty/netty/issues/5069
2016-04-06 12:32:14 -07:00
Scott Mitchell
9fb86a380d NIO/EPOLL readPending set to false incorrectly
Motivation:
441aa4c575 introduced a bug in transport-native-epoll where readPending is set to false before a read is attempted, but this should happen before fireChannelRead is called. The NIO transport also only sets the readPending variable to false on the first read in the event loop. This means that if the user only calls read() on the first channelRead(..) the select loop will still listen for read events even if the user does not call read() on subsequent channelRead() or channelReadComplete() in the same event loop run. If the user only needs 2 channelRead() calls then by default they will may get 14 more channelRead() calls in the current event loop, and then 16 more when the event loop is woken up for a read event. This will also read data off the TCP stack and allow the peer to queue more data in the local RECV buffers.

Modifications:
- readPending should be set to false before each call to channelRead()
- make NIO readPending set to false consistent with EPOLL

Result:
NIO and EPOLL transport set readPending to false at correct times which don't read more data than intended by the user.
Fixes https://github.com/netty/netty/issues/5082
2016-04-06 00:09:49 -07:00
Norman Maurer
7d60699a49 Fix compile error caused by 7fb475a223, because I missed to adjust jni code. 2016-04-06 08:39:27 +02:00
Norman Maurer
f46cfbc590 [#5059] Deprecate method with typo and introduce a new one without typo
Motivation:

There is a spelling error in FileRegion.transfered() as it should be transferred().

Modifications:

Deprecate old method and add a new one.

Result:

Fix typo and can remove the old method later.
2016-04-05 15:06:46 +02:00
Scott Mitchell
441aa4c575 EPOLL ET Missed Reads
Motivation:
bfbef036a8 made EPOLL respect autoRead while in ET mode. However it is possible that we may miss data pending on the RECV queue if autoRead is off. This is because maybeMoreDataToRead is updated after fireChannelRead and if a user calls read() from here maybeMoreDataToRead will be false because it is updated after the fireChannelRead call. The way maybeMoreDataToRead was updated also causes a single channel to continuously read on the event loop and not relinquish and give other channels to try reading.

Modifications:
- Ensure maybeMoreDataToRead is always set after all user events, and is evaluated with readPending to execute a epollInReady on the EventLoop
- Combine the checkResetEpollIn and maybeMoreDataToRead logic to invoke a epollInReady later into the epollInFinally method due to similar responsibilities
- Update unit tests to reflect the user calling read() on the event loop from channelRead()

Result:
EPOLL ET with autoRead set to false will not leave data on the RECV queue.
2016-04-01 13:06:19 -07:00
jiafu1115
3e5dcb5f3e [#3806] Setting WRITE_BUFFER_LOW_WATER_MARK before WRITE_BUFFER_HIGH_WATER_MARK results in an internal Exception
Motivation:

Setting the WRITE_BUFFER_LOW_WATER_MARK before WRITE_BUFFER_HIGH_WATER_MARK results in an internal Exception (appears only in the logs) if the value is larger than the default high water mark value. The WRITE_BUFFER_HIGH_WATER_MARK call appears to have no effect in this context.

Setting the values in the reverse order works.

Modifications:

- deprecated ChannelOption.WRITE_BUFFER_HIGH_WATER_MARK and
ChannelOption.WRITE_BUFFER_LOW_WATER_MARK.
- add one new option called ChannelOption.WRITE_BUFFER_WATER_MARK.

Result:
The high/low water mark values limits caused by default values are removed.

Setting the WRITE_BUFFER_LOW_WATER_MARK before WRITE_BUFFER_HIGH_WATER_MARK results in an internal Exception (appears only in the logs) if the value is larger than the default high water mark value. The WRITE_BUFFER_HIGH_WATER_MARK call appears to have no effect in this context.

Setting the values in the reverse order works.
2016-03-31 13:44:44 +02:00
Scott Mitchell
0c839d9e0a EPOLL SelectStrategy
Motivation:
NIO now supports a pluggable select strategy, but EPOLL currently doesn't support this. We should strive for feature parity for EPOLL.

Modifications:
- Add SelectStrategy to EPOLL transport.

Result:
EPOLL transport supports SelectStategy.
2016-03-30 15:11:35 -07:00
Norman Maurer
4950a523a7 Not attempt to read from fd when channel is closed during read loop. Related to [#5031]
Motivation:

We need to break out of the read loop for two reasons:

- If the input was shutdown in between (which may be the case when the user did it in the
  fireChannelRead(...) method we should not try to read again to not produce any
  miss-leading exceptions.

- If the user closes the channel we need to ensure we not try to read from it again as
  the filedescriptor may be re-used already by the OS if the system is handling a lot of
  concurrent connections and so needs a lot of filedescriptors. If not do this we risk
  reading data from a filedescriptor that belongs to another socket then the socket that
  was "wrapped" by this Channel implementation.

Modification:

Break the reading loop if the input was shutdown from within the channelRead(...) method.

Result:

No more meaningless exceptions and no risk to read data from wrong socket after the original was closed.
2016-03-29 10:50:38 +02:00
Scott Mitchell
5eab79a464 EPOLL Socket Shutdown Fix
Motivation:
8dbf5d02e5 modified the shutdown code for Socket but did not correctly calculate the change in shutdown state and only applying this change. This is significant because if sockets are being opening and closed quickly and the underlying FD happens to be reused we need to take care that we don't unintentionally change the state of the new FD by acting on an object which represents the old incarnation of that FD.

Modifications:
- Calculate the shutdown change, and only apply what has changed, or exit if no change.

Result:
Socket.shutdown can not inadvertently affect the state of another logical FD.
2016-03-25 12:01:03 -07:00
Scott Mitchell
99c85ef4f5 cf171ff525 Close Regression
Motivation:
cf171ff525 introduced a change in behavior when dealing with closing channel in the read loop. This changed behavior may use stale state to determine if a channel should be shutdown and may be incorrect.

Modifications:
- Revert the usage of potentially stale state

Result:
Closing a channel in the read loop is based upon current state instead of potentially stale state.
2016-03-24 14:52:04 -07:00
Norman Maurer
15f3b69b9e [#5033] Fix typo in exception message introduced by acbca192bd
Motivation:

I introduced a typo as part of acbca192bd.

Modifications:

Fix typo

Result:

Correct message in exception.
2016-03-24 16:25:32 +01:00
Norman Maurer
2c390ae66b [#5029] Fix type of EpollChannelOption.TCP_QUICKACK
Motivation:

TCP_QUICKACK uses Integer but needs to be Boolean

Modifications:

Fix type

Result:

Be able to use EpollChannelOption.TCP_QUICKACK
2016-03-18 01:08:13 +01:00
Norman Maurer
a11412fab0 Cleanup transport-native-epoll code.
Motivation:

The code of transport-native-epoll missed some things in terms of static keywords, @deprecated annotations and other minor things.

Modifications:

- Add missing @deprecated annotation
- Not using FQCN in javadocs
- Add static keyword where possible
- Use final fields when possible
- Remove throws IOException from method where it is not needed.

Result:

Cleaner code.
2016-03-23 10:59:42 +01:00
Norman Maurer
0320ccb59f Let getSoError() throw IOException as well
Motivation:

In commit acbca192bd we changed to have our native operations which either gall getsockopt or setsockopt throw IOExceptions (to be more specific we throw a ClosedChannelException in some cases). Unfortunally I missed to also do the same for getSoError() and missed to add throws IOException to the native methods.

Modifications:

- Correctly throw IOException from getSoError()
- Add throws IOException to native methods where it was missed.

Result:

Correct declaration of getSoError() and other native methods.
2016-03-17 20:09:15 +01:00
Scott Mitchell
8dbf5d02e5 EPOLL SO_LINGER=0 sends FIN+RST
Motivation:
If SO_LINGER is set to 0 the EPOLL transport will send a FIN followed by a RST. This is not consistent with the behavior of the NIO transport. This variation in behavior can cause protocol violations in streaming protocols (e.g. HTTP) where a FIN may be interpreted as a valid end to a data stream, but RST may be treated as the data is corrupted and should be discarded.

https://github.com/netty/netty/issues/4170 Claims the behavior of NIO always issues a shutdown when close occurs. I could not find any evidence of this in Netty's NIO transport nor in the JDK's SocketChannel.close() implementation.

Modifications:
- AbstractEpollChannel should be consistent with the NIO transport and not force a shutdown on every close
- FileDescriptor to keep state in a consistent manner with the JDK and not allow a shutdown after a close
- Unit tests for NIO and EPOLL to ensure consistent behavior

Result:
EPOLL is capable of sending just a RST to terminate a connection.
2016-03-16 22:35:04 -07:00
Norman Maurer
acbca192bd ChannelConfig operations should wrap ClosedChannelException if Channel was closed before.
Motivation:

To be consistent with the JDK we should ensure our native methods throw a ClosedChannelException if the Channel was previously closed. This will then be wrapped in a ChannelException as usual. For all other errors we continue to just throw a ChannelException directly.

Modifications:

Ensure getsockopt and setsockopt will throw a ClosedChannelException if the channel was closed before, on other errors we throw a ChannelException as before diretly.

Result:

Consistent with the NIO Channel implementations.
2016-03-15 14:47:39 +01:00
Norman Maurer
e3bf679998 Ensure connect promise is notifed before fireChannelActive() is called. Related to [#4927]
Motivation:

We should always first notify the promise before trigger an event through the pipeline to be consistent.

Modifications:

Ensure we notify the promise before fire event.

Result:

Consistent behavior
2016-03-14 14:18:11 +01:00
Norman Maurer
52bfaae1a0 Fix EpollServerSocketConfig.isFreebind()
Motivation:

EpollServerSocketConfig.isFreebind() throws an exception when called.

Modifications:

Use the correct getsockopt arguments.

Result:

No more exception when call EpollServerSocketConfig.isFreebind()
2016-03-14 12:12:41 +01:00
Norman Maurer
c65165c491 Remove TCP_MD5 from EpollServerChannelConfig.
Motivation:

TCP_MD5 is only supported by SocketChannels so remove it from EpollServerChannelConfig which is generic.

Modifications:

Remove invalid code.

Result:

Remove invalid / dead code.
2016-03-14 12:11:56 +01:00
Scott Mitchell
bfbef036a8 EPOLL ET AutoRead
Motivation:
EPOLL does not support autoread when in ET mode.

Modifications:
- EpollRecvByteAllocatorHandle should not unconditionally force reading just because ET is enabled
- AbstractEpollChannel and all derived classes which implement epollInReady must support a variable which indicates
there may be more data to read. The variable will be used when read is called to simulate a EPOLL wakeup and call epollInReady if necessary. This will ensure that if we don't read until EAGAIN that we will try to read again and not rely on EPOLL to notify us.

Result:
EPOLL ET supports auto read.
2016-03-11 07:42:30 -08:00
Norman Maurer
d09547deb8 Add support for TCP_DEFER_ACCEPT and TCP_QUICKACK
Motivation:

When using the native transport have support for TCP_DEFER_ACCEPT or / and TCP_QUICKACK can be useful.

Modifications:

- Add support for TCP_DEFER_ACCEPT and TCP_QUICKACK
- Ad unit tests

Result:

TCP_DEFER_ACCEPT and TCP_QUICKACK are supported now.
2016-03-08 13:46:06 +01:00
Norman Maurer
d8658989e1 Use smaller connect timeout to speed up tests.
Motivation:

For on tests we expected a ConnectTimeoutException but used the default timeout of 10 seconds. This slows down testing.

Modifications:

Use connect timeout of 1 second in unit test.

Result:

Faster execution of unit test.
2016-03-06 17:47:38 +01:00
Roman Timushev
23f7fc67a4 Enable shutdownOutput for EpollDomainSocketChannel 2016-02-18 18:05:51 -08:00
Norman Maurer
cd56f87ca1 Remove invalid return
Motivation:

JNI_OnUnload(...) does not return anything (has void in its signature) so we should not try to return something.

Modifications:

Remove return.

Result:

Fix incorrect but harmless code.
2016-02-10 16:48:39 -08:00
Scott Mitchell
b9682a26b1 EPOLL dladdr unexpected return value
Motivation:
netty_epoll_native.c uses dladdr in attempt to get the name of the library that the code is running in. However the address passed to this funciton (JNI_OnLoad) may not be unique in the context of the application which loaded it. For example if another JNI library is loaded this address may first resolve to the other JNI library and cause the path name parsing to fail, which will cause the library to fail.

Modifications:
- Pass an addresses which is local to the current library to dladdr

Result:
EPOLL JNI library can be loaded in an environment where multiple JNI libraries are loaded.
Fixes https://github.com/netty/netty/issues/4840
2016-02-06 19:43:57 +01:00
Norman Maurer
f10d66b45e Epoll.isAvailable() must return false if sun.misc.Unsafe is not present.
Motivation:

Currently our epoll native transport requires sun.misc.Unsafe and so we need to take this into account for Epoll.isAvailable().

Modifications:

Take into account if sun.misc.Unsafe is present.

Result:

Only return true for Epoll.isAvailable() if sun.misc.Unsafe is present.
2016-02-06 09:59:12 +01:00
Scott Mitchell
075a54af3e Native EPOLL Library Allows Shading
Motivation:
If Netty's class files are renamed and the type references are updated (shaded) the native libraries will not function. The native epoll module uses implicit JNI bindings which requires the fully qualified java type names to match the method signatures of the native methods. This means EPOLL cannot be used with a shaded Netty.

Modifications:
- Make the JNI method registration dynamic
- support a system property io.netty.packagePrefix which must be prepended to the name of the native library (to ensure the correct library is loaded) and all class names (to allow classes to be correctly referenced)
- remove system property io.netty.native.epoll.nettyPackagePrefix which was recently added and the code to support it was incomplete

Result:
transport-native-epoll can be used when Netty has been shaded.
Fixes https://github.com/netty/netty/issues/4800
2016-02-03 14:40:28 -08:00
Norman Maurer
3616d9e814 Correctly handle wildcard address when bind to socket and using native transport
Motivation:

When a wildcard address is used to bind a socket and ipv4 and ipv6 are usable we should accept both (just like JDK IO/NIO does).

Modifications:

Detect wildcard address and if so use in6addr_any

Result:

Correctly accept ipv4 and ipv6
2016-01-28 14:07:57 +01:00
Scott Mitchell
dea337b4cf Native Transport Netty Class Package Prefix
Motivation:
transport-native-epoll finds java classes from JNI using fully qualified class names. If a shaded version of Netty is used then these lookups will fail.

Modifications:
- Allow a prefix to be appended to Netty class names in JNI code.

Result:
JNI code can be used with shaded version of Netty.
2016-01-15 12:55:58 -08:00
Norman Maurer
bf24ffd335 [#4694] Ensure native transport can also be compiled on 32bit systems.
Motivation:

We should also be able to compile the native transport on 32bit systems.

Modifications:

Add cast to intptr_t for pointers

Result:

It's possible now to also compile on 32bit.
2016-01-15 10:54:24 +01:00
William Kemper
c205e2be76 Set DSCP bits for IPv6 when setting traffic class.
Motivation:

Linux uses different socket options to set the traffic class (DSCP) on IPv6

Modifications:

Also set IPV6_TCLASS for IPv6 sockets

Result:

TrafficClass will work on IPv4 and IPv6 correctly
2016-01-05 20:55:33 +01:00
Norman Maurer
80f45d6ae5 Ensure closing a Socket / FileDescriptor multiple times will not throw exception
Motivation:

If an user will close a Socket / FileDescriptor multiple times we should handle the extra close operations as NOOP.

Modifications:

Only do the actual closing one time

Result:

No exception if close is called multiple times.
2015-12-23 23:04:30 +01:00
Norman Maurer
e3d5ca82c0 [#4604] EpollSocketChannelConfig.isKeepAlive(...) throws UnsatisfieldLinkError
Motivation:

We missed to define the actual c function for isKeepAlive(...) and so throw UnsatisfieldLinkError.

Modifications:

- Add function
- Add unit test for Socket class

Result:

Correctly work isKeepAlive(...) when using native transport
2015-12-22 23:54:33 +01:00
Norman Maurer
89ff831a67 [#4449] Remove registered events from eventloop before close
Motivation:

We need to remove all registered events for a Channel from the EventLoop before doing the actual close to ensure we not produce a cpu spin when the actual close operation is delayed or executed outside of the EventLoop.

Modifications:

Deregister for events for NIO and EPOLL socket implementations when SO_LINGER is used.

Result:

No more cpu spin.
2015-12-13 09:55:50 +01:00
Norman Maurer
1f6b957377 Ensure we retain the original hostname when connect to a remote peer when using epoll transport.
Motivation:

We should retain the original hostname when connect to a remote peer so the user can still query the origin hostname if getHostString() is used.

Modifications:

Compute a InetSocketAddress from the original remote address and the one returned by the Os.

Result:

Same behavior when using epoll transport and nio transport.
2015-12-11 07:08:15 +01:00
Norman Maurer
0ec34b5f76 Fix race-condition when closing a NioSocketChannel or EpollSocketChannel
Motivation:

Fix a race-condition when closing NioSocketChannel or EpollSocketChannel while try to detect if a close executor should be used and the underlying socket was already closed. This could lead to an exception that then leave the channel / in an invalid state and so could lead to side-effects like heavy CPU usage.

Modifications:

Catch possible socket exception while try to get the SO_LINGER options from the underlying socket.

Result:

No more race-condition when closing the channel is possible with bad side-effects.
2015-11-26 22:56:00 +01:00
Scott Mitchell
6cb6282699 Lazy Initialization of epoll splice queue
Motivation:
AbstractEpollStreamChannel has a queue which collects splice events. Splice is assumed not to be the most common use case of this class and thus the splice queue could be initialized in a lazy fashion to save memory. This becomes more significant when the number of connections grows.

Modifications:
- AbstractEpollStreamChannel.spliceQueue will be initialized in a lazy fashion

Result:
Less memory consumption for most use cases
2015-11-20 15:09:53 -08:00
Norman Maurer
7bee318fc7 Use OneTimeTask where possible to reduce object creation
Motivation:

We should use OneTimeTask where possible to reduce object creation.

Modifications:

Replace Runnable with OneTimeTask

Result:

Less object creation
2015-11-20 14:39:06 -08:00
Norman Maurer
edb2250d35 Store reference to IovArray in the EpollEventLoop to reduce thread local access.
Motivation:

If we have a lot of writes going on we currently need to lookup the IovArray for each Channel that does writes. This can have quite some perf overhead. We should not need to do this and just store a reference of the IovArray on the EpollEventLoop itself.

Modifications:

- Remove IoArrayThreadLocal
- Store the IoArray in the EventLoop itself

Result:

Less FastThreadLocal lookups
2015-11-20 06:07:54 -08:00
Scott Mitchell
b640de2d94 Epoll Shutdown Input Exception Handling
Motivation:
If ChannelOption.ALLOW_HALF_CLOSURE is true and the shutdown input operation fails we should not propagate this exception, and instead consider this socket's read as half closed.

Modifications:
- AbstractEpollChannel.shutdownInput should not propagate exceptions when attempting to shutdown the input, but instead should just close the socket

Result:
Users expecting a ChannelInputShutdownEvent will get this event even if the socket is already shutdown, and the shutdown operation fails.
2015-11-19 16:03:10 -08:00
Scott Mitchell
c7cb104dc4 EPOLL Shutdown and Half Closed
Motivation:
The EPOLL module was not completly respecting the half closed state. It may have missed events, or procssed events when it should not have due to checking isOpen instead of the appropriate shutdown state.

Modifications:
- use FileDescriptor's isShutdown* methods instead of isOpen to check for processing events.

Result:
Half closed code in EPOLL module is more correct.
2015-11-02 13:01:24 -08:00
Scott Mitchell
dbbdbe11a6 Decouple Unix from Linux in Native Transport
Motivation:
transport-native-epoll is designed to be specific to Linux. However there is native code that can be extracted out and made to work on more Unix like distributions. There are a few steps to be completely decoupled but the first step is to extract out code that can run in a more general Unix environment from the Linux specific code base.

Modifications:
- Move all non-Linux specific stuff from Native.java into the io.netty.channel.unix package.
- io.netty.channel.unix.FileDescriptor will inherit all the native methods that are specific to file descriptors.
- io_netty_channel_epoll_Native.[c|h] will only have code that is specific to Linux.

Result:
Code is decoupled and design is streamlined in FileDescriptor.
2015-11-02 12:29:44 -08:00
Scott Mitchell
d5f502d940 Native getSoError bug
Motivation:
Java_io_netty_channel_epoll_Native_getSoError incorrectly returns the value from the get socket option function.

Modifications:
- return the value from the result of the get socket option call

Result:
Java_io_netty_channel_epoll_Native_getSoError returns the correct value.
2015-10-16 11:13:35 -07:00
Scott Mitchell
2f8bc24c62 Fix compile error introduced by 32231ee 2015-10-06 14:23:24 -07:00
Scott Mitchell
32231ee2e0 EPOLL RDHUP and IN at same time
Motivation:
If a RDHUP and IN event occurred at the same time it is possible we may not read all pending data on the channel. We should ensure we read data before processing the RDHUP event.

Modifications:
- Process the RDHUP event before the IN event.

Result:
Data will not be dropped.
Fixes https://github.com/netty/netty/issues/4317
2015-10-06 13:51:15 -07:00
Scott Mitchell
b2399c2475 EPOLL Shutdown Input Half Closed
Motivation:
EPOLL attempts to support half closed socket, but fails to call shutdown to close the read portion of the file descriptor.

Motivation:
- If half closed is supported shutting down the input should call underlying Native.shutdown(...) to make sure the peer is notified of the half closed state.

Result:
EPOLL half closed is more correct.
2015-10-06 12:45:57 -07:00