Commit Graph

175 Commits

Author SHA1 Message Date
Scott Mitchell
dbbdbe11a6 Decouple Unix from Linux in Native Transport
Motivation:
transport-native-epoll is designed to be specific to Linux. However there is native code that can be extracted out and made to work on more Unix like distributions. There are a few steps to be completely decoupled but the first step is to extract out code that can run in a more general Unix environment from the Linux specific code base.

Modifications:
- Move all non-Linux specific stuff from Native.java into the io.netty.channel.unix package.
- io.netty.channel.unix.FileDescriptor will inherit all the native methods that are specific to file descriptors.
- io_netty_channel_epoll_Native.[c|h] will only have code that is specific to Linux.

Result:
Code is decoupled and design is streamlined in FileDescriptor.
2015-11-02 12:29:44 -08:00
Scott Mitchell
d5f502d940 Native getSoError bug
Motivation:
Java_io_netty_channel_epoll_Native_getSoError incorrectly returns the value from the get socket option function.

Modifications:
- return the value from the result of the get socket option call

Result:
Java_io_netty_channel_epoll_Native_getSoError returns the correct value.
2015-10-16 11:13:35 -07:00
Scott Mitchell
2f8bc24c62 Fix compile error introduced by 32231ee 2015-10-06 14:23:24 -07:00
Scott Mitchell
32231ee2e0 EPOLL RDHUP and IN at same time
Motivation:
If a RDHUP and IN event occurred at the same time it is possible we may not read all pending data on the channel. We should ensure we read data before processing the RDHUP event.

Modifications:
- Process the RDHUP event before the IN event.

Result:
Data will not be dropped.
Fixes https://github.com/netty/netty/issues/4317
2015-10-06 13:51:15 -07:00
Scott Mitchell
b2399c2475 EPOLL Shutdown Input Half Closed
Motivation:
EPOLL attempts to support half closed socket, but fails to call shutdown to close the read portion of the file descriptor.

Motivation:
- If half closed is supported shutting down the input should call underlying Native.shutdown(...) to make sure the peer is notified of the half closed state.

Result:
EPOLL half closed is more correct.
2015-10-06 12:45:57 -07:00
Norman Maurer
2ff2806ada [maven-release-plugin] prepare for next development iteration 2015-10-02 09:03:29 +02:00
Norman Maurer
5a43de10f7 [maven-release-plugin] prepare release netty-4.1.0.Beta7 2015-10-02 09:02:58 +02:00
Norman Maurer
2ffe7bd72e Fail build on warnings in the native transport
Motivation:

We should fail the build on warnings in the JNI/c code.

Modifications:

- Add GCC flag to fail build on warnings.
- Fix warnings (which also fixed a bug when using splice with offsets).

Result:

Better code quality.
2015-09-30 20:26:36 +02:00
Norman Maurer
747533408d [#4170] Shutdown socket before close fd when using epoll transport
Motivation:

We should call shutdown(...) on the socket before closing the filedescriptor to ensure it is closed gracefully.

Modifications:

Call shutdown(...) before close.

Result:

Sockets are gracefully shutdown when using native transport.
2015-09-25 20:05:14 +02:00
Peeyush Aggarwal
4f40913b33 Use NetUtil.LOCALHOST4 instead of InetAddress.getLocalHost()
Motivation:

On ubuntu, InetAddress.getLocalHost() will return 127.0.1.1 this causes some tests to fail.
NetUtil.LOCALHOST4 is more portable.

Modifications:

Made changes in EpollSocketTcpMd5Test to make test passing on ubuntu.

Result:

EpollSocketTcpMd5Test now also passes on ubuntu.
2015-09-24 10:49:30 +02:00
nmittler
3ee44a3dbb Update Netty to latest netty-tcnative
Motivation:

The latest netty-tcnative fixes a bug in determining the version of the runtime openssl lib.  It also publishes an artificact with the classifier linux-<arch>-fedora for fedora-based systems.

Modifications:

Modified the build files to use the "-fedora" classifier when appropriate for tcnative. Care is taken, however, to not change the classifier for the native epoll transport.

Result:

Netty is updated the the new shiny netty-tcnative.
2015-09-18 12:07:21 -07:00
Norman Maurer
076d4ed514 [#4205] Correctly set EPOLLOUT flag whe writeBytes(...) was not able to write everything
Motivation:

writeBytes(...) missed to set EPOLLOUT flag when not all bytes were written. This could lead to have the EpollEventLoop not try to flush the remaining bytes once the socket becomes writable again.

Modifications:

- Move setting EPOLLOUT flag logic to one point so we are sure we always do it.
- Move OP_WRITE flag logic to one point as well.

Result:

Correctly try to write pending data if socket becomes writable again.
2015-09-16 07:30:17 +02:00
Peeyush Aggarwal
250a09df63 Add support for RFC7413 on linux for server sockets
Motivation:

TCP Fast Open allows data to be carried in the SYN and SYN-ACK packets and consumed by the receiving end during the initial connection handshake, and saves up to one full round-trip time (RTT) compared to the standard TCP, which requires a three-way handshake (3WHS) to complete before data can be exchanged. This commit enables support for TFO on server sockets.

Modifications:

Added new Integer Option TCP_FASTOPEN in EpollChannelOption.
Added getters/setters in EpollServerChannelConfig for TCP_FASTOPEN.
Added way to check if TCP_FASTOPEN is supported on server in Native.
Added setting on socket opt TCP_FASTOPEN if value is set on channel options in doBind in EpollServerSocketChannel.
Enhanced EpollSocketTestPermutation to contain a permutation for server socket containing fast open.

Result:

Users of native-epoll can set TCP_FASTOPEN on server sockets and thus leverage fast connect features of RFC7413 if client is capable of it.

Conflicts:
	transport-native-epoll/src/main/java/io/netty/channel/epoll/EpollChannelOption.java
2015-09-09 06:58:08 +02:00
Robert Varga
30a7701616 Add support for RFC2385 on Linux
Motivation:

There are protocols (BGP, SXP), which are typically deployed with TCP
MD5 authentication to protect sessions from being hijacked/torn down by
third parties. This facility is not available on most operating systems,
but is typically present on Linux.

Modifications:

- add a new EpollChannelOption, which is write-only
- teach Epoll(Server)SocketChannel to track which addresses have keys
  associated
- teach Native how to set the MD5 signature keys for a socket

Result:

Users of the native-epoll transport can set MD5 signature keys and thus
leverage RFC-2385 protection on TCP connections.
2015-09-03 08:50:12 +02:00
Norman Maurer
34de2667c7 [maven-release-plugin] prepare for next development iteration 2015-09-02 11:45:20 +02:00
Norman Maurer
2eb444ec1d [maven-release-plugin] prepare release netty-4.1.0.Beta6 2015-09-02 11:36:11 +02:00
Norman Maurer
141d4d0101 Remove extra empty line introduced by 0c83542000 2015-08-31 21:18:26 +02:00
Tomas Olvecky
de7f9d4913 Add TCP_USER_TIMEOUT
Motivation:

See #4174.

Modifications:

Modify transport-native-epoll to allow setting TCP_USER_TIMEOUT.

Result:

Hanging connections that are written into will get timeouted.

Conflicts:
	transport-native-epoll/src/main/java/io/netty/channel/epoll/EpollChannelOption.java
2015-08-31 14:02:55 +02:00
Norman Maurer
0c83542000 Use ChannelException when ChannelConfig operation fails in epoll.
Motivation:

In NIO and OIO we throw a ChannelException if a ChannelConfig operation fails. We should do the same with epoll to be consistent.

Modifications:

Use ChannelException

Result:

Consistent behaviour across different transport implementations.
2015-08-28 21:38:35 +02:00
Norman Maurer
97f5b43a3e Only try to obtain SO_LINGER on close if fd is still open.
Motivation:

When try to get SO_LINGER from a fd that is closed an Exception is thrown. We should only try to get SO_LINGER if the fd is still open otherwise an Exception is thrown that can be ignored anyway.

Modifications:

First check if the fd is still open before try to obtain SO_LINGER setting when get the closeExecutor. This is also the same that we do in the NIO transport.

Result:

No more exception when calling unsafe.close() on a channel that has a closed file descriptor.
2015-08-28 14:50:16 +02:00
Norman Maurer
e7f2abbd36 [#4127] Correctly set traffic class and so linger.
Motivation:

The method implementions for setSoLinger(...) and setTrafficClass(...) were swapped by mistake.

Modifications:

Use the correct implementation for setSoLinger(...) and setTrafficClass(...)

Result:

Correct behaviour when setSoLinger(...) and setTrafficClass(...) are used with the epoll transport.
2015-08-27 08:58:38 +02:00
Scott Mitchell
a55df36ae4 ce6931e0e5 buffer leak in EpollSocketChannelTest
Motivation:
ce6931e0e5 introduced a buffer leak in EpollSocketChannelTest.

Modifications:
- Fix buffer leak

Result:
No more buffer leak.
2015-08-21 11:29:15 -07:00
Scott Mitchell
ce6931e0e5 EPOLL exception processing feedback loop
Motivation:
Commit cf171ff525 changed the way read operations were done. This change introduced a feedback loop between fireException and epollInReady.

Modifications:
- All EPOLL*Channel* classes should not call fireException and also continue to read. Instead a read operation should be executed on the eventloop (if the channel's input is not closed, and other conditions are satisfied)

Result:
Exception processing and channelRead will not be in a feedback loop.
Fixes https://github.com/netty/netty/issues/4091
2015-08-20 13:11:02 -07:00
Jonas Konrad
e6ca9882a1 Add support for abstract domain sockets
Motivation:

Because of java custom UTF encoding, it was previously impossible to use
nul-bytes in domain socket names, which is required for abstract domain
sockets.

Modifications:

- Pass the encoded string byte array to the native code
- Modify native code accordingly to work with nul-bytes in the the
array.
- Move the string encoding to UTF-8 in java code.

Result:

Unix domain socket addresses will work properly if they contain nul-
bytes. Address encoding for these addresses changes from UTF-8-like to
real UTF-8.
2015-08-16 20:03:23 +02:00
Trustin Lee
75af257a62 Ignore EpollReuseAddrTest.testMultipleBindDatagramChannel
Motivation:

It keeps failing on some of our CI machines for an unknown reason.

Modifications:

Ignore it temporarily

Result:

Less noise in CI result
2015-08-15 09:07:17 +09:00
Scott Mitchell
d2683c3911 EPOLLRDHUP infinite loop
Motivation:
If  is enabled and a channel is half closed it is possible for the EPOLL event loop to get into an infinite loop by continuously being woken up on the EPOLLRDHUP event.

Modifications:
- Ensure that the EPOLLRDHUP event is unregistered for to prevent infinite loop.

Result:
1 less infinite loop.
2015-08-13 12:15:32 -07:00
Norman Maurer
deda3f5b65 Add GCC optimization flag when compile native transport
Motivation:

We not set any optimization flag when compile native transport

Modification:

Add -O3 to CFLAGS to have GCC do optimizations

Result:

Ship optimized native code
2015-08-13 12:31:20 +02:00
Scott Mitchell
cf171ff525 maxBytesPerRead channel configuration
Motiviation:
The current read loops don't fascilitate reading a maximum amount of bytes. This capability is useful to have more fine grain control over how much data is injested.

Modifications:
- Add a setMaxBytesPerRead(int) and getMaxBytesPerRead() to ChannelConfig
- Add a setMaxBytesPerIndividualRead(int) and getMaxBytesPerIndividualRead to ChannelConfig
- Add methods to RecvByteBufAllocator so that a pluggable scheme can be used to control the behavior of the read loop.
- Modify read loop for all transport types to respect the new RecvByteBufAllocator API

Result:
The ability to control how many bytes are read for each read operation/loop, and a more extensible read loop.
2015-08-05 23:59:54 -07:00
Norman Maurer
f65717be82 Add support for IP_FREEBIND when using native transport
Motivation:

IP_FREEBIND allows to bind to addresses without the address up yet or even the interface configured yet.

Modifications:

Add support for IP_FREEBIND.

Result:

It's now possible to use IP_FREEBIND when using the native epoll transport.
2015-07-30 20:57:53 +02:00
nmittler
93fc3c6e45 Make IntObjectHashMap extend Map
Motivation:

It would be useful to support the Java `Map` interface in our primitive maps.

Modifications:

Renamed current methods to "pXXX", where p is short for "primitive". Made the template for all primitive maps extend the appropriate Map interface.

Result:

Fixes #3970
2015-07-22 15:52:27 -07:00
Norman Maurer
1a5dac175e Correctly register for EPOLLRDHUP when construct EpollSocketChannel from FileDescriptor
Motivation:

We missed to register for EPOLLRDHUP events when construct the EpollSocketChannel from an existing FileDescriptor. This could cause to miss connection-resets.

Modifications:

Add Native.EPOLLRDHUP to the events we are interested in.

Result:

Connection-resets are detected correctly.
2015-07-21 18:41:05 +02:00
Norman Maurer
49643fb5b1 [#3848] Respect EPOLLERR event
Motivation:

Some glibc/kernel versions will trigger an EPOLLERR event to notify
about failed connect and not an EPOLLOUT. Also EPOLLERR may be triggered
when a connection is broke.

Modification:

React on EPOLLERR like if an EPOLLOUT / EPOLLIN was received, this will work in
all cases as we handle errors in EPOLLOUT / EPOLLIN anyway.

Result:

Correctly detect errors.
2015-06-06 10:34:58 +02:00
Scott Mitchell
09ecc34924 Linux EPOLL Channel Configuration test unsupported options
Motivation:
The unit tests should not fail due to using a channel option which is not supported by the underlying kernel.

Modifications:
- Ignore RuntimeExceptions which are thrown by JNI code when setsockopt or getsockopt fails.

Result:
Unit tests pass if socket option is not supported by kernel.
2015-06-02 12:54:36 -07:00
Scott Mitchell
3c432b5a19 TCP_NOTSENT_LOWAT older kernel compatibility
Motiviation:
TCP_NOTSENT_LOWAT is only supported in linux kernel 3.12 or newer. The addition of this socket option prevents older kernels from building.

Modifications:
- Conditionally define TCP_NOTSENT_LOWAT if it is not defined

Result:
Kernels older than 3.12 can still compile the EPOLL module.
2015-06-02 12:19:53 -07:00
Scott Mitchell
5121fba466 TCP_NOTSENT_LOWAT socket option support
Motiviation:
Linux provides the TCP_NOTSENT_LOWAT socket option. This can be used to control how much unsent data is queued in the tcp kernel buffers. This can be important when application level protocols (SPDY, HTTP/2) have their own priority mechanism and don't want data queued in the kernel.

Modifications:
- The epoll module will have an additional socket option TCP_NOTSENT_LOWAT
- There will be JNI methods to control the underlying linux socket option mechanism

Result:
Linux EPOLL module exposes the TCP_NOTSENT_LOWAT socket option.
2015-06-01 13:21:38 -07:00
Clebert Suconic
70e3d17620 fixing small leak on exception on the transport-epoll-native allocation
Motivation:

the JNI function ThrowNew won't release any allocated memory.
The method exceptionMessage is allocating a new string concatenating 2 constant strings
What is creating a small leak in case of these exceptions are happening.

Modifications:

Added new methods that will use exceptionMessage and free resources accordingly.
I am also removing the inline definition on these methods as they could be reused by
other added modules (e.g. libaio which should be coming soon)

Result:

No more leaks in case of failures.
2015-05-31 09:36:19 +02:00
Norman Maurer
f2ba221e7e [#3785] Correctly handle connection refused with native transport
Motivation:

Due a bug we not correctly handled connection refused errors and so failed the connect promise with the wrong exception.
Beside this we some times even triggered fireChannelActive() which is not correct.

Modifications:

- Add testcase
- correctly detect connect errors

Result:

Correct and consistent handling.
2015-05-27 13:38:50 +02:00
Norman Maurer
e903979a28 Not try to write more then Integer.MAX_VALUE / SSIZE_MAX via writev
Motivation:

When trying to write more then Integer.MAX_VALUE / SSIZE_MAX via writev(...) the OS may return EINVAL depending on the kernel or the actual OS (bsd / osx always return EINVAL). This will trigger an IOException.

Modifications:

Never try to write more then Integer.MAX_VALUE / SSIZE_MAX when using writev.

Result:

No more IOException when write more data then Integer.MAX_VALUE / SSIZE_MAX via writev.
2015-05-21 12:02:10 +02:00
Norman Maurer
f23b7b4efd [maven-release-plugin] prepare for next development iteration 2015-05-07 14:21:08 -04:00
Norman Maurer
871ce43b1f [maven-release-plugin] prepare release netty-4.1.0.Beta5 2015-05-07 14:20:38 -04:00
Norman Maurer
3c60c9ce32 Fix regression introduced by 08e4b076be. 2015-05-06 17:57:13 +02:00
Norman Maurer
d711eb827d [#3709] Ensure all data is read from socket when EPOLLRDUP is received
Motivation:

When EPOLLRDHUP is received we need to try to read at least one time to ensure
that we read all pending data from the socket. Otherwise we may loose data.

Modifications:

- Ensure we read all data from socket
- Ensure file descriptor is closed on doClose() even if doDeregister() throws an Exception.
- Only handle either EPOLLRDHUP or EPOLLIN as only one is needed to detect connection reset.

Result:

No more data loss on connection reset.
2015-05-06 08:45:39 +02:00
Norman Maurer
5f5cdd1089 Proper handling of epoll_ctl errors
Motivation:

When using epoll_ctl we should respect the return value and do the right thing depending on it.

Modifications:

Adjust java and native code to respect epoll_ctl return values.

Result:

Correct and cleaner code.
2015-05-06 08:20:32 +02:00
Norman Maurer
ad9bec4d9d Add support for splice(...)
Motivation:

Linux supports splice(...) to transfer data from one filedescriptor to another without
pass data through the user-space. This allows to write high-performant proxy code or to stream
stuff from the socket directly the the filesystem.

Modification:

Add AbstractEpollStreamChannel.spliceTo(...) method to support splice(...) system call

Result:

Splice is now supported when using the native linux transport.

Conflicts:

	transport-native-epoll/src/main/java/io/netty/channel/epoll/AbstractEpollStreamChannel.java
2015-04-30 07:10:46 +02:00
Norman Maurer
1cce998bb0 [#3662] Fail the connect future on close
Motivation:

Because of a bug we missed to fail the connect future when doClose() is called. This can lead to a future which is never notified and so may lead to deadlocks in user-programs.

Modifications:

Correctly fail the connect future when doClose() is called and the connection was not established yet.

Result:

Connect future is always notified.
2015-04-27 20:05:00 +02:00
Norman Maurer
b4b14ea19f Ensure backward-compability with 4.0
Motivation:

Each different *ChannelOption did extend ChannelOption in 4.0, which we changed in 4.1. This is a breaking change in terms of the API so we need to ensure we keep the old hierarchy.

Modifications:

- Let all *ChannelOption extend ChannelOption
- Add back constructor and mark it as @deprecated

Result:

No API breakage between 4.0 and 4.1
2015-04-19 13:25:42 +02:00
Norman Maurer
e48b8f5c49 [#3539] Correctly handle EPOLLRDHUP
Motivation:

As we missed to correctly handle EPOLLRDHUP we produce an IOException which is unnessary. This leads
to have exceptionCaught(...) methods called.

Modifications:

When EPOLLRDHUP was received just close the socket and fail all pending writes.

Result:

Correctly handle of EPOLLRDHUP and so not miss-leading exceptions.
2015-04-14 06:57:11 +02:00
Norman Maurer
c7827dc16a First load RuntimeException to prevent segfault on error
Motivation:

When an error happens during loading the native library it may try to generate a new RuntimeException before the RuntimeException is loaded.

Modifications:

- Load RuntimeException as first

Result:

No more segfaults possible
2015-03-13 18:28:40 +01:00
Norman Maurer
3df7b4dac7 Respect -Djava.net.preferIPv4Stack when using epoll transport
Motivation:

On a system where ipv4 and ipv6 are supported a user may want to use -Djava.net.preferIPv4Stack=true to restrict it to use ipv4 only.
This is currently ignored with the epoll transport.

Modifications:

Respect java.net.preferIPv4Stack system property.

Result:

-Djava.net.preferIPv4Stack=true will have the effect the user is looking for.
2015-03-11 02:50:11 +01:00
Norman Maurer
1a83232ab2 [#3463] EpollSocketChannel.localAddress() returns always null if Native.connect() was not able to connect directly
Motivation:

Due a a regression that was introduced by b898bdd we failed to set the localAddress if the connect did not success directly.

Modifications:

Correct set localAddress in doConnect(...)

Result:

Be able to get the localAddress in all cases.
2015-03-04 20:07:09 +01:00