Commit Graph

112 Commits

Author SHA1 Message Date
Trustin Lee
b175b3d8be Add more utility methods to check the availability of the epoll transport
Related issue: #2733

Motivation:

Unlike OpenSsl, Epoll lacks a couple useful availability checker
methods:

- ensureAvailability()
- unavailabilityCause()

Modifications:

Add missing methods

Result:

More ways to check the availability and to get the cause of
unavailability programatically.
2014-08-04 15:03:25 -07:00
Trustin Lee
16e50765d1 Fix a stall write in EpollSocketChannel
Motivation:

When a ChannelOutboundBuffer contains a series of entries whose messages
are all empty buffers, EpollSocketChannel sometimes fails to remove
them. As a result, the result of the write(EmptyByteBuf) is never
notified, making the user application hang.

Modifications:

- Add ChannelOutboundBuffer.removeBytes(long) method that updates the
  progress of the entries and removes them as much as the specified
  number of written bytes.  It also updates the reader index of
  partially flushed buffer.
  - Make both NioSocketChannel and EpollSocketChannel use it to reduce
    code duplication
  - Replace EpollSocketChannel.updateOutboundBuffer()
- Refactor EpollSocketChannel.doWrite() for simplicity
  - Split doWrite() into doWriteSingle() and doWriteMultiple()
- Do not add a zero-length buffer to IovArray
- Do not perform any real I/O when the size of IovArray is 0

Result:

Another regression is gone.
2014-08-01 16:58:12 -07:00
Trustin Lee
5e5d1a58fd Overall cleanup
- ChannelOutboundBuffer.Entry.buffers -> bufs for consistency
- Make Native.IOV_MAX final because it's a constant
- Naming changes
  - FlushedMessageProcessor -> MessageProcessor just in case we can
    reuse it for unflushed messages in the future
- Add ChannelOutboundBuffer.Entry.recycle() that does not return the
  next entry, and use it wherever possible
- Javadoc clean-up
2014-07-30 14:57:13 -07:00
Norman Maurer
e282e504f1 Optimize gathering write in the epoll transport
Motivation:

While benchmarking the native transport, I noticed that gathering write
is not as fast as expected.  It was due to the fact that we have to do a
lot of array copies to put the buffer addresses into the iovec struct
array.

Modifications:

Introduce a new class called IovArray, which allows to fill buffers
directly into an off-heap array of iovec structs, so that it can be
passed over to JNI without any extra array copies.

Result:

Big performance improvement when doing gathering writes:

Before:

[nmaurer@xxx]~% wrk/wrk -H 'Host: localhost' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' -H 'Connection: keep-alive' -d 120 -c 256 -t 16 --pipeline 256  http://xxx:8080/plaintext
Running 2m test @ http://xxx:8080/plaintext
  16 threads and 256 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    23.44ms   16.37ms 259.57ms   91.77%
    Req/Sec   181.99k    31.69k  304.60k    78.12%
  346544071 requests in 2.00m, 46.48GB read
Requests/sec: 2887885.09
Transfer/sec:    396.59MB

After:

[nmaurer@xxx]~% wrk/wrk -H 'Host: localhost' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' -H 'Connection: keep-alive' -d 120 -c 256 -t 16 --pipeline 256  http://xxx:8080/plaintext
Running 2m test @ http://xxx:8080/plaintext
  16 threads and 256 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    21.93ms   16.33ms 305.73ms   92.34%
    Req/Sec   194.56k    33.75k  309.33k    77.04%
  369617503 requests in 2.00m, 49.57GB read
Requests/sec: 3080169.65
Transfer/sec:    423.00MB
2014-07-30 14:57:13 -07:00
Trustin Lee
997d8c32d2 Fix a regression caused by 73dfd7c01b
Motivation:

73dfd7c01b introduced various test
failures because:

- EpollSocketChannel.doWrite() raised a NullPointerException when
  notifying the write progress.
- ChannelOutboundBuffer.nioBuffers() did not expand the internal array
  when the pending entries contained more than 1024 buffers, dropping
  the remainder.

Modifications:

- Fix the NPE in EpollSocketChannel by removing an unnecessary progress
  update
- Expand the thread-local buffer array if there is not enough room,
  which was the original behavior dropped by the offending commit

Result:

Regression is gone.
2014-07-30 13:49:17 -07:00
Norman Maurer
c90de50ea7 Use correct exception message when throw exception from native code
Motivation:

We sometimes not use the correct exception message when throw it from the native code.

Modifications:

Fixed the message.

Result:

Correct message in exception
2014-07-28 13:31:26 -07:00
Norman Maurer
35061a4332 [#2692] Allows notify ChannelFutureProgressListener on complete writes
Motivation:

We have some inconsistency when handling writes. Sometimes we call ChannelOutboundBuffer.progress(...) also for complete writes and sometimes not. We should call it always.

Modifications:

Correctly call ChannelOuboundBuffer.progress(...) for complete and incomplete writes.

Result:

Consistent behavior
2014-07-28 04:12:59 -07:00
Norman Maurer
eb34cbb2d8 [#2685] Epoll transport should use GetPrimitiveArrayCritical / ReleasePrimitiveArrayCritical
Motivation:

At the moment we use Get*ArrayElement all the time in the epoll transport which may be wasteful as the JVM may do a memory copy for this. For code-path that will get executed fast (without blocking) we should better make use of GetPrimitiveArrayCritical and ReleasePrimitiveArrayCritical as this signal the JVM that we not want to do any memory copy if not really needed. It is important to only do this on non-blocking code-path as this may even suspend the GC to disallow the JVM to move the arrays around.

See also http://docs.oracle.com/javase/7/docs/technotes/guides/jni/spec/functions.html#GetPrimitiveArrayCritical

Modification:

Make use of GetPrimitiveArrayCritical / ReleasePrimitiveArrayCritical as replacement for Get*ArrayElement / Release*ArrayElement where possible.

Result:

Better performance due less memory copies.
2014-07-21 07:07:09 +02:00
Norman Maurer
028128993d [#2684] EpollSocketChannel gathering writes should take fast-path if possible
Motivation:

In EpollSocketchannel.writeBytesMultiple(...) we loop over all buffers to see if we need to adjust the readerIndex for incomplete writes. We can skip this if we know that everything was written (a.k.a complete write).

Modification:

Use fast-path if all bytes are written and so no need to loop over buffers

Result:

Fast write path for the average use.
2014-07-21 06:43:49 +02:00
Norman Maurer
cb2246ee07 [#2680] ChannelOutboundBuffer.nioBuffers() should always return non-null array as stated in javadocs
Motivation:

At the moment ChannelOutboundBuffer.nioBuffers() returns null if something is contained in the ChannelOutboundBuffer which is not a ByteBuf. This is a problem for two reasons:
 1 - In the javadocs we state that it will never return null
 2 - We may do a not optimal write as there may be things that could be written via gathering writes

Modifications:

Change ChannelOutboundBuffer.nioBuffers() to never return null but have it contain all ByteBuffer that were found before the non ByteBuf. This way we can do a gathering write and also conform to the javadocs.

Result:

Better speed and also correct implementation in terms of the api.
2014-07-20 19:13:48 +02:00
Norman Maurer
72173f0d16 [#2667] Write until EAGAIN in native transport and only call setEpollOut() in this case
Motivation:

In the previous fix for #2667 I did introduce a bit overhead by calling setEpollOut() too often.

Modification:

Only call setEpollOut() if really needed and remove unused code.

Result:

Less overhead when saturate network.
2014-07-18 20:31:19 +02:00
Norman Maurer
530badb239 [#2665] Continue writing on IOException when using DatagramChannel
Motivation:

As a DatagramChannel supports to write to multiple remote peers we must not close the Channel once a IOException accours as this error may be only valid for one remote peer.

Modification:

Continue writing on IOException.

Result:

DatagramChannel can be used even after an IOException accours during writing.
2014-07-18 12:35:18 +02:00
Norman Maurer
4835ff2d85 [#2667] Write until EAGAIN in native transport
Motivation:

We need to continue write until we hit EAGAIN to make sure we not see an starvation

Modification:

Write until EAGAIN is returned

Result:

No starvation when using native transport with ET.
2014-07-18 09:45:08 +02:00
Norman Maurer
460bf37387 [#2666] Fix possible NPE when try to fullfill connect ChannelPromise
Motivation:

Because of a missing return statement we may produce a NPE when try to fullfill the connect ChannelPromise when it was fullfilled before.

Modification:

Add missing return statement.

Result:

No more NPE.
2014-07-18 07:10:29 +02:00
Norman Maurer
db790123fd [#2647] Handle IOV_MAX in java code
Motivation:

The handling of IOV_MAX was done in JNI code base which makes stuff really complicated to maintain etc.

Modifications:

Move handling of IOV_MAX to java code to simplify stuff

Result:

Cleaner code.
2014-07-17 16:00:53 +02:00
Norman Maurer
bc10131f71 [#2664] Support write-spinning also in native transport
Motivation:

In our nio implementation we use write-spinning for maximize throughput, but in the native implementation this is not used.

Modification:

Respect writeSpinCount in native transport.

Result:

Better throughput
2014-07-17 13:10:38 +02:00
Norman Maurer
b440fa840b [#2647] Respect IOV_MAX when call writev in native transport
Motivation:

epoll transport fails on gathering write of more then 1024 buffers. As linux supports max. 1024 iov entries when calling writev(...) the epoll transport throws an exception.

Thanks again to @blucas to provide me with a reproducer and so helped me to understand what the issue is.

Modifications:

Make sure we break down the writes if to many buffers are uses for gathering writes.

Result:

Gathering writes work with any number of buffers
2014-07-09 13:32:43 +02:00
Trustin Lee
0a8ff3b52d Fix most inspector warnings
Motivation:

It's good to minimize potentially broken windows.

Modifications:

Fix most inspector warnings from our profile
Update IntObjectHashMap

Result:

Cleaner code
2014-07-02 20:21:30 +09:00
Norman Maurer
893bc04eee [#2558] Define SO_REUSEPORT if not defined
Motivation:

Currently it is impossible to build netty on linux system that not define SO_REUSEPORT even if it is supported.

Modification:

Define SO_REUSEPORT if not defined.

Result:

Possible to build on more linux dists.
2014-07-02 09:39:27 +02:00
Norman Maurer
016bdfbf66 Correctly return from selector loop one a scheduled task is ready for processing
Motivation:

We use the nanoTime of the scheduledTasks to calculate the milli-seconds to wait for a select operation to select something. Once these elapsed we check if there was something selected or some task is ready for processing. Unfortunally we not take into account scheduled tasks here so the selection loop will continue if only scheduled tasks are ready for processing. This will delay the execution of these tasks.

Modification:

- Check if a scheduled task is ready after selecting
- also make a tiny change in NioEventLoop to not trigger a rebuild if nothing was selected because the timeout was reached a few times in a row.

Result:

Execute scheduled tasks on time.
2014-07-02 08:31:47 +02:00
Norman Maurer
8427aa723a [#2623] Release local references to guard against StackOverflow in JNI
Motivation:

When we do a (env*)->GetObjectArrayElement(...) call we may created many local references which will only be cleaned up once we exist the native method. Thus a lot of memory can be used and so a StackOverFlow may be triggered. Beside this the JNI specification only say that an implementation must cope with 16 local references.

Modification:

Call (env*)->ReleaseLocalRef(...) to release the resource once not needed anymore.

Result:

Less memory usage and guard against StackOverflow
2014-06-30 10:40:52 +02:00
Norman Maurer
384ba32b56 Let EpollReuseAddrTest also work with kernel versions that not have bugfix release part 2014-06-27 17:56:31 +02:00
Norman Maurer
8869f6d5c1 [#2598] Add Epoll.isAvailable() which allows to check if epoll can be used.
Motivation:

At the moment there is no simple way for a user to check if the native epoll transport can be used on the running platform. Thus the user can only try to instance it and catch any exception and fallback to nio transport.

Modification:

Add Epoll.isAvailable() which allows to check if epoll can be used.

Result:

User can easily check if epoll transport can be used or not
2014-06-26 12:27:40 +02:00
Norman Maurer
d1c8bcb40f [#2605] Use SO_REUSEADDR on EpollServerSocketChannel to match defaults of java.nio.ServerSocketChannel impl
Motivation:

When using openjdk and oracle jdk's nio (while using the nio transport) the ServerSocketChannel uses SO_REUSEADDR by default. Our native transport should do the same to make it easier to switch between the different implementations and get the expected result.

Modification:

Change EpollServerSocketChannelConfig to set SO_REUSEADDR on the created socket.

Result:

SO_REUSEADDR is used by default on servers.
2014-06-26 11:53:23 +02:00
Norman Maurer
c3f24444ef Use IntObjectMap to replace Map in EpollEventLoop.
Motivation:

We need to map from ints to AbstractEpollChannel in EpollEventLoop but there is no need for box to Integer.

Modification:

Replace Map with IntObjectMap.

Result:

No more auto-boxing needed.
2014-06-25 20:23:16 +02:00
Trustin Lee
c58f28dfdd Add an OpenSslEngine and the universal API for enabling SSL
Motivation:

Some users already use an SSLEngine implementation in finagle-native. It
wraps OpenSSL to get higher SSL performance.  However, to take advantage
of it, finagle-native must be compiled manually, and it means we cannot
pull it in as a dependency and thus we cannot test our SslHandler
against the OpenSSL-based SSLEngine.  For an instance, we had #2216.

Because the construction procedures of JDK SSLEngine and OpenSslEngine
are very different from each other, we also need to provide a universal
way to enable SSL in a Netty application.

Modifications:

- Pull netty-tcnative in as an optional dependency.
  http://netty.io/wiki/forked-tomcat-native.html
- Backport NativeLibraryLoader from 4.0
- Move OpenSSL-based SSLEngine implementation into our code base.
  - Copied from finagle-native; originally written by @jpinner et al.
  - Overall cleanup by @trustin.
- Run all SslHandler tests with both default SSLEngine and OpenSslEngine
- Add a unified API for creating an SSL context
  - SslContext allows you to create a new SSLEngine or a new SslHandler
    with your PKCS#8 key and X.509 certificate chain.
  - Add JdkSslContext and its subclasses
  - Add OpenSslServerContext
- Add ApplicationProtocolSelector to ensure the future support for NPN
  (NextProtoNego) and ALPN (Application Layer Protocol Negotiation) on
  the client-side.
- Add SimpleTrustManagerFactory to help a user write a
  TrustManagerFactory easily, which should be useful for those who need
  to write an alternative verification mechanism. For example, we can
  use it to implement an unsafe TrustManagerFactory that accepts
  self-signed certificates for testing purposes.
- Add InsecureTrustManagerFactory and FingerprintTrustManager for quick
  and dirty testing
- Add SelfSignedCertificate class which generates a self-signed X.509
  certificate very easily.
- Update all our examples to use SslContext.newClient/ServerContext()
- SslHandler now logs the chosen cipher suite when handshake is
  finished.

Result:

- Cleaner unified API for configuring an SSL client and an SSL server
  regardless of its internal implementation.
- When native libraries are available, OpenSSL-based SSLEngine
  implementation is selected automatically to take advantage of its
  performance benefit.
- Examples take advantage of this modification and thus are cleaner.
2014-05-18 02:54:23 +09:00
Norman Maurer
dd0782990b [#2485] Use RecvByteBufAllocator for all allocations related to read from Channel
Motivation:
At the moment we sometimes use only RecvByteBufAllocator.guess() to guess the next size and the use the ByteBufAllocator.* directly to allocate the buffer. We should always use RecvByteBufAllocator.allocate(...) all the time as this makes the behavior easier to adjust.

Modifications:
Change the read() implementations to make use of RecvByteBufAllocator.

Result:
Behavior is more consistent.
2014-05-10 15:28:02 +02:00
Trustin Lee
568295d6c0 Simplify native library resolution using os-maven-plugin
Motivation:

So far, we used a very simple platform string such as linux64 and
linux32.  However, this is far from perfection because it does not
include anything about the CPU architecture.

Also, the current build tries to put multiple versions of .so files into
a single JAR.  This doesn't work very well when we have to ship for many
different platforms.  Think about shipping .so/.dynlib files for both
Linux and Mac OS X.

Modification:

- Use os-maven-plugin as an extension to determine the current OS and
  CPU architecture reliable at build time
- Use Maven classifier instead of trying to put all shared libraries
  into a single JAR
- NativeLibraryLoader does not guess the OS and bit mode anymore and it
  always looks for the same location regardless of platform, because the
  Maven classifier does the job instead.

Result:

Better scalable native library deployment and retrieval
2014-05-02 04:22:11 +09:00
Norman Maurer
ee55d34cfe [#2414] Fix IllegalStateException when try to configure AbstractEpollChannel once it is deregistered
Motivation:
AbstractEpollChannel.clearEpollIn() throws an IllegalStateException if a user tries to change the autoRead configuration for the Channel and the Channel is not registered on an EventLoop yet. This makes it for example impossible to set AUTO_READ to false via the ServerBootstrap as the configuration is modifed before the Channel is registered.

Modification:
Check if the Channel is registered and if not just modify the flags directly so they are respected once the Channel is registered

Result:
It is possible now to configure AUTO_READ via the ServerBootstrap
2014-04-22 10:14:31 +02:00
Norman Maurer
12ac35cd57 [#2414] Fix RuntimeException during modify events via EpollEventLoop
Motivation:
We are currently try to modify the events via EpollEventLoop even when the channel was closed before and so the fd was set to -1. This fails with a RuntimeException in this case.

Modification:
Always check if the Channel is still open before try to modify the events.

Result:
No more RuntimeException because of a not open channel
2014-04-21 11:16:07 +02:00
Norman Maurer
302d116728 Use correct generics for TCP_KEEPIDLE, TCP_KEEPINTVL and TCP_KEEPCNT in EpollChannelOption. Part of [#2396]
Motivation:
Currently the generics used for TCP_KEEPIDLE, TCP_KEEPINTVL and TCP_KEEPCNT are incorrect.

Modifications:
Use Integer as type

Result:
User can use TCP_KEEPIDLE, TCP_KEEPINTVL and TCP_KEEPCNT as expected
2014-04-21 09:56:57 +02:00
Norman Maurer
886829d7e0 Fix buffer leak in EpollDatagramChannel
Motivation:
EpollDatagramChannel produced buffer leaks when tried to read from the channel and nothing was ready to be read.

Modifications:
Correctly release buffer if nothing was read

Result:
No buffer leak
2014-04-18 20:34:37 +02:00
Norman Maurer
b13953fd45 [#2396] Allow to set TCP_KEEPIDLE, TCP_KEEPINTVL and TCP_KEEPCNT in native transport
Motivation:
Allow to set TCP_KEEPIDLE, TCP_KEEPINTVL and TCP_KEEPCNT in native transport to offer the user with more flexibility.

Modifications:
Expose methods to set these options and write the JNI implementation.

Result:
User can now use TCP_KEEPIDLE, TCP_KEEPINTVL and TCP_KEEPCNT.
2014-04-18 11:27:58 +02:00
Norman Maurer
f5d4e6b10e [#2405] Add support for SO_REUSEPORT to EpollDatagramChannel
Motivation:
With SO_REUSEPORT it is possible to bind multiple sockets to the same port and so handle the processing of packets via multiple threads. This allows to handle DatagramPackets with more then one thread on the same port and so gives better performance.

Modifications:
Expose EpollDatagramChannelConfig.setReusePort(..) and isReusePort()

Result:
Allow to bind multiple times to the same local address and so archive better performance.
2014-04-17 11:19:00 +02:00
Norman Maurer
c2dc993c42 [#2375] [#2404] Fix bug in respecting ChannelConfig.setAutoRead(false) and also fix Channel.read() for OIO
Motivation:
At the moment ChanneConfig.setAutoRead(false) only is guaranteer to not have an extra channelRead(...) triggered when used from within the channelRead(...) or channelReadComplete(...) method. This is not the correct behaviour as it should also work from other methods that are triggered from within the EventLoop. For example a valid use case is to have it called from within a ChannelFutureListener, which currently not work as expected.

Beside this there is another bug which is kind of related. Currently Channel.read() will not work as expected for OIO as we will stop try to read even if nothing could be read there after one read operation on the socket (when the SO_TIMEOUT kicks in).

Modifications:
Implement the logic the right way for the NIO/OIO/SCTP and native transport, specific to the transport implementation. Also correctly handle Channel.read() for OIO transport by trigger a new read if SO_TIMEOUT was catched.

Result:
It is now also possible to use ChannelConfig.setAutoRead(false) from other methods that are called from within the EventLoop and have direct effect.
2014-04-17 07:37:34 +02:00
Norman Maurer
72a077b6c7 [#2377] Implement epoll based DatagramChannel
Motivation:
There is currently no epoll based DatagramChannel. We should add one to make the set of provided channels complete and also to be able to offer better performance compared to the NioDatagramChannel once SO_REUSEPORT is implemented.

Modifications:
Add implementation of DatagramChannel which uses epoll. This implementation does currently not support multicast yet which will me implemented later on. As most users will not use multicast anyway I think it is fair to just add the EpollDatagramChannel without the support for now. We shipped NioDatagramChannel without support earlier too ...

Result:
Be able to use EpollDatagramChannel for max. performance on linux
2014-04-16 15:28:04 +02:00
Norman Maurer
02c3e71012 [#2376] Add support for SO_REUSEPORT in native transport
Motivation:
In linux kernel 3.9 a new featured named SO_REUSEPORT was introduced which allows to have multiple sockets bind to the same port and so handle the accept() of new connections with multiple threads. This can greatly improve the performance when you not to accept a lot of connections.

Modifications:
Implement SO_REUSEPORT via JNI

Result:
Be able to use the SO_REUSEPORT feature when using the EpollServerSocketChannel
2014-04-16 14:25:02 +02:00
Norman Maurer
9cf92b3999 [#2371] Fix data corruption caused by EpollSocketChannel when writing
Motivation:
We sometimes see data corruption when writing to the EpollSocketChannel.

Modifications:
The problem was caused as we mixed writing via memory address and via ByteBuffer. This not works out pretty well because of how the position of the buffer is updated etc. To fix the problem we only write via ByteBuffer (this is true for normal and gathering writes). Before normal writes may write via the memory address and gathering writes always used the ByteBuffer.

Result:
Fix data-corruption which could happen on partial writes
2014-04-13 09:54:16 +02:00
Norman Maurer
ce3af65074 [#2371] Fix Potential data corruption in EpollSocketChannel when writing to the Channel
Motivation:
We sometimes see data corruption when writing to the EpollSocketChannel.

Modifications:
Correctly update the position of the ByteBuffer after something was written.

Result:
Fix data-corruption which could happen on partial writes
2014-04-09 14:22:23 +02:00
Norman Maurer
012166803a [#2353] Use a privileged block to get ClassLoader and System property if needed
Motivation:
When using System.getProperty(...) and various methods to get a ClassLoader it will fail when a SecurityManager is in place.

Modifications:
Use a priveled block if needed. This work is based in the PR #2353 done by @anilsaldhana .

Result:
Code works also when SecurityManager is present
2014-04-08 13:59:03 +02:00
Norman Maurer
a4c80211ea [#2359] EpollSocketChannel.remoteAddress0() is always null on accepted EpollSocketChannels
Motivation:
EpollSocketChannel.remoteAddress0() is always null on accepted EpollSocketChannels as we not set it excplicit.

Modifications:
Correctly retrieve the local and remote address when accept new channel and store it

Result:
EpollSocketchannel.remoteAddress0() and EpollSocketChannel.localAddress0() return correct addresses
2014-04-04 15:23:07 +02:00
Norman Maurer
9d695e03db [#2361] Native.epollCreate(...) fails on systems using a kernel < 2.6.27 / glibc < 2.9
Motivation:
Native.epollCreate(...) fails on systems using a kernel < 2.6.27 / glibc < 2.9 because it uses epoll_create1(...) without checking if it is present

Modifications:
Check if epoll_create1(...) exists abd if not fall back to use epoll_create(...)

Result:
Works even on systems with kernel < 2.6.27 / glibc < 2.9
2014-04-04 07:54:49 +02:00
Norman Maurer
76d091a884 Make sure the local / remote InetSocketAddres can be obtained. Part of [#2262]
Motivation:
Make sure the remote/local InetSocketAddress can be obtained correctly

Modifications:
Set the remote/local InetSocketAddress after a bind/connect operation was performed

Result:
It is possible to still access the informations even after the fd became invalid. This mirror the behaviour of NIO.
2014-03-22 15:01:49 +01:00
Trustin Lee
5acb8759e4 Perform cross-tests between NIO and epoll transport
Motivation:

The epoll testsuite tests the epoll transport only against itself (i.e. epoll x epoll only).  We should test the epoll transport also against the well-tested NIO transport, too.

Modifications:

- Make SocketTestPermutation extensible and reusable so that the epoll testsuite can take advantage of it.
- Rename EpollTestUtils to EpollSocketTestPermutation and make it extend SocketTestPermutation.
- Overall clean-up of SocketTestPermutation
  - Use Arrays.asList() for simplicity
  - Add combo() method to remove code duplication

Result:

The epoll transport is now also tested against the NIO transport.  SocketTestPermutation got cleaner.
2014-03-17 10:38:18 +09:00
Trustin Lee
bcd11d1ac2 Fix 'incompatible event loop' regression
Motivation:

Previous commit (2de65e25e9) introduced a regression that makes the epoll testsuite fail with an 'incompatible event loop' error.

Modifications:

Use the correct event loop type.

Result:

Build doesn't fail anymore.
2014-03-17 09:52:10 +09:00
Trustin Lee
eda702b6ab Set timeout for SocketSslEchoTest
Motivation:

We are seeing EpollSocketSslEchoTest does not finish itself while its I/O thread is busy.  Jenkins should have terminated them when the global build timeout reaches, but Jenkins seems to fail to do so.  What's more interesting is that Jenkins will start another job before the EpollSocketSslEchoTest is terminated, and Linux starts to oom-kill them, impacting the uptime of the CI service.

Modifications:

- Set timeout for all test cases in SocketSslEchoTest so that all SSL tests terminate themselves when they take too long.
- Fix a bug where the epoll testsuite uses non-daemon threads which can potentially prevent JVM from quitting.
- (Cleanup) Separate boss group and worker group just like we do for NIO/OIO transport testsuite.

Result:

Potentially more stable CI machine.
2014-03-17 09:31:48 +09:00
Norman Maurer
140f963d60 Replace usage of UnknownHostException with UnresolveableAddressException. Part of [#2262]
Motivation:
We better use UnresolveableAddressException as NIO does the same.

Modifications:
Replace usage of UnknownHostException with UnresolveableAddressException

Result:
epoll transport and nio transport behave the same way
2014-03-16 08:29:45 -07:00
Norman Maurer
b22d92057b [#2262] Fix NPE triggered by unresolveable InetSocketAddress in epoll transport
Motivation:
At the moment when an unresolvable InetSocketAddress is passed into the epoll transport a NPE is thrown

Modifications:
Add check in place which will throw an UnknownHostException if an InetSocketAddress could not been resolved.

Result:
Proper handling of unresolvable InetSocketAddresses
2014-03-16 06:25:52 -07:00
Norman Maurer
fc5d20ff2d [#2297] Correctly close all registered Channels on EpollEventLoop.closeAll() 2014-03-10 20:27:44 +01:00
Aaron Riekenberg
223deb255e [#2280] Correct logic in Native.finishConnect. Fix use of optval parameter in c getOption function. In epoll event loop, check that channel is open before processing event. 2014-03-09 19:37:32 +01:00
Norman Maurer
bdedde1294 [#1259] Add optimized queue for SCMP pattern and use it in NIO and native transport
This queue also produces less GC then CLQ when make use of OneTimeTask
2014-02-27 11:44:06 +01:00
Norman Maurer
c538fd5098 Fix check to clear READ_OP and EPOLLIN. Part of [#2254] 2014-02-22 20:06:35 +01:00
Norman Maurer
b00e2a19d3 [#2254] Correctly handle Channel.read() and ChannelHandlerContext.read()
This includes also when it is called from channelRead(...) and channelReadComplete(...) methods.
2014-02-22 18:45:36 +01:00
Norman Maurer
7fe10fe635 [#2254] Fix regression in handling autoRead and Channel.read()
This regression was introduced by e0b39159657c9eb711047bc32367537c4870d467
2014-02-21 08:39:41 +01:00
Norman Maurer
34e11d2e16 Correctly handle remove from epoll and EPOLLRDHUP. Also fix a bug with removing EPOLLIN and EPOLLOUT 2014-02-19 16:30:24 +01:00
Norman Maurer
b2a0a25b8f Use optimized write and read calls if memoryAddress is present. Part of [#2239] 2014-02-17 21:06:57 +01:00
Trustin Lee
61bfd46264 Fix an inspector warning 2014-02-17 05:25:58 -08:00
Trustin Lee
ab257b0ac6 Overall clean-up in EpollSocketChannel
- Extract writev part from doWrite() for simplicity
- Clearer comments
2014-02-17 05:21:34 -08:00
Norman Maurer
ea73d074f2 [#2238] Fix bug which caused a SIGSEGV when calling remoteAddress() or localAddress() 2014-02-17 11:59:28 +01:00
Norman Maurer
84b537519c Correctly update the ByteBuffers position on write and writev 2014-02-17 07:30:06 +01:00
Norman Maurer
04dab876b6 Use correct writerIndex when read for channel 2014-02-16 20:21:14 +01:00
Norman Maurer
1edca5f8d9 Introduce a native transport for linux using epoll ET
This transport use JNI (C) to directly make use of epoll in Edge-Triggered mode for maximal performance on Linux. Beside this it also support using TCP_CORK and produce less GC then the NIO transport using JDK NIO.
It only builds on linux and skip the build if linux is not used. The transport produce a jar which contains all needed .so files for 32bit and 64 bit. The user only need to include the jar as dependency as usually
to make use of it and use the correct classes.

This includes also some cleanup of @trustin
2014-02-15 22:27:43 +01:00