Commit Graph

444 Commits

Author SHA1 Message Date
Trustin Lee
5d11be58ea More meaningful assertion failure message
We see sporadic failure in EpollSocketConnectionAttemptTest. Generate
more useful failure message using hamcrest to know more.
2015-06-04 12:08:30 +09:00
Trustin Lee
4d79be44ef Fix sporadic failures in DatagramUnicastTest
Motivation:

DatagramUnitcastTest sometimes fails with BindException for an unknown reason.

Modifications:

Retry up to 3 times with a new free port when bind() fails with BindException

Result:

More build stability
2015-06-01 17:33:06 +09:00
Trustin Lee
c7b164b409 Fix sporadic assertion failure in SocketSslEchoTest
Motivation:

SocketSslEchoTest.testSslEcho() has a race condition where a renegotiation future can be done before:

    assertThat(renegoFuture.isDone(), is(false));

Modifications:

Remove the offending assertion.

Result:

More build stability
2015-06-01 13:56:47 +09:00
Norman Maurer
f2ba221e7e [#3785] Correctly handle connection refused with native transport
Motivation:

Due a bug we not correctly handled connection refused errors and so failed the connect promise with the wrong exception.
Beside this we some times even triggered fireChannelActive() which is not correct.

Modifications:

- Add testcase
- correctly detect connect errors

Result:

Correct and consistent handling.
2015-05-27 13:38:50 +02:00
Norman Maurer
f23b7b4efd [maven-release-plugin] prepare for next development iteration 2015-05-07 14:21:08 -04:00
Norman Maurer
871ce43b1f [maven-release-plugin] prepare release netty-4.1.0.Beta5 2015-05-07 14:20:38 -04:00
Norman Maurer
f963401d42 Allow rejection of remote initiated renegotiation
Motivation:

To prevent from DOS attacks it can be useful to disable remote initiated renegotiation.

Modifications:

Add new flag to OpenSslContext that can be used to disable it
Adding a testcase

Result:

Remote initiated renegotion requests can be disabled now.
2015-05-07 14:41:25 +02:00
Norman Maurer
b0a30cbf39 Fix race-condition in SocketSslEchoTest
Motivation:

Because we tried to grab the SSL renegotation future to early we could see test-failures.

Modifications:

Access the future at the correct time.

Result:

No more test-failures.
2015-05-06 06:25:25 +02:00
Trustin Lee
abaaa75ddc Reduce the memory requirement of heap dump compression
Motivation:

The current heap dump compression preset (9) requires way too much
memory (768 MiB at maximum for dictionary), resulting in OOME in many
cases.

Modifications:

- Use the default preset (6) which uses 8 MiB dictionary.
- Do not fail abruptly even when OOME has been raised.

Result:

More stable heap dump acquisition
2015-05-01 11:56:53 +09:00
Trustin Lee
518119d7c0 Fix the flaky SocketSslEchoTest
Motivation:

The SSL peer who did not initiate renegotiation sometimes does not get
the notification for renegotition due to an unknown reason.

Modification:

Until the exact cause is understood, relax the assertions of the flaky
tests.

Result:

Build stability
2015-05-01 11:48:02 +09:00
Ben Evans
6c025b2044 Add support for SCTP 'unordered' flag.
Motivation:

Some SCTP applications require the SCTP unordered flag.
This flag was not exposed by Netty so applications were unable
to use it.

Modifications:

- Add unordered flag to SctpMessage.
- {Nio,Oio}SctpChannel pass unordered flag to MessageInfo on write.
- SctpOutboundByteStreamHandler may optionally request unordered
  delivery for all outbound messages.
- Added test case to SctpEchoTest using unordered flag.

Result:

Fixes #3698. New constructors and methods added to SctpMessage and
SctpOutboundByteStreamHandler, but changes are backward compatible.
2015-04-30 08:46:07 +02:00
Norman Maurer
77e112a6d5 Correctly test for non-auto-read correctness in testsuite
Motiviation:

Our tests for non-auto-read did actually not test this correctly as auto-read was never disabled on the Bootstrap and ServerBootstrap.

Modifications:

- Correctly disable auto-read on Bootstrap and ServerBootstrap
- Fix tests to call ChannelHandlerContext.read() once a Channel becomes active.

Result:

Correctly test that non-auto-read works.
2015-04-20 09:10:55 +02:00
Norman Maurer
fce0989844 [maven-release-plugin] prepare for next development iteration 2015-03-03 02:06:47 -05:00
Norman Maurer
ca3b1bc4b7 [maven-release-plugin] prepare release netty-4.1.0.Beta4 2015-03-03 02:05:52 -05:00
Norman Maurer
afa9e71ed3 Allow to use WebSocketClientHandshaker and WebSocketServerHandshaker with HttpResponse / HttpRequest
Motivation:

To use WebSocketClientHandshaker / WebSocketServerHandshaker it's currently a requirement of having a HttpObjectAggregator in the ChannelPipeline. This is not a big deal when a user only wants to server WebSockets but is a limitation if the server serves WebSockets and normal HTTP traffic.

Modifications:

Allow to use WebSocketClientHandshaker and WebSocketServerHandshaker without HttpObjectAggregator in the ChannelPipeline.

Result:

More flexibility
2015-02-06 10:46:07 +01:00
Norman Maurer
f771a97592 Add support for Unix Domain Sockets when using native epoll transport
Motivation:

Using Unix Domain Sockets can be very useful when communication should take place on the same host and has less overhead then using loopback. We should support this with the native epoll transport.

Modifications:

- Add support for Unix Domain Sockets.
- Adjust testsuite to be able to reuse tests.

Result:

Unix Domain Sockets are now support when using native epoll transport.
2015-02-04 15:39:41 +01:00
Trustin Lee
1173699dff Enable client-side OpenSSL in SocketSslEchoTest 2014-12-30 23:56:31 +09:00
Frederic Bregier
2681112080 Fix big transfer and Write traffic shaping issues
Motivation:

Several issues were shown by various ticket (#2900 #2956).
Also use the improvement on writability user management from #3036.
And finally add a mixte handler, both for Global and Channels, with
the advantages of being uniquely created and using less memory and
less shaping.

Issue #2900

When a huge amount of data are written, the current behavior of the
TrafficShaping handler is to limit the delay to 15s, whatever the delay
the previous write has. This is wrong, and when a huge amount of writes
are done in a short time, the traffic is not correctly shapened.

Moreover, there is a high risk of OOM if one is not using in his/her own
handler for instance ChannelFuture.addListener() to handle the write
bufferisation in the TrafficShapingHandler.

This fix use the "user-defined writability flags" from #3036 to
allow the TrafficShapingHandlers to "user-defined" managed writability
directly, as for reading, thus using the default isWritable() and
channelWritabilityChanged().
This allows for instance HttpChunkedInput to be fully compatible.

The "bandwidth" compute on write is only on "acquired" write orders, not
on "real" write orders, which is wrong from statistic point of view.

Issue #2956

When using GlobalTrafficShaping, every write (and read) are
synchronized, thus leading to a drop of performance.
ChannelTrafficShaping is not touched by this issue since synchronized is
then correct (handler is per channel, so the synchronized).

Modifications:
The current write delay computation takes into account the previous
write delay and time to check is the 15s delay (maxTime) is really
exceeded or not (using last scheduled write time). The algorithm is
simplified and in the same time more accurate.

This proposal uses the #3036 improvement on user-defined writability
flags.

When the real write occurs, the statistics are update accordingly on a
new attribute (getRealWriteThroughput()).

To limit the synchronisations, all synchronized on
GlobalTrafficShapingHandler on submitWrite were removed. They are
replaced with a lock per channel (since synchronization is still needed
to prevent unordered write per channel), as in the sendAllValid method
for the very same reason.
Also all synchronized on TrafficCounter on read/writeTimeToWait() are
removed as they are unnecessary since already locked before by the
caller.
Still the creation and remove operations on lock per channel (PerChannel
object) are synchronized to prevent concurrency issue on this critical
part, but then limited.

Additionnal changes:
1) Use System.nanoTime() instead of System.currentTimeMillis() and
minimize calls
2) Remove / 10 ° 10 since no more sleep usage
3) Use nanoTime instead of currentTime such that time spend is computed,
not real time clock. Therefore the "now" relative time (nanoTime based)
is passed on all sub methods.
4) Take care of removal of the handler to force write all pending writes
and release read too
8) Review Javadoc to explicit:

- recommandations to take into account isWritable

- recommandations to provide reasonable message size according to
traffic shaping limit

- explicit "best effort" traffic shaping behavior when changing
configuration dynamically

Add a MixteGlobalChannelTrafficShapingHandler which allows to use only one
handler for mixing Global and Channel TSH. I enables to save more memory and
tries to optimize the traffic among various channels.

Result:
The traffic shaping is more stable, even with a huge number of writes in
short time by taking into consideration last scheduled write time.

The current implementation of TrafficShapingHandler using user-defined
writability flags and default isWritable() and
fireChannelWritabilityChanged works as expected.

The statistics are more valuable (asked write vs real write).

The Global TrafficShapingHandler should now have less "global"
synchronization, hoping to the minimum, but still per Channel as needed.

The GlobalChannel TrafficShapingHandler allows to have only one handler for all channels while still offering per channel in addition to global traffic shaping.

And finally maintain backward compatibility.
2014-12-29 15:47:06 +09:00
Norman Maurer
682df517c9 Add OpenSslClientContext to allow creating SslEngine for client side
Motivation:

We only support openssl for server side at the moment but it would be also useful for client side.

Modification:

* Upgrade to new netty-tcnative snapshot to support client side openssl support
* Add OpenSslClientContext which can be used to create SslEngine for client side usage
* Factor out common logic between OpenSslClientContext and OpenSslServerContent into new abstract base class called OpenSslContext
* Correctly detect handshake failures as soon as possible
* Guard against segfault caused by multiple calls to destroyPools(). This can happen if OpenSslContext throws an exception in the constructor and the finalize() method is called later during GC

Result:

openssl can be used for client and servers now.
2014-12-22 19:38:30 +01:00
Trustin Lee
563bf24424 Do not use SLF4J/Logback API directly
Motivation:

TrafficShapingHandlerTest uses Logback API directly, which is
discouraged.  Also, it overrides the global default log level, which
silences the DEBUG messages from other tests.

Modifications:

Remove the direct use of Logback API

Result:

The tests executed after TrafficShapingHandlerTest logs their DEBUG
messages correctly.
2014-12-17 10:15:52 +09:00
Trustin Lee
4f7632a853 Fix checkstyle 2014-12-16 20:34:18 +09:00
Trustin Lee
98a2bb62f5 Log detailed information about renegotiation and traffic
Motivation:

We need more information to understand why SocketSslEchoTest fails
sporadically in the CI machine.

Modifications:

- Refactor SocketSslEchoTest so that it is easier to retrieve the
  information about renegotiation and the current progress

Result:

We will get more information when the test fails.
2014-12-16 17:45:21 +09:00
Trustin Lee
c7b407e288 Compress the heap dump after tests are finished
Motivation:

Tests sometimes time out because it took too long to compress the
generated heap dump.

Modifications:

- Move the compression logic to a new method 'compressHeapDumps()'
- Call TestUtils.compressHeapDumps() at the end of the tests, so that
  the tests do not fail because of timeout

Result:

JUnit reports the real cause of the test failure instead of timeout
exception.
2014-12-16 16:16:17 +09:00
Trustin Lee
8d4a97e05b Generate non-test JAR for netty-testsuite
Motivation:

So far, we generated and deployed test JARs to Maven repositories. The
deployed JAR had the classifier 'test-jar'.  The test JAR is consumed by
transport-native-epoll as a test dependency.

The problem is, when netty-transport-native-epoll pulls the test JAR as
a dependency, that Maven resolves its transitive dependencies at
'compile' and 'runtime' scope only, which is incorrect.

I was bitten by this problem recently while trying to add a new
dependency to netty-testsuite.  Because I added a new dependency at the
'test' scope, the new dependency was not pulled transitively by
transport-native-epoll and caused an unexpected build failure.

- d6160208c3
- bf77bb4c3a

Modifications:

- Move all classes in netty-testsuite from src/test to src/main
- Update the 'compile' scope dependencies of netty-testsuite
- Override the test directory configuration properties of the surefire
  plugin
- Do not generate the test JAR anymore
- Update the dependency of netty-transport-native-epoll

Result:

It is less error-prone to add a new dependency to netty-testsuite.
2014-12-15 09:18:08 +09:00
Trustin Lee
5dd48cf3fe Fix build errors due to missing dependency 2014-12-14 21:29:52 +09:00
Trustin Lee
cbbf5eb96b Compress the heap dump generated by TestUtils.dump()
Motivation:

It takes too long to download the heap dump from the CI server.

Modifications:

Compress the heap dump as much as possible.

Result:

When heap dump is generated by certain test failure, the generated heap
dump file is about 3 times smaller than before, although the compression
time will increase the build time when the test fails.
2014-12-14 12:01:38 +09:00
Trustin Lee
dfd0cc5ea2 Add more assertions related with TLS renegotiation 2014-12-12 18:01:30 +09:00
Trustin Lee
e7cf1dcb98 Test TLS renegotiation with explicit cipher suite change
Motivation:

So far, our TLS renegotiation test did not test changing cipher suite
during renegotiation explicitly.

Modifications:

- Switch the cipher suite during renegotiation

Result:

We are now sure the cipher suite change works.
2014-12-12 17:49:16 +09:00
Trustin Lee
6ee069ffb4 Add log messages when dump starts
.. to make it easier to find the right dump file for a test when there
are multiple dump files.
2014-12-12 11:55:27 +09:00
Trustin Lee
e72b2235fb Make sure to notify handshake success even if SSLEngine is closed
Related:

e9685ea45a

Motivation:

SslHandler.unwrap() does not evaluate the handshake status of
SSLEngine.unwrap() when the status of SSLEngine.unwrap() is CLOSED.

It is not correct because the status does not reflect the state of the
handshake currently in progress, accoding to the API documentation of
SSLEngineResult.Status.

Also, sslCloseFuture can be notified earlier than handshake notification
because we call sslCloseFuture.trySuccess() before evaluating handshake
status.

Modifications:

- Notify sslCloseFuture after the unwrap loop is finished
- Add more assertions to SocketSslEchoTest

Result:

Potentially fix the regression caused by:
- e9685ea45a
2014-12-12 11:55:27 +09:00
Trustin Lee
3cd94822f5 Generate heap and thread dump when some tests fail
Motivation:

We have a few sporadic test failures which are only easily reproduceable
in our CI machine.  To get more information about the failure, we need
heap and full thread dump at the moment of failure.

Modifications:

- Add TestUtils.dump() method to dump heap and threads
- Modify SocketGatheringWriteTest and SocketSslEchoTest to call
  TestUtils.dump() on failure

Result:

We get more information about the test failure.
2014-12-12 10:35:05 +09:00
Trustin Lee
1dc1831abf Add SslHandler.renegotiate()
Related: #3125

Motivation:

We did not expose a way to initiate TLS renegotiation and to get
notified when the renegotiation is done.

Modifications:

- Add SslHandler.renegotiate() so that a user can initiate TLS
  renegotiation and get the future that's notified on completion
- Make SslHandler.handshakeFuture() return the future for the most
  recent handshake so that a user can get the future of the last
  renegotiation
- Add the test for renegotiation to SocketSslEchoTest

Result:

Both client-initiated and server-initiated renegotiations are now
supported properly.
2014-12-10 18:59:50 +09:00
Trustin Lee
4ce994dd4f Fix backward compatibility from the previous backport
Motivation:

The commit 50e06442c3 changed the type of
the constants in HttpHeaders.Names and HttpHeaders.Values, making 4.1
backward-incompatible with 4.0.

It also introduces newer utility classes such as HttpHeaderUtil, which
deprecates most static methods in HttpHeaders.  To ease the migration
between 4.1 and 5.0, we should deprecate all static methods that are
non-existent in 5.0, and provide proper counterpart.

Modification:

- Revert the changes in HttpHeaders.Names and Values
- Deprecate all static methods in HttpHeaders in favor of:
  - HttpHeaderUtil
  - the member methods of HttpHeaders
  - AsciiString
- Add integer and date access methods to HttpHeaders for easier future
  migration to 5.0
- Add HttpHeaderNames and HttpHeaderValues which provide standard HTTP
  constants in AsciiString
  - Deprecate HttpHeaders.Names and Values
  - Make HttpHeaderValues.WEBSOCKET lowercased because it's actually
    lowercased in all WebSocket versions but the oldest one
- Add RtspHeaderNames and RtspHeaderValues which provide standard RTSP
  constants in AsciiString
  - Deprecate RtspHeaders.*
- Do not use AsciiString.equalsIgnoreCase(CharSeq, CharSeq) if one of
  the parameters are AsciiString
- Avoid using AsciiString.toString() repetitively
  - Change the parameter type of some methods from String to
    CharSequence

Result:

Backward compatibility is recovered.  New classes and methods will make
the migration to 5.0 easier, once (Http|Rtsp)Header(Names|Values) are
ported to master.
2014-11-01 01:00:25 +09:00
Trustin Lee
f3a2c22738 Fix an infinite loop when writing a zero-length FileRegion
Related: #2964

Motivation:

Writing a zero-length FileRegion to an NIO channel will lead to an
infinite loop.

Modification:

- Do not write a zero-length FileRegion by protecting with proper 'if'.
- Update the testsuite

Result:

Another bug fixed
2014-10-17 16:06:51 +09:00
Trustin Lee
d63413754e Make TestUtils.getFreePort() check both TCP and UDP
Motivation:

We see occational failures in the datagram tests saying 'address already
in use' when we attempt to bind on a port returned by
TestUtils.getFreePort().

It turns out that TestUtils.getFreePort() only checks if TCP port is
available.

Modifications:

Also check if UDP port is available, so that the datagram tests do not
fail because of the 'address already in use' error during a bind
attempt.

Result:

Less chance of datagram test failures
2014-10-17 15:04:54 +09:00
Trustin Lee
e848066cab Name resolver API and DNS-based name resolver
Motivation:

So far, we relied on the domain name resolution mechanism provided by
JDK.  It served its purpose very well, but had the following
shortcomings:

- Domain name resolution is performed in a blocking manner.
  This becomes a problem when a user has to connect to thousands of
  different hosts. e.g. web crawlers
- It is impossible to employ an alternative cache/retry policy.
  e.g. lower/upper bound in TTL, round-robin
- It is impossible to employ an alternative name resolution mechanism.
  e.g. Zookeeper-based name resolver

Modification:

- Add the resolver API in the new module: netty-resolver
- Implement the DNS-based resolver: netty-resolver-dns
  .. which uses netty-codec-dns
- Make ChannelFactory reusable because it's now used by
  io.netty.bootstrap, io.netty.resolver.dns, and potentially by other
  modules in the future
  - Move ChannelFactory from io.netty.bootstrap to io.netty.channel
  - Deprecate the old ChannelFactory
  - Add ReflectiveChannelFactory

Result:

It is trivial to resolve a large number of domain names asynchronously.
2014-10-16 17:05:20 +09:00
Norman Maurer
6379075bb9 Fix buffer leak in DatagramUnicastTest caused by incorrect usage of CompositeByteBuf
Motivation:

Due incorrect usage of CompositeByteBuf a buffer leak was introduced.

Modifications:

Correctly handle tests with CompositeByteBuf.

Result:

No more buffer leaks
2014-09-10 11:42:26 +02:00
Norman Maurer
9428d51e2f Add support for sendmmsg(...) and so allow to write multiple DatagramPackets with one syscall. Related to [#2719]
Motivation:

On linux with glibc >= 2.14 it is possible to send multiple DatagramPackets with one syscall. This can be a huge performance win and so we should support it in our native transport.

Modification:

- Add support for sendmmsg by reuse IovArray
- Factor out ThreadLocal support of IovArray to IovArrayThreadLocal for better separation as we use IovArray also without ThreadLocal in NativeDatagramPacketArray now
- Introduce NativeDatagramPacketArray which is used for sendmmsg(...)
- Implement sendmmsg(...) via jni
- Expand DatagramUnicastTest to test also sendmmsg(...)

Result:

Netty now automatically use sendmmsg(...) if it is supported and we have more then 1 DatagramPacket in the ChannelOutboundBuffer and flush() is called.
2014-09-09 09:49:53 +02:00
Norman Maurer
646753c2eb Allow to write CompositeByteBuf directly via EpollDatagramChannel. Related to [#2719]
Motivation:

On linux it is possible to use the sendMsg(...) system call to write multiple buffers with one system call when using datagram/udp.

Modifications:

- Implement the needed changes and make use of sendMsg(...) if possible for max performance
- Add tests that test sending datagram packets with all kind of different ByteBuf implementations.

Result:

Performance improvement when using CompoisteByteBuf and EpollDatagramChannel.
2014-09-09 09:45:03 +02:00
Frédéric Brégier
bcbc23913e Better fix for TrafficShapingHandlerTest
Motivation:
The test procedure is unstable when testing quick time (factor less or equal to 1). Changing to default 10ms in this case will force time to be correct and time to be checked only when factor is >= 2.

Modifications:
When factor is <= 1, minimalWaitBetween is 10ms

Result:
Hoping this version is finally stable.
2014-08-16 18:10:10 +02:00
Frédéric Brégier
0ca84b3527 Better fix for TrafficShapingHandlerTest
Motivation:
It seems that in certain conditions, the write back from the server is so quick that the handler has no time to compute traffic shaping. So 10ms of wait before acknowledging is added in server side.

Modifications:
Add 10ms waiting before server ackonwledge the client.

Result:
The timing is now suppsed to be stable.
2014-08-16 10:54:54 +02:00
Trustin Lee
b5f61d0de5 [maven-release-plugin] prepare for next development iteration 2014-08-16 03:27:42 +09:00
Trustin Lee
76ac3b21a5 [maven-release-plugin] prepare release netty-4.1.0.Beta3 2014-08-16 03:27:37 +09:00
Trustin Lee
52232a7968 Fix checkstyle 2014-08-15 11:09:26 -07:00
Trustin Lee
88c273364d Fix for issue #2765 relative to unstable trafficshaping test procedure
Motivation:

The test procedure is unstable due to not enough precise timestamping
during the check.

Modifications:

Reducing the test cases and cibling "stable" test ("timestamp-able")
bring more stability to the tests.

Result:

Tests for TrafficShapingHandler seem more stable (whatever using JVM 6,
7 or 8).
2014-08-15 10:18:05 -07:00
Norman Maurer
dcfdad9e9e [#2769] Fix regression when writing different message types
Motivation:

Due a regression NioSocketChannel.doWrite(...) will throw a ClassCastException if you do something like:

channel.write(bytebuf);
channel.write(fileregion);
channel.flush();

Modifications:

Correctly handle writing of different message types by using the correct message count while loop over them.

Result:

No more ClassCastException
2014-08-15 11:56:12 +02:00
Trustin Lee
b3c1904cc9 [maven-release-plugin] prepare for next development iteration 2014-08-15 09:31:03 +09:00
Trustin Lee
e013b2400f [maven-release-plugin] prepare release netty-4.1.0.Beta2 2014-08-15 09:30:59 +09:00
Trustin Lee
061d5bc261 Fix data corruption in FileRegion transfer with epoll transport
Related issue: #2764

Motivation:

EpollSocketChannel.writeFileRegion() does not handle the case where the
position of a FileRegion is non-zero properly.

Modifications:

- Improve SocketFileRegionTest so that it tests the cases where the file
  transfer begins from the middle of the file
- Add another jlong parameter named 'base_off' so that we can take the
  position of a FileRegion into account

Result:

Improved test passes. Corruption is gone.
2014-08-13 16:58:28 -07:00
fbregier
bc1379d19d [#2721] Improve Traffic Shaping handler
Motivation:
Currently Traffic Shaping is using 1 timer only and could lead to
"partial" wrong bandwidth computation when "short" time occurs between
adding used bytes and when the TrafficCounter updates itself and finally
when the traffic is computed.
Indeed, the TrafficCounter is updated every x delay and it is at the
same time saved into "lastXxxxBytes" and set to 0. Therefore, when one
request the counter, it first updates the TrafficCounter with the added
used bytes. If this value is set just before the TrafficCounter is
updated, then the bandwidth computation will use the TrafficCounter with
a "0" value (this value being reset once the delay occurs). Therefore,
the traffic shaping computation is wrong in rare cases.

Secondly the traffic shapping should avoid if possible the "Timeout"
effect by not stopping reading or writing more than a maxTime, this
maxTime being less than the TimeOut limit.

Thirdly the traffic shapping in read had an issue since the readOp
was not set but should, turning in no read blocking from socket
point of view.

Modifications:
The TrafficCounter has 2 new methods that compute the time to wait
according to read or write) using in priority the currentXxxxBytes (as
before), but could used (if current is at 0) the lastXxxxxBytes, and
therefore having more chance to take into account the real traffic.

Moreover the Handler could change the default "max time to wait", which
is by default set to half of "standard" Time Out (30s:2 = 15s).

Finally we add the setAutoRead(boolean) accordingly to the situation,
as proposed in #2696 (this pull request is in error for unknown reason).

Result:
The Traffic Shaping is better take into account (no 0 value when it
shouldn't) and it tries to not block traffic more than Time Out event.

Moreover the read is really stopped from socket point of view.

This version is similar to #2388 and #2450.
This version is for V4.1, and includes the #2696 pull request
to ease the merge process.
It is compatible with master too.

Including also #2748

The test minimizes time check by reducing to 66ms steps (55s).
2014-08-13 01:40:32 +02:00
Norman Maurer
fb22d34925 [#2647] Respect IOV_MAX when call writev in native transport
Motivation:

epoll transport fails on gathering write of more then 1024 buffers. As linux supports max. 1024 iov entries when calling writev(...) the epoll transport throws an exception.

Thanks again to @blucas to provide me with a reproducer and so helped me to understand what the issue is.

Modifications:

Make sure we break down the writes if to many buffers are uses for gathering writes.

Result:

Gathering writes work with any number of buffers
2014-07-09 12:21:58 +02:00
Trustin Lee
e167b02d52 [maven-release-plugin] prepare for next development iteration 2014-07-04 17:26:02 +09:00
Trustin Lee
ba50cb829b [maven-release-plugin] prepare release netty-4.1.0.Beta1 2014-07-04 17:25:54 +09:00
Trustin Lee
787663a644 [maven-release-plugin] rollback the release of netty-4.1.0.Beta1 2014-07-04 17:11:14 +09:00
Trustin Lee
83eae705e1 [maven-release-plugin] prepare release netty-4.1.0.Beta1 2014-07-04 17:02:17 +09:00
Trustin Lee
41d44a8161 Remove 'get' prefix from all HTTP/SPDY messages
Motivation:

Persuit for the consistency in method naming

Modifications:

- Remove the 'get' prefix from all HTTP/SPDY message classes
- Fix some inspector warnings

Result:

Consistency
2014-06-24 18:03:33 +09:00
Trustin Lee
8b0a0f9a8f Introduce MessageAggregator and DecoderResultProvider
Motivation:

We have different message aggregator implementations for different
protocols, but they are very similar with each other.  They all stems
from HttpObjectAggregator.  If we provide an abstract class that provide
generic message aggregation functionality, we will remove their code
duplication.

Modifications:

- Add MessageAggregator which provides generic message aggregation
- Reimplement all existing aggregators using MessageAggregator
- Add DecoderResultProvider interface and extend it wherever possible so
  that MessageAggregator respects the state of the decoded message

Result:

Less code duplication
2014-06-05 16:51:14 +09:00
Trustin Lee
b6c0c0c95f Add an OpenSslEngine and the universal API for enabling SSL
Motivation:

Some users already use an SSLEngine implementation in finagle-native. It
wraps OpenSSL to get higher SSL performance.  However, to take advantage
of it, finagle-native must be compiled manually, and it means we cannot
pull it in as a dependency and thus we cannot test our SslHandler
against the OpenSSL-based SSLEngine.  For an instance, we had #2216.

Because the construction procedures of JDK SSLEngine and OpenSslEngine
are very different from each other, we also need to provide a universal
way to enable SSL in a Netty application.

Modifications:

- Pull netty-tcnative in as an optional dependency.
  http://netty.io/wiki/forked-tomcat-native.html
- Backport NativeLibraryLoader from 4.0
- Move OpenSSL-based SSLEngine implementation into our code base.
  - Copied from finagle-native; originally written by @jpinner et al.
  - Overall cleanup by @trustin.
- Run all SslHandler tests with both default SSLEngine and OpenSslEngine
- Add a unified API for creating an SSL context
  - SslContext allows you to create a new SSLEngine or a new SslHandler
    with your PKCS#8 key and X.509 certificate chain.
  - Add JdkSslContext and its subclasses
  - Add OpenSslServerContext
- Add ApplicationProtocolSelector to ensure the future support for NPN
  (NextProtoNego) and ALPN (Application Layer Protocol Negotiation) on
  the client-side.
- Add SimpleTrustManagerFactory to help a user write a
  TrustManagerFactory easily, which should be useful for those who need
  to write an alternative verification mechanism. For example, we can
  use it to implement an unsafe TrustManagerFactory that accepts
  self-signed certificates for testing purposes.
- Add InsecureTrustManagerFactory and FingerprintTrustManager for quick
  and dirty testing
- Add SelfSignedCertificate class which generates a self-signed X.509
  certificate very easily.
- Update all our examples to use SslContext.newClient/ServerContext()
- SslHandler now logs the chosen cipher suite when handshake is
  finished.

Result:

- Cleaner unified API for configuring an SSL client and an SSL server
  regardless of its internal implementation.
- When native libraries are available, OpenSSL-based SSLEngine
  implementation is selected automatically to take advantage of its
  performance benefit.
- Examples take advantage of this modification and thus are cleaner.
2014-05-18 02:54:06 +09:00
Norman Maurer
0aea9eaed5 Correctly write pending data after ssl handshake completes. Related to [#2437]
Motivation:
When writing data from a server before the ssl handshake completes may not be written at all to the remote peer
if nothing else is written after the handshake was done.

Modification:
Correctly try to write pending data after the handshake was complete

Result:
Correctly write out all pending data
2014-04-30 14:23:18 +02:00
Trustin Lee
db3709e652 Synchronized between 4.1 and master
Motivation:

4 and 5 were diverged long time ago and we recently reverted some of the
early commits in master.  We must make sure 4.1 and master are not very
different now.

Modification:

Fix found differences

Result:

4.1 and master got closer.
2014-04-25 00:38:02 +09:00
Norman Maurer
20ef4690e7 [#2375] [#2404] Fix bug in respecting ChannelConfig.setAutoRead(false) and also fix Channel.read() for OIO
Motivation:
At the moment ChanneConfig.setAutoRead(false) only is guaranteer to not have an extra channelRead(...) triggered when used from within the channelRead(...) or channelReadComplete(...) method. This is not the correct behaviour as it should also work from other methods that are triggered from within the EventLoop. For example a valid use case is to have it called from within a ChannelFutureListener, which currently not work as expected.

Beside this there is another bug which is kind of related. Currently Channel.read() will not work as expected for OIO as we will stop try to read even if nothing could be read there after one read operation on the socket (when the SO_TIMEOUT kicks in).

Modifications:
Implement the logic the right way for the NIO/OIO/SCTP and native transport, specific to the transport implementation. Also correctly handle Channel.read() for OIO transport by trigger a new read if SO_TIMEOUT was catched.

Result:
It is now also possible to use ChannelConfig.setAutoRead(false) from other methods that are called from within the EventLoop and have direct effect.

Conflicts:
	transport-sctp/src/main/java/io/netty/channel/sctp/nio/NioSctpChannel.java
	transport/src/main/java/io/netty/channel/socket/nio/NioDatagramChannel.java
	transport/src/main/java/io/netty/channel/socket/nio/NioSocketChannel.java
2014-04-17 07:50:51 +02:00
Jeff Pinner
7808b9926d SPDY: refactor frame codec implementation
Motivation:

Currently, the SPDY frame encoding and decoding code is based upon
the ChannelHandler abstraction. This requires maintaining multiple
versions for 3.x and 4.x (and possibly 5.x moving forward).

Modifications:

The SPDY frame encoding and decoding code is separated from the
ChannelHandler and SpdyFrame abstractions. Also test coverage is
improved.

Result:

SpdyFrameCodec now implements the ChannelHandler abstraction and is
responsible for creating and handling SpdyFrame objects.
2014-04-15 19:57:38 +02:00
Trustin Lee
c87db9b17a Reduce SO_TIMEOUT of testsuite so it finishes sooner
Motivation:

Testing the OIO transport takes longer time than other transports because it has to wait for SO_TIMEOUT if there is nothing to read.  In production, it's not a good idea to decrease this value (1000ms) because it will result in so many SocketTimeoutExceptions internally, but doing so in the testsuite should be fine.

Modifications:

Reduce the default SO_TIMEOUT of OIO channels to 10 ms.

Result:

Our testsuite finishes sooner.
2014-03-17 10:54:07 +09:00
Trustin Lee
bd9ad4b532 Perform cross-tests between NIO and epoll transport
Motivation:

The epoll testsuite tests the epoll transport only against itself (i.e. epoll x epoll only).  We should test the epoll transport also against the well-tested NIO transport, too.

Modifications:

- Make SocketTestPermutation extensible and reusable so that the epoll testsuite can take advantage of it.
- Rename EpollTestUtils to EpollSocketTestPermutation and make it extend SocketTestPermutation.
- Overall clean-up of SocketTestPermutation
  - Use Arrays.asList() for simplicity
  - Add combo() method to remove code duplication

Result:

The epoll transport is now also tested against the NIO transport.  SocketTestPermutation got cleaner.
2014-03-17 10:35:48 +09:00
Trustin Lee
2de65e25e9 Set timeout for SocketSslEchoTest
Motivation:

We are seeing EpollSocketSslEchoTest does not finish itself while its I/O thread is busy.  Jenkins should have terminated them when the global build timeout reaches, but Jenkins seems to fail to do so.  What's more interesting is that Jenkins will start another job before the EpollSocketSslEchoTest is terminated, and Linux starts to oom-kill them, impacting the uptime of the CI service.

Modifications:

- Set timeout for all test cases in SocketSslEchoTest so that all SSL tests terminate themselves when they take too long.
- Fix a bug where the epoll testsuite uses non-daemon threads which can potentially prevent JVM from quitting.
- (Cleanup) Separate boss group and worker group just like we do for NIO/OIO transport testsuite.

Result:

Potentially more stable CI machine.
2014-03-17 09:30:58 +09:00
Norman Maurer
7c4ad004f6 Ensure the HttpResponseEncoder is always placed before the HttpObjectAggregator. Part of [#2219] 2014-03-05 06:58:04 +01:00
Norman Maurer
78db65d0fb [#2254] Fix regression in handling autoRead and Channel.read()
This regression was introduced by e0b39159657c9eb711047bc32367537c4870d467
2014-02-21 09:19:36 +01:00
Jeff Pinner
b02531f0aa SPDY: remove SPDY/3 support 2014-02-20 15:00:32 -08:00
Trustin Lee
ac70dc4546 Update the version to 4.1.0.Alpha1-SNAPSHOT 2014-02-13 18:32:26 -08:00
Norman Maurer
7041a9238e Allow to cancel non-flushed writes 2014-02-11 19:42:49 +01:00
Norman Maurer
c93337b01a [#2212] Skip autobahntestsuite when using -DskipTests 2014-02-08 19:01:14 +01:00
Trustin Lee
ea345f6421 (Kind of) skip autobahn testsuite on -DskipTests
- autobahntestsuite-maven-plugin does not provide a way to skip all
  tests, so we run only one simple test when -DskipTests is set
2014-02-07 14:20:23 -08:00
Norman Maurer
d343a258e2 Allow to skip autobahntestsuite by specify property skipAutobahnTestsuite 2014-02-06 07:10:52 +01:00
Norman Maurer
293e34e13f Run autobahntestsuite as part of the build 2014-01-28 07:23:10 +01:00
Norman Maurer
d67184b488 [maven-release-plugin] prepare for next development iteration 2014-01-21 08:18:32 +01:00
Norman Maurer
287515210d [maven-release-plugin] prepare release netty-4.0.15.Final 2014-01-21 08:18:26 +01:00
Trustin Lee
2746688fd7 Fix occasional failure in SocketConnectionAttemptTest
- Use the real host name with a firewall so that the connection attempt always takes long time on all platforms.
- Fixes #2128
2014-01-19 14:30:01 +09:00
Norman Maurer
2c3f02c453 Allow to extend the provided tests by the testsuite
This allows custom transport implementations to just re-use the tests provided by netty and so make sure everything works like expected.
2014-01-10 15:53:25 +01:00
Trustin Lee
2ca7eae136 Do not deploy the testsuite 2014-01-10 15:14:17 +09:00
Trustin Lee
a3fb9fc823 Move the testsuite classes back to src/test 2014-01-10 15:07:02 +09:00
Trustin Lee
1b79781b4e tab -> spaces 2014-01-09 18:46:47 +09:00
Trustin Lee
de5abdeb48 Move the reusable classes in netty-testsuite to main/java/src and deploy them 2014-01-09 18:43:13 +09:00
Trustin Lee
bc21443ea9 Fix a regression in SslHandler where delegated tasks run in a different executor makes the session hang
- Fixes #2098
- Deprecate specifying an alternative Executor for delegated tasks for SslHandler
2014-01-09 18:08:05 +09:00
Trustin Lee
e83d2e0b4e [maven-release-plugin] prepare for next development iteration 2013-12-22 21:57:48 +09:00
Trustin Lee
cdb700c7a4 [maven-release-plugin] prepare release netty-4.0.14.Final 2013-12-22 21:57:40 +09:00
Trustin Lee
0b7aedb13b [maven-release-plugin] rollback the release of netty-4.0.14.Final 2013-12-22 21:53:24 +09:00
Trustin Lee
4bf6ec7171 [maven-release-plugin] prepare release netty-4.0.14.Final 2013-12-22 21:52:56 +09:00
Trustin Lee
9c1a49c58e [maven-release-plugin] rollback the release of netty-4.0.14.Final 2013-12-22 21:47:35 +09:00
Trustin Lee
008a049bf4 [maven-release-plugin] prepare for next development iteration 2013-12-22 21:43:55 +09:00
Trustin Lee
f6cb9088c6 [maven-release-plugin] prepare release netty-4.0.14.Final 2013-12-22 21:43:45 +09:00
Jeff Pinner
db59b38db3 SPDY: remove SPDY/2 support 2013-12-16 14:18:01 +09:00
Jeff Pinner
39ae2dd3f1 SPDY: add SPDY/3.1 support
- with Michael Schore <mschore@twitter.com>
2013-12-16 14:11:52 +09:00
Trustin Lee
dc9537e672 Add more diagnostics to SocketConnectionAttemptTest to fix the flaky test / Cleanup 2013-12-08 13:06:39 +09:00
Norman Maurer
b3d8c81557 Fix all leaks reported during tests
- One notable leak is from WebSocketFrameAggregator
- All other leaks are from tests
2013-12-07 00:44:56 +09:00
Norman Maurer
17f5865e38 [maven-release-plugin] prepare for next development iteration 2013-11-29 19:31:01 +01:00
Norman Maurer
ead617fdcc [maven-release-plugin] prepare release netty-4.0.14.Beta1 2013-11-29 19:30:55 +01:00
Norman Maurer
6cf2748dbb [maven-release-plugin] prepare for next development iteration 2013-11-28 15:04:51 +01:00
Norman Maurer
5fe7596f49 [maven-release-plugin] prepare release netty-4.0.13.Final 2013-11-28 15:04:46 +01:00
Norman Maurer
db78581bbb [maven-release-plugin] prepare for next development iteration 2013-11-07 18:11:45 +01:00
Norman Maurer
2386777af8 [maven-release-plugin] prepare release netty-4.0.12.Final 2013-11-07 18:11:38 +01:00
Trustin Lee
54db9ec725 Use StringUtil.simpleClassName(..) instead of Class.getSimpleName() where necessary
- Class.getSimpleName() doesn't render anonymous classes very well
- + some minor cleanup
2013-11-04 19:46:15 +09:00
Trustin Lee
2712ed9cbb Fix inspector warnings 2013-11-02 20:00:14 +09:00
Norman Maurer
ceab146b54 [maven-release-plugin] prepare for next development iteration 2013-10-21 07:43:42 +02:00
Norman Maurer
27a89d6032 [maven-release-plugin] prepare release netty-4.0.11.Final 2013-10-21 07:41:49 +02:00
Norman Maurer
a6b4eddbba Run tests with Pooled and Unpooled allocator 2013-10-16 11:51:16 +02:00
Norman Maurer
d7da19f745 [maven-release-plugin] prepare for next development iteration 2013-10-02 15:48:52 +02:00
Norman Maurer
d35768ae11 [maven-release-plugin] prepare release netty-4.0.10.Final 2013-10-02 15:48:45 +02:00
Norman Maurer
ae904a1f56 Introduce a new ChannelOption called DATAGRAM_CHANNEL_ACTIVE_ON_REGISTRATION. Related to [#1830]
This ChannelOption allows to tell the DatagramChannel implementation to be active as soon as they are registrated to their EventLoop. This can be used to make it possible to write to a not bound DatagramChannel.
The ChannelOption is marked as @deprecated as I'm looking for a better solution in master which breaks default behaviour with 4.0 branch.
2013-09-24 11:46:49 +02:00
Norman Maurer
357677d8fa [#1830] Add testcase for write to not bound DatagramChannel impls and revert change in OIO as it breaks things as the udnerlying socket lazy binds 2013-09-12 09:27:43 +02:00
Norman Maurer
a857994d82 Add test to show that writing a CompositeByteBuf when SslHandler is in use works. Related to [#1825] 2013-09-11 07:29:19 +02:00
Norman Maurer
ffab456aca Bump up version to reflect correct one 2013-09-09 11:20:12 +02:00
Norman Maurer
363531caf9 [maven-release-plugin] rollback the release of netty-4.0.9.Final 2013-09-06 09:18:34 +02:00
Norman Maurer
9d53573ee8 [maven-release-plugin] prepare for next development iteration 2013-09-06 09:17:15 +02:00
Norman Maurer
2e39b25cd4 [maven-release-plugin] prepare for next development iteration 2013-08-26 12:01:03 +02:00
Norman Maurer
b67659a866 [maven-release-plugin] prepare release netty-4.0.8.Final 2013-08-26 12:00:54 +02:00
Norman Maurer
1d3560e389 [maven-release-plugin] prepare for next development iteration 2013-08-08 13:53:28 +02:00
Norman Maurer
8e97e6c461 [maven-release-plugin] prepare release netty-4.0.7.Final 2013-08-08 13:53:19 +02:00
Norman Maurer
3f2000fa3a [maven-release-plugin] prepare for next development iteration 2013-08-01 10:59:55 +02:00
Norman Maurer
3f70d5caa4 [maven-release-plugin] prepare release netty-4.0.6.Final 2013-08-01 10:59:46 +02:00
Norman Maurer
e3410680de [maven-release-plugin] prepare for next development iteration 2013-07-31 20:08:14 +02:00
Norman Maurer
0e124583d6 [maven-release-plugin] prepare release netty-4.0.5.Final 2013-07-31 20:08:05 +02:00
Norman Maurer
0bc7d3f5d1 [maven-release-plugin] prepare for next development iteration 2013-07-23 10:04:23 +02:00
Norman Maurer
ca00182797 [maven-release-plugin] prepare release netty-4.0.4.Final 2013-07-23 10:04:14 +02:00
Trustin Lee
b130ee6a6c [maven-release-plugin] prepare for next development iteration 2013-07-18 11:17:42 +09:00
Trustin Lee
10d395e829 [maven-release-plugin] prepare release netty-4.0.3.Final 2013-07-18 11:17:31 +09:00
Norman Maurer
fc7c950b08 [maven-release-plugin] prepare for next development iteration 2013-07-17 15:58:36 +02:00
Norman Maurer
bbbf72359e [maven-release-plugin] prepare release netty-4.0.2.Final 2013-07-17 15:58:28 +02:00
Trustin Lee
b9bae2b5e4 Disable the timeout of some tests in SocketEchoTest to find out why it is hanging in CI 2013-07-17 16:02:50 +09:00
Trustin Lee
57eb531eb8 [maven-release-plugin] prepare for next development iteration 2013-07-16 17:16:10 +09:00
Trustin Lee
76cefcc421 [maven-release-plugin] prepare release netty-4.0.1.Final 2013-07-16 17:15:54 +09:00
Norman Maurer
5297eba280 [maven-release-plugin] prepare for next development iteration 2013-07-15 15:48:15 +02:00
Norman Maurer
c5d8af446a [maven-release-plugin] prepare release netty-4.0.0.Final 2013-07-15 15:48:05 +02:00
Trustin Lee
246a3ecdcb [maven-release-plugin] prepare for next development iteration 2013-07-15 20:58:33 +09:00
Trustin Lee
e8fd209115 [maven-release-plugin] prepare release netty-4.0.0.Final 2013-07-15 20:58:21 +09:00
Norman Maurer
b57d9f307f Allow per-write promises and disallow promises on flush()
- write() now accepts a ChannelPromise and returns ChannelFuture as most
  users expected.  It makes the user's life much easier because it is
  now much easier to get notified when a specific message has been
  written.
- flush() does not create a ChannelPromise nor returns ChannelFuture.
  It is now similar to what read() looks like.
2013-07-11 00:49:48 +09:00
Trustin Lee
26e9d70457 Remove channelReadSuspended / Rename messageReceived(Last) to channelRead(Complete)
- Remove channelReadSuspended because it's actually same with messageReceivedLast
- Rename messageReceived to channelRead
- Rename messageReceivedLast to channelReadComplete

We renamed messageReceivedLast to channelReadComplete because it
reflects what it really is for.  Also, we renamed messageReceived to
channelRead for consistency in method names.
2013-07-09 23:58:51 +09:00
Trustin Lee
cbd8817905 Remove MessageList from public API and change ChannelInbound/OutboundHandler accordingly
I must admit MesageList was pain in the ass.  Instead of forcing a
handler always loop over the list of messages, this commit splits
messageReceived(ctx, list) into two event handlers:

- messageReceived(ctx, msg)
- mmessageReceivedLast(ctx)

When Netty reads one or more messages, messageReceived(ctx, msg) event
is triggered for each message.  Once the current read operation is
finished, messageReceivedLast() is triggered to tell the handler that
the last messageReceived() was the last message in the current batch.

Similarly, for outbound, write(ctx, list) has been split into two:

- write(ctx, msg)
- flush(ctx, promise)

Instead of writing a list of message with a promise, a user is now
supposed to call write(msg) multiple times and then call flush() to
actually flush the buffered messages.

Please note that write() doesn't have a promise with it.  You must call
flush() to get notified on completion. (or you can use writeAndFlush())

Other changes:

- Because MessageList is completely hidden, codec framework uses
  List<Object> instead of MessageList as an output parameter.
2013-07-09 23:51:48 +09:00
Norman Maurer
ec5e793a2f [maven-release-plugin] prepare for next development iteration 2013-07-02 11:41:18 +02:00
Norman Maurer
ca73eaef0d [maven-release-plugin] prepare release netty-4.0.0.CR9 2013-07-02 11:41:09 +02:00
Norman Maurer
830c559405 [maven-release-plugin] rollback the release of netty-4.0.0.CR9 2013-07-02 11:34:29 +02:00
Norman Maurer
66a16b133c [maven-release-plugin] prepare release netty-4.0.0.CR9 2013-07-02 10:45:12 +02:00
Trustin Lee
7e3a01cc51 [maven-release-plugin] prepare for next development iteration 2013-07-02 10:26:48 +09:00
Trustin Lee
149db34c19 [maven-release-plugin] prepare release netty-4.0.0.CR8 2013-07-02 10:26:32 +09:00
Trustin Lee
4b11aff08f Less confusing log messages for system properties
- Fixes #1502
2013-07-02 09:23:29 +09:00
Trustin Lee
613547b0b9 [maven-release-plugin] prepare for next development iteration 2013-06-28 22:15:33 +09:00
Trustin Lee
a6abd2feb2 [maven-release-plugin] prepare release netty-4.0.0.CR7 2013-06-28 22:15:20 +09:00
Norman Maurer
1c57e3830c [#1474] Make sure testsuite also pass when no network/internet connection is present 2013-06-26 21:13:32 +02:00
Trustin Lee
a6795d7780 [maven-release-plugin] prepare for next development iteration 2013-06-25 11:07:15 +09:00
Trustin Lee
2221446425 [maven-release-plugin] prepare release netty-4.0.0.CR6 2013-06-25 11:07:15 +09:00
Norman Maurer
e06fcdbc6a [#1475] Correctly handle writes with empty MessageList in SslHandler 2013-06-25 11:07:14 +09:00