Commit Graph

6324 Commits

Author SHA1 Message Date
Norman Maurer
0027d830d4 Only run OpenSslEngineTests if OpenSsl is installed. Related to [#3732] 2015-05-06 10:41:44 +02:00
Norman Maurer
9dc236054e Add support for mutual auth when using OpenSslEngine.
Motivation:

Currently mutual auth is not supported when using OpenSslEngine.

Modification:

- Add support to OpenSslClientContext
- Correctly throw SSLHandshakeException when an error during handshake is detected

Result:

Mutual auth can be used with OpenSslEngine
2015-05-06 09:07:18 +02:00
Norman Maurer
59209f16bf [#3709] Ensure all data is read from socket when EPOLLRDUP is received
Motivation:

When EPOLLRDHUP is received we need to try to read at least one time to ensure
that we read all pending data from the socket. Otherwise we may loose data.

Modifications:

- Ensure we read all data from socket
- Ensure file descriptor is closed on doClose() even if doDeregister() throws an Exception.
- Only handle either EPOLLRDHUP or EPOLLIN as only one is needed to detect connection reset.

Result:

No more data loss on connection reset.
2015-05-06 08:44:56 +02:00
yz_liu
cf8ec39d51 fix a typo in RecyclableArrayList 2015-05-06 08:42:18 +02:00
Norman Maurer
588f5c2476 Proper handling of epoll_ctl errors
Motivation:

When using epoll_ctl we should respect the return value and do the right thing depending on it.

Modifications:

Adjust java and native code to respect epoll_ctl return values.

Result:

Correct and cleaner code.
2015-05-06 08:20:12 +02:00
Scott Mitchell
c9a0bf4859 DefaultChannelPipeline needs to release objects
Motiviation:
If user events or excpetions reach the tail end of the pipeline they are not released. This could result in buffer leaks.

Motivation:
- Use the ReferenceCountUtil.release to release objects for the userEventTriggered and exceptionCaught methods on DefaultChannelPipeline

Result:
2 less areas where buffer leaks can occur.
2015-05-06 06:46:35 +02:00
Norman Maurer
93f08aa28b Fix race-condition in SocketSslEchoTest
Motivation:

Because we tried to grab the SSL renegotation future to early we could see test-failures.

Modifications:

Access the future at the correct time.

Result:

No more test-failures.
2015-05-06 06:25:15 +02:00
Norman Maurer
9f6e06dc43 [#3675] Fix livelock issue in MpscLinkedQueue
Motivation:

All read operations should be safe to execute from multiple threads which was not the case and so could produce a livelock.

Modifications:

Modify methods so these are safe to be called from multiple threads.

Result:

No more livelock.
2015-05-06 06:21:02 +02:00
Trustin Lee
7b457a48f7 Add more profiles for newer JDK versions
Modifications:

- Add jetty.npn.version.latest and jetty.alpn.version.latest7/8
- Add npn-alpn-7 profile
- Use the *.latest7/8 version properties in alpn-8 and npn-alpn-7
- Add more profiles for newer JDK versions
- Reorder profiles
2015-05-04 14:33:56 +09:00
Trustin Lee
a8af0debcb Fix the flaky SocketSslEchoTest
Motivation:

The SSL peer who did not initiate renegotiation sometimes does not get
the notification for renegotition due to an unknown reason.

Modification:

Until the exact cause is understood, relax the assertions of the flaky
tests.

Result:

Build stability
2015-05-04 11:52:05 +09:00
Trustin Lee
3f5ce7c1d2 Reduce the memory requirement of heap dump compression
Motivation:

The current heap dump compression preset (9) requires way too much
memory (768 MiB at maximum for dictionary), resulting in OOME in many
cases.

Modifications:

- Use the default preset (6) which uses 8 MiB dictionary.
- Do not fail abruptly even when OOME has been raised.

Result:

More stable heap dump acquisition
2015-05-01 11:57:28 +09:00
Norman Maurer
b50f2d5294 [#3218] Add ChannelPool / ChannelPoolMap abstraction and implementations
Motivation:

Many projects need some kind a Channel/Connection pool implementation. While the protocols are different many things can be shared, so we should provide a generic API and implementation.

Modifications:

Add ChannelPool / ChannelPoolMap API and implementations.

Result:

Reusable / Generic pool implementation that users can use.
2015-04-30 12:04:19 +02:00
Ben Evans
44d64ce039 Add support for SCTP 'unordered' flag.
Motivation:

Some SCTP applications require the SCTP unordered flag.
This flag was not exposed by Netty so applications were unable
to use it.

Modifications:

- Add unordered flag to SctpMessage.
- {Nio,Oio}SctpChannel pass unordered flag to MessageInfo on write.
- SctpOutboundByteStreamHandler may optionally request unordered
  delivery for all outbound messages.
- Added test case to SctpEchoTest using unordered flag.

Result:

Fixes #3698. New constructors and methods added to SctpMessage and
SctpOutboundByteStreamHandler, but changes are backward compatible.
2015-04-30 08:06:53 +02:00
Norman Maurer
2c2b89d30f Add support for splice(...)
Motivation:

Linux supports splice(...) to transfer data from one filedescriptor to another without
pass data through the user-space. This allows to write high-performant proxy code or to stream
stuff from the socket directly the the filesystem.

Modification:

Add AbstractEpollStreamChannel.spliceTo(...) method to support splice(...) system call

Result:

Splice is now supported when using the native linux transport.

Conflicts:

	transport-native-epoll/src/main/java/io/netty/channel/epoll/AbstractEpollStreamChannel.java
2015-04-30 07:10:06 +02:00
Norman Maurer
55fbf007f0 Remove memory copy when extract frame in LengthFieldBasedFrameDecoder
Motivation:

We are currently doing a memory cop to extract the frame in LengthFieldBasedFrameDecoder which can be eliminated.

Modifications:

Use buffer.slice(...).retain() to eliminate the memory copy.

Result:

Better performance.
2015-04-29 08:42:55 +02:00
Norman Maurer
c72c71abbb [#3662] Fail the connect future on close
Motivation:

Because of a bug we missed to fail the connect future when doClose() is called. This can lead to a future which is never notified and so may lead to deadlocks in user-programs.

Modifications:

Correctly fail the connect future when doClose() is called and the connection was not established yet.

Result:

Connect future is always notified.
2015-04-27 20:04:34 +02:00
Norman Maurer
7cb9b9903d [#3680] Enabled SecurityManager results in ClassNotFoundError during io.netty.util.NetUtil initialization
Motivation:

When a SecurityManager is in place that preven reading the somaxconn file trying to bootstrap a channel later will result in a ClassNotFoundError.

Modifications:

- Reading the file in a privileged block.

Result:

No more ClassNotFoundError when a SecurityManager is in place.
2015-04-27 20:01:51 +02:00
Norman Maurer
a958e9b5d5 [#3652] Improve performance of StringUtil.simpleClassName()
Motivation:

static Package getPackage(Class<?> c) uses synchronized block internally.
Thanks to @jingene for the hint and initial report of the issue.

Modifications:

-Use simple lastIndexOf(...) and substring for a faster implementation

Result:

No more lock condition.
2015-04-22 09:11:11 +02:00
Alwayswithme
9a365b101e fix the discardedBytes counting on LineBasedFrameDecoder
Motivation:

The LineBasedFrameDecoder discardedBytes counting different compare to
DelimiterBasedFrameDecoder.

Modifications:

Add plus sign

Result:

DiscardedBytes counting correctly
2015-04-21 09:25:57 +02:00
Eric Anderson
97eba4eb66 Fix SslContextBuilder swapping client and server
The 'forClient' boolean was swapped to 'forServer' in code review of #3531.
Not all locations were updated.
2015-04-20 17:24:43 -07:00
Norman Maurer
f242bc5a1a [#3623] CompositeByteBuf.iterator() should return optimized Iterable
Motivation:

CompositeByteBuf.iterator() currently creates a new ArrayList and fill it with the ByteBufs, which is more expensive then it needs to be.

Modifications:

- Use special Iterator implementation

Result:

Less overhead when calling iterator()
2015-04-20 10:45:28 +02:00
Norman Maurer
5cd541c537 Fix handling of non-auto read for ByteToMessageDecoder and SslHandler
Motivation:

Our automatically handling of non-auto-read failed because it not detected the need of calling read again by itself if nothing was decoded. Beside this handling of non-auto-read never worked for SslHandler as it always triggered a read even if it decoded a message and auto-read was false.

This fixes [#3529] and [#3587].

Modifications:

- Implement handling of calling read when nothing was decoded (with non-auto-read) to ByteToMessageDecoder again
- Correctly respect non-auto-read by SslHandler

Result:

No more stales and correctly respecting of non-auto-read by SslHandler.
2015-04-20 09:02:47 +02:00
Norman Maurer
3b1bf6348a Correctly test for non-auto-read correctness in testsuite
Motiviation:

Our tests for non-auto-read did actually not test this correctly as auto-read was never disabled on the Bootstrap and ServerBootstrap.

Modifications:

- Correctly disable auto-read on Bootstrap and ServerBootstrap
- Fix tests to call ChannelHandlerContext.read() once a Channel becomes active.

Result:

Correctly test that non-auto-read works.
2015-04-20 09:02:40 +02:00
Norman Maurer
b35f2b4cb4 Revert "Ensure ctx.fireChannelReadComplete() is only called when something was produced"
This reverts commit 375b9e1307.
2015-04-20 09:02:34 +02:00
Norman Maurer
76dcda8259 Revert "Ensure channelReadComplete() is called only when necessary"
This reverts commit 8b2fb2b985.
2015-04-20 09:02:23 +02:00
Norman Maurer
f35158f402 Revert "Do not suppress channelReadComplete() when a handler was just added"
This reverts commit a41b46ff43.
2015-04-20 09:02:14 +02:00
Norman Maurer
2e1325aa9d Revert "Add another test case for channelReadComplete() suppression"
This reverts commit 5fca3de098.
2015-04-20 09:02:01 +02:00
Norman Maurer
d376bf6a58 Reduce object allocation during wrap/unwrap while handshake is in progress
Motivation:

Unnecessary object allocation is currently done during wrap/unwrap while a handshake is still in progress.

Modifications:

Use static instances when possible.

Result:

Less object creations.
2015-04-20 06:47:18 +02:00
Norman Maurer
19179f9fee Allow to get version of available OpenSSL library
Motivation:

Sometimes it's useful to get informations about the available OpenSSL library that is used for the OpenSslEngine.

Modifications:

Add two new methods which allows to get the available OpenSSL version as either
an int or an String.

Result:

Easy to access details about OpenSSL version.
2015-04-18 20:56:15 +02:00
Eric Anderson
6668b3c40d The "null" ClassLoader is the bootstrap ClassLoader
Motivation:
Class.forName() documents that null will use bootstrap loader:
http://docs.oracle.com/javase/8/docs/api/java/lang/Class.html#forName-java.lang.String-boolean-java.lang.ClassLoader-

But the link between "null" and bootstrap loader is even more explicit
in ClassLoader's documentation:
http://docs.oracle.com/javase/8/docs/api/java/lang/ClassLoader.html#getParent--

The current code is trying to use the bootstrap loader but seems to have
not been aware of the meaning of null.

Modifications:
Use "null" as the class loader when we want to load classes in the
bootstrap loader.

Result:
More reliable ALPN/NPN loading and simpler code.
2015-04-16 17:25:18 +02:00
Derek Troy-West
b7aef7bec8 Change AggregatedFullHttpMessage to contain a content ByteBuf
Motivation:

Other implementations of FullHttpMessage allow .toString to be called after the Message has been released
This brings AggregatedFullHttpMessage into line with those impls.

Modifications:

- Changed AggregatedFullHttpMessage to no longer be a sub-class of DefaultByteBufHolder
- Changes AggregatedFullHttpMessage to implement ByteBufHolder
- Hold the content buffer internally to AggregatedFullHttpMessage
- Implement the required content() and release() methods that were missing
- Do not check refcnt when accessing content() (similar to DefaultFullHttpMessage)

Result:

A released AggregatedFullHttpMessage can have .toString called without throwing an exception
2015-04-16 14:30:57 +02:00
Norman Maurer
9e5dd21d23 Fix regression introduced by cherry-pick bd224286f5 2015-04-14 17:59:33 +02:00
Norman Maurer
91a508664f Document the contract of Attribute.getAndSet(...) and set(...)
Motivation:

Attribute.getAndRemove() will return the value but also remove the AttributeKey itself from the AttributeMap. This may not
what you want as you may want to keep an instance of it and just set it later again. Document the contract so the user know what to expect.

Modifications:

- Make it clear when to use AttributeKey.getAndRemove() / AttributeKey.remove() and when AttributeKey.getAndSet(null) / AttributeKey.set(null).

Result:

Less suprising behaviour.
2015-04-14 09:53:04 +02:00
Norman Maurer
e907ae57a2 Add support for EC Keys when using SslServerContext
Motivation:

Sometimes it's useful to use EC keys and not DSA or RSA. We should support it.

Modifications:

Support EC keys and share the code between JDK and Openssl impl.

Result:

It's possible to use EC keys now.
2015-04-14 08:44:38 +02:00
Eric Anderson
fdc396674e [#3531] Create SslContext.Builder
Motivation:

SslContext factory methods have gotten out of control; it's past time to
swap to a builder.

Modifications:

New Builder class. The existing factory methods must be left as-is for
backward compatibility.

Result:

Fixes #3531
2015-04-14 07:27:55 +02:00
Norman Maurer
3307686bda [#3539] Correctly handle EPOLLRDHUP
Motivation:

As we missed to correctly handle EPOLLRDHUP we produce an IOException which is unnessary. This leads
to have exceptionCaught(...) methods called.

Modifications:

When EPOLLRDHUP was received just close the socket and fail all pending writes.

Result:

Correctly handle of EPOLLRDHUP and so not miss-leading exceptions.
2015-04-14 06:56:02 +02:00
garywu
8843d2b1a1 [#2925] Bug fix for NormalMemoryRegionCache overbooked for PoolThreadCache
Motivation:

When create NormalMemoryRegionCache for PoolThreadCache, we overbooked
cache array size. This means unnecessary overhead for thread local cache
as we will create multi cache enties for each element in cache array.

Modifications:

change:
int arraySize = Math.max(1, max / area.pageSize);
to:
int arraySize = Math.max(1, log2(max / area.pageSize) + 1);

Result:

Now arraySize won't introduce unnecessary overhead.

 Changes to be committed:
	modified:   buffer/src/main/java/io/netty/buffer/PoolThreadCache.java
2015-04-13 09:00:51 +02:00
Norman Maurer
bfb6189f77 Let CompositeByteBuf implement Iterable
Motivation:

CompositeByteBuf has an iterator() method but fails to implement Iterable

Modifications:

Let CompositeByteBuf implement Iterable<ByteBuf>

Result:

Easier usage
2015-04-12 13:38:39 +02:00
Norman Maurer
914d9ec095 [#3373] Rename class to match naming scheme
Motivation:

The ReplayingDecoderBuffer does not match the naming scheme we use for ByteBuf types.

Modifications:

Rename to ReplayingDecoderByteBuf to match naming scheme

Result:

Consistent naming
2015-04-12 13:32:24 +02:00
Norman Maurer
e4900cbdd3 Revert "Dereference when calling PooledByteBuf.deallocate()"
This reverts commit ddd19f4f21.
2015-04-11 06:44:10 +02:00
Norman Maurer
ddd19f4f21 Dereference when calling PooledByteBuf.deallocate()
Motivation:

We missed to dereference the chunk and tmpNioBuf when calling deallocate(). This means the GC can not collect these as we still hold a reference while have the PooledByteBuf in the recycler stack.

Modifications:

Dereference chunk and tmpNioBuf.

Result:

GC can collect things.
2015-04-10 21:46:43 +02:00
Norman Maurer
bfee39b318 [#3592] Flush when writing HttpChunkedInput
Motivation:

We missed to flush the channel when using HttpChunkedInput (this is done when using SSL). This will result in a stale.

Modifications:

Replace ctx.write(...) with ctx.writeAndFlush(...)

Result:

Correctly working example.
2015-04-10 21:19:02 +02:00
Norman Maurer
30f60ac68a Change PoolThreadCache to use LIFO for better cache performance
Motiviation:

At the moment we use FIFO for the PoolThreadCache which is sub-optimal as this may reduce the changes to have the cached memory actual still in the cpu-cache.

Modification:

- Change to use LIFO as this increase the chance to be able to serve buffers from the cpu-cache

Results:

Faster allocation out of the ThreadLocal cache.

Before the commit:
[xxx wrk]$ ./wrk -H 'Connection: keep-alive' -d 120 -c 256 -t 16 -s scripts/pipeline-many.lua  http://xxx:8080/plaintext
Running 2m test @ http://xxx:8080/plaintext
  16 threads and 256 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    14.69ms   10.06ms 131.43ms   80.10%
    Req/Sec   283.89k    40.37k  433.69k    66.81%
  533859742 requests in 2.00m, 72.09GB read
Requests/sec: 4449510.51
Transfer/sec:    615.29MB

After the commit:
[xxx wrk]$ ./wrk -H 'Connection: keep-alive' -d 120 -c 256 -t 16 -s scripts/pipeline-many.lua  http://xxx:8080/plaintext
Running 2m test @ http://xxx:8080/plaintext
  16 threads and 256 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    16.38ms   26.32ms 734.06ms   97.38%
    Req/Sec   283.86k    39.31k  361.69k    83.38%
  540836511 requests in 2.00m, 73.04GB read
Requests/sec: 4508150.18
Transfer/sec:    623.40MB
2015-04-10 20:57:31 +02:00
Norman Maurer
97a3308c7e Add support for ALPN when using openssl + NPN client mode and support for CipherSuiteFilter
Motivation:

To support HTTP2 we need APLN support. This was not provided before when using OpenSslEngine, so SSLEngine (JDK one) was the only bet.
Beside this CipherSuiteFilter was not supported

Modifications:

- Upgrade netty-tcnative and make use of new features to support ALPN and NPN in server and client mode.
- Guard against segfaults after the ssl pointer is freed
- support correctly different failure behaviours
- add support for CipherSuiteFilter

Result:

Be able to use OpenSslEngine for ALPN / NPN for server and client.
2015-04-10 18:59:58 +02:00
JongYoon Lim
ecf1b558d9 Motivation: If there are no readable bytes, it's unnecessary to go through javaChannel().write().
Modification:
If the readableBytes() is zero, then just return.

Result:
The logic can be easy to understand and a little faster than before.
2015-04-08 09:36:32 +02:00
Frederic Bregier
2ab2e2d6b7 Fix incorrect null value check in TrafficCounter
In TrafficCounter, a recent change makes the contract of the API (the
constructor) wrong and lead to issue with GlobalChannelTrafficCounter
where executor must be null.

Motivation:
TrafficCounter executor argument in constructor might be null, as
explained in the API, for some particular cases where no executor are
needed (relevant tasks being taken by the caller as in
GlobalChannelTrafficCounter).
A null pointer exception is raised while it should not since it is
legal.

Modifications:
Remove the 2 null checking for this particular attribute.
Note that when null, the attribute is not reached nor used (a null
checking condition later on is applied).

Result:
No more null exception raized while it should not.

This shall be made also to 4.0, 4.1 (present) and master. 3.10 is not
concerned.
2015-04-06 19:07:13 +02:00
Norman Maurer
f2fedbcdef [maven-release-plugin] prepare for next development iteration 2015-03-31 22:06:30 -04:00
Norman Maurer
054e7c5d17 [maven-release-plugin] prepare release netty-4.0.27.Final 2015-03-31 22:05:43 -04:00
Trustin Lee
4d8f824358 Fix unbounded expansion of cumulative buffer in SslHandler
Related: #3567

Motivation:

SslHandler.channelReadComplete() forgets to call
super.channelReadComplete(), which discards read bytes from the
cumulative buffer.  As a result, the cumulative buffer can expand its
capacity unboundedly.

Modifications:

Call super.channelReadComplete() instead of calling
ctx.fireChannelReadComplete()

Result:

Fixes #3567
2015-04-02 14:57:02 +09:00
Trustin Lee
28caa873b3 Don't trigger IOException at ChunkedStream.isEndOfInput()
Related: #3368

Motivation:

ChunkedWriteHandler checks if the return value of
ChunkedInput.isEndOfInput() after calling ChunkedInput.close().

This makes ChunkedStream.isEndOfInput() trigger an IOException, which is
originally triggered by PushBackInputStream.read().

By contract, ChunkedInput.isEndOfInput() should not raise an IOException
even when the underlying stream is closed.

Modifications:

Add a boolean flag that keeps track of whether the underlying stream has
been closed or not, so that ChunkedStream.isEndOfInput() does not
propagate the IOException from PushBackInputStream.

Result:

Fixes #3368
2015-03-31 11:31:54 +09:00