Commit Graph

6514 Commits

Author SHA1 Message Date
Trustin Lee
3f5ce7c1d2 Reduce the memory requirement of heap dump compression
Motivation:

The current heap dump compression preset (9) requires way too much
memory (768 MiB at maximum for dictionary), resulting in OOME in many
cases.

Modifications:

- Use the default preset (6) which uses 8 MiB dictionary.
- Do not fail abruptly even when OOME has been raised.

Result:

More stable heap dump acquisition
2015-05-01 11:57:28 +09:00
Norman Maurer
b50f2d5294 [#3218] Add ChannelPool / ChannelPoolMap abstraction and implementations
Motivation:

Many projects need some kind a Channel/Connection pool implementation. While the protocols are different many things can be shared, so we should provide a generic API and implementation.

Modifications:

Add ChannelPool / ChannelPoolMap API and implementations.

Result:

Reusable / Generic pool implementation that users can use.
2015-04-30 12:04:19 +02:00
Ben Evans
44d64ce039 Add support for SCTP 'unordered' flag.
Motivation:

Some SCTP applications require the SCTP unordered flag.
This flag was not exposed by Netty so applications were unable
to use it.

Modifications:

- Add unordered flag to SctpMessage.
- {Nio,Oio}SctpChannel pass unordered flag to MessageInfo on write.
- SctpOutboundByteStreamHandler may optionally request unordered
  delivery for all outbound messages.
- Added test case to SctpEchoTest using unordered flag.

Result:

Fixes #3698. New constructors and methods added to SctpMessage and
SctpOutboundByteStreamHandler, but changes are backward compatible.
2015-04-30 08:06:53 +02:00
Norman Maurer
2c2b89d30f Add support for splice(...)
Motivation:

Linux supports splice(...) to transfer data from one filedescriptor to another without
pass data through the user-space. This allows to write high-performant proxy code or to stream
stuff from the socket directly the the filesystem.

Modification:

Add AbstractEpollStreamChannel.spliceTo(...) method to support splice(...) system call

Result:

Splice is now supported when using the native linux transport.

Conflicts:

	transport-native-epoll/src/main/java/io/netty/channel/epoll/AbstractEpollStreamChannel.java
2015-04-30 07:10:06 +02:00
Norman Maurer
55fbf007f0 Remove memory copy when extract frame in LengthFieldBasedFrameDecoder
Motivation:

We are currently doing a memory cop to extract the frame in LengthFieldBasedFrameDecoder which can be eliminated.

Modifications:

Use buffer.slice(...).retain() to eliminate the memory copy.

Result:

Better performance.
2015-04-29 08:42:55 +02:00
Norman Maurer
c72c71abbb [#3662] Fail the connect future on close
Motivation:

Because of a bug we missed to fail the connect future when doClose() is called. This can lead to a future which is never notified and so may lead to deadlocks in user-programs.

Modifications:

Correctly fail the connect future when doClose() is called and the connection was not established yet.

Result:

Connect future is always notified.
2015-04-27 20:04:34 +02:00
Norman Maurer
7cb9b9903d [#3680] Enabled SecurityManager results in ClassNotFoundError during io.netty.util.NetUtil initialization
Motivation:

When a SecurityManager is in place that preven reading the somaxconn file trying to bootstrap a channel later will result in a ClassNotFoundError.

Modifications:

- Reading the file in a privileged block.

Result:

No more ClassNotFoundError when a SecurityManager is in place.
2015-04-27 20:01:51 +02:00
Norman Maurer
a958e9b5d5 [#3652] Improve performance of StringUtil.simpleClassName()
Motivation:

static Package getPackage(Class<?> c) uses synchronized block internally.
Thanks to @jingene for the hint and initial report of the issue.

Modifications:

-Use simple lastIndexOf(...) and substring for a faster implementation

Result:

No more lock condition.
2015-04-22 09:11:11 +02:00
Alwayswithme
9a365b101e fix the discardedBytes counting on LineBasedFrameDecoder
Motivation:

The LineBasedFrameDecoder discardedBytes counting different compare to
DelimiterBasedFrameDecoder.

Modifications:

Add plus sign

Result:

DiscardedBytes counting correctly
2015-04-21 09:25:57 +02:00
Eric Anderson
97eba4eb66 Fix SslContextBuilder swapping client and server
The 'forClient' boolean was swapped to 'forServer' in code review of #3531.
Not all locations were updated.
2015-04-20 17:24:43 -07:00
Norman Maurer
f242bc5a1a [#3623] CompositeByteBuf.iterator() should return optimized Iterable
Motivation:

CompositeByteBuf.iterator() currently creates a new ArrayList and fill it with the ByteBufs, which is more expensive then it needs to be.

Modifications:

- Use special Iterator implementation

Result:

Less overhead when calling iterator()
2015-04-20 10:45:28 +02:00
Norman Maurer
5cd541c537 Fix handling of non-auto read for ByteToMessageDecoder and SslHandler
Motivation:

Our automatically handling of non-auto-read failed because it not detected the need of calling read again by itself if nothing was decoded. Beside this handling of non-auto-read never worked for SslHandler as it always triggered a read even if it decoded a message and auto-read was false.

This fixes [#3529] and [#3587].

Modifications:

- Implement handling of calling read when nothing was decoded (with non-auto-read) to ByteToMessageDecoder again
- Correctly respect non-auto-read by SslHandler

Result:

No more stales and correctly respecting of non-auto-read by SslHandler.
2015-04-20 09:02:47 +02:00
Norman Maurer
3b1bf6348a Correctly test for non-auto-read correctness in testsuite
Motiviation:

Our tests for non-auto-read did actually not test this correctly as auto-read was never disabled on the Bootstrap and ServerBootstrap.

Modifications:

- Correctly disable auto-read on Bootstrap and ServerBootstrap
- Fix tests to call ChannelHandlerContext.read() once a Channel becomes active.

Result:

Correctly test that non-auto-read works.
2015-04-20 09:02:40 +02:00
Norman Maurer
b35f2b4cb4 Revert "Ensure ctx.fireChannelReadComplete() is only called when something was produced"
This reverts commit 375b9e1307.
2015-04-20 09:02:34 +02:00
Norman Maurer
76dcda8259 Revert "Ensure channelReadComplete() is called only when necessary"
This reverts commit 8b2fb2b985.
2015-04-20 09:02:23 +02:00
Norman Maurer
f35158f402 Revert "Do not suppress channelReadComplete() when a handler was just added"
This reverts commit a41b46ff43.
2015-04-20 09:02:14 +02:00
Norman Maurer
2e1325aa9d Revert "Add another test case for channelReadComplete() suppression"
This reverts commit 5fca3de098.
2015-04-20 09:02:01 +02:00
Norman Maurer
d376bf6a58 Reduce object allocation during wrap/unwrap while handshake is in progress
Motivation:

Unnecessary object allocation is currently done during wrap/unwrap while a handshake is still in progress.

Modifications:

Use static instances when possible.

Result:

Less object creations.
2015-04-20 06:47:18 +02:00
Norman Maurer
19179f9fee Allow to get version of available OpenSSL library
Motivation:

Sometimes it's useful to get informations about the available OpenSSL library that is used for the OpenSslEngine.

Modifications:

Add two new methods which allows to get the available OpenSSL version as either
an int or an String.

Result:

Easy to access details about OpenSSL version.
2015-04-18 20:56:15 +02:00
Eric Anderson
6668b3c40d The "null" ClassLoader is the bootstrap ClassLoader
Motivation:
Class.forName() documents that null will use bootstrap loader:
http://docs.oracle.com/javase/8/docs/api/java/lang/Class.html#forName-java.lang.String-boolean-java.lang.ClassLoader-

But the link between "null" and bootstrap loader is even more explicit
in ClassLoader's documentation:
http://docs.oracle.com/javase/8/docs/api/java/lang/ClassLoader.html#getParent--

The current code is trying to use the bootstrap loader but seems to have
not been aware of the meaning of null.

Modifications:
Use "null" as the class loader when we want to load classes in the
bootstrap loader.

Result:
More reliable ALPN/NPN loading and simpler code.
2015-04-16 17:25:18 +02:00
Derek Troy-West
b7aef7bec8 Change AggregatedFullHttpMessage to contain a content ByteBuf
Motivation:

Other implementations of FullHttpMessage allow .toString to be called after the Message has been released
This brings AggregatedFullHttpMessage into line with those impls.

Modifications:

- Changed AggregatedFullHttpMessage to no longer be a sub-class of DefaultByteBufHolder
- Changes AggregatedFullHttpMessage to implement ByteBufHolder
- Hold the content buffer internally to AggregatedFullHttpMessage
- Implement the required content() and release() methods that were missing
- Do not check refcnt when accessing content() (similar to DefaultFullHttpMessage)

Result:

A released AggregatedFullHttpMessage can have .toString called without throwing an exception
2015-04-16 14:30:57 +02:00
Norman Maurer
9e5dd21d23 Fix regression introduced by cherry-pick bd224286f5 2015-04-14 17:59:33 +02:00
Norman Maurer
91a508664f Document the contract of Attribute.getAndSet(...) and set(...)
Motivation:

Attribute.getAndRemove() will return the value but also remove the AttributeKey itself from the AttributeMap. This may not
what you want as you may want to keep an instance of it and just set it later again. Document the contract so the user know what to expect.

Modifications:

- Make it clear when to use AttributeKey.getAndRemove() / AttributeKey.remove() and when AttributeKey.getAndSet(null) / AttributeKey.set(null).

Result:

Less suprising behaviour.
2015-04-14 09:53:04 +02:00
Norman Maurer
e907ae57a2 Add support for EC Keys when using SslServerContext
Motivation:

Sometimes it's useful to use EC keys and not DSA or RSA. We should support it.

Modifications:

Support EC keys and share the code between JDK and Openssl impl.

Result:

It's possible to use EC keys now.
2015-04-14 08:44:38 +02:00
Eric Anderson
fdc396674e [#3531] Create SslContext.Builder
Motivation:

SslContext factory methods have gotten out of control; it's past time to
swap to a builder.

Modifications:

New Builder class. The existing factory methods must be left as-is for
backward compatibility.

Result:

Fixes #3531
2015-04-14 07:27:55 +02:00
Norman Maurer
3307686bda [#3539] Correctly handle EPOLLRDHUP
Motivation:

As we missed to correctly handle EPOLLRDHUP we produce an IOException which is unnessary. This leads
to have exceptionCaught(...) methods called.

Modifications:

When EPOLLRDHUP was received just close the socket and fail all pending writes.

Result:

Correctly handle of EPOLLRDHUP and so not miss-leading exceptions.
2015-04-14 06:56:02 +02:00
garywu
8843d2b1a1 [#2925] Bug fix for NormalMemoryRegionCache overbooked for PoolThreadCache
Motivation:

When create NormalMemoryRegionCache for PoolThreadCache, we overbooked
cache array size. This means unnecessary overhead for thread local cache
as we will create multi cache enties for each element in cache array.

Modifications:

change:
int arraySize = Math.max(1, max / area.pageSize);
to:
int arraySize = Math.max(1, log2(max / area.pageSize) + 1);

Result:

Now arraySize won't introduce unnecessary overhead.

 Changes to be committed:
	modified:   buffer/src/main/java/io/netty/buffer/PoolThreadCache.java
2015-04-13 09:00:51 +02:00
Norman Maurer
bfb6189f77 Let CompositeByteBuf implement Iterable
Motivation:

CompositeByteBuf has an iterator() method but fails to implement Iterable

Modifications:

Let CompositeByteBuf implement Iterable<ByteBuf>

Result:

Easier usage
2015-04-12 13:38:39 +02:00
Norman Maurer
914d9ec095 [#3373] Rename class to match naming scheme
Motivation:

The ReplayingDecoderBuffer does not match the naming scheme we use for ByteBuf types.

Modifications:

Rename to ReplayingDecoderByteBuf to match naming scheme

Result:

Consistent naming
2015-04-12 13:32:24 +02:00
Norman Maurer
e4900cbdd3 Revert "Dereference when calling PooledByteBuf.deallocate()"
This reverts commit ddd19f4f21.
2015-04-11 06:44:10 +02:00
Norman Maurer
ddd19f4f21 Dereference when calling PooledByteBuf.deallocate()
Motivation:

We missed to dereference the chunk and tmpNioBuf when calling deallocate(). This means the GC can not collect these as we still hold a reference while have the PooledByteBuf in the recycler stack.

Modifications:

Dereference chunk and tmpNioBuf.

Result:

GC can collect things.
2015-04-10 21:46:43 +02:00
Norman Maurer
bfee39b318 [#3592] Flush when writing HttpChunkedInput
Motivation:

We missed to flush the channel when using HttpChunkedInput (this is done when using SSL). This will result in a stale.

Modifications:

Replace ctx.write(...) with ctx.writeAndFlush(...)

Result:

Correctly working example.
2015-04-10 21:19:02 +02:00
Norman Maurer
30f60ac68a Change PoolThreadCache to use LIFO for better cache performance
Motiviation:

At the moment we use FIFO for the PoolThreadCache which is sub-optimal as this may reduce the changes to have the cached memory actual still in the cpu-cache.

Modification:

- Change to use LIFO as this increase the chance to be able to serve buffers from the cpu-cache

Results:

Faster allocation out of the ThreadLocal cache.

Before the commit:
[xxx wrk]$ ./wrk -H 'Connection: keep-alive' -d 120 -c 256 -t 16 -s scripts/pipeline-many.lua  http://xxx:8080/plaintext
Running 2m test @ http://xxx:8080/plaintext
  16 threads and 256 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    14.69ms   10.06ms 131.43ms   80.10%
    Req/Sec   283.89k    40.37k  433.69k    66.81%
  533859742 requests in 2.00m, 72.09GB read
Requests/sec: 4449510.51
Transfer/sec:    615.29MB

After the commit:
[xxx wrk]$ ./wrk -H 'Connection: keep-alive' -d 120 -c 256 -t 16 -s scripts/pipeline-many.lua  http://xxx:8080/plaintext
Running 2m test @ http://xxx:8080/plaintext
  16 threads and 256 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    16.38ms   26.32ms 734.06ms   97.38%
    Req/Sec   283.86k    39.31k  361.69k    83.38%
  540836511 requests in 2.00m, 73.04GB read
Requests/sec: 4508150.18
Transfer/sec:    623.40MB
2015-04-10 20:57:31 +02:00
Norman Maurer
97a3308c7e Add support for ALPN when using openssl + NPN client mode and support for CipherSuiteFilter
Motivation:

To support HTTP2 we need APLN support. This was not provided before when using OpenSslEngine, so SSLEngine (JDK one) was the only bet.
Beside this CipherSuiteFilter was not supported

Modifications:

- Upgrade netty-tcnative and make use of new features to support ALPN and NPN in server and client mode.
- Guard against segfaults after the ssl pointer is freed
- support correctly different failure behaviours
- add support for CipherSuiteFilter

Result:

Be able to use OpenSslEngine for ALPN / NPN for server and client.
2015-04-10 18:59:58 +02:00
JongYoon Lim
ecf1b558d9 Motivation: If there are no readable bytes, it's unnecessary to go through javaChannel().write().
Modification:
If the readableBytes() is zero, then just return.

Result:
The logic can be easy to understand and a little faster than before.
2015-04-08 09:36:32 +02:00
Frederic Bregier
2ab2e2d6b7 Fix incorrect null value check in TrafficCounter
In TrafficCounter, a recent change makes the contract of the API (the
constructor) wrong and lead to issue with GlobalChannelTrafficCounter
where executor must be null.

Motivation:
TrafficCounter executor argument in constructor might be null, as
explained in the API, for some particular cases where no executor are
needed (relevant tasks being taken by the caller as in
GlobalChannelTrafficCounter).
A null pointer exception is raised while it should not since it is
legal.

Modifications:
Remove the 2 null checking for this particular attribute.
Note that when null, the attribute is not reached nor used (a null
checking condition later on is applied).

Result:
No more null exception raized while it should not.

This shall be made also to 4.0, 4.1 (present) and master. 3.10 is not
concerned.
2015-04-06 19:07:13 +02:00
Norman Maurer
f2fedbcdef [maven-release-plugin] prepare for next development iteration 2015-03-31 22:06:30 -04:00
Norman Maurer
054e7c5d17 [maven-release-plugin] prepare release netty-4.0.27.Final 2015-03-31 22:05:43 -04:00
Trustin Lee
4d8f824358 Fix unbounded expansion of cumulative buffer in SslHandler
Related: #3567

Motivation:

SslHandler.channelReadComplete() forgets to call
super.channelReadComplete(), which discards read bytes from the
cumulative buffer.  As a result, the cumulative buffer can expand its
capacity unboundedly.

Modifications:

Call super.channelReadComplete() instead of calling
ctx.fireChannelReadComplete()

Result:

Fixes #3567
2015-04-02 14:57:02 +09:00
Trustin Lee
28caa873b3 Don't trigger IOException at ChunkedStream.isEndOfInput()
Related: #3368

Motivation:

ChunkedWriteHandler checks if the return value of
ChunkedInput.isEndOfInput() after calling ChunkedInput.close().

This makes ChunkedStream.isEndOfInput() trigger an IOException, which is
originally triggered by PushBackInputStream.read().

By contract, ChunkedInput.isEndOfInput() should not raise an IOException
even when the underlying stream is closed.

Modifications:

Add a boolean flag that keeps track of whether the underlying stream has
been closed or not, so that ChunkedStream.isEndOfInput() does not
propagate the IOException from PushBackInputStream.

Result:

Fixes #3368
2015-03-31 11:31:54 +09:00
Norman Maurer
0c6ba4aab6 Add supported for X509ExtendedTrustManager when using OpenSslEngine
Motivation:

For some use cases X509ExtendedTrustManager is needed as it allows to also access the SslEngine during validation.

Modifications:

Add support for X509ExtendedTrustManager on java >= 7

Result:

It's now possible to use X509ExtendedTrustManager with OpenSslEngine
2015-03-30 09:05:09 +02:00
Trustin Lee
0222f41233 Enforce a release is made with JDK 1.7 (and not w/ 1.8+)
Motivations:

JDK 1.8 adds default methods to collections classes that reference
classes that don't exist in JDK 7. That's binary compatible,
but not source compatible.

Modifications:

Enforce JDK version to be 1.7.* when releasing

Result:

Fixes #3548
2015-03-29 16:58:50 +09:00
David Dossot
cb8a607ba5 Fix example in CookieDecoder Javadoc
- CookieDecoder.decode() is a static method.
2015-03-26 11:49:30 +09:00
Scott Mitchell
d43c21dab0 Jetty ALNPN and NPN updates plus backport
Motivation:
There are new versions of the ALPN and NPN dependencies.  There was also some backport misses in the pom file related to ALPN/NPN.

Modifications:
- Add new versions for ALPN/NPN dependencies.
- Backport missed pieces from pom.xml.

Result:
Updated version of ALPN/NPN versions.
2015-03-25 08:55:13 -07:00
Pierre DAL-PRA
2221dc6b20 Small typos fixes in Channel's Javadoc 2015-03-21 16:13:05 +01:00
Wouter
f78c384fec Fix typo in javadoc 2015-03-18 17:15:45 +01:00
Trustin Lee
c1ac64fb82 Safely encode Strings to ASCII
(Ported @luciferous's changes against 3.10)

Motivation:

The current implementation of the encoder writes each character of the
String as a single byte to the buffer, however not all characters are
mappable to a single byte.

Modifications:

If a character is outside the ASCII range, it's converted to '?'.

Result:

A safer encoder for String to ASCII, which substitutes unmappable
2015-03-18 15:57:31 +09:00
Idel Pivnitskiy
abcb2b3661 Update jUnit version to 4.12
Motivation:

Too many new features in the new release of jUnit.
https://github.com/junit-team/junit/blob/master/doc/ReleaseNotes4.12.md

Modifications:

- Changed version of jUnit from 4.11 to 4.12 in the parent pom.

Result:

Allows using new testing features.
2015-03-17 16:17:59 +01:00
Trustin Lee
9091e66e15 Hide password in exception messages of SocksAuthRequest
Related: #3504

Motivation:

There are two places in the SocksAuthRequest constructor where an
IllegalArgumentException is thrown with a password as part of the
exception message.

This constitutes mishandling of confidential information, which can
compromise user privacy and is flagged as critical by security scanners.

Modifications:

Mask the password in the exception messages

Result:

No unexpected password leak
2015-03-17 17:23:30 +09:00
Leonardo Freitas Gomes
034e0bd8d2 Ensure server preference order in ALPN
Motivation:
With the current implementation the client protocol preference list
takes precedence over the one of the server, since the select method
will return the first item, in the client list, that matches any of the
protocols supported by the server. This violates the recommendation of
http://tools.ietf.org/html/rfc7301#section-3.2.

It will also fail with the current implementation of Chrome, which
sends back Extension application_layer_protocol_negotiation, protocols:
[http/1.1, spdy/3.1, h2-14]

Modifications:
Changed the protocol negotiator to prefer server’s list. Added a test
case that demonstrates the issue and that is fixed with the
modifications of this commit.

Result:
Server’s preference list is used.
2015-03-17 07:29:10 +01:00