Commit Graph

8391 Commits

Author SHA1 Message Date
Norman Maurer
32f497760f Ensure no java.lang.UnsupportedClassVersionError are thrown if running on Java7 and try to check if conscrypt is available.
Motivation:

We need to ensure we not try to load any conscrypt classes directly (which means without using reflection) in the same class that is used to check if conscrypt is available. This is needed as otherwise we will have the following problem when try to use netty on java7:

java.lang.UnsupportedClassVersionError: org/conscrypt/BufferAllocator : Unsupported major.minor version 52.0
	at io.netty.handler.ssl.ConscryptJdkSslEngineInteropTest.checkConscrypt(ConscryptJdkSslEngineInteropTest.java:49)

This regression was introduced by 4448b8f42f and detected on the CI when using:

mvn clean package -DtestJavaHome=$JAVA7_HOME

Modifications:

Move the detection code in an extra class and use it.

Result:

Works correctly also when using Java7.
2017-08-04 10:48:50 +02:00
Norman Maurer
4bb89dcc54 Correctly handle connect/disconnect in EpollDatagramChannel / KQueueDatagramChannel
Motivation:

We did not correctly handle connect() and disconnect() in EpollDatagramChannel / KQueueDatagramChannel and so the behavior was different compared to NioDatagramChannel.

Modifications:

- Correct implement connect and disconnect methods
- Share connect and related code
- Add tests

Result:

EpollDatagramChannel / KQueueDatagramChannel also supports correctly connect() and disconnect() methods.
2017-08-04 09:22:53 +02:00
Nathan Mittler
4448b8f42f Upgrading to Conscrypt 1.0.0.RC9. (#7044)
Motivation:

Starting with 1.0.0.RC9, conscrypt supports a buffer allocator.

Modifications:

- Updated the creation process for the engine to pass through the
ByteBufAllocator.
- Wrap a ByteBufAllocator with an adapter for conscrypt.
- Added a property to optionally control whether conscrypt uses
Netty's buffer allocator.

Result:

Netty+conscrypt will support using Netty's ByteBufAllocator.
2017-08-03 14:21:32 -07:00
Norman Maurer
52f384b37f [maven-release-plugin] prepare for next development iteration 2017-08-02 12:55:10 +00:00
Norman Maurer
8cc1071881 [maven-release-plugin] prepare release netty-4.1.14.Final 2017-08-02 12:54:51 +00:00
chhsiao90
8320a45c15 Configures HTTP2 pipeline with more proper way
Motivation:

When we use pipeline.replace and we still had ongoing inbound, then
there will be some problem that inbound message would go to wrong
handlers. So we add handler first, and remove self after add, so that
the next handler will be the correct one.

Modifications:

Uses remove after addAfter instead of replace.

Result:

Fixed #6881
2017-08-02 06:58:55 +02:00
Nikolay Fedorovskikh
6ab9c177ac Fix hash function and hash table size in Snappy
Motivation:

1. Hash function in the Snappy encoding is wrong probably: used '+' instead of '*'. See the reference implementation [1].
2. Size of the hash table is calculated, but not applied.

Modifications:

1. Fix hash function: replace addition by multiplication.
2. Allocate hash table with calculated size.
3. Use an `Integer.numberOfLeadingZeros` trick for calculate log2.
4. Release buffers in tests.

Result:

1. Better compression. In the test `encodeAndDecodeLongTextUsesCopy` now compressed size is 175 instead of 180 before this change.
2. No redundant allocations for hash table.
3. A bit faster the calc of shift (less an expensive math operations).

[1] 513df5fb5a/snappy.cc (L67)
2017-08-01 07:08:54 +02:00
Norman Maurer
2988fb8eeb Ensure Http2FrameCodec uses Http2Settings.defaultSettings()
Motivation:

Http2FrameCodec should use Http2Settings.defaultSettings() when no Http2Settings were specified by the user.

Modifications:

Replace new Http2Settings() with Http2Settings.defaultSettings()

Result:

Use correct Http2Settings by default when using Http2FrameCodec in all cases.
2017-08-01 07:07:05 +02:00
Nikolay Fedorovskikh
068e64dbcf Fix a potential NPE 2017-07-31 20:39:20 +02:00
Alberto K
21b7ab1f25 Allows IP_TRANSPARENT to be set on a redirecting socket
Motivation:

IP_TRANSPARENT support is not complete, the option can currently only be set on EpollServerSocket. Setting the option on an EpollSocket is also requires so as to be able to bind a socket to a non-local address as described in ip(7)
http://man7.org/linux/man-pages/man7/ip.7.html

"TProxy redirection with the iptables TPROXY target also
requires that this option be set on the redirected socket."

Modifications:

Added IP_TRANSPARENT socket option to EpollSocketChannelConfig

Result:

A redirecting socket can be created with a non-local IP address as required for TPROXY
2017-07-30 19:51:55 +02:00
Norman Maurer
580ac8cd41 Only flush on channelReadComplete(...) in http2 hello world examples.
Motivation:

In our http1 hello world example we only flush on channelReadComplete(...) to make better use of gathering writes. We should do the same in http2.

Modifications:

Only flush in channelReadComplete(...)

Result:

Better performance and more consistent examples.
2017-07-30 07:18:49 +02:00
Norman Maurer
8adb30bbe2 Correctly run all pending tasks for EmbeddedChannel when the Channel was closed.
Motivation:

When a user called ctx.close() and used the EmbeddedChannel we did not correctly run all pending tasks which means channelInactive was never called.

Modifications:

Ensure we run all pending tasks after all operations that may change the Channel state and are part of the Channel.Unsafe impl.

Result:

Fixes [#6894].
2017-07-30 06:57:18 +02:00
Norman Maurer
d9d3d65716 Add comment why the ResourceLeak creation is happening as last in the constructor. Followup of c5b5d36360 2017-07-30 06:55:22 +02:00
Norman Maurer
c5b5d36360 Fix false-positive leak detection report when ReferenceCountedOpenSslEngine constructor throws.
Motivation:

We need to ensure we only create the ResourceLeak when the constructor not throws.

Modifications:

Ensure ResourceLeakDetector.track(...) is only called if the constructor of ReferenceCoundedOpenSslEngine not throws.

Result:

No more false-positves.
2017-07-29 22:01:56 +02:00
Carl Mastrangelo
60250f3795 Make DelegatingChannelPromiseNotifier use Vararg overload
Motivation:
ErrorProne complains that the array override doesn't match the
vararg super call.  See http://errorprone.info/bugpattern/Overrides

Additionally, almost every other Future uses the vararg form, so
it would be stylistically consistent to keep it that way.

Modifications:
Use vararg override.

Result:
Cleaner, less naggy code.
2017-07-28 07:29:43 +02:00
Norman Maurer
339131c660 DefaultChannelPipeline.estimatorHandle needs to be volatile
Motivation:

DefaultChannelPipeline.estimatorHandle needs to be volatile as its accessed from different threads.

Modifications:

Make DefaultChannelPipeline.estimatorHandle volatile and correctly init it via CAS

Result:

No more race.
2017-07-27 06:57:22 +02:00
Vladimir Gordiychuk
fe8ecea366 Http2FrameLogger avoid hex dump of the ByteBufs when log disabled
Motivation:

Currentry logger create hex dump even if log write will not apply.
It's unecessary GC overhead.

Modifications:

Restore optimization from #3492

Result:

Fixes #7025
2017-07-26 21:21:04 +02:00
Spencer Fang
732b145842 Http2ConnectionHandler: allow graceful shutdown to wait forever
Motivation:

There should be a way to allow graceful shutdown to wait for all open streams to close without a timeout. Using gracefulShutdownTimeoutMillis with a large value is a bit of a hack, and has a gotcha that sufficiently large values will overflow the long, resulting in a ClosingChannelFutureListener that executes immediately.

Modification:

Allow to use gracefulShutdownTimeoutMillis(-1) to express waiting until all streams are closed.

Result:

We can now shutdown the connection without a forced timeout.
2017-07-26 20:40:24 +02:00
Norman Maurer
529025d9d5 Allow to use oldest Channel out of the Simple / FixedChannelPool on acquire
Motivation:

We previously used pollLast() to retrieve a Channel from the queue that backs SimpleChannelPool. This could lead to the problem that some Channels are very unfrequently used and so when these are used the connection was already be closed and so could not be reused.

Modifications:

Allow to configure if the last recent used Channel should be used or the "oldest".

Result:

More flexible usage of ChannelPools
2017-07-26 20:37:19 +02:00
Norman Maurer
359beff56f Choose ipv4 or ipv6 google dns servers as default fallback based on the settings for this system / jvm
Motivation:

We should not use ipv4 google dns servers if the app is configured to run ipv6.

Modifications:

Use either ipv4 or ipv6 dns servers depending on the system config.

Result:

More correct behaviour
2017-07-26 20:33:52 +02:00
Norman Maurer
efb2d141c1 Correctly handle unsigned int values returned from TCP_INFO
Motivation:

We used an int[] to store all values that are returned in the struct for TCP_INFO which is not good enough as it uses usigned int values.

Modifications:

- Change int[] to long[] and correctly cast values.

Result:

No more truncated values.
2017-07-25 16:18:52 +02:00
Norman Maurer
34fdc7a33e Skip invalid hostnames when construct default dns servers to use.
Motivation:

When the hostname portion can not be extracted we should just skip the server as otherwise we will produce and exception when trying to create the InetSocketAddress.

This was happing when trying to run the test-suite on a system and using java7:

java.lang.IllegalArgumentException: hostname can't be null
	at java.net.InetSocketAddress.checkHost(InetSocketAddress.java:149)
	at java.net.InetSocketAddress.<init>(InetSocketAddress.java:216)
	at io.netty.util.internal.SocketUtils$10.run(SocketUtils.java:171)
	at io.netty.util.internal.SocketUtils$10.run(SocketUtils.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at io.netty.util.internal.SocketUtils.socketAddress(SocketUtils.java:168)
	at io.netty.resolver.dns.DefaultDnsServerAddressStreamProvider.<clinit>(DefaultDnsServerAddressStreamProvider.java:74)
	at io.netty.resolver.dns.DnsServerAddressesTest.testDefaultAddresses(DnsServerAddressesTest.java:39)

Modifications:

Skip if hostname can not be extracted.

Result:

No more java.lang.ExceptionInInitializerError.
2017-07-25 08:43:53 +02:00
Norman Maurer
486f962252 Respect DNS port that is specified via JNDI
Motivation:

JNDI allows to specify an port so we should respect it.

Modifications:

Use the specified port and if none is specifed use 53.

Result:

Correct handling of JNDI configured DNS.
2017-07-25 08:26:59 +02:00
Carl Mastrangelo
d8f4547f5c Unify {Epoll,KQueue}EventLoopGroup initialization.
Motivation:
`Epoll.ensureAvailability()` is called multiple times, once in
static initialization and in a couple of the constructors.  This is
redundant and confusing to read.

Modifications:
Move `Epoll.ensureAvailability()` call into an instance initializer
and remove all other references.  This ensures that every EELG
checks availability, while still delaying the check until
construction.  This pattern is used when there are multiple ctors,
as in this class.

Result:
Easier to read code.
2017-07-24 20:14:54 +02:00
Scott Mitchell
452fd36240 ByteBufs which are not resizable should not throw in ensureWritable(int,boolean)
Motivation:
ByteBuf#ensureWritable(int,boolean) returns an int indicating the status of the resize operation. For buffers that are unmodifiable or cannot be resized this method shouldn't throw but just return 1.
ByteBuf#ensureWriteable(int) should throw unmodifiable buffers.

Modifications:
- ReadOnlyByteBuf should be updated as described above.
- Add a unit test to SslHandler which verifies the read only buffer can be tolerated in the aggregation algorithm.

Result:
Fixes https://github.com/netty/netty/issues/7002.
2017-07-22 08:44:48 -07:00
Eric Anderson
2fbce6d470 Delete temporary self-signed certs in SSLEngineTest-based tests
Motivation:

Lots of usages of SelfSignedCertificates were not deleting the certs at
the end of the test. This includes setupHandlers() which is used by
extending classes. Although these files will be deleted at JVM exit and
deleting them early does not free the JVM from trying to delete them at
shutdown, it's good practice to delete eagerly and since users sometimes
use tests as a form of documentation, it'd be good for them to see the
explicit deletes.

Modifications:

Add missing delete() calls to ½ of the SelfSignedCertificates-using
tests.

Result:

Tests that more clearly communicates which resources are created and
may accumulate without early delete.
2017-07-22 08:13:57 +02:00
Eric Anderson
8a25c35939 Filter user-provided ciphers using RFC cipher names
Motivation:

Previously filterCipherSuites was being passed the OpenSSL-formatted
cipher names. Commit 43ae974 introduced a regression as it swapped to the
RFC/JDK format, except that user-provided ciphers were not converted and
remained in the OpenSSL format.

This mis-match would cause all user-provided to be thrown away, leading
to failure trying to set zero ciphers:
Exception in thread "main" javax.net.ssl.SSLException: failed to set cipher suite: []
	at io.netty.handler.ssl.ReferenceCountedOpenSslContext.<init>(ReferenceCountedOpenSslContext.java:299)
	at io.netty.handler.ssl.OpenSslContext.<init>(OpenSslContext.java:43)
	at io.netty.handler.ssl.OpenSslServerContext.<init>(OpenSslServerContext.java:347)
	at io.netty.handler.ssl.OpenSslServerContext.<init>(OpenSslServerContext.java:335)
	at io.netty.handler.ssl.SslContext.newServerContextInternal(SslContext.java:421)
	at io.netty.handler.ssl.SslContextBuilder.build(SslContextBuilder.java:441)
Caused by: java.lang.Exception: Unable to configure permitted SSL ciphers (error:100000b1:SSL routines:OPENSSL_internal:NO_CIPHER_MATCH)
	at io.netty.internal.tcnative.SSLContext.setCipherSuite(Native Method)
	at io.netty.handler.ssl.ReferenceCountedOpenSslContext.<init>(ReferenceCountedOpenSslContext.java:295)
	... 7 more

Modifications:

Remove the reformatting of user-provided ciphers, as they are already in
the RFC/JDK format.

Result:

No regression, and the internals stay sane using the RFC/JDK format.
2017-07-21 19:20:07 -07:00
Norman Maurer
d141ba11bf Fix flacky multipart test introduced by 08748344d8.
Motivation:

08748344d8 introduced two new tests which did not take into account that the multipart delimiter can be between 2 and 16 bytes long.

Modifications:

Take the multipart delimiter length into account.

Result:

Fixes [#7001]
2017-07-21 14:28:32 +02:00
Norman Maurer
06f64948d5 Add tests to ensure an IllegalReferenceCountException is thrown if set/writeCharSequence is called on a released buffer
Motivation:

We need to ensure we not allow calling set/writeCharsequence on an released ByteBuf.

Modifications:

Add test-cases

Result:

Proves fix of [#6951].
2017-07-21 07:39:32 +02:00
martin vseticka
f897507b09 Handle handshake failure in Websocket Client example
Motivation:

We need to fail the promise if a failure during handshake happens.

Modification:

Correctly fail the promise.

Result:

Correct websocket client example. Fixes [#6998]
2017-07-21 07:36:36 +02:00
Norman Maurer
f23b2fc25d Use 4 spaces and not 2 spaces (cleanup of 3d22b24244) 2017-07-20 10:24:18 +02:00
Norman Maurer
3cdff36821 Update tests to not use TestUtils.getFreePort() and so ensure we not try to use a port that is used by the system in the meantime.
Motivation:

We should not try to detect a free port in tests put just use 0 when bind so there is no race in which the system my bind something to the port we choosen before.

Modifications:

- Remove the usage of TestUtils.getFreePort() in the testsuite
- Remove hack to workaround bind errors which will not happen anymore now

Result:

Less flacky tests.
2017-07-20 08:25:37 +02:00
Eric Anderson
e5a31a4282 Automatically detect shaded packagePrefix
Motivation:

Shading requires renaming binary components (.so, .dll; for tcnative,
epoll, etc). But the rename then requires setting the
io.netty.packagePrefix system property on the command line or runtime,
which is either a burden or not feasible.

If you don't rename the binary components everything appears to
work, until a dependency on a second version of the binary component is
added. At that point, only one version of the binary will be loaded...
which is what shading is supposed to prevent. So for valid shading, the
binaries must be renamed.

Modifications:

Automatically detect the package prefix by comparing the actual class
name to the non-shaded expected class name. The expected class name must
be obfuscated to prevent shading utilities from changing it.

Result:

When shading and using binary components, runtime configuration is no
longer necessary.

Pre-existing shading users that were not renaming the binary components
will break, because the packagePrefix previously defaulted to "". Since
these pre-existing users had broken configurations that only _appeared_
to work, this breakage is considered a Good Thing. Users may workaround
this breakage temporarily by setting -Dio.netty.packagePrefix= to
restore packagePrefix to "".

Fixes #6963
2017-07-19 18:38:02 -07:00
Carl Mastrangelo
3d22b24244 Allowed Netty Epoll/Kqueue to work in IPv6 Only environments
Motivation:
In some environments, IPv4 may be disabled (at a kernel level).
Google has such an environment for testing v4 -> v6 transition
paths.  This give confidence that code is v6 ready.

Modifications:
Change native socket code to ignore failures of trying to enter
dual stack mode.  This change has been made to Google's internal
JDK, and will/should be upstreamed to OpenJDK eventually.

Results:
Netty works in IPv6 only environments

Fixes: #6993
2017-07-19 18:26:37 -07:00
Scott Mitchell
a91df58ca1 HTTP/2 enforce HTTP message flow
Motivation:
codec-http2 currently does not strictly enforce the HTTP/1.x semantics with respect to the number of headers defined in RFC 7540 Section 8.1 [1]. We currently don't validate the number of headers nor do we validate that the trailing headers should indicate EOS.

[1] https://tools.ietf.org/html/rfc7540#section-8.1

Modifications:
- DefaultHttp2ConnectionDecoder should only allow decoding of a single headers and a single trailers
- DefaultHttp2ConnectionEncoder should only allow encoding of a single headers and optionally a single trailers

Result:
Constraints of RFC 7540 restricting the number of headers/trailers is enforced.
2017-07-19 13:37:23 -07:00
Norman Maurer
4af47f0ced AbstractByteBuf.setCharSequence(...) must not expand buffer
Motivation:

AbstractByteBuf.setCharSequence(...) must not expand the buffer if not enough writable space is present in the buffer to be consistent with all the other set operations.

Modifications:

- Ensure we only exand the buffer on writeCharSequence(...) but not on setCharSequence(...)
- Add unit tests.

Result:

Consistent and correct behavior.
2017-07-19 19:44:53 +02:00
Norman Maurer
ef22e65b57 Allow to delay registration when creating a EmbeddedChannel
Motivation:

Some ChannelOptions must be set before the Channel is really registered to have the desired effect.

Modifications:

Add another constructor argument which allows to not register the EmbeddedChannel to its EventLoop until the user calls register().

Result:

More flexible usage of EmbeddedChannel. Also Fixes [#6968].
2017-07-19 19:35:03 +02:00
Violeta Georgieva
08748344d8 Fix NPEs in HttpPostRequestEncoder#nextChunk
Motivation:

HttpPostRequestEncoder maintains an internal buffer that holds the
current encoded data. There are use cases when this internal buffer
becomes null, the next chunk processing implementation should take
this into consideration.

Modifications:

- When preparing the last chunk if currentBuffer is null, mark
isLastChunkSent as true and send LastHttpContent.EMPTY_LAST_CONTENT
- When calculating the remaining size take into consideration that the
currentBuffer might be null
- Tests are based on those provided in the issue by @nebhale and @bfiorini

Result:

Fixes #5478
2017-07-19 14:35:51 +02:00
Norman Maurer
deb5c45204 Correct typo in artifactId of dependency in bom pom.xml
Motivation:

There was a typo in a dependency in the bom pom.xml which lead to have it specify a non-existing artifact and also so not have the maven release plugin update the version correctly.

Modifications:

Rename netty-transport-unix-common to netty-transport-native-unix-common and also fix the version.

Result:

Fixes [#6979]
2017-07-19 14:20:11 +02:00
Norman Maurer
dbd82e07b1 Let Http2ServerUpgradeCodec support Http2FrameCodec
Motivation:

Http2ServerUpgradeCodec should support Http2FrameCodec.

Modifications:

- Add support for Http2FrameCodec
- Add example that uses Http2FrameCodec

Result:

More flexible use of Http2ServerUpgradeCodec
2017-07-19 11:12:10 +02:00
Scott Mitchell
0afe4e0964 Increase timeout for DnsNameResolverTest
Motivation:
DnsNameResolverTest has been observed to timeout on the CI servers. We should increase the timeout from 5 seconds to 30 seconds.

Modifications:
- Increase timeout from 5 to 30 seconds.

Result:
Less false failures due to slower CI machines.
2017-07-19 07:59:56 +02:00
Norman Maurer
d125adec38 AbstractByteBuf.ensureWritable(...) should check if buffer was released
Motivation:

AbstractByteBuf.ensureWritable(...) should check if buffer was released and if so throw an IllegalReferenceCountException

Modifications:

Ensure we throw in all cases.

Result:

More consistent and correct behaviour
2017-07-19 07:34:08 +02:00
louxiu
96e06aa74d Calculate correct lastRecords size
Motivation:
ResourceLeakDetector records at most MAX_RECORDS+1 records

Modifications:
Make room before add to lastRecords

Result:
ResourceLeakDetector will record at most MAX_RECORDS records
2017-07-18 19:14:31 -07:00
kashike
c43e09da5a Use the correct murmur3 C1 value
introduced in a7f7d9c8e0
2017-07-18 19:03:33 -07:00
ppatierno
b8d3d96550 MQTT unknown message type isn't handled as decoding error
Motivation:

MQTT unknown message type isn't handled as decoding error

Modification:

Catching exception during the MQTT decoding of the fixed header
Adding a unit test for unknown MQTT message type

Result:

Fixes #6984.
2017-07-18 15:48:09 +02:00
Nikolay Fedorovskikh
3e9f617504 Deduplicate and simplify code in HttpPostMultipartRequestDecoder
Motivation:

- A `HttpPostMultipartRequestDecoder` contains two pairs of the same methods: `readFileUploadByteMultipartStandard`+`readFileUploadByteMultipart` and `loadFieldMultipartStandard`+`loadFieldMultipart`.
- These methods use `NotEnoughDataDecoderException` to detecting not last data chunk (exception handling is very expensive).
- These methods can be greatly simplified.
- Methods `loadFieldMultipart` and `loadFieldMultipartStandard` has an unnecessary catching for the `IndexOutOfBoundsException`.

Modifications:

- Remove duplicate methods.
- Replace handling `NotEnoughDataDecoderException` by the return of a boolean result.
- Simplify code.

Result:

The code is cleaner and easier to support. Less exception handling logic.
2017-07-18 13:25:12 +02:00
Norman Maurer
64a3e6c69c SSLEngineTest should not depend on OpenSsl* class.
Motivation:

6152990073 introduced a test-case in SSLEngineTest which used OpenSsl.* which should not be done as this is am abstract bass class that is also used for non OpenSsl tests.

Modifications:

Move the protocol definations into SslUtils.

Result:

Cleaner code.
2017-07-18 13:21:27 +02:00
louxiu
0ad99310f5 Record release when enable detailed leak detection
Motivation:
It would be easier to find where is missing release call in several retain release calls on a ByteBuf

Modifications:
Remove final modifier on SimpleLeakAwareByteBuf and SimpleLeakAwareByteBuf release function and override it to record release in AdvancedLeakAwareByteBuf and AdvancedLeakAwareCompositeByteBuf

Result:
Release will be recorded when enable detailed leak detection
2017-07-18 09:28:56 +02:00
Norman Maurer
f1e14d0cb2 Only add / to uri if really needed.
Motivation:

We not need to include the start index in the check. See https://github.com/netty/netty/pull/6924#discussion_r125263918

Modifications:

Change <= to <

Result:

More correct code.
2017-07-18 09:24:19 +02:00
Norman Maurer
e6a399a778 Explicit specify hostaddress during tests to ensure testsuite pass on docker (mac)
Motivation:

When run the current testsuite on docker (mac) it will fail a few tests with:

io.netty.channel.AbstractChannel$AnnotatedConnectException: connect(..) failed: Cannot assign requested address: /0:0:0:0:0:0:0:0%0:46607
Caused by: java.net.ConnectException: connect(..) failed: Cannot assign requested address

Modifications:

Specify host explicit as done in other tests to only use ipv6 when really supported.

Result:

Build pass on docker as well
2017-07-18 07:24:04 +02:00