Commit Graph

9351 Commits

Author SHA1 Message Date
SplotyCode
5b654fb4f7 Fixed toString() exception in MqttSubscribePayload and MqttUnsubscribePayload (#9202)
Motivation:
The toString() methods of MqttSubscribePayload and MqttUnsubscribePayload are causing exceptions when no topics are set.

Modification:
The toString() methods will not throw Excpetions anymore.

Result:
Fixes #9197
2019-05-31 06:48:21 +02:00
Nick Hill
ed23cfae5e Simplify SingleThreadEventExecutor.awaitTermination() implementation (#9081)
Motivation

A Semaphore is currently dedicated to this purpose but a simple
CountDownLatch will do.

Modification

Remove private threadLock Semaphore from SingleThreadEventExecutor and just use a CountDownLatch.

Also eliminate use of PlatformDependent.throwException() in startThread
method, and combine some nested if clauses.

Result

Cleaner EventLoop termination notification.
2019-05-27 16:09:41 +02:00
Norman Maurer
be7c808ce1 Set the HOST header in Http2ClientInitializer when trying to start an upgrade request (#9177)
Motivation:

The io.netty.example.http2.helloworld.client.Http2Client example should work in the h2c (HTTP2 cleartext - non-TLS) mode, which is the default for this example unless you set a -Dssl VM param. As we do not set the HOST header some servers do reject the upgrade request.

Modifications:

Set the HOST header

Result:

Fixes https://github.com/netty/netty/issues/9115.
2019-05-27 16:04:04 +02:00
Nick Hill
1e575d7389 Fix redundant or missing checks and other inconsistencies in ByteBuf impls (#9119)
Motivation

There are a few minor inconsistencies / redundant operations in the
ByteBuf implementations which would be good to fix.

Modifications

- Unnecessary ByteBuffer.duplicate() performed in
CompositeByteBuf.nioBuffer(int,int)
- Add missing checkIndex(...) check to
ReadOnlyByteBufferBuf.nioBuffer(int,int)
- Remove duplicate bounds check in
ReadOnlyByteBufferBuf.getBytes(int,byte[],int,int)
- Omit redundant bounds check in
UnpooledHeapByteBuf.getBytes(int,ByteBuffer)

Result

More consistency and slightly less overhead
2019-05-27 15:32:42 +02:00
Norman Maurer
f966a0a041 Correctly detect InternetProtocolFamily when EpollDatagramChannel is created with existing FileDescriptor (#9185)
Motivation:

When EpollDatagramChannel is created with an existing FileDescriptor we should detect the correct InternetProtocolFamily.

Modifications:

Obtain the InternetProtocolFamily from the given FD

Result:

Use correct InternetProtocolFamily when EpollDatagramChannel is created via existing FileDescriptor
2019-05-26 20:30:05 +02:00
Steve Buzzard
33d1a91083 Added UDP multicast (with caveats: getInterface, getNetworkInterface, block or loopback-mode-disabled operations).
Motivation:

Provide epoll/native multicast to support high load multicast users (we are using it for a high load telecomm app at my day job).

Modification:

Added support for source specific and any source multicast for epoll transport. Some caveats: no support for disabling loop back mode, retrieval of interface and block operation, all of which tend to be less frequently used.

Result:

Provides epoll transport multicast for common use cases.

Co-authored-by: Norman Maurer <norman_maurer@apple.com>
2019-05-25 10:06:13 +02:00
Norman Maurer
5694dc3e2b Do not use static exceptions for websocket handshake timeout (#9174)
Motivation:

f17bfd0f64 removed the usage of static exception instances to reduce the risk of OOME due addSupressed calls. We should do the same for exceptions used to signal handshake timeouts.

Modifications:

Do not use static instances

Result:

No risk of OOME due addSuppressed calls
2019-05-23 08:36:37 +02:00
noSim
2e7e6d37d5 Updated jboss-marshalling dependency to current license (#9172)
Motivation:

The mentioned license for the jboss-marshalling dependency is outdated. The license has moved from LGPL v2.1 to Apache 2.0.
The version used by Netty (1.4.11Final) is on Apache 2.0 see https://github.com/jboss-remoting/jboss-marshalling/blob/1.4.11.Final/LICENSE.txt

Modification:

Updated NOTICE file with correct license for jboss-marshalling.

Result:

NOTICE file shows correct license.
2019-05-23 07:21:57 +02:00
Nick Hill
4fa7b99da2 Introduce ByteBuf.maxFastWritableBytes() method (#9086)
Motivation

ByteBuf capacity is automatically increased as needed up to maxCapacity
when writing beyond the buffer's current capacity. However there's no
way to tell in general whether such an increase will result in a
relatively costly internal buffer re-allocation.

For unpooled buffers it always does, in pooled cases it depends on the
size of the associated chunk of allocated memory, which I don't think is
currently exposed in any way.

It would sometimes be useful to know where this limit is when making
external decisions about whether to reuse or preemptively reallocate.

It would also be advantageous to take this limit into account when
auto-increasing the capacity during writes, to defer such reallocation
until really necessary.

Modifications

Introduce new AbstractByteBuf.maxFastWritableBytes() method which will
return a value >= writableBytes() and <= maxWritableBytes().

Make use of the new method in the sizing decision made by the
AbstractByteBuf.ensureWritable(...) methods.

Result

Less reallocation/copying.
2019-05-22 20:11:56 +02:00
Vojin Jovanovic
f7a8a6f1ae Remove deprecated GraalVM native-image flags (#9118)
Motivation:

The first final version of GraalVM was released which deprecated some flags. We should use the new ones.

Modifications:

Removes the use of deprecated GraalVM native-image flags
Adds a flag to initialize netty at build time.

Result:

Do not use deprecated flags
2019-05-22 19:22:03 +02:00
Norman Maurer
9c51a8c6d4 Correctly detect that KeyManagerFactory is not supported when using OpenSSL 1.1.0+ (#9170)
Motivation:

How we tried to detect if KeyManagerFactory is supported was not good enough for OpenSSL 1.1.0+ as it partly provided the API but not all of what is required.

This then lead to failures like:

[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 1.102 s <<< FAILURE! - in io.netty.channel.epoll.EpollDomainSocketStartTlsTest
[ERROR] initializationError(io.netty.channel.epoll.EpollDomainSocketStartTlsTest)  Time elapsed: 0.016 s  <<< ERROR!
javax.net.ssl.SSLException: failed to set certificate and key
	at io.netty.handler.ssl.ReferenceCountedOpenSslServerContext.newSessionContext(ReferenceCountedOpenSslServerContext.java:130)
	at io.netty.handler.ssl.OpenSslServerContext.<init>(OpenSslServerContext.java:353)
	at io.netty.handler.ssl.OpenSslServerContext.<init>(OpenSslServerContext.java:334)
	at io.netty.handler.ssl.SslContext.newServerContextInternal(SslContext.java:468)
	at io.netty.handler.ssl.SslContextBuilder.build(SslContextBuilder.java:457)
	at io.netty.testsuite.transport.socket.SocketStartTlsTest.data(SocketStartTlsTest.java:93)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.runners.Parameterized.allParameters(Parameterized.java:280)
	at org.junit.runners.Parameterized.<init>(Parameterized.java:248)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at org.junit.internal.builders.AnnotatedBuilder.buildRunner(AnnotatedBuilder.java:104)
	at org.junit.internal.builders.AnnotatedBuilder.runnerForClass(AnnotatedBuilder.java:86)
	at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:59)
	at org.junit.internal.builders.AllDefaultPossibilitiesBuilder.runnerForClass(AllDefaultPossibilitiesBuilder.java:26)
	at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:59)
	at org.junit.internal.requests.ClassRequest.getRunner(ClassRequest.java:33)
	at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:362)
	at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
	at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
	at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
	at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
	at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
Caused by: java.lang.Exception: Requires OpenSSL 1.0.2+
	at io.netty.internal.tcnative.SSLContext.setCertificateCallback(Native Method)
	at io.netty.handler.ssl.ReferenceCountedOpenSslServerContext.newSessionContext(ReferenceCountedOpenSslServerContext.java:126)
	... 32 more

Modifications:

Also try to set the certification callback and only if this works as well mark KeyManagerFactory support as enabled.

Result:

Also correctly work when OpenSSL 1.1.0 is used.
2019-05-22 19:08:40 +02:00
Norman Maurer
f81cd7d05b
Use correct object to synchronize on in DefaultChannelPipeline (#9171)
Motivation:

8e72071d76 did adjust how synchronization is done but missed to update one block and so used synchronized (this) while it should be synchronized (handlers) .

Modifications:

Use synchronized (handlers)

Result:

Correctly synchronize
2019-05-22 18:50:28 +02:00
秦世成
1465e3ce06 Support handshake timeout in websocket handlers (#8856)
Motivation:

Support handshake timeout option in websocket handlers. It makes sense to limit the time we need to move from `HANDSHAKE_ISSUED` to `HANDSHAKE_COMPLETE` states when upgrading to WebSockets

Modification:

- Add `handshakeTimeoutMillis` option in `WebSocketClientProtocolHandshakeHandler`  and `WebSocketServerProtocolHandshakeHandler`.
- Schedule a timeout task, the task will trigger user event `HANDSHAKE_TIMEOUT` if the handshake timed out.

Result:

Fixes issue https://github.com/netty/netty/issues/8841
2019-05-22 13:56:36 +02:00
Nick Hill
23554e6997 Ensure "full" ownership of msgs passed to EmbeddedChannel.writeInbound() (#9058)
Motivation

Pipeline handlers are free to "take control" of input buffers if they have singular refcount - in particular to mutate their raw data if non-readonly via discarding of read bytes, etc.

However there are various places (primarily unit tests) where a wrapped byte-array buffer is passed in and the wrapped array is assumed not to change (used after the wrapped buffer is passed to EmbeddedChannel.writeInbound()). This invalid assumption could result in unexpected errors, such as those exposed by #8931.

Modifications

Anywhere that the data passed to writeInbound() might be used again, ensure that either:
- A copy is used rather than wrapping a shared byte array, or
- The buffer is otherwise protected from modification by making it read-only

For the tests, copying is preferred since it still allows the "mutating" optimizations to be exercised.

Results

Avoid possible errors when pipeline assumes it has full control of input buffer.
2019-05-22 12:35:03 +02:00
Carl Mastrangelo
912aa7d867 Don't double record stacktrace in Annotated*Exception (#9117)
Motivation:
When initializing the AnnotatedSocketException in AbstractChannel, both
the cause and the stack trace are set, leaving a trailing "Caused By"
that is compressed when printing the trace.

Modification:
Don't include the stack trace in the exception, but leave it in the cause.

Result:
Clearer stack trace
2019-05-22 12:07:06 +02:00
Nick Hill
d29f91540d Fix possible unsafe sharing of internal NIO buffer in CompositeByteBuf (#9169)
Motivation

A small thread-safety bug was introduced during the internal
optimizations of ComponentByteBuf made a while back in #8437. When there
is a single component which was added as a slice,
internalNioBuffer(int,int) will currently return the unwrapped slice's
un-duplicated internal NIO buffer. This is not safe since it could be
modified concurrently with other usage of that parent buffer.

Modifications

Delegate internalNioBuffer to nioBuffer in this case, which returns a
duplicate. This matches what's done in derived buffers in general
(for the same reason). Add unit test.

Result

Fixed possible thread-safety bug
2019-05-22 11:07:27 +02:00
Fabien Renaud
73dae929e3 codec-memcache: copy metadata in binary full request response (#9160)
Motivations
-----------
Calling `copy()`, `duplicate()` or `replace()` on `FullBinaryMemcacheResponse`
or `FullBinaryMemcacheRequest` instances should copy status, opCode, etc.
that are defined in `AbstractBinaryMemcacheMessage`.

Modifications
-------------
 - Modified duplicate, copy and replace methods in
DefaultFullBinaryMemcacheRequest and DefaultFullBinaryMemcacheResponse
to always copy metadata from parent classes.
 - Unit tests verifying duplicate, copy and replace methods for
DefaultFullBinaryMemcacheRequest and DefaultFullBinaryMemcacheResponse
copy buffers and metadata as expected.

Result
------
Calling copy(), duplicate() or replace() methods on
DefaultFullBinaryMemcacheRequest or DefaultFullBinaryMemcacheResponse
produces valid copies with all expected metadata.

Fixes #9159
2019-05-22 11:06:16 +02:00
Norman Maurer
1176d10ebc Skip usage of Japicmp maven plugin for now
Motivation:

We did not release any version of netty 5 yet so we need to skip the usage of the Japicmp plugin.

Modifications:

Skip plugin execution

Result:

Build works
2019-05-22 11:04:52 +02:00
Julien Viet
f85583047e KQueueEventLoop | EpollEventLoop may incorrectly update registration when FD is reused.
Motivation:

The current KQueueEventLoop implementation does not process concurrent domain socket channel registration/unregistration in the order they actual
happen since unregistration are delated by an event loop task scheduling. When a domain socket is closed, it's file descriptor might be reused
quickly and therefore trigger a new channel registration using the same descriptor.

Consequently the KQueueEventLoop#add(AbstractKQueueChannel) method will overwrite the current inactive channels having the same descriptor
and the delayed KQueueEventLoop#remove(AbstractKQueueChannel) will remove the active channel that replaced the inactive one.

As active channels are registered, events for this file descriptor won't be processed anymore and the channels will never be closed.

The same problem can also happen in EpollEventLoop. Beside this we also may never remove the AbstractEpollChannel from the internal map
when it is unregistered which will prevent it from be GC'ed

Modifications:

- Change logic of native KQueue and Epoll implementations to ensure we correctly handle the case of FD reuse
- Only try to update kevent / epoll if the Channel is still open (as otherwise it will be handled by kqueue / epoll itself)
- Correctly remove AbstractEpollChannel from internal map in all cases
- Make implementation of closeAll() consistent for Epoll and KQueueEventLoop

Result:

KQueue and Epoll native transports correctly handle FD reuse

Co-authored-by: Norman Maurer <norman_maurer@apple.com>
2019-05-22 10:11:42 +02:00
Norman Maurer
bbb397ac5c Remove usage of io.netty.handler.ssl.openssl.useKeyManagerFactory system property
Motivation:

Usafe of io.netty.handler.ssl.openssl.useKeyManagerFactory system property was deprecated in 4.1 so let us remove it.

Modifications:

Remove io.netty.handler.ssl.openssl.useKeyManagerFactory usage.

Result:

Remove support of deprecated system property
2019-05-22 09:09:32 +02:00
Tim Brooks
e4ef8f6ff3 Prefer direct io buffers if direct buffers pooled (#9167)
Motivation

Direct buffers are normally preferred when interfacing with raw
sockets. Currently netty will only return direct io buffers (for reading
from a channel) when a platform has unsafe. However, this is
inconsistent with the write-side (filterOutboundMessage) where a direct
byte buffer will be returned if pooling is enabled. This means that
environments without unsafe (and no manual netty configurations) end up
with many pooled heap byte buffers for reading, many pooled direct byte
buffers for writing, and jdk pooled byte buffers (for reading).

Modifications

This commit modifies the AbstractByteBufAllocator to return a direct
byte buffer for io handling when the platform has unsafe or direct byte
buffers are pooled.

Result:

Use direct buffers when direct buffers are pooled for IO.
2019-05-22 07:33:06 +02:00
Norman Maurer
89e796cec6 Update to latest JDK13 EA release (#9166)
Motivation:

We should use the latest EA release when trying to compile with JDK13.

Modifications:

Update to latest release

Result:

Test with latest release on the CI
2019-05-21 20:10:34 +02:00
秦世成
a9159d28fb Format code to align unaligned code. (#9062)
Motivation:
Format code to align unaligned code.

Modification:
Reformat the code

Result:

Cleaner code
2019-05-20 12:07:45 +02:00
Norman Maurer
a65f231bc4 Only try to use reflection to access default nameservers when using Java8 and lower (#9157)
Motivation:

We should only try to use  reflection to access default nameservers when using Java8 and lower as otherwise we will produce an Illegal reflective access warning like:

WARNING: Illegal reflective access by io.netty.resolver.dns.DefaultDnsServerAddressStreamProvider

Modifications:

Add Java version check before try to use reflective access.

Result:

No more warning when Java9+ is used.
2019-05-18 08:21:59 +02:00
Norman Maurer
ed61e5f543 Only use static Exception instances when we can ensure addSuppressed … (#9152)
Motivation:

OOME is occurred by increasing suppressedExceptions because other libraries call Throwable#addSuppressed. As we have no control over what other libraries do we need to ensure this can not lead to OOME.

Modifications:

Only use static instances of the Exceptions if we can either dissable addSuppressed or we run on java6.

Result:

Not possible to OOME because of addSuppressed. Fixes https://github.com/netty/netty/issues/9151.
2019-05-17 22:42:53 +02:00
Norman Maurer
211dde4e08 Add support for TCP fallback when we receive a truncated DnsResponse (#9139)
Motivation:

Sometimes DNS responses can be very large which mean they will not fit in a UDP packet. When this is happening the DNS server will set the TC flag (truncated flag) to tell the resolver that the response was truncated. When a truncated response was received we should allow to retry via TCP and use the received response (if possible) as a replacement for the truncated one.

See https://tools.ietf.org/html/rfc7766.

Modifications:

- Add support for TCP fallback by allow to specify a socketChannelFactory / socketChannelType on the DnsNameResolverBuilder. If this is set to something different then null we will try to fallback to TCP.
- Add decoder / encoder for TCP
- Add unit tests

Result:

Support for TCP fallback as defined by https://tools.ietf.org/html/rfc7766 when using DnsNameResolver.
2019-05-17 14:47:10 +02:00
Norman Maurer
260a8a0e9e Add missing assume checks to skip tests if KeyManagerFactory can not be used (#9148)
Motivation:

Depending on what OpenSSL library version we use / system property that is set we need to skip tests that use KeyManagerFactory.

Modifications:

Add missing assume checks for tests that use KeyManagerFactory.

Result:

All tests pass even if KeyManagerFactory is not supported
2019-05-15 07:25:03 +02:00
RoganDawes
0a1786c32c Remove the Handler only after it has initialized the channel (#9132)
Motivation:

Previously, any 'relative' pipeline operations, such as
ctx.pipeline().replace(), .addBefore(), addAfter(), etc
would fail as the handler was not present in the pipeline.

Modification:

Used the pattern from ChannelInitializer when invoking configurePipeline().

Result:

Fixes #9131
2019-05-13 13:55:17 +02:00
Nick Hill
dd9e0d5a19 AsciiString.lastIndexOf(...) is implemented incorrectly (#9103)
Motivation

@xiaoheng1 reported incorrect behaviour of AsciiString.lastIndexOf in
#9099. Upon closer inspection it appears that it was never implemented
correctly and searches between the provided index and the end of the
string similar to indexOf(...), rather than between the provided index
and the beginning of the string as the javadoc states (and in line with
java.lang.String).

Modifications

Fix AsciiString.lastIndexOf implementation and corresponding unit tests
to behave the same as the equivalent String methods.

Result

Fixes #9099
2019-05-13 07:03:51 +02:00
Nick Hill
5e786e549e Fix incorrect behavior of ReadOnlyByteBufferBuf.getBytes(int,ByteBuffer) (#9125)
* Fix incorrect behavior of ReadOnlyByteBufferBuf.getBytes(int,ByteBuffer)

Motivation

It currently will succeed when the destination is larger than the source
range, but the ByteBuf javadoc states this should be a failure, as is
the case with all the other implementations.

Modifications

- Fix logic to fail the bounds check in this case
- Remove explicit null check which isn't done in any equivalent method
- Add unit test

Result

More correct/consistent behaviour
2019-05-13 07:03:10 +02:00
Norman Maurer
debc1cbc0a DnsNameResolver.resolveAll(DnsQuestion) should not try to filter duplicates (#9141)
Motivation:

https://github.com/netty/netty/pull/9021 did apply some changes to filter out duplicates InetAddress when calling resolveAll(...) to mimic JDK behaviour. Unfortunally this also introduced a regression as we should not filter duplicates when the user explicit calls resolveAll(DnsQuestion).

Modifications:

- Only filter duplicates if resolveAll(String) is used
- Add unit test

Result:

Fixes regressions introduces by https://github.com/netty/netty/pull/9021
2019-05-13 06:59:28 +02:00
SplotyCode
d3a13a0d6a Allow to specify KeyStore type in SslContext (#9003)
Motivation:

As brought up in https://github.com/netty/netty/issues/8998, JKS can be substantially faster than pkcs12, JDK's new default. Without an option to set the KeyStore type you must change the configuration of the entire JVM which is impractical.

Modification:

- Allow to specify KeyStore type
- Add test case

Result:

Fixes https://github.com/netty/netty/issues/8998.
2019-05-10 07:51:20 +02:00
Norman Maurer
83210c47ca Allow to have DnsNameResolver.resolveAll(...) notify as soon as the preferred records were resolved (#9136)
Motivation:

075cf8c02e introduced a change to allow resolve(...) to notify as soon as the preferred record was resolved. This works great but we should also allow the user to configure that we want to do the same for resolveAll(...), which means we should be able to notify as soon as all records for a preferred record were resolved.

Modifications:

- Add a new DnsNameResolverBuilder method to allow configure this (use false as default to not change default behaviour)
- Add unit test

Result:

Be able to speed up resolving.
2019-05-09 08:08:47 +02:00
Andrey Mizurov
a69ae1aced Fixed HttpHelloWorldServerHandler for handling HTTP 1.0/1.1 (#9124)
Motivation:

HttpHelloWorldServer example works incorrect for HTTP 1.1, the value of header connection is always set to close for each request.

Modification:

Correctly set header

Result:

Fixed HttpHelloWorldServerHandler for handling HTTP 1.0/1.1
2019-05-08 09:05:18 +02:00
Anuraag Agrawal
b45a1cd15b Add equality check to contentEquals instance methods. (#9130)
Motivation:

An instance is always equal to itself. It makes sense to skip processing for this case, which isn't uncommon since `AsciiString` is often memoized within an application when used as HTTP header names.

Modification:

`contentEquals` methods first check for instance equality before doing processing.

Result:

`contentEquals` will be faster when comparing an instance with itself.

I couldn't find any unit tests for these methods, only the static version. Let me know if I should add something to `AsciiStringCharacterTest`.

Came up here:
https://github.com/line/armeria/pull/1731#discussion_r280396280
2019-05-08 07:31:09 +02:00
Norman Maurer
b1f8f223f2 Always include classes from all native transports no matter on which platfrom netty-all is build (#9111)
Motivation:

While building netty-all we should always include all classes for native transports no matter if the native part can be build or not. This was it is easier to test locally with a installed snapshot of netty-all when the code that uses it does enable a specific native transport depending on if the native bits can be loaded or not.

Modifications:

Always include classes of native transports no matter on which platfrom we build. When a release is done we ensure we include the native bits by using the uber-staging profile.

Result:

Easier testing with netty-all snapshots.
2019-04-30 23:24:14 +02:00
Norman Maurer
c06ca367ca Introduce DynamicAddressConnectHandler which can be used to dynamically change remoteAddress / localAddress when a connect is issued (#8982)
Motivation:

Bootstrap allows you to set a localAddress for outbound TCP connections, either via the Bootstrap.localAddress(localAddress) or Bootstrap.connect(remoteAddress, localAddress) methods. This works well if you want to bind to just one IP address on an interface. Sometimes you want to bind to a specific address based on the resolved remote address which should be possible.

Modifications:

Add DynamicAddressConnectHandler and tests

Result:

Fixes https://github.com/netty/netty/issues/8940.
2019-04-30 07:59:40 +02:00
Ilya Maykov
1cd9f5e17b [openssl] fix refcount bug in OpenSslPrivateKeyMaterial ctor
Motivation:

Subclasses of `OpenSslKeyMaterial` implement `ReferenceCounted`. This means that a new object should have an initial refcount of 1. An `OpenSslPrivateKey.OpenSslPrivateKeyMaterial` object shares its refcount with the enclosing `OpenSslPrivateKey` object. This means the enclosing object's refcount must be incremented by 1 when an instance of `OpenSslPrivateKey.OpenSslPrivateKeyMaterial` is created. Otherwise, when the key material object is `release()`-ed, the refcount on the enclosing object will drop to 0 while it is still in use.

Modification:

- Increment the refcount in the constructor of `OpenSslPrivateKey.OpenSslPrivateKeyMaterial`
- Ensure we also always release the native certificates as well.

Result:

Refcount is now correct.
2019-04-29 23:11:53 +02:00
Norman Maurer
a4b05694d3 Http2MultiplexCodec.DefaultHttp2StreamChannel should handle ChannelConfig.isAutoClose() in a consistent way as AbstractChannel (#9108)
Motivation:

Http2MultiplexCodec.DefaultHttp2StreamChannel currently only act on ClosedChannelException exceptions when checking for isAutoClose(). We should widen the scope here to IOException to be more consistent with AbstractChannel.

Modifications:

Replace instanceof ClosedChannelException with instanceof IOException

Result:

More consistent handling of isAutoClose()
2019-04-29 18:50:44 +02:00
Norman Maurer
6de098bae8 Adjust pom.xml to be able to build with graalvm (#9107)
Motivation:

When trying to use graalvm and build netty we currently fail because our build configuration is not compatible with it.

Modification:

- Skip plugins that are not supported when graal is used
- Correctly configure surefire plugin for graal so it not produces a NPE

Result:

We can build and test with graalvm.
2019-04-29 18:40:47 +02:00
Paulo Lopes
f5c0d2eb56 Add SVM metadata and minimal substitutions to build graalvm native image applications. (#8963)
Motivation:

GraalVM native images are a new way to deliver java applications. Netty is one of the most popular libraries however there are a few limitations that make it impossible to use with native images out of the box. Adding a few metadata (in specific modules will allow the compilation to success and produce working binaries)

Modification:

Added properties files in `META-INF` and substitutions classes (under `internal.svm`) will solve the compilation issues. The substitutions classes are not visible and do not have a public constructor so they are not visible to end users.

Result:

Fixes #8959

This fix is very conservative as it applies the minimum config required to build:

* pure netty servers
* vert.x applications
* grpc applications

The build is having trouble due to checkstyle which does not seem to be able to find the copyright notice on property files.
2019-04-29 09:12:09 +02:00
Norman Maurer
0a8a52a9f1 Add docker-compose file to compile / test with graalvm (#9072)
Motivation:

We should try to compile / test with graalvm as well.

Modifications:

Add docker-compose file for graalvm

Result:

Be able to also compile / test with graalvm
2019-04-29 08:33:50 +02:00
Norman Maurer
aa0451d26d Fix flaky GlobalEventExecutorTest.* (#9074)
Motivation:

In GlobalEventExecutorTest we used Thread.sleep(...) which can produce flaky results (as seen on the CI). We should use another alternative during tests.

Modifications:

Replace Thread.sleep(...) with join()

Result:

No more flaky GlobalEventExecutor tests.
2019-04-29 08:33:18 +02:00
Norman Maurer
e3783c2332 Update to latest java releases (#9101)
Motivation:

There were new releases of various Java versions.

Modifications:

Adjust used java versions of the latest releases and so use these on our CI

Result:

Use latest java versions on our CI.
2019-04-29 08:32:39 +02:00
Norman Maurer
795fa8aef1 Throw SignatureException if OpenSslPrivateKeyMethod.* return null to prevent segfault (#9100)
Motivation:

While OpenSslPrivateKeyMethod.* should never return null we should still guard against it to prevent any possible segfault.

Modifications:

- Throw SignatureException if null is returned
- Add unit test

Result:

No segfault when user returns null.
2019-04-29 08:31:56 +02:00
Scott Mitchell
67518e306f Http2ConnectionHandler to allow decoupling close(..) from GOAWAY graceful close (#9094)
Motivation:
Http2ConnectionHandler#close(..) always runs the GOAWAY and graceful close
logic. This coupling means that a user would have to override
Http2ConnectionHandler#close(..) to modify the behavior, and the
Http2FrameCodec and Http2MultiplexCodec are not extendable so you cannot
override at this layer. Ideally we can totally decouple the close(..) of the
transport and the GOAWAY graceful closure process completely, but to preserve
backwards compatibility we can add an opt-out option to decouple where the
application is responsible for sending a GOAWAY with error code equal to
NO_ERROR as described in https://tools.ietf.org/html/rfc7540#section-6.8 in
order to initiate graceful close.

Modifications:
- Http2ConnectionHandler supports an additional boolean constructor argument to
opt out of close(..) going through the graceful close path.
- Http2FrameCodecBuilder and Http2MultiplexCodec expose
 gracefulShutdownTimeoutMillis but do not hook them up properly. Since these
are already exposed we should hook them up and make sure the timeout is applied
properly.
- Http2ConnectionHandler's goAway(..) method from Http2LifecycleManager should
initiate the graceful closure process after writing a GOAWAY frame if the error
code is NO_ERROR. This means that writing a Http2GoAwayFrame from
Http2FrameCodec will initiate graceful close.

Result:
Http2ConnectionHandler#close(..) can now be decoupled from the graceful close
process, and immediately close the underlying transport if desired.
2019-04-28 17:55:44 -07:00
Scott Mitchell
306a855d93 DefaultHttp2ConnectionEncoder async SETTINGS ACK SimpleChannelPromiseAggregator promise usage
Motivaiton:
DefaultHttp2ConnectionEncoder uses SimpleChannelPromiseAggregator to combine two
operations into a single future status. However it directly uses the
SimpleChannelPromiseAggregator object instead of using the newPromise() method
in one case. This may result in premature completion of the aggregated future.

Modifications:
- DefaultHttp2ConnectionEncoder to use
  SimpleChannelPromiseAggregator#newPromise() instead of directly using the
SimpleChannelPromiseAggregator instance when writing the settings ACK frame

Result:
More correct status for the SETTING ACK frame writing when auto settings ACK is
disabled.
2019-04-28 11:48:24 +02:00
Scott Mitchell
29661fdc96 HTTP/2 to support asynchronous SETTINGS ACK (#9069)
Motivation:
The HTTP/2 codec will synchronously respond to a SETTINGS frame with a SETTINGS
ACK before the application sees the SETTINGS frame. The application may need to
adjust its state depending upon what is in the SETTINGS frame before applying
the remote settings and responding with an ACK (e.g. to adjust for max
concurrent streams). In order to accomplish this the HTTP/2 codec should allow
for the application to opt-in to sending the SETTINGS ACK.

Modifications:
- DefaultHttp2ConnectionDecoder should support a mode where SETTINGS frames can
  be queued instead of immediately applying and ACKing.
- DefaultHttp2ConnectionEncoder should attempt to poll from the queue (if it
  exists) to apply the earliest received but not yet ACKed SETTINGS frame.
- AbstractHttp2ConnectionHandlerBuilder (and sub classes) should support a new
  option to enable the application to opt-in to managing SETTINGS ACK.

Result:
HTTP/2 allows for asynchronous SETTINGS ACK managed by the application.
2019-04-28 11:47:50 +02:00
Nick Hill
ddfe888173 Ensure channel handler close() is not skipped in !hasDisconnect case (#9098)
Motivation

The optimization in #8988 didn't correctly handle the specific case
where the channel hasDisconnect == false, and a
ChannelOutboundHandlerAdapter subclass overrides only the close(ctx,
promise) method without also overriding the disconnect(ctx, promise)
method.

Modifications

Adjust AbstractChannelHandler.disconnect(...) method to divert to
close(...) in !hasDisconnect case before computing target context for
the event.

Result

Fixes #9092
2019-04-28 10:47:25 +02:00
Scott Mitchell
3c11af965a DefaultHeaders#valueIterator doesn't remove from the in bucket list (#9090)
Motivation:
DefaultHeaders entries maintains two linked lists. 1 for overall insertion order
and 1 for "in bucket" order. DefaultHeaders#valueIterator removal (introduced in 1d9090aab2) only reliably
removes the entry from the overall insertion order, but may not remove from the
bucket unless the element is the first entry.

Modifications:
- DefaultHeaders$ValueIterator should track 2 elements behind the next entry so
that the single linked "in bucket" list can be patched up when removing the
previous entry.

Result:
More correct DefaultHeaders#valueIterator removal.
2019-04-27 21:16:04 +02:00