Commit Graph

9684 Commits

Author SHA1 Message Date
Norman Maurer
36b2477c30 Cleanup http2 example code to make clear it is fine to just use ctx directly. (#9276)
Motivation:

In our example we did use pipeline.context(this) to obtain the context of the handler while it was already passed in via ctx. This could confuse users and give the impression that the context is no the same.

Modifications:

Just use ctx directly.

Result:

Fix confusion in example code. This was brought up on stackoverflow:

https://stackoverflow.com/questions/56711128/when-is-a-channelhandlercontext-handed-to-a-channelhandler-not-that-channelhandl
2019-06-24 21:08:36 +02:00
Julien Viet
2d72a7c561 Preserve the original filename when encoding a multipart/form in mixed mode. (#9270)
Motivation:

The HttpPostRequestEncoder overwrites the original filename of file uploads sharing the same name encoded in mixed mode when it rewrites the multipart body header of the previous file. The original filename should be preserved instead.

Modifications:

Change the HttpPostRequestEncoder to reuse the correct filename when the encoder switches to mixed mode. The original test is incorrect and has been modified too, in addition it tests with an extra file upload since the current test was not testing the continuation of a mixed mode.

Result:

The HttpPostRequestEncoder will preserve the original filename of the first fileupload when switching to mixed mode
2019-06-24 10:40:39 +02:00
秦世成
dd6ba294e0 Fixed the haproxy message mem leak issue (#9250)
Motivation:

HAProxyMessage should be released as it contains a list of TLV which hold a ByteBuf, otherwise, it may cause memory leaks.

Modification:

- Let HAProxyMessage extend AbstractReferenceCounted
- Adjust tests.

Result:

Fixes #9201
2019-06-24 10:39:28 +02:00
Norman Maurer
bf72d6d2d9 Split multiplexing from frame decoding to allow easier customization of frame processing and better seperation of responsibilities (#9239)
Motivation:

In the past we had the following class hierarchy:

Http2ConnectionHandler --- Http2FrameCodec -- Http2MultiplexCodec

This hierarchy makes it impossible to plug in any code that would like to act on Http2Frame and Http2StreamFrame which can be quite useful for various situations (like metrics, logging etc). Beside this it also made the implementtion very hacky. To allow easier maintainance and also allow more flexible costumizations we should split Http2MultiplexCodec and Http2FrameCode.

Modifications:

- Introduce Http2MultiplexHandler (which is a replacement for Http2MultiplexCodec when used together with Http2FrameCodec)
- Mark Http2MultiplexCodecBuilder and Http2MultiplexCodec as deprecated. People should use Http2FrameCodecBuilder / Http2FrameCodec together with Http2MultiplexHandlder in the future
- Adjust / Add tests
- Adjust examples

Result:

More flexible usage possible and less hacky / coupled implementation for http2 multiplexing
2019-06-24 10:26:12 +02:00
ursa
a05adceae8 Bugfix #9257: WebSocketProtocolHandler does NOT support autoRead=false (#9258)
Motivation:

I need to control WebSockets inbound flow manually, when autoRead=false

Modification:

Add missed ctx.read() call into WebSocketProtocolHandler, where read request has been swallowed.

Result:

Fixes #9257
2019-06-24 09:08:30 +02:00
Norman Maurer
9b37be9550 Recycle RecyclableArrayDeque as fast as possible in FlowControlHandler (#9263)
Motivation:

FlowControlHandler does use a recyclable ArrayDeque internally but only recycles it when the channel is closed. We should better recycle it once it is empty.

Modifications:

Recycle the deque as fast as possible

Result:

Less RecyclableArrayDeque instances.
2019-06-22 07:27:29 +02:00
Alex Blewitt
99034a15b5 Return the result of the list.recycle() call (#9264)
Motivation:

Resolve the issue highlighted by SpotJMHBugs that the creation of the RecyclableArrayList may be elided by the JIT since the result isn't consumed or returned.

Modifications:

Return the result of `list.recycle()` so that the list isn't elided.

Result:

The JMH benchmark shows a change in performance indicating that the prior results of this may be unsound.
2019-06-22 07:21:51 +02:00
Nick Hill
c61fa9be70 Subsequence versions of ByteBufUtil#writeUtf8(...) methods (#9224)
Motivation

It would be useful to be able to write UTF-8 encoded subsequence of
CharSequence characters to a ByteBuf without needing to create a
temporary object via CharSequence#subSequence().

Modification

Add overloads of ByteBufUtil writeUtf8, reserveAndWriteUtf8 and
utf8Bytes methods which take explicit subsequence bounds.

Result

More efficient writing of substrings to byte buffers possible
2019-06-21 14:06:34 +02:00
Norman Maurer
780536ba72 Fix flaky DnsNameResolverTest.testTruncatedWithTcpFallback (#9262)
Motivation:

testTruncatedWithTcpFallback was flacky as we may end up closing the socket before we could read all data. We should only close the socket after we succesfully read all data.

Modifications:

Move socket.close() to finally block

Result:

Fix flaky test and so make the CI more stable again.
2019-06-21 09:29:15 +02:00
Nick Hill
ea2e4a7b3d De-duplicate PooledByteBuf implementations (#9120)
Motivation

There's quite a lot of duplicate/equivalent logic across the various
concrete ByteBuf implementations. We could take this even further but
for now I've focused on the PooledByteBuf sub-hierarchy.

Modifications

- Move common logic/methods into existing PooledByteBuf abstract
superclass
- Shorten PooledByteBuf.capacity(int) method implementation

Result

Less code to maintain
2019-06-19 20:50:57 +02:00
Kevin Oliver
750bd9f7b7 codec-http2: Lazily translate cookies for HTTP/1 (#9251)
Motivation:

For HTTP/2 messages with multiple cookies HttpConversionUtil.addHttp2ToHttpHeaders spends a good portion of time creating throwaway StringBuilders.

Modification:

Handle cookies lazily by using a ThreadLocal StringBuilder and then converting it to the H1 header at the end.

Result:

Less allocations.
2019-06-19 11:04:38 +02:00
Norman Maurer
e3be64fd9c Try to mark child channel writable again once the parent channel becomes writable (#9254)
Motivation:

f945a071db decoupled the writability state from the flow controller but could lead to the situation of a lot of writability updates events were propagated to the child channels. This change ensure we only take into account if the parent channel becomes writable again before we try to set the child channels to writable.

Modifications:

Only listen for channel writability changes for if the parent channel becomes writable again.

Result:

Less writability updates.
2019-06-18 20:31:18 +02:00
ursa
d9db218291 WebSocket is closed without an error on protocol violations (#9116)
Motivation:

Incorrect WebSockets closure affects our production system.
Enforced 'close socket on any protocol violation' prevents our custom termination sequence from execution.
Huge number of parameters is a nightmare both in usage and in support (decoders configuration).

Modification:

- Fix violations handling - send proper response codes.
- Fix for messages leak.
- Introduce decoder's option to disable default behavior (send close frame) on protocol violations.
- Encapsulate WebSocket response codes - WebSocketCloseStatus.
- Encapsulate decoder's configuration into a separate class - WebSocketDecoderConfig.

Result:

Fixes #8295.
2019-06-18 10:13:05 +02:00
Norman Maurer
aaf5ec1fbb Writability state of http2 child channels should be decoupled from the flow-controller (#9235)
Motivation:

We should decouple the writability state of the http2 child channels from the flow-controller and just tie it to its own pending bytes counter that is decremented by the parent Channel once the bytes were written.

Modifications:

- Decouple writability state of child channels from flow-contoller
- Update tests

Result:

Less coupling and more correct behavior. Fixes https://github.com/netty/netty/issues/8148.
2019-06-18 10:03:01 +02:00
Frédéric Brégier
1a487a0ff9 Change Scheduled to FixedRate in Traffic Counter (#9245)
Motivation:

Traffic shaping needs more accurate execution than scheduled one. So the
use of FixedRate instead.
Moreover the current implementation tends to create as many threads as
channels use a ChannelTrafficShapingHandlern, which is unnecessary.

Modifications:

Change the executor.schedule to executor.scheduleAtFixedRate in the
start and remove the reschedule call from run monitor thread since it
will be restarted by the Fixed rate executor.
Also fix a minor bug where restart was only doing start() without stop()
before.

Result:

Threads are more stable in number of cached and precision of traffic
shaping is enhanced.
2019-06-18 09:35:10 +02:00
Aleksey Yeschenko
09158a6420 Fix LZ4 encoder/decoder performance with (default) xxHash32 (#9249)
Motivation:

Lz4FrameEncoder and Lz4FrameDecoder in their default configuration use
an extremely inefficient way to checksum direct byte buffers. In
particular, for every byte checksummed, a single-element byte array is
being allocated and a JNI cal is made, which in some internal testing
makes a 25x difference in total throughput and allocates *a lot* of
garbage.

Modifications:

Lz4XXHash32, an implementation of ByteBufChecksum specifically for use
by Lz4FrameEncoder and Lz4FrameDecoder, is introduced. It utilises
xxHash32 block API which provides a hash() method that accepts a
ByteBuffer as an argument. Lz4FrameEncoder and Lz4FrameDecoder are
modified to use this implementation by default.

Result:

Lz4FrameEncoder and Lz4FrameDecoder perform well again when operating
on direct byte buffers with default checksum configuration; a public
implementation is provided for those who need to override the seed.
2019-06-18 09:33:25 +02:00
Aleksey Yeschenko
6390c6f2f6 Fix ReflectiveByteBufChecksum with direct buffers (#9244)
Motivation:

ReflectiveByteBufChecksum#update(buf, off, len) ignores provided offset
and length arguments when operating on direct buffers, leading to wrong
byte sequences being checksummed and ultimately incorrect checksum
values (unless checksumming the entire buffer).

Modifications:

Use the provided offset and length arguments to get the correct nio
buffer to checksum; add test coverage exercising the four meaningfully
different offset and length combinations.

Result:

Offset and length are respected and a correct checksum gets calculated;
simple unit test should prevent regressions in the future.
2019-06-17 20:38:06 +02:00
Scott Mitchell
bf7f41a993 SslHandler to fail handshake and pending writes if non-application write fails (#9240)
Motivation:
SslHandler must generate control data as part of the TLS protocol, for example
to do handshakes. SslHandler doesn't capture the status of the future
corresponding to the writes when writing this control (aka non-application
data). If there is another handler before the SslHandler that wants to fail
these writes the SslHandler will not detect the failure and we must wait until
the handshake timeout to detect a failure.

Modifications:
- SslHandler should detect if non application writes fail, tear down the
channel, and clean up any pending state.

Result:
SslHandler detects non application write failures and cleans up immediately.
2019-06-16 07:45:25 +02:00
Aleksey Yeschenko
db1e662933 Fix ByteBufChecksum optimisation for CRC32 and Adler32 (#9242)
Motivation:

Because of a simple bug in ByteBufChecksum#updateByteBuffer(Checksum),
ReflectiveByteBufChecksum is never used for CRC32 and Adler32, resulting
in direct ByteBuffers being checksummed byte by byte, which is
undesriable.

Modification:

Fix ByteBufChecksum#updateByteBuffer(Checksum) method to pass the
correct argument to Method#invoke(Checksum, ByteBuffer).

Result:

ReflectiveByteBufChecksum will now be used for Adler32 and CRC32 on
Java8+ and direct ByteBuffers will no longer be checksummed on slow
byte-by-byte basis.
2019-06-16 07:37:52 +02:00
Norman Maurer
acd13e1cf7 Allow to set parent Channel when constructing EmbeddedChannel (#9230)
Motivation:

Sometimes it is beneficial to be able to set a parent Channel in EmbeddedChannel if the handler that should be tested depend on the parent.

Modifications:

- Add another constructor which allows to specify a parent
- Add unit tests

Result:

Fixes https://github.com/netty/netty/issues/9228.
2019-06-08 09:20:19 -07:00
Stephane Landelle
576b1768f4 Introduce WebSocketClientHandshaker::absoluteUpgradeUrl, close #9205 (#9206)
Motivation:

When connecting through an HTTP proxy over clear HTTP, user agents must send requests with an absolute url. This hold true for WebSocket Upgrade request.

WebSocketClientHandshaker and subclasses currently always send requests with a relative url, which causes proxies to crash as request is malformed.

Modification:

Introduce a new parameter `absoluteUpgradeUrl` and expose it in constructors and WebSocketClientHandshakerFactory.

Result:

It's now possible to configure WebSocketClientHandshaker so it works properly with HTTP proxies over clear HTTP.
2019-06-07 16:01:49 -07:00
yipulash
68f2242ac4 delete Other "Content-" MIME Header Fields exception (#9122)
delete Other "Content-" MIME Header Fields exception

Motivation:

RFC7578 4.8. Other "Content-" Header Fields

The multipart/form-data media type does not support any MIME header
fields in parts other than Content-Type, Content-Disposition, and (in
limited circumstances) Content-Transfer-Encoding. Other header
fields MUST NOT be included and MUST be ignored.

Modification:

Ignore other Content types.

Result: 

Other "Content-" Header Fields should be ignored no exception
2019-06-07 13:51:50 -07:00
Norman Maurer
9d5420987a Add support for loopbackmode and accessing the configured interface when using epoll native transport with multicast (#9218)
Motivation:

We did not have support for enable / disable loopback mode in our native epoll transport and also missed the implemention to access the configured interface.

Modifications:

Add implementation and adjust test to cover it

Result:

More complete multicast support with native epoll transport
2019-06-07 13:45:45 -07:00
Carl Mastrangelo
c27fbb5ff2 Handle missing methods on ChannelHandlerMask (#9221)
Motivation:

When Netty is run through ProGuard, seemingly unused methods are removed.  This breaks reflection, making the Handler skipping throw a reflective error.

Modification:

If a method is seemingly absent, just disable the optimization.

Result:

Dealing with ProGuard sucks infinitesimally less.
2019-06-07 13:42:56 -07:00
Scott Mitchell
e0bfc4f91f HTTP/2 avoid closing connection when writing GOAWAY (#9227)
Motivation:
b4e3c12b8e introduced code to avoid coupling
close() to graceful close. It also added some code which attempted to infer when
a graceful close was being done in writing of a GOAWAY to preserve the
"connection is closed when all streams are closed behavior" for the child
channel API. However the implementation was too overzealous and may preemptively
close the connection if there are not currently any open streams (and close if
there are any frames which create streams in flight).

Modifications:
- Decouple writing a GOAWAY from trying to infer if a graceful close is being
  done and closing the connection. Even if we could enhance this logic (e.g.
wait to close until the second GOAWAY with no error) it is possible the user
doesn't want the connection to be closed yet. We can add a means for the codec
to orchestrate the graceful close in the future (e.g. write some special "close
the connection when all streams are closed") but for now we can just let the
application handle this.

Result:
Fixes https://github.com/netty/netty/issues/9207
2019-06-06 17:44:43 -07:00
Carl Mastrangelo
f01278616a Properly debounce wakeups (#9191)
Motivation:
The wakeup logic in EpollEventLoop is overly complex

Modification:
* Simplify the race to wakeup the loop
* Dont let the event loop wake up itself (it's already awake!)
* Make event loop check if there are any more tasks after preparing to
sleep.  There is small window where the non-eventloop writers can issue
eventfd writes here, but that is okay.

Result:
Cleaner wakeup logic.

Benchmarks:

```
BEFORE
Benchmark                                   Mode  Cnt       Score      Error  Units
EpollSocketChannelBenchmark.executeMulti   thrpt   20  408381.411 ± 2857.498  ops/s
EpollSocketChannelBenchmark.executeSingle  thrpt   20  157022.360 ± 1240.573  ops/s
EpollSocketChannelBenchmark.pingPong       thrpt   20   60571.704 ±  331.125  ops/s

Benchmark                                   Mode  Cnt       Score      Error  Units
EpollSocketChannelBenchmark.executeMulti   thrpt   20  440546.953 ± 1652.823  ops/s
EpollSocketChannelBenchmark.executeSingle  thrpt   20  168114.751 ± 1176.609  ops/s
EpollSocketChannelBenchmark.pingPong       thrpt   20   61231.878 ±  520.108  ops/s
```
2019-06-04 05:27:15 -07:00
EliyahuStern
5dacc5f3da Resolve the pid field in PeerCredentials of KQueueDomainSocketChannels. (#9219)
Motivation:

This resolves a TODO from the initial transport-native-kqueue implementation, supplying the user with the pid of the local peer client/server process.

Modification:

Inside netty_kqueue_bsdsocket_getPeerCredentials, Call getsockopt with LOCAL_PEERPID and pass it to PeerCredentials constructor.
Add a test case in KQueueSocketTest.

Result:

PeerCredentials now have pid field set. Fixes https://github.com/netty/netty/issues/9213
2019-06-04 05:16:11 -07:00
Jon Chambers
26275e8a3d Close delegate resolver from RoundRobinInetAddressResolver (#9214)
Motivation:

RoundRobinDnsAddressResolverGroup ultimately opens UDP
ports for DNS resolution. Callers likely expect that
RoundRobinDnsAddressResolverGroup#close() will close those
ports, but that is not currently true (see #9212).

Modifications:

Overrode RoundRobinInetAddressResolver#close() to close
the delegate name resolver, which in turn closes any UDP
ports used for name resolution.

Result:

RoundRobinDnsAddressResolverGroup#close() closes UDP ports
as expected. This fixes #9212.
2019-06-04 05:14:28 -07:00
Nick Hill
4ba75b99af De-duplicate UnpooledDirectByteBuf/UnpooledUnsafeDirectByteBuf (#9085)
Motivation

While digging around looking at something else I noticed that these
share a lot of logic and it would be nice to reduce that duplication.

Modifications

Have UnpooledUnsafeDirectByteBuf extend UnpooledDirectByteBuf and make
adjustments to ensure existing behaviour remains unchanged.

The most significant addition needed to UnpooledUnsafeDirectByteBuf was
re-overriding the getPrimitive/setPrimitive methods to revert back to
the AbstractByteBuf versions which include bounds checks
(UnpooledDirectByteBuf excludes these as an optimization, relying on
those done by underlying ByteBuffer).

Result

~200 fewer lines, less duplicate logic.
2019-06-03 13:15:43 +02:00
Norman Maurer
4b8db65b16 Allow null sender when using DatagramPacketEncoder (#9204)
Motivation:

It is valid to use null as sender so we should support it when DatagramPacketEncoder checks if it supports the message.

Modifications:

- Add null check
- Add unit test

Result:

Fixes https://github.com/netty/netty/issues/9199.
2019-06-03 08:44:48 +02:00
Norman Maurer
a63584715f ByteToMessageDecoder.handlerRemoved(...) should only call fireChannelReadComplete() if fireChannelRead(...) was called before (#9211)
Motivation:

At the moment ByteToMessageDecoder always calls fireChannelReadComplete() when the handler is removed from the pipeline and the cumulation buffer is not null. We should only call it when we also call fireChannelRead(...), which only happens if the cumulation buffer is not null and readable.

Modifications:

Only call fireChannelReadComplete() if fireChannelRead(...) is called before during removal of the handler.

Result:

More correct semantics
2019-06-03 08:44:06 +02:00
Idel Pivnitskiy
b5ff56b6ee Make UnpooledUnsafeHeapByteBuf class public (#9184)
Motivation:

1. Users will be able to use an optimized version of
`UnpooledHeapByteBuf` and override behavior of methods if required.
2. Consistency with `UnpooledDirectByteBuf`, `UnpooledHeapByteBuf`, and
`UnpooledUnsafeDirectByteBuf`.

Modifications:

- Add `public` access modifier to `UnpooledUnsafeHeapByteBuf` class and
ctor;

Result:

Public access for optimized version of `UnpooledHeapByteBuf`.
2019-05-31 07:04:52 +02:00
Norman Maurer
63f7854c1e Don't read from timerfd and eventfd on each EventLoop tick (#9192)
Motivation:

We do not need to issue a read on timerfd and eventfd when the EventLoop wakes up if we register these as Edge-Triggered. This removes the overhead of 2 syscalls and so helps to reduce latency.

Modifications:

- Ensure we register the timerfd and eventfd with EPOLLET flag
- If eventfd_write fails with EAGAIN, call eventfd_read and try eventfd_write again as we only use it as wake-up mechanism.

Result:

Less syscalls and so reducing overhead.

Co-authored-by: Carl Mastrangelo <carl@carlmastrangelo.com>
2019-05-31 07:00:44 +02:00
SplotyCode
5b654fb4f7 Fixed toString() exception in MqttSubscribePayload and MqttUnsubscribePayload (#9202)
Motivation:
The toString() methods of MqttSubscribePayload and MqttUnsubscribePayload are causing exceptions when no topics are set.

Modification:
The toString() methods will not throw Excpetions anymore.

Result:
Fixes #9197
2019-05-31 06:48:21 +02:00
Nick Hill
ed23cfae5e Simplify SingleThreadEventExecutor.awaitTermination() implementation (#9081)
Motivation

A Semaphore is currently dedicated to this purpose but a simple
CountDownLatch will do.

Modification

Remove private threadLock Semaphore from SingleThreadEventExecutor and just use a CountDownLatch.

Also eliminate use of PlatformDependent.throwException() in startThread
method, and combine some nested if clauses.

Result

Cleaner EventLoop termination notification.
2019-05-27 16:09:41 +02:00
Norman Maurer
be7c808ce1 Set the HOST header in Http2ClientInitializer when trying to start an upgrade request (#9177)
Motivation:

The io.netty.example.http2.helloworld.client.Http2Client example should work in the h2c (HTTP2 cleartext - non-TLS) mode, which is the default for this example unless you set a -Dssl VM param. As we do not set the HOST header some servers do reject the upgrade request.

Modifications:

Set the HOST header

Result:

Fixes https://github.com/netty/netty/issues/9115.
2019-05-27 16:04:04 +02:00
Nick Hill
1e575d7389 Fix redundant or missing checks and other inconsistencies in ByteBuf impls (#9119)
Motivation

There are a few minor inconsistencies / redundant operations in the
ByteBuf implementations which would be good to fix.

Modifications

- Unnecessary ByteBuffer.duplicate() performed in
CompositeByteBuf.nioBuffer(int,int)
- Add missing checkIndex(...) check to
ReadOnlyByteBufferBuf.nioBuffer(int,int)
- Remove duplicate bounds check in
ReadOnlyByteBufferBuf.getBytes(int,byte[],int,int)
- Omit redundant bounds check in
UnpooledHeapByteBuf.getBytes(int,ByteBuffer)

Result

More consistency and slightly less overhead
2019-05-27 15:32:42 +02:00
Norman Maurer
f966a0a041 Correctly detect InternetProtocolFamily when EpollDatagramChannel is created with existing FileDescriptor (#9185)
Motivation:

When EpollDatagramChannel is created with an existing FileDescriptor we should detect the correct InternetProtocolFamily.

Modifications:

Obtain the InternetProtocolFamily from the given FD

Result:

Use correct InternetProtocolFamily when EpollDatagramChannel is created via existing FileDescriptor
2019-05-26 20:30:05 +02:00
Steve Buzzard
33d1a91083 Added UDP multicast (with caveats: getInterface, getNetworkInterface, block or loopback-mode-disabled operations).
Motivation:

Provide epoll/native multicast to support high load multicast users (we are using it for a high load telecomm app at my day job).

Modification:

Added support for source specific and any source multicast for epoll transport. Some caveats: no support for disabling loop back mode, retrieval of interface and block operation, all of which tend to be less frequently used.

Result:

Provides epoll transport multicast for common use cases.

Co-authored-by: Norman Maurer <norman_maurer@apple.com>
2019-05-25 10:06:13 +02:00
Norman Maurer
5694dc3e2b Do not use static exceptions for websocket handshake timeout (#9174)
Motivation:

f17bfd0f64 removed the usage of static exception instances to reduce the risk of OOME due addSupressed calls. We should do the same for exceptions used to signal handshake timeouts.

Modifications:

Do not use static instances

Result:

No risk of OOME due addSuppressed calls
2019-05-23 08:36:37 +02:00
noSim
2e7e6d37d5 Updated jboss-marshalling dependency to current license (#9172)
Motivation:

The mentioned license for the jboss-marshalling dependency is outdated. The license has moved from LGPL v2.1 to Apache 2.0.
The version used by Netty (1.4.11Final) is on Apache 2.0 see https://github.com/jboss-remoting/jboss-marshalling/blob/1.4.11.Final/LICENSE.txt

Modification:

Updated NOTICE file with correct license for jboss-marshalling.

Result:

NOTICE file shows correct license.
2019-05-23 07:21:57 +02:00
Nick Hill
4fa7b99da2 Introduce ByteBuf.maxFastWritableBytes() method (#9086)
Motivation

ByteBuf capacity is automatically increased as needed up to maxCapacity
when writing beyond the buffer's current capacity. However there's no
way to tell in general whether such an increase will result in a
relatively costly internal buffer re-allocation.

For unpooled buffers it always does, in pooled cases it depends on the
size of the associated chunk of allocated memory, which I don't think is
currently exposed in any way.

It would sometimes be useful to know where this limit is when making
external decisions about whether to reuse or preemptively reallocate.

It would also be advantageous to take this limit into account when
auto-increasing the capacity during writes, to defer such reallocation
until really necessary.

Modifications

Introduce new AbstractByteBuf.maxFastWritableBytes() method which will
return a value >= writableBytes() and <= maxWritableBytes().

Make use of the new method in the sizing decision made by the
AbstractByteBuf.ensureWritable(...) methods.

Result

Less reallocation/copying.
2019-05-22 20:11:56 +02:00
Vojin Jovanovic
f7a8a6f1ae Remove deprecated GraalVM native-image flags (#9118)
Motivation:

The first final version of GraalVM was released which deprecated some flags. We should use the new ones.

Modifications:

Removes the use of deprecated GraalVM native-image flags
Adds a flag to initialize netty at build time.

Result:

Do not use deprecated flags
2019-05-22 19:22:03 +02:00
Norman Maurer
9c51a8c6d4 Correctly detect that KeyManagerFactory is not supported when using OpenSSL 1.1.0+ (#9170)
Motivation:

How we tried to detect if KeyManagerFactory is supported was not good enough for OpenSSL 1.1.0+ as it partly provided the API but not all of what is required.

This then lead to failures like:

[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 1.102 s <<< FAILURE! - in io.netty.channel.epoll.EpollDomainSocketStartTlsTest
[ERROR] initializationError(io.netty.channel.epoll.EpollDomainSocketStartTlsTest)  Time elapsed: 0.016 s  <<< ERROR!
javax.net.ssl.SSLException: failed to set certificate and key
	at io.netty.handler.ssl.ReferenceCountedOpenSslServerContext.newSessionContext(ReferenceCountedOpenSslServerContext.java:130)
	at io.netty.handler.ssl.OpenSslServerContext.<init>(OpenSslServerContext.java:353)
	at io.netty.handler.ssl.OpenSslServerContext.<init>(OpenSslServerContext.java:334)
	at io.netty.handler.ssl.SslContext.newServerContextInternal(SslContext.java:468)
	at io.netty.handler.ssl.SslContextBuilder.build(SslContextBuilder.java:457)
	at io.netty.testsuite.transport.socket.SocketStartTlsTest.data(SocketStartTlsTest.java:93)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.runners.Parameterized.allParameters(Parameterized.java:280)
	at org.junit.runners.Parameterized.<init>(Parameterized.java:248)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at org.junit.internal.builders.AnnotatedBuilder.buildRunner(AnnotatedBuilder.java:104)
	at org.junit.internal.builders.AnnotatedBuilder.runnerForClass(AnnotatedBuilder.java:86)
	at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:59)
	at org.junit.internal.builders.AllDefaultPossibilitiesBuilder.runnerForClass(AllDefaultPossibilitiesBuilder.java:26)
	at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:59)
	at org.junit.internal.requests.ClassRequest.getRunner(ClassRequest.java:33)
	at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:362)
	at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
	at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
	at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
	at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
	at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
Caused by: java.lang.Exception: Requires OpenSSL 1.0.2+
	at io.netty.internal.tcnative.SSLContext.setCertificateCallback(Native Method)
	at io.netty.handler.ssl.ReferenceCountedOpenSslServerContext.newSessionContext(ReferenceCountedOpenSslServerContext.java:126)
	... 32 more

Modifications:

Also try to set the certification callback and only if this works as well mark KeyManagerFactory support as enabled.

Result:

Also correctly work when OpenSSL 1.1.0 is used.
2019-05-22 19:08:40 +02:00
Norman Maurer
f81cd7d05b
Use correct object to synchronize on in DefaultChannelPipeline (#9171)
Motivation:

8e72071d76 did adjust how synchronization is done but missed to update one block and so used synchronized (this) while it should be synchronized (handlers) .

Modifications:

Use synchronized (handlers)

Result:

Correctly synchronize
2019-05-22 18:50:28 +02:00
秦世成
1465e3ce06 Support handshake timeout in websocket handlers (#8856)
Motivation:

Support handshake timeout option in websocket handlers. It makes sense to limit the time we need to move from `HANDSHAKE_ISSUED` to `HANDSHAKE_COMPLETE` states when upgrading to WebSockets

Modification:

- Add `handshakeTimeoutMillis` option in `WebSocketClientProtocolHandshakeHandler`  and `WebSocketServerProtocolHandshakeHandler`.
- Schedule a timeout task, the task will trigger user event `HANDSHAKE_TIMEOUT` if the handshake timed out.

Result:

Fixes issue https://github.com/netty/netty/issues/8841
2019-05-22 13:56:36 +02:00
Nick Hill
23554e6997 Ensure "full" ownership of msgs passed to EmbeddedChannel.writeInbound() (#9058)
Motivation

Pipeline handlers are free to "take control" of input buffers if they have singular refcount - in particular to mutate their raw data if non-readonly via discarding of read bytes, etc.

However there are various places (primarily unit tests) where a wrapped byte-array buffer is passed in and the wrapped array is assumed not to change (used after the wrapped buffer is passed to EmbeddedChannel.writeInbound()). This invalid assumption could result in unexpected errors, such as those exposed by #8931.

Modifications

Anywhere that the data passed to writeInbound() might be used again, ensure that either:
- A copy is used rather than wrapping a shared byte array, or
- The buffer is otherwise protected from modification by making it read-only

For the tests, copying is preferred since it still allows the "mutating" optimizations to be exercised.

Results

Avoid possible errors when pipeline assumes it has full control of input buffer.
2019-05-22 12:35:03 +02:00
Carl Mastrangelo
912aa7d867 Don't double record stacktrace in Annotated*Exception (#9117)
Motivation:
When initializing the AnnotatedSocketException in AbstractChannel, both
the cause and the stack trace are set, leaving a trailing "Caused By"
that is compressed when printing the trace.

Modification:
Don't include the stack trace in the exception, but leave it in the cause.

Result:
Clearer stack trace
2019-05-22 12:07:06 +02:00
Nick Hill
d29f91540d Fix possible unsafe sharing of internal NIO buffer in CompositeByteBuf (#9169)
Motivation

A small thread-safety bug was introduced during the internal
optimizations of ComponentByteBuf made a while back in #8437. When there
is a single component which was added as a slice,
internalNioBuffer(int,int) will currently return the unwrapped slice's
un-duplicated internal NIO buffer. This is not safe since it could be
modified concurrently with other usage of that parent buffer.

Modifications

Delegate internalNioBuffer to nioBuffer in this case, which returns a
duplicate. This matches what's done in derived buffers in general
(for the same reason). Add unit test.

Result

Fixed possible thread-safety bug
2019-05-22 11:07:27 +02:00
Fabien Renaud
73dae929e3 codec-memcache: copy metadata in binary full request response (#9160)
Motivations
-----------
Calling `copy()`, `duplicate()` or `replace()` on `FullBinaryMemcacheResponse`
or `FullBinaryMemcacheRequest` instances should copy status, opCode, etc.
that are defined in `AbstractBinaryMemcacheMessage`.

Modifications
-------------
 - Modified duplicate, copy and replace methods in
DefaultFullBinaryMemcacheRequest and DefaultFullBinaryMemcacheResponse
to always copy metadata from parent classes.
 - Unit tests verifying duplicate, copy and replace methods for
DefaultFullBinaryMemcacheRequest and DefaultFullBinaryMemcacheResponse
copy buffers and metadata as expected.

Result
------
Calling copy(), duplicate() or replace() methods on
DefaultFullBinaryMemcacheRequest or DefaultFullBinaryMemcacheResponse
produces valid copies with all expected metadata.

Fixes #9159
2019-05-22 11:06:16 +02:00