Commit Graph

9551 Commits

Author SHA1 Message Date
Norman Maurer
95230e01da
Try to reduce GC produced while writing headers (#9682)
Motivation:

bbc34d0eda introduced correct handling of "in process" setup of streams but there is some room for improvements. Often the writeHeaders(...) is completed directly which means there is not need to create the extra listener object.

Modifications:

- Only create the listener if we really need too.

Result:

Less GC
2019-10-17 10:12:20 -07:00
Nick Hill
19b4adf79c Avoid wrapping scheduled Runnables in Callable adapter (#9666)
Motivation

Currently when future tasks are scheduled via schedule(Runnable, ...)
methods, the supplied Runnable is wrapped in a newly allocated Callable
adapter prior to being wrapped in a ScheduledFutureTask.

This can be avoided which saves an object allocation per scheduled task.

Modifications

Change the Callable task field of ScheduledFutureTask to be of type
Object so that it can hold/run Runnables directly in addition to
Callables.

An "adapter" is still used in the case a Runnable is scheduled with an
explicit constant non-null completion value, assumed to be rare.

Result

Less garbage
2019-10-17 07:01:52 -07:00
Matthew Miller
bbc34d0eda HTTP/2: Prevent memory leak when trying to create new streams on a connection that received a GOAWAY. (#9674)
Motivation:

In https://github.com/netty/netty/issues/8692, `Http2FrameCodec` was
updated to keep track of all "being initialized" streams, allocating
memory before initialization begins, and releasing memory after
initialization completes successfully.

In some instances where stream initialization fails (e.g. because this
connection has received a GOAWAY frame), this memory is never released.

Modifications:

This change updates the `Http2FrameCodec` to use a separate promise
for monitoring the success of sending HTTP2 headers. When sending of
headers fails, we now make sure to release memory allocated for stream
initialization.

Result:

After this change, failures in writing HTTP2 Headers (e.g. because this
connection has received a GOAWAY frame) will no longer leak memory.
2019-10-16 19:32:45 -07:00
Norman Maurer
3d7dba373f
Eliminate unnessary copy of ByteBuf on ByteToMessageDecoder removal (#9662)
Motivation:

At the moment we do a ByteBuf.readBytes(...) on removal of the ByteToMessageDecoder if there are any bytes left and forward the returned ByteBuf to the next handler in the pipeline. This is not really needed as we can just forward the cumulation buffer directly and so eliminate the extra memory copy

Modifications:

Just forward the cumulation buffer directly on removal of the ByteToMessageDecoder

Result:

Less memory copies
2019-10-16 06:47:59 -07:00
Nick Hill
2e5dd28800 Fix DnsNameResolver TCP fallback test and message leaks (#9647)
Motivation

A memory leak related to DNS resolution was reported in #9634,
specifically linked to the TCP retry fallback functionality that was
introduced relatively recently. Upon inspection it's apparent that there
are some error paths where the original UDP response might not be fully
released, and more significantly the TCP response actually leaks every
time on the fallback success path.

It turns out that a bug in the unit test meant that the intended TCP
fallback path was not actually exercised, so it did not expose the main
leak in question.

Modifications

- Fix DnsNameResolverTest#testTruncated0 dummy server fallback logic to
first read transaction id of retried query and use it in replayed
response
- Adjust semantic of internal DnsQueryContext#finish method to always
take refcount ownership of passed in envelope
- Reorder some logic in DnsResponseHandler fallback handling to verify
the context of the response is expected, and ensure that the query
response are either released or propagated in all cases. This also
reduces a number of redundant retain/release pairings

Result

Fixes #9634
2019-10-14 16:10:15 +02:00
Norman Maurer
6c05d16967
Use @SuppressJava6Requirement for animal sniffer plugin to ensure we always guard correctly (#9655)
Motivation:

We can use the `@SuppressJava6Requirement` annotation to be more precise about when we use Java6+ APIs. This helps us to ensure we always protect these places.

Modifications:

Make use of `@SuppressJava6Requirement` explicit

Result:

Fixes https://github.com/netty/netty/issues/2509.
2019-10-14 15:54:49 +02:00
Norman Maurer
833f11be75
Remove usage of AtomicIntegerFieldUpdater in ReferenceCountedOpenSslE… (#9653)
Motivation:

There is not need to use a CAS as everything is synchronized anyway. We can simplify the code a bit by not using it.

Modifications:

- Just remove the CAS operation
- Change from int to boolean

Result:

Code cleanup
2019-10-13 11:02:52 +02:00
stroller
deb34a0f8b remove useless doc for 5.0 (#9546)
Fix javadoc issues

Motivation:

Remove useless doc for 5.0

Modifications:

Remove description for 5.0

Result:

Useless doc will be removed.
2019-10-12 20:17:18 +02:00
Nick Hill
166caf96ef Avoid unnecessary epoll event loop wake-ups (#9605)
Motivation

The recently-introduced event loop scheduling hooks can be exploited by
the epoll transport to avoid waking the event loop when scheduling
future tasks if there is a timer already set to wake up sooner.

There is also a "default" timeout which will wake the event
loop after 1 second if there are no pending future tasks. The
performance impact of these wakeups themselves is likely negligible but
there's significant overhead in having to re-arm the timer every time
the event loop goes to sleep (see #7816). It's not 100% clear why this
timeout was there originally but we're sure it's no longer needed.

Modification

Combine the existing volatile wakenUp and non-volatile prevDeadlineNanos
fields into a single AtomicLong that stores the next scheduled wakeup
time while the event loop is in epoll_wait, and is -1 while it is awake.

Use this as a guard to debounce wakeups from both immediate scheduled
tasks and future scheduled tasks, the latter using the new
before/afterScheduledTaskSubmitted overrides and based on whether the
new deadline occurs prior to an already-scheduled timer.

A similar optimization was already added to NioEventLoop, but it still
uses two separate volatiles. We should consider similar streamlining of
that in a future update.

Result

Fewer event loop wakeups when scheduling future tasks, greatly reduced
overhead when no future tasks are scheduled.
2019-10-12 20:16:16 +02:00
Tim Brooks
ec8d43c294 Do not mandate direct bytes in SslHandler queue (#9656)
Motivation:

Currently when the SslHandler coalesces outbound bytes it always
allocates a direct byte buffer. This does not make sense if the JDK
engine is being used as the bytes will have to be copied back to heap
bytes for the engine to operate on them.

Modifications:

Inspect engine type when coalescing outbound bytes and allocate heap
buffer if heap bytes are preferred by the engine.

Result:

Improved performance for JDK engine. Better performance in environments
without direct buffer pooling.
2019-10-12 20:12:12 +02:00
Norman Maurer
f4203c4339
Update commons-compress dependency (#9657)
Motivation:

We should use the latest commons-compress release to fix CVE-2019-12402 (even it is only a test dependency)

Modifications:

Update commons-compress to 1.19

Result:

Fix security alert
2019-10-12 20:10:59 +02:00
Jonathan Leitschuh
e0b15ed952 [DOC] Add CWE-113 warning to DefaultHttpHeaders constructor (#9646)
### Motivation:

I've now found two libraries that use Netty to be vulnerable to [CWE-113: Improper Neutralization of CRLF Sequences in HTTP Headers ('HTTP Response Splitting')](https://cwe.mitre.org/data/definitions/113.html) due to using `new DefaultHttpHeaders(false)`.

Some part of me hopes that this warning will help dissuade library authors from disabling this important security check.

### Modification:

Add documentation to `DefaultHttpHeaders(boolean)` to warn about the implications of `false`.

### Result:

This improves the documentation on `DefaultHttpHeaders`.
2019-10-10 22:47:28 +04:00
Nikolay Fedorovskikh
08e9b456a4 Fixes validation of input bytes in the Base64 decoder (#9623)
Motivation:
In the current implementation of Base64 decoder an invalid
character `\u00BD` treated as `=`.
Also character `\u007F` leads to ArrayIndexOutOfBoundsException.

Modification:
Explicitly checks that all input bytes are ASCII characters
(greater than zero). Fix `decodabet` tables.

Result:
Correctly validation input bytes in Base64 decoder.
2019-10-10 20:46:39 +02:00
Norman Maurer
35862cad7e
Fix SSL tests that use SslProvider.OPENSSL_REFCNT (#9649)
Motivation:

031c2e2e88 introduced some change to reduce the risk of have the `ReferenceCountedOpenSslContext` be destroyed while the `ReferenceCountedSslEngine` is still in us. Unfortunaly it missed to adjust a few tests which make assumptions about the refCnt of the context.

Modifications:

Adjust tests to take new semenatics into acount.

Result:

No more tests failures
2019-10-10 10:40:45 +04:00
Norman Maurer
a31a3e31da
Add vscode specific files / directory to .gitignore (#9652)
Motivation:

We should not commit vscode specific files, so at it to gitignore

Modifications:

Add files to .gitignore

Result:

Correctly ignore ide related files
2019-10-10 09:35:08 +04:00
康智冬
bd8cea644a Fix typos in javadocs (#9527)
Motivation:

We should have correct docs without typos

Modification:

Fix typos and spelling

Result:

More correct docs
2019-10-09 17:12:52 +04:00
Norman Maurer
b11aba354a
Use adopt@1.13.0-0 when building with Java 13 (#9641)
Motivation:

We should use adaptjdk 13 and not oracle openjdk 13 when building with Java 13

Modifications:

Use adopt@1.13.0-0

Result:

More consistent java vendor usage
2019-10-09 17:10:37 +04:00
Nick Hill
99a4cecec6 Fix NioEventLoopTest#testChannelsRegistered flakiness (#9650)
Motivation

NioEventLoopTest#testChannelsRegistered() fails intermittently due to
use of SingleThreadEventLoop#channelsRegistered() which is not
threadsafe and unreliable when called from outside the event loop.

Modifications

Add static registeredChannels method to NioEventLoopTest and
AbstractSingleThreadEventLoopTest to call from the tests via event loop
instead of directly.

Result

Hopefully fewer test failures
2019-10-09 16:57:54 +04:00
Norman Maurer
271d8de9ec
Ensure we finish setup mock before we use it in Http2ConnectionRoundtripTest.headersWriteForPeerStreamWhichWasResetShouldNotGoAway (#9645)
Motivation:

We did dispatch the client code before we did finish setup the mock and so may end up with org.mockito.exceptions.misusing.UnfinishedStubbingException if the connect happens quickly enough.

See https://ci.netty.io/job/netty-centos6-java8-prb/1637/testReport/junit/io.netty.handler.codec.http2/Http2ConnectionRoundtripTest/headersWriteForPeerStreamWhichWasResetShouldNotGoAway/

Modifications:

First finish setup the mock and the dispatch.

Result:

Fix flacky test
2019-10-09 09:10:38 +04:00
Carl Mastrangelo
27397e87b2 Remember to return writability events to flow controller in HTTP2 Multiplexer (#9642)
Motivation:

Http2MultiplexCodec extends Http2FrameCodec extends Http2ConnectionHandler.  It appears  Http2MultiplexCodec overrode the channelWritabilityChanged method, which prevented the flow controller from becoming active.  In the case the parent channel becomes unwritable, and then later becomes writable, it needs to indicate that the child channels can still write data.   This is slightly confusing, because the child channels may still themselves be unwritable, but should still drain their data to the parent channel.

Modification:

Still propagate writability changes to the HTTP/2 flow controller

Result:

Fixes https://github.com/netty/netty/issues/9636
2019-10-08 18:40:23 +04:00
Ran
ca915ae590 Initialize dynamicMethods before use (#9618)
Motivation:

There is a goto statement above the current position of initialize dynamicMethods, and dynamicMethods is used after the goto which might cause undefined behavior.

Modifications:

Initialize dynamicMehtods at the top.

Result:

No more undefined behavior.
2019-10-08 11:57:41 +04:00
Nick Hill
c591d03320 Remove redundant epollWaitNow() call in EpollEventLoop#closeAll() (#9614)
Motivation

This is a vestige that was removed in the original PR #9535 before it
was reverted, but we missed it when re-applying in #9586.

It means there is a possible race condition because a wakeup event could
be missed while shutting down, but the consequences aren't serious since
there's a 1 second safeguard timeout when waiting for it.

Modification

Remove call to epollWaitNow() in EpollEventLoop#closeAll()

Result

Cleanup redundant code, avoid shutdown delay race condition
2019-10-07 15:54:46 +04:00
Nick Hill
170e4deee6 Fix event loop shutdown timing fragility (#9616)
Motivation

The current event loop shutdown logic is quite fragile and in the
epoll/NIO cases relies on the default 1 second wait/select timeout that
applies when there are no scheduled tasks. Without this default timeout
the shutdown would hang indefinitely.

The timeout only takes effect in this case because queued scheduled
tasks are first cancelled in
SingleThreadEventExecutor#confirmShutdown(), but I _think_ even this
isn't robust, since the main task queue is subsequently serviced which
could result in some new scheduled task being queued with much later
deadline.

It also means shutdowns are unnecessarily delayed by up to 1 second.

Modifications

- Add/extend unit tests to expose the issue
- Adjust SingleThreadEventExecutor shutdown and confirmShutdown methods
to explicitly add no-op tasks to the taskQueue so that the subsequent
event loop iteration doesn't enter blocking wait (as looks like was
originally intended)

Results

Faster and more robust shutdown of event loops, allows removal of the
default wait timeout
2019-10-07 11:06:01 +04:00
Pete Woods
45be693889 Add io.netty.handler.codec.http2.Http2ConnectionHandler for runtime GraalVM compilation (#9621)
Motivation:

Native image compilation is failing without extra flags:

```
Warning: Aborting stand-alone image build. No instances of io.netty.buffer.UnpooledHeapByteBuf are allowed in the image heap as this class should be initialized at image runtime. Object has been initialized by the io.netty.handler.codec.http2.Http2ConnectionHandler class initializer with a trace: 
 	at io.netty.buffer.Unpooled.wrappedBuffer(Unpooled.java:157)
	at io.netty.handler.codec.http2.Http2ConnectionHandler.<clinit>(Http2ConnectionHandler.java:74)
.  To fix the issue mark io.netty.buffer.UnpooledHeapByteBuf for build-time initialization with --initialize-at-build-time=io.netty.buffer.UnpooledHeapByteBuf or use the the information from the trace to find the culprit and --initialize-at-run-time=<culprit> to prevent its instantiation.

Detailed message:
Trace: 	object io.netty.buffer.ReadOnlyByteBuf
	object io.netty.buffer.UnreleasableByteBuf
	method io.netty.handler.codec.http2.Http2ConnectionHandler.access$500()
Call path from entry point to io.netty.handler.codec.http2.Http2ConnectionHandler.access$500(): 
	at io.netty.handler.codec.http2.Http2ConnectionHandler.access$500(Http2ConnectionHandler.java:66)
	at io.netty.handler.codec.http2.Http2ConnectionHandler$PrefaceDecoder.readClientPrefaceString(Http2ConnectionHandler.java:299)
	at io.netty.handler.codec.http2.Http2ConnectionHandler$PrefaceDecoder.decode(Http2ConnectionHandler.java:239)
	at io.netty.handler.codec.http2.Http2ConnectionHandler.decode(Http2ConnectionHandler.java:438)
	at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:505)
	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:444)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:283)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
	at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:56)
	at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:365)
	at com.oracle.svm.core.jdk.RuntimeSupport.executeHooks(RuntimeSupport.java:144)
	at com.oracle.svm.core.jdk.RuntimeSupport.executeStartupHooks(RuntimeSupport.java:89)
	at com.oracle.svm.core.JavaMainWrapper.runCore(JavaMainWrapper.java:143)
	at com.oracle.svm.core.JavaMainWrapper.run(JavaMainWrapper.java:186)
	at com.oracle.svm.core.code.IsolateEnterStub.JavaMainWrapper_run_5087f5482cc9a6abc971913ece43acb471d2631b(generated:0)
```

Modification:

Add `io.netty.handler.codec.http2.Http2ConnectionHandler` for runtime compilation, as the buffer library's `io.netty.buffer.UnpooledHeapByteBuf` is also marked for runtime.

Result:

Native image compilation works again.
2019-10-07 09:03:07 +02:00
Nikolay Fedorovskikh
1fb5ff15a8 Fix possible NPE in DefaultHttp2UnknownFrame#equals (#9625)
Motivation:
`DefaultHttp2UnknownFrame#equals` may produce NPE due to
incorrect comparison of `stream` field.

Modification:
- Fix the `stream` field compare.
- Cleanup usage of class fields: use direct access instead of getters
(because the class is final).

Result:
No NPE in `equals` method.
2019-10-07 09:00:59 +02:00
Nikolay Fedorovskikh
4dc1eccf60 Make some inner classes static (#9624)
Motivation:
Classes `AbstractHttp2StreamChannel.Http2StreamChannelConfig`
and `DnsNameResolver.AddressedEnvelopeAdapter` may be static:
it doesn't reference its enclosing instance.

Modification:
Add `static` modifier.

Result:
Prevents a possible memory leak and uses less memory per class instance.
2019-10-07 08:14:02 +02:00
Bryce Anderson
031c2e2e88 Reference-counted SslEngines retain a reference to their parent SslContext (#9626)
Motivation:
With the Netty ref-counted OpenSSL implementation the parent SslContext
maintains state necessary for the SslEngine's it produces. However, it's
possible for the parent context to be closed and release those resources
before the child engines are finished which causes problems.

Modification:
Spawned ReferenceCountedOpenSslEngine's retain a reference to their
parent ReferenceCountedOpenSslContext.

Result:
The lifetime of the shared data is extended to include the lifetime of
the dependents.
2019-10-07 10:12:54 +04:00
Codrut Stancu
4980a6b304 Register sun.nio.ch.SelectorImpl fields for unsafe access. (#9631)
Motivation:

On JDK > 9 Netty uses Unsafe to write two internal JDK fields: sun.nio.ch.SelectorImp.selectedKeys and sun.nio.ch.SelectorImpl.publicSelectedKeys. This is done in transport/src/main/java/io/netty/channel/nio/NioEventLoop.java:225, in openSelector() method. The GraalVM analysis cannot do the Unsafe registration automatically because the object field offset computation is hidden behind two layers of calls.

Modifications:

This PR updates the Netty GraalVM configuration by registering those fields for unsafe access.
 
Result:

Improved support for Netty on GraalVM with JDK > 9.
2019-10-07 10:10:46 +04:00
Tatsushi Inagaki
d8b1a2d93f Allow to build on s390_64
Motivation:

It is not possible to build Netty on an s390_64 platform.

Modifications:

Modify pom.xml so that s390_64 is acceptable as os.detected.arch.

Result:

Netty can be built on an s390_64 platform.

Signed-off-by: Tatsushi Inagaki <e29253@jp.ibm.com>
2019-09-27 12:27:45 +02:00
Tatsushi Inagaki
ed142442a4 Enable Netty on a big endian platform
Motivation:

We would like to enable Netty also on a big endian platform such as
s390_64. We need to fix a function which assumes that the target
platform is little endian.

Modifications:

Modify netty_unix_socket_accept() to write an address length as jbyte
instead of jsize.

Result:

Netty can be enabled on a big endian platform.

Signed-off-by: Tatsushi Inagaki <e29253@jp.ibm.com>
2019-09-27 12:27:37 +02:00
bruce
b39ffed042 Fix incorrect calculation of next buffer size in AdaptiveRecvByteBufAllocator (#9555)
Motivation:

Due a bug we did not always correctly calculate the next buffer size in AdaptiveRecvByteBufAllocator.

Modification:

Fix calculation and add unit test

Result:

Correct calculation is always used.
2019-09-27 09:59:25 +02:00
Nick Hill
85a663fa52 Null out completed tasks to help with garbage collection (#9613)
Motivation

When ScheduledFutureTasks complete, there's no need to retain a ref to
the wrapped task. Clearing it could help in particular with the case
where many scheduled tasks have been cancelled but their queue removal
delayed (since it is done lazily).

Modifications

This comprises just the PromiseTask changes from #9580. Upon completion,
replace the task reference with a static sentinel depending on the type
of completion (so that it will be reflected by toString).

Result

More expedient collection of cancelled task objects
2019-09-27 09:54:42 +02:00
Tatsushi Inagaki
fed552d09d Fix broken pipe due to /usr/bin/ldd (#9606)
Motivation:

The build script for the module Netty Transport Native Epoll can cause
intermittent build break due to broken pipe by /usr/bin/ldd. This issue
likely to occur on a build environment with multiple processors.

Modifications:

The root cause is that the consumer head command finishes earlier the
producer ldd command. Buffering the outputs of the ldd command by an
intermediate tail command avoids the broken pipe.

Result:

A build on multiple processors can finish successfully.

Signed-off-by: Tatsushi Inagaki <e29253@jp.ibm.com>
2019-09-26 22:27:40 +02:00
Norman Maurer
622cc232f0
Use configured ByteBufAllocator in InboundHttp2ToHttpAdapter (#9611)
Motivation:

At the moment we use Unpooled.buffer(...) in InboundHttp2ToHttpAdapter when we need to do a copy of the message. We should better use the configured ByteBufAllocator for the Channel

Modifications:

Change internal interface to also take the ByteBufAllocator as argument and use it when we need to allocate a ByteBuf.

Result:

Use the "correct" ByteBufAllocator in InboundHttp2ToHttpAdapter in all cases
2019-09-26 22:11:41 +02:00
Norman Maurer
1f4b9e36ea
We should only disable releasing of the message once writeData(...) was called successfully (#9610)
Motivation:

At the moment we set release to false before we call writeData(...). This could let to the sitatuation that we will miss to release the message if writeData(...) throws. We should set release to false after we called writeData(...) to ensure the ownership of the buffer is correctly transferred.

Modifications:

- Set release to false after writeData(...) was successfully called only

Result:

No possibility for a buffer leak
2019-09-26 21:59:57 +02:00
Norman Maurer
299a682d3f
Correctly take Http2FrameCodecBuilder.isValidateHeaders() into account when creating a Http2FrameCodec from an existing Http2FrameWriter. (#9600)
Motivation:

We did miss to take Http2FrameCodecBuilder.isValidateHeaders() into account when a Http2FrameWriter was set on the builder and always assumed validation should be enabled.

Modifications:

Remove hardcode value and use configured value

Result:

Http2FrameCodecBuilder.isValidateHeaders() is respected in all cases
2019-09-26 21:57:05 +02:00
时无两丶
3ef00eaa06 Double check size to avoid ArrayIndexOutOfBoundsException (#9609)
Motivation:

Recycler$Stack.pop will occurs `ArrayIndexOutOfBoundsException` in some race cases, we should double check `size` even after `scavenge` called.

Modifications:

Double check `size` after `scavenge`

Result:

avoid ArrayIndexOutOfBoundsException in `pop`
2019-09-26 21:53:35 +02:00
root
92941cdcac [maven-release-plugin] prepare for next development iteration 2019-09-25 06:15:31 +00:00
root
bd907c3b3a [maven-release-plugin] prepare release netty-4.1.42.Final 2019-09-25 06:14:31 +00:00
Nick Hill
2791f0fefa Avoid use of global AtomicLong for ScheduledFutureTask ids (#9599)
Motivation

Currently a static AtomicLong is used to allocate a unique id whenever a
task is scheduled to any event loop. This could be a source of
contention if delayed tasks are scheduled at a high frequency and can be
easily avoided by having a non-volatile id counter per queue.

Modifications

- Replace static AtomicLong ScheduledFutureTask#nextTaskId with a long
field in AbstractScheduledExecutorService
- Set ScheduledFutureTask#id based on this when adding the task to the
queue (in event loop) instead of at construction time
- Add simple benchmark

Result

Less contention / cache-miss possibility when scheduling future tasks

Before:

Benchmark      (num)   Mode  Cnt    Score    Error  Units
scheduleLots  100000  thrpt   20  346.008 ± 21.931  ops/s

Benchmark      (num)   Mode  Cnt    Score    Error  Units
scheduleLots  100000  thrpt   20  654.824 ± 22.064  ops/s
2019-09-25 07:34:25 +02:00
liyixin
86ff76a4f7 Fix incorrect comment (#9598)
Motivation:

The comment is incorrect and so missleading

Modification:

Correct the comment

Result:

Correct comment in code
2019-09-24 10:00:55 +02:00
Norman Maurer
5e69a13c21
Cleanup JNI code to always correctly free memory when loading fails and also correctly respect out of memory in all cases (#9596)
Motivation:

At the moment we not consistently (and also not correctly) free allocated native memory in all cases during loading the JNI library. This can lead to native memory leaks in the unlikely case of failure while trying to load the library.

Beside this we also not always correctly handle the case when a new java object can not be created in native code because of out of memory.

Modification:

- Copy some macros from netty-tcnative to be able to handle errors in a more easy fashion
- Correctly account for New* functions to return NULL
- Share code

Result:

More robust and clean JNI code
2019-09-24 07:18:35 +02:00
Francesco Nigro
eb3c4bd926 ChunkedNioFile can use absolute FileChannel::read to read chunks (#9592)
Motivation:

Users can reuse the same FileChannel for different ChunkedNioFile
instances without being worried that FileChannel::position will be
changed concurrently by them.
In addition, FileChannel::read with absolute position allows to
use on *nix pread that is more efficient then fread.

Modifications:

Always use absolute FileChannel::read ops

Result:

Faster and more flexible uses of FileChannel for ChunkedNioFile
2019-09-24 07:17:09 +02:00
Norman Maurer
76592db0bd
Close eventfd shutdown/wakeup race by closely tracking epoll edges (#9586)
Motivation

This is another iteration of #9476.

Modifications

Instead of maintaining a count of all writes performed and then using
reads during shutdown to ensure all are accounted for, just set a flag
after each write and don't reset it until the corresponding event has
been returned from epoll_wait.

This requires that while a write is still pending we don't reset
wakenUp, i.e. continue to block writes from the wakeup() method.

Result

Race condition eliminated. Fixes #9362

Co-authored-by: Norman Maurer <norman_maurer@apple.com>
2019-09-23 15:30:42 +02:00
Pete Woods
0a2d85f1d3 Fix GraalVM native image build error (#9593)
Motivation:

Error: Class that is marked for delaying initialization to run time got initialized during image building: io.netty.handler.codec.http2.Http2CodecUtil. Try marking this class for build-time initialization with --initialize-at-build-time=io.netty.handler.codec.http2.Http2CodecUtil
Error: Use -H:+ReportExceptionStackTraces to print stacktrace of underlying exception
Error: Image build request failed with exit status 1
Modification:

After debugging, it seems the culprit is io.netty.handler.codec.http2.Http2ClientUpgradeCodec, which also needs runtime initialisation.

Result:

Fixes #micronaut-projects/micronaut-grpc#8
2019-09-23 14:42:42 +02:00
Norman Maurer
dc4de7fbb4
We need to use NewGloblRef when caching jclass instances (#9595)
Motivation:

It is not safe to cache a jclass without obtaining a global reference via NewGlobalRef.

Modifications:

Correctly use NewGlobalRef(...) before caching

Result:

Correctly cache jclass instance
2019-09-23 12:47:58 +02:00
Norman Maurer
4499384135
Update to netty-tcnative 2.0.26.Final (#9589)
Motivation:

We just released a new version of netty-tcnative.

Modifications:

Bump up to netty-tcnative 2.0.26.Final

Result:

Use latest netty-tcnative release
2019-09-21 18:09:34 +02:00
Norman Maurer
8648171abc
Fix *SslEngineTest to not throw ClassCastException and pass in all cases (#9588)
Motivation:

Due some bug we did endup with ClassCastExceptions in some cases. Beside this we also did not correctly handle the case when ReferenceCountedOpenSslEngineTest did produce tasks to run in on test.

Modifications:

- Correctly unwrap the engine before to fix ClassCastExceptions
- Run delegated tasks when needed.

Result:

All tests pass with different OpenSSL implementations (OpenSSL, BoringSSL etc)
2019-09-21 14:58:36 +02:00
Joe Ellis
aebe2064d5 Allow domain sockets to configure SO_SNDBUF and SO_RCVBUF (#9584)
Motivation:

Running tests with a `KQueueDomainSocketChannel` showed worse performance than an `NioSocketChannel`. It turns out that the default send buffer size for Nio sockets is 64k while for KQueue sockets it's 8k. I verified that manually setting the socket's send buffer size improved perf to expected levels.

Modification:

Plumb the `SO_SNDBUF` and `SO_RCVBUF` options into the `*DomainSocketChannelConfig`.

Result:

Can now configure send and receive buffer sizes for domain sockets.
2019-09-20 22:28:53 +02:00
liyixin
07fe1a299a Optimize the QueryStringEncoder performance (#9568)
Motivation:

Optimize the QueryStringEncoder for lower memory overhead and higher encode speed.

Modification:

Encode the space to + directly, and reuse the uriStringBuilder rather then create a new one.

Result:

Improved performance
2019-09-20 21:07:13 +02:00