Commit Graph

10093 Commits

Author SHA1 Message Date
Aayush Atharva
abd5a7d922
Add state in exception message (#10842)
Motivation:
We should add `state` in the exception message of `HttpObjectEncoder` because it makes debugging a little easier.

Modification:
Added `state` in the exception message.

Result:
Better exception message for smooth debugging.
2020-12-07 10:49:09 +01:00
Andrey Mizurov
f551db2bda
Provide ability to extend StompSubframeEncoder and improve full stomp frame encoding (allocate one buffer for full frame considering the size of the headers) (#10778)
Motivation:

At the moment `StompSubframeEncoder` encode a frame only to `ByteBuf` it is not convenient if further we need to convert it to another type of message,  e.g. `WebSocketFrame`. Also, if we send a full frame, it splits into two headers and a content what makes it difficult to convert it in the next handler. 

Modification:

Introduce additional converter methods e.g. (`Object protected convertFullFrame(StompFrame original, ByteBuf encoded`)...) for extending encoder functionality and allocate only one `ByteBuf` for full stomp frame. Change headers size calculation, previously used only 256 bytes that reallocate a new buffer each time when headers size more than this threshold. Add `StompEncoderBenchmark`.

Result:

Improved  `StompSubframeEncoder` fro extensions.

Previous version benchmark
```
Benchmark                              (contentLength)  (headersType)  (pooledAllocator)   Mode  Cnt        Score        Error  Units
StompEncoderBenchmark.writeStompFrame                0            ONE               true  thrpt   10  4432132.884 ± 178923.436  ops/s
StompEncoderBenchmark.writeStompFrame                0            ONE              false  thrpt   10  1281122.756 ±  52484.174  ops/s
StompEncoderBenchmark.writeStompFrame                0          THREE               true  thrpt   10  2980897.937 ± 130253.049  ops/s
StompEncoderBenchmark.writeStompFrame                0          THREE              false  thrpt   10  1116883.574 ±  35471.482  ops/s
StompEncoderBenchmark.writeStompFrame                0          SEVEN               true  thrpt   10  1988012.159 ±  74352.450  ops/s
StompEncoderBenchmark.writeStompFrame                0          SEVEN              false  thrpt   10   881772.343 ±  94633.870  ops/s
StompEncoderBenchmark.writeStompFrame                0         ELEVEN               true  thrpt   10  1048125.919 ± 151053.902  ops/s
StompEncoderBenchmark.writeStompFrame                0         ELEVEN              false  thrpt   10   429900.066 ±  47956.661  ops/s
StompEncoderBenchmark.writeStompFrame                0         TWENTY               true  thrpt   10   660584.122 ± 104973.439  ops/s
StompEncoderBenchmark.writeStompFrame                0         TWENTY              false  thrpt   10   278255.488 ±  20143.708  ops/s
StompEncoderBenchmark.writeStompFrame               10            ONE               true  thrpt   10  4251498.549 ± 625050.979  ops/s
StompEncoderBenchmark.writeStompFrame               10            ONE              false  thrpt   10  1214006.861 ±  60421.601  ops/s
StompEncoderBenchmark.writeStompFrame               10          THREE               true  thrpt   10  3117736.486 ± 173613.974  ops/s
StompEncoderBenchmark.writeStompFrame               10          THREE              false  thrpt   10  1046605.891 ±  94428.064  ops/s
StompEncoderBenchmark.writeStompFrame               10          SEVEN               true  thrpt   10  2006986.881 ± 108456.748  ops/s
StompEncoderBenchmark.writeStompFrame               10          SEVEN              false  thrpt   10   877983.112 ±  82919.387  ops/s
StompEncoderBenchmark.writeStompFrame               10         ELEVEN               true  thrpt   10  1132844.437 ±  84578.571  ops/s
StompEncoderBenchmark.writeStompFrame               10         ELEVEN              false  thrpt   10   429334.649 ±  35403.161  ops/s
StompEncoderBenchmark.writeStompFrame               10         TWENTY               true  thrpt   10   657093.390 ±  48092.947  ops/s
StompEncoderBenchmark.writeStompFrame               10         TWENTY              false  thrpt   10   252140.876 ±  37337.255  ops/s
StompEncoderBenchmark.writeStompFrame              100            ONE               true  thrpt   10  4720507.067 ± 100993.908  ops/s
StompEncoderBenchmark.writeStompFrame              100            ONE              false  thrpt   10  1266182.925 ±  85888.413  ops/s
StompEncoderBenchmark.writeStompFrame              100          THREE               true  thrpt   10  2898746.621 ± 452579.753  ops/s
StompEncoderBenchmark.writeStompFrame              100          THREE              false  thrpt   10  1019555.288 ±  65640.507  ops/s
StompEncoderBenchmark.writeStompFrame              100          SEVEN               true  thrpt   10  2259187.459 ±  20025.989  ops/s
StompEncoderBenchmark.writeStompFrame              100          SEVEN              false  thrpt   10   896405.412 ±  53750.148  ops/s
StompEncoderBenchmark.writeStompFrame              100         ELEVEN               true  thrpt   10  1110670.772 ± 107650.327  ops/s
StompEncoderBenchmark.writeStompFrame              100         ELEVEN              false  thrpt   10   445187.398 ±  28845.959  ops/s
StompEncoderBenchmark.writeStompFrame              100         TWENTY               true  thrpt   10   611506.846 ±  25304.240  ops/s
StompEncoderBenchmark.writeStompFrame              100         TWENTY              false  thrpt   10   247687.007 ±  43471.578  ops/s
StompEncoderBenchmark.writeStompFrame             1000            ONE               true  thrpt   10  4140949.576 ± 270274.087  ops/s
StompEncoderBenchmark.writeStompFrame             1000            ONE              false  thrpt   10  1154515.598 ± 134413.876  ops/s
StompEncoderBenchmark.writeStompFrame             1000          THREE               true  thrpt   10  3349996.875 ± 162309.889  ops/s
StompEncoderBenchmark.writeStompFrame             1000          THREE              false  thrpt   10  1141040.562 ±   5895.693  ops/s
StompEncoderBenchmark.writeStompFrame             1000          SEVEN               true  thrpt   10  2184632.248 ±   8957.833  ops/s
StompEncoderBenchmark.writeStompFrame             1000          SEVEN              false  thrpt   10   959545.704 ±   5835.161  ops/s
StompEncoderBenchmark.writeStompFrame             1000         ELEVEN               true  thrpt   10  1081113.327 ±   3957.527  ops/s
StompEncoderBenchmark.writeStompFrame             1000         ELEVEN              false  thrpt   10   467524.660 ±   1383.236  ops/s
StompEncoderBenchmark.writeStompFrame             1000         TWENTY               true  thrpt   10   568411.797 ± 108712.493  ops/s
StompEncoderBenchmark.writeStompFrame             1000         TWENTY              false  thrpt   10   260764.231 ±  43149.129  ops/s
StompEncoderBenchmark.writeStompFrame            10000            ONE               true  thrpt   10  4369787.147 ± 619367.939  ops/s
StompEncoderBenchmark.writeStompFrame            10000            ONE              false  thrpt   10  1246782.845 ±  47468.764  ops/s
StompEncoderBenchmark.writeStompFrame            10000          THREE               true  thrpt   10  3333328.810 ± 253061.481  ops/s
StompEncoderBenchmark.writeStompFrame            10000          THREE              false  thrpt   10  1108278.988 ±  81905.149  ops/s
StompEncoderBenchmark.writeStompFrame            10000          SEVEN               true  thrpt   10  2062961.266 ± 247096.284  ops/s
StompEncoderBenchmark.writeStompFrame            10000          SEVEN              false  thrpt   10   925199.985 ±  36734.594  ops/s
StompEncoderBenchmark.writeStompFrame            10000         ELEVEN               true  thrpt   10  1223240.034 ±  58833.801  ops/s
StompEncoderBenchmark.writeStompFrame            10000         ELEVEN              false  thrpt   10   460864.117 ±   2361.459  ops/s
StompEncoderBenchmark.writeStompFrame            10000         TWENTY               true  thrpt   10   655864.762 ±  35237.335  ops/s
StompEncoderBenchmark.writeStompFrame            10000         TWENTY              false  thrpt   10   286388.865 ±   1002.460  ops/s
```
A new version benchmark
```
Benchmark                              (contentLength)  (headersType)  (pooledAllocator)   Mode  Cnt        Score        Error  Units
StompEncoderBenchmark.writeStompFrame                0            ONE               true  thrpt   10  4366110.018 ± 420377.867  ops/s
StompEncoderBenchmark.writeStompFrame                0            ONE              false  thrpt   10  1289437.153 ± 215271.656  ops/s
StompEncoderBenchmark.writeStompFrame                0          THREE               true  thrpt   10  2818791.355 ± 218894.471  ops/s
StompEncoderBenchmark.writeStompFrame                0          THREE              false  thrpt   10  1040151.615 ±  75352.695  ops/s
StompEncoderBenchmark.writeStompFrame                0          SEVEN               true  thrpt   10  1842144.001 ±  94668.864  ops/s
StompEncoderBenchmark.writeStompFrame                0          SEVEN              false  thrpt   10   916742.825 ±  65467.820  ops/s
StompEncoderBenchmark.writeStompFrame                0         ELEVEN               true  thrpt   10  1310454.012 ± 100747.490  ops/s
StompEncoderBenchmark.writeStompFrame                0         ELEVEN              false  thrpt   10   679934.001 ±  82168.249  ops/s
StompEncoderBenchmark.writeStompFrame                0         TWENTY               true  thrpt   10   746867.549 ±  68373.269  ops/s
StompEncoderBenchmark.writeStompFrame                0         TWENTY              false  thrpt   10   483316.314 ±  50978.009  ops/s
StompEncoderBenchmark.writeStompFrame               10            ONE               true  thrpt   10  4791698.722 ± 263890.510  ops/s
StompEncoderBenchmark.writeStompFrame               10            ONE              false  thrpt   10  1289877.116 ± 128677.185  ops/s
StompEncoderBenchmark.writeStompFrame               10          THREE               true  thrpt   10  2984662.187 ± 395567.524  ops/s
StompEncoderBenchmark.writeStompFrame               10          THREE              false  thrpt   10  1079028.782 ±  43548.555  ops/s
StompEncoderBenchmark.writeStompFrame               10          SEVEN               true  thrpt   10  1806763.709 ±  59162.209  ops/s
StompEncoderBenchmark.writeStompFrame               10          SEVEN              false  thrpt   10   935274.980 ±  22064.148  ops/s
StompEncoderBenchmark.writeStompFrame               10         ELEVEN               true  thrpt   10  1284172.151 ± 119068.047  ops/s
StompEncoderBenchmark.writeStompFrame               10         ELEVEN              false  thrpt   10   687174.498 ±  30270.916  ops/s
StompEncoderBenchmark.writeStompFrame               10         TWENTY               true  thrpt   10   803843.483 ±  29106.133  ops/s
StompEncoderBenchmark.writeStompFrame               10         TWENTY              false  thrpt   10   502134.552 ±  23653.215  ops/s
StompEncoderBenchmark.writeStompFrame              100            ONE               true  thrpt   10  4337438.694 ± 378524.452  ops/s
StompEncoderBenchmark.writeStompFrame              100            ONE              false  thrpt   10  1289174.213 ±  50640.853  ops/s
StompEncoderBenchmark.writeStompFrame              100          THREE               true  thrpt   10  3232767.156 ± 311934.194  ops/s
StompEncoderBenchmark.writeStompFrame              100          THREE              false  thrpt   10  1115247.028 ±  15683.477  ops/s
StompEncoderBenchmark.writeStompFrame              100          SEVEN               true  thrpt   10  2213147.232 ±  86326.187  ops/s
StompEncoderBenchmark.writeStompFrame              100          SEVEN              false  thrpt   10   901120.188 ±  71344.491  ops/s
StompEncoderBenchmark.writeStompFrame              100         ELEVEN               true  thrpt   10  1238317.714 ±  68148.477  ops/s
StompEncoderBenchmark.writeStompFrame              100         ELEVEN              false  thrpt   10   671336.339 ±  72735.337  ops/s
StompEncoderBenchmark.writeStompFrame              100         TWENTY               true  thrpt   10   754565.791 ±  28574.382  ops/s
StompEncoderBenchmark.writeStompFrame              100         TWENTY              false  thrpt   10   498939.383 ±  38146.118  ops/s
StompEncoderBenchmark.writeStompFrame             1000            ONE               true  thrpt   10  3722594.471 ± 515861.000  ops/s
StompEncoderBenchmark.writeStompFrame             1000            ONE              false  thrpt   10  1265629.633 ±  84113.347  ops/s
StompEncoderBenchmark.writeStompFrame             1000          THREE               true  thrpt   10  2829696.349 ± 172520.267  ops/s
StompEncoderBenchmark.writeStompFrame             1000          THREE              false  thrpt   10  1111454.609 ±  26275.913  ops/s
StompEncoderBenchmark.writeStompFrame             1000          SEVEN               true  thrpt   10  1901506.449 ±  37701.353  ops/s
StompEncoderBenchmark.writeStompFrame             1000          SEVEN              false  thrpt   10   912528.888 ±  46221.215  ops/s
StompEncoderBenchmark.writeStompFrame             1000         ELEVEN               true  thrpt   10  1299674.123 ±  21889.002  ops/s
StompEncoderBenchmark.writeStompFrame             1000         ELEVEN              false  thrpt   10   724527.644 ±   2757.370  ops/s
StompEncoderBenchmark.writeStompFrame             1000         TWENTY               true  thrpt   10   811389.799 ±   2606.626  ops/s
StompEncoderBenchmark.writeStompFrame             1000         TWENTY              false  thrpt   10   504955.449 ±   6737.804  ops/s
StompEncoderBenchmark.writeStompFrame            10000            ONE               true  thrpt   10  3837912.649 ± 380742.919  ops/s
StompEncoderBenchmark.writeStompFrame            10000            ONE              false  thrpt   10  1375544.306 ±   3157.068  ops/s
StompEncoderBenchmark.writeStompFrame            10000          THREE               true  thrpt   10  3224743.448 ± 297369.719  ops/s
StompEncoderBenchmark.writeStompFrame            10000          THREE              false  thrpt   10  1125772.007 ±   4051.498  ops/s
StompEncoderBenchmark.writeStompFrame            10000          SEVEN               true  thrpt   10  2127352.136 ± 106787.777  ops/s
StompEncoderBenchmark.writeStompFrame            10000          SEVEN              false  thrpt   10   934848.418 ±   4564.147  ops/s
StompEncoderBenchmark.writeStompFrame            10000         ELEVEN               true  thrpt   10  1379672.772 ±   8778.640  ops/s
StompEncoderBenchmark.writeStompFrame            10000         ELEVEN              false  thrpt   10   723169.459 ±   2317.767  ops/s
StompEncoderBenchmark.writeStompFrame            10000         TWENTY               true  thrpt   10   802275.113 ±   4155.137  ops/s
StompEncoderBenchmark.writeStompFrame            10000         TWENTY              false  thrpt   10   517604.265 ±   3398.384  ops/s
```
For headers over 256 bytes we get a speedup.
2020-12-07 09:00:52 +01:00
Violeta Georgieva
8a7c580bf6
Add whitelist entry for SSLEngineImpl.wrap to BlockHound configuration (#10844)
Motivation:

Internally SSLEngineImpl.wrap(...) may call FileInputStream.read(...).
This will cause the error below when BlockHound is enabled
reactor.blockhound.BlockingOperationError: Blocking call! java.io.FileInputStream#readBytes
	at java.io.FileInputStream.readBytes(FileInputStream.java)
	at java.io.FileInputStream.read(FileInputStream.java:255)

Modifications:

- Add whitelist entry to BlockHound configuration
- Add test

Result:

Fixes #10837
2020-12-07 08:06:21 +01:00
Norman Maurer
44f85bba5f
Ensure we don't leak the ClassLoader in the backtrace of TrackRecord.BOTTOM (#10839)
Motivation:

We need to ensure we override fillInStacktrace() when we store exceptions in static fields to not leak the Classloader in the backtrace.

Came up in https://github.com/netty/netty/pull/10691#issuecomment-738331186. Thanks to @amir-shalem for notice this one.

Modifications:

- Add overrides of fillInStracktrace in TrackRecord.BOTTOM

Result:

Related fix to https://github.com/netty/netty/pull/10686
2020-12-05 07:01:03 +01:00
Johnny Lim
37267f95ce
Initialize Recycler.INITIAL_CAPACITY before invoking InternalLogger.debug() (#10836)
Motivation:

If Log4J2's `Filter` creates `Recycler.Stack` somehow, `Recycler.Stack()` will see uninitialized `Recycler.INITIAL_CAPACITY`. This has been raised originally in https://github.com/micrometer-metrics/micrometer/issues/2369.

Modification:

This PR changes to initialize `Recycler.INITIAL_CAPACITY` before invoking `InternalLogger.debug()` to avoid it.

Result:

Fixes the problem described in the "Motivation" section.
2020-12-04 14:57:06 +01:00
Chris Vest
d2e16fb621
Fix a bug in LongPriorityQueue internal remove (#10832)
Motivation:
We rely on this functionality in PoolChunk, and a bug was caught by a non-deterministic test failure

Modification:
Went back to the Algorithms book, and reimplemented remove() the way it was meant to.

Result:
No test failures after 200.000 runs, so we have some confidence the code is correct now.
2020-12-02 13:06:00 +01:00
valerauko
b27f0fccce
Let Http2ConnectionHandler close stream with voidPromise (#10819)
Motivation:

Http2ConnectionHandler tries to addListener to the future without checking if it's void. If it is void, this will fail and generate an exception. 
 
Modifications:
Unvoid the promise in writeData()
 
Result:

Fixes #10816, Writing with a voidPromise no longer generates exceptions.
2020-12-02 10:11:39 +01:00
Chris Vest
c41d46111d
Create bespoke long/long hashmap and long-valued priority queue for PoolChunk (#10826)
Motivation:
The uncached access to PoolChunk can be made faster, and avoid allocating boxed Longs, if we have a primitive hash map and priority queue implementation for it.

Modification:
Add bespoke primitive implementations of a hash map and a priority queue for PoolChunk.
Remove all the long-boxing caused by the previous implementation.
The hashmap is a linear probing map with a fairly short probe that keeps the search within a couple of cache lines.
The priority queue is the same binary heap algorithm that's described in Algorithms by Sedgewick and Wayne.
The implementation avoids the Long boxing by relying on a long[] array.
This makes the internal-remove method faster, which is an important operation in PoolChunk.

Result:
Roughly 13% performance uplift in buffer allocations that miss cache.
2020-11-29 11:29:46 +01:00
Norman Maurer
567b46fa01
Don't use the cname cache when using DnsRecordResolveContext (#10808)
Motivation:

The DnsNameResolver internally follows CNAME indirects for all records types, and supports caching for CNAME resolution and A* records. For DNS record types that are not cached (e.g. SRV records) the caching of CNAME records may result in failures at incorrect times. For example if a CNAME record has a larger TTL than the entries it resolves this may result in failures which don't occur if the CNAME cache is disabled.

Modifications:

- Don't cache CNAME and also dont use the cache for CNAME when using DnsRecordResolveContext
- Add unit test

Result:

More correct resolving and also not possible to have failures due CNAME still be in the cache while the queried record experied
2020-11-26 15:34:56 +01:00
Norman Maurer
221c1a1ed7
Fix caching for normal allocations (#10825)
Motivation:

https://github.com/netty/netty/pull/10267 introduced a change that reduced the fragmentation. Unfortunally it also introduced a regression when it comes to caching of normal allocations. This can have a negative performance impact depending on the allocation sizes.

Modifications:

- Fix algorithm to calculate the array size for normal allocation caches
- Correctly calculate indeox for normal caches
- Add unit test

Result:

Fixes https://github.com/netty/netty/issues/10805
2020-11-25 15:05:30 +01:00
Norman Maurer
0c2b761cfb
OpenSsl.memoryAddress(...) should use internalNioBuffer(...) if it can't access the memoryAddress (#10818)
Motivation:

We can make use of internalNioBuffer(...) if we cant access the memoryAddress. This at least will reduce the object creations.

Modifications:

Use internalNioBuffer(...) and so reduce the GC

Result:

Less object creation if we can't access the memory address.
2020-11-25 10:31:58 +01:00
Norman Maurer
9cfe3bf5e3
Fix NPE in ByteToMessageDecoder if the user removes the handler while channelInputClosed(...) is processing the buffer. (#10817)
Motivation:

We need to carefully check for null before we pass the cumulation buffer into decodeLast as callDecode(...) may have removed the codec already and so set cumulation to null.

Modifications:

- Check for null and if we see null use Unpooled.EMPTY_BUFFEr
- Only call decodeLast(...) if callDecode(...) didnt remove the handler yet.

Result:

Fixes https://github.com/netty/netty/issues/10802
2020-11-24 14:08:32 +01:00
Alexandre Dutra
02cd85181a
Allow blocking calls inside SingleThreadEventExecutor.addTask (#10811)
Motivation:

GlobalEventExecutor.addTask was rightfully allowed to block by commit
09d38c8. However the same should have been done for
SingleThreadEventExecutor.addTask.

BlockHound is currently intercepting that call, and as a consequence,
it prevents SingleThreadEventExecutor from working properly, if addTask is
called from a thread that cannot block.

The interception is due to LinkedBlockingQueue.offer implementation,
which uses a ReentrantLock internally.

Modifications:

* Added one BlockHound exception to
io.netty.util.internal.Hidden.NettyBlockHoundIntegration for
SingleThreadEventExecutor.addTask.
* Also added unit tests for both SingleThreadEventExecutor.addTask
and GlobalEventExecutor.addTask.

Result:

SingleThreadEventExecutor.addTask can now be invoked from any thread
when BlockHound is activated.
2020-11-23 19:20:18 +01:00
Norman Maurer
ba83a8840f
IovArray should support when there is no unsafe present (#10814)
Motivation:

In some enviroments sun.misc.Unsafe is not present. We should support these as well.

Modifications:

Fallback to JNI if we can't directly access the memoryAddress of the buffer.

Result:

Fixes https://github.com/netty/netty/issues/10813
2020-11-23 14:03:32 +01:00
Alexandre Dutra
2ff8973f69
Allow blocking calls inside HashedWheelTimer start() and stop() (#10810)
Motivation:

When a HashedWheelTimer instance is started or stopped, its working
thread is started or stopped. These operations block the calling
thread:
- start() calls java.util.concurrent.CountDownLatch.await() to wait
for the worker thread to finish initializing;
- stop() calls java.lang.Thread.join(long) to wait for the worker
thread to exit.
BlockHound detects these calls and as a consequence, prevents
HashedWheelTimer from working properly, if it is started or stopped
in a thread that is not allowed to block.

Modifications:

Added two more BlockHound exceptions to
io.netty.util.internal.Hidden.NettyBlockHoundIntegration: one
for HashedWheelTimer.start() and one for HashedWheelTimer.stop().

Result:

HashedWheelTimer can now be started and stopped properly when
BlockHound is activated.
2020-11-23 08:41:07 +01:00
Benjamin Roux
c1ba23933c
Add ABORT and COMMIT STOMP commands to the StompCommand enum (#10790)
Motivation:

ABORT and COMMIT commands were missing from the enum but they are part of the STOMP spec.

Modifications:

Modified the enum to add the missing commands.

Result:

ABORT and COMMIT commands can now be parsed properly and acted on.
2020-11-19 14:23:37 +01:00
Andrey Mizurov
f40b4f15a6
Override Sec-WebSocket-Protocol websocket handshake response header after custom headers to avoid duplication (#10793)
Motivation:

According rfc (https://tools.ietf.org/html/rfc6455#section-11.3.4), `Sec-WebSocket-Protocol` header field MUST NOT appear
more than once in an HTTP response.
At the moment we can pass `Sec-WebSocket-Protocol`  via custom headers and it will be added to response.

Modification:

Change method add() to set() for avoid duplication. If we pass sub protocols in handshaker constructor it means that they are preferred over custom ones.

Result:

Less error prone behavior.
2020-11-19 09:49:44 +01:00
Chris Vest
3354c7b0bf
Let object serialisation exceptions propagate in the Object Echo example (#10807)
Motivation:
People may use the object serialisation example as a vehicle to test out sending their own objects across the wire.
If those objects are not actually serialisable for some reason, then we need to let the exception propagate so that this becomes obvious to people.

Modification:
Add a listener to the future that sends the first serialisable message, so that we ensure that any exceptions that shows up during serialisation becomes visible.
Without this, the state of the future that sent the first message was never checked or inspected anywhere.

Result:
Serialisation bugs in code derived from the Object Echo example are much easier to diagnose.

This fixes #10777
2020-11-19 08:10:17 +01:00
Frédéric Brégier
1c230405fd
Fix for performance regression on HttpPost RequestDecoder (#10623)
Fix issue #10508 where PARANOID mode slow down about 1000 times compared to ADVANCED.
Also fix a rare issue when internal buffer was growing over a limit, it was partially discarded
using `discardReadBytes()` which causes bad changes within previously discovered HttpData.

Reasons were:

Too many `readByte()` method calls while other ways exist (such as keep in memory the last scan position when trying to find a delimiter or using `bytesBefore(firstByte)` instead of looping externally).

Changes done:
- major change on way buffer are parsed: instead of read byte per byte until found delimiter, try to find the delimiter using `bytesBefore()` and keep the last unfound position to skeep already parsed parts (algorithms are the same but implementation of scan are different)
- Change the condition to discard read bytes when refCnt is at most 1.

Observations using Async-Profiler:
==================================

1) Without optimizations, most of the time (more than 95%) is through `readByte()` method within `loadDataMultipartStandard` method.
2) With using `bytesBefore(byte)` instead of `readByte()` to find various delimiter, the `loadDataMultipartStandard` method is going down to 19 to 33% depending on the test used. the `readByte()` method or equivalent `getByte(pos)` method are going down to 15% (from 95%).

Times are confirming those profiling:
- With optimizations, in SIMPLE mode about 82% better, in ADVANCED mode about 79% better and in PARANOID mode about 99% better (most of the duplicate read accesses are removed or make internally through `bytesBefore(byte)` method)

A benchmark is added to show the behavior of the various cases (one big item, such as File upload, and many items) and various level of detection (Disabled, Simple, Advanced, Paranoid). This benchmark is intend to alert if new implementations make too many differences (such as the previous version where about PARANOID gives about 1000 times slower than other levels, while it is now about at most 10 times).

Extract of Benchmark run:
=========================

Run complete. Total time: 00:13:27

Benchmark                                                                           Mode  Cnt  Score   Error   Units
HttpPostMultipartRequestDecoderBenchmark.multipartRequestDecoderBigAdvancedLevel   thrpt    6  2,248 ± 0,198 ops/ms
HttpPostMultipartRequestDecoderBenchmark.multipartRequestDecoderBigDisabledLevel   thrpt    6  2,067 ± 1,219 ops/ms
HttpPostMultipartRequestDecoderBenchmark.multipartRequestDecoderBigParanoidLevel   thrpt    6  1,109 ± 0,038 ops/ms
HttpPostMultipartRequestDecoderBenchmark.multipartRequestDecoderBigSimpleLevel     thrpt    6  2,326 ± 0,314 ops/ms
HttpPostMultipartRequestDecoderBenchmark.multipartRequestDecoderHighAdvancedLevel  thrpt    6  1,444 ± 0,226 ops/ms
HttpPostMultipartRequestDecoderBenchmark.multipartRequestDecoderHighDisabledLevel  thrpt    6  1,462 ± 0,642 ops/ms
HttpPostMultipartRequestDecoderBenchmark.multipartRequestDecoderHighParanoidLevel  thrpt    6  0,159 ± 0,003 ops/ms
HttpPostMultipartRequestDecoderBenchmark.multipartRequestDecoderHighSimpleLevel    thrpt    6  1,522 ± 0,049 ops/ms
2020-11-19 08:00:35 +01:00
Aayush Atharva
8b2ed77042
Fix comment typo DelegatingDecompressorFrameListener (#10789)
Motivation:
`DelegatingDecompressorFrameListener#initDecompressor` has multiple dots `.` in comments. However, it should not have that.

Modification:
Removed multiple dots.

Result:
Clean comment
2020-11-16 09:03:37 +01:00
Ech0Fan
c717d4b97a
Fix UnsafeByteBufUtil#setBytes() cause JVM crash (#10791) (#10795)
Motivation:

Passing a null value of byte[] to the `Unsafe.copyMemory(xxx)` would cause the JVM crash 

Modification:

Add null checking before calling `PlatformDependent.copyMemory(src,  xxx)`

Result:

Fixes #10791 .
2020-11-16 09:01:01 +01:00
root
944a020586 [maven-release-plugin] prepare for next development iteration 2020-11-11 05:47:51 +00:00
root
715353ecd6 [maven-release-plugin] prepare release netty-4.1.54.Final 2020-11-11 05:47:37 +00:00
root
5557d77090 Use correct netty-build artifact
Motivation:

2d1b143dfa missed to change the artifactId in one place

Modification:

Change to netty-build-common

Result:

Release works
2020-11-10 12:04:40 +00:00
root
afca81a9d8 [maven-release-plugin] rollback the release of netty-4.1.54.Final 2020-11-10 12:02:24 +00:00
root
e256074e49 [maven-release-plugin] prepare for next development iteration 2020-11-10 11:12:23 +00:00
root
cea659bd8a [maven-release-plugin] prepare release netty-4.1.54.Final 2020-11-10 11:12:06 +00:00
Norman Maurer
5ffca6ef4a
Use http in xmlns URIs to make maven release plugin happy again (#10788)
Motivation:

https in xmlns URIs does not work and will let the maven release plugin fail:

```
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  1.779 s
[INFO] Finished at: 2020-11-10T07:45:21Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-release-plugin:2.5.3:prepare (default-cli) on project netty-parent: Execution default-cli of goal org.apache.maven.plugins:maven-release-plugin:2.5.3:prepare failed: The namespace xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" could not be added as a namespace to "project": The namespace prefix "xsi" collides with an additional namespace declared by the element -> [Help 1]
[ERROR]
```

See also https://issues.apache.org/jira/browse/HBASE-24014.

Modifications:

Use http for xmlns

Result:

Be able to use maven release plugin
2020-11-10 10:22:35 +01:00
Aayush Atharva
ff638cac9c
Add ByteBuf parameter HttpConversionUtil (#10785)
Motivation:
`HttpConversionUtil#toFullHttpResponse` and `HttpConversionUtil#toFullHttpRequest` has `ByteBufAllocator` which is used for building `FullHttpMessage` and then data can be appended with `FullHttpMessage#content`. However, there can be cases when we already have `ByteBuf` ready with data. So we need a parameter to add `ByteBuf` directly into `FullHttpMessage` while creating it.

Modification:
Added `ByteBuf` parameter,

Result:
More functionality for handling `FullHttpMessage` content.
2020-11-10 07:56:36 +01:00
Norman Maurer
1c37c8f072
Use special exception when failing because the SSLEngine was closed (#10783)
Motivation:

Sometimes it would be helpful to easily detect if an operation failed due the SSLEngine already be closed.

Modifications:

Add special exception that is used when the engine was closed

Result:

Easier to detect a failure caused by a closed exception
2020-11-09 15:33:14 +01:00
Chris Vest
b27d837974
Smaller output in codeql build (#10787)
Motivation:
Printing download progress in the build log makes it harder to see what's wrong when the build fails.

Modification:
Change the maven command to not show transfer progress, also enable batch mode so Maven don't print in colors that we can't see anyway.

Result:
More concise code analysis build logs.
2020-11-09 15:32:18 +01:00
Norman Maurer
2d1b143dfa
Update netty-build version (#10780)
Motivation:

We recently released a new netty-build version and changed the artifact name

Modifications:

Update version and artifact name

Result:

Use latest version
2020-11-08 09:01:22 +01:00
Aayush Atharva
76d5cdb9e8
Add HttpScheme Support in HttpToHttp2ConnectionHandler (#10641)
Motivation:
We should have a method to add `HttpScheme` if `HttpRequest` does not contain `x-http2-scheme` then we should use add it if `HttpToHttp2ConnectionHandler` is build using specified `HttpScheme`.

Modification:
Added `HttpScheme` in `HttpToHttp2ConnectionHandlerBuilder`.

Result:
Automatically add `HttpScheme` if missing in `HttpRequest`.
2020-11-05 15:53:32 +01:00
Eric Anderson
027a686042
codec-http2: Correct last-stream-id for HEADERS-triggered connection error (#10775)
Motivation:

When parsing HEADERS, connection errors can occur (e.g., too large of
headers, such that we don't want to HPACK decode them). These trigger a
GOAWAY with a last-stream-id telling the client which streams haven't
been processed.

Unfortunately that last-stream-id didn't include the stream for the
HEADERS that triggered the error. Since clients are free to silently
retry streams not included in last-stream-id, the client is free to
retransmit the request on a new connection, which will fail the
connection with the wrong last-stream-id, and the client is still free
to retransmit the request.

Modifications:

Have fatal connection errors (those that hard-cut the connection)
include all streams in last-stream-id, which guarantees the HEADERS'
stream is included and thus should not be silently retried by the HTTP/2
client.

This modification is heavy-handed, as it will cause racing streams to
also fail, but alternatives that provide precise last-stream-id tracking
are much more invasive. Hard-cutting the connection is already
heavy-handed and so is rare.

Result:

Fixes #10670
2020-11-05 09:07:28 +01:00
Arthur Gonigberg
b63e2dfb1b
Drop unknown frames on connection stream (#10771)
Motivation:

We received a [bug report](https://bugs.chromium.org/p/chromium/issues/detail?id=1143320) from the Chrome team at Google, their canary builds are failing [HTTP/2 GREASE](https://tools.ietf.org/html/draft-bishop-httpbis-grease-00) testing to netflix.com.

The reason it's failing is that Netty can't handle unknown frames without an active stream created. Let me know if you'd like more info, such as stack traces or repro steps. 

Modification:

The change is minor and simply ignores unknown frames on the connection stream, similarly to `onWindowUpdateRead`.

Result:

I figured I would just submit a PR rather than filing an issue, but let me know if you want me to do that for tracking purposes.
2020-11-04 14:01:08 +01:00
Aayush Atharva
23864d25b9
Fix License type of dnsinfo (#10773)
Motivation:
`dnsinfo` uses `Apple Public Source License 2.0` not `Apache License 2.0`.

Modification:
Changed `Apache License 2.0` to `Apple Public Source License 2.0`

Result:
Fixes #10772
2020-11-04 10:40:43 +01:00
Norman Maurer
ab56fb6f44
Update to latest java 8/11/15 versions (#10774)
Motivation:

There were new releases of java.

Modifications:

Update java versions so we use the latest on the CI

Result:

Use latest releases
2020-11-04 10:39:55 +01:00
Chris Vest
1c0662ea42
Use JUnit 5 for running all tests (#10764)
Motivation:
JUnit 5 is the new hotness. It's more expressive, extensible, and composable in many ways, and it's better able to run tests in parallel. But most importantly, it's able to directly run JUnit 4 tests.
This means we can update and start using JUnit 5 without touching any of our existing tests.
I'm also introducing a dependency on assertj-core, which is like hamcrest, but arguably has a nicer and more discoverable API.

Modification:
Add the JUnit 5 and assertj-core dependencies, without converting any tests at time time.

Result:
All our tests are now executed through the JUnit 5 Vintage Engine.
Also, the JUnit 5 test APIs are available, and any JUnit 5 tests that are added from now on will also be executed.
2020-11-04 10:19:59 +01:00
Norman Maurer
c6e2934357
Revert "Allow and skip null handlers when adding a vararg list of handlers (#10751)" (#10770)
This reverts commit 3b90b536bb.
2020-11-03 21:12:29 +01:00
Bennett Lynch
3b90b536bb
Allow and skip null handlers when adding a vararg list of handlers (#10751)
Motivation:

Allowing null handlers allows for more convenient idioms in
conditionally adding handlers, e.g.,

ch.pipeline().addLast(
        new FooHandler(),
        condition ? new BarHandler() : null,
        new BazHandler()
);

Modifications:

* Change addFirst(..) and addLast(..) to skip null handlers, rather than
break or short-circuit.
* Add new unit tests.

Result:

* Makes addFirst(..) and addLast(..) behavior more consistent
* Resolves https://github.com/netty/netty/issues/10728
2020-11-03 21:11:35 +01:00
Scott Mitchell
7e1147ea4f
Avoid auto boxing in PoolChunk#removeAvailRun (#10769)
Motivation:
PoolChunk maintains multiple PriorityQueue<Long> collections. The usage
of PoolChunk#removeAvailRun unboxes the Long values to long, and then
this method uses queue.remove(..) which will auto box the value back to
Long. This creates unnecessary allocations via Long.valueOf(long).

Modifications:
- Adjust method signature and usage of PoolChunk#removeAvailRun to avoid
boxing

Result:
Less allocations as a result of PoolChunk#removeAvailRun.
2020-11-03 21:08:43 +01:00
Aayush Atharva
c5077a3d87
HttpConversionUtil#toHttpResponse should use false in isRequest parameter (#10760)
Motivation:
`HttpConversionUtil#toHttpResponse` translates `Http2Headers` to `HttpResponse`. It uses `#addHttp2ToHttpHeaders(..., boolean isRequest)` to do so. However, `isRequest` field is set to `true` instead of `false`. It should be set to `false` because we're doing conversion of Response not Request.

Modification:
Changed `true` to `false`.

Result:
Correctly translates `Http2Headers` to `HttpResponse`.
2020-11-03 09:38:51 +01:00
Aayush Atharva
d1cf9774d5
Add toString method in DefaultHttp2WindowUpdateFrame (#10763)
Motivation:
We should have the `toString` method in `DefaultHttp2WindowUpdateFrame` because it makes debugging a little easy.

Modification:
Added `toString` method.

Result:
`toString` method to help in debugging.


Co-authored-by: Norman Maurer <norman_maurer@apple.com>
2020-11-03 09:37:18 +01:00
Benjamin Roux
175d1368e5
Revert "Add support for heartbeat in STOMP decoder/encoder. (#10695)" (#10766)
This reverts commit 81544ab94f.
2020-11-03 09:36:15 +01:00
Aayush Atharva
1492374f99
Remove extra empty line (#10754)
Motivation:
`Http2Frame` has extra empty line after `String name();`. However, it should not be there.

Modification:
Removed extra empty line.

Result:
Empty-line code style now matching with other classes.
2020-11-02 15:07:51 +01:00
Aayush Atharva
0b0b446d38
Fix typo in Http2FrameCodec#write(...) comment
Motivation:
`Http2FrameCodec#write(...)` has typo in comment.
`// In the event of manual SETTINGS ACK is is assumed the encoder will apply the earliest received but not`.
The typo is `is is`. However, it should be `it is`.

Modification:
Changed `is is` to `it is`.

Result:
Correct comment without typos.
2020-11-02 08:49:13 +01:00
Aayush Atharva
1efc5f81e2
Fix typo in Http2DataFrame javadocs (#10755)
Motivation:
`Http2DataFrame#isEndStream()` JavaDoc says `Returns {@code true} if the END_STREAM flag ist set.`. The typo is `ist` word. However, it should be `is`.

Modification:
Changed `ist` to `is`.

Result:
Better JavaDoc by fixing the typo.
2020-10-30 15:59:31 +01:00
Aayush Atharva
87c46113d1
Fix typo in Http2HeadersFrame javadocs (#10756)
Motivation:
`Http2HeadersFrame#isEndStream()` JavaDoc says `Returns {@code true} if the END_STREAM flag ist set.`. The typo is `ist` word. However, it should be `is`.

Modification:
Changed `ist` to `is`.

Result:
Better JavaDoc by fixing the typo.
2020-10-30 15:58:34 +01:00
Dave Syer
d62384b227
Add reflection config for JMX runtime (#10753)
Motivation:

`DefaultChannelId` uses reflection to access the JMX runtime. This needs some extra config for GraalVM.

Modification:

Add config for GraalVM

Result:

Works when using GraalVM native image
2020-10-30 15:26:41 +01:00
Benjamin Roux
81544ab94f
Add support for heartbeat in STOMP decoder/encoder. (#10695)
Motivation:

Heart-beat is a functionality of STOMP enabling clients and servers to know the healthiness of the connection. The current decoder didn't allow for heart-beat messages to be forwarded to the decoder and were simply swallowed as part of the frame decoding.

Modifications:

Adding support for heartbeat message parsing by introducing a new HEARTBEAT command (not a real STOMP command).
Heartbeat received on the channel will trigger a StompFrame with the command set to HEARTBEAT.
Sending heartbeat on the channel is achieved by creating a StompFrame with the command set to HEARTBEAT.

Result:

Heartbeat can now be received/sent and acted upon to determine the healthiness of the connection and terminate it if needed.
2020-10-30 14:32:57 +01:00