Commit Graph

10447 Commits

Author SHA1 Message Date
Norman Maurer
842e73f8d3
Migrate microbenchmark to junit5 (#11440) (#11443)
Motivation:

We should update to use junit5 in all modules.

Modifications:

Adjust microbenchmark to use junit5

Result:

Part of https://github.com/netty/netty/issues/10757
2021-07-02 08:05:03 +02:00
Norman Maurer
7c6be99da0
Migrate rest of epoll module to junit5 (#11442)
Motivation:

We should update to use junit5 in all modules.

Modifications:

Adjust missing epoll tests to use junit5

Result:

Part of https://github.com/netty/netty/issues/10757
2021-07-02 08:04:48 +02:00
Norman Maurer
5179e53294
Migrate codec-http to junit5 (#11440)
Motivation:

We should update to use junit5 in all modules.

Modifications:

Adjust codec-http tests to use junit5

Result:

Part of https://github.com/netty/netty/issues/10757
2021-07-01 18:53:41 +02:00
Norman Maurer
c82c17782c
Migrate common to junit5 (#11431) (#11438)
Motivation:

We should update to use junit5 in all modules.

Modifications:

Adjust common tests to use junit5

Result:

Part of https://github.com/netty/netty/issues/10757
2021-07-01 18:19:44 +02:00
Norman Maurer
8003ea8a03
Migrate transport-udt to junit5 (#11439)
Motivation:

We should update to use junit5 in all modules.

Modifications:

Adjust transport-udt tests to use junit5

Result:

Part of https://github.com/netty/netty/issues/10757
2021-07-01 18:19:24 +02:00
Aleksandar Gradinac
bc206ff173
Delay initialization of io.netty.handler.codec.compression.BrotliDecoder to runtime (#11428)
Motivation:
Including codec-http in the project and building a native-image out of it using a GraalVM 21.2 nightly can result in a failure.

Modification:
By delaying the initialization of `io.netty.handler.codec.compression.BrotliDecoder` to runtime, native-image will not try to eagerly initialize the class during the image build, avoiding the build failure described in the issue.

Result:
Fixes #11427
2021-07-01 15:08:51 +02:00
Aayush Atharva
750d23583c
Add ALPN Buffering to support HTTP/2 Prior Knowledge (#11407)
Motivation:
Currently, Netty cannot handle HTTP/2 Preface messages if the client used the Prior knowledge technique. In Prior knowledge, the client sends an HTTP/2 preface message immediately after finishing TLS Handshake. But in Netty, when TLS Handshake is finished, ALPNHandler is triggered to configure the pipeline. And between these 2 operations, if an HTTP/2 preface message arrives, it gets dropped.

Modification:

Buffer messages until we are done with the ALPN handling.

Result:
Fixes #11403.

Co-authored-by: Norman Maurer <norman_maurer@apple.com>
2021-07-01 14:10:52 +02:00
Norman Maurer
126e9f873b
Revert "Fix deprecated GC logging switch (#11395)" (#11436)
This reverts commit 077e5f282c.
2021-07-01 12:09:52 +02:00
Alex Budovski
077e5f282c
Fix deprecated GC logging switch (#11395)
Motivation:

This caused test failures due to the deprecation warning and produced a
dumpstream.

Modification:

Replace deprecated flag with recommended one.

Result:
Fix deprecation and cause of test failure in codec project.
2021-07-01 11:30:30 +02:00
Maksym Ostroverkhov
2abe20a6b5
Tail tasks queue: configure separately from tasks queue (#11400)
Motivation:

IO transports (primarily epoll, but also applies to kqueue, nio) cant be configured with separate tail tasks queue factory -
instead single queue factory is used for both normal tasks and tail tasks.

Modifications:

Add constructor accepting tail EventLoopTaskQueueFactory to aforementioned transports

Result:

IO transports can be configured with separate tail tasks
2021-07-01 10:51:10 +02:00
Norman Maurer
20e4ccbd33
Skip Brotli related tests on platforms which not support Brotli (#11435)
Motivation:

The native module is not yet available on aarch64 Mac / Windows thus causing tests in codec/ to fail (specifically all the Brotli ones, since the module could not be loaded).

Modification:

Disable Brotli tests when platform is not supported

Result:

Tests under codec/ now pass under Mac/aarch64 and Windows/aarch64
2021-07-01 09:59:24 +02:00
wujimin
3226e77485
Add support for GMSSL (#11406) (#11410)
__Motivation__

Add support for GMSSL protocol to SslUtils.

__Modification__

Modify `SslUtils.getEncryptedPacketLength(ByteBuf buffer, int offset)` to get packet length when protocol is GMSSL.
Modify `SslUtils.getEncryptedPacketLength(ByteBuffer buffer)` to get packet length when protocol is GMSSL.

__Result__

`SslUtils.getEncryptedPacketLength` now supports GMSSL protocol. Fixes https://github.com/netty/netty/issues/11406
2021-07-01 08:17:45 +02:00
Stephane Landelle
801819b359
Fix HttpHeaderValue#XML_HTTP_REQUEST case (#11433)
Motivation:

HTTP header values are case sensitive. The expected value for `x-request-with` header is `XMLHttpRequest`, not `XmlHttpRequest`.

Modification:

Fix constant's case.

Result:

Correct `XMLHttpRequest` HTTP header value.
2021-07-01 08:13:10 +02:00
Tamara Braun
194a81ff4f
Fix Buffer Overflow During Lz4FrameEncoder Close (#11429)
Motivation:

We failed to account for the last header when estimating the buffer
size. If the data does not compress enough to make space for the
last header we would exceed the ByteBuf's capacity.

Modifications:

Call #ensureWritable with appropriate capacity for footer ByteBuf
befor writing footer.

Result:

If there is not enough space left in the buffer, the buffer will be
expanded.
2021-06-30 18:47:07 +02:00
Norman Maurer
8d76f402b1
Migrate transport to junit5 (#11431) (#11432)
Motivation:

We should update to use junit5 in all modules.

Modifications:

Adjust transport tests to use junit5

Result:

Part of https://github.com/netty/netty/issues/10757
2021-06-30 16:17:57 +02:00
Norman Maurer
94a4880358
Migrate codec-mqtt to junit5 (#11431)
Motivation:

We should update to use junit5 in all modules.

Modifications:

Adjust codec-mqtt tests to use junit5

Result:

Part of https://github.com/netty/netty/issues/10757
2021-06-30 15:32:16 +02:00
Norman Maurer
3716575102
Migrate codec to junit5 (#11426) (#11430)
Motivation:

We should update to use junit5 in all modules.

Modifications:

Adjust codec tests to use junit5

Result:

Part of https://github.com/netty/netty/issues/10757
2021-06-30 14:12:50 +02:00
Norman Maurer
be606df4f0
Migrate codec-redis to junit5 (#11426)
Motivation:

We should update to use junit5 in all modules.

Modifications:

Adjust codec-redis tests to use junit5

Result:

Part of https://github.com/netty/netty/issues/10757
2021-06-30 13:59:57 +02:00
Norman Maurer
2c3e78566e
Migrate resolver to junit5 (#11425)
Motivation:

We should update to use junit5 in all modules.

Modifications:

Adjust resolver tests to use junit5

Result:

Part of https://github.com/netty/netty/issues/10757
2021-06-30 12:19:02 +02:00
Norman Maurer
29be99c538
Migrate the rest of codec-http2 to junit5 (#11424)
Motivation:

8c73dbe9bd did migrate the codec-http2 code to use junit5 but missed two classes.

Modifications:

Adjust the rest of codec-http2 tests to use junit5

Result:

Part of https://github.com/netty/netty/issues/10757
2021-06-30 11:11:25 +02:00
Norman Maurer
8c73dbe9bd
Migrate codec-http2 to junit5 (#11422)
Motivation:

We should update to use junit5 in all modules.

Modifications:

Adjust codec-http2 tests to use junit5

Result:

Part of https://github.com/netty/netty/issues/10757
2021-06-30 10:32:01 +02:00
Unev
d8ad931488 ByteBufFormat constructor for LoggingHandler (#11420)
__Motivation__

`LoggingHandler` misses a constructor variant that only takes `ByteBufFormat`

__Modification__

Added the missing constructor variant.

__Result__

`LoggingHandler` can be constructed with `ByteBufFormat` only.

Co-authored-by: Nitesh Kant <nitesh_kant@apple.com>
2021-06-29 10:28:27 -07:00
skyguard1
d99a8f75b4
Use Two way algorithm to optimize ByteBufUtil.indexOf() method (#11367)
Use Two way algorithm to optimize ByteBufUtil.indexOf() method

Motivation:

ByteBufUtil.indexOf can be inefficient for substring search on
ByteBuf, in terms of algorithm complexity (O(needle.readableBytes * haystack.readableBytes)), consider using the Two Way algorithm to optimize the ByteBufUtil.indexOf() method

Modification:

Use the Two Way algorithm to optimize ByteBufUtil.indexOf() method.

Result:

The performance of the ByteBufUtil.indexOf() method is higher than the original implementation
2021-06-28 11:07:17 +02:00
Norman Maurer
6c618e30af
CombinedChannelDuplexHandler.removeOutboundHandler() cause connect(...) to not pass the correct parameters (#11414)
Motivation:

Due a bug we did not pass the correct remote and localaddress to the next handler if the outbound portion of the CombinedChannelDuplexHandler was removed

Modifications:

- Call the correct connect(...) method
- Refactor tests to test that the parameters are correctly passed on
- Remvoe some code duplication in the tests

Result:

CombinedChannelDuplexHandler correctly pass parameters on
2021-06-24 13:58:17 +02:00
Norman Maurer
a71ec15fc4
Correctly use HandshakeStatus.NEED_WRAP when a handshake failed and a alert was produced (#11412)
Motivation:

We need to ensure we always "consumed" all alerts etc via SSLEngine.wrap(...) before we teardown the engine. Failing to do so may lead to a situation where the remote peer will not be able to see the actual cause of the handshake failure but just see the connection being closed.

Modifications:

Correctly return HandshakeStatus.NEED_WRAP when we need to wrap some data first before we shutdown the engine because of a handshake failure.

Result:

Fixes https://github.com/netty/netty/issues/11388
2021-06-24 10:05:47 +02:00
Norman Maurer
6edb231bf5
Update to netty-tcnative 2.0.40.Final (#11411)
Motivation:

A new version of netty-tcnative was released

Modifications:

Update to 2.0.40.Final

Result:

Use latest version
2021-06-23 14:21:59 +02:00
Nitesh Kant
5a658bb887
HttpUtil#normalizeAndGetContentLength() should handle empty value (#11409)
__Motivation__

`HttpUtil#normalizeAndGetContentLength()` throws `StringIndexOutOfBoundsException` for empty `content-length` values, it should instead throw `IllegalArgumentException` for all invalid values.

__Modification__

- Throw `IllegalArgumentException` if the `content-length` value is empty.
- Add tests

__Result__

Fixes https://github.com/netty/netty/issues/11408
2021-06-23 12:07:16 +02:00
ping
98a3a0c0cb
Recycler.WeakOrderQueue drop Object hasBeenRecycled (#11402)
Motivation:

WeakOrderQueue would drop object that has been recycled, even when it has space for it.
WeakOrderQueue#add should check DefaultHandler.hasBeenRecycler field  first

Modifications:

WeakOrderQueue test the DefaultHandler.hasBeenRecycler first

Result:

WeakOrderQueue would not drop object that has been recycled when there is space


Co-authored-by: Norman Maurer <norman_maurer@apple.com>
Co-authored-by: Trustin Lee <t@motd.kr>
2021-06-23 11:32:44 +02:00
Paulo Lopes
956e1da2ad
Update graal annotations dependencies GAV to allow license GPL2+CE (#11404)
Motivation:

We need to use a GraalVM dependency which uses GPL2 + CE.

Modifications:

- Update all graalvm dependencies to new GAV which introduces a license change from GPL2 to GPL2 + CE
- This also required a small bump on the general version from 19.2 to 19.3, which should be fine as 19.3 is an official maintained LTS version, while 19.2 wasn't.

Result:

Fixes: #11398

Signed-off-by: Paulo Lopes <pmlopes@gmail.com>
2021-06-21 16:10:39 +02:00
Chris Vest
ac177753d8
Add GraalVM to the PR build matrix (#11405)
Motivation:

Native image compatibility is fragile and breaks easily, so we need a PR build to tell us when this happens.

Modification:

Add a graalvm-based build to the PR build matrix.

Result:

Every PR is now also tested on Graal.
2021-06-21 16:08:40 +02:00
Norman Maurer
0c9a86db81
Log if the user tries to explicit set TLSv1.3 ciphers and using BoringSSL (#11392)
Motivation:

At the moment BoringSSL doesnt support explicit set the TLSv1.3 ciphers that should be used. If TLSv1.3 should be used it just enables all ciphers. We should better log if the user tries to explicit set a specific ciphers and using BoringSSL to inform the user that what is tried doesnt really work.

Modifications:

Log if the user tries to not use all TLSv1.3 ciphers and use BoringSSL

Result:

Easier for the user to understand why always all TLSv1.3 ciphers are enabled when using BoringSSL

Co-authored-by: Trustin Lee <trustin@gmail.com>
2021-06-21 08:54:58 +02:00
ktqco
f1742c0e43
Accept smaller server_max_window_bits than requested (#11394)
Motivation:

Netty will fail a handshake for the Per-Message Deflate WebSocket
extension if the server response contains a smaller
`server_max_window_bits` value than the client offered.

However, this is allowed by RFC 7692:

> A server accepts an extension negotiation offer with this parameter
> by including the “server_max_window_bits” extension parameter in the
> extension negotiation response to send back to the client with the
> same or smaller value as the offer.

Modifications:

- Allow the server to respond with a smaller value than offered.
- Change the unit tests to test for this.

Result:

The client will not fail when the server indicates it is using a
smaller window size than offered by the client.
2021-06-18 11:45:32 +02:00
Hylke van der Schaaf
2f4f7135fb
Validate fixed header bits in MQTT (#11389)
Motivation:
The MQTT spec states that the bits in the fixed header must be set to specific values depending on message type. If a client sends a message with the wrong bits, the server must treat the message as malformed. Netty did not check the value of the reserved bits in the fixed header.

See:
MQTT3.1.1: http://docs.oasis-open.org/mqtt/mqtt/v3.1.1/errata01/os/mqtt-v3.1.1-errata01-os-complete.html#_Toc442180835
MQTT 5.0: https://docs.oasis-open.org/mqtt/mqtt/v5.0/os/mqtt-v5.0-os.html#_Toc3901023


Modification:
Add validation checks to MqttDecoder.java
Add unit tests to MqttCodecTest.java 
Fixed two instances where messages were generated for other unit tests with an incorrect fixed header.

Result:
Fixes #11379.
2021-06-16 14:59:15 +02:00
Norman Maurer
0d5774a82b
Make all compression codecs support buffers that don't have arrays (#11383) (#11387)
Motivation:
Various compression codecs are currently hard-coded to only support buffers that are backed by byte-arrays that they are willing to expose.
This is efficient for most of the codecs, but compatibility suffers, as we are not able to freely choose our buffer implementations when compression codecs are involved.

Modification:
Add code to the compression codecs, that allow them to handle buffers that don't have arrays.
For many of the codecs, this unfortunately involves allocating temporary byte-arrays, and copying back-and-forth.
We have to do it that way since some codecs can _only_ work with byte-arrays.
Also add tests to verify that this works.

Result:
It is now possible to use all of our compression codecs with both on-heap and off-heap buffers.
The default buffer choice has not changed, however, so performance should be unaffected.

Co-authored-by: Chris Vest <christianvest_hansen@apple.com>
2021-06-15 08:04:44 +02:00
Violeta Georgieva
50e0e8c5ca
Skip the windows tests when there is an entry for localhost in the hosts file (#11385)
Motivation:

The tests must be executed only when there is no hosts file or
there is no entry for localhost in the hosts file. The tested functionality
is relevant only in these use cases.

Modifications:

Skip the windows tests when there is an entry for localhost in the hosts file.

Result:

Fix failing tests on Windows CI when using GitHub Actions
Related to #11384
2021-06-14 09:05:44 +02:00
Julien Viet
06f7deb030
The MqttDecoder incorrectly skip bytes before throwing TooLongFrameException (#11362)
Motivation:

Commit c32c520edd incorrectly skip the bytes of the replay decoder buffer. The number of bytes to skip is determined by ByteBuf#readableBytes() instead of using ByteToMessageDecoder#actualReadableBytes(). As result it throws an exception because the ByteBuf provided will return a too large value (Integer.MAX_VALUE - reader index) causing a bound check error in the skipBytes method. This is not detected by the tests because most tests are calling the decode(...) method with a regular ByteBuf. In practice when this method is called with a specialized ByteBuf when channelRead(...) is called. Such tests should actually use channelRead with proper mocking of the ChannelHandlerContext

Modification:

- Rewrite the MqttCodecTest to use channelRead(...) instead of decode(...) and use proper mocking of ChannelHandlerContext to get the message emitted by the decoder.
- Use actualReadableBytes() instead of buff.readableBytes() to compute the number of bytes to skip

Result:

Skip correctly the number of bytes when a too large message is found and improve testing. See #11361

Signed-off-by: Julien Viet <julien@julienviet.com>
2021-06-10 15:05:25 +02:00
Norman Maurer
c78afbc4f1
Remove tarball module (#11377)
Motivation:

We don't publish any tarballs these days so we can just remove the module

Modifications:

Remove tarball module and also adjust release scripts

Result:

Less code / config to mantain
2021-06-10 10:19:18 +02:00
anotherJJz
e12ae47666
Modify List to Map of pooled redis message in FixedRedisMessagePool (#11300)
Motivation:

To simplify retrieving pooled message messages, add enums that can be used as key.

Modifications:

- Modify pooled collections from List to Map in FixedRedisMessagePool
- Allow to use enum as the key to easy get pooled message.
- Add unit tests

Result:

Users can get pooled message by enum instead of the whole string

Co-authored-by: Norman Maurer <norman_maurer@apple.com>
2021-06-09 12:54:42 +02:00
Nitesh Kant
625a7a1075
HttpUtil.getCharset() fails for charset in double-quotes (#11373)
__Motivation__

As described in https://github.com/netty/netty/issues/11370 we should support quoted charset values

__Modification__

Modify `HttpUtil.getCharset(CharSequence contentTypeValue, Charset defaultCharset)` to trim the double-quotes if present.

__Result__

`HttpUtil.getCharset()` now supports quoted charsets. Fixes https://github.com/netty/netty/issues/11370
2021-06-09 12:37:23 +02:00
Violeta Georgieva
e69107ceaf
Fix ServerCookieDecoder javadoc (#11372)
Motivation:
When decoding the cookies on the server, the "Cookie" HTTP request header value should be considered.
The "Set-Cookie" HTTP response header is used to send cookies from the server to the user agent.

Modification:
- Specify in javadoc that the "Cookie" HTTP request header value should be considered and
not the "Set-Cookie" HTTP response header value.

Result:
Correct ServerCookieDecoder javadoc
2021-06-07 16:33:31 -07:00
Chris Vest
d5d6c3d3cf
Make build log output less chatty (#11365)
Motivation:
When Maven does not run in batch mode, it will continuously print its progress as it downloads dependencies.
This can produce a very large amount of log output, that makes it harder to debug build failures.

Modification:
Make all Maven builds run in batch mode by adding the `-B` command line flag, and have transfer progress suppressed with the `-ntp` flag.
Some builds were already running batch mode but had the flag in a different location – these have had their `-B` flag moved so all builds are consistent.

Result:
Much less output in our build logs where Maven is just downloading stuff.
2021-06-07 10:53:10 +02:00
Stuart Douglas
9f8bfa348e
Fix issue if encoding is enabled but not used (#11358)
Motivation:

Fixes an IllegalReferenceCountException

Modification:

Retained the buffers so the encoder works correctly.

Result:

Fixes #11357
2021-06-07 08:52:44 +02:00
Ikko Ashimine
10b3cee3fa
Fix typo in AbstractMultiSearchProcessorFactory (#11368)
Motivation:

There was a typo in the docs.

Modification:

occurence -> occurrence

Result:

Cleanup.
2021-06-07 08:45:55 +02:00
Aayush Atharva
113ed8fe4f
Fix typo in word accelerating (#11359)
Motivation:
There is a small typo in `pom.xml`. Typo is: `acclerating`, however it should be `accelerating`.

Modification:
Corrected the typo.

Result:
Typo-free `pom.xml`.
2021-06-07 08:08:37 +02:00
skyguard1
dbf56c527d
Fix IpSubnetFilterRule with IPv6 Default Route does not accept all IPv6 addresses (#11351)
Motivation:

In this issue(https://github.com/netty/netty/issues/11349 ),IpSubnetFilterRule needs to support ipv6 reserved addresses, such as 8000::, but the current implementation does not support

Modification:

Added support for default rule

Result:

Fixes https://github.com/netty/netty/issues/11349

Signed-off-by: xingrufei <xingrufei@sogou-inc.com>
2021-06-07 08:06:52 +02:00
skyguard1
6621e4a60f
Add default block in HttpObjectDecoder (#11342)
Motivation:

There should always be a default in switch blocks.

Modification:

Add default

Result:

Code cleanup.

Signed-off-by: xingrufei <xingrufei@sogou-inc.com>
2021-06-02 08:41:17 +02:00
skyguard1
8d815d55f9
Add default block in HttpClientCodec (#11352)
Motivation:

There should always be a default in switch blocks.

Modification:

Add default

Result:

Code cleanup.

Signed-off-by: xingrufei <xingrufei@sogou-inc.com>
2021-06-02 08:40:36 +02:00
skyguard1
d334146ccd
Add default block in IdleStateHandler (#11341)
Motivation:

We should have a default case in every switch block.

Modification:

Add default block in IdleStateHandler

Result:

Cleanup

Signed-off-by: xingrufei <xingrufei@sogou-inc.com>
2021-06-01 08:01:32 +02:00
skyguard1
a64a7eb39d
Remove useless code (#11339)
Signed-off-by: xingrufei <xingrufei@sogou-inc.com>

Co-authored-by: xingrufei <xingrufei@sogou-inc.com>
2021-05-31 15:35:42 +02:00
Frédéric Brégier
103f1d269e
HttpPostMultipartRequestDecoder IndexOutOfBoundsException error (#11335)
Motivation:

When searching for the delimiter, the decoder part within HttpPostBodyUtil
was not checking the left space to check if it could be included or not,
while it should.

Modifications:

Add a check on toRead being greater or equal than delimiterLength before
going within the loop. If the check is wrong, the delimiter is obviously not found.

Add a Junit test to preserve regression.

Result:

No more IndexOutOfBoundsException

Fixes #11334
2021-05-31 08:43:39 +02:00