Commit Graph

1502 Commits

Author SHA1 Message Date
Jeremy Kuhn
93484071d6
Fix support for optional encoders errors in HttpContentCompressor (#11582)
Motivation:

- Fix `HttpContentCompressor` errors due to missing optional compressor libraries such as Brotli and Zstd at runtime.
- Improve support for optional encoders by only considering the `CompressionOptions` provided to the constructor and ignoring those for which the encoder is unavailable.

Modification:

The `HttpContentCompressor` constructor now only creates encoder factories for the CompressionOptions passed to the constructor when the encoder is available which must be checked for Brotli and Zstd. In case of Brotli, it is not possible to create BrotliOptions if brotly4j is not available so there's actually nothing to check. In case of Zstd, I had to create class `io.netty.handler.codec.compression.Zstd` similar to `io.netty.handler.codec.compression.Brotli` which is used to check that zstd-jni is availabie at runtime.

The `determineEncoding()` method had to change as well in order to ignore encodings for which there's no `CompressionEncoderFactory` instance.

When the HttpContentCompressor is created using deprecated constructor (ie. with no CompressionOptions), we consider all available encoders.

Result:

Fixes #11581.
2021-08-19 08:40:25 +02:00
Netty Project Bot
f8f17f676d [maven-release-plugin] prepare for next development iteration 2021-08-16 08:23:23 +00:00
Netty Project Bot
b785fa9eed [maven-release-plugin] prepare release netty-4.1.67.Final 2021-08-16 08:23:20 +00:00
Norman Maurer
7903a44a14 Revert "[maven-release-plugin] prepare release netty-4.1.67.Final"
This reverts commit f773733af6.
2021-08-16 10:14:09 +02:00
Norman Maurer
ec23189e72 Revert "[maven-release-plugin] prepare for next development iteration"
This reverts commit 76c55caeb9.
2021-08-16 10:13:59 +02:00
Netty Project Bot
76c55caeb9 [maven-release-plugin] prepare for next development iteration 2021-08-12 17:12:05 +00:00
Netty Project Bot
f773733af6 [maven-release-plugin] prepare release netty-4.1.67.Final 2021-08-12 17:12:01 +00:00
DD
056eba4db4
server h2c upgrade fail when request doesn't have connection header (#11569)
__Motivation__
Since request.headers().getAll() will never return null. And the check null condition will not work as expected.

__Modification__

Add isEmpty() checking as well.

__Result__

Fixes #11568
2021-08-12 09:04:18 -07:00
Aayush Atharva
fe7b18ee83
Make variables final (#11548)
Motivation:
We should make variables `final` which are not reinstated again in code to match the code style and makes the code look better.

Modification:
Made couples of variables as `final`.

Result:
Variables marked as `final`.
2021-08-06 09:27:12 +02:00
Aayush Atharva
6e387e2f5d
Remove unnecessary toString calls (#11550)
Motivation:
We should get rid of the unnecessary toString calls because they're redundant in nature.

Modification:
Removed unnecessary toString calls.

Result:
Better code
2021-08-05 14:17:00 +02:00
Aayush Atharva
01b42c568d
Remove unnecessary semicolons (#11549)
Motivation:
We should get rid of unnecessary semicolons because they don't do anything in code.

Modification:
Removed unnecessary semicolons.

Result:
Better code
2021-08-05 14:09:54 +02:00
Aayush Atharva
1ce28e80d1
Remove Unused Imports (#11546)
Motivation:
There are lots of imports which are unused. We should get rid of them to make the code look better,

Modification:
Removed unused imports.

Result:
No unused imports.
2021-08-05 13:54:48 +02:00
Norman Maurer
26cdfe2cde
Add PromiseNotifier static method which takes care of cancel propagation (#11494)
Motivation:

At the moment we not correctly propagate cancellation in some case when we use the PromiseNotifier.

Modifications:

- Add PromiseNotifier static method which takes care of cancellation
- Add unit test
- Deprecate ChannelPromiseNotifier

Result:

Correctly propagate cancellation of operation

Co-authored-by: Nitesh Kant <nitesh_kant@apple.com>
2021-07-21 13:37:32 +02:00
Mads Johannessen
3a41a97b0e
Support large or variable chunk sizes (#11469)
Motivation:

Chunks are splitted up into even smaller chunks when the underlying
buffer's readable bytes are less than the chunk size.

The underlying buffer can be smaller than a chunk size if:

- The chunk size is larger than the maximum plaintext chunk allowed by the TLS RFC,
  see: io.netty.handler.ssl.SslHandler.MAX_PLAINTEXT_LENGTH.

- The chunk sizes are variable in size,
  which may cause Netty guess a buffer size that is smaller than a chunk size.

Modification:

Create a variable in HttpObjectDecoder: ByteBuf chunkedContent

- Initialize chunkedContent in READ_CHUNK_SIZE with chunkSize as buffer size.

- In READ_CHUNKED_CONTENT write bytes into chunkedContent

  - If the remaining chunk size is not 0 and toRead ==maxChunkSize,
    create a chunk using the chunkedContent and add it to the output messages
    before re-initializing chunkedContent with the remaining chunkSize as buffer size.

  - If the remaining chunk size is not 0 and toRead != maxChunkSize,
    return without adding any output messages.

  - If the remaining chunk size is 0,
    create a chunk using the chunkedContent and add it to the output messages;
    set chunkedContent = null and fall-through.

Result:

Support chunk sizes higher than the underlying buffer's readable bytes.

Co-authored-by: Nitesh Kant <nitesh_kant@apple.com>
Co-authored-by: Norman Maurer <norman_maurer@apple.com>
2021-07-20 15:21:26 +02:00
Netty Project Bot
2a184ab932 [maven-release-plugin] prepare for next development iteration 2021-07-16 06:23:23 +00:00
Netty Project Bot
e43d0d99f1 [maven-release-plugin] prepare release netty-4.1.66.Final 2021-07-16 06:23:17 +00:00
Aayush Atharva
3b8e7c3e60
Use Static Constants as Sample Data (#11485)
Motivation:
#11468 was merged but didn't fix tests completely. There is a fight between `LF` and `CRLF`. So to eliminate this, we should just get rid of them.

Modification:
Use a small sample dataset without `LF` and `CRLF`.

Result:
Simple and passing test.
2021-07-14 07:45:21 +02:00
Aayush Atharva
7ec198ddf9
Fix JavaDoc and Run Brotli tests on Windows (#11468)
Motivation:
JavaDoc of StandardCompressionOptions should point towards public methods. Also, Brotli tests were failing on Windows.

Modification:
Fixed JavaDoc and enabled Brotli tests on Windows.

Result:
Better JavaDoc and Brotli tests will run on Windows

Co-authored-by: Norman Maurer <norman_maurer@apple.com>
2021-07-13 16:24:58 +02:00
skyguard1
df22356a3a
Refactor HttpContentCompressor using CompressionEncoderFactory (#11480)
Motivation:

The `HttpContentCompressor.beginEncode()` method has too many if else, so consider refactoring

Modification:

Create the corresponding `CompressionEncoderFactory` according to the compression algorithm, remove the if else

Result:

The code of `HttpContentCompressor` is cleaner than the previous implementation

Signed-off-by: xingrufei <xingrufei@sogou-inc.com>
Co-authored-by: Norman Maurer <norman_maurer@apple.com>
2021-07-13 12:25:30 +02:00
Aayush Atharva
6085cd7ab6
Update HttpContentCompressor to pass correct message to ObjectUtil (#11482)
Motivation:
In #11256, We introduced `Iterable` as a parameter but later in review, it was removed. But we forgot to change `compressionOptionsIterable` to just `compressionOptions`.

Modification:
Changed `compressionOptionsIterable` to `compressionOptions`.

Result:
Correct ObjectUtil message
2021-07-13 08:11:01 +02:00
skyguard1
6ce36f1909
Add zstd http content compression support (#11470)
Motivation:

netty needs to support zstd content-encoding http content compression

Modification:

Add ZstdOptions, and modify HttpContentCompressor and CompressorHttp2ConnectionEncoder to support zstd compression

Result:

netty supports zstd http content compression

Signed-off-by: xingrufei <xingrufei@sogou-inc.com>
2021-07-12 08:49:04 +02:00
Aayush Atharva
fef761d03e
Introduce BrotliEncoder (#11256)
Motivation:
Currently, Netty only has BrotliDecoder which can decode Brotli encoded data. However, BrotliEncoder is missing which will encode normal data to Brotli encoded data.

Modification:
Added BrotliEncoder and CompressionOption

Result:
Fixes #6899.

Co-authored-by: Norman Maurer <norman_maurer@apple.com>
2021-07-08 11:51:27 +02:00
skyguard1
7825fa8d7a
Add zstd http header value (#11463)
Motivation:

ZSTD has a wide range of uses on the Internet, so should consider adding `application/zstd` HTTP media-type and `zstd` content-encoding, see  https://tools.ietf.org/html/rfc8478

Modification:

Add `application/zstd` HTTP media-type and `zstd` content-encoding

Result:

netty provides `application/zstd` HTTP media-type and `zstd content-encoding` as http headers

Signed-off-by: xingrufei <xingrufei@sogou-inc.com>
Co-authored-by: xingrufei <xingrufei@sogou-inc.com>
2021-07-08 11:44:54 +02:00
Kasimir Torri
c97981403d
Improve PerMessageDeflateClientExtensionHandler (#11413)
Motivation:

The `PerMessageDeflateClientExtensionHandler` has the following strange behaviors currently:

* The `requestedServerNoContext` parameter doesn't actually add the `server_no_context_takeover` parameter to the client offer; instead it depends on the requested server window size.
* The handshake will fail if the server responds with a `server_no_context_takeover` parameter and `requestedServerNoContext` is false. According to RFC 7692 (7.1.1.1) the server may do this, and this means that to cover both cases one needs to use two handshakers in the channel pipeline: one with `requestedServerNoContext = true` and one with `requestedServerNoContext = false`.
* The value of the `server_max_window_bits` parameter in the server response is never checked (should be between 8 and 15). And the value of `client_max_window_bits` is checked only in the branch handling the server window parameter.

Modification:

* Add the `server_no_context_takeover` parameter if `requestedServerNoContext` is true.
* Accept a server handshake response which includes the server no context takeover parameter even if we did not request it.
* Check the values of the client and server window size in their respective branches and fail the handshake if they are out of bounds.

Result:

There will be no need to use two handshakers in the pipeline to be lenient in what handshakes are accepted.
2021-07-02 14:47:59 +02:00
Norman Maurer
5179e53294
Migrate codec-http to junit5 (#11440)
Motivation:

We should update to use junit5 in all modules.

Modifications:

Adjust codec-http tests to use junit5

Result:

Part of https://github.com/netty/netty/issues/10757
2021-07-01 18:53:41 +02:00
Aleksandar Gradinac
bc206ff173
Delay initialization of io.netty.handler.codec.compression.BrotliDecoder to runtime (#11428)
Motivation:
Including codec-http in the project and building a native-image out of it using a GraalVM 21.2 nightly can result in a failure.

Modification:
By delaying the initialization of `io.netty.handler.codec.compression.BrotliDecoder` to runtime, native-image will not try to eagerly initialize the class during the image build, avoiding the build failure described in the issue.

Result:
Fixes #11427
2021-07-01 15:08:51 +02:00
Stephane Landelle
801819b359
Fix HttpHeaderValue#XML_HTTP_REQUEST case (#11433)
Motivation:

HTTP header values are case sensitive. The expected value for `x-request-with` header is `XMLHttpRequest`, not `XmlHttpRequest`.

Modification:

Fix constant's case.

Result:

Correct `XMLHttpRequest` HTTP header value.
2021-07-01 08:13:10 +02:00
Nitesh Kant
5a658bb887
HttpUtil#normalizeAndGetContentLength() should handle empty value (#11409)
__Motivation__

`HttpUtil#normalizeAndGetContentLength()` throws `StringIndexOutOfBoundsException` for empty `content-length` values, it should instead throw `IllegalArgumentException` for all invalid values.

__Modification__

- Throw `IllegalArgumentException` if the `content-length` value is empty.
- Add tests

__Result__

Fixes https://github.com/netty/netty/issues/11408
2021-06-23 12:07:16 +02:00
ktqco
f1742c0e43
Accept smaller server_max_window_bits than requested (#11394)
Motivation:

Netty will fail a handshake for the Per-Message Deflate WebSocket
extension if the server response contains a smaller
`server_max_window_bits` value than the client offered.

However, this is allowed by RFC 7692:

> A server accepts an extension negotiation offer with this parameter
> by including the “server_max_window_bits” extension parameter in the
> extension negotiation response to send back to the client with the
> same or smaller value as the offer.

Modifications:

- Allow the server to respond with a smaller value than offered.
- Change the unit tests to test for this.

Result:

The client will not fail when the server indicates it is using a
smaller window size than offered by the client.
2021-06-18 11:45:32 +02:00
Nitesh Kant
625a7a1075
HttpUtil.getCharset() fails for charset in double-quotes (#11373)
__Motivation__

As described in https://github.com/netty/netty/issues/11370 we should support quoted charset values

__Modification__

Modify `HttpUtil.getCharset(CharSequence contentTypeValue, Charset defaultCharset)` to trim the double-quotes if present.

__Result__

`HttpUtil.getCharset()` now supports quoted charsets. Fixes https://github.com/netty/netty/issues/11370
2021-06-09 12:37:23 +02:00
Violeta Georgieva
e69107ceaf
Fix ServerCookieDecoder javadoc (#11372)
Motivation:
When decoding the cookies on the server, the "Cookie" HTTP request header value should be considered.
The "Set-Cookie" HTTP response header is used to send cookies from the server to the user agent.

Modification:
- Specify in javadoc that the "Cookie" HTTP request header value should be considered and
not the "Set-Cookie" HTTP response header value.

Result:
Correct ServerCookieDecoder javadoc
2021-06-07 16:33:31 -07:00
Stuart Douglas
9f8bfa348e
Fix issue if encoding is enabled but not used (#11358)
Motivation:

Fixes an IllegalReferenceCountException

Modification:

Retained the buffers so the encoder works correctly.

Result:

Fixes #11357
2021-06-07 08:52:44 +02:00
skyguard1
6621e4a60f
Add default block in HttpObjectDecoder (#11342)
Motivation:

There should always be a default in switch blocks.

Modification:

Add default

Result:

Code cleanup.

Signed-off-by: xingrufei <xingrufei@sogou-inc.com>
2021-06-02 08:41:17 +02:00
skyguard1
8d815d55f9
Add default block in HttpClientCodec (#11352)
Motivation:

There should always be a default in switch blocks.

Modification:

Add default

Result:

Code cleanup.

Signed-off-by: xingrufei <xingrufei@sogou-inc.com>
2021-06-02 08:40:36 +02:00
skyguard1
a64a7eb39d
Remove useless code (#11339)
Signed-off-by: xingrufei <xingrufei@sogou-inc.com>

Co-authored-by: xingrufei <xingrufei@sogou-inc.com>
2021-05-31 15:35:42 +02:00
Frédéric Brégier
103f1d269e
HttpPostMultipartRequestDecoder IndexOutOfBoundsException error (#11335)
Motivation:

When searching for the delimiter, the decoder part within HttpPostBodyUtil
was not checking the left space to check if it could be included or not,
while it should.

Modifications:

Add a check on toRead being greater or equal than delimiterLength before
going within the loop. If the check is wrong, the delimiter is obviously not found.

Add a Junit test to preserve regression.

Result:

No more IndexOutOfBoundsException

Fixes #11334
2021-05-31 08:43:39 +02:00
skyguard1
cfe631ad05
Add default block in DefaultHttpHeaders (#11329)
Motivation:

Every switch block should also have a default case.

Modification:

Add default block in DefaultHttpHeaders to ensure we not fall-through by mistake

Result:

Cleanup

Signed-off-by: xingrufei <xingrufei@sogou-inc.com>
2021-05-28 08:47:53 +02:00
Riley Park
ac3f823cce Migrate codec-http tests to JUnit 5 (#11316)
Motivation:

JUnit 5 is more expressive, extensible, and composable in many ways, and it's better able to run tests in parallel.

Modifications:

Use JUnit5 in tests

Result:

Related to https://github.com/netty/netty/issues/10757
2021-05-27 09:19:33 +02:00
Norman Maurer
0ff93519a4
Add builds for windows (#11284)
Motivation:

Let's also build on windows during PR validation

Modifications:

Add build on windows during PR

Result:

Validate that all also pass on windows
2021-05-26 12:12:11 +02:00
Netty Project Bot
02142a2510 [maven-release-plugin] prepare for next development iteration 2021-05-19 12:09:18 +00:00
Netty Project Bot
9e06241ddb [maven-release-plugin] prepare release netty-4.1.65.Final 2021-05-19 12:09:14 +00:00
Trustin Lee
3972bacd60
Provide a way to pass through a certain HTTP upgrade request (#11267)
Motivation:

A user might want to handle a certain HTTP upgrade request differently
than what `HttpServerUpgradeHandler` does by default. For example, a
user could let `HttpServerUpgradeHandler` handle HTTP/2 upgrades but
not WebSocket upgrades.

Modifications:

- Added `HttpServerUpgradeHandler.isUpgrade(HttpRequest)` so a user can
  tell `HttpServerUpgradeHandler` to pass the request as it is to the
  next handler.

Result:

- A user can handle a certain upgrade request specially.
2021-05-18 11:42:38 +02:00
Netty Project Bot
d5c81d9a3e [maven-release-plugin] prepare for next development iteration 2021-05-17 07:04:14 +00:00
Netty Project Bot
67bedf0efe [maven-release-plugin] prepare release netty-4.1.64.Final 2021-05-17 07:04:08 +00:00
Stephane Landelle
400858cd5e
Introduce BrotliDecoder (#10960)
Motivation:

Netty lacks client side support for decompressing Brotli compressed response bodies.

Modification:

* Introduce optional dependency to brotli4j by @hyperxpro. It will be up to the user to provide the brotli4j libraries for the target platform in the classpath. brotli4j is currently available for Linux, OSX and Windows, all for x86 only.
* Introduce BrotliDecoder in codec module
* Plug it onto `HttpContentDecompressor` for HTTP/1 and `DelegatingDecompressorFrameListener` for HTTP/2
* Add test in `HttpContentDecoderTest`
* Add `BrotliDecoderTest` that doesn't extend `AbstractDecoderTest` that looks flaky

Result:

Netty now support decompressing Brotli compressed response bodies.
2021-05-10 15:25:24 +02:00
Norman Maurer
0bd9d43b18
Move HttpPostMultiPartRequestDecoder specific tests to HttpPostMultiPartRequestDecoderTest (#11211)
Motivation:

Some of the HttpPostMultiPartRequestDecoder specific tests were included in HttpPostRequestDecoderTest. We should better move these in the correct test class.

Modifications:

Move specific tests

Result:

Cleanup
2021-04-29 16:00:39 +02:00
Frédéric Brégier
e7330490e3
Fix Memory release not correctly in Multipart Decoder (#11188)
Motivation:
2 years ago a change remove the default clearing of all HttpData, whatever
they are disk based or memory based.

A lot of users were probably releasing HttpData directly, so there was no issue.
But now, it seems, and as the Javadoc said, that `decoder.destroy()` shall clean up
also Memory based HttpData, and not only Disk based HttpData as currently.

Change:
- Add in `destroy()` method the necessary code to release if necessary
the underlying Memory based HttpDatas.

- Change one Junit Test (using Mixed, Memory and Disk based factories)
in order to check the correctness of this behavior and to really act
as a handler (releasing buffers or requests).

- Modify one Junit core to check validity when a delimiter is present in the Chunk
but not CRLF/LF (false delimiter), to ensure correctness.

Result:
No more issue on memory leak

Note that still the List and the Map are not cleaned, since they were not
before. No change is done on this, since it could produce backward issue compatibility.

Fix issues #11175 and #11184
2021-04-29 12:13:45 +02:00
Norman Maurer
def8a3f17d
Destroy HttpPostMultipartRequestDecoder if contructor throws (#11207)
Motivation:

We need to call destroy() if the constructor of HttpPostMultipartRequestDecoder throws as otherwise we may leak memory.

Modifications:

- Call destroy() if we throw
- Add unit test

Result:

No more leaks when constructor throws

Co-authored-by: Frederic Bregier <frederic.bregier@waarp.fr>
2021-04-28 12:27:12 +02:00
Norman Maurer
3657805711
Correctly throw ErrorDataDecoderException for errors while decoding (#11198)
Motivation:

We didn't correctly handle the case when no content-type header was found or if the charset was illegal and just did throw a NPE or ICE. We should in both cases throw an ErrorDataDecoderException to reflect what is documented in the javadocs.

Modifications:

- Throw correct exception
- Merge private method into the constructor as it is only used there
- Add unit tests

Result:

Throw expected exceptions on decoding errors
2021-04-27 16:37:37 +02:00
roy
c1b922b224
adjust validation logic when websocket server check starts with '/' (#11191)
Motivation:

When create a WebSocketServerProtocolConfig to check URI path starts from '/',
only '/' or '//subPath' can be passed by the checker,but '/subPath' should be
passed as well

Modifications:

in `WebSocketServerProtocolHandshakeHandler.isWebSocketPath()` treat '/' a special case

Result:
'/subPath' can be passed
2021-04-26 10:32:10 +02:00