Commit Graph

1324 Commits

Author SHA1 Message Date
Gerd Riesselmann
f995426a59 Avoid possible comparison contract violation (#9883)
Motivation:

The current implementation causes IllegalArgumetExceptions to be thrown on Java 11.

The current implementation would violate comparison contract for two cookies C1 and C2 with same path length, since C1 < C2 and C2 < C1. Returning 0 (equality) does not since C1 == C2 and C2 == C1. See #9881

Modification:

Return equality instead of less than on same path length.

Result:

Fixes #9881.
2019-12-19 12:16:27 +01:00
Anuraag Agrawal
95b8db0633 Use array to buffer decoded query instead of ByteBuffer. (#9886)
Motivation:

In Java, it is almost always at least slower to use `ByteBuffer` than `byte[]` without pooling or I/O. `QueryStringDecoder` can use `byte[]` with arguably simpler code.

Modification:

Replace `ByteBuffer` / `CharsetDecoder` with `byte[]` and `new String`

Result:

After
```
Benchmark                                   Mode  Cnt  Score   Error   Units
QueryStringDecoderBenchmark.noDecoding     thrpt    6  5.612 ± 2.639  ops/us
QueryStringDecoderBenchmark.onlyDecoding   thrpt    6  1.393 ± 0.067  ops/us
QueryStringDecoderBenchmark.mixedDecoding  thrpt    6  1.223 ± 0.048  ops/us
```

Before
```
Benchmark                                   Mode  Cnt  Score   Error   Units
QueryStringDecoderBenchmark.noDecoding     thrpt    6  6.123 ± 0.250  ops/us
QueryStringDecoderBenchmark.onlyDecoding   thrpt    6  0.922 ± 0.159  ops/us
QueryStringDecoderBenchmark.mixedDecoding  thrpt    6  1.032 ± 0.178  ops/us
```

I notice #6781 switched from an array to `ByteBuffer` but I can't find any motivation for that in the PR. Unit tests pass fine with an array and we get a reasonable speed bump.
2019-12-18 21:11:28 +01:00
root
79d4e74019 [maven-release-plugin] prepare for next development iteration 2019-12-18 08:32:54 +00:00
root
5ddf45a2d5 [maven-release-plugin] prepare release netty-4.1.44.Final 2019-12-18 08:31:43 +00:00
Norman Maurer
d24dfb1051 Revert "Bugfix #9673: Origin header is always sent from WebSocket client (#9692)"
This reverts commit f48d9fa8d0 as this needs more thoughts.
2019-12-18 09:23:38 +01:00
Norman Maurer
8494b046ec
Verify we do not receive multiple content-length headers or a content-length and transfer-encoding: chunked header when using HTTP/1.1 (#9865)
Motivation:

RFC7230 states that we should not accept multiple content-length headers and also should not accept a content-length header in combination with transfer-encoding: chunked

Modifications:

- Check for multiple content-length headers and if found mark message as invalid
- Check if we found a content-length header and also a transfer-encoding: chunked and if so mark the message as invalid
- Add unit test

Result:

Fixes https://github.com/netty/netty/issues/9861
2019-12-13 08:53:19 +01:00
Dmitriy Dumanskiy
0992718f87 #9867 fix confusing method parameter name (#9874)
Motivation:

Parameter name is confusing and not match the actual type.

Modification:

Rename parameter.

Result:

Code cleanup
2019-12-12 14:41:41 +01:00
Norman Maurer
a7c18d44b4
Detect missing colon when parsing http headers with no value (#9871)
Motivation:

Technical speaking its valid to have http headers with no values so we should support it. That said we need to detect if these are "generated" because of an "invalid" fold.

Modifications:

- Detect if a colon is missing when parsing headers.
- Add unit test

Result:

Fixes https://github.com/netty/netty/issues/9866
2019-12-11 15:49:07 +01:00
Andrey Mizurov
cf63bc1005 Fix #9770, last frame may contain extra data that doesn't affect decompression (#9832)
Motivation:
Client can split data into different numbers of fragments and sometimes the last frame may contain trash data that doesn't affect decompression process.

Modification:
Added check if last frame is `ContinuationWebSocketFrame` and decompression data is empty
then don't throw an exception.

Result:
Fixes #9770
2019-12-11 15:00:52 +01:00
Carl Mastrangelo
4138cba861 Add Server Cookie Parser overload for multiple cookies. (#9856)
Motivation:

Multiple cookie values can be present in a single header.

Modification:

Add `decodeAll` overload which returns all cookies

Result:

Fixes #7210

Note:

This change is not as perscriptive as the ideas brought up in the linked issue.  Changing the Set implementation or the equals/compareTo definition is likely a breaking change, so they are practical.
2019-12-10 11:32:45 +01:00
时无两丶
0cde4d9cb4 Uniform null pointer check. (#9840)
Motivation:
Uniform null pointer check.

Modifications:

Use ObjectUtil.checkNonNull(...)

Result:
Less code, same result.
2019-12-09 09:47:35 +01:00
Norman Maurer
dcbfe17eeb
Prevent any leaks when HttpPostStandardRequestDecoder constructor throws (#9837)
Motivation:

HttpPostStandardRequestDecoder may throw multiple different exceptions in the constructor which could lead to memory leaks. We need to guard against this by explicit catch all of them and rethrow after we released any allocated memory.

Modifications:

- Catch, destroy and rethrow in any case
- Ensure we correctly wrap IllegalArgumentExceptions
- Add unit tests

Result:

Fixes https://github.com/netty/netty/issues/9829
2019-12-04 13:58:03 +01:00
ursa
19a4633859 Bugfix #9673: Origin header is always sent from WebSocket client (#9692)
### Motivation:

Those who need 'Origin' or 'Sec-WebSocket-Origin' headers should provide them explicitly, like it is stated in WebSocket specs.

E.g. through custom headers:

    HttpHeaders customHeaders = new DefaultHttpHeaders()
        .add(HttpHeaderNames.ORIGIN, "http://localhost:8080");
    new WebSocketClientProtocolHandler(
        new URI("ws://localhost:1234/test"), WebSocketVersion.V13, subprotocol, 
        allowExtensions, customHeaders, maxFramePayloadLength, handshakeTimeoutMillis)


### Modification:

* Remove enforced origin headers.
* Update tests

### Result:

Fixes #9673: Origin header is always sent from WebSocket client
2019-11-27 08:36:30 +01:00
Norman Maurer
2886bd6677
Simplify Deflate* implementations by using EmbeddedChannel.finishAndReleaseAll() (#9808)
Motivation:

We can simplify the code by just using finishAndReleaseAll()

Modifications:

Remove some code and simplify

Result:

Cleaner code
2019-11-27 06:54:55 +01:00
ursa
b5230c7b9c Send close frame on channel close, when this frame was not send manually (#9745)
Motivation:
By default CloseWebSocketFrames are handled automatically.
However I need manually manage their sending both on client- and on server-sides.

Modification:
Send close frame on channel close automatically, when it was not send before explicitly.

Result:
No more messages like "Connection closed by remote peer" for normal close flows.
2019-11-18 20:32:21 +01:00
ursa
8d99aa1235 Simplify WebSocket handlers constructor arguments hell #9698 (#9699)
### Motivation:

Introduction of `WebSocketDecoderConfig` made our server-side code more elegant and simpler for support.

However there is still some problem with maintenance and new features development for WebSocket codecs (`WebSocketServerProtocolHandler`, `WebSocketServerProtocolHandler`).

Particularly, it makes me ~~crying with blood~~ extremely sad to add new parameter and yet another one constructor into these handlers, when I want to contribute new feature.

### Modification:

I've extracted all parameters for client and server WebSocket handlers into config/builder structures, like it was made for decoders in PR #9116.

### Result:

* Fixes #9698: Simplify WebSocket handlers constructor arguments hell
* Unblock further development in this module (configurable close frame handling on server-side; automatic close-frame sending, when missed; memory leaks on protocol violations; etc...)

Bonuses:

* All defaults are gathered in one place and could be easily found/reused.
* New API greatly simplifies usage, but does NOT allow inheritance or modification.
* New API would simplify long-term maintenance of WebSockets module.

### Example

    WebSocketClientProtocolConfig config = WebSocketClientProtocolConfig.newBuilder()
        .webSocketUri("wss://localhost:8443/fx-spot")
        .subprotocol("trading")
        .handshakeTimeoutMillis(15000L)
        .build();
    ctx.pipeline().addLast(new WebSocketClientProtocolHandler(config));
2019-10-29 20:48:18 +01:00
Norman Maurer
a6681e74fb
HttpClientCodec need to keep request / response pairs in sync all the… (#9721)
Motivation:

At the moment we miss to poll the method queue when we see an Informational response code. This can lead to out-of-sync of request / response pairs when later try to compare these.

Modifications:

Always poll the queue correctly

Result:

Always compare the correct request / response pairs
2019-10-29 19:48:43 +01:00
Anuraag Agrawal
816350fba1 Respect all informational status codes. (#9712)
Motivation:

HTTP 102 (WebDAV) is not correctly treated as an informational response

Modification:

Delegate all `1XX` status codes to superclass, not just `100` and `101`.

Result:

Supports WebDAV response.
Removes a huge maintenance [headache](https://github.com/line/armeria/pull/2210) in Armeria which has forked the class for these features
2019-10-28 14:21:36 +01:00
Esteban Ginez
aa3c57508b Adds DeflateDecoder to native-image.properties of codec-http (#9708)
Motivation:
DeflateDecoder was found to be needed when building spring examples app
with graalvm's native image: https://github.com/spring-projects-experimental/spring-graal-native

Modification:
Adds extra native-image.properties to code-http package

Result:
Both:
https://github.com/spring-projects-experimental/spring-graal-native/tree/master/spring-graal-native-samples/spring-petclinic-jpa
and 
https://github.com/spring-projects-experimental/spring-graal-native/tree/master/spring-graal-native-samples/webflux-netty
Build and run
2019-10-25 11:15:34 -07:00
root
844b82b986 [maven-release-plugin] prepare for next development iteration 2019-10-24 12:57:00 +00:00
root
d066f163d7 [maven-release-plugin] prepare release netty-4.1.43.Final 2019-10-24 12:56:30 +00:00
Norman Maurer
7d6d953153
Support semicolons in query parameters as explain in the W3C recommentation (#9701)
Motivation:

Support semicolons in query parameters as explain in the W3C recommentation:
https://www.w3.org/TR/2014/REC-html5-20141028/forms.html#url-encoded-form-data

Modification:

- Add a new constructor arg that can be used to "switch" modes for decoding ;
- Add unit test

Result:

Fixes #8855
2019-10-23 23:44:37 -07:00
Andrey Mizurov
8674ccfcd2 Fix indexOutOfBoundsException when multipart/form-data is incorrect value (#9688)
Motivation:

HttpPostRequestDecoder.splitHeaderContentType() throws a StringIndexOutOfBoundsException when it parses a Content-Type header that starts with a semicolon ;. We should skip the execution for incorrect multipart form data.


Modification:

Avoid invocation of HttpPostRequestDecoder#splitHeaderContentType(...) for incorrect multipart form data content-type.

Result:

Fixes #8554
2019-10-23 00:03:20 -07:00
Norman Maurer
6c05d16967
Use @SuppressJava6Requirement for animal sniffer plugin to ensure we always guard correctly (#9655)
Motivation:

We can use the `@SuppressJava6Requirement` annotation to be more precise about when we use Java6+ APIs. This helps us to ensure we always protect these places.

Modifications:

Make use of `@SuppressJava6Requirement` explicit

Result:

Fixes https://github.com/netty/netty/issues/2509.
2019-10-14 15:54:49 +02:00
Jonathan Leitschuh
e0b15ed952 [DOC] Add CWE-113 warning to DefaultHttpHeaders constructor (#9646)
### Motivation:

I've now found two libraries that use Netty to be vulnerable to [CWE-113: Improper Neutralization of CRLF Sequences in HTTP Headers ('HTTP Response Splitting')](https://cwe.mitre.org/data/definitions/113.html) due to using `new DefaultHttpHeaders(false)`.

Some part of me hopes that this warning will help dissuade library authors from disabling this important security check.

### Modification:

Add documentation to `DefaultHttpHeaders(boolean)` to warn about the implications of `false`.

### Result:

This improves the documentation on `DefaultHttpHeaders`.
2019-10-10 22:47:28 +04:00
康智冬
bd8cea644a Fix typos in javadocs (#9527)
Motivation:

We should have correct docs without typos

Modification:

Fix typos and spelling

Result:

More correct docs
2019-10-09 17:12:52 +04:00
root
92941cdcac [maven-release-plugin] prepare for next development iteration 2019-09-25 06:15:31 +00:00
root
bd907c3b3a [maven-release-plugin] prepare release netty-4.1.42.Final 2019-09-25 06:14:31 +00:00
liyixin
86ff76a4f7 Fix incorrect comment (#9598)
Motivation:

The comment is incorrect and so missleading

Modification:

Correct the comment

Result:

Correct comment in code
2019-09-24 10:00:55 +02:00
liyixin
07fe1a299a Optimize the QueryStringEncoder performance (#9568)
Motivation:

Optimize the QueryStringEncoder for lower memory overhead and higher encode speed.

Modification:

Encode the space to + directly, and reuse the uriStringBuilder rather then create a new one.

Result:

Improved performance
2019-09-20 21:07:13 +02:00
Norman Maurer
39cafcb05c
Correctly handle whitespaces in HTTP header names as defined by RFC7230#section-3.2.4 (#9585)
Motivation:

When parsing HTTP headers special care needs to be taken when a whitespace is detected in the header name.

Modifications:

- Ignore whitespace when decoding response (just like before)
- Throw exception when whitespace is detected during parsing
- Add unit tests

Result:

Fixes https://github.com/netty/netty/issues/9571
2019-09-20 21:02:11 +02:00
root
01d805bb76 [maven-release-plugin] prepare for next development iteration 2019-09-12 16:09:55 +00:00
root
7cf69022d4 [maven-release-plugin] prepare release netty-4.1.41.Final 2019-09-12 16:09:00 +00:00
root
aef47bec7f [maven-release-plugin] prepare for next development iteration 2019-09-12 05:38:11 +00:00
root
267e5da481 [maven-release-plugin] prepare release netty-4.1.40.Final 2019-09-12 05:37:30 +00:00
Andrey Mizurov
bcb0d02248 Fix HttpContentEncoder does not handle multiple Accept-Encoding (#9557)
Motivation:
At the current moment HttpContentEncoder handle only first value of multiple accept-encoding headers.

Modification:

Join multiple accept-encoding headers to one separated by comma.

Result:

Fixes #9553
2019-09-11 08:46:06 +02:00
Norman Maurer
95527eec91
HttpPostStandardRequestDecoder leaks memory when constructor throws ErrorDataDecoderException. (#9517)
Motivation:

Currently when HttpPostStandardRequestDecoder throws a ErrorDataDecoderException during construction we leak memory. We need to ensure all is released correctly.

Modifications:

- Call destroy() if parseBody() throws and rethrow the ErrorDataDecoderException
- Add unit test

Result:

Fixes https://github.com/netty/netty/issues/9513.
2019-08-28 10:27:38 +02:00
Andrey Mizurov
491b1f428b Fix sending an empty String like "" causes an error #9429 (#9512)
Motivation:

Handle https://tools.ietf.org/html/rfc7692#section-7.2.3.6

Result:

The empty buffer is correctly handled in deflate  encoder/decoder.

 Fixes #9429 .
2019-08-28 08:21:38 +02:00
hengyunabc
4c14fa5c54 Correctly pass all parameters in WebSocketServerProtocolHandler constructor (#9506)
Motivation:

We did not correctly pass all supplied parameters to the called constructor and so did not apply the timeout.

Modification:

Correctly pass on the parameters.

Result:

Use timeout
2019-08-27 09:00:58 +02:00
Idel Pivnitskiy
9fa974f6a5 Update links to the latest HTTP/2 specifications (#9493)
Motivation:

Some of the links in javadoc point to the obsolete drafts of HTTP/2
specifications. We should point them to the latest RFC 7540 or 7541.

Modifications:

Update links from `draft-ietf-httpbis-*` to the `rfc7540` and `rfc7541`.

Result:

Correct links in javadoc.
2019-08-22 13:59:08 +02:00
Idel Pivnitskiy
85fcf4e581 Use AppendableCharSequence.charAtUnsafe(int) in HttpObjectDecoder (#9492)
Motivation:

`HttpObjectDecoder` pre-checks that it doesn't request characters
outside of the `AppendableCharSequence`'s length. `0` is always allowed
because the minimal length of `AppendableCharSequence` is `1`. We can
legally skip index check by using
`AppendableCharSequence.charAtUnsafe(int)` in all existing cases in
`HttpObjectDecoder`.

Modifications:

- Use `AppendableCharSequence.charAtUnsafe(int)` instead of
`AppendableCharSequence.charAt(int)` in `HttpObjectDecoder`.

Result:

No unnecessary index checks in `HttpObjectDecoder`.
2019-08-22 13:58:22 +02:00
Sergey S. Sergeev
9e2922b04d Fix unexpected IllegalReferenceCountException on decode multipart request. (#8575)
Motivation:

Http post request may be encoded as 'multipart/form-data' without any files and consist mixed attributes only.

Modifications:

- Do not double release attributes
- Add unit test

Result:

Code does not throw an IllegalReferenceCountException.
2019-08-19 15:08:40 +02:00
Norman Maurer
839aab8558
Ensure we replace WebSocketServerProtocolHandshakeHandler before doing the handshake (#9472)
Motivation:

We need to ensure we replace WebSocketServerProtocolHandshakeHandler before doing the actual handshake as the handshake itself may complete directly and so forward pending bytes through the pipeline.

Modifications:

Replace the handler before doing the actual handshake.

Result:

Fixes https://github.com/netty/netty/issues/9471.
2019-08-17 10:00:01 +02:00
Norman Maurer
d8e59ca638
Avoid creating FileInputStream and FileOutputStream for obtaining Fil… (#8110)
Motivation:

If all we need is the FileChannel we should better use RandomAccessFile as FileInputStream and FileOutputStream use a finalizer.

Modifications:

Replace FileInputStream and FileOutputStream with RandomAccessFile when possible.

Result:

Fixes https://github.com/netty/netty/issues/8078.
2019-08-17 09:43:01 +02:00
Norman Maurer
9ec3411c91
Fix possible NPE when using HttpClientCodec (#9465)
Motivation:

It was possible to produce a NPE when we for examples received more responses as requests as we did not check if the queue did not contain a method before trying to compare method names.

Modifications:

- Add extra null check
- Add unit tet

Result:

Fixes https://github.com/netty/netty/issues/9459
2019-08-16 08:17:18 +02:00
Norman Maurer
299954e138
Correctly respect mask parameters in all WebSocketClientHandshakerFactory#newHandshaker(...) methods (#9464)
Motivation:

We did not correctly pass the mask parameters in all cases.

Modifications:

Correctly pass on parameters

Result:

Fixes https://github.com/netty/netty/issues/9463.
2019-08-15 08:33:28 +02:00
root
d45a4ce01b [maven-release-plugin] prepare for next development iteration 2019-08-13 17:16:42 +00:00
root
88c2a4cab5 [maven-release-plugin] prepare release netty-4.1.39.Final 2019-08-13 17:15:20 +00:00
Andrey Mizurov
b8ac02d8ac Set the ORIGIN header from a custom headers if present (#9435)
Motivation:

Allow to set the ORIGIN header value from custom headers in WebSocketClientHandshaker

Modification:

Only override header if not present already

Result:

More flexible handshaker usage
2019-08-11 08:22:17 +02:00
noSim
cbc8fc4428 Fix HttpUtil.getCharset to not throw illegal charset exception (#9439)
Motivation:

If the HttpUtil.getCharset method is called with an illegal charset like
"charset=!illegal!" it throws an IllegalCharsetNameException. But the javadoc
states, that defaultCharset is returned if incorrect header value. Since the
client sending the request sets the header value this should not crash.

Modification:

HttpUtil.getCharset catches the IllegalCharsetNameException and returns the
defualt value.

Result:

HttpUtil.getCharset does not throw IllegalCharsetNameException any more.
2019-08-10 19:23:40 +02:00