Commit Graph

725 Commits

Author SHA1 Message Date
Aayush Atharva
76d5cdb9e8
Add HttpScheme Support in HttpToHttp2ConnectionHandler (#10641)
Motivation:
We should have a method to add `HttpScheme` if `HttpRequest` does not contain `x-http2-scheme` then we should use add it if `HttpToHttp2ConnectionHandler` is build using specified `HttpScheme`.

Modification:
Added `HttpScheme` in `HttpToHttp2ConnectionHandlerBuilder`.

Result:
Automatically add `HttpScheme` if missing in `HttpRequest`.
2020-11-05 15:53:32 +01:00
Eric Anderson
027a686042
codec-http2: Correct last-stream-id for HEADERS-triggered connection error (#10775)
Motivation:

When parsing HEADERS, connection errors can occur (e.g., too large of
headers, such that we don't want to HPACK decode them). These trigger a
GOAWAY with a last-stream-id telling the client which streams haven't
been processed.

Unfortunately that last-stream-id didn't include the stream for the
HEADERS that triggered the error. Since clients are free to silently
retry streams not included in last-stream-id, the client is free to
retransmit the request on a new connection, which will fail the
connection with the wrong last-stream-id, and the client is still free
to retransmit the request.

Modifications:

Have fatal connection errors (those that hard-cut the connection)
include all streams in last-stream-id, which guarantees the HEADERS'
stream is included and thus should not be silently retried by the HTTP/2
client.

This modification is heavy-handed, as it will cause racing streams to
also fail, but alternatives that provide precise last-stream-id tracking
are much more invasive. Hard-cutting the connection is already
heavy-handed and so is rare.

Result:

Fixes #10670
2020-11-05 09:07:28 +01:00
Arthur Gonigberg
b63e2dfb1b
Drop unknown frames on connection stream (#10771)
Motivation:

We received a [bug report](https://bugs.chromium.org/p/chromium/issues/detail?id=1143320) from the Chrome team at Google, their canary builds are failing [HTTP/2 GREASE](https://tools.ietf.org/html/draft-bishop-httpbis-grease-00) testing to netflix.com.

The reason it's failing is that Netty can't handle unknown frames without an active stream created. Let me know if you'd like more info, such as stack traces or repro steps. 

Modification:

The change is minor and simply ignores unknown frames on the connection stream, similarly to `onWindowUpdateRead`.

Result:

I figured I would just submit a PR rather than filing an issue, but let me know if you want me to do that for tracking purposes.
2020-11-04 14:01:08 +01:00
Aayush Atharva
c5077a3d87
HttpConversionUtil#toHttpResponse should use false in isRequest parameter (#10760)
Motivation:
`HttpConversionUtil#toHttpResponse` translates `Http2Headers` to `HttpResponse`. It uses `#addHttp2ToHttpHeaders(..., boolean isRequest)` to do so. However, `isRequest` field is set to `true` instead of `false`. It should be set to `false` because we're doing conversion of Response not Request.

Modification:
Changed `true` to `false`.

Result:
Correctly translates `Http2Headers` to `HttpResponse`.
2020-11-03 09:38:51 +01:00
Aayush Atharva
d1cf9774d5
Add toString method in DefaultHttp2WindowUpdateFrame (#10763)
Motivation:
We should have the `toString` method in `DefaultHttp2WindowUpdateFrame` because it makes debugging a little easy.

Modification:
Added `toString` method.

Result:
`toString` method to help in debugging.


Co-authored-by: Norman Maurer <norman_maurer@apple.com>
2020-11-03 09:37:18 +01:00
Aayush Atharva
1492374f99
Remove extra empty line (#10754)
Motivation:
`Http2Frame` has extra empty line after `String name();`. However, it should not be there.

Modification:
Removed extra empty line.

Result:
Empty-line code style now matching with other classes.
2020-11-02 15:07:51 +01:00
Aayush Atharva
0b0b446d38
Fix typo in Http2FrameCodec#write(...) comment
Motivation:
`Http2FrameCodec#write(...)` has typo in comment.
`// In the event of manual SETTINGS ACK is is assumed the encoder will apply the earliest received but not`.
The typo is `is is`. However, it should be `it is`.

Modification:
Changed `is is` to `it is`.

Result:
Correct comment without typos.
2020-11-02 08:49:13 +01:00
Aayush Atharva
1efc5f81e2
Fix typo in Http2DataFrame javadocs (#10755)
Motivation:
`Http2DataFrame#isEndStream()` JavaDoc says `Returns {@code true} if the END_STREAM flag ist set.`. The typo is `ist` word. However, it should be `is`.

Modification:
Changed `ist` to `is`.

Result:
Better JavaDoc by fixing the typo.
2020-10-30 15:59:31 +01:00
Aayush Atharva
87c46113d1
Fix typo in Http2HeadersFrame javadocs (#10756)
Motivation:
`Http2HeadersFrame#isEndStream()` JavaDoc says `Returns {@code true} if the END_STREAM flag ist set.`. The typo is `ist` word. However, it should be `is`.

Modification:
Changed `ist` to `is`.

Result:
Better JavaDoc by fixing the typo.
2020-10-30 15:58:34 +01:00
Aayush Atharva
e9b28e76a3
Change variable name of Http2Headers (#10743)
Motivation:
`addHttp2ToHttpHeaders(int streamId, Http2Headers sourceHeaders, FullHttpMessage destinationMessage, boolean addToTrailer)` 
should match 
`addHttp2ToHttpHeaders(int streamId, Http2Headers inputHeaders, HttpHeaders outputHeaders, HttpVersion httpVersion, boolean isTrailer, boolean isRequest)`.

However, the `Http2Headers` variable name is different.

Modification:
Changed `sourceHeaders` to `inputHeaders`.

Result:
Variable and JavaDoc naming now correct.
2020-10-29 10:40:49 +01:00
Aayush Atharva
8a10adaa55
Make Http2ToHttpHeaderTranslator method package-private. (#10736)
Motivation:
Http2ToHttpHeaderTranslator is a private class but translateHeaders(Iterable<Entry<CharSequence, CharSequence>>) is public but it should be package-private.

Modification:
Removed public.

Result:
Correct access modifer.
2020-10-29 10:39:02 +01:00
Aayush Atharva
ee3b9a5f7b
Fix possible NPEs and IndexOutOfBoundsExceptions in HTTP/2 Codec (#10640)
Motivation:

There are possible NPEs and IndexOutOfBoundsExceptions in HTTP/2 code. 

Modification:
Fixed possible NPEs and IOOBEs

Result:
Better code
2020-10-26 14:41:49 +01:00
Aayush Atharva
6ae7f9e1a8
Fix JavaDoc of Http2Headers (#10711)
Motivation:
Http2Headers has JavaDoc error which says Sets the {@link PseudoHeaderName#AUTHORITY} header or {@code null} if there is no such header however it should be Sets the {@link PseudoHeaderName#AUTHORITY} header in Http2Headers#authority(CharSequence) methods because it only sets CharSequence.

This is true for all setters in Http2Headers.

Modification:
Fixed all JavaDoc errors.

Result:
Better JavaDoc.
2020-10-23 15:35:43 +02:00
Artem Smotrakov
e5951d46fc
Enable nohttp check during the build (#10708)
Motivation:

HTTP is a plaintext protocol which means that someone may be able
to eavesdrop the data. To prevent this, HTTPS should be used whenever
possible. However, maintaining using https:// in all URLs may be
difficult. The nohttp tool can help here. The tool scans all the files
in a repository and reports where http:// is used.

Modifications:

- Added nohttp (via checkstyle) into the build process.
- Suppressed findings for the websites
  that don't support HTTPS or that are not reachable

Result:

- Prevent using HTTP in the future.
- Encourage users to use HTTPS when they follow the links they found in
  the code.
2020-10-23 14:44:18 +02:00
Norman Maurer
ffbddcd842
Replace deprecated Assert.assertThat(...) with MatcherAssert.assertThat(...) (#10699)
Motivation:

junit deprecated Assert.assertThat(...)

Modifications:

Use MatcherAssert.assertThat(...) as replacement for deprecated method

Result:

Less deprecation warnings
2020-10-18 14:30:52 +02:00
Norman Maurer
6b613682ba
Ensure we don't leak the ClassLoader in the backtrace (#10691)
Motivation:

We have a few classes in which we store and reuse static instances of various exceptions. When doing so it is important to also override fillInStacktrace() and so prevent the leak of the ClassLoader in the internal backtrace field.

Modifications:

- Add overrides of fillInStracktrace when needed
- Move ThrowableUtil usage in the static methods

Result:

Fixes https://github.com/netty/netty/pull/10686
2020-10-15 20:41:29 +02:00
Aayush Atharva
6cb72be2dd
Use ObjectUtil for multiple operations (#10679)
Motivation:
We should use ObjectUtil for checking if Compression parameters are in range. This will reduce LOC and make code more readable.

Modification:
Used ObjectUtil

Result:
More readable code
2020-10-14 12:01:42 +02:00
root
01b7e18632 [maven-release-plugin] prepare for next development iteration 2020-10-13 06:29:26 +00:00
root
d4a0050ef3 [maven-release-plugin] prepare release netty-4.1.53.Final 2020-10-13 06:29:02 +00:00
Aayush Atharva
4c231a2e77
Simplify and Remove useless Bit operations from HpackHuffmanDecoder (#10656)
Motivation:
We should simplify and remove useless bit operations to make code more efficient, faster, and easier to understand.

Modification:
Simplified and Removed Useless Bit Operations

Result:
Simpler Code.
2020-10-12 11:44:44 +02:00
Aayush Atharva
33cfcd1cfa
Remove extra line from end (#10673)
Motivation:
`Http2MultiplexHandler` have to 2 empty lines at the end instead of 1.

Modification:
Removed 1 extra line.

Result:
Little better code style.
2020-10-12 09:22:15 +02:00
Aayush Atharva
3298f5efee
Use ObjectUtil#checkPositive and Add missing JavaDoc In InboundHttp2ToHttpAdapter (#10637)
Motivation:
We can use ObjectUtil#checkPositive instead of manually checking maxContentLength. Also, there was a missing JavaDoc for Http2Exception,

Modification:
Used ObjectUtil#checkPositive for checking maxContentLength.
Added missing JavaDoc.

Result:
More readable code.
2020-10-07 11:12:00 +02:00
root
957ef746d8 [maven-release-plugin] prepare for next development iteration 2020-09-08 05:26:25 +00:00
root
ada9c38c0a [maven-release-plugin] prepare release netty-4.1.52.Final 2020-09-08 05:26:05 +00:00
Maksym Ostroverkhov
fb39fc450e
Http2ConnectionHandlerBuilder: make initialSettings() public (#10526)
Motivation:
be able to modify Http2Settings of Http2ConnectionHandlerBuilder.

Modifications:
make initialSettings() of Http2ConnectionHandlerBuilder public.

Result:
public initialSettings() of Http2ConnectionHandlerBuilder allow Http2Settings modification.
2020-09-02 21:12:35 +02:00
Nick Hill
8e8f01e01d
Use ByteBuf#isAccessible() in more places (#10506)
Motivation

ByteBuf has an isAccessible method which was introduced as part of ref
counting optimizations but there are some places still doing
accessibility checks by accessing the volatile refCnt() directly.

Modifications

- Have PooledNonRetained(Duplicate|Sliced)ByteBuf#isAccessible() use
their refcount delegate's isAccessible() method
- Add static isAccessible(buf) and ensureAccessible(buf) methods to
ByteBufUtil
(since ByteBuf#isAccessible() is package-private)
- Adjust DefaultByteBufHolder and similar classes to use these methods
rather than access refCnt() directly

Result

- More efficient accessibility checks in more places
2020-08-28 09:18:13 +02:00
Violeta Georgieva
4be9387151
Add validation when constructing Http2FrameLogger (#10495)
Motivation:

There should be a validation for the input arguments when constructing Http2FrameLogger

Modification:

Check that the provided arguments are not null

Result:

Proper validation when constructing Http2FrameLogger
2020-08-21 14:26:08 +02:00
Norman Maurer
fdd4706783
Include TLSv1.3 ciphers as recommented ciphers for HTTP2 (#10480)
Motivation:

We should include TLSv1.3 ciphers as well as recommented ciphers these days for HTTP/2. That is especially true as Java supports TLSv1.3 these days out of the box

Modifications:

- Add TLSv1.3 ciphers that are recommended by mozilla as was for HTTP/2
- Add unit test

Result:

Include TLSv1.3 ciphers as well
2020-08-13 20:33:07 +02:00
Eric Anderson
61b0fa97d7
Permit h2 PRIORITY frames with a dependency on the connection (#10473)
Motivation:

Setting a dependency on the connection is normal and permitted; streams
actually default to depending on the connection. Using a PRIORITY frame
with a dependency on the connection could reset a previous PRIORITY,
change the relative weight, or make all streams dependent on one stream.

The previous code was disallowing these usages as it considered
depending on the connection to be a validation failure.

Modifications:

Loosen validation check to also allow depending on the connection. Fix
error message when the validation check fails.

Result:

Setting a dependency on connection would be permitted. Fixes #10416
2020-08-11 13:50:56 -07:00
Idel Pivnitskiy
5aea78950f
Create a new h2 stream only if the parent channel is still active (#10444)
Motivation:

If a request to open a new h2 stream was made from outside of the
EventLoop it will be scheduled for future execution on the EventLoop.
However, during the time before the `open0` task will be executed the
parent channel may already be closed. As the result,
`Http2MultiplexHandler#newOutboundStream()` will throw an
`IllegalStateException` with the message that is hard to
interpret correctly for this use-case: "Http2FrameCodec not found. Has
the handler been added to a pipeline?".

Modifications:

- Check that the parent h2 `Channel` is still active before creating a
new stream when `open0` task is picked up by EventLoop;

Result:

Users see a correct `ClosedChannelException` in case the parent h2
`Channel` was closed concurrently with a request for a new stream.
2020-08-06 13:54:46 +02:00
Idel Pivnitskiy
de4627852f
New H2 stream promise may never complete (#10443)
Motivation:

`Http2StreamChannelBootstrap#open0` invokes
`Http2MultiplexHandler#newOutboundStream()` which may throw an
`IllegalStateException`. In this case, it will never complete
the passed promise.

Modifications:

- `try-catch` all invocations of `newOutboundStream()` and fail
promise in case of any exception;

Result:

New H2 stream promise always completes.
2020-08-06 08:34:10 +02:00
Idel Pivnitskiy
dd168f406e
Make DefaultHttp2FrameStream.stream private (#10442)
Motivation:

`DefaultHttp2FrameStream.stream` is not used outside of its class and
therefore can be private.

Modifications:

- Make `DefaultHttp2FrameStream.stream` private;

Result:

Correct visibility scoping for `DefaultHttp2FrameStream.stream`.
2020-08-03 07:55:13 +02:00
Esun Kim
573ff0a33e
Making AES_128_GCM_SHA256 have a higher priority over CHACHA20_POLY1305_SHA256 on HTTP2 (#10418)
Motivation:

From the recent benchmark using gRPC-Java based on Netty's HTTP2, it appears that it prefers `ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256` over `ECDHE_ECDSA_WITH_AES_128_GCM_SHA256` since it uses the Netty HTTPS Cipher list as is. Both are considered safe but `ECDHE_ECDSA_WITH_AES_128_GCM_SHA256` has a good chance to get more optimized implementation. (e.g. AES-NI) When running both on GCP Intel Haswell VM, `TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256` spent 3x CPU time than `TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256`. (Note that this VM supports AES-NI)

From the cipher suites listed on `Intermediate compatibility (recommended)` of [Security/Server Side TLS](https://wiki.mozilla.org/Security/Server_Side_TLS#Modern_compatibility), they have a cipher preference which is aligned with this PR.

```
0x13,0x01  -  TLS_AES_128_GCM_SHA256         TLSv1.3  Kx=any   Au=any    Enc=AESGCM(128)             Mac=AEAD
0x13,0x02  -  TLS_AES_256_GCM_SHA384         TLSv1.3  Kx=any   Au=any    Enc=AESGCM(256)             Mac=AEAD
0x13,0x03  -  TLS_CHACHA20_POLY1305_SHA256   TLSv1.3  Kx=any   Au=any    Enc=CHACHA20/POLY1305(256)  Mac=AEAD
0xC0,0x2B  -  ECDHE-ECDSA-AES128-GCM-SHA256  TLSv1.2  Kx=ECDH  Au=ECDSA  Enc=AESGCM(128)             Mac=AEAD
0xC0,0x2F  -  ECDHE-RSA-AES128-GCM-SHA256    TLSv1.2  Kx=ECDH  Au=RSA    Enc=AESGCM(128)             Mac=AEAD
0xC0,0x2C  -  ECDHE-ECDSA-AES256-GCM-SHA384  TLSv1.2  Kx=ECDH  Au=ECDSA  Enc=AESGCM(256)             Mac=AEAD
0xC0,0x30  -  ECDHE-RSA-AES256-GCM-SHA384    TLSv1.2  Kx=ECDH  Au=RSA    Enc=AESGCM(256)             Mac=AEAD
0xCC,0xA9  -  ECDHE-ECDSA-CHACHA20-POLY1305  TLSv1.2  Kx=ECDH  Au=ECDSA  Enc=CHACHA20/POLY1305(256)  Mac=AEAD
0xCC,0xA8  -  ECDHE-RSA-CHACHA20-POLY1305    TLSv1.2  Kx=ECDH  Au=RSA    Enc=CHACHA20/POLY1305(256)  Mac=AEAD
0x00,0x9E  -  DHE-RSA-AES128-GCM-SHA256      TLSv1.2  Kx=DH    Au=RSA    Enc=AESGCM(128)             Mac=AEAD
0x00,0x9F  -  DHE-RSA-AES256-GCM-SHA384      TLSv1.2  Kx=DH    Au=RSA    Enc=AESGCM(256)             Mac=AEAD
```

Modification:

Moving up `AES_128_GCM_SHA256` in the CIPHERS of HTTPS so that it gets priority.

Result:

When connecting to the server supporting both `ECDHE_ECDSA_WITH_AES_128_GCM_SHA256` and `ECDSA_WITH_CHACHA20_POLY1305_SHA256` and respecting the client priority of cipher suites, it will be able to save significant cpu time when running it on machines with AES-NI support.
2020-08-03 07:49:55 +02:00
feijermu
77101ab8af
Eliminate a redundant method call in HpackDynamicTable.add(...) (#10399)
Motivation:

The result of `header.size()` is already cached in `headerSize`. There is no need to call it again actually.

Modification:

Replace the second `header.size()` with `headerSize` directly.

Result:

Improve performance slightly.
2020-07-13 17:34:29 +02:00
root
bfbeb2dec6 [maven-release-plugin] prepare for next development iteration 2020-07-09 12:27:06 +00:00
root
646934ef0a [maven-release-plugin] prepare release netty-4.1.51.Final 2020-07-09 12:26:30 +00:00
feijermu
7a05aa1cf8
Add detailed error message corresponding to the IndexOutOfBoundsException while calling getEntry(...) (#10386)
Motivation:
`getEntry(...)` may throw an IndexOutOfBoundsException without any error messages.


Modification:

Add detailed error message corresponding to the IndexOutOfBoundsException while calling `getEntry(...)` in HpackDynamicTable.java.

Result: 

Easier to debug
2020-07-06 10:19:22 +02:00
feijermu
a608437089
Add unit test for HpackDynamicTable. (#10389)
Motivation:

`HpackDynamicTable` needs some test cases to ensure bug-free.

Modification:

Add unit test for `HpackDynamicTable`.

Result:

Improve test coverage slightly.
2020-07-06 09:07:30 +02:00
root
caf51b7284 [maven-release-plugin] prepare for next development iteration 2020-05-13 06:00:23 +00:00
root
8c5b72aaf0 [maven-release-plugin] prepare release netty-4.1.50.Final 2020-05-13 05:59:55 +00:00
feijermu
d10c946e1b
Remove a unused private method with empty body in HttpConversionUtil.java (#10266)
Motivation:

After searching the whole netty project, I found that the private method `translateHeader(...)` with empty body is never used actually. So I think it could be safely removed.

Modification:

Just remove this unused method.

Result:

Clean up the code.
2020-05-11 15:31:56 +02:00
feijermu
4c758a214d
Remove unused imports in DefaultHttp2LocalFlowController.java and HpackStaticTable.java (#10265)
Motivation:

`io.netty.handler.codec.http2.Http2Stream.State` is never used in DefaultHttp2LocalFlowController.java, and `io.netty.handler.codec.http2.HpackUtil.equalsConstantTime` is never used in HpackStaticTable.java.

Modification:

Just remove these unused imports.

Result:

Make imports cleaner.
2020-05-11 15:10:20 +02:00
feijermu
40448db5bb
Remove a unused import in DefaultHttp2ConnectionEncoder.java (#10249)
Motivation:

`io.netty.handler.codec.http2.Http2CodecUtil.DEFAULT_PRIORITY_WEIGHT` is never used in DefaultHttp2ConnectionEncoder.java.

Modification:

Just remove this unused import.

Result:

Make the DefaultHttp2ConnectionEncoder.java's imports clean.
2020-05-07 08:10:36 +02:00
Norman Maurer
0cf919c389
HTTP2: Guard against multiple ctx.close(...) calls with the same ChannelPromise (#10201)
Motivation:

Http2ConnectionHandler may call ctx.close(...) with the same promise instance multiple times if the timeout for gracefulShutdown elapse and the listener itself is notified. This can cause problems as other handlers in the pipeline may queue these promises and try to notify these later via setSuccess() or setFailure(...) which will then throw an IllegalStateException if the promise was notified already

Modification:

- Add boolean flag to ensure doClose() will only try to call ctx.close(...) one time

Result:

Don't call ctx.close(...) with the same promise multiple times when gradefulShutdown timeout elapses.
2020-04-23 11:25:39 +02:00
root
9c5008b109 [maven-release-plugin] prepare for next development iteration 2020-04-22 09:57:54 +00:00
root
d0ec961cce [maven-release-plugin] prepare release netty-4.1.49.Final 2020-04-22 09:57:26 +00:00
root
14e4afeba2 [maven-release-plugin] prepare for next development iteration 2020-03-17 09:20:54 +00:00
root
c10c697e5b [maven-release-plugin] prepare release netty-4.1.48.Final 2020-03-17 09:18:28 +00:00
feijermu
679c606f13
Add test case for Http2StreamChannelId. (#10108)
Motivation:

Http2StreamChannelId is Serializable. A test case is needed to ensure that it works.

Modification:

Add a test case about serialization.

Result:

Improve test coverage slightly.
2020-03-16 09:31:38 +01:00
root
c623a50d19 [maven-release-plugin] prepare for next development iteration 2020-03-09 12:13:56 +00:00
root
a401b2ac92 [maven-release-plugin] prepare release netty-4.1.47.Final 2020-03-09 12:13:26 +00:00
Bryce Anderson
7b946a781e
Make sure we always flush window update frames in AbstractHttp2StreamChannel (#10075)
Motivation:

Under certain read patters the AbstractHttp2StreamChannel can fail to
flush, resulting in flow window starvation.

Modifications:

- Ensure we flush if we exit the `doBeginRead()` method.
- Account for the Http2FrameCodec always synchronously finishing writes
  of window update frames.

Result:

Fixes #10072
2020-03-04 10:50:41 +01:00
root
e0d73bca4d [maven-release-plugin] prepare for next development iteration 2020-02-28 06:37:33 +00:00
root
ebe7af5102 [maven-release-plugin] prepare release netty-4.1.46.Final 2020-02-28 06:36:45 +00:00
Hyunjin Choi
1859170e69
Change String comparison to equals() from == (#10022)
Motivation:

Even if it was stored in the string constant pool, I thought it was safe to compare it through the Equals() method.

Modification:

So, I changed "==" comparison to equals() comparison

Result:

It has become safer to compare String values with different references and with the same values.
2020-02-13 09:57:14 +01:00
Norman Maurer
2d4b5abc99
HTTP/2 child channel may discard flush when done from an arbitrary thread (#10019)
Motivation:

We need to carefully manage flushes to ensure we not discard these by mistake due wrongly implemented consolidation of flushes.

Modifications:

- Ensure we reset flag before we actually call flush0(...)
- Add unit test

Result:

Fixes https://github.com/netty/netty/issues/10015
2020-02-11 14:43:04 +01:00
Bryce Anderson
238aa73e36 DefaultHttp2ConnectionDecoder notifies frame listener before connection of GOAWAYS (#10009)
Motivation:

Users of the DefaultHttp2ConnectionDecodcer are notified of inbound GoAwayFrames
after the connection has already closed any ignored streams, potentially
losing the signal that some streams may have been ignored by the peer and
are thus retryable.

Modifications:

Reorder the notifications of the frame and connection listeners to
propagate the frame first, giving the frame listeners the opportunity to
clean up ignored streams in their own way.

Result:
Fixes #9986
2020-02-10 10:26:54 +01:00
Norman Maurer
e7f291629b ClearTextHttp2ServerUpgradeHandler can be merged with inner PriorKnowledgeHandler (#10008)
Motivation:

ClearTextHttp2ServerUpgradeHandler is currently more complex then needed. We can simplify it by directly implement the prior-knowledge logic as part of the handler.

Modifications:

Merge inner PriorKnowledgeHandler logic into ClearTextHttp2ServerUpgradeHandler by extending ByteToMessageDecoder directly

Result:

Cleaner code and less pipeline operations
2020-02-08 08:49:55 +01:00
Norman Maurer
5bc644ba41
Ensure ChannelOptions are applied in the same order as configured in Http2StreamChannelBootstrap (#9998) (#10001)
Motivation:

https://github.com/netty/netty/pull/9848 changed how we handled ChannelOptions internally to use a ConcurrentHashMap. This unfortunally had the side-effect that the ordering may be affected and not stable anymore. Here the problem is that sometimes we do validation based on two different ChannelOptions (for example we validate high and low watermarks against each other). Thus even if the user specified the options in the same order we may fail to configure them.

Modifications:

- Use again a LinkedHashMap to preserve order

Result:

Apply ChannelOptions in correct and expected order
2020-02-07 09:14:16 +01:00
root
9b1ea10a12 [maven-release-plugin] prepare for next development iteration 2020-01-13 09:13:53 +00:00
root
136db8680a [maven-release-plugin] prepare release netty-4.1.45.Final 2020-01-13 09:13:30 +00:00
Ikhun Um
7f241f1f3c Fix typos in javadocs (#9900)
Motivation:

Javadocs should have no typos.

Modifications:

Fix two typos

Result:

Less typos.
2019-12-23 08:34:16 +01:00
root
79d4e74019 [maven-release-plugin] prepare for next development iteration 2019-12-18 08:32:54 +00:00
root
5ddf45a2d5 [maven-release-plugin] prepare release netty-4.1.44.Final 2019-12-18 08:31:43 +00:00
Norman Maurer
3cad3c0ae7
Revert "Validate pseudo and conditional HTTP/2 headers (#8619)" (#9893)
This reverts commit ffc3b2da72.
2019-12-17 17:43:52 +01:00
Sergey Skrobotov
84547029d9 Change DefaultByteBufHolder.equals() to treat instances of different classes as not equal (#9855)
# Motivation:
`DefaultByteBufHolder.equals()` considers another object equal if it's an instance of `ByteBufferHolder` and if the contents of two objects are equal. However, the behavior of `equals` method is not a part of the `ByteBufHolder` contract so `DefaultByteBufHolder`'s version may be causing violation of the symmetric property if other classes have different logic.
There are already a few classes that are affected by this: `DefaultHttp2GoAwayFrame`, `DefaultHttp2UnknownFrame`, and `SctpMessage` are all overriding `equals` method breaking the symmetric property.
Another effect of this behavior is that all instances with empty data are considered equal. That may not be desireable in the situations when instances are created for predefined constants, e.g. `FullBulkStringRedisMessage.NULL_INSTANCE` and `FullBulkStringRedisMessage.EMPTY_INSTANCE` in `codec-redis`. 

# Modification:
Make `DefaultByteBufHolder.equals()` implementation only work for the objects of the same class.

# Result:
- The symmetric property of the `equals` method is restored for the classes in question.
- Instances of different classes are not considered equal even if the content of the data they hold are the same.
2019-12-10 11:29:44 +01:00
时无两丶
0cde4d9cb4 Uniform null pointer check. (#9840)
Motivation:
Uniform null pointer check.

Modifications:

Use ObjectUtil.checkNonNull(...)

Result:
Less code, same result.
2019-12-09 09:47:35 +01:00
Norman Maurer
e875e59a9b
DefaultHttp2ConnectionEncoder writeHeaders method always send an head frame with a priority (#9852)
Motivation:

The current implementation delegates to writeHeaders(ChannelHandlerContext ctx, int streamId, Http2Headers headers, int streamDependency, short weight, boolean exclusive, int padding, boolean endStream, ChannelPromise promise) that will send an header frame with the priority flag set and the default priority values even if the user didnt want too.

Modifications:

- Change DefaultHttp2ConnectionEncoder to call the correct Http2FrameWriter method depending on if the user wants to use priorities or not
- Adjust tests

Result:

Fixes https://github.com/netty/netty/issues/9842
2019-12-08 07:43:11 +01:00
Norman Maurer
e69c4173ea
Replace synchronized with ConcurrentHashMap in Http2StreamChannelBootstrap (#9848)
Motivation:

97361fa2c8 replace synchronized with ConcurrentHashMap in *Bootstrap classes but missed to do the same for the Http2 variant.

Modifications:

- Use ConcurrentHashMap
- Simplify code in *Bootstrap classes

Result:

Less contention
2019-12-06 10:59:55 +01:00
Norman Maurer
d0f94200e8
Call ctx.flush() when onStreamClosed(...) produces a window update frame (#9818)
Motivation:

We use the onStreamClosed(...) callback to return unconsumed bytes back to the window of the connection when needed. When this happens we will write a window update frame but not automatically call ctx.flush(). As the user has no insight into this it could in the worst case result in a "deadlock" as the frame is never written out ot the socket.

Modifications:

- If onStreamClosed(...) produces a window update frame call ctx.flush()
- Add unit test

Result:

No stales possible due unflushed window update frames produced by onStreamClosed(...) when not all bytes were consumed before the stream was closed
2019-11-28 11:11:32 +01:00
Norman Maurer
3654d2c245
Use EmbeddedChannel.finishAndReleaseAll() to remove boiler-plate code (#9824)
Motivation:

We can make use of EmbeddedChannel.finishAndReleaseAll() and so remove some code

Modifications:

Use finishAndReleaseAll()

Result:

Less code to maintain
2019-11-28 11:03:39 +01:00
Norman Maurer
98615a0de5
Don't send window update frame for unconsumed bytes when stream is already closed (#9816)
Motivation:

At the moment we send a window update frame for the connection + stream when a stream is closed and there are unconsumed bytes left. While we need to do this for the connection it makes no sense to write a window update frame for the stream itself as it is already closed

Modifications:

- Don't write the window update frame for the stream when the stream is closed
- Add unit test

Result:

Don't write the window frame for closed streams
2019-11-28 09:06:32 +01:00
Norman Maurer
5f5776c3e6
Correctly guard against multiple RST frames for the same stream (#9811)
Motivation:

Http2ConnectionHandler tries to guard against sending multiple RST frames for the same stream. Unfortunally the code is not 100 % correct as it only updates the state after it calls write. This may lead to the situation of have an extra RST frame slip through if the second write for the RST frame is done from a listener that is attached to the promise.

Modifications:

- Update state before calling write
- Add unit test

Result:

Only ever send one RST frame per stream
2019-11-27 06:54:19 +01:00
Bryce Anderson
09eed19438 Log expected STREAM_CLOSED exceptions for already closed streams at DEBUG level (#9798)
Motivation:

There is an intrinsic race between a local session resetting a stream
and the peer no longer sending any frames. This can result in the
session receiving frames for a stream that the local peer no longer
tracks. This results in a StreamException being thrown which triggers a
RST_STREAM frame, which is a good thing, but also logging at level WARN,
which is noisy for an expected and benign condition.

Modification:

Change the log level to DEBUG when logging stream errors with code
STREAM_CLOSED. All others are more interesting and will continue to be
logged at level WARN.

Additionally, it was found that DATA frames for streams that could not
have existed only resulted in a StreamException when the spec is clear
that such a situation should be fatal to the connection, resulting in a
GOAWAY(PROTOCOL_ERROR).

Fixes #8025.
2019-11-25 09:01:42 +01:00
monkey-mas
dc618d6046 Add test to check Connection-Specific headers are removed in HTTP/2 (by HttpConversionUtil.toHttp2Headers) (#9766)
Motivation:
To avoid regression regarding connection-specific headers[1], we should add a test.
[1] https://tools.ietf.org/html/rfc7540#section-8.1.2.2

Modification:
Add test that checks the following headers are removed.
- Connection
- Host
- Keep-Alive
- Proxy-Connection
- Transfer-Encoding
- Upgrade

Result:
There's no functional change.
2019-11-08 10:14:26 +01:00
zhangheng
044beae313 Remove padding when writing CONTINUATION frame (#9752)
Motivation:

Padding was removed from CONTINUATION frame in http2-spec, as showed in [PR](https://github.com/http2/http2-spec/pull/510). We should follow it.

Modifications:

- Remove padding when writing CONTINUATION frame in DefaultHttp2FrameWriter
- Add a unit test for writing large header with padding

Result:

More spec-compliant
2019-11-05 15:20:36 +01:00
monkey-mas
2796407fe8 Remove unnecessary line in Http2ClientUpgradeCodec (#9750)
Motivation:
To clean up code.

Modification:
Remove unnecessary line.

Result:
There's no functional change.
2019-11-04 11:20:04 +01:00
Norman Maurer
4f6c21fa97
Fix Http2Headers.method(...) javadocs (#9718)
Motivation:

The javadocs of Http2Headers.method(...) are incorrect, we should fix these.

Modifications:

Correct javadocs

Result:

Fixes https://github.com/netty/netty/issues/8068.
2019-10-29 19:51:28 +01:00
Norman Maurer
bf07592668 Fix typo in test which did introduce a failing test after ffc3b2da72 2019-10-28 09:26:29 +01:00
Julien Hoarau
ffc3b2da72 Validate pseudo and conditional HTTP/2 headers (#8619)
Motivation:

Netty HTTP/2 implementation is not 100% compliant to the spec. This
commit improves the compliance regarding headers validation,
in particular pseudo-headers and connection ones.

According to the spec:
   All HTTP/2 requests MUST include exactly one valid value for the
   ":method", ":scheme", and ":path" pseudo-header fields, unless it is
   a CONNECT request (Section 8.3).  An HTTP request that omits
   mandatory pseudo-header fields is malformed (Section 8.1.2.6).

Modifications:

- Introduce Http2HeadersValidator class capable of validating HTTP/2
headers
- Invoke validation from DefaultHttp2ConnectionDecoder#onHeadersRead
- Modify tests to use valid headers when required
- Modify HttpConversionUtil#toHttp2Headers to not add :scheme and
:path header on CONNECT method in order to conform to the spec

Result:

- Initial requests without :method, :path, :scheme will fail
- Initial requests with multiple values for :method, :path, :scheme
will fail
- Initial requests with an empty :path fail
- Requests with connection-specific header field will fail
- Requests with TE header different than "trailers" will fail
-
- Fixes 8.1.2.2 tests from h2spec #5761
- Fixes 8.1.2.3 tests from h2spec #5761
2019-10-27 16:13:01 +01:00
root
844b82b986 [maven-release-plugin] prepare for next development iteration 2019-10-24 12:57:00 +00:00
root
d066f163d7 [maven-release-plugin] prepare release netty-4.1.43.Final 2019-10-24 12:56:30 +00:00
Carl Mastrangelo
e8e7a206b3 Use fast HPACK comparisons when not checking sensitive headers (#9259)
Motivation:
Constant time comparison functions are used to compare HTTP/2 header
values, even if they are not sensitive.

Modification:
After checking for sensitivity, use fast comparison.

Result: Faster HPACK table reads/writes
2019-10-23 23:37:57 -07:00
Norman Maurer
9bf10f2dc5
Correctly propagate failures while update the flow-controller to the … (#9664)
Motivation:

We may fail to update the flow-controller and in this case need to notify the stream channel and close it.

Modifications:

Attach a future to the write of the update frame and in case of a failure propagate it to the channel and close it

Result:

Fixes https://github.com/netty/netty/issues/9663
2019-10-22 05:40:14 -07:00
Norman Maurer
95230e01da
Try to reduce GC produced while writing headers (#9682)
Motivation:

bbc34d0eda introduced correct handling of "in process" setup of streams but there is some room for improvements. Often the writeHeaders(...) is completed directly which means there is not need to create the extra listener object.

Modifications:

- Only create the listener if we really need too.

Result:

Less GC
2019-10-17 10:12:20 -07:00
Matthew Miller
bbc34d0eda HTTP/2: Prevent memory leak when trying to create new streams on a connection that received a GOAWAY. (#9674)
Motivation:

In https://github.com/netty/netty/issues/8692, `Http2FrameCodec` was
updated to keep track of all "being initialized" streams, allocating
memory before initialization begins, and releasing memory after
initialization completes successfully.

In some instances where stream initialization fails (e.g. because this
connection has received a GOAWAY frame), this memory is never released.

Modifications:

This change updates the `Http2FrameCodec` to use a separate promise
for monitoring the success of sending HTTP2 headers. When sending of
headers fails, we now make sure to release memory allocated for stream
initialization.

Result:

After this change, failures in writing HTTP2 Headers (e.g. because this
connection has received a GOAWAY frame) will no longer leak memory.
2019-10-16 19:32:45 -07:00
康智冬
bd8cea644a Fix typos in javadocs (#9527)
Motivation:

We should have correct docs without typos

Modification:

Fix typos and spelling

Result:

More correct docs
2019-10-09 17:12:52 +04:00
Norman Maurer
271d8de9ec
Ensure we finish setup mock before we use it in Http2ConnectionRoundtripTest.headersWriteForPeerStreamWhichWasResetShouldNotGoAway (#9645)
Motivation:

We did dispatch the client code before we did finish setup the mock and so may end up with org.mockito.exceptions.misusing.UnfinishedStubbingException if the connect happens quickly enough.

See https://ci.netty.io/job/netty-centos6-java8-prb/1637/testReport/junit/io.netty.handler.codec.http2/Http2ConnectionRoundtripTest/headersWriteForPeerStreamWhichWasResetShouldNotGoAway/

Modifications:

First finish setup the mock and the dispatch.

Result:

Fix flacky test
2019-10-09 09:10:38 +04:00
Carl Mastrangelo
27397e87b2 Remember to return writability events to flow controller in HTTP2 Multiplexer (#9642)
Motivation:

Http2MultiplexCodec extends Http2FrameCodec extends Http2ConnectionHandler.  It appears  Http2MultiplexCodec overrode the channelWritabilityChanged method, which prevented the flow controller from becoming active.  In the case the parent channel becomes unwritable, and then later becomes writable, it needs to indicate that the child channels can still write data.   This is slightly confusing, because the child channels may still themselves be unwritable, but should still drain their data to the parent channel.

Modification:

Still propagate writability changes to the HTTP/2 flow controller

Result:

Fixes https://github.com/netty/netty/issues/9636
2019-10-08 18:40:23 +04:00
Pete Woods
45be693889 Add io.netty.handler.codec.http2.Http2ConnectionHandler for runtime GraalVM compilation (#9621)
Motivation:

Native image compilation is failing without extra flags:

```
Warning: Aborting stand-alone image build. No instances of io.netty.buffer.UnpooledHeapByteBuf are allowed in the image heap as this class should be initialized at image runtime. Object has been initialized by the io.netty.handler.codec.http2.Http2ConnectionHandler class initializer with a trace: 
 	at io.netty.buffer.Unpooled.wrappedBuffer(Unpooled.java:157)
	at io.netty.handler.codec.http2.Http2ConnectionHandler.<clinit>(Http2ConnectionHandler.java:74)
.  To fix the issue mark io.netty.buffer.UnpooledHeapByteBuf for build-time initialization with --initialize-at-build-time=io.netty.buffer.UnpooledHeapByteBuf or use the the information from the trace to find the culprit and --initialize-at-run-time=<culprit> to prevent its instantiation.

Detailed message:
Trace: 	object io.netty.buffer.ReadOnlyByteBuf
	object io.netty.buffer.UnreleasableByteBuf
	method io.netty.handler.codec.http2.Http2ConnectionHandler.access$500()
Call path from entry point to io.netty.handler.codec.http2.Http2ConnectionHandler.access$500(): 
	at io.netty.handler.codec.http2.Http2ConnectionHandler.access$500(Http2ConnectionHandler.java:66)
	at io.netty.handler.codec.http2.Http2ConnectionHandler$PrefaceDecoder.readClientPrefaceString(Http2ConnectionHandler.java:299)
	at io.netty.handler.codec.http2.Http2ConnectionHandler$PrefaceDecoder.decode(Http2ConnectionHandler.java:239)
	at io.netty.handler.codec.http2.Http2ConnectionHandler.decode(Http2ConnectionHandler.java:438)
	at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:505)
	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:444)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:283)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
	at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:56)
	at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:365)
	at com.oracle.svm.core.jdk.RuntimeSupport.executeHooks(RuntimeSupport.java:144)
	at com.oracle.svm.core.jdk.RuntimeSupport.executeStartupHooks(RuntimeSupport.java:89)
	at com.oracle.svm.core.JavaMainWrapper.runCore(JavaMainWrapper.java:143)
	at com.oracle.svm.core.JavaMainWrapper.run(JavaMainWrapper.java:186)
	at com.oracle.svm.core.code.IsolateEnterStub.JavaMainWrapper_run_5087f5482cc9a6abc971913ece43acb471d2631b(generated:0)
```

Modification:

Add `io.netty.handler.codec.http2.Http2ConnectionHandler` for runtime compilation, as the buffer library's `io.netty.buffer.UnpooledHeapByteBuf` is also marked for runtime.

Result:

Native image compilation works again.
2019-10-07 09:03:07 +02:00
Nikolay Fedorovskikh
1fb5ff15a8 Fix possible NPE in DefaultHttp2UnknownFrame#equals (#9625)
Motivation:
`DefaultHttp2UnknownFrame#equals` may produce NPE due to
incorrect comparison of `stream` field.

Modification:
- Fix the `stream` field compare.
- Cleanup usage of class fields: use direct access instead of getters
(because the class is final).

Result:
No NPE in `equals` method.
2019-10-07 09:00:59 +02:00
Nikolay Fedorovskikh
4dc1eccf60 Make some inner classes static (#9624)
Motivation:
Classes `AbstractHttp2StreamChannel.Http2StreamChannelConfig`
and `DnsNameResolver.AddressedEnvelopeAdapter` may be static:
it doesn't reference its enclosing instance.

Modification:
Add `static` modifier.

Result:
Prevents a possible memory leak and uses less memory per class instance.
2019-10-07 08:14:02 +02:00
Norman Maurer
622cc232f0
Use configured ByteBufAllocator in InboundHttp2ToHttpAdapter (#9611)
Motivation:

At the moment we use Unpooled.buffer(...) in InboundHttp2ToHttpAdapter when we need to do a copy of the message. We should better use the configured ByteBufAllocator for the Channel

Modifications:

Change internal interface to also take the ByteBufAllocator as argument and use it when we need to allocate a ByteBuf.

Result:

Use the "correct" ByteBufAllocator in InboundHttp2ToHttpAdapter in all cases
2019-09-26 22:11:41 +02:00
Norman Maurer
1f4b9e36ea
We should only disable releasing of the message once writeData(...) was called successfully (#9610)
Motivation:

At the moment we set release to false before we call writeData(...). This could let to the sitatuation that we will miss to release the message if writeData(...) throws. We should set release to false after we called writeData(...) to ensure the ownership of the buffer is correctly transferred.

Modifications:

- Set release to false after writeData(...) was successfully called only

Result:

No possibility for a buffer leak
2019-09-26 21:59:57 +02:00
Norman Maurer
299a682d3f
Correctly take Http2FrameCodecBuilder.isValidateHeaders() into account when creating a Http2FrameCodec from an existing Http2FrameWriter. (#9600)
Motivation:

We did miss to take Http2FrameCodecBuilder.isValidateHeaders() into account when a Http2FrameWriter was set on the builder and always assumed validation should be enabled.

Modifications:

Remove hardcode value and use configured value

Result:

Http2FrameCodecBuilder.isValidateHeaders() is respected in all cases
2019-09-26 21:57:05 +02:00
root
92941cdcac [maven-release-plugin] prepare for next development iteration 2019-09-25 06:15:31 +00:00
root
bd907c3b3a [maven-release-plugin] prepare release netty-4.1.42.Final 2019-09-25 06:14:31 +00:00
Pete Woods
0a2d85f1d3 Fix GraalVM native image build error (#9593)
Motivation:

Error: Class that is marked for delaying initialization to run time got initialized during image building: io.netty.handler.codec.http2.Http2CodecUtil. Try marking this class for build-time initialization with --initialize-at-build-time=io.netty.handler.codec.http2.Http2CodecUtil
Error: Use -H:+ReportExceptionStackTraces to print stacktrace of underlying exception
Error: Image build request failed with exit status 1
Modification:

After debugging, it seems the culprit is io.netty.handler.codec.http2.Http2ClientUpgradeCodec, which also needs runtime initialisation.

Result:

Fixes #micronaut-projects/micronaut-grpc#8
2019-09-23 14:42:42 +02:00
root
01d805bb76 [maven-release-plugin] prepare for next development iteration 2019-09-12 16:09:55 +00:00
root
7cf69022d4 [maven-release-plugin] prepare release netty-4.1.41.Final 2019-09-12 16:09:00 +00:00