Commit Graph

519 Commits

Author SHA1 Message Date
Norman Maurer
93267f9b98 Revert "Validate pseudo and conditional HTTP/2 headers (#8619)"
This reverts commit dd5d4887ed.
2019-12-17 17:45:17 +01:00
Norman Maurer
0e4c073bcf
Remove the intermediate List from ByteToMessageDecoder (and sub-class… (#8626)
Motivation:

ByteToMessageDecoder requires using an intermediate List to put results into. This intermediate list adds overhead (memory/CPU) which grows as the number of objects increases. This overhead can be avoided by directly propagating events through the ChannelPipeline via ctx.fireChannelRead(...). This also makes the semantics more clear and allows us to keep track if we need to call ctx.read() in all cases.

Modifications:

- Remove List from the method signature of ByteToMessageDecoder.decode(...) and decodeLast(...)
- Adjust all sub-classes
- Adjust unit tests
- Fix javadocs.

Result:

Adjust ByteToMessageDecoder as noted in https://github.com/netty/netty/issues/8525.
2019-12-16 21:00:32 +01:00
Norman Maurer
4a8476af67 Revert "Protect ChannelHandler from reentrancee issues (#9358)"
This reverts commit 48634f1466.
2019-12-12 14:42:30 +01:00
Sergey Skrobotov
f10bee9057 Change DefaultByteBufHolder.equals() to treat instances of different classes as not equal (#9855)
# Motivation:
`DefaultByteBufHolder.equals()` considers another object equal if it's an instance of `ByteBufferHolder` and if the contents of two objects are equal. However, the behavior of `equals` method is not a part of the `ByteBufHolder` contract so `DefaultByteBufHolder`'s version may be causing violation of the symmetric property if other classes have different logic.
There are already a few classes that are affected by this: `DefaultHttp2GoAwayFrame`, `DefaultHttp2UnknownFrame`, and `SctpMessage` are all overriding `equals` method breaking the symmetric property.
Another effect of this behavior is that all instances with empty data are considered equal. That may not be desireable in the situations when instances are created for predefined constants, e.g. `FullBulkStringRedisMessage.NULL_INSTANCE` and `FullBulkStringRedisMessage.EMPTY_INSTANCE` in `codec-redis`. 

# Modification:
Make `DefaultByteBufHolder.equals()` implementation only work for the objects of the same class.

# Result:
- The symmetric property of the `equals` method is restored for the classes in question.
- Instances of different classes are not considered equal even if the content of the data they hold are the same.
2019-12-10 11:30:23 +01:00
Norman Maurer
ff8846f1b5
Replace ObjectUtil.checkNonNull(...) with Objects.requireNonNull(...) (#9864)
Motivation:

We should use Objects.requireNonNull(...) as we require java8

Modifications:

Replace ObjectUtil.checkNonNull(...) with Objects.requireNonNull(...)

Result:

Code cleanup
2019-12-10 11:27:32 +01:00
Norman Maurer
8e281dc54e DefaultHttp2ConnectionEncoder writeHeaders method always send an head frame with a priority (#9852)
Motivation:

The current implementation delegates to writeHeaders(ChannelHandlerContext ctx, int streamId, Http2Headers headers, int streamDependency, short weight, boolean exclusive, int padding, boolean endStream, ChannelPromise promise) that will send an header frame with the priority flag set and the default priority values even if the user didnt want too.

Modifications:

- Change DefaultHttp2ConnectionEncoder to call the correct Http2FrameWriter method depending on if the user wants to use priorities or not
- Adjust tests

Result:

Fixes https://github.com/netty/netty/issues/9842
2019-12-08 08:00:10 +01:00
Norman Maurer
254732b43c Replace synchronized with ConcurrentHashMap in Http2StreamChannelBootstrap (#9848)
Motivation:

97361fa2c8 replace synchronized with ConcurrentHashMap in *Bootstrap classes but missed to do the same for the Http2 variant.

Modifications:

- Use ConcurrentHashMap
- Simplify code in *Bootstrap classes

Result:

Less contention
2019-12-06 12:02:57 +01:00
Norman Maurer
8cfd71c354 Call ctx.flush() when onStreamClosed(...) produces a window update frame (#9818)
Motivation:

We use the onStreamClosed(...) callback to return unconsumed bytes back to the window of the connection when needed. When this happens we will write a window update frame but not automatically call ctx.flush(). As the user has no insight into this it could in the worst case result in a "deadlock" as the frame is never written out ot the socket.

Modifications:

- If onStreamClosed(...) produces a window update frame call ctx.flush()
- Add unit test

Result:

No stales possible due unflushed window update frames produced by onStreamClosed(...) when not all bytes were consumed before the stream was closed
2019-11-28 11:11:49 +01:00
Norman Maurer
16c88fd9ed Use EmbeddedChannel.finishAndReleaseAll() to remove boiler-plate code (#9824)
Motivation:

We can make use of EmbeddedChannel.finishAndReleaseAll() and so remove some code

Modifications:

Use finishAndReleaseAll()

Result:

Less code to maintain
2019-11-28 11:04:09 +01:00
Norman Maurer
371c431b6b Don't send window update frame for unconsumed bytes when stream is already closed (#9816)
Motivation:

At the moment we send a window update frame for the connection + stream when a stream is closed and there are unconsumed bytes left. While we need to do this for the connection it makes no sense to write a window update frame for the stream itself as it is already closed

Modifications:

- Don't write the window update frame for the stream when the stream is closed
- Add unit test

Result:

Don't write the window frame for closed streams
2019-11-28 09:06:44 +01:00
Norman Maurer
621af5ce27 Correctly guard against multiple RST frames for the same stream (#9811)
Motivation:

Http2ConnectionHandler tries to guard against sending multiple RST frames for the same stream. Unfortunally the code is not 100 % correct as it only updates the state after it calls write. This may lead to the situation of have an extra RST frame slip through if the second write for the RST frame is done from a listener that is attached to the promise.

Modifications:

- Update state before calling write
- Add unit test

Result:

Only ever send one RST frame per stream
2019-11-27 06:54:17 +01:00
Bryce Anderson
3ebc946c35 Log expected STREAM_CLOSED exceptions for already closed streams at DEBUG level (#9798)
Motivation:

There is an intrinsic race between a local session resetting a stream
and the peer no longer sending any frames. This can result in the
session receiving frames for a stream that the local peer no longer
tracks. This results in a StreamException being thrown which triggers a
RST_STREAM frame, which is a good thing, but also logging at level WARN,
which is noisy for an expected and benign condition.

Modification:

Change the log level to DEBUG when logging stream errors with code
STREAM_CLOSED. All others are more interesting and will continue to be
logged at level WARN.

Additionally, it was found that DATA frames for streams that could not
have existed only resulted in a StreamException when the spec is clear
that such a situation should be fatal to the connection, resulting in a
GOAWAY(PROTOCOL_ERROR).

Fixes #8025.
2019-11-25 09:02:55 +01:00
monkey-mas
fc60fa4426 Add test to check Connection-Specific headers are removed in HTTP/2 (by HttpConversionUtil.toHttp2Headers) (#9766)
Motivation:
To avoid regression regarding connection-specific headers[1], we should add a test.
[1] https://tools.ietf.org/html/rfc7540#section-8.1.2.2

Modification:
Add test that checks the following headers are removed.
- Connection
- Host
- Keep-Alive
- Proxy-Connection
- Transfer-Encoding
- Upgrade

Result:
There's no functional change.
2019-11-08 10:15:19 +01:00
zhangheng
86a533576f Remove padding when writing CONTINUATION frame (#9752)
Motivation:

Padding was removed from CONTINUATION frame in http2-spec, as showed in [PR](https://github.com/http2/http2-spec/pull/510). We should follow it.

Modifications:

- Remove padding when writing CONTINUATION frame in DefaultHttp2FrameWriter
- Add a unit test for writing large header with padding

Result:

More spec-compliant
2019-11-05 15:21:10 +01:00
monkey-mas
83b35db2e5 Remove unnecessary line in Http2ClientUpgradeCodec (#9750)
Motivation:
To clean up code.

Modification:
Remove unnecessary line.

Result:
There's no functional change.
2019-11-04 11:20:41 +01:00
Nick Hill
7df012884f Rename SimpleChannelInboundHandler.channelRead0() to messageReceived() (#8819)
Motivation

Per javadoc in 4.1.x SimpleChannelInboundHandler:

"Please keep in mind that channelRead0(ChannelHandlerContext, I) will be
renamed to messageReceived(ChannelHandlerContext, I) in 5.0."

Modifications

Rename aforementioned method and all references/overrides.

Result

Method is renamed.
2019-11-01 07:23:07 +01:00
Norman Maurer
63729a310f Fix Http2Headers.method(...) javadocs (#9718)
Motivation:

The javadocs of Http2Headers.method(...) are incorrect, we should fix these.

Modifications:

Correct javadocs

Result:

Fixes https://github.com/netty/netty/issues/8068.
2019-10-29 19:51:53 +01:00
Norman Maurer
34fa8cbc3d Fix typo in test which did introduce a failing test after ffc3b2da72 2019-10-28 09:26:51 +01:00
Julien Hoarau
dd5d4887ed Validate pseudo and conditional HTTP/2 headers (#8619)
Motivation:

Netty HTTP/2 implementation is not 100% compliant to the spec. This
commit improves the compliance regarding headers validation,
in particular pseudo-headers and connection ones.

According to the spec:
   All HTTP/2 requests MUST include exactly one valid value for the
   ":method", ":scheme", and ":path" pseudo-header fields, unless it is
   a CONNECT request (Section 8.3).  An HTTP request that omits
   mandatory pseudo-header fields is malformed (Section 8.1.2.6).

Modifications:

- Introduce Http2HeadersValidator class capable of validating HTTP/2
headers
- Invoke validation from DefaultHttp2ConnectionDecoder#onHeadersRead
- Modify tests to use valid headers when required
- Modify HttpConversionUtil#toHttp2Headers to not add :scheme and
:path header on CONNECT method in order to conform to the spec

Result:

- Initial requests without :method, :path, :scheme will fail
- Initial requests with multiple values for :method, :path, :scheme
will fail
- Initial requests with an empty :path fail
- Requests with connection-specific header field will fail
- Requests with TE header different than "trailers" will fail
-
- Fixes 8.1.2.2 tests from h2spec #5761
- Fixes 8.1.2.3 tests from h2spec #5761
2019-10-27 16:20:18 +01:00
Carl Mastrangelo
03ad809a3b Use fast HPACK comparisons when not checking sensitive headers (#9259)
Motivation:
Constant time comparison functions are used to compare HTTP/2 header
values, even if they are not sensitive.

Modification:
After checking for sensitivity, use fast comparison.

Result: Faster HPACK table reads/writes
2019-10-24 08:43:40 +02:00
Norman Maurer
d71661ff88 Correctly propagate failures while update the flow-controller to the … (#9664)
Motivation:

We may fail to update the flow-controller and in this case need to notify the stream channel and close it.

Modifications:

Attach a future to the write of the update frame and in case of a failure propagate it to the channel and close it

Result:

Fixes https://github.com/netty/netty/issues/9663
2019-10-22 14:40:43 +02:00
Norman Maurer
c25eec0658 Try to reduce GC produced while writing headers (#9682)
Motivation:

bbc34d0eda introduced correct handling of "in process" setup of streams but there is some room for improvements. Often the writeHeaders(...) is completed directly which means there is not need to create the extra listener object.

Modifications:

- Only create the listener if we really need too.

Result:

Less GC
2019-10-17 10:19:28 -07:00
Matthew Miller
d36a2a65e1 HTTP/2: Prevent memory leak when trying to create new streams on a connection that received a GOAWAY. (#9674)
Motivation:

In https://github.com/netty/netty/issues/8692, `Http2FrameCodec` was
updated to keep track of all "being initialized" streams, allocating
memory before initialization begins, and releasing memory after
initialization completes successfully.

In some instances where stream initialization fails (e.g. because this
connection has received a GOAWAY frame), this memory is never released.

Modifications:

This change updates the `Http2FrameCodec` to use a separate promise
for monitoring the success of sending HTTP2 headers. When sending of
headers fails, we now make sure to release memory allocated for stream
initialization.

Result:

After this change, failures in writing HTTP2 Headers (e.g. because this
connection has received a GOAWAY frame) will no longer leak memory.
2019-10-16 19:42:35 -07:00
康智冬
1c69448e2e Fix typos in javadocs (#9527)
Motivation:

We should have correct docs without typos

Modification:

Fix typos and spelling

Result:

More correct docs
2019-10-09 15:25:41 +02:00
Norman Maurer
3a48010c42 Ensure we finish setup mock before we use it in Http2ConnectionRoundtripTest.headersWriteForPeerStreamWhichWasResetShouldNotGoAway (#9645)
Motivation:

We did dispatch the client code before we did finish setup the mock and so may end up with org.mockito.exceptions.misusing.UnfinishedStubbingException if the connect happens quickly enough.

See https://ci.netty.io/job/netty-centos6-java8-prb/1637/testReport/junit/io.netty.handler.codec.http2/Http2ConnectionRoundtripTest/headersWriteForPeerStreamWhichWasResetShouldNotGoAway/

Modifications:

First finish setup the mock and the dispatch.

Result:

Fix flacky test
2019-10-09 07:23:47 +02:00
Carl Mastrangelo
3f6af1fc1d Remember to return writability events to flow controller in HTTP2 Multiplexer (#9642)
Motivation:

Http2MultiplexCodec extends Http2FrameCodec extends Http2ConnectionHandler.  It appears  Http2MultiplexCodec overrode the channelWritabilityChanged method, which prevented the flow controller from becoming active.  In the case the parent channel becomes unwritable, and then later becomes writable, it needs to indicate that the child channels can still write data.   This is slightly confusing, because the child channels may still themselves be unwritable, but should still drain their data to the parent channel.

Modification:

Still propagate writability changes to the HTTP/2 flow controller

Result:

Fixes https://github.com/netty/netty/issues/9636
2019-10-08 16:41:22 +02:00
Pete Woods
540798b970 Add io.netty.handler.codec.http2.Http2ConnectionHandler for runtime GraalVM compilation (#9621)
Motivation:

Native image compilation is failing without extra flags:

```
Warning: Aborting stand-alone image build. No instances of io.netty.buffer.UnpooledHeapByteBuf are allowed in the image heap as this class should be initialized at image runtime. Object has been initialized by the io.netty.handler.codec.http2.Http2ConnectionHandler class initializer with a trace: 
 	at io.netty.buffer.Unpooled.wrappedBuffer(Unpooled.java:157)
	at io.netty.handler.codec.http2.Http2ConnectionHandler.<clinit>(Http2ConnectionHandler.java:74)
.  To fix the issue mark io.netty.buffer.UnpooledHeapByteBuf for build-time initialization with --initialize-at-build-time=io.netty.buffer.UnpooledHeapByteBuf or use the the information from the trace to find the culprit and --initialize-at-run-time=<culprit> to prevent its instantiation.

Detailed message:
Trace: 	object io.netty.buffer.ReadOnlyByteBuf
	object io.netty.buffer.UnreleasableByteBuf
	method io.netty.handler.codec.http2.Http2ConnectionHandler.access$500()
Call path from entry point to io.netty.handler.codec.http2.Http2ConnectionHandler.access$500(): 
	at io.netty.handler.codec.http2.Http2ConnectionHandler.access$500(Http2ConnectionHandler.java:66)
	at io.netty.handler.codec.http2.Http2ConnectionHandler$PrefaceDecoder.readClientPrefaceString(Http2ConnectionHandler.java:299)
	at io.netty.handler.codec.http2.Http2ConnectionHandler$PrefaceDecoder.decode(Http2ConnectionHandler.java:239)
	at io.netty.handler.codec.http2.Http2ConnectionHandler.decode(Http2ConnectionHandler.java:438)
	at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:505)
	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:444)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:283)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
	at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:56)
	at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:365)
	at com.oracle.svm.core.jdk.RuntimeSupport.executeHooks(RuntimeSupport.java:144)
	at com.oracle.svm.core.jdk.RuntimeSupport.executeStartupHooks(RuntimeSupport.java:89)
	at com.oracle.svm.core.JavaMainWrapper.runCore(JavaMainWrapper.java:143)
	at com.oracle.svm.core.JavaMainWrapper.run(JavaMainWrapper.java:186)
	at com.oracle.svm.core.code.IsolateEnterStub.JavaMainWrapper_run_5087f5482cc9a6abc971913ece43acb471d2631b(generated:0)
```

Modification:

Add `io.netty.handler.codec.http2.Http2ConnectionHandler` for runtime compilation, as the buffer library's `io.netty.buffer.UnpooledHeapByteBuf` is also marked for runtime.

Result:

Native image compilation works again.
2019-10-07 11:02:13 +04:00
Nikolay Fedorovskikh
873761f9bd Fix possible NPE in DefaultHttp2UnknownFrame#equals (#9625)
Motivation:
`DefaultHttp2UnknownFrame#equals` may produce NPE due to
incorrect comparison of `stream` field.

Modification:
- Fix the `stream` field compare.
- Cleanup usage of class fields: use direct access instead of getters
(because the class is final).

Result:
No NPE in `equals` method.
2019-10-07 11:00:37 +04:00
Nikolay Fedorovskikh
47040bd791 Make some inner classes static (#9624)
Motivation:
Classes `AbstractHttp2StreamChannel.Http2StreamChannelConfig`
and `DnsNameResolver.AddressedEnvelopeAdapter` may be static:
it doesn't reference its enclosing instance.

Modification:
Add `static` modifier.

Result:
Prevents a possible memory leak and uses less memory per class instance.
2019-10-07 10:13:48 +04:00
Norman Maurer
43f7a9ea2a Use configured ByteBufAllocator in InboundHttp2ToHttpAdapter (#9611)
Motivation:

At the moment we use Unpooled.buffer(...) in InboundHttp2ToHttpAdapter when we need to do a copy of the message. We should better use the configured ByteBufAllocator for the Channel

Modifications:

Change internal interface to also take the ByteBufAllocator as argument and use it when we need to allocate a ByteBuf.

Result:

Use the "correct" ByteBufAllocator in InboundHttp2ToHttpAdapter in all cases
2019-09-26 22:12:02 +02:00
Norman Maurer
8f23e27bd2 We should only disable releasing of the message once writeData(...) was called successfully (#9610)
Motivation:

At the moment we set release to false before we call writeData(...). This could let to the sitatuation that we will miss to release the message if writeData(...) throws. We should set release to false after we called writeData(...) to ensure the ownership of the buffer is correctly transferred.

Modifications:

- Set release to false after writeData(...) was successfully called only

Result:

No possibility for a buffer leak
2019-09-26 22:00:49 +02:00
Norman Maurer
6f584ec9de Correctly take Http2FrameCodecBuilder.isValidateHeaders() into account when creating a Http2FrameCodec from an existing Http2FrameWriter. (#9600)
Motivation:

We did miss to take Http2FrameCodecBuilder.isValidateHeaders() into account when a Http2FrameWriter was set on the builder and always assumed validation should be enabled.

Modifications:

Remove hardcode value and use configured value

Result:

Http2FrameCodecBuilder.isValidateHeaders() is respected in all cases
2019-09-26 21:57:21 +02:00
Pete Woods
92809db288 Fix GraalVM native image build error (#9593)
Motivation:

Error: Class that is marked for delaying initialization to run time got initialized during image building: io.netty.handler.codec.http2.Http2CodecUtil. Try marking this class for build-time initialization with --initialize-at-build-time=io.netty.handler.codec.http2.Http2CodecUtil
Error: Use -H:+ReportExceptionStackTraces to print stacktrace of underlying exception
Error: Image build request failed with exit status 1
Modification:

After debugging, it seems the culprit is io.netty.handler.codec.http2.Http2ClientUpgradeCodec, which also needs runtime initialisation.

Result:

Fixes #micronaut-projects/micronaut-grpc#8
2019-09-23 14:42:10 +02:00
Norman Maurer
3099bbcc13
Change semantics of EmbeddedChannel to match other transports more closely. (#9529)
Motiviation:

EmbeddedChannel currently is quite differently in terms of semantics to other Channel implementations. We should better change it to be more closely aligned and so have the testing code be more robust.

Modifications:

- Change EmbeddedEventLoop.inEventLoop() to only return true if we currenlty run pending / scheduled tasks
- Change EmbeddedEventLoop.execute(...) to automatically process pending tasks if not already doing so
- Adjust a few tests for the new semantics (which is closer to other Channel implementations)

Result:

EmbeddedChannel works more like other Channel implementations
2019-09-04 12:00:06 +02:00
Norman Maurer
48634f1466
Protect ChannelHandler from reentrancee issues (#9358)
Motivation:

At the moment it is quite easy to hit reentrance issues when you have multiple handlers in the pipeline and each of the handlers does not correctly protect against these. To make it easier for the user we should try to protect from these. The issue is usually if and inbound event will trigger and outbound event and this outbound event then against triggeres an inbound event. This may result in having methods in a ChannelHandler re-enter some method and so state can be corrupted or messages be re-ordered.

Modifications:

- Keep track of inbound / outbound operations in DefaultChannelHandlerContext and if reentrancy is detected break it by scheduling the action on the EventLoop. This will then be picked up once the method returns and so the reentrancy is broken up.
- Adjust tests which made strange assumptions about execution order

Result:

No more reentrancy of handlers possible.
2019-09-03 10:28:08 +02:00
Bryce Anderson
bf086c1aa3 Support cancellation in the Http2StreamChannelBootstrap (#9519)
Motivation:

Right now you can cancel the Future returned by
`Http2StreamChannelBootstrap.open()` and that will race with the
registration of the stream channel with the event loop, potentially
culminating in an `IllegalStateException` and potential resource leak.

Modification:

Ensure that the returned promise is uncancellable.

Result:

Should no longer see `IllegalStateException`s.
2019-08-27 20:42:48 +02:00
nizarm
130522be84 Correctly handle client side http2 upgrades when Http2FrameCodec …(9495) (#9501)
Motivation:

In the release (4.1.37) we introduced Http2MultiplexHandler as a
replacement of Http2MultiplexCodec. This did split the frame parsing from
the multiplexing to allow a more flexible way to handle frames and to make
the code cleaner. Unfortunally we did miss to special handle this in
Http2ClientUpgradeCodec and so did not correctly add Http2MultiplexHandler
to the pipeline before calling Http2FrameCodec.onHttpClientUpgrade(...).
This did lead to the situation that we did not correctly receive the event
on the Http2MultiplexHandler and so did not correctly created the
Http2StreamChannel for the upgrade stream. Because of this we ended up
with an NPE if a frame was dispatched to the upgrade stream later on.

Modifications:

- Correctly add Http2MultiplexHandler to the pipeline before calling Http2FrameCodec.onHttpClientUpgrade(...)

Result:

Fixes #9495.
2019-08-23 19:26:30 +02:00
Idel Pivnitskiy
05bbecff49 Update links to the latest HTTP/2 specifications (#9493)
Motivation:

Some of the links in javadoc point to the obsolete drafts of HTTP/2
specifications. We should point them to the latest RFC 7540 or 7541.

Modifications:

Update links from `draft-ietf-httpbis-*` to the `rfc7540` and `rfc7541`.

Result:

Correct links in javadoc.
2019-08-22 13:59:44 +02:00
Norman Maurer
7a375a492c HTTP2: Update local flow-controller on Channel.read() if needed (#9400)
Motivation:

We should better update the flow-controller on Channel.read() to reduce overhead and memory overhead.

See https://github.com/netty/netty/pull/9390#issuecomment-513008269

Modifications:

Move updateLocalWindowIfNeeded() to doBeginRead()

Result:

Reduce memory overhead
2019-08-16 09:28:21 +02:00
Norman Maurer
e4d9a17ad9 Http2EmptyDataFrameConnectionDecoder.frameListener() should return unwrapped Http2FrameListener (#9467)
Motivation:

As we decorate the Http2FrameListener under the covers we should ensure the user can still access the original Http2FrameListener.

Modifications:

- Unwrap the Http2FrameListener in frameListener()
- Add unit test

Result:

Less suprises for users.
2019-08-16 08:16:46 +02:00
Norman Maurer
9727cd8452 Fix ByteBuf leak in Http2ControlFrameLimitEncoderTest (#9466)
Motivation:

We recently introduced Http2ControlFrameLimitEncoderTest which did not correctly notify the goAway promises and so leaked buffers.

Modifications:

Correctly notify all promises and so release the debug data.

Result:

Fixes leak in HTTP2 test
2019-08-14 13:28:36 +02:00
Norman Maurer
6283a78e4f HTTP2: Guard against empty DATA frames (without end_of_stream flag) set (#9461)
Motivation:

It is possible for a remote peer to flood the server / client with empty DATA frames (without end_of_stream flag) set and so cause high CPU usage without the possibility to ever hit a limit. We need to guard against this.

See CVE-2019-9518

Modifications:

- Add a new config option to AbstractHttp2ConnectionBuilder and sub-classes which allows to set the max number of consecutive empty DATA frames (without end_of_stream flag). After this limit is hit we will close the connection. A limit of 10 is used by default.
- Add unit tests

Result:

Guards against CVE-2019-9518
2019-08-14 10:02:32 +02:00
Norman Maurer
c6c679597f HTTP2: Add protection against remote control frames that are triggered by a remote peer (#9460)
Motivation:

Due how http2 spec is defined it is possible by a remote peer to flood us with frames that will trigger control frames as response, the problem here is that the remote peer can also just stop reading these (while still produce more of these) and so may drive us to the pointer where we either run out of memory or burn all CPU. To protect against this we need to implement some kind of limit that will tear down connections that cause the above mentioned situation.

See CVE-2019-9512 / CVE-2019-9514 / CVE-2019-9515

Modifications:

- Add Http2ControlFrameLimitEncoder which limits the number of queued control frames that were caused because of the remote peer.
- Allow to insert ths Http2ControlFrameLimitEncoder by setting AbstractHttp2ConnectionBuilder.encoderEnforceMaxQueuedControlFrames(...) to a number higher then 0. The default is 10000 which provides some protection by default but will hopefully not cause too many false-positives.
- Add unit tests

Result:

Protect against DDOS due control frames. Fixes CVE-2019-9512 / CVE-2019-9514 / CVE-2019-9515 .
2019-08-14 10:02:24 +02:00
Norman Maurer
02408b306c Delay Http2ConnectionPrefaceAndSettingsFrameWrittenEvent by one EventLoop tick when using the Http2FrameCodec (#9442)
Motivation:

We should delay the firing of the Http2ConnectionPrefaceAndSettingsFrameWrittenEvent by one EventLoop tick when using the Http2FrameCodec to ensure all handlers are added to the pipeline before the event is passed through it.

This is needed to workaround a race that could happen when the preface is send in handlerAdded(...) but a later handler wants to act on the event.

Modifications:

Offload firing of the event to the EventExecutor.

Result:

Fixes https://github.com/netty/netty/issues/9432.
2019-08-13 10:50:40 +02:00
Norman Maurer
d6c05e9b31 HTTP/2: Ensure newStream() is called only once per connection upgrade and the correct handler is used (#9396)
Motivation:

306299323c introduced some code change to move the responsibility of creating the stream for the upgrade to Http2FrameCodec. Unfortunaly this lead to the situation of having newStream().setStreamAndProperty(...) be called twice. Because of this we only ever saw the channelActive(...) on Http2StreamChannel but no other events as the mapping was replaced on the second newStream().setStreamAndProperty(...) call.

Beside this we also did not use the correct handler for the upgrade stream in some cases

Modifications:

- Just remove the Http2FrameCodec.onHttpClientUpgrade() method and so let the base class handle all of it. The stream is created correctly as part of the ConnectionListener implementation of Http2FrameCodec already.
- Consolidate logic of creating stream channels
- Adjust unit test to capture the bug

Result:

Fixes https://github.com/netty/netty/issues/9395
2019-07-23 21:13:33 +02:00
Norman Maurer
a677e62dd0 HTTP/2 multiplex: Correctly process buffered inbound data even if autoRead is false (#9389)
Motivation:

When using the HTTP/2 multiplex implementation we need to ensure we correctly drain the buffered inbound data even if the RecvByteBufallocator.Handle tells us to stop reading in between.

Modifications:

Correctly loop through the buffered inbound data until the user does stop to request from it.

Result:

Fixes https://github.com/netty/netty/issues/9387.

Co-authored-by: Bryce Anderson <banderson@twitter.com>
2019-07-21 20:58:50 +02:00
Norman Maurer
f6005c5b2d Reuse Http2FrameStreamEvent instances to reduce GC pressure (#9392)
Motivation:

We can easily reuse the Http2FrameStreamEvent instances and so reduce GC pressure as there may be multiple events per streams over the life-time.

Modifications:

Reuse instances

Result:

Less allocations
2019-07-21 20:37:28 +02:00
Norman Maurer
82ac20bec0 Cache the ChannelHandlerContext used in Http2StreamChannelBootstrap (#9382)
Motivation:

At the moment we lookup the ChannelHandlerContext used in Http2StreamChannelBootstrap each time the open(...) method is invoked. This is not needed and we can just cache it for later usage.

Modifications:

Cache ChannelHandlerContext in volatile field.

Result:

Speed up open(...) method implementation when called multiple times
2019-07-18 10:29:14 +02:00
Bryce Anderson
8e370fbbcd Fix an NPE in AbstractHttp2StreamChannel (#9379)
Motivation:

If a read triggers a AbstractHttp2StreamChannel to close we can
get an NPE in the read loop.

Modifications:

Make sure that the inboundBuffer isn't null before attempting to
continue the loop.

Result:

No NPE.
Fixes #9337
2019-07-17 20:13:09 +02:00
Norman Maurer
7e818352e2 Move responsibility for creating upgrade stream to Http2FrameCodec (#9360)
Motivation:

The Http2FrameCodec should be responsible to create the upgrade stream.

Modifications:

Move code to create stream to Http2FrameCodec

Result:

More correct responsibility
2019-07-16 13:27:51 +02:00