Motivation:
DefaultHttp2ConnectionDecoder writes a ACK when receiving a ping frame and sends the same data buffer it received. The data buffer is also passed to the listener, but the indexes are shared between the send and the listener. We should ensure the indexes are independent for these two operations.
Modifications:
- Call slice on the buffer that is being sent
Result:
Listener now has access to a buffer that will not appear to be already consumed.
Motivation:
JDK SslEngine supports renegotion, so we should at least support it server-side with OpenSslEngine as well.
That said OpenSsl does not support sending messages asynchronly while the renegotiation is still in progress, so the application need to ensure there are not writes going on while the renegotiation takes place. See also https://rt.openssl.org/Ticket/Display.html?id=1019 .
Modifications:
- Add support for renegotiation when OpenSslEngine is used in server mode
- Add unit tests.
- Upgrade to netty-tcnative 1.1.33.Fork9
Result:
Better compatibility with the JDK SSLEngine implementation.
Motivation:
Using the builder pattern for Http2ConnectionHandler (and subclasses) would be advantageous for the following reasons:
1. Provides the consistent construction afforded by the builder pattern for 'optional' arguments. Users can specify these options 1 time in the builder and then re-use the builder after this.
2. Enforces that the Http2ConnectionHandler's internals (decoder Http2FrameListener) are initialized after construction.
Modifications:
- Add an extensible builder which can be used to build Http2ConnectionHandler objects
- Update classes which inherit from Http2ConnectionHandler
Result:
It is easier to specify options and construct Http2ConnectionHandler objects.
Motivation:
We missed to correctly update the internal handshake state on beginHandshake() if we was able to finish the handshake directly. Also we not handled the case correctly when beginHandshake() was called after the first handshake was finished, which incorrectly throw an Error.
Modifications:
- Correctly set internal handshake state in all cases
- Correctly handle beginHandshake() once first handshake was finished.
Result:
Correctly handle OpenSslEngine.beginHandshake()
Motivation:
It is often the case that implementations of Http2FrameListener will want to send responses when data is read. The Http2FrameListener needs access to the Http2ConnectionHandler (or the encoder contained within) to be able to send responses. However the Http2ConnectionHandler requires a Http2FrameListener instance to be passed in during construction time. This creates a cyclic dependency which can make it difficult to cleanly accomplish this relationship.
Modifications:
- Add Http2ConnectionDecoder.frameListener(..) method to set the frame listener. This will allow the listener to be set after construction.
Result:
Classes which inherit from Http2ConnectionHandler can more cleanly set the Http2FrameListener.
Motivation:
We should fail the build on warnings in the JNI/c code.
Modifications:
- Add GCC flag to fail build on warnings.
- Fix warnings (which also fixed a bug when using splice with offsets).
Result:
Better code quality.
Motivation:
A new version of netty-tcnative was released with some important bug-fixes.
Modifications:
Bump up version.
Result:
Using latest netty-tcnative version
Motivation:
We should prevent to add/set DefaultHttpHeaders to itself to prevent unexpected side-effects.
Modifications:
Throw IllegalArgumentException if user tries to pass the same instance to set/add.
Result:
No surprising side-effects.
Motivation:
The STOMP decoder used to fail when parsing a frame with no content-length
and a body split across multiple packets.
Modifications:
Support contentLength of -1 (indicating indeterminate length) and added a
check to getContentLength. Moved the NUL byte searching from the
readHeaders() method out to the main decoder loop.
Result:
A STOMP frame can be properly parsed even if it's missing the
content-length header and the NUL byte is in a later packet.
Motivation:
If a remote peer writes fast enough it may take a long time to have fireChannelReadComplete(...) triggered. Because of this we need to take special care and ensure we try to discard some bytes if channelRead(...) is called to often in ByteToMessageDecoder.
Modifications:
- Add ByteToMessageDecoder.setDiscardAfterReads(...) which allows to set the number of reads after which we try to discard the read bytes
- Use default value of 16 for max reads.
Result:
No risk of OOME.
Motivation:
For implementations that want to manage flow control down to the stream level it is useful to be notified when stream writability changes.
Modifications:
- Add writabilityChanged to Http2RemoteFlowController.Listener
- Add isWritable to Http2RemoteFlowController
Result:
The Http2RemoteFlowController provides notification when writability of a stream changes.
Motivation:
The DefaultHttp2RemoteFlowController has become very large and is getting difficult to understand and maintain. It is also desirable for some applications to be able to disable the priority algorithm altogether for performance reasons.
Modifications:
Abstract the stream byte assignment logic (renamed allocation->assignment for clarity) behind an interface `StreamByteAssigner` with a single implementation `PriorityStreamByteAssigner`.
Result:
Goes some way towards supporting #4246
Motivation:
We should provide a better way to set session keys that not use the deprecated method of netty-tcnative.
Modifications:
- Add OpenSslSessionTicketKey
- Expose new method on OpenSslServerContext and deprecate the old method.
Result:
Easier to use and can remove the deprecated method later on.
Motivation:
DefaultHttp2RemoteFlowController's allocation algorithm may not allocate all bytes that are available in the connection window. If the 'fair share' based upon weight is not fully used by sibling nodes it was not correctly re-distributed to other sibilings which may be able to utilize part / all of that share.
Modifications:
- Add a unit test which demonstrates the issue.
- Modify the allocation algorithm to ensure all available bytes are allocated.
Result:
Fixes https://github.com/netty/netty/issues/4266
Motivation:
We should call shutdown(...) on the socket before closing the filedescriptor to ensure it is closed gracefully.
Modifications:
Call shutdown(...) before close.
Result:
Sockets are gracefully shutdown when using native transport.
Motivation:
We need to ensure all markers are reset when doing an allocation via the PooledByteBufAllocator. This was not the always the case.
Modifications:
Move all logic that needs to get executed when reuse a PooledByteBuf into one place and call it.
Result:
Correct behavior
Motivation:
Some DNS servers in DnsNameResolverTest are outdated and some of them
returns NoError for non-existent domains.
Modifications:
- Update the DNS server list from http://meo.ws/dnsrec.php again
- Update the web-scraper script
Result:
DnsNameResolverTest.testNegativeTtl() should not fail anymore.
Motivation:
PR https://github.com/netty/netty/pull/4257 introduced paramters and didn't use them.
Modifications:
- Use the new paramters
Result:
No warnings and correct behavior
Motivation:
OpenSslEngine.unwrap(...) / wrap(...) must return HandhsakeStatus.FINISHED if an unwrap or wrap finishes a handshake to behave like descripted in the SSLEngine docs.
Modifications:
- Ensure we return HandshakeStatus.FINISHED
Result:
Behave correctly.
Motivation:
On ubuntu, InetAddress.getLocalHost() will return 127.0.1.1 this causes some tests to fail.
NetUtil.LOCALHOST4 is more portable.
Modifications:
Made changes in EpollSocketTcpMd5Test to make test passing on ubuntu.
Result:
EpollSocketTcpMd5Test now also passes on ubuntu.
Motiviation:
The http2 spec https://tools.ietf.org/html/rfc7540#section-8.1.2.3 states that the :authority header should be copied into the HOST header when converting from HTTP/2 to HTTP/1.x. We currently have an extension header to preserve the authority.
Modifications:
- Remove AUTHORITY extension header
- HTTP/2 :authority should map to HOST header when converting to HTTP/1.x.
Result:
More spec compliant.
Motivation:
Users may want to control the valid dates for SelfSignedCertificate.
Modifications:
- Allow NOT_BEFORE and NOT_AFTER to be controlled via java system properties.
Result:
Fixes https://github.com/netty/netty/issues/3978
Motivation:
Buffer leak in StreamBufferingEncoderTest
Modifications:
- Make sure buffers are released in StreamBufferingEncoderTest
Result:
Fixes https://github.com/netty/netty/issues/4230
Motivation:
Http2LifecycleManager.onException takes a Throwable as a paramter and not an Exception. There are also onConnectionError and onStreamError methods in the codec. We should rename this method to onError for consistency and clarity.
Modifications:
- Rename Http2LifecycleManager.onException to Http2LifecycleManager.onError
Result:
More consistent and clarified interface.
Motivation:
The HashingStrategy for DefaultStompHeaders was using the java .equals() method which would fail to compare String, AsciiString, and other CharSequence objects as equal.
Modification:
- Use AsciiString.CASE_SENSITIVE_HASHER for DefaultStompHeaders
Result:
DefaultStompHeaders work with all CharSequence objects.
Fixes https://github.com/netty/netty/issues/4247
Motivation:
DefaultHttp2RemoteFlowController attempts to write as many bytes as possible to transition the channel to not writable, and then relies on notification of channelWritabilityChange to continue writing. However the amount of bytes written by DefaultHttp2RemoteFlowController may not be the same number of bytes that is actually written to the channel due to other ChannelHandlers (SslHandler, compression, etc...) in the pipeline. This means there is a potential for the DefaultHttp2RemoteFlowController to be waiting for a channel writaiblity change event that will never come, and thus not write all queued data.
Modifications:
- DefaultHttp2RemoteFlowController should write pending bytes until there are no more, or until the channel is not writable.
Result:
DefaultHttp2RemoteFlowController will write all pending data.
Fixes https://github.com/netty/netty/issues/4242
Motivation:
The last os-maven-plugin had a bug that sometimes missed to correctly detect fedora based linux.
Modifications:
Upgrade to 1.4.1
Result:
Correctly detect on all fedora based linux.
Motivation:
RC4 is not supported by default in more recent java versions as RC4 is considered insecure. We should not use it in tests as these test will fail on more recent java version.
Modifications:
Use SSL_RSA_WITH_3DES_EDE_CBC_SHA for test.
Result:
Non failing test on more recent java versions.
Motivation:
We currently set the flow controller ChannelHandlerContexts to null when the channel becomes inactive. This is bad :)
Modifications:
Just remove that code in Http2ConnectionHandler
Result:
Fixes#4240
Motivation:
HttpConversionUtil.toHttp2Headers does not convert urlencoded uri to http2 path properly.
Modifications:
Use getRawPath(), getRawQuery(), getRawFragment() in java.net.URI when converts to http2 path
Result:
HttpConversionUtil.toHttp2Headers does not urldecode uri unproperly.
Motivation:
To simplify the use of client auth, we need to add it to the SslContextBuilder.
Modifications:
Added a ClientAuth enum and plumbed it through the builder, down into the contexts/engines.
Result:
Client auth can be configured when building an SslContext.
Motivation:
The latest netty-tcnative fixes a bug in determining the version of the runtime openssl lib. It also publishes an artificact with the classifier linux-<arch>-fedora for fedora-based systems.
Modifications:
Modified the build files to use the "-fedora" classifier when appropriate for tcnative. Care is taken, however, to not change the classifier for the native epoll transport.
Result:
Netty is updated the the new shiny netty-tcnative.
Motivation:
Http2CodecUtils has some static variables which are defined as Strings instead of CharSequence. One of these defines is used as a header name and should be AsciiString.
Modifications:
- Change the String defines in Http2CodecUtils to CharSequence
Result:
Types are more consistently using CharSequence and adding the upgrade header will require less work.
Motivation:
The DefaultHttp2Headers code is throwing a IllegalArgumentException if an invalid character is detected. This is being ignored by the HTTP/2 codec instead of generating a GOAWAY.
Modifications:
- Throw a Http2Exception of type PROTOCOL_ERROR in accordance with https://tools.ietf.org/html/rfc7540#section-8.1.2.6
- Update examples which were building invalid headers
Result:
More compliant with https://tools.ietf.org/html/rfc7540#section-8.1.2.6
Motiviation:
We need to ensure the actual close to the transport takes place before the promsie of the write is notified that triggered it. This is needed as otherwise Channel.isActive(), isOpen() and isWritable() may return true even if the Channel should be closed already.
Modifications:
- Ensure the close takes place first
Result:
ChannelFutureListener will see the correct state of the Channel.
Motivation:
According to the SPDY spec https://www.chromium.org/spdy/spdy-protocol/spdy-protocol-draft3-1#TOC-3.2.1-Request header names must be lowercase. Our predefined SPDY extension headers are not lowercase.
Modifications
- SpdyHttpHeaders should define header names in lower case
Result:
Compliant with SPDY spec, and header validation code does not detect errors for our own header names.
Motivation:
The HTTP/2 spec states that the ping frame length must be 8 and is otherwise an error https://tools.ietf.org/html/rfc7540#section-6.7. The DefaultHttp2FrameReader enforces this, but the DefaultHttp2FrameWriter allows invalid frames to be written. We should not allow invalid ping frames to be written to the network.
Modifications:
- DefaultHttp2FrameWriter checks the frame size to be 8, or throws an exception
Result:
Fixes https://github.com/netty/netty/issues/3721
Motivation:
Currently there is a HttpConversionUtil.addHttp2ToHttpHeaders which requires a FullHttpMessage, but this may not always be available. There is no interface that can be used with just Http2Headers and HttpHeaders.
Modifications:
- add an overload for HttpConversionUtil.addHttp2ToHttpHeaders which does not take FullHttpMessage
Result:
An overload for HttpConversionUtil.addHttp2ToHttpHeaders exists which does not require FullHttpMessage.
Motivation:
writeBytes(...) missed to set EPOLLOUT flag when not all bytes were written. This could lead to have the EpollEventLoop not try to flush the remaining bytes once the socket becomes writable again.
Modifications:
- Move setting EPOLLOUT flag logic to one point so we are sure we always do it.
- Move OP_WRITE flag logic to one point as well.
Result:
Correctly try to write pending data if socket becomes writable again.
Motivation:
When AsciiString is used we can optimize the write operation done by ByteBufUtil.writeUsAscii(...)
Modifications:
Sepcial handle AsciiString.
Result:
Faster writing of AsciiString.
Motivation:
SSLSession.getLocalCertificates() and getLocalPrincipal() was not supported when using OpenSSL, which can produce problems when switch from JDK to OpenSSL impl.
Modifications:
Implement SSLSession.getLocalCertificates() and getLocalPrincipal() for OpenSslEngine.
Result:
More consistent behaving between JDK and OpenSSL based SSLEngine.
Motivation:
As stated in the SSLSession javadocs getPeer* methods need to throw a SSLPeerUnverifiedException if peers identity has not be verified.
Modifications:
- Correctly throw SSLPeerUnverifiedException
- Add test for it.
Result:
Correctly behave like descripted in javadocs.
Motivation:
As we stored the WebSocketServerHandshaker in the ChannelHandlerContext it was always null and so no close frame was send if WebSocketServerProtocolHandler was used.
Modifications:
Store WebSocketServerHAndshaker in the Channel attributes and so make it visibile between different handlers.
Result:
Correctly send close frame.
Motivation:
https://github.com/twitter/hpack released version v1.0.1.
Modifications:
- Update pom files to pull in new version
Results:
Depend on the most recent hpack library.
Motivation:
The HTTP/2 codec has a few static buffers sent over the network which are allocated on the heap. This results in a copy operation when the buffer is sent out on the network.
Modifications:
- Ensure these static buffers are allocated using direct memory.
Result:
No copy operation necessary when writing static buffers to network.
Motivaion:
The HttpHeaders and DefaultHttpHeaders have methods deprecated due to being removed in future releases, but no replacement method to use in the current release. The deprecation policy should not be so aggressive as to not provide any non-deprecated method to use.
Modifications:
- Remove deprecated annotations and javadocs from methods which are the best we can do in terms of matching the master's api for 4.1
Result:
There should be non-deprecated methods available for HttpHeaders in 4.1.
Motivation:
The HTTP/2 header name validation was removed, and does not currently exist.
Modifications:
- Header name validation for HTTP/2 should be restored and set to the default mode of operation.
Result:
HTTP/2 header names are validated according to https://tools.ietf.org/html/rfc7540
Motivation:
TCP Fast Open allows data to be carried in the SYN and SYN-ACK packets and consumed by the receiving end during the initial connection handshake, and saves up to one full round-trip time (RTT) compared to the standard TCP, which requires a three-way handshake (3WHS) to complete before data can be exchanged. This commit enables support for TFO on server sockets.
Modifications:
Added new Integer Option TCP_FASTOPEN in EpollChannelOption.
Added getters/setters in EpollServerChannelConfig for TCP_FASTOPEN.
Added way to check if TCP_FASTOPEN is supported on server in Native.
Added setting on socket opt TCP_FASTOPEN if value is set on channel options in doBind in EpollServerSocketChannel.
Enhanced EpollSocketTestPermutation to contain a permutation for server socket containing fast open.
Result:
Users of native-epoll can set TCP_FASTOPEN on server sockets and thus leverage fast connect features of RFC7413 if client is capable of it.
Conflicts:
transport-native-epoll/src/main/java/io/netty/channel/epoll/EpollChannelOption.java
Motivation:
Related to issue #4185.
HTTP has the option to disable header validation for optimisation purposes. Introduce the same option for SPDY headers.
Also, optimise SpdyHttpEncoder by allowing the user to specify whether or not the encoder needs to convert header names to lowercase.
Modifications:
Added flags for validation and conversion.
Result:
SpdyHeader validation and conversion can be disabled.
