Motivation:
The latest netty-tcnative fixes a bug in determining the version of the runtime openssl lib. It also publishes an artificact with the classifier linux-<arch>-fedora for fedora-based systems.
Modifications:
Modified the build files to use the "-fedora" classifier when appropriate for tcnative. Care is taken, however, to not change the classifier for the native epoll transport.
Result:
Netty is updated the the new shiny netty-tcnative.
Motivation:
Http2CodecUtils has some static variables which are defined as Strings instead of CharSequence. One of these defines is used as a header name and should be AsciiString.
Modifications:
- Change the String defines in Http2CodecUtils to CharSequence
Result:
Types are more consistently using CharSequence and adding the upgrade header will require less work.
Motivation:
The DefaultHttp2Headers code is throwing a IllegalArgumentException if an invalid character is detected. This is being ignored by the HTTP/2 codec instead of generating a GOAWAY.
Modifications:
- Throw a Http2Exception of type PROTOCOL_ERROR in accordance with https://tools.ietf.org/html/rfc7540#section-8.1.2.6
- Update examples which were building invalid headers
Result:
More compliant with https://tools.ietf.org/html/rfc7540#section-8.1.2.6
Motiviation:
We need to ensure the actual close to the transport takes place before the promsie of the write is notified that triggered it. This is needed as otherwise Channel.isActive(), isOpen() and isWritable() may return true even if the Channel should be closed already.
Modifications:
- Ensure the close takes place first
Result:
ChannelFutureListener will see the correct state of the Channel.
Motivation:
According to the SPDY spec https://www.chromium.org/spdy/spdy-protocol/spdy-protocol-draft3-1#TOC-3.2.1-Request header names must be lowercase. Our predefined SPDY extension headers are not lowercase.
Modifications
- SpdyHttpHeaders should define header names in lower case
Result:
Compliant with SPDY spec, and header validation code does not detect errors for our own header names.
Motivation:
The HTTP/2 spec states that the ping frame length must be 8 and is otherwise an error https://tools.ietf.org/html/rfc7540#section-6.7. The DefaultHttp2FrameReader enforces this, but the DefaultHttp2FrameWriter allows invalid frames to be written. We should not allow invalid ping frames to be written to the network.
Modifications:
- DefaultHttp2FrameWriter checks the frame size to be 8, or throws an exception
Result:
Fixes https://github.com/netty/netty/issues/3721
Motivation:
Currently there is a HttpConversionUtil.addHttp2ToHttpHeaders which requires a FullHttpMessage, but this may not always be available. There is no interface that can be used with just Http2Headers and HttpHeaders.
Modifications:
- add an overload for HttpConversionUtil.addHttp2ToHttpHeaders which does not take FullHttpMessage
Result:
An overload for HttpConversionUtil.addHttp2ToHttpHeaders exists which does not require FullHttpMessage.
Motivation:
writeBytes(...) missed to set EPOLLOUT flag when not all bytes were written. This could lead to have the EpollEventLoop not try to flush the remaining bytes once the socket becomes writable again.
Modifications:
- Move setting EPOLLOUT flag logic to one point so we are sure we always do it.
- Move OP_WRITE flag logic to one point as well.
Result:
Correctly try to write pending data if socket becomes writable again.
Motivation:
When AsciiString is used we can optimize the write operation done by ByteBufUtil.writeUsAscii(...)
Modifications:
Sepcial handle AsciiString.
Result:
Faster writing of AsciiString.
Motivation:
SSLSession.getLocalCertificates() and getLocalPrincipal() was not supported when using OpenSSL, which can produce problems when switch from JDK to OpenSSL impl.
Modifications:
Implement SSLSession.getLocalCertificates() and getLocalPrincipal() for OpenSslEngine.
Result:
More consistent behaving between JDK and OpenSSL based SSLEngine.
Motivation:
As stated in the SSLSession javadocs getPeer* methods need to throw a SSLPeerUnverifiedException if peers identity has not be verified.
Modifications:
- Correctly throw SSLPeerUnverifiedException
- Add test for it.
Result:
Correctly behave like descripted in javadocs.
Motivation:
As we stored the WebSocketServerHandshaker in the ChannelHandlerContext it was always null and so no close frame was send if WebSocketServerProtocolHandler was used.
Modifications:
Store WebSocketServerHAndshaker in the Channel attributes and so make it visibile between different handlers.
Result:
Correctly send close frame.
Motivation:
https://github.com/twitter/hpack released version v1.0.1.
Modifications:
- Update pom files to pull in new version
Results:
Depend on the most recent hpack library.
Motivation:
The HTTP/2 codec has a few static buffers sent over the network which are allocated on the heap. This results in a copy operation when the buffer is sent out on the network.
Modifications:
- Ensure these static buffers are allocated using direct memory.
Result:
No copy operation necessary when writing static buffers to network.
Motivaion:
The HttpHeaders and DefaultHttpHeaders have methods deprecated due to being removed in future releases, but no replacement method to use in the current release. The deprecation policy should not be so aggressive as to not provide any non-deprecated method to use.
Modifications:
- Remove deprecated annotations and javadocs from methods which are the best we can do in terms of matching the master's api for 4.1
Result:
There should be non-deprecated methods available for HttpHeaders in 4.1.
Motivation:
The HTTP/2 header name validation was removed, and does not currently exist.
Modifications:
- Header name validation for HTTP/2 should be restored and set to the default mode of operation.
Result:
HTTP/2 header names are validated according to https://tools.ietf.org/html/rfc7540
Motivation:
TCP Fast Open allows data to be carried in the SYN and SYN-ACK packets and consumed by the receiving end during the initial connection handshake, and saves up to one full round-trip time (RTT) compared to the standard TCP, which requires a three-way handshake (3WHS) to complete before data can be exchanged. This commit enables support for TFO on server sockets.
Modifications:
Added new Integer Option TCP_FASTOPEN in EpollChannelOption.
Added getters/setters in EpollServerChannelConfig for TCP_FASTOPEN.
Added way to check if TCP_FASTOPEN is supported on server in Native.
Added setting on socket opt TCP_FASTOPEN if value is set on channel options in doBind in EpollServerSocketChannel.
Enhanced EpollSocketTestPermutation to contain a permutation for server socket containing fast open.
Result:
Users of native-epoll can set TCP_FASTOPEN on server sockets and thus leverage fast connect features of RFC7413 if client is capable of it.
Conflicts:
transport-native-epoll/src/main/java/io/netty/channel/epoll/EpollChannelOption.java
Motivation:
Related to issue #4185.
HTTP has the option to disable header validation for optimisation purposes. Introduce the same option for SPDY headers.
Also, optimise SpdyHttpEncoder by allowing the user to specify whether or not the encoder needs to convert header names to lowercase.
Modifications:
Added flags for validation and conversion.
Result:
SpdyHeader validation and conversion can be disabled.
Motivation:
Invoking the javax.net.ssl.SSLEngine.closeInbound() method will send a
fatal alert and invalidate the SSL session if a close_notify alert has
not been received.
From the javadoc:
If the application initiated the closing process by calling
closeOutbound(), under some circumstances it is not required that the
initiator wait for the peer's corresponding close message. (See section
7.2.1 of the TLS specification (RFC 2246) for more information on
waiting for closure alerts.) In such cases, this method need not be
called.
Always invoking the closeInbound() method without regard to whether or
not the closeOutbound() method has been invoked could lead to
invalidating perfectly valid SSL sessions.
Modifications:
Added an instance variable to track whether the
SSLEngine.closeOutbound() method has been invoked. When the instance
variable is true, the SSLEngine.closeInbound() method doesn't need to be
invoked.
Result:
SSL sessions will not be invalidated if the outbound side has been
closed but a close_notify alert hasn't been received.
Motivation:
When SpdyHttpEncoder attempts to create an SpdyHeadersFrame from a HttpResponse an IllegalArgumentException is thrown if the original HttpResponse contains a header that includes uppercase characters. The IllegalArgumentException is thrown due to the additional validation check introduced by #4047.
Previous versions of the SPDY codec would handle this by converting the HTTP header name to lowercase before adding the header to the SpdyHeadersFrame.
Modifications:
Convert the header name to lowercase before adding it to SpdyHeaders
Result:
SpdyHttpEncoder can now convert a valid HttpResponse into a valid SpdyFrame
Motivation:
RecvByteBufAllocator.DelegatingHandle does not provide an accessor to get the delegate handle. This may be useful for classes that extend DelegatingHandle.
Modifications:
- add delegate() method to DelegatingHandle
Result:
Classes which inherit from DelegatingHandle can now access the delegate Handle.
Motivation:
The javadoc comments on Http2Headers.iterator() are incorrect.
Modifications:
- Correct and clarify the javadoc for Http2Headers.iterator()
Result:
Javadoc for Http2Headers.iterator() is more correct.
Motivation:
There are protocols (BGP, SXP), which are typically deployed with TCP
MD5 authentication to protect sessions from being hijacked/torn down by
third parties. This facility is not available on most operating systems,
but is typically present on Linux.
Modifications:
- add a new EpollChannelOption, which is write-only
- teach Epoll(Server)SocketChannel to track which addresses have keys
associated
- teach Native how to set the MD5 signature keys for a socket
Result:
Users of the native-epoll transport can set MD5 signature keys and thus
leverage RFC-2385 protection on TCP connections.
Motivation:
If LocalChannel doWrite executes while the peer's state changes from CONNECTED to CLOSED it is possible that some promise's won't be completed and buffers will be leaked.
Modifications:
- Check the peer's state in doWrite to avoid a race condition
Result:
All write operations should release, and the associated promise should be completed.
Motivation:
InboundHttp2ToHttpAdapterTest.bootstrapEnv does not wait for the serverConnectedChannel to be initialized before returning. Some methods rely only this behavior and throw a NPE because it may not be set.
Modifications:
- Add a CountDownLatch to ensure the serverConnectedChannel is initialized
Result:
No more NPE.
Motivation:
The SimplePromiseAggregator.setFailure allows a failure to occur before newPromise is called, but tryFailure doesn't. These methods should be consistent.
Modifications:
- tryFailure should use the same logic as setFailure
Result:
Consistent failure routines.
Motivation:
The configurable property value recently added was not logged like others properties.
Modifications:
Added debug log with effective value applied.
Result:
Consistent with other properties
Motivation:
See #4174.
Modifications:
Modify transport-native-epoll to allow setting TCP_USER_TIMEOUT.
Result:
Hanging connections that are written into will get timeouted.
Conflicts:
transport-native-epoll/src/main/java/io/netty/channel/epoll/EpollChannelOption.java
Motivation:
Leak detector, when it detects a leak, will print the last 5 stack
traces that touched the ByteBuf. In some cases that might not be enough
to identify the root cause of the leak.
Also, sometimes users might not be interested in tracing all the
operations on the buffer, but just the ones that are affecting the
reference count.
Modifications:
Added command line properties to override default values:
* Allow to configure max number of stack traces to collect
* Allow to only record retain/release operation on buffers
Result:
Users can increase the number of stack traces to debug buffer leaks
with lot of retain/release operations.
Motivation:
DNS lookups in DnsNameResolverTest can take longer than expected due to
retries. The hard limit of 5 seconds is being applied to
testNegativeTtl(), making the first uncached lookup cause a timeout.
Modifications:
Do not use JUnit's Timeout annotation but implement simple timeout
mechanism that apples only to cached lookups.
Result:
testNegativeTtl() should not fail when an initial negative lookup
requires a retry.
Motivation:
As all methods in the ChannelHandler are executed by the same thread there is no need to use synchronized.
Modifications:
Remove synchronized keyword.
Result:
No more unnessary synchronized in SpdySessionHandler.
Motivation:
In NIO and OIO we throw a ChannelException if a ChannelConfig operation fails. We should do the same with epoll to be consistent.
Modifications:
Use ChannelException
Result:
Consistent behaviour across different transport implementations.
Motivation:
https://github.com/netty/netty/pull/4143 addressed a few ordering issues but an ordering issue still remained if the Promise for a write completes, and a listener of that promise does a write on a peer channel. The ordering was subject to how potentially 2 different executors would run a task, but it should be coordinated such that the first write is read first.
Modifications:
- Keep track of the finishPeerRead task run on the executor if necessary and ensure it completes before current channel read occurs
Result:
Ordering of events for echo type situations is preserved.
Motivation:
When a LocalChannel write operation occurs, the promise associated with the write operation is marked successful when it is added to the peer's queue, but before the peer has actually received the data. If the promise callback closes the channel then a race condition exists where the close event may occur before the data is delivered. We should preserve ordering of events.
Modifications:
- LocalChannel should track when a write is in progress, and if a close operation happens make sure the peer gets all pending read operations.
Result:
LocalChannel preserves order of operations.
Fixes https://github.com/netty/netty/issues/4118
Motivation:
The alpn / npn dependency versions are dependent on java version. If a java version 1.8+ is used that is not explicitly listed in the pom file then ALPN tests will fail because the java 1.7 version of alpn will be loaded by out pom file.
Modifications:
- Ensure there is a latest version to fall back up for npn 1.7+
- Ensure there is a latest version to fall back upon from alpn 1.8+
Result:
Build can complete despite having a newer jdk which is not listed in our pom file.
Motivation:
DefaultPropertyKey.index is currently private and accessed outside the class's scope.
Modifications:
- Change access level to package private
Result:
No chance of synthetic method generation for accessing this field
Motivation:
The latches in InboundHttp2ToHttpAdapterTest were volatile and reset during the tests. This resulted in race conditions and sometimes the tests would be waiting on old latches that were not the same latches being counted down when messages were received.
Modifications:
- Remove volatile latches from tests
Result:
More reliable tests with less race conditions.
Motivation:
for debugging and metrics reasons its sometimes useful to be able to get details of the the Thread that powers a SingleThreadEventExecutor.
Modifications:
- Expose ThreadProperties
- Add unit test.
Result:
It's now possible to get details of the Thread that powers a SingleThreadEventExecutor.
Motivation:
Sometimes it is useful to disable recycling completely if memory constraints are very tight.
Modifications:
Allow to use -Dio.netty.recycler.maxCapacity=0 to disable recycling completely.
Result:
It's possible to disable recycling now.
Motivation:
When try to get SO_LINGER from a fd that is closed an Exception is thrown. We should only try to get SO_LINGER if the fd is still open otherwise an Exception is thrown that can be ignored anyway.
Modifications:
First check if the fd is still open before try to obtain SO_LINGER setting when get the closeExecutor. This is also the same that we do in the NIO transport.
Result:
No more exception when calling unsafe.close() on a channel that has a closed file descriptor.
