Commit Graph

7201 Commits

Author SHA1 Message Date
Norman Maurer
8ec594c6eb Change HttpServerUpgradeHandler.UpgradeCodec to allow aborting upgrade
Motivation:

HttpServerUpgradeHandler.UpgradeCodec.prepareUpgradeResponse should allow to abort the upgrade and so just continue with using HTTP. Beside this we should only pass in the response HttpHeaders as this is inline with the docs.

Modifications:

- UpgradeCodec.prepareUpgradeResponse now allows to return a boolean and so allows to specifiy if the upgrade should take place.
- Change the param from FullHttpResponse to HttpHeaders to be inline with the javadocs.

Result:

More flexible and correct handling of upgrades.
2016-03-18 17:01:59 +01:00
Stephane Landelle
d747438366 Add ! to allowed cookie value chars
Motivation:

! is missing from allowed cookie value chars, as per https://tools.ietf.org/html/rfc6265#section-4.1.1.
Issue was originally reported on Play!, see https://github.com/playframework/playframework/issues/4460#issuecomment-198177302.

Modifications:

Stick to RFC6265 ranges.

Result:

RFC6265 compliance, ! is supported
2016-03-18 16:58:54 +01:00
Norman Maurer
0320ccb59f Let getSoError() throw IOException as well
Motivation:

In commit acbca192bd we changed to have our native operations which either gall getsockopt or setsockopt throw IOExceptions (to be more specific we throw a ClosedChannelException in some cases). Unfortunally I missed to also do the same for getSoError() and missed to add throws IOException to the native methods.

Modifications:

- Correctly throw IOException from getSoError()
- Add throws IOException to native methods where it was missed.

Result:

Correct declaration of getSoError() and other native methods.
2016-03-17 20:09:15 +01:00
Norman Maurer
ed9d6c79bc [#4972] Remove misleading argument from HttpServerUpgradeHandler.UpgradeCodec.upgradeTo
Motivation:

upgradeTo(...) takes the response as paramater, but the respone itself was already written to the Channel. This gives the user the impression the response can be changed or even act on it which may not be safe anymore once it was written and has been released.

Modifications:

Remove the response param from the method.

Result:

Less confusion and safer usage.
2016-03-17 10:50:07 +01:00
Norman Maurer
8d499a2419 Fix calculation of PoolArena metrics after introducing a regression in 89da788fd2 2016-03-17 10:37:22 +01:00
Norman Maurer
5c02397689 Support private key encrypted with empty password
Motivation:

A user may use a private key which is encrypted with an empty password. Because of this we should only handle a null password in a special way.

Modifications:

- Correctly handle private key that is encrypted with empty password.
- Make OpenSsl*Context implementions consistent in terms of initialization in the constructor.

Result:

Correctly support private key that is encrypted with empty password.
2016-03-17 09:07:28 +01:00
Norman Maurer
daa4efcfef Add proper synchronization when access metrics.
Motivation:

We also need to add synchronization when access fields to ensure we see the latest updates.

Modifications:

Add synchronization when read fields that are written concurrently.

Result:

Ensure correct visibility of updated.
2016-03-17 09:05:48 +01:00
Scott Mitchell
8dbf5d02e5 EPOLL SO_LINGER=0 sends FIN+RST
Motivation:
If SO_LINGER is set to 0 the EPOLL transport will send a FIN followed by a RST. This is not consistent with the behavior of the NIO transport. This variation in behavior can cause protocol violations in streaming protocols (e.g. HTTP) where a FIN may be interpreted as a valid end to a data stream, but RST may be treated as the data is corrupted and should be discarded.

https://github.com/netty/netty/issues/4170 Claims the behavior of NIO always issues a shutdown when close occurs. I could not find any evidence of this in Netty's NIO transport nor in the JDK's SocketChannel.close() implementation.

Modifications:
- AbstractEpollChannel should be consistent with the NIO transport and not force a shutdown on every close
- FileDescriptor to keep state in a consistent manner with the JDK and not allow a shutdown after a close
- Unit tests for NIO and EPOLL to ensure consistent behavior

Result:
EPOLL is capable of sending just a RST to terminate a connection.
2016-03-16 22:35:04 -07:00
Xiaoyan Lin
01835fdf18 Add LineEncoder to append a line separator automatically
Motivation:

See #1811

Modifications:

Add LineEncoder and LineSeparator

Result:

The user can use LineEncoder to write a String with a line separator automatically
2016-03-16 20:31:01 +01:00
Xiaoyan Lin
abbdc70d8b Validate MQTT CONNECT reserved flag in variable header
Motivation:

According to the MQTT 3.1.1 Protocol Specification: The Server MUST validate that the reserved flag in the CONNECT Control Packet is set to zero and disconnect the Client if it is not zero. (http://docs.oasis-open.org/mqtt/mqtt/v3.1.1/os/mqtt-v3.1.1-os.html#_Toc385349230)

Resolves #4182

Modifications:

Check the CONNECT reserved flag for MQTT 3.1.1. If it's not 0, throw an exception.

Result:

If the CONNECT reserved flag, a decode failure will be emitted.
2016-03-16 20:23:31 +01:00
Tibor Csögör
6e840d8e62 trivial javadoc fixes
- fix the formatting of the diagram in ChannelFuture's javadoc
- update external link in AutobahnServer
- fix various spelling issues
2016-03-16 20:18:29 +01:00
Norman Maurer
c3c1b4a6d2 [#4937] [#4935] Correctly valid domain name length and convert to ASCII.
Motivation:

Domain name labels must be converted to ASCII and not be longer then 63 chars.

Modifications:

Correctly convert to ASCII which also will enforce the 63 chars length.

Result:

Correctly guard against invalid input.
2016-03-16 11:55:19 +01:00
Norman Maurer
6796604f46 Remove double spacing
Motivation:

We had some double spacing in the methods which should be removed to keep things consistent.

Modifications:

Remove redundant spaces.

Result:

Cleaner / consistent coding style.
2016-03-16 10:25:48 +01:00
Norman Maurer
7d12333c38 Add final keyword which was missing in 47b598e6ce
Motivation:

The two fields should have final keyword.

Modifications:

Add final keyword

Result:

Cleaner code.
2016-03-16 10:25:00 +01:00
buchgr
83c349ffa9 Fix wrong use of assertTrue in unit test.
Motivation:

My previous commit b88a980482 introduced a flawed unit test,
that executes an assertion in a different thread than the test thread.
If this assertion fails, the test doesn't fail.

Modifications:

Replace the assertion by a proper workaround.

Result:

More correct unit test
2016-03-15 16:02:33 +01:00
Norman Maurer
acbca192bd ChannelConfig operations should wrap ClosedChannelException if Channel was closed before.
Motivation:

To be consistent with the JDK we should ensure our native methods throw a ClosedChannelException if the Channel was previously closed. This will then be wrapped in a ChannelException as usual. For all other errors we continue to just throw a ChannelException directly.

Modifications:

Ensure getsockopt and setsockopt will throw a ClosedChannelException if the channel was closed before, on other errors we throw a ChannelException as before diretly.

Result:

Consistent with the NIO Channel implementations.
2016-03-15 14:47:39 +01:00
buchgr
b88a980482 Change arena to thread cache mapping algorithm to be closer to ideal.
Motivation:
Circular assignment of arenas to thread caches can lead to less than optimal
mappings in cases where threads are (frequently) shutdown and started.

Example Scenario:
There are a total of 2 arenas. The first two threads performing an allocation
would lead to the following mapping:

Thread 0 -> Arena 0
Thread 1 -> Arena 1

Now, assume Thread 1 is shut down and another Thread 2 is started. The current
circular assignment algorithm would lead to the following mapping:

Thread 0 -> Arena 0
Thread 2 -> Arena 0

Ideally, we want Thread 2 to use Arena 1 though.

Presumably, this is not much of an issue for most Netty applications that do all
the allocations inside the eventloop, as eventloop threads are seldomly shut down
and restarted. However, applications that only use the netty-buffer package
or implement their own threading model outside the eventloop might suffer from
increased contention. For example, gRPC Java when using the blocking stub
performs some allocations outside the eventloop and within its own thread pool
that is dynamically sized depending on system load.

Modifications:

Implement a linear scan algorithm that assigns a new thread cache to the arena
that currently backs the fewest thread caches.

Result:

Closer to ideal mappings between thread caches and arenas. In order to always
get an ideal mapping, we would have to re-balance the mapping whenever a thread
dies. However, that's difficult because of deallocation.
2016-03-15 14:16:34 +01:00
johnou
26811b53ab Adding support for tcnative fedora flavour in uber jar
Motivation:

We want to allow the use of an uber jar that contains shared dynamic libraries for all platforms (including fedora).

Modifications:

Modified OpenSsl to try and load the fedora library if the OS is Linux and the platform specified library fails before using the default lib.

Result:

True uber support.
2016-03-15 13:56:41 +01:00
Scott Mitchell
6a2425b846 HTTP/2 SimpleChannelPromiseAggregator don't fail fast
Motivation:
Http2Codec.SimpleChannelPromiseAggregator currently fails fast if as soon as a tryFailure or setFailure method is called. This can lead to write operations which pass the result of SimpleChannelPromiseAggregator.newPromise to multiple channel.write calls throwing exceptions due to the promise being already done. This behavior is not expected by most of the Netty codecs (SslHandler) and can also create unexpected leaks in the http2 codec (DefaultHttp2FrameWriter).

Modifications:
- Http2Codec.SimpleChannelPromiseAggregator shouldn't complete the promise until doneAllocatingPromises is called
- Usages of Http2Codec.SimpleChannelPromiseAggregator should be adjusted to handle the change in behavior
- What were leaks in DefaultHttp2FrameWriter should be fixed to catch any other cases where ctx.write may throw

Result:
SimpleChannelPromiseAggregator won't generate promises which are done when newPromise is called.
2016-03-14 11:00:21 -07:00
Scott Mitchell
45849b2fa8 Deprecate PromiseAggregator
Motivation:
PromiseAggregator's API allows for the aggregate promise to complete before the user is done adding promises. In order to support this use case the API structure would need to change in a breaking manner.

Modifications:
- Deprecate PromiseAggregator and subclasses
- Introduce PromiseCombiner which corrects these issues

Result:
PromiseCombiner corrects the deficiencies in PromiseAggregator.
2016-03-14 10:53:30 -07:00
Norman Maurer
e3bf679998 Ensure connect promise is notifed before fireChannelActive() is called. Related to [#4927]
Motivation:

We should always first notify the promise before trigger an event through the pipeline to be consistent.

Modifications:

Ensure we notify the promise before fire event.

Result:

Consistent behavior
2016-03-14 14:18:11 +01:00
Xiaoyan Lin
4fb585965c Add DatagramPacketEncoder and DatagramPacketDecoder
Motivation:

UDP-oriented codec reusing the existing encoders and decoders would be helpful. See #1350

Modifications:

Add DatagramPacketEncoder and DatagramPacketDecoder to reuse the existing encoders and decoders.

Result:

People can use DatagramPacketEncoder and DatagramPacketDecoder to wrap existing encoders and decoders to create UDP-oriented codec.
2016-03-14 12:14:57 +01:00
Norman Maurer
52bfaae1a0 Fix EpollServerSocketConfig.isFreebind()
Motivation:

EpollServerSocketConfig.isFreebind() throws an exception when called.

Modifications:

Use the correct getsockopt arguments.

Result:

No more exception when call EpollServerSocketConfig.isFreebind()
2016-03-14 12:12:41 +01:00
Norman Maurer
c65165c491 Remove TCP_MD5 from EpollServerChannelConfig.
Motivation:

TCP_MD5 is only supported by SocketChannels so remove it from EpollServerChannelConfig which is generic.

Modifications:

Remove invalid code.

Result:

Remove invalid / dead code.
2016-03-14 12:11:56 +01:00
Norman Maurer
97df3cb039 [#4936] NetUtil can prevent using Netty due to SecurityManager denial
Motivation:

A custom SecurityManager may prevent calling File.exists() and so throw a SecurityException in the static init block of NetUtil.

Modifications:

Correctly catch the exception and so allow to static init NetUtil.

Result:

Allow static init method of NetUtil to work even with custom SecurityManager.
2016-03-14 12:10:16 +01:00
buchgr
35771dd1cd Fix race in PoolArena.allocate. Fixes #4829
Motivation:

The statistic counters PoolArena.(allocationsTiny|allocationsSmall) are
not protected by a per arena lock, but by a per size class lock. Thus,
two concurrent allocations of different size (class) could lead to a
race and ultimately to wrong statistics.

Modifications:

Use a thread-safe LongCounter instead of a plain long data type.

Result:

Fewer data races.
2016-03-14 08:57:46 +01:00
Max Ng
e7ee6abd70 Guard against re-entrance in PendingWriteQueue.
Motivation:

PendingWriteQueue should guard against re-entrant writes once
removeAndFailAll() is run.

Modifications:

removeAndFailAll() should repeat until the queue is finally empty.

Result:

assertEmpty() will always hold.
2016-03-14 08:52:43 +01:00
Scott Mitchell
404666d247 HTTP/2 ByteBufUtil.writeUtf8 cleanup
Motiviation:
691bc1690e made writeUtf8 consistent with String.getBytes() so that it never throws.
94f27be59b provided a writeUtf8 method which takes a ByteBufAllocator to do an appropriately sized buffer allocation.

Result:
- Assume writeUtf8 will not throw in HTTP/2 codec
- Use the new writeUtf8 method

Result:
Cleaner code in codec-http2.
2016-03-11 14:24:45 -08:00
Scott Mitchell
bfbef036a8 EPOLL ET AutoRead
Motivation:
EPOLL does not support autoread when in ET mode.

Modifications:
- EpollRecvByteAllocatorHandle should not unconditionally force reading just because ET is enabled
- AbstractEpollChannel and all derived classes which implement epollInReady must support a variable which indicates
there may be more data to read. The variable will be used when read is called to simulate a EPOLL wakeup and call epollInReady if necessary. This will ensure that if we don't read until EAGAIN that we will try to read again and not rely on EPOLL to notify us.

Result:
EPOLL ET supports auto read.
2016-03-11 07:42:30 -08:00
Scott Mitchell
bd6040a36e HTTP/2 DefaultHttp2Connection NPE
Motivation:
If while iterating the active streams a close operation occurs this will be queued and process after the iteration has completed to avoid a concurrent modification exception. However it is possible that during the iteration the stream which was closed could have been removed from the priority tree and its parent would be set to null. Then after the iteration completes the close operation will attempt to dereference the parent and results in a NPE.

Modifications:
- pending close operations should verify the stream's parent is not null before processing the event

Result:
No More NPE.
2016-03-11 07:41:04 -08:00
Mahdi Mohammadi
6536c7c4ef Updating Branches to look section to match the current branching structure of the project 2016-03-10 22:08:01 +01:00
Julien Viet
3d7cec6376 Bug fix for HttpPostMultipartRequestDecoder part decoding with an invalid charset not reported as an ErrorDataDecoderException
Motivation:

The current HttpPostMultipartRequestDecoder can decode multipart/form-data parts with a Content-Type that specifies a charset. When this charset is invalid the Charset.forName() throws an unchecked UnsupportedCharsetException. This exception is not catched by the decoder. It should actually be rethrown as an ErrorDataDecoderException, because the developer using the API would expect this validation failure to be reported as such.

Modifications:

Add a catch block for UnsupportedCharsetException and rethrow it as an ErrorDataDecoderException.

Result:

UnsupportedCharsetException are now rethrown as ErrorDataDecoderException.
2016-03-10 18:33:06 +01:00
Fabian Lange
58c63b7888 Add support for directly decoding/encoding DNS PTR Records
Motivation:

The current implementation will provide a DnsRawRecord, which, while
containing the host name it resolves to, would require the user to
decode the name using the decode method currently private to
DefaultDnsRecordDecoder, which in fact means copying it.

Modifications:

Introduce DnsPtrRecord, which is a specialization of DnsRecord which
provides a decoded host name.

Result:

PTR Records are much easier to work with, as the name is decoded already.
2016-03-08 14:59:46 +01:00
Norman Maurer
45d291bb15 Add asserts so users will see errors when try to use methods from outside the EventLoop.
Motivation:

We should guard users from using Unsafe methods from outside the EventLoop if not designed to do so.

Modifications:

Add asserts

Result:

Easier for users to detect miss-use.
2016-03-08 14:59:43 +01:00
Norman Maurer
d09547deb8 Add support for TCP_DEFER_ACCEPT and TCP_QUICKACK
Motivation:

When using the native transport have support for TCP_DEFER_ACCEPT or / and TCP_QUICKACK can be useful.

Modifications:

- Add support for TCP_DEFER_ACCEPT and TCP_QUICKACK
- Ad unit tests

Result:

TCP_DEFER_ACCEPT and TCP_QUICKACK are supported now.
2016-03-08 13:46:06 +01:00
Scott Mitchell
d0f7f98d22 e2f5012 unit test cleanup
Motivation:
e2f5012 added unit tests which did not verify the buffer was released as it was intended to.

Modification:
- Unit tests must verify release is called

Result:
Unit tests enforce that ByteBufs are released.
2016-03-07 09:33:45 -08:00
Scott Mitchell
e2f5012f3b DefaultChannelHandlerInvoker write leak
Motivation:
DefaultChannelHandlerInvoker.invokeWrite calls a utility method validatePromise which may throw if the arguments are not valid. If this method throws then the message will not be released.

Modifications:
- If an exception is thrown the message should be released

Result:
No more leak in DefaultChannelHandlerInvoker.invokeWrite
2016-03-07 09:16:12 -08:00
Trustin Lee
ef8dcae9af Fix potential infinite loop when resolving CNAME records
Related: #4771

Motivation:

A malicious or misconfigured DNS server can send the CNAME records that
resolve into each other, causing an unexpected infinite loop in
DnsNameResolverContext.onResponseCNAME().

Modifications:

- Remove the dereferenced CNAME from the alias map so that infinite loop
  is impossible.
- Fix inspection warnings and typos in DnsNameResolverTest

Result:

Fixes #4771
2016-03-07 15:12:26 +00:00
Xiaoyan Lin
e2d4e22243 Add CharsetUtil.encoder/decoder() methods
Motivation:

See #3321

Modifications:

1. Add CharsetUtil.encoder/decoder() methods
2. Deprecate CharsetUtil.getEncoder/getDecoder() methods

Result:

Users can use new CharsetUtil.encoder/decoder() to specify error actions
2016-03-07 10:48:31 +00:00
Norman Maurer
d8658989e1 Use smaller connect timeout to speed up tests.
Motivation:

For on tests we expected a ConnectTimeoutException but used the default timeout of 10 seconds. This slows down testing.

Modifications:

Use connect timeout of 1 second in unit test.

Result:

Faster execution of unit test.
2016-03-06 17:47:38 +01:00
Mike Smith
4095cb253a Just a couple of minor javadoc fixes 2016-03-06 17:45:48 +01:00
Scott Mitchell
900353af52 HTTP/2 Reduce Log Level
Motivation:
83c4aa6ad8 changed the log level to warn, but should have changed to debug.

Modifications:
- Change the log level to debug in Http2ConnectionHandler if the GO_AWAY fails to send. The write failure could be the result of the channel already being closed.

Result:
Fixes https://github.com/netty/netty/issues/4930.
2016-03-04 17:25:04 -08:00
Xiaoyan Lin
c295f22390 Add junit timeout listener to print the full thread dump on test timeout
Motivation:

See #3172

Modifications:

https://github.com/netty/netty-build/pull/6 added a junit timeout listener to the netty-build project. This patch just set it up.

Result:

If a test is set the timeout parameter using junit's @Test(timeout = ...) and the timeout is triggered, a full stack trace dump will be outputted and also output the deadlocks if any.
2016-03-04 10:47:11 +01:00
Sergey Polovko
68bbd4e966 Handle only those http requests that equal to adjusted websocket path
Motivation:

It will be easier to support websockets in server application by using WebSocketServerProtocolHandshakeHandler class and not reinvent its functionality. But currently it handles all http requests as if they were websocket handshake requests.

Modifications:

Check if http request path is equals to adjusted websocket path.
Fixed example of websocket server implementation.

Result:

WebSocketServerProtocolHandshakeHandler handles only websocket handshake requests.
2016-03-04 08:36:14 +01:00
Dmitry Spikhalskiy
0d3eda38e1 Helper method to get mime-type from Content-Type header of HttpMessage 2016-03-03 15:18:39 +01:00
Scott Mitchell
4b5b230802 HTTP/2 DefaultHttp2HeadersDecoder weighted average error
Motiviation:
cfcee5798d introduced code to resize the headers based upon a weighted average. The weight used for new entries was initialized using integer arithmetic when it should have been floating point arithmetic and so new values contribute 0 weight.

Modifications:
- Cast to float when initializing

Result:
Weighted average does not give 0 weight to new headers in DefaultHttp2HeadersDecoder.
2016-03-02 15:07:38 -08:00
Norman Maurer
9aac6dac2e [#4386] ByteToMessage.decodeLast(...) should not call decode(...) if buffer is empty.
Motivation:

If the input buffer is empty we should not have decodeLast(...) call decode(...) as the user may not expect this.

Modifications:

- Not call decode(...) in decodeLast(...) if the input buffer is empty.
- Add testcases.

Result:

decodeLast(...) will not call decode(...) if input buffer is empty.
2016-03-01 08:42:26 +01:00
Scott Mitchell
c4fbc0642d HTTP/2 stream removed from map before onStreamClosed called
Motivation:
The interface contract of Http2Connection.Listener.onStreamClosed says that the stream will be removed from the active stream map, and not necessarily the stream map. If the channel becomes inactive we may remove from the stream map before calling onStreamClosed.

Modifications:
- Don't remove from the stream map during iteration until after onStreamClosed is called

Result:
Expectations of onStreamClosed interface are not violated
2016-02-29 08:46:22 -08:00
Scott Mitchell
94f27be59b ByteBufUtil writeUtf8 and writeAscii helper methods
Motivation:
Utility methods in ByteBufUtil to writeUtf8 and writeAscii expect a buffer to already be allocated. If the user does not have a buffer allocated they have to know details of the encoding in order to know the size of the buffer to allocate.

Modifications:
- Add writeUtf8 and writeAscii which take a ByteBufAllocator and allocate a ByteBuf of the correct size for the user

Result:
ByteBufUtil methods which are easier to use if the user doesn't already have a ByteBuf.
2016-02-29 10:09:06 +01:00
Xiaoyan Lin
4a5e484c5a Add xml-maven-plugin to check indentation and fix violations
Motivation:

See https://github.com/netty/netty-build/issues/5

Modifications:

Add xml-maven-plugin to check indentation and fix violations

Result:
pom.xml will be checked in the PR build
2016-02-29 09:46:32 +01:00