Commit Graph

7193 Commits

Author SHA1 Message Date
Xiaoyan Lin
01835fdf18 Add LineEncoder to append a line separator automatically
Motivation:

See #1811

Modifications:

Add LineEncoder and LineSeparator

Result:

The user can use LineEncoder to write a String with a line separator automatically
2016-03-16 20:31:01 +01:00
Xiaoyan Lin
abbdc70d8b Validate MQTT CONNECT reserved flag in variable header
Motivation:

According to the MQTT 3.1.1 Protocol Specification: The Server MUST validate that the reserved flag in the CONNECT Control Packet is set to zero and disconnect the Client if it is not zero. (http://docs.oasis-open.org/mqtt/mqtt/v3.1.1/os/mqtt-v3.1.1-os.html#_Toc385349230)

Resolves #4182

Modifications:

Check the CONNECT reserved flag for MQTT 3.1.1. If it's not 0, throw an exception.

Result:

If the CONNECT reserved flag, a decode failure will be emitted.
2016-03-16 20:23:31 +01:00
Tibor Csögör
6e840d8e62 trivial javadoc fixes
- fix the formatting of the diagram in ChannelFuture's javadoc
- update external link in AutobahnServer
- fix various spelling issues
2016-03-16 20:18:29 +01:00
Norman Maurer
c3c1b4a6d2 [#4937] [#4935] Correctly valid domain name length and convert to ASCII.
Motivation:

Domain name labels must be converted to ASCII and not be longer then 63 chars.

Modifications:

Correctly convert to ASCII which also will enforce the 63 chars length.

Result:

Correctly guard against invalid input.
2016-03-16 11:55:19 +01:00
Norman Maurer
6796604f46 Remove double spacing
Motivation:

We had some double spacing in the methods which should be removed to keep things consistent.

Modifications:

Remove redundant spaces.

Result:

Cleaner / consistent coding style.
2016-03-16 10:25:48 +01:00
Norman Maurer
7d12333c38 Add final keyword which was missing in 47b598e6ce
Motivation:

The two fields should have final keyword.

Modifications:

Add final keyword

Result:

Cleaner code.
2016-03-16 10:25:00 +01:00
buchgr
83c349ffa9 Fix wrong use of assertTrue in unit test.
Motivation:

My previous commit b88a980482 introduced a flawed unit test,
that executes an assertion in a different thread than the test thread.
If this assertion fails, the test doesn't fail.

Modifications:

Replace the assertion by a proper workaround.

Result:

More correct unit test
2016-03-15 16:02:33 +01:00
Norman Maurer
acbca192bd ChannelConfig operations should wrap ClosedChannelException if Channel was closed before.
Motivation:

To be consistent with the JDK we should ensure our native methods throw a ClosedChannelException if the Channel was previously closed. This will then be wrapped in a ChannelException as usual. For all other errors we continue to just throw a ChannelException directly.

Modifications:

Ensure getsockopt and setsockopt will throw a ClosedChannelException if the channel was closed before, on other errors we throw a ChannelException as before diretly.

Result:

Consistent with the NIO Channel implementations.
2016-03-15 14:47:39 +01:00
buchgr
b88a980482 Change arena to thread cache mapping algorithm to be closer to ideal.
Motivation:
Circular assignment of arenas to thread caches can lead to less than optimal
mappings in cases where threads are (frequently) shutdown and started.

Example Scenario:
There are a total of 2 arenas. The first two threads performing an allocation
would lead to the following mapping:

Thread 0 -> Arena 0
Thread 1 -> Arena 1

Now, assume Thread 1 is shut down and another Thread 2 is started. The current
circular assignment algorithm would lead to the following mapping:

Thread 0 -> Arena 0
Thread 2 -> Arena 0

Ideally, we want Thread 2 to use Arena 1 though.

Presumably, this is not much of an issue for most Netty applications that do all
the allocations inside the eventloop, as eventloop threads are seldomly shut down
and restarted. However, applications that only use the netty-buffer package
or implement their own threading model outside the eventloop might suffer from
increased contention. For example, gRPC Java when using the blocking stub
performs some allocations outside the eventloop and within its own thread pool
that is dynamically sized depending on system load.

Modifications:

Implement a linear scan algorithm that assigns a new thread cache to the arena
that currently backs the fewest thread caches.

Result:

Closer to ideal mappings between thread caches and arenas. In order to always
get an ideal mapping, we would have to re-balance the mapping whenever a thread
dies. However, that's difficult because of deallocation.
2016-03-15 14:16:34 +01:00
johnou
26811b53ab Adding support for tcnative fedora flavour in uber jar
Motivation:

We want to allow the use of an uber jar that contains shared dynamic libraries for all platforms (including fedora).

Modifications:

Modified OpenSsl to try and load the fedora library if the OS is Linux and the platform specified library fails before using the default lib.

Result:

True uber support.
2016-03-15 13:56:41 +01:00
Scott Mitchell
6a2425b846 HTTP/2 SimpleChannelPromiseAggregator don't fail fast
Motivation:
Http2Codec.SimpleChannelPromiseAggregator currently fails fast if as soon as a tryFailure or setFailure method is called. This can lead to write operations which pass the result of SimpleChannelPromiseAggregator.newPromise to multiple channel.write calls throwing exceptions due to the promise being already done. This behavior is not expected by most of the Netty codecs (SslHandler) and can also create unexpected leaks in the http2 codec (DefaultHttp2FrameWriter).

Modifications:
- Http2Codec.SimpleChannelPromiseAggregator shouldn't complete the promise until doneAllocatingPromises is called
- Usages of Http2Codec.SimpleChannelPromiseAggregator should be adjusted to handle the change in behavior
- What were leaks in DefaultHttp2FrameWriter should be fixed to catch any other cases where ctx.write may throw

Result:
SimpleChannelPromiseAggregator won't generate promises which are done when newPromise is called.
2016-03-14 11:00:21 -07:00
Scott Mitchell
45849b2fa8 Deprecate PromiseAggregator
Motivation:
PromiseAggregator's API allows for the aggregate promise to complete before the user is done adding promises. In order to support this use case the API structure would need to change in a breaking manner.

Modifications:
- Deprecate PromiseAggregator and subclasses
- Introduce PromiseCombiner which corrects these issues

Result:
PromiseCombiner corrects the deficiencies in PromiseAggregator.
2016-03-14 10:53:30 -07:00
Norman Maurer
e3bf679998 Ensure connect promise is notifed before fireChannelActive() is called. Related to [#4927]
Motivation:

We should always first notify the promise before trigger an event through the pipeline to be consistent.

Modifications:

Ensure we notify the promise before fire event.

Result:

Consistent behavior
2016-03-14 14:18:11 +01:00
Xiaoyan Lin
4fb585965c Add DatagramPacketEncoder and DatagramPacketDecoder
Motivation:

UDP-oriented codec reusing the existing encoders and decoders would be helpful. See #1350

Modifications:

Add DatagramPacketEncoder and DatagramPacketDecoder to reuse the existing encoders and decoders.

Result:

People can use DatagramPacketEncoder and DatagramPacketDecoder to wrap existing encoders and decoders to create UDP-oriented codec.
2016-03-14 12:14:57 +01:00
Norman Maurer
52bfaae1a0 Fix EpollServerSocketConfig.isFreebind()
Motivation:

EpollServerSocketConfig.isFreebind() throws an exception when called.

Modifications:

Use the correct getsockopt arguments.

Result:

No more exception when call EpollServerSocketConfig.isFreebind()
2016-03-14 12:12:41 +01:00
Norman Maurer
c65165c491 Remove TCP_MD5 from EpollServerChannelConfig.
Motivation:

TCP_MD5 is only supported by SocketChannels so remove it from EpollServerChannelConfig which is generic.

Modifications:

Remove invalid code.

Result:

Remove invalid / dead code.
2016-03-14 12:11:56 +01:00
Norman Maurer
97df3cb039 [#4936] NetUtil can prevent using Netty due to SecurityManager denial
Motivation:

A custom SecurityManager may prevent calling File.exists() and so throw a SecurityException in the static init block of NetUtil.

Modifications:

Correctly catch the exception and so allow to static init NetUtil.

Result:

Allow static init method of NetUtil to work even with custom SecurityManager.
2016-03-14 12:10:16 +01:00
buchgr
35771dd1cd Fix race in PoolArena.allocate. Fixes #4829
Motivation:

The statistic counters PoolArena.(allocationsTiny|allocationsSmall) are
not protected by a per arena lock, but by a per size class lock. Thus,
two concurrent allocations of different size (class) could lead to a
race and ultimately to wrong statistics.

Modifications:

Use a thread-safe LongCounter instead of a plain long data type.

Result:

Fewer data races.
2016-03-14 08:57:46 +01:00
Max Ng
e7ee6abd70 Guard against re-entrance in PendingWriteQueue.
Motivation:

PendingWriteQueue should guard against re-entrant writes once
removeAndFailAll() is run.

Modifications:

removeAndFailAll() should repeat until the queue is finally empty.

Result:

assertEmpty() will always hold.
2016-03-14 08:52:43 +01:00
Scott Mitchell
404666d247 HTTP/2 ByteBufUtil.writeUtf8 cleanup
Motiviation:
691bc1690e made writeUtf8 consistent with String.getBytes() so that it never throws.
94f27be59b provided a writeUtf8 method which takes a ByteBufAllocator to do an appropriately sized buffer allocation.

Result:
- Assume writeUtf8 will not throw in HTTP/2 codec
- Use the new writeUtf8 method

Result:
Cleaner code in codec-http2.
2016-03-11 14:24:45 -08:00
Scott Mitchell
bfbef036a8 EPOLL ET AutoRead
Motivation:
EPOLL does not support autoread when in ET mode.

Modifications:
- EpollRecvByteAllocatorHandle should not unconditionally force reading just because ET is enabled
- AbstractEpollChannel and all derived classes which implement epollInReady must support a variable which indicates
there may be more data to read. The variable will be used when read is called to simulate a EPOLL wakeup and call epollInReady if necessary. This will ensure that if we don't read until EAGAIN that we will try to read again and not rely on EPOLL to notify us.

Result:
EPOLL ET supports auto read.
2016-03-11 07:42:30 -08:00
Scott Mitchell
bd6040a36e HTTP/2 DefaultHttp2Connection NPE
Motivation:
If while iterating the active streams a close operation occurs this will be queued and process after the iteration has completed to avoid a concurrent modification exception. However it is possible that during the iteration the stream which was closed could have been removed from the priority tree and its parent would be set to null. Then after the iteration completes the close operation will attempt to dereference the parent and results in a NPE.

Modifications:
- pending close operations should verify the stream's parent is not null before processing the event

Result:
No More NPE.
2016-03-11 07:41:04 -08:00
Mahdi Mohammadi
6536c7c4ef Updating Branches to look section to match the current branching structure of the project 2016-03-10 22:08:01 +01:00
Julien Viet
3d7cec6376 Bug fix for HttpPostMultipartRequestDecoder part decoding with an invalid charset not reported as an ErrorDataDecoderException
Motivation:

The current HttpPostMultipartRequestDecoder can decode multipart/form-data parts with a Content-Type that specifies a charset. When this charset is invalid the Charset.forName() throws an unchecked UnsupportedCharsetException. This exception is not catched by the decoder. It should actually be rethrown as an ErrorDataDecoderException, because the developer using the API would expect this validation failure to be reported as such.

Modifications:

Add a catch block for UnsupportedCharsetException and rethrow it as an ErrorDataDecoderException.

Result:

UnsupportedCharsetException are now rethrown as ErrorDataDecoderException.
2016-03-10 18:33:06 +01:00
Fabian Lange
58c63b7888 Add support for directly decoding/encoding DNS PTR Records
Motivation:

The current implementation will provide a DnsRawRecord, which, while
containing the host name it resolves to, would require the user to
decode the name using the decode method currently private to
DefaultDnsRecordDecoder, which in fact means copying it.

Modifications:

Introduce DnsPtrRecord, which is a specialization of DnsRecord which
provides a decoded host name.

Result:

PTR Records are much easier to work with, as the name is decoded already.
2016-03-08 14:59:46 +01:00
Norman Maurer
45d291bb15 Add asserts so users will see errors when try to use methods from outside the EventLoop.
Motivation:

We should guard users from using Unsafe methods from outside the EventLoop if not designed to do so.

Modifications:

Add asserts

Result:

Easier for users to detect miss-use.
2016-03-08 14:59:43 +01:00
Norman Maurer
d09547deb8 Add support for TCP_DEFER_ACCEPT and TCP_QUICKACK
Motivation:

When using the native transport have support for TCP_DEFER_ACCEPT or / and TCP_QUICKACK can be useful.

Modifications:

- Add support for TCP_DEFER_ACCEPT and TCP_QUICKACK
- Ad unit tests

Result:

TCP_DEFER_ACCEPT and TCP_QUICKACK are supported now.
2016-03-08 13:46:06 +01:00
Scott Mitchell
d0f7f98d22 e2f5012 unit test cleanup
Motivation:
e2f5012 added unit tests which did not verify the buffer was released as it was intended to.

Modification:
- Unit tests must verify release is called

Result:
Unit tests enforce that ByteBufs are released.
2016-03-07 09:33:45 -08:00
Scott Mitchell
e2f5012f3b DefaultChannelHandlerInvoker write leak
Motivation:
DefaultChannelHandlerInvoker.invokeWrite calls a utility method validatePromise which may throw if the arguments are not valid. If this method throws then the message will not be released.

Modifications:
- If an exception is thrown the message should be released

Result:
No more leak in DefaultChannelHandlerInvoker.invokeWrite
2016-03-07 09:16:12 -08:00
Trustin Lee
ef8dcae9af Fix potential infinite loop when resolving CNAME records
Related: #4771

Motivation:

A malicious or misconfigured DNS server can send the CNAME records that
resolve into each other, causing an unexpected infinite loop in
DnsNameResolverContext.onResponseCNAME().

Modifications:

- Remove the dereferenced CNAME from the alias map so that infinite loop
  is impossible.
- Fix inspection warnings and typos in DnsNameResolverTest

Result:

Fixes #4771
2016-03-07 15:12:26 +00:00
Xiaoyan Lin
e2d4e22243 Add CharsetUtil.encoder/decoder() methods
Motivation:

See #3321

Modifications:

1. Add CharsetUtil.encoder/decoder() methods
2. Deprecate CharsetUtil.getEncoder/getDecoder() methods

Result:

Users can use new CharsetUtil.encoder/decoder() to specify error actions
2016-03-07 10:48:31 +00:00
Norman Maurer
d8658989e1 Use smaller connect timeout to speed up tests.
Motivation:

For on tests we expected a ConnectTimeoutException but used the default timeout of 10 seconds. This slows down testing.

Modifications:

Use connect timeout of 1 second in unit test.

Result:

Faster execution of unit test.
2016-03-06 17:47:38 +01:00
Mike Smith
4095cb253a Just a couple of minor javadoc fixes 2016-03-06 17:45:48 +01:00
Scott Mitchell
900353af52 HTTP/2 Reduce Log Level
Motivation:
83c4aa6ad8 changed the log level to warn, but should have changed to debug.

Modifications:
- Change the log level to debug in Http2ConnectionHandler if the GO_AWAY fails to send. The write failure could be the result of the channel already being closed.

Result:
Fixes https://github.com/netty/netty/issues/4930.
2016-03-04 17:25:04 -08:00
Xiaoyan Lin
c295f22390 Add junit timeout listener to print the full thread dump on test timeout
Motivation:

See #3172

Modifications:

https://github.com/netty/netty-build/pull/6 added a junit timeout listener to the netty-build project. This patch just set it up.

Result:

If a test is set the timeout parameter using junit's @Test(timeout = ...) and the timeout is triggered, a full stack trace dump will be outputted and also output the deadlocks if any.
2016-03-04 10:47:11 +01:00
Sergey Polovko
68bbd4e966 Handle only those http requests that equal to adjusted websocket path
Motivation:

It will be easier to support websockets in server application by using WebSocketServerProtocolHandshakeHandler class and not reinvent its functionality. But currently it handles all http requests as if they were websocket handshake requests.

Modifications:

Check if http request path is equals to adjusted websocket path.
Fixed example of websocket server implementation.

Result:

WebSocketServerProtocolHandshakeHandler handles only websocket handshake requests.
2016-03-04 08:36:14 +01:00
Dmitry Spikhalskiy
0d3eda38e1 Helper method to get mime-type from Content-Type header of HttpMessage 2016-03-03 15:18:39 +01:00
Scott Mitchell
4b5b230802 HTTP/2 DefaultHttp2HeadersDecoder weighted average error
Motiviation:
cfcee5798d introduced code to resize the headers based upon a weighted average. The weight used for new entries was initialized using integer arithmetic when it should have been floating point arithmetic and so new values contribute 0 weight.

Modifications:
- Cast to float when initializing

Result:
Weighted average does not give 0 weight to new headers in DefaultHttp2HeadersDecoder.
2016-03-02 15:07:38 -08:00
Norman Maurer
9aac6dac2e [#4386] ByteToMessage.decodeLast(...) should not call decode(...) if buffer is empty.
Motivation:

If the input buffer is empty we should not have decodeLast(...) call decode(...) as the user may not expect this.

Modifications:

- Not call decode(...) in decodeLast(...) if the input buffer is empty.
- Add testcases.

Result:

decodeLast(...) will not call decode(...) if input buffer is empty.
2016-03-01 08:42:26 +01:00
Scott Mitchell
c4fbc0642d HTTP/2 stream removed from map before onStreamClosed called
Motivation:
The interface contract of Http2Connection.Listener.onStreamClosed says that the stream will be removed from the active stream map, and not necessarily the stream map. If the channel becomes inactive we may remove from the stream map before calling onStreamClosed.

Modifications:
- Don't remove from the stream map during iteration until after onStreamClosed is called

Result:
Expectations of onStreamClosed interface are not violated
2016-02-29 08:46:22 -08:00
Scott Mitchell
94f27be59b ByteBufUtil writeUtf8 and writeAscii helper methods
Motivation:
Utility methods in ByteBufUtil to writeUtf8 and writeAscii expect a buffer to already be allocated. If the user does not have a buffer allocated they have to know details of the encoding in order to know the size of the buffer to allocate.

Modifications:
- Add writeUtf8 and writeAscii which take a ByteBufAllocator and allocate a ByteBuf of the correct size for the user

Result:
ByteBufUtil methods which are easier to use if the user doesn't already have a ByteBuf.
2016-02-29 10:09:06 +01:00
Xiaoyan Lin
4a5e484c5a Add xml-maven-plugin to check indentation and fix violations
Motivation:

See https://github.com/netty/netty-build/issues/5

Modifications:

Add xml-maven-plugin to check indentation and fix violations

Result:
pom.xml will be checked in the PR build
2016-02-29 09:46:32 +01:00
Sergey Polovko
a2e113b987 Use ByteProcessor.FIND_LF to find end of line
Motivation:

Each call of ByteBuf.getByte(int) method does boundary checking. This can be eliminated by using ByteBuf.forEachByte(ByteProcessor) method and ByteProcessor.FIND_LF processor.

Modifications:

Find end of line with ByteProcessor.FIND_LF

Result:

A little better performance of LineBasedFrameDecoder.
2016-02-29 09:38:36 +01:00
cnapagoda
a4846e6153 issues/4909 - Adding paxexam version
Motivation:

Build warning message is shown when building netty project. Due to missing paxexam version below[1] warning message is shown when building netty project.

Modifications:

Added paxexam plugin version into root pom

Result:

paxexam related warning will not be displayed when building the project.

[1]. #4909
2016-02-29 09:17:18 +01:00
Norman Maurer
0bea10b0b0 [#4892] Make Snappy class public
Motivation:

Some people may want to use the Snappy class directly to encode / decode ByteBufs.

Modifications:

Make the Snappy class public and final.

Result:

Easier for people to reuse parts of Netty.
2016-02-26 12:15:10 +01:00
Trustin Lee
5ce504070f Remove a unused import 2016-02-24 11:17:35 +09:00
nmittler
6423e1b9c8 Adding support for tcnative uber jar
Motivation:

We want to allow the use of an uber jar that contains the shared libraries for all platforms.

Modifications:

Modified OpenSsl to first check for a platform-specific lib before using the default lib.

Result:

uber support.
2016-02-23 09:28:40 -08:00
Lukáš Karas
b426566617 DnsNameResolver: makes possible to define additional records in DNS query
Motivation:

Current DnsNameResolver api don't allow to define additional records in DNS query.
It can be useful in many cases. For example when we want to query dns server with
real client address (EDNS-CLIENT-SUBNET extension:
http://tools.ietf.org/html/draft-vandergaast-edns-client-subnet-02 )

Modifications:

This change add new query methods with list of additional DnsRecord-s for query.

Result:

It is possible to create dns query with EDNS-CLIENT-SUBNET extension for example.
2016-02-23 10:36:30 +01:00
Xiaoyan Lin
c7aadb5469 Fork SpscLinkedQueue and SpscLinkedAtomicQueue from JCTools
Motivation:

See #3746.

Modifications:

Fork SpscLinkedQueue and SpscLinkedAtomicQueue from JCTools based on 7846450e28

Result:

Add SpscLinkedQueue and SpscLinkedAtomicQueue and apply it in LocalChannel.
2016-02-22 15:55:52 +01:00
Norman Maurer
ca443e42e0 [maven-release-plugin] prepare for next development iteration 2016-02-19 23:00:11 +01:00