Commit Graph

8236 Commits

Author SHA1 Message Date
Scott Mitchell
6cd086050f DNS Resolver Search Domain Bugs
Motivation:
The DNS resolver supports search domains. However the ndots are not correctly enforced. The search domain should only be appended under the following scenario [1]:

> Resolver queries having fewer than ndots dots (default is 1) in them will be attempted using each component of the search path in turn until a match is found.

The DNS resolver current appends the search domains if ndots is 0 which should never happen (because no domain can have less than 0 dots).

[1] https://linux.die.net/man/5/resolv.conf

Modifications:
- Parse /etc/resolv.conf to get the default value for ndots on Unix platforms
- The search domain shouldn't be used if ndots is 0
- Avoid failing a promise to trigger the search domain queries in DnsNameResolverContext#resolve

Result:
More correct usage of search domains in the DNS resolver.
Fixes https://github.com/netty/netty/issues/6844.
2017-06-22 00:05:43 -07:00
Scott Mitchell
5934ae8fd2 Http2FrameLogger Updates
Motivation:
The Http2FrameLogger uses a custom format when logging events. We should use the more familiar format of 'channel event type: details' and single line logging for more consistent debugging.

Modifications:
- Http2FrameLogger should not use a StringBuilder and instead should directly use the Logger
- Http2FrameLogger should use the more consistent format defined above

Result:
Http2FrameLogger's logging formate is more consistent with other log events.
2017-06-21 17:12:38 -07:00
Nikolay Fedorovskikh
8f8be31226 Remove unnecessary conversions
Motivation:

In a `HttpConversionUtil#toHttp2Headers` a status code conversion can be replaced with using `HttpResponseStatus#codeAsText` method.

Modifications:

Apply `HttpResponseStatus#codeAsText` method.

Result:

Less allocations.
2017-06-21 21:53:48 +02:00
Scott Mitchell
6b25909f6b HTTP/2 HelloWorld clear text NPE
Motivation:
HelloWorldHttp2Handler throws a NPE when converting from HTTP/1.x headers to HTTP/2 headers because there is no Host header.

Modifications:
- HelloWorldHttp2Handler should check if the Host header is present before setting it in the HTTP/2 headers

Result:
No more NPE in HelloWorldHttp2Handler.
2017-06-21 06:55:28 +02:00
Norman Maurer
568fa998b1 Deploy transport-native-unix-common-tests
Motivation:

To be able to easily build only one of the native sub-modules its needed that all the dependencies can be fetched from maven. At the moment we dont deploy transport-native-unix-common and so an attempt to just build for example the native epoll transport fails with:

[ERROR] Failed to execute goal on project netty-transport-native-epoll: Could not resolve dependencies for project io.netty:netty-transport-native-epoll:jar:4.1.13.Final-SNAPSHOT: Could not find artifact io.netty:netty-transport-native-unix-common-tests:jar:4.1.13.Final-SNAPSHOT in sonatype-nexus-snapshots (https://oss.sonatype.org/content/repositories/snapshots) -> [Help 1]

Modifications:

Deploy jar

Result:

All dependencies on maven repository.
2017-06-21 06:52:18 +02:00
Scott Mitchell
6d029ad3ac OpenSSL CHACHA20 CipherSuiteConverter updates
Motivation:
For historical reasons OpenSSL's internal naming convention for CHACHA20 based cipher suites does not include the HMAC algorithm in the cipher name. This will prevent the CHACHA20 cipher suites from being used if the RFC cipher names are specified.

Modifications:
- Add a special case for CHACHA20 cipher name conversions in CipherSuiteConverter
- Update OPENSSL_CIPHERSUITE_PATTERN to accommodate the new naming scheme for CHACHA20 cipher suites

Result:
CipherSuiteConverter now works with CHACHA20 cipher suites.
2017-06-21 06:50:43 +02:00
Dmitriy Dumanskiy
81f9434ec7 Added test for multi header, HttpObjectDecoder performance improvement for multi header, removed empty else block.
Motivation:

For multi-line headers HttpObjectDecoder uses StringBuilder.append(a).append(b) pattern that could be easily replaced with regular a + b. Also oparations with a and b moved out from concat operation to make it friendly for StringOptimizeConcat optimization and thus - faster.

Modification:

StringBuilder.append(a).append(b) reaplced with a + b. Operations with a and b moved out from concat oparation.

Result:
Code simpler to read and faster.
2017-06-20 07:11:02 +02:00
Norman Maurer
82a43727c3 Fix testcase introduced by a2bd9a4 2017-06-20 07:01:13 +02:00
Scott Mitchell
14ea69cdc1 NullPointerException in Lz4FrameEncoder
Motivation:
Lz4FrameEncoder maintains internal state, but the life cycle of the buffer is not consistently managed. The buffer is allocated in handlerAdded but freed in close, but the buffer can still be used until handlerRemoved is called.

Modifications:
- Move the cleanup of the buffer from close to handlerRemoved
- Explicitly throw an EncoderException from Lz4FrameEncoder if the encode operation has finished and there isn't enough space to write data

Result:
No more NPE in Lz4FrameEncoder on the buffer.
2017-06-19 14:24:09 -07:00
Roger Kapsi
7460d90a67 Add listener to returned Future rather than passed in Promise
Motivation

It's cleaner to add listeners to returned Futures rather than provided Promises because the latter can have strange side effects in terms of listeners firing and called methods returning. Adding listeners preemtively may yield also to more OPS than necessary when there's an Exception in the to be called method.

Modifications

Add listener to returned ChannelFuture rather than given ChannelPromise

Result

Cleaner completion and exception handling
2017-06-16 13:21:19 +01:00
Norman Maurer
575baf5050 Use more aggressive expanding strategy in HpackHuffmanDecoder
Motivation:

Before we always expanded the buffer by the initialCapacity which by default is 32 bytes. This may lead to many expansions of the buffer until we finally reached the point that the buffer can fin everything.

Modifications:

Double the buffer size until the threshold of >= 1024 is hit. After this will grow it by the initialCapacity

Result:

Less expansion of the buffer (and so allocations / copies) when the intialCapacity is not big enough. Fixes [#6864].
2017-06-15 06:56:44 +02:00
Norman Maurer
e597756a56 Remove synchronized (ReferenceCountedOpenSslContext.class) blocks
Motivation:

We had some useless synchronized (ReferenceCountedOpenSslContext.class) blocks in our code which could slow down concurrent collecting and creating of ReferenceCountedOpenSslContext instances. Beside this we missed a few guards.

Modifications:

Use ReadWriteLock to correctly guard. A ReadWriteLock was choosen as SSL.newSSL(...) will be called from multiple threads all the time so using synchronized would be worse and there would be no way for the JIT to optimize it away

Result:

Faster concurrent creating and collecting of ReferenceCountedOpenSslContext instances and correctly guard in all cases.
2017-06-15 06:29:14 +02:00
Stephane Nicoll
2af895994d Add dependency management for missing entries
Motivation:

The bom does not provide entries for a number of netty modules, in
particular those that are deployed with classifiers. As a result, they
can't be used without defining a version.

Modifications:

Provide dependency management for the missing modules.

Result:

Fixes [#6852]
2017-06-14 20:26:33 +02:00
Nikolay Fedorovskikh
b8a418d53d Remove redundant code block in HttpPostRequestEncoder and make some cleanup
Motivation:

The class `HttpPostRequestEncoder` has minor issues:
- The `encodeNextChunkMultipart()` method contains two identical blocks of code with a difference only in the cast interfaces: `Attribute` vs `HttpData`. Because the `Attribute` is extended by `HttpData`, the block with the `Attribute` can be safely deleted.
- The `getNewMultipartDelimiter()` method contains a redundant `toLowerCase()`.
- The `addBodyFileUploads()` method throws `NPE` instead of `IllegalArgumentException`.

Modifications:

- Remove duplicated code block from `encodeNextChunkMultipart()`.
- Remove redundant `toLowerCase()` from `getNewMultipartDelimiter()`.
- Replace `NPE` with `IllegalArgumentException` in `addBodyFileUploads()`.
- Use `ObjectUtil#checkNotNull` where possible.

Result:

More correct and clean code.
2017-06-14 06:49:20 +02:00
Nikolay Fedorovskikh
aa38b6a769 Prevent unnecessary allocations in the StringUtil#escapeCsv
Motivation:

A `StringUtil#escapeCsv` creates new `StringBuilder` on each value even if the same string is returned in the end.

Modifications:

Create new `StringBuilder` only if it really needed. Otherwise, return the original string (or just trimmed substring).

Result:

Less GC load. Up to 4x faster work for not changed strings.
2017-06-13 14:57:38 -07:00
Norman Maurer
94c0ef3c96 Not fail the promise when a closed Channel is offered back to the ChannelPool
Motivation:

We should not fail the promise when a closed Channel is offereed back to the ChannelPool as we explicit mention that the Channel must always be returned.

Modifications:

- Not fail the promise
- Add test-case

Result:

Fixes [#6831]
2017-06-13 18:50:20 +02:00
Subhobrata Dey
66c83f7b74 codec.mqtt: password and willMessage field types should be byte[]
Motivation:

Update the mqtt-codec based on mqtt spec (3.1.3.5).

Modification:

Changes made to the file MqttConnectPayload.java.
Subsequent changes have been made to files MqttDecoder.java, MqttEncoder.java, MqttMessageBuilders.java.
Test cases have been updated.
Result:

Fixes #6750 .
2017-06-13 18:49:13 +02:00
Andy Wilkinson
ea2af3593c Correct bom entry for netty-transport-native-unix-common
Motivation:

The entry for the netty-transport-native-unix-common module in the bom
was using the wrong artifact ID and version.

Modifications:

Correct the artifact ID for the netty-transport-native-unix-common
module in the bom.

Result:

Fixes [#6849]
2017-06-13 15:47:36 +02:00
Stephane Landelle
ca5ed7c114 Let DnsNameResolver constructor use a default value for searchDomains
Motivation:

It’s currently complicated to extend `DnsNameResolver` as the default
value for `searchDomain` is package private.

Modifications:

* let `DnsNameResolver` accept a null `searchDomains` and then default
to `DEFAULT_SEARCH_DOMAINS`, just like it’s being done with
`resolvedAddressTypes`.
* set default `DnsNameResolverBuilder#searchDomains` value to null to
avoid cloning internal `DnsNameResolver.DEFAULT_SEARCH_DOMAINS` in
`DnsNameResolver` constructor.

Result:

More versatile `DnsNameResolver` constructor.
No array copy when using default search domains.
2017-06-13 15:43:08 +02:00
Norman Maurer
3f0085c267 Do proper bounds-checking in HpackHuffmanDecoder to reduce overhead of IndexOutOfBoundsException creation
Motivation:

HpackHuffmanDecoder.Decoder did not do any bound-checking but just catched IndexOutOfBoundsException to detect if the array needs to grow. This can be very expensive because of fillInStackTrace()

Modifications:

Add proper bounds checking and grow the array if needed without catching IndexOutOfBoundsException.

Result:

Less overhead if the array needs to grow.
2017-06-13 07:43:29 +02:00
Scott Mitchell
f00638af52 AbstractHttp2ConnectionHandlerBuilder support for HPACK huffman decoder initial size
Motivation:
Depending on the use case it may make sense to increase or decrease the initial size of the buffer used during the HPACK huffman decode process. This is currently not exposed through the AbstractHttp2ConnectionHandlerBuilder.

Modifications:
- Add a method to AbstractHttp2ConnectionHandlerBuilder which allows the initial size of the buffer used during the HPACK huffman decode prcoess to be configured.

Result:
AbstractHttp2ConnectionHandlerBuilder provides more control of codec-http2 knobs.
2017-06-12 16:36:43 -07:00
Scott Mitchell
1cc4607f07 AppendableCharSequence not to depend upon IndexOutOfBoundsException for resize
Motivation:
AppendableCharSequence depends upon IndexOutOfBoundsException to trigger a resize operation under the assumption that the resize operation will be rare if the initial size guess is good. However if the initial size guess is not good then the performance will be more unpredictable and likely suffer.

Modifications:
- Check the position in AppendableCharSequence#append to determine if a resize is necessary

Result:
More predictable performance in AppendableCharSequence#append.
2017-06-12 12:42:20 -07:00
Rogan Dawes
051e0ad4be Add support for IP_TRANSPARENT socket option
Motivation:

This allows netty to operate in 'transparent proxy' mode, intercepting connections
to other addresses by means of Linux firewalling rules, as per

https://www.kernel.org/doc/Documentation/networking/tproxy.txt

The original destination address can be obtained by referencing
ch.localAddress().

Modification:

Add methods similar to those for ipFreeBind, to set the IP_TRANSPARENT option.

Result:

Allows setting and getting of the IP_TRANSPARENT option, which allows retrieval of the ultimate socket address originally requested.
2017-06-12 08:03:39 +02:00
Norman Maurer
b6c27b9f6b Not force to run autoconf and compile multiple times
Motivation:

We should not force autoconf and compile as this will result in multiple executions and so slow down the build.

Modifications:

Remove force declarations

Result:

Faster build of native modules
2017-06-09 20:34:28 +02:00
Dmitriy Dumanskiy
acc07fac32 disabling leak detection micro benchmark
Motivation:

When I run Netty micro benchmarks I get many warnings like:

WARNING: -Dio.netty.noResourceLeakDetection is deprecated. Use '-Dio.netty.leakDetection.level=simple' instead.

Modification:

-Dio.netty.noResourceLeakDetection replaced with -Dio.netty.leakDetection.level=disabled.

Result:

No warnings.
2017-06-09 18:03:54 +02:00
Norman Maurer
fd67a2354d [maven-release-plugin] prepare for next development iteration 2017-06-08 21:06:24 +02:00
Norman Maurer
3acd5c68ea [maven-release-plugin] prepare release netty-4.1.12.Final 2017-06-08 21:06:01 +02:00
Norman Maurer
80aa5dcdcc Revert "Not add ChannelHandler to ChannelPipeline once the pipeline was destroyed."
This reverts commit 4aa8002596.
2017-06-08 19:50:17 +02:00
Norman Maurer
f208b147a6 Use FQCN to prevent classloader issues on java6
Motivation:

We need to use FQCN to prevent classloader issues for classes that are > Java6. This is a cleanup of ed5fcbb773.

Modifications:

Just remove the imports and use FQCN.

Result:

No classloader issues with java6
2017-06-08 12:04:17 +02:00
Norman Maurer
cdf3acb6a2 Update to latest netty-tcnative release 2017-06-07 20:06:55 +02:00
Norman Maurer
047da11086 Correctly handle ByteBuf implementations which have no memoryAddress when writing to native transport
Motivation:

Commit 3c4dfed08a introduced a regression in handling buffers that have no memoryAddress.

Modifications:

Fix regression and also add unit tests.

Result:

It's possible again to write buffers without memory address.
2017-06-07 19:36:05 +02:00
Norman Maurer
7922757575 Allow to access memoryAddress of wrapped ByteBuf for ReadOnlyByteBuf
Motivation:

We should allow to access the memoryAddress of the wrapped ByteBuf when using ReadOnlyByteBuf for peformance reasons. If a user act on a memoryAddress its his responsible anyway to do nothing "stupid".

Modifications:

Delegate to wrapped ByteBuf.

Result:

Less performance overhead for various operations and also when writing to a native transport (which needs the memoryAddress).
2017-06-07 18:45:26 +02:00
Renjie Sun
629b83e0a5 Move QueryStringDecoder.decodeHexByte into ByteBufUtil
Motivations:
1. There are duplicated implementations of decoding hex strings. #6797
2. ByteBufUtil.HexUtil.decodeHexDump does not handle substring start
index properly and does not decode hex byte rigorously.

Modifications:
1. Function decodeHexByte is moved from QueryStringDecoder into ByteBufUtil.
2. ByteBufUtil.HexUtil.decodeHexDump is changed to use decodeHexByte.
3. Tests are Updated accordingly.

Result:
Fixed #6797 and made hex decoding functions more robust.
2017-06-07 09:27:36 -07:00
Michael K. Werle
ed5fcbb773 Add explicit message when noexec prevents library loading.
Motivation:

Docker's `--tmpfs` flag mounts the temp volume with `noexec` by default,
resulting in an UnsatisfiedLinkError.  While this is good security
practice, it is a surprising failure from a seemingly innocuous flag.

Modifications:

Add a best-effort attempt in `NativeLibraryLoader` to detect when temp
files beng loaded cannot be executed even when execution permissions
are set, often because the `noexec` flag is set on the volume.

Requires numerous additional exclusions to the Animal Sniffer config
for Java7 POSIX permissions manipulation.

Result:

Fixes [#6678].
2017-06-07 09:20:05 -07:00
Georg Held
1f0d47dee7 Added PROXY Protocol TLV support
Motivation:

The current PROXY protocol implementation does not have support for optional Type-Length-Value fields. This pull requests adds the TLV values as specified in the PROXY protocol specification (http://www.haproxy.org/download/1.5/doc/proxy-protocol.txt) and adds support for arbitrary TLVs.

Modifications:

The existing HAProxyMessage implements an additional TLV reading operation. A small bug in the AF_UNIX reader which didn’t set the reader index correctly was also fixed.

Result:

The PROXY protocol supports TLVs
2017-06-07 09:09:29 -07:00
Nikolay Fedorovskikh
b03b0f22d1 Removing a SeekAheadNoBackArrayException to avoid exception handling
Motivation:

A `SeekAheadNoBackArrayException` used as check for `ByteBuf#hasArray`. The catch of exceptions carries a large overhead on stack trace filling, and this should be avoided.

Modifications:

- Remove the class `SeekAheadNoBackArrayException` and replace its usage with `if` statements.
- Use methods from `ObjectUtils` for better readability.
- Make private methods static where it make sense.
- Remove unused private methods.

Result:

Less of exception handling logic, better performance.
2017-06-06 19:30:04 -07:00
Scott Mitchell
e06cb82c4c JdkZlibDecoder and JZlibDecoder consistency
Motivation:
JdkZlibDecoder will allocate a new buffer when the previous buffer is filled with inflated data, but JZlibDecoder will attempt to use the same buffer by resizing. This leads to inconsistent results when these two decoders that are intended to be functionality equivalent.

Modifications:
- JdkZlibDecoder should attempt to resize and reuse the existing buffer instead of creating multiple buffers

Result:
Fixes https://github.com/netty/netty/issues/6804
2017-06-06 14:18:10 -07:00
Norman Maurer
474bf036ff Update to new netty-tcnative release 2017-06-06 22:39:34 +02:00
Bryce Anderson
9fa3e556f3 Adjust Content-Length header when encoding Full Responses
Motivation:
If a full HttpResponse with a Content-Length header is encoded by the HttpContentEncoder subtypes the Content-Length header is removed and the message is set to Transfer-Encoder: chunked. This is an unnecessary loss of information about the message content.

Modifications:
- If a full HttpResponse has a Content-Length header, the header is adjusted after encoding.

Result:
Complete messages continue to have the Content-Length header after encoding.
2017-06-06 22:07:29 +02:00
Pavel Drankov
f8788a9f6c Issue №6802. Not specified field in MQTT codec (#6807) 2017-06-05 13:21:49 -07:00
Scott Mitchell
24f801c7d1 OpenSslEngine return NEED_WRAP if the destination buffered filled
Motivation:
If the destination buffer is completely filled during a call to OpenSslEngine#wrap(..) we may return NEED_UNWRAP because there is no data pending in the SSL buffers. However during a handshake if the SSL buffers were just drained, and filled up the destination buffer it is possible OpenSSL may produce more data on the next call to SSL_write. This means we should keep trying to call SSL_write as long as the destination buffer is filled and only return NEED_UNWRAP when the destination buffer is not full and there is no data pending in OpenSSL's buffers.

Modifications:
- If the handshake produces data in OpenSslEngine#wrap(..) we should return NEED_WRAP if the destination buffer is completely filled

Result:
OpenSslEngine returns the correct handshake status from wrap().
Fixes https://github.com/netty/netty/issues/6796.
2017-06-02 07:47:35 -07:00
Nikolay Fedorovskikh
270e9d66c5 Fixes in QueryStringDecoder
Motivation:

QueryStringDecoder has several problems:
- doesn't decode correctly path part with `+` (plus) sign in it,
- doesn't cut a `fragment` (after `#`) from query string (see RFC 3986),
- doesn't work correctly with encoding,
- treat `%%` as a percent character escaping (it's don't described in RFC).

Modifications:

- leave `+` chars in a `path` part of uri string,
- ignore `fragment` part (after `#`),
- correctly work with encoding.
- don't treat `%%` as escaping for the `%`.

Result:

Fixed issues from #6745.
2017-05-31 13:54:56 -07:00
Norman Maurer
4aa8002596 Not add ChannelHandler to ChannelPipeline once the pipeline was destroyed.
Motivation:

ChannelPipeline will happily add a handler to a closed Channel's pipeline and will call handlerAdded(...) but will not call handlerRemoved(...).

Modifications:

Check if pipeline was destroyed and if so not add the handler at all but propergate an exception.

Result:

Fixes [#6768]
2017-05-31 07:37:39 +02:00
Julien Viet
eee0ec3902 HttpObjectEncoder#isContentAlwaysEmpty cannot be overridden by subclasses
Motivation:

Allow subclasses of HttpObjectEncoder other than HttpServerCodec to override the isContentAlwaysEmpty method

Modification:

Change the method visibility from package private to protected

Result:

Fixes #6761
2017-05-31 07:36:12 +02:00
Norman Maurer
1504abd474 Correctly exclude netty-tcnative classes from the all jar.
Motivation:

We need to exclude the netty-tcnative classes from the all jar when building it as otherwise there may be classloader issues.

Modifications:

Add io/netty/internal/tcnative to the exclude list.

Result:

Fixes [#6791]
2017-05-30 16:00:01 -07:00
Scott Mitchell
b71abcedd1 ByteBufUtil#decodeHexDump
Motivation:
ByteBufUtil provides a hexDump method. For debugging purposes it is often useful to decode that hex dump to get the original content, but no such method exists.

Modifications:
- Add ByteBufUtil#decodeHexDump

Result:
ByteBufUtil#decodeHexDump is available to make debugging easier.
2017-05-30 15:20:54 -07:00
Scott Mitchell
742ee76424 RedisDecoder infinite loop
Motivation:
RedisDecoder can get into an infinite loop while decoding bulk strings if the final \r and \n to indicate the end of content are split on ByteBuf boundaries.

Modifications:
- We should break out of the decode loop if remainingBulkLength is 0 and we don't have enough data to read EOL

Result:
No more infinite loop in RedisDecoder#decodeBulkStringContent.
2017-05-30 08:53:36 -07:00
Norman Maurer
0b0309624a Correctly parse package-prefix if the jni lib is contained in a path that matches the library name.
Motivation:

We used strstr to find the path to the library, which fails if the library is contained in a directory that also matches the library name.

Modifications:

- Introduce netty_unix_util_strstr_last which will return a pointer which points to the last accourance and so not fails if the direct also matches the library name.

Result:

Be able to load the library in all cases.
2017-05-29 20:09:57 +02:00
Nolan O'Brien
d56a7560ea Netty force encodes already encoded responses
Motivation:

Fix the regression recently introduced that causes already encoded responses to be encoded again as gzip

Modification:

instead of just looking for IDENTITY, anything set for Content-Encoding should be respected and left as-is

added unit tests to capture this use case

Result:

Fixes #6784
2017-05-27 08:26:14 +02:00
louxiu
3c4dfed08a Fix handle of ByteBuf with multi nioBuffer in EpollDatagramChannel and KQueueDatagramChannel
Motivation:
1. special handling of ByteBuf with multi nioBuffer rather than type of CompositeByteBuf (eg. DuplicatedByteBuf with CompositeByteBuf)
2. EpollDatagramUnicastTest and KQueueDatagramUnicastTest passed because CompositeByteBuf is converted to DuplicatedByteBuf before write to channel
3. uninitalized struct msghdr will raise error

Modifications:
1. isBufferCopyNeededForWrite(like isSingleDirectBuffer in NioDatgramChannel) checks wether a new direct buffer is needed
2. special handling of ByteBuf with multi nioBuffer in EpollDatagramChannel, AbstractEpollStreamChannel, KQueueDatagramChannel, AbstractKQueueStreamChannel and IovArray
3. initalize struct msghdr

Result:
handle of ByteBuf with multi nioBuffer in EpollDatagramChannel and KQueueDatagramChannel are ok
2017-05-26 07:56:34 +02:00