Commit Graph

8445 Commits

Author SHA1 Message Date
Nikolay Fedorovskikh
c3bd1245c5 SocksCommonUtils cleanup
Motivation:

1. Internal class `SocksCommonUtils` contains a method `intToIp` that also exists in the `NetUtil`.
2. A `SocksCommonUtils#ipv6toCompressedForm` is never used.

Modifications:

1. Replace `intToIp` method usage with `NetUtil#intToIpAddress`.
2. Remove unused methods from `SocksCommonUtils`.

Result:

Less code for supports.
2017-06-23 19:04:03 +02:00
Nikolay Fedorovskikh
01eb428b39 Move methods for decode hex dump into StringUtil
Motivation:

PR #6811 introduced a public utility methods to decode hex dump and its parts, but they are not visible from netty-common.

Modifications:

1. Move the `decodeHexByte`, `decodeHexDump` and `decodeHexNibble` methods into `StringUtils`.
2. Apply these methods where applicable.
3. Remove similar methods from other locations (e.g. `HpackHex` test class).

Result:

Less code duplication.
2017-06-23 18:52:42 +02:00
Scott Mitchell
1767814a46 Replace DnsNameResolverContext#trace special code with an implementation of DnsQueryLifecycleObserver
Motivation:
DnsQueryLifecycleObserver is designed to capture the life cycle of every query. DnsNameResolverContext has a custom trace mechanism which consists of a StringBuilder and manual calls throughout the class. We can remove some special case code in DnsNameResolverContext and instead use a special implementation of DnsQueryLifecycleObserver when trace is enabled.

Modifications:
- Remove all references to the boolean trace variables in DnsNameResolverContext and DnsNameResolver
- Introduce TraceDnsQueryLifecycleObserver which will be used when trace is enabled and will log similar data as what trace currently provides

Result:
Less special case code in DnsNameResolverContext and instead delegate to TraceDnsQueryLifecycleObserver to capture trace information.
2017-06-23 09:04:59 -07:00
Scott Mitchell
1df722f65b kqueue version of 7baef4fbe8 2017-06-23 08:23:40 -07:00
Norman Maurer
58386aea43 Upgrade to netty-tcnative 2.0.4.Final 2017-06-23 13:43:51 +02:00
Carl Mastrangelo
cbde38f6e9 Add cause to thrown exception in SelfSignedCert
Motivation:
Exceptions with causes are easier to debug

Modification:
Add the cause when generating a SelfSignedCert

Results:
More debugging context
2017-06-23 07:24:11 +02:00
Carl Mastrangelo
7baef4fbe8 Move "fallthrough" statement to where fall actually happens Motivation: Static analysis looks for error prone switch case statements. Accidental fall through is one such case, but it is sometimes intentional. To indicate this, the "//fallthrough" comment can be added before the fall.
The code in question has this comment, but it is *after* the fall
so the static analysis flags it.

This is described in http://errorprone.info/bugpattern/FallThrough

Modifications:
Move fall through comment to where the fall actually occurs

Result:
More compatible with Error Prone tools
2017-06-23 07:22:47 +02:00
Carl Mastrangelo
b985615522 Fix compiler warnings in netty Epoll and unix common
Motivation:
Google requires stricter compilation by adding -Werror and enabling many other warnings.

Modification:

* fix warning caused by -Wmissing-braces

* Use the address of `sendmmsg` rather than the function itself when
checking for presence.  This resovles the warning caused by
`-Wpointer-bool-conversion`.

More detail:
When compiling on Linux, `sendmmsg` is always present, so the
function is always nonnull.  When compiling elsewhere, the
function is defined as `__attribute__((weak))` which means it
may be absent at link time.  This is controlled by
`IO_NETTY_SENDMMSG_NOT_FOUND`, which is off by default.

The reason for the error is due to the risk of accidentally not
calling the function.  By adding `&` before the function, there
is no ambiguity.  (the result of the fn call cannot have its
address taken.)

* use != to check for sendmmsg

Result:
Easier compilation.
2017-06-23 07:19:46 +02:00
Jason Tedor
9ad74e72e6 Remove content-length header leniency
Motivation:

If the content-length does not parse as a number, leniency causes this
to instead be parsed as the default value. This leads to bodies being
silently ignored on requests which can be incredibly dangerous. Instead,
if the content-length header is invalid, an exception should be thrown
for upstream handling.

Modifications:

This commit removes the leniency in parsing the content-length header by
allowing a number format exception, if thrown, to escape from the method
rather than falling back to the default value.

Result:

In invalid content-length header will not be silently ignored.
2017-06-22 09:20:11 -07:00
Scott Mitchell
6cd086050f DNS Resolver Search Domain Bugs
Motivation:
The DNS resolver supports search domains. However the ndots are not correctly enforced. The search domain should only be appended under the following scenario [1]:

> Resolver queries having fewer than ndots dots (default is 1) in them will be attempted using each component of the search path in turn until a match is found.

The DNS resolver current appends the search domains if ndots is 0 which should never happen (because no domain can have less than 0 dots).

[1] https://linux.die.net/man/5/resolv.conf

Modifications:
- Parse /etc/resolv.conf to get the default value for ndots on Unix platforms
- The search domain shouldn't be used if ndots is 0
- Avoid failing a promise to trigger the search domain queries in DnsNameResolverContext#resolve

Result:
More correct usage of search domains in the DNS resolver.
Fixes https://github.com/netty/netty/issues/6844.
2017-06-22 00:05:43 -07:00
Scott Mitchell
5934ae8fd2 Http2FrameLogger Updates
Motivation:
The Http2FrameLogger uses a custom format when logging events. We should use the more familiar format of 'channel event type: details' and single line logging for more consistent debugging.

Modifications:
- Http2FrameLogger should not use a StringBuilder and instead should directly use the Logger
- Http2FrameLogger should use the more consistent format defined above

Result:
Http2FrameLogger's logging formate is more consistent with other log events.
2017-06-21 17:12:38 -07:00
Nikolay Fedorovskikh
8f8be31226 Remove unnecessary conversions
Motivation:

In a `HttpConversionUtil#toHttp2Headers` a status code conversion can be replaced with using `HttpResponseStatus#codeAsText` method.

Modifications:

Apply `HttpResponseStatus#codeAsText` method.

Result:

Less allocations.
2017-06-21 21:53:48 +02:00
Scott Mitchell
6b25909f6b HTTP/2 HelloWorld clear text NPE
Motivation:
HelloWorldHttp2Handler throws a NPE when converting from HTTP/1.x headers to HTTP/2 headers because there is no Host header.

Modifications:
- HelloWorldHttp2Handler should check if the Host header is present before setting it in the HTTP/2 headers

Result:
No more NPE in HelloWorldHttp2Handler.
2017-06-21 06:55:28 +02:00
Norman Maurer
568fa998b1 Deploy transport-native-unix-common-tests
Motivation:

To be able to easily build only one of the native sub-modules its needed that all the dependencies can be fetched from maven. At the moment we dont deploy transport-native-unix-common and so an attempt to just build for example the native epoll transport fails with:

[ERROR] Failed to execute goal on project netty-transport-native-epoll: Could not resolve dependencies for project io.netty:netty-transport-native-epoll:jar:4.1.13.Final-SNAPSHOT: Could not find artifact io.netty:netty-transport-native-unix-common-tests:jar:4.1.13.Final-SNAPSHOT in sonatype-nexus-snapshots (https://oss.sonatype.org/content/repositories/snapshots) -> [Help 1]

Modifications:

Deploy jar

Result:

All dependencies on maven repository.
2017-06-21 06:52:18 +02:00
Scott Mitchell
6d029ad3ac OpenSSL CHACHA20 CipherSuiteConverter updates
Motivation:
For historical reasons OpenSSL's internal naming convention for CHACHA20 based cipher suites does not include the HMAC algorithm in the cipher name. This will prevent the CHACHA20 cipher suites from being used if the RFC cipher names are specified.

Modifications:
- Add a special case for CHACHA20 cipher name conversions in CipherSuiteConverter
- Update OPENSSL_CIPHERSUITE_PATTERN to accommodate the new naming scheme for CHACHA20 cipher suites

Result:
CipherSuiteConverter now works with CHACHA20 cipher suites.
2017-06-21 06:50:43 +02:00
Dmitriy Dumanskiy
81f9434ec7 Added test for multi header, HttpObjectDecoder performance improvement for multi header, removed empty else block.
Motivation:

For multi-line headers HttpObjectDecoder uses StringBuilder.append(a).append(b) pattern that could be easily replaced with regular a + b. Also oparations with a and b moved out from concat operation to make it friendly for StringOptimizeConcat optimization and thus - faster.

Modification:

StringBuilder.append(a).append(b) reaplced with a + b. Operations with a and b moved out from concat oparation.

Result:
Code simpler to read and faster.
2017-06-20 07:11:02 +02:00
Norman Maurer
82a43727c3 Fix testcase introduced by a2bd9a4 2017-06-20 07:01:13 +02:00
Scott Mitchell
14ea69cdc1 NullPointerException in Lz4FrameEncoder
Motivation:
Lz4FrameEncoder maintains internal state, but the life cycle of the buffer is not consistently managed. The buffer is allocated in handlerAdded but freed in close, but the buffer can still be used until handlerRemoved is called.

Modifications:
- Move the cleanup of the buffer from close to handlerRemoved
- Explicitly throw an EncoderException from Lz4FrameEncoder if the encode operation has finished and there isn't enough space to write data

Result:
No more NPE in Lz4FrameEncoder on the buffer.
2017-06-19 14:24:09 -07:00
Roger Kapsi
7460d90a67 Add listener to returned Future rather than passed in Promise
Motivation

It's cleaner to add listeners to returned Futures rather than provided Promises because the latter can have strange side effects in terms of listeners firing and called methods returning. Adding listeners preemtively may yield also to more OPS than necessary when there's an Exception in the to be called method.

Modifications

Add listener to returned ChannelFuture rather than given ChannelPromise

Result

Cleaner completion and exception handling
2017-06-16 13:21:19 +01:00
Norman Maurer
575baf5050 Use more aggressive expanding strategy in HpackHuffmanDecoder
Motivation:

Before we always expanded the buffer by the initialCapacity which by default is 32 bytes. This may lead to many expansions of the buffer until we finally reached the point that the buffer can fin everything.

Modifications:

Double the buffer size until the threshold of >= 1024 is hit. After this will grow it by the initialCapacity

Result:

Less expansion of the buffer (and so allocations / copies) when the intialCapacity is not big enough. Fixes [#6864].
2017-06-15 06:56:44 +02:00
Norman Maurer
e597756a56 Remove synchronized (ReferenceCountedOpenSslContext.class) blocks
Motivation:

We had some useless synchronized (ReferenceCountedOpenSslContext.class) blocks in our code which could slow down concurrent collecting and creating of ReferenceCountedOpenSslContext instances. Beside this we missed a few guards.

Modifications:

Use ReadWriteLock to correctly guard. A ReadWriteLock was choosen as SSL.newSSL(...) will be called from multiple threads all the time so using synchronized would be worse and there would be no way for the JIT to optimize it away

Result:

Faster concurrent creating and collecting of ReferenceCountedOpenSslContext instances and correctly guard in all cases.
2017-06-15 06:29:14 +02:00
Stephane Nicoll
2af895994d Add dependency management for missing entries
Motivation:

The bom does not provide entries for a number of netty modules, in
particular those that are deployed with classifiers. As a result, they
can't be used without defining a version.

Modifications:

Provide dependency management for the missing modules.

Result:

Fixes [#6852]
2017-06-14 20:26:33 +02:00
Nikolay Fedorovskikh
b8a418d53d Remove redundant code block in HttpPostRequestEncoder and make some cleanup
Motivation:

The class `HttpPostRequestEncoder` has minor issues:
- The `encodeNextChunkMultipart()` method contains two identical blocks of code with a difference only in the cast interfaces: `Attribute` vs `HttpData`. Because the `Attribute` is extended by `HttpData`, the block with the `Attribute` can be safely deleted.
- The `getNewMultipartDelimiter()` method contains a redundant `toLowerCase()`.
- The `addBodyFileUploads()` method throws `NPE` instead of `IllegalArgumentException`.

Modifications:

- Remove duplicated code block from `encodeNextChunkMultipart()`.
- Remove redundant `toLowerCase()` from `getNewMultipartDelimiter()`.
- Replace `NPE` with `IllegalArgumentException` in `addBodyFileUploads()`.
- Use `ObjectUtil#checkNotNull` where possible.

Result:

More correct and clean code.
2017-06-14 06:49:20 +02:00
Nikolay Fedorovskikh
aa38b6a769 Prevent unnecessary allocations in the StringUtil#escapeCsv
Motivation:

A `StringUtil#escapeCsv` creates new `StringBuilder` on each value even if the same string is returned in the end.

Modifications:

Create new `StringBuilder` only if it really needed. Otherwise, return the original string (or just trimmed substring).

Result:

Less GC load. Up to 4x faster work for not changed strings.
2017-06-13 14:57:38 -07:00
Norman Maurer
94c0ef3c96 Not fail the promise when a closed Channel is offered back to the ChannelPool
Motivation:

We should not fail the promise when a closed Channel is offereed back to the ChannelPool as we explicit mention that the Channel must always be returned.

Modifications:

- Not fail the promise
- Add test-case

Result:

Fixes [#6831]
2017-06-13 18:50:20 +02:00
Subhobrata Dey
66c83f7b74 codec.mqtt: password and willMessage field types should be byte[]
Motivation:

Update the mqtt-codec based on mqtt spec (3.1.3.5).

Modification:

Changes made to the file MqttConnectPayload.java.
Subsequent changes have been made to files MqttDecoder.java, MqttEncoder.java, MqttMessageBuilders.java.
Test cases have been updated.
Result:

Fixes #6750 .
2017-06-13 18:49:13 +02:00
Andy Wilkinson
ea2af3593c Correct bom entry for netty-transport-native-unix-common
Motivation:

The entry for the netty-transport-native-unix-common module in the bom
was using the wrong artifact ID and version.

Modifications:

Correct the artifact ID for the netty-transport-native-unix-common
module in the bom.

Result:

Fixes [#6849]
2017-06-13 15:47:36 +02:00
Stephane Landelle
ca5ed7c114 Let DnsNameResolver constructor use a default value for searchDomains
Motivation:

It’s currently complicated to extend `DnsNameResolver` as the default
value for `searchDomain` is package private.

Modifications:

* let `DnsNameResolver` accept a null `searchDomains` and then default
to `DEFAULT_SEARCH_DOMAINS`, just like it’s being done with
`resolvedAddressTypes`.
* set default `DnsNameResolverBuilder#searchDomains` value to null to
avoid cloning internal `DnsNameResolver.DEFAULT_SEARCH_DOMAINS` in
`DnsNameResolver` constructor.

Result:

More versatile `DnsNameResolver` constructor.
No array copy when using default search domains.
2017-06-13 15:43:08 +02:00
Norman Maurer
3f0085c267 Do proper bounds-checking in HpackHuffmanDecoder to reduce overhead of IndexOutOfBoundsException creation
Motivation:

HpackHuffmanDecoder.Decoder did not do any bound-checking but just catched IndexOutOfBoundsException to detect if the array needs to grow. This can be very expensive because of fillInStackTrace()

Modifications:

Add proper bounds checking and grow the array if needed without catching IndexOutOfBoundsException.

Result:

Less overhead if the array needs to grow.
2017-06-13 07:43:29 +02:00
Scott Mitchell
f00638af52 AbstractHttp2ConnectionHandlerBuilder support for HPACK huffman decoder initial size
Motivation:
Depending on the use case it may make sense to increase or decrease the initial size of the buffer used during the HPACK huffman decode process. This is currently not exposed through the AbstractHttp2ConnectionHandlerBuilder.

Modifications:
- Add a method to AbstractHttp2ConnectionHandlerBuilder which allows the initial size of the buffer used during the HPACK huffman decode prcoess to be configured.

Result:
AbstractHttp2ConnectionHandlerBuilder provides more control of codec-http2 knobs.
2017-06-12 16:36:43 -07:00
Scott Mitchell
1cc4607f07 AppendableCharSequence not to depend upon IndexOutOfBoundsException for resize
Motivation:
AppendableCharSequence depends upon IndexOutOfBoundsException to trigger a resize operation under the assumption that the resize operation will be rare if the initial size guess is good. However if the initial size guess is not good then the performance will be more unpredictable and likely suffer.

Modifications:
- Check the position in AppendableCharSequence#append to determine if a resize is necessary

Result:
More predictable performance in AppendableCharSequence#append.
2017-06-12 12:42:20 -07:00
Rogan Dawes
051e0ad4be Add support for IP_TRANSPARENT socket option
Motivation:

This allows netty to operate in 'transparent proxy' mode, intercepting connections
to other addresses by means of Linux firewalling rules, as per

https://www.kernel.org/doc/Documentation/networking/tproxy.txt

The original destination address can be obtained by referencing
ch.localAddress().

Modification:

Add methods similar to those for ipFreeBind, to set the IP_TRANSPARENT option.

Result:

Allows setting and getting of the IP_TRANSPARENT option, which allows retrieval of the ultimate socket address originally requested.
2017-06-12 08:03:39 +02:00
Norman Maurer
b6c27b9f6b Not force to run autoconf and compile multiple times
Motivation:

We should not force autoconf and compile as this will result in multiple executions and so slow down the build.

Modifications:

Remove force declarations

Result:

Faster build of native modules
2017-06-09 20:34:28 +02:00
Dmitriy Dumanskiy
acc07fac32 disabling leak detection micro benchmark
Motivation:

When I run Netty micro benchmarks I get many warnings like:

WARNING: -Dio.netty.noResourceLeakDetection is deprecated. Use '-Dio.netty.leakDetection.level=simple' instead.

Modification:

-Dio.netty.noResourceLeakDetection replaced with -Dio.netty.leakDetection.level=disabled.

Result:

No warnings.
2017-06-09 18:03:54 +02:00
Norman Maurer
fd67a2354d [maven-release-plugin] prepare for next development iteration 2017-06-08 21:06:24 +02:00
Norman Maurer
3acd5c68ea [maven-release-plugin] prepare release netty-4.1.12.Final 2017-06-08 21:06:01 +02:00
Norman Maurer
80aa5dcdcc Revert "Not add ChannelHandler to ChannelPipeline once the pipeline was destroyed."
This reverts commit 4aa8002596.
2017-06-08 19:50:17 +02:00
Norman Maurer
f208b147a6 Use FQCN to prevent classloader issues on java6
Motivation:

We need to use FQCN to prevent classloader issues for classes that are > Java6. This is a cleanup of ed5fcbb773.

Modifications:

Just remove the imports and use FQCN.

Result:

No classloader issues with java6
2017-06-08 12:04:17 +02:00
Norman Maurer
cdf3acb6a2 Update to latest netty-tcnative release 2017-06-07 20:06:55 +02:00
Norman Maurer
047da11086 Correctly handle ByteBuf implementations which have no memoryAddress when writing to native transport
Motivation:

Commit 3c4dfed08a introduced a regression in handling buffers that have no memoryAddress.

Modifications:

Fix regression and also add unit tests.

Result:

It's possible again to write buffers without memory address.
2017-06-07 19:36:05 +02:00
Norman Maurer
7922757575 Allow to access memoryAddress of wrapped ByteBuf for ReadOnlyByteBuf
Motivation:

We should allow to access the memoryAddress of the wrapped ByteBuf when using ReadOnlyByteBuf for peformance reasons. If a user act on a memoryAddress its his responsible anyway to do nothing "stupid".

Modifications:

Delegate to wrapped ByteBuf.

Result:

Less performance overhead for various operations and also when writing to a native transport (which needs the memoryAddress).
2017-06-07 18:45:26 +02:00
Renjie Sun
629b83e0a5 Move QueryStringDecoder.decodeHexByte into ByteBufUtil
Motivations:
1. There are duplicated implementations of decoding hex strings. #6797
2. ByteBufUtil.HexUtil.decodeHexDump does not handle substring start
index properly and does not decode hex byte rigorously.

Modifications:
1. Function decodeHexByte is moved from QueryStringDecoder into ByteBufUtil.
2. ByteBufUtil.HexUtil.decodeHexDump is changed to use decodeHexByte.
3. Tests are Updated accordingly.

Result:
Fixed #6797 and made hex decoding functions more robust.
2017-06-07 09:27:36 -07:00
Michael K. Werle
ed5fcbb773 Add explicit message when noexec prevents library loading.
Motivation:

Docker's `--tmpfs` flag mounts the temp volume with `noexec` by default,
resulting in an UnsatisfiedLinkError.  While this is good security
practice, it is a surprising failure from a seemingly innocuous flag.

Modifications:

Add a best-effort attempt in `NativeLibraryLoader` to detect when temp
files beng loaded cannot be executed even when execution permissions
are set, often because the `noexec` flag is set on the volume.

Requires numerous additional exclusions to the Animal Sniffer config
for Java7 POSIX permissions manipulation.

Result:

Fixes [#6678].
2017-06-07 09:20:05 -07:00
Georg Held
1f0d47dee7 Added PROXY Protocol TLV support
Motivation:

The current PROXY protocol implementation does not have support for optional Type-Length-Value fields. This pull requests adds the TLV values as specified in the PROXY protocol specification (http://www.haproxy.org/download/1.5/doc/proxy-protocol.txt) and adds support for arbitrary TLVs.

Modifications:

The existing HAProxyMessage implements an additional TLV reading operation. A small bug in the AF_UNIX reader which didn’t set the reader index correctly was also fixed.

Result:

The PROXY protocol supports TLVs
2017-06-07 09:09:29 -07:00
Nikolay Fedorovskikh
b03b0f22d1 Removing a SeekAheadNoBackArrayException to avoid exception handling
Motivation:

A `SeekAheadNoBackArrayException` used as check for `ByteBuf#hasArray`. The catch of exceptions carries a large overhead on stack trace filling, and this should be avoided.

Modifications:

- Remove the class `SeekAheadNoBackArrayException` and replace its usage with `if` statements.
- Use methods from `ObjectUtils` for better readability.
- Make private methods static where it make sense.
- Remove unused private methods.

Result:

Less of exception handling logic, better performance.
2017-06-06 19:30:04 -07:00
Scott Mitchell
e06cb82c4c JdkZlibDecoder and JZlibDecoder consistency
Motivation:
JdkZlibDecoder will allocate a new buffer when the previous buffer is filled with inflated data, but JZlibDecoder will attempt to use the same buffer by resizing. This leads to inconsistent results when these two decoders that are intended to be functionality equivalent.

Modifications:
- JdkZlibDecoder should attempt to resize and reuse the existing buffer instead of creating multiple buffers

Result:
Fixes https://github.com/netty/netty/issues/6804
2017-06-06 14:18:10 -07:00
Norman Maurer
474bf036ff Update to new netty-tcnative release 2017-06-06 22:39:34 +02:00
Bryce Anderson
9fa3e556f3 Adjust Content-Length header when encoding Full Responses
Motivation:
If a full HttpResponse with a Content-Length header is encoded by the HttpContentEncoder subtypes the Content-Length header is removed and the message is set to Transfer-Encoder: chunked. This is an unnecessary loss of information about the message content.

Modifications:
- If a full HttpResponse has a Content-Length header, the header is adjusted after encoding.

Result:
Complete messages continue to have the Content-Length header after encoding.
2017-06-06 22:07:29 +02:00
Pavel Drankov
f8788a9f6c Issue №6802. Not specified field in MQTT codec (#6807) 2017-06-05 13:21:49 -07:00
Scott Mitchell
24f801c7d1 OpenSslEngine return NEED_WRAP if the destination buffered filled
Motivation:
If the destination buffer is completely filled during a call to OpenSslEngine#wrap(..) we may return NEED_UNWRAP because there is no data pending in the SSL buffers. However during a handshake if the SSL buffers were just drained, and filled up the destination buffer it is possible OpenSSL may produce more data on the next call to SSL_write. This means we should keep trying to call SSL_write as long as the destination buffer is filled and only return NEED_UNWRAP when the destination buffer is not full and there is no data pending in OpenSSL's buffers.

Modifications:
- If the handshake produces data in OpenSslEngine#wrap(..) we should return NEED_WRAP if the destination buffer is completely filled

Result:
OpenSslEngine returns the correct handshake status from wrap().
Fixes https://github.com/netty/netty/issues/6796.
2017-06-02 07:47:35 -07:00