Commit Graph

9068 Commits

Author SHA1 Message Date
Karl Pauls
5256d17363 Make osgi testing work with java>=9 by updating to felix framework 5.6.10 (#7923)
Motivation:

Currently, the testing-osgi is set to skip if run with java>=9. That is not necessary when using a newer version of Felix.

Modification:

Update to Felix framework 5.6.10 (which has better jpms support), add some more --add-opens to not have WARN messages, and remove the skipOsgiTestsuite setting from the parent pom.

Result:

OSGi tests run and pass on java>=9.
2018-05-09 18:26:38 +02:00
Norman Maurer
369d64c3e6
Always follow cnames even if a matching A or AAAA record was found. (#7919)
Motivation:

At the moment if you do a resolveAll and at least one A / AAAA record is present we will not follow any CNAMEs that are also present. This is different to how the JDK behaves.

Modifications:

- Allows follow CNAMEs.
- Add unit test.

Result:

Fixes https://github.com/netty/netty/issues/7915.
2018-05-09 13:48:20 +02:00
Norman Maurer
cbe9ed8cc1
Add test that we handle thread.interrupt() in NioEventLoop (#7917)
Motivation:

We added some code to guard against thread.interrupt() in NioEventLoop but did not added a test.

Modifications:

Add testcase.

Result:

Verify that we correctly handle interrupt().
2018-05-09 08:57:53 +02:00
Norman Maurer
f01a590154
Enfore upper limit for minTtl when using DefaultCacheEntry. (#7920)
Motivation:

a598c3b69b added a upper limit for ttl but missed to also do the same for minTtl.

Modifications:

- Add upper limit for minTtl
- Add testcase.

Result:

No more IllegalArgumentException possible.
2018-05-09 08:57:20 +02:00
Xiaoyan Lin
a0ed6ec06c Fix the error message in ReferenceCounted.release (#7921)
Motivation:

When a buffer is over-released, the current error message of `IllegalReferenceCountException` is `refCnt: XXX, increment: XXX`, which is confusing. The correct message should be `refCnt: XXX, decrement: XXX`.

Modifications:

Pass `-decrement` to create `IllegalReferenceCountException`.

Result:

The error message will be `refCnt: XXX, decrement: XXX` when a buffer is over-released.
2018-05-08 20:09:16 +02:00
Konstantin Lutovich
437849bd93 Fix tiny typo in FixedChannelPool error message (#7918)
Motivation:

Closed `FixedChannelPool` fails acquire and release operations with
`IllegalStateException`s. These exceptions had message
"FixedChannelPooled was closed". Here "FixedChannelPooled" looks like
a typo and should probably be "FixedChannelPool".

Modifications:

Changed exception message to "FixedChannelPool was closed".

Result:

A tiny bit cleaner exception message.
2018-05-08 15:58:23 +02:00
Norman Maurer
18b170bd7a
Cleanup resolver-dns test code. (#7916)
Motivation:

We had some stuff that could be cleaned up in the resolver-dns test classes.

Modifications:

Cleanup code.

Result:

Cleaner code.
2018-05-08 08:55:17 +02:00
Norman Maurer
54c3de0f8a
Add test to validate SSLEngine does not zero out src buffer on wrap. (#7914)
Motivation:

We had a bug-report that claimed the src buffer used by OpenSslEngine will be zero out.

Modifications:

Add testcase to ensure correct behaviour

Result:

Testcase for https://github.com/netty/netty/issues/7753
2018-05-07 20:10:06 +02:00
Norman Maurer
d03bd44e9d
Add test for caching failed queries in DefaultDnsCache. (#7909)
Motivation:

We had no test that validated the handling of caching failures for DefaultDnsCache.

Modifications:

Add testcase.

Result:

More tests FTW.
2018-05-04 20:09:38 +02:00
Norman Maurer
358249e5c9
Allow to disable native transport and native ssl support via system property. (#7903)
Motivation:

Sometimes it's useful to disable native transports / native ssl to debug a problem. We should allow to do so with a system property so people not need to adjust code for this.

Modifications:

Add system properties which allow to disable native transport and native ssl.

Result:

Easier to disable native code usage without code changes.
2018-05-04 14:44:44 +02:00
Norman Maurer
b83328606f
DefaultDnsCache should store more then one Entry per hostname. (#7906)
Motivation:

Due a bug we did never store more then one address per hostname in DefaultDnsCache.

Modifications:

- Correctly store multiple entries per hostname
- Add tests

Result:

DefaultDnsCache correctly stores more then one entry. Also fixes https://github.com/netty/netty/issues/7882 .
2018-05-04 14:19:26 +02:00
Norman Maurer
a598c3b69b
Enforce sane upper limit for TTL in DefaultDnsCache. (#7907)
Motivation:

In b47fb81799 we limited the max supported delay to match what our internal implementat can support. Because of this it was possible that DefaultDnsCache produced an IllegalArgumentException when it tried to schedule a expiration > 3 years.

Modifications:

Limit the max supported TTL to 2 years which is safe for all our EventLoop implementations.

Result:

No more exceptions when adding records to the cache with a huge TTL.
2018-05-04 13:38:08 +02:00
Norman Maurer
095dadcbec
Fix race in ChannelReadHandler used during LocalChannel testing. (#7904)
Motivation:

ChannelReadHandler is used in tests added via f4d7e8de14. In the handler we verify the number of messages we receive per read() call but missed to sometimes reset the counter which resulted in exceptions.

Modifications:

Correctly reset read counter in all cases.

Result:

No more unexpected exceptions when running LocalChannel tests.
2018-05-04 07:36:33 +02:00
Norman Maurer
c60263e8a3
Correctly handle responses with status 205 and payload. (#7891)
Motivation:

HTTP responses with status of 205 should not contain a payload. We should enforce this.

Modifications:

Correctly handle responses with status 205 and payload by set Content-Length: 0 header and stripping out the content.

Result:

Fixes https://github.com/netty/netty/issues/7888
2018-05-03 11:25:06 +02:00
Daniel Schobel
bd800fa7e7 Add null-check to Htt2FrameCodec#consumeBytes. (#7899)
Motivation:

Streams can be deregistered so we can't assume their existence in the stream map.

Modifications:

Add a null-check in case a stream has been deregistered.

Result:

Fixes #7898.
2018-05-02 13:01:26 +02:00
Norman Maurer
0261e00662
Allow to call AbstractDiskHttpData.setContent(Unpooled.EMPTY_BUFFER) multiple times. (#7890)
Motivation:

It should be possible to call setContent(Unpooled.EMPTY_BUFFER) multiple times just like its possible to do the same with a non empty buffer.

Modifications:

- Correctly reset underlying storage if called multiple times.
- Add tests

Result:

Fixes https://github.com/netty/netty/issues/6418
2018-04-30 08:39:24 +02:00
Norman Maurer
b5fde3e0cd
Add #ifdef statements so the compilation of the native code not fails on MacOS (#7895)
Motivation:

b818852cdb added support for IP_RECVORIGDSTADDR but did not include any #ifdef statements to ensure its usable at all (which is not the case on MacOS).

Modifications:

Add #ifdef statements to check for IP_RECVORIGDSTADDR.

Result:

Compilation works again on MacOS.
2018-04-28 22:04:35 +02:00
Norman Maurer
dfeb4b15b5
Fix IllegalReferenceCountException when using Http2MultiplexCodec and a DefaultHttp2GoAwayFrame with a non empty ByteBuffer is received. (#7894)
Motivation:

We incorrectly called frame.release() in onHttp2GoAwayFrame which could lead to IllegalReferenceCountExceptions.  The call of release() is inappropriate because the fireChannelRead() in onHttp2Frame() will handle it.

Modifications:

- Not call frame.release()
- Add a unit test

Result:

Fxies https://github.com/netty/netty/issues/7892.
2018-04-28 22:03:49 +02:00
Alexey Kachayev
c0ae5e697c DnsAddressResolverGroup to use pluggable DnsNameResolverBuilder (#7793)
Motivation:

Right now to customize DNS name resolver when using DnsAddressResolverGroup
one should subclass implementation and override newNameResolver method when
in fact it's possible to collect all settings in a DnsNameResolverBuilder
instance. Described in #7749.

Modifications:

- Added new constructor for DnsNameResolverBuilder in order to delay
  EventLoop specification

- Added copy() method to DnsNameResolverBuilder to provide an immutable
  copy of the builder

- Added new single-argument constructor for DnsAddressResolverGroup and
  RoundRobinDnsAddressResolverGroup accepting DnsNameResolverBuilder
  instance

- DnsAddressResolverGroup to build a new resolver using DnsNameResolverBuilder
  given instead of creating a new one

- Test cases to check that changing channelFactory after the builder was passed
  to create a DnsNameResolverGroup would not propagate to the name resolver

Result:

Much easier to customize DNS settings w/o subclassing DnsAddressResolverGroup
2018-04-26 08:04:01 +02:00
Devrim Åžahin
b818852cdb Update DatagramPacket.recipient() to return the actual destination IP (#7879)
Motivation:

DatagramPacket.recipient() doesn't return the actual destination IP, but the IP the app is bound to.

Modification:

- IP_RECVORIGDSTADDR option is enabled for UDP sockets, which allows retrieval of ancillary information containing the original recipient.
- _recvFrom(...) function from transport-native-unix-common/src/main/c/netty_unix_socket.c is modified such that if IP_RECVORIGDSTADDR is set, recvmsg is used instead of recvfrom; enabling the retrieval of the original recipient.
- DatagramSocketAddress also contains a 'local' address, representing the recipient.
- EpollDatagramChannel is updated to return the retrieved recipient address instead of the address the channel is bound to.

Result:

Fixes #4950.
2018-04-26 08:00:36 +02:00
Norman Maurer
f4d7e8de14
Respect MAX_MESSAGES_PER_READ in LocalChannel / LocalServerChannel. (#7885)
Motivation:

LocalChannel / LocalServerChannel did not respect read limits and just always read all of the messages.

Modifications:

- Correct respect MAX_MESSAGES_PER_READ settings
- Add unit tests

Result:

Fixes https://github.com/netty/netty/issues/7880.
2018-04-26 07:58:56 +02:00
Norman Maurer
eaf1771336
Don't use VM.maxDirectMemory() on z/OS to retrieve direct memory limit. (#7886)
Motivation:

On z/OS netty initializes this value with 64M, even the direct accessible memory is actually unbounded.

Modifications:

Skip usage of VM.maxDirectMemory() on z/OS.

Result:

More correct direct memory limit detection. Fixes https://github.com/netty/netty/issues/7654.
2018-04-25 07:33:07 +02:00
Norman Maurer
b47fb81799
EventLoop.schedule with big delay fails (#7402)
Motivation:

Using a very huge delay when calling schedule(...) may cause an Selector error when calling select(...) later on. We should gaurd against such a big value.

Modifications:

- Add guard against a very huge value.
- Added tests.

Result:

Fixes [#7365]
2018-04-24 11:15:20 +02:00
Robin Wang
010dbe3c73 Optimize space usage of FastThreadLocal (#7861)
Motivation:
A FastThreadLocal instance currently occupies 2 slots of InternalThreadLocalMap of every thread. Actually for a FastThreadLocalThread, it does not need to store cleaner flags of FastThreadLocal instances. Besides, using BitSet to store cleaner flags is also better for space usage.

Modification:
Use BitSet to optimize space usage of FastThreadLocal.

Result:
Avoid unnecessary slot occupancy. Cleaner flags are now stored into a BitSet.
2018-04-24 09:19:01 +02:00
Rikki Gibson
1b1f7677ac Implement isWritable and ensureWritable on ReadOnlyByteBufferBuf (#7883)
Motivation:

It should be possible to write a ReadOnlyByteBufferBuf to a channel without errors. However, ReadOnlyByteBufferBuf does not override isWritable and ensureWritable, which can cause some handlers to mistakenly assume they can write to the ReadOnlyByteBufferBuf, resulting in ReadOnlyBufferException.

Modification:

Added isWritable and ensureWritable method overrides on ReadOnlyByteBufferBuf to indicate that it is never writable. Added tests for these methods.

Result:

Can successfully write ReadOnlyByteBufferBuf to a channel with an SslHandler (or any other handler which may attempt to write to the ByteBuf it receives).
2018-04-23 08:31:30 +02:00
Trustin Lee
48fe14402a Update os-maven-plugin to 1.6.0 (#7884)
Motivation:

A new version of os-maven-plugin has been released.

Modifications:

- Update os-maven-plugin from 1.5.0 to 1.6.0

Result:

- No visible changes
- Feels good
2018-04-23 08:30:42 +02:00
Bryce Anderson
f9604eeff5 Motivation: (#7848)
It is possible to create streams in the half-closed state where the
stream state doesn't reflect that the request headers have been sent by
the client or the server hasn't received the request headers. This
state isn't possible in the H2 spec as a half closed stream must have
either received a full request or have received the headers from a
pushed stream. In the current implementation, this can cause the stream
created as part of an h2c upgrade request to be in this invalid state
and result in the omission of RST frames as the client doesn't believe
it has sent the request to begin with.

Modification:

The `DefaultHttp2Connection.activate` method checks the state and
modifies the status of the request headers as appropriate.

Result:

Fixes #7847.
2018-04-21 08:23:15 +02:00
Norman Maurer
b75f44db9a [maven-release-plugin] prepare for next development iteration 2018-04-19 11:56:07 +00:00
Norman Maurer
04fac00c8c [maven-release-plugin] prepare release netty-4.1.24.Final 2018-04-19 11:55:47 +00:00
Anuraag Agrawal
4cd39cc4b3 Filter DNS results so they only contain the expected type when multiple types are present. (#7875)
Motivation:

Currently, if a DNS server returns a non-preferred address type before the preferred one, then both will be returned as the result, and when only taking a single one, this usually ends up being the non-preferred type. However, the JDK requires lookups to only return the preferred type when possible to allow for backwards compatibility.

To allow a client to be able to resolve the appropriate address when running on a machine that does not support IPv6 but the DNS server returns IPv6 addresses before IPv4 addresses when querying.

Modification:

Filter the returned records to the expected type when both types are present.

Result:

Allows a client to run on a machine with IPv6 disabled even when a server returns both IPv4 and IPv6 results. Netty-based code can be a drop-in replacement for JDK-based code in such circumstances.

This PR filters results before returning them to respect JDK expectations.
2018-04-19 10:52:22 +02:00
Norman Maurer
3ec29455af
Fix eternal loop in Recycler.WeakOrderQueue.Head#run() that blocks ObjectCleaner thread (#7878)
Motivation:

When trying to cleanup WeakOrderQueue by the ObjectCleaner we end up in an endless loop which will cause the ObjectCleaner to be not able to cleanup any other resources anymore.This bug was introduced by 6eb9674bf5.

Modifications:

Correctly update link while cleanup

Result:

Fixes https://github.com/netty/netty/issues/7877
2018-04-19 09:40:38 +02:00
Norman Maurer
0f34d887b7
Only reset readIsPending if outboundBuffer is not empty. (#7874)
Motivation:

We need to ensure we only reset readInProgress if the outboundBuffer is not empty as otherwise we may miss to call fireChannelRead(...) later on when using the LocalChannel.

Modifications:

Also check if the outboundBuffer is not empty before setting readInProgress to false again

Result:

Fixes https://github.com/netty/netty/issues/7855
2018-04-19 09:39:16 +02:00
Devrim Åžahin
0b690a991f Added IP_TRANSPARENT option for UDP (#7872)
Motivation:

This allows netty to operate in 'transparent proxy' mode for UDP, intercepting connections
to other addresses by means of Linux firewalling rules, as per

https://www.kernel.org/doc/Documentation/networking/tproxy.txt


Modification:

Add IP_TRANSPARENT option.

Result:

Allows setting and getting of the IP_TRANSPARENT option, which allows retrieval of the ultimate socket address originally requested.
2018-04-17 09:07:02 +02:00
JLofgren
a4146b706c Do not enforce arbitrary max header list size in HpackEncoder (#7853)
Motivation:

When connecting to an HTTP/2 server that did not set any value for the
SETTINGS_MAX_HEADER_LIST_SIZE in the settings frame, the netty client was
imposing an arbitrary maximum header list size of 8kB. There should be no need
for the client to enforce such a limit if the server has not specified any
limit. This caused an issue for a grpc-java client that needed to send a large
header to a server via an Envoy proxy server. The error condition is
demonstrated here: https://github.com/JLofgren/demo-grpc-java-bug-4284

Fixes grpc-java issue #4284 - https://github.com/grpc/grpc-java/issues/4284
and netty issue #7825 - https://github.com/netty/netty/issues/7825

Modifications:

In HpackEncoder use MAX_HEADER_LIST_SIZE as default maxHeader list size.

Result:

HpackEncoder will only enforce a max header list size if the server has
specified a limit in its settings frame.
2018-04-16 14:27:36 -07:00
Nikolay Fedorovskikh
81a7d1413b Makes EmptyByteBuf#hashCode and AbstractByteBuf#hashCode consistent (#7870)
Motivation:
The `AbstractByteBuf#equals` method doesn't take into account the
class of buffer instance. So the two buffers with different classes
must have the same `hashCode` values if `equals` method returns `true`.
But `EmptyByteBuf#hashCode` is not consistent with `#hashCode`
of the empty `AbstractByteBuf`, that is violates the contract and
can lead to errors.

Modifications:
Return `1` in `EmptyByteBuf#hashCode`.

Result:
Consistent behavior of `EmptyByteBuf#hashCode` and `AbstractByteBuf#hashCode`.
2018-04-16 12:11:42 +02:00
Gustavo Fernandes
f874a37ecb Fixes NPE in Corshandler for unauthorized prefligt requests (#7865)
Motivation:
NPE in `CorsHandler` if a pre-flight request is done using an Origin header which is not allowed by any `CorsConfig` passed to the handler on creation.

Modifications:
During the pre-flight, check the `CorsConfig` for `null` and handle it correctly by not returning any access-control header

Result:
No more NPE for pre-flight requests with unauthorized origins.
2018-04-13 14:36:45 +02:00
Norman Maurer
da2e91b33a
Allow to write Http2UnkownFrame when using Http2FrameCodec / Http2MultiplexCodec (#7867)
Motivation:

We should allow to write Http2UnkownFrame to allow custom extensions.

Modifications:

Allow to write Http2UnkownFrame
Add unit test

Result:

Fixes https://github.com/netty/netty/issues/7860.
2018-04-13 07:50:44 +02:00
Scott Mitchell
3f244a94fe
AsciiString#indexOf out of bounds (#7866)
Motivation:
The bounds checking for AsciiString#indexOf and AsciiString#lastIndexOf is not correct and may lead to ArrayIndexOutOfBoundsException.

Modifications:
- Correct the bounds checking for AsciiString#indexOf and AsciiString#lastIndexOf

Result
Fixes https://github.com/netty/netty/issues/7863
2018-04-12 12:06:12 -07:00
Nikolay Fedorovskikh
401b196623 Extract common parts from if statements (#7831)
Motivation:
Some `if` statements contains common parts that can be extracted.

Modifications:
Extract common parts from `if` statements.

Result:
Less code and bytecode. The code is simpler and more clear.
2018-04-11 14:36:56 +02:00
Gustavo Fernandes
76c5f6cd03 Enable per origin Cors configuration (#7800)
Motivation:

Finer granularity when configuring CorsHandler, enabling different policies for different origins.

Modifications:

The CorsHandler has an extra constructor that accepts a List<CorsConfig> that are evaluated sequentially when processing a Cors request

Result:

The changes don't break backwards compatibility. The extra ctor can be used to provide more than one CorsConfig object.
2018-04-11 10:06:13 +02:00
Nikolay Fedorovskikh
f8ff834f03 Checks accessibility in the #slice and #duplicate methods of ByteBuf (#7846)
Motivation:
The `ByteBuf#slice` and `ByteBuf#duplicate` methods should check
an accessibility to prevent creation slice or duplicate
of released buffer. At now this works not in the all scenarios.

Modifications:
Add missed checks.

Result:
More correct and consistent behavior of `ByteBuf` methods.
2018-04-10 10:41:50 +02:00
Norman Maurer
5ab8342798
Add dev-tools dependency for commons (#7858)
Motivation:

We need to add a dev-tools dependecy for commons as otherwise we may fail to fetch it before we try to use it.

Modifications:

Add dependency.

Result:

Fixes https://github.com/netty/netty/issues/7842
2018-04-10 10:37:02 +02:00
Nikolay Fedorovskikh
587afddb27 Fixes NPE in ClientCookieDecoder
Motivation:
NPE in `ClientCookieDecoder` if cookie starts with comma.

Modifications:
Check `cookieBuilder` for `null` in the return.

Result:
No fails NPE on invalid cookies.
2018-04-05 19:44:08 +02:00
ikurovsky
c58069f284 Better handling of streaming JSON data in JsonObjectDecoder (#7821)
Motivation:

When the JsonObjectDecoder determines that the incoming buffer had some data discarded, it resets the internal index to readerIndex and attempts to adjust the state which does not correctly work for streams of JSON objects.

Modifications:

Reset the internal index to the value considering the previous reads.

Result:

JsonObjectDecoder correctly handles streams of both JSON objects and arrays with no state adjustments or repeatable reads.
2018-04-05 07:58:14 +02:00
Dave Moten
0f4001d598 add task before starting thread in SingleThreadEventExecutor.execute (#7841)
Motivation:

Minor performance optimisation that prevents thread from blocking due to task not having been added to queue. Discussed #7815.

Modification:

add task to the queue before starting the thread.

Result:

No additional tests.
2018-04-05 07:57:21 +02:00
root
0a61f055f5 [maven-release-plugin] prepare for next development iteration 2018-04-04 10:44:46 +00:00
root
8c549bad38 [maven-release-plugin] prepare release netty-4.1.23.Final 2018-04-04 10:44:15 +00:00
Nikolay Fedorovskikh
a95fd91bc6 Don't check accessible in the #capacity method (#7830)
Motivation:
The `#ensureAccessible` method in `UnpooledHeapByteBuf#capacity` used
to prevent NPE if buffer is released and `array` is `null`. In all
other implementations of `ByteBuf` the accessible is not checked by
`capacity` method. We can assign an empty array to `array`
in the `deallocate` and don't worry about NPE in the `#capacity`.
This will help reduce the number of repeated calls of the
`#ensureAccessible` in many operations with `UnpooledHeapByteBuf`.

Modifications:
1. Remove `#ensureAccessible` call from `UnpooledHeapByteBuf#capacity`.
Use the `EmptyArrays#EMPTY_BYTES` instead of `null` in `#deallocate`.

2. Fix access checks in `AbstractUnsafeSwappedByteBuf` and
`AbstractByteBuf#slice` that relied on `#ensureAccessible`
in `UnpooledHeapByteBuf#capacity`. This was found by unit tests.

Result:
Less double calls of `#ensureAccessible` for `UnpooledHeapByteBuf`.
2018-04-03 21:35:02 +02:00
Norman Maurer
ee9057ad99
CorsHandler.write(...) should not cause a flush. (#7839)
Motivation:

Unnecessary flushes reduce the amount of flush coalescing that can happen at higher levels and thus can increase number of packets (because of TCP_NODELAY) and lower throughput (due to syscalls, TLS frames, etc)

Modifications:

Replace writeAndFlush(...) with write(...)

Result:

Fixes https://github.com/netty/netty/issues/7837.
2018-04-03 21:11:51 +02:00
Norman Maurer
473f6a6edb
SctpMessageCompletionHandler may leak ByteBuf for fragmented messages. (#7832)
Motivation:

SctpMessageCompletionHandler stores fragments in a Map but not release the stored ByteBuf when the handler is removed.

Modifications:

Release all buffers that are still in the Map when handlerRemoved(...) is called.

Result:

No more leaks for fragemented messages.
2018-04-02 21:37:03 +02:00