Commit Graph

10068 Commits

Author SHA1 Message Date
feijermu
f66412c84c
Dequeue all cached messages and destroy the queue instance after removing the FlowControlHandler from channel pipeline. (#10304)
Motivation:

The `FlowControlHandler` may cache the received messages in a queue in order to do the flow control. However, if this handler is manually removed from pipeline during runtime, those cached messages might not be passed to the next channel handler forever.

Modification:

Dequeue all these cached messages and call `ChannelHandlerContext.fireChannelRead(...)` in `handlerRemoved(...)` method.

Result:
Avoid losing the received messages.
2020-05-19 09:46:12 +02:00
Norman Maurer
d1b99b702c
Use allocation free algorithm to detect CNAME cache loops (#10291)
Motivation:

We did use a HashSet to detect CNAME cache loops which needs allocations. We can use an algorithm that doesnt need any allocations

Modifications:

Use algorithm that doesnt need allocations

Result:

Less allocations on the slow path
2020-05-18 14:28:30 +02:00
Idel Pivnitskiy
877db52e37
Do not require BoringSSL for testSessionTicketsWithTLSv12AndNoKey (#10301)
Motivation:

`SslHandlerTest.testSessionTicketsWithTLSv12AndNoKey` does not require
BoringSSL and works with OpenSSL as well.

Modifications:

- Remove assume statement that expected BoringSSL;

Result:

Test works for any implementation of `OPENSSL` provider.
2020-05-18 14:25:04 +02:00
Norman Maurer
1b6595b358
Check if SSL pointer was freed before using it in RefereceCountedOpenSslEngine in all cases (#10299)
Motivation:

To ensure we not crash in all cases we should better check that the SSL pointer was not freed before using it.

Modifications:

Add missing `isDestroyed()` checks

Result:

Ensure we not crash due usage of freed pointer.
2020-05-18 09:40:31 +02:00
Robert Varga
bdbf68f988
Include aarch64 packages in netty-bom (#10292)
Motivation:

The linux-aarch64 packages are not declared in netty-bom. There are no consistency checks for netty bom, hence it can easily miss updates when artifacts are added.

Modifications:

- Add declarations.
- Modify netty-all to depend on netty-bom for version declarations,
thus requiring netty-bom to be uptodate.

Result:

Be able to reference aarch64 packages without an explicit version. The content of netty-all is guaranteed to be declared in netty-bom, adding a safety net.

Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
2020-05-15 10:12:46 +02:00
sanjaypujare
20b8685c8b
Make ReferenceCountedOpenSslContext.setUseTasks public (#10289)
Motivation:
make the existing setter `ReferenceCountedOpenSslContext.setUseTasks` public

Modification:

Added the `public` qualified and removed the comment "for tests only"

Result:

Fixes #10288
2020-05-15 10:04:08 +02:00
Norman Maurer
69db5bff71
Respect jdk.tls.client.enableSessionTicketExtension and jdk.tls.server.enableSessionTicketExtension when using native SSL impl (#10296)
Motivation:

We should respect jdk.tls.client.enableSessionTicketExtension and jdk.tls.server.enableSessionTicketExtension when using the native SSL implementation as well to make the usage of it easier and more consistent. These properties were introduced by JDK13:

https://seanjmullan.org/blog/2019/08/05/jdk13

Modifications:

Check if the properties are set to true and if so enable tickets

Result:

Easier to enable tickets and be more consistent
2020-05-15 10:01:09 +02:00
Norman Maurer
75df58a7e1
Guard against re-entrancy issues while draining AbstractCoalescingBufferQueue (#10294)
Motivation:

AbstractCoalescingBufferQueue had a bug which could lead to an empty queue while still report bytes left. This was due the fact that we decremented the pending bytes before draining the queue one-by-one. The problem here is that while the queue is drained we may notify the promise which may add again buffers to the queue for which we never decrement the bytes while we drain these

Modifications:

- Decrement the pending bytes every time we drain a buffer from the queue
- Add unit tests

Result:

Fixes https://github.com/netty/netty/issues/10286
2020-05-15 09:51:33 +02:00
feijermu
91a7f49f0d
Add DNS client examples for run-example.sh (#10283)
Motivation:

There exists no DNS client examples in run-example.sh for the moment.

Modification:

Add DNS client examples for run-example.sh.

Result:

Help run the examples.
2020-05-14 12:10:32 +02:00
Norman Maurer
91ca3d332f
Allow to have the session tickets automatically managed by the native… (#10280)
Motivation:

BoringSSL supports to automatically manage the session tickets to be used and so also rotate them etc. This is often prefered by users as it removed some complexity. We should support to make use of this.

Modifications:

- Allow to have setSessionTickets() called without an argument or an empty array
- Add tests

Result:

Easier usage of session tickets
2020-05-14 12:09:26 +02:00
prgitpr
2183b37892
Fix a potential fd leak in AbstractDiskHttpData.getChunk (#10270)
Motivation:

`FileChannel.read()` may throw an IOException. We must deal with this in case of the occurrence of `I/O` error.

Modification:

Place the `FileChannel.read()` method call in the `try-finally` block.

Result:

Advoid fd leak.


Co-authored-by: Norman Maurer <norman_maurer@apple.com>
2020-05-14 10:16:16 +02:00
Norman Maurer
9411f2a434
Fix classifier for aarch64 (#10279)
Motivation:

The defined classifier for aarch64 was not correct

Modifications:

Fix classifier

Result:

Be able to correctly include the aarch64 native libs
2020-05-14 09:31:06 +02:00
Fabien Renaud
d5087deec6
Fix regression in HttpPostStandardRequestDecoder to always decode + to whitespace (#10285)
Motivations
-----------
HttpPostStandardRequestDecoder was changed in 4.1.50 to provide its own
ByteBuf UrlDecoder. Prior to this change, it was using the decodeComponent
method from QueryStringDecoder which decoded + characters to
whitespaces. This behavior needs to be preserved to maintain backward
compatibility.

Modifications
-------------
Changed HttpPostStandardRequestDecoder to detect + bytes and decode them
toe whitespaces. Added a test.

Results
-------
Addresses issues#10284
2020-05-14 09:28:40 +02:00
root
caf51b7284 [maven-release-plugin] prepare for next development iteration 2020-05-13 06:00:23 +00:00
root
8c5b72aaf0 [maven-release-plugin] prepare release netty-4.1.50.Final 2020-05-13 05:59:55 +00:00
Norman Maurer
8f617b6305
Use correct JDK 13 version (#10276)
Motivation:

We had a typo in the JDK 13 version to use.

Modifications:

Use the correct version string

Result:

Be able to run CI with JDK13 again
2020-05-13 07:18:02 +02:00
Norman Maurer
71467892bf
OpenSslSession.getLocalCertificates() and getLocalPrincipal() must r… (#10275)
Motivation:

OpenSslSession.getLocalCertificates() and  getLocalPrincipal() must return null on client side if mTLS is not used as stated in the API documentation. At the moment this is not always the case

Modifications:

- Ensure we only return non-null if mTLS is used
- Add unit tests

Result:

Follow SSLSession API contract
2020-05-13 07:16:27 +02:00
Norman Maurer
1c21733fb9
Update Java versions (#10273)
Motivation:

We should use the latest patch release of each java version

Modifications:

Update versions

Result:

Use latest versions on CI
2020-05-12 08:48:54 +02:00
Norman Maurer
68105b257d
Select correct nameserver for CNAME (#10272)
Motivation:

The nameserver that should / must be used to resolve a CNAME may be different then the nameserver that was selected for the hostname to resolve. Failing to select the correct nameserver may result in problems during resolution.

Modifications:

Use the correct DnsServerAddressStream for CNAMEs

Result:

Always use the correct DnsServerAddressStream for CNAMEs and so fix resolution failures which could accour when CNAMEs are in the mix that use a different domain then the original hostname that we try to resolve
2020-05-12 08:46:45 +02:00
feijermu
d10c946e1b
Remove a unused private method with empty body in HttpConversionUtil.java (#10266)
Motivation:

After searching the whole netty project, I found that the private method `translateHeader(...)` with empty body is never used actually. So I think it could be safely removed.

Modification:

Just remove this unused method.

Result:

Clean up the code.
2020-05-11 15:31:56 +02:00
feijermu
4c758a214d
Remove unused imports in DefaultHttp2LocalFlowController.java and HpackStaticTable.java (#10265)
Motivation:

`io.netty.handler.codec.http2.Http2Stream.State` is never used in DefaultHttp2LocalFlowController.java, and `io.netty.handler.codec.http2.HpackUtil.equalsConstantTime` is never used in HpackStaticTable.java.

Modification:

Just remove these unused imports.

Result:

Make imports cleaner.
2020-05-11 15:10:20 +02:00
Richard Nguyen
322d96ef92
Fix date format in headers to use 2-digit day of month (#10259)
Motivation:

`Date`, `Expires`, and `Set-Cookie` headers are being generated with a 1-digit day of month,
e.g. `Sun, 6 Nov 1994 08:49:37 GMT`. RFC 2616 specifies that `Date` and `Expires` headers should
use "a fixed-length subset of that defined by RFC 1123" which includes a 2-digit day of month.
RFC6265 is lax in it's specification of the `Set-Cookie` header and permits a 2-digit day of month.

See: https://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html
See: https://tools.ietf.org/html/rfc1123#page-55
See: https://tools.ietf.org/html/rfc6265#section-5.1.1

Modifications:

- Update `DateFormatter` to correctly implement RFC 2616 headers

Result:

```
Date: Sun, 06 Nov 1994 08:49:37 GMT
Expires: Sun, 06 Nov 1994 08:49:37 GMT
Set-Cookie: id=a3fWa; Expires=Sun, 06 Nov 1994 08:49:37 GMT
```
2020-05-11 08:57:17 +02:00
Piotr Betkier
09d38c87df
Add GlobalEventExecutor#addTask to BlockHound exceptions. (#10262)
Motivation:

GlobalEventExecutor#addTask may be called during SingleThreadEventExecutor shutdown.
May result in a blocking call, because GlobalEventExecutor#taskQueue is a BlockingQueue.

Modifications:

Add allowBlockingCallsInside configuration for GlobalEventExecutor#addTask.

Result:

Fixes #10257.
When BlockHound is installed, GlobalEventExecutor#addTask is not reported as a blocking call.
2020-05-11 08:51:38 +02:00
Piyush Goyal
79915347c6
Cleanup test case (#10232)
* Motivation:

JsonObjectDecoderTest did include 3 println(...) call which was leftover from debugging.

Modifications:

Removed println(...)

Result:

Cleanup

Co-authored-by: Norman Maurer <norman_maurer@apple.com>
2020-05-07 16:10:25 +02:00
Aayush Atharva
cf589a8f31
Add DoT and TCP DNS Client Example (#10256)
Motivation:

[DNS-over-TLS (DoT)](https://tools.ietf.org/html/rfc7858.html) encrypts DNS queries and sends it over TLS connection to make sure queries are secure in transit.

[TCP DNS](https://tools.ietf.org/html/rfc7766) sends DNS queries over TCP connection (unencrypted).

Modification:

Add DNS-over-TLS (DoT) Client Example which uses TLSv1.2 and TLSv1.3.
Add TCP DNS Client Example

Result:

DNS-over-TLS (DoT) Client Example
TCP DNS Client Example
2020-05-07 15:26:14 +02:00
louxiu
909e7c9c29
Add option to configure recycler delayed queue drop ratio (#10251) (#10255)
Motivation

- Recycler stack and delayed queue drop ratio can only be configured with
the same value. The overall drop ratio is ratio^2.

- #10251 shows that enable drop in `WeakOrderQueue` may introduce
performance degradation. Though the final reason is not clear now,
it would be better to add option to configure delayed queue drop ratio separately.

Modification

- "io.netty.recycler.delayedQueue.ratio" as the drop ratio of delayed queue
- default "delayedQueue.ratio" is same as "ratio"

Results

Able to configure recycler delayed queue drop ratio separately
2020-05-07 15:08:35 +02:00
feijermu
731d33070f
Add a DNS client example. (#10237)
Motivation:

It seems that there is no DNS client example in Netty project so far.

Modification:

Add a Netty DNS client example.

Result:

More examples
2020-05-07 10:46:41 +02:00
louxiu
4c9a30d5f9
Use io.netty.recycler.ratio directly (#10253)
Motivation

1. It's inable to collect all object because RATIO is always >=1 after
`safeFindNextPositivePowerOfTwo`

2. Enable drop object in `WeakOrderQueue`(commit:
71860e5b94) enlarge the drop ratio. We
can subtly control the overall drop ratio by using `io.netty.recycler.ratio` directly,

Modification

- Remove `safeFindNextPositivePowerOfTwo` before set the ratio

Results

Able to disable drop when recycle object
2020-05-07 10:29:05 +02:00
Norman Maurer
f00160bca3
Don't reuse ChannelPromise in WebSocketProtocolHandler (#10248)
Motivation:

We cant reuse the ChannelPromise as it will cause an error when trying to ful-fill it multiple times.

Modifications:

- Use a new promise and chain it with the old one
- Add unit test

Result:

Fixes https://github.com/netty/netty/issues/10240
2020-05-07 08:11:33 +02:00
feijermu
40448db5bb
Remove a unused import in DefaultHttp2ConnectionEncoder.java (#10249)
Motivation:

`io.netty.handler.codec.http2.Http2CodecUtil.DEFAULT_PRIORITY_WEIGHT` is never used in DefaultHttp2ConnectionEncoder.java.

Modification:

Just remove this unused import.

Result:

Make the DefaultHttp2ConnectionEncoder.java's imports clean.
2020-05-07 08:10:36 +02:00
Nick Hill
c3db0391af
Correctly handle WrappedCompositeByteBufs in addFlattenedComponents() (#10247)
Motivation

An NPE was reported in #10245, caused by a regression introduced in
#8939. This in particular affects ByteToMessageDecoders that use the
COMPOSITE_CUMULATOR.

Modification

- Unwrap WrappedCompositeByteBufs passed to
CompositeByteBuf#addFlattenedComponents(...) method before accessing
internal components field
- Extend unit test to cover this case and ensure more of the
CompositeByteBuf tests are also run on the wrapped variant

Results

Fixes #10245
2020-05-05 13:56:40 +02:00
Norman Maurer
cfcd7a4fde
Rename testmethods to make these more clear (#10231)
Motivation:

The currently used method names don't make a lot of sense.

Modifications:

Rename to cleanup

Result:

Cleanup
2020-04-29 17:52:43 +02:00
Aayush Atharva
9427255ffb
Specify algorithm for key pair in self signed certificate to generate EC or RSA based certificate. (#10223)
Motivation:

EC is better than RSA because of the small key size, efficient and secure which makes it perfect for testing purposes.

Modification:

Added support to specify an algorithm (EC or RSA) in constructors for key pair generation. The default key size is 256-bits as promulgated by NSA Suite B.

Result:
Able to generate a self-signed certificate of EC or RSA.
2020-04-29 16:52:07 +02:00
Norman Maurer
8f7ca2b4ef
Reuse the same allocator as used by the ByteBuf that is used during… (#10226)
Motivation:

We should not use Unpooled to allocate buffers if possible to ensure we can make use of pooling etc.

Modifications:

- Only allocate a buffer if really needed
- Use the ByteBufAllocator of the offered ByteBuf
- Ensure we not use buffer.copy() but explicitly allocate a buffer and then copy into it to not hit the limit of maxCapacity()

Result:

Improve allocations
2020-04-29 14:39:14 +02:00
Norman Maurer
ff36f2826c
Remove some debugging cruft (#10229)
Motivation:

RtspDecoderTest did include a println(...) call which was a left over from debugging.

Modifications:

Remove println(...)

Result:

Cleanup
2020-04-29 11:35:22 +02:00
Norman Maurer
987a68eb02
Fix memory leak in HttpPostMultipartRequestDecoder (#10227)
Motivation:

We need to release all ByteBufs that we allocate to prevent leaks. We missed to release the ByteBufs that are used to aggregate in two cases

Modifications:

Add release() calls

Result:

No more memory leak in HttpPostMultipartRequestDecoder
2020-04-29 08:23:41 +02:00
Norman Maurer
6cd193e83f
Don't log with warn level in the DnsNameResolver in most cases (#10225)
Motivation:

We should only log with warn level if something really critical happens as otherwise we may spam logs and confuse the user.

Modifications:

- Change log level to debug for most cases

Result:

Less noisy logging
2020-04-29 08:00:14 +02:00
Norman Maurer
387e451c82
Detect CNAME loops in the CNAME cache while trying to resolve (#10221)
Motivation:

We need to detect CNAME loops during lookup the DnsCnameCache as otherwise we may try to follow cnames forever.

Modifications:

- Correctly detect CNAME loops in the cache
- Add unit test

Result:

Fixes https://github.com/netty/netty/issues/10220
2020-04-28 10:47:10 +02:00
Norman Maurer
83012a038b
Update to latest Conscrypt release and add workarounds for bugs (#10211)
Motivation:

We are far behind with the version of Conscrypt we are using during testing. We should ensure we use the latest.

Modifications:

- Update conscrypt dependency
- Ensure we use conscrypt provider in tests
- Add workarounds for conscrypt bugs in testsuite

Result:

Use latest Conscrypt release
2020-04-28 09:50:05 +02:00
Fabien Renaud
c354fa48e1
HttpPostRequestDecoder: retain instead of copy when first buf is last (#10209)
Motivations
-----------
There is no need to copy the "offered" ByteBuf in HttpPostRequestDecoder
when the first HttpContent ByteBuf is also the last (LastHttpContent) as
the full content can immediately be decoded. No extra bookeeping needed.

Modifications
-------------
HttpPostMultipartRequestDecoder
 - Retain the first ByteBuf when it is both the first HttpContent offered
to the decoder and is also LastHttpContent.
 - Retain slices of the final buffers values

Results
-------
ByteBufs of FullHttpMessage decoded by HttpPostRequestDecoder are no longer
unnecessarily copied. Attributes are extracted as retained slices when
the content is multi-part. Non-multi-part content continues to return
Unpooled buffers.

Partially addresses issue #10200
2020-04-28 09:43:05 +02:00
feijermu
9751bb3ebc
Move up the size check in AbstractDiskHttpData.setContent. (#10222)
Motivation:

`AbstractHttpData.checkSize` may throw an IOException if we set the max size limit via `AbstractHttpData.setMaxSize`. However, if this exception happens, the `AbstractDiskHttpData.file` and the `AbstractHttpData.size` are still be modified. In other words, it may break the failure atomicity here.

Modification:

Just move up the size check.

Result:

Keep the failure atomicity even if `AbstractHttpData.checkSize` fails.
2020-04-28 09:34:33 +02:00
Fabien Renaud
4f72cdf233
Dns resolver: honor resolv.conf timeout, rotate and attempts options (#10207)
Motivations
-----------
DnsNameResolverBuilder and DnsNameResolver do not auto-configure
themselves uing default options define in /etc/resolv.conf.
In particular, rotate, timeout and attempts options are ignored.

Modifications
-------------
 - Modified UnixResolverDnsServerAddressStreamProvider to parse ndots,
attempts and timeout options all at once and use these defaults to
configure DnsNameResolver when values are not provided by the
DnsNameResolverBuilder.
 - When rotate option is specified, the DnsServerAddresses returned by
UnixResolverDnsServerAddressStreamProvider is rotational.
 - Amend resolv.conf options with the RES_OPTIONS environment variable
when present.

Result: 

Fixes https://github.com/netty/netty/issues/10202
2020-04-28 09:28:05 +02:00
Saranya Krishnakumar
5fa5ce34e1
Add check for DefaultFileRegion to calculate size of msg in AbstractTrafficShapingHandler (#10215)
Motivation:

Currently calculateSize method in AbstractTrafficShapingHandler calculates size for object of type ByteBuf or ByteBufHolder. Adding a check for FileRegion, makes it possible to do traffic shaping for FileRegion objects as well

Modification:

Check if object to be sent is of type FileRegion, if yes calculate the size using its count() method.


Co-authored-by: Dinesh Joshi <dinesh.joshi@apple.com>
2020-04-27 13:59:01 +02:00
feijermu
be28b266d5
Remove unused import in JsonObjectDecoder.java (#10213)
Motivation:

`io.netty.channel.ChannelHandler` is never used in JsonObjectDecoder.java.

Modification:

Just remove this unused import.

Result:

Make the JsonObjectDecoder.java's imports simple and clean.
2020-04-27 13:58:02 +02:00
feijermu
eb3721b971
Fix a potential fd leak in AbstractDiskHttpData.delete (#10212)
Motivation:

An unexpected IOException may be thrown from `FileChannel.force`. If it happens, the `FileChannel.close` may not be invoked.

Modification:

Place the `FileChannel.close` in a finally block.

Result:

Avoid fd leak.
2020-04-27 07:03:45 +02:00
Norman Maurer
9778f05e14
Update testsuite / pom.xml to be able to build with Java15 (#10210)
Motivation:

We need to make some slightly changes to be able to build on Java15 as some previous deprecated methods now throw UnsupportedOperationException

Modifications:

- Add code to handle UnsupportedOperationException
- Revert previous applied workaround for bug that was fixed in Java15
- Add maven profile

Result:

Be able to build with latest Java15 EA release
2020-04-27 06:27:54 +02:00
Norman Maurer
5a08dc0d9a
Add fastpath implementation for Unpooled.copiedBuffer(CharSequence, Charset) when UTF-8 or US-ASCII is used (#10206)
Motivation:

We can make use of our optimized implementations for UTF-8 and US-ASCII if the user request a copy of a sequence for these charsets

Modifications:

- Add fastpath implementation
- Add unit tests

Result:

Fixes https://github.com/netty/netty/issues/10205
2020-04-23 17:47:16 +02:00
wangxiyuan
660611c450
Add epoll aarch64 maven config and Dockerfile (#9804)
Motivation:

`transport-native-epoll` doesn't have ARM release package. 

Modification:

This PR added cross compile profile for epoll. Then we can easily build aarch64 package on X86 machine. 

Result:
Fixes #8279
2020-04-23 13:58:08 +02:00
Norman Maurer
0cf919c389
HTTP2: Guard against multiple ctx.close(...) calls with the same ChannelPromise (#10201)
Motivation:

Http2ConnectionHandler may call ctx.close(...) with the same promise instance multiple times if the timeout for gracefulShutdown elapse and the listener itself is notified. This can cause problems as other handlers in the pipeline may queue these promises and try to notify these later via setSuccess() or setFailure(...) which will then throw an IllegalStateException if the promise was notified already

Modification:

- Add boolean flag to ensure doClose() will only try to call ctx.close(...) one time

Result:

Don't call ctx.close(...) with the same promise multiple times when gradefulShutdown timeout elapses.
2020-04-23 11:25:39 +02:00
root
9c5008b109 [maven-release-plugin] prepare for next development iteration 2020-04-22 09:57:54 +00:00