Commit Graph

10136 Commits

Author SHA1 Message Date
luffyke
7e823e3842
Replace MQTT deprecated API usage
Motivation:

Mqtt procotol defines packetId instead of messageId, replace deprecated method to match protocol definition

Modification:

Replace messageId() to packetId() in MqttEncoder

Result:

Match mqtt protocol definition and improve readability.
2020-03-30 21:23:23 +02:00
Romain Manni-Bucau
84ebf47515
making DefaultHttpDataFactory able to configure basedir and deleteonexit (#10146)
Motivation: 

currently (http) disk based attributes or uploads are globally configured in a single directory and can also only globally be deleted on exit or not. it does not fit well multiple cases, in particular the case you have multiple servers in the same JVM.

Modification: 
make it configurable per attribute/fileupload.

Result:

This PR duplicates Disk* constructor to add basedir and deleteonexit parameters and wires it in default http daa factory.
2020-03-30 13:11:46 +02:00
feijermu
aed5a74e09
Release the ByteBuf when IOException occurs in AbstractMemoryHttpData. (#10133)
Motivation:

An IOException may be thrown from InputStream.read or checkSize method, and cause the ByteBuf leak.

Modification:

Release the ByteBuf when IOException occurs.

Result:
Avoid ByteBuf leak.
2020-03-30 11:26:03 +02:00
Norman Maurer
81513c3728
Add JDK 14 and update JDK 13 for builds (#10136)
Motivation:

JDK 14 was released so we should include it in our build matrix. Beside this there was also a JDK 13 update

Modifications:

- Add docker-compose files for JDK 14
- Update JDK 13 version

Result:

Build with JDK 14 as well and use latest JDK 13 release
2020-03-26 10:26:20 +01:00
Norman Maurer
62e3c463c4
Don't produce multiple calls to exceptionCaught(...) on SSL handshake failure (#10134)
Motivation:

Before release 4.1.23, there was only ONE call to exceptionCaught method when an ssl handshake failure occurs, now we have two when using the JDK provider.

Modifications:

- Ensure we only propagate one exception fi we already failed the handshake and channelInactive(...) produce an exception again
- Add unit test

Result:

Fixes https://github.com/netty/netty/issues/10119
2020-03-26 09:12:39 +01:00
feijermu
0c3eae34ec
fix a potential ByteBuf leak in HttpProxyHandler. (#10130)
Motivation:

An exception may occur between ByteBuf's allocation and release. For example:
```java
java.lang.OutOfMemoryError: Java heap space
	at java.lang.String.<init>(String.java:325)
	at io.netty.buffer.ByteBufUtil.decodeString(ByteBufUtil.java:838)
	at io.netty.buffer.AbstractByteBuf.toString(AbstractByteBuf.java:1247)
	at io.netty.buffer.AbstractByteBuf.toString(AbstractByteBuf.java:1242)
	at io.netty.handler.proxy.HttpProxyHandler.<init>(HttpProxyHandler.java:105)
	at io.netty.handler.proxy.HttpProxyHandler.<init>(HttpProxyHandler.java:90)
	at io.netty.handler.proxy.HttpProxyHandler.<init>(HttpProxyHandler.java:85)
```
It may cause the ByteBuf variable authz and authzBase64's leak.

Modification:

Release the ByteBuf in a finally block as soon as possible.

Result:

Fix a potential ByteBuf leak.
2020-03-24 11:17:40 +01:00
Dmitry Konstantinov
913cae6fef
Optimize log2 in PoolThreadCache (#10128)
Motivation:
The current implementation of log2 in PoolThreadCache uses a loop and less efficient than an version based on Integer.numberOfLeadingZeros (intrinsic).

Modifications:
Replace the current log2 implementation in PoolThreadCache with a version based on Integer.numberOfLeadingZeros

Result:
It can improve performance slightly during allocation and de-allocation of ByteBuf using pooled allocator.
2020-03-24 11:13:52 +01:00
水木今山
4d7d249cc3
Fix a mistake in javadoc of ReplayingDecoder (#10129)
Motivation:

In the code example of ReplayingDecoder, an input parameter List<Object> out is missing.

Modification:

Just add this parameter.

Result:

The right doc.
2020-03-23 13:09:21 +01:00
Stephane Landelle
fcf7144f7c
Add some HTTP header constants (#10127)
Motivation:

Add some missing HTTP header names and values constants.

Modification:

* names:
  * dnt (Do Not Track)
  * upgrade-insecure-requests
  * x-requested-with
* values:
  * application/xhtml+xml
  * application/xml
  * text/css
  * text/html
  * text/event-stream
  * XmlHttpRequest

Result:

More constants available
2020-03-23 13:05:58 +01:00
Norman Maurer
99aca1879f
Update to netty-tcnative 2.0.30.Final to fix small memory leak (#10118)
Motivation:

A new netty-tcnative version was just released which fixes a small memory leak.

Modifications:

Update to 2.0.30.Final

Result:

Small memory leak fixed
2020-03-19 12:47:46 +01:00
root
14e4afeba2 [maven-release-plugin] prepare for next development iteration 2020-03-17 09:20:54 +00:00
root
c10c697e5b [maven-release-plugin] prepare release netty-4.1.48.Final 2020-03-17 09:18:28 +00:00
Norman Maurer
0fb58d3c54
HttpObjectDecoder should limit the number of control chars (#10112)
Motivation:

At the moment HttpObjectDecoder does not limit the number of controls chars. These should be counted with the initial line and so a limit should be exposed

Modifications:

- Change LineParser to also be used when skipping control chars and so enforce a limit
- Add various tests to ensure that limit is enforced

Result:

Fixes https://github.com/netty/netty/issues/10111
2020-03-17 10:16:45 +01:00
Bo Zhang
286f14f04a
Close Channel upon exceptions during bootstrap to avoid leaking (#10110)
Motivation:

We need to ensure we always close the Channel when we see an exception during bootstrapping it.

Modification:

Ensure we correct close the Channel if we see an exception during retrieving the Resolver from the group while bootstrapping.

Result:

Fixes #10109


Co-authored-by: Norman Maurer <norman_maurer@apple.com>
2020-03-16 13:37:29 +01:00
Norman Maurer
c74469bc6c
Ensure WebSocket*Handshaker can not corrupt pipeline when HttpProxyHa… (#10103)
Motivation:

HttpProxyHandler itself will add a HttpClientCodec into the ChannelPipeline and so confuse the WebSocket*Handshaker when it tries to modify the pipeline as it will replace the wrong HttpClientCodec.

Modifications:

Wrap the internal HttpClientCodec that is added by HttpProxyHandler so it will not be replaced by HttpProxyHandler.

Result:

Fixes  https://github.com/netty/netty/issues/5201 and https://github.com/netty/netty/issues/5070
2020-03-16 11:38:41 +01:00
feijermu
427a641299
Replace oio's default thread factory with netty's DefaultThreadFactory. (#10107)
Motivation:

Netty's DefaultThreadFactory that creates FastThreadLocalThread instance is widely used in NioEventLoopGroup, EpollEventLoopGroup, etc, but not OioEventLoopGroup. Although oio is quite stale, I still think this change may be useful.

Modification:

Replace oio's default thread factory with netty's DefaultThreadFactory just like NioEventLoopGroup, EpollEventLoopGroup, etc.

Result:

Faster access to FastThreadLocal in oio.
2020-03-16 11:37:29 +01:00
feijermu
679c606f13
Add test case for Http2StreamChannelId. (#10108)
Motivation:

Http2StreamChannelId is Serializable. A test case is needed to ensure that it works.

Modification:

Add a test case about serialization.

Result:

Improve test coverage slightly.
2020-03-16 09:31:38 +01:00
Norman Maurer
4b22d8ab35
Use WebSocketVersion.toAsciiString() as header value when possible (#10105)
Motivation:

In our WebSocketClientHandshaker* implementations we "hardcode" the version number to use. This is error-prone, we should better use the WebSocketVersion so we dont need to maintain the value multiple times. Beside this we can also use an AsciiString to improve performance

Modifications:

- Use WebSocketVersion.toAsciiString

Result:

Less stuff to maintain and small performance win
2020-03-13 07:48:58 +01:00
Norman Maurer
260540b25a
Don't override HOST header if provided by user already in WebSocketClientHandshaker (#10104)
Motivation:

The user may need to provide a specific HOST header. We should not override it when specified during handshake.

Modifications:

Check if a custom HOST header is already provided by the user and if so dont override it

Result:

Fixes https://github.com/netty/netty/issues/10101
2020-03-12 20:25:53 +01:00
David Latorre
e4af5c3631
Added support for the SameSite attribute in Cookies (#10050)
Motivation:

Netty currently does not support the SameSite attribute for response cookies (see issue #8161 for discussion).

Modifications:

The attribute has been added to the DefaultCookie class as a quick fix since adding new methods to the Cookie interface would be backwards-incompatible.
ServerCookieEncoder and ClientCookieDecoder have been updated accordingly to process this value. No validation for allowed values (Lax, None, Strict) has been implemented.

Result:

Response cookies with the SameSite attribute set can be read or written by Netty.

Co-authored-by: David Latorre <a-dlatorre@hotels.com>
2020-03-12 09:48:30 +01:00
时无两丶
60cbe8b7b2
Let LzfEncoder support length aware ability. (#10082)
Motivation:

Since the LZF support non-compress and compress format, we can let LzfEncoder support length aware ability. It can let the user control compress.

Modification:

When the data length over compressThreshold, LzfEncoder use compress format to compress data. Otherwise, only use non-compress format. Whatever compress format the encoder use, the LzfDecoder can decompress data well.

Result:

Gives users control over compression capabilities
2020-03-11 21:05:22 +01:00
Stephane Landelle
2576a2dd74
Fix WebSocketClientHandshaker not generating correct handshake request when path is empty (#10095)
Motivation:

WebSocketClientHandshaker#upgradeUrl doesn't comperly compute relative url when path is empty and produces url such as `?access_token=foo` instead of `/?access_token=foo`.

Modifications:

* fix WebSocketClientHandshaker#upgradeUrl
* add tests for urls without path, with and without query

Result:

WebSocketClientHandshaker properly connects to url without path.
2020-03-10 15:17:22 +01:00
root
c623a50d19 [maven-release-plugin] prepare for next development iteration 2020-03-09 12:13:56 +00:00
root
a401b2ac92 [maven-release-plugin] prepare release netty-4.1.47.Final 2020-03-09 12:13:26 +00:00
feijermu
8df19ce1f3
Replace several magic numbers. (#10094)
Motivation:

Magic numbers seem hard to read or understand.

Modification:

Replace several magic numbers with named constants.

Result:

Improve readability and make it easier to maintain.
2020-03-09 11:48:59 +01:00
Tristan Perry
59c8ce317a
Add a HTTP/2 client example using the newer frames approach the current HTTP/2 client example uses the older 'HTTP/1 <--> HTTP/2' translation approach. (#10081)
**Motivation:**

When I was previously working on a project using Netty's HTTP/2 support, I used the newer frames approach but I struggled to find any good examples or documentation online. I did, however, see a few people ask the same (or similar) questions as me on StackOverflow and a couple of older Netty Github issues.

Reading issue [9733](https://github.com/netty/netty/issues/9733) therefore prompted me to pull together a few bits of code into this HTTP/2 frame client example.

**Modification:**

Populated the previously-empty `example/src/main/java/io/netty/example/http2/helloworld/frame/client/` folder with a HTTP/2 frame client example.

**Result:**

Gives a clear example of how the newer HTTP/2 support can be used for Netty clients.
2020-03-09 08:59:31 +01:00
feijermu
b892722efb
Add a log level check simply before logging. (#10093)
Motivation:

ThrowableUtil.stackTraceToString is an expensive method call. So I think a log level check before this logging statement is quite needed especially in a environment with the warning log disabled.

Modification:

Add log level check simply before logging.

Result:

Improve performance in a environment with the warning log disabled.
2020-03-09 08:47:53 +01:00
Norman Maurer
660759b997
Use MacOSDnsServerAddressStreamProvider when on the classpath and we … (#10079)
Motivation:

939e928312 introduced MacOSDnsServerAddressStreamProvider which will ensure the right nameservers are selected when running on MacOS. To ensure this is done automatically on MacOS we should use it by default on these platforms.

Modifications:

Try to use MacOSDnsServerAddressStreamProvider when on MacOS via reflection and fallback if not possible

Result:

Ensure the right nameservers are used on MacOS even when a VPN (for example) is used.
2020-03-06 10:41:35 +01:00
zlm0125
161c237fb9
http multipart decode with chinese chars should work (#10089)
Motivation:

I am receiving a mutlipart/form_data upload from postman. The filename contains Chinese, and so some invalid chars. We should ensure all of these are removed before trying to decode.

Modification:

Ensure all invalid characters are removed

Result:

Fixes #10087

Co-authored-by: liming.zhang <liming.zhang@luckincoffee.com>
2020-03-06 10:33:58 +01:00
Norman Maurer
15fa45a84b
Add log level check simply before logging. (#10080)
Motivation:

In general, we will close the debug log in a product environment. However, logging without external level check may still affect performance as varargs will need to allocate an array.

Modification:

Add log level check simply before logging.

Result:

Improve performance slightly in a product environment.
2020-03-05 14:38:57 +01:00
Norman Maurer
65b1713ce7
Update ProtocolDetectionResult to fix typo (#10086)
Motivation:

Correct a typo.

Modification:

Changed a name of a private constant variable, 'NEEDS_MORE_DATE' to 'NEEDS_MORE_DATA'

Result:

Cleanup
2020-03-05 12:42:46 +01:00
Bryce Anderson
7b946a781e
Make sure we always flush window update frames in AbstractHttp2StreamChannel (#10075)
Motivation:

Under certain read patters the AbstractHttp2StreamChannel can fail to
flush, resulting in flow window starvation.

Modifications:

- Ensure we flush if we exit the `doBeginRead()` method.
- Account for the Http2FrameCodec always synchronously finishing writes
  of window update frames.

Result:

Fixes #10072
2020-03-04 10:50:41 +01:00
djelinski
1b0e3d95f4
Fix AssertionError in ScheduledFutureTask (#10073)
Motivation:

Some JVMs (like OpenJDK 8 on Windows) use a low resolution timer in System.nanoTime() and may return the same value more than once. This triggered an assertion failure when deadlineNanos was equal to nanoTime and AbstractScheduledEventExecutor#pollScheduledTask called #setConsumed.

Modifications: 
With this change the assertion checks exactly the same condition as AbstractScheduledEventExecutor#pollScheduledTask, and will no longer fail under these circumstances.

Result:

Fixes #10070.
2020-03-03 14:34:51 +01:00
feijermu
7ab68adbe8
Add test cases for StringUtil. (#10074)
Motivation:

StringUtil needs more test cases.

Modification:

Add several test cases for StringUtil.

Result:

Improve test coverage slightly.
2020-03-03 10:57:41 +01:00
Antony T Curtis
7fce06a0d1
Preserve order when using alternate event loops (#10069)
Motivation:

When the HttpContentCompressor is put on an alternate EventExecutor, the order of events should be
preserved so that the compressed content is correctly created.

Modifications:
- checking that the executor in the ChannelHandlerContext is the same executor as the current executor when evaluating if the handler should be skipped.
- Add unit test

Result:

Fixes https://github.com/netty/netty/issues/10067

Co-authored-by: Norman Maurer <norman_maurer@apple.com>
2020-03-03 10:42:41 +01:00
feijermu
0453222015
Add test cases for MathUtil. (#10071)
Motivation:

MathUtil needs more test cases.

Modification:

Add several test cases for MathUtil.

Result:

Improve test coverage slightly.
2020-03-02 08:38:45 +01:00
root
e0d73bca4d [maven-release-plugin] prepare for next development iteration 2020-02-28 06:37:33 +00:00
root
ebe7af5102 [maven-release-plugin] prepare release netty-4.1.46.Final 2020-02-28 06:36:45 +00:00
Norman Maurer
c9f38f855e
PromiseTask.isCancelled performs an unsynchronized read (#10066)
Motivation:

PromiseTask.isCancelled performs an unsynchronized read and so may trigger race-detectors. As this was just done for a small optimization which should not really make a lot of difference we should just remove it.

Modifications:

Remove unsynchronized read optimization

Result:

Fixes https://github.com/netty/netty/issues/10026.
2020-02-28 07:06:02 +01:00
Ning Xie
a304d61b26
fix snappy crc32c checksum (#10048)
Motivation:

The Snappy crc32c checksum produced by SnappyFrameEncoder maybe failed to be validated on other languages snappy decoder, such as golang/snappy.

Modification: 

- make the 4-byte cast later after the mask operation. Because whether retaining the higher 4-7 bytes in a long java type will make difference in (checksum >> 15 | checksum << 17) + 0xa282ead8 result.

Result:

Checksum correctly calculated
2020-02-27 09:03:50 +01:00
Norman Maurer
9b7e091b8d
Add SslHandshakeTimeoutException and use it for handshake timeouts (#10062)
Motivation:

Often it is useful to be able to detect different sorts of SSL errors that cause the handshake to fail. To make this easier we should throw and explicit exception type for handshake timeouts.

Modifications:

- Add SslHandshakeTimeoutException (which extends SSLHandshakeException) and use it for handshake timeouts
- Adjust testcases

Result:

Easier to detect that handshake failed because of a timeout
2020-02-27 08:57:01 +01:00
Norman Maurer
b64abd0e52
Correctly calculate the produced bytes in all cases when calling Refe… (#10063)
Motivation:

We did not correctly account for produced bytes when SSL_write(...) returns -1 in all cases. This could lead to lost data and so a corrupt SSL connection.

Modifications:

- Always ensure we calculate the produced bytes correctly
- Add unit tests

Result:

Fixes https://github.com/netty/netty/issues/10041
2020-02-27 08:49:53 +01:00
feijermu
f451295c75
Add test cases for ImmediateExecutor. (#10060)
Motivation:

ImmediateExecutor needs more test cases.

Modification:

Add several test cases for ImmediateExecutor.

Result:

Improve test coverage slightly.
2020-02-26 11:08:36 +01:00
Norman Maurer
9ae782d632
More strict parsing of initial line / http headers (#10058)
Motivation:

Our parsing of the initial line / http headers did treat some characters as separators which should better trigger an exception during parsing.

Modifications:

- Tighten up parsing of the inital line by follow recommentation of RFC7230
- Restrict separators to OWS for http headers
- Add unit test

Result:

Stricter parsing of HTTP1
2020-02-26 09:49:39 +01:00
feijermu
6d09298d1d
Add a null check to method TestUtils.compressHeapDumps (#10053)
Motivation:

java.io.File.listFiles() may return null and cause a unexpected NPE.

Modification:

Add a null check for variable files. And if variable files is null, the compressHeapDumps method will just return after logging a error message.

Result:

Fix the potential NPE.
2020-02-25 09:37:52 +01:00
Norman Maurer
880e1239c5
NetworkInterface.getByInetAddress() may return null on Android platform (#10056)
Motivation:

NetworkInterface.getByInetAddress() may return null on Android. This is incorrect by the API but still happens. To help our users we should provide a workaround

Modifications:

Just return an empty Enumeration when null is returned.

Result:

Fixes https://github.com/netty/netty/issues/10045
2020-02-25 09:34:38 +01:00
feijermu
914b433a8d
Replace the type of fragments field with IntObjectMap in SctpMessageCompletionHandler (#10057)
Motivation:

The type IntObjectMap with the key of primitive int can help lower the cost of boxing and unboxing,  and improve performance. I think it's more suitable for fragments field in SctpMessageCompletionHandler.java.

Modification:

Just replace the type of fragments field with IntObjectMap.

Result:
Improve performance slightly while decoding sctp message.
2020-02-24 19:57:55 +01:00
Norman Maurer
b1c57257af
Correctly calculate the initial size of the LinkedHashMap during DefaultChannelGroup.write* (#10055)
Motivation:

We should not include the number of ServerChannel that are part of the DefaultChannelGroup when specify the initial size of the LinkedHashMap

Modifications:

Only use the number of the non ServerChannel

Result:

Reduce memory-footprint
2020-02-24 19:29:08 +01:00
Norman Maurer
60082c9553
Update to netty-tcnative 2.0.29.Final (#10044)
Motivation:

A new release of netty-tcnative is out

Modifications:

Update to latest version

Result:

Use latest netty-tcnative
2020-02-24 09:24:55 +01:00
Norman Maurer
5c458c9a98
Workaround Android bug that cause AbstractDnsRecord to throw when the name is only a ROOT label (#10039)
Motivation:

Having only the ROOT label (".") as the name is valid, but Android 9 and prior does not correctly handle the case. We should add a workaround for it.

Modifications:

Detect if we are on Android and if so check if the name is the ROOT label and if so just return the name without trying to convert it

Result:

Fixes https://github.com/netty/netty/issues/10034
2020-02-18 15:05:52 +01:00