Commit Graph

1333 Commits

Author SHA1 Message Date
Norman Maurer
2ef4b16138 Add websocket encoder / decoder in correct order to the pipeline when HttpServerCodec is used (#9386)
Motivation:

We need to ensure we place the encoder before the decoder when doing the websockets upgrade as the decoder may produce a close frame when protocol violations are detected.

Modifications:

- Correctly place encoder before decoder
- Add unit test

Result:

Fixes https://github.com/netty/netty/issues/9300
2019-07-18 10:19:33 +02:00
Emily Littleworth
0f8685ae9b Return null in HttpPostRequestEncoder (#9352)
Motivation:

If the encoded value of a form element happens to exactly hit
the chunk limit (8096 bytes), the post request encoder will
throw a NullPointerException.

Modifications:

Catch the null case and return.

Result:

No NPE.
2019-07-16 13:30:07 +02:00
Dmitriy Dumanskiy
ef24732640 Cleanup in websockets, throw exception before allocating response if possible (#9361)
Motivation:

While fixing #9359 found few places that could be patched / improved separately.

Modification:

On handshake response generation - throw exception before allocating response objects if request is invalid.

Result:

No more leaks when exception is thrown.
2019-07-16 13:21:16 +02:00
Andrey Mizurov
e9fec0a710 Fixed incorrect Sec-WebSocket-Origin header for v13, see #9134 (#9312)
Motivation:

Based on https://tools.ietf.org/html/rfc6455#section-1.3 - for non-browser
clients, Origin header field may be sent if it makes sense in the context of those clients.

Modification:

Replace Sec-WebSocket-Origin to Origin

Result:

Fixes #9134 .
2019-07-12 12:06:11 +02:00
jingene
af614e4d6e Change the netty.io homepage scheme(http -> https) (#9344)
Motivation:

Netty homepage(netty.io) serves both "http" and "https".
It's recommended to use https than http.
Modification:

I changed from "http://netty.io" to "https://netty.io"
Result:

No effects.
2019-07-09 21:10:14 +02:00
jimin
d54678d645 Remove unnecessary code (#9303)
Motivation:

There are is some unnecessary code (like toString() calls) which can be cleaned up.

Modifications:

- Remove not needed toString() calls
- Simplify subString(...) calls
- Remove some explicit casts when not needed.

Result:

Cleaner code
2019-07-04 09:02:24 +02:00
jimin
3e836bd3fe Call to ‘asList’ with only one argument could be replaced with ‘singletonList’ (#9288)
Motivation:

asList should only be used if there are multiple elements.

Modification:

Call to asList with only one argument could be replaced with singletonList

Result:

Cleaner code and a bit of memory savings
2019-06-26 21:07:11 +02:00
Julien Viet
2d72a7c561 Preserve the original filename when encoding a multipart/form in mixed mode. (#9270)
Motivation:

The HttpPostRequestEncoder overwrites the original filename of file uploads sharing the same name encoded in mixed mode when it rewrites the multipart body header of the previous file. The original filename should be preserved instead.

Modifications:

Change the HttpPostRequestEncoder to reuse the correct filename when the encoder switches to mixed mode. The original test is incorrect and has been modified too, in addition it tests with an extra file upload since the current test was not testing the continuation of a mixed mode.

Result:

The HttpPostRequestEncoder will preserve the original filename of the first fileupload when switching to mixed mode
2019-06-24 10:40:39 +02:00
ursa
a05adceae8 Bugfix #9257: WebSocketProtocolHandler does NOT support autoRead=false (#9258)
Motivation:

I need to control WebSockets inbound flow manually, when autoRead=false

Modification:

Add missed ctx.read() call into WebSocketProtocolHandler, where read request has been swallowed.

Result:

Fixes #9257
2019-06-24 09:08:30 +02:00
ursa
d9db218291 WebSocket is closed without an error on protocol violations (#9116)
Motivation:

Incorrect WebSockets closure affects our production system.
Enforced 'close socket on any protocol violation' prevents our custom termination sequence from execution.
Huge number of parameters is a nightmare both in usage and in support (decoders configuration).

Modification:

- Fix violations handling - send proper response codes.
- Fix for messages leak.
- Introduce decoder's option to disable default behavior (send close frame) on protocol violations.
- Encapsulate WebSocket response codes - WebSocketCloseStatus.
- Encapsulate decoder's configuration into a separate class - WebSocketDecoderConfig.

Result:

Fixes #8295.
2019-06-18 10:13:05 +02:00
Stephane Landelle
576b1768f4 Introduce WebSocketClientHandshaker::absoluteUpgradeUrl, close #9205 (#9206)
Motivation:

When connecting through an HTTP proxy over clear HTTP, user agents must send requests with an absolute url. This hold true for WebSocket Upgrade request.

WebSocketClientHandshaker and subclasses currently always send requests with a relative url, which causes proxies to crash as request is malformed.

Modification:

Introduce a new parameter `absoluteUpgradeUrl` and expose it in constructors and WebSocketClientHandshakerFactory.

Result:

It's now possible to configure WebSocketClientHandshaker so it works properly with HTTP proxies over clear HTTP.
2019-06-07 16:01:49 -07:00
yipulash
68f2242ac4 delete Other "Content-" MIME Header Fields exception (#9122)
delete Other "Content-" MIME Header Fields exception

Motivation:

RFC7578 4.8. Other "Content-" Header Fields

The multipart/form-data media type does not support any MIME header
fields in parts other than Content-Type, Content-Disposition, and (in
limited circumstances) Content-Transfer-Encoding. Other header
fields MUST NOT be included and MUST be ignored.

Modification:

Ignore other Content types.

Result: 

Other "Content-" Header Fields should be ignored no exception
2019-06-07 13:51:50 -07:00
Norman Maurer
5694dc3e2b Do not use static exceptions for websocket handshake timeout (#9174)
Motivation:

f17bfd0f64 removed the usage of static exception instances to reduce the risk of OOME due addSupressed calls. We should do the same for exceptions used to signal handshake timeouts.

Modifications:

Do not use static instances

Result:

No risk of OOME due addSuppressed calls
2019-05-23 08:36:37 +02:00
Vojin Jovanovic
f7a8a6f1ae Remove deprecated GraalVM native-image flags (#9118)
Motivation:

The first final version of GraalVM was released which deprecated some flags. We should use the new ones.

Modifications:

Removes the use of deprecated GraalVM native-image flags
Adds a flag to initialize netty at build time.

Result:

Do not use deprecated flags
2019-05-22 19:22:03 +02:00
秦世成
1465e3ce06 Support handshake timeout in websocket handlers (#8856)
Motivation:

Support handshake timeout option in websocket handlers. It makes sense to limit the time we need to move from `HANDSHAKE_ISSUED` to `HANDSHAKE_COMPLETE` states when upgrading to WebSockets

Modification:

- Add `handshakeTimeoutMillis` option in `WebSocketClientProtocolHandshakeHandler`  and `WebSocketServerProtocolHandshakeHandler`.
- Schedule a timeout task, the task will trigger user event `HANDSHAKE_TIMEOUT` if the handshake timed out.

Result:

Fixes issue https://github.com/netty/netty/issues/8841
2019-05-22 13:56:36 +02:00
Nick Hill
23554e6997 Ensure "full" ownership of msgs passed to EmbeddedChannel.writeInbound() (#9058)
Motivation

Pipeline handlers are free to "take control" of input buffers if they have singular refcount - in particular to mutate their raw data if non-readonly via discarding of read bytes, etc.

However there are various places (primarily unit tests) where a wrapped byte-array buffer is passed in and the wrapped array is assumed not to change (used after the wrapped buffer is passed to EmbeddedChannel.writeInbound()). This invalid assumption could result in unexpected errors, such as those exposed by #8931.

Modifications

Anywhere that the data passed to writeInbound() might be used again, ensure that either:
- A copy is used rather than wrapping a shared byte array, or
- The buffer is otherwise protected from modification by making it read-only

For the tests, copying is preferred since it still allows the "mutating" optimizations to be exercised.

Results

Avoid possible errors when pipeline assumes it has full control of input buffer.
2019-05-22 12:35:03 +02:00
秦世成
a9159d28fb Format code to align unaligned code. (#9062)
Motivation:
Format code to align unaligned code.

Modification:
Reformat the code

Result:

Cleaner code
2019-05-20 12:07:45 +02:00
Norman Maurer
ed61e5f543 Only use static Exception instances when we can ensure addSuppressed … (#9152)
Motivation:

OOME is occurred by increasing suppressedExceptions because other libraries call Throwable#addSuppressed. As we have no control over what other libraries do we need to ensure this can not lead to OOME.

Modifications:

Only use static instances of the Exceptions if we can either dissable addSuppressed or we run on java6.

Result:

Not possible to OOME because of addSuppressed. Fixes https://github.com/netty/netty/issues/9151.
2019-05-17 22:42:53 +02:00
Paulo Lopes
f5c0d2eb56 Add SVM metadata and minimal substitutions to build graalvm native image applications. (#8963)
Motivation:

GraalVM native images are a new way to deliver java applications. Netty is one of the most popular libraries however there are a few limitations that make it impossible to use with native images out of the box. Adding a few metadata (in specific modules will allow the compilation to success and produce working binaries)

Modification:

Added properties files in `META-INF` and substitutions classes (under `internal.svm`) will solve the compilation issues. The substitutions classes are not visible and do not have a public constructor so they are not visible to end users.

Result:

Fixes #8959

This fix is very conservative as it applies the minimum config required to build:

* pure netty servers
* vert.x applications
* grpc applications

The build is having trouble due to checkstyle which does not seem to be able to find the copyright notice on property files.
2019-04-29 09:12:09 +02:00
Oleksii Kachaiev
a57f73e1fe WebSocket client handshaker to support "force close" after timeout (#8896)
Motivation:

RFC 6455 defines that, generally, a WebSocket client should not close a TCP
connection as far as a server is the one who's responsible for doing that.
In practice tho', it's not always possible to control the server. Server's
misbehavior may lead to connections being leaked (if the server does not
comply with the RFC).

RFC 6455 #7.1.1 says

> In abnormal cases (such as not having received a TCP Close from the server
after a reasonable amount of time) a client MAY initiate the TCP Close.

Modifications:

* WebSocket client handshaker additional param `forceCloseAfterMillis`

* Use 10 seconds as default

Result:

WebSocket client handshaker to comply with RFC. Fixes #8883.
2019-04-10 15:48:24 +02:00
Norman Maurer
66f30874e3 Do not manually reset HttpObjectDecoder in HttpObjectAggregator.handleOversizedMessage(...) (#9017)
Motivation:

We did manually call HttpObjectDecoder.reset() in HttpObjectAggregator.handleOversizedMessage(...) which is incorrect and will prevent correct parsing of the next message.

Modifications:

- Remove call to HttpObjectDecoder.reset()
- Add unit test

Result:

Verify that we can correctly parse the next request after we rejected a request.
2019-04-08 21:11:23 +02:00
Norman Maurer
077b1fc4d9 Correctly discard messages after oversized message is detected. (#9015)
Motivation:

32563bfcc1 introduced a regression in which we did now not longer discard the messages after we handled an oversized message.

Modifications:

- Do not set aggregating to false after handleOversizedMessage is called
- Adjust unit tests to verify the behaviour is correct again.

Result:

Fixes https://github.com/netty/netty/issues/9007.
2019-04-08 21:09:30 +02:00
Norman Maurer
0f34345347
Merge ChannelInboundHandler and ChannelOutboundHandler into ChannelHa… (#8957)
Motivation:

In 42742e233f we already added default methods to Channel*Handler and deprecated the Adapter classes to simplify the class hierarchy. With this change we go even further and merge everything into just ChannelHandler. This simplifies things even more in terms of class-hierarchy.

Modifications:

- Merge ChannelInboundHandler | ChannelOutboundHandler into ChannelHandler
- Adjust code to just use ChannelHandler
- Deprecate old interfaces.

Result:

Cleaner and simpler code in terms of class-hierarchy.
2019-03-28 09:28:27 +00:00
Andrey Mizurov
d3906d862e Add user possibility to skip the evaluation of a certain websocket ex… (#8910)
Motivation:

Add user possibility to skip the evaluation of certain web socket extension,
for example we can skip compression extension for messages that already compressed or very small and etc.

Modification:

This pull request is related with #5669

Result:

User can set to WebSocketClientExtensionHandshaker or WebSocketServerExtensionHandshaker a filter to skip the evaluation of certain extension.
2019-03-22 15:02:54 +01:00
violetagg
a44c33136f Fix HttpUtil.isKeepAlive to behave correctly when Connection is a comma separated list (#8924)
Motivation:

According to the specification, the "Connection" header's syntax is:

"
The Connection header field's value has the following grammar:

     Connection        = 1#connection-option
     connection-option = token

Connection options are case-insensitive.
"
https://tools.ietf.org/html/rfc7230#section-6.1

This means that Connection's value can have at least one element or
a comma separated list with elements
When calculating whether the connection can remain open,
HttpUtil.isKeepAlive(HttpMessage) should take this into account.

Modifications:

- Check for "close" and "keep-alive" in a comma separated list
- Add unit test

Result:

HttpUtil.isKeepAlive(HttpMessage) works correctly when "Connection: Upgrade, close"
2019-03-13 14:30:46 +01:00
Norman Maurer
42742e233f
Deprecate ChannelInboundHandlerAdapter and ChannelOutboundHandlerAdapter (#8929)
Motivation:

As we now us java8 as minimum java version we can deprecate ChannelInboundHandlerAdapter / ChannelOutboundHandlerAdapter and just move the default implementations into the interfaces. This makes things a bit more flexible for the end-user and also simplifies the class-hierarchy.

Modifications:

- Mark ChannelInboundHandlerAdapter and ChannelOutboundHandlerAdapter as deprecated
- Add default implementations to ChannelInboundHandler / ChannelOutboundHandler
- Refactor our code to not use ChannelInboundHandlerAdapter / ChannelOutboundHandlerAdapter anymore

Result:

Cleanup class-hierarchy and make things a bit more flexible.
2019-03-13 09:46:10 +01:00
Norman Maurer
10f06a04e6 HttpContentDecoder must continue read when it did not produce any mes… (#8922)
Motivation:

When HttpContentDecoder (and so HttpContentDecompressor) does not produce any message we need to make sure it calls ctx.read() if auto read is false to not stale.

Modifications:

- Keep track if we need to call ctx.read() or not
- Add unit test

Result:

Fixes https://github.com/netty/netty/issues/8915.
2019-03-07 10:41:44 +01:00
Norman Maurer
c6b372f517 Use maven plugin to prevent API/ABI breakage as part of build process (#8904)
Motivation:

Netty is very widely used which can lead to a lot of pain when we break API / ABI. We should make use japicmp-maven-plugin during the build to verify we do not introduce breakage by mistake.

Modifications:

- Add japicmp-maven-plugin to the build process
- Fix a method signature change in HttpProxyHandler that was flagged as a possible problem.

Result:

Ensure no API/ABI breakage accour between releases.
2019-03-01 19:48:29 +01:00
Dmitriy Dumanskiy
da45e07e80 Avoid unnecessary char casts for CookieEncoder (#8827)
Motivation:

Avoid unnecessary (char) casts by changing variables types.

Modifications:

Use chars directly.

Result:

Less casts.
2019-02-25 19:50:46 +01:00
Dmitriy Dumanskiy
dd88f0ec48 Simplify WebSocketVersion code
Motivation:

WebSocketVersion can be simplified by directly store the string representation in the enum.

Modification:

Pass in the string representation when creating the enum.

Result:

Cleaner code.
2019-02-25 19:49:05 +01:00
Norman Maurer
d036b24a4b Do not depend on the implementation detail of Unpooled.buffer(int) when accessing backing array. (#8865)
Motivation:

We should not depend on the implementation detail of Unpooled.buffer(int) to allocate the exact size of backing byte[] as depending on the implementation it may return a buffer with a bigger backing array.

Modifications:

Explicit allocate the byte[] and wrap it in the ByteBuf. This way we are sure that ByteBuf.array() returns an byte[] which has the exact length and content we expect.

Result:

More correct and safe usage of ByteBuf.array()
2019-02-15 09:39:04 -08:00
Artem Morozov
1bc7c4900c Handle null "origin" header in "Old Hixie 75 handshake" as proper bad request. (#8864)
Motivation:

Gracefully respond on bad client request.
We have a set of errors produced by Android 7.1.1/7.1.2 clients where both headers `HttpHeaderNames.SEC_WEBSOCKET_VERSION` and `HttpHeaderNames.ORIGIN` are not present. Absence of the first headers leads to WebSocketServerHandshaker00 be applied as a handshaker. However, null 2nd header causes

```
java.lang.NullPointerException: value
 io.netty.util.internal.ObjectUtil.checkNotNull(ObjectUtil.java:33)
 io.netty.handler.codec.DefaultHeaders.addObject(DefaultHeaders.java:327)
 io.netty.handler.codec.http.DefaultHttpHeaders.add(DefaultHttpHeaders.java:123)
 io.netty.handler.codec.http.websocketx.WebSocketServerHandshaker00.newHandshakeResponse(WebSocketServerHandshaker00.java:162)
```
Which causes connection close with unclear reason.

Modification:

Added null-check, and in case of null an appropriate WebSocketHandshakeException is thrown.

Result:

In case of null `HttpHeaderNames.ORIGIN` header a WebSocketHandshakeException is caught by WebSocketServerProtocolHandler which sends a graceful `BAD_REQUEST`.
2019-02-13 17:16:24 -08:00
Dmitriy Dumanskiy
900d00d5db Java 8 migration. Move WebSocketUtil.base64 to java version instead of netty (#8837)
Motivation:

Since Java 8, JDK has `java.util.Base64` that could replace custom netty implementation. It is faster (3x for this particular PR) and simpler.

Modifications:

Use Base64 directly.


Result:

```
Benchmark                   Mode  Cnt        Score        Error  Units
Base64Benchmark.base64New  thrpt    6  7739181.025 ± 114230.467  ops/s
Base64Benchmark.base64Old  thrpt    6  2689783.304 ± 454710.641  ops/s
```
2019-02-12 10:10:06 -08:00
Rukshani Athapathu
dd96b4a876 Fix h2c upgrade failure when multiple connection headers are present in upgrade request (#8848)
Motivation:

When more than one connection header is present in h2c upgrade request, upgrade fails. This is to fix that.

Modification:
In HttpServerUpgradeHandler's upgrade() method, check whether any of the connection header value is upgrade, not just the first header value which might return a different value other than upgrade.

Result:
Fixes #8846.

With this PR, now when multiple connection headers are sent with the upgrade request, upgrade will not fail.
2019-02-12 08:10:47 -08:00
Stephane Landelle
ee4e46e6e2 Drop SPDY support (#8845)
Motivation:

SPDY has been superseded by HTTP/2. Chrome has dropped support in 2016 and GFE no longer negociate it.

Modifications:

* drop codec
* drop examples
* drop constants from `ApplicationProtocolNames`

Result:

SPDY support dropped from Netty 5
2019-02-07 09:25:31 +01:00
田欧
db79983874 use checkPositive/checkPositiveOrZero (#8835)
Motivation:

We can replace some "hand-rolled" integer checks with our own static utility method to simplify the code.

Modifications:

Use methods provided by `ObjectUtil`.

Result:

Cleaner code and less duplication
2019-02-04 15:55:07 +01:00
Dmitriy Dumanskiy
dcaec33a86 Typo fix in WebSocketServerExtension (#8829)
Motivation:

We had a typo in the name

Modifications:

Fix typo in method name

Result:

Method name has no typo
2019-02-04 12:48:07 +01:00
Dmitriy Dumanskiy
8069226ef1 Java 8 migration. Remove WebSocketUtil.randomNumber (#8830)
Motivation:

We can use ThreadLocalRandom.current().nextInt() directly .

Motivation:

Use ThreadLocalRandom.current().nextInt() directly instead of (int) ThreadLocalRandom.current().nextDouble()

Result:

Less custom code to maintain
2019-02-04 11:00:46 +01:00
田欧
e8efcd82a8 migrate java8: use requireNonNull (#8840)
Motivation:

We can just use Objects.requireNonNull(...) as a replacement for ObjectUtil.checkNotNull(....)

Modifications:

- Use Objects.requireNonNull(...)

Result:

Less code to maintain.
2019-02-04 10:32:25 +01:00
Roger Kapsi
cfa60552fb Selective Message Aggregation (#8793)
Motivation

Implementations of MessageAggregator (HttpObjectAggregator in particular) may wish to
selectively aggrerage requests and responses on a case-by-case basis such as for example
only POST requests or only responses of a certain content-type.

Modifications

Adding a flag to MessageAggregator that toggles between true/false depending on if aggregation
is desired for the current message or not.

Result

Fixes #8772
2019-02-04 09:58:40 +01:00
Dmitriy Dumanskiy
c9dacf9b85 Replace custome readAllFile method with Files.readAllBytes (#8828)
Motivation:

We can re-use Files.readAllBytes(....) and so can remove our own implementation.

Modification:

Replaced custom readAllBytes with Files.readAllBytes

Result:

Less custom code.
2019-02-01 16:45:20 +01:00
Dmitriy Dumanskiy
245cccd5e0 Old cookies classes removed (#8812)
Motivation:

We currently include two different cookie implementations, one is deprecated and one is not. We should remove the deprecated implentation.

Modifications:

Remove deprecated cookies classes.

Result:

Less code to maintain.
2019-01-31 20:35:56 +01:00
Norman Maurer
9725cae033 HttpObjectDecoder ignores HTTP trailer header when empty line is rece… (#8799)
* HttpObjectDecoder ignores HTTP trailer header when empty line is received in seperate ByteBuf

Motivation:

When the empty line that termines the trailers was sent in a seperate ByteBuf we did ignore the previous parsed trailers and just returned none.

Modifications:

- Correct respect previous parsed trailers.
- Add unit test.

Result:

Fixes https://github.com/netty/netty/issues/8736
2019-01-31 20:28:03 +01:00
Dmitriy Dumanskiy
ab0c33adff Remove deprecated HttpHeaders sub classes and related classes (#8813)
Motivation:

Code cleanup;

Changes:

- Removed deprecated HttpHeaders.Names and HttpHeaders.Values sub classes;
- Removed cross reference between EmptyHttpHeaders and HttpHeaders;
- Removed depraced Rtsp* classes;

Result:

Removed deprecated code.
2019-01-31 20:26:45 +01:00
Dmitriy Dumanskiy
67b23ab056 Remove HttpHeaderDateFormat class (#8807)
Motivation:

HttpHeaderDateFormat was replaced with DateFormatter many days ago and now can be easily removed.

Modification:

Remove deprecated class and related test / benchmark

Result:

Less code to maintain
2019-01-31 07:22:20 +01:00
Dmitriy Dumanskiy
b7ceeb1797 Compare HttpMethod by reference (#8815)
Motivation:

In most cases, HttpMethod instance is built from the factory method and the same instance is taken for known Http Methods. So we can implement fast path for equals().

Modification:

Replace == checks with HttpMethod.equals;
Use this == o within HttpMethod.equals;
Replaced known new HttpMethod with HttpMethod.valueOf;
Result:

Comparisons should be a bit faster in some cases.
2019-01-30 21:17:24 +01:00
田欧
6222101924 migrate java8: use lambda and method reference (#8781)
Motivation:

We can use lambdas now as we use Java8.

Modification:

use lambda function for all package, #8751 only migrate transport package.

Result:

Code cleanup.
2019-01-29 14:06:05 +01:00
田欧
e941cbe27a remove unused import statement (#8792)
Motivation:
The code contained some unused import statements.

Modification:
Remove unused import statements.

Result:
Code cleanup
2019-01-28 16:50:15 +01:00
田欧
934a07fbe2 migrate java8 (#8779)
Motivation:

We can omit argument types when using Java8.

Modification:

Omit arguments where possible.

Result:

Cleaner code.
2019-01-28 05:55:30 +01:00
kezhenxu94
7b6336f1fd Java 8 Migration: remove uneccessary if statement (#8755)
Motivation:

As netty 4.x supported Java 6 we had various if statements to check for java versions < 8. We can remove these now.

Modification:

Remove unnecessary if statements that check for java versions < 8.

Result:

Cleanup code.
2019-01-25 08:57:11 +01:00
Norman Maurer
310f31b392
Update to new checkstyle plugin (#8777)
Motivation:

We need to update to a new checkstyle plugin to allow the usage of lambdas.

Modifications:

- Update to new plugin version.
- Fix checkstyle problems.

Result:

Be able to use checkstyle plugin which supports new Java syntax.
2019-01-24 16:24:19 +01:00
Dmitriy Dumanskiy
9e9ee2ecd0 Java 8 migration. Join similar catch blocks (#8767)
Motivation:

We can now use multi-catch blocks.

Modification:

Join similar catch blocks + small cleanup.

Result:

Cleaner code.
2019-01-23 13:03:07 +01:00
Norman Maurer
3d6e6136a9
Decouple EventLoop details from the IO handling for each transport to… (#8680)
* Decouble EventLoop details from the IO handling for each transport to allow easy re-use of code and customization

Motiviation:

As today extending EventLoop implementations to add custom logic / metrics / instrumentations is only possible in a very limited way if at all. This is due the fact that most implementations are final or even package-private. That said even if these would be public there are the ability to do something useful with these is very limited as the IO processing and task processing are very tightly coupled. All of the mentioned things are a big pain point in netty 4.x and need improvement.

Modifications:

This changeset decoubled the IO processing logic from the task processing logic for the main transport (NIO, Epoll, KQueue) by introducing the concept of an IoHandler. The IoHandler itself is responsible to wait for IO readiness and process these IO events. The execution of the IoHandler itself is done by the SingleThreadEventLoop as part of its EventLoop processing. This allows to use the same EventLoopGroup (MultiThreadEventLoupGroup) for all the mentioned transports by just specify a different IoHandlerFactory during construction.

Beside this core API change this changeset also allows to easily extend SingleThreadEventExecutor / SingleThreadEventLoop to add custom logic to it which then can be reused by all the transports. The ideas are very similar to what is provided by ScheduledThreadPoolExecutor (that is part of the JDK). This allows for example things like:

  * Adding instrumentation / metrics:
    * how many Channels are registered on an SingleThreadEventLoop
    * how many Channels were handled during the IO processing in an EventLoop run
    * how many task were handled during the last EventLoop / EventExecutor run
    * how many outstanding tasks we have
    ...
    ...
  * Implementing custom strategies for choosing the next EventExecutor / EventLoop to use based on these metrics.
  * Use different Promise / Future / ScheduledFuture implementations
  * decorate Runnable / Callables when submitted to the EventExecutor / EventLoop

As a lot of functionalities are folded into the MultiThreadEventLoopGroup and SingleThreadEventLoopGroup this changeset also removes:

  * AbstractEventLoop
  * AbstractEventLoopGroup
  * EventExecutorChooser
  * EventExecutorChooserFactory
  * DefaultEventLoopGroup
  * DefaultEventExecutor
  * DefaultEventExecutorGroup

Result:

Fixes https://github.com/netty/netty/issues/8514 .
2019-01-23 08:32:05 +01:00
Dmitriy Dumanskiy
7b92ff2500 Java 8 migration. Remove ThreadLocalProvider and inline java.util.concurrent.ThreadLocalRandom.current() where necessary. (#8762)
Motivation:

Custom Netty ThreadLocalRandom and ThreadLocalRandomProvider classes are no longer needed and can be removed.

Modification:

Remove own ThreadLocalRandom

Result:

Less code to maintain
2019-01-22 20:14:28 +01:00
Dmitriy Dumanskiy
32d96a7f79 Java 8 migration. Similar catch blocks joined (#8759)
Motivation:

Avoid IDE warnings, easier to read.

Modification:

Same catch blocks joined.

Result:

Cleanup
2019-01-22 18:00:10 +01:00
Dmitriy Dumanskiy
42376c052a Java 8 migration. Inline PlatformDependent.newConcurrentHashMap() (#8760)
Motivation:

PlatformDependent.newConcurrentHashMap() is no longer needed so it could be easily removed and new ConcurrentHashMap<>() inlined instead of invoking PlatformDependent.newConcurrentHashMap().

Modification:

Use ConcurrentHashMap provided by the JDK directly.

Result:

Less code to maintain.
2019-01-22 17:18:50 +01:00
Dmitriy Dumanskiy
afc03da93b Java 8 migration. Use reference to StandardCharsets in CharsetUtil. (#8757)
Motivation:

No need to initialize charsets from the string. We can take already allocated charset from StandardCharsets class.

Modification:

Replace Charset.forName("US-ASCII") with StandardCharsets.US_ASCII.
Removed Charset[] values() method and internal static variable as it was used only in tests.

Result:

Reuse what the JDK provides
2019-01-22 16:50:36 +01:00
田欧
9d62deeb6f Java 8 migration: Use diamond operator (#8749)
Motivation:

We can use the diamond operator these days.

Modification:

Use diamond operator whenever possible.

Result:

More modern code and less boiler-plate.
2019-01-22 16:07:26 +01:00
Dmitriy Dumanskiy
5fb515f4af Java 8 migration. Use string switch where possible (#8753)
Motivation:

Replace "if else" conditions with string switch. It is easier to read the code, for large "if else" constructions switch also could be faster.

Modification:

Replaced "if else" with a string switch.

Result:

Use new language features
2019-01-22 15:58:49 +01:00
Dmitriy Dumanskiy
c34340fff8 Java 8 migration. Auto close for try catch blocks (#8752)
Motivation:

Since Java 7 we can automatically close resources in try () construction.

Modification:

Changed all try catches in the code with autoclose try (resource)

Result:

Less boiler-plate
2019-01-22 15:57:30 +01:00
Stephane Landelle
bc5924f550 HttpUtil#is100ContinueExpected clean up (#8740)
Motivation:

Current implementation extract header value as String. We have an idiomatic way for checking presence of a header value.

Modification:

Use HttpHeaders#contains for checking if if contains Expect: 100-continue.

Result:

Use idiomatic way + simplify boolean logic.
2019-01-22 08:50:31 +01:00
Norman Maurer
4a82d107d2
Require Java8 as minimum (#8739)
Motivation:

While we are not yet quite sure if we want to require Java11 as minimum we are at least sure we want to use java8 as minimum.

Modifications:

Change minimum version to java8 and update some tests which failed compilation after this change.

Result:

Use Java8 as minimum and be able to use Java8 features.
2019-01-22 08:48:18 +01:00
Bartek Kowalczyk
8bcea35d2e Set result for decoded request and add test for #8721 (#8721)
Motivation:
I want to fix bug in vert.x project (eclipse-vertx/vert.x#2562) caused by ComposedLastHttpContent result being null. I don't know if it is intentional that this last decoded chuck in the issue returns null, but if not - I am providing fix for that.

Modification:
* Added new constructor in ComposedLastHttpContent allowing to pass DecoderResult
* set DecoderResult.SUCCESS for created ComposedLastHttpContent in HttpContentEncoder
* set DecoderResult.SUCCESS for created ComposedLastHttpContent in HttpContentDecoder

Result:
Fixes eclipse-vertx/vert.x#2562
2019-01-21 07:45:21 +01:00
Riyafa Abdul Hameed
d5aba6d342 Close connection for CorruptedFrameException (#8705)
Motivation:

The CorruptedFrameException from the finish() method of the Utf8Validator gets propagated to other handlers while the connection is still open.

Modification:

Override exceptionCaught method of the Utf8FrameValidator and close the connection if it is a CorruptedFrameException.

Result:

The CorruptedFrameException gets propagated to other handlers only after properly closing the connection.
2019-01-17 07:19:21 +01:00
Norman Maurer
c10ccc5dec
Tighten contract between Channel and EventLoop by require the EventLoop on Channel construction. (#8587)
Motivation:

At the moment it’s possible to have a Channel in Netty that is not registered / assigned to an EventLoop until register(...) is called. This is suboptimal as if the Channel is not registered it is also not possible to do anything useful with a ChannelFuture that belongs to the Channel. We should think about if we should have the EventLoop as a constructor argument of a Channel and have the register / deregister method only have the effect of add a Channel to KQueue/Epoll/... It is also currently possible to deregister a Channel from one EventLoop and register it with another EventLoop. This operation defeats the threading model assumptions that are wide spread in Netty, and requires careful user level coordination to pull off without any concurrency issues. It is not a commonly used feature in practice, may be better handled by other means (e.g. client side load balancing), and therefore we propose removing this feature.

Modifications:

- Change all Channel implementations to require an EventLoop for construction ( + an EventLoopGroup for all ServerChannel implementations)
- Remove all register(...) methods from EventLoopGroup
- Add ChannelOutboundInvoker.register(...) which now basically means we want to register on the EventLoop for IO.
- Change ChannelUnsafe.register(...) to not take an EventLoop as parameter (as the EventLoop is supplied on custruction).
- Change ChannelFactory to take an EventLoop to create new Channels and introduce ServerChannelFactory which takes an EventLoop and one EventLoopGroup to create new ServerChannel instances.
- Add ServerChannel.childEventLoopGroup()
- Ensure all operations on the accepted Channel is done in the EventLoop of the Channel in ServerBootstrap
- Change unit tests for new behaviour

Result:

A Channel always has an EventLoop assigned which will never change during its life-time. This ensures we are always be able to call any operation on the Channel once constructed (unit the EventLoop is shutdown). This also simplifies the logic in DefaultChannelPipeline a lot as we can always call handlerAdded / handlerRemoved directly without the need to wait for register() to happen.

Also note that its still possible to deregister a Channel and register it again. It's just not possible anymore to move from one EventLoop to another (which was not really safe anyway).

Fixes https://github.com/netty/netty/issues/8513.
2019-01-14 20:11:13 +01:00
kashike
c0aa1ea5c7 Fix minor spelling issues in javadocs (#8701)
Motivation:

Javadocs contained some spelling errors, we should fix these.

Modification:

Fix spelling

Result:

Javadoc cleanup.
2019-01-14 07:25:13 +01:00
Bryce Anderson
44b919b698 Remove the maxChunkSize parameter for HTTP codecs (#8671)
Motivation:

The `maxChunkSize` only serves to split available content into even
smaller chunks which is not all that useful since the data is already
buffered.

Modification:

Remove the parameter.

Result:

Things are simpler.

Fixes #8430.
2018-12-20 07:37:51 +01:00
Stephane Landelle
9ed2884d3a Support 1012, 1013 and 1014 WebSocket close status code (#8664)
Motivation:

RFC 6455 doesn't define close status codes 1012, 1013 and 1014.
Yet, since then, IANA has defined them and web browsers support them.

From https://www.iana.org/assignments/websocket/websocket.xhtml:

* 1012: Service Restart
* 1013: Try Again Later
* 1014: The server was acting as a gateway or proxy and received an invalid response from the upstream server. This is similar to 502 HTTP Status Code.

Modification:

Make status codes 1012, 1013 and 1014 legit.

Result:

WebSocket status codes as defined by IANA are supported.
2018-12-17 19:43:10 +01:00
Norman Maurer
d9a6cf341c
Remove support for marking reader and writerIndex in ByteBuf to reduce overhead and complexity. (#8636)
Motivation:

ByteBuf supports “marker indexes”. The intended use case for these is if a speculative operation (e.g. decode) is in process the user can “mark” and interface and refer to it later if the operation isn’t successful (e.g. not enough data). However this is rarely used in practice,
requires extra memory to maintain, and introduces complexity in the state management for derived/pooled buffer initialization, resizing, and other operations which may modify reader/writer indexes.

Modifications:

Remove support for marking and adjust testcases / code.

Result:

Fixes https://github.com/netty/netty/issues/8535.
2018-12-11 14:00:49 +01:00
Feri73
5618229203 Correcting Maven Dependencies (#8622)
Motivation:

Most of the maven modules do not explicitly declare their
dependencies and rely on transitivity, which is not always correct.

Modifications:

For all maven modules, add all of their dependencies to pom.xml

Result:

All of the (essentially non-transitive) depepdencies of the modules are explicitly declared in pom.xml
2018-12-06 09:02:00 +01:00
Julien Hoarau
849bdc8a84 Set-Cookie headers should not be combined (#8611)
Motivation:

According to the HTTP spec set-cookie headers should not be combined
because they are not using the list syntax.

Modifications:

Do not combine set-cookie headers.

Result:

Set-Cookie headers won't be combined anymore
2018-12-01 10:47:37 +01:00
Norman Maurer
2c78dde749 Update version number to start working on Netty 5 2018-11-20 15:49:57 +01:00
root
3e7ddb36c7 [maven-release-plugin] prepare for next development iteration 2018-10-29 15:38:51 +00:00
root
9e50739601 [maven-release-plugin] prepare release netty-4.1.31.Final 2018-10-29 15:37:47 +00:00
Christian Lang
a6f807dd68 Fix context and window sizes sides. (#8395)
Motivation:

As mentioned in RFC 7692 :

The "server_no_context_takeover" Extension Parameter should be used on server side for compression and on client side for decompression.

The "client_no_context_takeover" Extension Parameter should be used on client side for compression and on server side for decompression.

Right now, in PerMessageDeflateClientExtensionHandshaker, the decoder uses clientNoContext instead of serverNoContext and the encoder uses serverNoContext instead of clientNoContext.

The same inversion is present in PerMessageDeflateServerExtensionHandshaker: the decoder uses
serverNoContext instead of clientNoContext, while the encoder uses serverNoContext instead of clientNoContext. Besides the context inversion, the sliding window sizes seem to be inversed as well.

Modification:

Inverse clientNoContext with serverNoContext and clientWindowSize with serverWindowSize for both the Decoder and Encoder in PerMessageDeflateServerExtensionHandshaker and PerMessageDeflateClientExtensionHandshaker.

Result:

This fixes the decompression fail in the case that one of the contexts is set and the other one is not.
2018-10-18 13:55:30 +02:00
root
2d7cb47edd [maven-release-plugin] prepare for next development iteration 2018-09-27 19:00:45 +00:00
root
3a9ac829d5 [maven-release-plugin] prepare release netty-4.1.30.Final 2018-09-27 18:56:12 +00:00
Andrey Mizurov
2ab3e13f08 Fix get charset from content-type header with multiple parameters (#8286)
Motivation:

Get charset from Content-Type header even it contains multiple parameters.

Modification:

Extract charset value from the charset parameter if it is not last.

Result:

Fixes #8273
2018-09-14 21:39:01 +02:00
root
a580dc7585 [maven-release-plugin] prepare for next development iteration 2018-08-24 06:36:33 +00:00
root
3fc789e83f [maven-release-plugin] prepare release netty-4.1.29.Final 2018-08-24 06:36:06 +00:00
root
fcb19cb589 [maven-release-plugin] prepare for next development iteration 2018-07-27 04:59:28 +00:00
root
ff785fbe39 [maven-release-plugin] prepare release netty-4.1.28.Final 2018-07-27 04:59:06 +00:00
root
b4dbdc2036 [maven-release-plugin] prepare for next development iteration 2018-07-11 15:37:40 +00:00
root
1c16519ac8 [maven-release-plugin] prepare release netty-4.1.27.Final 2018-07-11 15:37:21 +00:00
root
7bb9e7eafe [maven-release-plugin] prepare for next development iteration 2018-07-10 05:21:24 +00:00
root
8ca5421bd2 [maven-release-plugin] prepare release netty-4.1.26.Final 2018-07-10 05:18:13 +00:00
Norman Maurer
83710cb2e1
Replace toArray(new T[size]) with toArray(new T[0]) to eliminate zero-out and allow the VM to optimize. (#8075)
Motivation:

Using toArray(new T[0]) is usually the faster aproach these days. We should use it.

See also https://shipilev.net/blog/2016/arrays-wisdom-ancients/#_conclusion.

Modifications:

Replace toArray(new T[size]) with toArray(new T[0]).

Result:

Faster code.
2018-06-29 07:56:04 +02:00
Alexey Kachayev
fa4e28ba1c Fix random number generators in WebSocketUtil
Motivation:

Implementation of WebSocketUtil/randomNumber is incorrect and might violate
the API returning values > maximum specified.

Modifications:

* WebSocketUtil/randomNumber is reimplemented, the idea of the solution described
  in the comment in the code

* Implementation of WebSocketUtil/randomBytes changed to nextBytes method

* PlatformDependet.threadLocalRandom is used instead of Math.random to improve efficiency

* Added test cases to check random numbers generator

* To ensure corretness, we now assert that min < max when generating random number

Result:

WebSocketUtil/randomNumber always produces correct result.

Covers https://github.com/netty/netty/issues/8023
2018-06-25 14:42:59 +02:00
Stefan Lance
4c709be1ab Print correct invalid character after unwrapping value in CookieEncoder
Motivation:

If a wrapped cookie value with an invalid charcater is passed to the strict
encoder, an exception is thrown on validation but the error message contains
a character at the wrong position.

Modifications:

Print `unwrappedValue.charAt(pos)` instead of `value.charAt(pos)`.

Result:

The exception indicates the correct invalid character in the unwrapped cookie.
2018-06-21 08:19:01 +02:00
Roger
6208c9b0d6 Mention the HttpObjectEncoder's state in the message of the IllegalStateException (#7996)
Motivation

The HttpObjectEncoder raises an IllegalStateException due to an illegal state but doesn't mention what the state was. It could be useful for debugging purposes to figure out what happened.

Modifications

Mention the HttpObjectEncoder's state in the message of the IllegalStateException.

Result

An exception with more information what caused it.
2018-06-04 10:14:58 +02:00
Nick Travers
48911e0b63 Set (and override) websocket handshake headers after custom headers (#7975)
Motivation:

Currently, when passing custom headers to a WebSocketClientHandshaker,
if values are added for headers that are reserved for use in the
websocket handshake performed with the server, these custom values can
be used by the server to compute the websocket handshake challenge. If
the server computes the response to the challenge with the custom header
values, rather than the values computed by the client handshaker, the
handshake may fail.

Modifications:

Update the client handshaker implementations to add the custom header
values first, and then set the reserved websocket header values.

Result:

Reserved websocket handshake headers, if present in the custom headers
passed to the client handshaker, will not be propagated to the server.
Instead the client handshaker will propagate the values it generates.

Fixes #7973.
2018-05-30 19:52:40 +02:00
Nick Travers
9a3311506e Run the WebSocketClientHandshaker08Test on the 08 implementation (#7974)
Motivation:

The websockets abstract test suite does not run against the 08
implementation in the 08 version of the test suite.

Modifications:

Update the WebSocketClientHandshaker08Test to instantiate a new
WebSocketClientHandshaker08 rather than an 07 handshaker.

Result:

The WebSocketClientHandshaker08Test now tests the 08 implementation.
2018-05-27 10:02:49 +02:00
Nick Travers
19d1f4ea62 Propagate pong frames in WebSocketProtocolHandler (#7955)
Motivation:

Currently, on recipt of a PongWebSocketFrame, the
WebSocketProtocolHandler will drop the frame, rather than passing it
along so it can be referenced by other handlers.

Modifications:

Add boolean field to WebSocketProtocolHandler to indicate whether Pong
frames should be dropped or propagated, defaulting to "true" to preserve
existing functionality.

Add new constructors to the client and server implementations of
WebSocketProtocolHandler that allow for overriding the behavior for the
handling of Pong frames.

Result:

PongWebSocketFrames are passed along the channel, if specified.
2018-05-24 20:27:29 +02:00
Norman Maurer
4d6b006fe6
Correctly take status into account when compare DefaultHttpResponse (#7965)
Motivation:

DefaultHttpResponse did not respect its status when compute the hashCode and check for equality.

Modifications:

Correctly implement hashCode and equals

Result:

Fixes https://github.com/netty/netty/issues/7964.
2018-05-24 20:13:21 +02:00
Norman Maurer
64bb279f47 [maven-release-plugin] prepare for next development iteration 2018-05-14 11:11:45 +00:00
Norman Maurer
c67a3b0507 [maven-release-plugin] prepare release netty-4.1.25.Final 2018-05-14 11:11:24 +00:00
Norman Maurer
c60263e8a3
Correctly handle responses with status 205 and payload. (#7891)
Motivation:

HTTP responses with status of 205 should not contain a payload. We should enforce this.

Modifications:

Correctly handle responses with status 205 and payload by set Content-Length: 0 header and stripping out the content.

Result:

Fixes https://github.com/netty/netty/issues/7888
2018-05-03 11:25:06 +02:00
Norman Maurer
0261e00662
Allow to call AbstractDiskHttpData.setContent(Unpooled.EMPTY_BUFFER) multiple times. (#7890)
Motivation:

It should be possible to call setContent(Unpooled.EMPTY_BUFFER) multiple times just like its possible to do the same with a non empty buffer.

Modifications:

- Correctly reset underlying storage if called multiple times.
- Add tests

Result:

Fixes https://github.com/netty/netty/issues/6418
2018-04-30 08:39:24 +02:00
Norman Maurer
b75f44db9a [maven-release-plugin] prepare for next development iteration 2018-04-19 11:56:07 +00:00
Norman Maurer
04fac00c8c [maven-release-plugin] prepare release netty-4.1.24.Final 2018-04-19 11:55:47 +00:00