Commit Graph

6610 Commits

Author SHA1 Message Date
Norman Maurer
bac2e3a6d2 Reduce calls to System.nanoTime() and object creation in IdleStateHandler. Related to [#3808]
Motivation:

Calling System.nanoTime() for each channelRead(...) is very expensive. See [#3808] for more detailed description.
Also we always do extra work for each write and read even if read or write idle states should not be handled.

Modifications:

- Move System.nanoTime() call to channelReadComplete(...).
- Reuse ChannelFutureListener for writes
- Only add ChannelFutureListener to writes if write and all idle states should be handled.
- Only call System.nanoTime() for reads if idle state events for read and all states should be handled.

Result:

Less overhead when using the IdleStateHandler.
2015-05-27 14:07:39 +02:00
Norman Maurer
6fce3b79c3 Do not try to init TrustManagerFactory if trustCertChainFile is null.
Motivation:

We called TrustManagerFactory.init(...) even when the trustCertChainFile is null. This could lead to exceptions during the handshake.

Modifications:

Correctly only call TurstManagerFactory.init() if trustCertcChainFail is not null.

Result:

Correct behavior.
2015-05-27 13:45:57 +02:00
Norman Maurer
f2ba221e7e [#3785] Correctly handle connection refused with native transport
Motivation:

Due a bug we not correctly handled connection refused errors and so failed the connect promise with the wrong exception.
Beside this we some times even triggered fireChannelActive() which is not correct.

Modifications:

- Add testcase
- correctly detect connect errors

Result:

Correct and consistent handling.
2015-05-27 13:38:50 +02:00
Norman Maurer
ab1bb9136b [#3654] Synchronize on PoolSubpage head when allocate / free PoolSubpages
Motivation:

Currently we hold a lock on the PoolArena when we allocate / free PoolSubpages, which is wasteful as this also affects "normal" allocations. The same is true vice-verse.

Modifications:

Ensure we synchronize on the head of the PoolSubPages pool. This is done per size and so it is possible to concurrently allocate / deallocate PoolSubPages with different sizes, and also normal allocations.

Result:

Less condition and so faster allocation/deallocation.

Before this commit:
xxx:~/wrk $ ./wrk -H 'Connection: keep-alive' -d 120 -c 256 -t 16 -s scripts/pipeline-many.lua  http://xxx:8080/plaintext
Running 2m test @ http://xxx:8080/plaintext
  16 threads and 256 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    17.61ms   29.52ms 689.73ms   97.27%
    Req/Sec   278.93k    41.97k  351.04k    84.83%
  530527460 requests in 2.00m, 71.64GB read
Requests/sec: 4422226.13
Transfer/sec:    611.52MB

After this commit:
xxx:~/wrk $ ./wrk -H 'Connection: keep-alive' -d 120 -c 256 -t 16 -s scripts/pipeline-many.lua  http://xxx:8080/plaintext
Running 2m test @ http://xxx:8080/plaintext
  16 threads and 256 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    15.85ms   24.50ms 681.61ms   97.42%
    Req/Sec   287.14k    38.39k  360.33k    85.88%
  547902773 requests in 2.00m, 73.99GB read
Requests/sec: 4567066.11
Transfer/sec:    631.55MB

This is reproducable every time.
2015-05-27 10:33:12 +02:00
Norman Maurer
dce0dd9b78 [#3654] No need to hold lock while destroy a chunk
Motiviation:

At the moment we sometimes hold the lock on the PoolArena during destroy a PoolChunk. This is not needed.

Modification:

- Ensure we not hold the lock during destroy a PoolChunk
- Move all synchronized usage in PoolArena
- Cleanup

Result:

Less condition.
2015-05-27 09:47:53 +02:00
Scott Mitchell
d5f1dc66aa Consistent use of SSLHandshakeException for ALPN
Motiviation:
The OpenSSL engine uses SSLHandshakeException in the event of failures that occur during the handshake process. The alpn-boot project's getSSLException will also map the no_application_protocol to a SSLHandshakeException exception. We should be consistent and use SSLHandshakeException for handshake failure events.

Modifications:
-Update JdkAlpnSslEngine to propagate an SSLHandshakeException in the event of a failure.

Result:
Consistent usage of SSLHandshakeException during a handshake failure event.
2015-05-26 16:10:58 -07:00
Scott Mitchell
0f28bdf7bb Update HTTP/2 to RFCs
Motivation:
HTTP/2 RFC 7540 has been released.

Modifications:
- Make changes RFC 7540 introduced since draft 17.

Result:
HTTP/2 RFC 7540 compliant code
2015-05-26 16:08:09 -07:00
Roelof Naude
757671b7cc Support empty http responses when using compression
Motivation:

Found a bug in that netty would generate a 20 byte body when returing a response
to an HTTP HEAD. the 20 bytes seems to be related to the compression footer.

RFC2616, section 9.4 states that responses to an HTTP HEAD MUST not return a message
body in the response.

Netty's own client implementation expected an empty response. The extra bytes lead to a
2nd response with an error decoder result:
java.lang.IllegalArgumentException: invalid version format: 14

Modifications:

Track the HTTP request method. When processing the response we determine if the response
is passthru unnchanged. This decision now takes into account the request method and passthru
responses related to HTTP HEAD requests.

Result:

Netty's http client works and better RFC conformance.
2015-05-26 09:55:34 +02:00
Norman Maurer
08d234cdf0 [#3805] Fix incorrect javadoc in PlatformDependent 2015-05-25 21:42:31 +02:00
johnou
ad7f033c06 Allow writing with void promise if IdleStateHandler is configured in pipeline.
Motivation:

Allow writing with void promise if IdleStateHandler is configured in the pipeline for read timeout events.

Modifications:

Better performance.

Result:

No more ChannelFutureListeners are created if IdleStateHandler is only configured for read timeouts allowing for writing to the channel with void promise.
2015-05-25 21:09:47 +02:00
Norman Maurer
9d675def81 Only call System.nanoTime() if no read batch is ongoing. Related to [#3808]
Motivation:

[#3808] introduced some improvements to reduce the calls to System.nanoTime() but missed one possible optimization.

Modifications:

Only call System.nanoTime() if no reading patch is in process.

Result:

Less System.nanoTime() calls.
2015-05-25 18:21:00 +02:00
Jestan Nirojan
4877fffa0a SCTP echo client/server multi-homing examples
Motivation:
There are no Netty SCTP examples on multi-homing.

Modifications:
- Added new example classes based on echo client/server example

Result:
Better documentation
2015-05-22 22:24:09 +02:00
nmittler
e4af176be7 Upgrading Jetty alpn-api version
Motivation:

Discussion is in https://github.com/jetty-project/jetty-alpn/issues/8. The new API allows protocol negotiation to properly throw SSLHandshakeException.

Modifications:

Updated the parent pom.xml with the new version.

Result:

Upgraded alpn-api now allows throwing SSLHandshakeException.
2015-05-22 13:13:14 -07:00
nmittler
c20b38378b Allow manual configuration of initial HTTP/2 SETTINGS frame.
Motivation:

There is currently no good way to configure the initial SETTINGS frame. The individual settings can be configured on the various components, but doing this bypasses the proper setting update logic in the encoder.

Modifications:

Updated Http2ConnectionHandler to optionally take initial settings in the constructor. If not provided, it will default to current behavior.

Result:

Easy manual configuration of initial settings.
2015-05-21 13:18:52 -07:00
Norman Maurer
e903979a28 Not try to write more then Integer.MAX_VALUE / SSIZE_MAX via writev
Motivation:

When trying to write more then Integer.MAX_VALUE / SSIZE_MAX via writev(...) the OS may return EINVAL depending on the kernel or the actual OS (bsd / osx always return EINVAL). This will trigger an IOException.

Modifications:

Never try to write more then Integer.MAX_VALUE / SSIZE_MAX when using writev.

Result:

No more IOException when write more data then Integer.MAX_VALUE / SSIZE_MAX via writev.
2015-05-21 12:02:10 +02:00
Ruslan Sennov
1a5254b597 QueryStringDecoder's javadoc fix 2015-05-21 11:45:36 +02:00
Robert Varga
f3dcad3230 Do not call System.nanoTime() in ReadTimeoutHandler.channelRead()
Motivation:

We mitigate callouts to System.nanoTime() in SingleThreadEventExecutor
as it is 'relatively expensive'. On a modern system, tak translates to
about 20ns per call. With channelReadComplete() we can side-step this in
channelRead().

Modifications:

Introduce a boolean flag, which indicates that a read batch is currently
on-going, which acts as a flush guard for lastReadTime. Update
lastReadTime in channelReadComplete() just before setting the flag to
false. We set the flag to true in channelRead().

The periodic task examines the flag, and if it observes it to be true,
it will reschedule the task for the full duration. If it observes as
false, it will read lastReadTime and adjust the delay accordingly.

Result:

ReadTimeoutHandler calls System.nanoTime() only once per read batch.
2015-05-21 07:14:08 +02:00
Norman Maurer
271af7c624 Expose metrics for PooledByteBufAllocator
Motivation:

The PooledByteBufAllocator is more or less a black-box atm. We need to expose some metrics to allow the user to get a better idea how to tune it.

Modifications:

- Expose different metrics via PooledByteBufAllocator
- Add *Metrics interfaces

Result:

It is now easy to gather metrics and detail about the PooledByteBufAllocator and so get a better understanding about resource-usage etc.
2015-05-20 21:06:17 +02:00
nmittler
b034cf18f9 Fixing logging of ping acks in Http2OutboundFrameLogger
Motivation:

The Http2OutboundFrameLogger logs all PING frames as not acks.

Modifications:

Changed the logger to correctly log PING acks.

Result:

PING acks are logged correctly.
2015-05-20 08:59:17 -07:00
Norman Maurer
88b8558ec8 Add PooledSlicedByteBuf and PooledDuplicatedByteBuf
Motivation:

At the moment when calling slice(...) or duplicate(...) on a Pooled*ByteBuf a new SlicedByteBuf or DuplicatedByteBuf. This can create a lot of GC.

Modifications:

Add PooledSlicedByteBuf and PooledDuplicatedByteBuf which will be used when a PooledByteBuf is used.

Result:

Less GC.
2015-05-20 11:37:50 +02:00
Norman Maurer
33e443e71a Clarify ByteBuf.duplicate() semantics.
Motivation:

From the javadocs of ByteBuf.duplicate() it is not clear if the reader and writer marks will be duplicated.

Modifications:

Add sentence to clarify that marks will not be duplicated.

Result:

Clear semantics.
2015-05-20 08:32:52 +02:00
Norman Maurer
9d568586db Reset markers when obtain PooledByteBuf.
Motivation:

When allocate a PooledByteBuf we need to ensure to also reset the markers for the readerIndex and writerIndex.

Modifications:

- Correct reset the markers
- Add test-case for it

Result:

Correctly reset markers.
2015-05-20 07:29:32 +02:00
Norman Maurer
92bfeeca1b No need to release lock and acquire again when allocate normal size.
Motiviation:

When tried to allocate tiny and small sized and failed to serve these out of the PoolSubPage we exit the synchronization
block just to enter it again when call allocateNormal(...).

Modification:

Not exit the synchronized block until allocateNormal(...) is done.

Result:

Better performance.
2015-05-20 07:29:32 +02:00
Leonardo Freitas Gomes
781a85520c Add HTTP/2 Netty tiles example
Motivation:

Adding an example that showcases Netty’s HTTP/2 codec and that is
slightly more complex than the existing hello-world example. It is
based on the Gopher tiles example available here:
https://http2.golang.org/gophertiles?latency=0

Modifications:

Moved current http2 example to http2/helloworld.
Added http2 tiles example under http2/tiles.

Result:

A Netty tiles example is available.
2015-05-18 14:16:54 -07:00
Scott Mitchell
c6d61f9b43 Fix merge issue introduced by 04c0d77
Motiviation:
Interface changes between master and 4.1 branch resulted in a compile failure.

Modifications:
- change messageReceived to channelRead0

Result:
No more compile error.
2015-05-18 08:44:08 -07:00
Scott Mitchell
04c0d77287 HTTP/2 Server Example No Response for HTTP/1.x Only Clients
Motiviation:
The HTTP/2 server example just hangs when a client is using only HTTP with no ALPN or upgrade attempts. We should still send some kind of response.

Modifications:
The HTTP/2 server example has a special handler to detect no upgrade HTTP clients and generate a response.

Result:
Clients that just use HTTP with no upgrade will no appear hung when interacting with the HTTP/2 server example.
2015-05-18 08:40:06 -07:00
Robin Stocker
9bf636076a Fix typo in FingerprintTrustManagerFactory docs 2015-05-18 08:30:25 +02:00
Norman Maurer
b934257796 [#3784] Support hostname verification when using OpenSSLEngine
Motivation:

At the moment hostname verification is not supported with OpenSSLEngine.

Modifications:

- Allow to create OpenSslEngine with peerHost and peerPort informations.
- Respect endPointIdentificationAlgorithm and algorithmConstraints when set and get SSLParamaters.

Result:

hostname verification is supported now.
2015-05-18 08:16:49 +02:00
Scott Mitchell
fd22af8377 HTTP/2 GOAWAY Reference Count Issue
Motiviation:
The Http2ConnectionHandler is incrementing the reference count in the goAway method for the debugData buffer after it has already been sent and maybe consumed. This may result in an IllegalRefCountException to be thrown. The unit tests also encounter buffer leaks because they have not been updated to invoke the listener which releases the buffer in the goAway method.

Modifications:
- The retain() call should be before the frameWriter().writeGoAway(...) call
- The unit tests which call goAway must also invoke the operationComplete(..) method for the listener.

Result:
No IllegalRefCountException. Less buffer leaks in tests.
2015-05-15 10:48:41 -07:00
Stephane Landelle
f6299d942c Minor ClientCookieDecoder improvements
Motivation:

* Path attribute should be null, not empty String, if it's passed as "Path=".
* Only extract attribute value when the name is recognized.
* Only extract Expires attribute value String if MaxAge is undefined as it has precedence.

Modification:

Modify ClientCookieDecoder.
Add "testIgnoreEmptyPath" test in ClientCookieDecoderTest.

Result:

More idyomatic Path behavior (like Domain).
Minor performance improvement in some corner cases.
2015-05-12 11:25:28 +02:00
Eric Anderson
864f196c67 Add missing SslContextBuilder.forServer(KeyManagerFactory)
Motivation:

keyManager() is required on server-side, and so there is a forServer()
method for each override of keyManager(). However, one of the
forServer() overrides was missing, which meant that if you wanted to use
a KeyManagerFactory you were forced to provide garbage configuration
just to get past null checks.

Modifications:

Add missing override.

Result:

No hacks to use SslContextBuilder on server-side with KeyManagerFactory.
Resolves #3775
2015-05-11 22:10:34 +02:00
Idel Pivnitskiy
cbbcdaa249 Update compression dependencies
Motivation:

New versions of compression libraries, which improve their performance and fix some bugs.

Modifications:

Updated versions of jzlib, compress-lzf, lz4 and commons-compres libraries.

Result:

Better stability and performance of compression codecs.
2015-05-11 21:59:56 +02:00
Frederic Bregier
61dced3dcd Proposal to fix issue #3768 (3.10)
Motivations:
When using HttpPostRequestEncoder and trying to set an attribute if a
charset is defined, currenlty implicit Charset.toStrng() is used, given
wrong format.
As in Android for UTF-16 = "com.ibm.icu4jni.charset.CharsetICU[UTF-16]".

Modifications:
Each time charset is used to be printed as its name, charset.name() is
used to get the canonical name.

Result:
Now get "UTF-16" instead.
(3.10 version)
2015-05-11 06:36:27 +02:00
nmittler
bb5f9fe078 Scoring loopback addresses as 0 in DefaultChannelId
Motivation:

As described in #3490, [compareAddresses](https://github.com/netty/netty/blob/master/transport/src/main/java/io/netty/channel/DefaultChannelId.java#L182)  seems to allow loopback addresses to score high, disallowing replacing the loopback address.

Modifications:

Changed [scoreAddress](https://github.com/netty/netty/blob/master/transport/src/main/java/io/netty/channel/DefaultChannelId.java#L283) to score loopback addresses as 0.

Result:

Fixes #3490
2015-05-08 13:14:36 -07:00
Scott Mitchell
a2cd01b32b ByteString test failure bug
Motivation:
There is an error in the ByteString test logic which is resulting in test failures.

Modifications:
- Fix the loop iteration to use the loop iteration variable instead of a fixed index.

Result:
Tests are less buggy.
2015-05-08 09:44:02 -07:00
Scott Mitchell
2dd59e44b7 ALPN/NPN Examples Not using failure behavior compatible with all SslProviders
Motivation:
Examples that are using ALPN/NPN are using a failure mode which is not supported by the JDK SslProvider. The examples fail to run and throw an exception if the JDK SslProvider is used.

Modifications:
- Use SelectorFailureBehavior.NO_ADVERTISE
- Use SelectedListenerFailureBehavior.ACCEPT

Result:
Examples can be run with both OpenSsl and JDK SslProviders.
2015-05-08 09:39:09 -07:00
Eric Anderson
2927cdd504 Swap to SslContextBuilder in examples
Motivation:

Using factory methods of SslContext is deprecated. Code should be using
SslContextBuilder instead. This would have been done when the old
methods were deprecated, but memcache and http2 examples didn't exist in
the 4.0 branch which the PR was against.

Modifications:

Swap to the new construction pattern.

Result:

No more deprecated warnings during build of examples. Users are
instructed to use the new pattern.
2015-05-07 15:57:25 -07:00
Norman Maurer
f23b7b4efd [maven-release-plugin] prepare for next development iteration 2015-05-07 14:21:08 -04:00
Norman Maurer
871ce43b1f [maven-release-plugin] prepare release netty-4.1.0.Beta5 2015-05-07 14:20:38 -04:00
Scott Mitchell
74627483d7 [#3724] HTTP/2 Headers END_STREAM results in RST_STREAM
Motivation:
If headers are sent on a stream that does not yet exist and the END_STREAM flag is set we will send a RST_STREAM frame. We should send the HEADERS frame and no RST_STREAM.

Modifications:
DefaultHttp2RemoteFlowController should allow frames to be sent if stream is created in the 'half closed (local)' state.

Result:
We can send HEADERS frame with the END_STREAM flag sent without sending a RST_STREAM frame.
2015-05-07 08:31:05 -07:00
nmittler
bace371ca5 Addressing a few more comments from #3749.
Motivation:

There were a few outstanding comments that were left unaddressed after committing the changes for #3749.

Modifications:

Changes to Http2ConnectionHandler.goAway():

- Retaining the debugData buffer, rather than always converting it to a string immediately.
- Changing log level for sending a GOAWAY with error to debug.

Result:

Remaining comments from #3749 are addressed.
2015-05-07 07:57:33 -07:00
Norman Maurer
f963401d42 Allow rejection of remote initiated renegotiation
Motivation:

To prevent from DOS attacks it can be useful to disable remote initiated renegotiation.

Modifications:

Add new flag to OpenSslContext that can be used to disable it
Adding a testcase

Result:

Remote initiated renegotion requests can be disabled now.
2015-05-07 14:41:25 +02:00
Alwayswithme
79c17cf1fd ByteBufUtil use IndexOfProcessor to find occurrence.
Motivation:
The way of firstIndexOf and lastIndexOf iterating the ByteBuf is similar to forEachByte and forEachByteDesc, but have many range checks.
Modifications:
Use forEachByte and a IndexOfProcessor to find occurrence.
Result:
eliminate range checks
2015-05-07 09:36:47 +02:00
Stephane Landelle
97d871a755 Validate cookie name and value characters Motivation:
RFC6265 specifies which characters are allowed in a cookie name and value.

Netty is currently too lax, which can used for HttpOnly escaping.

Modification:

In ServerCookieDecoder: discard cookie key-value pairs that contain invalid characters.
In ClientCookieEncoder: throw an exception when trying to encode cookies with invalid characters.

Result:

The problem described in the motivation section is fixed.
2015-05-07 06:33:36 +02:00
JongYoon Lim
c85b770ac4 Remove duplicated code in AcceptorChannel classes.
Motivation:

NioUdtByteAcceptorChannel and NioUdtMessageAcceptorChannel have almost same code.
For maintainability, it's better to remove it.

Motification:

- Pulled a member(METADATA) and methods(doReadMessage() and metadata() up.
- Added newConnectorChannel().

Result:

Cleaner code.
2015-05-07 06:32:01 +02:00
nmittler
77d0042310 Allow override of HTTP/2 graceful connection shutdown.
Motivation:

Currently the graceful shutdown of the HTTP/2 connection waits until there are no active streams. There may be use cases that buffer stream creation (due to limits imposed by MAX_CONCURRENT_STREAMS), in which case they may still want those streams to complete before closing.

Modifications:

Added a isGracefulShutdownComplete method to Http2ConnectionHandler, which can be overridden by a subclass.

Result:

Graceful shutdown logic can be overridden.
2015-05-06 14:42:07 -07:00
Norman Maurer
f6c2c99efc [#3740] Add missing parentheses so the fix works as expected. 2015-05-06 23:03:53 +02:00
Norman Maurer
e71e40057f Fix possible IllegalStateException caused by closeNotifyTimeout when using SslHandler
Motivation:

In the SslHandler we schedule a timeout at which we close the Channel if a timeout was detected during close_notify. Because this can race with notify the flushFuture we can see an IllegalStateException when the Channel is closed.

Modifications:

- Use a trySuccess() and tryFailure(...) to guard against race.

Result:

No more race.
2015-05-06 21:50:16 +02:00
Norman Maurer
57e7571c65 Not trigger channelWritabilityChanged if fail messages before close Channel.
Motivation:

We should not trigger channelWritabilityChanged during failing message when we are about to close the Channel as otherwise the use may try again writing even if the Channel is about to get closed.

Modifications:

Add new boolean param to ChannelOutboundBuffer.failFlushed(...) which allows to specify if we should notify or not.

Result:

channelWritabilityChanged is not triggered anymore if we cloe the Channel because of an IOException during write.
2015-05-06 21:36:43 +02:00
Norman Maurer
f839f65c15 Do not defer closing of Channel when in flush
Motivation:

Previously, we deferred the closing of the Channel when we were flushing. This is problematic as this means that if the user adds a ChannelFutureListener, that will close the Channel, the closing will not happen until we are done with flushing. This can lead to more data is sent than expected.

Modifications:

- Do not defer closing when in flush

Result:

Correctly respect order of events and closing the Channel ASAP
2015-05-06 21:07:10 +02:00