Commit Graph

6881 Commits

Author SHA1 Message Date
Norman Maurer
5a6238ed4c Minimize reference count checks in SlicedByteBuf
Motivation:

SlicedByteBuf did double reference count checking for various bulk operations, which affects performance.

Modifications:

- Add package private method to AbstractByteBuf that can be used to check indexes without check the reference count
- Use this new method in the bulk operation os SlicedByteBuf as the reference count checks take place on the wrapped buffer anyway
- Fix test-case to not try to read data that is out of the bounds of the buffer.

Result:

Better performance on bulk operations when using SlicedByteBuf (and sub-classes)
2015-10-16 21:09:03 +02:00
Norman Maurer
b7e947709e [#4357] Fix possible assert error in GlobalEventExecutor
Motivation:

We started the thread before store it in a field which could lead to an assert error when the thread is executed before we actually store it.

Modifications:

Store thread before start it.

Result:

No more assert error possible.
2015-10-16 20:56:25 +02:00
Norman Maurer
bd4df20eb3 Cleanup buffer tests.
Motivation:

Some of the tests in the buffer module contained unused code. Some of the tests also used unnecessary inheritance which could be avoided to simplify code.

Modifications:

Cleanup the test cases.

Result:

Cleaner code, less cruft.
2015-10-16 20:52:28 +02:00
Norman Maurer
8f13e333dd Always return a real slice even when the length is 0
Motivation:

We need to always return a real slice even when the requested length is 0. This is needed as otherwise we not correctly share the reference count and so may leak a buffer if the user call release() on the returned slice and expect it to decrement the reference count of the "parent" buffer.

Modifications:

- Always return a real slice
- Add unit test for the bug.

Result:

No more leak possible when a user requests a slice of length 0 of a SlicedByteBuf.
2015-10-16 20:46:05 +02:00
Scott Mitchell
d5f502d940 Native getSoError bug
Motivation:
Java_io_netty_channel_epoll_Native_getSoError incorrectly returns the value from the get socket option function.

Modifications:
- return the value from the result of the get socket option call

Result:
Java_io_netty_channel_epoll_Native_getSoError returns the correct value.
2015-10-16 11:13:35 -07:00
Norman Maurer
4c287d4e27 Added SlicedAbstractByteBuf that can provide fast-path for _get* and _set* methods
Motivation:

SlicedByteBuf can be used for any ByteBuf implementations and so can not do any optimizations that could be done
when AbstractByteBuf is sliced.

Modifications:

- Add SlicedAbstractByteBuf that can eliminate range and reference count checks for _get* and _set* methods.

Result:

Faster SlicedByteBuf implementations for AbstractByteBuf sub-classes.
2015-10-16 09:12:20 +02:00
Norman Maurer
8c93f4b1ef Added DuplicatedAbstractByteBuf that can provide fast-path for _get* and _set* methods
Motivation:

DuplicatedByteBuf can be used for any ByteBuf implementations and so can not do any optimizations that could be done
when AbstractByteBuf is duplicated.

Modifications:

- Add DuplicatedAbstractByteBuf that can eliminate range and reference count checks for _get* and _set* methods.

Result:

Faster DuplicatedByteBuf implementations for AbstractByteBuf sub-classes.
2015-10-16 08:56:35 +02:00
Norman Maurer
2aef4a504f Minimize object allocation when calling AbstractByteBuf.toString(..., Charset)
Motivation:

Calling AbstractByteBuf.toString(..., Charset) is used quite frequently by users but produce a lot of GC.

Modification:

- Use a FastThreadLocal to store the CharBuffer that are needed for decoding.
- Use internalNioBuffer(...) when possible

Result:

Less object creation / Less GC
2015-10-15 17:51:57 +02:00
Norman Maurer
4aa19a09bd Implement SSLSession.invalidate() and isValid() for OpenSSLEngine.
Motivation:

The SSLSession allows to invalidate a SSLSession and so disallow resume of a session. We should support this for OpenSSLEngine as well.

Modifications:

- Correctly implement SSLSession.isValid() and invalidate() in OpenSSLEngine
- Add unit test.

Result:

Invalidate of SSL sessions is supported when using OpenSSL now.
2015-10-15 12:02:19 +02:00
Scott Mitchell
0555b0aefd HTTP Headers Over Deprecation
Motivation:
As part of recent efforts to rectify performance and make 4.1 headers more similar to 5.0 some methods were deprecated. Some of these methods were deprecated because they used String instead of CharSequence in the signature, which may require casting at the user level. Some of the deprecated methods have no direct alternatives and were done to inform a user the method will go away in future releases.

Modifications:
- Remove the deprecated qualifier from methods where no direct replacement exists

Result:
Less warnings in user code.
2015-10-15 10:23:35 +02:00
Norman Maurer
9697afc106 Allow to disable reference count checks on every access of the ByteBuf
Motiviation:

Checking reference count on every access on a ByteBuf can have some big performance overhead depending on how the access pattern is. If the user is sure that there are no reference count errors on his side it should be possible to disable the check and so gain the max performance.

Modification:

- Add io.netty.buffer.bytebuf.checkAccessible system property which allows to disable the checks. Enabled by default.
- Add microbenchmark

Result:

Increased performance for operations on the ByteBuf.
2015-10-15 10:21:16 +02:00
Norman Maurer
ffe7aafd82 Optimize and minimize bound checks
Motivation:

We should minimize and optimize bound checks as much as possible to get the most out of performance.

Modifications:

- Use bitwise operations to remove branching
- Remove branches when possible

Result:

Better performance for various operations.
2015-10-15 10:18:13 +02:00
Norman Maurer
077af8c019 Remove encoderMaxConcurrentStreams
Motivation:

Remove encoderMaxConcurrentStreams(...) and use the default settings. Also throw an exception if server mode is used.

Modifications:

- Remove encoderMaxConcurrentStreams(...) method
- Throw exception if server mode is used and trying to enforce conncurrent streams.

Result:

Correctly support settings stuff via builder
2015-10-15 10:11:39 +02:00
Norman Maurer
6ebf4d1f5b [#4347] Remove not needed write operation from example.
Motivation:

The proxy example contains some code that is not needed. This can confuse the reader.

Modifications:

Remove the not needed ctx.write(...).

Result:

Less confusing code.
2015-10-14 10:57:04 +02:00
Richard DiCroce
b58036aeea Improve flexibility of EmbeddedChannel ID
Motivation:

EmbeddedChannelId#hashCode() and equals() do not behave correctly if an
instance is serialized and then deserialized. Additionally,
EmbeddedChannel does not allow use of any other type of ChannelId, and
EmbeddedChannelId is (mostly) a singleton instance. This creates a
problem for unit tests that construct multiple EmbeddedChannels and
expect each channel to have a unique ID.

Modifications:

EmbeddedChannelId is modified so equals() will return true and
hashCode() will return the same value for any instance of the class.
EmbeddedChannel is modified to allow a ChannelId to be specified when
the channel is constructed. Tests added for both changes.

Result:

EmbeddedChannelId now behaves correctly when deserialized, and
EmbeddedChannels can now have unique IDs.
2015-10-14 10:40:02 +02:00
Norman Maurer
30dc1c1fa4 [#4313] ByteBufUtil.writeUtf8 should use fast-path for WrappedByteBuf
Motivation:

ByteBufUtil.writeUtf8(...) / writeUsAscii(...) can use a fast-path when writing into AbstractByteBuf. We should try to unwrap WrappedByteBuf implementations so
we are able to do the same on wrapped AbstractByteBuf instances.

Modifications:

- Try to unwrap WrappedByteBuf to use the fast-path

Result:

Faster writing of utf8 and usascii for WrappedByteBuf instances.
2015-10-13 12:00:37 +02:00
Norman Maurer
42e6b8fa86 [#4289] Use a mock DNS Server for dns tests.
Motivation:

As relaying on external DNS Server can result to test-failures we should better use a mock DNS Server for the dns tests.

Modifications:

- Refactor the DnsNameResolverTest to use a mock DNS Server which is using apacheds.
- Allow to disable adding an opt resources as some servers not support it.

Result:

More stable testsuite.
2015-10-10 20:27:34 +02:00
Sky Ao
0740f703c1 change type definition of pipeline from DefaultChannelPipeline to ChannelPipeline 2015-10-10 20:15:10 +02:00
Norman Maurer
99b11c95b4 [#4327] Ensure toString() will not throw IllegalReferenceCountException
Motivation:

As toString() is often used while logging we need to ensure this produces no exception.

Modifications:

Ensure we never throw an IllegalReferenceCountException.

Result:

Be able to log without produce exceptions.
2015-10-10 20:12:43 +02:00
Norman Maurer
2d7e957a23 Cleanup PendingWriteQueueTest
Motivation:

PendingWriteQueueTest needs some cleanup.

Modifications:

- Cleanup code to remove deprecation warnings
- use static imports

Result:

No more warnings
2015-10-10 19:59:23 +02:00
Scott Mitchell
81a913ced1 ByteBufUtilTest bug
Motivation:
The logic in ByteBufUtilTest.ByteBufUtilTest is wrong. It is attempting to ensure at least 1 byte is different in the ranges that will be subsequently compared, but does so before the copy operation.

Modifications:
- Move the code which ensures there is a difference to after the copy
- Simplify the logic which ensures there is a difference

Result:
Unit test now operates as designed.
2015-10-09 10:43:12 -07:00
Norman Maurer
11e8163aa9 [#4284] Forward decoded messages more frequently
Motivation:

At the moment we only forward decoded messages that were added the out List once the full decode loop was completed. This has the affect that resources may not be released as fast as possible and as an application may incounter higher latency if the user triggeres a writeAndFlush(...) as a result of the decoded messages.

Modifications:

- forward decoded messages after each decode call

Result:

Forwarding decoded messages through the pipeline in a more eager fashion.
2015-10-07 14:15:53 +02:00
Norman Maurer
66c3c58d3e Reduce object creation for for unwrap/wrap if no ByteBuffer[] is used.
Motivation:

Often unwrap(...), wrap(...) is used with a single ByteBuffer and not with a ByteBuffer[]. We should reduce the array creations in this case.

Modifications:

Reuse ByteBuffer[1] for dst/src ByteBuffer.

Result:

Less object creation and so less GC
2015-10-07 13:35:53 +02:00
Norman Maurer
fbaf5d06e6 [#4316] Ensure pending tasks are run when EmbeddedChannel.close(...) or disconnect(...) is called.
Motivation:

We missed to run all pending tasks when EmbeddedChannel.close(...) or disconnect(...) was called. Because of this channelInactive(...) / channelUnregistered(...) of the handlers were never called.

Modifications:

Correctly run all pending tasks and cancel all not ready scheduled tasks when close or disconnect was called.

Result:

Correctly run tasks on close / disconnect and have channelInactive(...) / channelUnregistered(...) called.
2015-10-07 09:28:00 +02:00
Norman Maurer
e419533498 Remove unused parameter from method declaration.
Motivation:

We had an unused paramter on a method, we should just remove it to keep code clean.

Modifications:

- Remove parameter
- Fix typo in javadoc

Result:

Cleanup done.
2015-10-07 09:18:22 +02:00
Scott Mitchell
2f8bc24c62 Fix compile error introduced by 32231ee 2015-10-06 14:23:24 -07:00
Scott Mitchell
32231ee2e0 EPOLL RDHUP and IN at same time
Motivation:
If a RDHUP and IN event occurred at the same time it is possible we may not read all pending data on the channel. We should ensure we read data before processing the RDHUP event.

Modifications:
- Process the RDHUP event before the IN event.

Result:
Data will not be dropped.
Fixes https://github.com/netty/netty/issues/4317
2015-10-06 13:51:15 -07:00
Scott Mitchell
b2399c2475 EPOLL Shutdown Input Half Closed
Motivation:
EPOLL attempts to support half closed socket, but fails to call shutdown to close the read portion of the file descriptor.

Motivation:
- If half closed is supported shutting down the input should call underlying Native.shutdown(...) to make sure the peer is notified of the half closed state.

Result:
EPOLL half closed is more correct.
2015-10-06 12:45:57 -07:00
Norman Maurer
2ff2806ada [maven-release-plugin] prepare for next development iteration 2015-10-02 09:03:29 +02:00
Norman Maurer
5a43de10f7 [maven-release-plugin] prepare release netty-4.1.0.Beta7 2015-10-02 09:02:58 +02:00
Norman Maurer
dc6cb7545b Lazy compute SSLSession creation time.
Motivation:

As a SSL session may be created later at some time we should compute the creation time in a lazy fashion.

Modifications:

- Lazy compute creation time
- Add some unit test

Result:

More correct behavior
2015-10-03 10:42:00 +02:00
Scott Mitchell
67ddb61eca ByteBufUtilTest test failure
Motivation:
ByteBufUtilTest.notEqualsBufferSubsections is testing non-equality but just uses random numbers to assume they will not be equal. Even after the random bytes are generated we should check they are infact not equal so the test has no chance of failing when it should not.

Modifications:
- Loop through bytes in notEqualsBufferSubsections after they are randomly generated to ensure there is atleast 1 difference.

Result:
More reliable unit tests.
2015-10-02 17:49:09 -07:00
Scott Mitchell
d4680c55d8 AsciiString contains utility methods
Motivation:
When dealing with case insensitive headers it can be useful to have a case insensitive contains method for CharSequence.

Modifications:
- Add containsCaseInsensative to AsciiString

Result:
More expressive utility method for case insensitive CharSequence.
2015-10-02 12:50:11 -07:00
Scott Mitchell
06c3ae07a0 DefaultHttp2ConnectionDecoder write ping buffer
Motivation:
DefaultHttp2ConnectionDecoder writes a ACK when receiving a ping frame and sends the same data buffer it received. The data buffer is also passed to the listener, but the indexes are shared between the send and the listener. We should ensure the indexes are independent for these two operations.

Modifications:
- Call slice on the buffer that is being sent

Result:
Listener now has access to a buffer that will not appear to be already consumed.
2015-10-02 11:40:21 -07:00
Norman Maurer
5deec9631f Add support for server-side renegotiation when using OpenSslEngine.
Motivation:

JDK SslEngine supports renegotion, so we should at least support it server-side with OpenSslEngine as well.

That said OpenSsl does not support sending messages asynchronly while the renegotiation is still in progress, so the application need to ensure there are not writes going on while the renegotiation takes place. See also https://rt.openssl.org/Ticket/Display.html?id=1019 .

Modifications:

- Add support for renegotiation when OpenSslEngine is used in server mode
- Add unit tests.
- Upgrade to netty-tcnative 1.1.33.Fork9

Result:

Better compatibility with the JDK SSLEngine implementation.
2015-10-02 11:24:51 +02:00
Scott Mitchell
284e3702d8 Http2ConnectionHandler Builder instead of constructors
Motivation:
Using the builder pattern for Http2ConnectionHandler (and subclasses) would be advantageous for the following reasons:
1. Provides the consistent construction afforded by the builder pattern for 'optional' arguments. Users can specify these options 1 time in the builder and then re-use the builder after this.
2. Enforces that the Http2ConnectionHandler's internals (decoder Http2FrameListener) are initialized after construction.

Modifications:
- Add an extensible builder which can be used to build Http2ConnectionHandler objects
- Update classes which inherit from Http2ConnectionHandler

Result:
It is easier to specify options and construct Http2ConnectionHandler objects.
2015-10-01 13:51:03 -07:00
Norman Maurer
179cd9a4a1 Correctly update internal handshake state on beginHandshake()
Motivation:

We missed to correctly update the internal handshake state on beginHandshake() if we was able to finish the handshake directly. Also we not handled the case correctly when beginHandshake() was called after the first handshake was finished, which incorrectly throw an Error.

Modifications:

- Correctly set internal handshake state in all cases
- Correctly handle beginHandshake() once first handshake was finished.

Result:

Correctly handle OpenSslEngine.beginHandshake()
2015-10-01 17:41:46 +02:00
Scott Mitchell
1485a87e25 Http2ConnectionHandler and Http2FrameListener cyclic dependency
Motivation:
It is often the case that implementations of Http2FrameListener will want to send responses when data is read. The Http2FrameListener needs access to the Http2ConnectionHandler (or the encoder contained within) to be able to send responses. However the Http2ConnectionHandler requires a Http2FrameListener instance to be passed in during construction time. This creates a cyclic dependency which can make it difficult to cleanly accomplish this relationship.

Modifications:
- Add Http2ConnectionDecoder.frameListener(..) method to set the frame listener. This will allow the listener to be set after construction.

Result:
Classes which inherit from Http2ConnectionHandler can more cleanly set the Http2FrameListener.
2015-09-30 15:41:15 -07:00
Norman Maurer
2ffe7bd72e Fail build on warnings in the native transport
Motivation:

We should fail the build on warnings in the JNI/c code.

Modifications:

- Add GCC flag to fail build on warnings.
- Fix warnings (which also fixed a bug when using splice with offsets).

Result:

Better code quality.
2015-09-30 20:26:36 +02:00
Norman Maurer
173ebb9538 Fix enforcer rule for new os detection plugin 2015-09-30 10:01:43 +02:00
Norman Maurer
06fcb56980 Upgrade to netty-tcnative 1.1.33.Fork8
Motivation:

A new version of netty-tcnative was released with some important bug-fixes.

Modifications:

Bump up version.

Result:

Using latest netty-tcnative version
2015-09-30 09:14:36 +02:00
Norman Maurer
ca44436ce6 [#4265] Not allow to add/set DefaultHttpHeaders to itself.
Motivation:

We should prevent to add/set DefaultHttpHeaders to itself to prevent unexpected side-effects.

Modifications:

Throw IllegalArgumentException if user tries to pass the same instance to set/add.

Result:

No surprising side-effects.
2015-09-30 08:57:53 +02:00
Scott Van Wart
2adf6e5358 Better parsing for STOMP body with no length.
Motivation:

The STOMP decoder used to fail when parsing a frame with no content-length
and a body split across multiple packets.

Modifications:

Support contentLength of -1 (indicating indeterminate length) and added a
check to getContentLength.  Moved the NUL byte searching from the
readHeaders() method out to the main decoder loop.

Result:

A STOMP frame can be properly parsed even if it's missing the
content-length header and the NUL byte is in a later packet.
2015-09-29 16:09:54 +02:00
Norman Maurer
c8a941d01e [#4275] Discard bytes after X reads to guard against OOME.
Motivation:

If a remote peer writes fast enough it may take a long time to have fireChannelReadComplete(...) triggered. Because of this we need to take special care and ensure we try to discard some bytes if channelRead(...) is called to often in ByteToMessageDecoder.

Modifications:

- Add ByteToMessageDecoder.setDiscardAfterReads(...) which allows to set the number of reads after which we try to discard the read bytes
- Use default value of 16 for max reads.

Result:

No risk of OOME.
2015-09-29 12:02:39 +02:00
Scott Mitchell
0e9545e94d Http2RemoteFlowController stream writibility listener
Motivation:
For implementations that want to manage flow control down to the stream level it is useful to be notified when stream writability changes.

Modifications:
- Add writabilityChanged to Http2RemoteFlowController.Listener
- Add isWritable to Http2RemoteFlowController

Result:
The Http2RemoteFlowController provides notification when writability of a stream changes.
2015-09-28 13:47:24 -07:00
nmittler
7ab132f28a Making HTTP/2 stream byte assignment pluggable
Motivation:

The DefaultHttp2RemoteFlowController has become very large and is getting difficult to understand and maintain. It is also desirable for some applications to be able to disable the priority algorithm altogether for performance reasons.

Modifications:

Abstract the stream byte assignment logic (renamed allocation->assignment for clarity) behind an interface `StreamByteAssigner` with a single implementation `PriorityStreamByteAssigner`.

Result:

Goes some way towards supporting #4246
2015-09-25 14:00:12 -07:00
Norman Maurer
2766fc49e2 Expose new way of setting session keys
Motivation:

We should provide a better way to set session keys that not use the deprecated method of netty-tcnative.

Modifications:

- Add OpenSslSessionTicketKey
- Expose new method on OpenSslServerContext and deprecate the old method.

Result:

Easier to use and can remove the deprecated method later on.
2015-09-25 20:58:04 +02:00
Scott Mitchell
93011dd315 DefaultHttp2RemoteFlowController not allocating all available bytes
Motivation:
DefaultHttp2RemoteFlowController's allocation algorithm may not allocate all bytes that are available in the connection window. If the 'fair share' based upon weight is not fully used by sibling nodes it was not correctly re-distributed to other sibilings which may be able to utilize part / all of that share.

Modifications:
- Add a unit test which demonstrates the issue.
- Modify the allocation algorithm to ensure all available bytes are allocated.

Result:
Fixes https://github.com/netty/netty/issues/4266
2015-09-25 11:12:04 -07:00
Norman Maurer
747533408d [#4170] Shutdown socket before close fd when using epoll transport
Motivation:

We should call shutdown(...) on the socket before closing the filedescriptor to ensure it is closed gracefully.

Modifications:

Call shutdown(...) before close.

Result:

Sockets are gracefully shutdown when using native transport.
2015-09-25 20:05:14 +02:00
Norman Maurer
956a757d37 [#3789] Correctly reset markers for all allocations when using PooledByteBufAllocator
Motivation:

We need to ensure all markers are reset when doing an allocation via the PooledByteBufAllocator. This was not the always the case.

Modifications:

Move all logic that needs to get executed when reuse a PooledByteBuf into one place and call it.

Result:

Correct behavior
2015-09-25 19:57:33 +02:00