Commit Graph

705 Commits

Author SHA1 Message Date
Norman Maurer
34ff9cf5f2 Fix possible overflow when calculate in the size of the out buffer in Base64
Motivation:

We not correctly guarded against overflow and so call Base64.encode(...) with a big buffer may lead to an overflow when calculate the size of the out buffer.

Modifications:

Correctly guard against overflow.

Result:

Fixes [#6620].
2017-04-21 08:11:17 +02:00
Nikolay Fedorovskikh
0692bf1b6a fix the typos 2017-04-20 04:56:09 +02:00
Norman Maurer
38b054c65c Correctly handle read-only ByteBuf in ByteToMessageDecoder
Motivation:

If a read-only ByteBuf is passed to the ByteToMessageDecoder.channelRead(...) method we need to make a copy of it once we try to merge buffers for cumulation. This usually is not the case but can for example happen if the local transport is used. This was the cause of the leak report we sometimes saw during the codec-http2 tests, as we are using the local transport and write a read-only buffer. This buffer will then be passed to the peer channel and fired through the pipeline and so end up as the cumulation buffer in the ByteToMessageDecoder. Once the next fragement is received we tried to merge these and failed with a ReadOnlyBufferException which then produced a leak.

Modifications:

Ensure we copy the buffer if its read-only.

Result:

No more exceptions and so leak when a read-only buffer is passed to ByteToMessageDecoder.channelRead(...)
2017-04-19 07:26:26 +02:00
Vladimir Kostyukov
4c77e7c55a netty-codec: Manage read-flow explicitly in MessageAggregator 2017-04-17 19:37:43 +02:00
Jeff Evans
476d2aea76 Adding method to assert XML decoder framing works
Motivation:

In an effort to better understand how the XmlFrameDecoder works, I consulted the tests to find a method that would reframe the inputs as per the Javadocs for that class. I couldn't find any methods that seemed to be doing it, so I wanted to add one to reinforce my understanding.

Modification:

Add a new test method to XmlFrameDecoder to assert that the reframing works as described.

Result:

New test method is added to XmlFrameDecoder
2017-03-19 08:08:07 -07:00
Norman Maurer
2b8c8e0805 [maven-release-plugin] prepare for next development iteration 2017-03-10 07:46:17 +01:00
Norman Maurer
1db58ea980 [maven-release-plugin] prepare release netty-4.1.9.Final 2017-03-10 07:45:28 +01:00
ulinnuha
a9d7c72cde Update links from code.google.com to github.com
Motivation:

This pull request does not solve any problem but we find that several links in the code refer to project websites under the domain of http://code.google.com which are either moved to github or not maintained anymore.

Modification:

Update the project links from code.google.com to the relevant project in github.com
2017-03-07 10:21:03 +01:00
Scott Mitchell
675980c7ff Lz4FrameEncoder incorrect usage of internalNioBuffer
Motivation:
Lz4FrameEncoder uses internalNioBuffer but always passes in a value of 0 for the index. This should be readerIndex().

Modifications:
- change 0 to readerIndex()

Result:
More correct usage of internalNioBuffer in Lz4FrameEncoder.
2017-03-02 12:50:40 -08:00
Norman Maurer
4f78bae2eb DatagramPacketEncoder|Decoder should take into account if wrapped handler is sharable
Motivation:

DatagramPacketEncoder|Decoder should respect if the wrapped handler is sharable or not and depending on that be sharable or not.

Modifications:

- Delegate isSharable() to wrapped handler
- Add test-cases

Result:

Correct behavior
2017-02-23 20:22:34 +01:00
Scott Mitchell
77e65fe6bb Base64 reduce byte manipulation operations
Motivation:
Base64#decode4to3 generally calculates an int value where the contents of the decodabet straddle bytes, and then uses a byte shifting or a full byte swapping operation to get the resulting contents. We can directly calculate the contents and avoid any intermediate int values and full byte swap operations. This will reduce the number of operations required during the decode operation.

Modifications:
- remove the intermediate int in the Base64#decond4to3 method.
- manually do the byte shifting since we are already doing bit/byte manipulations here anyways.

Result:
Base64#decode4to3 requires less operations to compute the end result.
2017-02-22 21:32:44 -08:00
Norman Maurer
7feb92959e Improve performance of Base64.decode and encode methods.
Motivation:

The decode and encode method uses getByte(...) and setByte(...) in loops which can be very expensive because of bounds / reference-count checking. Beside this it also slows-down a lot when paranoid leak-detection is enabled as it will track each access.

Modifications:

- Pack bytes into int / short and so reduce operations on the ByteBuf
- Use ByteProcessor to reduce getByte calls.

Result:

Better performance in general. Also when you run the build with -Pleak the handler module will build in 1/4 of the time it took before.
2017-02-22 20:12:12 +01:00
Nikolay Fedorovskikh
0623c6c533 Fix javadoc issues
Motivation:

Invalid javadoc in project

Modifications:

Fix it

Result:

More correct javadoc
2017-02-22 07:31:07 +01:00
Norman Maurer
fbf0e5f4dd Prefer JDK ThreadLocalRandom implementation over ours.
Motivation:

We have our own ThreadLocalRandom implementation to support older JDKs . That said we should prefer the JDK provided when running on JDK >= 7

Modification:

Using ThreadLocalRandom implementation of the JDK when possible.

Result:

Make use of JDK implementations when possible.
2017-02-16 15:44:00 -08:00
Norman Maurer
78586a99b6 Ensure CombinedChannelDuplexHandler can not be shared.
Motivation:

CombinedChannelDuplexHandler must not be shared as it contains state.

Modifications:

Enforce that it is not shared.

Result:

Fixes [#6333]
2017-02-14 08:42:23 +01:00
Norman Maurer
974a251de8 Not fail tests when running on JDK9+ and init of MarshallingFactory fails
Motivation:

To use jboss-marshalling extra command-line arguments are needed on JDK9+ as it makes use of reflection internally.

Modifications:

Skip jboss-marshalling tests when running on JDK9+ and init of MarshallingFactory fails.

Result:

Be able to build on latest JDK9 release.
2017-02-14 08:27:58 +01:00
Norman Maurer
9b2b3e2512 Ensure tests pass when sun.misc.Unsafe is not present
Motivation:

We need to ensure we pass all tests when sun.misc.Unsafe is not present.

Modifications:

- Make *ByteBufAllocatorTest work whenever sun.misc.Unsafe is present or not
- Let Lz4FrameEncoderTest not depend on AbstractByteBufAllocator implementation details which take into account if sun.misc.Unsafe is present or not

Result:

Tests pass even without sun.misc.Unsafe.
2017-02-14 07:52:07 +01:00
fenik17
ccd1d77714 Simplification of boolean expression
Motivation:

Simplify code and reduction the logical operations.

Modifications:

Simplify boolean expression.

Result:

Code less and simpler.
2017-02-07 14:35:16 -08:00
Norman Maurer
a7c0ff665c Only use Mockito for mocking.
Motivation:

We used various mocking frameworks. We should only use one...

Modifications:

Make usage of mocking framework consistent by only using Mockito.

Result:

Less dependencies and more consistent mocking usage.
2017-02-07 08:47:22 +01:00
Dmitriy Dumanskiy
b9abd3c9fc Cleanup : for loops for arrays to make code easier to read and removed unnecessary toLowerCase() 2017-02-06 07:47:59 +01:00
Norman Maurer
735d6dd636 [maven-release-plugin] prepare for next development iteration 2017-01-30 15:14:02 +01:00
Norman Maurer
76e22e63f3 [maven-release-plugin] prepare release netty-4.1.8.Final 2017-01-30 15:12:36 +01:00
Tim Brooks
3344cd21ac Wrap operations requiring SocketPermission with doPrivileged blocks
Motivation:

Currently Netty does not wrap socket connect, bind, or accept
operations in doPrivileged blocks. Nor does it wrap cases where a dns
lookup might happen.

This prevents an application utilizing the SecurityManager from
isolating SocketPermissions to Netty.

Modifications:

I have introduced a class (SocketUtils) that wraps operations
requiring SocketPermissions in doPrivileged blocks.

Result:

A user of Netty can grant SocketPermissions explicitly to the Netty
jar, without granting it to the rest of their application.
2017-01-19 21:12:52 +01:00
Jason Brown
3ea807e375 Flush LZ4FrameEncoder buffer when channel flush() is received.
Motivation:

LZ4FrameEncoder maintains an internal buffer of incoming data compress, and only writes out compressed data when a size threshold is reached. LZ4FrameEncoder does not override the flush() method, and thus the only way to flush data down the pipeline is via more data or close the channel.

Modifications:

Override the flush() function to flush on demand. Also overrode the allocateBuffer() function so we can more accurately size the output buffer (instead of needing to potatntially realloc via buffer.ensureWritable()).

Result:

Implementation works as described.
2017-01-18 10:57:21 -08:00
Norman Maurer
7f01da8d0f [maven-release-plugin] prepare for next development iteration 2017-01-12 11:36:51 +01:00
Norman Maurer
7a21eb1178 [maven-release-plugin] prepare release netty-4.1.7.Final 2017-01-12 11:35:58 +01:00
Johno Crawford
84410f97af Add unit test that shows LineBasedFrameDelimiter correctly handles fragmented data.
Motivation:

Verify everything works as expected.

Modifications:

Added testcase.

Result:

More test-coverage.
2017-01-12 07:50:31 +01:00
Norman Maurer
7a4b0c3297 Add unit test that shows LineBasedFrameDelimiter correctly splits line.
Motivation:

Thought there may be a bug so added a testcase to verify everything works as expected.

Modifications:

Added testcase

Result:

More test-coverage.
2017-01-11 08:00:47 +01:00
Scott Mitchell
06e7627b5f Read Only Http2Headers
Motivation:
A read only implementation of Http2Headers can allow for a more efficient usage of memory and more performant combined construction and iteration during serialization.

Modifications:
- Add a new ReadOnlyHttp2Headers class

Result:
ReadOnlyHttp2Headers exists and can be used for performance reasons when appropriate.

```
Benchmark                                            (headerCount)  Mode  Cnt    Score   Error  Units
ReadOnlyHttp2HeadersBenchmark.defaultClientHeaders               1  avgt   20   96.156 ± 1.902  ns/op
ReadOnlyHttp2HeadersBenchmark.defaultClientHeaders               5  avgt   20  157.925 ± 3.847  ns/op
ReadOnlyHttp2HeadersBenchmark.defaultClientHeaders              10  avgt   20  236.257 ± 2.663  ns/op
ReadOnlyHttp2HeadersBenchmark.defaultClientHeaders              20  avgt   20  392.861 ± 3.932  ns/op
ReadOnlyHttp2HeadersBenchmark.defaultServerHeaders               1  avgt   20   48.759 ± 0.466  ns/op
ReadOnlyHttp2HeadersBenchmark.defaultServerHeaders               5  avgt   20  113.122 ± 0.948  ns/op
ReadOnlyHttp2HeadersBenchmark.defaultServerHeaders              10  avgt   20  192.698 ± 1.936  ns/op
ReadOnlyHttp2HeadersBenchmark.defaultServerHeaders              20  avgt   20  348.974 ± 3.111  ns/op
ReadOnlyHttp2HeadersBenchmark.defaultTrailers                    1  avgt   20   35.694 ± 0.271  ns/op
ReadOnlyHttp2HeadersBenchmark.defaultTrailers                    5  avgt   20   98.993 ± 2.933  ns/op
ReadOnlyHttp2HeadersBenchmark.defaultTrailers                   10  avgt   20  171.035 ± 5.068  ns/op
ReadOnlyHttp2HeadersBenchmark.defaultTrailers                   20  avgt   20  330.621 ± 3.381  ns/op
ReadOnlyHttp2HeadersBenchmark.readOnlyClientHeaders              1  avgt   20   40.573 ± 0.474  ns/op
ReadOnlyHttp2HeadersBenchmark.readOnlyClientHeaders              5  avgt   20   56.516 ± 0.660  ns/op
ReadOnlyHttp2HeadersBenchmark.readOnlyClientHeaders             10  avgt   20   76.890 ± 0.776  ns/op
ReadOnlyHttp2HeadersBenchmark.readOnlyClientHeaders             20  avgt   20  117.531 ± 1.393  ns/op
ReadOnlyHttp2HeadersBenchmark.readOnlyServerHeaders              1  avgt   20   29.206 ± 0.264  ns/op
ReadOnlyHttp2HeadersBenchmark.readOnlyServerHeaders              5  avgt   20   44.587 ± 0.312  ns/op
ReadOnlyHttp2HeadersBenchmark.readOnlyServerHeaders             10  avgt   20   64.458 ± 1.169  ns/op
ReadOnlyHttp2HeadersBenchmark.readOnlyServerHeaders             20  avgt   20  107.179 ± 0.881  ns/op
ReadOnlyHttp2HeadersBenchmark.readOnlyTrailers                   1  avgt   20   21.563 ± 0.202  ns/op
ReadOnlyHttp2HeadersBenchmark.readOnlyTrailers                   5  avgt   20   41.019 ± 0.440  ns/op
ReadOnlyHttp2HeadersBenchmark.readOnlyTrailers                  10  avgt   20   64.053 ± 0.785  ns/op
ReadOnlyHttp2HeadersBenchmark.readOnlyTrailers                  20  avgt   20  113.737 ± 4.433  ns/op
```
2016-12-18 09:32:24 -08:00
Stephane Landelle
ba95c401a7 Misc clean up
Motivation:
IntelliJ issues several warnings.

Modifications:

* `ClientCookieDecoder` and `ServerCookieDecoder`:
  * `nameEnd`, `valueBegin` and `valueEnd` don't need to be initialized
  * `keyValLoop` loop doesn't been to be labelled, as it's the most inner one (same thing for labelled breaks)
  * Remove `if (i != headerLen)` as condition is always true
* `ClientCookieEncoder` javadoc still mention old logic
* `DefaultCookie`, `ServerCookieEncoder` and `DefaultHttpHeaders` use ternary ops that can be turned into simple boolean ones
* `DefaultHeaders` uses a for(int) loop over an array. It can be turned into a foreach one as javac doesn't allocate an iterator to iterate over arrays
* `DefaultHttp2Headers` and `AbstractByteBuf` `equal` can be turned into a single boolean statement
Result:

Cleaner code
2016-11-22 15:17:05 -08:00
Stephane Landelle
f755e58463 Clean up following #6016
Motivation:

* DefaultHeaders from netty-codec has some duplicated logic for header date parsing
* Several classes keep on using deprecated HttpHeaderDateFormat

Modifications:

* Move HttpHeaderDateFormatter to netty-codec and rename it into HeaderDateFormatter
* Make DefaultHeaders use HeaderDateFormatter
* Replace HttpHeaderDateFormat usage with HeaderDateFormatter

Result:

Faster and more consistent code
2016-11-21 12:35:40 -08:00
Norman Maurer
0bc30a123e Eliminate usage of releaseLater(...) to reduce memory usage during tests
Motiviation:

We used ReferenceCountUtil.releaseLater(...) in our tests which simplifies a bit the releasing of ReferenceCounted objects. The problem with this is that while it simplifies stuff it increase memory usage a lot as memory may not be freed up in a timely manner.

Modifications:

- Deprecate releaseLater(...)
- Remove usage of releaseLater(...) in tests.

Result:

Less memory needed to build netty while running the tests.
2016-11-18 09:34:11 +01:00
Scott Mitchell
e7631867d3 LzmaFrameEncoderTest double release
Motivation:
2c78902ebc ensured buffers were released in the general case but didn't clean up an extra release in LzmaFrameEncoderTest#testCompressionOfBatchedFlowOfData which lead to a double release.

Modifications:
LzmaFrameEncoderTest#testCompressionOfBatchedFlowOfData should not explicitly release the buffer because decompress will release the buffer

Result:
No more reference count exception and failed test.
2016-11-16 09:55:38 -08:00
Scott Mitchell
2c78902ebc LzmaFrameEncoderTest leak due to LzmaInputStream close behavior
Motivation:
c1932a8537 made an assumption that the LzmaInputStream which wraps a ByteBufInputStream would delegate the close operation to the wrapped stream. This assumption is not true and thus we still had a leak. An issue has been logged with our LZMA dependency https://github.com/jponge/lzma-java/issues/14.

Modifications:
- Force a close on the wrapped stream

Result:
No more leak.
2016-11-15 17:07:09 -08:00
Scott Mitchell
c1932a8537 ByteBuf Input Stream Reference Count Ownership
Motivation:
Netty provides a adaptor from ByteBuf to Java's InputStream interface. The JDK Stream interfaces have an explicit lifetime because they implement the Closable interface. This lifetime may be differnt than the ByteBuf which is wrapped, and controlled by the interface which accepts the JDK Stream. However Netty's ByteBufInputStream currently does not take reference count ownership of the underlying ByteBuf. There may be no way for existing classes which only accept the InputStream interface to communicate when they are done with the stream, other than calling close(). This means that when the stream is closed it may be appropriate to release the underlying ByteBuf, as the ownership of the underlying ByteBuf resource may be transferred to the Java Stream.

Motivation:
- ByteBufInputStream.close() supports taking reference count ownership of the underyling ByteBuf

Result:
ByteBufInputStream can assume reference count ownership so the underlying ByteBuf can be cleaned up when the stream is closed.
2016-11-14 16:29:55 -08:00
Scott Mitchell
d479e939b0 Buffer Leaks in Compression Tests
Motivation:
The unit tests for the compression encoders/decoders may write buffers to an EmbeddedChannel but then may not release buffer or close the channel after the test. This may result in buffer leaks.

Modifications:
- Call channel.finishAndReleaseAll() after each test

Result:
Fixes https://github.com/netty/netty/issues/6007
2016-11-14 16:24:22 -08:00
Scott Mitchell
e47da7be77 CompatibleObjectEncoder cached ObjectOutputStream backed by release buffer bug
Motivation:
ObjectOutputStream uses a Channel Attribute to cache a ObjectOutputStream which is backed by a ByteBuf that may be released after an object is encoded and the underlying buffer is written to the channel. On subsequent encode operations the cached ObjectOutputStream will be invalid and lead to a reference count exception.

Modifications:
- CompatibleObjectEncoder should not cache a ObjectOutputStream.

Result:
CompatibleObjectEncoder doesn't use a cached object backed by a released ByteBuf.
2016-11-10 10:04:01 -08:00
Norman Maurer
5f533b7358 [maven-release-plugin] prepare for next development iteration 2016-10-14 13:20:41 +02:00
Norman Maurer
35fb0babe2 [maven-release-plugin] prepare release netty-4.1.6.Final 2016-10-14 12:47:19 +02:00
knoyrok
0b939a96cf Fix typo in ProtobufDecoder comment 2016-10-10 07:40:35 +02:00
radai-rosenblatt
15ac6c4a1f Clean-up unused imports
Motivation:

the build doesnt seem to enforce this, so they piled up

Modifications:

removed unused import lines

Result:

less unused imports

Signed-off-by: radai-rosenblatt <radai.rosenblatt@gmail.com>
2016-09-30 09:08:50 +02:00
Scott Mitchell
e3462a79c7 MessageAggregator Potential Leak
Motivation:
MessageAggregator has a potential to leak if a new message is received before the existing message has completed, and if a HttpContent is received but maxContentLength has been exceeded, or the content length is too long.

Modifications:

- Make the HttpObjectAggregator more robust to leaks
- Reduce dependance on handlingOversizedMessage but instead rely on the more general check of a null currentMessage

Result:
More robust MessageAggregator with less chance of leaks
2016-09-14 10:13:49 -07:00
Norman Maurer
54b1a100f4 [maven-release-plugin] prepare for next development iteration 2016-08-26 10:06:32 +02:00
Norman Maurer
1208b90f57 [maven-release-plugin] prepare release netty-4.1.5.Final 2016-08-26 04:59:35 +02:00
Dmitriy Dumanskiy
a4d8f930af small performance fixes : unnecessary unboxing operations removed
Motivation :

Unboxing operations allocate unnecessary objects when it could be avoided.

Modifications:
Replaced Float.valueOf with Number.parseFloat where possible.

Result:

Less unnecessary objects allocations.
2016-08-01 07:10:25 +02:00
Norman Maurer
cb7cf4491c [maven-release-plugin] prepare for next development iteration 2016-07-27 13:29:56 +02:00
Norman Maurer
9466b32d05 [maven-release-plugin] prepare release netty-4.1.4.Final 2016-07-27 13:16:59 +02:00
Norman Maurer
f25f1f255d No need to do a memory copy to verify snappy identifier
Motivation:

We are currently doing a memory copy to verify the snapy version. This is not needed.

Modifications:

Remove memory copy and just compare byte per byte.

Result:

Less memory copies and allocations
2016-07-24 20:39:46 +02:00
Norman Maurer
7db3e01498 Ensure uncompressed ByteBuf is released when an exception happens during decoding.
Motivation:

We need to ensure the uncompressed ByteBuf is released if an exception happens while calling decode(...). If we miss to do so we leak buffers.

Modifications:

Correctly release buffer on exception.

Result:

No more memory leak.
2016-07-24 20:28:40 +02:00
Norman Maurer
3ebbd96820 Remove memory copies when doing CRC32 processing in JdkZlibDecoder
Motivation:

We not need to do any memory copies when doing CRC32 processing.

Modifications:

Use ByteBufChecksum to eliminate memory copies.

Result:

Less memory copies.
2016-07-22 20:02:12 +02:00