Commit Graph

646 Commits

Author SHA1 Message Date
Scott Mitchell
a7f7d9c8e0 Remove unsafe char[] access in PlatformDependent
Motivation:
PlatformDependent attempts to use reflection to get the underlying char[] (or byte[]) from String objects. This is fragile as if the String implementation does not utilize the full array, and instead uses a subset of the array, this optimization is invalid. OpenJDK6 and some earlier versions of OpenJDK7 String have the capability to use a subsection of the underlying char[].

Modifications:
- PlatformDependent should not attempt to use the underlying array from String (or other data types) via reflection

Result:
PlatformDependent hash code generation for CharSequence does not depend upon specific JDK implementation details.
2016-06-30 08:58:28 -07:00
Guido Medina
f0a5ee068f Update dependencies and plugins to latest possible versions.
Motivation:
It is good to have used dependencies and plugins up-to-date to fix any undiscovered bug fixed by the authors.

Modification:
Scanned dependencies and plugins and carefully updated one by one.

Result:
Dependencies and plugins are up-to-date.
2016-06-27 13:35:35 +02:00
Guido Medina
c3abb9146e Use shaded dependency on JCTools instead of copy and paste
Motivation:
JCTools supports both non-unsafe, unsafe versions of queues and JDK6 which allows us to shade the library in netty-common allowing it to stay "zero dependency".

Modifications:
- Remove copy paste JCTools code and shade the library (dependencies that are shaded should be removed from the <dependencies> section of the generated POM).
- Remove usage of OneTimeTask and remove it all together.

Result:
Less code to maintain and easier to update JCTools and less GC pressure as the queue implementation nt creates so much garbage
2016-06-10 13:19:45 +02:00
Norman Maurer
4dec7f11b7 [maven-release-plugin] prepare for next development iteration 2016-06-07 18:52:34 +02:00
Norman Maurer
cf670fab75 [maven-release-plugin] prepare release netty-4.1.1.Final 2016-06-07 18:52:22 +02:00
Norman Maurer
dc816e9807 Ensure we null out cumulation buffer before fire through the pipeline in handlerRemoved(...)
Motivation:

We should ensure we null out the cumulation buffer before we fire it through the pipleine in handlerRemoved(...) as in theory it could be possible that another method is triggered as result of the fireChannelRead(...) or fireChannelReadComplete() that will try to access the cumulation.

Modifications:

Null out cumulation buffer early in handlerRemoved(...)

Result:

No possible to access the cumulation buffer that was already handed over.
2016-06-04 09:18:07 +02:00
buchgr
af7b0a04a0 Fix DefaultHeaders.toString() for keys with multiple values.
Motivation:

For example,

DefaultHttp2Headers headers = new DefaultHttp2Headers();
headers.add("key1", "value1");
headers.add("key1", "value2");
headers.add("key1", "value3");
headers.add("key2", "value4");

produces:

DefaultHttp2Headers[key1: value1key1: value2key1: value3, key2: value4]

while correctly it should be

DefaultHttp2Headers[key1: value1, key1: value2, key1: value3, key2: value4]

Modifications:

Change the toString() method to produce the beforementioned output.

Result:

toString() format is correct also for keys with multiple values.
2016-06-02 17:24:51 +02:00
Norman Maurer
6ca49d1336 [maven-release-plugin] prepare for next development iteration 2016-05-25 19:16:44 +02:00
Norman Maurer
446b38db52 [maven-release-plugin] prepare release netty-4.1.0.Final 2016-05-25 19:14:15 +02:00
Norman Maurer
7b25402e80 Add CompositeByteBuf.addComponent(boolean ...) method to simplify usage
Motivation:

At the moment the user is responsible to increase the writer index of the composite buffer when a new component is added. We should add some methods that handle this for the user as this is the most popular usage of the composite buffer.

Modifications:

Add new methods that autoamtically increase the writerIndex when buffers are added.

Result:

Easier usage of CompositeByteBuf.
2016-05-21 19:52:16 +02:00
Norman Maurer
e59d0e9efb Introduce CodecOutputList to reduce overhead of encoder/decoder
Motivation:

99dfc9ea79 introduced some code that will more frequently try to forward messages out of the list of decoded messages to reduce latency and memory footprint. Unfortunally this has the side-effect that RecycleableArrayList.clear() will be called more often and so introduce some overhead as ArrayList will null out the array on each call.

Modifications:

- Introduce a CodecOutputList which allows to not null out the array until we recycle it and also allows to access internal array with extra range checks.
- Add benchmark that add elements to different List implementations and clear them

Result:

Less overhead when decode / encode messages.

Benchmark                                     (elements)   Mode  Cnt         Score        Error  Units
CodecOutputListBenchmark.arrayList                     1  thrpt   20  24853764.609 ± 161582.376  ops/s
CodecOutputListBenchmark.arrayList                     4  thrpt   20  17310636.508 ± 930517.403  ops/s
CodecOutputListBenchmark.codecOutList                  1  thrpt   20  26670751.661 ± 587812.655  ops/s
CodecOutputListBenchmark.codecOutList                  4  thrpt   20  25166421.089 ± 166945.599  ops/s
CodecOutputListBenchmark.recyclableArrayList           1  thrpt   20  24565992.626 ± 210017.290  ops/s
CodecOutputListBenchmark.recyclableArrayList           4  thrpt   20  18477881.775 ± 157003.777  ops/s

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 246.748 sec - in io.netty.handler.codec.CodecOutputListBenchmark
2016-05-20 09:12:07 +02:00
Trustin Lee
3a9f472161 Make retained derived buffers recyclable
Related: #4333 #4421 #5128

Motivation:

slice(), duplicate() and readSlice() currently create a non-recyclable
derived buffer instance. Under heavy load, an application that creates a
lot of derived buffers can put the garbage collector under pressure.

Modifications:

- Add the following methods which creates a non-recyclable derived buffer
  - retainedSlice()
  - retainedDuplicate()
  - readRetainedSlice()
- Add the new recyclable derived buffer implementations, which has its
  own reference count value
- Add ByteBufHolder.retainedDuplicate()
- Add ByteBufHolder.replace(ByteBuf) so that..
  - a user can replace the content of the holder in a consistent way
  - copy/duplicate/retainedDuplicate() can delegate the holder
    construction to replace(ByteBuf)
- Use retainedDuplicate() and retainedSlice() wherever possible
- Miscellaneous:
  - Rename DuplicateByteBufTest to DuplicatedByteBufTest (missing 'D')
  - Make ReplayingDecoderByteBuf.reject() return an exception instead of
    throwing it so that its callers don't need to add dummy return
    statement

Result:

Derived buffers are now recycled when created via retainedSlice() and
retainedDuplicate() and derived from a pooled buffer
2016-05-17 11:16:13 +02:00
Norman Maurer
9f5eb7d698 Add CharSequence operations to ByteBuf
Motivation:

Often users either need to read or write CharSequences to a ByteBuf. We should add methods for this to ByteBuf as we can do some optimizations for this depending on the implementation.

Modifications:

Add setCharSequence, writeCharSequence, getCharSequence and readCharSequence

Result:

Easier reading / writing of CharSequence with ByteBuf.
2016-05-06 16:52:00 +02:00
Xiaoyan Lin
ce1ae0eb8b Handle the backslash with double quote in JsonObjectDecoder
Motivation:

The double quote may be escaped in a JSON string, but JsonObjectDecoder doesn't handle it. Resolves #5157.

Modifications:

Don't end a JSON string when processing an escaped double quote.

Result:

JsonObjectDecoder can handle backslash and double quote in a JSON string correctly.
2016-05-04 14:04:39 +02:00
Norman Maurer
d698746609 Add ByteBuf.asReadOnly()
Motivation:

We lately added ByteBuf.isReadOnly() which allows to detect if a buffer is read-only or not. We should add ByteBuf.asReadOnly() to allow easily access a read-only version of a buffer.

Modifications:

- Add ByteBuf.asReadOnly()
- Deprecate Unpooled.unmodifiableBuffer(Bytebuf)

Result:

More consistent api.
2016-04-14 10:51:20 +02:00
Norman Maurer
6108b7297b Correctly handle ChannelInputShutdownEvent in ReplayingDecoder
Motivation:

b112673554 added ChannelInputShutdownEvent support to ByteToMessageDecoder but missed updating the code for ReplayingDecoder. This has the effect:

- If a ChannelInputShutdownEvent is fired ByteToMessageDecoder (the super-class of ReplayingDecoder) will call the channelInputClosed(...) method which will pass the incorrect buffer to the decode method of ReplayingDecoder.

Modifications:

Share more code between ByteToMessageDEcoder and ReplayingDecoder and so also support ChannelInputShutdownEvent correctly in ReplayingDecoder

Result:

ChannelInputShutdownEvent is corrrectly handle in ReplayingDecoder as well.
2016-04-14 10:20:58 +02:00
Norman Maurer
7c734fcf73 Fix resource leak in tests introduced by 69070c37ba. 2016-04-14 10:13:55 +02:00
Norman Maurer
1861461db2 [#4635] Stop decoding if decoder was removed in ReplayingDecoder
We need to check if this handler was removed before continuing with decoding.
If it was removed, it is not safe to continue to operate on the buffer. This was already fixed for ByteToMessageDecoder in 4cdbe39284  but missed for ReplayingDecoder.

Modifications:

Check if decoder was removed after fire messages through the pipeline.

Result:

No illegal buffer access when decoder was removed.
2016-04-14 08:25:59 +02:00
Trustin Lee
0b078314b2 Add ByteBuf.isReadOnly()
Motivation:

It is sometimes useful to determins if a buffer is read-only.

Modifications:

Add ByteBuf.isReadOnly()

Result:

One more feature
2016-04-13 21:41:27 +09:00
Norman Maurer
572bdfb494 [maven-release-plugin] prepare for next development iteration 2016-04-10 08:37:18 +02:00
Norman Maurer
c6121a6f49 [maven-release-plugin] prepare release netty-4.1.0.CR7 2016-04-10 08:36:56 +02:00
Norman Maurer
6e919f70f8 [maven-release-plugin] rollback the release of netty-4.1.0.CR7 2016-04-09 22:13:44 +02:00
Norman Maurer
4cdd51509a [maven-release-plugin] prepare release netty-4.1.0.CR7 2016-04-09 22:05:34 +02:00
Norman Maurer
d081851156 Remove ByteBuf.readBytes(int) calls when possible
Motivation:

We use ByteBuf.readBytes(int) in various places where we could either remove it completely or use readSlice(int).retain().

Modifications:

- Remove ByteBuf.readBytes(int) when possible or replace by readSlice(int).retain().

Result:

Faster code.
2016-04-09 18:40:57 +02:00
Trustin Lee
3b941c2a7c [maven-release-plugin] prepare for next development iteration 2016-04-02 01:25:05 -04:00
Trustin Lee
7368ccc539 [maven-release-plugin] prepare release netty-4.1.0.CR6 2016-04-02 01:24:55 -04:00
Norman Maurer
cee38ed2b6 [maven-release-plugin] prepare for next development iteration 2016-03-29 16:45:13 +02:00
Norman Maurer
9cd9e7daeb [maven-release-plugin] prepare release netty-4.1.0.CR5 2016-03-29 16:44:33 +02:00
Norman Maurer
28d03adbfe [maven-release-plugin] prepare for next development iteration 2016-03-21 11:51:50 +01:00
Norman Maurer
4653dc1d05 [maven-release-plugin] prepare release netty-4.1.0.CR4 2016-03-21 11:51:12 +01:00
Xiaoyan Lin
01835fdf18 Add LineEncoder to append a line separator automatically
Motivation:

See #1811

Modifications:

Add LineEncoder and LineSeparator

Result:

The user can use LineEncoder to write a String with a line separator automatically
2016-03-16 20:31:01 +01:00
Xiaoyan Lin
4fb585965c Add DatagramPacketEncoder and DatagramPacketDecoder
Motivation:

UDP-oriented codec reusing the existing encoders and decoders would be helpful. See #1350

Modifications:

Add DatagramPacketEncoder and DatagramPacketDecoder to reuse the existing encoders and decoders.

Result:

People can use DatagramPacketEncoder and DatagramPacketDecoder to wrap existing encoders and decoders to create UDP-oriented codec.
2016-03-14 12:14:57 +01:00
Norman Maurer
9aac6dac2e [#4386] ByteToMessage.decodeLast(...) should not call decode(...) if buffer is empty.
Motivation:

If the input buffer is empty we should not have decodeLast(...) call decode(...) as the user may not expect this.

Modifications:

- Not call decode(...) in decodeLast(...) if the input buffer is empty.
- Add testcases.

Result:

decodeLast(...) will not call decode(...) if input buffer is empty.
2016-03-01 08:42:26 +01:00
Sergey Polovko
a2e113b987 Use ByteProcessor.FIND_LF to find end of line
Motivation:

Each call of ByteBuf.getByte(int) method does boundary checking. This can be eliminated by using ByteBuf.forEachByte(ByteProcessor) method and ByteProcessor.FIND_LF processor.

Modifications:

Find end of line with ByteProcessor.FIND_LF

Result:

A little better performance of LineBasedFrameDecoder.
2016-02-29 09:38:36 +01:00
Norman Maurer
0bea10b0b0 [#4892] Make Snappy class public
Motivation:

Some people may want to use the Snappy class directly to encode / decode ByteBufs.

Modifications:

Make the Snappy class public and final.

Result:

Easier for people to reuse parts of Netty.
2016-02-26 12:15:10 +01:00
Norman Maurer
ca443e42e0 [maven-release-plugin] prepare for next development iteration 2016-02-19 23:00:11 +01:00
Norman Maurer
f39eb9a6b2 [maven-release-plugin] prepare release netty-4.1.0.CR3 2016-02-19 22:59:52 +01:00
Norman Maurer
0b1c82b254 Ensure read components can be discarded when COMPOSITE_CUMULATOR is used.
Motivation:

ByteToMessageDecoder must ensure that read components of the CompositeByteBuf can be discard by default when discardSomeReadBytes() is called. This may not be the case before as because of the default maxNumComponents that will cause consolidation.

Modifications:

Ensure we not do any consolidation to actually be abel to discard read components

Result:

Less memory usage and allocations.
2016-02-17 19:51:08 -08:00
Xiaoyan Lin
ccb0870600 Add methods with position independent FileChannel calls to ByteBuf
Motivation

See ##3229

Modifications:

Add methods with position independent FileChannel calls to ByteBuf and its subclasses.

Results:

The user can use these new methods to read/write ByteBuff without updating FileChannel's position.
2016-02-14 20:37:37 -08:00
Scott Mitchell
b112673554 ByteToMessageDecoder ChannelInputShutdownEvent support
Motivation:
b714297a44 introduced ChannelInputShutdownEvent support for HttpObjectDecoder. However this should have been added to the super class ByteToMessageDecoder, and ByteToMessageDecoder should not propegate a channelInactive event through the pipeline in this case.

Modifications:
- Move the ChannelInputShutdownEvent handling from HttpObjectDecoder to ByteToMessageDecoder
- ByteToMessageDecoder doesn't call ctx.fireChannelInactive() on ChannelInputShutdownEvent

Result:
Half closed events are treated more generically, and don't get translated into a channelInactive pipeline event.
2016-02-12 16:15:17 -08:00
Norman Maurer
65b3470456 [#4793] Correctly add newlines when encode base64
Motivation:

We not correctly added newlines if the src data needed to be padded. This regression was introduced by '63426fc3ed083513c07a58b45381f5c10dd47061'

Modifications:

- Correctly handling newlines
- Add unit test that proves the fix.

Result:

No more invalid base64 encoded data.
2016-02-06 09:56:21 +01:00
Norman Maurer
75a2ddd61c [maven-release-plugin] prepare for next development iteration 2016-02-04 16:51:44 +01:00
Norman Maurer
7eb3a60dba [maven-release-plugin] prepare release netty-4.1.0.CR2 2016-02-04 16:37:06 +01:00
Norman Maurer
a0758e7e60 [#4794] Support window size flag by default if ZlibCodecFactory supports it.
Motivation:

If the ZlibCodecFactory can support using a custom window size we should support it by default in the websocket extensions as well.

Modifications:

Detect if a custom window size can be handled by the ZlibCodecFactory and if so enable it by default for PerMessageDeflate*ExtensionHandshaker.

Result:

Support window size flag by default in most installations.
2016-02-04 14:01:40 +01:00
Norman Maurer
1c417e5f82 [maven-release-plugin] prepare for next development iteration 2016-01-21 15:35:55 +01:00
Norman Maurer
c681a40a78 [maven-release-plugin] prepare release netty-4.1.0.CR1 2016-01-21 15:28:21 +01:00
Xiaoyan Lin
d1ef33b8f4 Change 64 to 63 in Snappy.decodeLiteral
Motivation:

According to https://github.com/google/snappy/blob/master/format_description.txt#L55 , Snappy.decodeLiteral should handle the cases of 60, 61, 62 and 63. However right now it processes 64 instead of 63. I believe it's a typo since `tag >> 2 & 0x3F` must be less than 64.

Modifications:

Use the correct value 63.

Result:

Snappy.decodeLiteral handles the correct case.
2016-01-21 09:59:10 +01:00
Robert Borg
3785ca9311 added support for Protobuf codec nano runtime
Motivation:

Netty was missing support for Protobuf nano runtime targeted at
weaker systems such as Android devices.

Modifications:

Added ProtobufDecoderNano and ProtobufDecoderNano
in order to provide support for Nano runtime.

modified ProtobufVarint32FrameDecoder and
ProtobufLengthFieldPrepender in order to remove any
on either Nano or Lite runtime by copying the code
for handling Protobuf varint32 in from Protobuf
library.

modified Licenses and NOTICE in order to reflect the
changes i made.

added Protobuf Nano runtime as optional dependency

Result:

Netty now supports Protobuf Nano runtime.
2016-01-19 21:39:17 +01:00
Stephane Landelle
f7c83c8565 Don't enforce JZlibDecoder on JDK7+, close #4707
Motivation:

`JdkZlibDecoder` is available since Netty 4.0.8 and works with JDK7+.
However, `io.netty.noJdkZlibDecoder` System prop evaluation always defaults to
true, causing Netty to always use JZLib when decompressing on the
client side when the property insn't explictly set to `false`.

Modifications:

Default to `false` instead of `true` when JDK7+.

Result:

JZLib optional as expected on JDK7+.
2016-01-14 21:04:15 +01:00
Norman Maurer
4cdbe39284 [#4635] Stop decoding if decoder was removed
Motivation:

We need to check if this handler was removed before continuing with decoding.
If it was removed, it is not safe to continue to operate on the buffer.

Modifications:

Check if decoder was removed after fire messages through the pipeline.

Result:

No illegal buffer access when decoder was removed.
2016-01-05 11:01:24 +01:00