Commit Graph

727 Commits

Author SHA1 Message Date
Trustin Lee
8cb6b779c9 Add io.netty.noJdkZlibEncoder system property
Related issue: #2821

Motivation:

There's no way for a user to change the default ZlibEncoder
implementation.

It is already possible to change the default ZlibDecoder implementation.

Modification:

Add a new system property 'io.netty.noJdkZlibEncoder'.

Result:

A user can disable JDK ZlibEncoder, just like he or she can disable JDK
ZlibDecoder.
2014-08-26 15:51:11 +09:00
Norman Maurer
b201877734 Move duplicated code into CodecUtil
Motivation:

We have some duplicated code that can be reused.

Modifications:

Create package private class called CodecUtil that now contains the shared code / helper method.

Result:

Less code-duplication
2014-08-25 07:37:23 +02:00
Norman Maurer
fbf8533759 [#2812] Ensure we call checkForSharableAnnotation in all constructors of ByteToMessageCodec
Motivation:

ByteToMessageCodec miss to check for @Sharable annotation in one of its constructors.

Modifications:

Ensure we call checkForSharableAnnotation in all constructors.

Result:

After your change, what will change.
2014-08-23 21:03:25 +02:00
Trustin Lee
b5f61d0de5 [maven-release-plugin] prepare for next development iteration 2014-08-16 03:27:42 +09:00
Trustin Lee
76ac3b21a5 [maven-release-plugin] prepare release netty-4.1.0.Beta3 2014-08-16 03:27:37 +09:00
Trustin Lee
b3c1904cc9 [maven-release-plugin] prepare for next development iteration 2014-08-15 09:31:03 +09:00
Trustin Lee
e013b2400f [maven-release-plugin] prepare release netty-4.1.0.Beta2 2014-08-15 09:30:59 +09:00
Trustin Lee
1971bd1da6 Rename SnappyFramedEncoder/Decoder to SnappyFrameEncoder/Decoder
Related issue: #2766

Motivation:

Forgot to rename them before the final release by mistake.

Modifications:

Rename and then re-introduce the deprecated version that extends the
renamed class.

Result:

Better naming
2014-08-14 15:17:10 -07:00
Idel Pivnitskiy
c8841bc9de Implemented LZ4 compression codec
Motivation:

LZ4 compression codec provides sending and receiving data encoded by very fast LZ4 algorithm.

Modifications:

- Added `lz4` library which implements LZ4 algorithm.
- Implemented Lz4FramedEncoder which extends MessageToByteEncoder and provides compression of outgoing messages.
- Added tests to verify the Lz4FramedEncoder and how it can compress data for the next uncompression using the original library.
- Implemented Lz4FramedDecoder which extends ByteToMessageDecoder and provides uncompression of incoming messages.
- Added tests to verify the Lz4FramedDecoder and how it can uncompress data after compression using the original library.
- Added integration tests for Lz4FramedEncoder/Decoder.

Result:

Full LZ4 compression codec which can compress/uncompress data using LZ4 algorithm.
2014-08-14 15:05:24 -07:00
Trustin Lee
1aa20e8609 Fix class description of FastLzFrameDecoder 2014-08-13 22:58:45 -07:00
Trustin Lee
f311012455 Rename FastLzFramed* to FastLzFrame* 2014-08-13 22:55:34 -07:00
Norman Maurer
d315bbaa14 Fix and clearify javadocs
Motivation:

ByteToMessageDecoder and ReplayingDecoder have incorrect javadocs in some places.

Modifications:

Fix incorrect javadocs for both classes.

Result:

Correct javadocs for both classes
2014-08-14 06:46:29 +02:00
Idel Pivnitskiy
a0c466a276 Implemented FastLZ compression codec
Motivation:

FastLZ compression codec provides sending and receiving data encoded by fast FastLZ algorithm using block mode.

Modifications:

- Added part of `jfastlz` library which implements FastLZ algorithm. See FastLz class.
- Implemented FastLzFramedEncoder which extends MessageToByteEncoder and provides compression of outgoing messages.
- Implemented FastLzFramedDecoder which extends ByteToMessageDecoder and provides uncompression of incoming messages.
- Added integration tests for `FastLzFramedEncoder/Decoder`.

Result:

Full FastLZ compression codec which can compress/uncompress data using FastLZ algorithm.
2014-08-12 15:14:59 -07:00
Trustin Lee
60764200d7 Do not throw an exception when failed to get a header
Motivation:

It is often very expensive to instantiate an exception. TextHeader
should not raise an exception when it failed to find a header or when
its header value is not valid.

Modification:

- Change the return type of the getter methods to Integer and Long so
  that null is returned when no header is found or its value is invalid
- Update Javadoc

Result:

- Fixes #2758
- No unnecessary instantiation of exceptions
2014-08-12 11:14:06 -07:00
Trustin Lee
de724063f3 Reduce the initial capacity of the value list from 4 to 2
Motivation:

DefaultTextHeaders.getAll*() methods create an ArrayList whose initial
capacity is 4.  However, it is more likely that the actual number of
values is smaller than that.

Modifications:

Reduce the initial capacity of the value list from 4 to 2

Result:

Slightly reduced memory footprint
2014-08-12 10:36:13 -07:00
jxu
2d36caa9f6 Add TextHeaders.getAndRemove(...) and its variants
Related issue: #2649 and #2745

Motivation:

At the moment there is no way to get and remove a header with one call.
This means you need to search the headers two times. We should add
getAndRemove(...) to allow doing so with one call.

Modifications:

Add getAndRemove(...) and getUnconvertedAndRemove(...) and their
variants

Result:

More efficient API
2014-08-12 10:33:04 -07:00
Idel Pivnitskiy
073ec8d10a Consider writerIndex when LzfDecoder writes into a new heap buffer
Motivation:

Now LzfDecoder do not consider writerIndex when it writes into array of a new heap buffer (when it decodes a compressed chuck of data)
2014-08-05 22:51:02 +02:00
Idel Pivnitskiy
c13419750d Improve Bzip2BitReader/Writer
Motivation:

Before this changes Bzip2BitReader and Bzip2BitWriter accessed to ByteBuf byte by byte. So tests for Bzip2 compression codec takes a lot of time if we ran them with paranoid level of resource leak detection. For more information see comments to #2681 and #2689.

Modifications:

- Increased size of bit buffers from 8 to 64 bits.
- Improved reading and writing operations.
- Save link to incoming ByteBuf inside Bzip2BitReader.
- Added methods to check possible readable bits and bytes in Bzip2BitReader.
- Updated Bzip2 classes to use new API of Bzip2BitReader.
- Added new constants to Bzip2Constants.

Result:

Increased size of bit buffers and improved performance of Bzip2 compression codec (for general work by 13% and for tests with paranoid level of resource leak detection by 55%).
2014-08-04 07:52:40 +02:00
Norman Maurer
e1cc1fbabc [#2705] Call fireChannelReadComplete() if channelActive(...) decodes messages in ReplayingDecoder / ByteToMessageDecoder
Motivation:

In ReplayingDecoder / ByteToMessageDecoder channelInactive(...) method we try to decode a last time and fire all decoded messages throw the pipeline before call ctx.fireChannelInactive(...). To keep the correct order of events we also need to call ctx.fireChannelReadComplete() if we read anything.

Modifications:

- Channel channelInactive(...) to call ctx.fireChannelReadComplete() if something was decoded
- Move out.recycle() to finally block

Result:

Correct order of events.
2014-07-24 14:38:46 +02:00
Idel Pivnitskiy
4816533638 Refactor Bzip2 tests
Motivation:

Complicated code of Bzip2 tests with some unnecessary actions.

Modifications:

- Reduce size of BYTES_LARGE array of random test data for Bzip2  tests.
- Removed unnecessary creations of EmbeddedChannel instances in Bzip2 tests.
- Simplified tests in Bzip2DecoderTest which expect exception.
- Removed unnecessary testStreamInitialization() from Bzip2EncoderTest.

Result:

Reduced time to test the 'codec' package by 7 percent, simplified code of Bzip2 tests.
2014-07-23 19:46:00 +02:00
Idel Pivnitskiy
99cf6f0732 Refactor integration tests of compression codecs
Motivation:

Duplicated code of integration tests for different compression codecs.

Modifications:

- Added abstract class IntegrationTest which contains common tests for any compression codec.
- Removed common tests from Bzip2IntegrationTest and LzfIntegrationTest.
- Implemented abstract methods of IntegrationTest in Bzip2IntegrationTest, LzfIntegrationTest and SnappyIntegrationTest.

Result:

Removed duplicated code of integration tests for compression codecs and simplified an addition of integration tests for new compression codecs.
2014-07-23 19:44:10 +02:00
Idel Pivnitskiy
ca87cc887e Simplify Bzip2 tests
Motivation:

Sometimes we have a 'build time out' error because tests for bzip2 codec take a long time.

Modifications:

Removed cycles from Bzip2EncoderTest.testCompression(byte[]) and Bzip2DecoderTest.testDecompression(byte[]).

Result:

Reduced time to test the 'codec' package by 30 percent.
2014-07-22 18:00:26 +02:00
Idel Pivnitskiy
dc9d933d74 Fixes for compression codecs
Motivation:

Fixed founded mistakes in compression codecs.

Modifications:

- Changed return type of ZlibUtil.inflaterException() from CompressionException to DecompressionException
- Updated @throws in javadoc of JZlibDecoder to throw DecompressionException instead of CompressionException
- Fixed JdkZlibDecoder to throw DecompressionException instead of CompressionException
- Removed unnecessary empty lines in JdkZlibEncoder and JZlibEncoder
- Removed public modifier from Snappy class
- Added MAX_UNCOMPRESSED_DATA_SIZE constant in SnappyFramedDecoder
- Used in.readableBytes() instead of (in.writerIndex() - in.readerIndex()) in SnappyFramedDecoder
- Added private modifier for enum ChunkType in SnappyFramedDecoder
- Fixed potential bug (sum overflow) in Bzip2HuffmanAllocator.first(). For more info, see http://googleresearch.blogspot.ru/2006/06/extra-extra-read-all-about-it-nearly.html

Result:

Fixed sum overflow in Bzip2HuffmanAllocator, improved exceptions in ZlibDecoder implementations, hid Snappy class
2014-07-20 09:32:53 +02:00
Idel Pivnitskiy
0cc3eccc2b Close ObjectInputStream in ObjectDecoder.decode(...)
Motivation:

We create a new CompactObjectInputStream with ByteBufInputStream in ObjectDecoder.decode(...) method and don't close this InputStreams before return statement.

Modifications:

Save link to the ObjectInputStream and close it before return statement.

Result:

Close InputStreams and clean up unused resources. It will be better for GC.
2014-07-20 09:23:35 +02:00
Norman Maurer
53141b04a8 Fix buffer leak in Bzip2EncoderTest 2014-07-19 14:41:18 +02:00
Idel Pivnitskiy
ed7240b597 Implemented a Bzip2Encoder
Motivation:

Bzip2Encoder provides sending data compressed in bzip2 format.

Modifications:

Added classes:
- Bzip2Encoder
- Bzip2BitWriter
- Bzip2BlockCompressor
- Bzip2DivSufSort
- Bzip2HuffmanAllocator
- Bzip2HuffmanStageEncoder
- Bzip2MTFAndRLE2StageEncoder
- Bzip2EncoderTest

Modified classes:
- Bzip2Constants (splited BLOCK_HEADER_MAGIC and END_OF_STREAM_MAGIC)
- Bzip2Decoder (use splited magic numbers)

Added integration tests for Bzip2Encoder/Decoder

Result:

Implemented new encoder which can compress outgoing data in bzip2 format.
2014-07-17 16:19:39 +02:00
Idel Pivnitskiy
3c6017a9b1 Implemented LZF compression codec
Motivation:

LZF compression codec provides sending and receiving data encoded by very fast LZF algorithm.

Modifications:

- Added Compress-LZF library which implements LZF algorithm
- Implemented LzfEncoder which extends MessageToByteEncoder and provides compression of outgoing messages
- Added tests to verify the LzfEncoder and how it can compress data for the next uncompression using the original library
- Implemented LzfDecoder which extends ByteToMessageDecoder and provides uncompression of incoming messages
- Added tests to verify the LzfDecoder and how it can uncompress data after compression using the original library
- Added integration tests for LzfEncoder/Decoder

Result:

Full LZF compression codec which can compress/uncompress data using LZF algorithm.
2014-07-17 07:18:07 +02:00
Norman Maurer
81126ab429 Fix checkstyle error introduced by 52cb55d388 2014-07-10 07:12:40 +02:00
Norman Maurer
17280116c4 [#2643] Throw TooLongFrameException instead of using fireExceptionCaught
Motivation:

It's not always the case that there is another handler in the pipeline that will intercept the exceptionCaught event because sometimes users just sub-class. In this case the exception will just hit the end of the pipeline.

Modification:
Throw the TooLongFrameException so that sub-classes can handle it in the exceptionCaught(...) method directly.

Result:
Sub-classes can correctly handle the exception,
2014-07-10 06:56:28 +02:00
Idel Pivnitskiy
deda8f15a2 Moved bit-level read operations from Bzip2Decoder to the new Bzip2BitReader
Motivation:

Collect all bit-level read operations in one class is better. And now it's easy to use not only in Bzip2Decoder. For example, in Bzip2HuffmanStageDecoder.

Modifications:

Created a new class - Bzip2BitReader which provides bit-level reads.
Removed bit-level read operations from Bzip2Decoder.
Improved javadoc.

Result:

Bzip2BitReader allows the reading of single bit booleans, bit strings of arbitrary length (up to 24 bits), and bit aligned 32-bit integers.
2014-07-09 13:50:30 +02:00
Trustin Lee
e167b02d52 [maven-release-plugin] prepare for next development iteration 2014-07-04 17:26:02 +09:00
Trustin Lee
ba50cb829b [maven-release-plugin] prepare release netty-4.1.0.Beta1 2014-07-04 17:25:54 +09:00
Trustin Lee
787663a644 [maven-release-plugin] rollback the release of netty-4.1.0.Beta1 2014-07-04 17:11:14 +09:00
Trustin Lee
83eae705e1 [maven-release-plugin] prepare release netty-4.1.0.Beta1 2014-07-04 17:02:17 +09:00
Trustin Lee
97825598d2 Fix another buffer leaks in JsonObjectDecoderTest 2014-07-04 16:12:06 +09:00
Trustin Lee
cde7157c39 Make JsonObjectDecoder discard everything after stream corruption
Motivation:

There's no way to recover from a corrupted JSON stream. The current
implementation will raise an infinite exception storm when a peer sends
a large corrupted stream.

Modification:

Discard everything once stream corruption is detected.

Result:

Fixes a buffer leak
Fixes exception storm
2014-07-04 11:16:00 +09:00
Trustin Lee
a1974ef35b Fix an inspector warning in JsonObjectDecoder 2014-07-03 20:01:48 +09:00
Trustin Lee
d0b355b26e Fix a buffer leak in JsonObjectDecoderTest 2014-07-03 19:58:06 +09:00
Jakob Buchgraber
aed13ba5ef Split a JSON byte stream into JSON objects/arrays. Fixes #2536
Motivation:

See GitHub Issue #2536.

Modifications:

Introduce the class JsonObjectDecoder to split a JSON byte stream
into individual JSON objets/arrays.

Result:

A Netty application can now handle a byte stream where multiple JSON
documents follow eachother as opposed to only a single JSON document
per request.
2014-07-03 18:34:29 +09:00
Trustin Lee
d0912f2709 Fix most inspector warnings
Motivation:

It's good to minimize potentially broken windows.

Modifications:

Fix most inspector warnings from our profile
Update IntObjectHashMap

Result:

Cleaner code
2014-07-02 19:55:07 +09:00
xmxsuperstar
79195da0d7 fix example missing break statement in ReplayingDecoder 2014-06-28 21:42:23 +02:00
Zhihui Jiao
c4441642bb Fix inconsistent code in the doc 2014-06-27 06:48:40 +02:00
Trustin Lee
d2e385acc2 Fix incorrect bytesBefore/indexOf() in ReplayingDecoderBuffer
Motivation:

bytesBefore(length, ...), bytesBefore(index, length, ...), and
indexOf(fromIndex, toIndex,...) in ReplayingDecoderBuffer are buggy.
They trigger 'REPLAY even when they don't need to.

Modification:

Implement the buggy methods properly so that REPLAYs are not triggered
unnecessarily.

Result:

Correct behvaior
2014-06-26 18:56:33 +09:00
Norman Maurer
4d2b78ca3c Reduce the memory copies in JdkZlibEncoder
Motivation:

At the moment we use a lot of unnecessary memory copies in JdkZlibEncoder. This is caused by either allocate a to small ByteBuf and expand it later or using a temporary byte array.
Beside this the memory footprint of JdkZlibEncoder is pretty high because of the byte[] used for compressing.

Modification:

- Override allocateBuffer(...) and calculate the estimatedsize in there, this reduce expanding of the ByteBuf later
- Not use byte[] in the instance itself but allocate a heap ByteBuf and write directly into the byte array

Result:

Less memory copies and smaller memory footprint
2014-06-26 11:12:19 +02:00
Trustin Lee
937f790f70 Checkstyle 2014-06-26 17:48:32 +09:00
Trustin Lee
5f889d92a1 Fix buffer leaks in Bzip2Decoder(Test)
If decompression fails, the buffer that contains the decompressed data
is not released.  Bzip2DecoderTest.testStreamCrcError() also does not
release the partial output Bzip2Decoder produces.
2014-06-26 17:48:32 +09:00
Norman Maurer
56d732d439 Fix buffer leaks in Bzip2DecoderTest 2014-06-26 09:21:44 +02:00
Trustin Lee
bf85af5743 Fix buffer leaks in Bzip2DecoderTest 2014-06-24 16:47:14 +09:00
Idel Pivnitskiy
f9021a6061 Implement a Bzip2Decoder
Motivation:

Bzip2Decoder provides receiving data compressed in bzip2 format.

Modifications:

Added classes:
- Bzip2Decoder
- Bzip2Constants
- Bzip2BlockDecompressor
- Bzip2HuffmanStageDecoder
- Bzip2MoveToFrontTable
- Bzip2Rand
- Crc32
- Bzip2DecoderTest

Result:

Implemented and tested new decoder which can uncompress incoming data in bzip2 format.
2014-06-24 14:50:09 +09:00
Norman Maurer
12a3e23e47 MessageToByteEncoder always starts with ByteBuf that use initalCapacity == 0
Motivation:

MessageToByteEncoder always starts with ByteBuf that use initalCapacity == 0 when preferDirect is used. This is really wasteful in terms of performance as every first write into the buffer will cause an expand of the buffer itself.

Modifications:

 - Change ByteBufAllocator.ioBuffer() use the same default initialCapacity as heapBuffer() and directBuffer()
 - Add new allocateBuffer method to MessageToByteEncoder that allow the user to do some smarter allocation based on the message that will be encoded.

Result:

Less expanding of buffer and more flexibilty when allocate the buffer for encoding.
2014-06-24 13:55:21 +09:00
Trustin Lee
8c25830b0b Move haproxy codec to a separate module 2014-06-21 15:59:21 +09:00
Jon Keys
d7b2affe32 Add HAProxy protocol decoder
Motivation:

The proxy protocol provides client connection information for proxied
network services. Several implementations exist (e.g. Haproxy, Stunnel,
Stud, Postfix), but the primary motivation for this implementation is to
support the proxy protocol feature of Amazon Web Services Elastic Load
Balancing.

Modifications:

This commit adds a proxy protocol decoder for proxy protocol version 1
as specified at:

  http://haproxy.1wt.eu/download/1.5/doc/proxy-protocol.txt

The foundation for version 2 support is also in this commit but it is
explicitly NOT supported due to a lack of external implementations to
test against.

Result:

The proxy protocol decoder can be used to send client connection
information to inbound handlers in a channel pipeline from services
which support the proxy protocol.
2014-06-21 15:59:21 +09:00
Trustin Lee
085a61a310 Refactor FastThreadLocal to simplify TLV management
Motivation:

When Netty runs in a managed environment such as web application server,
Netty needs to provide an explicit way to remove the thread-local
variables it created to prevent class loader leaks.

FastThreadLocal uses different execution paths for storing a
thread-local variable depending on the type of the current thread.
It increases the complexity of thread-local removal.

Modifications:

- Moved FastThreadLocal and FastThreadLocalThread out of the internal
  package so that a user can use it.
- FastThreadLocal now keeps track of all thread local variables it has
  initialized, and calling FastThreadLocal.removeAll() will remove all
  thread-local variables of the caller thread.
- Added FastThreadLocal.size() for diagnostics and tests
- Introduce InternalThreadLocalMap which is a mixture of hard-wired
  thread local variable fields and extensible indexed variables
- FastThreadLocal now uses InternalThreadLocalMap to implement a
  thread-local variable.
- Added ThreadDeathWatcher.unwatch() so that PooledByteBufAllocator
  tells it to stop watching when its thread-local cache has been freed
  by FastThreadLocal.removeAll().
- Added FastThreadLocalTest to ensure that removeAll() works
- Added microbenchmark for FastThreadLocal and JDK ThreadLocal
- Upgraded to JMH 0.9

Result:

- A user can remove all thread-local variables Netty created, as long as
  he or she did not exit from the current thread. (Note that there's no
  way to remove a thread-local variable from outside of the thread.)
- FastThreadLocal exposes more useful operations such as isSet() because
  we always implement a thread local variable via InternalThreadLocalMap
  instead of falling back to JDK ThreadLocal.
- FastThreadLocalBenchmark shows that this change improves the
  performance of FastThreadLocal even more.
2014-06-19 21:13:55 +09:00
Norman Maurer
984b0aa961 [#2572] Correctly calculate length of output buffer before inflate to fix IndexOutOfBoundException
Motivation:

JdkZlibDecoder fails to decode because the length of the output buffer is not calculated correctly.
This can cause an IndexOutOfBoundsException or data-corruption when the PooledByteBuffAllocator is used.

Modifications:

Correctly calculate the length

Result:

No more IndexOutOfBoundsException or data-corruption.
2014-06-16 10:17:02 +02:00
Trustin Lee
776ac4ba19 Use FastThreadLocal in more places 2014-06-14 17:46:10 +09:00
Trustin Lee
c076c33901 Backport the additional AsciiString/TextHeader changes from master
- Add useful static methods to AsciiString
- Add more getters in TextHeaders
- Remove unnecessary utility methods in SpdyHttpHeaders
2014-06-14 17:33:34 +09:00
Trustin Lee
681d460938 Introduce TextHeaders and AsciiString
Motivation:

We have quite a bit of code duplication between HTTP/1, HTTP/2, SPDY,
and STOMP codec, because they all have a notion of 'headers', which is a
multimap of string names and values.

Modifications:

- Add TextHeaders and its default implementation
- Add AsciiString to replace HttpHeaderEntity
  - Borrowed some portion from Apache Harmony's java.lang.String.
- Reimplement HttpHeaders, SpdyHeaders, and StompHeaders using
  TextHeaders
- Add AsciiHeadersEncoder to reuse the encoding a TextHeaders
  - Used a dedicated encoder for HTTP headers for better performance
    though
- Remove shortcut methods in SpdyHeaders
- Replace SpdyHeaders.getStatus() with HttpResponseStatus.parseLine()

Result:

- Removed quite a bit of code duplication in the header implementations.
- Slightly better performance thanks to improved header validation and
  hash code calculation
2014-06-14 15:36:19 +09:00
belliottsmith
2a2a21ec59 Introduce FastThreadLocal which uses an EnumMap and a predefined fixed set of possible thread locals
Motivation:
Provide a faster ThreadLocal implementation

Modification:
Add a "FastThreadLocal" which uses an EnumMap and a predefined fixed set of possible thread locals (all of the static instances created by netty) that is around 10-20% faster than standard ThreadLocal in my benchmarks (and can be seen having an effect in the direct PooledByteBufAllocator benchmark that uses the DEFAULT ByteBufAllocator which uses this FastThreadLocal, as opposed to normal instantiations that do not, and in the new RecyclableArrayList benchmark);

Result:
Improved performance
2014-06-13 10:56:18 +02:00
Trustin Lee
8b0a0f9a8f Introduce MessageAggregator and DecoderResultProvider
Motivation:

We have different message aggregator implementations for different
protocols, but they are very similar with each other.  They all stems
from HttpObjectAggregator.  If we provide an abstract class that provide
generic message aggregation functionality, we will remove their code
duplication.

Modifications:

- Add MessageAggregator which provides generic message aggregation
- Reimplement all existing aggregators using MessageAggregator
- Add DecoderResultProvider interface and extend it wherever possible so
  that MessageAggregator respects the state of the decoded message

Result:

Less code duplication
2014-06-05 16:51:14 +09:00
Norman Maurer
b0ddfb9b65 [#2525] Use VoidChannelPromise in MessageToMessageEncoder when possible
Motivation:
At the moment MessageToMessageEncoder uses ctx.write(msg) when have more then one message was produced. This may produce more GC pressure then necessary as when the original ChannelPromise is a VoidChannelPromise we can safely also use one when write messages.

Modifications:
Use VoidChannelPromise when the original ChannelPromise was of this type

Result:
Less object creation and GC pressure
2014-06-01 19:26:20 +02:00
Norman Maurer
880acbca72 Use ByteBuf.readSlice(...).retain() to minimize memory copies.
Motivation:
At the moment we call ByteBuf.readBytes(...) in these handlers but with optimizations done as part of 25e0d9d we can just use readSlice(...).retain() and eliminate the memory copy.

Modifications:
Replace ByteBuf.readBytes(...) usage with readSlice(...).retain().

Result:
Less memory copies.
2014-05-10 17:20:53 +02:00
Trustin Lee
db3709e652 Synchronized between 4.1 and master
Motivation:

4 and 5 were diverged long time ago and we recently reverted some of the
early commits in master.  We must make sure 4.1 and master are not very
different now.

Modification:

Fix found differences

Result:

4.1 and master got closer.
2014-04-25 00:38:02 +09:00
Martin Krüger
d854d3a617 Fix chunk type for stream identifier
Motivation:
The problem with the current snappy implementation is that it does
not comply with framing format definition found on
https://code.google.com/p/snappy/source/browse/trunk/framing_format.txt

The document describes that chunk type of the stream identifier is defined
as 0xff. The current implentation uses 0x80.

Modifications:
This patch replaces the first byte of the chunk type of the stream identifier
with 0xff.

Result:
After this modification the snappy implementation is compliant to the
framing format described at
https://code.google.com/p/snappy/source/browse/trunk/framing_format.txt.
This results in a better compatibility with other implementations.
2014-04-19 21:06:28 +02:00
Norman Maurer
88481131be [#2353] Use a privileged block to get ClassLoader and System property if needed
Motivation:
When using System.getProperty(...) and various methods to get a ClassLoader it will fail when a SecurityManager is in place.

Modifications:
Use a priveled block if needed. This work is based in the PR #2353 done by @anilsaldhana .

Result:
Code works also when SecurityManager is present
2014-04-08 14:12:25 +02:00
Norman Maurer
cb9660f83d Allow the user to call slice().retain() or duplicate.retain() in his/her ByteToMessageDecoder.decode(...) method.
Motivation:
At the moment a user can not safetly call slice().retain() or duplicate.retain()in the ByteToMessageDecoder.decode(...) implementation without the risk to see coruption because we may call discardSomeReadBytes() to make room on the buffer once the handling is done.

Modifications:
Check for the refCnt() before call discardSomeReadBytes() and also check before call decode(...) to create a copy if needed.

Result:
The user can safetly call slice().retain() or duplicate.retain() in his/her ByteToMessageDecoder.decode(...) method.
2014-04-07 11:53:28 +02:00
Alexey Diomin
2a4999b6b8 [#2339] Reduce memory usage in ProtobufVarint32LengthFieldPrepender
Motivation:

Reduce memory usage in ProtobufVarint32LengthFieldPrepender.

Modifications:

Explicit set the buffer size that is needed for the header (between 1 and 5 bytes).

Result:

Less memory usage in ProtobufVarint32LengthFieldPrepender.
2014-03-28 19:57:06 +01:00
Norman Maurer
adcc2d4b4b Remove condition in ChannelHandlerAdapter.isSharable() by caching the result of the annotation lookup.
Motivation:
Remove the synchronization bottleneck and so speed up things

Modifications:
Introduce a ThreadLocal cache that holds mappings between classes of ChannelHandlerAdapater implementations and the result of checking if the @Sharable annotation is present.
This way we only will need to do the real check one time and server the other calls via the cache. A ThreadLocal and WeakHashMap combo is used to implement the cache
as this way we can minimize the conditions while still be sure we not leak class instances in containers.

Result:
Less conditions during adding ChannelHandlerAdapter to the ChannelPipeline
2014-03-12 12:31:52 +01:00
Norman Maurer
99995876dc Fix buffer leak in test which was introduced while implement ZLIB_OR_NONE support. Related to [#2269] 2014-03-10 06:25:42 +01:00
Norman Maurer
d89bfc593e Fix buffer leak in test which was introduced while implement ZLIB_OR_NONE support. Related to [#2269] 2014-03-06 20:13:30 +01:00
Jakob Buchgraber
9fb235459e Add ZLIB_OR_NONE support to JdkZlibDecoder [#2016] 2014-03-03 06:37:47 +01:00
Trustin Lee
a0378af850 Fix resource leaks in ByteArrayEncoderTest 2014-02-16 11:50:09 -08:00
Trustin Lee
ac70dc4546 Update the version to 4.1.0.Alpha1-SNAPSHOT 2014-02-13 18:32:26 -08:00
Trustin Lee
8837afddf8 Enable a user specify an arbitrary information with ReferenceCounted.touch()
- Related: #2163
- Add ResourceLeakHint to allow a user to provide a meaningful information about the leak when touching it
- DefaultChannelHandlerContext now implements ResourceLeakHint to tell where the message is going.
- Cleaner resource leak report by excluding noisy stack trace elements
2014-02-13 18:16:25 -08:00
Trustin Lee
45e70d9935 Add ReferenceCounted.touch() / Add missing retain() overrides
- Fixes #2163
- Inspector warnings
2014-02-13 18:10:11 -08:00
Trustin Lee
df346a023b Change the return type of EmbeddedChannel.read*() from Object to an ad-hoc type parameter
.. so that there's no need to explicitly down-cast.

Fixes #2067
2014-02-13 17:19:26 -08:00
Trustin Lee
5e69955d23 Fix another buffer leak in XmlFrameDecoderTest 2014-02-13 17:15:06 -08:00
Trustin Lee
457cd2f6fa Fix buffer leaks in XmlFrameDecoderTest 2014-02-13 17:14:59 -08:00
Trustin Lee
502ccabab3 Fix inspector warnings 2014-02-13 17:13:55 -08:00
Mirko Caserta
ee8571824b CDATA support 2014-02-13 17:13:49 -08:00
Mirko Caserta
086dbd1ba1 Fixed the XML decoder 2014-02-13 17:13:39 -08:00
Trustin Lee
2d5a3b5898 Add XML decoder
- based on @mcaserta's work at https://github.com/netty/netty/pull/1121
- not ready for a merge yet
2014-02-13 17:13:29 -08:00
Trustin Lee
499033d44f Add a shortcut method for collision-free naming 2014-02-13 15:17:09 -08:00
Norman Maurer
573b54a93d [#1907] LengthFieldPrepender should better extend MessageToMessageEncoder for less memory copies 2014-02-13 14:52:12 -08:00
Vladimir Schafer
3d531231fe #2183 Fix for releasing of the internal cumulation buffer in ByteToMessageDecoder 2014-02-06 20:07:56 +01:00
Norman Maurer
85a997d5ec Correctly calculate checksum when using GZIP. Part of [#2168] 2014-02-06 10:47:59 +01:00
Norman Maurer
0f7379157a [#2168] Eliminate unnessary memory copy for heap buffers in JdkZlibEncoder
* Also adjust tests so it test with direct and heap buffers
2014-01-30 07:02:14 +01:00
Norman Maurer
d67184b488 [maven-release-plugin] prepare for next development iteration 2014-01-21 08:18:32 +01:00
Norman Maurer
287515210d [maven-release-plugin] prepare release netty-4.0.15.Final 2014-01-21 08:18:26 +01:00
Norman Maurer
faf8becf2e Make use of ByteBufProcessor for extract initial line and headers
This gives some nice performance boost as readByte() is quite expensive because of the index / replay checks.
2014-01-16 20:14:08 +01:00
Trustin Lee
e83d2e0b4e [maven-release-plugin] prepare for next development iteration 2013-12-22 21:57:48 +09:00
Trustin Lee
cdb700c7a4 [maven-release-plugin] prepare release netty-4.0.14.Final 2013-12-22 21:57:40 +09:00
Trustin Lee
0b7aedb13b [maven-release-plugin] rollback the release of netty-4.0.14.Final 2013-12-22 21:53:24 +09:00
Trustin Lee
4bf6ec7171 [maven-release-plugin] prepare release netty-4.0.14.Final 2013-12-22 21:52:56 +09:00
Trustin Lee
9c1a49c58e [maven-release-plugin] rollback the release of netty-4.0.14.Final 2013-12-22 21:47:35 +09:00
Trustin Lee
008a049bf4 [maven-release-plugin] prepare for next development iteration 2013-12-22 21:43:55 +09:00
Trustin Lee
f6cb9088c6 [maven-release-plugin] prepare release netty-4.0.14.Final 2013-12-22 21:43:45 +09:00
Norman Maurer
b3d8c81557 Fix all leaks reported during tests
- One notable leak is from WebSocketFrameAggregator
- All other leaks are from tests
2013-12-07 00:44:56 +09:00
Norman Maurer
17f5865e38 [maven-release-plugin] prepare for next development iteration 2013-11-29 19:31:01 +01:00
Norman Maurer
ead617fdcc [maven-release-plugin] prepare release netty-4.0.14.Beta1 2013-11-29 19:30:55 +01:00
Trustin Lee
3f7b674db8 Fix bugs in ZLIB codec where they produce malformed stream or their streams are not flushed on time
- Fixes #2014
- Add the tests that mix JDK ZLIB codec and JZlib codecs
- Fix a bug where JdkZlibEncoder does not encode the GZIP header when nothing was written to te channel
- Fix a bug where the encoders do not consider the overhead of the wrapper format when calculating the estimated compressed output size.
- Fix a bug where the decoders do not discard the received data after the compressed stream is finished
2013-11-29 18:09:04 +09:00
Norman Maurer
6cf2748dbb [maven-release-plugin] prepare for next development iteration 2013-11-28 15:04:51 +01:00
Norman Maurer
5fe7596f49 [maven-release-plugin] prepare release netty-4.0.13.Final 2013-11-28 15:04:46 +01:00
Trustin Lee
407f0a36f5 Simplify bundle generation / Add io.netty.versions.properties to all JARs
- Fixes #2003 properly
- Instead of using 'bundle' packaging, use 'jar' packaging.  This is
  more robust because some strict build tools fail to retrieve the
  artifacts from a Maven repository unless their packaging is not 'jar'.
- All artifacts now contain META-INF/io.netty.version.properties, which
  provides the detailed information about the build and repository.
- Removed OSGi testsuite temporarily because it gives false errors
  during split package test and examination.
- Add io.netty.util.Version for easy retrieval of version information
2013-11-26 22:01:46 +09:00
Norman Maurer
db78581bbb [maven-release-plugin] prepare for next development iteration 2013-11-07 18:11:45 +01:00
Norman Maurer
2386777af8 [maven-release-plugin] prepare release netty-4.0.12.Final 2013-11-07 18:11:38 +01:00
Trustin Lee
54db9ec725 Use StringUtil.simpleClassName(..) instead of Class.getSimpleName() where necessary
- Class.getSimpleName() doesn't render anonymous classes very well
- + some minor cleanup
2013-11-04 19:46:15 +09:00
Trustin Lee
4f5c52572b Allow a user to define his/her own length field decoder
- Fixes #1960
2013-11-02 14:29:51 +01:00
Luke Wood
31c1513229 [#1958] Use CRC32-C in Snappy as per the spec 2013-11-01 20:54:55 +01:00
Trustin Lee
1c2352e6a0 Replace constructor calls on UniqueName and its subtypes with valueOf() wherever possible 2013-10-25 20:58:53 +09:00
Norman Maurer
8930cefab8 [#1812] Rework ByteToMessageDecoder.channelRead(..) method to allow for inlining 2013-10-25 13:32:33 +02:00
Norman Maurer
ceab146b54 [maven-release-plugin] prepare for next development iteration 2013-10-21 07:43:42 +02:00
Norman Maurer
27a89d6032 [maven-release-plugin] prepare release netty-4.0.11.Final 2013-10-21 07:41:49 +02:00
Norman Maurer
d946659520 [#1906] Use a ByteBuf allocator from the ByteBufAllocator when encode Strings 2013-10-09 21:18:08 +02:00
Norman Maurer
00f99dbff6 [#1893] Fix bug in JdkZlibDecoder which did not let it handle large data 2013-10-07 07:31:21 +02:00
Norman Maurer
d7da19f745 [maven-release-plugin] prepare for next development iteration 2013-10-02 15:48:52 +02:00
Norman Maurer
d35768ae11 [maven-release-plugin] prepare release netty-4.0.10.Final 2013-10-02 15:48:45 +02:00
Norman Maurer
ffab456aca Bump up version to reflect correct one 2013-09-09 11:20:12 +02:00
Norman Maurer
363531caf9 [maven-release-plugin] rollback the release of netty-4.0.9.Final 2013-09-06 09:18:34 +02:00
Norman Maurer
9d53573ee8 [maven-release-plugin] prepare for next development iteration 2013-09-06 09:17:15 +02:00
Norman Maurer
aedd631dbb [#1798] Correctly handle skipping of too big frames 2013-08-29 13:51:50 +02:00
Norman Maurer
51a536fd30 Free up all buffers after the tests 2013-08-28 11:00:55 +02:00
Norman Maurer
7a38163743 [#1788] Correctly decode CRC32 and ISIZE when using JdkZlibDecoder
Because of incorrect decoding a CompressionException was thrown before
2013-08-28 11:00:47 +02:00
Derek Troy-West
6a4131c10f Currently the system variable to chose which type of zlib decoder is being negated 2013-08-27 08:27:34 +02:00
Norman Maurer
2e39b25cd4 [maven-release-plugin] prepare for next development iteration 2013-08-26 12:01:03 +02:00
Norman Maurer
b67659a866 [maven-release-plugin] prepare release netty-4.0.8.Final 2013-08-26 12:00:54 +02:00
Norman Maurer
65d24a7647 [#1735] Disable usage of JdkZlibDecoder by default, will be enabled in 4.1.0.Final 2013-08-26 08:15:30 +02:00
Norman Maurer
39cabcd36c [#1481] Add a JdkZlibDecoder which has no dependencies 2013-08-14 07:41:28 +02:00
Norman Maurer
7d31943326 Remove some dead code 2013-08-11 21:37:43 +02:00
Norman Maurer
73755d1233 [#1729] Let StringEncoder extend MessageToMessageEncoder and so safe a memory copy 2013-08-11 21:24:02 +02:00
Norman Maurer
9a88c50ffb [#1728] Recycle list after done with it in ReplayingDecoder/ByteToMessageDecoder channelInactive(..) 2013-08-11 21:04:55 +02:00
Norman Maurer
26ddf0849f [#1714] Make sure ByteArrayDecoder also works correctly with pooled buffers 2013-08-09 08:40:43 +02:00
Norman Maurer
1547876e97 Tiny optimizations 2013-08-09 08:24:24 +02:00
Norman Maurer
1d3560e389 [maven-release-plugin] prepare for next development iteration 2013-08-08 13:53:28 +02:00
Norman Maurer
8e97e6c461 [maven-release-plugin] prepare release netty-4.0.7.Final 2013-08-08 13:53:19 +02:00
Trustin Lee
87b17e195a Grammar 2013-08-05 17:40:45 +09:00
Norman Maurer
4d064d85df Rename class 2013-08-02 07:24:49 +02:00
Norman Maurer
ebee56ef49 More tests related to self removal from pipeline in ByteToMessageDecoder and ReplayingDecoder 2013-08-02 07:23:46 +02:00
Norman Maurer
487c28d93e Add testcases to prove that remove ReplayingDecoder/ByteToMessageDecoder from within the decode(...) method works 2013-08-01 20:42:46 +02:00
Norman Maurer
3f2000fa3a [maven-release-plugin] prepare for next development iteration 2013-08-01 10:59:55 +02:00
Norman Maurer
3f70d5caa4 [maven-release-plugin] prepare release netty-4.0.6.Final 2013-08-01 10:59:46 +02:00
Norman Maurer
a06295fe0a Correctly fix problem in ByteToMessageDecoder and ReplayingDecoder which could let to have a released buffer passed to the decode methods.
This fixes #1664 and revert also the original commit which was meant to fix it 3b1881b523 . The problem with the original commit was that it could delay handlerRemove(..) calls and so mess up the order or forward bytes to late.
2013-08-01 09:54:07 +02:00
Norman Maurer
e3410680de [maven-release-plugin] prepare for next development iteration 2013-07-31 20:08:14 +02:00
Norman Maurer
0e124583d6 [maven-release-plugin] prepare release netty-4.0.5.Final 2013-07-31 20:08:05 +02:00
Norman Maurer
6ce8571df3 [#1668] Remove synchronized usage in JZlibEncoder and JdkZlibEncoder 2013-07-29 07:08:49 +02:00
Norman Maurer
6278d88d2f [#1656] Fix IndexOutOfBoundsException in MessageToMessageException 2013-07-26 11:46:26 +02:00
Norman Maurer
0bc7d3f5d1 [maven-release-plugin] prepare for next development iteration 2013-07-23 10:04:23 +02:00
Norman Maurer
ca00182797 [maven-release-plugin] prepare release netty-4.0.4.Final 2013-07-23 10:04:14 +02:00
Norman Maurer
eb3283c59c [#1613] Allow to specify if direct buffers are prefered in ByteToMessageCodec 2013-07-19 09:39:06 +02:00
Trustin Lee
7215c011ca Fix resource leaks in the tests 2013-07-18 23:42:51 +09:00
Trustin Lee
fe59ab1b9c Fix resource leak in LengthFieldBasedFrameDecoderTest 2013-07-18 20:55:02 +09:00
Trustin Lee
b130ee6a6c [maven-release-plugin] prepare for next development iteration 2013-07-18 11:17:42 +09:00
Trustin Lee
10d395e829 [maven-release-plugin] prepare release netty-4.0.3.Final 2013-07-18 11:17:31 +09:00
Norman Maurer
fc7c950b08 [maven-release-plugin] prepare for next development iteration 2013-07-17 15:58:36 +02:00
Norman Maurer
bbbf72359e [maven-release-plugin] prepare release netty-4.0.2.Final 2013-07-17 15:58:28 +02:00
Trustin Lee
57eb531eb8 [maven-release-plugin] prepare for next development iteration 2013-07-16 17:16:10 +09:00
Trustin Lee
76cefcc421 [maven-release-plugin] prepare release netty-4.0.1.Final 2013-07-16 17:15:54 +09:00
Norman Maurer
22be797054 Use Unpooled to allocate buffer in Base64 2013-07-16 07:24:45 +02:00
Norman Maurer
5297eba280 [maven-release-plugin] prepare for next development iteration 2013-07-15 15:48:15 +02:00
Norman Maurer
c5d8af446a [maven-release-plugin] prepare release netty-4.0.0.Final 2013-07-15 15:48:05 +02:00
Trustin Lee
246a3ecdcb [maven-release-plugin] prepare for next development iteration 2013-07-15 20:58:33 +09:00
Trustin Lee
e8fd209115 [maven-release-plugin] prepare release netty-4.0.0.Final 2013-07-15 20:58:21 +09:00
Norman Maurer
7522d530ac Use the allocator of the src ByteBuf to allocate the new ByteBuf 2013-07-15 10:08:44 +02:00
Norman Maurer
2f7759a264 Remove unused package private classes 2013-07-14 23:50:27 +02:00
Norman Maurer
3140282ae3 Use the ByteBufAllocator when possible 2013-07-14 23:34:36 +02:00
Norman Maurer
ecb215c12f Fix buffer leaks 2013-07-14 22:50:53 +02:00
Norman Maurer
a07abee55f Add and correct javadocs 2013-07-13 19:42:02 +02:00
Norman Maurer
910c5fd594 Hide RecyclableArrayList from sub-classes 2013-07-13 16:57:44 +02:00
Norman Maurer
a8af577423 Add javadocs to methods that user may override and make the others final 2013-07-12 13:49:54 +02:00
Trustin Lee
765d470500 Remove UnreplayableOperationException
- Can't find a reason to keep it.
2013-07-12 20:40:39 +09:00
Norman Maurer
edb6786a5c Make the cumulation field in ByteToMessageDecoder package private to hide it 2013-07-12 13:29:54 +02:00
Norman Maurer
b57d9f307f Allow per-write promises and disallow promises on flush()
- write() now accepts a ChannelPromise and returns ChannelFuture as most
  users expected.  It makes the user's life much easier because it is
  now much easier to get notified when a specific message has been
  written.
- flush() does not create a ChannelPromise nor returns ChannelFuture.
  It is now similar to what read() looks like.
2013-07-11 00:49:48 +09:00
Norman Maurer
da5c6add14 Rename CodecOutput to RecyclableArrayList and move it to internal package.
* Also reuse it in SslHandler
2013-07-10 07:50:26 +02:00
Norman Maurer
768152cf88 Correctly handle forward of remaining data on removal 2013-07-09 22:41:56 +02:00
Trustin Lee
26e9d70457 Remove channelReadSuspended / Rename messageReceived(Last) to channelRead(Complete)
- Remove channelReadSuspended because it's actually same with messageReceivedLast
- Rename messageReceived to channelRead
- Rename messageReceivedLast to channelReadComplete

We renamed messageReceivedLast to channelReadComplete because it
reflects what it really is for.  Also, we renamed messageReceived to
channelRead for consistency in method names.
2013-07-09 23:58:51 +09:00
Trustin Lee
cbd8817905 Remove MessageList from public API and change ChannelInbound/OutboundHandler accordingly
I must admit MesageList was pain in the ass.  Instead of forcing a
handler always loop over the list of messages, this commit splits
messageReceived(ctx, list) into two event handlers:

- messageReceived(ctx, msg)
- mmessageReceivedLast(ctx)

When Netty reads one or more messages, messageReceived(ctx, msg) event
is triggered for each message.  Once the current read operation is
finished, messageReceivedLast() is triggered to tell the handler that
the last messageReceived() was the last message in the current batch.

Similarly, for outbound, write(ctx, list) has been split into two:

- write(ctx, msg)
- flush(ctx, promise)

Instead of writing a list of message with a promise, a user is now
supposed to call write(msg) multiple times and then call flush() to
actually flush the buffered messages.

Please note that write() doesn't have a promise with it.  You must call
flush() to get notified on completion. (or you can use writeAndFlush())

Other changes:

- Because MessageList is completely hidden, codec framework uses
  List<Object> instead of MessageList as an output parameter.
2013-07-09 23:51:48 +09:00
Trustin Lee
65c2a6ed46 Make ByteBuf an abstract class rather than an interface
- 5% improvement in throughput (HelloWorldServer example)
- Made CompositeByteBuf a concrete class (renamed from DefaultCompositeByteBuf) because there's no multiple inheritance in Java

Fixes #1536
2013-07-08 14:59:52 +09:00
Trustin Lee
da5a5af520 Add Test case for truncated SSL packets
- Related issue: #1534
- Also made sure ByteToMessageDecoder does not wrap EncoderException.
2013-07-07 11:52:34 +09:00
Norman Maurer
7dda4b9ce4 [#1532] Remove @deprecated ByteBufIndexFinder and all methods that take it as argument 2013-07-06 20:14:53 +02:00
Norman Maurer
ec5e793a2f [maven-release-plugin] prepare for next development iteration 2013-07-02 11:41:18 +02:00
Norman Maurer
ca73eaef0d [maven-release-plugin] prepare release netty-4.0.0.CR9 2013-07-02 11:41:09 +02:00
Norman Maurer
830c559405 [maven-release-plugin] rollback the release of netty-4.0.0.CR9 2013-07-02 11:34:29 +02:00
Norman Maurer
66a16b133c [maven-release-plugin] prepare release netty-4.0.0.CR9 2013-07-02 10:45:12 +02:00
Trustin Lee
7e3a01cc51 [maven-release-plugin] prepare for next development iteration 2013-07-02 10:26:48 +09:00
Trustin Lee
149db34c19 [maven-release-plugin] prepare release netty-4.0.0.CR8 2013-07-02 10:26:32 +09:00
Trustin Lee
a6bdd7cda4 Fix compilation errors 2013-07-01 17:27:45 +09:00
Norman Maurer
4dab51b619 Remove un-used method parameter 2013-06-28 20:41:10 +02:00
Norman Maurer
e234abdbf7 Add javadocs 2013-06-28 20:28:29 +02:00
Trustin Lee
613547b0b9 [maven-release-plugin] prepare for next development iteration 2013-06-28 22:15:33 +09:00
Trustin Lee
a6abd2feb2 [maven-release-plugin] prepare release netty-4.0.0.CR7 2013-06-28 22:15:20 +09:00
Trustin Lee
0dcf352f4c Vastly simplified ByteBufProcessor and MessageListProcessor
- Related: #1378
- They now accept only one argument.
- A user who wants to use a buffer for more complex use cases, he or she can always access the buffer directly via memoryAddress() and array()
2013-06-28 20:29:00 +09:00
Norman Maurer
094d01873b Throw IllegalStateException if implementations of ByteToMessageDecoder and ByteToMessageCodec use @Sharable 2013-06-27 22:38:35 +02:00
Trustin Lee
ac39cad5ff Split ByteBuf.forEachByte() into forEachByte() and forEachByteDesc()
- Related: #1378
- As suggested by @liqweed
2013-06-27 18:48:09 +09:00
Trustin Lee
b5bb36c087 Use (fromIndex, toIndex) instead of (index, length) for ByteBuf.forEachByte(...)
- Related: #1378
2013-06-27 17:30:19 +09:00
Trustin Lee
792edf631c Deprecate ByteBufIndexFinder
- Prefer ByteBufProcessor
- Related: #1378
2013-06-27 14:26:58 +09:00
Trustin Lee
4dd9b6ef2e Add ByteBufProcessor and ByteBuf.forEach(...)
- Fixes #1378
- Needs to provide optimized forEach implementations though.
2013-06-27 13:55:42 +09:00
Trustin Lee
a6795d7780 [maven-release-plugin] prepare for next development iteration 2013-06-25 11:07:15 +09:00
Trustin Lee
2221446425 [maven-release-plugin] prepare release netty-4.0.0.CR6 2013-06-25 11:07:15 +09:00
Trustin Lee
96c5ab62e6 Fix IndexOutOfBoundsException raised by JZlibEncoder.encode()
- Fixes #1471
2013-06-25 11:07:13 +09:00
Norman Maurer
14f2e29af9 [#1450] Fix examples in ReplayDecoder javadocs 2013-06-16 16:53:03 +02:00
Trustin Lee
a5871dfd86 [maven-release-plugin] prepare for next development iteration 2013-06-14 12:55:15 +09:00
Trustin Lee
f5377cc8d7 [maven-release-plugin] prepare release netty-4.0.0.CR5 2013-06-14 12:55:05 +09:00
Trustin Lee
30bfb989c1 Fix memory leak 2013-06-14 10:44:22 +09:00
Trustin Lee
e5ca6518ba [maven-release-plugin] prepare for next development iteration 2013-06-13 17:02:32 +09:00
Trustin Lee
381063e09c [maven-release-plugin] prepare release netty-4.0.0.CR4 2013-06-13 17:02:19 +09:00
Trustin Lee
fa277eda20 Fix reported leaks 2013-06-13 15:24:19 +09:00
Trustin Lee
427d9c4bf2 Fix test failures and reported leaks 2013-06-13 15:18:11 +09:00
Trustin Lee
01d9f10af6 Remove the volatile modifiers where they are unnecessary 2013-06-13 14:43:39 +09:00
Trustin Lee
6d1cd0d0cd ReferenceCountException -> IllegalReferenceCountException 2013-06-13 14:00:15 +09:00
Trustin Lee
7eb0f6105d Fix memory leaks 2013-06-13 13:32:47 +09:00
Trustin Lee
175526b6bd Move ReferenceCounted and AbstractReferenceCounted to io.netty.util
- Fixes #1441
- Also move and rename IllegalBufferAccessException to ReferenceCountException
- Prettier reference count exception messages
2013-06-13 13:14:21 +09:00
Trustin Lee
283feda119 Reduce even more garbage by exposing ByteBuf.internalNioBuffer() 2013-06-13 12:40:26 +09:00
Norman Maurer
78c6925921 Clearify when you need to call retain() 2013-06-12 23:13:36 +02:00
Norman Maurer
68c737f0c0 Optimize the way messages are added from one MessageList to another one 2013-06-10 14:07:25 +02:00
Norman Maurer
3be25694d0 Add ChannelHandlerContext.isRemoved() to easily detect the removal of a ChannelHandler while in a method. 2013-06-10 11:16:05 +02:00
Trustin Lee
14158070bf Revamp the core API to reduce memory footprint and consumption
The API changes made so far turned out to increase the memory footprint
and consumption while our intention was actually decreasing them.

Memory consumption issue:

When there are many connections which does not exchange data frequently,
the old Netty 4 API spent a lot more memory than 3 because it always
allocates per-handler buffer for each connection unless otherwise
explicitly stated by a user.  In a usual real world load, a client
doesn't always send requests without pausing, so the idea of having a
buffer whose life cycle if bound to the life cycle of a connection
didn't work as expected.

Memory footprint issue:

The old Netty 4 API decreased overall memory footprint by a great deal
in many cases.  It was mainly because the old Netty 4 API did not
allocate a new buffer and event object for each read.  Instead, it
created a new buffer for each handler in a pipeline.  This works pretty
well as long as the number of handlers in a pipeline is only a few.
However, for a highly modular application with many handlers which
handles connections which lasts for relatively short period, it actually
makes the memory footprint issue much worse.

Changes:

All in all, this is about retaining all the good changes we made in 4 so
far such as better thread model and going back to the way how we dealt
with message events in 3.

To fix the memory consumption/footprint issue mentioned above, we made a
hard decision to break the backward compatibility again with the
following changes:

- Remove MessageBuf
- Merge Buf into ByteBuf
- Merge ChannelInboundByte/MessageHandler and ChannelStateHandler into ChannelInboundHandler
  - Similar changes were made to the adapter classes
- Merge ChannelOutboundByte/MessageHandler and ChannelOperationHandler into ChannelOutboundHandler
  - Similar changes were made to the adapter classes
- Introduce MessageList which is similar to `MessageEvent` in Netty 3
- Replace inboundBufferUpdated(ctx) with messageReceived(ctx, MessageList)
- Replace flush(ctx, promise) with write(ctx, MessageList, promise)
- Remove ByteToByteEncoder/Decoder/Codec
  - Replaced by MessageToByteEncoder<ByteBuf>, ByteToMessageDecoder<ByteBuf>, and ByteMessageCodec<ByteBuf>
- Merge EmbeddedByteChannel and EmbeddedMessageChannel into EmbeddedChannel
- Add SimpleChannelInboundHandler which is sometimes more useful than
  ChannelInboundHandlerAdapter
- Bring back Channel.isWritable() from Netty 3
- Add ChannelInboundHandler.channelWritabilityChanges() event
- Add RecvByteBufAllocator configuration property
  - Similar to ReceiveBufferSizePredictor in Netty 3
  - Some existing configuration properties such as
    DatagramChannelConfig.receivePacketSize is gone now.
- Remove suspend/resumeIntermediaryDeallocation() in ByteBuf

This change would have been impossible without @normanmaurer's help. He
fixed, ported, and improved many parts of the changes.
2013-06-10 16:10:39 +09:00
Luke Wood
c1cbe8bec3 Clarify SnappyFramedDecoder javadoc to make it as clear as possible regarding the default settings 2013-05-20 17:31:04 +01:00
Norman Maurer
81e3c1719a [maven-release-plugin] prepare for next development iteration 2013-05-18 09:59:13 +02:00
Norman Maurer
99caefdf39 [maven-release-plugin] prepare release netty-4.0.0.CR3 2013-05-18 09:57:11 +02:00
Trustin Lee
e58af1bc94 Readability / Fix inception year 2013-05-17 17:59:13 +09:00
kxbmap
3a5aa12000 Add LengthFieldPrepender.lengthAdjustment parameter 2013-05-17 17:58:53 +09:00
Luke Wood
f841056752 Fix SnappyFramedDecoder checksum validation and add a pair of suitable unit tests 2013-05-16 14:10:54 +01:00
Trustin Lee
80f4c0b334 Merge SnappyChecksumUtil into Snappy and make calculateChecksum() public / Overall clean up 2013-05-09 15:23:56 +09:00
Norman Maurer
c43950a03f [maven-release-plugin] prepare for next development iteration 2013-05-08 18:19:51 +02:00
Norman Maurer
ae76502040 [maven-release-plugin] prepare release netty-4.0.0.CR2 2013-05-08 18:19:38 +02:00
Norman Maurer
108c7d9b44 No need to pass the next BufType to the constructor 2013-05-06 07:46:26 +02:00
Norman Maurer
73c35aef4e [#1309] Make sure ReplayDecoder respect isSingleDecode()
* This could cause for example corrupt WebSocketFrame's if they was written from the server
  to the client directly after it send the handshake response.
2013-04-25 08:32:07 +02:00
Trustin Lee
7884574c7b Remove freeInboundBuffer() and freeOutboundBuffer() which has no value
- Fixes #1308

freeInboundBuffer() and freeOutboundBuffer() were introduced in the early days of the new API when we did not have reference counting mechanism in the buffer. A user did not want Netty to free the handler buffers had to override these methods.

However, now that we have reference counting mechanism built into the buffer, a user who wants to retain the buffers beyond handler's life cycle can simply return the buffer whose reference count is greater than 1 in newInbound/OutboundBuffer().
2013-04-25 09:15:55 +09:00
Trustin Lee
1b3d7f5325 Make sure handlerAdded() is called before forwarding the buffer content of the removed handler
- Added a test case that reproduces the problem in ReplayingDecoderTest
- Call newHandler.handlerAdded() *before* oldHandler.handlerRemoved() to ensure newHandlerAdded() is called before forwarding the buffer content of the old handler in replace0().
2013-04-24 19:25:43 +09:00
Trustin Lee
5dd35448a2 Add a failing test that will pass once #1306 is fixed 2013-04-24 18:47:36 +09:00
Trustin Lee
b5989e2449 Reduce exception instantiation overhead in SslHandler / Reduce unnecessary empty array creation
- Added EmptyArrays as an internal utility class
2013-04-24 09:32:53 +09:00
Trustin Lee
8e2e22c270 Change the thread model slightly for new/freeInbound/OutboundBuffer() for future improvement
- Related: #1283
- Make ReplayingDecoder work with the modified thread model
2013-04-23 13:06:27 +09:00
Norman Maurer
9c4bfa44d9 [#1294] Make sure ByteBuf is released once written to channel 2013-04-22 11:04:56 +02:00
Norman Maurer
ca5554dfe7 [#1236] Fix problem where adding a new ChannelHandler could block the eventloop
This change also introduce a few other changes which was needed:
 * ChannelHandler.beforeAdd(...) and ChannelHandler.beforeRemove(...) were removed
 * ChannelHandler.afterAdd(...) -> handlerAdded(...)
 * ChannelHandler.afterRemoved(...) -> handlerRemoved(...)
 * SslHandler.handshake() -> SslHandler.hanshakeFuture() as the handshake is triggered automatically after
   the Channel becomes active
2013-04-19 07:00:50 +02:00
Trustin Lee
4a5dc32224 Fix a bug where LineBasedFrameDecoder does not handle too long lines correctly
- Related: #1287
2013-04-19 13:04:20 +09:00
Andrei Pozolotin
a3e760a003 fix #1234 - duplicate package-info.java errors in eclipse requires release of netty-build v 19 and netty-parent update. 2013-04-05 05:38:05 +09:00
Trustin Lee
6dfa455f9e Fix compiler warnings 2013-04-04 16:31:41 +09:00
Trustin Lee
c03179c81c Clean up try/catch blocks / Prefer 'throw' to 'fireExceptionCaught' 2013-04-04 15:00:07 +09:00
Trustin Lee
3b9994455a Saner field order 2013-04-04 14:53:32 +09:00
Trustin Lee
03955428bc Ensure BufUtil.release() is called even if writeBytes() fails in writeAndRelease() 2013-04-04 14:50:25 +09:00
Trustin Lee
015e60b00f Clean up catch blocks
- No need to have nested try blocks because the catch block catches everything and rethrows it
- No need to do instanceof-checks
2013-04-04 14:48:30 +09:00
Trustin Lee
c25fd78ca0 No need to have nested try blocks
.. because the catch block catches everything and rethrows it
2013-04-04 14:44:52 +09:00
Trustin Lee
788b88b7af Make sure to release the buffer when it's written to the next byte buffer
.. otherwise it will leak.
2013-04-04 14:37:54 +09:00
Trustin Lee
46540578fc Optimize OutputMessageBuf.drainToNextInbound/Outbound()
- Return early when the buffer is empty
- Keep only the number of byte buffers
- Remove unnecessary null check in the loop (because we know buffer is not empty at certain point)
2013-04-04 14:32:48 +09:00
Trustin Lee
8a672c2800 Proper assertion instead of blindly clearing the OutputMessageBuf 2013-04-04 14:11:24 +09:00
Norman Maurer
7ee2adb587 Move drain logic to OutputMessageBuf and optimize it as far as possible 2013-04-03 18:52:57 +02:00
Norman Maurer
1675e61f5b Share same ThreadLocal for all decoder/encoders to minimize memory usage 2013-04-03 18:19:35 +02:00
Norman Maurer
94ef7dc1b9 Optimize to minimize volatile reads to access next buffer in codec framework 2013-04-03 18:03:55 +02:00
Norman Maurer
9828267165 Fix performance regression which was caused by calling the fireInboundBufferUpdated() a way to often
* Also use a ThreadLocal for the decoder buffer to safe space as it is cheap
2013-04-03 17:07:52 +02:00
Norman Maurer
af4b71a00e Remove special handling of Object[] in codec framework (a.k.a unfolding)
- Fixes #1229
- Primarily written by @normanmaurer and revised by @trustin

This commit removes the notion of unfolding from the codec framework
completely.  Unfolding was introduced in Netty 3.x to work around the
shortcoming of the codec framework where encode() and decode() did not
allow generating multiple messages.

Such a shortcoming can be fixed by changing the signature of encode()
and decode() instead of introducing an obscure workaround like
unfolding.  Therefore, we changed the signature of them in 4.0.

The change is simple, but backward-incompatible.  encode() and decode()
do not return anything.  Instead, the codec framework will pass a
MessageBuf<Object> so encode() and decode() can add the generated
messages into the MessageBuf.
2013-04-03 21:44:54 +09:00
Norman Maurer
2e3b8f13ea Revert "Let ProtobufEncoder extend MessageToByteEncoder which makes more sense. Related to [#1222]"
This reverts commit 6a9d055f57.
2013-04-02 07:56:49 +02:00