Commit Graph

300 Commits

Author SHA1 Message Date
Trustin Lee
03955428bc Ensure BufUtil.release() is called even if writeBytes() fails in writeAndRelease() 2013-04-04 14:50:25 +09:00
Trustin Lee
015e60b00f Clean up catch blocks
- No need to have nested try blocks because the catch block catches everything and rethrows it
- No need to do instanceof-checks
2013-04-04 14:48:30 +09:00
Trustin Lee
c25fd78ca0 No need to have nested try blocks
.. because the catch block catches everything and rethrows it
2013-04-04 14:44:52 +09:00
Trustin Lee
788b88b7af Make sure to release the buffer when it's written to the next byte buffer
.. otherwise it will leak.
2013-04-04 14:37:54 +09:00
Trustin Lee
46540578fc Optimize OutputMessageBuf.drainToNextInbound/Outbound()
- Return early when the buffer is empty
- Keep only the number of byte buffers
- Remove unnecessary null check in the loop (because we know buffer is not empty at certain point)
2013-04-04 14:32:48 +09:00
Trustin Lee
8a672c2800 Proper assertion instead of blindly clearing the OutputMessageBuf 2013-04-04 14:11:24 +09:00
Norman Maurer
7ee2adb587 Move drain logic to OutputMessageBuf and optimize it as far as possible 2013-04-03 18:52:57 +02:00
Norman Maurer
1675e61f5b Share same ThreadLocal for all decoder/encoders to minimize memory usage 2013-04-03 18:19:35 +02:00
Norman Maurer
94ef7dc1b9 Optimize to minimize volatile reads to access next buffer in codec framework 2013-04-03 18:03:55 +02:00
Norman Maurer
9828267165 Fix performance regression which was caused by calling the fireInboundBufferUpdated() a way to often
* Also use a ThreadLocal for the decoder buffer to safe space as it is cheap
2013-04-03 17:07:52 +02:00
Norman Maurer
af4b71a00e Remove special handling of Object[] in codec framework (a.k.a unfolding)
- Fixes #1229
- Primarily written by @normanmaurer and revised by @trustin

This commit removes the notion of unfolding from the codec framework
completely.  Unfolding was introduced in Netty 3.x to work around the
shortcoming of the codec framework where encode() and decode() did not
allow generating multiple messages.

Such a shortcoming can be fixed by changing the signature of encode()
and decode() instead of introducing an obscure workaround like
unfolding.  Therefore, we changed the signature of them in 4.0.

The change is simple, but backward-incompatible.  encode() and decode()
do not return anything.  Instead, the codec framework will pass a
MessageBuf<Object> so encode() and decode() can add the generated
messages into the MessageBuf.
2013-04-03 21:44:54 +09:00
Norman Maurer
2e3b8f13ea Revert "Let ProtobufEncoder extend MessageToByteEncoder which makes more sense. Related to [#1222]"
This reverts commit 6a9d055f57.
2013-04-02 07:56:49 +02:00
Norman Maurer
6a9d055f57 Let ProtobufEncoder extend MessageToByteEncoder which makes more sense. Related to [#1222] 2013-04-02 07:17:45 +02:00
Norman Maurer
a0fca93128 [#1222] Correctly handle the case when a MessageToMessageEncoder encodes to ByteBuf and the next outbound buffer is of type ByteBuf 2013-04-02 07:17:10 +02:00
Prajwal Tuladhar
05850da863 enable checkstyle for test source directory and fix checkstyle errors 2013-03-30 13:18:57 +01:00
Trustin Lee
8b722d29a7 Add constructor parameters that do not perform type parameter auto-detection for the languages without type parameters
- Fixes #1177
- Add TypeParameterMatcher.get(parameterType)
- Add alternative constructors
2013-03-21 16:11:47 +09:00
Trustin Lee
97f2fa7341 Allow a user specify the byte order of the length field for LengthFieldBasedFrameDecoder
- Fixes #1164
2013-03-18 13:58:48 +09:00
Trustin Lee
e3d10ad493 Break the decode loop if decoder raises an exception to give a chance to close the connection to a user handler
- Fixes: #1161
2013-03-16 18:28:58 +09:00
Michael Nitschinger
b186342f52 Fixing 2 typos in codec documentation. 2013-03-16 17:18:21 +09:00
Trustin Lee
289d474a23 Make ByteArrayEncoder sharable 2013-03-14 17:44:32 +09:00
Trustin Lee
42e31a4445 Revert "[#1131] Codecs must not cache next buffer during processing"
This reverts commit b1775a3223.
2013-03-13 15:17:06 +09:00
Trustin Lee
c25513d5e1 Upgrade to protobuf 2.5 and take advantage of MessageLite.getParserFromType()
- also fall back to MessageBuilder if getParserFromType() is not available.
2013-03-12 16:25:35 +09:00
Trustin Lee
559b860ff6 Fix grammar 2013-03-12 15:23:09 +09:00
Trustin Lee
397830d238 Allow SingleOutboundMessageHandler.beginFlush() to reject the flush request by returning false / Replace PartialFlushException with IncompleteFlushException which is more correct. 2013-03-12 15:20:46 +09:00
Trustin Lee
c660002b4e Add DecoderResult.UNFINISHED to represent the case where a decoder generated a message that was not decoded completely / Remove partial failure in DecoderResult which is not very useful but confusing 2013-03-12 13:04:53 +09:00
Norman Maurer
806e9b1f8c Make sure we handle outbound messages of type ByteBuf special 2013-03-11 08:59:00 +09:00
Norman Maurer
b1775a3223 [#1131] Codecs must not cache next buffer during processing 2013-03-08 15:38:17 +01:00
Norman Maurer
fd3f923b52 Allow to specify the used buffer type for ChannelInboundByteBufHandler and ChannelOutboundByteBufHandler by configuration. As default it tries to use a direct ByteBuf 2013-03-08 08:20:46 +01:00
Trustin Lee
a9a29bdf3f Use I/O buffer whenever possible now that our direct buffers are as fast as heap buffers 2013-03-08 11:21:08 +09:00
Trustin Lee
88df53ec1a Fix infinite recursion when transferring data between different type of buffers / Add ByteBuf.hasMemoryAddress/memoryAddress()
- Fixes: #1109 and #1110
2013-03-06 18:22:16 +09:00
Trustin Lee
a8a7c4f576 Provide a way to implement an ChannelInbound/OutboundMessageHandler conveniently without extending an adapter class
- Add ChannelHandlerUtil and move the core logic of ChannelInbound/OutboundMessageHandler to ChannelHandlerUtil
- Add ChannelHandlerUtil.SingleInbound/OutboundMessageHandler and make ChannelInbound/OutboundMessageHandlerAdapter implement them.  This is a backward incompatible change because it forces all handler methods to be public (was protected previously)
- Fixes: #1119
2013-03-05 17:27:53 +09:00
Norman Maurer
42dad6d9d4 [#1106] Also handle FileRegion in ByteToByteEncoder and SslHandler 2013-02-28 19:40:34 +01:00
Trustin Lee
671f9d48d4 Use ConcurrentHashMapV8 wherever possible
- Fixes #1052
2013-02-26 15:54:51 -08:00
Luke Wood
e45db60b5e [#1069] Snappy decoding fixes
* Correct reading offset of 1-byte-offset copies
* Keep track of how much we've written so far in order to validate offsets
* Uncomment and reduce number of tests
2013-02-25 16:36:50 +00:00
Trustin Lee
58c72acd42 Add more failing tests to SnappyIntegrationTest 2013-02-22 10:06:24 -08:00
Luke Wood
f6d88ca5bf Use Short.MAX_VALUE, as it accurately represents what we're trying to limit ourselves to 2013-02-22 15:23:47 +00:00
Luke Wood
0bfa9159e3 Do not attempt to compress trailing data that is less than the MIN_COMPRESSIBLE_LENGTH 2013-02-22 14:24:50 +00:00
Luke Wood
7a8e9d7993 Ensure that at least 4 bytes are remaining in input buffer when compiling our hash table in search of duplicates, and restrict encoded frame sizes to 32,767 bytes (2^15 - 1) 2013-02-22 14:24:50 +00:00
Trustin Lee
17641d52fb Integrate @timboudreau's SnappyIntegrationTest improvements 2013-02-21 18:38:58 -08:00
Norman Maurer
c93f5afa99 [#1012] Cleanup 2013-02-20 12:52:55 +01:00
Atsuhiko Yamanaka
8fdf788cbd [#1012] Replace forked jzlib with official jzlib and add a test. 2013-02-20 12:49:05 +01:00
Trustin Lee
1011227b88 Remove apiviz tags - we are focusing on user guide instead and putting diagrams there 2013-02-14 12:09:16 -08:00
Norman Maurer
6983f704c0 Fix unreplayble error 2013-02-14 11:18:12 +01:00
Norman Maurer
5370573400 Change ReferenceCounted.retain* to return itself and so allow method chaining 2013-02-14 07:39:44 +01:00
Trustin Lee
53c27ef5ae More robust type parameter detection
- Also removed unnecessary constructors which were added due to incomplete type parameter detection logic
2013-02-13 19:02:55 -08:00
Luke Wood
c1783cc8d3 More fixes to snappy: * Make Snappy.decode stateful instead of relying on the uncompressed length being equal to the compressed length * Correctly handle copies where offset < length * Take copies from the output buffer in decoding * Make the maximum encoded chunk size 32kB for compressed data 2013-02-11 15:05:27 +01:00
Trustin Lee
b4f4b95739 Move io.netty.logging to io.netty.internal / Move Signal out of internal because we use it in Channel*MessageAdapters 2013-02-11 20:08:18 +09:00
Trustin Lee
a2e5cd94be Prettify APIviz / Tighten visibility / Move subclasses to top level / Remove unused UnknownSocksMessage 2013-02-11 19:42:23 +09:00
Trustin Lee
6c7bd6d174 Tighten visibility 2013-02-11 18:42:31 +09:00
Trustin Lee
d5105834d6 Hide SnappyChecksumUtil 2013-02-11 18:41:37 +09:00
Trustin Lee
2f1a0b0593 Remove freeInbound/OutboundMessage(), replaced by ReferenceCounted.retain/release()
- Related: #1029
2013-02-10 13:31:31 +09:00
Trustin Lee
b9996908b1 Implement reference counting
- Related: #1029
- Replace Freeable with ReferenceCounted
- Add AbstractReferenceCounted
- Add AbstractReferenceCountedByteBuf
- Add AbstractDerivedByteBuf
- Add EmptyByteBuf
2013-02-10 13:10:09 +09:00
Trustin Lee
46a249a26b Add more failing tests for Snappy codec 2013-02-10 00:33:41 +09:00
Trustin Lee
bd87284829 Clean up SnappyIntegrationTest / Fix checksum encoding 2013-02-10 00:20:49 +09:00
Trustin Lee
8bd7ea2f93 Fix SnappyFramedEncoderTest & SnappyFramedEncoder
- chunkLength is 3 bytes
2013-02-10 00:03:47 +09:00
Trustin Lee
3268781607 Fix SnappyFramedDecoderTest 2013-02-09 23:57:14 +09:00
Trustin Lee
319b7fa69a Fix SnappyFramedEncoder/Decoder / Fix Snappy preamble encoding / Add test for #1002
- The new test still fails due to a bug in Snappy.encode/decode()
2013-02-09 23:39:33 +09:00
Trustin Lee
36f8630512 Make SnappyFramedDecoder more robust against corrupt frame 2013-02-09 20:58:55 +09:00
Trustin Lee
affd514b8c Rename ByteBufUtil to BufUtil and move ChannelHandlerUtil.freeMessage() there / Remove ChannelHandlerUtil 2013-02-08 23:23:26 +09:00
Trustin Lee
44ea0a116f Replace ChannelHandlerUtil.unfoldAndAdd() with MessageBuf.unfoldAndAdd() / Remove unused methods in ChannelHandlerUtil 2013-02-08 23:07:20 +09:00
Trustin Lee
ff5aec0c78 Replace TypeParameterFinder with TypeParameterMatcher
- We can avoid reflective matching using byte code generation.
 - Better matching performance when message type is Object
2013-02-08 18:28:06 +09:00
Trustin Lee
01e65a01c7 Make ReplayingDecoder.newInboundBuffer/discardInboundReadByte() final for safety 2013-02-08 17:41:43 +09:00
Trustin Lee
82c46180c9 Tighten access modifier of encode/decode() 2013-02-08 17:37:16 +09:00
Trustin Lee
76eb40a4d2 Make ChannelOutboundMessageHandlerAdapter similar to ChannelInboundMessageHandlerAdapter 2013-02-08 17:07:01 +09:00
Trustin Lee
1640b1fea6 Automatically detect the message types in MessageToMessageCodec 2013-02-08 16:12:32 +09:00
Trustin Lee
fa1b49de98 More robust automatic messageType detection for ChannelInboundMessageHandlerAdapter and MessageToMessageDecoder 2013-02-08 15:45:17 +09:00
Trustin Lee
f9eff51683 Fix missing ChannelHandlerUtil.acceptedMessageTypes() in ByteToMessageCodec constructor 2013-02-08 13:24:27 +09:00
Trustin Lee
d8c0bf3be2 Add the 'nextBufferType' parameter to ByteArrayEncoder like did to StringEncoder / Consistent parameter order 2013-02-08 01:36:41 +09:00
Trustin Lee
d4742bbe16 Clean up abstract ChannelHandler impls / Remove ChannelHandlerContext.hasNext*()
- Rename ChannelHandlerAdapter to ChannelDuplexHandler
- Add ChannelHandlerAdapter that implements only ChannelHandler
- Rename CombinedChannelHandler to CombinedChannelDuplexHandler and
  improve runtime validation
- Remove ChannelInbound/OutboundHandlerAdapter which are not useful
- Make ChannelOutboundByteHandlerAdapter similar to
  ChannelInboundByteHandlerAdapter
- Make the tail and head handler of DefaultChannelPipeline accept both
  bytes and messages.  ChannelHandlerContext.hasNext*() were removed
  because they always return true now.
- Removed various unnecessary null checks.
- Correct method/field names:
  inboundBufferSuspended -> channelReadSuspended
2013-02-07 23:47:45 +09:00
Luke Wood
bfe44180f9 Fix SnappyFramedDecoder issues
- Checksum header was being incorrectly read due to incorrect order of
  shift and masking operations.
- Length field of 1-byte copy was being incorrectly interpreted due to a
  typo in the binary mask used to extract it.
- Use ByteBuf.readUnsignedByte() instead of readByte() & 0xff
- Use bitwise-OR wherever possible
- Use EmbeddedByteChannel to test
- Use ByteBuf comparison instead of array comparison
- Work done by @lw346 and then revised by @trustin
2013-02-01 13:10:34 +09:00
Trustin Lee
2ec932798f Replace .readable() and .writable() to .isReadable() and .isWritable() 2013-01-31 18:24:33 +01:00
Trustin Lee
42c65cca3a Make MessageBuf bounded
- Move common methods from ByteBuf to Buf
- Rename ensureWritableBytes() to ensureWritable()
- Rename readable() to isReadable()
- Rename writable() to isWritable()
- Add isReadable(int) and isWritable(int)
- Add AbstractMessageBuf
- Rewrite DefaultMessageBuf and QueueBackedMessageBuf
  - based on Josh Bloch's public domain ArrayDeque impl
2013-01-31 18:11:06 +01:00
Norman Maurer
ec013bf2d3 [#983] Force the user to implement an actual ChannelInboundHandler or ChannelOutboundHandler
For this ChannelInboundHandler* and ChannelOutboundHandler* was made package private
2013-01-31 15:50:27 +01:00
Luke Wood
bd0339ce1a Fix buffer over-run in Snappy codec when emitting a copy that leaves < 4 bytes remaining 2013-01-30 12:15:57 +01:00
Norman Maurer
291293a6dc [#980] Automatically trigger a read operation if isAutoRead() == false but we only had a partial decode 2013-01-30 09:16:45 +01:00
Norman Maurer
d7bfd44e10 [#982] [#977] [#858] Allow to transfer the content a ChannelHandlers inbound/outbound buffer on removal/replacement
This changes the behavior of the ChannelPipeline.remove(..) and ChannelPipeline.replace(..) methods in that way
that after invocation it is not possible anymore to access any data in the inbound or outbound buffer. This is
because it empty it now to prevent side-effects. If a user want to preserve the content and forward it to the
next handler in the pipeline it is adviced to use one of the new methods which where introduced.

 - ChannelPipeline.removeAndForward(..)
 - ChannelPipeline.replaceAndForward(..)
2013-01-28 10:25:38 +01:00
Norman Maurer
b9aaf9a167 [#977] Stop processing messages/bytes in a loop when the handler was removed from the ChannelPipeline 2013-01-23 07:35:44 +01:00
Norman Maurer
f2d84f75d6 [#952] Allow to switch to single message decoding mode on the fly 2013-01-18 09:56:12 +01:00
Trustin Lee
5a82dccbc5 Tighten visibility 2013-01-17 14:43:59 +09:00
Trustin Lee
64ae8b6a37 Replace and merge DetectionUtil and DirectByteBufUtil into PlatformDependent and PlatformDependent0
PlatformDependent delegates the operations requires sun.misc.* to PlatformDependent0 to avoid runtime errors due to missing sun.misc.* classes.
2013-01-11 14:03:27 +09:00
Norman Maurer
b742dcc209 [#902] Remove usage of generics for output of Encoder/Decoder to make them more flexible again 2013-01-09 07:13:31 +01:00
Trustin Lee
5d2e0688ab Fix checkstyle 2013-01-09 15:12:25 +09:00
Trustin Lee
5c2be33cae Delegate to the actual encoder/decoder if possible 2013-01-09 15:09:20 +09:00
Trustin Lee
102563ec8f No need to override ByteToMessageCodec.decoder.discardInboundReadBytes() 2013-01-09 15:05:03 +09:00
Trustin Lee
ef692b0c38 Add missing decodeLast() 2013-01-09 14:58:44 +09:00
Trustin Lee
b58a8f0106 Add missing codec operations in ByteToMessageCodec 2013-01-09 14:56:07 +09:00
Trustin Lee
4a3d73724f Make all encoder/decoder methods overridable in MessageToMessageCodec 2013-01-09 14:44:19 +09:00
Trustin Lee
8d0785da36 Rename MessageToMessageEncoder.freeInboundMessage() to freeOutboundMessage() 2013-01-09 14:35:50 +09:00
Trustin Lee
8d5fd0839f Make ByteToMessageCodec.isEncodable() public like other similar methods 2013-01-09 14:28:42 +09:00
Trustin Lee
aa69d628f1 Add ByteToMessageCodec.isEncodable() 2013-01-09 14:26:15 +09:00
Trustin Lee
dd6b7969b7 Give a handler more control over how its buffers' read bytes are discarded.
This pull request adds two new handler methods: discardInboundReadBytes(ctx) and discardOutboundReadBytes(ctx) to ChannelInboundByteHandler and ChannelOutboundByteHandler respectively. They are called between every inboundBufferUpdated() and flush() respectively. Their default implementation is to call discardSomeReadBytes() on their buffers and a user can override this behavior easily. For example, ReplayingDecoder.discardInboundReadBytes() looks like the following:

    @Override
    public void discardInboundReadBytes(ChannelHandlerContext ctx) throws Exception {
        ByteBuf in = ctx.inboundByteBuffer();
        final int oldReaderIndex = in.readerIndex();
        super.discardInboundReadBytes(ctx);
        final int newReaderIndex = in.readerIndex();
        checkpoint -= oldReaderIndex - newReaderIndex;
    }

If a handler, which has its own buffer index variable, extends ReplayingDecoder or ByteToMessageDecoder, the handler can also override discardInboundReadBytes() and adjust its index variable accordingly.
2013-01-09 13:34:09 +09:00
kxbmap
cf2fbf7883 Fix a link in ZlibWrapper 2013-01-05 21:28:03 +01:00
Norman Maurer
5526153459 [#882] Add a PartialFlushException which will allow to notify the user that the flush/write was only partial succesful 2013-01-05 20:30:48 +01:00
Norman Maurer
ccb5409f58 [#884] Split SCTP transport into extra module 2013-01-03 22:19:06 +01:00
Norman Maurer
37a3f2e3b8 [#887] [#866] [#883] Add unified interface for Message oriented protocols and also use direct buffers for them 2013-01-03 18:15:53 +01:00
Norman Maurer
4e77bacdf7 [#873] [#868] Split ChannelFuture into ChannelFuture and ChannelPromise 2012-12-31 23:27:16 +09:00
Trustin Lee
0909878581 Read only when requested (read-on-demand)
This pull request introduces a new operation called read() that replaces the existing inbound traffic control method. EventLoop now performs socket reads only when the read() operation has been issued. Once the requested read() operation is actually performed, EventLoop triggers an inboundBufferSuspended event that tells the handlers that the requested read() operation has been performed and the inbound traffic has been suspended again. A handler can decide to continue reading or not.

Unlike other outbound operations, read() does not use ChannelFuture at all to avoid GC cost. If there's a good reason to create a new future per read at the GC cost, I'll change this.

This pull request consequently removes the readable property in ChannelHandlerContext, which means how the traffic control works changed significantly.

This pull request also adds a new configuration property ChannelOption.AUTO_READ whose default value is true. If true, Netty will call ctx.read() for you. If you need a close control over when read() is called, you can set it to false.

Another interesting fact is that non-terminal handlers do not really need to call read() at all. Only the last inbound handler will have to call it, and that's just enough. Actually, you don't even need to call it at the last handler in most cases because of the ChannelOption.AUTO_READ mentioned above.

There's no serious backward compatibility issue. If the compiler complains your handler does not implement the read() method, add the following:

public void read(ChannelHandlerContext ctx) throws Exception {
    ctx.read();
}

Note that this pull request certainly makes bounded inbound buffer support very easy, but itself does not add the bounded inbound buffer support.
2012-12-31 23:26:00 +09:00
Norman Maurer
64351ad7d6 [#861] Let SctpMessageCompletionHandler fire inboundMessageBufferUpdated() only if needed 2012-12-27 23:08:54 +01:00
Norman Maurer
a20aba87ab Remove get prefix from Sctp methods to be more inline with the rest 2012-12-27 22:53:42 +01:00
Norman Maurer
852f546b5b [#846] Tighten up visibility 2012-12-25 18:54:55 +01:00