Commit Graph

345 Commits

Author SHA1 Message Date
Trustin Lee
40003ed250 Resurrect Channel.id() with global uniqueness
- Fixes #1810
- Add a new interface ChannelId and its default implementation which generates globally unique channel ID.
- Replace AbstractChannel.hashCode with ChannelId.hashCode() and ChannelId.shortValue()
- Add variants of ByteBuf.hexDump() which accept byte[] instead of ByteBuf.
2014-02-13 15:53:07 -08:00
Trustin Lee
ef4bc99849 Remove unnecessary parenthesis 2014-02-13 15:52:06 -08:00
Norman Maurer
f23d68b42f [#2187] Always do a volatile read on the refCnt 2014-02-07 09:23:16 +01:00
Norman Maurer
9bee78f91c Provide an optimized AtomicIntegerFieldUpdater, AtomicLongFieldUpdater and AtomicReferenceFieldUpdater 2014-02-06 20:08:45 +01:00
Norman Maurer
b3d8c81557 Fix all leaks reported during tests
- One notable leak is from WebSocketFrameAggregator
- All other leaks are from tests
2013-12-07 00:44:56 +09:00
Trustin Lee
2102cb062b Fix false-positive leaks
- All derived buffers and swapped buffers of a leak-aware buffer must be wrapped again with the leak-aware buffer
2013-12-06 21:32:56 +09:00
Trustin Lee
e506581eb1 Add ReferenceCountUtil.releaseLater() to make writing tests easy with ReferenceCounteds 2013-12-06 15:13:00 +09:00
Trustin Lee
128c4b96b5 Checkstyle 2013-12-06 13:54:36 +09:00
Trustin Lee
5d39b1fc3d Also record retain() and release() 2013-12-06 13:45:24 +09:00
Trustin Lee
e88172495a Ensure backward compatibility
.. by resurrecting the removed methods and system properties.
2013-12-05 01:02:38 +09:00
Trustin Lee
65b522a2a7 Better buffer leak reporting
- Remove the reference to ResourceLeak from the buffer implementations
  and use wrappers instead:
  - SimpleLeakAwareByteBuf and AdvancedLeakAwareByteBuf
  - It is now allocator's responsibility to create a leak-aware buffer.
  - Added AbstractByteBufAllocator.toLeakAwareBuffer() for easier
    implementation
- Add WrappedByteBuf to reduce duplication between *LeakAwareByteBuf and
  UnreleasableByteBuf
- Raise the level of leak reports to ERROR - because it will break the
  app eventually
- Replace enabled/disabled property with the leak detection level
  - Only print stack trace when level is ADVANCED or above to avoid user
    confusion
- Add the 'leak' build profile, which enables highly detailed leak
  reporting during the build
- Remove ResourceLeakException which is unsed anymore
2013-12-05 00:51:39 +09:00
Norman Maurer
053c512f6d Fix checkstyle 2013-12-02 08:23:57 +01:00
Norman Maurer
14600167d6 [#2021] No need to synchronize for unpooled chunks 2013-12-02 08:02:48 +01:00
Norman Maurer
7231be592a Also allow to override how direct ByteBuffers are freed 2013-11-12 12:40:41 +01:00
Norman Maurer
e83fb821d5 Allow to override how wrapped direct ByteBuffer are allocated to make it easier to extend 2013-11-12 12:13:38 +01:00
Norman Maurer
b00f8c6390 [#1976] Fix IndexOutOfBoundsException when calling CompositeByteBuf.discardReadComponents() 2013-11-09 20:13:24 +01:00
Alex Petrov
e4f391f626 Improve docstrings for and of 2013-11-08 12:15:41 +01:00
Trustin Lee
ba3bc0c020 Simpler toString() for ByteBufAllocators 2013-11-08 17:54:34 +09:00
Norman Maurer
77b4ec7e1b [#1800] [#1802] Correctly expand capacity of ByteBuf while preserve content 2013-11-04 15:18:21 +01:00
Trustin Lee
54db9ec725 Use StringUtil.simpleClassName(..) instead of Class.getSimpleName() where necessary
- Class.getSimpleName() doesn't render anonymous classes very well
- + some minor cleanup
2013-11-04 19:46:15 +09:00
Norman Maurer
4ce49a6195 [#1943] Unpooled.copiedBuffer(ByteBuf pooled) should always return unpooled ByteBuf 2013-10-22 20:20:38 +02:00
Norman Maurer
68b616728a [#1925] Only expose sub-region of ByteBuf on nioBuffer(...) 2013-10-16 10:34:33 +02:00
Norman Maurer
d946659520 [#1906] Use a ByteBuf allocator from the ByteBufAllocator when encode Strings 2013-10-09 21:18:08 +02:00
Norman Maurer
1c73be21fc Remove redundant index check 2013-10-08 07:21:01 +02:00
Bill Gallagher
8f612660b2 disable debugging output during test 2013-10-04 06:45:26 +02:00
Norman Maurer
ee192f0321 [#1880] Use ByteBufAllocator when read bytes into new chunks 2013-10-01 10:10:43 +02:00
Norman Maurer
6d09e57be7 [#1875] Correctly check the readerIndex when try to read a byte from AbstractByteBuf 2013-09-30 14:47:49 +02:00
Norman Maurer
b4fa8af079 Cache underlying ByteBuffers and count in ChannelOutboundBuffer.Entry to reduce object creation and so GC pressure
Beside this it also helps to reduce CPU usage as nioBufferCount() is quite expensive when used on CompositeByteBuf which are
nested and contains a lot of components
2013-09-26 20:37:39 +02:00
Norman Maurer
2b9a07cac9 CompositeByteBuf.isDirect() should return true if its only backed by direct buffers 2013-09-26 20:37:31 +02:00
Norman Maurer
a74149e984 [#1865] Only use internalNioBuffer when one of the read* or write* methods are used. This is neccessary to prevent races as those can happen when a slice or duplicate is shared between different Channels
that are not assigned to the same EventLoop. In general get* operations should always be safe to be used from different Threads.

This aslo include unit tests that show the issue
2013-09-25 17:27:26 +02:00
Norman Maurer
910ed31a1b [#1851] EmptyByteBuf.isWritable(..) and isReadable(...) should not throw IndexOutOfBoundsException 2013-09-21 20:40:22 +02:00
Norman Maurer
23baef8fb4 [#1853] Optimize gathering writes for CompositeByteBuf that are only backed by one ByteBuffer 2013-09-19 07:29:58 +02:00
Norman Maurer
c0bbde48b7 [#1852] Fix bug in UnpooledDirectByteBuf.nioBuffer(...) implementation 2013-09-18 20:47:57 +02:00
Greg Soltis
f1d4f813ed Fix nioBuffer implementation for CompositeByteBuf 2013-09-16 06:41:08 +02:00
Norman Maurer
451e91d142 [#1821] Fix IndexOutOfBoundsException which was thrown if the last component was removed but other components was left 2013-09-09 20:29:30 +02:00
Norman Maurer
25c226a835 Make sure only direct ByteBuffer are passed to the underlying jdk Channel.
This is needed because of otherwise the JDK itself will do an extra ByteBuffer copy with it's own pool implementation. Even worth it will be done
multiple times if the ByteBuffer is always only partial written. With this change the copy is done inside of netty using it's own allocator and
only be done one time in all cases.
2013-09-02 20:17:53 +02:00
Norman Maurer
5416f2315e [#1797] No use internalNioBuffer() in derived buffers as it is not meant for concurrent access 2013-09-02 14:15:19 +02:00
Norman Maurer
0007fb81ef Add tests to try to track down some buffer issues 2013-09-02 13:50:47 +02:00
Norman Maurer
795182843d Remove legancy code which we not need anymore as we use gathering writes anyway everywhere 2013-09-01 11:00:58 +02:00
Norman Maurer
5ddd7cee90 [#1797] Throw IllegalArgumentException if AbstractByteBuf.skipBytes(...) is used with a negative value 2013-08-29 11:14:36 +02:00
Trustin Lee
20894bc99e Fix a bug in internalNioBuffer() implementations of derived buffers
- A user can create multiple duplicates of a buffer and access their internal NIO buffers. (e.g. write multiple duplicates to multiple channels assigned to different event loop.)  Because the derived buffers' internalNioBuffer() simply delegates the call to the original buffer, all derived buffers and the original buffer's internalNioBuffer() will return the same buffer, which will lead to a race condition.
- Fixes #1739
2013-08-20 14:28:50 +09:00
bgallagher
9f88552f12 remove some dead code 2013-08-10 20:46:48 +02:00
Norman Maurer
194b64cff1 [#1708] Correctly set the writerIndex in ReadOnlyByteBufferBuf if it is constructed with a buffer which has non zero position 2013-08-08 06:54:32 +02:00
Norman Maurer
ea1dca8105 [#1704] Make sure SwappedByteBuf.readSlice(..) returns ByteBuf with correct ByteOrder 2013-08-06 12:22:22 +02:00
Norman Maurer
8a673db92b [#1644] Fixed IndexOutOfBoundException when calling copy() on a empty CompositeByteBuf 2013-07-24 07:35:51 +02:00
Trustin Lee
764741c5ce Change the contract of ResourceLeakDetector.open() so that unsampled resources are recycled
- This also fixes the problem introduced while trying to implement #1612 (Allow to disable resource leak detection).
2013-07-23 14:06:58 +09:00
Norman Maurer
35802207e1 Fix compile error 2013-07-23 06:42:41 +02:00
kerr
ada07cb9e0 Fix types in javadocs 2013-07-22 19:14:36 +02:00
Norman Maurer
f478fcd3b0 [#1628] Fix bug in ReadOnlyByteByteBufferBuf where get operations threw ReadOnlyBufferException 2013-07-22 07:12:05 +02:00
Norman Maurer
feb8d101bd [#1626] Use static fields for default values 2013-07-22 06:47:29 +02:00
Shawn Silverman
674f4bce51 netty-1597: Rewrite ByteBufInputStream.readLine() to avoid IndexOutOfBoundsException and to behave more correctly for lines ending in '\r'. 2013-07-20 08:05:54 +02:00
Trustin Lee
f0a3f849f7 Fix a bug in AbstractByteBuf.writeZero() where the capacity is not auto-expanded 2013-07-18 09:55:02 +09:00
Norman Maurer
df5daadd0f Remove unused import 2013-07-15 10:09:17 +02:00
Norman Maurer
7254a5c2c6 Just some tiny javadocs optimizations 2013-07-14 16:02:03 +02:00
Trustin Lee
65c2a6ed46 Make ByteBuf an abstract class rather than an interface
- 5% improvement in throughput (HelloWorldServer example)
- Made CompositeByteBuf a concrete class (renamed from DefaultCompositeByteBuf) because there's no multiple inheritance in Java

Fixes #1536
2013-07-08 14:59:52 +09:00
Norman Maurer
086ae3536c [#1533] Introduce ByteBufHolder.duplicate() and make use of it in DefaultChannelGroup.write(...) 2013-07-06 21:17:51 +02:00
Norman Maurer
7dda4b9ce4 [#1532] Remove @deprecated ByteBufIndexFinder and all methods that take it as argument 2013-07-06 20:14:53 +02:00
Trustin Lee
dfc05a6ed7 Fix documentation error in ByteBuf
- Fixes #1531
- Thanks to @daschl
2013-07-05 17:03:34 +09:00
Trustin Lee
0b9235f072 Simplify ByteBufProcessor and MessageListProcessor and Add internal component accessors to CompositeByteBuf
- Fixes #1528

It's not really easy to provide a general-purpose abstraction for fast-yet-safe iteration. Instead of making forEachByte() less optimal, let's make it do what it does really well, and allow a user to implement potentially unsafe-yet-fast loop using unsafe operations.
2013-07-05 14:00:46 +09:00
Norman Maurer
7ec12d327f Remove deprecated ByteBufUtil.release(..) and ByteBufUtil.retain(..) methods and its usage. Also fix a problem where an object would have been released two times.
* The problem with the release(..) calls here was that it would have called release on an unsupported message and then throw an exception. This exception will trigger ChannelOutboundBuffer.fail(..), which will also try to release the message again.
* Also use the same exception type for unsupported messages as in other channel impls.
2013-07-03 10:00:13 +02:00
Trustin Lee
4b11aff08f Less confusing log messages for system properties
- Fixes #1502
2013-07-02 09:23:29 +09:00
Norman Maurer
5d88c423df [#1500] Remove @deprecated methods 2013-07-01 08:53:02 +02:00
Trustin Lee
0dcf352f4c Vastly simplified ByteBufProcessor and MessageListProcessor
- Related: #1378
- They now accept only one argument.
- A user who wants to use a buffer for more complex use cases, he or she can always access the buffer directly via memoryAddress() and array()
2013-06-28 20:29:00 +09:00
Trustin Lee
52691488ee Update Javadoc of ByteBufProcessor and MessageListProcessor
- in response to @shacharo's suggestion
2013-06-27 19:01:01 +09:00
Trustin Lee
ac39cad5ff Split ByteBuf.forEachByte() into forEachByte() and forEachByteDesc()
- Related: #1378
- As suggested by @liqweed
2013-06-27 18:48:09 +09:00
Trustin Lee
9804741fb3 Fix test failure in SlicedByteBuf / Add tests for built-in ByteBufProcessor impls
- Related: #1378
2013-06-27 17:49:46 +09:00
Trustin Lee
98531313de Optimize derived buffers' forEachByte(...) implementation
- Related: #1378
2013-06-27 17:36:22 +09:00
Trustin Lee
b5bb36c087 Use (fromIndex, toIndex) instead of (index, length) for ByteBuf.forEachByte(...)
- Related: #1378
2013-06-27 17:30:19 +09:00
Trustin Lee
792edf631c Deprecate ByteBufIndexFinder
- Prefer ByteBufProcessor
- Related: #1378
2013-06-27 14:26:58 +09:00
Trustin Lee
4dd9b6ef2e Add ByteBufProcessor and ByteBuf.forEach(...)
- Fixes #1378
- Needs to provide optimized forEach implementations though.
2013-06-27 13:55:42 +09:00
Norman Maurer
58b968b603 [#1454] Fix IndexOutOfBoundsException which was thrown if last component of a CompositeByteBuf was removed 2013-06-25 09:32:00 +02:00
Trustin Lee
dbab41cc50 Improve the utilization of subpage pools
.. by avoiding the overly frequent removal of a subpage from a pool

This change makes sure that the unused subpage is not removed when there's no subpage left in the pool.  If the last subpage is removed from the pool, it is very likely that the allocator will create a new subpage very soon again, so it's better not remove it.
2013-06-25 11:07:15 +09:00
Trustin Lee
a2f232720b Make AdaptiveRecvByteBufAllocator's lookup table simpler / Optimize buffer size normalization
- No need to have fine-grained lookup table because the buffer pool has
  much more coarse capacities available
- No need to use a loop to normalize a buffer capacity
2013-06-25 11:07:14 +09:00
Norman Maurer
6a9f965f9b Introduce new utility class calles ReferenceCountUtil and move utility methods from ByteBufUtil to it.
The ones in ByteBufUtil were marked as @deprecated
2013-06-14 07:07:33 +02:00
Trustin Lee
0da48e7e7f Determine the default number of heap/direct arenas of PooledByteBufAllocator conservatively
- Fixes #1445
- Add PlatformDependent.maxDirectMemory()
- Ensure the default number or arenas is decreased if the max memory of the VM is not large enough.
2013-06-14 12:14:45 +09:00
Trustin Lee
6d1cd0d0cd ReferenceCountException -> IllegalReferenceCountException 2013-06-13 14:00:15 +09:00
Trustin Lee
7eb0f6105d Fix memory leaks 2013-06-13 13:32:47 +09:00
Trustin Lee
175526b6bd Move ReferenceCounted and AbstractReferenceCounted to io.netty.util
- Fixes #1441
- Also move and rename IllegalBufferAccessException to ReferenceCountException
- Prettier reference count exception messages
2013-06-13 13:14:21 +09:00
Trustin Lee
283feda119 Reduce even more garbage by exposing ByteBuf.internalNioBuffer() 2013-06-13 12:40:26 +09:00
Trustin Lee
2d7c6f8ee1 Make PooledByteBuf recyclable regardless its maxCapacity
- Make AbstractByteBuf.maxCapacity internally mutable so that PooledByteBuf is completely recyclable
2013-06-12 04:18:40 +09:00
Trustin Lee
9396246fe9 " " -> " " / Cleanup 2013-06-12 04:07:09 +09:00
Norman Maurer
341f7757aa Fix checkstyle 2013-06-11 16:12:34 +02:00
Norman Maurer
bf046492fb [#1439] Fix CompositeByteBuf.nioBufferCount() to return the correct number 2013-06-11 16:07:40 +02:00
Trustin Lee
bf5960e9eb Fix #1435 and #1436 by reverting 7f7bf304b0
Different PooledByteBufs can have the reference to the same PooledChunk (and its ByteBuffer), so it's incorrect not to create a duplicate.
2013-06-11 00:57:23 +09:00
Trustin Lee
7f7bf304b0 Optimize PooledUnsafeDirectByteBuf.newInternalNioBuffer()
- No need to produce garbage
2013-06-10 22:08:30 +09:00
Trustin Lee
6732c6761b Recycle PooledByteBuf partially
- Related issue: #1397
- Resource leak detection should be turned off and the maxCapacity has to be Integer.MAX_VALUE
- It's technically possible to pool PooledByteBufs with different maxCapacity, which will be addressed in another commit.
2013-06-10 19:52:56 +09:00
Derek Troy-West
f2d8a745b1 [#1422] ReadOnlyByteBuffer.isWritable() should return false 2013-06-10 09:23:57 +02:00
Trustin Lee
14158070bf Revamp the core API to reduce memory footprint and consumption
The API changes made so far turned out to increase the memory footprint
and consumption while our intention was actually decreasing them.

Memory consumption issue:

When there are many connections which does not exchange data frequently,
the old Netty 4 API spent a lot more memory than 3 because it always
allocates per-handler buffer for each connection unless otherwise
explicitly stated by a user.  In a usual real world load, a client
doesn't always send requests without pausing, so the idea of having a
buffer whose life cycle if bound to the life cycle of a connection
didn't work as expected.

Memory footprint issue:

The old Netty 4 API decreased overall memory footprint by a great deal
in many cases.  It was mainly because the old Netty 4 API did not
allocate a new buffer and event object for each read.  Instead, it
created a new buffer for each handler in a pipeline.  This works pretty
well as long as the number of handlers in a pipeline is only a few.
However, for a highly modular application with many handlers which
handles connections which lasts for relatively short period, it actually
makes the memory footprint issue much worse.

Changes:

All in all, this is about retaining all the good changes we made in 4 so
far such as better thread model and going back to the way how we dealt
with message events in 3.

To fix the memory consumption/footprint issue mentioned above, we made a
hard decision to break the backward compatibility again with the
following changes:

- Remove MessageBuf
- Merge Buf into ByteBuf
- Merge ChannelInboundByte/MessageHandler and ChannelStateHandler into ChannelInboundHandler
  - Similar changes were made to the adapter classes
- Merge ChannelOutboundByte/MessageHandler and ChannelOperationHandler into ChannelOutboundHandler
  - Similar changes were made to the adapter classes
- Introduce MessageList which is similar to `MessageEvent` in Netty 3
- Replace inboundBufferUpdated(ctx) with messageReceived(ctx, MessageList)
- Replace flush(ctx, promise) with write(ctx, MessageList, promise)
- Remove ByteToByteEncoder/Decoder/Codec
  - Replaced by MessageToByteEncoder<ByteBuf>, ByteToMessageDecoder<ByteBuf>, and ByteMessageCodec<ByteBuf>
- Merge EmbeddedByteChannel and EmbeddedMessageChannel into EmbeddedChannel
- Add SimpleChannelInboundHandler which is sometimes more useful than
  ChannelInboundHandlerAdapter
- Bring back Channel.isWritable() from Netty 3
- Add ChannelInboundHandler.channelWritabilityChanges() event
- Add RecvByteBufAllocator configuration property
  - Similar to ReceiveBufferSizePredictor in Netty 3
  - Some existing configuration properties such as
    DatagramChannelConfig.receivePacketSize is gone now.
- Remove suspend/resumeIntermediaryDeallocation() in ByteBuf

This change would have been impossible without @normanmaurer's help. He
fixed, ported, and improved many parts of the changes.
2013-06-10 16:10:39 +09:00
Norman Maurer
bc20107b68 Use correct value to disable/enable direct arenas in PooledByteBufAllocator 2013-05-30 20:24:11 +02:00
Trustin Lee
f92cfba388 Remove DefaultCompositeByteBuf.lastAccessed and use binary search instead
- Fixes #1364
- Even if a user creates a duplicate/slice, lastAccessed was shared between the derived buffers and it's updated even by a read operation, which made multithread access impossible
2013-05-17 16:54:47 +09:00
Trustin Lee
2a15f658d6 Ensure UnreleasableByteBuf does not expose the wrapped buffer to the caller
- Fixes #1324
2013-05-01 20:43:23 +09:00
Trustin Lee
1e0c83db23 Introduce AddressedEnvelope message type for generic representation of an addressed message
- Fixes #1282 (not perfectly, but to the extent it's possible with the current API)
- Add AddressedEnvelope and DefaultAddressedEnvelope
- Make DatagramPacket extend DefaultAddressedEnvelope<ByteBuf, InetSocketAddress>
- Rename ByteBufHolder.data() to content() so that a message can implement both AddressedEnvelope and ByteBufHolder (DatagramPacket does) without introducing two getter methods for the content
- Datagram channel implementations now understand ByteBuf and ByteBufHolder as a message with unspecified remote address.
2013-05-01 17:04:43 +09:00
Trustin Lee
2e0dd65250 Fix a bug where the unpooled buffer returned by the pooled allocator reports an incorrect allocator 2013-05-01 11:14:21 +09:00
Trustin Lee
a218eb6f6f Allow to disable only heap or direct buffer pool
- Fixes #1315

If a user specifies the arena size of 0, the pool is now disabled
instead of raising an IllegalArgumentException. Using this, you can
disable only heap or direct buffer pool easily. Once disabled,
PooledByteBufAllocator will delegate the allocation request to
UnpooledByteBufAllocator.
2013-04-27 08:55:16 +09:00
Trustin Lee
b5989e2449 Reduce exception instantiation overhead in SslHandler / Reduce unnecessary empty array creation
- Added EmptyArrays as an internal utility class
2013-04-24 09:32:53 +09:00
Trustin Lee
32fa4c07f3 Do not unwrap a CompositeByteBuf when it is added as a component of another CompositeByteBuf
.. because Reference counting introduces life cycle issues to the CompositeByteBuf being added.

 - Fixes #1266
2013-04-23 21:53:51 +09:00
Trustin Lee
87007d4eb8 Fix another memory leak in AbstractByteBufTest 2013-04-23 13:46:34 +09:00
Norman Maurer
9a5f45a0c1 [#1297] Make sure ResourceLeakDetector.open(...) is only used after constructing was successful 2013-04-22 10:07:22 +02:00
Norman Maurer
ab685de7a3 [#1273] Fix resource leaks in test 2013-04-22 09:47:44 +02:00
Trustin Lee
18dca2a8a4 Fix checkstyle 2013-04-19 04:37:51 +09:00