Commit Graph

971 Commits

Author SHA1 Message Date
garywu
4d02c3a040 [#2925] Bug fix for NormalMemoryRegionCache overbooked for PoolThreadCache
Motivation:

When create NormalMemoryRegionCache for PoolThreadCache, we overbooked
cache array size. This means unnecessary overhead for thread local cache
as we will create multi cache enties for each element in cache array.

Modifications:

change:
int arraySize = Math.max(1, max / area.pageSize);
to:
int arraySize = Math.max(1, log2(max / area.pageSize) + 1);

Result:

Now arraySize won't introduce unnecessary overhead.

 Changes to be committed:
	modified:   buffer/src/main/java/io/netty/buffer/PoolThreadCache.java
2015-04-13 09:02:10 +02:00
Norman Maurer
18627749a9 Let CompositeByteBuf implement Iterable
Motivation:

CompositeByteBuf has an iterator() method but fails to implement Iterable

Modifications:

Let CompositeByteBuf implement Iterable<ByteBuf>

Result:

Easier usage
2015-04-12 13:38:27 +02:00
Norman Maurer
d8e5d421e1 Revert "Dereference when calling PooledByteBuf.deallocate()"
This reverts commit 7094c7b797.
2015-04-11 06:44:32 +02:00
Norman Maurer
7094c7b797 Dereference when calling PooledByteBuf.deallocate()
Motivation:

We missed to dereference the chunk and tmpNioBuf when calling deallocate(). This means the GC can not collect these as we still hold a reference while have the PooledByteBuf in the recycler stack.

Modifications:

Dereference chunk and tmpNioBuf.

Result:

GC can collect things.
2015-04-10 21:47:14 +02:00
Norman Maurer
3e42292d8b Change PoolThreadCache to use LIFO for better cache performance
Motiviation:

At the moment we use FIFO for the PoolThreadCache which is sub-optimal as this may reduce the changes to have the cached memory actual still in the cpu-cache.

Modification:

- Change to use LIFO as this increase the chance to be able to serve buffers from the cpu-cache

Results:

Faster allocation out of the ThreadLocal cache.

Before the commit:
[xxx wrk]$ ./wrk -H 'Connection: keep-alive' -d 120 -c 256 -t 16 -s scripts/pipeline-many.lua  http://xxx:8080/plaintext
Running 2m test @ http://xxx:8080/plaintext
  16 threads and 256 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    14.69ms   10.06ms 131.43ms   80.10%
    Req/Sec   283.89k    40.37k  433.69k    66.81%
  533859742 requests in 2.00m, 72.09GB read
Requests/sec: 4449510.51
Transfer/sec:    615.29MB

After the commit:
[xxx wrk]$ ./wrk -H 'Connection: keep-alive' -d 120 -c 256 -t 16 -s scripts/pipeline-many.lua  http://xxx:8080/plaintext
Running 2m test @ http://xxx:8080/plaintext
  16 threads and 256 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    16.38ms   26.32ms 734.06ms   97.38%
    Req/Sec   283.86k    39.31k  361.69k    83.38%
  540836511 requests in 2.00m, 73.04GB read
Requests/sec: 4508150.18
Transfer/sec:    623.40MB
2015-04-10 20:57:54 +02:00
Scott Mitchell
0d3a6e0511 HTTP/2 Decoder reduce preface conditional checks
Motivation:
The DefaultHttp2ConnectionDecoder class is calling verifyPrefaceReceived() for almost every frame event at all times.
The Http2ConnectionHandler class is calling readClientPrefaceString() on every decode event.

Modifications:
- DefaultHttp2ConnectionDecoder should not have to continuously call verifyPrefaceReceived() because it transitions boolean state 1 time for each connection.
- Http2ConnectionHandler should not have to continuously call readClientPrefaceString() because it transitions boolean state 1 time for each connection.

Result:
- Less conditional checks for the mainstream usage of the connection.
2015-03-28 18:52:35 -07:00
Leo Gomes
4500adb6e0 Updates the javadoc of Unpooled to remove mention to methods it does not provide
Motivation:

`Unpooled` javadoc's mentioned the generation of hex dump and swapping an integer's byte order,
which are actually provided by `ByteBufUtil`.

Modifications:

 Sentence moved to `ByteBufUtil` javadoc.

Result:

`Unpooled` javadoc is correct.
2015-03-04 12:04:14 +09:00
Norman Maurer
fce0989844 [maven-release-plugin] prepare for next development iteration 2015-03-03 02:06:47 -05:00
Norman Maurer
ca3b1bc4b7 [maven-release-plugin] prepare release netty-4.1.0.Beta4 2015-03-03 02:05:52 -05:00
Norman Maurer
41fd857a7c Ensure CompositeByteBuf.addComponent* handles buffer in consistent way and not causes leaks
Motivation:

At the moment we have two problems:
 - CompositeByteBuf.addComponent(...) will not add the supplied buffer to the CompositeByteBuf if its empty, which means it will not be released on CompositeByteBuf.release() call. This is a problem as a user will expect everything added will be released (the user not know we not added it).
 - CompositeByteBuf.addComponents(...) will either add no buffers if none is readable and so has the same problem as addComponent(...) or directly release the ByteBuf if at least one ByteBuf is readable. Again this gives inconsistent handling and may lead to memory leaks.

Modifications:

 - Always add the buffer to the CompositeByteBuf and so release it on release call.

Result:

Consistent handling and no buffer leaks.
2015-02-12 16:09:41 +01:00
Trustin Lee
155c0e2f36 Implement internal memory access methods of CompositeByteBuf correctly
Motivation:

When a CompositeByteBuf is empty (i.e. has no component), its internal
memory access operations do not always behave as expected.

Modifications:

Check if the nunmber of components is zero. If so, return an empty
array or an empty NIO buffer, etc.

Result:

More robustness
2014-12-30 15:56:53 +09:00
Trustin Lee
a666acce6d Add more tests to EmptyByteBufTest
- Ensure an EmptyByteBuf has an array, an NIO buffer, and a memory
  address at the same time
- Add an assertion that checks if EMPTY_BUFFER is an EmptyByteBuf,
  just in case we make a mistake in the future
2014-12-30 15:51:45 +09:00
Norman Maurer
fe796fc8ab Provide helper methods in ByteBufUtil to write UTF-8/ASCII CharSequences. Related to [#909]
Motivation:

We expose no methods in ByteBuf to directly write a CharSequence into it. This leads to have the user either convert the CharSequence first to a byte array or use CharsetEncoder. Both cases have some overheads and we can do a lot better for well known Charsets like UTF-8 and ASCII.

Modifications:

Add ByteBufUtil.writeAscii(...) and ByteBufUtil.writeUtf8(...) which can do the task in an optimized way. This is especially true if the passed in ByteBuf extends AbstractByteBuf which is true for all of our implementations which not wrap another ByteBuf.

Result:

Writing an ASCII and UTF-8 CharSequence into a AbstractByteBuf is a lot faster then what the user could do by himself as we can make use of some package private methods and so eliminate reference and range checks. When the Charseq is not ASCII or UTF-8 we can still do a very good job and are on par in most of the cases with what the user would do.

The following benchmark shows the improvements:

Result: 2456866.966 ?(99.9%) 59066.370 ops/s [Average]
  Statistics: (min, avg, max) = (2297025.189, 2456866.966, 2586003.225), stdev = 78851.914
  Confidence interval (99.9%): [2397800.596, 2515933.336]

Benchmark                                                        Mode   Samples        Score  Score error    Units
i.n.m.b.ByteBufUtilBenchmark.writeAscii                         thrpt        50  9398165.238   131503.098    ops/s
i.n.m.b.ByteBufUtilBenchmark.writeAsciiString                   thrpt        50  9695177.968   176684.821    ops/s
i.n.m.b.ByteBufUtilBenchmark.writeAsciiStringViaArray           thrpt        50  4788597.415    83181.549    ops/s
i.n.m.b.ByteBufUtilBenchmark.writeAsciiStringViaArrayWrapped    thrpt        50  4722297.435    98984.491    ops/s
i.n.m.b.ByteBufUtilBenchmark.writeAsciiStringWrapped            thrpt        50  4028689.762    66192.505    ops/s
i.n.m.b.ByteBufUtilBenchmark.writeAsciiViaArray                 thrpt        50  3234841.565    91308.009    ops/s
i.n.m.b.ByteBufUtilBenchmark.writeAsciiViaArrayWrapped          thrpt        50  3311387.474    39018.933    ops/s
i.n.m.b.ByteBufUtilBenchmark.writeAsciiWrapped                  thrpt        50  3379764.250    66735.415    ops/s
i.n.m.b.ByteBufUtilBenchmark.writeUtf8                          thrpt        50  5671116.821   101760.081    ops/s
i.n.m.b.ByteBufUtilBenchmark.writeUtf8String                    thrpt        50  5682733.440   111874.084    ops/s
i.n.m.b.ByteBufUtilBenchmark.writeUtf8StringViaArray            thrpt        50  3564548.995    55709.512    ops/s
i.n.m.b.ByteBufUtilBenchmark.writeUtf8StringViaArrayWrapped     thrpt        50  3621053.671    47632.820    ops/s
i.n.m.b.ByteBufUtilBenchmark.writeUtf8StringWrapped             thrpt        50  2634029.071    52304.876    ops/s
i.n.m.b.ByteBufUtilBenchmark.writeUtf8ViaArray                  thrpt        50  3397049.332    57784.119    ops/s
i.n.m.b.ByteBufUtilBenchmark.writeUtf8ViaArrayWrapped           thrpt        50  3318685.262    35869.562    ops/s
i.n.m.b.ByteBufUtilBenchmark.writeUtf8Wrapped                   thrpt        50  2473791.249    46423.114    ops/s
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1,387.417 sec - in io.netty.microbench.buffer.ByteBufUtilBenchmark

Results :

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

Results :

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

The *ViaArray* benchmarks are basically doing a toString().getBytes(Charset) which the others are using ByteBufUtil.write*(...).
2014-12-26 15:58:18 +09:00
Norman Maurer
66294892a0 CompositeByteBuf.nioBuffers(...) must not return an empty ByteBuffer array
Motivation:

CompositeByteBuf.nioBuffers(...) returns an empty ByteBuffer array if the specified length is 0. This is not consistent with other ByteBuf implementations which return an ByteBuffer array of size 1 with an empty ByteBuffer included.

Modifications:

Make CompositeByteBuf.nioBuffers(...) consistent with other ByteBuf implementations.

Result:

Consistent and correct behaviour of nioBufffers(...)
2014-12-22 11:18:32 +01:00
Norman Maurer
a69a39c849 Always return SliceByteBuf on slice(...) to eliminate possible leak
Motivation:

When calling slice(...) on a ByteBuf the returned ByteBuf should be the slice of a ByteBuf and shares it's reference count. This is important as it is perfect legal to use buf.slice(...).release() and have both, the slice and the original ByteBuf released. At the moment this is only the case if the requested slice size is > 0. This makes the behavior inconsistent and so may lead to a memory leak.

Modifications:

- Never return Unpooled.EMPTY_BUFFER when calling slice(...).
- Adding test case for buffer.slice(...).release() and buffer.duplicate(...).release()

Result:

Consistent behaviour and so no more leaks possible.
2014-12-22 11:15:50 +01:00
Norman Maurer
182c91f06c Ensure buffer is not released when call array() / memoryAddress()
Motivation:

Before we missed to check if a buffer was released before we return the backing byte array or memoryaddress. This could lead to JVM crashes when someone tried various bulk operations on the Unsafe*ByteBuf implementations.

Modifications:

Always check if the buffer is released before all to return the byte array and memoryaddress.

Result:

No more JVM crashes because of released buffers when doing bulk operations on Unsafe*ByteBuf implementations.
2014-12-11 11:30:31 +01:00
Idel Pivnitskiy
35db3c6710 Small performance improvements
Motivation:

Found performance issues via FindBugs and PMD.

Modifications:

- Removed unnecessary boxing/unboxing operations in DefaultTextHeaders.convertToInt(CharSequence) and DefaultTextHeaders.convertToLong(CharSequence). A boxed primitive is created from a string, just to extract the unboxed primitive value.
- Added a static modifier for DefaultHttp2Connection.ParentChangedEvent class. This class is an inner class, but does not use its embedded reference to the object which created it. This reference makes the instances of the class larger, and may keep the reference to the creator object alive longer than necessary.
- Added a static compiled Pattern to avoid compile it each time it is used when we need to replace some part of authority.
- Improved using of StringBuilders.

Result:

Performance improvements.
2014-11-20 00:10:06 -05:00
Norman Maurer
48f1398869 Disable caching of PooledByteBuf for different threads.
Motivation:

We introduced a PoolThreadCache which is used in our PooledByteBufAllocator to reduce the synchronization overhead on PoolArenas when allocate / deallocate PooledByteBuf instances. This cache is used for both the allocation path and deallocation path by:
  - Look for cached memory in the PoolThreadCache for the Thread that tries to allocate a new PooledByteBuf and if one is found return it.
  - Add the memory that is used by a PooledByteBuf to the PoolThreadCache of the Thread that release the PooledByteBuf

This works out very well when all allocation / deallocation is done in the EventLoop as the EventLoop will be used for read and write. On the otherside this can lead to surprising side-effects if the user allocate from outside the EventLoop and and pass the ByteBuf over for writing. The problem here is that the memory will be added to the PoolThreadCache that did the actual write on the underlying transport and not on the Thread that previously allocated the buffer.

Modifications:

Don't cache if different Threads are used for allocating/deallocating

Result:

Less confusing behavior for users that allocate PooledByteBufs from outside the EventLoop.
2014-09-22 13:39:31 +02:00
Norman Maurer
858de5699b [#2924] Correctly update head in MemoryRegionCache.trim()
Motivation:
When MemoryRegionCache.trim() is called, some unused cache entries will be freed (started from head). However, in MeoryRegionCache.trim() the head is not updated, which make entry list's head point to an entry whose chunk is null now and following allocate of MeoryRegionCache will return false immediately.

In other word, cache is no longer usable once trim happen.

Modifications:

Update head to correct idx after free entries in trim().

Result:

MemoryRegionCache behaves correctly even after calling trim().
2014-09-22 11:04:21 +02:00
Norman Maurer
4e62b51c6d [#2843] Add test-case to show correct behavior of ByteBuf.refCnt() and ByteBuf.release(...)
Motivation:

We received a bug-report that the ByteBuf.refCnt() does sometimes not show the correct value when release() and refCnt() is called from different Threads.

Modifications:

Add test-case which shows that all is working like expected

Result:

Test-case added which shows everything is ok.
2014-09-01 08:50:21 +02:00
Trustin Lee
b5f61d0de5 [maven-release-plugin] prepare for next development iteration 2014-08-16 03:27:42 +09:00
Trustin Lee
76ac3b21a5 [maven-release-plugin] prepare release netty-4.1.0.Beta3 2014-08-16 03:27:37 +09:00
Trustin Lee
b3c1904cc9 [maven-release-plugin] prepare for next development iteration 2014-08-15 09:31:03 +09:00
Trustin Lee
e013b2400f [maven-release-plugin] prepare release netty-4.1.0.Beta2 2014-08-15 09:30:59 +09:00
Trustin Lee
0dc6a8dccf Use heap buffers for Unpooled.copiedBuffer()
Related issue: #2028

Motivation:

Some copiedBuffer() methods in Unpooled allocated a direct buffer.  An
allocation of a direct buffer is an expensive operation, and thus should
be avoided for unpooled buffers.

Modifications:

- Use heap buffers in all copiedBuffer() methods

Result:

Unpooled.copiedBuffers() are less expensive now.
2014-08-13 15:10:11 -07:00
Norman Maurer
ef572d859d Change back default allocator to pooled.
Motivation:

While porting some changes from 4.0 to 4.1 and master branch I changed the default allocator from pooled to unpooled by mistake. This should be reverted. The guilty commit is 4a3ef90381.

Thanks to @blucas for spotting this.

Modifications:

Revert changes related to allocator.

Result:

Use the correct default allocator again.
2014-08-13 12:07:06 +02:00
Norman Maurer
869687bd71 Port ChannelOutboundBuffer and related changes from 4.0
Motivation:

We did various changes related to the ChannelOutboundBuffer in 4.0 branch. This commit port all of them over and so make sure our branches are synced in terms of these changes.

Related to [#2734], [#2709], [#2729], [#2710] and [#2693] .

Modification:
Port all changes that was done on the ChannelOutboundBuffer.

This includes the port of the following commits:
 - 73dfd7c01b
 - 997d8c32d2
 - e282e504f1
 - 5e5d1a58fd
 - 8ee3575e72
 - d6f0d12a86
 - 16e50765d1
 - 3f3e66c31a

Result:
 - Less memory usage by ChannelOutboundBuffer
 - Same code as in 4.0 branch
 - Make it possible to use ChannelOutboundBuffer with Channel implementation that not extends AbstractChannel
2014-08-05 15:00:45 +02:00
Trustin Lee
9a654d8a61 Remove duplicate range check in AbstractByteBuf.skipBytes() 2014-07-29 15:58:28 -07:00
Idel Pivnitskiy
ad1389be9d Small performance improvements
Modifications:

- Added a static modifier for CompositeByteBuf.Component.
This class is an inner class, but does not use its embedded reference to the object which created it. This reference makes the instances of the class larger, and may keep the reference to the creator object alive longer than necessary.
- Removed unnecessary boxing/unboxing operations in HttpResponseDecoder, RtspResponseDecoder, PerMessageDeflateClientExtensionHandshaker and PerMessageDeflateServerExtensionHandshaker
A boxed primitive is created from a String, just to extract the unboxed primitive value.
- Removed unnecessary 3 times calculations in DiskAttribute.addContent(...).
- Removed unnecessary checks if file exists before call mkdirs() in NativeLibraryLoader and PlatformDependent.
Because the method mkdirs() has this check inside.
- Removed unnecessary `instanceof AsciiString` check in StompSubframeAggregator.contentLength(StompHeadersSubframe) and StompSubframeDecoder.getContentLength(StompHeaders, long).
Because StompHeaders.get(CharSequence) always returns java.lang.String.
2014-07-20 09:26:04 +02:00
Norman Maurer
f88dfd0430 [#2653] Remove unnecessary ensureAccessible() calls
Motivation:

I introduced ensureAccessible() class as part of 6c47cc9711 in some places. Unfortunally I also added some where these are not needed and so caused a performance regression.

Modification:

Remove calls where not needed.

Result:

Fixed performance regression.
2014-07-14 21:04:12 +02:00
Norman Maurer
93c306602a [#2653] Remove uncessary range checks for performance reasons
Motivation:

I introduced range checks as part of 6c47cc9711 in some places. Unfortunally I also added some where these are not needed and so caused a performance regression.

Modification:

Remove range checks where not needed

Result:

Fixed performance regression.
2014-07-14 11:43:19 +02:00
Brendt Lucas
ac8ac59148 [#2642] CompositeByteBuf.deallocate memory/GC improvement
Motivation:

CompositeByteBuf.deallocate generates unnecessary GC pressure when using the 'foreach' loop, as a 'foreach' loop creates an iterator when looping.

Modification:

Convert 'foreach' loop into regular 'for' loop.

Result:

Less GC pressure (and possibly more throughput) as the 'for' loop does not create an iterator
2014-07-08 21:08:14 +02:00
Trustin Lee
e167b02d52 [maven-release-plugin] prepare for next development iteration 2014-07-04 17:26:02 +09:00
Trustin Lee
ba50cb829b [maven-release-plugin] prepare release netty-4.1.0.Beta1 2014-07-04 17:25:54 +09:00
Trustin Lee
787663a644 [maven-release-plugin] rollback the release of netty-4.1.0.Beta1 2014-07-04 17:11:14 +09:00
Trustin Lee
83eae705e1 [maven-release-plugin] prepare release netty-4.1.0.Beta1 2014-07-04 17:02:17 +09:00
Trustin Lee
fbf1bdbef1 Fix the build timeout when 'leak' profile is active
Motivation:

AbstractByteBufTest.testInternalBuffer() uses writeByte() operations to
populate the sample data.  Usually, this isn't a problem, but it starts
to take a lot of time when the resource leak detection level gets
higher.

In our CI machine, testInternalBuffer() takes more than 30 minutes,
causing the build timeout when the 'leak' profile is active (paranoid
level resource detection.)

Modification:

Populate the sample data using ThreadLocalRandom.nextBytes() instead of
using millions of writeByte() operations.

Result:

Test runs much faster when leak detection level is high.
2014-07-03 17:55:10 +09:00
Trustin Lee
d0912f2709 Fix most inspector warnings
Motivation:

It's good to minimize potentially broken windows.

Modifications:

Fix most inspector warnings from our profile
Update IntObjectHashMap

Result:

Cleaner code
2014-07-02 19:55:07 +09:00
Norman Maurer
9594a81b95 [#2622] Correctly check reference count before try to work on the underlying memory
Motivation:

Because of how we use reference counting we need to check for the reference count before each operation that touches the underlying memory. This is especially true as we use sun.misc.Cleaner.clean() to release the memory ASAP when possible. Because of this the user may cause a SEGFAULT if an operation is called that tries to access the backing memory after it was released.

Modification:

Correctly check the reference count on all methods that access the underlying memory or expose it via a ByteBuffer.

Result:

Safer usage of ByteBuf
2014-06-30 07:14:25 +02:00
Trustin Lee
c0462c0c3b Optimize PoolChunk
- Using short[] for memoryMap did not improve performance. Reverting
  back to the original dual-byte[] structure in favor of simplicity.
- Optimize allocateRun() which yields small performence improvement
- Use local variable when member fields are accessed more than once
2014-06-26 17:06:10 +09:00
Trustin Lee
dbc011c3f4 Fix inspector warnings 2014-06-26 17:06:10 +09:00
Pavan Kumar
69a6ad940a Improve the allocation algorithm in PoolChunk
Motivation:

Depth-first search is not always efficient for buddy allocation.

Modification:

Employ a new faster search algorithm with different memoryMap layout.

Result:

With thread-local cache disabled, we see a lot of performance
improvment, especially when the size of the allocation is as small as
the page size, which had the largest search space previously.
2014-06-26 17:06:10 +09:00
Trustin Lee
41d44a8161 Remove 'get' prefix from all HTTP/SPDY messages
Motivation:

Persuit for the consistency in method naming

Modifications:

- Remove the 'get' prefix from all HTTP/SPDY message classes
- Fix some inspector warnings

Result:

Consistency
2014-06-24 18:03:33 +09:00
Norman Maurer
12a3e23e47 MessageToByteEncoder always starts with ByteBuf that use initalCapacity == 0
Motivation:

MessageToByteEncoder always starts with ByteBuf that use initalCapacity == 0 when preferDirect is used. This is really wasteful in terms of performance as every first write into the buffer will cause an expand of the buffer itself.

Modifications:

 - Change ByteBufAllocator.ioBuffer() use the same default initialCapacity as heapBuffer() and directBuffer()
 - Add new allocateBuffer method to MessageToByteEncoder that allow the user to do some smarter allocation based on the message that will be encoded.

Result:

Less expanding of buffer and more flexibilty when allocate the buffer for encoding.
2014-06-24 13:55:21 +09:00
Trustin Lee
37b07a04d4 Revert "Improve the allocation algorithm in PoolChunk"
This reverts commit 36305d7dce, which
seems to cause an assertion failure on our CI machine.
2014-06-21 19:19:35 +09:00
Pavan Kumar
6bd8c5d4d0 Improve the allocation algorithm in PoolChunk
Motivation:

Depth-first search is not always efficient for buddy allocation.

Modification:

Employ a new faster search algorithm with different memoryMap layout.

Result:

With thread-local cache disabled, we see a lot of performance
improvment, especially when the size of the allocation is as small as
the page size, which had the largest search space previously:

-- master head --
Benchmark                (size) Mode    Score  Error Units
pooledDirectAllocAndFree  8192 thrpt  215.392  1.565 ops/ms
pooledDirectAllocAndFree 16384 thrpt  594.625  2.154 ops/ms
pooledDirectAllocAndFree 65536 thrpt 1221.520 18.965 ops/ms
pooledHeapAllocAndFree    8192 thrpt  217.175  1.653 ops/ms
pooledHeapAllocAndFree   16384 thrpt  587.250 14.827 ops/ms
pooledHeapAllocAndFree   65536 thrpt 1217.023 44.963 ops/ms

-- changes --
Benchmark                (size) Mode    Score  Error Units
pooledDirectAllocAndFree  8192 thrpt 3656.744 94.093 ops/ms
pooledDirectAllocAndFree 16384 thrpt 4087.152 22.921 ops/ms
pooledDirectAllocAndFree 65536 thrpt 4058.814 29.276 ops/ms
pooledHeapAllocAndFree    8192 thrpt 3640.355 44.418 ops/ms
pooledHeapAllocAndFree   16384 thrpt 4030.206 24.365 ops/ms
pooledHeapAllocAndFree   65536 thrpt 4103.991 70.991 ops/ms
2014-06-21 13:20:25 +09:00
Norman Maurer
f05510063e Remove System.out.println(...) debug messages 2014-06-20 19:42:38 +02:00
Norman Maurer
371f8066d2 [#2580] [#2587] Fix buffer corruption regression when ByteBuf.order(LITTLE_ENDIAN) is used
Motivation:

To improve the speed of ByteBuf with order LITTLE_ENDIAN and where the native order is also LITTLE_ENDIAN (intel) we introduces a new special SwappedByteBuf before in commit 4ad3984c8b. Unfortunally the commit has a flaw which does not handle correctly the case when a ByteBuf expands. This was caused because the memoryAddress was cached and never changed again even if the underlying buffer expanded. This can lead to corrupt data or even to SEGFAULT the JVM if you are lucky enough.

Modification:

Always lookup the actual memoryAddress of the wrapped ByteBuf.

Result:

No more data-corruption for ByteBuf with order LITTLE_ENDIAN and no JVM crashes.
2014-06-20 18:24:44 +02:00
Trustin Lee
085a61a310 Refactor FastThreadLocal to simplify TLV management
Motivation:

When Netty runs in a managed environment such as web application server,
Netty needs to provide an explicit way to remove the thread-local
variables it created to prevent class loader leaks.

FastThreadLocal uses different execution paths for storing a
thread-local variable depending on the type of the current thread.
It increases the complexity of thread-local removal.

Modifications:

- Moved FastThreadLocal and FastThreadLocalThread out of the internal
  package so that a user can use it.
- FastThreadLocal now keeps track of all thread local variables it has
  initialized, and calling FastThreadLocal.removeAll() will remove all
  thread-local variables of the caller thread.
- Added FastThreadLocal.size() for diagnostics and tests
- Introduce InternalThreadLocalMap which is a mixture of hard-wired
  thread local variable fields and extensible indexed variables
- FastThreadLocal now uses InternalThreadLocalMap to implement a
  thread-local variable.
- Added ThreadDeathWatcher.unwatch() so that PooledByteBufAllocator
  tells it to stop watching when its thread-local cache has been freed
  by FastThreadLocal.removeAll().
- Added FastThreadLocalTest to ensure that removeAll() works
- Added microbenchmark for FastThreadLocal and JDK ThreadLocal
- Upgraded to JMH 0.9

Result:

- A user can remove all thread-local variables Netty created, as long as
  he or she did not exit from the current thread. (Note that there's no
  way to remove a thread-local variable from outside of the thread.)
- FastThreadLocal exposes more useful operations such as isSet() because
  we always implement a thread local variable via InternalThreadLocalMap
  instead of falling back to JDK ThreadLocal.
- FastThreadLocalBenchmark shows that this change improves the
  performance of FastThreadLocal even more.
2014-06-19 21:13:55 +09:00
Norman Maurer
ad86ec798d Move calculateNewCapacity(...) to ByteBufAllocator
Motivation:

Currently we have the algorithm of calculate the new capacity of a ByteBuf implemented in AbstractByteBuf. The problem with this is that it is impossible for a user to change it if it not fits well it's use-case. We should better move it to ByteBufAllocator and so let the user implement it's own by either write his/her own ByteBufAllocator or just override the default implementation in one of our provided ByteBufAllocators.

Modifications:

Move calculateNewCapacity(...) to ByteBufAllocator and move the implementation (which was part of AbstractByteBuf) to AbstractByteBufAllocator.

Result:

The user can now override the default calculation algorithm when needed.
2014-06-17 09:35:45 +02:00
Norman Maurer
066f95d047 [#2573] UnpooledUnsafeDirectByteBuf.setBytes(int,ByteBuf,int,int) fails to use fast-path when src has array
Motivation:

UnpooledUnsafeDirectByteBuf.setBytes(int,ByteBuf,int,int) fails to use fast-path when src uses an array as backing storage. This is because the if else uses the wrong ByteBuf for its check.

Modifications:

- Use correct ByteBuf when check for array as backing storage
- Also eliminate unecessary check in UnpooledDirectByteBuf which always fails anyway

Result:

Faster setBytes(...) when src ByteBuf is backed by an array.

No more IndexOutOfBoundsException or data-corruption.
2014-06-16 11:11:41 +02:00
belliottsmith
2a2a21ec59 Introduce FastThreadLocal which uses an EnumMap and a predefined fixed set of possible thread locals
Motivation:
Provide a faster ThreadLocal implementation

Modification:
Add a "FastThreadLocal" which uses an EnumMap and a predefined fixed set of possible thread locals (all of the static instances created by netty) that is around 10-20% faster than standard ThreadLocal in my benchmarks (and can be seen having an effect in the direct PooledByteBufAllocator benchmark that uses the DEFAULT ByteBufAllocator which uses this FastThreadLocal, as opposed to normal instantiations that do not, and in the new RecyclableArrayList benchmark);

Result:
Improved performance
2014-06-13 10:56:18 +02:00
Norman Maurer
61dbc353ca [#2436] Unsafe*ByteBuf implementation should only invert bytes if ByteOrder differ from native ByteOrder
Motivation:
Our Unsafe*ByteBuf implementation always invert bytes when the native ByteOrder is LITTLE_ENDIAN (this is true on intel), even when the user calls order(ByteOrder.LITTLE_ENDIAN). This is not optimal for performance reasons as the user should be able to set the ByteOrder to LITTLE_ENDIAN and so write bytes without the extra inverting.

Modification:
- Introduce a new special SwappedByteBuf (called UnsafeDirectSwappedByteBuf) that is used by all the Unsafe*ByteBuf implementation and allows to write without inverting the bytes.
- Add benchmark
- Upgrade jmh to 0.8

Result:
The user is be able to get the max performance even on servers that have ByteOrder.LITTLE_ENDIAN as their native ByteOrder.
2014-06-05 10:59:22 +02:00
Trustin Lee
7d9374a582 Use Java 5 foreach for arrays for brevity at no cost 2014-06-02 18:25:25 +09:00
Trustin Lee
af4c30fa56 Remove the deprecated constructor 2014-06-02 18:24:19 +09:00
Trustin Lee
e79ca269b8 Introduce ThreadDeathWatcher
Motivation:

PooledByteBufAllocator's thread local cache and
ReferenceCountUtil.releaseLater() are in need of a way to run an
arbitrary logic when a certain thread is terminated.

Modifications:

- Add ThreadDeathWatcher, which spawns a low-priority daemon thread
  that watches a list of threads periodically (every second) and
  invokes the specified tasks when the associated threads are not alive
  anymore
  - Start-stop logic based on CAS operation proposed by @tea-dragon
- Add debug-level log messages to see if ThreadDeathWatcher works

Result:

- Fixes #2519 because we don't use GlobalEventExecutor anymore
- Cleaner code
2014-06-02 18:23:23 +09:00
Trustin Lee
ea3dac0753 Do not use a pseudo random for tree traversal
Motivation:

If we make allocateRun/SubpageSimple() always try the left node first and make allocateRun/Subpage() always tries the right node first,  it is more likely that allocateRun/Subpage() will find a node with ST_UNUSED sooner.

Modifications:

- Make allocateRunSimple() and allocateSubpageSimple() always try the left node first.
- Make allocateRun() and allocateSubpage() always try the right node first.
- Remove randome

Result:

We get the same performance without using random numbers.
2014-05-30 11:24:16 +09:00
Trustin Lee
e5ed69241b Optimize PooledByteBufAllocator
Motivation:

We still have a room for improvement in PoolChunk.allocateRun() and
Subpage.allocate().

Modifications:

- Unroll the recursion in PoolChunk.allocateRun()
- Subpage.allocate() makes use of the 'nextAvail' value set by previous
  free().

Result:

- PoolChunk.allocateRun() optimization yields 10%+ improvements in
  allocation throughput for non-subpage allocations.
- Subpage.allocate() optimization makes the subpage allocations for
  tiny buffers as fast as non-tiny buffers even when the pageSize is
  huge (e.g. 1048576) because it doesn't need to perform a linear search
  in most cases.
2014-05-30 10:51:21 +09:00
Jake Luciani
d547b5d51d Fix capacity check bug affecting offheap buffers 2014-05-13 07:25:15 +02:00
Trustin Lee
db3709e652 Synchronized between 4.1 and master
Motivation:

4 and 5 were diverged long time ago and we recently reverted some of the
early commits in master.  We must make sure 4.1 and master are not very
different now.

Modification:

Fix found differences

Result:

4.1 and master got closer.
2014-04-25 00:38:02 +09:00
ian
15d11289b0 Fix error that causes (up to) double memory usage
Motivation:

PoolArena's 'normalizeCapacity' function was micro-optimized some
time ago to remove a while loop. However, there was a change of
behavior in the function as a result. Capacities passed into it
that are already powers of 2 (and >= 512) are doubled in size. So
if I ask for a buffer with a capacity of 1024, I will get back one
that actually uses 2048 bytes (stored in maxLength).

Aligning to powers of two for book keeping ease is reasonable,
and if someone tries to expand a buffer, you might as well use some
of the previously wasted space. However, since this distinction
between 'easily expanded' and 'costly to expand' space is not
supported at all by the APIs, I cannot imagine this change to
doubling is desirable or intentional.

This is especially costly when using composite buffers. They
frequently allocate components with a capacity that is a power of
2, and they never attempt to expand components themselves. The end
result is that heavy use of pool-backed composite buffers wastes
almost half of the memory pool (the smaller / initial components are
<512 and so are not affected by the off-by-one bug).

Modifications:

Although I find it difficult to believe that such an optimization
is really helpful, I left it in and fixed the off-by-one issue by
decrementing the value at the start.

I also added a simple test to both attempt to verify that the
decrement fixes the issue without introducing any other change, and
to make it easy for a reviewer to test the existing behavior. PoolArena
does not seem to have much testing or testability support though so
the test is kind of a hack and will break for unrelated changes. I
suggest either removing it or factoring out the single non-static
portion of normalizeCapacity so that the fragile dummy PoolArena is
not required.

Result:

Pooled allocators will allocate less resources to the highly
inefficient and undocumented buffer section between length and
maxLength.

Composite buffers of non-trivial size that are backed by pooled
allocators will use about half as much memory.
2014-04-15 07:03:13 +02:00
Norman Maurer
ceffa82d0d [#2370] Periodically check for not alive Threads and free up their ThreadPoolCache
Motivation:
At the moment we create new ThreadPoolCache whenever a Thread tries either allocate or release something on the PooledByteBufAllocator. When something is released we put it then in its ThreadPoolCache. The problem is we never check if a Thread is not alive anymore and so we may end up with memory that is never freed again if a user create many short living Threads that use the PooledByteBufAllocator.

Modifications:
Periodically check if the Thread is still alive that has a ThreadPoolCache assinged and if not free it.

Result:
Memory is freed up correctly even for short living Threads.
2014-04-09 11:45:11 +02:00
Norman Maurer
8429ecfcc4 Implement Thread caches for pooled buffers to minimize conditions. This fixes [#2264] and [#808].
Motivation:
Remove the synchronization bottleneck in PoolArena and so speed up things

Modifications:

This implementation uses kind of the same technics as outlined in the jemalloc paper and jemalloc
blogpost https://www.facebook.com/notes/facebook-engineering/scalable-memory-allocation-using-jemalloc/480222803919.

At the moment we only cache for "known" Threads (that powers EventExecutors) and not for others to keep the overhead
minimal when need to free up unused buffers in the cache and free up cached buffers once the Thread completes. Here
we use multi-level caches for tiny, small and normal allocations. Huge allocations are not cached at all to keep the
memory usage at a sane level. All the different cache configurations can be adjusted via system properties or the constructor
directly where it makes sense.

Result:
Less conditions as most allocations can be served by the cache itself
2014-03-20 09:30:57 -07:00
Trustin Lee
19422972e3 Fix and simplify freeing a direct buffer / Fix Android support
Motivation:

6e8ba291cf introduced a regression in Android because Android does not have sun.nio.ch.DirectBuffer (see #2330.)  I also found PlatformDependent0.freeDirectBuffer() and freeDirectBufferUnsafe() are pretty much same after the commit and the unsafe version should be removed.

Modifications:

- Do not use the pooled allocator in Android because it's too resource hungry for Androids.
- Merge PlatformDependent0.freeDirectBuffer() and freeDirectBufferUnsafe() into one method.
- Make the Unsafe unavailable when sun.nio.ch.DirectBuffer is unavailable.  We could keep the Unsafe available and handle the sun.nio.ch.DirectBuffer case separately, but I don't want to complicate our code just because of that.  All supported JDK versions have sun.nio.ch.DirectBuffer if the Unsafe is available.

Result:

Simpler code. Fixes Android support (#2330)
2014-03-20 11:11:07 +09:00
Jakob Buchgraber
1bce46dbb3 Bit tricks to check for and calculate power of two.
Motivation:
I was studying the code and thought this was simpler and easier to
understand.

Modifications:
Replaced the for loop and if conditions, with a simple implementation.

Result:
Code is easier to understand.
2014-03-18 15:59:34 +09:00
Bourne, Geoff
1334d34e9d Fix limit computation of NIO ByteBuffers obtained via ReadOnlyByteBufferBuf.nioBuffer
Motivation:

When starting with a read-only NIO buffer, wrapping it in a ByteBuf,
and then later retrieving a re-wrapped NIO buffer the limit was getting
too short.

Modifications:

Changed ReadOnlyByteBufferBuf.nioBuffer(int,int) to compute the
limit in the same manner as the internalNioBuffer method.

Result:

Round-trip conversion from NIO to ByteBuf to NIO will work reliably.
2014-03-14 08:10:18 +01:00
Trustin Lee
fbd4385506 Determine the default allocator from system property
- Add ByteBufAllocator.DEFAULT
- The default allocator is now 'pooled'
2014-02-14 13:04:12 -08:00
Trustin Lee
dea5c688fd Fix checkstyle 2014-02-13 18:51:32 -08:00
Trustin Lee
ac70dc4546 Update the version to 4.1.0.Alpha1-SNAPSHOT 2014-02-13 18:32:26 -08:00
Trustin Lee
51349352e2 Fix a bug that CompositeByteBuf.touch() does nothing 2014-02-13 18:24:36 -08:00
Trustin Lee
8837afddf8 Enable a user specify an arbitrary information with ReferenceCounted.touch()
- Related: #2163
- Add ResourceLeakHint to allow a user to provide a meaningful information about the leak when touching it
- DefaultChannelHandlerContext now implements ResourceLeakHint to tell where the message is going.
- Cleaner resource leak report by excluding noisy stack trace elements
2014-02-13 18:16:25 -08:00
Trustin Lee
45e70d9935 Add ReferenceCounted.touch() / Add missing retain() overrides
- Fixes #2163
- Inspector warnings
2014-02-13 18:10:11 -08:00
Trustin Lee
2b84314fdd Add Recycler.Handle.recycle() so that it's possible to recycle an object without an explicit reference to Recycler 2014-02-13 17:24:37 -08:00
Trustin Lee
86c4166c24 Fixed various buffer leaks in FixedCompositeByteBufTest 2014-02-13 17:03:13 -08:00
Trustin Lee
26fc84529f Also record retain() and release() 2014-02-13 16:58:45 -08:00
Norman Maurer
2351b8ddd9 Add FixedCompositeByteBuf which can be used to write an array of ByteBuf in an efficient way.
This implementation does not produce as much GC pressure as CompositeByteBuf and so is prefered,
for writing an array of ByteBufs. Be aware that FixedCompositeByteBuf is readonly.

When using this in a project that make heavy use of CompositeByteBuf for writes we was able to cut
down allocation to a half.
2014-02-13 16:52:31 -08:00
Trustin Lee
40003ed250 Resurrect Channel.id() with global uniqueness
- Fixes #1810
- Add a new interface ChannelId and its default implementation which generates globally unique channel ID.
- Replace AbstractChannel.hashCode with ChannelId.hashCode() and ChannelId.shortValue()
- Add variants of ByteBuf.hexDump() which accept byte[] instead of ByteBuf.
2014-02-13 15:53:07 -08:00
Trustin Lee
ef4bc99849 Remove unnecessary parenthesis 2014-02-13 15:52:06 -08:00
Norman Maurer
f23d68b42f [#2187] Always do a volatile read on the refCnt 2014-02-07 09:23:16 +01:00
Norman Maurer
9bee78f91c Provide an optimized AtomicIntegerFieldUpdater, AtomicLongFieldUpdater and AtomicReferenceFieldUpdater 2014-02-06 20:08:45 +01:00
Norman Maurer
d67184b488 [maven-release-plugin] prepare for next development iteration 2014-01-21 08:18:32 +01:00
Norman Maurer
287515210d [maven-release-plugin] prepare release netty-4.0.15.Final 2014-01-21 08:18:26 +01:00
Trustin Lee
e83d2e0b4e [maven-release-plugin] prepare for next development iteration 2013-12-22 21:57:48 +09:00
Trustin Lee
cdb700c7a4 [maven-release-plugin] prepare release netty-4.0.14.Final 2013-12-22 21:57:40 +09:00
Trustin Lee
0b7aedb13b [maven-release-plugin] rollback the release of netty-4.0.14.Final 2013-12-22 21:53:24 +09:00
Trustin Lee
4bf6ec7171 [maven-release-plugin] prepare release netty-4.0.14.Final 2013-12-22 21:52:56 +09:00
Trustin Lee
9c1a49c58e [maven-release-plugin] rollback the release of netty-4.0.14.Final 2013-12-22 21:47:35 +09:00
Trustin Lee
008a049bf4 [maven-release-plugin] prepare for next development iteration 2013-12-22 21:43:55 +09:00
Trustin Lee
f6cb9088c6 [maven-release-plugin] prepare release netty-4.0.14.Final 2013-12-22 21:43:45 +09:00
Norman Maurer
b3d8c81557 Fix all leaks reported during tests
- One notable leak is from WebSocketFrameAggregator
- All other leaks are from tests
2013-12-07 00:44:56 +09:00
Trustin Lee
2102cb062b Fix false-positive leaks
- All derived buffers and swapped buffers of a leak-aware buffer must be wrapped again with the leak-aware buffer
2013-12-06 21:32:56 +09:00
Trustin Lee
e506581eb1 Add ReferenceCountUtil.releaseLater() to make writing tests easy with ReferenceCounteds 2013-12-06 15:13:00 +09:00
Trustin Lee
128c4b96b5 Checkstyle 2013-12-06 13:54:36 +09:00
Trustin Lee
5d39b1fc3d Also record retain() and release() 2013-12-06 13:45:24 +09:00
Trustin Lee
e88172495a Ensure backward compatibility
.. by resurrecting the removed methods and system properties.
2013-12-05 01:02:38 +09:00
Trustin Lee
65b522a2a7 Better buffer leak reporting
- Remove the reference to ResourceLeak from the buffer implementations
  and use wrappers instead:
  - SimpleLeakAwareByteBuf and AdvancedLeakAwareByteBuf
  - It is now allocator's responsibility to create a leak-aware buffer.
  - Added AbstractByteBufAllocator.toLeakAwareBuffer() for easier
    implementation
- Add WrappedByteBuf to reduce duplication between *LeakAwareByteBuf and
  UnreleasableByteBuf
- Raise the level of leak reports to ERROR - because it will break the
  app eventually
- Replace enabled/disabled property with the leak detection level
  - Only print stack trace when level is ADVANCED or above to avoid user
    confusion
- Add the 'leak' build profile, which enables highly detailed leak
  reporting during the build
- Remove ResourceLeakException which is unsed anymore
2013-12-05 00:51:39 +09:00
Norman Maurer
053c512f6d Fix checkstyle 2013-12-02 08:23:57 +01:00
Norman Maurer
14600167d6 [#2021] No need to synchronize for unpooled chunks 2013-12-02 08:02:48 +01:00
Norman Maurer
17f5865e38 [maven-release-plugin] prepare for next development iteration 2013-11-29 19:31:01 +01:00
Norman Maurer
ead617fdcc [maven-release-plugin] prepare release netty-4.0.14.Beta1 2013-11-29 19:30:55 +01:00
Norman Maurer
6cf2748dbb [maven-release-plugin] prepare for next development iteration 2013-11-28 15:04:51 +01:00
Norman Maurer
5fe7596f49 [maven-release-plugin] prepare release netty-4.0.13.Final 2013-11-28 15:04:46 +01:00
Trustin Lee
407f0a36f5 Simplify bundle generation / Add io.netty.versions.properties to all JARs
- Fixes #2003 properly
- Instead of using 'bundle' packaging, use 'jar' packaging.  This is
  more robust because some strict build tools fail to retrieve the
  artifacts from a Maven repository unless their packaging is not 'jar'.
- All artifacts now contain META-INF/io.netty.version.properties, which
  provides the detailed information about the build and repository.
- Removed OSGi testsuite temporarily because it gives false errors
  during split package test and examination.
- Add io.netty.util.Version for easy retrieval of version information
2013-11-26 22:01:46 +09:00
Norman Maurer
7231be592a Also allow to override how direct ByteBuffers are freed 2013-11-12 12:40:41 +01:00
Norman Maurer
e83fb821d5 Allow to override how wrapped direct ByteBuffer are allocated to make it easier to extend 2013-11-12 12:13:38 +01:00
Norman Maurer
b00f8c6390 [#1976] Fix IndexOutOfBoundsException when calling CompositeByteBuf.discardReadComponents() 2013-11-09 20:13:24 +01:00
Alex Petrov
e4f391f626 Improve docstrings for and of 2013-11-08 12:15:41 +01:00
Trustin Lee
ba3bc0c020 Simpler toString() for ByteBufAllocators 2013-11-08 17:54:34 +09:00
Norman Maurer
db78581bbb [maven-release-plugin] prepare for next development iteration 2013-11-07 18:11:45 +01:00
Norman Maurer
2386777af8 [maven-release-plugin] prepare release netty-4.0.12.Final 2013-11-07 18:11:38 +01:00
Norman Maurer
77b4ec7e1b [#1800] [#1802] Correctly expand capacity of ByteBuf while preserve content 2013-11-04 15:18:21 +01:00
Trustin Lee
54db9ec725 Use StringUtil.simpleClassName(..) instead of Class.getSimpleName() where necessary
- Class.getSimpleName() doesn't render anonymous classes very well
- + some minor cleanup
2013-11-04 19:46:15 +09:00
Norman Maurer
4ce49a6195 [#1943] Unpooled.copiedBuffer(ByteBuf pooled) should always return unpooled ByteBuf 2013-10-22 20:20:38 +02:00
Norman Maurer
ceab146b54 [maven-release-plugin] prepare for next development iteration 2013-10-21 07:43:42 +02:00
Norman Maurer
27a89d6032 [maven-release-plugin] prepare release netty-4.0.11.Final 2013-10-21 07:41:49 +02:00
Norman Maurer
68b616728a [#1925] Only expose sub-region of ByteBuf on nioBuffer(...) 2013-10-16 10:34:33 +02:00
Norman Maurer
d946659520 [#1906] Use a ByteBuf allocator from the ByteBufAllocator when encode Strings 2013-10-09 21:18:08 +02:00
Norman Maurer
1c73be21fc Remove redundant index check 2013-10-08 07:21:01 +02:00
Bill Gallagher
8f612660b2 disable debugging output during test 2013-10-04 06:45:26 +02:00
Norman Maurer
d7da19f745 [maven-release-plugin] prepare for next development iteration 2013-10-02 15:48:52 +02:00
Norman Maurer
d35768ae11 [maven-release-plugin] prepare release netty-4.0.10.Final 2013-10-02 15:48:45 +02:00
Norman Maurer
ee192f0321 [#1880] Use ByteBufAllocator when read bytes into new chunks 2013-10-01 10:10:43 +02:00
Norman Maurer
6d09e57be7 [#1875] Correctly check the readerIndex when try to read a byte from AbstractByteBuf 2013-09-30 14:47:49 +02:00
Norman Maurer
b4fa8af079 Cache underlying ByteBuffers and count in ChannelOutboundBuffer.Entry to reduce object creation and so GC pressure
Beside this it also helps to reduce CPU usage as nioBufferCount() is quite expensive when used on CompositeByteBuf which are
nested and contains a lot of components
2013-09-26 20:37:39 +02:00
Norman Maurer
2b9a07cac9 CompositeByteBuf.isDirect() should return true if its only backed by direct buffers 2013-09-26 20:37:31 +02:00
Norman Maurer
a74149e984 [#1865] Only use internalNioBuffer when one of the read* or write* methods are used. This is neccessary to prevent races as those can happen when a slice or duplicate is shared between different Channels
that are not assigned to the same EventLoop. In general get* operations should always be safe to be used from different Threads.

This aslo include unit tests that show the issue
2013-09-25 17:27:26 +02:00
Norman Maurer
910ed31a1b [#1851] EmptyByteBuf.isWritable(..) and isReadable(...) should not throw IndexOutOfBoundsException 2013-09-21 20:40:22 +02:00
Norman Maurer
23baef8fb4 [#1853] Optimize gathering writes for CompositeByteBuf that are only backed by one ByteBuffer 2013-09-19 07:29:58 +02:00
Norman Maurer
c0bbde48b7 [#1852] Fix bug in UnpooledDirectByteBuf.nioBuffer(...) implementation 2013-09-18 20:47:57 +02:00
Greg Soltis
f1d4f813ed Fix nioBuffer implementation for CompositeByteBuf 2013-09-16 06:41:08 +02:00
Norman Maurer
451e91d142 [#1821] Fix IndexOutOfBoundsException which was thrown if the last component was removed but other components was left 2013-09-09 20:29:30 +02:00
Norman Maurer
ffab456aca Bump up version to reflect correct one 2013-09-09 11:20:12 +02:00
Norman Maurer
363531caf9 [maven-release-plugin] rollback the release of netty-4.0.9.Final 2013-09-06 09:18:34 +02:00
Norman Maurer
9d53573ee8 [maven-release-plugin] prepare for next development iteration 2013-09-06 09:17:15 +02:00
Norman Maurer
25c226a835 Make sure only direct ByteBuffer are passed to the underlying jdk Channel.
This is needed because of otherwise the JDK itself will do an extra ByteBuffer copy with it's own pool implementation. Even worth it will be done
multiple times if the ByteBuffer is always only partial written. With this change the copy is done inside of netty using it's own allocator and
only be done one time in all cases.
2013-09-02 20:17:53 +02:00
Norman Maurer
5416f2315e [#1797] No use internalNioBuffer() in derived buffers as it is not meant for concurrent access 2013-09-02 14:15:19 +02:00
Norman Maurer
0007fb81ef Add tests to try to track down some buffer issues 2013-09-02 13:50:47 +02:00
Norman Maurer
795182843d Remove legancy code which we not need anymore as we use gathering writes anyway everywhere 2013-09-01 11:00:58 +02:00
Norman Maurer
5ddd7cee90 [#1797] Throw IllegalArgumentException if AbstractByteBuf.skipBytes(...) is used with a negative value 2013-08-29 11:14:36 +02:00
Norman Maurer
2e39b25cd4 [maven-release-plugin] prepare for next development iteration 2013-08-26 12:01:03 +02:00
Norman Maurer
b67659a866 [maven-release-plugin] prepare release netty-4.0.8.Final 2013-08-26 12:00:54 +02:00
Trustin Lee
20894bc99e Fix a bug in internalNioBuffer() implementations of derived buffers
- A user can create multiple duplicates of a buffer and access their internal NIO buffers. (e.g. write multiple duplicates to multiple channels assigned to different event loop.)  Because the derived buffers' internalNioBuffer() simply delegates the call to the original buffer, all derived buffers and the original buffer's internalNioBuffer() will return the same buffer, which will lead to a race condition.
- Fixes #1739
2013-08-20 14:28:50 +09:00
bgallagher
9f88552f12 remove some dead code 2013-08-10 20:46:48 +02:00
Norman Maurer
1d3560e389 [maven-release-plugin] prepare for next development iteration 2013-08-08 13:53:28 +02:00
Norman Maurer
8e97e6c461 [maven-release-plugin] prepare release netty-4.0.7.Final 2013-08-08 13:53:19 +02:00
Norman Maurer
194b64cff1 [#1708] Correctly set the writerIndex in ReadOnlyByteBufferBuf if it is constructed with a buffer which has non zero position 2013-08-08 06:54:32 +02:00
Norman Maurer
ea1dca8105 [#1704] Make sure SwappedByteBuf.readSlice(..) returns ByteBuf with correct ByteOrder 2013-08-06 12:22:22 +02:00
Norman Maurer
3f2000fa3a [maven-release-plugin] prepare for next development iteration 2013-08-01 10:59:55 +02:00
Norman Maurer
3f70d5caa4 [maven-release-plugin] prepare release netty-4.0.6.Final 2013-08-01 10:59:46 +02:00
Norman Maurer
e3410680de [maven-release-plugin] prepare for next development iteration 2013-07-31 20:08:14 +02:00
Norman Maurer
0e124583d6 [maven-release-plugin] prepare release netty-4.0.5.Final 2013-07-31 20:08:05 +02:00
Norman Maurer
8a673db92b [#1644] Fixed IndexOutOfBoundException when calling copy() on a empty CompositeByteBuf 2013-07-24 07:35:51 +02:00
Norman Maurer
0bc7d3f5d1 [maven-release-plugin] prepare for next development iteration 2013-07-23 10:04:23 +02:00
Norman Maurer
ca00182797 [maven-release-plugin] prepare release netty-4.0.4.Final 2013-07-23 10:04:14 +02:00
Trustin Lee
764741c5ce Change the contract of ResourceLeakDetector.open() so that unsampled resources are recycled
- This also fixes the problem introduced while trying to implement #1612 (Allow to disable resource leak detection).
2013-07-23 14:06:58 +09:00
Norman Maurer
35802207e1 Fix compile error 2013-07-23 06:42:41 +02:00
kerr
ada07cb9e0 Fix types in javadocs 2013-07-22 19:14:36 +02:00
Norman Maurer
f478fcd3b0 [#1628] Fix bug in ReadOnlyByteByteBufferBuf where get operations threw ReadOnlyBufferException 2013-07-22 07:12:05 +02:00
Norman Maurer
feb8d101bd [#1626] Use static fields for default values 2013-07-22 06:47:29 +02:00
Shawn Silverman
674f4bce51 netty-1597: Rewrite ByteBufInputStream.readLine() to avoid IndexOutOfBoundsException and to behave more correctly for lines ending in '\r'. 2013-07-20 08:05:54 +02:00
Trustin Lee
b130ee6a6c [maven-release-plugin] prepare for next development iteration 2013-07-18 11:17:42 +09:00
Trustin Lee
10d395e829 [maven-release-plugin] prepare release netty-4.0.3.Final 2013-07-18 11:17:31 +09:00
Trustin Lee
f0a3f849f7 Fix a bug in AbstractByteBuf.writeZero() where the capacity is not auto-expanded 2013-07-18 09:55:02 +09:00
Norman Maurer
fc7c950b08 [maven-release-plugin] prepare for next development iteration 2013-07-17 15:58:36 +02:00
Norman Maurer
bbbf72359e [maven-release-plugin] prepare release netty-4.0.2.Final 2013-07-17 15:58:28 +02:00
Trustin Lee
57eb531eb8 [maven-release-plugin] prepare for next development iteration 2013-07-16 17:16:10 +09:00
Trustin Lee
76cefcc421 [maven-release-plugin] prepare release netty-4.0.1.Final 2013-07-16 17:15:54 +09:00
Norman Maurer
5297eba280 [maven-release-plugin] prepare for next development iteration 2013-07-15 15:48:15 +02:00
Norman Maurer
c5d8af446a [maven-release-plugin] prepare release netty-4.0.0.Final 2013-07-15 15:48:05 +02:00
Trustin Lee
246a3ecdcb [maven-release-plugin] prepare for next development iteration 2013-07-15 20:58:33 +09:00
Trustin Lee
e8fd209115 [maven-release-plugin] prepare release netty-4.0.0.Final 2013-07-15 20:58:21 +09:00
Norman Maurer
df5daadd0f Remove unused import 2013-07-15 10:09:17 +02:00
Norman Maurer
7254a5c2c6 Just some tiny javadocs optimizations 2013-07-14 16:02:03 +02:00
Trustin Lee
65c2a6ed46 Make ByteBuf an abstract class rather than an interface
- 5% improvement in throughput (HelloWorldServer example)
- Made CompositeByteBuf a concrete class (renamed from DefaultCompositeByteBuf) because there's no multiple inheritance in Java

Fixes #1536
2013-07-08 14:59:52 +09:00
Norman Maurer
086ae3536c [#1533] Introduce ByteBufHolder.duplicate() and make use of it in DefaultChannelGroup.write(...) 2013-07-06 21:17:51 +02:00
Norman Maurer
7dda4b9ce4 [#1532] Remove @deprecated ByteBufIndexFinder and all methods that take it as argument 2013-07-06 20:14:53 +02:00
Trustin Lee
dfc05a6ed7 Fix documentation error in ByteBuf
- Fixes #1531
- Thanks to @daschl
2013-07-05 17:03:34 +09:00
Trustin Lee
0b9235f072 Simplify ByteBufProcessor and MessageListProcessor and Add internal component accessors to CompositeByteBuf
- Fixes #1528

It's not really easy to provide a general-purpose abstraction for fast-yet-safe iteration. Instead of making forEachByte() less optimal, let's make it do what it does really well, and allow a user to implement potentially unsafe-yet-fast loop using unsafe operations.
2013-07-05 14:00:46 +09:00
Norman Maurer
7ec12d327f Remove deprecated ByteBufUtil.release(..) and ByteBufUtil.retain(..) methods and its usage. Also fix a problem where an object would have been released two times.
* The problem with the release(..) calls here was that it would have called release on an unsupported message and then throw an exception. This exception will trigger ChannelOutboundBuffer.fail(..), which will also try to release the message again.
* Also use the same exception type for unsupported messages as in other channel impls.
2013-07-03 10:00:13 +02:00
Norman Maurer
ec5e793a2f [maven-release-plugin] prepare for next development iteration 2013-07-02 11:41:18 +02:00
Norman Maurer
ca73eaef0d [maven-release-plugin] prepare release netty-4.0.0.CR9 2013-07-02 11:41:09 +02:00
Norman Maurer
830c559405 [maven-release-plugin] rollback the release of netty-4.0.0.CR9 2013-07-02 11:34:29 +02:00
Norman Maurer
66a16b133c [maven-release-plugin] prepare release netty-4.0.0.CR9 2013-07-02 10:45:12 +02:00
Trustin Lee
7e3a01cc51 [maven-release-plugin] prepare for next development iteration 2013-07-02 10:26:48 +09:00
Trustin Lee
149db34c19 [maven-release-plugin] prepare release netty-4.0.0.CR8 2013-07-02 10:26:32 +09:00
Trustin Lee
4b11aff08f Less confusing log messages for system properties
- Fixes #1502
2013-07-02 09:23:29 +09:00
Norman Maurer
5d88c423df [#1500] Remove @deprecated methods 2013-07-01 08:53:02 +02:00
Trustin Lee
613547b0b9 [maven-release-plugin] prepare for next development iteration 2013-06-28 22:15:33 +09:00
Trustin Lee
a6abd2feb2 [maven-release-plugin] prepare release netty-4.0.0.CR7 2013-06-28 22:15:20 +09:00
Trustin Lee
0dcf352f4c Vastly simplified ByteBufProcessor and MessageListProcessor
- Related: #1378
- They now accept only one argument.
- A user who wants to use a buffer for more complex use cases, he or she can always access the buffer directly via memoryAddress() and array()
2013-06-28 20:29:00 +09:00
Trustin Lee
52691488ee Update Javadoc of ByteBufProcessor and MessageListProcessor
- in response to @shacharo's suggestion
2013-06-27 19:01:01 +09:00
Trustin Lee
ac39cad5ff Split ByteBuf.forEachByte() into forEachByte() and forEachByteDesc()
- Related: #1378
- As suggested by @liqweed
2013-06-27 18:48:09 +09:00
Trustin Lee
9804741fb3 Fix test failure in SlicedByteBuf / Add tests for built-in ByteBufProcessor impls
- Related: #1378
2013-06-27 17:49:46 +09:00
Trustin Lee
98531313de Optimize derived buffers' forEachByte(...) implementation
- Related: #1378
2013-06-27 17:36:22 +09:00
Trustin Lee
b5bb36c087 Use (fromIndex, toIndex) instead of (index, length) for ByteBuf.forEachByte(...)
- Related: #1378
2013-06-27 17:30:19 +09:00
Trustin Lee
792edf631c Deprecate ByteBufIndexFinder
- Prefer ByteBufProcessor
- Related: #1378
2013-06-27 14:26:58 +09:00
Trustin Lee
4dd9b6ef2e Add ByteBufProcessor and ByteBuf.forEach(...)
- Fixes #1378
- Needs to provide optimized forEach implementations though.
2013-06-27 13:55:42 +09:00
Norman Maurer
58b968b603 [#1454] Fix IndexOutOfBoundsException which was thrown if last component of a CompositeByteBuf was removed 2013-06-25 09:32:00 +02:00
Trustin Lee
dbab41cc50 Improve the utilization of subpage pools
.. by avoiding the overly frequent removal of a subpage from a pool

This change makes sure that the unused subpage is not removed when there's no subpage left in the pool.  If the last subpage is removed from the pool, it is very likely that the allocator will create a new subpage very soon again, so it's better not remove it.
2013-06-25 11:07:15 +09:00
Trustin Lee
a6795d7780 [maven-release-plugin] prepare for next development iteration 2013-06-25 11:07:15 +09:00
Trustin Lee
2221446425 [maven-release-plugin] prepare release netty-4.0.0.CR6 2013-06-25 11:07:15 +09:00
Trustin Lee
a2f232720b Make AdaptiveRecvByteBufAllocator's lookup table simpler / Optimize buffer size normalization
- No need to have fine-grained lookup table because the buffer pool has
  much more coarse capacities available
- No need to use a loop to normalize a buffer capacity
2013-06-25 11:07:14 +09:00
Norman Maurer
6a9f965f9b Introduce new utility class calles ReferenceCountUtil and move utility methods from ByteBufUtil to it.
The ones in ByteBufUtil were marked as @deprecated
2013-06-14 07:07:33 +02:00
Trustin Lee
a5871dfd86 [maven-release-plugin] prepare for next development iteration 2013-06-14 12:55:15 +09:00
Trustin Lee
f5377cc8d7 [maven-release-plugin] prepare release netty-4.0.0.CR5 2013-06-14 12:55:05 +09:00
Trustin Lee
0da48e7e7f Determine the default number of heap/direct arenas of PooledByteBufAllocator conservatively
- Fixes #1445
- Add PlatformDependent.maxDirectMemory()
- Ensure the default number or arenas is decreased if the max memory of the VM is not large enough.
2013-06-14 12:14:45 +09:00
Trustin Lee
e5ca6518ba [maven-release-plugin] prepare for next development iteration 2013-06-13 17:02:32 +09:00
Trustin Lee
381063e09c [maven-release-plugin] prepare release netty-4.0.0.CR4 2013-06-13 17:02:19 +09:00
Trustin Lee
6d1cd0d0cd ReferenceCountException -> IllegalReferenceCountException 2013-06-13 14:00:15 +09:00
Trustin Lee
7eb0f6105d Fix memory leaks 2013-06-13 13:32:47 +09:00
Trustin Lee
175526b6bd Move ReferenceCounted and AbstractReferenceCounted to io.netty.util
- Fixes #1441
- Also move and rename IllegalBufferAccessException to ReferenceCountException
- Prettier reference count exception messages
2013-06-13 13:14:21 +09:00
Trustin Lee
283feda119 Reduce even more garbage by exposing ByteBuf.internalNioBuffer() 2013-06-13 12:40:26 +09:00
Trustin Lee
2d7c6f8ee1 Make PooledByteBuf recyclable regardless its maxCapacity
- Make AbstractByteBuf.maxCapacity internally mutable so that PooledByteBuf is completely recyclable
2013-06-12 04:18:40 +09:00
Trustin Lee
9396246fe9 " " -> " " / Cleanup 2013-06-12 04:07:09 +09:00
Norman Maurer
341f7757aa Fix checkstyle 2013-06-11 16:12:34 +02:00
Norman Maurer
bf046492fb [#1439] Fix CompositeByteBuf.nioBufferCount() to return the correct number 2013-06-11 16:07:40 +02:00
Trustin Lee
bf5960e9eb Fix #1435 and #1436 by reverting 7f7bf304b0
Different PooledByteBufs can have the reference to the same PooledChunk (and its ByteBuffer), so it's incorrect not to create a duplicate.
2013-06-11 00:57:23 +09:00
Trustin Lee
7f7bf304b0 Optimize PooledUnsafeDirectByteBuf.newInternalNioBuffer()
- No need to produce garbage
2013-06-10 22:08:30 +09:00
Trustin Lee
6732c6761b Recycle PooledByteBuf partially
- Related issue: #1397
- Resource leak detection should be turned off and the maxCapacity has to be Integer.MAX_VALUE
- It's technically possible to pool PooledByteBufs with different maxCapacity, which will be addressed in another commit.
2013-06-10 19:52:56 +09:00
Derek Troy-West
f2d8a745b1 [#1422] ReadOnlyByteBuffer.isWritable() should return false 2013-06-10 09:23:57 +02:00
Trustin Lee
14158070bf Revamp the core API to reduce memory footprint and consumption
The API changes made so far turned out to increase the memory footprint
and consumption while our intention was actually decreasing them.

Memory consumption issue:

When there are many connections which does not exchange data frequently,
the old Netty 4 API spent a lot more memory than 3 because it always
allocates per-handler buffer for each connection unless otherwise
explicitly stated by a user.  In a usual real world load, a client
doesn't always send requests without pausing, so the idea of having a
buffer whose life cycle if bound to the life cycle of a connection
didn't work as expected.

Memory footprint issue:

The old Netty 4 API decreased overall memory footprint by a great deal
in many cases.  It was mainly because the old Netty 4 API did not
allocate a new buffer and event object for each read.  Instead, it
created a new buffer for each handler in a pipeline.  This works pretty
well as long as the number of handlers in a pipeline is only a few.
However, for a highly modular application with many handlers which
handles connections which lasts for relatively short period, it actually
makes the memory footprint issue much worse.

Changes:

All in all, this is about retaining all the good changes we made in 4 so
far such as better thread model and going back to the way how we dealt
with message events in 3.

To fix the memory consumption/footprint issue mentioned above, we made a
hard decision to break the backward compatibility again with the
following changes:

- Remove MessageBuf
- Merge Buf into ByteBuf
- Merge ChannelInboundByte/MessageHandler and ChannelStateHandler into ChannelInboundHandler
  - Similar changes were made to the adapter classes
- Merge ChannelOutboundByte/MessageHandler and ChannelOperationHandler into ChannelOutboundHandler
  - Similar changes were made to the adapter classes
- Introduce MessageList which is similar to `MessageEvent` in Netty 3
- Replace inboundBufferUpdated(ctx) with messageReceived(ctx, MessageList)
- Replace flush(ctx, promise) with write(ctx, MessageList, promise)
- Remove ByteToByteEncoder/Decoder/Codec
  - Replaced by MessageToByteEncoder<ByteBuf>, ByteToMessageDecoder<ByteBuf>, and ByteMessageCodec<ByteBuf>
- Merge EmbeddedByteChannel and EmbeddedMessageChannel into EmbeddedChannel
- Add SimpleChannelInboundHandler which is sometimes more useful than
  ChannelInboundHandlerAdapter
- Bring back Channel.isWritable() from Netty 3
- Add ChannelInboundHandler.channelWritabilityChanges() event
- Add RecvByteBufAllocator configuration property
  - Similar to ReceiveBufferSizePredictor in Netty 3
  - Some existing configuration properties such as
    DatagramChannelConfig.receivePacketSize is gone now.
- Remove suspend/resumeIntermediaryDeallocation() in ByteBuf

This change would have been impossible without @normanmaurer's help. He
fixed, ported, and improved many parts of the changes.
2013-06-10 16:10:39 +09:00
Norman Maurer
bc20107b68 Use correct value to disable/enable direct arenas in PooledByteBufAllocator 2013-05-30 20:24:11 +02:00
Norman Maurer
81e3c1719a [maven-release-plugin] prepare for next development iteration 2013-05-18 09:59:13 +02:00
Norman Maurer
99caefdf39 [maven-release-plugin] prepare release netty-4.0.0.CR3 2013-05-18 09:57:11 +02:00
Trustin Lee
f92cfba388 Remove DefaultCompositeByteBuf.lastAccessed and use binary search instead
- Fixes #1364
- Even if a user creates a duplicate/slice, lastAccessed was shared between the derived buffers and it's updated even by a read operation, which made multithread access impossible
2013-05-17 16:54:47 +09:00
Norman Maurer
c43950a03f [maven-release-plugin] prepare for next development iteration 2013-05-08 18:19:51 +02:00
Norman Maurer
ae76502040 [maven-release-plugin] prepare release netty-4.0.0.CR2 2013-05-08 18:19:38 +02:00
Trustin Lee
2a15f658d6 Ensure UnreleasableByteBuf does not expose the wrapped buffer to the caller
- Fixes #1324
2013-05-01 20:43:23 +09:00
Trustin Lee
1e0c83db23 Introduce AddressedEnvelope message type for generic representation of an addressed message
- Fixes #1282 (not perfectly, but to the extent it's possible with the current API)
- Add AddressedEnvelope and DefaultAddressedEnvelope
- Make DatagramPacket extend DefaultAddressedEnvelope<ByteBuf, InetSocketAddress>
- Rename ByteBufHolder.data() to content() so that a message can implement both AddressedEnvelope and ByteBufHolder (DatagramPacket does) without introducing two getter methods for the content
- Datagram channel implementations now understand ByteBuf and ByteBufHolder as a message with unspecified remote address.
2013-05-01 17:04:43 +09:00
Trustin Lee
2e0dd65250 Fix a bug where the unpooled buffer returned by the pooled allocator reports an incorrect allocator 2013-05-01 11:14:21 +09:00
Trustin Lee
a218eb6f6f Allow to disable only heap or direct buffer pool
- Fixes #1315

If a user specifies the arena size of 0, the pool is now disabled
instead of raising an IllegalArgumentException. Using this, you can
disable only heap or direct buffer pool easily. Once disabled,
PooledByteBufAllocator will delegate the allocation request to
UnpooledByteBufAllocator.
2013-04-27 08:55:16 +09:00
Trustin Lee
b5989e2449 Reduce exception instantiation overhead in SslHandler / Reduce unnecessary empty array creation
- Added EmptyArrays as an internal utility class
2013-04-24 09:32:53 +09:00
Trustin Lee
32fa4c07f3 Do not unwrap a CompositeByteBuf when it is added as a component of another CompositeByteBuf
.. because Reference counting introduces life cycle issues to the CompositeByteBuf being added.

 - Fixes #1266
2013-04-23 21:53:51 +09:00
Trustin Lee
87007d4eb8 Fix another memory leak in AbstractByteBufTest 2013-04-23 13:46:34 +09:00
Norman Maurer
9a5f45a0c1 [#1297] Make sure ResourceLeakDetector.open(...) is only used after constructing was successful 2013-04-22 10:07:22 +02:00
Norman Maurer
ab685de7a3 [#1273] Fix resource leaks in test 2013-04-22 09:47:44 +02:00
Trustin Lee
18dca2a8a4 Fix checkstyle 2013-04-19 04:37:51 +09:00
Trustin Lee
8884e311f1 Fix a bug where DefaultCompositeByteBuf.nioBuffers() fails when its component's nioBufferCount() is greater than 1
- Fixes #1267
2013-04-19 04:35:44 +09:00
Trustin Lee
9e890f0ab8 Ensure to release the component when it's removed from CompositeByteBuf / Add tests for reference counting of CompositeByteBuf 2013-04-18 16:40:22 +09:00
Trustin Lee
5bfb408b7d Add setRefCnt(int) method to AbstractReferenceCounted(ByteBuf)
- Fixes #1265
2013-04-11 19:36:27 +09:00
Norman Maurer
3b8673733e [#1262] Respect adjustment when using SlicedByteBuf.nioBuffers() 2013-04-10 14:19:03 +02:00
Norman Maurer
d34daebeca [#1238] Correctly log the content of the MessageBuf and not depend on MessageBuf.toString()
Add an extra static method to BufUtil which will convert the content of any MessaBuf implementation to a String
2013-04-08 09:45:00 +02:00
Andrei Pozolotin
a3e760a003 fix #1234 - duplicate package-info.java errors in eclipse requires release of netty-build v 19 and netty-parent update. 2013-04-05 05:38:05 +09:00
Trustin Lee
0ac31ae846 Make Unpooled*ByteBuf public so that ByteBufAllocator implementor can extend it 2013-04-04 17:26:14 +09:00
Norman Maurer
94ef7dc1b9 Optimize to minimize volatile reads to access next buffer in codec framework 2013-04-03 18:03:55 +02:00
Trustin Lee
c3559ddbda Fix misleading example in ByteBuf 2013-04-03 22:18:38 +09:00
Trustin Lee
baf9ecfe7b Fix IndexOutOfBoundsException raised when numHeapArenas and numDirectArenas differ
- Fixes #1227
2013-04-03 22:15:34 +09:00
Trustin Lee
8fef511390 Fix typo 2013-04-03 22:08:43 +09:00
Norman Maurer
af4b71a00e Remove special handling of Object[] in codec framework (a.k.a unfolding)
- Fixes #1229
- Primarily written by @normanmaurer and revised by @trustin

This commit removes the notion of unfolding from the codec framework
completely.  Unfolding was introduced in Netty 3.x to work around the
shortcoming of the codec framework where encode() and decode() did not
allow generating multiple messages.

Such a shortcoming can be fixed by changing the signature of encode()
and decode() instead of introducing an obscure workaround like
unfolding.  Therefore, we changed the signature of them in 4.0.

The change is simple, but backward-incompatible.  encode() and decode()
do not return anything.  Instead, the codec framework will pass a
MessageBuf<Object> so encode() and decode() can add the generated
messages into the MessageBuf.
2013-04-03 21:44:54 +09:00
Trustin Lee
2ffa083d3c Allow overriding the default allocator properties and log them / Prettier log 2013-04-03 12:08:01 +09:00
Prajwal Tuladhar
05850da863 enable checkstyle for test source directory and fix checkstyle errors 2013-03-30 13:18:57 +01:00
Adam Vandenberg
8e23ab6886 Fix typo in BufType javadoc 2013-03-29 21:47:55 +09:00
Norman Maurer
5a7f049fe2 [#1202] Fix javadoc 2013-03-28 10:01:26 +01:00
Norman Maurer
71727e42de [#1210] Allow to use derived buffers with DefaultBufferHolder and so fix broken SpdySessionHandler 2013-03-27 07:30:12 +01:00
Norman Maurer
f53db96a3e [#1198] Fix references to ChannelBuffer and ChannelBuffers 2013-03-25 11:06:58 +01:00
Norman Maurer
4eb0172251 [#1196] Make it clear that addComponent(..) of CompositeByteBuf does NOT increase the writerIndex 2013-03-25 10:58:50 +01:00
Norman Maurer
59012390f6 Fix version numbering 2013-03-25 08:01:11 +01:00
Norman Maurer
f136fb3673 [#1197] Add Unpooled.unreleasableBuffer(...) to create a unreleasable view on a ByteBuf) 2013-03-23 11:25:01 +01:00
Norman Maurer
c71dc9d4b6 [#1195] Fix Unpooled.wrappedBuffer(..) with non-direct read-only ByteBuffer 2013-03-23 10:57:07 +01:00
Norman Maurer
7d7b676eeb [maven-release-plugin] prepare for next development iteration 2013-03-22 15:20:35 +01:00
Norman Maurer
60fc7dac4d [maven-release-plugin] prepare release netty-4.0.0.CR1 2013-03-22 15:20:11 +01:00
Trustin Lee
28576aa41e Fix incorrect exception message
- Thanks @hepin1989
2013-03-22 16:07:10 +09:00
Trustin Lee
1bad0b48cf Fix memory leak in the test 2013-03-22 12:37:57 +09:00
Trustin Lee
6869a2bd23 Fix memory leak in AbstractCompositeByteBufTest 2013-03-22 09:01:28 +09:00
Norman Maurer
52c4e042d6 Correctly handle read-only direct ByteBuffer when wrap them 2013-03-22 06:49:57 +09:00
Trustin Lee
2a87950784 [maven-release-plugin] prepare for next development iteration 2013-03-16 18:41:36 +09:00
Trustin Lee
adfb29330b [maven-release-plugin] prepare release netty-4.0.0.Beta3 2013-03-16 18:40:59 +09:00
Trustin Lee
5fe2e7fc9d Fix more memory leaks in buffer tests 2013-03-14 17:21:31 +09:00
Trustin Lee
0f351d2c47 Fix memory leak in DefaultCompositeByteBuf when a component is another CompositeByteBuf / Allow retain() and release() on a derived buffer 2013-03-14 16:37:20 +09:00
Trustin Lee
60d9984db1 Fix memory leak in DefaultCompositeByteBuf when a component is another CompositeByteBuf 2013-03-14 16:06:38 +09:00
Trustin Lee
d2b137649d Fix more memory leaks in the buffer tests 2013-03-14 15:25:22 +09:00
Trustin Lee
b86d3d692a Fix a bug where AbstractByteBuf.order() doesn't return a swapped buffer if capacity is 0.
- Fixes #1152
2013-03-14 06:58:14 +09:00
Trustin Lee
5830875b42 Fix a memory leak in AbstractCompositeByteBufTest
- Fixed #1147
2013-03-13 15:09:26 +09:00
Trustin Lee
94a9096be5 Fix a memory leak in AbstractCompositeByteBufTest 2013-03-12 17:57:23 +09:00
Trustin Lee
b271774c90 Fix memory leak in UnpooledTest
- nothing critical. It's a test that leaks.  Not CompositeByteBuf implementation.
2013-03-12 16:49:14 +09:00
Trustin Lee
e1dd149ca6 Reschedule the streaming API for later
- Will release as a part of http_next
2013-03-12 13:08:10 +09:00
Trustin Lee
8f5eaaa740 Make StreamTest finish sooner to make CI happy 2013-03-11 09:46:36 +09:00
Trustin Lee
5d65bbc0a9 Add package-info.java for the Stream API / Print maven version on travis CI 2013-03-11 09:09:46 +09:00
Trustin Lee
32efba34d8 Initial implementation of the Streaming API
This pull request provides a framework for exchanging a very large
stream between handlers, typically between a decoder and an inbound
handler (or between a handler that writes a message and an encoder that
encodes that message).

For example, an HTTP decoder, previously, generates multiple
micro-messages to decode an HTTP message (i.e. HttpRequest +
HttpChunks). With the streaming API, The HTTP decoder can simply
generate a single HTTP message whose content is a Stream. And then the
inbound handler can consume the Stream via the buffer you created when
you begin to read the stream. If you create a buffer whose capacity is
bounded, you can handle a very large stream without allocating a lot of
memory. If you just want to wait until the whole content is ready, you
can also do that with an unbounded buffer.

The streaming API also supports a limited form of communication between
a producer (i.e. decoder) and a consumer. A producer can abort the
stream if the stream is not valid anymore. A consumer can choose to
reject or discard the stream, where rejection is for unrecoverable
failure and discard is for recoverable failure.

P.S. Special thanks to @jpinner for the initial input.
2013-03-11 08:57:17 +09:00
Norman Maurer
0a1bc86569 Javadocs cleanup / added 2013-03-10 21:07:19 +01:00
Norman Maurer
0ac5fd9f18 Let BufUtil.retain(...) return the given object 2013-03-10 19:50:26 +01:00
Trustin Lee
63116239ac Always use EmptyByteBuf when a user attempts to construct a buffer with 0 maxCapacity / Make EmptyByteBuf remember the allocator it came from / Optimize EmptyByteBuf a little bit 2013-03-08 11:03:11 +09:00
Trustin Lee
096e4c95ef Fix checkstyle 2013-03-08 10:39:52 +09:00
Trustin Lee
12f1d96914 Relaxed memory access constraint of ReferenceCounted.refCnt() for potentially better performance / More precise reference counting for MessageBuf 2013-03-08 10:32:20 +09:00
Trustin Lee
303f83043b Fix checkstyle 2013-03-06 18:23:08 +09:00
Trustin Lee
88df53ec1a Fix infinite recursion when transferring data between different type of buffers / Add ByteBuf.hasMemoryAddress/memoryAddress()
- Fixes: #1109 and #1110
2013-03-06 18:22:16 +09:00
Trustin Lee
3d6d9f394d Fix checkstyle 2013-03-06 17:10:26 +09:00
Trustin Lee
81ce0555e6 Add UnpooledUnsafeDirectByteBuf and use it when low-level access is available
- Remove PooledUnsafeDirectByteBuf.setMedium() which is redundant
- Fix constructor visibility
2013-03-06 16:01:46 +09:00
Trustin Lee
6c3d5ed907 Use _set* in AbstractByteBuf.write*() 2013-03-06 14:56:27 +09:00
Trustin Lee
1c1570ffc4 Make field access via ByteBuf.read/write*() faster by avoiding unnecessary boundary checks
- also disabled a time consuming test that is actually a regression test
2013-03-06 10:32:29 +09:00
Trustin Lee
5f2c2cdc9b Fix a bug in PoolArena and PoolSubpage where subpage pools are not updated correctly
- Make PoolSubpage a linked list node in the pool
- Now that a subpage is added to and removed from the pool correctly, allocating a subpage from the pool became vastly simpler.
2013-03-05 23:55:41 +09:00
Trustin Lee
4cb023f190 Add more variants of ByteBufAllocator.ioBuffer() / Update Javadoc 2013-03-05 17:59:31 +09:00
Trustin Lee
8d88acb4a7 Change ByteBufAllocator.buffer() to allocate a direct buffer only when the platform can handle a direct buffer reliably
- Rename directbyDefault to preferDirect
 - Add a system property 'io.netty.prederDirect' to allow a user from changing the preference on launch-time
 - Merge UnpooledByteBufAllocator.DEFAULT_BY_* to DEFAULT
2013-03-05 17:55:24 +09:00
Trustin Lee
307e6c47d8 Make hasUnsafe() return true only when all necessary low level operations are available for reliable direct buffer access 2013-03-05 17:25:54 +09:00
Trustin Lee
7e17f71b30 Make PlatformDependent work with the platforms without unaligned access support 2013-03-05 14:27:52 +09:00
Trustin Lee
d4d01ba52c Optimize SlicedByteBuf a little bit 2013-02-27 15:19:25 -08:00
Trustin Lee
49aa907bd0 [maven-release-plugin] prepare for next development iteration 2013-02-26 16:55:07 -08:00
Trustin Lee
5026c2f359 [maven-release-plugin] prepare release netty-4.0.0.Beta2 2013-02-26 16:54:53 -08:00
Trustin Lee
a9a8d5d8c2 Clean-up 2013-02-21 14:17:04 -08:00
Norman Maurer
74738fbd08 [#1061] Add workaround to not use PooledUnsafeDirectByteBuf when running on latest OpenJDK6 because of missing Unsafe method 2013-02-19 12:21:08 +01:00
Norman Maurer
4ed5b07e4e [#1060] Fix bug in CompositeByteBuf which let the buffer expand in a incorrect way and so result in corrupted data 2013-02-19 09:43:31 +01:00
Norman Maurer
1c6ed9b2ce Add javadocs 2013-02-17 20:12:48 +01:00
Trustin Lee
d68a04a879 [maven-release-plugin] prepare for next development iteration 2013-02-14 12:56:24 -08:00
Trustin Lee
59e638f8f5 [maven-release-plugin] prepare release netty-4.0.0.Beta1 2013-02-14 12:56:15 -08:00
Trustin Lee
1011227b88 Remove apiviz tags - we are focusing on user guide instead and putting diagrams there 2013-02-14 12:09:16 -08:00
Norman Maurer
7c002504a1 [#1051] Increment by the correct value 2013-02-14 14:28:47 +01:00
Norman Maurer
5370573400 Change ReferenceCounted.retain* to return itself and so allow method chaining 2013-02-14 07:39:44 +01:00
Trustin Lee
a1540a49d3 More safety in DefaultMessageBuf 2013-02-13 14:58:21 -08:00
Norman Maurer
a8216e7ce0 Fix possible NPE 2013-02-13 09:52:07 +01:00
Trustin Lee
4aacf50758 Prettify APIviz diagrams / Remove an empty package 2013-02-11 18:33:15 +09:00
Trustin Lee
aca0d5fa68 Make CompositeyteBuf and MessageBuf call release() on its elements when it is deallocated 2013-02-11 17:19:53 +09:00
Trustin Lee
bf0bfe9a69 Fix inspector warnings 2013-02-11 16:52:43 +09:00
Trustin Lee
0e341c9d72 Fix covariant return types in ChannelConfig interfaces for method chaining
- Does it really worth doing this? I'm tempted to remove support for method chaining for ChannelConfig.
2013-02-11 15:08:51 +09:00
Trustin Lee
4f6d05365a Fix a race condition in reference counter implementation / Reference count never goes below 0 2013-02-10 14:22:14 +09:00
Trustin Lee
b9996908b1 Implement reference counting
- Related: #1029
- Replace Freeable with ReferenceCounted
- Add AbstractReferenceCounted
- Add AbstractReferenceCountedByteBuf
- Add AbstractDerivedByteBuf
- Add EmptyByteBuf
2013-02-10 13:10:09 +09:00
Trustin Lee
43af599d52 Faster memory copy between direct buffer and byte array (part 2) 2013-02-09 02:09:37 +09:00
Trustin Lee
9a676bc7d5 Faster memory copy between direct buffer and byte array 2013-02-09 01:55:01 +09:00
Trustin Lee
affd514b8c Rename ByteBufUtil to BufUtil and move ChannelHandlerUtil.freeMessage() there / Remove ChannelHandlerUtil 2013-02-08 23:23:26 +09:00
Trustin Lee
44ea0a116f Replace ChannelHandlerUtil.unfoldAndAdd() with MessageBuf.unfoldAndAdd() / Remove unused methods in ChannelHandlerUtil 2013-02-08 23:07:20 +09:00
Trustin Lee
d4742bbe16 Clean up abstract ChannelHandler impls / Remove ChannelHandlerContext.hasNext*()
- Rename ChannelHandlerAdapter to ChannelDuplexHandler
- Add ChannelHandlerAdapter that implements only ChannelHandler
- Rename CombinedChannelHandler to CombinedChannelDuplexHandler and
  improve runtime validation
- Remove ChannelInbound/OutboundHandlerAdapter which are not useful
- Make ChannelOutboundByteHandlerAdapter similar to
  ChannelInboundByteHandlerAdapter
- Make the tail and head handler of DefaultChannelPipeline accept both
  bytes and messages.  ChannelHandlerContext.hasNext*() were removed
  because they always return true now.
- Removed various unnecessary null checks.
- Correct method/field names:
  inboundBufferSuspended -> channelReadSuspended
2013-02-07 23:47:45 +09:00
Trustin Lee
7eed272e57 Add FilteredMessageBuf 2013-02-07 23:37:04 +09:00
Norman Maurer
fd75615d7a [#870] Convert all modules into osgi bundles 2013-02-06 07:57:11 +01:00
Trustin Lee
2ec932798f Replace .readable() and .writable() to .isReadable() and .isWritable() 2013-01-31 18:24:33 +01:00
Trustin Lee
42c65cca3a Make MessageBuf bounded
- Move common methods from ByteBuf to Buf
- Rename ensureWritableBytes() to ensureWritable()
- Rename readable() to isReadable()
- Rename writable() to isWritable()
- Add isReadable(int) and isWritable(int)
- Add AbstractMessageBuf
- Rewrite DefaultMessageBuf and QueueBackedMessageBuf
  - based on Josh Bloch's public domain ArrayDeque impl
2013-01-31 18:11:06 +01:00
Norman Maurer
ee9f30a2b9 *ChannelBuffer* -> *ByteBuf* and ChannelBuffersTest -> UnpooledTest 2013-01-21 21:02:25 +01:00
Courtney Robinson
7b6cbdbb1e [#964] ByteBuf.readLine() must return null when no more data is available in ByteBuf 2013-01-21 20:56:00 +01:00
Trustin Lee
ba8c8171fa Fix leaks in buffer tests 2013-01-18 13:49:17 +09:00
Trustin Lee
b5c87b0f9c Fix failing tests 2013-01-18 13:34:00 +09:00
Trustin Lee
d0afe6cad0 Consistent exception messages 2013-01-18 13:10:54 +09:00
Trustin Lee
ad15155f04 Ensure cascaded derivation of a buffer does not result in an infinitely nested buffer. 2013-01-17 13:55:48 +09:00
Trustin Lee
d806e3bf81 SwappedByteBuf.unwrap() should return null
Fixes #945.  For swapped wrapped buffers, it should not return null.
2013-01-17 12:05:43 +09:00
Trustin Lee
34820511ff Second HTTP overhaul
- Rename message types for clarity
  - HttpMessage -> FullHttpMessage
  - HttpHeader -> HttpMessage
  - HttpRequest -> FullHttpRequest
  - HttpResponse -> FulllHttpResponse
  - HttpRequestHeader -> HttpRequest
  - HttpResponseHeader -> HttpResponse
- HttpContent now extends ByteBufHolder; no more content() method
- Make HttpHeaders abstract, make its header access methods public, and
  add DefaultHttpHeaders
- Header accessor methods in HttpMessage and LastHttpContent are
  replaced with HttpMessage.headers() and
  LastHttpContent.trailingHeaders(). Both methods return HttpHeaders.
- Remove setters wherever possible and remove 'get' prefix
- Instead of calling setContent(), a user can either specify the content
  when constructing a message or write content into the buffer.
  (e.g. m.content().writeBytes(...))
- Overall cleanup & fixes
2013-01-16 23:46:02 +09:00
Trustin Lee
337f5bbb8e Automatic diagnosis of resource leaks
Now that we are going to use buffer pooling by default, it is obvious
that a user will forget to call .free() and report memory leak. In this
case, we should have a tool to determine if it is a bug in our allocator
implementation or in the user's code.

This pull request adds a system property flag called
'io.netty.resourceLeakDetection'. If set, when a user forgets to call
.free(), the ResourceLeakDetector will detect it and log a message with
detailed stack trace to tell where the leaked buffer has been allocated.

Because obtaining stack trace is an expensive operation, I used sampling
technique. Allocation is recorded only for every 113th allocation. I
chose 113 because it's a prime number.

In production, a user might not want to enable this option due to
potential performance impact. If a user does not specify the
'-Dio.netty.resourceLeakDetection' option leak detection is disabled.

Even if the leak detection is enabled, the overhead should be less than
5% because only ~1% of allocations are monitored.

I also replaced SharedResourceMisuseDetector with ResourceLeakDetector.
2013-01-15 14:15:27 +09:00
Trustin Lee
04bae9bceb Use sun.misc.Unsafe to access a direct ByteBuffer
- Add PooledUnsafeDirectByteBuf, a variant of PooledDirectByteBuf, which
  accesses its underlying direct ByteBuffer using sun.misc.Unsafe.
- To decouple Netty from sun.misc.*, sun.misc.Unsafe is accessed via
  PlatformDependent.
- This change solely introduces about 8+% improvement in direct memory
  access according to the tests conducted as described in #918
2013-01-11 16:25:12 +09:00
Norman Maurer
e564157381 Fix one checkstyle and one compile error caused by the last commit 2013-01-11 07:45:22 +01:00
Norman Maurer
75b2dd592a Merge branch 'master' of github.com:netty/netty 2013-01-11 07:33:54 +01:00
Trustin Lee
64ae8b6a37 Replace and merge DetectionUtil and DirectByteBufUtil into PlatformDependent and PlatformDependent0
PlatformDependent delegates the operations requires sun.misc.* to PlatformDependent0 to avoid runtime errors due to missing sun.misc.* classes.
2013-01-11 14:03:27 +09:00
Trustin Lee
896bbb67b6 Use sun.misc.Unsafe to access a direct buffer if possible 2013-01-10 21:30:11 +01:00
Norman Maurer
5b5b39a606 [#916] Only access Cleaner if it is really present to prevent errors on android 2013-01-10 20:03:54 +01:00
Trustin Lee
5bd8b41a58 Use Number.reverseBytes() instead of custom impl 2013-01-10 18:47:21 +09:00
Trustin Lee
340da3e97b Fix infinite recursion 2013-01-10 18:43:20 +09:00
Trustin Lee
eb337ff5a7 Fix various inspection warnings 2013-01-10 15:23:58 +09:00
Trustin Lee
8f7fba2d39 Optimize AbstractByteBuf.adjustMarkers() 2013-01-04 23:51:11 +09:00
Norman Maurer
364b7d1430 Make sure the Unpooled.EMTPY_BUFFER can not be freed and the capacity can not be changed 2013-01-03 22:49:25 +01:00
Norman Maurer
37a3f2e3b8 [#887] [#866] [#883] Add unified interface for Message oriented protocols and also use direct buffers for them 2013-01-03 18:15:53 +01:00
Norman Maurer
8a7bc2c606 Add a lot of javadocs to make usage more clear 2012-12-21 22:22:40 +01:00
Trustin Lee
67da6e4bf9 Remove the notion of ByteBufAllocator.bufferMaxCapacity()
- Allocate the unpooled memory if the requested capacity is greater then the chunkSize
- Fixes #834
2012-12-19 17:35:32 +09:00
Trustin Lee
0e017db89a Return the new buffer's capacity is same with the requested capacity
- Rename capacity variables to reqCapacity or normCapacity to distinguish if its the request capacity or the normalized capacity
- Do not reallocate on ByteBuf.capacity(int) if reallocation is unnecessary; just update the index range.
- Revert the workaround in DefaultChannelHandlerContext
2012-12-19 16:50:05 +09:00
Trustin Lee
a8f5efdb26 Add proper boundary / freeness check on ByteBuf impls
- Fixes #827
2012-12-17 18:27:30 +09:00
Trustin Lee
ca93b624ff Add IllegalBufferAccessException and checks on MessageBuf impls
- Related: #827
2012-12-17 18:03:31 +09:00
Trustin Lee
def12a171c Rename ChannelBuf to Buf and ChannelBufType to BufType
- Fixes #825
2012-12-17 17:43:45 +09:00
Trustin Lee
03e68482bb Remove ChannelBuf/ByteBuf.Unsafe
- Fixes #826
Unsafe.isFreed(), free(), suspend/resumeIntermediaryAllocations() are not that dangerous. internalNioBuffer() and internalNioBuffers() are dangerous but it seems like nobody is using it even inside Netty. Removing those two methods also removes the necessity to keep Unsafe interface at all.
2012-12-17 17:41:21 +09:00
Trustin Lee
33134b1343 Ensure PooledByteBuf.init() is not called with null memory 2012-12-17 16:02:21 +09:00
Trustin Lee
5ffb495746 Do not allow suspendIntermediaryDeallocations() after free() 2012-12-14 18:39:00 +09:00
Trustin Lee
5a4a59406b Merge ByteBuf.hasNioBuffer() and hasNioBuffers()
- Fixes #797
2012-12-14 12:20:33 +09:00
Trustin Lee
8e1a6c6cf5 Add ByteBuf.maxWritableBytes()
- Fixes #806
2012-12-14 11:49:01 +09:00
Trustin Lee
ad10518fca Fix the incorrect snapshot version number 2012-12-13 22:49:31 +09:00
Trustin Lee
b47fc77522 Add PooledByteBufAllocator + microbenchmark module
This pull request introduces the new default ByteBufAllocator implementation based on jemalloc, with a some differences:

* Minimum possible buffer capacity is 16 (jemalloc: 2)
* Uses binary heap with random branching (jemalloc: red-black tree)
* No thread-local cache yet (jemalloc has thread-local cache)
* Default page size is 8 KiB (jemalloc: 4 KiB)
* Default chunk size is 16 MiB (jemalloc: 2 MiB)
* Cannot allocate a buffer bigger than the chunk size (jemalloc: possible) because we don't have control over memory layout in Java. A user can work around this issue by creating a composite buffer, but it's not always a feasible option. Although 16 MiB is a pretty big default, a user's handler might need to deal with the bounded buffers when the user wants to deal with a large message.

Also, to ensure the new allocator performs good enough, I wrote a microbenchmark for it and made it a dedicated Maven module. It uses Google's Caliper framework to run and publish the test result (example)

Miscellaneous changes:

* Made some ByteBuf implementations public so that those who implements a new allocator can make use of them.
* Added ByteBufAllocator.compositeBuffer() and its variants.
* ByteBufAllocator.ioBuffer() creates a buffer with 0 capacity.
2012-12-13 22:35:06 +09:00
Trustin Lee
51e6519b67 Replace UnsafeByteBuf with ByteBuf.unsafe() again
* UnsafeByteBuf is gone. I added ByteBuf.unsafe() back.
* To avoid extra instantiation, all ByteBuf implementations implement the ByteBuf.Unsafe interface.
* To hide this implementation detail, all ByteBuf implementations are package-private.
* AbstractByteBuf and SwappedByteBuf are public and they do not implement ByteBuf.Unsafe because they don't need to.
* unwrap() is not an unsafe operation anymore.
* ChannelBuf also has unsafe() and Unsafe. ByteBuf.Unsafe extends ChannelBuf.unsafe(). ChannelBuf.unsafe() provides free() operation so that a user does not need to down-cast the buffer in freeInbound/OutboundBuffer().
2012-12-05 19:28:56 +09:00
Norman Maurer
85c570505b [maven-release-plugin] prepare for next development iteration 2012-12-03 20:34:05 +01:00
Norman Maurer
17d77ed160 [maven-release-plugin] prepare release netty-4.0.0.Alpha8 2012-12-03 20:33:49 +01:00
Trustin Lee
33c0c89fef Remove unnecessary empty lines 2012-12-03 19:58:13 +09:00
Trustin Lee
e45ab1d182 Fix a checkstyle error 2012-12-02 22:03:11 +09:00
Trustin Lee
5f9090a7f0 Fix invalid memory access in AIO writes
To perform writes in AioSocketChannel, we get a ByteBuffer view of the
outbound buffer and specify it as a parameter when we call
AsynchronousSocketChannel.write().

In most cases, the write() operation is finished immediately.  However,
sometimes, it is scheduled for later execution.  In such a case, there's
a chance for a user's handler to append more data to the outbound
buffer.

When more data is appended to the outbound buffer, the outbound buffer
can expand its capacity by itself.  Changing the capacity of a buffer is
basically made of the following steps:

1. Allocate a larger new internal memory region.
2. Copy the current content of the buffer to the new memory region.
3. Rewire the buffer so that it refers to the new region.
4. Deallocate the old memory region.

Because the old memory region is deallocated at the step 4, the write
operation scheduled later will access the deallocated region, leading
all sort of data corruption or even segfaults.

To prevent this situation, I added suspendIntermediaryDeallocations()
and resumeIntermediaryDeallocations() to UnsafeByteBuf.

AioSocketChannel.doFlushByteBuf() now calls suspendIntermediaryDealloc()
to defer the deallocation of the old memory regions until the completion
handler is notified.
2012-12-02 21:50:33 +09:00
Trustin Lee
bfe2a96505 Fix AssertionError from AsyncSocketChannel.beginRead()
An AssertionError is triggered by a ByteBuf when beginRead() attempts to
access the buffer which has been freed already.  This commit ensures the
buffer is not freed before performing an I/O operation.

To determine if the buffer has been freed, UnsafeByteBuf.isFreed() has
been added.
2012-12-02 20:17:53 +09:00
Trustin Lee
6208c62888 Fix inspector warnings introduced by recent mergences 2012-11-30 23:01:57 +09:00
Trustin Lee
81e2db10fa ByteBufAllocator API w/ ByteBuf perf improvements
This commit introduces a new API for ByteBuf allocation which fixes
issue #643 along with refactoring of ByteBuf for simplicity and better
performance. (see #62)

A user can configure the ByteBufAllocator of a Channel via
ChannelOption.ALLOCATOR or ChannelConfig.get/setAllocator().  The
default allocator is currently UnpooledByteBufAllocator.HEAP_BY_DEFAULT.

To allocate a buffer, do not use Unpooled anymore. do the following:

  ctx.alloc().buffer(...); // allocator chooses the buffer type.
  ctx.alloc().heapBuffer(...);
  ctx.alloc().directBuffer(...);

To deallocate a buffer, use the unsafe free() operation:

  ((UnsafeByteBuf) buf).free();

The following is the list of the relevant changes:

- Add ChannelInboundHandler.freeInboundBuffer() and
  ChannelOutboundHandler.freeOutboundBuffer() to let a user free the
  buffer he or she allocated. ChannelHandler adapter classes implement
  is already, so most users won't need to call free() by themselves.
  freeIn/OutboundBuffer() methods are invoked when a Channel is closed
  and deregistered.

- All ByteBuf by contract must implement UnsafeByteBuf. To access an
  unsafe operation: ((UnsafeByteBuf) buf).internalNioBuffer()

- Replace WrappedByteBuf and ByteBuf.Unsafe with UnsafeByteBuf to
  simplify overall class hierarchy and to avoid unnecesary instantiation
  of Unsafe instances on an unsafe operation.

- Remove buffer reference counting which is confusing

- Instantiate SwappedByteBuf lazily to avoid instantiation cost

- Rename ChannelFutureFactory to ChannelPropertyAccess and move common
  methods between Channel and ChannelHandlerContext there. Also made it
  package-private to hide it from a user.

- Remove unused unsafe operations such as newBuffer()

- Add DetectionUtil.canFreeDirectBuffer() so that an allocator decides
  which buffer type to use safely
2012-11-22 15:10:59 +09:00
dantran
4107b08f29 Only generate OSGi manifest only at all-in-on sub module to reduce the complexity to the build 2012-11-19 06:27:18 +01:00
dantran
105f952f5d Clean up maven-bungle-plugin warnings 2012-11-12 11:42:42 +01:00
dantran
e236f5b77d [#154] [#727] Use maven-plugin-plugin to generate OSGi manifest 2012-11-12 09:15:36 +01:00
Trustin Lee
6f2840193a Fix inspection warnings related with JUnit usage 2012-11-12 12:45:06 +09:00
Trustin Lee
aedf8790c3 Fix various Javadoc issues / Do not use argN parameter names 2012-11-12 12:26:19 +09:00
Trustin Lee
ea4a0e3535 Prefer {@code ...} to <code>...</code> / Fix deprecation warnings 2012-11-12 11:51:23 +09:00
Trustin Lee
b195190b2a Remove an unused import 2012-11-12 11:26:29 +09:00
Trustin Lee
4dce19b814 Replace a variable with a constant wherever possible 2012-11-12 09:43:14 +09:00
Trustin Lee
aa7cd691df Remove redundant 'else' branches. 2012-11-12 09:31:40 +09:00
Trustin Lee
b4f796c5e3 Use 'x' over "x" wherever possible / String.equals("") -> isEmpty() 2012-11-10 08:03:52 +09:00
Trustin Lee
4e0f455e69 Remove methods overridden but identical with the super implementation / Make constructors of abstract classes protected rather than non-sense public
AbstractWrappedByteBuf.capacity(int) should raise a UnsupportedOperationException rather than ReadOnlyBufferException.
2012-11-10 07:10:30 +09:00
Trustin Lee
58ba0de659 Remove unnecessarily qualified static access 2012-11-10 01:32:21 +09:00
Trustin Lee
2ab38d8685 Remove pointless bitwise expressions 2012-11-09 17:26:11 +09:00
Norman Maurer
313f777491 [maven-release-plugin] prepare for next development iteration 2012-11-05 23:08:39 +01:00
Norman Maurer
57da8222a4 [maven-release-plugin] prepare release netty-4.0.0.Alpha7 2012-11-05 23:08:28 +01:00
Norman Maurer
d293e6c389 [#707] Hopefully now the correct fix for it which also takes the index into account when consolidate 2012-11-04 21:21:26 +01:00
Veebs
36ac52a4bd Port http multipart package. See #709 2012-11-04 13:59:50 +01:00
Norman Maurer
4e3258973f [#707] Fix IndexOutOfBoundsException which could accour when the buffer is consolitated in the middle of an operation 2012-11-03 20:00:48 +01:00
Norman Maurer
122051ba44 Only a few javadocs 2012-10-28 20:45:00 +01:00
Norman Maurer
87cc67306f [maven-release-plugin] prepare for next development iteration 2012-10-28 18:41:25 +01:00
Norman Maurer
7315490fca [maven-release-plugin] prepare release netty-4.0.0.Alpha6 2012-10-28 18:41:17 +01:00
Norman Maurer
afc687436a Revert "[maven-release-plugin] prepare release netty-4.0.0.Alpha6"
This reverts commit 95de4db0f1.
2012-10-28 18:36:15 +01:00
Norman Maurer
16eb4ec713 Revert "[maven-release-plugin] prepare for next development iteration"
This reverts commit e3e0776c20.
2012-10-28 18:35:47 +01:00
Norman Maurer
e3e0776c20 [maven-release-plugin] prepare for next development iteration 2012-10-28 13:06:07 +01:00
Norman Maurer
95de4db0f1 [maven-release-plugin] prepare release netty-4.0.0.Alpha6 2012-10-28 13:05:59 +01:00
Norman Maurer
cbcabaf29b Add support for method chaining to ByteBuf 2012-10-18 08:57:23 +02:00
Trustin Lee
820af50b63 [maven-release-plugin] prepare for next development iteration 2012-09-28 17:57:40 +09:00
Trustin Lee
595e1067c7 [maven-release-plugin] prepare release netty-4.0.0.Alpha5 2012-09-28 17:57:04 +09:00
Norman Maurer
b8ae8be96a Fix IndexOutOfBoundException when using CompositeChannelBuffer and the readerIndex is at the last position and an empty array is passed to read to. See #474 2012-09-22 21:23:58 +02:00
norman
3295145e88 [maven-release-plugin] prepare for next development iteration 2012-09-13 10:40:52 +02:00
norman
42685759de [maven-release-plugin] prepare release netty-4.0.0.Alpha4 2012-09-13 10:40:44 +02:00
Trustin Lee
f2538a996d [maven-release-plugin] prepare for next development iteration 2012-08-30 16:47:52 +09:00
Trustin Lee
628c5598b3 [maven-release-plugin] prepare release netty-4.0.0.Alpha3 2012-08-30 16:46:58 +09:00
Norman Maurer
5dbb799a92 Merge pull request #554 from jfallows/composite-buffer
Prevent Unpooled.EMPTY_BUFFER from being stored in the composite buffer ...
2012-08-23 22:26:03 -07:00
Trustin Lee
73720c422d [maven-release-plugin] prepare for next development iteration 2012-08-21 15:41:04 +09:00
Trustin Lee
68bef8cb99 [maven-release-plugin] prepare release netty-4.0.0.Alpha2 2012-08-21 15:40:45 +09:00
Trustin Lee
0a43350c66 [maven-release-plugin] prepare for next development iteration 2012-08-21 14:41:45 +09:00
Trustin Lee
56211fee59 [maven-release-plugin] prepare release netty-4.0.0.Alpha2 2012-08-21 14:39:59 +09:00
Trustin Lee
a0e34fd93a [maven-release-plugin] prepare for next development iteration 2012-08-21 14:13:38 +09:00
Trustin Lee
72ccf65093 [maven-release-plugin] prepare release netty-4.0.0.Alpha2 2012-08-21 14:13:17 +09:00
John Fallows
83c21d70ee Prevent Unpooled.EMPTY_BUFFER from being stored in the composite buffer due to a zero length slice, which causes reference counting problems on subsequent discardReadBytes. 2012-08-19 22:59:49 -07:00
Trustin Lee
df0aee22cb [#540] Add a variant of ByteBuf.ensureWritableBytes() which does not throw an exception 2012-08-20 12:52:38 +09:00
Trustin Lee
aa8a761fe2 Better exception message
.. as advised by @jpinner
2012-08-20 12:30:38 +09:00
Trustin Lee
505e767a09 [#529] ByteBuf.ensureWritableBytes() can trigger IllegalArgumentException
- Remove redundant boundary check in ensureWritableBytes()
- Ensure calculateNewCapacity() never returns the value that exceeds
  maxCapacity
2012-08-17 22:08:36 +09:00
John Fallows
06fd869711 Add support for AIO scattering reads and gathering writes.
o Add ByteBuf.hasNioBuffers() method
 o Promote CompositeByteBuf.nioBuffers() methods to ByteBuf
 o Use ByteBuf.nioBuffers() methods from AioSocketChannel
2012-08-12 17:29:07 -07:00
Trustin Lee
501746aeff Fix an infinite loop in DefaultCompositeByteBuf.setBytes() 2012-08-08 17:42:58 +09:00
Trustin Lee
a2aadef4da Add ByteBuf.Unsafe.discardSomeReadBytes() to reduce discardReadBytes() 2012-08-08 17:34:00 +09:00
norman
f7e0366bae Add nioBuffers() and nioBuffers(..) method which will be used to support gathering writes for the AIO transport. See #492 2012-08-07 15:39:46 +02:00
Trustin Lee
728306b64f Add CompositeByteBuf.consolidate()
- also added test cases that test automatic / full / ranged
  consolidation
2012-08-05 18:32:30 +09:00
Trustin Lee
26e64eb305 Fix IOOBE after buffer truncation / Add CompositeByteBuf.addComponents()
- plus some bug fixes while running unit tests
2012-07-20 13:18:17 +09:00
Trustin Lee
8d813b127c Replace free() with reference counting / Fix SlicedByteBuf.unsafe()
- based on @normanmaurer's feed back
- Added Unpooled.compositeBuffer(int)
2012-07-20 12:33:17 +09:00
Trustin Lee
5a613f379e Make ByteBuf dynamic / Introduce an interface for composite buffers
- Replace ByteBufferBackedByteBuf with DirectByteBuf
- Make DirectByteBuf and HeapByteBuf dynamic
- Remove DynamicByteBuf
- Replace Unpooled.dynamicBuffer() with Unpooled.buffer() and
  directBuffer()
- Remove ByteBufFactory (will be replaced with ByteBufPool later)
- Add ByteBuf.Unsafe (might change in the future)
2012-07-19 20:25:47 +09:00
Trustin Lee
d801459cb8 [maven-release-plugin] prepare for next development iteration 2012-07-10 23:11:33 +09:00
Trustin Lee
527f2f6c6e [maven-release-plugin] prepare release netty-4.0.0.Alpha1 2012-07-10 23:10:48 +09:00
Trustin Lee
917ed5173e Fix a bug where discardReadBytes() doesn't adjust markers
.. if buffer is empty
2012-07-10 14:23:31 +09:00
Cruz Julian Bishop
b9d16663d9 Fixes javadoc from #414 / #415 (@trustin) 2012-06-30 20:30:56 +10:00
Norman Maurer
27b83a480b Fix checkstyle 2012-06-29 12:49:58 +02:00
Norman Maurer
07095a41f4 Rename method and make it more clear thats an expert method. See #414 #415 2012-06-29 12:28:08 +02:00
Cruz Julian Bishop
c52cb0ea21 Assuming that @trustin will want this changed :) 2012-06-29 19:27:41 +10:00
Cruz Julian Bishop
c55e10c12a Fix a bug where a potential overflow occurs 2012-06-29 13:38:53 +10:00
Cruz Julian Bishop
b11d4fa37a Two tentative last asserts in the test 2012-06-29 13:37:41 +10:00
Cruz Julian Bishop
557f1c85df Little bit more testing 2012-06-29 13:35:14 +10:00
Cruz Julian Bishop
3d9be81377 Provide a basic test for getBufferFor() 2012-06-29 13:31:02 +10:00
Cruz Julian Bishop
e2989719e1 Comply with line width a bit more 2012-06-28 19:34:52 +10:00
Cruz Julian Bishop
a504aa1159 Documentation and checkstyle fixes from @fredericBregier
This is part of #414
2012-06-28 19:33:54 +10:00
Cruz Julian Bishop
42f3a6750a Add documentation and changes to ComposityByteBuf.getBufferFor(index)
Thanks to @kimchy and @fredericBregier
This is part of #414
2012-06-28 19:01:16 +10:00
Cruz Julian Bishop
49f0e32428 Actually throw the correct Exception type. Whoops!
This should be done, now.
2012-06-28 10:31:49 +10:00
Cruz Julian Bishop
332a35f9ed Simplify the process of getting the component 2012-06-28 10:29:11 +10:00
Cruz Julian Bishop
cb8c9767f6 Adds a method to get the buffer for a specific index in CompositeByteBuf
This fixes #414
2012-06-28 10:24:18 +10:00
Trustin Lee
1be9bbf62c Add CompositeByteBuf.numComponents() 2012-06-24 22:14:17 +09:00
Trustin Lee
8b7af2ca3b Generate Javadoc with APIviz 2012-06-13 12:11:11 +09:00
Trustin Lee
3b562c917d Move ChannelBufferType to netty-buffers and rename it to ChannelBufType
- Also add ChannelBuf.type()
2012-06-12 17:05:28 +09:00
Trustin Lee
ecd0ae5406 Prefer MessageBuf over Queue where possible
- Also replaced thread safe queues with non-thread-safe ones where
  possible
- Unpooled.wrappedBuffer(Queue<T>) does not wrap MessageBuf anymore
2012-06-12 17:02:00 +09:00
Trustin Lee
6211e53e86 Code clean-up based on IntelliJ code analysis 2012-06-11 22:54:28 +09:00
Trustin Lee
40a7659784 Move utility methods in Unpooled to ByteBufUtil 2012-06-11 20:41:19 +09:00
Trustin Lee
754392aaa9 Add ByteBuf.order(ByteOrder) method to simplify little endian access
- Removed all methods that requires ByteOrder as a parameter
  from Unpooled (formerly ByteBufs/ChannelBuffers)
  - Instead, a user calls order(ByteOrder) to get a little endian
    version of the user's buffer
  - This gives less overwhelming number of methods in Unpooled.
2012-06-11 20:24:44 +09:00
Trustin Lee
876847fd20 Merge MessageBufs and ByteBufs into Unpooled
- e.g. Unpooled.messageBuffer()
- It will make much more sense once we introduce pooling:
  - i.e. Pooled.buffer()
2012-06-11 17:02:29 +09:00
Trustin Lee
574d84e98e Remove ChannelBufferHolder / Add more handler interfaces for type safety
- ChannelInboundHandler and ChannelOutboundHandler does not have a type
  parameter anymore.  
- User should implement ChannelInboundMessageHandler or
  ChannelOutboundMessageHandler.
2012-06-10 12:22:32 +09:00
Trustin Lee
a849d11877 ChannelBuffers -> ByteBufs / Add MessageBuf & ChannelBuf
- Add MessageBuf which replaces java.util.Queue
- Add ChannelBuf which is common type of ByteBuf and ChannelBuf
- ChannelBuffers was renamed to ByteBufs
- Add MessageBufs
- All these changes are going to replace ChannelBufferHolder.
2012-06-10 11:31:39 +09:00
Trustin Lee
5164d91255 Rename ChannelBuffer to ByteBuf as discussed before
- ChannelBuffer gives a perception that it's a buffer of a
  channel, but channel's buffer is now a byte buffer or a message
  buffer.  Therefore letting it be as is is going to be confusing.
2012-06-10 11:08:43 +09:00
Trustin Lee
468a3228a4 Fit every line into 120 columns 2012-06-08 19:28:12 +09:00
Trustin Lee
c2e65016fd Fixed some checkstyle errors 2012-06-04 13:43:02 -07:00
Trustin Lee
1eced1e9e3 Update license headers 2012-06-04 13:31:44 -07:00
Trustin Lee
cc4f705029 Replace ChannelBuffer.toByteBuffer() with hasNioBuffer() and nioBuffer()
... just like we do with byte arrays.  toByteBuffer() and
toByteBuffers() had an indeterministic behavior and thus it could not
tell when the returned NIO buffer is shared or not.  nioBuffer() always
returns a view buffer of the Netty buffer.  The only case where
hasNioBuffer() returns false and nioBuffer() fails is the
CompositeChannelBuffer, which is not very commonly used and *slow*.
2012-06-02 01:30:55 -07:00
Trustin Lee
77274ae743 Automated code clean-up 2012-05-31 12:03:01 -07:00
Trustin Lee
c8fa42beaf Rename wrapPrimitive() to copyPrimitive() / Add tests / Tidy up (#167)
- Add ChannelBuffers.copyShort(short...)
2012-05-31 11:13:01 -07:00
Trustin Lee
429535e6c8 More primitive wrapper methods (#167)
- ChannelBuffers.wrap(Float|Double)(...)
- Handle the case where there is only one value.
2012-05-31 10:51:27 -07:00
Trustin Lee
c32df371ab Fix compilation errors 2012-05-31 01:55:30 -07:00
norman
7fb64e2046 A small optimization for the fix of #360 2012-05-30 23:06:06 -07:00
Sun Ning
f2df20ddff fix #360, add check for empty buffer; also add unit test for this scenario 2012-05-30 23:05:43 -07:00
Cruz Julian Bishop
f6f246cac0 Added messages to all IndexOutOfBoundsExceptions
I need to implement this to help myself finish more future pull requests which, so far, are plagued by these exceptions with no information available.
2012-05-30 19:25:45 -07:00
norman
a8d63a4ad7 Make sure CompositeChanneBuffer does not throw a UnsupportedOperationException if discardReadBytes() discard the whole content of the buffer. See #325 2012-05-30 19:09:41 -07:00
Trustin Lee
57d3d0cbb5 Update ChunkedWriteHandler to the latest revision at branch 3 2012-05-30 18:47:55 -07:00
Trustin Lee
b5d715343a Fail if DynamicChannelBuffer attempts to exceed the max capacity (#258)
- Contributed by @normanmaurer
2012-05-30 15:51:24 -07:00
Trustin Lee
e2d69120bb Ported OIO socket/datagram transport to the new API 2012-05-25 13:58:56 -07:00
Trustin Lee
02cb7adf03 Reduce GC in ChannelBuffer bulk operations
- Pre-create an NIO ByteBuffer for reuse instead of creating a new one
  every time
2012-05-25 06:10:10 -07:00
Trustin Lee
83026f29a4 Make EventLoop a ScheduledExecutorService
- SingleThreadEventLoop now implements ScheduledExecutorService
  - Scheduled tasks are automatically fetched into taskQueue by
    pollTask() and takeTask()
- Removed MapBackedSet because Java 6 provides it
2012-05-11 20:19:57 +09:00
Trustin Lee
129a2af86a Initial working version of the echo server example
- Optimized AbstractChannelBuffer.discardReadBytes()
- Split ChannelHandlerInvoker into ChannelInboundInvoker and
  ChannelOutboundInvoker
  - Channel implements ChannelOutboundInvoker
  - ChannelOutboundInvoker.nextOut() is now out()
  - ChannelOutboundHandlerContext.out() is now prevOut()
  - Added the outbound operations without future
    parameter to ChannelOutboundInvoker for user convenience
- All async operations which requires a ChannelFuture as a parameter
  now returns ChannelFuture for user convenience
- Added ChannelFutureFactory.newVoidFuture() to allow a user specify
  a dummy future that is of no use
  - I'm unsure if it is actually a good idea to introduce it. It might
    go away later.
- Made the contract of AbstractChannel.doXXX() much simpler and moved
  all common code up to AbstractChannel.DefaultUnsafe
- Added Channel.isOpen()
- Fixed a bug where MultithreadEventLoop always shut down its child
  event loops on construction
- Maybe more changes I don't remember :-)
2012-05-09 22:09:06 +09:00
Trustin Lee
fd0b0a4e2b Code cleanup 2012-03-30 12:48:28 +09:00
Trustin Lee
c28137b0b2 Improve ChannelBufferInputStream documentation 2012-03-26 18:10:16 +09:00
Norman Maurer
2b9df060dd Add support to wrap primitives via ChannelBuffers.wrap*(..) easily. See
#167
2012-02-26 20:51:53 +01:00
Trustin Lee
303c1b5f79 Overall cleanup / Add lost old jzlib headers 2012-01-13 17:41:18 +09:00
Trustin Lee
ebfc4513e0 Apply checkstyle to the build
Please note that the build will fail at the moment due to various checkstyle
violations which should be fixed soon
2012-01-11 20:16:14 +09:00
Trustin Lee
8663716d38 Issue #60: Make the project multi-module
Split the project into the following modules:
* common
* buffer
* codec
* codec-http
* transport
* transport-*
* handler
* example
* testsuite (integration tests that involve 2+ modules)
* all (does nothing yet, but will make it generate netty.jar)

This commit also fixes the compilation errors with transport-sctp on
non-Linux systems.  It will at least compile without complaints.
2011-12-28 19:44:04 +09:00