Commit Graph

5873 Commits

Author SHA1 Message Date
Trustin Lee
d0bc3be79c Add missing m2eclipse life cycle mapping
Also, use ignore instead of execution because those plugins are not
really useful when building from Eclipse anyway.
2014-07-06 16:53:53 +09:00
Trustin Lee
0623df12a5 Log the time taken for generating the initialSeedUniquifier
- Sometimes useful to know it how long it takes from the log, to make
  sure it's not something else that is blocking.
2014-07-04 13:27:18 +09:00
Norman Maurer
cd688c5cde [#2586] Use correct EventLoop to notify delayed bind failures
Motivation:

When a bind fails AbstractBootstrap will use the GlobalEventExecutor to notify the ChannelPromise. We should use the EventLoop of the Channel if possible.

Modification:

Use EventLoop of the Channel if possible to use the correct Thread to notify and so guaranteer the right order of events.

Result:

Use the correct EventLoop for notification
2014-07-03 21:24:15 +02:00
Trustin Lee
1464abe765 Let OkResponseHandler extend SimpleChannelInboundHandler
Motivation:

OkResponseHandler is the last handler in the pipeline of the HTTP CORS
example.  It is responsible for releasing all messages it handled.

Modification:

Extend SimpleChannelInboundHandler instead of
ChannelInboundHandlerAdapter

Result:

Fixed a leak
2014-07-03 18:13:03 +09:00
Trustin Lee
3d8d843198 Fix the build timeout when 'leak' profile is active
Motivation:

AbstractByteBufTest.testInternalBuffer() uses writeByte() operations to
populate the sample data.  Usually, this isn't a problem, but it starts
to take a lot of time when the resource leak detection level gets
higher.

In our CI machine, testInternalBuffer() takes more than 30 minutes,
causing the build timeout when the 'leak' profile is active (paranoid
level resource detection.)

Modification:

Populate the sample data using ThreadLocalRandom.nextBytes() instead of
using millions of writeByte() operations.

Result:

Test runs much faster when leak detection level is high.
2014-07-03 17:55:18 +09:00
Trustin Lee
0a8ff3b52d Fix most inspector warnings
Motivation:

It's good to minimize potentially broken windows.

Modifications:

Fix most inspector warnings from our profile
Update IntObjectHashMap

Result:

Cleaner code
2014-07-02 20:21:30 +09:00
Norman Maurer
893bc04eee [#2558] Define SO_REUSEPORT if not defined
Motivation:

Currently it is impossible to build netty on linux system that not define SO_REUSEPORT even if it is supported.

Modification:

Define SO_REUSEPORT if not defined.

Result:

Possible to build on more linux dists.
2014-07-02 09:39:27 +02:00
Norman Maurer
016bdfbf66 Correctly return from selector loop one a scheduled task is ready for processing
Motivation:

We use the nanoTime of the scheduledTasks to calculate the milli-seconds to wait for a select operation to select something. Once these elapsed we check if there was something selected or some task is ready for processing. Unfortunally we not take into account scheduled tasks here so the selection loop will continue if only scheduled tasks are ready for processing. This will delay the execution of these tasks.

Modification:

- Check if a scheduled task is ready after selecting
- also make a tiny change in NioEventLoop to not trigger a rebuild if nothing was selected because the timeout was reached a few times in a row.

Result:

Execute scheduled tasks on time.
2014-07-02 08:31:47 +02:00
Norman Maurer
e8f4def2a3 [maven-release-plugin] prepare for next development iteration 2014-06-30 14:31:08 +02:00
Norman Maurer
25e3c8ce3d [maven-release-plugin] prepare release netty-4.0.21.Final 2014-06-30 14:29:15 +02:00
Norman Maurer
8427aa723a [#2623] Release local references to guard against StackOverflow in JNI
Motivation:

When we do a (env*)->GetObjectArrayElement(...) call we may created many local references which will only be cleaned up once we exist the native method. Thus a lot of memory can be used and so a StackOverFlow may be triggered. Beside this the JNI specification only say that an implementation must cope with 16 local references.

Modification:

Call (env*)->ReleaseLocalRef(...) to release the resource once not needed anymore.

Result:

Less memory usage and guard against StackOverflow
2014-06-30 10:40:52 +02:00
Norman Maurer
6c47cc9711 [#2622] Correctly check reference count before try to work on the underlying memory
Motivation:

Because of how we use reference counting we need to check for the reference count before each operation that touches the underlying memory. This is especially true as we use sun.misc.Cleaner.clean() to release the memory ASAP when possible. Because of this the user may cause a SEGFAULT if an operation is called that tries to access the backing memory after it was released.

Modification:

Correctly check the reference count on all methods that access the underlying memory or expose it via a ByteBuffer.

Result:

Safer usage of ByteBuf
2014-06-30 07:10:12 +02:00
Jay
0332fd1589 Maintain decoder result in HttpObjectAggregator
Motivation:
DecodeResult is dropped when aggregate HTTP messages.

Modification:

Make sure we not drop the DecodeResult while aggregate HTTP messages.

Result:

Correctly include the DecodeResult for later processing.
2014-06-28 21:47:49 +02:00
xmxsuperstar
cb1143ec20 fix example missing break statement in ReplayingDecoder 2014-06-28 21:42:46 +02:00
Norman Maurer
3cdfe95608 Cleanup comment / code 2014-06-27 21:35:54 +02:00
Norman Maurer
384ba32b56 Let EpollReuseAddrTest also work with kernel versions that not have bugfix release part 2014-06-27 17:56:31 +02:00
Norman Maurer
ac348956e3 [#2615] Correctly update SelectionKey after selector rebuild
Motivation:

When a select rebuild was triggered the reference to the SelectionKey is not updated in AbstractNioChannel. This will cause a CancelledKeyException later.

Modification:

Correctly update SelectionKey reference after rebuild

Result:

Fix exception
2014-06-27 17:09:27 +02:00
Trustin Lee
ff5b15acbd Upgrade JACOCO to the latest version
.. to fix the build errors triggered by the old JACOCO release
2014-06-27 17:23:03 +09:00
Trustin Lee
4f82b6e393 Enable verbose:gc only when 'leak' profile is disabled
The leak profile generates a lot of garbage which makes the verbose:gc
output too large.
2014-06-27 17:10:45 +09:00
Trustin Lee
7b2d2e22ba Remove the deprecated methods in SPDY
SPDY is an experimental package. No need to keep ABI compatibility.
2014-06-27 16:46:45 +09:00
Trustin Lee
2bdcedeffd Partially revert 4a13f66e13
Revert the removal of 'get' prefix from HTTP classes to ensure ABI
compatibility.  Note that this commit does not revert the changes in
SPDY, which is considered experimental.
2014-06-27 16:40:43 +09:00
Norman Maurer
522b3e1b92 [#2604] Not try to use sun.misc.Cleaner when on android
Motivation:

When a user tries to use netty on android it currently fails with "Could not find class 'sun.misc.Cleaner'"

Modification:

Encapsulate sun.misc.Cleaner usage in extra class to workaround this isssue.

Result:
Netty can be used on android again
2014-06-27 08:18:21 +02:00
Zhihui Jiao
d062500ca0 Fix inconsistent code in the doc 2014-06-27 06:48:47 +02:00
Norman Maurer
8869f6d5c1 [#2598] Add Epoll.isAvailable() which allows to check if epoll can be used.
Motivation:

At the moment there is no simple way for a user to check if the native epoll transport can be used on the running platform. Thus the user can only try to instance it and catch any exception and fallback to nio transport.

Modification:

Add Epoll.isAvailable() which allows to check if epoll can be used.

Result:

User can easily check if epoll transport can be used or not
2014-06-26 12:27:40 +02:00
Trustin Lee
a5097b937e Fix incorrect bytesBefore/indexOf() in ReplayingDecoderBuffer
Motivation:

bytesBefore(length, ...), bytesBefore(index, length, ...), and
indexOf(fromIndex, toIndex,...) in ReplayingDecoderBuffer are buggy.
They trigger 'REPLAY even when they don't need to.

Modification:

Implement the buggy methods properly so that REPLAYs are not triggered
unnecessarily.

Result:

Correct behvaior
2014-06-26 18:57:20 +09:00
Norman Maurer
d1c8bcb40f [#2605] Use SO_REUSEADDR on EpollServerSocketChannel to match defaults of java.nio.ServerSocketChannel impl
Motivation:

When using openjdk and oracle jdk's nio (while using the nio transport) the ServerSocketChannel uses SO_REUSEADDR by default. Our native transport should do the same to make it easier to switch between the different implementations and get the expected result.

Modification:

Change EpollServerSocketChannelConfig to set SO_REUSEADDR on the created socket.

Result:

SO_REUSEADDR is used by default on servers.
2014-06-26 11:53:23 +02:00
Norman Maurer
658fdffbad Reduce the memory copies in JdkZlibEncoder
Motivation:

At the moment we use a lot of unnecessary memory copies in JdkZlibEncoder. This is caused by either allocate a to small ByteBuf and expand it later or using a temporary byte array.
Beside this the memory footprint of JdkZlibEncoder is pretty high because of the byte[] used for compressing.

Modification:

- Override allocateBuffer(...) and calculate the estimatedsize in there, this reduce expanding of the ByteBuf later
- Not use byte[] in the instance itself but allocate a heap ByteBuf and write directly into the byte array

Result:

Less memory copies and smaller memory footprint
2014-06-26 11:12:06 +02:00
Trustin Lee
d8d0bbfc26 Optimize PoolChunk
- Using short[] for memoryMap did not improve performance. Reverting
  back to the original dual-byte[] structure in favor of simplicity.
- Optimize allocateRun() which yields small performence improvement
- Use local variable when member fields are accessed more than once
2014-06-26 17:06:29 +09:00
Trustin Lee
c41538050c Fix inspector warnings 2014-06-26 17:06:29 +09:00
Pavan Kumar
d2e36a49c7 Improve the allocation algorithm in PoolChunk
Motivation:

Depth-first search is not always efficient for buddy allocation.

Modification:

Employ a new faster search algorithm with different memoryMap layout.

Result:

With thread-local cache disabled, we see a lot of performance
improvment, especially when the size of the allocation is as small as
the page size, which had the largest search space previously.
2014-06-26 17:06:29 +09:00
Norman Maurer
c3f24444ef Use IntObjectMap to replace Map in EpollEventLoop.
Motivation:

We need to map from ints to AbstractEpollChannel in EpollEventLoop but there is no need for box to Integer.

Modification:

Replace Map with IntObjectMap.

Result:

No more auto-boxing needed.
2014-06-25 20:23:16 +02:00
Norman Maurer
b4b61c1f41 [#2599] Not use sun.nio.ch.DirectBuffer as it not exists on android
Motivation:

During some refactoring we changed PlatformDependend0 to use sun.nio.ch.DirectBuffer for release direct buffers. This broke support for android as the class does not exist there and so an exception is thrown.

Modification:

Use again the fieldoffset to get access to Cleaner for release direct buffers.

Result:
Netty can be used on android again
2014-06-25 15:01:57 +02:00
Alexey Parfenov
063ca10d87 Fix integer overflow in HttpObjectEncoder when handling chunked encoding and FileRegion > Integer.MAX_VALUE
Motivation:

Due to integer overflow bug, writes of FileRegions to http server pipeline (eg like one from HttpStaticFileServer example) with length greater than Integer.MAX_VALUE are ignored in 1/2 of cases (ie no data gets sent to client)

Modification:

Correctly handle chunk sized > Integer.MAX_VALUE

Result:

Be able to use FileRegion > Integer.MAX_VALUE when using chunked encoding.
2014-06-24 12:14:46 +02:00
Trustin Lee
4a13f66e13 Remove 'get' prefix from all HTTP/SPDY messages
Motivation:

Persuit for the consistency in method naming

Modifications:

- Remove the 'get' prefix from all HTTP/SPDY message classes
- Fix some inspector warnings

Result:

Consistency
Fixes #2594
2014-06-24 18:33:30 +09:00
onlychoice
30e22f5da3 Fix a typo in comment 2014-06-24 11:02:12 +02:00
Trustin Lee
7457801020 Deprecate SocksMessage.encodeAsByteBuf()
It was an internal use only method which became public by a mistake
during the review process.
2014-06-24 16:41:47 +09:00
Trustin Lee
71dce0193f Rename fromByte() to valueOf()
Motivation:

Persuit the consistency in method naming

Modifications:

Rename fromByte(byte) to valueOf(byte)

Result:

Consistency
2014-06-24 16:36:10 +09:00
Norman Maurer
790c63e8d2 Improve performance of Recycler
Motivation:

Recycler is used in many places to reduce GC-pressure but is still not as fast as possible because of the internal datastructures used.

Modification:

 - Rewrite Recycler to use a WeakOrderQueue which makes minimal guaranteer about order and visibility for max performance.
 - Recycling of the same object multiple times without acquire it will fail.
 - Introduce a RecyclableMpscLinkedQueueNode which can be used for MpscLinkedQueueNodes that use Recycler

These changes are based on @belliottsmith 's work that was part of #2504.

Result:

Huge increase in performance.

4.0 branch without this commit:

Benchmark                                                (size)   Mode   Samples        Score  Score error    Units
i.n.m.i.RecyclableArrayListBenchmark.recycleSameThread    00000  thrpt        20 116026994.130  2763381.305    ops/s
i.n.m.i.RecyclableArrayListBenchmark.recycleSameThread    00256  thrpt        20 110823170.627  3007221.464    ops/s
i.n.m.i.RecyclableArrayListBenchmark.recycleSameThread    01024  thrpt        20 118290272.413  7143962.304    ops/s
i.n.m.i.RecyclableArrayListBenchmark.recycleSameThread    04096  thrpt        20 120560396.523  6483323.228    ops/s
i.n.m.i.RecyclableArrayListBenchmark.recycleSameThread    16384  thrpt        20 114726607.428  2960013.108    ops/s
i.n.m.i.RecyclableArrayListBenchmark.recycleSameThread    65536  thrpt        20 119385917.899  3172913.684    ops/s
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 297.617 sec - in io.netty.microbench.internal.RecyclableArrayListBenchmark

4.0 branch with this commit:

Benchmark                                                (size)   Mode   Samples        Score  Score error    Units
i.n.m.i.RecyclableArrayListBenchmark.recycleSameThread    00000  thrpt        20 204158855.315  5031432.145    ops/s
i.n.m.i.RecyclableArrayListBenchmark.recycleSameThread    00256  thrpt        20 205179685.861  1934137.841    ops/s
i.n.m.i.RecyclableArrayListBenchmark.recycleSameThread    01024  thrpt        20 209906801.437  8007811.254    ops/s
i.n.m.i.RecyclableArrayListBenchmark.recycleSameThread    04096  thrpt        20 214288320.053  6413126.689    ops/s
i.n.m.i.RecyclableArrayListBenchmark.recycleSameThread    16384  thrpt        20 215940902.649  7837706.133    ops/s
i.n.m.i.RecyclableArrayListBenchmark.recycleSameThread    65536  thrpt        20 211141994.206  5017868.542    ops/s
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 297.648 sec - in io.netty.microbench.internal.RecyclableArrayListBenchmark
2014-06-24 08:09:19 +02:00
Norman Maurer
22f16e52bf MessageToByteEncoder always starts with ByteBuf that use initalCapacity == 0
Motivation:

MessageToByteEncoder always starts with ByteBuf that use initalCapacity == 0 when preferDirect is used. This is really wasteful in terms of performance as every first write into the buffer will cause an expand of the buffer itself.

Modifications:

 - Change ByteBufAllocator.ioBuffer() use the same default initialCapacity as heapBuffer() and directBuffer()
 - Add new allocateBuffer method to MessageToByteEncoder that allow the user to do some smarter allocation based on the message that will be encoded.

Result:

Less expanding of buffer and more flexibilty when allocate the buffer for encoding.
2014-06-24 13:55:02 +09:00
Norman Maurer
f0e2d0b77c Make use of HttpChunkedInput as this will also work when compression is used 2014-06-23 09:38:31 +02:00
Trustin Lee
7162d96ed5 Revert "Improve the allocation algorithm in PoolChunk"
This reverts commit 36305d7dce, which
seems to cause an assertion failure on our CI machine.
2014-06-21 19:19:49 +09:00
Trustin Lee
a1b87411fb Make sure OpenSslEngine is tested against transport-native-epoll 2014-06-21 18:29:00 +09:00
Trustin Lee
3775a6124e Remove padding utility classes
- It's not used anywhere
2014-06-21 17:59:20 +09:00
Trustin Lee
6834c2da51 Add missing last padding / Comment 2014-06-21 17:57:29 +09:00
Trustin Lee
b1a5ced729 Checkstyle / Overall clean-up / Fix serialization 2014-06-21 17:57:29 +09:00
nitsanw
a5d585a8ef Fix false sharing between head and tail reference in MpscLinkedQueue
Motivation:

The tail node reference writes (by producer threads) are very likely to
invalidate the cache line holding the headRef which is read by the
consumer threads in order to access the padded reference to the head
node. This is because the resulting layout for the object is:

- header
- Object AtomicReference.value -> Tail node
- Object MpscLinkedQueue.headRef -> PaddedRef -> Head node

This is 'passive' false sharing where one thread reads and the other
writes.  The current implementation suffers from further passive false
sharing potential from any and all neighbours to the queue object as no
pre/post padding is provided for the class fields.

Modifications:

Fix the memory layout by adding pre-post padding for the head node and
putting the tail node reference in the same object.

Result:

Fixed false sharing
2014-06-21 17:57:29 +09:00
nmittler
fd895e53f4 Adding int-to-object map implementation.
Motivation:

Maps with integer keys are used in several places (HTTP/2 code, for
example). To reduce the memory footprint of these structures, we need a
specialized map class that uses ints as keys.

Modifications:

Added IntObjectHashMap, which is uses open addressing and double hashing
for collision resolution.

Result:

A new int-based map class that can be shared across Netty.
2014-06-21 08:36:06 +02:00
Trustin Lee
cb994dd926 Fix the inconsistencies between performance tests in ByteBufAllocatorBenchmark
Motivation:

default*() tests are performing a test in a different way, and they must be same with other tests.

Modification:

Make sure default*() tests are same with the others

Result:

Easier to compare default and non-default allocators
2014-06-21 13:28:11 +09:00
Pavan Kumar
004ffbad90 Improve the allocation algorithm in PoolChunk
Motivation:

Depth-first search is not always efficient for buddy allocation.

Modification:

Employ a new faster search algorithm with different memoryMap layout.

Result:

With thread-local cache disabled, we see a lot of performance
improvment, especially when the size of the allocation is as small as
the page size, which had the largest search space previously:

-- master head --
Benchmark                (size) Mode    Score  Error Units
pooledDirectAllocAndFree  8192 thrpt  215.392  1.565 ops/ms
pooledDirectAllocAndFree 16384 thrpt  594.625  2.154 ops/ms
pooledDirectAllocAndFree 65536 thrpt 1221.520 18.965 ops/ms
pooledHeapAllocAndFree    8192 thrpt  217.175  1.653 ops/ms
pooledHeapAllocAndFree   16384 thrpt  587.250 14.827 ops/ms
pooledHeapAllocAndFree   65536 thrpt 1217.023 44.963 ops/ms

-- changes --
Benchmark                (size) Mode    Score  Error Units
pooledDirectAllocAndFree  8192 thrpt 3656.744 94.093 ops/ms
pooledDirectAllocAndFree 16384 thrpt 4087.152 22.921 ops/ms
pooledDirectAllocAndFree 65536 thrpt 4058.814 29.276 ops/ms
pooledHeapAllocAndFree    8192 thrpt 3640.355 44.418 ops/ms
pooledHeapAllocAndFree   16384 thrpt 4030.206 24.365 ops/ms
pooledHeapAllocAndFree   65536 thrpt 4103.991 70.991 ops/ms
2014-06-21 13:20:56 +09:00
Norman Maurer
58f4b4b7d9 [#2589] LocalServerChannel.doClose() throws NPE when localAddress == null
Motivation:

LocalServerChannel.doClose() calls LocalChannelRegistry.unregister(localAddress); without check if localAddress is null and so produce a NPE when pass null the used ConcurrentHashMapV8

Modification:
Check for localAddress != null before try to remove it from Map. Also added a unit test which showed the stacktrace of the error.

Result:

No more NPE during doClose().
2014-06-20 20:07:00 +02:00