Commit Graph

2114 Commits

Author SHA1 Message Date
Nick Hill
ba140ac6bc Fix SimpleChannelPoolTest#testCloseAsync() test flake (#9943)
Motivation

This test is failing intermittently and upon inspection has a race
condition.

Modification

Fix race condition by waiting for async release calls to complete prior
to closing the pool.

Result

Hopefully fixed flakey test
2020-01-11 06:03:45 +01:00
root
79d4e74019 [maven-release-plugin] prepare for next development iteration 2019-12-18 08:32:54 +00:00
root
5ddf45a2d5 [maven-release-plugin] prepare release netty-4.1.44.Final 2019-12-18 08:31:43 +00:00
Scott Mitchell
e7091c04d7 Channel Pool LocalAddress initialization and cleanup (#9879)
Motivation:
The Channel Pool tests commonly use the same fixed local address String. This
has been observed to result in test failures if a single test fails and cleanup
is not done properly, or if tests are run in parallel. Also each test should
close any channel pool objects or maps to make sure resources are reclaimed.

Modifications:
- Use a random string for the address on each test to reduce the chance of
  collision.
- close all ChannelPool and ChannelPoolMap objects at the end of each test

Result:
Less likely to observe spurious failures due to LocalAddress collisions and more
complete test cleanup.
2019-12-13 13:45:22 +01:00
Robert Mihaly
af58bfcfb8 Synchronously close pools when closing AbstractChannelPoolMap (#9857)
Motivation:

In #9830 the get/remove/close methods implementation changed to avoid
deadlocks on event loops. The change involved modifying the methods to
close the managed ChannelPools asynchronously and return immediately.
While this behavior might be fine for get/remove, it is changing what
a user expects from a close() method and after returning from close()
there might be still resources open.

Modifications:

This change is a follow-up for #9830 to preserve the synchronous
behavior of the AbstractChannelPoolMap#close() method.

Result:

AbstractChannelPoolMap#close() returns once the managed pools have
been closed.
2019-12-11 09:28:44 +01:00
梦典
66fca3b58e Update ChannelOutboundBuffer.java (#9863)
Motivation:

fix comment of ChannelOutboundBuffer.CHANNEL_OUTBOUND_BUFFER_ENTRY_OVERHEAD

Modifications:

6 (NOT 8) reference fields is right

Result:
Correct comment
2019-12-11 09:26:21 +01:00
时无两丶
0cde4d9cb4 Uniform null pointer check. (#9840)
Motivation:
Uniform null pointer check.

Modifications:

Use ObjectUtil.checkNonNull(...)

Result:
Less code, same result.
2019-12-09 09:47:35 +01:00
Norman Maurer
e69c4173ea
Replace synchronized with ConcurrentHashMap in Http2StreamChannelBootstrap (#9848)
Motivation:

97361fa2c8 replace synchronized with ConcurrentHashMap in *Bootstrap classes but missed to do the same for the Http2 variant.

Modifications:

- Use ConcurrentHashMap
- Simplify code in *Bootstrap classes

Result:

Less contention
2019-12-06 10:59:55 +01:00
Robert Mihaly
a322addb17 Asynchronously close pools in AbstractChannelPoolMap (#8238) (#9830)
Motivation:

In certain scenarios mutliple concurrent AbstractChannelPoolMap
operations might be called from event loops that handle also
ChannelPool close operations. If the map uses synchronous close
it could end up blocking the event loop and if multiple threads
are waiting for each other a deadlock might occur.

Modifications:

Previously #9226 introduced a closeAsync operation for
FixedChannelPool, which is now extended to SimpleChannelPool class.
The AbstractChannelPoolMap now uses the closeAsync operations when
closing redundant or removed SimpleChannelPool instances.

Result:

The AbstractChannelPoolMap get/remove operations will not wait
until the pools are closed as they will happen asynchronously and
avoid situations that could cause the event loop being blocked in
deadlocks.
2019-12-05 09:14:10 +01:00
Nick Hill
6d0c0e991b Small simplification to WriteTask optimization (#9807)
Motiviation

#9800 was just merged which consolidates the flush/no-flush WriteTasks
in AbstractChannelHandlerContext, but after looking at the changes again
I noticed a tiny simplification that would be good to make imo.

Modification

Remove use of conditional operator in decrementPendingOutboundBytes()

Result

Simpler code, one less branch
2019-11-27 06:55:22 +01:00
Nick Hill
e208e96f12 Clean up NioEventLoop (#9799)
Motivation

The event loop implementations had become somewhat tangled over time and
work was done recently to streamline EpollEventLoop. NioEventLoop would
benefit from the same treatment and it is more straighforward now that
we can follow the same structure as was done for epoll.

Modifications

Untangle NioEventLoop logic and mirror what's now done in EpollEventLoop
w.r.t. the volatile selector wake-up guard and scheduled task deadline
handling.

Some common refinements to EpollEventLoop have also been included - to
use constants for the "special" deadline/wakeup volatile values and to
avoid some unnecessary calls to System.nanoTime() on task-only
iterations.

Result

Hopefully cleaner, more efficient and less fragile NIO transport
implementation.
2019-11-26 08:25:59 +01:00
Nick Hill
b413464091 Optimize WriteTask recycling (#9800)
Motivation

AbstractChannelHandlerContext uses recyclable tasks when performing
writes from outside of the event loop. There's currently two distinct
classes WriteTask and WriteAndFlushTask used for executing writes versus
writeAndFlushes, and these are recycled in separate pools. However it is
straightforward to just have a single class / recycler pool, with a
flush flag.

Modifications

- Unify WriteTasks into a single class using the sign bit of the
existing size field to indicate whether a flush should be performed
- Use the new executor lazyExecute() method to lazily execute the
non-flush write tasks explicitly
- Change AbstractChannelHandlerContext#invokeWrite and
AbstractChannelHandlerContext#invokeWriteAndFlush from private to
package-private to avoid synthetic methods
- Correct the default object size estimate for WriteTask

Results

- Possibly improved reuse of recycled write tasks
- Fewer virtual method calls and shorter path lengths
- Less code
2019-11-23 20:45:35 +01:00
stroller
aa2a9931e8 Add one new constructor with threadFactory only (#9773)
Motivation:
In most cases, we want to use MultithreadEventLoopGroup such as NioEventLoopGroup without setting thread numbers but thread name only. So we need to use followed code:
NioEventLoopGroup boss = new NioEventLoopGroup(0, new DefaultThreadFactory("boss"));
It looks a bit confuse or strange for the number 0 due to we only want to set thread name. So it will be better to add new constructor for this case.

Modifications:
add new constructor into all event loop groups, for example: public NioEventLoopGroup(ThreadFactory threadFactory)

Result:
User can only set thread factory without setting the thread number to 0:
NioEventLoopGroup boss = new NioEventLoopGroup(new DefaultThreadFactory("boss"));
2019-11-18 09:42:44 +01:00
Matthew Miller
394e7f0287 Prevent memory leak when SimpleChannelPools are destroyed (#9760)
Motivation:

https://github.com/netty/netty/pull/9548 introduced a change that creates a new AttributeKey
for each SimpleChannelPool instance created. AttributeKeys are cached statically in a ConstantPool
by the AttributeKey.newInstance method. Because of this, creating a SimpleChannelPool instance will
allocate memory that will never be released, even after the SimpleChannelPool is closed.

Modifications:

This change goes back to a single AttributeKey per SimpleChannelPool, just using a more specific
name to reduce the chance of conflicts with user code.

Result:

No memory is leaked after a SimpleChannelPool instance is created and destroyed.
2019-11-07 14:48:56 +01:00
Nick Hill
a18c57ea87 Externalize lazy execution semantic for EventExecutors (#9587)
Motivation

This is already done internally for various reasons but it would make
sense i.m.o. as a top level concept: submitting a task to be run on the
event loop which doesn't need to run immediately but must still be
executed in FIFO order relative all other submitted tasks (be those
"lazy" or otherwise).

It's nice to separate this abstract "relaxed" semantic from concrete
implementations - the simplest is to just delegate to existing execute,
but for the main EL impls translates to whether a wakeup is required
after enqueuing.

Having a "global" abstraction also allows for simplification of our
internal use - for example encapsulating more of the common scheduled
future logic within AbstractScheduledEventExecutor.

Modifications

- Introduce public LazyRunnable interface and
AbstractEventExecutor#lazyExecute method (would be nice for this to be
added to EventExecutor interface in netty 5)
- Tweak existing SingleThreadEventExecutor mechanics to support these
- Replace internal use of NonWakeupRunnable (such as for pre-flush
channel writes)
- Uplift scheduling-related hooks into AbstractScheduledEventExecutor,
eliminating intermediate executeScheduledRunnable method

Result

Simpler code, cleaner and more useful/flexible abstractions - cleaner in
that they fully communicate the intent in a more general way, without
implying/exposing/restricting implementation details
2019-10-31 10:01:53 +01:00
SimpleYoung
82376fd889 Remove an unnecessary expression in NioEventLoop (#9729)
Motivation:

Remove an unnecessary expression to make the code more readable

Modification:

Remove `eventLoop == null`

Result:

Fixes #9725
2019-10-30 12:24:03 +01:00
Norman Maurer
6c061abc49
Hide Recycler implemention to allow experimenting with different implementions of an Object pool (#9715)
Motivation:

At the moment we directly extend the Recycler base class in our code which makes it hard to experiment with different Object pool implementation. It would be nice to be able to switch from one to another by using a system property in the future. This would also allow to more easily test things like https://github.com/netty/netty/pull/8052.

Modifications:

- Introduce ObjectPool class with static method that we now use internally to obtain an ObjectPool implementation.
- Wrap the Recycler into an ObjectPool and return it for now

Result:

Preparation for different ObjectPool implementations
2019-10-26 00:34:30 -07:00
root
844b82b986 [maven-release-plugin] prepare for next development iteration 2019-10-24 12:57:00 +00:00
root
d066f163d7 [maven-release-plugin] prepare release netty-4.1.43.Final 2019-10-24 12:56:30 +00:00
Norman Maurer
6c05d16967
Use @SuppressJava6Requirement for animal sniffer plugin to ensure we always guard correctly (#9655)
Motivation:

We can use the `@SuppressJava6Requirement` annotation to be more precise about when we use Java6+ APIs. This helps us to ensure we always protect these places.

Modifications:

Make use of `@SuppressJava6Requirement` explicit

Result:

Fixes https://github.com/netty/netty/issues/2509.
2019-10-14 15:54:49 +02:00
stroller
deb34a0f8b remove useless doc for 5.0 (#9546)
Fix javadoc issues

Motivation:

Remove useless doc for 5.0

Modifications:

Remove description for 5.0

Result:

Useless doc will be removed.
2019-10-12 20:17:18 +02:00
Nick Hill
99a4cecec6 Fix NioEventLoopTest#testChannelsRegistered flakiness (#9650)
Motivation

NioEventLoopTest#testChannelsRegistered() fails intermittently due to
use of SingleThreadEventLoop#channelsRegistered() which is not
threadsafe and unreliable when called from outside the event loop.

Modifications

Add static registeredChannels method to NioEventLoopTest and
AbstractSingleThreadEventLoopTest to call from the tests via event loop
instead of directly.

Result

Hopefully fewer test failures
2019-10-09 16:57:54 +04:00
Nick Hill
170e4deee6 Fix event loop shutdown timing fragility (#9616)
Motivation

The current event loop shutdown logic is quite fragile and in the
epoll/NIO cases relies on the default 1 second wait/select timeout that
applies when there are no scheduled tasks. Without this default timeout
the shutdown would hang indefinitely.

The timeout only takes effect in this case because queued scheduled
tasks are first cancelled in
SingleThreadEventExecutor#confirmShutdown(), but I _think_ even this
isn't robust, since the main task queue is subsequently serviced which
could result in some new scheduled task being queued with much later
deadline.

It also means shutdowns are unnecessarily delayed by up to 1 second.

Modifications

- Add/extend unit tests to expose the issue
- Adjust SingleThreadEventExecutor shutdown and confirmShutdown methods
to explicitly add no-op tasks to the taskQueue so that the subsequent
event loop iteration doesn't enter blocking wait (as looks like was
originally intended)

Results

Faster and more robust shutdown of event loops, allows removal of the
default wait timeout
2019-10-07 11:06:01 +04:00
Codrut Stancu
4980a6b304 Register sun.nio.ch.SelectorImpl fields for unsafe access. (#9631)
Motivation:

On JDK > 9 Netty uses Unsafe to write two internal JDK fields: sun.nio.ch.SelectorImp.selectedKeys and sun.nio.ch.SelectorImpl.publicSelectedKeys. This is done in transport/src/main/java/io/netty/channel/nio/NioEventLoop.java:225, in openSelector() method. The GraalVM analysis cannot do the Unsafe registration automatically because the object field offset computation is hidden behind two layers of calls.

Modifications:

This PR updates the Netty GraalVM configuration by registering those fields for unsafe access.
 
Result:

Improved support for Netty on GraalVM with JDK > 9.
2019-10-07 10:10:46 +04:00
bruce
b39ffed042 Fix incorrect calculation of next buffer size in AdaptiveRecvByteBufAllocator (#9555)
Motivation:

Due a bug we did not always correctly calculate the next buffer size in AdaptiveRecvByteBufAllocator.

Modification:

Fix calculation and add unit test

Result:

Correct calculation is always used.
2019-09-27 09:59:25 +02:00
root
92941cdcac [maven-release-plugin] prepare for next development iteration 2019-09-25 06:15:31 +00:00
root
bd907c3b3a [maven-release-plugin] prepare release netty-4.1.42.Final 2019-09-25 06:14:31 +00:00
Norman Maurer
3ad037470e
Correctly reset cached local and remote address when disconnect() is called (#9545)
Motivation:

We should correctly reset the cached local and remote address when a Channel.disconnect() is called and the channel has a notion of disconnect vs close (for example DatagramChannel implementations).

Modifications:

- Correctly reset cached kicak abd remote address
- Update testcase to cover it and so ensure all transports work in a consistent way

Result:

Correctly handle disconnect()
2019-09-19 08:51:10 +02:00
stroller
d01282e5f9 Remove duplicated calculation (#9565)
Motivation:

calculateMaxBytesPerGatheringWrite() contains duplicated calculation:  getSendBufferSize() << 1

Modifications:

Remove the duplicated calculation

Result:

The method will be clear and better
2019-09-15 08:05:21 +02:00
root
01d805bb76 [maven-release-plugin] prepare for next development iteration 2019-09-12 16:09:55 +00:00
root
7cf69022d4 [maven-release-plugin] prepare release netty-4.1.41.Final 2019-09-12 16:09:00 +00:00
root
aef47bec7f [maven-release-plugin] prepare for next development iteration 2019-09-12 05:38:11 +00:00
root
267e5da481 [maven-release-plugin] prepare release netty-4.1.40.Final 2019-09-12 05:37:30 +00:00
Nizar Mankulangara
572b6a8f19 SimpleChannelPool POOL_KEY attribute name is easy to get conflict from user code (#9542) (#9548)
Motivation:

    It is noticed that SimpleChannelPool's POOL_KEY attribute name channelPool is easy to get conflict with user code and throws an exception 'channelPool' is already in use. Being a generic framework - it would be great if we can name the attribute something unique - may be use UUID for the name since the name is not required later.

    Modifications:

    This change make sure that the POOL_KEY used inside SimpleChannelPool is unique by appending the object hashcode in the name.

    Result:

    No unwanted channel attribute name conflict with user code.
2019-09-09 09:47:23 +02:00
Nitsan Wakart
d446765b84 Replace synchronized blocks + assert with synchronized method (#9538)
Motivation:

Following up on discussion with @normanmaurer with suggestion to improve code clarity.

Modification:

Method is synchronized, no need for assert or verbose sync blocks around calls.

Result:

Reduce verbosity and more idiomatic use of keyword. Also rename the method to better describe what it's for.
2019-09-05 09:12:16 +02:00
Norman Maurer
affbdf7125
Correctly protect DefaultChannelPipeline nodes when concurrent removals happen due handlerAdded(...) throwing (#9530)
Motivation:

We need to update the doubly-linked list nodes while holding a lock via synchronized in all cases as otherwise we may end-up with a corrupted pipeline. We missed this when calling remove0(...) due handlerAdded(...) throwing an exception.

Modifications:

- Correctly hold lock while update node
- Add assert
- Add unit test

Result:

Fixes https://github.com/netty/netty/issues/9528
2019-09-03 08:35:11 +02:00
Xiaoqin Fu
21b7e29ea7 Remove extra checks to fix #9456 (#9523)
Motivation:

There are some extra log level checks (logger.isWarnEnabled()).

Modification:

Remove log level checks (logger.isWarnEnabled()) from io.netty.channel.epoll.AbstractEpollStreamChannel, io.netty.channel.DefaultFileRegion, io.netty.channel.nio.AbstractNioChannel, io.netty.util.HashedWheelTimer, io.netty.handler.stream.ChunkedWriteHandler and io.netty.channel.udt.nio.NioUdtMessageConnectorChannel

Result:

Fixes #9456
2019-08-30 10:37:30 +02:00
Nick Hill
a22d4ba859 Simplify EventLoop abstractions for timed scheduled tasks (#9470)
Motivation

The epoll transport was updated in #7834 to decouple setting of the
timerFd from the event loop, so that scheduling delayed tasks does not
require waking up epoll_wait. To achieve this, new overridable hooks
were added in the AbstractScheduledEventExecutor and
SingleThreadEventExecutor superclasses.

However, the minimumDelayScheduledTaskRemoved hook has no current
purpose and I can't envisage a _practical_ need for it. Removing
it would reduce complexity and avoid supporting this specific
API indefinitely. We can add something similar later if needed
but the opposite is not true.

There also isn't a _nice_ way to use the abstractions for
wakeup-avoidance optimizations in other EventLoops that don't have a
decoupled timer.

This PR replaces executeScheduledRunnable and
wakesUpForScheduledRunnable
with two new methods before/afterFutureTaskScheduled that have slightly
different semantics:
 - They only apply to additions; given the current internals there's no
practical use for removals
 - They allow per-submission wakeup decisions via a boolean return val,
which makes them easier to exploit from other existing EL impls (e.g.
NIO/KQueue)
 - They are subjectively "cleaner", taking just the deadline parameter
and not exposing Runnables
 - For current EL/queue impls, only the "after" hook is really needed,
but specialized blocking queue impls can conditionally wake on task
submission (I have one lined up)

Also included are further optimization/simplification/fixes to the
timerFd manipulation logic.

Modifications

- Remove AbstractScheduledEventExecutor#minimumDelayScheduledTaskRemoved()
and supporting methods
- Uplift NonWakeupRunnable and corresponding default wakesUpForTask()
impl from SingleThreadEventLoop to SingleThreadEventExecutor
- Change executeScheduledRunnable() to be package-private, and have a
final impl in SingleThreadEventExecutor which triggers new overridable
hooks before/afterFutureTaskScheduled()
- Remove unnecessary use of bookend tasks while draining the task queue
- Use new hooks to add simpler wake-up avoidance optimization to
NioEventLoop (primarily to demonstrate utility/simplicity)
- Reinstate removed EpollTest class

In EpollEventLoop:
 - Refactor to use only the new afterFutureTaskScheduled() hook for
updating timerFd
 - Fix setTimerFd race condition using a monitor
 - Set nextDeadlineNanos to a negative value while the EL is awake and
use this to block timer changes from outside the EL. Restore the
known-set value prior to sleeping, updating timerFd first if necessary
 - Don't read from timerFd when processing expiry event

Result

- Cleaner API for integrating with different EL/queue timing impls
- Fixed race condition to avoid missing scheduled wakeups
- Eliminate unnecessary timerFd updates while EL is awake, and
unnecessary expired timerFd reads
- Avoid unnecessary scheduled-task wakeups when using NIO transport

I did not yet further explore the suggestion of using
TFD_TIMER_ABSTIME for the timerFd.
2019-08-21 12:34:22 +02:00
lwlee2608
cb739b2619 Update javadoc for NioEventLoop.setRatio() #9481 (#9484)
Motivation:

Improve java apidoc for NioEventLoop.setRatio

Modification:

javadoc

Result:

Fixes #9481
2019-08-21 09:13:42 +02:00
Norman Maurer
97361fa2c8
Replace synchronized usage with ConcurrentHashMap in *Bootstrap classes (#9458)
Motivation:

In AbstractBoostrap, options and attrs are LinkedHashMap that are synchronized on for every read, copy/clone, write operation.
When a lot of connections are triggered concurrently on the same bootstrap instance, the synchronized blocks lead to contention, Netty IO threads get blocked, and performance may be severely degraded.

Modifications:

Use ConcurrentHashMap

Result:

Less contention. Fixes https://github.com/netty/netty/issues/9426
2019-08-16 15:18:17 +02:00
Scott Mitchell
1fa7a5e697 EPOLL - decouple schedule tasks from epoll_wait life cycle (#7834)
Motivation:
EPOLL supports decoupling the timed wakeup mechanism from the selector call. The EPOLL transport takes advantage of this in order to offer more fine grained timer resolution. However we are current calling timerfd_settime on each call to epoll_wait and this is expensive. We don't have to re-arm the timer on every call to epoll_wait and instead only have to arm the timer when a task is scheduled with an earlier expiration than any other existing scheduled task.

Modifications:
- Before scheduled tasks are added to the task queue, we determine if the new
  duration is the soonest to expire, and if so update with timerfd_settime. We
also drain all the tasks at the end of the event loop to make sure we service
any expired tasks and get an accurate next time delay.
- EpollEventLoop maintains a volatile variable which represents the next deadline to expire. This variable is modified inside the event loop thread (before calling epoll_wait) and out side the event loop thread (immediately to ensure proper wakeup time).
- Execute the task queue before the schedule task priority queue. This means we
  may delay the processing of scheduled tasks but it ensures we transfer all
pending tasks from the task queue to the scheduled priority queue to run the
soonest to expire scheduled task first.
- Deprecate IORatio on EpollEventLoop, and drain the executor and scheduled queue on each event loop wakeup. Coupling the amount of time we are allowed to drain the executor queue to a proportion of time we process inbound IO may lead to unbounded queue sizes and unpredictable latency.

Result:
Fixes https://github.com/netty/netty/issues/7829
- In most cases this results in less calls to timerfd_settime
- Less event loop wakeups just to check for scheduled tasks executed outside the event loop
- More predictable executor queue and scheduled task queue draining
- More accurate and responsive scheduled task execution
2019-08-14 10:11:04 +02:00
root
d45a4ce01b [maven-release-plugin] prepare for next development iteration 2019-08-13 17:16:42 +00:00
root
88c2a4cab5 [maven-release-plugin] prepare release netty-4.1.39.Final 2019-08-13 17:15:20 +00:00
Dmitriy Dumanskiy
005fd55b88 #7285 Improved "Discarded inbound message" warning for embedded channel (#9414)
Motivation:

Look like `EmbeddedChannelPipeline` should also override `onUnhandledInboundMessage(ChannelHandlerContext ctx, Object msg)` in order to do not print "Discarded message pipeline" because in case of `EmbeddedChannelPipeline` discarding actually not happens.

This fixes next warning in the latest netty version with websocket and `WebSocketServerCompressionHandler`:

```
13:36:36.231 DEBUG- Decoding WebSocket Frame opCode=2
13:36:36.231 DEBUG- Decoding WebSocket Frame length=5
13:36:36.231 DEBUG- Discarded message pipeline : [JdkZlibDecoder#0, DefaultChannelPipeline$TailContext#0]. Channel : [id: 0xembedded, L:embedded - R:embedded].
```

Modification:

Override correct method

Result:
Follow up fix after https://github.com/netty/netty/pull/9286
2019-08-03 12:20:41 +02:00
root
718b7626e6 [maven-release-plugin] prepare for next development iteration 2019-07-24 09:05:57 +00:00
root
465c900c04 [maven-release-plugin] prepare release netty-4.1.38.Final 2019-07-24 09:05:23 +00:00
Dmitriy Dumanskiy
5ded050f7b #7285 Improved "Discarded inbound message" warning (#9286)
Motivation:

On servers with many pipelines or dynamic pipelines, it is easy for end user to make mistake during pipeline configuration. Current message:

`Discarded inbound message PooledUnsafeDirectByteBuf(ridx: 0, widx: 2, cap: 2) that reached at the tail of the pipeline. Please check your pipeline configuration.`

Is not always meaningful and doesn't allow to find the wrong pipeline quickly.

Modification:

Added additional log placeholder that identifies pipeline handlers and channel info. This will allow for the end users quickly find the problem pipeline.

Result:

Meaningful warning when the message reaches the end of the pipeline. Fixes #7285
2019-07-01 20:38:58 +02:00
Norman Maurer
47eb9c3bf4
Ensure sc.close() is executed before FixedChannelPoolTest.testCloseAsync() returns (#9298)
Motivation:

We observed some test-failues sometimes in the CI which happened if sc.close() was not completed before the next test did run. If this happened we would fail the bind(...) as the LocalAddress was still in use.

Modifications:

Await the close before return

Result:

Fixes race in testsuite which resulted in FixedChannelPoolTest.testAcquireNewConnection to fail if FixedChannelPoolTest.testCloseAsync() did run before it.
2019-06-29 09:21:11 +02:00
root
5b58b8e6b5 [maven-release-plugin] prepare for next development iteration 2019-06-28 05:57:21 +00:00
root
35e0843376 [maven-release-plugin] prepare release netty-4.1.37.Final 2019-06-28 05:56:28 +00:00
jimin
9621a5b981 remove unused imports (#9287)
Motivation:

Some imports are not used

Modification:

remove unused imports

Result:

Code cleanup
2019-06-26 21:08:31 +02:00
Graeme Rocher
18b7bdff12 Add null check to isSkippable. Fixes #9278 (#9280)
Motivation:

Currently GraalVM substrate returns null for reflective calls if the reflection access is not declared up front.

A change introduced in Netty 4.1.35 results in needing to register every Netty handler for reflection. This complicates matters as it is difficult to know all the possible handlers that need to be registered.

Modification:

This change adds a simple
null check such that Netty does not break on GraalVM substrate without the reflection information registration.

Result:

Fixes #9278
2019-06-25 14:45:30 +02:00
Norman Maurer
517a93d87d Make EventLoopTaskQueueFactory a top-level interface
Motivation:

c9aaa93d83 added the ability to specify an EventLoopTaskQueueFactory but did place it under MultithreadEventLoopGroup while not really belongs there.

Modifications:

Make EventLoopTaskQueueFactory a top-level interface

Result:

More logical code layout.
2019-06-22 07:38:03 +02:00
Norman Maurer
c9aaa93d83
Allow to specify a EventLoopTaskQueueFactory for various EventLoopGroup implementations (#9247)
Motivation:

Sometimes it is desirable to be able to use a different Queue implementation for the EventLoop of a Channel. This is currently not possible without resort to reflection.

Modifications:

- Add a new constructor to Nio|Epoll|KQueueEventLoopGroup which allows to specify a factory which is used to create the task queue. This was the user can override the default implementation.
- Add test

Result:

Be able to change Queue that is used for the EventLoop.
2019-06-21 09:05:19 +02:00
Divij Vaidya
fa1dedcc0f Make sync close for FixedChannelPool truly synchronous (#9226)
Motivation:

In the current implementation, the synchronous close() method for FixedChannelPool returns
after scheduling the channels to close via a single threaded executor asynchronously. Closing a channel
requires event loop group, however, there might be a scenario when the application has closed
the event loop group after the sync close() completes. In this scenario an exception is thrown
(event loop rejected the execution) when the single threaded executor tries to close the channel.

Modifications:

Complete the close function only after all the channels have been close and introduce
closeAsync() method for cases when the current/existing behaviour is desired.

Result:

Close function would completely when the channels have been closed
2019-06-14 12:01:14 +02:00
Norman Maurer
dc2649e95d
Allow to set parent Channel when constructing EmbeddedChannel (#9230)
Motivation:

Sometimes it is beneficial to be able to set a parent Channel in EmbeddedChannel if the handler that should be tested depend on the parent.

Modifications:

- Add another constructor which allows to specify a parent
- Add unit tests

Result:

Fixes https://github.com/netty/netty/issues/9228.
2019-06-08 09:11:31 -07:00
Carl Mastrangelo
67ad79d080 Handle missing methods on ChannelHandlerMask (#9221)
Motivation:

When Netty is run through ProGuard, seemingly unused methods are removed.  This breaks reflection, making the Handler skipping throw a reflective error.

Modification:

If a method is seemingly absent, just disable the optimization.

Result:

Dealing with ProGuard sucks infinitesimally less.
2019-06-07 13:39:47 -07:00
Carl Mastrangelo
dea4e33c52 Don't double record stacktrace in Annotated*Exception (#9117)
Motivation:
When initializing the AnnotatedSocketException in AbstractChannel, both
the cause and the stack trace are set, leaving a trailing "Caused By"
that is compressed when printing the trace.

Modification:
Don't include the stack trace in the exception, but leave it in the cause.

Result:
Clearer stack trace
2019-05-22 12:06:30 +02:00
Norman Maurer
f17bfd0f64
Only use static Exception instances when we can ensure addSuppressed … (#9152)
Motivation:

OOME is occurred by increasing suppressedExceptions because other libraries call Throwable#addSuppressed. As we have no control over what other libraries do we need to ensure this can not lead to OOME.

Modifications:

Only use static instances of the Exceptions if we can either dissable addSuppressed or we run on java6.

Result:

Not possible to OOME because of addSuppressed. Fixes https://github.com/netty/netty/issues/9151.
2019-05-17 22:23:02 +02:00
秦世成
cf2f1f54b6 Replace all logic that checks Null with the ObjectUtil utility class (#9145)
Motivation:

Clean the code , replace all logic that checks Null with the ObjectUtil utility class in bootstrap package

Modification:
Replace all logic that checks null with the ObjectUtil utility class

Result:

Less verbose code.
2019-05-13 19:53:45 +02:00
root
ba06eafa1c [maven-release-plugin] prepare for next development iteration 2019-04-30 16:42:29 +00:00
root
49a451101c [maven-release-plugin] prepare release netty-4.1.36.Final 2019-04-30 16:41:28 +00:00
Divij Vaidya
b9c4e17291 Invoke channelAcquired callback on first time channel acquire (#9093)
Motivation:

SimpleChannelPool provides ability to provide custom callbacks/handlers
on major events such as "channel acquired", "channel created" and
"channel released". In the current implementation, when a request to
acquire a channel is made for the first time, the internal channel pool
creates the channel lazily. This triggers the "channel created" callback
but does not invoke the "channel acquired" callback. This is contrary to
caller expectations who assumes that "channel acquired" will be invoked
at the end of every successful acquire call. It also leads to an
inconsistent API experience where the acquired callback is sometimes
invoked and sometimes it isn't depending on wheather the internal
mechanism is creating a new channel or re-using an existing one.

Modifications:

Invoke acquired callback consistenly even when creating a new channel
and modify the tests to support this behaviour

Result:

Consistent experience for the caller of acquire API. Every time they
call the API, the acquired callback will be invoked.
2019-04-29 20:45:49 +02:00
Paulo Lopes
f1495e1945 Add SVM metadata and minimal substitutions to build graalvm native image applications. (#8963)
Motivation:

GraalVM native images are a new way to deliver java applications. Netty is one of the most popular libraries however there are a few limitations that make it impossible to use with native images out of the box. Adding a few metadata (in specific modules will allow the compilation to success and produce working binaries)

Modification:

Added properties files in `META-INF` and substitutions classes (under `internal.svm`) will solve the compilation issues. The substitutions classes are not visible and do not have a public constructor so they are not visible to end users.

Result:

Fixes #8959 

This fix is very conservative as it applies the minimum config required to build:

* pure netty servers
* vert.x applications
* grpc applications

The build is having trouble due to checkstyle which does not seem to be able to find the copyright notice on property files.
2019-04-29 08:39:42 +02:00
Nick Hill
00a9a25f29 Ensure channel handler close() is not skipped in !hasDisconnect case (#9098)
Motivation

The optimization in #8988 didn't correctly handle the specific case
where the channel hasDisconnect == false, and a
ChannelOutboundHandlerAdapter subclass overrides only the close(ctx,
promise) method without also overriding the disconnect(ctx, promise)
method.

Modifications

Adjust AbstractChannelHandler.disconnect(...) method to divert to
close(...) in !hasDisconnect case before computing target context for
the event.

Result

Fixes #9092
2019-04-28 10:41:51 +02:00
root
baab215f66 [maven-release-plugin] prepare for next development iteration 2019-04-17 07:26:24 +00:00
root
dfe657e2d4 [maven-release-plugin] prepare release netty-4.1.35.Final 2019-04-17 07:25:40 +00:00
Daniel Anderson
bedc8a6ea5 Documentation update to MessageSizeEstimator (#9034)
Motivation:

Did not understand the context of "ca".

Modification:

Clarified "CA" to "approximately".

Result:

Fixes #9031
2019-04-12 19:16:23 +02:00
Norman Maurer
6ed203b7ba
NioServerSocketChannel.isActive() must return false after close() completes. (#9030)
Motivation:

When a Channel was closed its isActive() method must return false.

Modifications:

First check for isOpen() before isBound() as isBound() will continue to return true even after the underyling fd was closed.

Result:

Fixes https://github.com/netty/netty/issues/9026.
2019-04-11 18:54:31 +02:00
秦世成
51112e2b36 Avoid IdleStateHandler triggering unexpected idle events when flushing large entries to slow clients (#9020)
Motivation:

IdleStateHandler may trigger unexpected idle events when flushing large entries to slow clients.

Modification:

In netty design, we check the identity hash code and total pending write bytes of the current flush entry to determine whether there is a change in output. But if a large entry has been flushing slowly (for some reason, the network speed is slow, or the client processing speed is too slow to cause the TCP sliding window to be zero), the total pending write bytes size and identity hash code would remain unchanged.

Avoid this issue by adding checks for the current entry flush progress.

Result:

Fixes #8912 .
2019-04-09 16:26:27 +02:00
Norman Maurer
8f7ef1cabb
Skip execution of Channel*Handler method if annotated with @Skip and … (#8988)
Motivation:

Invoking ChannelHandlers is not free and can result in some overhead when the ChannelPipeline becomes very long. This is especially true if most handlers will just forward the call to the next handler in the pipeline. When the user extends Channel*HandlerAdapter we can easily detect if can just skip the handler and invoke the next handler in the pipeline directly. This reduce the overhead of dispatch but also reduce the call-stack in many cases.

This backports https://github.com/netty/netty/pull/8723 and https://github.com/netty/netty/pull/8987 to 4.1

Modifications:

Detect if we can skip the handler when walking the pipeline.

Result:

Reduce overhead for long pipelines.

Benchmark                                       (extraHandlers)   Mode  Cnt       Score      Error  Units
DefaultChannelPipelineBenchmark.propagateEventOld             4  thrpt   10  267313.031 ± 9131.140  ops/s
DefaultChannelPipelineBenchmark.propagateEvent                4  thrpt   10  824825.673 ± 12727.594  ops/s
2019-04-09 09:36:52 +02:00
Norman Maurer
60d135f0c8
Deprecate ChannelOption.newInstance(...) (#8997)
Motivation:

Deprecate ChannelOption.newInstance(...) as it is not used.

Modifications:

Deprecate ChannelOption.newInstance(...) as valueOf(...) should be used as a replacement.

Result:

Fixes https://github.com/netty/netty/issues/8983.
2019-04-05 12:09:54 +02:00
Norman Maurer
ad928c19eb
Mark flaky test as @Ignore (#9010)
Motivation:

0a0da67f43 introduced a testcase which is flacky. We need to fix it and enable it again.

Modifications:

Mark flaky test as ignore.

Result:

No flaky build anymore.
2019-04-04 21:05:36 +02:00
Norman Maurer
20042b6522
Add @SupressWarnings("deprecation") to ChannelInboundHandlerAdapter and clarify deprecation in ChannelHandler (#9001)
Motivation:

https://github.com/netty/netty/pull/8826 added @Deprecated to the exceptionCaught(...) method but we missed to add @SupressWarnings(...) to it's sub-types. Beside this we can make the deprecated docs a bit more clear.

Modifications:

- Add @SupressWarnings("deprecated")
- Clarify docs.

Result:

Less warnings and more clear deprecated docs.
2019-04-02 20:52:06 +02:00
Vladimir Kostyukov
0a0da67f43 Introduce SingleThreadEventLoop.registeredChannels (#8428)
Motivation:

Systems depending on Netty may benefit (telemetry, alternative even loop scheduling algorithms) from knowing the number of channels assigned to each EventLoop.

Modification:

Expose the number of channels registered in the EventLoop via SingleThreadEventLoop.registeredChannels.

Result:

Fixes #8276.
2019-03-28 11:33:12 +00:00
Nick Hill
daf63373bf AbstractChannelHandlerContext doesn't need to extend DefaultAttributeMap (#8960)
Motivation:

It appears this was an oversight, maybe was valid at some point in the past. Noticed while reviewing #8958.

Modifications:

Change AbstractChannelHandlerContext to not extend DefaultAttributeMap.

Result:

Simpler hierarchy, eliminate unused attributes field from each context instance.
2019-03-21 08:49:26 +01:00
Norman Maurer
cb231e9796 Remove test.log file that was commited by mistake.
Motivation:

We commit a test.log file by mistake.

Modifications:

Remove the file.

Result:

Cleanup repo.
2019-03-19 17:56:59 +01:00
root
92b19cfedd [maven-release-plugin] prepare for next development iteration 2019-03-08 08:55:45 +00:00
root
ff7a9fa091 [maven-release-plugin] prepare release netty-4.1.34.Final 2019-03-08 08:51:34 +00:00
Norman Maurer
625c4e8286
Tighten up contract of PromiseCombiner and so make it more safe to use (#8886)
Motivation:

PromiseCombiner is not thread-safe and even assumes all added Futures are using the same EventExecutor. This is kind of fragile as we do not enforce this. We need to enforce this contract to ensure it's safe to use and easy to spot concurrency problems.

Modifications:

- Add new contructor to PromiseCombiner that takes an EventExecutor and deprecate the old non-arg constructor.
- Check if methods are called from within the EventExecutor thread and if not fail
- Correctly dispatch on the right EventExecutor if the Future uses a different EventExecutor to eliminate concurrency issues.

Result:

More safe use of PromiseCombiner + enforce correct usage / contract.
2019-02-28 20:32:04 +01:00
Norman Maurer
81e43d5088
DefaultFileRegion.transferTo with invalid count may cause busy-spin (#8885)
Motivation:

`DefaultFileRegion.transferTo` will return 0 all the time when we request more data then the actual file size. This may result in a busy spin while processing the fileregion during writes.

Modifications:

- If we wrote 0 bytes check if the underlying file size is smaller then the requested count and if so throw an IOException
- Add DefaultFileRegionTest
- Add a test to the testsuite

Result:

Fixes https://github.com/netty/netty/issues/8868.
2019-02-26 11:08:09 +01:00
Norman Maurer
f176384a72
Include the original Exception that caused the Channel to be closed in the ClosedChannelException (#8863)
Motivation:

To make it easier to understand why a Channel was closed previously and so why the operation failed with a ClosedChannelException we should include the original Exception.

Modifications:

- Store the original exception that lead to the closed Channel and include it in the ClosedChannelException that is used to fail the operation.
- Add unit test

Result:

Fixes https://github.com/netty/netty/issues/8862.
2019-02-15 13:13:17 -08:00
Norman Maurer
ad922fa47e
Mark ChannelHandlerAdapter.exceptionCaught(...) as @deprecated. (#8826)
Motivation:

41e03adf24 marked ChannelHandler.exceptionCaught(...) as @deprecated but missed to also mark ChannelHandlerAdapter.exceptionCaught(...) as @deprecated. We should do so as most people extend the base classes and not implement the interfaces directly.

Modifications:

Mark ChannelHandlerAdapter.exceptionCaught(...) as @deprecated as well.

Result:

Mark method as @deprecated to warn users about its removal.
2019-02-01 10:23:54 +01:00
田欧
a33200ca38 use checkPositive/checkPositiveOrZero (#8803)
Motivation:

We have a utility method to check for > 0 and >0 arguments. We should use it.

Modification:

use checkPositive/checkPositiveOrZero instead of if statement.

Result:

Re-use utility method.
2019-01-31 09:07:14 +01:00
Norman Maurer
948d4a9ec5
Minimize memory footprint for AbstractChannelHandlerContext for handlers that execute in the EventExecutor. (#8786)
Motivation:

We cache the Runnable for some tasks to reduce GC pressure in 4 different fields. This gives overhead in terms of memory usage in all cases, even if we always execute in the EventExecutor (which is the case most of the times).

Modifications:

Move the 4 fields to another class and only have one reference to this in AbstractChannelHandlerContext. This gives a small overhead in the case of execution that is done outside of the EventExecutor but reduce memory footprint in the more likily execution case.

Result:

Less memory used per AbstractChannelHandlerContext in most cases.
2019-01-28 19:45:38 +01:00
Norman Maurer
cd3254df88
Update to new checkstyle plugin (#8777) (#8780)
Motivation:

We need to update to a new checkstyle plugin to allow the usage of lambdas.

Modifications:

- Update to new plugin version.
- Fix checkstyle problems.

Result:

Be able to use checkstyle plugin which supports new Java syntax.
2019-01-25 11:58:42 +01:00
Norman Maurer
3c2b86303a
Release message when validation of passed in ChannelPromise fails when calling write(...) / writeAndFlush(...) (#8769)
Motivation:

We need to release the message when we throw an IllegalArgumentException because of a validation failure of the promise to eliminate the risk of a memory leak.

Modifications:

- Consistently release the message before rethrow
- Add testcase.

Result:

Fixes https://github.com/netty/netty/issues/8765.
2019-01-24 07:43:04 +01:00
root
cf03ed0478 [maven-release-plugin] prepare for next development iteration 2019-01-21 12:26:44 +00:00
root
37484635cb [maven-release-plugin] prepare release netty-4.1.33.Final 2019-01-21 12:26:12 +00:00
Norman Maurer
fabc6ee1bc
Fix flaky ChannelInitializerTest.testChannelInitializerEventExecutor() (#8738)
Motivation:

testChannelInitializerEventExecutor() did sometimes fail as we sometimes miss to count down the latch. This can happen when we remove the handler from the pipeline before channelUnregistered(...) was called for it.

Modifications:

Countdown the latch in handlerRemoved(...).

Result:

Fix flaky test.
2019-01-21 09:01:04 +01:00
kezhenxu94
a2cd246f00 cleanup: fix indent (#8734)
Motivation:

Clean up to make the code style unified.

Modification:

Fix indent

Result:

Indents are unified
2019-01-19 17:40:55 +01:00
Norman Maurer
bce0784e5e
Fix racy ChannelOutboundBuffer.testWriteTaskRejected test. (#8735)
Motivation:

testWriteTaskRejected was racy as we did not ensure we dispatched all events to the executor before shutting it down.

Modifications:

Add a latch to ensure we dispatched everything.

Result:

Fix racy test that failed sometimes before.
2019-01-19 17:17:03 +01:00
kezhenxu94
8ebaa1b972 enhancement: extract duplicate code (#8732)
Motivation:

Clean up code to increase readability.

Modification:

Extract duplicate code and remove unnecessary throws

Result:

Share more code.
2019-01-18 19:44:47 +01:00
Norman Maurer
c424599593
Access the Constructor of the Channel in the constructor of ReflectiveChannelFactory. (#8718)
Motivation:

We should access the Constructor of the passed in class in the Constructor of ReflectiveChannelFactory only to reduce the overhead but also fail-fast.

Modifications:

Access the Constructor early.

Result:

Fails fast and less performance overhead.
2019-01-15 08:38:13 +01:00
Norman Maurer
250e2494d9
Only call handlerRemoved(...) if handlerAdded(...) was called during adding the handler to the pipeline. (#8684)
Motivation:

Due a race in DefaultChannelPipeline / AbstractChannelHandlerContext it was possible to have only handlerRemoved(...) called during tearing down the pipeline, even when handlerAdded(...) was never called. We need to ensure we either call both of none to guarantee a proper lifecycle of the handler.

Modifications:

- Enforce handlerAdded(...) / handlerRemoved(...) semantics / ordering
- Add unit test.

Result:

Fixes https://github.com/netty/netty/issues/8676 / https://github.com/netty/netty/issues/6536 .
2019-01-14 08:19:48 +01:00
kashike
6fdd7fcddb Fix minor spelling issues in javadocs (#8701)
Motivation:

Javadocs contained some spelling errors, we should fix these.

Modification:

Fix spelling

Result:

Javadoc cleanup.
2019-01-14 07:24:34 +01:00
kezhenxu94
66addd485f Use camel-case in NioEventLoop (#8713)
Motivation:

Java uses camel-case by convention.

Modification:

Consistently use camel-case.

Result:

More consistent code styling.
2019-01-14 07:20:57 +01:00
Norman Maurer
a564b70d51
More correct fix for using ChannelInitializer with custom EventExecutor. (#8633)
Motivation:

8331248671 did make some changes to fix a race in ChannelInitializer when using with a custom EventExecutor. Unfortunally these where a bit racy and so the testcase failed sometimes.

Modifications:

- More correct fix when using a custom EventExecutor
- Adjust the testcase to be more correct.

Result:

Proper fix for https://github.com/netty/netty/issues/8616.
2018-12-07 19:12:06 +01:00
Feri73
5df235c083 Correcting Maven Dependencies (#8622)
Motivation:

Most of the maven modules do not explicitly declare their
dependencies and rely on transitivity, which is not always correct.

Modifications:

For all maven modules, add all of their dependencies to pom.xml

Result:

All of the (essentially non-transitive) depepdencies of the modules are explicitly declared in pom.xml
2018-12-06 09:01:14 +01:00
Norman Maurer
8331248671
ChannelInitializer may be invoked multiple times when used with custom EventExecutor. (#8620)
Motivation:

The ChannelInitializer may be invoked multipled times when used with a custom EventExecutor as removal operation may be done asynchronously. We need to guard against this.

Modifications:

- Change Map to Set which is more correct in terms of how we use it.
- Ensure we only modify the internal Set when the handler was removed yet
- Add unit test.

Result:

Fixes https://github.com/netty/netty/issues/8616.
2018-12-05 19:30:17 +01:00
Norman Maurer
6739755d39
NioEventLoop.register(...) should offload to the EventLoop if not alr… (#8612)
Motivation:

java.nio.channels.spi.AbstractSelectableChannel.register(...) need to obtain multiple locks during execution which may produce a long wait time if we currently select. This lead to multiple CI failures in the past.

Modifications:

Ensure the register call takes place on the EventLoop.

Result:

No more flacky CI test timeouts.
2018-12-05 15:31:21 +01:00
root
8eb313072e [maven-release-plugin] prepare for next development iteration 2018-11-29 11:15:09 +00:00
root
afcb4a37d3 [maven-release-plugin] prepare release netty-4.1.32.Final 2018-11-29 11:14:20 +00:00
Norman Maurer
af63626777
Factor out less common code-path into own method to allow inlining. (#8590)
Motivation:

During benchmarks two methods showed up as "hot method too big". We can easily make these smaller by factor out some less common code-path to an extra method and so allow inlining.

Modifications:

Factor out less common code path to an extra method.

Result:

Hot methods can be inlined.
2018-11-25 21:46:14 +01:00
Norman Maurer
af34287fd1
HeadContext is inbound and outbound (#8592)
Motivation:

Our HeadContext in DefaultChannelPipeline does handle inbound and outbound but we only marked it as outbound. While this does not have any effect in the current code-base it can lead to problems when we change our internals (this is also how I found the bug).

Modifications:

Construct HeadContext so it is also marked as handling inbound.

Result:

More correct code.
2018-11-24 10:47:56 +01:00
Norman Maurer
2a2bc21067
Remove @Deprecated from package-info.java file (#8591)
Motivation:

31fd66b617 added @Deprecated to some classes but also to the package-info.java files. IntelliJ does not like to have these annotations on package-info.java

Modifications:

Remove annotation from package-info.java

Result:

Be able to compile against via IntelliJ
2018-11-23 17:03:29 +01:00
Norman Maurer
31fd66b617
Mark OIO based transports as deprecated as preparation for removal in Netty 5. (#8579)
Motivation:

We plan to remove the OIO based transports in Netty 5 so we should mark these as deprecated already.

Modifications:

Mark all OIO based transports as deprecated.

Result:

Give the user a heads-up for removal.
2018-11-21 15:15:01 +01:00
Norman Maurer
cd689ee775
Fix javadoc to correctly explain how ChannelDuplexHandler.deregister(...) works. (#8577)
Motivation:

We had an error in the javadoc which was most likely caused by copy and paste.

Modifications:

Fix javadoc.

Result:

Correct javadoc.
2018-11-20 16:45:15 +01:00
Norman Maurer
278b49b2a7
Recover from Selector IOException (#8569)
Motivation:

When the Selector throws an IOException during our EventLoop processing we should rebuild it and transfer the registered Channels. At the moment we will continue trying to use it which will never work.

Modifications:

- Rebuild Selector when an IOException is thrown during any select*(...) methods.
- Add unit test.

Result:

Fixes https://github.com/netty/netty/issues/8566.
2018-11-19 07:41:43 +01:00
Andremoniy
abc8a08c96 Rethrow Error during retrieving remoteAddress / localAddress
Motivation:

Besides an error caused by closing socket in Windows a bunch of other errors may happen at this place which won't be somehow logged. For instance any VirtualMachineError as OutOfMemoryError will be simply ignored. The library should at least log the problem.

Modification:

Added logging of the throwable object.

Result:

Fixes #8499.
2018-11-16 17:20:28 +01:00
Norman Maurer
845a65b31c
Nio|Epoll|KqueueEventLoop task execution might throw UnsupportedOperationException on shutdown. (#8476)
Motivation:

There is a racy UnsupportedOperationException instead because the task removal is delegated to MpscChunkedArrayQueue that does not support removal. This happens with SingleThreadEventExecutor that overrides the newTaskQueue to return an MPSC queue instead of the LinkedBlockingQueue returned by the base class such as NioEventLoop, EpollEventLoop and KQueueEventLoop.

Modifications:

- Catch the UnsupportedOperationException
- Add unit test.

Result:

Fix #8475
2018-11-15 07:19:28 +01:00
root
3e7ddb36c7 [maven-release-plugin] prepare for next development iteration 2018-10-29 15:38:51 +00:00
root
9e50739601 [maven-release-plugin] prepare release netty-4.1.31.Final 2018-10-29 15:37:47 +00:00
Nick Hill
d7fa7be67f Exploit PlatformDependent.allocateUninitializedArray() in more places (#8393)
Motivation:

There are currently many more places where this could be used which were
possibly not considered when the method was added.

If https://github.com/netty/netty/pull/8388 is included in its current
form, a number of these places could additionally make use of the same
BYTE_ARRAYS threadlocal.

There's also a couple of adjacent places where an optimistically-pooled
heap buffer is used for temp byte storage which could use the
threadlocal too in preference to allocating a temp heap bytebuf wrapper.
For example
https://github.com/netty/netty/blob/4.1/buffer/src/main/java/io/netty/buffer/ByteBufUtil.java#L1417.

Modifications:

Replace new byte[] with PlatformDependent.allocateUninitializedArray()
where appropriate; make use of ByteBufUtil.getBytes() in some places
which currently perform the equivalent logic, including avoiding copy of
backing array if possible (although would be rare).

Result:

Further potential speed-up with java9+ and appropriate compile flags.
Many of these places could be on latency-sensitive code paths.
2018-10-27 10:43:28 -05:00
Norman Maurer
a542d4d78b
Increase test timeout (#8385)
Motivation:

It has shown that the used test timeout may be too low when the CI is busy.

Modifications:

Increase timeout to 3 seconds.

Result:

Less false-positives.
2018-10-15 15:10:22 +02:00
Norman Maurer
652650b0db
Correctly decrement pending bytes when submitting AbstractWriteTask fails. (#8349)
Motivation:

Currently we may end up in the situation that we incremented the pending bytes before submitting the AbstractWriteTask but never decrement these again if the submitting of the task fails. This may result in incorrect watermark handling.

Modifications:

- Correctly decrement pending bytes if subimitting of task fails and also ensure we recycle it correctly.
- Add unit test.

Result:

Fixes https://github.com/netty/netty/issues/8343.
2018-10-11 18:46:10 +02:00
Jussi Virtanen
fc28bccdf1 Fix SelectableChannel support in NioEventLoop (#8344)
Motivation:

Unless the 'io.netty.noKeySetOptimization' system property is set,
registering a SelectableChannel instance to a NioEventLoop results
in a ClassCastException:

    io.netty.channel.nio.SelectedSelectionKeySetSelector cannot be cast
        to java.nio.channels.spi.AbstractSelector

Modifications:

Instead of 'selector', pass 'unwrappedSelector' to SelectableChannel.

Result:

It is possible to register a SelectableChannel instance without
setting the 'io.netty.noKeySetOptimization' system property.
2018-10-09 12:11:07 +08:00
Matteo Merli
3a96e7373b Added option to do busy-wait on epoll (#8267)
Motivation:

Add an option (through a SelectStrategy return code) to have the Netty event loop thread to do busy-wait on the epoll.

The reason for this change is to avoid the context switch cost that comes when the event loop thread is blocked on the epoll_wait() call.

On average, the context switch has a penalty of ~13usec.

This benefits both:

The latency when reading from a socket
Scheduling tasks to be executed on the event loop thread.
The tradeoff, when enabling this feature, is that the event loop thread will be using 100% cpu, even when inactive.

Modification:

Added SelectStrategy option to return BUSY_WAIT
Epoll loop will do a epoll_wait() with no timeout
Use pause instruction to hint to processor that we're in a busy loop
Result:

When enabled, minimizes impact of context switch in the critical path
2018-09-28 22:52:00 +02:00
root
2d7cb47edd [maven-release-plugin] prepare for next development iteration 2018-09-27 19:00:45 +00:00
root
3a9ac829d5 [maven-release-plugin] prepare release netty-4.1.30.Final 2018-09-27 18:56:12 +00:00
Norman Maurer
3c2dbdb5db
NioEventLoop should also use our special SelectionKeySet on Java9 and later. (#8260)
Motivation:

In Java8 and earlier we used reflection to replace the used key set if not otherwise told. This does not work on Java9 and later without special flags as its not possible to call setAccessible(true) on the Field anymore.

Modifications:

- Use Unsafe to instrument the Selector with out special set when sun.misc.Unsafe is present and we are using Java9+.

Result:

NIO transport produce less GC on Java9 and later as well.
2018-09-05 07:23:03 +02:00
Norman Maurer
c74b3f3a3b Correctly implement SelectedSelectionKeySet.Iterator remove()
Motivation:

We need to implement remove() by ourselves to make it work on Java7 as otherwise it will throw an AbstractMethodError. This is a followup of c1a335446d.

Modifications:

Just implemented remove()

Result:

Works on Java7 as well.
2018-09-01 08:59:08 +02:00
Norman Maurer
187b1b8a55 Correctly implement SelectedSelectionKeySet.remove(...) / contains(...) again so it works with the NIO Selector.
Motivation:

c1a335446d reimplemented remove(...) and contains(...) in a way which made it not work anymore when used by the Selector.

Modifications:

Partly revert changes in c1a335446d.

Result:

Works again as expected
2018-09-01 08:43:50 +02:00
Norman Maurer
c1a335446d
Correctly implement SelectedSelectionKeySet iterator(), contains(...) and remove(...) (#8244)
Motivation:

Our SelectedSelectionKeySet does not correctly implement various methods which can be done without any performance overhead.

Modifications:

Implement iterator(), contains(...) and remove(...)

Result:

Related to https://github.com/netty/netty/issues/8242.
2018-09-01 08:10:02 +02:00
Norman Maurer
1bd9e662dd
Clarify deprecation docs a bit. (#8226)
Motivation:

It seems to sometimes confuse people what to do to replace setMaxMessagePerRead(...).

Modifications:

Add some more details to the javadocs about the correct replacement.

Result:

Related to https://github.com/netty/netty/issues/8214.
2018-08-25 08:14:38 +02:00
root
a580dc7585 [maven-release-plugin] prepare for next development iteration 2018-08-24 06:36:33 +00:00
root
3fc789e83f [maven-release-plugin] prepare release netty-4.1.29.Final 2018-08-24 06:36:06 +00:00
Ziyan Mo
785473788f (Nio|Epoll)EventLoop.pendingTasks does not need to dispatch to the EventLoop (#8197)
Motivation:

EventLoop.pendingTasks should be (reasonably) cheap to invoke so it can be used within observability. 

Modifications:

Remove code that dispatch access to the internal taskqueue to the EventLoop when invoked as this is not needed anymore with the current MPSC queues we are using. 

See https://github.com/netty/netty/issues/8196#issuecomment-413653286.

Result:

Fixes https://github.com/netty/netty/issues/8196
2018-08-18 07:28:31 +02:00
Norman Maurer
bd25fd03e3
Add testcase for ChannelInitializer.initChannel(...) when throwing an Exception (#8188)
Motivation:

We had a report that the exception may not be correctly propagated. This test shows it is.

Modifications:

Add testcase.

Result:

Test for https://github.com/netty/netty/issues/8158
2018-08-10 08:54:21 +02:00
Norman Maurer
534de73d28
Workaround JDK bug that will cause an AssertionError when calling ServerSocketChannel.config().getOptions(). (#8183)
Motivation:

There is a JDK bug which will return IP_TOS as supported option for ServerSocketChannel even if its not supported afterwards and cause an AssertionError.
See http://mail.openjdk.java.net/pipermail/nio-dev/2018-August/005365.html.

Modifications:

Add a workaround for the JDK bug.

Result:

ServerSocketChannel.config().getOptions() will not throw anymore and work as expected.
2018-08-09 13:11:08 +02:00
Norman Maurer
2c13f71c73
Ensure NIO transport can be used on Java6 again. (#8168)
Motivation:

952eeb8e1e introduced the possibility to use any JDK SocketOption when using the NIO transport but broke the possibility to use netty with java6.

Modifications:

Do not use java7 types in method signatures of the static methods in NioChannelOption to prevent class-loader issues on java6.

Result:

Fixes https://github.com/netty/netty/issues/8166.
2018-08-03 07:07:09 +02:00
root
fcb19cb589 [maven-release-plugin] prepare for next development iteration 2018-07-27 04:59:28 +00:00
root
ff785fbe39 [maven-release-plugin] prepare release netty-4.1.28.Final 2018-07-27 04:59:06 +00:00
Norman Maurer
952eeb8e1e
Support the usage of SocketOption when nio is used and the java versi… (#8085)
* Support the usage of SocketOption when nio is used and the java version >= 7.

Motivation:

The JDK uses SocketOption since java7 to support configuration options on the underyling Channel. We should allow to create a ChannelOption from a given SocketOption if nio is used. This also allows us to expose the same featureset in terms of configuration as the java nio implementation does without any extra effort.

Modifications:

- Add NioChannelOption which allows to wrap an existing SocketOption which then can be applied to the nio transport.
- Add test-cases

Result:

Support the same configuration options as the JDK. Also fixes https://github.com/netty/netty/issues/8072.
2018-07-25 12:32:28 +08:00
root
b4dbdc2036 [maven-release-plugin] prepare for next development iteration 2018-07-11 15:37:40 +00:00
root
1c16519ac8 [maven-release-plugin] prepare release netty-4.1.27.Final 2018-07-11 15:37:21 +00:00
Norman Maurer
301e22eafb
Fix incorrect code in javadocs of ChannelHandler. (#8115)
Motivation:

Some code that was shown as part of the ChannelHandler javadoc was not 100 % correct and used some constructs that we used in netty 3. Also we never called flush() in the code which is a bad example for users.

Modifications:

- Remove netty 3 code references
- Replace channel.write(...) with ctx.writeAndFlush(...)

Result:

More correct code in the javadocs.
2018-07-11 10:19:20 +01:00
时无两丶
cf713d0368 Remove extra 'should' word in docs of continueReading() method 2018-07-11 10:19:03 +01:00
root
7bb9e7eafe [maven-release-plugin] prepare for next development iteration 2018-07-10 05:21:24 +00:00
root
8ca5421bd2 [maven-release-plugin] prepare release netty-4.1.26.Final 2018-07-10 05:18:13 +00:00
Bennett Lynch
253522fc91 Create SimpleUserEventChannelHandler convenience class (#7991)
Motivation:

Currently, the vast majority of userEventTriggered() implementations
require the user to supply the boilerplate behavior of performing an
instanceof check, handling if appropriate, and calling
fireUserEventTriggered() otherwise.

We can simplify this very common use case by creating a class that only
matches user events of a given type, similar to the existing
SimpleChannelInboundHandler class.

Modifications:

Create a new SimpleUserEventChannelHandler class
Create accompanying SimpleUserEventChannelHandlerTest class

Result:

Users will be able to handle most events in a less verbose manner.
2018-06-29 08:25:23 +02:00
Norman Maurer
83710cb2e1
Replace toArray(new T[size]) with toArray(new T[0]) to eliminate zero-out and allow the VM to optimize. (#8075)
Motivation:

Using toArray(new T[0]) is usually the faster aproach these days. We should use it.

See also https://shipilev.net/blog/2016/arrays-wisdom-ancients/#_conclusion.

Modifications:

Replace toArray(new T[size]) with toArray(new T[0]).

Result:

Faster code.
2018-06-29 07:56:04 +02:00
Tyler Rockwood
34b25dc94c Expose channel count for FixedChannelPool (#8059)
Motivation:

We use FixedChannelPool in production, and we believe we have a leak that doesn't return sockets to the pool (but they should be closed), thus blocking us from creating new connections when we need them. I haven't confirmed this yet, but right now I have to resort to reflection to access this field which makes me sad.

Modification:

Expose the acquiredChannelCount field through a getter method.

Result:

Allows introspection of the pool size in FixedChannelPool.
2018-06-28 08:12:26 +02:00
radai-rosenblatt
b1d4b2af1c fix grammar in javadoc
Signed-off-by: radai-rosenblatt <radai.rosenblatt@gmail.com>
2018-06-20 10:45:49 +02:00
Roger
3e3e5155b9 Check if Log level is enabled before creating log statements (#8022)
Motivation

There is a cost to concatenating strings and calling methods that will be wasted if the Logger's level is not enabled.

Modifications

Check if Log level is enabled before producing log statement. These are just a few cases found by RegEx'ing in the code.

Result

Tiny bit more efficient code.
2018-06-13 23:21:53 -07:00
时无两丶
b53cf045a7 SelectedSelectionKeySet should only be created if the set can be instrumented
Motivation:

If we can not replace the internal used Set of the Selector there is no need to create an SelectedSelectionKeySet instance.

Modification:

Only create SelectedSelectionKeySet if we will replace the internal set.

Result:

Less object creation in some cases and cleaner code.
2018-05-30 15:33:39 +02:00
Norman Maurer
d133bf06a4
Allow to schedule tasks up to Long.MAX_VALUE (#7972)
Motivation:

We should allow to schedule tasks with a delay up to Long.MAX_VALUE as we did pre 4.1.25.Final.

Modifications:

Just ensure we not overflow and put the correct max limits in place when schedule a timer. At worse we will get a wakeup to early and then schedule a new timeout.

Result:

Fixes https://github.com/netty/netty/issues/7970.
2018-05-30 11:11:42 +02:00
Norman Maurer
c3637ff42c
AUTO_CLOSE should not be marked as deprecated. (#7967)
Motivation:

A long time ago we deprecated AUTO_CLOSE but it turned out this feature is still useful because if a write error is detected there still maybe data to read, and if we close the channel automatically we will lose data

Modifications:

- Remove `@Deprecated` tag for AUTO_CLOSE, setAutoClose(...) and isAutoClose(...)
- Fix javadocs on ChannelConfig to correctly tell the default value of AUTO_CLOSE.

Result:

Less warnings.
2018-05-24 21:25:30 +02:00
Norman Maurer
583fc272f2
Fixed|SimpleChannelPool.close() should only return after complete. (#7927)
Motivation:

We need to ensure we only return from close() after all work is done as otherwise we may close the EventExecutor before we dispatched everything.

Modifications:

Correctly wait on operations to complete before return.

Result:

Fixes https://github.com/netty/netty/issues/7901.
2018-05-21 19:22:31 +02:00
Norman Maurer
64bb279f47 [maven-release-plugin] prepare for next development iteration 2018-05-14 11:11:45 +00:00