Commit Graph

1723 Commits

Author SHA1 Message Date
Norman Maurer
7547a448e0 [#4906] Ensure addLast(...) works as expected in EmbeddedChannel
Motivation:

If the user will use addLast(...) on the ChannelPipeline of EmbeddedChannel after its constructor was run it will break the EmbeddedChannel as it will not be able to collect inbound messages and exceptions.

Modifications:

Ensure addLast(...) work as expected by move the logic of handling messages and exceptions ti protected methods of DefaultChannelPipeline and use a custom implementation for EmbeddedChannel

Result:

addLast(...) works as expected when using EmbeddedChannel.
2016-05-21 10:33:25 +02:00
Norman Maurer
e10dca7601 Mark Recycler.recycle(...) deprecated and update usage.
Motivation:

Recycler.recycle(...) should not be used anymore and be replaced by Handle.recycle().

Modifications:

Mark it as deprecated and update usage.

Result:

Correctly document deprecated api.
2016-05-20 22:11:31 +02:00
Norman Maurer
2e352b75ac Remove unused class
Motivation:

We should remove unused classes that are package-private

Modifications:

Remove unused class

Result:

Less cruft in code-base.
2016-05-20 21:53:59 +02:00
Norman Maurer
a425a8551d [#5174] Expose Bootstrap getter methods and add some additional ones
Motivation:

The Bootstrap class (applies also to AbstractBootstrap and ServerBootstrap) has a few package private getter methods and some things such as #attr() and #options() aren't exposed at all.

Modifications:

Expose "getters" for configured things in a safe-manner via the config() method.

Result:

Easier for the user to check what is configured for a Bootstrap/ServerBootstrap.
2016-05-18 13:26:43 +02:00
Scott Mitchell
2b340df452 DuplexChannel to support shutdownInput
Motivation:
The DuplexChannel is currently incomplete and only supports shutting down the output side of a channel. This interface should also support shutting down the input side of the channel.

Modifications:
- Add shutdownInput and shutdown methods to the DuplexChannel interface
- Remove state in NIO and OIO for tracking input being shutdown independent of the underlying transport's socket type. Tracking the state independently may lead to inconsistent state.

Result:
DuplexChannel supports shutting down the input side of the channel
Fixes https://github.com/netty/netty/issues/5175
2016-05-18 09:11:49 +02:00
Norman Maurer
8fe3c83e4c [#5104] Fix possible deadlock in DefaultChannelPipeline
Motivation:

When a user has multiple EventLoops in an EventLoopGroup and calls pipeline.add* / remove* / replace from an EventLoop that belongs to another Channel it is possible to deadlock if the other EventLoop does the same.

Modification:

- Only ensure the actual modification takes place in a synchronized block and not wait until the handlerAdded(...) / handlerRemoved(...) method is called. This is ok as we submit the task to the executor while still holding the look and so ensure correct order of pipeline modifications.
- Ensure if an AbstractChannelHandlerContext is put in the linked-list structure but the handlerAdded(...) method was not called we skip it until handlerAdded(...) was called. This is needed to ensure handlerAdded(...) is always called first.

Result:

Its not possible to deadlock when modify the DefaultChannelPipeline.
2016-05-17 14:24:46 +02:00
Trustin Lee
3a9f472161 Make retained derived buffers recyclable
Related: #4333 #4421 #5128

Motivation:

slice(), duplicate() and readSlice() currently create a non-recyclable
derived buffer instance. Under heavy load, an application that creates a
lot of derived buffers can put the garbage collector under pressure.

Modifications:

- Add the following methods which creates a non-recyclable derived buffer
  - retainedSlice()
  - retainedDuplicate()
  - readRetainedSlice()
- Add the new recyclable derived buffer implementations, which has its
  own reference count value
- Add ByteBufHolder.retainedDuplicate()
- Add ByteBufHolder.replace(ByteBuf) so that..
  - a user can replace the content of the holder in a consistent way
  - copy/duplicate/retainedDuplicate() can delegate the holder
    construction to replace(ByteBuf)
- Use retainedDuplicate() and retainedSlice() wherever possible
- Miscellaneous:
  - Rename DuplicateByteBufTest to DuplicatedByteBufTest (missing 'D')
  - Make ReplayingDecoderByteBuf.reject() return an exception instead of
    throwing it so that its callers don't need to add dummy return
    statement

Result:

Derived buffers are now recycled when created via retainedSlice() and
retainedDuplicate() and derived from a pooled buffer
2016-05-17 11:16:13 +02:00
Norman Maurer
68cd670eb9 Remove ChannelHandlerInvoker
Motivation:

We tried to provide the ability for the user to change the semantics of the threading-model by delegate the invoking of the ChannelHandler to the ChannelHandlerInvoker. Unfortunually this not really worked out quite well and resulted in just more complexity and splitting of code that belongs together. We should remove the ChannelHandlerInvoker again and just do the same as in 4.0

Modifications:

Remove ChannelHandlerInvoker again and replace its usage in Http2MultiplexCodec

Result:

Easier code and less bad abstractions.
2016-05-17 11:14:00 +02:00
Hyangtack Lee
f44f3e7926 NioEventLoop ensures that all submitted tasks are executed immediately.
Motivation:
If a task was submitted when wakenUp value was true, the task didn't get a chance to call Selector#wakeup.
So we need to check task queue again before executing select operation. If we don't, the task might be pended until select operation was timed out.
It might be pended until idle timeout if IdleStateHandler existed in pipeline.

Modifications:
Execute Selector#select in a non-blocking manner if there's a task submitted when wakenUp value was true.

Result:
Every tasks in NioEventLoop will not be pended.
2016-05-17 07:43:46 +02:00
Norman Maurer
c249926784 [#3127] Allow to write with VoidPromise to Channels in ChannelGroup
Motivation:

Users sometimes want to use Channel.voidPromise() when write to a Channel to reduce GC-pressure. This should be also possible when write via a ChannelGroup.

Modifications:

Add new write(...) and writeAndFlush(...) overloads which allow to signale that a VoidPromise should be used to write to the Channel

Result:

Users can write with VoidPromise when using ChannelGroup
2016-05-14 20:53:25 +02:00
Norman Maurer
27a392b877 Add ChannelInboundInvoker and ChannelOutboundInvoker
Motivation:

ChannelHandlerContext, ChannelPipeline and Channel share various method signatures. We should provide an interface to share.

Modifications:

Add ChannelInboundInvoker and ChannelOutboundInvoker and extend these.

Result:

Better API abstraction.
2016-05-14 20:41:13 +02:00
Norman Maurer
01109dd635 Fix test-failures introduces by c1827114e9. 2016-05-14 20:38:31 +02:00
Norman Maurer
c1827114e9 Use ConnectException when connection failed for LocalChannel
Motivation:

To be more consistent we should use ConnectException when we fail the connect attempt because no LocalServerChannel exists with the given address.

Modifications:

Use correct exception.

Result:

More consistent handling of connection refused between different transports.
2016-05-14 07:23:55 +02:00
Norman Maurer
775dd139ea Ensure Bootstrap.connect(...) not throw IllegalStateException when registration is delayed.
Motivation:

Bootstrap.connect(...) tries to obtain the EventLoop of a Channel before it may be registered. This will cause an IllegalStateException. We need to ensure we handle the cause of late registration and not throw in this case.

Modifications:

Ensure we only try to access the EventLoop after the Channel is registered and handle the case of late registration.

Result:

Bootstrap.connect(...) not fails on late registration.
2016-05-14 07:22:38 +02:00
Scott Mitchell
1f8e192e21 Remove EventExecutor.children
Motivation:
EventExecutor.children uses generics in such a way that an entire colleciton must be cast to a specific type of object. This interface is not very flexible and is impossible to implement if the EventExecutor type must be wrapped. The current usage of this method also does not have any clear need within Netty. The Iterator interface allows for EventExecutor to be wrapped and forces the caller to make assumptions about types instead of building the assumptions into the interface.

Motivation:
- Remove EventExecutor.children and undeprecate the iterator() interface

Result:
EventExecutor interface has one less method and is easier to wrap.
2016-05-13 18:17:22 -07:00
Norman Maurer
690ab563e7 Ensure ChannelHandlerContext.attr(...) and ChannelHandlerContext.hasAttr(...) has no semantic change
Motivation:

The ChannelHandlerContext.attr(...) and ChannelHandlerContext.hasAttr(...) delegated to Channel for the attributes which is a semantic change compared to 4.0 releases. We should not change the semantic to not break users applications when upgrading to 4.1.0

Modifications:

- Revert semantic change
- Mark ChannelHandlerContext.attr(...) and hasAttr(...) as @deprecated so we can remove these later

Result:

Semantic of attribute operations on ChannelHandlerContext is the same in 4.1 as in 4.0 again.
2016-05-13 15:51:58 +02:00
Norman Maurer
5ffa3b1c46 Only try to bind if late registration not failed.
Motivation:

We should not try to call bind if registration failed.

Modifications:

Only call doBind0(...) when the registration not failed.

Result:

Not try to to bind if the registration failed.
2016-05-13 12:10:31 +02:00
Norman Maurer
ea61b5b5b3 [#5223] Remove indirection in DefaultChannelPipeline.executorSafe(...)
Motivation:

We use channel.unsafe().invoker().executor() in DefaultChannelPipeline.executorSafe(...) which is an unnecessary indirection to channel.eventLoop().

Modifications:

Remove indirection by using channel.eventLoop().

Result:

Cleaner code.
2016-05-13 08:42:36 +02:00
Scott Mitchell
f2ed3e6ce8 DefaultPromise LateListener Logic Issues
Motivation:
The LateListener logic is prone to infinite loops and relies on being processed in the EventExecutor's thread for synchronization, but this EventExecutor may not be constant. An infinite loop can occur if the EventExecutor's execute method does not introduce a context switch in LateListener.run. The EventExecutor can be changed by classes which inherit from DefaultPromise. For example the DefaultChannelPromise will return w/e EventLoop the channel is registered to, but this EventLoop can change (re-registration).

Modifications:
- Remove the LateListener concept and instead use a single Object to maintain the listeners while still preserving notification order
- Make the result member variable an atomic variable so it can be outside the synchronized(this) blocks
- Cleanup/simplify existing state management code

Result:
Fixes https://github.com/netty/netty/issues/5185
2016-05-09 10:33:40 -07:00
Norman Maurer
b2bb72b2de [#5171] Ensure NioDatagramChannelConfig can be instanced on android
Motivation:

NioDatagramChannelConfig currently uses NetworkChannel in its static { } block and so fails to init on android which not has this class.

Modifications:

Use reflection to load the NetworkChannel.class

Result:

Be able to use NIO Datagram on android as well.
2016-05-06 08:04:36 +02:00
Norman Maurer
c745ac0e16 [#3297] Fix race while resolve remote address that can cause NPE
Motivation:

We may produce a NPE due a race that can happen while check if a resolution was done and failed.

Modifications:

Correctly first check if the resultion is done before try to detect if it failed and so remove the race that can produce a NPE.

Result:

No more race possible while resolve address during connect.
2016-05-06 08:00:15 +02:00
Carl Mastrangelo
cf07f984b1 Add @Deprecated when the javadoc says its deprecated
Motivation:

Reduce nag warnings when compiling, make it easier for IDEs to display what's deprecated.

Modifications:

Added @Deprecated in a few places

Result:

No more warnings.
2016-05-01 20:30:13 +02:00
Scott Mitchell
04e33fd2d8 NioEventLoop closes channel even if channel is not registered
Motivation:
If a channel is deregistered from an NioEventLoop the associated SelectionKey is cancelled. If the NioEventLoop has yet to process a pending event as a result of that SelectionKey then the NioEventLoop will see the SelecitonKey is invalid and close the channel. The NioEventLoop should not close a channel if it is not registered with that NioEventLoop.

Modifications:
- NioEventLoop.processSelectedKeys should check that the channel is still registered to itself before closing the channel

Result:
NioEventLoop doesn't close a channel that is no longer registered to it when the SelectionKey is invalid
Fixes https://github.com/netty/netty/issues/5125
2016-04-21 09:17:14 -07:00
Norman Maurer
572bdfb494 [maven-release-plugin] prepare for next development iteration 2016-04-10 08:37:18 +02:00
Norman Maurer
c6121a6f49 [maven-release-plugin] prepare release netty-4.1.0.CR7 2016-04-10 08:36:56 +02:00
Norman Maurer
6e919f70f8 [maven-release-plugin] rollback the release of netty-4.1.0.CR7 2016-04-09 22:13:44 +02:00
Norman Maurer
4cdd51509a [maven-release-plugin] prepare release netty-4.1.0.CR7 2016-04-09 22:05:34 +02:00
Scott Mitchell
abce89d1bc Revert "[#5028] Fix re-entrance issue with channelWritabilityChanged(...) and write(...)"
Motivation:
Revert d0943dcd30. Delaying the notification of writability change may lead to notification being missed. This is a ABA type of concurrency problem.

Modifications:
- Revert d0943dcd30.

Result:
channelWritabilityChange will be called on every change, and will not be suppressed due to ABA scenario.
2016-04-09 20:32:47 +02:00
Scott Mitchell
a0b28d6c82 Fix potential assertion error introduced by 0bc93dd
Motivation:
Commit 0bc93dd introduced a potential assertion failure, if the deprecated method would be used.

Modifications:
Fix the potential assertion error.

Result:
Regression removed
2016-04-08 09:44:44 -07:00
Trustin Lee
32cfe25132 Fix checkstyle 2016-04-08 17:44:52 +09:00
Trustin Lee
c1a3cc32e7 Fix an assertion error introduced by 0bc93dda08
Motivation:
    
Commit 0bc93dda08 introduced an assertion
failure.

Modifications:

Fix the assertion error.

Result:

Regression removed
2016-04-08 17:23:32 +09:00
stroller
84e281ced9 fix one java doc issue: extra }
Motivation:

There is one extra } for WriteBufferWaterMark's javadoc:
{@linkplain #high}  high water mark}

The generated javadoc will show the content: "the high high water mark}"

Modifications:

remove the }

Result:
The generated javadoc will show the content: "the high water mark" instead of "the high high water mark}"
2016-04-08 09:47:26 +02:00
Scott Mitchell
0bc93dda08 NIO autoReadClear should also clear the SelectionKey
Motivation:
Prior to 5b48fc284e setting readPending to false when autoReadClear was called was enough because when/if the EventLoop woke up with a read event we would first check if readPending was true and if it is not, we would return early. Now that readPending will only be set in the EventLoop it is not necessary to check readPending at the start of the read methods, and we should instead remove the OP_READ from the SelectionKey when we also set readPending to false.

Modifications:
- autoReadCleared should call AbstractNioUnsafe.removeReadOp

Result:
NIO is now consistent with EPOLL and removes the READ operation on the selector when autoRead is cleared.
2016-04-08 00:03:37 -07:00
Scott Mitchell
5b48fc284e Make OIO/NIO/EPOLL autoReadClear consistent
Motivation:
OIO/NIO use a volatile variable to track if a read is pending. EPOLL does not use a volatile an executes a Runnable on the event loop thread to set readPending to false. These mechansims should be consistent, and not using a volatile variable is preferable because the variable is written to frequently in the event loop thread.
OIO also does not set readPending to false before each fireChannelRead operation and may result in reading more data than the user desires.

Modifications:
- OIO/NIO should not use a volatile variable for readPending
- OIO should set readPending to false before each fireChannelRead

Result:
OIO/NIO/EPOLL are more consistent w.r.t. readPending and volatile variable operations are reduced
Fixes https://github.com/netty/netty/issues/5069
2016-04-06 12:32:14 -07:00
Norman Maurer
d0943dcd30 [#5028] Fix re-entrance issue with channelWritabilityChanged(...) and write(...)
Motivation:

When always triggered fireChannelWritabilityChanged() directly when the update the pending bytes in the ChannelOutboundBuffer was made from within the EventLoop. This is problematic as this can cause some re-entrance issue if the user has a custom ChannelOutboundHandler that does multiple writes from within the write(...) method and also has a handler that will intercept the channelWritabilityChanged event and trigger another write when the Channel is writable. This can also easily happen if the user just use a MessageToMessageEncoder subclass and triggers a write from channelWritabilityChanged().

Beside this we also triggered fireChannelWritabilityChanged() too often when a user did a write from outside the EventLoop. In this case we increased the pending bytes of the outboundbuffer before scheduled the actual write and decreased again before the write then takes place. Both of this may trigger a fireChannelWritabilityChanged() event which then may be re-triggered once the actual write ends again in the ChannelOutboundBuffer.

The third gotcha was that a user may get multiple events even if the writability of the channel not changed.

Modification:

- Always invoke the fireChannelWritabilityChanged() later on the EventLoop.
- Only trigger the fireChannelWritabilityChanged() if the channel is still active and if the writability of the channel changed. No need to cause events that were already triggered without a real writability change.
- when write(...) is called from outside the EventLoop we only increase the pending bytes in the outbound buffer (so that Channel.isWritable() is updated directly) but not cause a fireChannelWritabilityChanged(). The fireChannelWritabilityChanged() is then triggered once the task is picked up by the EventLoop as usual.

Result:

No more re-entrance possible because of writes from within channelWritabilityChanged(...) method and no events without a real writability change.
2016-04-06 10:07:13 +02:00
Scott Mitchell
9fb86a380d NIO/EPOLL readPending set to false incorrectly
Motivation:
441aa4c575 introduced a bug in transport-native-epoll where readPending is set to false before a read is attempted, but this should happen before fireChannelRead is called. The NIO transport also only sets the readPending variable to false on the first read in the event loop. This means that if the user only calls read() on the first channelRead(..) the select loop will still listen for read events even if the user does not call read() on subsequent channelRead() or channelReadComplete() in the same event loop run. If the user only needs 2 channelRead() calls then by default they will may get 14 more channelRead() calls in the current event loop, and then 16 more when the event loop is woken up for a read event. This will also read data off the TCP stack and allow the peer to queue more data in the local RECV buffers.

Modifications:
- readPending should be set to false before each call to channelRead()
- make NIO readPending set to false consistent with EPOLL

Result:
NIO and EPOLL transport set readPending to false at correct times which don't read more data than intended by the user.
Fixes https://github.com/netty/netty/issues/5082
2016-04-06 00:09:49 -07:00
Norman Maurer
d602277204 Include cause that was used to notify the promise when logging an failed try to notify it.
Motivation:

When a promise is notified that was already added to the ChannelOutboundBuffer and we try to notify it later on we only see a warning that it was notified before. This is often not very useful as we have no idea where it was notified at all. We can do better in case it was failed before (which is most of the times the case) and just also log the cause that was used for it.

Modifications:

Add the cause that was used to notify the promise when we fail to notify it as part of the ChannelOutboundBuffer.

Result:

Easier to debug user errors.
2016-04-05 21:13:51 +02:00
Norman Maurer
7fb475a223 Fix typo missed in f46cfbc590 2016-04-05 15:32:54 +02:00
Norman Maurer
f46cfbc590 [#5059] Deprecate method with typo and introduce a new one without typo
Motivation:

There is a spelling error in FileRegion.transfered() as it should be transferred().

Modifications:

Deprecate old method and add a new one.

Result:

Fix typo and can remove the old method later.
2016-04-05 15:06:46 +02:00
Norman Maurer
ea94336689 DefaultChannelHandlerInvoker should work with non AbstractChannelHandlerContext sub-classes.
Motivation:

DefaultChannelHandlerInvoker currently blindly cast to AbstractChannelHandlerContext without checking if the ChannelHandlerContext is really a sub-type of it. It should check it first and if not just use slow-path implementation.

Modifications:

Do instanceof check first and if it fails just create a new Runnable instance of used the cached.

Result:

DefaultChannelHandlerInvoker works with any ChannelHandlerContext implementations.
2016-04-05 13:21:07 +02:00
Trustin Lee
3b941c2a7c [maven-release-plugin] prepare for next development iteration 2016-04-02 01:25:05 -04:00
Trustin Lee
7368ccc539 [maven-release-plugin] prepare release netty-4.1.0.CR6 2016-04-02 01:24:55 -04:00
jiafu1115
3e5dcb5f3e [#3806] Setting WRITE_BUFFER_LOW_WATER_MARK before WRITE_BUFFER_HIGH_WATER_MARK results in an internal Exception
Motivation:

Setting the WRITE_BUFFER_LOW_WATER_MARK before WRITE_BUFFER_HIGH_WATER_MARK results in an internal Exception (appears only in the logs) if the value is larger than the default high water mark value. The WRITE_BUFFER_HIGH_WATER_MARK call appears to have no effect in this context.

Setting the values in the reverse order works.

Modifications:

- deprecated ChannelOption.WRITE_BUFFER_HIGH_WATER_MARK and
ChannelOption.WRITE_BUFFER_LOW_WATER_MARK.
- add one new option called ChannelOption.WRITE_BUFFER_WATER_MARK.

Result:
The high/low water mark values limits caused by default values are removed.

Setting the WRITE_BUFFER_LOW_WATER_MARK before WRITE_BUFFER_HIGH_WATER_MARK results in an internal Exception (appears only in the logs) if the value is larger than the default high water mark value. The WRITE_BUFFER_HIGH_WATER_MARK call appears to have no effect in this context.

Setting the values in the reverse order works.
2016-03-31 13:44:44 +02:00
Tibor Csögör
9d4fae308c ChannelInitializer: change propagation of channelRegistered event
Motivation:

If a handler is added to the pipeline within ChannelInitializer::initChannel via
addFirst(...) then it will not receive the channelRegistered event.  The same
handler added via addLast(...) will receive the event.  This different behavior
is unlikely to be expected by users and can cause confusion.

Modifications:

Let ChannelInitializer::channelRegistered propagate the event by passing it to
the pipeline instead of firing it on the ChannelHandlerContext.

Result:

The channelRegistered event is propagated to handlers regardless of the method
used to add it to the pipeline (addFirst/addLast).
2016-03-31 09:01:00 +02:00
Scott Mitchell
0c839d9e0a EPOLL SelectStrategy
Motivation:
NIO now supports a pluggable select strategy, but EPOLL currently doesn't support this. We should strive for feature parity for EPOLL.

Modifications:
- Add SelectStrategy to EPOLL transport.

Result:
EPOLL transport supports SelectStategy.
2016-03-30 15:11:35 -07:00
Michael Nitschinger
5d76daf33b Allow to customize NIO (channel) select strategies.
Motivation:

Under high throughput/low latency workloads, selector wakeups are
degrading performance when the incoming operations are triggered
from outside of the event loop. This is a common scenario for
"client" applications where the originating input is coming from
application threads rather from the socket attached inside the
event loops.

As a result, it can be desirable to defer the blocking select
so that incoming tasks (write/flush) do not need to wakeup
the selector.

Modifications:

This changeset adds the notion of a generic SelectStrategy which,
based on its contract, allows the implementation to optionally
defer the blocking select based on some custom criteria.

The default implementation resembles the original behaviour, that
is if tasks are in the queue `selectNow()` and move on, and if no
tasks need to be processed go into the blocking select and wait
for wakeup.

The strategy can be customized per `NioEventLoopGroup` in the
constructor.

Result:

High performance client applications are now given the chance to
customize for how long the actual selector blocking should be
deferred by employing a custom select strategy.
2016-03-30 15:01:25 -07:00
Norman Maurer
2facb7affd Change DefaultChannelId visibility to default. Related to [#5053]
Motivation:

There is no need to make DefaultChannelId package private as it may be useful for the user. For example EmbeddedChannel allows to inject a ChannelId when it is constructed. For this case the user can just use DefaultChannelId.

Modifications:

Change visibility of DefaultChannelId to public.

Result:

It's possible to create a new instance of DefaultChannelId by the user.
2016-03-30 17:39:32 +02:00
Norman Maurer
cee38ed2b6 [maven-release-plugin] prepare for next development iteration 2016-03-29 16:45:13 +02:00
Norman Maurer
9cd9e7daeb [maven-release-plugin] prepare release netty-4.1.0.CR5 2016-03-29 16:44:33 +02:00
Norman Maurer
86b9656167 Correctly run pending tasks before flush and also remove incorrect assert.
Motivation:

We need to ensure we run all pending tasks before doing any flush in writeOutbound(...) to ensure all pending tasks are run first. Also we should remove the assert of the future and just add a listener to it so it is processed later if needed. This is true as a user may schedule a write for later execution.

Modifications:

- Remove assert of future in writeOutbound(...)
- Correctly run pending tasks before doing the flush and also before doing the close of the channel.
- Add unit tests to proof the defect is fixed.

Result:

Correclty handle the situation of delayed writes.
2016-03-29 14:30:23 +02:00
Scott Mitchell
99c85ef4f5 cf171ff525 Close Regression
Motivation:
cf171ff525 introduced a change in behavior when dealing with closing channel in the read loop. This changed behavior may use stale state to determine if a channel should be shutdown and may be incorrect.

Modifications:
- Revert the usage of potentially stale state

Result:
Closing a channel in the read loop is based upon current state instead of potentially stale state.
2016-03-24 14:52:04 -07:00
Norman Maurer
9a183ec38f Add methods to easily release messages from inbound / outbound buffer of EmbeddedChannel
Motivation:

Often the user uses EmbeddedChannel within unit tests where the only "important" thing is to know if any pending messages were in the buffer and then release these.
We should provide methods for this so the user not need to manually loop through these and release.

Modifications:

Add methods to easily handle releasing of messages.

Result:

Less boiler-plate code for the user to write.
2016-03-24 11:03:30 +01:00
Norman Maurer
28d03adbfe [maven-release-plugin] prepare for next development iteration 2016-03-21 11:51:50 +01:00
Norman Maurer
4653dc1d05 [maven-release-plugin] prepare release netty-4.1.0.CR4 2016-03-21 11:51:12 +01:00
Scott Mitchell
fc099292fd HTTP/2 DefaultHttp2ConnectionEncoder data frame size incorrect if error
Motivation:
If an error occurs during a write operation then DefaultHttp2ConnectionEncoder.FlowControlledData will clear the CoalescingBufferQueue which will reset the queue's readable bytes to 0. To recover from an error the DefaultHttp2RemoteFlowController will attempt to return bytes to the flow control window, but since the frame has reset its own size this will lead to invalid flow control accounting.

Modifications:
- DefaultHttp2ConnectionEncoder.FlowControlledData should not reset its size if an error occurs

Result:
No more flow controller errors due to DefaultHttp2ConnectionEncoder.FlowControlledData setting its size to 0 if an error occurs.
2016-03-18 11:38:01 -07:00
Tibor Csögör
6e840d8e62 trivial javadoc fixes
- fix the formatting of the diagram in ChannelFuture's javadoc
- update external link in AutobahnServer
- fix various spelling issues
2016-03-16 20:18:29 +01:00
Scott Mitchell
45849b2fa8 Deprecate PromiseAggregator
Motivation:
PromiseAggregator's API allows for the aggregate promise to complete before the user is done adding promises. In order to support this use case the API structure would need to change in a breaking manner.

Modifications:
- Deprecate PromiseAggregator and subclasses
- Introduce PromiseCombiner which corrects these issues

Result:
PromiseCombiner corrects the deficiencies in PromiseAggregator.
2016-03-14 10:53:30 -07:00
Max Ng
e7ee6abd70 Guard against re-entrance in PendingWriteQueue.
Motivation:

PendingWriteQueue should guard against re-entrant writes once
removeAndFailAll() is run.

Modifications:

removeAndFailAll() should repeat until the queue is finally empty.

Result:

assertEmpty() will always hold.
2016-03-14 08:52:43 +01:00
Norman Maurer
45d291bb15 Add asserts so users will see errors when try to use methods from outside the EventLoop.
Motivation:

We should guard users from using Unsafe methods from outside the EventLoop if not designed to do so.

Modifications:

Add asserts

Result:

Easier for users to detect miss-use.
2016-03-08 14:59:43 +01:00
Scott Mitchell
d0f7f98d22 e2f5012 unit test cleanup
Motivation:
e2f5012 added unit tests which did not verify the buffer was released as it was intended to.

Modification:
- Unit tests must verify release is called

Result:
Unit tests enforce that ByteBufs are released.
2016-03-07 09:33:45 -08:00
Scott Mitchell
e2f5012f3b DefaultChannelHandlerInvoker write leak
Motivation:
DefaultChannelHandlerInvoker.invokeWrite calls a utility method validatePromise which may throw if the arguments are not valid. If this method throws then the message will not be released.

Modifications:
- If an exception is thrown the message should be released

Result:
No more leak in DefaultChannelHandlerInvoker.invokeWrite
2016-03-07 09:16:12 -08:00
Xiaoyan Lin
c7aadb5469 Fork SpscLinkedQueue and SpscLinkedAtomicQueue from JCTools
Motivation:

See #3746.

Modifications:

Fork SpscLinkedQueue and SpscLinkedAtomicQueue from JCTools based on 7846450e28

Result:

Add SpscLinkedQueue and SpscLinkedAtomicQueue and apply it in LocalChannel.
2016-02-22 15:55:52 +01:00
Norman Maurer
ca443e42e0 [maven-release-plugin] prepare for next development iteration 2016-02-19 23:00:11 +01:00
Norman Maurer
f39eb9a6b2 [maven-release-plugin] prepare release netty-4.1.0.CR3 2016-02-19 22:59:52 +01:00
Roman Timushev
23f7fc67a4 Enable shutdownOutput for EpollDomainSocketChannel 2016-02-18 18:05:51 -08:00
Norman Maurer
c6a3729e4c Ensure handlerAdded(...) and handlerRemoved(...) is always called from the right thread
Motiviation:

We should ensure that handlerAdded(...) and handlerRemoved(...) is always called from the EventExecutor that also invokes the other methods of the ChannelHandler. Also we need to ensure we always call handlerAdded(...) before any other method can be calld to ensure correct ordering.

Motifications:

- Ensure that the right thread is used to call the methods
- Ensure correct ordering
- Add tests

Result:

Respect the thread-model for handlerAdded(...) and handlerRemoved(...) and preserve correct ordering in all cases.
2016-02-12 11:18:40 -08:00
Fabian Lange
a51e2c8769 Expose Helper to obtain the "best" mac address.
Motivation:

The implementation of obtaining the best possible mac address is very good. There are many sub-par implementations proposed on stackoverflow.
While not strictly a netty concern, it would be nice to offer this util also to netty users.

Modifications:

extract DefaultChannelId#defaultMachineId code obtaining the "best" mac into a new helper called MacAddress, keep the random bytes fallback in DefaultChannelID.

Result:

New helper available.
2016-02-05 09:27:43 +01:00
Norman Maurer
75a2ddd61c [maven-release-plugin] prepare for next development iteration 2016-02-04 16:51:44 +01:00
Norman Maurer
7eb3a60dba [maven-release-plugin] prepare release netty-4.1.0.CR2 2016-02-04 16:37:06 +01:00
Norman Maurer
465a190c3f [#4805] Respect isAutoRead() once re-register Channel
Motivation:

When a channel was registered before and is re-registered we need to respect ChannelConfig.isAutoRead() and so start reading one the registration task completes. This was done "by luck" before 15162202fb.

Modifications:

Explicit start reading once a Channel was re-registered if isAutoRead() is true.

Result:

Correctly receive data after re-registration completes.
2016-02-04 15:34:24 +01:00
Norman Maurer
08a7ca3747 Correctly pass ChannelPromise on to the next ChannelOutboundHandler when use CombinedChannelDuplexHandler.
Motivation:

Due a regression introduced by e969b6917c we missed to pass the original ChannelPromise to the next ChannelOutboundHandler and so
may never notify the origin ChannelPromise. This is related to #4805.

Modifications:

- Correctly pass the ChannelPromise
- Add unit test.

Result:

Correctly pass the ChannelPromise on deregister(...)
2016-02-04 15:28:46 +01:00
fu.jian
a06708f81b fix the issue netty#2944 in 4.1
Motivation:

fix the issue netty#2944

Modifications:

use - instead of =>, use ! instead of :> due to the connection is bidirectional. What's more, toString() method don't know the direction or there is no need to know the direction when only log channel information.
add L: before local address and R: before remote address.

Result:

after the fix, log won't confuse the user
2016-02-02 21:43:38 +01:00
Travis Haagen
a75dcb2756 Made it easier to use custom ChannelId instances with Channel implementations that rely on the AbstractChannel(Channel parent) constructor.
Motivation:

The AbstractChannel(Channel parent) constructor was previously hard-coded to always
call DefaultChannelId.newInstance(), and this made it difficult to use a custom
ChannelId implementation with some commonly used Channel implementations.

Modifications:

Introduced newId() method in AbstractChannel, which by default returns
DefaultChannelId.newInstance() but can be overridden by subclasses. Added
ensureDefaultChannelId() test to AbstractChannelTest, to ensure the prior
behavior of calling DefaultChannelId.newInstance() still holds with the
AbstractChannel(Channel parent) constructor.

Result:

AbstractChannel now has the protected newId() method, but there is no functional
difference.
2016-02-02 08:33:42 +01:00
Scott Mitchell
8ba4b63cb6 OioServerChannel Default Max Messages Per Read Too High
Motivation:
A few implementations of OioServerChannel have a default max messages per read set to 16. We should set the default to 1 to prevent blocking on a read before setting a socket that has just been accepted.

Modifications:
- OioSctpServerChannel and OioServerSocketChannel metadata changed to use the default (1) max messages per read

Result:
Oio based servers will complete accepting a socket before potentially blocking waiting to accept other sockets.
2016-01-29 11:34:00 -08:00
Norman Maurer
af39cb6b12 Ensure ChannelHandler.handlerAdded(...) is always called as first method of the handler
Motivation:

If a user adds a ChannelHandler from outside the EventLoop it is possible to get into the situation that handlerAdded(...) is scheduled on the EventLoop and so called after another methods of the ChannelHandler as the EventLoop may already be executing on this point in time.

Modification:

- Ensure we always check if the handlerAdded(...) method was called already and if not add the currently needed call to the EventLoop so it will be picked up after handlerAdded(...) was called. This works as if the handler is added to the ChannelPipeline from outside the EventLoop the actual handlerAdded(...) operation is scheduled on the EventLoop.
- Some cleanup in the DefaultChannelPipeline

Result:

Correctly order of method executions of ChannelHandler.
2016-01-28 13:24:36 +01:00
Xiaoyan Lin
58a038d398 Fix ChannelOutboundHandlerAdapter javadoc
Motivation:

ChannelOutboundHandlerAdapter's javadoc has some minor issues.

Modifications:

Fix the minor javadoc issues and resolves #4752.

Result:

ChannelOutboundHandlerAdapter's javadoc issues are fixed.
2016-01-26 10:13:16 +01:00
Norman Maurer
1c417e5f82 [maven-release-plugin] prepare for next development iteration 2016-01-21 15:35:55 +01:00
Norman Maurer
c681a40a78 [maven-release-plugin] prepare release netty-4.1.0.CR1 2016-01-21 15:28:21 +01:00
Eric Anderson
6dbb610f5b Add ChannelHandlerContext.invoker()
Motivation:

Being able to access the invoker() is useful when adding additional
handlers that should be running in the same thread. Since an application
may be using a threading model unsupported by the default invoker, they
can specify their own. Because of that, in a handler that auto-adds
other handlers:

// This is a good pattern
ctx.pipeline().addBefore(ctx.invoker(), ctx.name(), null, newHandler);
// This will generally work, but prevents using custom invoker.
ctx.pipeline().addBefore(ctx.executor(), ctx.name(), null, newHandler);

That's why I believe in commit 110745b0, for the now-defunct 5.0 branch,
when ChannelHandlerAppender was added the invoker() method was also
necessary.

There is a side-benefit to exposing the invoker: in certain advanced
use-cases using the invoker for a particular handler is useful. Using
the invoker you are able to invoke a _particular_ handler, from possibly
a different thread yet still using standard exception processing.

ChannelHandlerContext does part of that, but is unwieldy when trying to
invoke a particular handler because it invokes the prev or next handler,
not the one the context is for. A workaround is to use the next or prev
context (respectively), but this breaks when the pipeline changes.

This came up during writing the Http2MultiplexCodec which uses a
separate child channel for each http/2 stream and wants to send messages
from the child channel directly to the Http2MultiplexCodec handler that
created it.

Modifications:

Add the invoker() method to ChannelHandlerContext. It was already being
implemented by AbstractChannelHandlerContext. The two other
implementations of ChannelHandlerContext needed minor tweaks.

Result:

Access to the invoker used for a particular handler, for either reusing
for other handlers or for advanced use-cases. Fixes #4738
2016-01-22 14:04:35 +01:00
Norman Maurer
e969b6917c Let CombinedChannelDuplexHandler correctly handle exceptionCaught. Related to [#4528]
Motivation:

ChannelInboundHandler and ChannelOutboundHandler both can implement exceptionCaught(...) method and so we need to dispatch to both of them.

Modifications:

- Correctly first dispatch exceptionCaught to the ChannelInboundHandler but also make sure the next handler it will be dispatched to will be the ChannelOutboundHandler
- Add removeInboundHandler() and removeOutboundHandler() which allows to remove one of the combined handlers
- Let *Codec extends it and not ChannelHandlerAppender
- Remove ChannelHandlerAppender

Result:

Correctly handle events and also have same behavior as in 4.0
2016-01-18 09:54:48 +01:00
Norman Maurer
1848e73ce6 Ensure connectPromise is not notified before fireChannelActive() is called.
Motivation:

Our contract in Channels is that the promise should always be notified before the actual callbacks of the ChannelInboundHandler are called. This was not done in the LocalChannel and so the behavior was different to other Channel implementations.

Modifications:

- First complete the ChannelPromise then call fireChannelActive()
- Guard against NPE when doClose() was called before the task was executed.

Result:

Consistent behavior between LocalChannel and other Channel implementations.
2016-01-18 09:28:43 +01:00
Norman Maurer
8b123a5546 Fix AbstractChannelTest errors caused by incorrect mocking
Motivation:

In AbstractChannelTest we not correctly mocked some methods which could lead to test errors. That said it only showed up here when running from the IDE and not from the cmdLine.

Modifications:

Mock methods that are needed for the test

Result:

Test pass in the IDE as well.
2016-01-14 20:57:42 +01:00
Norman Maurer
4d854cc149 Allow to change if EmbeddedChannel should handle close() and disconnect() different.
Motivation:

At the moment EmbeddedChannel always handle close() and disconnect() the same way which also means that ChannelOutboundHandler.disconnect(...) will never called. We should allow to specify if these are handle different or not to make the use of EmbeddedChannel more flexible.

Modifications:

Add 2 other constructors which allow to specify if disconnect / close are handled the same way or differently.

Result:

More flexible usage of EmbeddedChannel possible.
2016-01-14 07:19:23 +01:00
Fernando van Loenhout
5c05629da1 Fix issue #4676
Fixed spelling mistake at EmbeddedChannel#readOutbound()
2016-01-08 18:12:31 -08:00
Xiaoyan Lin
751ed6cc94 Avoid unnecessary boxing/unboxing
Motivation:

Boxing/unboxing can be avoided.

Modifications:

Use parseInt/parseLong to avoid unnecessary boxing/unboxing.

Result:

Remove unnecessary boxing/unboxing.
2016-01-08 17:38:20 +01:00
fu.jian
0c733e1425 [#2363] Correctly null out SelectionKey[] when selectAgain
Motivation:

The prefix fix of #2363 did not correctly handle the case when selectAgain is true and so missed to null out entries.

Modifications:

Move the i++ from end of loop to beginning of loop

Result:

Entries in the array will be null out so allow to have these GC'ed once the Channel close
2016-01-08 09:04:10 +01:00
Norman Maurer
bf2a99518c Simplify synchronized syntax
Motivation:

We often used synchronized(this) while the whole method was synchronized, which can be simplified by just mark the whole method as synchronized.

Modifications:

Replace synchronized(this) with synchronized on the method

Result:

Cleaner code
2016-01-05 08:45:20 +01:00
Fabian Lange
619d82b56f Removed unused imports
Motivation:

Warnings in IDE, unclean code, negligible performance impact.

Modification:

Deletion of unused imports

Result:

No more warnings in IDE, cleaner code, negligible performance improvement.
2016-01-04 14:32:29 +01:00
Sergey Polovko
bc2559ceb1 fix links to github issues in javadoc 2016-01-04 08:47:00 +01:00
Xiaoyan Lin
475d901131 Fix errors reported by javadoc
Motivation:

Javadoc reports errors about invalid docs.

Modifications:

Fix some errors reported by javadoc.

Result:

A lot of javadoc errors are fixed by this patch.
2015-12-27 08:36:45 +01:00
Xiaoyan Lin
a96d52fe66 Fix javadoc links and tags
Motivation:

There are some wrong links and tags in javadoc.

Modifications:

Fix the wrong links and tags in javadoc.

Result:

These links will work correctly in javadoc.
2015-12-26 08:34:31 +01:00
Norman Maurer
d59bf84ef2 [#4435] Always invoke the actual deregisteration later in the EventLoop.
Motivation:

As a user may call deregister() from within any method while doing processing in the ChannelPipeline,  we need to ensure we do the actual deregister operation later. This is needed as for example,  we may be in the ByteToMessageDecoder.callDecode(...) method and so still try to do processing in the old EventLoop while the user already registered the Channel to a new EventLoop. Without delay, the deregister operation this could lead to have a handler invoked by different EventLoop and so threads.

Modifications:

Ensure the actual deregister will be done later on and not directly when invoked.

Result:

Calling deregister() within ByteToMessageDecoder.decode(..) is safe.
2015-12-24 14:20:17 +01:00
Alexey Ermakov
cfd6793bb7 Customizable estimation for messages written outside the EventLoop
Motivation:

Estimation algorithm currently used for WriteTasks is complicated and
wrong. Additionally, some code relies on outbound buffer size
incremented only on actual writes to the outbound buffer.

Modifications:

- Throw away the old estimator and replace with a simple algorithm that
  uses the client-provided estimator along with a statically configured
  WriteTask overhead (io.netty.transport.writeTaskSizeOverhead system
  property with the default value of 48 bytes)
- Add a io.netty.transport.estimateSizeOnSubmit boolean system property
  allowing the clients to disable the message estimation outside the
  event loop

Result:

Task estimation is user controllable and produces better results by
default
2015-12-23 23:04:23 +01:00
Jonas Berlin
e4cb163be2 Fix javadoc link 2015-12-22 20:53:41 +01:00
Sky Ao
9b171beb92 Trivial javadoc fixes in ChannelHandlerContext 2015-12-18 14:00:28 +01:00
Stephane Landelle
6393506b97 Extract SocketAdress logic from NameResolver
Motivation:

As discussed in #4529, NameResolver design shouldn't be resolving SocketAddresses (or String name + port) and return InetSocketAddresses. It should resolve String names and return InetAddresses.
This SocketAddress to InetSocketAddresses resolution is actually a different concern, used by Bootstrap.

Modifications:

Extract SocketAddress to InetSocketAddresses resolution concern to a new class hierarchy named AddressResolver.
These AddressResolvers delegate to NameResolvers.

Result:

Better separation of concerns.

Note that new AddressResolvers generate a bit more allocations because of the intermediate Promise and List<InetAddress>.
2015-12-14 14:03:50 +01:00
Norman Maurer
89ff831a67 [#4449] Remove registered events from eventloop before close
Motivation:

We need to remove all registered events for a Channel from the EventLoop before doing the actual close to ensure we not produce a cpu spin when the actual close operation is delayed or executed outside of the EventLoop.

Modifications:

Deregister for events for NIO and EPOLL socket implementations when SO_LINGER is used.

Result:

No more cpu spin.
2015-12-13 09:55:50 +01:00
Norman Maurer
0ec34b5f76 Fix race-condition when closing a NioSocketChannel or EpollSocketChannel
Motivation:

Fix a race-condition when closing NioSocketChannel or EpollSocketChannel while try to detect if a close executor should be used and the underlying socket was already closed. This could lead to an exception that then leave the channel / in an invalid state and so could lead to side-effects like heavy CPU usage.

Modifications:

Catch possible socket exception while try to get the SO_LINGER options from the underlying socket.

Result:

No more race-condition when closing the channel is possible with bad side-effects.
2015-11-26 22:56:00 +01:00
Scott Mitchell
641505a5d2 DefaultChannelConfig maxMessagesPerRead default not always set
Motivation:
ChannelMetadata has a field minMaxMessagesPerRead which can be confusing. There are also some cases where static instances are used and the default value for channel type is not being applied.

Modifications:
- use a default value which is set unconditionally to simplify
- make sure static instances of MaxMessagesRecvByteBufAllocator are not used if the intention is that the default maxMessagesPerRead should be derived from the channel type.

Result:
Less confusing interfaces in ChannelMetadata and ChannelConfig. Default maxMessagesPerRead is correctly applied.
2015-11-25 15:14:07 -08:00
nmittler
227e67900e Fixing spammy logging for CoalescingBufferQueueTest
Motivation:

The CoalescingBufferQueueTest is somewhat relaxed with its releasing of test buffers, using safeRelease to generically deal with tests that may or may not release the buffers. SafeRelease generates logs, however, when the release fails.

Modifications:

Tightened up the individual test methods to verify that the buffers are released properly.

Result:

Fixes #4497
2015-11-25 07:16:05 -08:00
Scott Mitchell
3fa4603120 Cleanup ChannelOption.AUTO_CLOSE javadocs
Motivation:
The javadocs for ChannelOption.AUTO_CLOSE say the default is false, but the default is currently true.

Modifications:
- Make javadocs consistent with code

Result:
Less confusing docs.
2015-11-24 15:23:15 -08:00
Norman Maurer
41e03adf24 Mark ChannelHandler.exceptionCaught(...) as deprected.
Motivation:

exceptionCaught(...) will only handle inbound exceptions which means it makes not much sense to have it also on ChannelOutboundHandler. Because of this we should move it to ChannelInboundHandler.

Modifications:

Add @deprecated annotation to ChannelHandler.exceptionCaught(...).

Result:

Preapre to cleanup the API in later release.
2015-11-23 09:59:22 +01:00
Norman Maurer
7bee318fc7 Use OneTimeTask where possible to reduce object creation
Motivation:

We should use OneTimeTask where possible to reduce object creation.

Modifications:

Replace Runnable with OneTimeTask

Result:

Less object creation
2015-11-20 14:39:06 -08:00
Norman Maurer
cfa76f6326 Remove unnecessary reference to AbstractChannel from AbstractChannelHandlerContext
Motivation:

We not need to store another reference to AbstractChannel as we can access it through DefaultChannelHandlerContext.

Modifications:

Remove reference.

Result:

Cleaner code.
2015-11-20 06:41:11 -08:00
Norman Maurer
2d2e07578a Remove HashMap for lookup name / ctx from DefaultChannelPipeline to reduce memory footprint
Motivation:

If you start to have 1M+ concurrent connections memory footprint can be come a big issue. We should try to reduce it as much as possible in the core of netty.

Modifications:

- Remove HashMap that was used to store name to ctx mapping. This was only used for validation and access a handler by name. As a pipeline is not expected to be very long (like 100+ handlers) we can just walk the linked list structure to find the ctx with a given name.

Result:

Less memory footprint of the DefaultChannelPipeline.
2015-11-20 06:29:28 -08:00
pieteradejong
64409ad10b fixed word - issue #4469 2015-11-19 07:33:02 -08:00
Norman Maurer
2ecce8fa56 [maven-release-plugin] prepare for next development iteration 2015-11-10 22:59:33 +01:00
Norman Maurer
6a93f331d3 [maven-release-plugin] prepare release netty-4.1.0.Beta8 2015-11-10 22:50:57 +01:00
Sergio Bossa
187efca9aa Improved DefaultChannelPipeline#destroy() to avoid spinning continuously in case of custom executors.
Motivation:

The previous DefaultChannelPipeline#destroy() implementation, introduced in #3156, is suboptimal as it can cause the for loop to continuously spin if the executor used by a given handler is unable to "recognize" the event loop.
It could be objected that it's the custom executor responsibility to properly implement the inEventLoop() method, but some implementetaions might not be able to do that for performance reasons, and even so, it's always better to be safe against API misuse, in particular when it is not possible to fail fast and the alternative is rather some sutle behaviour.

Modifications:

The patch simply avoids the recursive spin by explicitly passing the "in event loop" condition as a boolean parameter, preserving the same guarantees offered by #3156. A unit test has also been added.

Result:

All channel events are correctly called and no high CPU usage is seen anymore.
2015-11-06 19:37:53 +01:00
Norman Maurer
98eb69f169 Remove synchronization overhead on generateName.
Motivation:

Changing the chache of generated names to use a cache per thread. This will remove the bottleneck when many eventloops are used and names need to generate.

Modifications:

Use a FastThreadLocal to store the cached names.

Result:

Less locking between threads.
2015-11-04 20:41:41 +01:00
Norman Maurer
94e907fa39 Only call ReferenceCountUtil.touch(...) if ResourceLeakDetection was enabled when Channel was created.
Motivation:

We should only call ReferenceCountUtil.touch(...) if needed as otherwise we pay the overhead of instanceof and cast
everytime.

Modifications:

Add boolean flag which indicates if touch(...) should be called.

Result:

Less overhead when leak detection is not enabled.
2015-10-29 19:23:27 +01:00
Norman Maurer
2100a83e30 [#4363] Improve size calculation of messages when written from outside the EventLoop
Motiviation:

If a user writes from outside the EventLoop we increase the pending bytes of the outbound buffer before submitting the write request. This is done so the user can stop writing asap once the channel turns unwritable. Unfortunally this doesn't take the overhead of adding the task into the account and so it is very easy for an user to full up the task queue. Beside this we use a value of 0 for an unown message by default which is not ideal.

Modifications:

- port the message calculation we used in netty 3.x into AbstractChannelHandlerContext and so better calculate the overhead of a message that is submitted from outside the EventLoop
- change the default estimated size for an unknown message to 8.

Result:

Better behaviour when submiting writes from outside the EventLoop.
2015-10-28 21:54:31 +01:00
Norman Maurer
8687475eb7 [#4373] Fix assert error when trying to release Channel to closed FixedChannelPool
Motivation:

Once a FixedChannelPool was closed we must not allow to acquire or release Channels to prevent assert errors.

Modifications:

Fail release and acquire calls when FixedChannelPool is closed.

Result:

No more assert errors.1
2015-10-24 11:56:46 +02:00
Norman Maurer
b90685d3c9 Not share FixedRecvByteBufAllocator.HandleImpl
Motivation:

As MaxMessageHandle is stateful we can not share the same HandleImpl instance as otherwise we will see race conditions.

Modifications:

Create a new HandleImpl instance on each newHandle() call.

Result:

No more races.
2015-10-22 09:47:23 +02:00
Richard DiCroce
b58036aeea Improve flexibility of EmbeddedChannel ID
Motivation:

EmbeddedChannelId#hashCode() and equals() do not behave correctly if an
instance is serialized and then deserialized. Additionally,
EmbeddedChannel does not allow use of any other type of ChannelId, and
EmbeddedChannelId is (mostly) a singleton instance. This creates a
problem for unit tests that construct multiple EmbeddedChannels and
expect each channel to have a unique ID.

Modifications:

EmbeddedChannelId is modified so equals() will return true and
hashCode() will return the same value for any instance of the class.
EmbeddedChannel is modified to allow a ChannelId to be specified when
the channel is constructed. Tests added for both changes.

Result:

EmbeddedChannelId now behaves correctly when deserialized, and
EmbeddedChannels can now have unique IDs.
2015-10-14 10:40:02 +02:00
Sky Ao
0740f703c1 change type definition of pipeline from DefaultChannelPipeline to ChannelPipeline 2015-10-10 20:15:10 +02:00
Norman Maurer
2d7e957a23 Cleanup PendingWriteQueueTest
Motivation:

PendingWriteQueueTest needs some cleanup.

Modifications:

- Cleanup code to remove deprecation warnings
- use static imports

Result:

No more warnings
2015-10-10 19:59:23 +02:00
Norman Maurer
fbaf5d06e6 [#4316] Ensure pending tasks are run when EmbeddedChannel.close(...) or disconnect(...) is called.
Motivation:

We missed to run all pending tasks when EmbeddedChannel.close(...) or disconnect(...) was called. Because of this channelInactive(...) / channelUnregistered(...) of the handlers were never called.

Modifications:

Correctly run all pending tasks and cancel all not ready scheduled tasks when close or disconnect was called.

Result:

Correctly run tasks on close / disconnect and have channelInactive(...) / channelUnregistered(...) called.
2015-10-07 09:28:00 +02:00
Norman Maurer
2ff2806ada [maven-release-plugin] prepare for next development iteration 2015-10-02 09:03:29 +02:00
Norman Maurer
5a43de10f7 [maven-release-plugin] prepare release netty-4.1.0.Beta7 2015-10-02 09:02:58 +02:00
Norman Maurer
2a27d581a9 Ensure close caused by write will happen before write promise is notified
Motiviation:

We need to ensure the actual close to the transport takes place before the promsie of the write is notified that triggered it. This is needed as otherwise Channel.isActive(), isOpen() and isWritable() may return true even if the Channel should be closed already.

Modifications:

- Ensure the close takes place first

Result:

ChannelFutureListener will see the correct state of the Channel.
2015-09-16 20:36:04 +02:00
Norman Maurer
076d4ed514 [#4205] Correctly set EPOLLOUT flag whe writeBytes(...) was not able to write everything
Motivation:

writeBytes(...) missed to set EPOLLOUT flag when not all bytes were written. This could lead to have the EpollEventLoop not try to flush the remaining bytes once the socket becomes writable again.

Modifications:

- Move setting EPOLLOUT flag logic to one point so we are sure we always do it.
- Move OP_WRITE flag logic to one point as well.

Result:

Correctly try to write pending data if socket becomes writable again.
2015-09-16 07:30:17 +02:00
Scott Mitchell
983920f25f RecvByteBufAllocator.DelegatingHandle accessor
Motivation:
RecvByteBufAllocator.DelegatingHandle does not provide an accessor to get the delegate handle. This may be useful for classes that extend DelegatingHandle.

Modifications:
- add delegate() method to DelegatingHandle

Result:
Classes which inherit from DelegatingHandle can now access the delegate Handle.
2015-09-04 12:44:26 -07:00
Norman Maurer
34de2667c7 [maven-release-plugin] prepare for next development iteration 2015-09-02 11:45:20 +02:00
Norman Maurer
2eb444ec1d [maven-release-plugin] prepare release netty-4.1.0.Beta6 2015-09-02 11:36:11 +02:00
Scott Mitchell
1e763b6504 LocalChannel write when peer closed leak
Motivation:
If LocalChannel doWrite executes while the peer's state changes from CONNECTED to CLOSED it is possible that some promise's won't be completed and buffers will be leaked.

Modifications:
- Check the peer's state in doWrite to avoid a race condition

Result:
All write operations should release, and the associated promise should be completed.
2015-09-01 13:13:02 -07:00
Scott Mitchell
d3dcc7f658 LocalChannelWrite event sequencing issue
Motivation:
https://github.com/netty/netty/pull/4143 addressed a few ordering issues but an ordering issue still remained if the Promise for a write completes, and a listener of that promise does a write on a peer channel. The ordering was subject to how potentially 2 different executors would run a task, but it should be coordinated such that the first write is read first.

Modifications:
- Keep track of the finishPeerRead task run on the executor if necessary and ensure it completes before current channel read occurs

Result:
Ordering of events for echo type situations is preserved.
2015-08-28 11:26:59 -07:00
Scott Mitchell
e37069b947 LocalChannel Event Ordering Error
Motivation:
When a LocalChannel write operation occurs, the promise associated with the write operation is marked successful when it is added to the peer's queue, but before the peer has actually received the data. If the promise callback closes the channel then a race condition exists where the close event may occur before the data is delivered. We should preserve ordering of events.

Modifications:
- LocalChannel should track when a write is in progress, and if a close operation happens make sure the peer gets all pending read operations.

Result:
LocalChannel preserves order of operations.
Fixes https://github.com/netty/netty/issues/4118
2015-08-28 09:03:46 -07:00
Christopher Probst
c8fb2a84c5 Fix race condition of DefaultChannelGroup by introducing a closed flag.
Motivation:

Doc of ChannelGroup says, that it can be used to manage server and child channels at once.
However, in DefaultChannelGroup, there is a race condition. When a server channel accepts a child, it schedules its
registration on an event loop, which takes some time. If the ChannelGroup, which is supposed
to close server and child channels at once, is closed after the child channel has been scheduled
for registration and before this registration actually happens, this child channel is not closed
and remains connected. This could lead to connection leaks.

Modifications:

To fix this, the DefaultChannelGroup is changed to has a closed flag.
This flag is set to true, just before the close() method is actually closing channels.
The add() method checks after adding a new channel, if this flag has been set to true.
If yes, the new channel is closed. If not, we have the guarantee, that this channel will be
closed by the ChannelGroup, because setting the closed flag to true happens-before closing any channels.

This behaviour can be activated by two new constructors. The old constructors are still there and behave like before.
Therefore, no existing code should be affected directly.

Result:

If activating this feature, the DefaultChannelGroup can be used, for managing server and child channels at once.
But this activating this feature means also, that a ChannelGroup cannot be reused after calling close().
2015-08-27 09:50:09 +02:00
Trustin Lee
fdfe3149ba Provide more control over DnsNameResolver.query() / Add NameResolver.resolveAll()
Related issues:
- #3971
- #3973
- #3976
- #4035

Motivation:

1. Previously, DnsNameResolver.query() retried the request query by its
own. It prevents a user from deciding when to retry or stop. It is also
impossible to get the response object whose code is not NOERROR.

2. NameResolver does not have an operation that resolves a host name
into multiple addresses, like InetAddress.getAllByName()

Modifications:

- Changes related with DnsNameResolver.query()
  - Make query() not retry
    - Move the retry logic to DnsNameResolver.resolve() instead.
  - Make query() fail the promise only when I/O error occurred or it
    failed to get a response
  - Add DnsNameResolverException and use it when query() fails so that
    the resolver can give more information about the failure
  - query() does not cache anymore.

- Changes related with NameResolver.resolveAll()
  - Add NameResolver.resolveAll()
  - Add SimpleNameResolver.doResolveAll()

- Changes related with DnsNameResolver.resolve() and resolveAll()
  - Make DnsNameResolveContext abstract so that DnsNameResolver can
    decide to get single or multiple addresses from it
  - Re-implement cache so that the cache works for resolve() and
    resolveAll()
  - Add 'traceEnabled' property to enable/disable trace information

- Miscellaneous changes
  - Use ObjectUtil.checkNotNull() wherever possible
  - Add InternetProtocolFamily.addressType() to remove repetitive
    switch-case blocks in DnsNameResolver(Context)
  - Do not raise an exception when decoding a truncated DNS response

Result:

- Full control over query()
- A user can now retrieve all addresses via (Dns)NameResolver.resolveAll()
- DNS cache works only for resolve() and resolveAll() now.
2015-08-18 17:40:13 +09:00
Trustin Lee
fd9ca8bbb3 Fix compilation errors 2015-08-18 12:42:33 +09:00
Ivan Bahdanau
08b73bf914 Unhealthy channel is not offered back to the pool.
Motivation:
When releasing unhealthy channel back to a pool we don't have to offer it since on acquire it will be discarded anyways.
 Also checking healthiness at release is a good idea so we don't end up having tons of unhealthy channels in the pool(unless they became unhealthy after being offered)

Modifications:
private SimpleChannelPool.offerIfHealthy() method added that is called from SimpleChannelPool.doReleaseChannel(). SimpleChannelPool.offerIfHealthy() offers channel back to pool only if channel is healthy.
Otherwise it throws setFailure exception to the promise.

 Result:
The pool is now much cleaner and not spammed with unhealthy channels.

Added ability to choose if channel health has to be validated on release by passing boolean flag.

Motivation:
Depending on performance preferences and individual use cases sometimes we would like to be able force health check of a channel at release time and do not offer it back to the pool. Other times we would want to just release channel and offer it back to the pool and check health only when we try to acquire that channel from the pool. See more details here: https://github.com/netty/netty/issues/4077#issuecomment-130461684

Modifications:
Future<Void> release(Channel channel, Promise<Void> promise, boolean offerHealthyOnly);
The offerHealthyOnly boolean flag allows developers to choose whether to do channel validation before offering it back to pool or not.
Appropriate modifications made to hierarchy of implementations of ChannelPool. offerHealthyOnly=true will force channel health to be checked before offering back to pool. offerHealthyOnly=false  will ignore channel health check and will just try just offer it back to the pool
 offerHealthyOnly=true by default.

Result:
Channel health check before offer back to pool is controlled by a flag now.

Code changed to satisfy checkstyle requirements.

Motivation:
Code needs to satisfy checkstyle requirements.

Modifications:
 SimpleChannelPool.java:279 line split to be less then 120 characters.
 SimpleChannelPool.java:280:31 space added after '{'
 SimpleChannelPool.java:282:17 space added after '{'
 SimpleChannelPoolTest.java:198 - extra white space line removed.

Result:
Code satisfies checkstyle requirements.

 offerHealthyOnly is passed as a constructor parameter now.

Motivation:
Instead of passing offerHealthyOnly as a method parameter it is better to pass it in as SimpleChannelPool or FixedChannelPool constructor.

Modifications:
 Redundant release method that takes offerHealthyOnly removed from ChannelPool.
 offerHealthyOnly parameter added to constructor for FixedChannelPool and SimpleChannelPool.

Result:
SimpleChannelPool and FixedChannelPool are now take offerHealthyOnly as a constructor parameter. Default behavior is: offerHealthyOnly=true.

Code changed to satisfy checkstyle requirements.

Motivation:
Code needs to satisfy checkstyle requirements.

Modifications:
 SimpleChannelPool.java:84: line made to be no longer then 120 characters.
 SimpleChannelPool.java:237: extra white space line removed.

Result:
Code satisfies checkstyle requirements.

Tests do not need to be too  copled to the code. Exception message should not be validated

Motivation:
We don't need our tests to be too coupled to the code. Exception type validation in tests is just good enough.

Modifications:
Exception validation message removed from SimpleChannelPoolTest.testUnhealthyChannelIsNotOffered() test.

Result:
The SimpleChannelPoolTest test is less coupled to the code now.

Stack trace set to empty for UNHEALTHY_NON_OFFERED_TO_POOL.

Motivation:
We don't need stack trace for UNHEALTHY_NON_OFFERED_TO_POOL.

Modifications:
Added  UNHEALTHY_NON_OFFERED_TO_POOL.setStackTrace(EmptyArrays.EMPTY_STACK_TRACE) to static init block.

Result:
UNHEALTHY_NON_OFFERED_TO_POOL's stack trace set to empty.

Minor code re-factorings.

Motivation:
For better code readability we need to apply several minor code re-factorings.

Modifications:
javadocs true -> {@code true}
offerHealthyOnly variable name changed to releaseHeathCheck
<p/> -> <p> in javadocs
offerHealthyOnly removed from doReleaseChannel as it not needed there.

Result:
Code quality is improved.

Code changed to satisfy checkstyle requirements.

Motivation:
Code needs to satisfy checkstyle requirements.

Modifications:
SimpleChannelPool.java:87: line made to be no longer then 120 characters.

Result:
Code satisfies checkstyle requirements.

Pull request needs to contain only necessary changes

Motivation:
The pull request should not contain unnecessary changes that are not needed as part of required functionality of pull request.

Modifications:
private void doReleaseChannel(final Channel channel, final Promise<Void> promise) - >  private void doReleaseChannel(Channel channel, Promise<Void> promise)

Result:
Pull request contains less unnecessary modifications.
2015-08-17 15:43:08 -07:00
Norman Maurer
9d417c1626 Fixing compile error, introduce by 5d011b8895 2015-08-12 14:25:45 +02:00
Ivan Bahdanau
5d011b8895 Improve the logic around acquire channel function is improved.
Motivation:
The acquire channel function resulted in calling itself several times in case when channel polled from the pool queue was unhealthy, which resulted FixedChannelPool to be called several times which in it's turn caused FixedChannelPool.acquire() to be called and resulted into acquireChannelCount to be unnecessary increased.
Example use case:
1) Create FixedChannelPool instance with one channel in the pool: new FixedChannelPool(cb, handler, 1)
2) Acquire channel A from the pool
3) close the channel A
4) Return it back to the pool
5) Acquire channel from the same pool again
Expected result:
new channel created and acquired, channel A that has been closed discarded and removed from the pool from being unhealthy
Actual result:
Channel A had been removed from the pool, how ever the new channel had never be acquired, instead the request to acquire had been added to the pending queue in FixedChannelPool and the acquireChannelCount is increased by one. The reason is that at the time when SimpleChannelPool figured out that the channel was unhealthy called FixedChannelPool.acquire to try to acquire new channel, how ever the request was added to the pendingTakQueue because by the time when FixedChannelPool.acquire was called, the acquireChannelCount was already "1" so new channel ould not be created cause of maxChannelsLimit=1.

Modifications:
The suggested approach modifies the SimpleChannelPool in a way so that when channel detected to be unhealthy it calls private method SimpleChannelPool.acquireHealthyFromPoolOrNew() which guarantees that SimpleChannelPool actually either finds a healthy channel in the pool and returns it or causes the promise.cause() in case when new channel was failed to be created.

 Result:
The  ```acquiredChannelCount``` is now calculated correctly as a result of SimpleChannelPool.acquire() of not being recursive on overridable acquire method.
2015-08-12 06:35:56 +02:00
Scott Mitchell
cf171ff525 maxBytesPerRead channel configuration
Motiviation:
The current read loops don't fascilitate reading a maximum amount of bytes. This capability is useful to have more fine grain control over how much data is injested.

Modifications:
- Add a setMaxBytesPerRead(int) and getMaxBytesPerRead() to ChannelConfig
- Add a setMaxBytesPerIndividualRead(int) and getMaxBytesPerIndividualRead to ChannelConfig
- Add methods to RecvByteBufAllocator so that a pluggable scheme can be used to control the behavior of the read loop.
- Modify read loop for all transport types to respect the new RecvByteBufAllocator API

Result:
The ability to control how many bytes are read for each read operation/loop, and a more extensible read loop.
2015-08-05 23:59:54 -07:00
fratboy
083dbe5c59 Correctly count acquired channels when timeout occurs in FixedChannelPool
Motivation:

We don't decrease acquired channel count in FixedChannelPool when timeout occurs by AcquireTimeoutAction.NEW and eventually fails.

Modifications:

Set AcquireTask.acquired=true to call decrementAndRunTaskQueue when timeout action fails.

Result:

Acquired channel count decreases correctly.
2015-07-30 07:55:46 +02:00
Norman Maurer
bd42987143 [#3988] Correctly count acquired channels in FixedChannelPool
Motivation:

We missed to correctly count acquired channels in FixedChannelPool which could produce an assert error.

Modifications:

Only try to decrement acquired count if the channel was really acuired.

Result:

No more assert error possible.
2015-07-21 21:07:13 +02:00
Minwoo Jung
5fb349ca5a Update ChannelConfig.java
{@link ByteBufAllocator} -> {@link MessageSizeEstimator} on
https://github.com/netty/netty/blob/4.0/transport/src/main/java/io/netty/channel/ChannelConfig.java#L248
2015-07-19 18:10:24 +02:00
Norman Maurer
f1c5f0e0c7 [#3967] Guard against NPE in PendingWriteQueue
Motivation:

If the Channel is already closed when the PendingWriteQueue is created it will generate a NPE when add or remove is called later.

Modifications:

Add null checks to guard against NPE.

Result:

No more NPE possible.
2015-07-17 21:31:27 +02:00
Louis Ryan
e6ed3ce6b1 Make CoalescingBufferQueueTest properly release all buffers including the composites removed from the queue.
Motivation:
Test was leaving composite buffers taken from the queue unreleased.

Modifications:
Make the test release buffers.

Result:
Nagging about leaked buffers should stop.
2015-07-16 21:46:45 +02:00
Effective Light
80c59864cf Fix DatagramChannel javadoc
there seems to be an extra arrow incorrectly placed there when trying to link "Channel."
2015-07-12 20:19:59 +02:00
Louis Ryan
cdbf43596d Add generic utility for enqueuing buffers with promises and dequeueing them in arbitrary byte ranges.
Motivation:
Simplifies writing code that needs to merge or slice a sequence of buffer & promise pairs into chunks of arbitrary sizes.
For example in HTTP2 we merge or split buffers across fixed-size DATA frame boundaries.

Modifications:
Add new utility class CoalescingBufferQueue

Result:
Following this change HTTP2 code will switch to use it instead of CompositeByteBuffer for DATA frame coalescing.
2015-07-09 07:50:02 -07:00
Norman Maurer
8650679f29 Fix regression introduced by 11f9e9084b
Motivation:

While cherry-picked 11f9e9084b I changed the EmbeddedChannel implementation to not allow no ChannelHandlers when constructing it.
This was done by mistake.

Modifications:

Revert change and add unit test.

Result:

Restore old behavior.
2015-07-08 21:37:58 +02:00
Norman Maurer
ab43c78c85 Reduce memory usage by EmbeddedChannel
Motivation:

When using an EmbeddedChannel often it either does inbound or outbound processing which means we only often need one queue.

Modifications:

Lazy init the inbound and outbound message queues.

Result:

Less memory usage.
2015-07-08 10:49:40 +02:00
Norman Maurer
0f6559d188 [#3780] Handle ChannelInitializer exception in exceptionCaught()
Motivation:

At the moment we directly closed the Channel when an exception accoured durring initChannel(...) without giving the user any way to do extra or special handling.

Modifications:

Handle the exception in exceptionCaught(...) of the ChannelInitializer which will by default log and close the Channel. This way the user can override this.

Result:

More felixible handling of exceptions.
2015-07-07 09:13:22 +02:00
Norman Maurer
11f9e9084b [#3921] EmbeddedChannel should add ChannelHandlers once registered
Motivation:

Currently in EmbeddedChannel we add the ChannelHandlers before the Channel is registered which leads to have the handlerAdded(...) callback
be called from outside the EventLoop and also prevent the user to obtain a reference to the EventLoop in the callback itself.

Modifications:

Delay adding ChannelHandlers until EmbeddedChannel is registered.

Result:

Correctly call handlerAdded(...) after EmbeddedChannel is registered.
2015-07-07 08:54:27 +02:00
Norman Maurer
dd6ac55fa0 ServerBootstrap.handler(...) will add handler before Channel is registered.
Motivation:

If you set a ChannelHandler via ServerBootstrap.handler(...) it is added to the ChannelPipeline before the Channel is registered. This will lead to and IllegalStateException if a user tries to access the EventLoop in the ChannelHandler.handlerAdded(...) method.

Modifications:

Delay the adding of the ChannelHandler until the Channel was registered.

Result:

No more IllegalStateException.
2015-07-07 08:44:13 +02:00
a-mkarjalainen
e1e8a59ad9 Fix broken constructor chaining for FixedChannelPool class.
Motivation:

Only one of the three FixedChannelPool constructors checks for the constructor
arguments. Therfore it was possible to create a pool with zero maxConnections.

This change chains all constructors together, so that the last one
in the chain always checks the validity of the arguments, regardless of the
constructor used.

Result:

It is no longer possible to create a FixedChannelPool instance with invalid
maxConnections or maxPendingAcquires parameters.
2015-06-18 20:11:04 +02:00
Norman Maurer
4a0d68c50b [#3881] FixedChannelPool creates 1 more channel than maxConnections
Motivation:

FixedChannelPool should enforce a number of maximal used channels, but due a bug we fail to correctly enforce this.

Modifications:

Change check to correctly only acquire channel if we not hit the limit yet.

Result:

Correct limiting.
2015-06-16 20:10:27 +02:00
Scott Mitchell
09d826ed46 Channel writable bytes feature
Motivation:
To avoid buffering too much it would be useful to get an estimate of how many bytes can be written to a Channel before it becomes unwritable.

Modifications:
- Update the Channel interface to support 2 new methods. 1 to give how many bytes before unwritable. 1 to give how many bytes before writable.
- Update the AbstractChannel implementation to delegate to the ChannelOutboundBuffer.

Result:
The Channel interface supports 2 new methods which provide more visibility into writability.
2015-06-12 12:39:53 -07:00