Commit Graph

1900 Commits

Author SHA1 Message Date
Roger Kapsi
149916d052 Adding ability omit the implicit #flush() call in EmbeddedChannel#writeOutbound() and
the implicit #fireChannelReadComplete() in EmbeddedChannel#writeInbound().

Motivation

We use EmbeddedChannels to implement a ProxyChannel of some sorts that shovels
messages between a source and a destination Channel. The latter are real network
channels (such as Epoll) and they may or may not be managed in a ChannelPool. We
could fuse both ends directly together but the EmbeddedChannel provides a nice
disposable section of a ChannelPipeline that can be used to instrument the messages
that are passing through the proxy portion.

The ideal flow looks abount like this:

source#channelRead() -> proxy#writeOutbound() -> destination#write()
source#channelReadComplete() -> proxy#flushOutbound() -> destination#flush()

destination#channelRead() -> proxy#writeInbound() -> source#write()
destination#channelReadComplete() -> proxy#flushInbound() -> source#flush()

The problem is that #writeOutbound() and #writeInbound() emit surplus #flush()
and #fireChannelReadComplete() events which in turn yield to surplus #flush()
calls on both ends of the pipeline.

Modifications

Introduce a new set of write methods that reain the same sematics as the #write()
method and #flushOutbound() and #flushInbound().

Result

It's possible to implement the above ideal flow.

Fix for EmbeddedChannel#ensureOpen() and Unit Tests for it

Some PR stuff.
2016-09-24 12:33:04 -07:00
ChuntaoLu
9008e72c2b Fix javadoc
Removes unmatched brace
2016-09-12 06:43:10 -07:00
Norman Maurer
6bbf32134a Log more details if notification of promise fails in PromiseNotifier and AbstractChannelHandlerContext
Motivation:

To make it easier to debug why notification of a promise failed we should log extra info and make it consistent.

Modifications:

- Create a new PromiseNotificationUtil that has static methods that can be used to try notify a promise and log.
- Reuse this in AbstractChannelHandlerContext, ChannelOutboundBuffer and PromiseNotifier

Result:

Easier to debug why a promise could not be notified.
2016-09-07 06:55:38 +02:00
Norman Maurer
dfa3bbbf00 Add support for Client Subnet in DNS Queries (RFC7871)
Motivation:

RFC7871 defines an extension which allows to request responses for a given subset.

Modifications:

- Add DnsOptPseudoRrRecord which can act as base class for extensions based on EDNS(0) as defined in RFC6891
- Add DnsOptEcsRecord to support the Client Subnet in DNS Queries extension
- Add tests

Result:

Client Subnet in DNS Queries extension is now supported.
2016-09-06 07:16:57 +02:00
buchgr
4af6855695 Remove @Deprecated for primitive WriteWaterMark getters and setters
Motivation:

For use cases that demand frequent updates of the write watermarks, an
API that requires immutable WriteWaterMark objects is not ideal, as it
implies a lot of object allocation.

For example, the HTTP/2 child channel API uses write watermarks for outbound
flow control and updates the write watermarks on every DATA frame write.

Modifications:

Remote @Deprecated tag from primitive getters and setters, however the corresponding
channel options remain deprecated.

Result:

Primitive getters and setters for write watermarks are no longer marked @Deprecated.
2016-09-05 10:26:05 +02:00
Norman Maurer
30fe2e868f Call finishConnect() before try to call read(...) / write(...) when using NIO
Motivation:

The JDK implementation of SocketChannel has an internal state that is tracked for its operations. Because of this we need to ensure we call finishConnect() before try to call read(...) / write(...) as otherwise it may produce a NotYetConnectedException.

Modifications:

First process OP_CONNECT flag.

Result:

No more possibility of NotYetConnectedException because OP_CONNECT is handled not early enough when processing interestedOps for a Channel.
2016-09-01 08:55:02 +02:00
Norman Maurer
54b1a100f4 [maven-release-plugin] prepare for next development iteration 2016-08-26 10:06:32 +02:00
Norman Maurer
6fd8bb8c63 [#5763] DefaultEventLoopGroup doesn't expose ctor variant that accepts custom Executor
Motivation:

The DefaultEventLoopGroup class extends MultithreadEventExecutorGroup but doesn't expose the ctor variants that accept a custom Executor like NioEventLoopGroup and EpollEventLoopGroup do.

Modifications:

Add missing constructor.

Result:

Be able to use custom Executor with DefaultEventLoopGroup.
2016-09-01 08:20:05 +02:00
Jason Tedor
00c0664ef8 Avoid inaccessible object exception replacing set
Motivation:

When attempting to set the selectedKeys fields on the selector
implementation, JDK 9 can throw an inaccessible object exception.

Modications:

Catch and log this exception as an possible course of action if the
sun.nio.ch package is not exported from java.base.

Result:

The selector replacement will fail gracefully as an expected course of
action if the sun.nio.ch package is not exported from java.base.
2016-08-31 13:59:48 +02:00
Norman Maurer
1208b90f57 [maven-release-plugin] prepare release netty-4.1.5.Final 2016-08-26 04:59:35 +02:00
Norman Maurer
d3cb95ef00 Make NIO and EPOLL transport connect errors more consistent with the JDK
Motivation:

The NIO transport used an IllegalStateException if a user tried to issue another connect(...) while the connect was still in process. For this case the JDK specified a ConnectPendingException which we should use. The same issues exists in the EPOLL transport. Beside this the EPOLL transport also does not throw the right exceptions for ENETUNREACH and EISCONN errno codes.

Modifications:

- Replace IllegalStateException with ConnectPendingException in NIO and EPOLL transport
- throw correct exceptions for ENETUNREACH and EISCONN in EPOLL transport
- Add test case

Result:

More correct error handling for connect attempts when using NIO and EPOLL transport
2016-08-27 20:57:36 +02:00
buchgr
4accd300e5 Fix write watermarks comparison to use less than and greater than.
Motivation:

The API documentation in ChannelConfig states that a a channel is writable,
if the number of pending bytes is below the low watermark and a
channel is not writable, if the number of pending bytes exceeds the high
watermark.

Therefore, we should use < operators instead of <= as well as > instead of >=.

Using <= and >= is also problematic, if the low watermark is equal to the high watermark,
as then a channel could be both writable and unwritable with the same number of pending
bytes (depending on whether remove() or addMessage() is called first).

The use of <= and >= was introduced in PR https://github.com/netty/netty/pull/3036, but
I don't understand why, as there doesn't seem to have been any discussion around that.

Modifications:

Use < and > operators instead of <= and >=.

Result:

High and low watermarks are treated as stated in the API docs.
2016-08-24 15:58:02 +02:00
Norman Maurer
5e148d5670 [#5639] Ensure fireChannelActive() is also called if Channel is closed in connect promise.
Motivation:

We need to ensure we also call fireChannelActive() if the Channel is directly closed in a ChannelFutureListener that is belongs to the promise for the connect. Otherwise we will see missing active events.

Modifications:

Ensure we always call fireChannelActive() if the Channel was active.

Result:

No missing events.
2016-08-24 08:47:49 +02:00
Norman Maurer
43626ac6ea Use NIO methods when using Java7+ in the NIO transport
Motivation:

We use often javachannel().socket().* in NIO as these methods exists in java6. The problem is that these will throw often very general Exceptions (Like SocketException) while it is more expected to throw the Exceptions listed in the nio interfaces. When possible we should use the new methods available in java7+ which throw the correct exceptions.

Modifications:

Check for java version and depending on it using the socket or the javachannel.

Result:

Throw expected Exceptions.
2016-08-24 07:36:14 +02:00
Norman Maurer
8ce7e73e78 Prevent extra peformance hit by fillInStackTrace() when create a new annotated connect exception.
Motivation:

To make it easier to debug connect exceptions we create new exceptions which also contain the remote address. For this we basically created a new instance and call setStackTrace(...). When doing this we pay an extra penality because it calls fillInStackTrace() when calling the super constructor.

Modifications:

Create special sub-classes of Exceptions that override the fillInStackTrace() method and so eliminate the overhead.

Result:

Less overhead when "annotate" connect exceptions.
2016-08-24 07:34:43 +02:00
buchgr
8d1e46ffd1 Remove reference to 5.0 release.
Motivation:

Comments stating that AUTO_CLOSE will be removed in Netty 5.0 are wrong,
as there is no Netty 5.0.

Modifications:

Removed comment.

Result:

No more references to Netty 5.0
2016-08-23 09:46:57 +02:00
Norman Maurer
6d70f4b38a Guard against re-entrance in PendingWriteQueue.removeAndWriteAll()
Motivation:

PendingWriteQueue should guard against re-entrant writes once removeAndWriteAll() is run.

Modifications:

Continue writing until queue is empty.

Result:

Correctly guard against re-entrance.
2016-08-18 07:13:19 +02:00
tbcs
95ee705e17 fix typo in javadoc 2016-08-12 20:42:37 +02:00
Jason Tedor
a6dfd08812 Mark initialization of selector as privileged
Motivation:

Instrumenting the NIO selector implementation requires special
permissions. Yet, the code for performing this instrumentation is
executed in a manner that would require all code leading up to the
initialization to have the requisite permissions. In a restrictive
environment (e.g., under a security policy that only grants the
requisite permissions the Netty transport jar but not to application
code triggering the Netty initialization), then instrumeting the
selector will not succeed even if the security policy would otherwise
permit it.

Modifications:

This commit marks the necessary blocks as privileged. This enables
access to the necessary resources for instrumenting the selector. The
idea is that we are saying the Netty code is trusted, and as long as the
Netty code has been granted the necessary permissions, then we will
allow the caller access to these resources even though the caller itself
might not have the requisite permissions.

Result:

The selector can be instrumented in a restrictive security environment.
2016-08-05 19:01:57 +02:00
Jason Tedor
32629078a2 Mark setting of sun.nio.ch.bugLevel as privileged
Motivation:

Writing to a system property requires permissions. Yet the code for
setting sun.nio.ch.bugLevel is not marked as privileged. In a
restrictive environment (e.g., under a security policy that only grants
the requisite permissions the Netty transport jar but not to application
code triggering the Netty initialization), writing to this system
property will not succeed even if the security policy would otherwise
permit it.

Modifications:

This commt marks the necessary code block as privileged. This enables
writing to this system property. The idea is that we are saying the
Netty code is trusted, and as long as the Netty code has been granted
the necessary permissions, then we will allow the caller access to these
resources even though the caller itself might not have the requisite
permissions.

Result:

The system property sun.nio.ch.bugLevel can be written to in a
restrictive security environment.
2016-08-05 18:58:34 +02:00
Norman Maurer
aa6e6ae307 [#4241] Ensure NioEventLoopGroup.shutdownGracefully(...) with no quiet period shutdown as fast as expected.
Motivation:

If the user uses 0 as quiet period we should shutdown without any delay if possible.

Modifications:

Ensure we not introduce extra delay when a shutdown quit period of 0 is used.

Result:

EventLoop shutdown as fast as expected.
2016-08-05 07:21:17 +02:00
Norman Maurer
26aa34853a Ensure correct ordering if a ChannelInitializer adds another ChannelInitializer
Motivation:

At the moment we call initChannel(...) in the channelRegistered(...) method which has the effect that if another ChannelInitializer is added within the initChannel(...) method the ordering of the added handlers is not correct and surprising. This is as the whole initChannel(...) method block is executed before the initChannel(...) block of the added ChannelInitializer is handled.

Modifications:

Call initChannel(...) from within handlerAdded(...) if the Channel is registered already. This is true in all cases for our DefaultChannelPipeline implementation. This way the ordering is always as expected. We still keep the old behaviour as well to not break code for other ChannelPipeline implementations (if someone ever wrote one).

Result:

Correct and expected ordering of ChannelHandlers.
2016-08-03 07:50:46 +02:00
Norman Maurer
f585806a74 [#5598] Ensure SslHandler not log false-positives when try to close the channel due timeout.
Motivation:

When we try to close the Channel due a timeout we need to ensure we not log if the notification of the promise fails as it may be completed in the meantime.

Modifications:

Add another constructor to ChannelPromiseNotifier and PromiseNotifier which allows to log on notification failure.

Result:

No more miss-leading logs.
2016-07-30 21:15:09 +02:00
Norman Maurer
cb7cf4491c [maven-release-plugin] prepare for next development iteration 2016-07-27 13:29:56 +02:00
Norman Maurer
9466b32d05 [maven-release-plugin] prepare release netty-4.1.4.Final 2016-07-27 13:16:59 +02:00
Norman Maurer
b97a36a10f Add test to verify that its possible to add another ChannelInitializer in the initChannel(...) method.
Motivation:

I received a report the its not possible to add another ChannelInitialiter in the initChannel(...) method, so we should add a test case for it.

Modifications:

Added testcase.

Result:

Validate that all works as expected.
2016-07-27 09:25:13 +02:00
Norman Maurer
4638df2062 [#5566] Ensure using a ChannelInitializer via ServerBootstrap.handler(...) produce correct ordering.
Motivation:

When a ChannelInitializer is used via ServerBootstrap.handler(...) the users handlers may be added after the internal ServerBootstrapAcceptor. This should not happen.

Modifications:

Delay the adding of the ServerBootstrapAcceptor until the initChannel(....) method returns.

Result:

Correct order of handlers in the ServerChannels ChannelPipeline.
2016-07-27 08:34:31 +02:00
Norman Maurer
dc6c6d956a [#5541] Ensure failing a Promise in SimpleChannelPool will not result in stack overflow.
Motivation:

We used Promise.setFailure(...) when fail a Promise in SimpleChannelPool. As this happens in multiple levels this can result in stackoverflow as setFailure(...) may throw an IllegalStateException which then again is propergated.

Modifications:

Use tryFailure(...)

Result:

No more possibility to cause a stack overflow when failing the promise.
2016-07-27 07:57:58 +02:00
Norman Maurer
b8400f9628 [#5553] SimpleChannelPool#notifyConnect() may leak Channels
Motivation:

The SimpleChannelPool#notifyConnect() method will leak Channels if the user cancelled the Promise in between.

Modifications:

Release the channel if the Promise was complete before.

Result:

No more channel leaks.
2016-07-20 20:17:19 +02:00
Jason Tedor
d262f7c189 Reduce permissions needed for process ID
Motiviation:

DefaultChannelId attempts to acquire a default process ID by determining
the process PID. However, to do this it attempts to punch through to the
system classloader, a permission that in the face of a restrictive
security manager is unlikely to be granted. Looking past this, it then
attempts to load a declared method off a reflectively loaded class,
another permission that is not likely to be granted in the face of a
restrictive security manager. However, neither of these permissions are
necessary as the punching through to the system security manager is
completely unneeded, and there is no need to load a public method as a
declared method.

Modifications:

Instead of punching through to the system classloader requiring
restricted permissions, we can just use current classloader. To address
the access declared method permission, we instead just reflectively
obtain the desired public method via Class#getMethod.

Result:

Acquiring the default process ID from the PID will succeed without
requiring the runtime permissions "getClassLoader" and
"accessDeclaredMembers".
2016-07-20 19:47:56 +02:00
Scott Mitchell
3d7ae97359 Make Epoll ChannelMetadata more consistent with NIO
Motivation:
In 4.0 AbstractNioByteChannel has a default of 16 max messages per read. However in 4.1 that constraint was applied at the NioSocketChannel which is not equivalent. In 4.1 AbstractEpollStreamChannel also did not have the default of 16 max messages per read applied.

Modifications:
- Make Nio consistent with 4.0
- Make Epoll consistent with Nio

Result:
Nio and Epoll both have consistent ChannelMetadata and are consistent with 4.0.
2016-07-18 13:26:05 +02:00
Norman Maurer
047f6aed28 [maven-release-plugin] prepare for next development iteration 2016-07-15 09:09:13 +02:00
Norman Maurer
b2adea87a0 [maven-release-plugin] prepare release netty-4.1.3.Final 2016-07-15 09:08:53 +02:00
Nitesh Kant
77770374fb Ability to run a task at the end of an eventloop iteration.
Motivation:

This change is part of the change done in PR #5395 to provide an `AUTO_FLUSH` capability.
Splitting this change will enable to try other ways of implementing `AUTO_FLUSH`.

Modifications:



Two methods:

```java
void executeAfterEventLoopIteration(Runnable task);


boolean removeAfterEventLoopIterationTask(Runnable task);
```
are added to `SingleThreadEventLoop` class for adding/removing a task to be executed at the end of current/next iteration of this `eventloop`.

In order to support the above, a few methods are added to `SingleThreadEventExecutor`

```java
protected void afterRunningAllTasks() { }
```

This is invoked after all tasks are run for this executor OR if the passed timeout value for `runAllTasks(long timeoutNanos)` is expired.

Added a queue of `tailTasks` to `SingleThreadEventLoop` to hold all tasks to be executed at the end of every iteration.


Result:



`SingleThreadEventLoop` now has the ability to execute tasks at the end of an eventloop iteration.
2016-07-12 10:22:15 +02:00
Norman Maurer
b37a41a535 Allow to get the number of bytes queued in PendingWriteQueue
Motivation:

For some use-cases it would be useful to know the number of bytes queued in the PendingWriteQueue without the need to dequeue them.

Modifications:

Add PendingWriteQueue.bytes().

Result:

Be able to get the number of bytes queued.
2016-07-11 09:05:20 +02:00
Norman Maurer
50a74e95f2 Ensure ChannelHandler.handlerAdded(...) callback is executed directly when added from ChannelFutureListener added to the registration future.
Motivation:

Commit 4c048d069d moved the logic of calling handlerAdded(...) to the channelRegistered(...) callback of the head of the DefaultChannelPipeline. Unfortunatlly this may execute the callbacks to late as a user may add handlers to the pipeline in the ChannelFutureListener attached to the registration future. This can lead to incorrect ordering.

Modifications:

Ensure we always invoke ChannelHandler.handlerAdded(...) for all handlers before the registration promise is notified.

Result:

Not possible of incorrect ordering or missed events.
2016-07-09 08:04:15 +02:00
Norman Maurer
061899f2a7 Allow to remove pinning of EventExecutor for EventExecutorGroup
Motivation:

We pinned the EventExecutor for a Channel in DefaultChannelPipeline. Which means if the user added multiple handlers with the same EventExecutorGroup to the ChannelPipeline it will use the same EventExecutor for all of these handlers. This may be unexpected and even not what the user wants. If the user want to use the same one for all of them it can be done by obtain an EventExecutor and pass the same instance to the add methods. Because of this we should allow to not pin.

Modifications:

Allow to disable pinning of EventExecutor for Channel based on EventExecutorGroup via ChannelOption.

Result:

Less confusing and more flexible usage of EventExecutorGroup when adding ChannelHandlers to the ChannelPipeline.
2016-07-08 20:09:16 +02:00
Norman Maurer
64bf167423 Fix flacky test introducd by 29fdb160f3 2016-07-07 22:53:08 +02:00
Roger Kapsi
02850da480 Rename ChannelHandlerContext#fireUserEventTriggered() argument from event to evt so it matches the ChannelInboundHandler#userEventTriggered() argument's name.
Motivation

When I override ChannelHandler methods I usually (always) refire events myself via
ChannelHandlerContext instead of relieing on calling the super method (say
`super.write(ctx, ...)`). This works great and the IDE actually auto completes/generates
the right code for it except `#fireUserEventTriggered()` and `#userEventTriggered()`
which have a mismatching argument names and I have to manually "intervene".

Modification

Rename `ChannelHandlerContext#fireUserEventTriggered()` argument from `event` to `evt`
to match its handler counterpart.

Result

The IDE's auto generated code will reference the correct variable.
2016-07-07 17:01:19 +02:00
Norman Maurer
29fdb160f3 [#5486] Not operate on serial execution assumption when using EventExecutor in the DefaultChannelPipeline.
Motivation:

In commit f984870ccc I made a change which operated under invalide assumption that tasks executed by an EventExecutor will always be processed in a serial fashion. This is true for SingleThreadEventExecutor sub-classes but not part of the EventExecutor interface contract.

Because of this change implementations of EventExecutor which not strictly execute tasks in a serial fashion may miss events before handlerAdded(...) is called. This is strictly speaking not correct as there is not guarantee in this case that handlerAdded(...) will be called as first task (as there is no ordering guarentee).

Cassandra itself ships such an EventExecutor implementation which has no strict ordering to spread load across multiple threads.

Modifications:

- Add new OrderedEventExecutor interface and let SingleThreadEventExecutor / EventLoop implement / extend it.
- Only expose "restriction" of skipping events until handlerAdded(...) is called for OrderedEventExecutor implementations
- Add ThreadPoolEventExecutor implementation which executes tasks in an unordered fashion. This is used in added unit test but can also be used for protocols which not expose an strict ordering.
- Add unit test.

Result:

Resurrect the possibility to implement an EventExecutor which does not enforce serial execution of events and be able to use it with the DefaultChannelPipeline.
2016-07-07 15:01:56 +02:00
Norman Maurer
c393374cf5 [#5455] Clarify ChannelPool javadocs
Motivation:

We should make it clear that each acquired Channel needs to be released in all cases.

Modifications:

More clear javadocs.

Result:

Harder for users to leak Channel.
2016-07-07 06:46:28 +02:00
Norman Maurer
195d7476d5 Deprecate methods in AbstractChannel that have no real usage.
Motivation:

We should deprecate methods that are not used.

Modifications:

Add @Deprecated to methods

Result:

Be able to cleanup stuff sooner.
2016-07-07 06:41:43 +02:00
buchgr
0fbb791ad6 Make AbstractChannel.outboundBuffer volatile.
Motivation:

The field can be read from arbitrary threads via Channel.(isWritable()|bytesBeforeWritable()|bytesBeforeUnwritable()), WriteAndFlushTask.newInstance(), PendingWriteQueue, etc.

Modifications:

Make AbstractChannel.outboundBuffer volatile.

Result:

More correct in a concurrent use case.
2016-07-06 21:25:33 +02:00
Norman Maurer
44b942e507 Rename future to promise in the ChannelDuplexHandler method arguments.
Motivation:

We used future in many method of ChannelDuplexHandler as argument name of ChannelPromise. We should make it more consistent and correct.

Modifications:

Replace future with promise.

Result:

More correct and consistent naming.
2016-07-06 08:55:29 +02:00
Norman Maurer
4676a2271c [maven-release-plugin] prepare for next development iteration 2016-07-01 10:33:32 +02:00
Norman Maurer
ad270c02b9 [maven-release-plugin] prepare release netty-4.1.2.Final 2016-07-01 09:07:40 +02:00
Norman Maurer
731f52fdf7 Allow to inject RejectedExecutionHandler for different EventLoops and EventExecutors
Motiviation:

Sometimes it is useful to allow to specify a custom strategy to handle rejected tasks. For example if someone tries to add tasks from outside the eventloop it may make sense to try to backoff and retries and so give the executor time to recover.

Modification:

Add RejectedEventExecutor interface and implementations and allow to inject it.

Result:

More flexible handling of executor overload.
2016-06-24 17:08:30 +02:00
Norman Maurer
a0d4fb16fc Allow to set max capacity for task queue for EventExecutors and EventLoops
Motivation:

To restrict the memory usage of a system it is sometimes needed to adjust the number of max pending tasks in the tasks queue.

Modifications:

- Add new constructors to modify the number of allowed pending tasks.
- Add system properties to configure the default values.

Result:

More flexible configuration.
2016-06-24 14:00:50 +02:00
Norman Maurer
278c36af0c Merge ThrowableUtils into ThrowableUtil.
Motivation:

We should merge ThrowableUtils into ThrowableUtil as this name is more consistent with the naming of utility classes in netty.

Modifications:

Merge classes.

Result:

More consistent naming
2016-06-23 10:05:41 +02:00
Tim Brooks
d964bf6f18 Remove usages of deprecated methods group() and childGroup().
Motivation:

These methods were recently deprecated. However, they remained in use in several locations in Netty's codebase.

Modifications:

Netty's code will now access the bootstrap config to get the group or child group.

Result:

No impact on functionality.
2016-06-21 14:06:57 +02:00
Norman Maurer
31e0ac1d22 Expose the SelectorProvider used by the NioEventLoop
Motivation:

If a user writes an own nio based transport which uses a special SelectorProvider it is useful to be able to get the SelectorProvider that is used by a NioEventLoop. This way this can be used when implement AbstractChannel.isCompatible(...) and check that the SelectorProvider is the correct one.

Modifications:

Expose the SelectorProvider.

Result:

Be able to get the SelectorProvider used by a NioEventLoop.
2016-06-21 14:05:44 +02:00
Norman Maurer
e845670043 Set some StackTraceElement on pre-instantiated static exceptions
Motivation:

We use pre-instantiated exceptions in various places for performance reasons. These exceptions don't include a stacktrace which makes it hard to know where the exception was thrown. This is especially true as we use the same exception type (for example ChannelClosedException) in different places. Setting some StackTraceElements will provide more context as to where these exceptions original and make debugging easier.

Modifications:

Set a generated StackTraceElement on these pre-instantiated exceptions which at least contains the origin class and method name. The filename and linenumber are specified as unkown (as stated in the javadocs of StackTraceElement).

Result:

Easier to find the origin of a pre-instantiated exception.
2016-06-20 11:33:05 +02:00
Norman Maurer
8f3a5e5b18 Fix typo in log message
Motivation:

We had a typo in the log message.

Modifications:

Remove extra "s" in log message.

Result:

Correct spelling in log message.
2016-06-17 06:23:00 +02:00
Norman Maurer
418550914a Log Selector instance when Selector needs to be rebuild
Motivation:

To better debug why a Selector need to be rebuild it is useful to also log the instance of the Selector.

Modifications:

Add logger instance to the log message.

Result:

More useful log message.
2016-06-17 06:19:23 +02:00
Stephane Landelle
9bfeab2c8a AbstractBootstrap can crash instead of failing promise, close #5387
Motivation:
When `ChannelFactory#newChannel` crashed, `AbstractBootstrap#initAndRegister` propagates the exception to the caller instead of failing the promise.

Modifications:
- Catch exceptions from `ChannelFactory#newChannel`.
- Notify promise of such failure.

Result:
`AbstractBootstrap` gracefully handles connect failures.
2016-06-13 18:59:09 +02:00
Dmitry Spikhalskiy
428c61673b Logs in invokeExceptionCaught have been made consistent and full
Motivation:

In case of exception in invokeExceptionCaught() only original exception passed to invokeExceptionCaught() will be logged on any log level.
+ AbstractChannelHandlerContext and CombinedChannelDuplexHandler log different exceptions.

Modifications:

Fix inconsistent logging code and add ability to see both stacktraces on DEBUG level.

Result:

Both handlers log now both original exception and thrown from invokeExceptionCaught. To see full stacktrace of exception thrown from invokeExceptionCaught DEBUG log level must be enabled.
2016-06-11 20:11:11 +02:00
Guido Medina
c3abb9146e Use shaded dependency on JCTools instead of copy and paste
Motivation:
JCTools supports both non-unsafe, unsafe versions of queues and JDK6 which allows us to shade the library in netty-common allowing it to stay "zero dependency".

Modifications:
- Remove copy paste JCTools code and shade the library (dependencies that are shaded should be removed from the <dependencies> section of the generated POM).
- Remove usage of OneTimeTask and remove it all together.

Result:
Less code to maintain and easier to update JCTools and less GC pressure as the queue implementation nt creates so much garbage
2016-06-10 13:19:45 +02:00
Jon Chambers
829be86223 Use a default resolver with bootstrap.resolver(null).
Motivation:

`Bootstrap` has a notion of a default resolver group, but it's hidden from the public. To allow callers to reset a `Bootstrap` instance's resolver group, we could either make `DEFAULT_RESOLVER` public, or we could allow callers to pass `null` as an argument to `Bootstrap#resolver(AddressResolverGroup<?>)`. This pull request does the latter.

Modifications:

- Allow `Bootstrap#resolver(AddressResolverGroup<?>)` to accept `null` as an argument

Result:

Callers may pass `null` to `Bootstrap#resolver(AddressResolverGroup<?>)` to cause the `Bootstrap` instance to use its default resolver group.
2016-06-08 06:57:55 +02:00
Norman Maurer
4dec7f11b7 [maven-release-plugin] prepare for next development iteration 2016-06-07 18:52:34 +02:00
Norman Maurer
cf670fab75 [maven-release-plugin] prepare release netty-4.1.1.Final 2016-06-07 18:52:22 +02:00
Yuri Schimke
a14eda7db0 typo: Skelton 2016-06-06 16:28:21 -07:00
Norman Maurer
b461c9d54c Allow to specify a custom EventExecutorChooserFactory. Related to [#1230]
Motivation:

Sometimes it may be benefitially for an user to specify a custom algorithm when choose the next EventExecutor/EventLoop.

Modifications:

Allow to specify a custom EventExecutorChooseFactory that allows to customize algorithm.

Result:

More flexible api.
2016-06-06 11:04:56 +02:00
Norman Maurer
e847ac0443 Fix possible deadlock in DefaultChannelPipeline.destroyDown(...)
Motivation:

We need to ensure we not hold a lock while executor callHandlerRemoved(...) as this may lead to a deadlock if handlerRemoved(...) will call another method in DEfaultChannelPipeline from another thread that will need to obtain the lock as well and wait for the result.

Modifications:

Release the lock before call handlerRemoved0(...).

Result:

No more deadlock possible
2016-06-04 09:13:54 +02:00
Norman Maurer
f8b306f61c [#5313] Correctly catch errors during bootstrap.
Motivation:

We not correctly catched errors during resolving in bootstrap and so may not have notified the future correctly.

Modifications:

Move code into try / catch block and try to fail the promise.

Result:

Promise is always notified
2016-06-04 09:12:54 +02:00
Norman Maurer
c43b424f56 Add missing null check that was missed in 844976a0a2 2016-05-31 14:05:03 +02:00
Norman Maurer
844976a0a2 Ensure the same ByteBufAllocator is used in the EmbeddedChannel when compress / decompress. Related to [#5294]
Motivation:

The user may specify to use a different allocator then the default. In this case we need to ensure it is shared when creating the EmbeddedChannel inside of a ChannelHandler

Modifications:

Use the config of the "original" Channel in the EmbeddedChannel and so share the same allocator etc.

Result:

Same type of buffers are used.
2016-05-31 09:08:33 +02:00
Norman Maurer
339b512e70 Fix small race in DefaultChannelPipeline introduced by a729e0fcd9
Motivation:

There is a small race while adding handlers to the pipeline because callHandlerAddedForAllHandlers() may not be run when the user calls add* but the Channel is already registered.

Modifications:

Ensure we always delay handlerAdded(..) / handlerRemoved(...) until callHandlerAddedForAllHandlers() was called.

Result:

No more race on pipeline modifications possible.
2016-05-30 15:05:48 +02:00
Norman Maurer
dcd93e3be0 Remove volatile where not needed.
Motivation:

We can remove the volatile keyword from the cached Runnables as at worse these will just be re-created.

Modifications:

Remove volatile.

Result:

Less overhead.
2016-05-30 07:32:45 +02:00
Norman Maurer
86f53083e7 [#5297] Ensure calling NioEventLoop.pendingTasks() and EpollEventLoop.pendingTasks() will not produce livelock
Motivation:

SingleThreadEventExecutor.pendingTasks() will call taskQueue.size() to get the number of pending tasks in the queue. This is not safe when using MpscLinkedQueue as size() is only allowed to be called by a single consumer.

Modifications:

Ensure size() is only called from the EventLoop.

Result:

No more livelock possible when call pendingTasks, no matter from which thread it is done.
2016-05-28 20:04:07 +02:00
Norman Maurer
6ca49d1336 [maven-release-plugin] prepare for next development iteration 2016-05-25 19:16:44 +02:00
Norman Maurer
446b38db52 [maven-release-plugin] prepare release netty-4.1.0.Final 2016-05-25 19:14:15 +02:00
Norman Maurer
2618c2a649 [#5239] Allow adding handlers to pipeline with null name.
Motivation:

While doing 8fe3c83e4c I made a change which disallowed using null as name for handlers in the pipeline (this generated a new name before).

Modifications:

Revert to old behaviour and adding test case.

Result:

Allow null name again
2016-05-24 06:22:44 +02:00
Norman Maurer
26a175cd94 Fix flacky test which was missed when commit 8fe3c83e4c 2016-05-22 19:33:48 +02:00
Norman Maurer
7b25402e80 Add CompositeByteBuf.addComponent(boolean ...) method to simplify usage
Motivation:

At the moment the user is responsible to increase the writer index of the composite buffer when a new component is added. We should add some methods that handle this for the user as this is the most popular usage of the composite buffer.

Modifications:

Add new methods that autoamtically increase the writerIndex when buffers are added.

Result:

Easier usage of CompositeByteBuf.
2016-05-21 19:52:16 +02:00
Xiaoyan Lin
a5006c1969 Add a EventLoopGroup.register(ChannelPromise)
Motivation:

EventLoopGroup.register doesn't need the Channel paramter when ChannelPromise is provided as we can get the Channel from ChannelPromise. Resolves #2422.

Modifications:

- Add EventLoopGroup.register(ChannelPromise)
- Deprecate EventLoopGroup.register(Channel, ChannelPromise)

Result:

EventLoopGroup.register is more convenient as people only need to set one parameter.
2016-05-21 18:40:17 +02:00
Norman Maurer
a729e0fcd9 Decouple DefaultChannelPipeline from AbstractChannel
Motivation:

DefaultChannelPipeline was tightly coupled to AbstractChannel which is not really needed.

Modifications:

Move logic of calling handlerAdded(...) for handlers that were added before the Channel was registered to DefaultChannelPipeline by making it part of the head context.

Result:

Less coupling and so be able to use DefaultChannelPipeline also with other Channel implementations that not extend AbstractChannel
2016-05-21 17:14:00 +02:00
Norman Maurer
0838f223e1 Decouple AbstractChannel and AbstractChannelHandlerContext
Motivation:

We do a "blind" cast to AbstractChannel in AbstractChannelHandlerContext which we should better no do. It would be better to decouble AbstractChannelHandlerContext from AbstractChannel.

Modifications:

Decouble AbstractChannelHandlerContext from AbstractChannel by move logic to DefaultChannelPipeline

Result:

Less coubling and less casting.
2016-05-21 10:46:21 +02:00
Norman Maurer
7547a448e0 [#4906] Ensure addLast(...) works as expected in EmbeddedChannel
Motivation:

If the user will use addLast(...) on the ChannelPipeline of EmbeddedChannel after its constructor was run it will break the EmbeddedChannel as it will not be able to collect inbound messages and exceptions.

Modifications:

Ensure addLast(...) work as expected by move the logic of handling messages and exceptions ti protected methods of DefaultChannelPipeline and use a custom implementation for EmbeddedChannel

Result:

addLast(...) works as expected when using EmbeddedChannel.
2016-05-21 10:33:25 +02:00
Norman Maurer
e10dca7601 Mark Recycler.recycle(...) deprecated and update usage.
Motivation:

Recycler.recycle(...) should not be used anymore and be replaced by Handle.recycle().

Modifications:

Mark it as deprecated and update usage.

Result:

Correctly document deprecated api.
2016-05-20 22:11:31 +02:00
Norman Maurer
2e352b75ac Remove unused class
Motivation:

We should remove unused classes that are package-private

Modifications:

Remove unused class

Result:

Less cruft in code-base.
2016-05-20 21:53:59 +02:00
Norman Maurer
a425a8551d [#5174] Expose Bootstrap getter methods and add some additional ones
Motivation:

The Bootstrap class (applies also to AbstractBootstrap and ServerBootstrap) has a few package private getter methods and some things such as #attr() and #options() aren't exposed at all.

Modifications:

Expose "getters" for configured things in a safe-manner via the config() method.

Result:

Easier for the user to check what is configured for a Bootstrap/ServerBootstrap.
2016-05-18 13:26:43 +02:00
Scott Mitchell
2b340df452 DuplexChannel to support shutdownInput
Motivation:
The DuplexChannel is currently incomplete and only supports shutting down the output side of a channel. This interface should also support shutting down the input side of the channel.

Modifications:
- Add shutdownInput and shutdown methods to the DuplexChannel interface
- Remove state in NIO and OIO for tracking input being shutdown independent of the underlying transport's socket type. Tracking the state independently may lead to inconsistent state.

Result:
DuplexChannel supports shutting down the input side of the channel
Fixes https://github.com/netty/netty/issues/5175
2016-05-18 09:11:49 +02:00
Norman Maurer
8fe3c83e4c [#5104] Fix possible deadlock in DefaultChannelPipeline
Motivation:

When a user has multiple EventLoops in an EventLoopGroup and calls pipeline.add* / remove* / replace from an EventLoop that belongs to another Channel it is possible to deadlock if the other EventLoop does the same.

Modification:

- Only ensure the actual modification takes place in a synchronized block and not wait until the handlerAdded(...) / handlerRemoved(...) method is called. This is ok as we submit the task to the executor while still holding the look and so ensure correct order of pipeline modifications.
- Ensure if an AbstractChannelHandlerContext is put in the linked-list structure but the handlerAdded(...) method was not called we skip it until handlerAdded(...) was called. This is needed to ensure handlerAdded(...) is always called first.

Result:

Its not possible to deadlock when modify the DefaultChannelPipeline.
2016-05-17 14:24:46 +02:00
Trustin Lee
3a9f472161 Make retained derived buffers recyclable
Related: #4333 #4421 #5128

Motivation:

slice(), duplicate() and readSlice() currently create a non-recyclable
derived buffer instance. Under heavy load, an application that creates a
lot of derived buffers can put the garbage collector under pressure.

Modifications:

- Add the following methods which creates a non-recyclable derived buffer
  - retainedSlice()
  - retainedDuplicate()
  - readRetainedSlice()
- Add the new recyclable derived buffer implementations, which has its
  own reference count value
- Add ByteBufHolder.retainedDuplicate()
- Add ByteBufHolder.replace(ByteBuf) so that..
  - a user can replace the content of the holder in a consistent way
  - copy/duplicate/retainedDuplicate() can delegate the holder
    construction to replace(ByteBuf)
- Use retainedDuplicate() and retainedSlice() wherever possible
- Miscellaneous:
  - Rename DuplicateByteBufTest to DuplicatedByteBufTest (missing 'D')
  - Make ReplayingDecoderByteBuf.reject() return an exception instead of
    throwing it so that its callers don't need to add dummy return
    statement

Result:

Derived buffers are now recycled when created via retainedSlice() and
retainedDuplicate() and derived from a pooled buffer
2016-05-17 11:16:13 +02:00
Norman Maurer
68cd670eb9 Remove ChannelHandlerInvoker
Motivation:

We tried to provide the ability for the user to change the semantics of the threading-model by delegate the invoking of the ChannelHandler to the ChannelHandlerInvoker. Unfortunually this not really worked out quite well and resulted in just more complexity and splitting of code that belongs together. We should remove the ChannelHandlerInvoker again and just do the same as in 4.0

Modifications:

Remove ChannelHandlerInvoker again and replace its usage in Http2MultiplexCodec

Result:

Easier code and less bad abstractions.
2016-05-17 11:14:00 +02:00
Hyangtack Lee
f44f3e7926 NioEventLoop ensures that all submitted tasks are executed immediately.
Motivation:
If a task was submitted when wakenUp value was true, the task didn't get a chance to call Selector#wakeup.
So we need to check task queue again before executing select operation. If we don't, the task might be pended until select operation was timed out.
It might be pended until idle timeout if IdleStateHandler existed in pipeline.

Modifications:
Execute Selector#select in a non-blocking manner if there's a task submitted when wakenUp value was true.

Result:
Every tasks in NioEventLoop will not be pended.
2016-05-17 07:43:46 +02:00
Norman Maurer
c249926784 [#3127] Allow to write with VoidPromise to Channels in ChannelGroup
Motivation:

Users sometimes want to use Channel.voidPromise() when write to a Channel to reduce GC-pressure. This should be also possible when write via a ChannelGroup.

Modifications:

Add new write(...) and writeAndFlush(...) overloads which allow to signale that a VoidPromise should be used to write to the Channel

Result:

Users can write with VoidPromise when using ChannelGroup
2016-05-14 20:53:25 +02:00
Norman Maurer
27a392b877 Add ChannelInboundInvoker and ChannelOutboundInvoker
Motivation:

ChannelHandlerContext, ChannelPipeline and Channel share various method signatures. We should provide an interface to share.

Modifications:

Add ChannelInboundInvoker and ChannelOutboundInvoker and extend these.

Result:

Better API abstraction.
2016-05-14 20:41:13 +02:00
Norman Maurer
01109dd635 Fix test-failures introduces by c1827114e9. 2016-05-14 20:38:31 +02:00
Norman Maurer
c1827114e9 Use ConnectException when connection failed for LocalChannel
Motivation:

To be more consistent we should use ConnectException when we fail the connect attempt because no LocalServerChannel exists with the given address.

Modifications:

Use correct exception.

Result:

More consistent handling of connection refused between different transports.
2016-05-14 07:23:55 +02:00
Norman Maurer
775dd139ea Ensure Bootstrap.connect(...) not throw IllegalStateException when registration is delayed.
Motivation:

Bootstrap.connect(...) tries to obtain the EventLoop of a Channel before it may be registered. This will cause an IllegalStateException. We need to ensure we handle the cause of late registration and not throw in this case.

Modifications:

Ensure we only try to access the EventLoop after the Channel is registered and handle the case of late registration.

Result:

Bootstrap.connect(...) not fails on late registration.
2016-05-14 07:22:38 +02:00
Scott Mitchell
1f8e192e21 Remove EventExecutor.children
Motivation:
EventExecutor.children uses generics in such a way that an entire colleciton must be cast to a specific type of object. This interface is not very flexible and is impossible to implement if the EventExecutor type must be wrapped. The current usage of this method also does not have any clear need within Netty. The Iterator interface allows for EventExecutor to be wrapped and forces the caller to make assumptions about types instead of building the assumptions into the interface.

Motivation:
- Remove EventExecutor.children and undeprecate the iterator() interface

Result:
EventExecutor interface has one less method and is easier to wrap.
2016-05-13 18:17:22 -07:00
Norman Maurer
690ab563e7 Ensure ChannelHandlerContext.attr(...) and ChannelHandlerContext.hasAttr(...) has no semantic change
Motivation:

The ChannelHandlerContext.attr(...) and ChannelHandlerContext.hasAttr(...) delegated to Channel for the attributes which is a semantic change compared to 4.0 releases. We should not change the semantic to not break users applications when upgrading to 4.1.0

Modifications:

- Revert semantic change
- Mark ChannelHandlerContext.attr(...) and hasAttr(...) as @deprecated so we can remove these later

Result:

Semantic of attribute operations on ChannelHandlerContext is the same in 4.1 as in 4.0 again.
2016-05-13 15:51:58 +02:00
Norman Maurer
5ffa3b1c46 Only try to bind if late registration not failed.
Motivation:

We should not try to call bind if registration failed.

Modifications:

Only call doBind0(...) when the registration not failed.

Result:

Not try to to bind if the registration failed.
2016-05-13 12:10:31 +02:00
Norman Maurer
ea61b5b5b3 [#5223] Remove indirection in DefaultChannelPipeline.executorSafe(...)
Motivation:

We use channel.unsafe().invoker().executor() in DefaultChannelPipeline.executorSafe(...) which is an unnecessary indirection to channel.eventLoop().

Modifications:

Remove indirection by using channel.eventLoop().

Result:

Cleaner code.
2016-05-13 08:42:36 +02:00
Scott Mitchell
f2ed3e6ce8 DefaultPromise LateListener Logic Issues
Motivation:
The LateListener logic is prone to infinite loops and relies on being processed in the EventExecutor's thread for synchronization, but this EventExecutor may not be constant. An infinite loop can occur if the EventExecutor's execute method does not introduce a context switch in LateListener.run. The EventExecutor can be changed by classes which inherit from DefaultPromise. For example the DefaultChannelPromise will return w/e EventLoop the channel is registered to, but this EventLoop can change (re-registration).

Modifications:
- Remove the LateListener concept and instead use a single Object to maintain the listeners while still preserving notification order
- Make the result member variable an atomic variable so it can be outside the synchronized(this) blocks
- Cleanup/simplify existing state management code

Result:
Fixes https://github.com/netty/netty/issues/5185
2016-05-09 10:33:40 -07:00
Norman Maurer
b2bb72b2de [#5171] Ensure NioDatagramChannelConfig can be instanced on android
Motivation:

NioDatagramChannelConfig currently uses NetworkChannel in its static { } block and so fails to init on android which not has this class.

Modifications:

Use reflection to load the NetworkChannel.class

Result:

Be able to use NIO Datagram on android as well.
2016-05-06 08:04:36 +02:00
Norman Maurer
c745ac0e16 [#3297] Fix race while resolve remote address that can cause NPE
Motivation:

We may produce a NPE due a race that can happen while check if a resolution was done and failed.

Modifications:

Correctly first check if the resultion is done before try to detect if it failed and so remove the race that can produce a NPE.

Result:

No more race possible while resolve address during connect.
2016-05-06 08:00:15 +02:00
Carl Mastrangelo
cf07f984b1 Add @Deprecated when the javadoc says its deprecated
Motivation:

Reduce nag warnings when compiling, make it easier for IDEs to display what's deprecated.

Modifications:

Added @Deprecated in a few places

Result:

No more warnings.
2016-05-01 20:30:13 +02:00
Scott Mitchell
04e33fd2d8 NioEventLoop closes channel even if channel is not registered
Motivation:
If a channel is deregistered from an NioEventLoop the associated SelectionKey is cancelled. If the NioEventLoop has yet to process a pending event as a result of that SelectionKey then the NioEventLoop will see the SelecitonKey is invalid and close the channel. The NioEventLoop should not close a channel if it is not registered with that NioEventLoop.

Modifications:
- NioEventLoop.processSelectedKeys should check that the channel is still registered to itself before closing the channel

Result:
NioEventLoop doesn't close a channel that is no longer registered to it when the SelectionKey is invalid
Fixes https://github.com/netty/netty/issues/5125
2016-04-21 09:17:14 -07:00
Norman Maurer
572bdfb494 [maven-release-plugin] prepare for next development iteration 2016-04-10 08:37:18 +02:00
Norman Maurer
c6121a6f49 [maven-release-plugin] prepare release netty-4.1.0.CR7 2016-04-10 08:36:56 +02:00
Norman Maurer
6e919f70f8 [maven-release-plugin] rollback the release of netty-4.1.0.CR7 2016-04-09 22:13:44 +02:00
Norman Maurer
4cdd51509a [maven-release-plugin] prepare release netty-4.1.0.CR7 2016-04-09 22:05:34 +02:00
Scott Mitchell
abce89d1bc Revert "[#5028] Fix re-entrance issue with channelWritabilityChanged(...) and write(...)"
Motivation:
Revert d0943dcd30. Delaying the notification of writability change may lead to notification being missed. This is a ABA type of concurrency problem.

Modifications:
- Revert d0943dcd30.

Result:
channelWritabilityChange will be called on every change, and will not be suppressed due to ABA scenario.
2016-04-09 20:32:47 +02:00
Scott Mitchell
a0b28d6c82 Fix potential assertion error introduced by 0bc93dd
Motivation:
Commit 0bc93dd introduced a potential assertion failure, if the deprecated method would be used.

Modifications:
Fix the potential assertion error.

Result:
Regression removed
2016-04-08 09:44:44 -07:00
Trustin Lee
32cfe25132 Fix checkstyle 2016-04-08 17:44:52 +09:00
Trustin Lee
c1a3cc32e7 Fix an assertion error introduced by 0bc93dda08
Motivation:
    
Commit 0bc93dda08 introduced an assertion
failure.

Modifications:

Fix the assertion error.

Result:

Regression removed
2016-04-08 17:23:32 +09:00
stroller
84e281ced9 fix one java doc issue: extra }
Motivation:

There is one extra } for WriteBufferWaterMark's javadoc:
{@linkplain #high}  high water mark}

The generated javadoc will show the content: "the high high water mark}"

Modifications:

remove the }

Result:
The generated javadoc will show the content: "the high water mark" instead of "the high high water mark}"
2016-04-08 09:47:26 +02:00
Scott Mitchell
0bc93dda08 NIO autoReadClear should also clear the SelectionKey
Motivation:
Prior to 5b48fc284e setting readPending to false when autoReadClear was called was enough because when/if the EventLoop woke up with a read event we would first check if readPending was true and if it is not, we would return early. Now that readPending will only be set in the EventLoop it is not necessary to check readPending at the start of the read methods, and we should instead remove the OP_READ from the SelectionKey when we also set readPending to false.

Modifications:
- autoReadCleared should call AbstractNioUnsafe.removeReadOp

Result:
NIO is now consistent with EPOLL and removes the READ operation on the selector when autoRead is cleared.
2016-04-08 00:03:37 -07:00
Scott Mitchell
5b48fc284e Make OIO/NIO/EPOLL autoReadClear consistent
Motivation:
OIO/NIO use a volatile variable to track if a read is pending. EPOLL does not use a volatile an executes a Runnable on the event loop thread to set readPending to false. These mechansims should be consistent, and not using a volatile variable is preferable because the variable is written to frequently in the event loop thread.
OIO also does not set readPending to false before each fireChannelRead operation and may result in reading more data than the user desires.

Modifications:
- OIO/NIO should not use a volatile variable for readPending
- OIO should set readPending to false before each fireChannelRead

Result:
OIO/NIO/EPOLL are more consistent w.r.t. readPending and volatile variable operations are reduced
Fixes https://github.com/netty/netty/issues/5069
2016-04-06 12:32:14 -07:00
Norman Maurer
d0943dcd30 [#5028] Fix re-entrance issue with channelWritabilityChanged(...) and write(...)
Motivation:

When always triggered fireChannelWritabilityChanged() directly when the update the pending bytes in the ChannelOutboundBuffer was made from within the EventLoop. This is problematic as this can cause some re-entrance issue if the user has a custom ChannelOutboundHandler that does multiple writes from within the write(...) method and also has a handler that will intercept the channelWritabilityChanged event and trigger another write when the Channel is writable. This can also easily happen if the user just use a MessageToMessageEncoder subclass and triggers a write from channelWritabilityChanged().

Beside this we also triggered fireChannelWritabilityChanged() too often when a user did a write from outside the EventLoop. In this case we increased the pending bytes of the outboundbuffer before scheduled the actual write and decreased again before the write then takes place. Both of this may trigger a fireChannelWritabilityChanged() event which then may be re-triggered once the actual write ends again in the ChannelOutboundBuffer.

The third gotcha was that a user may get multiple events even if the writability of the channel not changed.

Modification:

- Always invoke the fireChannelWritabilityChanged() later on the EventLoop.
- Only trigger the fireChannelWritabilityChanged() if the channel is still active and if the writability of the channel changed. No need to cause events that were already triggered without a real writability change.
- when write(...) is called from outside the EventLoop we only increase the pending bytes in the outbound buffer (so that Channel.isWritable() is updated directly) but not cause a fireChannelWritabilityChanged(). The fireChannelWritabilityChanged() is then triggered once the task is picked up by the EventLoop as usual.

Result:

No more re-entrance possible because of writes from within channelWritabilityChanged(...) method and no events without a real writability change.
2016-04-06 10:07:13 +02:00
Scott Mitchell
9fb86a380d NIO/EPOLL readPending set to false incorrectly
Motivation:
441aa4c575 introduced a bug in transport-native-epoll where readPending is set to false before a read is attempted, but this should happen before fireChannelRead is called. The NIO transport also only sets the readPending variable to false on the first read in the event loop. This means that if the user only calls read() on the first channelRead(..) the select loop will still listen for read events even if the user does not call read() on subsequent channelRead() or channelReadComplete() in the same event loop run. If the user only needs 2 channelRead() calls then by default they will may get 14 more channelRead() calls in the current event loop, and then 16 more when the event loop is woken up for a read event. This will also read data off the TCP stack and allow the peer to queue more data in the local RECV buffers.

Modifications:
- readPending should be set to false before each call to channelRead()
- make NIO readPending set to false consistent with EPOLL

Result:
NIO and EPOLL transport set readPending to false at correct times which don't read more data than intended by the user.
Fixes https://github.com/netty/netty/issues/5082
2016-04-06 00:09:49 -07:00
Norman Maurer
d602277204 Include cause that was used to notify the promise when logging an failed try to notify it.
Motivation:

When a promise is notified that was already added to the ChannelOutboundBuffer and we try to notify it later on we only see a warning that it was notified before. This is often not very useful as we have no idea where it was notified at all. We can do better in case it was failed before (which is most of the times the case) and just also log the cause that was used for it.

Modifications:

Add the cause that was used to notify the promise when we fail to notify it as part of the ChannelOutboundBuffer.

Result:

Easier to debug user errors.
2016-04-05 21:13:51 +02:00
Norman Maurer
7fb475a223 Fix typo missed in f46cfbc590 2016-04-05 15:32:54 +02:00
Norman Maurer
f46cfbc590 [#5059] Deprecate method with typo and introduce a new one without typo
Motivation:

There is a spelling error in FileRegion.transfered() as it should be transferred().

Modifications:

Deprecate old method and add a new one.

Result:

Fix typo and can remove the old method later.
2016-04-05 15:06:46 +02:00
Norman Maurer
ea94336689 DefaultChannelHandlerInvoker should work with non AbstractChannelHandlerContext sub-classes.
Motivation:

DefaultChannelHandlerInvoker currently blindly cast to AbstractChannelHandlerContext without checking if the ChannelHandlerContext is really a sub-type of it. It should check it first and if not just use slow-path implementation.

Modifications:

Do instanceof check first and if it fails just create a new Runnable instance of used the cached.

Result:

DefaultChannelHandlerInvoker works with any ChannelHandlerContext implementations.
2016-04-05 13:21:07 +02:00
Trustin Lee
3b941c2a7c [maven-release-plugin] prepare for next development iteration 2016-04-02 01:25:05 -04:00
Trustin Lee
7368ccc539 [maven-release-plugin] prepare release netty-4.1.0.CR6 2016-04-02 01:24:55 -04:00
jiafu1115
3e5dcb5f3e [#3806] Setting WRITE_BUFFER_LOW_WATER_MARK before WRITE_BUFFER_HIGH_WATER_MARK results in an internal Exception
Motivation:

Setting the WRITE_BUFFER_LOW_WATER_MARK before WRITE_BUFFER_HIGH_WATER_MARK results in an internal Exception (appears only in the logs) if the value is larger than the default high water mark value. The WRITE_BUFFER_HIGH_WATER_MARK call appears to have no effect in this context.

Setting the values in the reverse order works.

Modifications:

- deprecated ChannelOption.WRITE_BUFFER_HIGH_WATER_MARK and
ChannelOption.WRITE_BUFFER_LOW_WATER_MARK.
- add one new option called ChannelOption.WRITE_BUFFER_WATER_MARK.

Result:
The high/low water mark values limits caused by default values are removed.

Setting the WRITE_BUFFER_LOW_WATER_MARK before WRITE_BUFFER_HIGH_WATER_MARK results in an internal Exception (appears only in the logs) if the value is larger than the default high water mark value. The WRITE_BUFFER_HIGH_WATER_MARK call appears to have no effect in this context.

Setting the values in the reverse order works.
2016-03-31 13:44:44 +02:00
Tibor Csögör
9d4fae308c ChannelInitializer: change propagation of channelRegistered event
Motivation:

If a handler is added to the pipeline within ChannelInitializer::initChannel via
addFirst(...) then it will not receive the channelRegistered event.  The same
handler added via addLast(...) will receive the event.  This different behavior
is unlikely to be expected by users and can cause confusion.

Modifications:

Let ChannelInitializer::channelRegistered propagate the event by passing it to
the pipeline instead of firing it on the ChannelHandlerContext.

Result:

The channelRegistered event is propagated to handlers regardless of the method
used to add it to the pipeline (addFirst/addLast).
2016-03-31 09:01:00 +02:00
Scott Mitchell
0c839d9e0a EPOLL SelectStrategy
Motivation:
NIO now supports a pluggable select strategy, but EPOLL currently doesn't support this. We should strive for feature parity for EPOLL.

Modifications:
- Add SelectStrategy to EPOLL transport.

Result:
EPOLL transport supports SelectStategy.
2016-03-30 15:11:35 -07:00
Michael Nitschinger
5d76daf33b Allow to customize NIO (channel) select strategies.
Motivation:

Under high throughput/low latency workloads, selector wakeups are
degrading performance when the incoming operations are triggered
from outside of the event loop. This is a common scenario for
"client" applications where the originating input is coming from
application threads rather from the socket attached inside the
event loops.

As a result, it can be desirable to defer the blocking select
so that incoming tasks (write/flush) do not need to wakeup
the selector.

Modifications:

This changeset adds the notion of a generic SelectStrategy which,
based on its contract, allows the implementation to optionally
defer the blocking select based on some custom criteria.

The default implementation resembles the original behaviour, that
is if tasks are in the queue `selectNow()` and move on, and if no
tasks need to be processed go into the blocking select and wait
for wakeup.

The strategy can be customized per `NioEventLoopGroup` in the
constructor.

Result:

High performance client applications are now given the chance to
customize for how long the actual selector blocking should be
deferred by employing a custom select strategy.
2016-03-30 15:01:25 -07:00
Norman Maurer
2facb7affd Change DefaultChannelId visibility to default. Related to [#5053]
Motivation:

There is no need to make DefaultChannelId package private as it may be useful for the user. For example EmbeddedChannel allows to inject a ChannelId when it is constructed. For this case the user can just use DefaultChannelId.

Modifications:

Change visibility of DefaultChannelId to public.

Result:

It's possible to create a new instance of DefaultChannelId by the user.
2016-03-30 17:39:32 +02:00
Norman Maurer
cee38ed2b6 [maven-release-plugin] prepare for next development iteration 2016-03-29 16:45:13 +02:00
Norman Maurer
9cd9e7daeb [maven-release-plugin] prepare release netty-4.1.0.CR5 2016-03-29 16:44:33 +02:00
Norman Maurer
86b9656167 Correctly run pending tasks before flush and also remove incorrect assert.
Motivation:

We need to ensure we run all pending tasks before doing any flush in writeOutbound(...) to ensure all pending tasks are run first. Also we should remove the assert of the future and just add a listener to it so it is processed later if needed. This is true as a user may schedule a write for later execution.

Modifications:

- Remove assert of future in writeOutbound(...)
- Correctly run pending tasks before doing the flush and also before doing the close of the channel.
- Add unit tests to proof the defect is fixed.

Result:

Correclty handle the situation of delayed writes.
2016-03-29 14:30:23 +02:00
Scott Mitchell
99c85ef4f5 cf171ff525 Close Regression
Motivation:
cf171ff525 introduced a change in behavior when dealing with closing channel in the read loop. This changed behavior may use stale state to determine if a channel should be shutdown and may be incorrect.

Modifications:
- Revert the usage of potentially stale state

Result:
Closing a channel in the read loop is based upon current state instead of potentially stale state.
2016-03-24 14:52:04 -07:00
Norman Maurer
9a183ec38f Add methods to easily release messages from inbound / outbound buffer of EmbeddedChannel
Motivation:

Often the user uses EmbeddedChannel within unit tests where the only "important" thing is to know if any pending messages were in the buffer and then release these.
We should provide methods for this so the user not need to manually loop through these and release.

Modifications:

Add methods to easily handle releasing of messages.

Result:

Less boiler-plate code for the user to write.
2016-03-24 11:03:30 +01:00
Norman Maurer
28d03adbfe [maven-release-plugin] prepare for next development iteration 2016-03-21 11:51:50 +01:00
Norman Maurer
4653dc1d05 [maven-release-plugin] prepare release netty-4.1.0.CR4 2016-03-21 11:51:12 +01:00
Scott Mitchell
fc099292fd HTTP/2 DefaultHttp2ConnectionEncoder data frame size incorrect if error
Motivation:
If an error occurs during a write operation then DefaultHttp2ConnectionEncoder.FlowControlledData will clear the CoalescingBufferQueue which will reset the queue's readable bytes to 0. To recover from an error the DefaultHttp2RemoteFlowController will attempt to return bytes to the flow control window, but since the frame has reset its own size this will lead to invalid flow control accounting.

Modifications:
- DefaultHttp2ConnectionEncoder.FlowControlledData should not reset its size if an error occurs

Result:
No more flow controller errors due to DefaultHttp2ConnectionEncoder.FlowControlledData setting its size to 0 if an error occurs.
2016-03-18 11:38:01 -07:00
Tibor Csögör
6e840d8e62 trivial javadoc fixes
- fix the formatting of the diagram in ChannelFuture's javadoc
- update external link in AutobahnServer
- fix various spelling issues
2016-03-16 20:18:29 +01:00
Scott Mitchell
45849b2fa8 Deprecate PromiseAggregator
Motivation:
PromiseAggregator's API allows for the aggregate promise to complete before the user is done adding promises. In order to support this use case the API structure would need to change in a breaking manner.

Modifications:
- Deprecate PromiseAggregator and subclasses
- Introduce PromiseCombiner which corrects these issues

Result:
PromiseCombiner corrects the deficiencies in PromiseAggregator.
2016-03-14 10:53:30 -07:00
Max Ng
e7ee6abd70 Guard against re-entrance in PendingWriteQueue.
Motivation:

PendingWriteQueue should guard against re-entrant writes once
removeAndFailAll() is run.

Modifications:

removeAndFailAll() should repeat until the queue is finally empty.

Result:

assertEmpty() will always hold.
2016-03-14 08:52:43 +01:00
Norman Maurer
45d291bb15 Add asserts so users will see errors when try to use methods from outside the EventLoop.
Motivation:

We should guard users from using Unsafe methods from outside the EventLoop if not designed to do so.

Modifications:

Add asserts

Result:

Easier for users to detect miss-use.
2016-03-08 14:59:43 +01:00
Scott Mitchell
d0f7f98d22 e2f5012 unit test cleanup
Motivation:
e2f5012 added unit tests which did not verify the buffer was released as it was intended to.

Modification:
- Unit tests must verify release is called

Result:
Unit tests enforce that ByteBufs are released.
2016-03-07 09:33:45 -08:00
Scott Mitchell
e2f5012f3b DefaultChannelHandlerInvoker write leak
Motivation:
DefaultChannelHandlerInvoker.invokeWrite calls a utility method validatePromise which may throw if the arguments are not valid. If this method throws then the message will not be released.

Modifications:
- If an exception is thrown the message should be released

Result:
No more leak in DefaultChannelHandlerInvoker.invokeWrite
2016-03-07 09:16:12 -08:00
Xiaoyan Lin
c7aadb5469 Fork SpscLinkedQueue and SpscLinkedAtomicQueue from JCTools
Motivation:

See #3746.

Modifications:

Fork SpscLinkedQueue and SpscLinkedAtomicQueue from JCTools based on 7846450e28

Result:

Add SpscLinkedQueue and SpscLinkedAtomicQueue and apply it in LocalChannel.
2016-02-22 15:55:52 +01:00
Norman Maurer
ca443e42e0 [maven-release-plugin] prepare for next development iteration 2016-02-19 23:00:11 +01:00
Norman Maurer
f39eb9a6b2 [maven-release-plugin] prepare release netty-4.1.0.CR3 2016-02-19 22:59:52 +01:00
Roman Timushev
23f7fc67a4 Enable shutdownOutput for EpollDomainSocketChannel 2016-02-18 18:05:51 -08:00
Norman Maurer
c6a3729e4c Ensure handlerAdded(...) and handlerRemoved(...) is always called from the right thread
Motiviation:

We should ensure that handlerAdded(...) and handlerRemoved(...) is always called from the EventExecutor that also invokes the other methods of the ChannelHandler. Also we need to ensure we always call handlerAdded(...) before any other method can be calld to ensure correct ordering.

Motifications:

- Ensure that the right thread is used to call the methods
- Ensure correct ordering
- Add tests

Result:

Respect the thread-model for handlerAdded(...) and handlerRemoved(...) and preserve correct ordering in all cases.
2016-02-12 11:18:40 -08:00
Fabian Lange
a51e2c8769 Expose Helper to obtain the "best" mac address.
Motivation:

The implementation of obtaining the best possible mac address is very good. There are many sub-par implementations proposed on stackoverflow.
While not strictly a netty concern, it would be nice to offer this util also to netty users.

Modifications:

extract DefaultChannelId#defaultMachineId code obtaining the "best" mac into a new helper called MacAddress, keep the random bytes fallback in DefaultChannelID.

Result:

New helper available.
2016-02-05 09:27:43 +01:00
Norman Maurer
75a2ddd61c [maven-release-plugin] prepare for next development iteration 2016-02-04 16:51:44 +01:00
Norman Maurer
7eb3a60dba [maven-release-plugin] prepare release netty-4.1.0.CR2 2016-02-04 16:37:06 +01:00
Norman Maurer
465a190c3f [#4805] Respect isAutoRead() once re-register Channel
Motivation:

When a channel was registered before and is re-registered we need to respect ChannelConfig.isAutoRead() and so start reading one the registration task completes. This was done "by luck" before 15162202fb.

Modifications:

Explicit start reading once a Channel was re-registered if isAutoRead() is true.

Result:

Correctly receive data after re-registration completes.
2016-02-04 15:34:24 +01:00
Norman Maurer
08a7ca3747 Correctly pass ChannelPromise on to the next ChannelOutboundHandler when use CombinedChannelDuplexHandler.
Motivation:

Due a regression introduced by e969b6917c we missed to pass the original ChannelPromise to the next ChannelOutboundHandler and so
may never notify the origin ChannelPromise. This is related to #4805.

Modifications:

- Correctly pass the ChannelPromise
- Add unit test.

Result:

Correctly pass the ChannelPromise on deregister(...)
2016-02-04 15:28:46 +01:00
fu.jian
a06708f81b fix the issue netty#2944 in 4.1
Motivation:

fix the issue netty#2944

Modifications:

use - instead of =>, use ! instead of :> due to the connection is bidirectional. What's more, toString() method don't know the direction or there is no need to know the direction when only log channel information.
add L: before local address and R: before remote address.

Result:

after the fix, log won't confuse the user
2016-02-02 21:43:38 +01:00
Travis Haagen
a75dcb2756 Made it easier to use custom ChannelId instances with Channel implementations that rely on the AbstractChannel(Channel parent) constructor.
Motivation:

The AbstractChannel(Channel parent) constructor was previously hard-coded to always
call DefaultChannelId.newInstance(), and this made it difficult to use a custom
ChannelId implementation with some commonly used Channel implementations.

Modifications:

Introduced newId() method in AbstractChannel, which by default returns
DefaultChannelId.newInstance() but can be overridden by subclasses. Added
ensureDefaultChannelId() test to AbstractChannelTest, to ensure the prior
behavior of calling DefaultChannelId.newInstance() still holds with the
AbstractChannel(Channel parent) constructor.

Result:

AbstractChannel now has the protected newId() method, but there is no functional
difference.
2016-02-02 08:33:42 +01:00
Scott Mitchell
8ba4b63cb6 OioServerChannel Default Max Messages Per Read Too High
Motivation:
A few implementations of OioServerChannel have a default max messages per read set to 16. We should set the default to 1 to prevent blocking on a read before setting a socket that has just been accepted.

Modifications:
- OioSctpServerChannel and OioServerSocketChannel metadata changed to use the default (1) max messages per read

Result:
Oio based servers will complete accepting a socket before potentially blocking waiting to accept other sockets.
2016-01-29 11:34:00 -08:00
Norman Maurer
af39cb6b12 Ensure ChannelHandler.handlerAdded(...) is always called as first method of the handler
Motivation:

If a user adds a ChannelHandler from outside the EventLoop it is possible to get into the situation that handlerAdded(...) is scheduled on the EventLoop and so called after another methods of the ChannelHandler as the EventLoop may already be executing on this point in time.

Modification:

- Ensure we always check if the handlerAdded(...) method was called already and if not add the currently needed call to the EventLoop so it will be picked up after handlerAdded(...) was called. This works as if the handler is added to the ChannelPipeline from outside the EventLoop the actual handlerAdded(...) operation is scheduled on the EventLoop.
- Some cleanup in the DefaultChannelPipeline

Result:

Correctly order of method executions of ChannelHandler.
2016-01-28 13:24:36 +01:00
Xiaoyan Lin
58a038d398 Fix ChannelOutboundHandlerAdapter javadoc
Motivation:

ChannelOutboundHandlerAdapter's javadoc has some minor issues.

Modifications:

Fix the minor javadoc issues and resolves #4752.

Result:

ChannelOutboundHandlerAdapter's javadoc issues are fixed.
2016-01-26 10:13:16 +01:00
Norman Maurer
1c417e5f82 [maven-release-plugin] prepare for next development iteration 2016-01-21 15:35:55 +01:00
Norman Maurer
c681a40a78 [maven-release-plugin] prepare release netty-4.1.0.CR1 2016-01-21 15:28:21 +01:00
Eric Anderson
6dbb610f5b Add ChannelHandlerContext.invoker()
Motivation:

Being able to access the invoker() is useful when adding additional
handlers that should be running in the same thread. Since an application
may be using a threading model unsupported by the default invoker, they
can specify their own. Because of that, in a handler that auto-adds
other handlers:

// This is a good pattern
ctx.pipeline().addBefore(ctx.invoker(), ctx.name(), null, newHandler);
// This will generally work, but prevents using custom invoker.
ctx.pipeline().addBefore(ctx.executor(), ctx.name(), null, newHandler);

That's why I believe in commit 110745b0, for the now-defunct 5.0 branch,
when ChannelHandlerAppender was added the invoker() method was also
necessary.

There is a side-benefit to exposing the invoker: in certain advanced
use-cases using the invoker for a particular handler is useful. Using
the invoker you are able to invoke a _particular_ handler, from possibly
a different thread yet still using standard exception processing.

ChannelHandlerContext does part of that, but is unwieldy when trying to
invoke a particular handler because it invokes the prev or next handler,
not the one the context is for. A workaround is to use the next or prev
context (respectively), but this breaks when the pipeline changes.

This came up during writing the Http2MultiplexCodec which uses a
separate child channel for each http/2 stream and wants to send messages
from the child channel directly to the Http2MultiplexCodec handler that
created it.

Modifications:

Add the invoker() method to ChannelHandlerContext. It was already being
implemented by AbstractChannelHandlerContext. The two other
implementations of ChannelHandlerContext needed minor tweaks.

Result:

Access to the invoker used for a particular handler, for either reusing
for other handlers or for advanced use-cases. Fixes #4738
2016-01-22 14:04:35 +01:00
Norman Maurer
e969b6917c Let CombinedChannelDuplexHandler correctly handle exceptionCaught. Related to [#4528]
Motivation:

ChannelInboundHandler and ChannelOutboundHandler both can implement exceptionCaught(...) method and so we need to dispatch to both of them.

Modifications:

- Correctly first dispatch exceptionCaught to the ChannelInboundHandler but also make sure the next handler it will be dispatched to will be the ChannelOutboundHandler
- Add removeInboundHandler() and removeOutboundHandler() which allows to remove one of the combined handlers
- Let *Codec extends it and not ChannelHandlerAppender
- Remove ChannelHandlerAppender

Result:

Correctly handle events and also have same behavior as in 4.0
2016-01-18 09:54:48 +01:00
Norman Maurer
1848e73ce6 Ensure connectPromise is not notified before fireChannelActive() is called.
Motivation:

Our contract in Channels is that the promise should always be notified before the actual callbacks of the ChannelInboundHandler are called. This was not done in the LocalChannel and so the behavior was different to other Channel implementations.

Modifications:

- First complete the ChannelPromise then call fireChannelActive()
- Guard against NPE when doClose() was called before the task was executed.

Result:

Consistent behavior between LocalChannel and other Channel implementations.
2016-01-18 09:28:43 +01:00
Norman Maurer
8b123a5546 Fix AbstractChannelTest errors caused by incorrect mocking
Motivation:

In AbstractChannelTest we not correctly mocked some methods which could lead to test errors. That said it only showed up here when running from the IDE and not from the cmdLine.

Modifications:

Mock methods that are needed for the test

Result:

Test pass in the IDE as well.
2016-01-14 20:57:42 +01:00
Norman Maurer
4d854cc149 Allow to change if EmbeddedChannel should handle close() and disconnect() different.
Motivation:

At the moment EmbeddedChannel always handle close() and disconnect() the same way which also means that ChannelOutboundHandler.disconnect(...) will never called. We should allow to specify if these are handle different or not to make the use of EmbeddedChannel more flexible.

Modifications:

Add 2 other constructors which allow to specify if disconnect / close are handled the same way or differently.

Result:

More flexible usage of EmbeddedChannel possible.
2016-01-14 07:19:23 +01:00
Fernando van Loenhout
5c05629da1 Fix issue #4676
Fixed spelling mistake at EmbeddedChannel#readOutbound()
2016-01-08 18:12:31 -08:00
Xiaoyan Lin
751ed6cc94 Avoid unnecessary boxing/unboxing
Motivation:

Boxing/unboxing can be avoided.

Modifications:

Use parseInt/parseLong to avoid unnecessary boxing/unboxing.

Result:

Remove unnecessary boxing/unboxing.
2016-01-08 17:38:20 +01:00
fu.jian
0c733e1425 [#2363] Correctly null out SelectionKey[] when selectAgain
Motivation:

The prefix fix of #2363 did not correctly handle the case when selectAgain is true and so missed to null out entries.

Modifications:

Move the i++ from end of loop to beginning of loop

Result:

Entries in the array will be null out so allow to have these GC'ed once the Channel close
2016-01-08 09:04:10 +01:00
Norman Maurer
bf2a99518c Simplify synchronized syntax
Motivation:

We often used synchronized(this) while the whole method was synchronized, which can be simplified by just mark the whole method as synchronized.

Modifications:

Replace synchronized(this) with synchronized on the method

Result:

Cleaner code
2016-01-05 08:45:20 +01:00
Fabian Lange
619d82b56f Removed unused imports
Motivation:

Warnings in IDE, unclean code, negligible performance impact.

Modification:

Deletion of unused imports

Result:

No more warnings in IDE, cleaner code, negligible performance improvement.
2016-01-04 14:32:29 +01:00
Sergey Polovko
bc2559ceb1 fix links to github issues in javadoc 2016-01-04 08:47:00 +01:00
Xiaoyan Lin
475d901131 Fix errors reported by javadoc
Motivation:

Javadoc reports errors about invalid docs.

Modifications:

Fix some errors reported by javadoc.

Result:

A lot of javadoc errors are fixed by this patch.
2015-12-27 08:36:45 +01:00
Xiaoyan Lin
a96d52fe66 Fix javadoc links and tags
Motivation:

There are some wrong links and tags in javadoc.

Modifications:

Fix the wrong links and tags in javadoc.

Result:

These links will work correctly in javadoc.
2015-12-26 08:34:31 +01:00
Norman Maurer
d59bf84ef2 [#4435] Always invoke the actual deregisteration later in the EventLoop.
Motivation:

As a user may call deregister() from within any method while doing processing in the ChannelPipeline,  we need to ensure we do the actual deregister operation later. This is needed as for example,  we may be in the ByteToMessageDecoder.callDecode(...) method and so still try to do processing in the old EventLoop while the user already registered the Channel to a new EventLoop. Without delay, the deregister operation this could lead to have a handler invoked by different EventLoop and so threads.

Modifications:

Ensure the actual deregister will be done later on and not directly when invoked.

Result:

Calling deregister() within ByteToMessageDecoder.decode(..) is safe.
2015-12-24 14:20:17 +01:00
Alexey Ermakov
cfd6793bb7 Customizable estimation for messages written outside the EventLoop
Motivation:

Estimation algorithm currently used for WriteTasks is complicated and
wrong. Additionally, some code relies on outbound buffer size
incremented only on actual writes to the outbound buffer.

Modifications:

- Throw away the old estimator and replace with a simple algorithm that
  uses the client-provided estimator along with a statically configured
  WriteTask overhead (io.netty.transport.writeTaskSizeOverhead system
  property with the default value of 48 bytes)
- Add a io.netty.transport.estimateSizeOnSubmit boolean system property
  allowing the clients to disable the message estimation outside the
  event loop

Result:

Task estimation is user controllable and produces better results by
default
2015-12-23 23:04:23 +01:00
Jonas Berlin
e4cb163be2 Fix javadoc link 2015-12-22 20:53:41 +01:00
Sky Ao
9b171beb92 Trivial javadoc fixes in ChannelHandlerContext 2015-12-18 14:00:28 +01:00
Stephane Landelle
6393506b97 Extract SocketAdress logic from NameResolver
Motivation:

As discussed in #4529, NameResolver design shouldn't be resolving SocketAddresses (or String name + port) and return InetSocketAddresses. It should resolve String names and return InetAddresses.
This SocketAddress to InetSocketAddresses resolution is actually a different concern, used by Bootstrap.

Modifications:

Extract SocketAddress to InetSocketAddresses resolution concern to a new class hierarchy named AddressResolver.
These AddressResolvers delegate to NameResolvers.

Result:

Better separation of concerns.

Note that new AddressResolvers generate a bit more allocations because of the intermediate Promise and List<InetAddress>.
2015-12-14 14:03:50 +01:00
Norman Maurer
89ff831a67 [#4449] Remove registered events from eventloop before close
Motivation:

We need to remove all registered events for a Channel from the EventLoop before doing the actual close to ensure we not produce a cpu spin when the actual close operation is delayed or executed outside of the EventLoop.

Modifications:

Deregister for events for NIO and EPOLL socket implementations when SO_LINGER is used.

Result:

No more cpu spin.
2015-12-13 09:55:50 +01:00
Norman Maurer
0ec34b5f76 Fix race-condition when closing a NioSocketChannel or EpollSocketChannel
Motivation:

Fix a race-condition when closing NioSocketChannel or EpollSocketChannel while try to detect if a close executor should be used and the underlying socket was already closed. This could lead to an exception that then leave the channel / in an invalid state and so could lead to side-effects like heavy CPU usage.

Modifications:

Catch possible socket exception while try to get the SO_LINGER options from the underlying socket.

Result:

No more race-condition when closing the channel is possible with bad side-effects.
2015-11-26 22:56:00 +01:00
Scott Mitchell
641505a5d2 DefaultChannelConfig maxMessagesPerRead default not always set
Motivation:
ChannelMetadata has a field minMaxMessagesPerRead which can be confusing. There are also some cases where static instances are used and the default value for channel type is not being applied.

Modifications:
- use a default value which is set unconditionally to simplify
- make sure static instances of MaxMessagesRecvByteBufAllocator are not used if the intention is that the default maxMessagesPerRead should be derived from the channel type.

Result:
Less confusing interfaces in ChannelMetadata and ChannelConfig. Default maxMessagesPerRead is correctly applied.
2015-11-25 15:14:07 -08:00
nmittler
227e67900e Fixing spammy logging for CoalescingBufferQueueTest
Motivation:

The CoalescingBufferQueueTest is somewhat relaxed with its releasing of test buffers, using safeRelease to generically deal with tests that may or may not release the buffers. SafeRelease generates logs, however, when the release fails.

Modifications:

Tightened up the individual test methods to verify that the buffers are released properly.

Result:

Fixes #4497
2015-11-25 07:16:05 -08:00
Scott Mitchell
3fa4603120 Cleanup ChannelOption.AUTO_CLOSE javadocs
Motivation:
The javadocs for ChannelOption.AUTO_CLOSE say the default is false, but the default is currently true.

Modifications:
- Make javadocs consistent with code

Result:
Less confusing docs.
2015-11-24 15:23:15 -08:00
Norman Maurer
41e03adf24 Mark ChannelHandler.exceptionCaught(...) as deprected.
Motivation:

exceptionCaught(...) will only handle inbound exceptions which means it makes not much sense to have it also on ChannelOutboundHandler. Because of this we should move it to ChannelInboundHandler.

Modifications:

Add @deprecated annotation to ChannelHandler.exceptionCaught(...).

Result:

Preapre to cleanup the API in later release.
2015-11-23 09:59:22 +01:00
Norman Maurer
7bee318fc7 Use OneTimeTask where possible to reduce object creation
Motivation:

We should use OneTimeTask where possible to reduce object creation.

Modifications:

Replace Runnable with OneTimeTask

Result:

Less object creation
2015-11-20 14:39:06 -08:00
Norman Maurer
cfa76f6326 Remove unnecessary reference to AbstractChannel from AbstractChannelHandlerContext
Motivation:

We not need to store another reference to AbstractChannel as we can access it through DefaultChannelHandlerContext.

Modifications:

Remove reference.

Result:

Cleaner code.
2015-11-20 06:41:11 -08:00
Norman Maurer
2d2e07578a Remove HashMap for lookup name / ctx from DefaultChannelPipeline to reduce memory footprint
Motivation:

If you start to have 1M+ concurrent connections memory footprint can be come a big issue. We should try to reduce it as much as possible in the core of netty.

Modifications:

- Remove HashMap that was used to store name to ctx mapping. This was only used for validation and access a handler by name. As a pipeline is not expected to be very long (like 100+ handlers) we can just walk the linked list structure to find the ctx with a given name.

Result:

Less memory footprint of the DefaultChannelPipeline.
2015-11-20 06:29:28 -08:00
pieteradejong
64409ad10b fixed word - issue #4469 2015-11-19 07:33:02 -08:00
Norman Maurer
2ecce8fa56 [maven-release-plugin] prepare for next development iteration 2015-11-10 22:59:33 +01:00
Norman Maurer
6a93f331d3 [maven-release-plugin] prepare release netty-4.1.0.Beta8 2015-11-10 22:50:57 +01:00
Sergio Bossa
187efca9aa Improved DefaultChannelPipeline#destroy() to avoid spinning continuously in case of custom executors.
Motivation:

The previous DefaultChannelPipeline#destroy() implementation, introduced in #3156, is suboptimal as it can cause the for loop to continuously spin if the executor used by a given handler is unable to "recognize" the event loop.
It could be objected that it's the custom executor responsibility to properly implement the inEventLoop() method, but some implementetaions might not be able to do that for performance reasons, and even so, it's always better to be safe against API misuse, in particular when it is not possible to fail fast and the alternative is rather some sutle behaviour.

Modifications:

The patch simply avoids the recursive spin by explicitly passing the "in event loop" condition as a boolean parameter, preserving the same guarantees offered by #3156. A unit test has also been added.

Result:

All channel events are correctly called and no high CPU usage is seen anymore.
2015-11-06 19:37:53 +01:00
Norman Maurer
98eb69f169 Remove synchronization overhead on generateName.
Motivation:

Changing the chache of generated names to use a cache per thread. This will remove the bottleneck when many eventloops are used and names need to generate.

Modifications:

Use a FastThreadLocal to store the cached names.

Result:

Less locking between threads.
2015-11-04 20:41:41 +01:00
Norman Maurer
94e907fa39 Only call ReferenceCountUtil.touch(...) if ResourceLeakDetection was enabled when Channel was created.
Motivation:

We should only call ReferenceCountUtil.touch(...) if needed as otherwise we pay the overhead of instanceof and cast
everytime.

Modifications:

Add boolean flag which indicates if touch(...) should be called.

Result:

Less overhead when leak detection is not enabled.
2015-10-29 19:23:27 +01:00
Norman Maurer
2100a83e30 [#4363] Improve size calculation of messages when written from outside the EventLoop
Motiviation:

If a user writes from outside the EventLoop we increase the pending bytes of the outbound buffer before submitting the write request. This is done so the user can stop writing asap once the channel turns unwritable. Unfortunally this doesn't take the overhead of adding the task into the account and so it is very easy for an user to full up the task queue. Beside this we use a value of 0 for an unown message by default which is not ideal.

Modifications:

- port the message calculation we used in netty 3.x into AbstractChannelHandlerContext and so better calculate the overhead of a message that is submitted from outside the EventLoop
- change the default estimated size for an unknown message to 8.

Result:

Better behaviour when submiting writes from outside the EventLoop.
2015-10-28 21:54:31 +01:00
Norman Maurer
8687475eb7 [#4373] Fix assert error when trying to release Channel to closed FixedChannelPool
Motivation:

Once a FixedChannelPool was closed we must not allow to acquire or release Channels to prevent assert errors.

Modifications:

Fail release and acquire calls when FixedChannelPool is closed.

Result:

No more assert errors.1
2015-10-24 11:56:46 +02:00
Norman Maurer
b90685d3c9 Not share FixedRecvByteBufAllocator.HandleImpl
Motivation:

As MaxMessageHandle is stateful we can not share the same HandleImpl instance as otherwise we will see race conditions.

Modifications:

Create a new HandleImpl instance on each newHandle() call.

Result:

No more races.
2015-10-22 09:47:23 +02:00
Richard DiCroce
b58036aeea Improve flexibility of EmbeddedChannel ID
Motivation:

EmbeddedChannelId#hashCode() and equals() do not behave correctly if an
instance is serialized and then deserialized. Additionally,
EmbeddedChannel does not allow use of any other type of ChannelId, and
EmbeddedChannelId is (mostly) a singleton instance. This creates a
problem for unit tests that construct multiple EmbeddedChannels and
expect each channel to have a unique ID.

Modifications:

EmbeddedChannelId is modified so equals() will return true and
hashCode() will return the same value for any instance of the class.
EmbeddedChannel is modified to allow a ChannelId to be specified when
the channel is constructed. Tests added for both changes.

Result:

EmbeddedChannelId now behaves correctly when deserialized, and
EmbeddedChannels can now have unique IDs.
2015-10-14 10:40:02 +02:00
Sky Ao
0740f703c1 change type definition of pipeline from DefaultChannelPipeline to ChannelPipeline 2015-10-10 20:15:10 +02:00
Norman Maurer
2d7e957a23 Cleanup PendingWriteQueueTest
Motivation:

PendingWriteQueueTest needs some cleanup.

Modifications:

- Cleanup code to remove deprecation warnings
- use static imports

Result:

No more warnings
2015-10-10 19:59:23 +02:00
Norman Maurer
fbaf5d06e6 [#4316] Ensure pending tasks are run when EmbeddedChannel.close(...) or disconnect(...) is called.
Motivation:

We missed to run all pending tasks when EmbeddedChannel.close(...) or disconnect(...) was called. Because of this channelInactive(...) / channelUnregistered(...) of the handlers were never called.

Modifications:

Correctly run all pending tasks and cancel all not ready scheduled tasks when close or disconnect was called.

Result:

Correctly run tasks on close / disconnect and have channelInactive(...) / channelUnregistered(...) called.
2015-10-07 09:28:00 +02:00
Norman Maurer
2ff2806ada [maven-release-plugin] prepare for next development iteration 2015-10-02 09:03:29 +02:00
Norman Maurer
5a43de10f7 [maven-release-plugin] prepare release netty-4.1.0.Beta7 2015-10-02 09:02:58 +02:00
Norman Maurer
2a27d581a9 Ensure close caused by write will happen before write promise is notified
Motiviation:

We need to ensure the actual close to the transport takes place before the promsie of the write is notified that triggered it. This is needed as otherwise Channel.isActive(), isOpen() and isWritable() may return true even if the Channel should be closed already.

Modifications:

- Ensure the close takes place first

Result:

ChannelFutureListener will see the correct state of the Channel.
2015-09-16 20:36:04 +02:00
Norman Maurer
076d4ed514 [#4205] Correctly set EPOLLOUT flag whe writeBytes(...) was not able to write everything
Motivation:

writeBytes(...) missed to set EPOLLOUT flag when not all bytes were written. This could lead to have the EpollEventLoop not try to flush the remaining bytes once the socket becomes writable again.

Modifications:

- Move setting EPOLLOUT flag logic to one point so we are sure we always do it.
- Move OP_WRITE flag logic to one point as well.

Result:

Correctly try to write pending data if socket becomes writable again.
2015-09-16 07:30:17 +02:00
Scott Mitchell
983920f25f RecvByteBufAllocator.DelegatingHandle accessor
Motivation:
RecvByteBufAllocator.DelegatingHandle does not provide an accessor to get the delegate handle. This may be useful for classes that extend DelegatingHandle.

Modifications:
- add delegate() method to DelegatingHandle

Result:
Classes which inherit from DelegatingHandle can now access the delegate Handle.
2015-09-04 12:44:26 -07:00
Norman Maurer
34de2667c7 [maven-release-plugin] prepare for next development iteration 2015-09-02 11:45:20 +02:00
Norman Maurer
2eb444ec1d [maven-release-plugin] prepare release netty-4.1.0.Beta6 2015-09-02 11:36:11 +02:00
Scott Mitchell
1e763b6504 LocalChannel write when peer closed leak
Motivation:
If LocalChannel doWrite executes while the peer's state changes from CONNECTED to CLOSED it is possible that some promise's won't be completed and buffers will be leaked.

Modifications:
- Check the peer's state in doWrite to avoid a race condition

Result:
All write operations should release, and the associated promise should be completed.
2015-09-01 13:13:02 -07:00
Scott Mitchell
d3dcc7f658 LocalChannelWrite event sequencing issue
Motivation:
https://github.com/netty/netty/pull/4143 addressed a few ordering issues but an ordering issue still remained if the Promise for a write completes, and a listener of that promise does a write on a peer channel. The ordering was subject to how potentially 2 different executors would run a task, but it should be coordinated such that the first write is read first.

Modifications:
- Keep track of the finishPeerRead task run on the executor if necessary and ensure it completes before current channel read occurs

Result:
Ordering of events for echo type situations is preserved.
2015-08-28 11:26:59 -07:00
Scott Mitchell
e37069b947 LocalChannel Event Ordering Error
Motivation:
When a LocalChannel write operation occurs, the promise associated with the write operation is marked successful when it is added to the peer's queue, but before the peer has actually received the data. If the promise callback closes the channel then a race condition exists where the close event may occur before the data is delivered. We should preserve ordering of events.

Modifications:
- LocalChannel should track when a write is in progress, and if a close operation happens make sure the peer gets all pending read operations.

Result:
LocalChannel preserves order of operations.
Fixes https://github.com/netty/netty/issues/4118
2015-08-28 09:03:46 -07:00
Christopher Probst
c8fb2a84c5 Fix race condition of DefaultChannelGroup by introducing a closed flag.
Motivation:

Doc of ChannelGroup says, that it can be used to manage server and child channels at once.
However, in DefaultChannelGroup, there is a race condition. When a server channel accepts a child, it schedules its
registration on an event loop, which takes some time. If the ChannelGroup, which is supposed
to close server and child channels at once, is closed after the child channel has been scheduled
for registration and before this registration actually happens, this child channel is not closed
and remains connected. This could lead to connection leaks.

Modifications:

To fix this, the DefaultChannelGroup is changed to has a closed flag.
This flag is set to true, just before the close() method is actually closing channels.
The add() method checks after adding a new channel, if this flag has been set to true.
If yes, the new channel is closed. If not, we have the guarantee, that this channel will be
closed by the ChannelGroup, because setting the closed flag to true happens-before closing any channels.

This behaviour can be activated by two new constructors. The old constructors are still there and behave like before.
Therefore, no existing code should be affected directly.

Result:

If activating this feature, the DefaultChannelGroup can be used, for managing server and child channels at once.
But this activating this feature means also, that a ChannelGroup cannot be reused after calling close().
2015-08-27 09:50:09 +02:00
Trustin Lee
fdfe3149ba Provide more control over DnsNameResolver.query() / Add NameResolver.resolveAll()
Related issues:
- #3971
- #3973
- #3976
- #4035

Motivation:

1. Previously, DnsNameResolver.query() retried the request query by its
own. It prevents a user from deciding when to retry or stop. It is also
impossible to get the response object whose code is not NOERROR.

2. NameResolver does not have an operation that resolves a host name
into multiple addresses, like InetAddress.getAllByName()

Modifications:

- Changes related with DnsNameResolver.query()
  - Make query() not retry
    - Move the retry logic to DnsNameResolver.resolve() instead.
  - Make query() fail the promise only when I/O error occurred or it
    failed to get a response
  - Add DnsNameResolverException and use it when query() fails so that
    the resolver can give more information about the failure
  - query() does not cache anymore.

- Changes related with NameResolver.resolveAll()
  - Add NameResolver.resolveAll()
  - Add SimpleNameResolver.doResolveAll()

- Changes related with DnsNameResolver.resolve() and resolveAll()
  - Make DnsNameResolveContext abstract so that DnsNameResolver can
    decide to get single or multiple addresses from it
  - Re-implement cache so that the cache works for resolve() and
    resolveAll()
  - Add 'traceEnabled' property to enable/disable trace information

- Miscellaneous changes
  - Use ObjectUtil.checkNotNull() wherever possible
  - Add InternetProtocolFamily.addressType() to remove repetitive
    switch-case blocks in DnsNameResolver(Context)
  - Do not raise an exception when decoding a truncated DNS response

Result:

- Full control over query()
- A user can now retrieve all addresses via (Dns)NameResolver.resolveAll()
- DNS cache works only for resolve() and resolveAll() now.
2015-08-18 17:40:13 +09:00
Trustin Lee
fd9ca8bbb3 Fix compilation errors 2015-08-18 12:42:33 +09:00
Ivan Bahdanau
08b73bf914 Unhealthy channel is not offered back to the pool.
Motivation:
When releasing unhealthy channel back to a pool we don't have to offer it since on acquire it will be discarded anyways.
 Also checking healthiness at release is a good idea so we don't end up having tons of unhealthy channels in the pool(unless they became unhealthy after being offered)

Modifications:
private SimpleChannelPool.offerIfHealthy() method added that is called from SimpleChannelPool.doReleaseChannel(). SimpleChannelPool.offerIfHealthy() offers channel back to pool only if channel is healthy.
Otherwise it throws setFailure exception to the promise.

 Result:
The pool is now much cleaner and not spammed with unhealthy channels.

Added ability to choose if channel health has to be validated on release by passing boolean flag.

Motivation:
Depending on performance preferences and individual use cases sometimes we would like to be able force health check of a channel at release time and do not offer it back to the pool. Other times we would want to just release channel and offer it back to the pool and check health only when we try to acquire that channel from the pool. See more details here: https://github.com/netty/netty/issues/4077#issuecomment-130461684

Modifications:
Future<Void> release(Channel channel, Promise<Void> promise, boolean offerHealthyOnly);
The offerHealthyOnly boolean flag allows developers to choose whether to do channel validation before offering it back to pool or not.
Appropriate modifications made to hierarchy of implementations of ChannelPool. offerHealthyOnly=true will force channel health to be checked before offering back to pool. offerHealthyOnly=false  will ignore channel health check and will just try just offer it back to the pool
 offerHealthyOnly=true by default.

Result:
Channel health check before offer back to pool is controlled by a flag now.

Code changed to satisfy checkstyle requirements.

Motivation:
Code needs to satisfy checkstyle requirements.

Modifications:
 SimpleChannelPool.java:279 line split to be less then 120 characters.
 SimpleChannelPool.java:280:31 space added after '{'
 SimpleChannelPool.java:282:17 space added after '{'
 SimpleChannelPoolTest.java:198 - extra white space line removed.

Result:
Code satisfies checkstyle requirements.

 offerHealthyOnly is passed as a constructor parameter now.

Motivation:
Instead of passing offerHealthyOnly as a method parameter it is better to pass it in as SimpleChannelPool or FixedChannelPool constructor.

Modifications:
 Redundant release method that takes offerHealthyOnly removed from ChannelPool.
 offerHealthyOnly parameter added to constructor for FixedChannelPool and SimpleChannelPool.

Result:
SimpleChannelPool and FixedChannelPool are now take offerHealthyOnly as a constructor parameter. Default behavior is: offerHealthyOnly=true.

Code changed to satisfy checkstyle requirements.

Motivation:
Code needs to satisfy checkstyle requirements.

Modifications:
 SimpleChannelPool.java:84: line made to be no longer then 120 characters.
 SimpleChannelPool.java:237: extra white space line removed.

Result:
Code satisfies checkstyle requirements.

Tests do not need to be too  copled to the code. Exception message should not be validated

Motivation:
We don't need our tests to be too coupled to the code. Exception type validation in tests is just good enough.

Modifications:
Exception validation message removed from SimpleChannelPoolTest.testUnhealthyChannelIsNotOffered() test.

Result:
The SimpleChannelPoolTest test is less coupled to the code now.

Stack trace set to empty for UNHEALTHY_NON_OFFERED_TO_POOL.

Motivation:
We don't need stack trace for UNHEALTHY_NON_OFFERED_TO_POOL.

Modifications:
Added  UNHEALTHY_NON_OFFERED_TO_POOL.setStackTrace(EmptyArrays.EMPTY_STACK_TRACE) to static init block.

Result:
UNHEALTHY_NON_OFFERED_TO_POOL's stack trace set to empty.

Minor code re-factorings.

Motivation:
For better code readability we need to apply several minor code re-factorings.

Modifications:
javadocs true -> {@code true}
offerHealthyOnly variable name changed to releaseHeathCheck
<p/> -> <p> in javadocs
offerHealthyOnly removed from doReleaseChannel as it not needed there.

Result:
Code quality is improved.

Code changed to satisfy checkstyle requirements.

Motivation:
Code needs to satisfy checkstyle requirements.

Modifications:
SimpleChannelPool.java:87: line made to be no longer then 120 characters.

Result:
Code satisfies checkstyle requirements.

Pull request needs to contain only necessary changes

Motivation:
The pull request should not contain unnecessary changes that are not needed as part of required functionality of pull request.

Modifications:
private void doReleaseChannel(final Channel channel, final Promise<Void> promise) - >  private void doReleaseChannel(Channel channel, Promise<Void> promise)

Result:
Pull request contains less unnecessary modifications.
2015-08-17 15:43:08 -07:00
Norman Maurer
9d417c1626 Fixing compile error, introduce by 5d011b8895 2015-08-12 14:25:45 +02:00
Ivan Bahdanau
5d011b8895 Improve the logic around acquire channel function is improved.
Motivation:
The acquire channel function resulted in calling itself several times in case when channel polled from the pool queue was unhealthy, which resulted FixedChannelPool to be called several times which in it's turn caused FixedChannelPool.acquire() to be called and resulted into acquireChannelCount to be unnecessary increased.
Example use case:
1) Create FixedChannelPool instance with one channel in the pool: new FixedChannelPool(cb, handler, 1)
2) Acquire channel A from the pool
3) close the channel A
4) Return it back to the pool
5) Acquire channel from the same pool again
Expected result:
new channel created and acquired, channel A that has been closed discarded and removed from the pool from being unhealthy
Actual result:
Channel A had been removed from the pool, how ever the new channel had never be acquired, instead the request to acquire had been added to the pending queue in FixedChannelPool and the acquireChannelCount is increased by one. The reason is that at the time when SimpleChannelPool figured out that the channel was unhealthy called FixedChannelPool.acquire to try to acquire new channel, how ever the request was added to the pendingTakQueue because by the time when FixedChannelPool.acquire was called, the acquireChannelCount was already "1" so new channel ould not be created cause of maxChannelsLimit=1.

Modifications:
The suggested approach modifies the SimpleChannelPool in a way so that when channel detected to be unhealthy it calls private method SimpleChannelPool.acquireHealthyFromPoolOrNew() which guarantees that SimpleChannelPool actually either finds a healthy channel in the pool and returns it or causes the promise.cause() in case when new channel was failed to be created.

 Result:
The  ```acquiredChannelCount``` is now calculated correctly as a result of SimpleChannelPool.acquire() of not being recursive on overridable acquire method.
2015-08-12 06:35:56 +02:00
Scott Mitchell
cf171ff525 maxBytesPerRead channel configuration
Motiviation:
The current read loops don't fascilitate reading a maximum amount of bytes. This capability is useful to have more fine grain control over how much data is injested.

Modifications:
- Add a setMaxBytesPerRead(int) and getMaxBytesPerRead() to ChannelConfig
- Add a setMaxBytesPerIndividualRead(int) and getMaxBytesPerIndividualRead to ChannelConfig
- Add methods to RecvByteBufAllocator so that a pluggable scheme can be used to control the behavior of the read loop.
- Modify read loop for all transport types to respect the new RecvByteBufAllocator API

Result:
The ability to control how many bytes are read for each read operation/loop, and a more extensible read loop.
2015-08-05 23:59:54 -07:00
fratboy
083dbe5c59 Correctly count acquired channels when timeout occurs in FixedChannelPool
Motivation:

We don't decrease acquired channel count in FixedChannelPool when timeout occurs by AcquireTimeoutAction.NEW and eventually fails.

Modifications:

Set AcquireTask.acquired=true to call decrementAndRunTaskQueue when timeout action fails.

Result:

Acquired channel count decreases correctly.
2015-07-30 07:55:46 +02:00
Norman Maurer
bd42987143 [#3988] Correctly count acquired channels in FixedChannelPool
Motivation:

We missed to correctly count acquired channels in FixedChannelPool which could produce an assert error.

Modifications:

Only try to decrement acquired count if the channel was really acuired.

Result:

No more assert error possible.
2015-07-21 21:07:13 +02:00
Minwoo Jung
5fb349ca5a Update ChannelConfig.java
{@link ByteBufAllocator} -> {@link MessageSizeEstimator} on
https://github.com/netty/netty/blob/4.0/transport/src/main/java/io/netty/channel/ChannelConfig.java#L248
2015-07-19 18:10:24 +02:00
Norman Maurer
f1c5f0e0c7 [#3967] Guard against NPE in PendingWriteQueue
Motivation:

If the Channel is already closed when the PendingWriteQueue is created it will generate a NPE when add or remove is called later.

Modifications:

Add null checks to guard against NPE.

Result:

No more NPE possible.
2015-07-17 21:31:27 +02:00
Louis Ryan
e6ed3ce6b1 Make CoalescingBufferQueueTest properly release all buffers including the composites removed from the queue.
Motivation:
Test was leaving composite buffers taken from the queue unreleased.

Modifications:
Make the test release buffers.

Result:
Nagging about leaked buffers should stop.
2015-07-16 21:46:45 +02:00
Effective Light
80c59864cf Fix DatagramChannel javadoc
there seems to be an extra arrow incorrectly placed there when trying to link "Channel."
2015-07-12 20:19:59 +02:00
Louis Ryan
cdbf43596d Add generic utility for enqueuing buffers with promises and dequeueing them in arbitrary byte ranges.
Motivation:
Simplifies writing code that needs to merge or slice a sequence of buffer & promise pairs into chunks of arbitrary sizes.
For example in HTTP2 we merge or split buffers across fixed-size DATA frame boundaries.

Modifications:
Add new utility class CoalescingBufferQueue

Result:
Following this change HTTP2 code will switch to use it instead of CompositeByteBuffer for DATA frame coalescing.
2015-07-09 07:50:02 -07:00
Norman Maurer
8650679f29 Fix regression introduced by 11f9e9084b
Motivation:

While cherry-picked 11f9e9084b I changed the EmbeddedChannel implementation to not allow no ChannelHandlers when constructing it.
This was done by mistake.

Modifications:

Revert change and add unit test.

Result:

Restore old behavior.
2015-07-08 21:37:58 +02:00
Norman Maurer
ab43c78c85 Reduce memory usage by EmbeddedChannel
Motivation:

When using an EmbeddedChannel often it either does inbound or outbound processing which means we only often need one queue.

Modifications:

Lazy init the inbound and outbound message queues.

Result:

Less memory usage.
2015-07-08 10:49:40 +02:00
Norman Maurer
0f6559d188 [#3780] Handle ChannelInitializer exception in exceptionCaught()
Motivation:

At the moment we directly closed the Channel when an exception accoured durring initChannel(...) without giving the user any way to do extra or special handling.

Modifications:

Handle the exception in exceptionCaught(...) of the ChannelInitializer which will by default log and close the Channel. This way the user can override this.

Result:

More felixible handling of exceptions.
2015-07-07 09:13:22 +02:00
Norman Maurer
11f9e9084b [#3921] EmbeddedChannel should add ChannelHandlers once registered
Motivation:

Currently in EmbeddedChannel we add the ChannelHandlers before the Channel is registered which leads to have the handlerAdded(...) callback
be called from outside the EventLoop and also prevent the user to obtain a reference to the EventLoop in the callback itself.

Modifications:

Delay adding ChannelHandlers until EmbeddedChannel is registered.

Result:

Correctly call handlerAdded(...) after EmbeddedChannel is registered.
2015-07-07 08:54:27 +02:00
Norman Maurer
dd6ac55fa0 ServerBootstrap.handler(...) will add handler before Channel is registered.
Motivation:

If you set a ChannelHandler via ServerBootstrap.handler(...) it is added to the ChannelPipeline before the Channel is registered. This will lead to and IllegalStateException if a user tries to access the EventLoop in the ChannelHandler.handlerAdded(...) method.

Modifications:

Delay the adding of the ChannelHandler until the Channel was registered.

Result:

No more IllegalStateException.
2015-07-07 08:44:13 +02:00
a-mkarjalainen
e1e8a59ad9 Fix broken constructor chaining for FixedChannelPool class.
Motivation:

Only one of the three FixedChannelPool constructors checks for the constructor
arguments. Therfore it was possible to create a pool with zero maxConnections.

This change chains all constructors together, so that the last one
in the chain always checks the validity of the arguments, regardless of the
constructor used.

Result:

It is no longer possible to create a FixedChannelPool instance with invalid
maxConnections or maxPendingAcquires parameters.
2015-06-18 20:11:04 +02:00
Norman Maurer
4a0d68c50b [#3881] FixedChannelPool creates 1 more channel than maxConnections
Motivation:

FixedChannelPool should enforce a number of maximal used channels, but due a bug we fail to correctly enforce this.

Modifications:

Change check to correctly only acquire channel if we not hit the limit yet.

Result:

Correct limiting.
2015-06-16 20:10:27 +02:00
Scott Mitchell
09d826ed46 Channel writable bytes feature
Motivation:
To avoid buffering too much it would be useful to get an estimate of how many bytes can be written to a Channel before it becomes unwritable.

Modifications:
- Update the Channel interface to support 2 new methods. 1 to give how many bytes before unwritable. 1 to give how many bytes before writable.
- Update the AbstractChannel implementation to delegate to the ChannelOutboundBuffer.

Result:
The Channel interface supports 2 new methods which provide more visibility into writability.
2015-06-12 12:39:53 -07:00
Norman Maurer
3a4d64bb84 Allow to receive a ChannelGroupFuture that will be notified once all Channels are closed.
Motivation:

It's useful to be able to be notified once all Channels that are part of the ChannelGroup are notified. This can for example be useful if you want to do a graceful shutdown.

Modifications:

- Add ChannelGroup.newCloseFuture(...) which will be notified once all Channels are notified that are part of the ChannelGroup at the time of calling.

Result:

Easier to be notified once all Channels within a ChannelGroup are closed.
2015-06-12 13:56:33 +02:00
Scott Mitchell
43a558b9e1 ChannelOutboundBuffer bytes before writable accessor
Motiviation:
There are currently no accessors which provide visbility into how many bytes must be written in order for a writability change to occur. This feature would be useful for codecs which intent to control how many bytes are queued at any given time.

Modifications:
- add bytesBeforeUnWritable() which will give the number of bytes before the buffer (and associated channel) transitions to not writable
- add bytesBeforeWritable() which will give the number of bytes that must be drained from the queue until the channel becomes writable.

Result:
More visibility into writability for the ChannelOutboundBuffer.
2015-06-10 08:44:03 -07:00
Norman Maurer
ed3d26cf7d [#3837] Null out ByteBuffer[] array once done
Motivation:

the ByteBuffer[] that we keep in the ThreadLocal are never nulled out which can lead to have ByteBuffer instances sit there forever.
This is even a bigger problem if nioBuffer() of ByteBuffer returns a new ByteBuffer that can not be destroyed by ByteBuffer.release().

Modifications:

Null out ByteBuffer array after processing.

Result:

No more dangling references after done.
2015-06-04 12:33:25 +02:00
Jean-Rémi Desjardins
0bd6acc5d8 Fix typo 2015-06-02 13:02:01 +02:00
Trustin Lee
5f59591a72 Fix sporadic assertion failure in SingleThreadEventLoopTest
Motivation:

SingleThreadEventLoopTest.testScheduleTaskAtFixedRate() fails often due to:

- too little tolerance
- incorrect assertion (it compares only with the previous timestamp)

Modifications:

- Increase the timestamp difference tolerance from 10ms to 20ms
- Improve the timestamp assertion so that the comparison is performed against the first recorded timestamp
- Misc: Fix broken Javadoc tag

Result:

More build stability
2015-06-01 14:45:36 +09:00
Norman Maurer
2a848ce1c9 Mention correct order in SimplechannelPool javadocs 2015-05-29 20:56:52 +02:00
Norman Maurer
e903979a28 Not try to write more then Integer.MAX_VALUE / SSIZE_MAX via writev
Motivation:

When trying to write more then Integer.MAX_VALUE / SSIZE_MAX via writev(...) the OS may return EINVAL depending on the kernel or the actual OS (bsd / osx always return EINVAL). This will trigger an IOException.

Modifications:

Never try to write more then Integer.MAX_VALUE / SSIZE_MAX when using writev.

Result:

No more IOException when write more data then Integer.MAX_VALUE / SSIZE_MAX via writev.
2015-05-21 12:02:10 +02:00
nmittler
bb5f9fe078 Scoring loopback addresses as 0 in DefaultChannelId
Motivation:

As described in #3490, [compareAddresses](https://github.com/netty/netty/blob/master/transport/src/main/java/io/netty/channel/DefaultChannelId.java#L182)  seems to allow loopback addresses to score high, disallowing replacing the loopback address.

Modifications:

Changed [scoreAddress](https://github.com/netty/netty/blob/master/transport/src/main/java/io/netty/channel/DefaultChannelId.java#L283) to score loopback addresses as 0.

Result:

Fixes #3490
2015-05-08 13:14:36 -07:00
Norman Maurer
f23b7b4efd [maven-release-plugin] prepare for next development iteration 2015-05-07 14:21:08 -04:00
Norman Maurer
871ce43b1f [maven-release-plugin] prepare release netty-4.1.0.Beta5 2015-05-07 14:20:38 -04:00
Norman Maurer
f6c2c99efc [#3740] Add missing parentheses so the fix works as expected. 2015-05-06 23:03:53 +02:00
Norman Maurer
e71e40057f Fix possible IllegalStateException caused by closeNotifyTimeout when using SslHandler
Motivation:

In the SslHandler we schedule a timeout at which we close the Channel if a timeout was detected during close_notify. Because this can race with notify the flushFuture we can see an IllegalStateException when the Channel is closed.

Modifications:

- Use a trySuccess() and tryFailure(...) to guard against race.

Result:

No more race.
2015-05-06 21:50:16 +02:00
Norman Maurer
57e7571c65 Not trigger channelWritabilityChanged if fail messages before close Channel.
Motivation:

We should not trigger channelWritabilityChanged during failing message when we are about to close the Channel as otherwise the use may try again writing even if the Channel is about to get closed.

Modifications:

Add new boolean param to ChannelOutboundBuffer.failFlushed(...) which allows to specify if we should notify or not.

Result:

channelWritabilityChanged is not triggered anymore if we cloe the Channel because of an IOException during write.
2015-05-06 21:36:43 +02:00
Norman Maurer
f839f65c15 Do not defer closing of Channel when in flush
Motivation:

Previously, we deferred the closing of the Channel when we were flushing. This is problematic as this means that if the user adds a ChannelFutureListener, that will close the Channel, the closing will not happen until we are done with flushing. This can lead to more data is sent than expected.

Modifications:

- Do not defer closing when in flush

Result:

Correctly respect order of events and closing the Channel ASAP
2015-05-06 21:07:10 +02:00
Norman Maurer
e276cb796c Correct semantic of LocalChannel.doWrite(...) and remove memory copy
Motivation:

The semantic of LocalChannel.doWrite(...) were a bit off as it notified the ChannelFuture before the data was actual moved to the peer buffer.

Modifications:

- Use our MPSC queue as inbound buffer
- Directly copy to data to the inbound buffer of the peer and either success or fail the promise after each copy.

Result:

Correct semantic and less memory copies.
2015-05-06 10:44:31 +02:00
Scott Mitchell
c94e7f744a DefaultChannelPipeline needs to release objects
Motiviation:
If user events or excpetions reach the tail end of the pipeline they are not released. This could result in buffer leaks.

Motivation:
- Use the ReferenceCountUtil.release to release objects for the userEventTriggered and exceptionCaught methods on DefaultChannelPipeline

Result:
2 less areas where buffer leaks can occur.
2015-05-06 06:46:52 +02:00
Trustin Lee
63a02fc04e Revamp DNS codec
Motivation:

There are various known issues in netty-codec-dns:

- Message types are not interfaces, which can make it difficult for a
  user to implement his/her own message implementation.
- Some class names and field names do not match with the terms in the
  RFC.
- The support for decoding a DNS record was limited. A user had to
  encode and decode by him/herself.
- The separation of DnsHeader from DnsMessage was unnecessary, although
  it is fine conceptually.
- Buffer leak caused by DnsMessage was difficult to analyze, because the
  leak detector tracks down the underlying ByteBuf rather than the
  DnsMessage itself.
- DnsMessage assumes DNS-over-UDP.
- To send an EDNS message, a user have to create a new DNS record class
  instance unnecessarily.

Modifications:

- Make all message types interfaces and add default implementations
- Rename some classes, properties, and constants to match the RFCs
  - DnsResource -> DnsRecord
  - DnsType -> DnsRecordType
  - and many more
- Remove DnsClass and use an integer to support EDNS better
- Add DnsRecordEncoder/DnsRecordDecoder and their default
  implementations
  - DnsRecord does not require RDATA to be ByteBuf anymore.
  - Add DnsRawRecord as the catch-all record type
- Merge DnsHeader into DnsMessage
- Make ResourceLeakDetector track AbstractDnsMessage
- Remove DnsMessage.sender/recipient properties
  - Wrap DnsMessage with AddressedEnvelope
  - Add DatagramDnsQuest and DatagramDnsResponse for ease of use
  - Rename DnsQueryEncoder to DatagramDnsQueryEncoder
  - Rename DnsResponseDecoder to DatagramDnsResponseDecoder
- Miscellaneous changes
  - Add StringUtil.TAB

Result:

- Cleaner APi
- Can support DNS-over-TCP more easily in the future
- Reduced memory footprint in the default DnsQuery/Response
  implementations
- Better leak tracking for DnsMessages
- Possibility to introduce new DnsRecord types in the future and provide
  full record encoder/decoder implementation.
- No unnecessary instantiation for an EDNS pseudo resource record
2015-05-01 11:33:16 +09:00
Norman Maurer
56c98839c3 [#3218] Add ChannelPool / ChannelPoolMap abstraction and implementations
Motivation:

Many projects need some kind a Channel/Connection pool implementation. While the protocols are different many things can be shared, so we should provide a generic API and implementation.

Modifications:

Add ChannelPool / ChannelPoolMap API and implementations.

Result:

Reusable / Generic pool implementation that users can use.
2015-04-30 12:13:19 +02:00
Norman Maurer
1cce998bb0 [#3662] Fail the connect future on close
Motivation:

Because of a bug we missed to fail the connect future when doClose() is called. This can lead to a future which is never notified and so may lead to deadlocks in user-programs.

Modifications:

Correctly fail the connect future when doClose() is called and the connection was not established yet.

Result:

Connect future is always notified.
2015-04-27 20:05:00 +02:00
Norman Maurer
ccde870b38 Revert "Ensure channelReadComplete() is called only when necessary"
This reverts commit 27a25e29f7.
2015-04-20 09:10:41 +02:00
Norman Maurer
bb692816d2 Revert "Do not suppress channelReadComplete() when a handler was just added"
This reverts commit 720faa4df1.
2015-04-20 09:10:29 +02:00
Norman Maurer
a123c495e1 Revert "Add another test case for channelReadComplete() suppression"
This reverts commit daa04cb4f1.
2015-04-20 09:10:14 +02:00
Norman Maurer
b4b14ea19f Ensure backward-compability with 4.0
Motivation:

Each different *ChannelOption did extend ChannelOption in 4.0, which we changed in 4.1. This is a breaking change in terms of the API so we need to ensure we keep the old hierarchy.

Modifications:

- Let all *ChannelOption extend ChannelOption
- Add back constructor and mark it as @deprecated

Result:

No API breakage between 4.0 and 4.1
2015-04-19 13:25:42 +02:00