Commit Graph

1812 Commits

Author SHA1 Message Date
Jason Tedor
98beb777f8 Enable configuring available processors
Motivation:

In cases when an application is running in a container or is otherwise
constrained to the number of processors that it is using, the JVM
invocation Runtime#availableProcessors will not return the constrained
value but rather the number of processors available to the virtual
machine. Netty uses this number in sizing various resources.
Additionally, some applications will constrain the number of threads
that they are using independenly of the number of processors available
on the system. Thus, applications should have a way to globally
configure the number of processors.

Modifications:

Rather than invoking Runtime#availableProcessors, Netty should rely on a
method that enables configuration when the JVM is started or by the
application. This commit exposes a new class NettyRuntime for enabling
such configuraiton. This value can only be set once. Its default value
is Runtime#availableProcessors so that there is no visible change to
existing applications, but enables configuring either a system property
or configuring during application startup (e.g., based on settings used
to configure the application).

Additionally, we introduce the usage of forbidden-apis to prevent future
uses of Runtime#availableProcessors from creeping. Future work should
enable the bundled signatures and clean up uses of deprecated and
other forbidden methods.

Result:

Netty can be configured to not use the underlying number of processors,
but rather the constrained number of processors.
2017-04-23 10:31:17 +02:00
Norman Maurer
c663a94359 Fix buffer leak in local transport when a close triggers the close of a remote peer and there are still messages in the inbound buffer.
Motivation:

We need to release all the buffers that may be put into our inbound queue since we closed the Channel to ensure we not leak any memory. This is fine as it basically gives the same guarantees as TCP which  means even if the promise was notified before its not really guaranteed that the "remote peer" will see the buffer at all.

Modifications:

Ensure we release all buffers in the inbound buffer if a doClose() is called.

Result:

No more leaks.
2017-04-21 07:42:29 +02:00
Nikolay Fedorovskikh
970d310ec9 Regulation of the InternetProtocolFamily usage
Motivation:

1. The use of InternetProtocolFamily is not consistent:
   the DnsNameResolverContext and DnsNameResolver contains switches
   instead of appropriate methods usage.
2. The InternetProtocolFamily class contains redundant switches in the
   constructor.

Modifications:

1. Replacing switches to the use of an appropriate methods.
2. Simplifying the InternetProtocolFamily constructor.

Result:

Code is cleaner and simpler.
2017-04-20 05:22:24 +02:00
Nikolay Fedorovskikh
0692bf1b6a fix the typos 2017-04-20 04:56:09 +02:00
Norman Maurer
119383873d VoidChannelPromise not notified when exception is thrown.
Motivation:

When a VoidChannelPromise is used by the user we need to ensure we propergate the exception through the ChannelPipeline otherwise the exception will just be swallowed and so the user has no idea whats going on.

Modifications:

- Always call tryFailure / trySuccess even when we use the VoidChannelPromise
- Add unit test

Result:

Fixes [#6622].
2017-04-19 11:25:59 +02:00
Scott Mitchell
c37267d682 NioEventLoop#rebuildSelector0 throws ClassCastException
Motivation:
Commit 795f318 simplified some code related to the special case Set for the selected keys and introduced a Selector wrapper to make sure this set was properly reset. However the JDK makes assumptions about the type of Selector and this type is not extensible. This means whenever we call into the JDK we must provide the unwrapped version of the Selector or we get a ClassCastException. We missed a case of unwrapping in NioEventLoop#rebuildSelector0.

Modificaitons:
- NioEventLoop#openSelector should return a tuple so we can atomically set the wrapped and unwrapped Selector
- NioEventLoop#rebuildSelector0 should use the unwrapped version of the selector

Result:
Fixes https://github.com/netty/netty/issues/6607.
2017-04-05 19:34:49 -07:00
Boaz Avital
bf08ed0b93 fix failure logging of value on channel option set
Motivation:

The code accidentally passes channel twice instead of value, resulting in logs like:
Failed to set channel option 'SO_SNDBUF' with value '[id: 0x2c5b2eb4]' for channel '[id: 0x2c5b2eb4]'

Modifications:

Pass value instead of channel where it needs to be.

Result:

Failed to set channel option 'SO_SNDBUF' with value '0' for channel '[id: 0x9bd3c5b8]'
2017-03-21 22:02:23 -07:00
Norman Maurer
2b8c8e0805 [maven-release-plugin] prepare for next development iteration 2017-03-10 07:46:17 +01:00
Norman Maurer
1db58ea980 [maven-release-plugin] prepare release netty-4.1.9.Final 2017-03-10 07:45:28 +01:00
Norman Maurer
9ade81ab5b Use system property to detect if root is running the program
Motivation:

We forked a new process to detect if the program is run by root. We should better just use user.name system property

Modifications:

- Change PlatformDependent.isRoot0() to read the user.name system property to detect if root runs the program and rename it to maybeSuperUser0().
- Rename PlatformDependent.isRoot() to maybeSuperUser() and let it init directly in the static block

Result:

Less heavy way to detect if the program is run by root.
2017-03-09 11:16:10 +01:00
Nikolay Fedorovskikh
2993760e92 Fix misordered 'assertEquals' arguments in tests
Motivation:

Wrong argument order in some 'assertEquals' applying.

Modifications:

Flip compared arguments.

Result:

Correct `assertEquals` usage.
2017-03-08 22:48:37 -08:00
Jason Brown
3861b7de2b Motivation:
Make the FileRegion comments about which transports are supported more accurate.
Also, eleminate any outstanding references to FileRegion.transfered as the method was renamed for spelling.

Modifications:

Class-level comment on FileRegion, can call renamed method.

Result:

More accurate documentation and less calls to deprecated methods.
2017-03-06 14:43:52 -08:00
Scott Mitchell
2cff918044 Correct usages of internalNioBuffer
Motivation:
There are numerous usages of internalNioBuffer which hard code 0 for the index when the intention was to use the readerIndex().

Modifications:
- Remove hard coded 0 for the index and use readerIndex()

Result:
We are less susceptible to using the wrong index, and don't make assumptions about the ByteBufAllocator.
2017-03-02 12:51:22 -08:00
Nikolay Fedorovskikh
943f4ec7ff Make methods 'static' where it missed
Motivation:

Calling a static method is faster then dynamic

Modifications:

Add 'static' keyword for methods where it missed

Result:

A bit faster method calls
2017-02-23 11:01:57 +01:00
Nikolay Fedorovskikh
0623c6c533 Fix javadoc issues
Motivation:

Invalid javadoc in project

Modifications:

Fix it

Result:

More correct javadoc
2017-02-22 07:31:07 +01:00
Nikolay Fedorovskikh
634a8afa53 Fix some warnings at generics usage
Motivation:

Existing warnings from java compiler

Modifications:

Add/fix type parameters

Result:

Less warnings
2017-02-22 07:29:59 +01:00
Hanson
d0a3877535 Ensure trying to recover from exceptionCaught on the ServerChannel works as expected
Motivation:

When "Too many open files" happens,the URLClassLoader cannot do any classloading because URLClassLoader need a FD  for findClass. Because of this the anonymous inner class that is created to re-enable auto read may cause a problem.

Modification:

Pre-create Runnable that is scheduled and so ensure it is not lazy loaded.

Result:

No more problems when try to recover.
2017-02-19 13:41:11 +01:00
Norman Maurer
fbf0e5f4dd Prefer JDK ThreadLocalRandom implementation over ours.
Motivation:

We have our own ThreadLocalRandom implementation to support older JDKs . That said we should prefer the JDK provided when running on JDK >= 7

Modification:

Using ThreadLocalRandom implementation of the JDK when possible.

Result:

Make use of JDK implementations when possible.
2017-02-16 15:44:00 -08:00
Scott Mitchell
795f318c3c Use a single array in SelectedSelectionKeySet
Motivation:
SelectedSelectionKeySet currently uses 2 arrays internally and users are expected to call flip() to access the underlying array and switch the active array. However we do not concurrently use 2 arrays at the same time and we can get away with using a single array if we are careful about when we reset the elements of the array.

Modifications:
- Introduce SelectedSelectionKeySetSelector which wraps a Selector and ensures we reset the underlying SelectedSelectionKeySet data structures before we select
- The loop bounds in NioEventLoop#processSelectedKeysOptimized can be defined more precisely because we know the real size of the underlying array

Result:
Fixes https://github.com/netty/netty/issues/6058
2017-02-16 15:10:37 -08:00
Scott Mitchell
544d771152 Checkstyle cleanup from d59b4840c1 2017-02-15 12:03:11 -08:00
Jason Tedor
d59b4840c1 Cleanup ChannelId handling of basic methods
Motiviation:

Simplify implementation of compareTo/equals/hashCode for ChannelIds.

Modifications:

We simplfy the hashCode implementation for DefaultChannelId by not
making it random, but making it based on the underlying data. We fix the
compareTo implementation for DefaultChannelId by using lexicographic
comparison of the underlying data array. We fix the compareTo
implementation for CustomChannelId to avoid the possibility of overflow.

Result:

Cleaner code that is easier to maintain.
2017-02-15 11:53:36 -08:00
Norman Maurer
90bc605477 Initialization of PlatformDependent0 fails on Java 9
Motivation:

Initialization of PlatformDependent0 fails on Java 9 in static initializer when calling setAccessible(true).

Modifications:

Add RefelectionUtil which can be used to safely try if setAccessible(true) can be used or not and if not fail back to non reflection.

Result:

Fixed [#6345]
2017-02-14 10:15:27 +01:00
Norman Maurer
78586a99b6 Ensure CombinedChannelDuplexHandler can not be shared.
Motivation:

CombinedChannelDuplexHandler must not be shared as it contains state.

Modifications:

Enforce that it is not shared.

Result:

Fixes [#6333]
2017-02-14 08:42:23 +01:00
fenik17
0cf3f54a8d Adding 'final' keyword for private fields where possible
Motivation

Missing 'final' keyword for fields

Modifications

Add 'final' for fields where possible

Result

More safe and consistent code
2017-02-14 08:29:15 +01:00
Scott Mitchell
413d6eba53 EPOLL include error description and cause in exceptions
Motivation:
EPOLL annotates some exceptions to provide the remote address, but the original exception is not preserved. This may make determining a root cause more difficult. The static EPOLL exceptions references the native method that failed, but does not provide a description of the actual error number. Without the description users have to know intimate details about the native calls and how they may fail to debug issues.

Modifications:
- annotated exceptions should preserve the original exception
- static exceptions should include the string description of the expected errno

Result:
EPOLL exceptions provide more context and are more useful to end users.
2017-02-13 18:37:16 -08:00
Scott Mitchell
a1b5b5dcca EpollRecvByteAllocatorHandle doesn't inform delegate of more data
Motivation:
EpollRecvByteAllocatorHandle intends to override the meaning of "maybe more data to read" which is a concept also used in all existing implementations of RecvByteBufAllocator$Handle but the interface doesn't support overriding. Because the interfaces lack the ability to propagate this computation EpollRecvByteAllocatorHandle attempts to implement a heuristic on top of the delegate which may lead to reading when we shouldn't or not reading data.

Modifications:
- Create a new interface ExtendedRecvByteBufAllocator and ExtendedHandle which allows the "maybe more data to read" between interfaces
- Deprecate RecvByteBufAllocator and change all existing implementations to extend ExtendedRecvByteBufAllocator
- transport-native-epoll should require ExtendedRecvByteBufAllocator so the "maybe more data to read" can be propagated to the ExtendedHandle

Result:
Fixes https://github.com/netty/netty/issues/6303.
2017-02-13 17:42:24 -08:00
Dmitriy Dumanskiy
64838f1505 Cleanup : validatePromise ranamed to isNotValidPromise and added more tests for corner cases.
Motivation:

Result of validatePromise() is always inverted with if (!validatePromise()).

Modification:

validatePromise() renamed to isNotValidPromise() and now returns inverted state so you don't need to invert state in conditions. Also name is now more meaningful according to returned result.
Added more tests for validatePromise corner cases with Exceptions.

Result:

Code easier to read. No need in inverted result.
2017-02-10 12:39:58 +01:00
Norman Maurer
a7c0ff665c Only use Mockito for mocking.
Motivation:

We used various mocking frameworks. We should only use one...

Modifications:

Make usage of mocking framework consistent by only using Mockito.

Result:

Less dependencies and more consistent mocking usage.
2017-02-07 08:47:22 +01:00
Norman Maurer
8a5e42ad2e Correct fail write with NotYetConnectedException when OioDatagramChannel is not connected yet.
Motivation:

NioDatagramChannel fails a write with NotYetConnectedException when the DatagramChannel was not yet connected and a ByteBuf is written. The same should be done for OioDatagramChannel as well.

Modifications:

Make OioDatagramChannel consistent with NioDatagramChannel

Result:

Correct and consistent implementations of DatagramChannel
2017-02-06 11:06:00 +01:00
周岑
48f6541cb3 delete no useful intermediate variables
delete no useful intermediate variables
2017-02-06 07:55:29 +01:00
Norman Maurer
735d6dd636 [maven-release-plugin] prepare for next development iteration 2017-01-30 15:14:02 +01:00
Norman Maurer
76e22e63f3 [maven-release-plugin] prepare release netty-4.1.8.Final 2017-01-30 15:12:36 +01:00
Tim Brooks
3344cd21ac Wrap operations requiring SocketPermission with doPrivileged blocks
Motivation:

Currently Netty does not wrap socket connect, bind, or accept
operations in doPrivileged blocks. Nor does it wrap cases where a dns
lookup might happen.

This prevents an application utilizing the SecurityManager from
isolating SocketPermissions to Netty.

Modifications:

I have introduced a class (SocketUtils) that wraps operations
requiring SocketPermissions in doPrivileged blocks.

Result:

A user of Netty can grant SocketPermissions explicitly to the Netty
jar, without granting it to the rest of their application.
2017-01-19 21:12:52 +01:00
Norman Maurer
cd9008f95b Remove unnecessray for loop missed by fac0ca8319 2017-01-19 19:13:57 +01:00
Scott Mitchell
9a4aa617f4 PlatformDependent#getClassLoader fails in restrictive classloader environment
Motivation:
https://github.com/netty/netty/pull/6042 only addressed PlatformDependent#getSystemClassLoader but getClassLoader is also called in an optional manner in some common code paths but fails to catch a general enough exception to continue working.

Modifications:
- Calls to getClassLoader which can continue if results fail should catch Throwable

Result:
More resilient code in the presense of restrictive class loaders.
Fixes https://github.com/netty/netty/issues/6246.
2017-01-19 09:01:36 -08:00
周岑
86020a2858 Typo error: Method invoker() no longer exists
Method invoker() no longer exists
2017-01-19 12:16:31 +01:00
Norman Maurer
fac0ca8319 Warn about not-supported ChannelOption when bootstrap Channels.
Motivation:

We not warned about not-supported ChannelOptions when set the options for the ServerChannel.

Modifications:

- Share code for setting ChannelOptions during bootstrap

Result:

Warning is logged when a ChannelOption is used that is not supported during bootstrap a Channel. See also [#6192]
2017-01-19 08:27:19 +01:00
Norman Maurer
7f01da8d0f [maven-release-plugin] prepare for next development iteration 2017-01-12 11:36:51 +01:00
Norman Maurer
7a21eb1178 [maven-release-plugin] prepare release netty-4.1.7.Final 2017-01-12 11:35:58 +01:00
Norman Maurer
c1830c8b4e Fix missleading comment in AbstractChannelHandlerContext.invokeHandler()
Motivation:

The comment on AbstractChannelHandlerContext.invokeHandler() is incorrect and missleading. See [#6177]

Modifications:

Change true to false to correct the comment.

Result:

Fix missleading and incorrect comment.
2017-01-10 12:05:24 +01:00
Jon Chambers
074075de7e Expose channel pool configuration to subclasses.
Motivation:

`SimpleChannelPool` subclasses are likely to override the `connectChannel` method, and are likely to clobber the cloned `Bootstrap` handler in the process. To allow subclasses to properly notify the pool listener of new connections, we should expose (at least) the `handler` property of the pool to subclasses.

Modifications:

Expose `SimpleChannelPool` properties to subclasses via `protected` getters.

Result:

Subclasses can now use the bootstrap, handler, health checker, and health-check-on-release preoperties from their superclass.
2016-12-21 20:45:01 +01:00
Scott Mitchell
3d11334151 Fix DefaultChannelId MAC address parsing bug
Motivation:
DefaultChannelId provides a regular expression which validates if a user provided MAC address is valid. This regular expression may allow invalid MAC addresses and also not allow valid MAC addresses.

Modifications:
- Introduce a MacAddressUtil#parseMac method which can parse and validate the MAC address at the same time. The regular expression check before hand is additional overhead if we have to parse the MAC address.

Result:
Fixes https://github.com/netty/netty/issues/6132.
2016-12-20 17:06:27 -08:00
Norman Maurer
cfd8fb10db [#6134] Do not limit the PID to be <= 4194304
Motivation:

On some platforms the PID my be bigger then 4194304 so we should not limit it to 4194304.

Modifications:

Only check that the PID is a valid Integer

Result:

No more warnings on systems where the PID is bigger then 4194304.
2016-12-20 10:31:16 +01:00
Norman Maurer
89e93968ac Remove usage of own Atomic*FieldUpdater in favor of JDKs
Motivation:

In later Java8 versions our Atomic*FieldUpdater are slower then the JDK implementations so we should not use ours anymore. Even worse the JDK implementations provide for example an optimized version of addAndGet(...) using intrinsics which makes it a lot faster for this use-case.

Modifications:

- Remove methods that return our own Atomic*FieldUpdaters.
- Use the JDK implementations everywhere.

Result:

Faster code.
2016-12-15 08:09:06 +00:00
Scott Mitchell
002c99e751 NIO ServerChannel shouldn't close because of Exception
Motivation:
e102a008b6 changed a conditional where previously the NIO ServerChannel would not be closed in the event of an exception.

Modifications:
- Restore the logic prior to e102a008b6 which does not automatically close ServerChannels for IOExceptions

Result:
NIO ServerChannel doesn't close automatically for an IOException.
2016-12-05 20:51:05 -08:00
Norman Maurer
eed6791f8e Cleanup after commit fc1cdc991e 2016-12-05 12:18:35 +01:00
Norman Maurer
fc1cdc991e [#6095] Remove catching of ConcurrentModificationException as this can not happen.
Motivation:

We should not catch ConcurrentModificationException as this can never happen because things are executed on the EventLoop thread.

Modifications:

Remove try / catch

Result:

Cleaner code.
2016-12-04 18:59:10 +01:00
Derbylock
f6b37a38ba Removed final keyword from FixedChannelPool 2016-11-23 13:37:37 +01:00
Scott Mitchell
a25101dd0b Now that LocalChannel#releaseInboundBuffers is only called from the EventLoop (eb4d317b9d) it should clear readInProgress and drain/release the queue. Otherwise if a read event is pending (doBeginRead) was called we may later call channelRead or channelReadComplete after we have closed the channel.
Modifications:
LocalChannel#releaseInboundBuffers should always clear/release the queue and set readInProgress to false

Result:
LocalChannel queue is more reliably cleaned up.
2016-11-21 11:11:43 -08:00
Scott Mitchell
eb4d317b9d Fix LocalChannel close sequence
Motivation:
LocalChannel attempts to close its peer socket when ever it is closed. However if the channels are on different EventLoops we may attempt to process events for the peer channel on the wrong EventLoop.

Modifications:
- Ensure the close process ensures we are on the correct thread before accessing data

Result:
More correct LocalChannel close code.
2016-11-21 10:33:55 -08:00
Scott Mitchell
a043cf4a98 Catch exceptions from PlatformDependent#getSystemClassLoader
Motivation:
PlatformDependent#getSystemClassLoader may throw a wide variety of exceptions based upon the environment. We should handle all exceptions and continue initializing the slow path if an exception occurs.

Modifications:
- Catch Throwable in cases where PlatformDependent#getSystemClassLoader is used

Result:
Fixes https://github.com/netty/netty/issues/6038
2016-11-19 09:23:25 -08:00
Scott Mitchell
c1932a8537 ByteBuf Input Stream Reference Count Ownership
Motivation:
Netty provides a adaptor from ByteBuf to Java's InputStream interface. The JDK Stream interfaces have an explicit lifetime because they implement the Closable interface. This lifetime may be differnt than the ByteBuf which is wrapped, and controlled by the interface which accepts the JDK Stream. However Netty's ByteBufInputStream currently does not take reference count ownership of the underlying ByteBuf. There may be no way for existing classes which only accept the InputStream interface to communicate when they are done with the stream, other than calling close(). This means that when the stream is closed it may be appropriate to release the underlying ByteBuf, as the ownership of the underlying ByteBuf resource may be transferred to the Java Stream.

Motivation:
- ByteBufInputStream.close() supports taking reference count ownership of the underyling ByteBuf

Result:
ByteBufInputStream can assume reference count ownership so the underlying ByteBuf can be cleaned up when the stream is closed.
2016-11-14 16:29:55 -08:00
Norman Maurer
e3cb9935c0 Take memory overhead of ChannelOutboundBuffer / PendingWriteQueue into account
Motivation:

To guard against the case that a user will enqueue a lot of empty or small buffers and so raise an OOME we need to also take the overhead of the ChannelOutboundBuffer / PendingWriteQueue into account when detect if a Channel is writable or not. This is related to #5856.

Modifications:

When calculate the memory for an message that is enqueued also add some extra bytes depending on the implementation.

Result:

Better guard against OOME.
2016-11-03 15:54:00 +01:00
Roger Kapsi
b2379e62f4 Allow customization of LocalChannel instances that are being created by LocalServerChannel.
Motivation

It's possible to extend LocalChannel as well as LocalServerChannel but the LocalServerChannel's serve(peer) method is hardcoded to create only instances of LocalChannel.

Modifications

Add a protected factory method that returns by default new LocalChannel(...) but users may override it to customize it.

Result

It's possible to customize the LocalChannel instance on either end of the virtual connection.
2016-10-30 08:32:49 +01:00
Norman Maurer
5f533b7358 [maven-release-plugin] prepare for next development iteration 2016-10-14 13:20:41 +02:00
Norman Maurer
35fb0babe2 [maven-release-plugin] prepare release netty-4.1.6.Final 2016-10-14 12:47:19 +02:00
Scott Mitchell
eb1dfb8f5e SingleThreadEventLoopTest failures
Motivation:
Some unit tests in SingleThreadEventLoopTest rely upon Thread.sleep for sequencing events between threads. This can be unreliable and result in spurious test failures if thread scheduling does not occur in a fair predictable manner.

Modifications:
- Reduce the reliance on Thread.sleep in SingleThreadEventLoopTest

Result:
Fixes https://github.com/netty/netty/issues/5851
2016-10-11 09:09:53 +02:00
Norman Maurer
6c8cd023bf [#5770] Use heapbuffers by default when using LocalChannel and LocalServerChannel.
Motivation:

The local transport is used to communicate in the same JVM so we should use heap buffers.

Modifications:

Use heapbuffers by default if not requested otherwise.

Result:

No allocating of direct buffers by default when using local transport
2016-10-10 11:16:17 +02:00
Norman Maurer
e102a008b6 [#5893] Ensure we not close NioDatagramChannel when SocketException is received.
Motivation:

When using java.nio.DatagramChannel we should not close the channel when a SocketException was thrown as we can still use the channel.

Modifications:

Not close the Channel when SocketException is thrown

Result:

More robust and correct handling of exceptions when using NioDatagramChannel.
2016-10-10 10:24:28 +02:00
Norman Maurer
a09e56850e [#5882] Ensure we even process tasks if processing of ready channels throws an Exception in event loop.
Motivation:

If an exception is thrown while processing the ready channels in the EventLoop we should still run all tasks as this may allow to recover. For example a OutOfMemoryError may be thrown and runAllTasks() will free up memory again. Beside this we should also ensure we always allow to shutdown even if an exception was thrown.

Modifications:

- Call runAllTasks() in a finally block
- Ensure shutdown is always handles.

Result:

More robust EventLoop implementations for NIO and Epoll.
2016-10-10 07:49:57 +02:00
Norman Maurer
3cf7ccbd3c Process OP_WRITE before OP_READ to free memory faster
Motivation:

We should better first process OP_WRITE before OP_READ as this may allow us to free memory in a faster fashion for previous queued writes.

Modifications:

Process OP_WRITE before OP_READ

Result:

Free memory faster for queued writes.
2016-10-10 07:42:39 +02:00
radai-rosenblatt
15ac6c4a1f Clean-up unused imports
Motivation:

the build doesnt seem to enforce this, so they piled up

Modifications:

removed unused import lines

Result:

less unused imports

Signed-off-by: radai-rosenblatt <radai.rosenblatt@gmail.com>
2016-09-30 09:08:50 +02:00
Roger Kapsi
149916d052 Adding ability omit the implicit #flush() call in EmbeddedChannel#writeOutbound() and
the implicit #fireChannelReadComplete() in EmbeddedChannel#writeInbound().

Motivation

We use EmbeddedChannels to implement a ProxyChannel of some sorts that shovels
messages between a source and a destination Channel. The latter are real network
channels (such as Epoll) and they may or may not be managed in a ChannelPool. We
could fuse both ends directly together but the EmbeddedChannel provides a nice
disposable section of a ChannelPipeline that can be used to instrument the messages
that are passing through the proxy portion.

The ideal flow looks abount like this:

source#channelRead() -> proxy#writeOutbound() -> destination#write()
source#channelReadComplete() -> proxy#flushOutbound() -> destination#flush()

destination#channelRead() -> proxy#writeInbound() -> source#write()
destination#channelReadComplete() -> proxy#flushInbound() -> source#flush()

The problem is that #writeOutbound() and #writeInbound() emit surplus #flush()
and #fireChannelReadComplete() events which in turn yield to surplus #flush()
calls on both ends of the pipeline.

Modifications

Introduce a new set of write methods that reain the same sematics as the #write()
method and #flushOutbound() and #flushInbound().

Result

It's possible to implement the above ideal flow.

Fix for EmbeddedChannel#ensureOpen() and Unit Tests for it

Some PR stuff.
2016-09-24 12:33:04 -07:00
ChuntaoLu
9008e72c2b Fix javadoc
Removes unmatched brace
2016-09-12 06:43:10 -07:00
Norman Maurer
6bbf32134a Log more details if notification of promise fails in PromiseNotifier and AbstractChannelHandlerContext
Motivation:

To make it easier to debug why notification of a promise failed we should log extra info and make it consistent.

Modifications:

- Create a new PromiseNotificationUtil that has static methods that can be used to try notify a promise and log.
- Reuse this in AbstractChannelHandlerContext, ChannelOutboundBuffer and PromiseNotifier

Result:

Easier to debug why a promise could not be notified.
2016-09-07 06:55:38 +02:00
Norman Maurer
dfa3bbbf00 Add support for Client Subnet in DNS Queries (RFC7871)
Motivation:

RFC7871 defines an extension which allows to request responses for a given subset.

Modifications:

- Add DnsOptPseudoRrRecord which can act as base class for extensions based on EDNS(0) as defined in RFC6891
- Add DnsOptEcsRecord to support the Client Subnet in DNS Queries extension
- Add tests

Result:

Client Subnet in DNS Queries extension is now supported.
2016-09-06 07:16:57 +02:00
buchgr
4af6855695 Remove @Deprecated for primitive WriteWaterMark getters and setters
Motivation:

For use cases that demand frequent updates of the write watermarks, an
API that requires immutable WriteWaterMark objects is not ideal, as it
implies a lot of object allocation.

For example, the HTTP/2 child channel API uses write watermarks for outbound
flow control and updates the write watermarks on every DATA frame write.

Modifications:

Remote @Deprecated tag from primitive getters and setters, however the corresponding
channel options remain deprecated.

Result:

Primitive getters and setters for write watermarks are no longer marked @Deprecated.
2016-09-05 10:26:05 +02:00
Norman Maurer
30fe2e868f Call finishConnect() before try to call read(...) / write(...) when using NIO
Motivation:

The JDK implementation of SocketChannel has an internal state that is tracked for its operations. Because of this we need to ensure we call finishConnect() before try to call read(...) / write(...) as otherwise it may produce a NotYetConnectedException.

Modifications:

First process OP_CONNECT flag.

Result:

No more possibility of NotYetConnectedException because OP_CONNECT is handled not early enough when processing interestedOps for a Channel.
2016-09-01 08:55:02 +02:00
Norman Maurer
54b1a100f4 [maven-release-plugin] prepare for next development iteration 2016-08-26 10:06:32 +02:00
Norman Maurer
6fd8bb8c63 [#5763] DefaultEventLoopGroup doesn't expose ctor variant that accepts custom Executor
Motivation:

The DefaultEventLoopGroup class extends MultithreadEventExecutorGroup but doesn't expose the ctor variants that accept a custom Executor like NioEventLoopGroup and EpollEventLoopGroup do.

Modifications:

Add missing constructor.

Result:

Be able to use custom Executor with DefaultEventLoopGroup.
2016-09-01 08:20:05 +02:00
Jason Tedor
00c0664ef8 Avoid inaccessible object exception replacing set
Motivation:

When attempting to set the selectedKeys fields on the selector
implementation, JDK 9 can throw an inaccessible object exception.

Modications:

Catch and log this exception as an possible course of action if the
sun.nio.ch package is not exported from java.base.

Result:

The selector replacement will fail gracefully as an expected course of
action if the sun.nio.ch package is not exported from java.base.
2016-08-31 13:59:48 +02:00
Norman Maurer
1208b90f57 [maven-release-plugin] prepare release netty-4.1.5.Final 2016-08-26 04:59:35 +02:00
Norman Maurer
d3cb95ef00 Make NIO and EPOLL transport connect errors more consistent with the JDK
Motivation:

The NIO transport used an IllegalStateException if a user tried to issue another connect(...) while the connect was still in process. For this case the JDK specified a ConnectPendingException which we should use. The same issues exists in the EPOLL transport. Beside this the EPOLL transport also does not throw the right exceptions for ENETUNREACH and EISCONN errno codes.

Modifications:

- Replace IllegalStateException with ConnectPendingException in NIO and EPOLL transport
- throw correct exceptions for ENETUNREACH and EISCONN in EPOLL transport
- Add test case

Result:

More correct error handling for connect attempts when using NIO and EPOLL transport
2016-08-27 20:57:36 +02:00
buchgr
4accd300e5 Fix write watermarks comparison to use less than and greater than.
Motivation:

The API documentation in ChannelConfig states that a a channel is writable,
if the number of pending bytes is below the low watermark and a
channel is not writable, if the number of pending bytes exceeds the high
watermark.

Therefore, we should use < operators instead of <= as well as > instead of >=.

Using <= and >= is also problematic, if the low watermark is equal to the high watermark,
as then a channel could be both writable and unwritable with the same number of pending
bytes (depending on whether remove() or addMessage() is called first).

The use of <= and >= was introduced in PR https://github.com/netty/netty/pull/3036, but
I don't understand why, as there doesn't seem to have been any discussion around that.

Modifications:

Use < and > operators instead of <= and >=.

Result:

High and low watermarks are treated as stated in the API docs.
2016-08-24 15:58:02 +02:00
Norman Maurer
5e148d5670 [#5639] Ensure fireChannelActive() is also called if Channel is closed in connect promise.
Motivation:

We need to ensure we also call fireChannelActive() if the Channel is directly closed in a ChannelFutureListener that is belongs to the promise for the connect. Otherwise we will see missing active events.

Modifications:

Ensure we always call fireChannelActive() if the Channel was active.

Result:

No missing events.
2016-08-24 08:47:49 +02:00
Norman Maurer
43626ac6ea Use NIO methods when using Java7+ in the NIO transport
Motivation:

We use often javachannel().socket().* in NIO as these methods exists in java6. The problem is that these will throw often very general Exceptions (Like SocketException) while it is more expected to throw the Exceptions listed in the nio interfaces. When possible we should use the new methods available in java7+ which throw the correct exceptions.

Modifications:

Check for java version and depending on it using the socket or the javachannel.

Result:

Throw expected Exceptions.
2016-08-24 07:36:14 +02:00
Norman Maurer
8ce7e73e78 Prevent extra peformance hit by fillInStackTrace() when create a new annotated connect exception.
Motivation:

To make it easier to debug connect exceptions we create new exceptions which also contain the remote address. For this we basically created a new instance and call setStackTrace(...). When doing this we pay an extra penality because it calls fillInStackTrace() when calling the super constructor.

Modifications:

Create special sub-classes of Exceptions that override the fillInStackTrace() method and so eliminate the overhead.

Result:

Less overhead when "annotate" connect exceptions.
2016-08-24 07:34:43 +02:00
buchgr
8d1e46ffd1 Remove reference to 5.0 release.
Motivation:

Comments stating that AUTO_CLOSE will be removed in Netty 5.0 are wrong,
as there is no Netty 5.0.

Modifications:

Removed comment.

Result:

No more references to Netty 5.0
2016-08-23 09:46:57 +02:00
Norman Maurer
6d70f4b38a Guard against re-entrance in PendingWriteQueue.removeAndWriteAll()
Motivation:

PendingWriteQueue should guard against re-entrant writes once removeAndWriteAll() is run.

Modifications:

Continue writing until queue is empty.

Result:

Correctly guard against re-entrance.
2016-08-18 07:13:19 +02:00
tbcs
95ee705e17 fix typo in javadoc 2016-08-12 20:42:37 +02:00
Jason Tedor
a6dfd08812 Mark initialization of selector as privileged
Motivation:

Instrumenting the NIO selector implementation requires special
permissions. Yet, the code for performing this instrumentation is
executed in a manner that would require all code leading up to the
initialization to have the requisite permissions. In a restrictive
environment (e.g., under a security policy that only grants the
requisite permissions the Netty transport jar but not to application
code triggering the Netty initialization), then instrumeting the
selector will not succeed even if the security policy would otherwise
permit it.

Modifications:

This commit marks the necessary blocks as privileged. This enables
access to the necessary resources for instrumenting the selector. The
idea is that we are saying the Netty code is trusted, and as long as the
Netty code has been granted the necessary permissions, then we will
allow the caller access to these resources even though the caller itself
might not have the requisite permissions.

Result:

The selector can be instrumented in a restrictive security environment.
2016-08-05 19:01:57 +02:00
Jason Tedor
32629078a2 Mark setting of sun.nio.ch.bugLevel as privileged
Motivation:

Writing to a system property requires permissions. Yet the code for
setting sun.nio.ch.bugLevel is not marked as privileged. In a
restrictive environment (e.g., under a security policy that only grants
the requisite permissions the Netty transport jar but not to application
code triggering the Netty initialization), writing to this system
property will not succeed even if the security policy would otherwise
permit it.

Modifications:

This commt marks the necessary code block as privileged. This enables
writing to this system property. The idea is that we are saying the
Netty code is trusted, and as long as the Netty code has been granted
the necessary permissions, then we will allow the caller access to these
resources even though the caller itself might not have the requisite
permissions.

Result:

The system property sun.nio.ch.bugLevel can be written to in a
restrictive security environment.
2016-08-05 18:58:34 +02:00
Norman Maurer
aa6e6ae307 [#4241] Ensure NioEventLoopGroup.shutdownGracefully(...) with no quiet period shutdown as fast as expected.
Motivation:

If the user uses 0 as quiet period we should shutdown without any delay if possible.

Modifications:

Ensure we not introduce extra delay when a shutdown quit period of 0 is used.

Result:

EventLoop shutdown as fast as expected.
2016-08-05 07:21:17 +02:00
Norman Maurer
26aa34853a Ensure correct ordering if a ChannelInitializer adds another ChannelInitializer
Motivation:

At the moment we call initChannel(...) in the channelRegistered(...) method which has the effect that if another ChannelInitializer is added within the initChannel(...) method the ordering of the added handlers is not correct and surprising. This is as the whole initChannel(...) method block is executed before the initChannel(...) block of the added ChannelInitializer is handled.

Modifications:

Call initChannel(...) from within handlerAdded(...) if the Channel is registered already. This is true in all cases for our DefaultChannelPipeline implementation. This way the ordering is always as expected. We still keep the old behaviour as well to not break code for other ChannelPipeline implementations (if someone ever wrote one).

Result:

Correct and expected ordering of ChannelHandlers.
2016-08-03 07:50:46 +02:00
Norman Maurer
f585806a74 [#5598] Ensure SslHandler not log false-positives when try to close the channel due timeout.
Motivation:

When we try to close the Channel due a timeout we need to ensure we not log if the notification of the promise fails as it may be completed in the meantime.

Modifications:

Add another constructor to ChannelPromiseNotifier and PromiseNotifier which allows to log on notification failure.

Result:

No more miss-leading logs.
2016-07-30 21:15:09 +02:00
Norman Maurer
cb7cf4491c [maven-release-plugin] prepare for next development iteration 2016-07-27 13:29:56 +02:00
Norman Maurer
9466b32d05 [maven-release-plugin] prepare release netty-4.1.4.Final 2016-07-27 13:16:59 +02:00
Norman Maurer
b97a36a10f Add test to verify that its possible to add another ChannelInitializer in the initChannel(...) method.
Motivation:

I received a report the its not possible to add another ChannelInitialiter in the initChannel(...) method, so we should add a test case for it.

Modifications:

Added testcase.

Result:

Validate that all works as expected.
2016-07-27 09:25:13 +02:00
Norman Maurer
4638df2062 [#5566] Ensure using a ChannelInitializer via ServerBootstrap.handler(...) produce correct ordering.
Motivation:

When a ChannelInitializer is used via ServerBootstrap.handler(...) the users handlers may be added after the internal ServerBootstrapAcceptor. This should not happen.

Modifications:

Delay the adding of the ServerBootstrapAcceptor until the initChannel(....) method returns.

Result:

Correct order of handlers in the ServerChannels ChannelPipeline.
2016-07-27 08:34:31 +02:00
Norman Maurer
dc6c6d956a [#5541] Ensure failing a Promise in SimpleChannelPool will not result in stack overflow.
Motivation:

We used Promise.setFailure(...) when fail a Promise in SimpleChannelPool. As this happens in multiple levels this can result in stackoverflow as setFailure(...) may throw an IllegalStateException which then again is propergated.

Modifications:

Use tryFailure(...)

Result:

No more possibility to cause a stack overflow when failing the promise.
2016-07-27 07:57:58 +02:00
Norman Maurer
b8400f9628 [#5553] SimpleChannelPool#notifyConnect() may leak Channels
Motivation:

The SimpleChannelPool#notifyConnect() method will leak Channels if the user cancelled the Promise in between.

Modifications:

Release the channel if the Promise was complete before.

Result:

No more channel leaks.
2016-07-20 20:17:19 +02:00
Jason Tedor
d262f7c189 Reduce permissions needed for process ID
Motiviation:

DefaultChannelId attempts to acquire a default process ID by determining
the process PID. However, to do this it attempts to punch through to the
system classloader, a permission that in the face of a restrictive
security manager is unlikely to be granted. Looking past this, it then
attempts to load a declared method off a reflectively loaded class,
another permission that is not likely to be granted in the face of a
restrictive security manager. However, neither of these permissions are
necessary as the punching through to the system security manager is
completely unneeded, and there is no need to load a public method as a
declared method.

Modifications:

Instead of punching through to the system classloader requiring
restricted permissions, we can just use current classloader. To address
the access declared method permission, we instead just reflectively
obtain the desired public method via Class#getMethod.

Result:

Acquiring the default process ID from the PID will succeed without
requiring the runtime permissions "getClassLoader" and
"accessDeclaredMembers".
2016-07-20 19:47:56 +02:00
Scott Mitchell
3d7ae97359 Make Epoll ChannelMetadata more consistent with NIO
Motivation:
In 4.0 AbstractNioByteChannel has a default of 16 max messages per read. However in 4.1 that constraint was applied at the NioSocketChannel which is not equivalent. In 4.1 AbstractEpollStreamChannel also did not have the default of 16 max messages per read applied.

Modifications:
- Make Nio consistent with 4.0
- Make Epoll consistent with Nio

Result:
Nio and Epoll both have consistent ChannelMetadata and are consistent with 4.0.
2016-07-18 13:26:05 +02:00
Norman Maurer
047f6aed28 [maven-release-plugin] prepare for next development iteration 2016-07-15 09:09:13 +02:00
Norman Maurer
b2adea87a0 [maven-release-plugin] prepare release netty-4.1.3.Final 2016-07-15 09:08:53 +02:00
Nitesh Kant
77770374fb Ability to run a task at the end of an eventloop iteration.
Motivation:

This change is part of the change done in PR #5395 to provide an `AUTO_FLUSH` capability.
Splitting this change will enable to try other ways of implementing `AUTO_FLUSH`.

Modifications:



Two methods:

```java
void executeAfterEventLoopIteration(Runnable task);


boolean removeAfterEventLoopIterationTask(Runnable task);
```
are added to `SingleThreadEventLoop` class for adding/removing a task to be executed at the end of current/next iteration of this `eventloop`.

In order to support the above, a few methods are added to `SingleThreadEventExecutor`

```java
protected void afterRunningAllTasks() { }
```

This is invoked after all tasks are run for this executor OR if the passed timeout value for `runAllTasks(long timeoutNanos)` is expired.

Added a queue of `tailTasks` to `SingleThreadEventLoop` to hold all tasks to be executed at the end of every iteration.


Result:



`SingleThreadEventLoop` now has the ability to execute tasks at the end of an eventloop iteration.
2016-07-12 10:22:15 +02:00
Norman Maurer
b37a41a535 Allow to get the number of bytes queued in PendingWriteQueue
Motivation:

For some use-cases it would be useful to know the number of bytes queued in the PendingWriteQueue without the need to dequeue them.

Modifications:

Add PendingWriteQueue.bytes().

Result:

Be able to get the number of bytes queued.
2016-07-11 09:05:20 +02:00
Norman Maurer
50a74e95f2 Ensure ChannelHandler.handlerAdded(...) callback is executed directly when added from ChannelFutureListener added to the registration future.
Motivation:

Commit 4c048d069d moved the logic of calling handlerAdded(...) to the channelRegistered(...) callback of the head of the DefaultChannelPipeline. Unfortunatlly this may execute the callbacks to late as a user may add handlers to the pipeline in the ChannelFutureListener attached to the registration future. This can lead to incorrect ordering.

Modifications:

Ensure we always invoke ChannelHandler.handlerAdded(...) for all handlers before the registration promise is notified.

Result:

Not possible of incorrect ordering or missed events.
2016-07-09 08:04:15 +02:00
Norman Maurer
061899f2a7 Allow to remove pinning of EventExecutor for EventExecutorGroup
Motivation:

We pinned the EventExecutor for a Channel in DefaultChannelPipeline. Which means if the user added multiple handlers with the same EventExecutorGroup to the ChannelPipeline it will use the same EventExecutor for all of these handlers. This may be unexpected and even not what the user wants. If the user want to use the same one for all of them it can be done by obtain an EventExecutor and pass the same instance to the add methods. Because of this we should allow to not pin.

Modifications:

Allow to disable pinning of EventExecutor for Channel based on EventExecutorGroup via ChannelOption.

Result:

Less confusing and more flexible usage of EventExecutorGroup when adding ChannelHandlers to the ChannelPipeline.
2016-07-08 20:09:16 +02:00
Norman Maurer
64bf167423 Fix flacky test introducd by 29fdb160f3 2016-07-07 22:53:08 +02:00