Commit Graph

206 Commits

Author SHA1 Message Date
Nick Hill
170e4deee6 Fix event loop shutdown timing fragility (#9616)
Motivation

The current event loop shutdown logic is quite fragile and in the
epoll/NIO cases relies on the default 1 second wait/select timeout that
applies when there are no scheduled tasks. Without this default timeout
the shutdown would hang indefinitely.

The timeout only takes effect in this case because queued scheduled
tasks are first cancelled in
SingleThreadEventExecutor#confirmShutdown(), but I _think_ even this
isn't robust, since the main task queue is subsequently serviced which
could result in some new scheduled task being queued with much later
deadline.

It also means shutdowns are unnecessarily delayed by up to 1 second.

Modifications

- Add/extend unit tests to expose the issue
- Adjust SingleThreadEventExecutor shutdown and confirmShutdown methods
to explicitly add no-op tasks to the taskQueue so that the subsequent
event loop iteration doesn't enter blocking wait (as looks like was
originally intended)

Results

Faster and more robust shutdown of event loops, allows removal of the
default wait timeout
2019-10-07 11:06:01 +04:00
root
92941cdcac [maven-release-plugin] prepare for next development iteration 2019-09-25 06:15:31 +00:00
root
bd907c3b3a [maven-release-plugin] prepare release netty-4.1.42.Final 2019-09-25 06:14:31 +00:00
Norman Maurer
5e69a13c21
Cleanup JNI code to always correctly free memory when loading fails and also correctly respect out of memory in all cases (#9596)
Motivation:

At the moment we not consistently (and also not correctly) free allocated native memory in all cases during loading the JNI library. This can lead to native memory leaks in the unlikely case of failure while trying to load the library.

Beside this we also not always correctly handle the case when a new java object can not be created in native code because of out of memory.

Modification:

- Copy some macros from netty-tcnative to be able to handle errors in a more easy fashion
- Correctly account for New* functions to return NULL
- Share code

Result:

More robust and clean JNI code
2019-09-24 07:18:35 +02:00
Norman Maurer
dc4de7fbb4
We need to use NewGloblRef when caching jclass instances (#9595)
Motivation:

It is not safe to cache a jclass without obtaining a global reference via NewGlobalRef.

Modifications:

Correctly use NewGlobalRef(...) before caching

Result:

Correctly cache jclass instance
2019-09-23 12:47:58 +02:00
Joe Ellis
aebe2064d5 Allow domain sockets to configure SO_SNDBUF and SO_RCVBUF (#9584)
Motivation:

Running tests with a `KQueueDomainSocketChannel` showed worse performance than an `NioSocketChannel`. It turns out that the default send buffer size for Nio sockets is 64k while for KQueue sockets it's 8k. I verified that manually setting the socket's send buffer size improved perf to expected levels.

Modification:

Plumb the `SO_SNDBUF` and `SO_RCVBUF` options into the `*DomainSocketChannelConfig`.

Result:

Can now configure send and receive buffer sizes for domain sockets.
2019-09-20 22:28:53 +02:00
Norman Maurer
3ad037470e
Correctly reset cached local and remote address when disconnect() is called (#9545)
Motivation:

We should correctly reset the cached local and remote address when a Channel.disconnect() is called and the channel has a notion of disconnect vs close (for example DatagramChannel implementations).

Modifications:

- Correctly reset cached kicak abd remote address
- Update testcase to cover it and so ensure all transports work in a consistent way

Result:

Correctly handle disconnect()
2019-09-19 08:51:10 +02:00
root
01d805bb76 [maven-release-plugin] prepare for next development iteration 2019-09-12 16:09:55 +00:00
root
7cf69022d4 [maven-release-plugin] prepare release netty-4.1.41.Final 2019-09-12 16:09:00 +00:00
root
aef47bec7f [maven-release-plugin] prepare for next development iteration 2019-09-12 05:38:11 +00:00
root
267e5da481 [maven-release-plugin] prepare release netty-4.1.40.Final 2019-09-12 05:37:30 +00:00
Norman Maurer
7a547aab65
Correctly handle IPV6-mapped-IPV4 addresses in native code when receiving datagrams (#9560)
Motivation:

291f80733a introduced a change to use a byte[] to construct the InetAddress when receiving datagram messages to reduce the overhead. Unfortunally it introduced a regression when handling IPv6-mapped-IPv4 addresses and so produced an IndexOutOfBoundsException when trying to fill the byte[] in native code.

Modifications:

- Correctly use the offset on the pointer of the address.
- Add testcase
- Make tests more robust and include more details when the test fails

Result:

No more IndexOutOfBoundsException
2019-09-11 20:30:28 +02:00
Norman Maurer
394a1b3485
Add support for recvmmsg when using epoll transport (#9509)
Motivation:

When using datagram sockets which need to handle a lot of packets it makes sense to use recvmmsg to be able to read multiple datagram packets with one syscall.

Modifications:

- Add support for recvmmsg on linux
- Add new EpollChannelOption.MAX_DATAGRAM_PACKET_SIZE
- Add tests

Result:

Fixes https://github.com/netty/netty/issues/8446.
2019-09-03 08:40:17 +02:00
Norman Maurer
d9d39f1394
Include c source files in source jar (#9497)
Motivation:

We should not only include the java source files but also the c source file in our source jars.

Modifications:

Add files from src/main/c as well

Result:

Fixes https://github.com/netty/netty/issues/9494
2019-08-23 09:30:27 +02:00
root
d45a4ce01b [maven-release-plugin] prepare for next development iteration 2019-08-13 17:16:42 +00:00
root
88c2a4cab5 [maven-release-plugin] prepare release netty-4.1.39.Final 2019-08-13 17:15:20 +00:00
Nico Kruber
3f0f322562 Fix native-build/target/lib wanted but build in native-build/target/lib64 (#9410)
Motivation:

On openSUSE (probably more), 64 bit builds use lib64, e.g. /usr/lib64, and
configure picks this up and builds the native library in
native-build/target/lib64 where maven is not looking.

Modifications:

Explicitly specify --libdir=${project.build.directory}/native-build/target/lib
during configuration.

Result:

Maven uses the correct lib directory.
2019-08-07 09:56:28 +02:00
root
718b7626e6 [maven-release-plugin] prepare for next development iteration 2019-07-24 09:05:57 +00:00
root
465c900c04 [maven-release-plugin] prepare release netty-4.1.38.Final 2019-07-24 09:05:23 +00:00
jingene
c0f9364870 Change the netty.io homepage scheme(http -> https) (#9344)
Motivation:

Netty homepage(netty.io) serves both "http" and "https".
It's recommended to use https than http.
Modification:

I changed from "http://netty.io" to "https://netty.io"
Result:

No effects.
2019-07-09 21:09:42 +02:00
root
5b58b8e6b5 [maven-release-plugin] prepare for next development iteration 2019-06-28 05:57:21 +00:00
root
35e0843376 [maven-release-plugin] prepare release netty-4.1.37.Final 2019-06-28 05:56:28 +00:00
jimin
856f1185e1 All override methods must be added @override (#9285)
Motivation:

Some methods that either override others or are implemented as part of implementation an interface did miss the `@Override` annotation

Modifications:

Add missing `@Override`s

Result:

Code cleanup
2019-06-27 13:51:26 +02:00
jimin
9621a5b981 remove unused imports (#9287)
Motivation:

Some imports are not used

Modification:

remove unused imports

Result:

Code cleanup
2019-06-26 21:08:31 +02:00
Norman Maurer
517a93d87d Make EventLoopTaskQueueFactory a top-level interface
Motivation:

c9aaa93d83 added the ability to specify an EventLoopTaskQueueFactory but did place it under MultithreadEventLoopGroup while not really belongs there.

Modifications:

Make EventLoopTaskQueueFactory a top-level interface

Result:

More logical code layout.
2019-06-22 07:38:03 +02:00
Norman Maurer
c9aaa93d83
Allow to specify a EventLoopTaskQueueFactory for various EventLoopGroup implementations (#9247)
Motivation:

Sometimes it is desirable to be able to use a different Queue implementation for the EventLoop of a Channel. This is currently not possible without resort to reflection.

Modifications:

- Add a new constructor to Nio|Epoll|KQueueEventLoopGroup which allows to specify a factory which is used to create the task queue. This was the user can override the default implementation.
- Add test

Result:

Be able to change Queue that is used for the EventLoop.
2019-06-21 09:05:19 +02:00
EliyahuStern
6f602cbd14 Resolve the pid field in PeerCredentials of KQueueDomainSocketChannels. (#9219)
Motivation:

This resolves a TODO from the initial transport-native-kqueue implementation, supplying the user with the pid of the local peer client/server process.

Modification:

Inside netty_kqueue_bsdsocket_getPeerCredentials, Call getsockopt with LOCAL_PEERPID and pass it to PeerCredentials constructor.
Add a test case in KQueueSocketTest.

Result:

PeerCredentials now have pid field set. Fixes https://github.com/netty/netty/issues/9213
2019-06-04 05:15:42 -07:00
Julien Viet
e348bd9217 KQueueEventLoop | EpollEventLoop may incorrectly update registration when FD is reused.
Motivation:

The current KQueueEventLoop implementation does not process concurrent domain socket channel registration/unregistration in the order they actual
happen since unregistration are delated by an event loop task scheduling. When a domain socket is closed, it's file descriptor might be reused
quickly and therefore trigger a new channel registration using the same descriptor.

Consequently the KQueueEventLoop#add(AbstractKQueueChannel) method will overwrite the current inactive channels having the same descriptor
and the delayed KQueueEventLoop#remove(AbstractKQueueChannel) will remove the active channel that replaced the inactive one.

As active channels are registered, events for this file descriptor won't be processed anymore and the channels will never be closed.

The same problem can also happen in EpollEventLoop. Beside this we also may never remove the AbstractEpollChannel from the internal map
when it is unregistered which will prevent it from be GC'ed

Modifications:

- Change logic of native KQueue and Epoll implementations to ensure we correctly handle the case of FD reuse
- Only try to update kevent / epoll if the Channel is still open (as otherwise it will be handled by kqueue / epoll itself)
- Correctly remove AbstractEpollChannel from internal map in all cases
- Make implementation of closeAll() consistent for Epoll and KQueueEventLoop

Result:

KQueue and Epoll native transports correctly handle FD reuse

Co-authored-by: Norman Maurer <norman_maurer@apple.com>
2019-05-22 09:23:09 +02:00
Norman Maurer
f17bfd0f64
Only use static Exception instances when we can ensure addSuppressed … (#9152)
Motivation:

OOME is occurred by increasing suppressedExceptions because other libraries call Throwable#addSuppressed. As we have no control over what other libraries do we need to ensure this can not lead to OOME.

Modifications:

Only use static instances of the Exceptions if we can either dissable addSuppressed or we run on java6.

Result:

Not possible to OOME because of addSuppressed. Fixes https://github.com/netty/netty/issues/9151.
2019-05-17 22:23:02 +02:00
Norman Maurer
71c184076c Revert "KQueueEventLoop won't unregister active channels reusing a file descriptor (#9114)"
This reverts commit 909a3d942e.
2019-05-07 16:44:41 +02:00
Julien Viet
909a3d942e KQueueEventLoop won't unregister active channels reusing a file descriptor (#9114)
Motivation:

The current KQueueEventLoop implementation does not process concurrent domain socket channel registration/unregistration in the order they actual
happen since unregistration are delated by an event loop task scheduling. When a domain socket is closed, it's file descriptor might be reused
quickly and therefore trigger a new channel registration using the same descriptor.

Consequently the KQueueEventLoop#add(AbstractKQueueChannel) method will overwrite the current inactive channels having the same descriptor
and the delayed KQueueEventLoop#remove(AbstractKQueueChannel) will remove the active channel that replaced the inactive one.

As active channels are registered, events for this file descriptor won't be processed anymore and the channels will never be closed.

Modifications:

Change the logic of KQueueEventLoop#remove(AbstractKQueueChannel) channels so it will check channels equality prior removal.

Result:

KQueueEventLoop won't remove anymore active channels reusing a file descriptor.
2019-05-07 10:19:42 +02:00
root
ba06eafa1c [maven-release-plugin] prepare for next development iteration 2019-04-30 16:42:29 +00:00
root
49a451101c [maven-release-plugin] prepare release netty-4.1.36.Final 2019-04-30 16:41:28 +00:00
root
baab215f66 [maven-release-plugin] prepare for next development iteration 2019-04-17 07:26:24 +00:00
root
dfe657e2d4 [maven-release-plugin] prepare release netty-4.1.35.Final 2019-04-17 07:25:40 +00:00
Nick Hill
9ed41db1d7 Have (Epoll|KQueue)RecvByteAllocatorHandle extend DelegatingHandle (#9060)
Motivation

These implementations delegate most of their methods to an existing Handle and previously extended RecvByteBufAllocator.DelegatingHandle. This was reverted in #6322 with the introduction of ExtendedHandle but it's not clear to me why it needed to be - the code looks a lot cleaner.

Modifications

Have (Epoll|KQueue)RecvByteAllocatorHandle extend DelegatingHandle again, while still implementing ExtendedHandle.

Result

Less code.
2019-04-16 09:14:09 +02:00
Norman Maurer
778ff2057e
Add IPv6 multicast test to testsuite (#9037)
Motivation:

We currently only cover ipv4 multicast in the testsuite but we should also have tests for ipv6.

Modifications:

- Add test for ipv6
- Ensure we only try to run multicast test for ipv4 / ipv6 if the loopback interface supports it.

Result:

Better test coverage
2019-04-12 12:29:08 +02:00
Scott Mitchell
ac023da16d Correctly handle overflow in Native.kevent(...) when EINTR is detected (#9024)
Motivation:
When kevent(...) returns with EINTR we do not correctly decrement the timespec
structure contents to account for the time duration. This may lead to negative
values for tv_nsec which will result in an EINVAL and raise an IOException to
the event loop selection loop.

Modifications:
Correctly calculate new timeoutTs when EINTR is detected

Result:
Fixes #9013.
2019-04-10 11:04:13 +02:00
Vladimir Kostyukov
0a0da67f43 Introduce SingleThreadEventLoop.registeredChannels (#8428)
Motivation:

Systems depending on Netty may benefit (telemetry, alternative even loop scheduling algorithms) from knowing the number of channels assigned to each EventLoop.

Modification:

Expose the number of channels registered in the EventLoop via SingleThreadEventLoop.registeredChannels.

Result:

Fixes #8276.
2019-03-28 11:33:12 +00:00
Lunfu Zhong
e7b3195570 Support ALLOW_HALF_CLOSURE channel option on Unix domain socket. (#8932)
Motivation:

Since DomainSocketChannel is a DuplexChannel,  which be able to shutdown input or output individually on demands, but ALLOW_HALF_CLOSURE channel option has not been supported yet.

I thought this could be a missing feature of Unix domain socket, so here the PR for it.

Modifications:

1. Added allHalfClosure property both in  EpollDomainSocketChannelConfig and KQueueDomainSocketChannelConfig,
2. Enabled isAllowHalfClosure method of native channel to support domain channel config,
3. Created EpollDomainSocketShutdownOutputByPeerTest and KQueueDomainSocketShutdownOutputByPeerTest to verify the change.

Result:

ALLOW_HALF_CLOSURE channel option can be set with DomainSocketChannel, and no more warning of Unknown channel option 'ALLOW_HALF_CLOSURE'.
2019-03-19 11:24:07 +01:00
Norman Maurer
5eb91d9ca1
Remove --add-opens=java.base/java.nio=ALL-UNNAMED when running tests as it is not needed anymore since a long time (#8934)
Motivation:

At some point we needed --add-opens=java.base/java.nio=ALL-UNNAMED to run our native tests but this is not true anymore.

Modifications:

Remove --add-opens=java.base/java.nio=ALL-UNNAMED when running native tests.

Result:

Remove obsolate jvm arg.
2019-03-13 08:25:10 +01:00
root
92b19cfedd [maven-release-plugin] prepare for next development iteration 2019-03-08 08:55:45 +00:00
root
ff7a9fa091 [maven-release-plugin] prepare release netty-4.1.34.Final 2019-03-08 08:51:34 +00:00
Norman Maurer
81e43d5088
DefaultFileRegion.transferTo with invalid count may cause busy-spin (#8885)
Motivation:

`DefaultFileRegion.transferTo` will return 0 all the time when we request more data then the actual file size. This may result in a busy spin while processing the fileregion during writes.

Modifications:

- If we wrote 0 bytes check if the underlying file size is smaller then the requested count and if so throw an IOException
- Add DefaultFileRegionTest
- Add a test to the testsuite

Result:

Fixes https://github.com/netty/netty/issues/8868.
2019-02-26 11:08:09 +01:00
Norman Maurer
d02b51965f
Don't deregister Channel as part of closing it when using native kqueue transport (#8881)
Motivation:

In https://github.com/netty/netty/pull/8665 we changed how we handle the registration of Channels to KQueue but missed to removed some code which would deregister the Channel before it actual closed the underlying socket. This could lead to have events triggered still while not have a mapping to the Channel anymore.

Modifications:

Remove deregister call during socket closure.

Result:

Fixes https://github.com/netty/netty/issues/8849.
2019-02-25 08:55:55 +01:00
Norman Maurer
fa6a8cb09c
Support using an Executor to offload blocking / long-running tasks wh… (#8847)
Motivation:

The SSLEngine does provide a way to signal to the caller that it may need to execute a blocking / long-running task which then can be offloaded to an Executor to ensure the I/O thread is not blocked. Currently how we handle this in SslHandler is not really optimal as while we offload to the Executor we still block the I/O Thread.

Modifications:

- Correctly support offloading the task to the Executor while suspending processing of SSL in the I/O Thread
- Add new methods to SslContext to specify the Executor when creating a SslHandler
- Remove @deprecated annotations from SslHandler constructor that takes an Executor
- Adjust tests to also run with the Executor to ensure all works as expected.

Result:

Be able to offload long running tasks to an Executor when using SslHandler. Partly fixes https://github.com/netty/netty/issues/7862 and https://github.com/netty/netty/issues/7020.
2019-02-11 09:47:44 +01:00
Norman Maurer
c6a90d90a6
Add more tests to KQueue and Epoll testsuites. (#8851)
Motivation:

We missed to extend a few tests from the testsuite and so also run these with our native KQueue and Epoll transport.

Modifications:

Extend tests and so run these for our native transports as well.

Result:

More tests.
2019-02-08 20:08:34 +01:00
Norman Maurer
7bba4f49cf
Reduce GC produced by native DatagramChannel implementations when in connected mode. (#8806)
Motivation:

In the native code EpollDatagramChannel / KQueueDatagramChannel creates a DatagramSocketAddress object for each received UDP datagram even when in connected mode as it uses the recvfrom(...) / recvmsg(...)  method. Creating these is quite heavy in terms of allocations as internally, char[], String, Inet4Address, InetAddressHolder, InetSocketAddressHolder, InetAddress[], byte[] objects are getting generated when constructing the object. When in connected mode we can just use regular read(...) calls which do not need to allocate all of these.

Modifications:

- When in connected mode use read(...) and NOT recvfrom(..) / readmsg(...) to reduce allocations when possible.
- Adjust tests to ensure read works as expected when in connected mode.

Result:

Less allocations and GC when using native datagram channels in connected mode. Fixes https://github.com/netty/netty/issues/8770.
2019-02-01 10:29:36 +01:00
田欧
a33200ca38 use checkPositive/checkPositiveOrZero (#8803)
Motivation:

We have a utility method to check for > 0 and >0 arguments. We should use it.

Modification:

use checkPositive/checkPositiveOrZero instead of if statement.

Result:

Re-use utility method.
2019-01-31 09:07:14 +01:00
Norman Maurer
cd3254df88
Update to new checkstyle plugin (#8777) (#8780)
Motivation:

We need to update to a new checkstyle plugin to allow the usage of lambdas.

Modifications:

- Update to new plugin version.
- Fix checkstyle problems.

Result:

Be able to use checkstyle plugin which supports new Java syntax.
2019-01-25 11:58:42 +01:00
root
cf03ed0478 [maven-release-plugin] prepare for next development iteration 2019-01-21 12:26:44 +00:00
root
37484635cb [maven-release-plugin] prepare release netty-4.1.33.Final 2019-01-21 12:26:12 +00:00
kashike
6fdd7fcddb Fix minor spelling issues in javadocs (#8701)
Motivation:

Javadocs contained some spelling errors, we should fix these.

Modification:

Fix spelling

Result:

Javadoc cleanup.
2019-01-14 07:24:34 +01:00
Norman Maurer
d77bdeaa7d
Fix ClassCastException and native crash when using kqueue transport. (#8665)
Motivation:

How we did the mapping from native code to AbstractKQueueChannel was not safe and could lead to heap corruption. This then sometimes produced ClassCastExceptions or could also lead to crashes. This happened sometimes when running the testsuite.

Modifications:

Use a Map for the mapping (just as we do in the native epoll transport).

Result:

No more heap corruption / crashes.
2018-12-19 12:13:56 +01:00
Feri73
5df235c083 Correcting Maven Dependencies (#8622)
Motivation:

Most of the maven modules do not explicitly declare their
dependencies and rely on transitivity, which is not always correct.

Modifications:

For all maven modules, add all of their dependencies to pom.xml

Result:

All of the (essentially non-transitive) depepdencies of the modules are explicitly declared in pom.xml
2018-12-06 09:01:14 +01:00
root
8eb313072e [maven-release-plugin] prepare for next development iteration 2018-11-29 11:15:09 +00:00
root
afcb4a37d3 [maven-release-plugin] prepare release netty-4.1.32.Final 2018-11-29 11:14:20 +00:00
root
3e7ddb36c7 [maven-release-plugin] prepare for next development iteration 2018-10-29 15:38:51 +00:00
root
9e50739601 [maven-release-plugin] prepare release netty-4.1.31.Final 2018-10-29 15:37:47 +00:00
Norman Maurer
adb4ce1f31
Use MACOSX_DEPLOYMENT_TARGET=10.6 when compile native transport on MacOS (#8379)
Motivation:

MACOSX_DEPLOYMENT_TARGET=10.6 needs to be used as everything before is not supported in 10.14 anymore. 10.6 was released 2009 so this should be a safe thing to do.

Modifications:

Use MACOSX_DEPLOYMENT_TARGET=10.6

Result:

Be able to compile on MacOS 10.14
2018-10-13 04:25:56 +02:00
Matteo Merli
3a96e7373b Added option to do busy-wait on epoll (#8267)
Motivation:

Add an option (through a SelectStrategy return code) to have the Netty event loop thread to do busy-wait on the epoll.

The reason for this change is to avoid the context switch cost that comes when the event loop thread is blocked on the epoll_wait() call.

On average, the context switch has a penalty of ~13usec.

This benefits both:

The latency when reading from a socket
Scheduling tasks to be executed on the event loop thread.
The tradeoff, when enabling this feature, is that the event loop thread will be using 100% cpu, even when inactive.

Modification:

Added SelectStrategy option to return BUSY_WAIT
Epoll loop will do a epoll_wait() with no timeout
Use pause instruction to hint to processor that we're in a busy loop
Result:

When enabled, minimizes impact of context switch in the critical path
2018-09-28 22:52:00 +02:00
Roger
6138541033 Avoid repeating the same field and hiding it (#8335)
Motivation

The EpollChannelConfig (same for KQueues) and its subclasses repeatetly declare their own channel field which leads to a 3x repetition for each config instance. Given the fields are protected or package-private it's exposing the code code to "field hiding" bugs.

Modifications

Use the the existing protected channel field from the DefaultChannelConfig class and simply cast it when needed.

Result

Fixes #8331
2018-09-28 17:37:14 +02:00
root
2d7cb47edd [maven-release-plugin] prepare for next development iteration 2018-09-27 19:00:45 +00:00
root
3a9ac829d5 [maven-release-plugin] prepare release netty-4.1.30.Final 2018-09-27 18:56:12 +00:00
Norman Maurer
b73f785631
We should call the UnLoad methods when we detect an error during calling OnLoad (#8237)
Motivation:

We should ensure we call *UnLoad when we detect an error during calling *OnLoad and previous *OnLoad calls were succesfull.

Modifications:

Correctly call *UnLoad when needed.

Result:

More correct code and no leaks when an error happens during loading the native lib.
2018-08-30 06:56:42 +02:00
Norman Maurer
54f565ac67
Allow to use native transports when sun.misc.Unsafe is not present on… (#8231)
* Allow to use native transports when sun.misc.Unsafe is not present on the system

Motivation:

We should be able to use the native transports (epoll / kqueue) even when sun.misc.Unsafe is not present on the system. This is especially important as Java11 will be released soon and does not allow access to it by default.

Modifications:

- Correctly disable usage of sun.misc.Unsafe when -PnoUnsafe is used while running the build
- Correctly increment metric when UnpooledDirectByteBuf is allocated. This was uncovered once -PnoUnsafe usage was fixed.
- Implement fallbacks in all our native transport code for when sun.misc.Unsafe is not present.

Result:

Fixes https://github.com/netty/netty/issues/8229.
2018-08-29 19:36:33 +02:00
root
a580dc7585 [maven-release-plugin] prepare for next development iteration 2018-08-24 06:36:33 +00:00
root
3fc789e83f [maven-release-plugin] prepare release netty-4.1.29.Final 2018-08-24 06:36:06 +00:00
Norman Maurer
ea4c315b45
Ensure multiple shaded version of the same netty artifact can be loaded as long as the shaded prefix is different (#8207)
Motivation:

We should support to load multiple shaded versions of the same netty artifact as netty is often used in multiple dependencies.

This is related to https://github.com/netty/netty/issues/7272.

Modifications:

- Use -fvisibility=hidden when compiling and use JNIEXPORT for things we really want to have exported
- Ensure fields are declared as static so these are not exported
- Adjust testsuite-shading to use install_name_tool on MacOS to change the id of the lib. Otherwise the wrong may be used.

Result:

Be able to use multiple shaded versions of the same netty artifact.
2018-08-21 07:53:45 +02:00
Ziyan Mo
785473788f (Nio|Epoll)EventLoop.pendingTasks does not need to dispatch to the EventLoop (#8197)
Motivation:

EventLoop.pendingTasks should be (reasonably) cheap to invoke so it can be used within observability. 

Modifications:

Remove code that dispatch access to the internal taskqueue to the EventLoop when invoked as this is not needed anymore with the current MPSC queues we are using. 

See https://github.com/netty/netty/issues/8196#issuecomment-413653286.

Result:

Fixes https://github.com/netty/netty/issues/8196
2018-08-18 07:28:31 +02:00
root
fcb19cb589 [maven-release-plugin] prepare for next development iteration 2018-07-27 04:59:28 +00:00
root
ff785fbe39 [maven-release-plugin] prepare release netty-4.1.28.Final 2018-07-27 04:59:06 +00:00
root
b4dbdc2036 [maven-release-plugin] prepare for next development iteration 2018-07-11 15:37:40 +00:00
root
1c16519ac8 [maven-release-plugin] prepare release netty-4.1.27.Final 2018-07-11 15:37:21 +00:00
root
7bb9e7eafe [maven-release-plugin] prepare for next development iteration 2018-07-10 05:21:24 +00:00
root
8ca5421bd2 [maven-release-plugin] prepare release netty-4.1.26.Final 2018-07-10 05:18:13 +00:00
Scott Mitchell
12f6500a4f Epoll and Kqueue shouldn't read by default (#8024)
Motivation:
Epoll and Kqueue channels have internal state which forces
a single read operation after channel construction. This
violates the Channel#read() interface which indicates that
data shouldn't be delivered until this method is called.
The behavior is also inconsistent with the NIO transport.

Modifications:
- Epoll and Kqueue shouldn't unconditionally read upon
initialization, and instead should rely upon Channel#read()
or auto_read.

Result:
Epoll and Kqueue are more consistent with NIO.
2018-06-15 10:28:50 +02:00
Norman Maurer
d133bf06a4
Allow to schedule tasks up to Long.MAX_VALUE (#7972)
Motivation:

We should allow to schedule tasks with a delay up to Long.MAX_VALUE as we did pre 4.1.25.Final.

Modifications:

Just ensure we not overflow and put the correct max limits in place when schedule a timer. At worse we will get a wakeup to early and then schedule a new timeout.

Result:

Fixes https://github.com/netty/netty/issues/7970.
2018-05-30 11:11:42 +02:00
Norman Maurer
030318e53c
Read until all data is consumed when EOF is detected even if readPend… (#7961)
* Read until all data is consumed when EOF is detected even if readPending is false and auto-read is disabled.

Motivation:

We should better always notify the user of EOF even if the user did not request any data as otherwise we may never be notified when the remote peer closes the connection. This should be ok as the amount of extra data we may read and so fire through the pipeline is limited by SO_RECVBUF.

Modifications:

- Always drain the socket when EOF is detected.
- Add testcase

Result:

No risk for the user to be not notified of EOF.
2018-05-24 20:29:29 +02:00
Norman Maurer
64bb279f47 [maven-release-plugin] prepare for next development iteration 2018-05-14 11:11:45 +00:00
Norman Maurer
c67a3b0507 [maven-release-plugin] prepare release netty-4.1.25.Final 2018-05-14 11:11:24 +00:00
Norman Maurer
358249e5c9
Allow to disable native transport and native ssl support via system property. (#7903)
Motivation:

Sometimes it's useful to disable native transports / native ssl to debug a problem. We should allow to do so with a system property so people not need to adjust code for this.

Modifications:

Add system properties which allow to disable native transport and native ssl.

Result:

Easier to disable native code usage without code changes.
2018-05-04 14:44:44 +02:00
Norman Maurer
b47fb81799
EventLoop.schedule with big delay fails (#7402)
Motivation:

Using a very huge delay when calling schedule(...) may cause an Selector error when calling select(...) later on. We should gaurd against such a big value.

Modifications:

- Add guard against a very huge value.
- Added tests.

Result:

Fixes [#7365]
2018-04-24 11:15:20 +02:00
Norman Maurer
b75f44db9a [maven-release-plugin] prepare for next development iteration 2018-04-19 11:56:07 +00:00
Norman Maurer
04fac00c8c [maven-release-plugin] prepare release netty-4.1.24.Final 2018-04-19 11:55:47 +00:00
root
0a61f055f5 [maven-release-plugin] prepare for next development iteration 2018-04-04 10:44:46 +00:00
root
8c549bad38 [maven-release-plugin] prepare release netty-4.1.23.Final 2018-04-04 10:44:15 +00:00
Scott Mitchell
ed0668384b NIO read spin event loop spin when half closed (#7801)
Motivation:
AbstractNioByteChannel will detect that the remote end of the socket has
been closed and propagate a user event through the pipeline. However if
the user has auto read on, or calls read again, we may propagate the
same user events again. If the underlying transport continuously
notifies us that there is read activity this will happen in a spin loop
which consumes unnecessary CPU.

Modifications:
- AbstractNioByteChannel's unsafe read() should check if the input side
of the socket has been shutdown before processing the event. This is
consistent with EPOLL and KQUEUE transports.
- add unit test with @normanmaurer's help, and make transports consistent with respect to user events

Result:
No more read spin loop in NIO when the channel is half closed.
2018-03-28 20:02:57 +02:00
Norman Maurer
0a8e1aaf19 Flush task should not flush messages that were written since last flush attempt.
Motivation:

The flush task is currently using flush() which will have the affect of have the flush traverse the whole ChannelPipeline and also flush messages that were written since we gave up flushing. This is not really correct as we should only continue to flush messages that were flushed at the point in time when the flush task was submitted for execution if the user not explicit call flush() by him/herself.

Modification:

Call *Unsafe.flush0() via the flush task which will only continue flushing messages that were marked as flushed before.

Result:

More correct behaviour when the flush task is used.
2018-03-02 10:09:40 +09:00
Norman Maurer
69582c0b6c [maven-release-plugin] prepare for next development iteration 2018-02-21 12:52:33 +00:00
Norman Maurer
786f35c6c9 [maven-release-plugin] prepare release netty-4.1.22.Final 2018-02-21 12:52:19 +00:00
Scott Mitchell
d2d3e6ef0c
KQueue write filter initial state (#7738)
Motivation:
KQueue implementations current have inconsistent behavior with Epoll implementations with respect to asynchronous sockets and connecting. In the Epoll transport we attempt to connect, if the connect call does not synchornously fail/succeed we set the EPOLLOUT which will be triggered by the kernel if the connection attempt succeeds or an error occurs. The connect API provides no way to asynchronously communicate an error so the Epoll implementation fires a EPOLLOUT event and puts the connect status in getsockopt(SO_ERROR). KQueue provides the same APIs but different behavior. If the EVFILT_WRITE is not enabled and the EVFILT_READ is enabled before connect is called, and there is an error the kernel may fire the EVFILT_READ filter and provide the Connection Refused error via read(). This is even true if we set the EVFILT_WRITE filter after calling connect because connect didn't synchornously complete. After the error has been delievered via read() a call to getsockopt(SO_ERROR) will return 0 indicating there is no error. This means we cannot rely upon the KQueue based kernel to deliver connection errors via the EVFILT_WRITE filter in the same way that the linux kernel does with the EPOLLOUT flag.
ce241bd introduced a change which depends upon the behavior of the EVFILT_WRITE being set and may prematurely stop writing to the OS as a result, becaues we assume the OS will notify us when the socket is writable. However the current work around for the above described behavior is to initialize the EVFILT_WRITE to true for connection oriented protocols. This leads to prematurely exiting from the flush() which may lead to deadlock.

Modifications:
- KQueue should check when an error is obtained from read() if the connectPromise has not yet been completed, and if not complete it with a ConnectException

Result:
No more deadlock in KQueue due to asynchronous connect workaround.
2018-02-20 11:01:49 -08:00
Scott Mitchell
ce241bd11e Epoll flush/writabilityChange deadlock
Motivation:
b215794de3 recently introduced a change in behavior where writeSpinCount provided a limit for how many write operations were attempted per flush operation. However when the write quantum was meet the selector write flag was not cleared, and the channel unsafe flush0 method has an optimization which prematurely exits if the write flag is set. This may lead to no write progress being made under the following scenario:
- flush is called, but the socket can't accept all data, we set the write flag
- the selector wakes us up because the socket is writable, we write data and use the writeSpinCount quantum
- we then schedule a flush() on the EventLoop to execute later, however it the flush0 optimization prematurely exits because the write flag is still set

In this scenario the socket is still writable so the EventLoop may never notify us that the socket is writable, and therefore we may never attempt to flush data to the OS.

Modifications:
- When the writeSpinCount quantum is exceeded we should clear the selector write flag

Result:
Fixes https://github.com/netty/netty/issues/7729
2018-02-20 11:40:58 +01:00
Norman Maurer
e71fa1e7b6 [maven-release-plugin] prepare for next development iteration 2018-02-05 12:02:35 +00:00
Norman Maurer
41ebb5fcca [maven-release-plugin] prepare release netty-4.1.21.Final 2018-02-05 12:02:19 +00:00
Norman Maurer
ea58dc7ac7 [maven-release-plugin] prepare for next development iteration 2018-01-21 12:53:51 +00:00
Norman Maurer
96c7132dee [maven-release-plugin] prepare release netty-4.1.20.Final 2018-01-21 12:53:34 +00:00
Scott Mitchell
33ddb83dc1
IovArray#add return value resulted in more ByteBufs being added during iteration
Motivation:
IovArray implements MessageProcessor, and the processMessage method will continue to be called during iteration until it returns true. A recent commit b215794de3 changed the return value to only return true if any component of a CompositeByteBuf was added as a result of the method call. However this results in the iteration continuing, and potentially subsequent smaller buffers maybe added, which will result in out of order writes and generally corrupts data.

Modifications:
- IovArray#add should return false so that the MessageProcessor#processMessage will stop iterating.

Result:
Native transports which use IovArray will not corrupt data during gathering writes of CompositeByteBuf objects.
2018-01-04 08:04:32 -08:00
Norman Maurer
264a5daa41 [maven-release-plugin] prepare for next development iteration 2017-12-15 13:10:54 +00:00
Norman Maurer
0786c4c8d9 [maven-release-plugin] prepare release netty-4.1.19.Final 2017-12-15 13:09:30 +00:00