Go to file
Nick Hill a22d4ba859 Simplify EventLoop abstractions for timed scheduled tasks (#9470)
Motivation

The epoll transport was updated in #7834 to decouple setting of the
timerFd from the event loop, so that scheduling delayed tasks does not
require waking up epoll_wait. To achieve this, new overridable hooks
were added in the AbstractScheduledEventExecutor and
SingleThreadEventExecutor superclasses.

However, the minimumDelayScheduledTaskRemoved hook has no current
purpose and I can't envisage a _practical_ need for it. Removing
it would reduce complexity and avoid supporting this specific
API indefinitely. We can add something similar later if needed
but the opposite is not true.

There also isn't a _nice_ way to use the abstractions for
wakeup-avoidance optimizations in other EventLoops that don't have a
decoupled timer.

This PR replaces executeScheduledRunnable and
wakesUpForScheduledRunnable
with two new methods before/afterFutureTaskScheduled that have slightly
different semantics:
 - They only apply to additions; given the current internals there's no
practical use for removals
 - They allow per-submission wakeup decisions via a boolean return val,
which makes them easier to exploit from other existing EL impls (e.g.
NIO/KQueue)
 - They are subjectively "cleaner", taking just the deadline parameter
and not exposing Runnables
 - For current EL/queue impls, only the "after" hook is really needed,
but specialized blocking queue impls can conditionally wake on task
submission (I have one lined up)

Also included are further optimization/simplification/fixes to the
timerFd manipulation logic.

Modifications

- Remove AbstractScheduledEventExecutor#minimumDelayScheduledTaskRemoved()
and supporting methods
- Uplift NonWakeupRunnable and corresponding default wakesUpForTask()
impl from SingleThreadEventLoop to SingleThreadEventExecutor
- Change executeScheduledRunnable() to be package-private, and have a
final impl in SingleThreadEventExecutor which triggers new overridable
hooks before/afterFutureTaskScheduled()
- Remove unnecessary use of bookend tasks while draining the task queue
- Use new hooks to add simpler wake-up avoidance optimization to
NioEventLoop (primarily to demonstrate utility/simplicity)
- Reinstate removed EpollTest class

In EpollEventLoop:
 - Refactor to use only the new afterFutureTaskScheduled() hook for
updating timerFd
 - Fix setTimerFd race condition using a monitor
 - Set nextDeadlineNanos to a negative value while the EL is awake and
use this to block timer changes from outside the EL. Restore the
known-set value prior to sleeping, updating timerFd first if necessary
 - Don't read from timerFd when processing expiry event

Result

- Cleaner API for integrating with different EL/queue timing impls
- Fixed race condition to avoid missing scheduled wakeups
- Eliminate unnecessary timerFd updates while EL is awake, and
unnecessary expired timerFd reads
- Avoid unnecessary scheduled-task wakeups when using NIO transport

I did not yet further explore the suggestion of using
TFD_TIMER_ABSTIME for the timerFd.
2019-08-21 12:34:22 +02:00
.github Change the netty.io homepage scheme(http -> https) (#9344) 2019-07-09 21:09:42 +02:00
.mvn support publishing snapshots from docker based ci (#8634) 2018-12-07 05:43:06 +01:00
all [maven-release-plugin] prepare for next development iteration 2019-08-13 17:16:42 +00:00
bom [maven-release-plugin] prepare for next development iteration 2019-08-13 17:16:42 +00:00
buffer Avoid creating FileInputStream and FileOutputStream for obtaining Fil… (#8110) 2019-08-17 09:43:01 +02:00
codec [maven-release-plugin] prepare for next development iteration 2019-08-13 17:16:42 +00:00
codec-dns Detect truncated responses caused by EDNS0 and MTU miss-match (#9468) 2019-08-17 09:58:22 +02:00
codec-haproxy [maven-release-plugin] prepare for next development iteration 2019-08-13 17:16:42 +00:00
codec-http Fix unexpected IllegalReferenceCountException on decode multipart request. (#8575) 2019-08-19 15:08:40 +02:00
codec-http2 HTTP2: Update local flow-controller on Channel.read() if needed (#9400) 2019-08-16 09:27:47 +02:00
codec-memcache [maven-release-plugin] prepare for next development iteration 2019-08-13 17:16:42 +00:00
codec-mqtt [maven-release-plugin] prepare for next development iteration 2019-08-13 17:16:42 +00:00
codec-redis [maven-release-plugin] prepare for next development iteration 2019-08-13 17:16:42 +00:00
codec-smtp [maven-release-plugin] prepare for next development iteration 2019-08-13 17:16:42 +00:00
codec-socks [maven-release-plugin] prepare for next development iteration 2019-08-13 17:16:42 +00:00
codec-stomp [maven-release-plugin] prepare for next development iteration 2019-08-13 17:16:42 +00:00
codec-xml [maven-release-plugin] prepare for next development iteration 2019-08-13 17:16:42 +00:00
common Simplify EventLoop abstractions for timed scheduled tasks (#9470) 2019-08-21 12:34:22 +02:00
dev-tools [maven-release-plugin] prepare for next development iteration 2019-08-13 17:16:42 +00:00
docker Cleanup docker / docker-compose configs (#9473) 2019-08-16 13:59:36 +02:00
example [maven-release-plugin] prepare for next development iteration 2019-08-13 17:16:42 +00:00
handler Add tests for using Amazon Corretto Crypto Provider with Netty (#9480) 2019-08-20 14:55:26 +02:00
handler-proxy [maven-release-plugin] prepare for next development iteration 2019-08-13 17:16:42 +00:00
license Use Table lookup for HPACK decoder (#9307) 2019-07-02 20:09:44 +02:00
microbench [maven-release-plugin] prepare for next development iteration 2019-08-13 17:16:42 +00:00
resolver [maven-release-plugin] prepare for next development iteration 2019-08-13 17:16:42 +00:00
resolver-dns Detect truncated responses caused by EDNS0 and MTU miss-match (#9468) 2019-08-17 09:58:22 +02:00
tarball [maven-release-plugin] prepare for next development iteration 2019-08-13 17:16:42 +00:00
testsuite Avoid creating FileInputStream and FileOutputStream for obtaining Fil… (#8110) 2019-08-17 09:43:01 +02:00
testsuite-autobahn [maven-release-plugin] prepare for next development iteration 2019-08-13 17:16:42 +00:00
testsuite-http2 [maven-release-plugin] prepare for next development iteration 2019-08-13 17:16:42 +00:00
testsuite-native-image [maven-release-plugin] prepare for next development iteration 2019-08-13 17:16:42 +00:00
testsuite-osgi [maven-release-plugin] prepare for next development iteration 2019-08-13 17:16:42 +00:00
testsuite-shading [maven-release-plugin] prepare for next development iteration 2019-08-13 17:16:42 +00:00
transport Simplify EventLoop abstractions for timed scheduled tasks (#9470) 2019-08-21 12:34:22 +02:00
transport-native-epoll Simplify EventLoop abstractions for timed scheduled tasks (#9470) 2019-08-21 12:34:22 +02:00
transport-native-kqueue [maven-release-plugin] prepare for next development iteration 2019-08-13 17:16:42 +00:00
transport-native-unix-common [maven-release-plugin] prepare for next development iteration 2019-08-13 17:16:42 +00:00
transport-native-unix-common-tests [maven-release-plugin] prepare for next development iteration 2019-08-13 17:16:42 +00:00
transport-rxtx [maven-release-plugin] prepare for next development iteration 2019-08-13 17:16:42 +00:00
transport-sctp [maven-release-plugin] prepare for next development iteration 2019-08-13 17:16:42 +00:00
transport-udt [maven-release-plugin] prepare for next development iteration 2019-08-13 17:16:42 +00:00
.fbprefs Updated Find Bugs configuration 2009-03-04 10:33:09 +00:00
.gitattributes Include mvn wrapper to make setup of development env easier 2018-01-26 08:13:17 +01:00
.gitignore Add .gitignore for docker-sync stuff 2019-03-19 14:03:15 +01:00
CONTRIBUTING.md Change the netty.io homepage scheme(http -> https) (#9344) 2019-07-09 21:09:42 +02:00
LICENSE.txt Relicensed to Apache License v2 2009-08-28 07:15:49 +00:00
mvnw Include mvn wrapper to make setup of development env easier 2018-01-26 08:13:17 +01:00
mvnw.cmd Include mvn wrapper to make setup of development env easier 2018-01-26 08:13:17 +01:00
NOTICE.txt Change the netty.io homepage scheme(http -> https) (#9344) 2019-07-09 21:09:42 +02:00
pom.xml [maven-release-plugin] prepare for next development iteration 2019-08-13 17:16:42 +00:00
README.md Change the netty.io homepage scheme(http -> https) (#9344) 2019-07-09 21:09:42 +02:00
run-example.sh Add UptimeServer and adjust UptimeClient's code style. 2017-04-28 07:41:07 +02:00

Netty Project

Netty is an asynchronous event-driven network application framework for rapid development of maintainable high performance protocol servers & clients.

How to build

For the detailed information about building and developing Netty, please visit the developer guide. This page only gives very basic information.

You require the following to build Netty:

Note that this is build-time requirement. JDK 5 (for 3.x) or 6 (for 4.0+) is enough to run your Netty-based application.

Branches to look

Development of all versions takes place in each branch whose name is identical to <majorVersion>.<minorVersion>. For example, the development of 3.9 and 4.0 resides in the branch '3.9' and the branch '4.0' respectively.

Usage with JDK 9

Netty can be used in modular JDK9 applications as a collection of automatic modules. The module names follow the reverse-DNS style, and are derived from subproject names rather than root packages due to historical reasons. They are listed below:

  • io.netty.all
  • io.netty.buffer
  • io.netty.codec
  • io.netty.codec.dns
  • io.netty.codec.haproxy
  • io.netty.codec.http
  • io.netty.codec.http2
  • io.netty.codec.memcache
  • io.netty.codec.mqtt
  • io.netty.codec.redis
  • io.netty.codec.smtp
  • io.netty.codec.socks
  • io.netty.codec.stomp
  • io.netty.codec.xml
  • io.netty.common
  • io.netty.handler
  • io.netty.handler.proxy
  • io.netty.resolver
  • io.netty.resolver.dns
  • io.netty.transport
  • io.netty.transport.epoll (native omitted - reserved keyword in Java)
  • io.netty.transport.kqueue (native omitted - reserved keyword in Java)
  • io.netty.transport.unix.common (native omitted - reserved keyword in Java)
  • io.netty.transport.rxtx
  • io.netty.transport.sctp
  • io.netty.transport.udt

Automatic modules do not provide any means to declare dependencies, so you need to list each used module separately in your module-info file.