Go to file
Nick Hill 10539f4dc7 Streamline CompositeByteBuf internals (#8437)
Motivation:

CompositeByteBuf is a powerful and versatile abstraction, allowing for
manipulation of large data without copying bytes. There is still a
non-negligible cost to reading/writing however relative to "singular"
ByteBufs, and this can be mostly eliminated with some rework of the
internals.

My use case is message modification/transformation while zero-copy
proxying. For example replacing a string within a large message with one
of a different length

Modifications:

- No longer slice added buffers and unwrap added slices
   - Components store target buf offset relative to position in
composite buf
   - Less allocations, object footprint, pointer indirection, offset
arithmetic
- Use Component[] rather than ArrayList<Component>
   - Avoid pointer indirection and duplicate bounds check, more
efficient backing array growth
   - Facilitates optimization when doing bulk-inserts - inserting n
ByteBufs behind m is now O(m + n) instead of O(mn)
- Avoid unnecessary casting and method call indirection via superclass
- Eliminate some duplicate range/ref checks via non-checking versions of
toComponentIndex and findComponent
- Add simple fast-path for toComponentIndex(0); add racy cache of
last-accessed Component to findComponent(int)
- Override forEachByte0(...) and forEachByteDesc0(...) methods
- Make use of RecyclableArrayList in nioBuffers(int, int) (in line with
FasterCompositeByteBuf impl)
- Modify addComponents0(boolean,int,Iterable) to use the Iterable
directly rather than copy to an array first (and possibly to an
ArrayList before that)
- Optimize addComponents0(boolean,int,ByteBuf[],int) to not perform
repeated array insertions and avoid second loop for offset updates
- Simplify other logic in various places, in particular the general
pattern used where a sub-range is iterated over
- Add benchmarks to demonstrate some improvements

While refactoring I also came across a couple of clear bugs. They are
fixed in these changes but I will open another PR with unit tests and
fixes to the current version.

Result:

Much faster creation, manipulation, and access; many fewer allocations
and smaller footprint. Benchmark results to follow.
2018-11-03 10:37:07 +01:00
.github Use GitHub Issue/PR Template Feature 2016-12-07 11:40:26 -08:00
.mvn/wrapper Include mvn wrapper to make setup of development env easier 2018-01-26 08:13:17 +01:00
all [maven-release-plugin] prepare for next development iteration 2018-10-29 15:38:51 +00:00
bom [maven-release-plugin] prepare for next development iteration 2018-10-29 15:38:51 +00:00
buffer Streamline CompositeByteBuf internals (#8437) 2018-11-03 10:37:07 +01:00
codec Don't swallow intermediate write failures in MessageToMessageEncoder (#8454) 2018-11-03 10:36:26 +01:00
codec-dns [maven-release-plugin] prepare for next development iteration 2018-10-29 15:38:51 +00:00
codec-haproxy [maven-release-plugin] prepare for next development iteration 2018-10-29 15:38:51 +00:00
codec-http [maven-release-plugin] prepare for next development iteration 2018-10-29 15:38:51 +00:00
codec-http2 Remove uninterpolated {} in DefaultHttp2ConnectionDecoder log message (#8441) 2018-10-30 10:09:27 +01:00
codec-memcache [maven-release-plugin] prepare for next development iteration 2018-10-29 15:38:51 +00:00
codec-mqtt [maven-release-plugin] prepare for next development iteration 2018-10-29 15:38:51 +00:00
codec-redis [maven-release-plugin] prepare for next development iteration 2018-10-29 15:38:51 +00:00
codec-smtp [maven-release-plugin] prepare for next development iteration 2018-10-29 15:38:51 +00:00
codec-socks [maven-release-plugin] prepare for next development iteration 2018-10-29 15:38:51 +00:00
codec-stomp [maven-release-plugin] prepare for next development iteration 2018-10-29 15:38:51 +00:00
codec-xml [maven-release-plugin] prepare for next development iteration 2018-10-29 15:38:51 +00:00
common Make it clear that HashedWheelTimer only support millis. (#8322) 2018-11-02 08:10:18 +01:00
dev-tools [maven-release-plugin] prepare for next development iteration 2018-10-29 15:38:51 +00:00
docker Update to latest openjdk 12 ea release. (#8459) 2018-11-03 09:29:44 +01:00
example #7695 no need to manually release chunk during upload (#7696) 2018-11-02 08:12:10 +01:00
handler Don't double release ByteBuf when parsing of the X509Certificate fails (#8457) 2018-11-02 17:08:53 +01:00
handler-proxy [maven-release-plugin] prepare for next development iteration 2018-10-29 15:38:51 +00:00
license Add the NOTICE of the forked portion of Apache Harmony 2018-01-30 11:22:51 +01:00
microbench Streamline CompositeByteBuf internals (#8437) 2018-11-03 10:37:07 +01:00
resolver [maven-release-plugin] prepare for next development iteration 2018-10-29 15:38:51 +00:00
resolver-dns Fix NPE when trying to build a DnsNameResolver with a null resolvedAddressTypes (#8445) 2018-10-30 13:15:16 +01:00
tarball [maven-release-plugin] prepare for next development iteration 2018-10-29 15:38:51 +00:00
testsuite [maven-release-plugin] prepare for next development iteration 2018-10-29 15:38:51 +00:00
testsuite-autobahn [maven-release-plugin] prepare for next development iteration 2018-10-29 15:38:51 +00:00
testsuite-http2 [maven-release-plugin] prepare for next development iteration 2018-10-29 15:38:51 +00:00
testsuite-osgi [maven-release-plugin] prepare for next development iteration 2018-10-29 15:38:51 +00:00
testsuite-shading [maven-release-plugin] prepare for next development iteration 2018-10-29 15:38:51 +00:00
transport [maven-release-plugin] prepare for next development iteration 2018-10-29 15:38:51 +00:00
transport-native-epoll Add testcase for epollWait(...) with negative timerfd values. (#8447) 2018-10-30 19:38:02 +01:00
transport-native-kqueue [maven-release-plugin] prepare for next development iteration 2018-10-29 15:38:51 +00:00
transport-native-unix-common [maven-release-plugin] prepare for next development iteration 2018-10-29 15:38:51 +00:00
transport-native-unix-common-tests [maven-release-plugin] prepare for next development iteration 2018-10-29 15:38:51 +00:00
transport-rxtx [maven-release-plugin] prepare for next development iteration 2018-10-29 15:38:51 +00:00
transport-sctp [maven-release-plugin] prepare for next development iteration 2018-10-29 15:38:51 +00:00
transport-udt [maven-release-plugin] prepare for next development iteration 2018-10-29 15:38:51 +00:00
.fbprefs Updated Find Bugs configuration 2009-03-04 10:33:09 +00:00
.gitattributes Include mvn wrapper to make setup of development env easier 2018-01-26 08:13:17 +01:00
.gitignore Exclude mainframer related files from git 2018-10-14 13:20:18 +02:00
CONTRIBUTING.md Move the pull request guide to the developer guide 2014-03-12 13:13:58 +09:00
LICENSE.txt Relicensed to Apache License v2 2009-08-28 07:15:49 +00:00
mvnw Include mvn wrapper to make setup of development env easier 2018-01-26 08:13:17 +01:00
mvnw.cmd Include mvn wrapper to make setup of development env easier 2018-01-26 08:13:17 +01:00
NOTICE.txt Add the NOTICE of the forked portion of Apache Harmony 2018-01-30 11:22:51 +01:00
pom.xml Update to maven-surefire-plugin 2.22.1 (#8418) 2018-11-02 08:09:54 +01:00
README.md Provide an Automatic-Module-Name for the netty-all artifact fixes #7644 2018-01-27 20:31:16 +01:00
run-example.sh Add UptimeServer and adjust UptimeClient's code style. 2017-04-28 07:41:07 +02:00

Netty Project

Netty is an asynchronous event-driven network application framework for rapid development of maintainable high performance protocol servers & clients.

How to build

For the detailed information about building and developing Netty, please visit the developer guide. This page only gives very basic information.

You require the following to build Netty:

Note that this is build-time requirement. JDK 5 (for 3.x) or 6 (for 4.0+) is enough to run your Netty-based application.

Branches to look

Development of all versions takes place in each branch whose name is identical to <majorVersion>.<minorVersion>. For example, the development of 3.9 and 4.0 resides in the branch '3.9' and the branch '4.0' respectively.

Usage with JDK 9

Netty can be used in modular JDK9 applications as a collection of automatic modules. The module names follow the reverse-DNS style, and are derived from subproject names rather than root packages due to historical reasons. They are listed below:

  • io.netty.all
  • io.netty.buffer
  • io.netty.codec
  • io.netty.codec.dns
  • io.netty.codec.haproxy
  • io.netty.codec.http
  • io.netty.codec.http2
  • io.netty.codec.memcache
  • io.netty.codec.mqtt
  • io.netty.codec.redis
  • io.netty.codec.smtp
  • io.netty.codec.socks
  • io.netty.codec.stomp
  • io.netty.codec.xml
  • io.netty.common
  • io.netty.handler
  • io.netty.handler.proxy
  • io.netty.resolver
  • io.netty.resolver.dns
  • io.netty.transport
  • io.netty.transport.epoll (native omitted - reserved keyword in Java)
  • io.netty.transport.kqueue (native omitted - reserved keyword in Java)
  • io.netty.transport.unix.common (native omitted - reserved keyword in Java)
  • io.netty.transport.rxtx
  • io.netty.transport.sctp
  • io.netty.transport.udt

Automatic modules do not provide any means to declare dependencies, so you need to list each used module separately in your module-info file.