Go to file
Frédéric Brégier 3a58063fe7 Fix for performance regression on HttpPost RequestDecoder (#10623)
Fix issue #10508 where PARANOID mode slow down about 1000 times compared to ADVANCED.
Also fix a rare issue when internal buffer was growing over a limit, it was partially discarded
using `discardReadBytes()` which causes bad changes within previously discovered HttpData.

Reasons were:

Too many `readByte()` method calls while other ways exist (such as keep in memory the last scan position when trying to find a delimiter or using `bytesBefore(firstByte)` instead of looping externally).

Changes done:
- major change on way buffer are parsed: instead of read byte per byte until found delimiter, try to find the delimiter using `bytesBefore()` and keep the last unfound position to skeep already parsed parts (algorithms are the same but implementation of scan are different)
- Change the condition to discard read bytes when refCnt is at most 1.

Observations using Async-Profiler:
==================================

1) Without optimizations, most of the time (more than 95%) is through `readByte()` method within `loadDataMultipartStandard` method.
2) With using `bytesBefore(byte)` instead of `readByte()` to find various delimiter, the `loadDataMultipartStandard` method is going down to 19 to 33% depending on the test used. the `readByte()` method or equivalent `getByte(pos)` method are going down to 15% (from 95%).

Times are confirming those profiling:
- With optimizations, in SIMPLE mode about 82% better, in ADVANCED mode about 79% better and in PARANOID mode about 99% better (most of the duplicate read accesses are removed or make internally through `bytesBefore(byte)` method)

A benchmark is added to show the behavior of the various cases (one big item, such as File upload, and many items) and various level of detection (Disabled, Simple, Advanced, Paranoid). This benchmark is intend to alert if new implementations make too many differences (such as the previous version where about PARANOID gives about 1000 times slower than other levels, while it is now about at most 10 times).

Extract of Benchmark run:
=========================

Run complete. Total time: 00:13:27

Benchmark                                                                           Mode  Cnt  Score   Error   Units
HttpPostMultipartRequestDecoderBenchmark.multipartRequestDecoderBigAdvancedLevel   thrpt    6  2,248 ± 0,198 ops/ms
HttpPostMultipartRequestDecoderBenchmark.multipartRequestDecoderBigDisabledLevel   thrpt    6  2,067 ± 1,219 ops/ms
HttpPostMultipartRequestDecoderBenchmark.multipartRequestDecoderBigParanoidLevel   thrpt    6  1,109 ± 0,038 ops/ms
HttpPostMultipartRequestDecoderBenchmark.multipartRequestDecoderBigSimpleLevel     thrpt    6  2,326 ± 0,314 ops/ms
HttpPostMultipartRequestDecoderBenchmark.multipartRequestDecoderHighAdvancedLevel  thrpt    6  1,444 ± 0,226 ops/ms
HttpPostMultipartRequestDecoderBenchmark.multipartRequestDecoderHighDisabledLevel  thrpt    6  1,462 ± 0,642 ops/ms
HttpPostMultipartRequestDecoderBenchmark.multipartRequestDecoderHighParanoidLevel  thrpt    6  0,159 ± 0,003 ops/ms
HttpPostMultipartRequestDecoderBenchmark.multipartRequestDecoderHighSimpleLevel    thrpt    6  1,522 ± 0,049 ops/ms
2020-11-19 08:01:05 +01:00
.github Smaller output in codeql build (#10787) 2020-11-09 15:32:49 +01:00
.mvn Use latest maven release (#9820) 2019-11-27 14:45:48 +01:00
all Use correct netty-build artifact 2020-11-10 14:10:48 +01:00
bom Use http in xmlns URIs to make maven release plugin happy again (#10788) 2020-11-10 10:51:05 +01:00
buffer Fix compilation failure introduced by bad cherry-picking of 057eb121f4 2020-11-16 09:26:07 +01:00
codec Use http in xmlns URIs to make maven release plugin happy again (#10788) 2020-11-10 10:51:05 +01:00
codec-dns Use http in xmlns URIs to make maven release plugin happy again (#10788) 2020-11-10 10:51:05 +01:00
codec-haproxy Use http in xmlns URIs to make maven release plugin happy again (#10788) 2020-11-10 10:51:05 +01:00
codec-http Fix for performance regression on HttpPost RequestDecoder (#10623) 2020-11-19 08:01:05 +01:00
codec-http2 Fix comment typo DelegatingDecompressorFrameListener (#10789) 2020-11-16 09:03:54 +01:00
codec-memcache Use http in xmlns URIs to make maven release plugin happy again (#10788) 2020-11-10 10:51:05 +01:00
codec-mqtt Use http in xmlns URIs to make maven release plugin happy again (#10788) 2020-11-10 10:51:05 +01:00
codec-redis Use http in xmlns URIs to make maven release plugin happy again (#10788) 2020-11-10 10:51:05 +01:00
codec-smtp Use http in xmlns URIs to make maven release plugin happy again (#10788) 2020-11-10 10:51:05 +01:00
codec-socks Use http in xmlns URIs to make maven release plugin happy again (#10788) 2020-11-10 10:51:05 +01:00
codec-stomp Use http in xmlns URIs to make maven release plugin happy again (#10788) 2020-11-10 10:51:05 +01:00
codec-xml Use http in xmlns URIs to make maven release plugin happy again (#10788) 2020-11-10 10:51:05 +01:00
common Use http in xmlns URIs to make maven release plugin happy again (#10788) 2020-11-10 10:51:05 +01:00
dev-tools Use http in xmlns URIs to make maven release plugin happy again (#10788) 2020-11-10 10:51:05 +01:00
docker Update to latest java 8/11/15 versions (#10774) 2020-11-04 10:41:33 +01:00
example Use http in xmlns URIs to make maven release plugin happy again (#10788) 2020-11-10 10:51:05 +01:00
handler Use http in xmlns URIs to make maven release plugin happy again (#10788) 2020-11-10 10:51:05 +01:00
handler-proxy Use http in xmlns URIs to make maven release plugin happy again (#10788) 2020-11-10 10:51:05 +01:00
license Enable nohttp check during the build (#10708) 2020-10-23 15:26:25 +02:00
microbench Fix for performance regression on HttpPost RequestDecoder (#10623) 2020-11-19 08:01:05 +01:00
resolver Use http in xmlns URIs to make maven release plugin happy again (#10788) 2020-11-10 10:51:05 +01:00
resolver-dns Use http in xmlns URIs to make maven release plugin happy again (#10788) 2020-11-10 10:51:05 +01:00
resolver-dns-native-macos Use http in xmlns URIs to make maven release plugin happy again (#10788) 2020-11-10 10:51:05 +01:00
tarball Use http in xmlns URIs to make maven release plugin happy again (#10788) 2020-11-10 10:51:05 +01:00
testsuite Use http in xmlns URIs to make maven release plugin happy again (#10788) 2020-11-10 10:51:05 +01:00
testsuite-autobahn Use http in xmlns URIs to make maven release plugin happy again (#10788) 2020-11-10 10:51:05 +01:00
testsuite-http2 Use http in xmlns URIs to make maven release plugin happy again (#10788) 2020-11-10 10:51:05 +01:00
testsuite-native-image Use http in xmlns URIs to make maven release plugin happy again (#10788) 2020-11-10 10:51:05 +01:00
testsuite-native-image-client Use http in xmlns URIs to make maven release plugin happy again (#10788) 2020-11-10 10:51:05 +01:00
testsuite-osgi Use http in xmlns URIs to make maven release plugin happy again (#10788) 2020-11-10 10:51:05 +01:00
testsuite-shading Use http in xmlns URIs to make maven release plugin happy again (#10788) 2020-11-10 10:51:05 +01:00
transport Use http in xmlns URIs to make maven release plugin happy again (#10788) 2020-11-10 10:51:05 +01:00
transport-blockhound-tests Use http in xmlns URIs to make maven release plugin happy again (#10788) 2020-11-10 10:51:05 +01:00
transport-native-epoll Use http in xmlns URIs to make maven release plugin happy again (#10788) 2020-11-10 10:51:05 +01:00
transport-native-kqueue Use http in xmlns URIs to make maven release plugin happy again (#10788) 2020-11-10 10:51:05 +01:00
transport-native-unix-common Use http in xmlns URIs to make maven release plugin happy again (#10788) 2020-11-10 10:51:05 +01:00
transport-native-unix-common-tests Use http in xmlns URIs to make maven release plugin happy again (#10788) 2020-11-10 10:51:05 +01:00
transport-sctp Use http in xmlns URIs to make maven release plugin happy again (#10788) 2020-11-10 10:51:05 +01:00
transport-udt/lib/bin/lib/x86_64-MacOSX-gpp/jni Replace reflection usage with MethodHandles when performance matters (#10097) 2020-03-11 21:04:40 +01:00
.fbprefs Updated Find Bugs configuration 2009-03-04 10:33:09 +00:00
.gitattributes Include mvn wrapper to make setup of development env easier 2018-01-26 08:13:17 +01:00
.gitignore Ignore .shelf/ folder generated by IntelliJ IDEA (#10445) 2020-08-03 07:52:18 +02:00
.lgtm.yml Enables lgtm.com to process this project and create a CodeQL database 2020-01-20 19:22:49 +01:00
CONTRIBUTING.md Change the netty.io homepage scheme(http -> https) (#9344) 2019-07-09 21:10:14 +02:00
LICENSE.txt Enable nohttp check during the build (#10708) 2020-10-23 15:26:25 +02:00
mvnw Enable nohttp check during the build (#10708) 2020-10-23 15:26:25 +02:00
mvnw.cmd Enable nohttp check during the build (#10708) 2020-10-23 15:26:25 +02:00
nohttp-checkstyle-suppressions.xml Use http in xmlns URIs to make maven release plugin happy again (#10788) 2020-11-10 10:51:05 +01:00
nohttp-checkstyle.xml Enable nohttp check during the build (#10708) 2020-10-23 15:26:25 +02:00
NOTICE.txt Fix License type of dnsinfo (#10773) 2020-11-04 10:41:41 +01:00
pom.xml Use http in xmlns URIs to make maven release plugin happy again (#10788) 2020-11-10 10:51:05 +01:00
README.md Enable nohttp check during the build (#10708) 2020-10-23 15:26:25 +02:00
run-example.sh Add DNS client examples for run-example.sh (#10283) 2020-05-14 12:10:48 +02:00
SECURITY.md Added a security policy (#10692) 2020-10-15 20:40:05 +02:00

Netty Project

Netty is an asynchronous event-driven network application framework for rapid development of maintainable high performance protocol servers & clients.

How to build

For the detailed information about building and developing Netty, please visit the developer guide. This page only gives very basic information.

You require the following to build Netty:

Note that this is build-time requirement. JDK 5 (for 3.x) or 6 (for 4.0+) is enough to run your Netty-based application.

Branches to look

Development of all versions takes place in each branch whose name is identical to <majorVersion>.<minorVersion>. For example, the development of 3.9 and 4.0 resides in the branch '3.9' and the branch '4.0' respectively.

Usage with JDK 9

Netty can be used in modular JDK9 applications as a collection of automatic modules. The module names follow the reverse-DNS style, and are derived from subproject names rather than root packages due to historical reasons. They are listed below:

  • io.netty.all
  • io.netty.buffer
  • io.netty.codec
  • io.netty.codec.dns
  • io.netty.codec.haproxy
  • io.netty.codec.http
  • io.netty.codec.http2
  • io.netty.codec.memcache
  • io.netty.codec.mqtt
  • io.netty.codec.redis
  • io.netty.codec.smtp
  • io.netty.codec.socks
  • io.netty.codec.stomp
  • io.netty.codec.xml
  • io.netty.common
  • io.netty.handler
  • io.netty.handler.proxy
  • io.netty.resolver
  • io.netty.resolver.dns
  • io.netty.transport
  • io.netty.transport.epoll (native omitted - reserved keyword in Java)
  • io.netty.transport.kqueue (native omitted - reserved keyword in Java)
  • io.netty.transport.unix.common (native omitted - reserved keyword in Java)
  • io.netty.transport.rxtx
  • io.netty.transport.sctp
  • io.netty.transport.udt

Automatic modules do not provide any means to declare dependencies, so you need to list each used module separately in your module-info file.