Commit Graph

1284 Commits

Author SHA1 Message Date
Roman Puchkovskiy
80c28a3c33
Fix native image build for the cases when io.netty.util.NetUtil is initialized at run-time (#10799)
Motivation:

In #10630, field substitutions were introduced for NetUtil.LOCALHOST4, NetUtil.LOCALHOST6 and NetUtil.LOCALHOST fields. They were required to allow a native image be built with most of Netty (including NetUtil) initialized at build time.

The substitutions created in #10630 only define getters, so the 3 fields can only be read in a native image.

But when NetUtil is initialized at run-time (this is what happens in #10797), its static initialization block is executed, and this block writes to all 3 fields. As the substitutions do not provide any setters, field stores are not valid, and such builds fail.

Modifications:

- Add netty-testsuite-native-image-client-runtime-init Maven module that builds a native image deferring NetUtil class initialization till run-time; this module is used to demonstrate the problem and verify the problem is gone with the fix
- Add no-op setters to substitutions for NetUtil.LOCALHOST4, NetUtil.LOCALHOST6 and NetUtil.LOCALHOST

Result:

A native image initializing NetUtil at run-time builds successfully.

Fixes #10797
2020-12-07 18:16:30 +01:00
Violeta Georgieva
8a7c580bf6
Add whitelist entry for SSLEngineImpl.wrap to BlockHound configuration (#10844)
Motivation:

Internally SSLEngineImpl.wrap(...) may call FileInputStream.read(...).
This will cause the error below when BlockHound is enabled
reactor.blockhound.BlockingOperationError: Blocking call! java.io.FileInputStream#readBytes
	at java.io.FileInputStream.readBytes(FileInputStream.java)
	at java.io.FileInputStream.read(FileInputStream.java:255)

Modifications:

- Add whitelist entry to BlockHound configuration
- Add test

Result:

Fixes #10837
2020-12-07 08:06:21 +01:00
Norman Maurer
44f85bba5f
Ensure we don't leak the ClassLoader in the backtrace of TrackRecord.BOTTOM (#10839)
Motivation:

We need to ensure we override fillInStacktrace() when we store exceptions in static fields to not leak the Classloader in the backtrace.

Came up in https://github.com/netty/netty/pull/10691#issuecomment-738331186. Thanks to @amir-shalem for notice this one.

Modifications:

- Add overrides of fillInStracktrace in TrackRecord.BOTTOM

Result:

Related fix to https://github.com/netty/netty/pull/10686
2020-12-05 07:01:03 +01:00
Johnny Lim
37267f95ce
Initialize Recycler.INITIAL_CAPACITY before invoking InternalLogger.debug() (#10836)
Motivation:

If Log4J2's `Filter` creates `Recycler.Stack` somehow, `Recycler.Stack()` will see uninitialized `Recycler.INITIAL_CAPACITY`. This has been raised originally in https://github.com/micrometer-metrics/micrometer/issues/2369.

Modification:

This PR changes to initialize `Recycler.INITIAL_CAPACITY` before invoking `InternalLogger.debug()` to avoid it.

Result:

Fixes the problem described in the "Motivation" section.
2020-12-04 14:57:06 +01:00
Alexandre Dutra
02cd85181a
Allow blocking calls inside SingleThreadEventExecutor.addTask (#10811)
Motivation:

GlobalEventExecutor.addTask was rightfully allowed to block by commit
09d38c8. However the same should have been done for
SingleThreadEventExecutor.addTask.

BlockHound is currently intercepting that call, and as a consequence,
it prevents SingleThreadEventExecutor from working properly, if addTask is
called from a thread that cannot block.

The interception is due to LinkedBlockingQueue.offer implementation,
which uses a ReentrantLock internally.

Modifications:

* Added one BlockHound exception to
io.netty.util.internal.Hidden.NettyBlockHoundIntegration for
SingleThreadEventExecutor.addTask.
* Also added unit tests for both SingleThreadEventExecutor.addTask
and GlobalEventExecutor.addTask.

Result:

SingleThreadEventExecutor.addTask can now be invoked from any thread
when BlockHound is activated.
2020-11-23 19:20:18 +01:00
Alexandre Dutra
2ff8973f69
Allow blocking calls inside HashedWheelTimer start() and stop() (#10810)
Motivation:

When a HashedWheelTimer instance is started or stopped, its working
thread is started or stopped. These operations block the calling
thread:
- start() calls java.util.concurrent.CountDownLatch.await() to wait
for the worker thread to finish initializing;
- stop() calls java.lang.Thread.join(long) to wait for the worker
thread to exit.
BlockHound detects these calls and as a consequence, prevents
HashedWheelTimer from working properly, if it is started or stopped
in a thread that is not allowed to block.

Modifications:

Added two more BlockHound exceptions to
io.netty.util.internal.Hidden.NettyBlockHoundIntegration: one
for HashedWheelTimer.start() and one for HashedWheelTimer.stop().

Result:

HashedWheelTimer can now be started and stopped properly when
BlockHound is activated.
2020-11-23 08:41:07 +01:00
root
944a020586 [maven-release-plugin] prepare for next development iteration 2020-11-11 05:47:51 +00:00
root
715353ecd6 [maven-release-plugin] prepare release netty-4.1.54.Final 2020-11-11 05:47:37 +00:00
root
afca81a9d8 [maven-release-plugin] rollback the release of netty-4.1.54.Final 2020-11-10 12:02:24 +00:00
root
e256074e49 [maven-release-plugin] prepare for next development iteration 2020-11-10 11:12:23 +00:00
root
cea659bd8a [maven-release-plugin] prepare release netty-4.1.54.Final 2020-11-10 11:12:06 +00:00
Norman Maurer
5ffca6ef4a
Use http in xmlns URIs to make maven release plugin happy again (#10788)
Motivation:

https in xmlns URIs does not work and will let the maven release plugin fail:

```
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  1.779 s
[INFO] Finished at: 2020-11-10T07:45:21Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-release-plugin:2.5.3:prepare (default-cli) on project netty-parent: Execution default-cli of goal org.apache.maven.plugins:maven-release-plugin:2.5.3:prepare failed: The namespace xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" could not be added as a namespace to "project": The namespace prefix "xsi" collides with an additional namespace declared by the element -> [Help 1]
[ERROR]
```

See also https://issues.apache.org/jira/browse/HBASE-24014.

Modifications:

Use http for xmlns

Result:

Be able to use maven release plugin
2020-11-10 10:22:35 +01:00
Norman Maurer
d371b1bbaa
Rethrow NoSuchMethodError with more hints about incompatible native library versions (#10740)
Motivation:

03aafb9cff did ensure we fail while loading a natibe library which is not compatible. While this is great it is still sometimes hard for people to understand what NoSuchMethodError means in this context.

Modifications:

If possible rethrow the NoSuchMethodError and provide some more hints about multiple versions of the shared library

Result:

Easier to understand for people why loading fails
2020-10-28 19:22:08 +01:00
Norman Maurer
d4defc30b8
Add PlatformDependent* methods that are needed by io_uring (#10744)
Motivation:

ddebc10 added a few adjustments that are needed by io_uring that we will add as an incubator repository. Unfortunally we missed the changed in PlatformDependent*.

Modifications:

Add missing methods

Result:

Be able to compile io_uring code against core netty
2020-10-28 19:20:52 +01:00
Roman Puchkovskiy
4ecd78e104
Fix native image build on modern GraalVM versions for the cases when the program uses netty-dns (#10630)
Motivation:

Since GraalVM version 19.3.0, instances of java.net.InetAddress (and its subclasses Inet4Address and Inet6Address) are not allowed in native image heap (that is, they cannot be stored in static fields of classes initialized at build time or be reachable through static fields of such classes). When building a native image, it makes sense to initialize at build time as many classes as possible.
But some fields of some classes in Netty (for example, NetUtil.LOCALHOST4) contain InetAddress instances. If a program is using code path that makes it possible to reach such fields at build time initialization, it becomes impossible to build a native image initializing core Netty classes initialized at runtime. An example of such a program is a client that uses netty-dns.

Modifications:

- Add netty-testsuite-native-image-client Maven module to test that such an example program can be built after the corresponding fixes
- Add native-image.properties to resolver-dns module to move initialization of some classes to runtime (some of them are parsing configuration during initialization, so it makes no sense to initialize them at build time; for others, it's needed to avoid InetAddress reachability at build time)
- Add substitutions for NetUtil.LOCALHOST4, NetUtil.LOCALHOST6 and NetUtil.LOCALHOST to overcome the InetAddress-related prohibition
- Extract some initialization code from NetUtil to NetUtilInitializations to allow it to be used by the substitutions

Result:

A client program using netty-dns with --initialize-at-build-time=io.netty builds successfully
2020-10-26 08:34:31 +01:00
Artem Smotrakov
e5951d46fc
Enable nohttp check during the build (#10708)
Motivation:

HTTP is a plaintext protocol which means that someone may be able
to eavesdrop the data. To prevent this, HTTPS should be used whenever
possible. However, maintaining using https:// in all URLs may be
difficult. The nohttp tool can help here. The tool scans all the files
in a repository and reports where http:// is used.

Modifications:

- Added nohttp (via checkstyle) into the build process.
- Suppressed findings for the websites
  that don't support HTTPS or that are not reachable

Result:

- Prevent using HTTP in the future.
- Encourage users to use HTTPS when they follow the links they found in
  the code.
2020-10-23 14:44:18 +02:00
Norman Maurer
ffbddcd842
Replace deprecated Assert.assertThat(...) with MatcherAssert.assertThat(...) (#10699)
Motivation:

junit deprecated Assert.assertThat(...)

Modifications:

Use MatcherAssert.assertThat(...) as replacement for deprecated method

Result:

Less deprecation warnings
2020-10-18 14:30:52 +02:00
Artem Smotrakov
1ca7d5db81
Fix or suppress LGTM findings (#10689)
Motivation:

LGTM reports multiple issues. They need to be triaged,
and real ones should be fixed.

Modifications:
- Fixed multiple issues reported by LGTM, such as redundant conditions,
  resource leaks, typos, possible integer overflows.
- Suppressed false-positives.
- Added a few testcases.

Result:

Fixed several possible issues, get rid of false alarms in the LGTM report.
2020-10-17 09:49:44 +02:00
Norman Maurer
cc79f5f4ff
Fix unit tests that sometimes failed due timeouts (#10698)
Motivation:

We had two unit tests that sometimes failed due timeouts. After insepecting these I noticed these can be improved to run faster while still do the right validation

Modifications:

- Only submit one task for execution per execute
- Cleanup

Result:

No test failures due timeout
2020-10-16 21:00:26 +02:00
Norman Maurer
6b613682ba
Ensure we don't leak the ClassLoader in the backtrace (#10691)
Motivation:

We have a few classes in which we store and reuse static instances of various exceptions. When doing so it is important to also override fillInStacktrace() and so prevent the leak of the ClassLoader in the internal backtrace field.

Modifications:

- Add overrides of fillInStracktrace when needed
- Move ThrowableUtil usage in the static methods

Result:

Fixes https://github.com/netty/netty/pull/10686
2020-10-15 20:41:29 +02:00
Chris Vest
fd8c1874b4
Fix #10614 by making UnorderedTPEExecutor.scheduleAtFixedRate run tasks more than once (#10659)
Motivation:
All scheduled executors should behave in accordance to their API.
The bug here is that scheduled tasks were not run more than once because we executed the runnables directly, instead of through the provided runnable future.

Modification:
We now run tasks through the provided future, so that when each run completes, the internal state of the task is reset and the ScheduledThreadPoolExecutor is informed of the completion.
This allows the executor to prepare the next run.

Result:
The UnorderedThreadPoolEventExecutor is now able to run scheduled tasks more than once.
Which is what one would expect from the API.
2020-10-14 11:09:16 +02:00
root
01b7e18632 [maven-release-plugin] prepare for next development iteration 2020-10-13 06:29:26 +00:00
root
d4a0050ef3 [maven-release-plugin] prepare release netty-4.1.53.Final 2020-10-13 06:29:02 +00:00
Aayush Atharva
9d457c3f0f
Add checkInRange in ObjectUtil (#10668)
Motivation:
We check lots of numbers if it lies in a range. So it's better to add a method in `ObjectUtil` to check if a number lies inside a range.

Modification:
Added Range check method.

Result:
A faster and better way to check if a number lies inside a range.
2020-10-12 18:25:58 +02:00
Francesco Nigro
69b44c6d06
Reduce DefaultAttributeMap lookup cost (#10530)
Motivation:

DefaultAttributeMap::attr has a blocking behaviour on lookup of an existing attribute:
it can be made non-blocking.

Modification:

Replace the existing fixed bucket table using a locked intrusive linked list
with an hand-rolled copy-on-write ordered single array

Result:
Non blocking behaviour for the lookup happy path
2020-10-02 18:24:35 +02:00
Aayush Atharva
836bb74051
Call long as l instead of i (#10578)
Motivation:

Long should be called `l` instead of `i` because `i` is generally used for `int`.

Modification:

Changed `i` to `l`.

Result:
Better naming
2020-09-16 07:58:03 +02:00
Francesco Nigro
162e59848a
Improve predictability of writeUtf8/writeAscii performance (#10368)
Motivation:

writeUtf8 can suffer from inlining issues and/or megamorphic call-sites on the hot path due to ByteBuf hierarchy

Modifications:

Duplicate and specialize the code paths to reduce the need of polymorphic calls

Result:

Performance are more stable in user code
2020-09-09 16:10:26 +02:00
root
957ef746d8 [maven-release-plugin] prepare for next development iteration 2020-09-08 05:26:25 +00:00
root
ada9c38c0a [maven-release-plugin] prepare release netty-4.1.52.Final 2020-09-08 05:26:05 +00:00
Aayush Atharva
fde6bc8885
Add ipv4AddressToInt(Inet4Address) in NetUtils (#10500)
Motivation:

We're converting `Inet4Address` to `Integer` quite frequently so it's a good idea to keep that code in `NetUtils`.

Modification:

Added ipv4AddressToInt(Inet4Address) in NetUtils

Result:
Easy conversion of  `Inet4Address` to `Integer`.
2020-08-31 09:13:14 +02:00
Norman Maurer
98c98d53d7
Don't try to remove the task when the underlying executor fails the execution of self (#10505)
Motivation:

It makes no sense to remove the task when the underlying executor fails as we may be able to pick it up later. Beside this the used Queue doesnt support remove(...) and so will throw.

Modifications:

Remove the queue.remove(...) call

Result:

Fixes https://github.com/netty/netty/issues/10501.
2020-08-27 08:22:41 +02:00
Chris Vest
cfeda0fef2
Fix overflow bug in GenericEventExecutorChooser (#10468)
Motivation:
 The executor chooser should pluck executors in round-robin, but at the 32-bit
 overflow boundary, the round-robin sequence was disrupted when the number of
 executors are not a power of 2.

Modification:
 Changed the index counter from a 32-bit to a 64-bit long. The overflow bug is
 still technically there, but it now takes so long to reach that it will never
 happen in practice. For example, 2^63 nanoseconds is almost 300 years.

Result:
 The round-robin behaviour for all EventExecutorChoosers is now preserved in
 practice.
 This fixes #10423.
2020-08-11 20:53:48 +02:00
Chris Vest
3d7ec896ac
Avoid name-clash with future java.lang.Record (#10470)
Motivation:
 Recent Intellij versions are starting to anticipate
 future versions of Java that include a
 `java.lang.Record` class, and the Intellij compiler
 gets confused by the `Record` class in our
 `ResorceLeakDetector`.

Modification:
 Rename our `Record` class to `TracerRecord`.
 This matches what the class is doing, while avoiding
 any future name clashes.

Result:
 Intellij can now compile the project again, even when
 configured to use a future (snapshot or early access)
 version of Java.
2020-08-11 20:51:39 +02:00
Norman Maurer
cd7552f551
Add whitelist entry to BlockHound config to workaround issue when TLS… (#10459)
Motivation:

SSLEngineImpl.unwrap(...) may call FileInputStream.read(...) internally when TLS1.3 is used. This will cause an BlockingOperationError when BlockHound is enabled.

For more details see https://mail.openjdk.java.net/pipermail/security-dev/2020-August/022271.html

Modifications:

- Add whitelist entry to BlockHound config for now
- Update NettyBlockHoundIntegrationTest to include testing for this workaround

Result:

No BlockingOperationError when TLS1.3 is used with JDK SSL implementation and BlockHound is enabled
2020-08-10 11:10:35 +02:00
Roman Puchkovskiy
0601389766
Do not try to use Unsafe.staticFieldOffset() method under a native image. (#10428)
Motivation:

GraalVM's native images built with native-image tool do not support Unsafe.staticFieldOffset() method (at least, currently). If an application using Netty (and causing initialization of io.netty.util.internal.PlatformDependent0 class) is built into a native image and run, this results in the following error thrown during initialization:

Exception in thread "main" com.oracle.svm.core.jdk.UnsupportedFeatureError: Unsupported method of Unsafe
	at com.oracle.svm.core.util.VMError.unsupportedFeature(VMError.java:86)
	at jdk.internal.misc.Unsafe.staticFieldOffset(Unsafe.java:230)
	at sun.misc.Unsafe.staticFieldOffset(Unsafe.java:662)
	at io.netty.util.internal.PlatformDependent0$5.run(PlatformDependent0.java:294)
	at java.security.AccessController.doPrivileged(AccessController.java:83)
	at io.netty.util.internal.PlatformDependent0.<clinit>(PlatformDependent0.java:279)

This seems to be the reason of the behavior described in #10051.

Modification:

The idea of this commit is to only invoke Unsafe.staticFieldOffset() is we are not in a native image; if we are, behave like if we could not find the field at all.

GraalDetector is borrowed from Spring framework.

Result:

Fixes #10051
2020-08-03 19:49:06 +02:00
skyguard1
9f89eb6429
Add IndexOutOfBoundsException error message (#10405)
Motivation:

We should provide details about why an IOOBE was thrown

Modification:

Add IndexOutOfBoundsException error information in io.netty.util.internal.AppendableCharSequence and io.netty.handler.codec.CodecOutputList class

Result:

Easier to debug
2020-07-16 11:37:03 +02:00
skyguard1
cd0203b5c7
Add default handling for switch statement (#10408)
Motivation:

When a switch statement is used we should always define a `default:` so we don't introduce bugs due fall-through.

Modification:

Add missing `default:`s

Result:

Less error-prone code
2020-07-16 10:54:50 +02:00
Gene
9da40ec311
Simple fix typo (#10403)
Motivation:

Wrong typo in annotation at line 925.

Modifications:

Fix typo. *then -> than.

Result:

Fix typo.
2020-07-14 10:59:02 +02:00
root
bfbeb2dec6 [maven-release-plugin] prepare for next development iteration 2020-07-09 12:27:06 +00:00
root
646934ef0a [maven-release-plugin] prepare release netty-4.1.51.Final 2020-07-09 12:26:30 +00:00
violetagg
9893ac726f
Do not report ReferenceCountedOpenSslClientContext$ExtendedTrustManagerVerifyCallback.verify as blocking call (#10387)
Motivation:

When BlockHound is installed,
ReferenceCountedOpenSslClientContext$ExtendedTrustManagerVerifyCallback.verify
is reported as blocking call.

Modifications:

Add allowBlockingCallsInside configuration for
ReferenceCountedOpenSslClientContext$ExtendedTrustManagerVerifyCallback.verify

Result:

Fixes #10384
2020-07-06 09:08:44 +02:00
skyguard1
523dc5c269
Fix #10378,ResourceLeakDetectorFactory.newResourceLeakDetector(Class, int) ignores  sampling interval (#10383)
Motivation:

newResourceLeakDetector(...) did not correctly pass the samplingInterval parameter and so it was ignored.

Modification:

ResourceLeakDetectorFactory.newResourceLeakDetector(Class, int) use the second parameter as the sampling interval of the newly created ResourceLeakDetector.

Result:

Fixes #10378
2020-07-01 10:24:41 +02:00
Francesco Nigro
eccb87b1f7
The bounded Mpsc Queue for no Unsafe users behave differently from Unsafe ones (#10377)
Motivation:

Unsafe users are getting MpscChunkedArrayQueue while no Unsafe ones MpscGrowableAtomicArrayQueue

Modifications:

MpscChunkedAtomicArrayQueue should be used for no Unsafe users (matching MpscChunkedArrayQueue behaviour)

Result:

no Unsafe users uses MpscChunkedAtomicArrayQueue while allocating bounded Mpsc Queues
2020-06-25 21:54:29 +02:00
离诌
4dc6764d7b
version: remove Duplicating managed version (#10329)
Motivation:

remove Duplicating managed version, cause it is already defined in the parent project.

Modification:

- origin 
```
<dependency>
      <groupId>io.netty</groupId>
      <artifactId>netty-dev-tools</artifactId>
      <scope>test</scope>
      <version>${project.version}</version>
      <optional>true</optional>
</dependency>

<plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>build-helper-maven-plugin</artifactId>
        <version>1.10</version>
</plugin>
```

- after modify

```
<dependency>
      <groupId>io.netty</groupId>
      <artifactId>netty-dev-tools</artifactId>
      <scope>test</scope>
      <optional>true</optional>
</dependency>

<plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>build-helper-maven-plugin</artifactId>
</plugin>
```

Result:

remove Duplicating managed version
2020-06-12 11:12:30 +02:00
louxiu
0375e6e01b
Fix very tiny comment error in Recycler (#10309)
Motivation:

Fix very tiny comment error in Recycler

Modifications:

Fix very tiny comment error in Recycler

Result:

Correctly comment about drop in WeakOrderQueue#add
2020-05-25 07:39:52 +02:00
root
caf51b7284 [maven-release-plugin] prepare for next development iteration 2020-05-13 06:00:23 +00:00
root
8c5b72aaf0 [maven-release-plugin] prepare release netty-4.1.50.Final 2020-05-13 05:59:55 +00:00
Piotr Betkier
09d38c87df
Add GlobalEventExecutor#addTask to BlockHound exceptions. (#10262)
Motivation:

GlobalEventExecutor#addTask may be called during SingleThreadEventExecutor shutdown.
May result in a blocking call, because GlobalEventExecutor#taskQueue is a BlockingQueue.

Modifications:

Add allowBlockingCallsInside configuration for GlobalEventExecutor#addTask.

Result:

Fixes #10257.
When BlockHound is installed, GlobalEventExecutor#addTask is not reported as a blocking call.
2020-05-11 08:51:38 +02:00
louxiu
909e7c9c29
Add option to configure recycler delayed queue drop ratio (#10251) (#10255)
Motivation

- Recycler stack and delayed queue drop ratio can only be configured with
the same value. The overall drop ratio is ratio^2.

- #10251 shows that enable drop in `WeakOrderQueue` may introduce
performance degradation. Though the final reason is not clear now,
it would be better to add option to configure delayed queue drop ratio separately.

Modification

- "io.netty.recycler.delayedQueue.ratio" as the drop ratio of delayed queue
- default "delayedQueue.ratio" is same as "ratio"

Results

Able to configure recycler delayed queue drop ratio separately
2020-05-07 15:08:35 +02:00
louxiu
4c9a30d5f9
Use io.netty.recycler.ratio directly (#10253)
Motivation

1. It's inable to collect all object because RATIO is always >=1 after
`safeFindNextPositivePowerOfTwo`

2. Enable drop object in `WeakOrderQueue`(commit:
71860e5b94) enlarge the drop ratio. We
can subtly control the overall drop ratio by using `io.netty.recycler.ratio` directly,

Modification

- Remove `safeFindNextPositivePowerOfTwo` before set the ratio

Results

Able to disable drop when recycle object
2020-05-07 10:29:05 +02:00
root
9c5008b109 [maven-release-plugin] prepare for next development iteration 2020-04-22 09:57:54 +00:00
root
d0ec961cce [maven-release-plugin] prepare release netty-4.1.49.Final 2020-04-22 09:57:26 +00:00
Linas Medžiūnas
fb5e2cd3aa
Efficient BytBuf search algorithms (#9914) (#9955)
Motivation:

We have found out that ByteBufUtil.indexOf can be inefficient for substring search on
ByteBuf, both in terms of algorithm complexity (worst case O(needle.readableBytes *
haystack.readableBytes)), and in constant factor (esp. on Composite buffers).
With implementation of more performant search algorithms we have seen improvements on
the order of magnitude.

Modifications:

This change introduces three search algorithms:
1. Knuth Morris Pratt - classical textbook algorithm, a good default choice.
2. Bit mask based algorithm - stable performance on any input, but limited to maximum
search substring (the needle) length of 64 bytes.
3. Aho–Corasick - worse performance and higher memory consumption than [1] and [2], but
it supports multiple substring (the needles) search simultaneously, by inspecting every
byte of the haystack only once.

Each algorithm processes every byte of underlying buffer only once, they are implemented
as ByteProcessor.

Result:

Efficient search algorithms with linear time complexity available in Netty (I will share
benchmark results in a comment on a PR).
2020-04-15 10:21:24 +02:00
Cenjie Ho
363c4ecc7f
Add the missing comment (#10169)
Motivation:

The comment of SingleThreadEventExecutor#run() is missing

Modification:

Just fill it in.

Result:

Method comment is not blank now.
2020-04-06 09:05:29 +02:00
Norman Maurer
4b235a9600
Introduce DomainWildcardMappingBuilder to fix wildcard matching accor… (#10132)
Motivation:

How we did wildcard matching was not correct according to RFC6125. Beside this our implementation was quite CPU heavy.

Modifications:

- Add new DomainWildcardMappingBuilder which correctly does wildcard matching. See https://tools.ietf.org/search/rfc6125#section-6.4
- Add unit tests
- Deprecate old implementations

Result:

Correctly implement wildcard matching and improve performance
2020-03-31 16:57:42 +02:00
root
14e4afeba2 [maven-release-plugin] prepare for next development iteration 2020-03-17 09:20:54 +00:00
root
c10c697e5b [maven-release-plugin] prepare release netty-4.1.48.Final 2020-03-17 09:18:28 +00:00
root
c623a50d19 [maven-release-plugin] prepare for next development iteration 2020-03-09 12:13:56 +00:00
root
a401b2ac92 [maven-release-plugin] prepare release netty-4.1.47.Final 2020-03-09 12:13:26 +00:00
feijermu
8df19ce1f3
Replace several magic numbers. (#10094)
Motivation:

Magic numbers seem hard to read or understand.

Modification:

Replace several magic numbers with named constants.

Result:

Improve readability and make it easier to maintain.
2020-03-09 11:48:59 +01:00
feijermu
b892722efb
Add a log level check simply before logging. (#10093)
Motivation:

ThrowableUtil.stackTraceToString is an expensive method call. So I think a log level check before this logging statement is quite needed especially in a environment with the warning log disabled.

Modification:

Add log level check simply before logging.

Result:

Improve performance in a environment with the warning log disabled.
2020-03-09 08:47:53 +01:00
Norman Maurer
15fa45a84b
Add log level check simply before logging. (#10080)
Motivation:

In general, we will close the debug log in a product environment. However, logging without external level check may still affect performance as varargs will need to allocate an array.

Modification:

Add log level check simply before logging.

Result:

Improve performance slightly in a product environment.
2020-03-05 14:38:57 +01:00
djelinski
1b0e3d95f4
Fix AssertionError in ScheduledFutureTask (#10073)
Motivation:

Some JVMs (like OpenJDK 8 on Windows) use a low resolution timer in System.nanoTime() and may return the same value more than once. This triggered an assertion failure when deadlineNanos was equal to nanoTime and AbstractScheduledEventExecutor#pollScheduledTask called #setConsumed.

Modifications: 
With this change the assertion checks exactly the same condition as AbstractScheduledEventExecutor#pollScheduledTask, and will no longer fail under these circumstances.

Result:

Fixes #10070.
2020-03-03 14:34:51 +01:00
feijermu
7ab68adbe8
Add test cases for StringUtil. (#10074)
Motivation:

StringUtil needs more test cases.

Modification:

Add several test cases for StringUtil.

Result:

Improve test coverage slightly.
2020-03-03 10:57:41 +01:00
feijermu
0453222015
Add test cases for MathUtil. (#10071)
Motivation:

MathUtil needs more test cases.

Modification:

Add several test cases for MathUtil.

Result:

Improve test coverage slightly.
2020-03-02 08:38:45 +01:00
root
e0d73bca4d [maven-release-plugin] prepare for next development iteration 2020-02-28 06:37:33 +00:00
root
ebe7af5102 [maven-release-plugin] prepare release netty-4.1.46.Final 2020-02-28 06:36:45 +00:00
Norman Maurer
c9f38f855e
PromiseTask.isCancelled performs an unsynchronized read (#10066)
Motivation:

PromiseTask.isCancelled performs an unsynchronized read and so may trigger race-detectors. As this was just done for a small optimization which should not really make a lot of difference we should just remove it.

Modifications:

Remove unsynchronized read optimization

Result:

Fixes https://github.com/netty/netty/issues/10026.
2020-02-28 07:06:02 +01:00
feijermu
f451295c75
Add test cases for ImmediateExecutor. (#10060)
Motivation:

ImmediateExecutor needs more test cases.

Modification:

Add several test cases for ImmediateExecutor.

Result:

Improve test coverage slightly.
2020-02-26 11:08:36 +01:00
Norman Maurer
9ae782d632
More strict parsing of initial line / http headers (#10058)
Motivation:

Our parsing of the initial line / http headers did treat some characters as separators which should better trigger an exception during parsing.

Modifications:

- Tighten up parsing of the inital line by follow recommentation of RFC7230
- Restrict separators to OWS for http headers
- Add unit test

Result:

Stricter parsing of HTTP1
2020-02-26 09:49:39 +01:00
Norman Maurer
880e1239c5
NetworkInterface.getByInetAddress() may return null on Android platform (#10056)
Motivation:

NetworkInterface.getByInetAddress() may return null on Android. This is incorrect by the API but still happens. To help our users we should provide a workaround

Modifications:

Just return an empty Enumeration when null is returned.

Result:

Fixes https://github.com/netty/netty/issues/10045
2020-02-25 09:34:38 +01:00
feijermu
e648aa259d
Add a null check to NetUtil. sysctlGetInt(...) (#10027)
Motivation:

BufferedReader.readLine() may return null and cause a NPE.

Modification:

Simply add a null check.

Result:

If BufferedReader.readLine() returns null, the sysctlGetInt will just return null rather than cause NPE.
2020-02-14 09:08:27 +01:00
Norman Maurer
713e0aa68a
java.security.AccessControlException: access denied ("java.io.FilePermission" "/etc/os-release" "read") (#10018)
Motivation:

Modifications:

- Wrap the code and execute with an AccessController
- Ignore SecurityException (by just logging it)
- Add some more debug logging

Result:

Fixes https://github.com/netty/netty/issues/10017
2020-02-13 11:48:40 +01:00
violetagg
fcf55fcf71
When BlockHound is installed, do not report GlobalEventExecutor/SingleThreadEventExecutor#takeTask as blocking call. (#10020)
Motivation:

GlobalEventExecutor/SingleThreadEventExecutor#taskQueue is BlockingQueue.

Modifications:

Add allowBlockingCallsInside configuration for GlobalEventExecutor/SingleThreadEventExecutor#takeTask.

Result:

Fixes #9984
When BlockHound is installed, GlobalEventExecutor/SingleThreadEventExecutor#takeTask is not reported as a blocking call.
2020-02-11 20:24:41 +01:00
Ruwei
6e5f229589
fix bug: scheduled tasks may not be executed (#9980)
Motivation:

If there was always a task in the taskQueue of GlobalEvenExecutor, scheduled tasks in the
scheduledTaskQueue will never be executed.

Related to  #1614

Modifications:

fix bug in GlobalEventExecutor#takeTask

Result:

fix bug
2020-01-31 10:57:38 +01:00
Johno Crawford
0671b18e24
SSL / BlockHound works out of the box with the default SSL provider (#9969)
Motivation:

JDK is the default SSL provider and internally uses blocking IO operations.

Modifications:

Add allowBlockingCallsInside configuration for SslHandler runAllDelegate function.

Result:

When BlockHound is installed, SSL works out of the box with the default SSL provider.


Co-authored-by: violetagg <milesg78@gmail.com>
2020-01-30 11:35:16 +01:00
Norman Maurer
eb50e5148a
Initialize ThreadLocalRandom at runtime to improve GraalVM support (#9977)
Motivation:

We need to initialize ThreadLocalRandom at runtime as it uses System.nanoTime() in a static block to init the seed.

Modifications:

Add io.netty.util.internal.ThreadLocalRandom to properties file

Result:

Better support for GraalVM
2020-01-29 15:00:12 +01:00
Iván López
066a180a43 Initialize some classes at runtime to improve GraalVM support (#9963)
Motivation:

Deploying a Micronaut application as GraalVM native image to AWS Lambda with custom runtime fails when using Micronaut Http Client.

This PR initializes at runtime some classes needed to fix the issue. There is more information in our original issue in Micronaut https://github.com/micronaut-projects/micronaut-core/issues/2335#issuecomment-570151944

At this moment I've added those classes into Micronaut (b383d3ab14) as a workaround but this should be included in Netty so it's available for everyone.

Modification:

Mark 3 classes to be initialized at runtime for GraalVM.

Result:

Mark 3 classes to be initialized at runtime for GraalVM.
2020-01-24 06:40:39 -08:00
root
9b1ea10a12 [maven-release-plugin] prepare for next development iteration 2020-01-13 09:13:53 +00:00
root
136db8680a [maven-release-plugin] prepare release netty-4.1.45.Final 2020-01-13 09:13:30 +00:00
时无两丶
b82258b72f Introduce needReport for ResourceLeakDetector. (#9910)
Motivation:

We can extend `ResourceLeakDetector` through `ResourceLeakDetectorFactory`, and then report the leaked information by covering `reportTracedLeak` and `reportUntracedLeak`. However, the behavior of `reportTracedLeak` and `reportUntracedLeak` is controlled by `logger.isErrorEnabled()`, which is not reasonable. In the case of extending `ResourceLeakDetector`, we sometimes need `needReport` to always return true instead of relying on `logger.isErrorEnabled ()`.

Modification:

introduce `needReport` method and let it be `protected`

Result:

We can control the report leak behavior.
2020-01-10 05:21:24 +01:00
Francesco Nigro
bc026ef8ba Faster decodeHexNibble (#9896)
Motivation:

decodeHexNibble can be a lot faster using a lookup table

Modifications:

decodeHexNibble is made faster by using a lookup table

Result:

decodeHexNibble is faster
2019-12-23 21:15:56 +01:00
Ikhun Um
7f241f1f3c Fix typos in javadocs (#9900)
Motivation:

Javadocs should have no typos.

Modifications:

Fix two typos

Result:

Less typos.
2019-12-23 08:34:16 +01:00
Norman Maurer
68cfab472e
Ignore inline comments when parsing nameservers (#9894)
Motivation:

The resolv.conf file may contain inline comments which should be ignored

Modifications:

- Detect if we have a comment after the ipaddress and if so skip it
- Add unit test

Result:

Fixes https://github.com/netty/netty/issues/9889
2019-12-18 21:07:23 +01:00
root
79d4e74019 [maven-release-plugin] prepare for next development iteration 2019-12-18 08:32:54 +00:00
root
5ddf45a2d5 [maven-release-plugin] prepare release netty-4.1.44.Final 2019-12-18 08:31:43 +00:00
Robert Mihaly
ed16da2e84 Ensure scheduled tasks are executed before shutdown (#9858)
Motivation:

In #9603 the executor hung on shutdown because of an abandoned task
on another executor the first was waiting for.

Modifications:

This commit modifies the executor shutdown sequence to include
switching to SHUTDOWN state and then running all remaining tasks.
This ensures that no more tasks are scheduled after SHUTDOWN and
the last pass of running remaining tasks will take it all.
Any tasks scheduled after SHUTDOWN will be rejected.

This change preserves the functionality of graceful shutdown with
quiet period and only adds one more pass of task execution after
the default shutdown process has finished and the executor is
ready for termination.

Result:

After this change tasks that succeed to be added to the executor will
be always executed. Tasks which come late will be rejected instead of
abandoned.
2019-12-11 10:48:38 +01:00
时无两丶
0cde4d9cb4 Uniform null pointer check. (#9840)
Motivation:
Uniform null pointer check.

Modifications:

Use ObjectUtil.checkNonNull(...)

Result:
Less code, same result.
2019-12-09 09:47:35 +01:00
Norman Maurer
385a17b103
Include JCTools sources for shaded classes in the sources jar (#9838)
Motivation:

We should include the shaded sources for JCTools in our sources jar to make it easier to debug.

Modifications:

- Adjust plugin configuration to execute plugins in correct order
- Update source plugin
- Add configuration for shade plugin to generate source jar content

Result:

Fixes https://github.com/netty/netty/issues/6640.
2019-12-05 09:10:32 +01:00
时无两丶
df121e5e55 Replace map with set. (#9833)
Motivation:
Replace Map with Set. `reportedLeaks` has better semantics as a Set, and if it is a Map, it seems that the value of this Map has no meaning to us.

Modifications:

Use Set.

Result:
Cleaner code
2019-12-04 13:53:11 +01:00
Nick Hill
e208e96f12 Clean up NioEventLoop (#9799)
Motivation

The event loop implementations had become somewhat tangled over time and
work was done recently to streamline EpollEventLoop. NioEventLoop would
benefit from the same treatment and it is more straighforward now that
we can follow the same structure as was done for epoll.

Modifications

Untangle NioEventLoop logic and mirror what's now done in EpollEventLoop
w.r.t. the volatile selector wake-up guard and scheduled task deadline
handling.

Some common refinements to EpollEventLoop have also been included - to
use constants for the "special" deadline/wakeup volatile values and to
avoid some unnecessary calls to System.nanoTime() on task-only
iterations.

Result

Hopefully cleaner, more efficient and less fragile NIO transport
implementation.
2019-11-26 08:25:59 +01:00
Norman Maurer
804c33ef46
Cleanup Recycler to better encapsulate stuff (#9739)
Motivation:

We can move some methods etc to make encapsulation better in Recycler

Modifications:

Move / rename methods to make usage more clear

Result:

Code cleanup
2019-11-04 17:43:53 +01:00
Nick Hill
feb804dca8 Avoid extra Runnable allocs when scheduling tasks outside event loop (#9744)
Motivation

Currently when future tasks are scheduled via EventExecutors from a
different thread, at least two allocations are performed - the
ScheduledFutureTask wrapping the to-be-run task, and a Runnable wrapping
the action to add to the scheduled task priority queue. The latter can
be avoided by incorporating this logic into the former.

Modification

- When scheduling or cancelling a future task from outside the event
loop, enqueue the task itself rather than wrapping in a Runnable
- Have ScheduledFutureTask#run first verify the task's deadline has
passed and if not add or remove it from the scheduledTaskQueue depending
on its cancellation state
- Add new outside-event-loop benchmarks to ScheduleFutureTaskBenchmark

Result

Fewer allocations when scheduling/cancelling future tasks
2019-11-04 11:57:53 +01:00
Nick Hill
17bffce90e Use interval instead of mask comparison for Recycler ratio (#9748)
Motivation

The recycling ratio is currently implemented by comparing with a masked
count. The mask operation is not free and also not necessary.

Modification

Change the count(s) to just iterate over the corresponding interval,
which requires only a comparison and no mask.

Also make "first time recycle" behaviour consistent and revert change to
RecyclerTest made in #9727.

Result

Less recycling overhead
2019-11-04 11:24:20 +01:00
Norman Maurer
71860e5b94
Enforce ratioMask also for WeakOrderQueue (#9727)
Motivation:

At the moment we only enfore ratioMask for the Stack which means that we only guard against recycle burts when recycled from the same Thread. We should also enforce the ratioMask in the WeakOrderQueue so we also guard against the bursts when recycle from other threads.

Modifications:

- Keep counter in WeakOrderQueue to enforce ratioMask as well
- Adjust unit test

Result:

Better guard against recycle bursts which could pollute the heap unnecessary.
2019-11-01 07:10:42 +01:00
Nick Hill
fb0d34a2f9 Avoid synthetic methods in Recycler (#9736)
Motivation

Currently the visibility of the various Recycler inner classes and their
fields isn't optimal. Some private members are accessed by other classes
resulting in synthetic methods, and other non-private classes/members
are only accessed privately and so can be made private.

Modifications

- Increase/reduce visibility of various fields/methods/classes within
Recycler
- Have WeakOrderQueue extend WeakReference<Thread> to eliminate the
owner field
- Change local DefaultHandle var to DefaultHandle<?> to avoid raw type
compiler warning

Result

Tidier code, fewer implicit methods on hot paths (reducing inlining
depths)
2019-10-31 11:22:05 +01:00
Nick Hill
a18c57ea87 Externalize lazy execution semantic for EventExecutors (#9587)
Motivation

This is already done internally for various reasons but it would make
sense i.m.o. as a top level concept: submitting a task to be run on the
event loop which doesn't need to run immediately but must still be
executed in FIFO order relative all other submitted tasks (be those
"lazy" or otherwise).

It's nice to separate this abstract "relaxed" semantic from concrete
implementations - the simplest is to just delegate to existing execute,
but for the main EL impls translates to whether a wakeup is required
after enqueuing.

Having a "global" abstraction also allows for simplification of our
internal use - for example encapsulating more of the common scheduled
future logic within AbstractScheduledEventExecutor.

Modifications

- Introduce public LazyRunnable interface and
AbstractEventExecutor#lazyExecute method (would be nice for this to be
added to EventExecutor interface in netty 5)
- Tweak existing SingleThreadEventExecutor mechanics to support these
- Replace internal use of NonWakeupRunnable (such as for pre-flush
channel writes)
- Uplift scheduling-related hooks into AbstractScheduledEventExecutor,
eliminating intermediate executeScheduledRunnable method

Result

Simpler code, cleaner and more useful/flexible abstractions - cleaner in
that they fully communicate the intent in a more general way, without
implying/exposing/restricting implementation details
2019-10-31 10:01:53 +01:00
Norman Maurer
1d57241565
Remove usage of finalizer in Recycler (#9726)
Motivation:

We currently use a finalizer to ensure we correctly return the reserved back to the Stack but this is not really needed as we can ensure we return it when needed before dropping the WeakOrderQueue

Modifications:

Use explicit method call to ensure we return the reserved space back before dropping the object

Result:

Less finalizer usage and so less work for the GC
2019-10-31 09:58:23 +01:00
Norman Maurer
7c85c9ea0b
Correctly update size of the Stack before doing any validation in Recycler (#9731)
Motivation:

We null out the element in the array after we decrement the current size of the Stack but not directly write back the updated size to the stored field. This is problematic as we do some validation before we write it back and so may never do so if the validation fails. This then later can lead to have null objects returned where not expected

Modifications:

Update size directly after null out object

Result:

No more unexpected null value possible
2019-10-31 09:52:07 +01:00
Bennett Lynch
9976ab7fe8 Prefer Log4J2 over Log4J1 for default InternalLoggerFactory (#9734)
##Motivation

The InternalLoggerFactory attempts to instantiate different logger
implementations to discover what is available on the class path,
accepting the first implementation that does not throw an exception.

Currently, the default ordering will attempt to instantiate a Log4j1
logger before Log4j2. For environments where both Log4j1 and Log4j2 are
available, this will result in using the older version. It seems that it
would be more intuitive to prefer the newer version, when possible.

##Modifications

Change the default ordering to attempt to use the Log4J2LoggerFactory
before the Log4JLoggerFactory.

##Result

For environments where both Log4j1 and Log4j2 are available on the class
path (but Slf4J is not available), Netty will now use Log4j2 instead of
Log4j1.
2019-10-30 19:35:28 +01:00