Commit Graph

1119 Commits

Author SHA1 Message Date
kezhenxu94
7b6336f1fd Java 8 Migration: remove uneccessary if statement (#8755)
Motivation:

As netty 4.x supported Java 6 we had various if statements to check for java versions < 8. We can remove these now.

Modification:

Remove unnecessary if statements that check for java versions < 8.

Result:

Cleanup code.
2019-01-25 08:57:11 +01:00
Norman Maurer
310f31b392
Update to new checkstyle plugin (#8777)
Motivation:

We need to update to a new checkstyle plugin to allow the usage of lambdas.

Modifications:

- Update to new plugin version.
- Fix checkstyle problems.

Result:

Be able to use checkstyle plugin which supports new Java syntax.
2019-01-24 16:24:19 +01:00
Dmitriy Dumanskiy
9e9ee2ecd0 Java 8 migration. Join similar catch blocks (#8767)
Motivation:

We can now use multi-catch blocks.

Modification:

Join similar catch blocks + small cleanup.

Result:

Cleaner code.
2019-01-23 13:03:07 +01:00
Norman Maurer
3d6e6136a9
Decouple EventLoop details from the IO handling for each transport to… (#8680)
* Decouble EventLoop details from the IO handling for each transport to allow easy re-use of code and customization

Motiviation:

As today extending EventLoop implementations to add custom logic / metrics / instrumentations is only possible in a very limited way if at all. This is due the fact that most implementations are final or even package-private. That said even if these would be public there are the ability to do something useful with these is very limited as the IO processing and task processing are very tightly coupled. All of the mentioned things are a big pain point in netty 4.x and need improvement.

Modifications:

This changeset decoubled the IO processing logic from the task processing logic for the main transport (NIO, Epoll, KQueue) by introducing the concept of an IoHandler. The IoHandler itself is responsible to wait for IO readiness and process these IO events. The execution of the IoHandler itself is done by the SingleThreadEventLoop as part of its EventLoop processing. This allows to use the same EventLoopGroup (MultiThreadEventLoupGroup) for all the mentioned transports by just specify a different IoHandlerFactory during construction.

Beside this core API change this changeset also allows to easily extend SingleThreadEventExecutor / SingleThreadEventLoop to add custom logic to it which then can be reused by all the transports. The ideas are very similar to what is provided by ScheduledThreadPoolExecutor (that is part of the JDK). This allows for example things like:

  * Adding instrumentation / metrics:
    * how many Channels are registered on an SingleThreadEventLoop
    * how many Channels were handled during the IO processing in an EventLoop run
    * how many task were handled during the last EventLoop / EventExecutor run
    * how many outstanding tasks we have
    ...
    ...
  * Implementing custom strategies for choosing the next EventExecutor / EventLoop to use based on these metrics.
  * Use different Promise / Future / ScheduledFuture implementations
  * decorate Runnable / Callables when submitted to the EventExecutor / EventLoop

As a lot of functionalities are folded into the MultiThreadEventLoopGroup and SingleThreadEventLoopGroup this changeset also removes:

  * AbstractEventLoop
  * AbstractEventLoopGroup
  * EventExecutorChooser
  * EventExecutorChooserFactory
  * DefaultEventLoopGroup
  * DefaultEventExecutor
  * DefaultEventExecutorGroup

Result:

Fixes https://github.com/netty/netty/issues/8514 .
2019-01-23 08:32:05 +01:00
Dmitriy Dumanskiy
7b92ff2500 Java 8 migration. Remove ThreadLocalProvider and inline java.util.concurrent.ThreadLocalRandom.current() where necessary. (#8762)
Motivation:

Custom Netty ThreadLocalRandom and ThreadLocalRandomProvider classes are no longer needed and can be removed.

Modification:

Remove own ThreadLocalRandom

Result:

Less code to maintain
2019-01-22 20:14:28 +01:00
Dmitriy Dumanskiy
32d96a7f79 Java 8 migration. Similar catch blocks joined (#8759)
Motivation:

Avoid IDE warnings, easier to read.

Modification:

Same catch blocks joined.

Result:

Cleanup
2019-01-22 18:00:10 +01:00
Dmitriy Dumanskiy
42376c052a Java 8 migration. Inline PlatformDependent.newConcurrentHashMap() (#8760)
Motivation:

PlatformDependent.newConcurrentHashMap() is no longer needed so it could be easily removed and new ConcurrentHashMap<>() inlined instead of invoking PlatformDependent.newConcurrentHashMap().

Modification:

Use ConcurrentHashMap provided by the JDK directly.

Result:

Less code to maintain.
2019-01-22 17:18:50 +01:00
Dmitriy Dumanskiy
afc03da93b Java 8 migration. Use reference to StandardCharsets in CharsetUtil. (#8757)
Motivation:

No need to initialize charsets from the string. We can take already allocated charset from StandardCharsets class.

Modification:

Replace Charset.forName("US-ASCII") with StandardCharsets.US_ASCII.
Removed Charset[] values() method and internal static variable as it was used only in tests.

Result:

Reuse what the JDK provides
2019-01-22 16:50:36 +01:00
Oleksii Kachaiev
b3500f50a6 Java 8 migration. Replace Collection.newSetFromMap with ConcurrentHashMap.newKeySet (#8756)
Motivation:

Simplify code with newer Java API. It also reduce levels of indirection, but I don't
think we can gain any visible performance improvements out of this.

Modifications:

Replace Collection.newSetFromMap(new ConcurrentHashMap()) with ConcurrentHashMap.newKeySet().

Result:

Use Java8 APIs.
2019-01-22 16:41:15 +01:00
田欧
9d62deeb6f Java 8 migration: Use diamond operator (#8749)
Motivation:

We can use the diamond operator these days.

Modification:

Use diamond operator whenever possible.

Result:

More modern code and less boiler-plate.
2019-01-22 16:07:26 +01:00
Dmitriy Dumanskiy
c34340fff8 Java 8 migration. Auto close for try catch blocks (#8752)
Motivation:

Since Java 7 we can automatically close resources in try () construction.

Modification:

Changed all try catches in the code with autoclose try (resource)

Result:

Less boiler-plate
2019-01-22 15:57:30 +01:00
Dmitriy Dumanskiy
b2a9a8e9fe Java 8 migration. Removed custom LongCounter and LongAdderCounter classes and replaced with direct reference to LongAdder (#8750)
Motivation:

After switching to Java 8 we no longer need LongCounter and LongAdderCounter. We can directly replace them with LongAdder.

Modification:

Removed LongCounter and LongAdderCounter classes.

Result:

Less code to maintain
2019-01-22 13:53:28 +01:00
Dmitriy Dumanskiy
82b0db5013 Java 8 migration. Removed custom MathUtil.compare methods (#8748)
Motivation:

Netty uses own Integer.compare and Long.compare methods. Since Java 7 we can use Java implementation instead.

Modification:

Remove own implementation

Result:

Less code to maintain
2019-01-22 12:37:09 +01:00
Dmitriy Dumanskiy
91e7e3d8c9 Not used IntegerHolder class removed (#8747)
Motivation:

IntergerHolder was left from some old days and not used anymore.

Modification:

Remove IntergerHolder from the codebase.

Result:

Removed unused code
2019-01-22 11:25:08 +01:00
Dmitriy Dumanskiy
45c0ea543f Java 8 migration. Replace netty ConcurrentSet with Java KeySet. (#8745)
Motivation:

The concurrent set is present in Java 8 and above so we can use it instead of own implementation.

Modification:

io.netty.utik.internal.ConcurrentSet replaced with ConcurrentHashMap.newKeySet().

Result:

Less code to maintain.
2019-01-22 10:40:40 +01:00
kezhenxu94
2df8bca8da fix typo (#8741)
Motivation:

Correct typo

Modification:

Correct typo

Result:

JavaDoc and method name are more readable
2019-01-22 08:51:54 +01:00
kezhenxu94
ae1340fd72 extract duplicate code into method (#8720)
Motivation:

Clean up code to increase readability.

Modification:

Extract duplicate code blocks into method.

Result:

Less code duplication
2019-01-16 10:56:36 +01:00
Derek Lewis
f08e80ffd6 Remove unnecessary loop variable from AsciiString. (#8711)
Motivation:

Incrementing two variables in sync is not necessary when only one will do.

Modifications:

- Remove `j` from `for` loop and replace with `i`.
- Add more unit testing scenarios to cover changed code.

Results:

Unnecessary variable removed.
2019-01-15 08:33:48 +01:00
kashike
c0aa1ea5c7 Fix minor spelling issues in javadocs (#8701)
Motivation:

Javadocs contained some spelling errors, we should fix these.

Modification:

Fix spelling

Result:

Javadoc cleanup.
2019-01-14 07:25:13 +01:00
Norman Maurer
71430d8bbd Call FastThreadLocal.removeAll() before notify termination future of … (#8666)
Motivation:

We should try removing all FastThreadLocals for the Thread before we notify the termination. future. The user may block on the future and once it unblocks the JVM may terminate and start unloading classes.

Modifications:

Remove all FastThreadLocals for the Thread before notify termination future.

Result:

Fixes https://github.com/netty/netty/issues/6596.
2018-12-21 11:11:05 +01:00
Norman Maurer
7f22743d92
Revert "Ability to run a task at the end of an eventloop iteration." (#8637)
Motivation:

executeAfterEventLoopIteration is an Unstable API and isnt used in Netty. We should remove it to reduce complexity.

Changes:

This reverts commit 77770374fb.

Result:

Simplify implementation / cleanup.
2018-12-12 10:37:07 +01:00
Feri73
563793688f Adding support for whitespace in resource path in tests (#8606)
Motivation:

In windows if the project is in a path that contains whitespace,
resources cannot be accessed and tests fail.

Modifications:

Adds ResourcesUtil.java in netty-common. Tests use ResourcesUtil.java to access a resource.

Result:

Being able to build netty in a path containing whitespace
2018-12-12 10:29:19 +01:00
Nick Hill
fc0a4e15ab Harden ref-counting concurrency semantics (#8583)
Motivation

#8563 highlighted race conditions introduced by the prior optimistic
update optimization in 83a19d5650. These
were known at the time but considered acceptable given the perf
benefit in high contention scenarios.

This PR proposes a modified approach which provides roughly half the
gains but stronger concurrency semantics. Race conditions still exist
but their scope is narrowed to much less likely cases (releases
coinciding with retain overflow), and even in those
cases certain guarantees are still assured. Once release() returns true,
all subsequent release/retains are guaranteed to throw, and in
particular deallocate will be called at most once.

Modifications

- Use even numbers internally (including -ve) for live refcounts
- "Final" release changes to odd number (equivalent to refcount 0)
- Retain still uses faster getAndAdd, release uses CAS loop
- First CAS attempt uses non-volatile read
- Thread.yield() after a failed CAS provides a net gain

Result

More (though not completely) robust concurrency semantics for ref
counting; increased latency under high contention, but still roughly
twice as fast as the original logic. Bench results to follow
2018-11-29 08:32:50 +01:00
Norman Maurer
cb1090653f LocationAwareSlf4jLogger does not correctly format log message. (#8595)
Motivation:

We did miss to use MessageFormatter inside LocationAwareSlf4jLogger and so {} was not correctly replaced in log messages when using slf4j.
This regression was introduced by afe0767e9c.

Modifications:

- Make use of MessageFormatter
- Add unit test.

Result:

Fixes https://github.com/netty/netty/issues/8483.
2018-11-27 11:45:11 +01:00
Norman Maurer
2e0922d773 Use addAndGet(...) as a replacement for compareAndSet(...) when tracking the direct memory usage. (#8596)
Motivation:

We can change from using compareAndSet to addAndGet, which emits a different CPU instruction on x86 (CMPXCHG to XADD) when count direct memory usage. This instruction is cheaper in general and so produce less overhead on the "happy path". If we detect too much memory usage we just rollback the change before throwing the Error.

Modifications:

Replace compareAndSet(...) with addAndGet(...)

Result:

Less overhead when tracking direct memory.
2018-11-27 08:38:01 +01:00
Norman Maurer
2c78dde749 Update version number to start working on Netty 5 2018-11-20 15:49:57 +01:00
Jake Luciani
63dc1f5aaa Allow adjusting of lead detection sampling interval. (#8568)
Motivation:

We should allow adjustment of the leak detecting sampling interval when in SAMPLE mode.

Modifications:

Added new int property io.netty.leakDetection.samplingInterval

Result:

Be able to consume changes made by the user.
2018-11-16 17:22:03 +01:00
Norman Maurer
845a65b31c
Nio|Epoll|KqueueEventLoop task execution might throw UnsupportedOperationException on shutdown. (#8476)
Motivation:

There is a racy UnsupportedOperationException instead because the task removal is delegated to MpscChunkedArrayQueue that does not support removal. This happens with SingleThreadEventExecutor that overrides the newTaskQueue to return an MPSC queue instead of the LinkedBlockingQueue returned by the base class such as NioEventLoop, EpollEventLoop and KQueueEventLoop.

Modifications:

- Catch the UnsupportedOperationException
- Add unit test.

Result:

Fix #8475
2018-11-15 07:19:28 +01:00
时无两丶
28f9136824 Replace ConcurrentHashMap at allLeaks with a thread-safe set (#8467)
Motivation:
allLeaks is to store the DefaultResourceLeak. When we actually use it, the key is DefaultResourceLeak, and the value is actually a meaningless value.
We only care about the keys of allLeaks and don't care about the values. So Set is more in line with this scenario.
Using Set as a container is more consistent with the definition of a container than Map.

Modification:

Replace allLeaks with set. Create a thread-safe set using 'Collections.newSetFromMap(new ConcurrentHashMap<DefaultResourceLeak<?>, Boolean>()).'
2018-11-06 11:21:56 +01:00
Norman Maurer
bde2865ef8
Make it clear that HashedWheelTimer only support millis. (#8322)
Motivation:

HWT does not support anything smaller then 1ms so we should make it clear that this is the case.

Modifications:

Log a warning if < 1ms is used.

Result:

Less suprising behaviour.
2018-11-02 08:10:18 +01:00
Norman Maurer
d533befa96
PlatformDependent.maxDirectMemory() must respect io.netty.maxDirectMemory (#8452)
Motivation:

In netty we use our own max direct memory limit that can be adjusted by io.netty.maxDirectMemory but we do not take this in acount when maxDirectMemory() is used. That will lead to non optimal configuration of PooledByteBufAllocator in some cases.

This came up on stackoverflow:
https://stackoverflow.com/questions/53097133/why-is-default-num-direct-arena-derived-from-platformdependent-maxdirectmemory

Modifications:

Correctly respect io.netty.maxDirectMemory and so configure PooledByteBufAllocator correctly by default.

Result:

Correct value for max direct memory.
2018-11-02 07:19:43 +01:00
root
3e7ddb36c7 [maven-release-plugin] prepare for next development iteration 2018-10-29 15:38:51 +00:00
root
9e50739601 [maven-release-plugin] prepare release netty-4.1.31.Final 2018-10-29 15:37:47 +00:00
Nick Hill
d7fa7be67f Exploit PlatformDependent.allocateUninitializedArray() in more places (#8393)
Motivation:

There are currently many more places where this could be used which were
possibly not considered when the method was added.

If https://github.com/netty/netty/pull/8388 is included in its current
form, a number of these places could additionally make use of the same
BYTE_ARRAYS threadlocal.

There's also a couple of adjacent places where an optimistically-pooled
heap buffer is used for temp byte storage which could use the
threadlocal too in preference to allocating a temp heap bytebuf wrapper.
For example
https://github.com/netty/netty/blob/4.1/buffer/src/main/java/io/netty/buffer/ByteBufUtil.java#L1417.

Modifications:

Replace new byte[] with PlatformDependent.allocateUninitializedArray()
where appropriate; make use of ByteBufUtil.getBytes() in some places
which currently perform the equivalent logic, including avoiding copy of
backing array if possible (although would be rare).

Result:

Further potential speed-up with java9+ and appropriate compile flags.
Many of these places could be on latency-sensitive code paths.
2018-10-27 10:43:28 -05:00
almson
1cc692dd7d Fix incorrect reachability assumption in ResourceLeakDetector (#8410)
Motivation:

trackedObject != null gives no guarantee that trackedObject remains reachable. This may cause problems related to premature finalization: false leak detector warnings.
 
Modifications:

Add private method reachabilityFence0 that works on JDK 8 and can be factored out into PlatformDependent. Later, it can be swapped for the real Reference.reachabilityFence.
 
Result:

No false leak detector warnings in future versions of JDK.
2018-10-24 22:15:13 +02:00
almson
fc35e20e2c Include correct duped value in DefaultResourceLeak.toString() (#8413)
Motivation:

DefaultResourceLeak.toString() did include the wrong value for duplicated records.

Modifications:

Include the correct value.

Result:

Correct toString() implementation.
2018-10-22 15:01:38 +02:00
Dmitriy Dumanskiy
b59336142f deprecate own ConcurrentSet for removal (#8340)
Motivation:

Java since version 6 has the wrapper for the ConcurrentHashMap that could be created via Collections.newSetFromMap(map). So no need to create own ConcurrentSet class. Also, since netty plans to switch to Java 8 soon there is another method for that - ConcurrentHashMap.newKeySet().
For now, marking this class @deprecated would be enough, just to warn users who use netty's ConcurrentSet. After switching to Java 8 ConcurrentSet should be removed and replaced with ConcurrentHashMap.newKeySet().

Modification:

ConcurrentSet deprecated.
2018-10-15 19:36:05 +02:00
Dmitriy Dumanskiy
0e4186c552 deprecate IntegerHolder for removal (#8339)
Motivation:

Seems like IntegerHolder counterHashCode field is the very old legacy field that is no longer used. Should be marked as deprecated and removed in the future versions.

Modification:

IntegerHolder class, InternalThreadLocalMap.counterHashCode() and InternalThreadLocalMap.setCounterHashCode(IntegerHolder counterHashCode) are now deprecated.
2018-10-11 14:59:47 +08:00
Norman Maurer
a208f6dc7c
Do the same extended checks as the JDK when a X509TrustManager is used with the OpenSSL provider. (#8307)
Motivation:

When a X509TrustManager is used while configure the SslContext the JDK automatically does some extra checks during validation of provided certs by the remote peer. We should do the same when our native implementation is used.

Modification:

- Automatically wrap a X509TrustManager and so do the same validations as the JDK does.
- Add unit tests.

Result:

More consistent behaviour. Fixes https://github.com/netty/netty/issues/6664.
2018-09-28 09:19:58 +02:00
root
2d7cb47edd [maven-release-plugin] prepare for next development iteration 2018-09-27 19:00:45 +00:00
root
3a9ac829d5 [maven-release-plugin] prepare release netty-4.1.30.Final 2018-09-27 18:56:12 +00:00
Carl Mastrangelo
1dff107de1 Don't re-arm timerfd each epoll_wait (#7816)
Motivation:
The Epoll transport checks to see if there are any scheduled tasks
before entering epoll_wait, and resets the timerfd just before.
This causes an extra syscall to timerfd_settime before doing any
actual work.   When scheduled tasks aren't added frequently, or
tasks are added with later deadlines, this is unnecessary.

Modification:
Check the *deadline* of the peeked task in EpollEventLoop, rather
than the *delay*.  If it hasn't changed since last time, don't
re-arm the timer

Result:
About 2us faster on gRPC RTT 50pct latency benchmarks.

Before (2 runs for 5 minutes, 1 minute of warmup):

```
50.0%ile Latency (in nanos):		64267
90.0%ile Latency (in nanos):		72851
95.0%ile Latency (in nanos):		78903
99.0%ile Latency (in nanos):		92327
99.9%ile Latency (in nanos):		119691
100.0%ile Latency (in nanos):		13347327
QPS:                           14933

50.0%ile Latency (in nanos):		63907
90.0%ile Latency (in nanos):		73055
95.0%ile Latency (in nanos):		79443
99.0%ile Latency (in nanos):		93739
99.9%ile Latency (in nanos):		123583
100.0%ile Latency (in nanos):		14028287
QPS:                           14936
```

After:
```
50.0%ile Latency (in nanos):		62123
90.0%ile Latency (in nanos):		70795
95.0%ile Latency (in nanos):		76895
99.0%ile Latency (in nanos):		90887
99.9%ile Latency (in nanos):		117819
100.0%ile Latency (in nanos):		14126591
QPS:                           15387

50.0%ile Latency (in nanos):		61021
90.0%ile Latency (in nanos):		70311
95.0%ile Latency (in nanos):		76687
99.0%ile Latency (in nanos):		90887
99.9%ile Latency (in nanos):		119527
100.0%ile Latency (in nanos):		6351615
QPS:                           15571
```
2018-09-11 13:38:38 +02:00
Norman Maurer
afe0767e9c
Log the correct line-number when using SLF4j with netty if possible. (#8258)
* Log the correct line-number when using SLF4j with netty if possible.

Motivation:

At the moment we do not log the correct line number in many cases as it will log the line number of the logger wrapper itself. Slf4j does have an extra interface that can be used to filter out these nad make it more usable with logging wrappers.

Modifications:

Detect if the returned logger implements LocationAwareLogger and if so make use of its extra methods to be able to log the correct origin of the log request.

Result:

Better logging when using slf4j.
2018-09-07 07:34:22 +02:00
Norman Maurer
3c2dbdb5db
NioEventLoop should also use our special SelectionKeySet on Java9 and later. (#8260)
Motivation:

In Java8 and earlier we used reflection to replace the used key set if not otherwise told. This does not work on Java9 and later without special flags as its not possible to call setAccessible(true) on the Field anymore.

Modifications:

- Use Unsafe to instrument the Selector with out special set when sun.misc.Unsafe is present and we are using Java9+.

Result:

NIO transport produce less GC on Java9 and later as well.
2018-09-05 07:23:03 +02:00
Norman Maurer
ade60c11e1
PlatformDependent0 should be able to better detect if unaligned access is supported on java9 and later. (#8255)
Motivation:

In Java8 and earlier we used reflection to detect if unaligned access is supported. This fails in Java9 and later as we would need to change the accessible level of the method.
Lucky enough we can use Unsafe directly to read the content of the static field here.

Modifications:

Add special handling for detecting if unaligned access is supported on Java9 and later which does not fail due jigsaw.

Result:

Better and more correct detection on Java9 and later.
2018-09-05 07:22:16 +02:00
Norman Maurer
3eec66a974
Do not fail on runtime when an older version of Log4J2 is on the classpath. (#8240)
Motivation:

At the moment we will just assume the correct version of log4j2 is used when we find it on the classpath. This may lead to an AbstractMethodError at runtime. We should not use log4j2 if the version is not correct.

Modifications:

Check on class loading if we can use Log4J2 or not.

Result:

Fixes #8217.
2018-09-03 18:07:53 +02:00
Norman Maurer
9d8846cfce
Cleanup Log4J2Logger (#8245)
Motivation:

Log4J2Logger had some code-duplication with AbstractInternalLogger

Modifications:

Reuse AbstractInternaLogger.EXCEPTION_MESSAGE in Log4J2Logger and so remove some code-duplication

Result:

Less duplicated code.
2018-08-31 17:08:38 +02:00
Norman Maurer
38eee409c8
We should be able to use the ByteBuffer cleaner on java8 (and earlier… (#8234)
* We should be able to use the ByteBuffer cleaner on java8 (and earlier versions) even if sun.misc.Unsafe is not present.

Motivation:

At the moment we have a hard dependency on sun.misc.Unsafe to use the Cleaner on Java8 and earlier. This is not really needed as we can still use pure reflection if sun.misc.Unsafe is not present.

Modifications:

Refactor Cleaner6 to fallback to pure reflection if sun.misc.Unsafe is not present on system.

Result:

More timely releasing of direct memory on Java8 and earlier when sun.misc.Unsafe is not present.
2018-08-30 07:43:10 +02:00
Norman Maurer
4a5b61fc13
Fix log message about using non-direct buffers by default (#8235)
Motivation:

f77891cc17 changed slightly how we detect if we should prefer direct buffers or not but did miss to also take this into account when logging.

Modifications:

Fix branch for log message to reflect changes in f77891cc17.

Result:

Correct logging.
2018-08-30 06:57:12 +02:00
Terence Yim
79706357c7 Fix race condition in the NonStickyEventExecutorGroup (#8232)
Motivation:

There was a race condition between the task submitter and task executor threads such that the last Runnable submitted may not get executed. 

Modifications:

The bug was fixed by checking the task queue and state in the task executor thread after it saw the task queue was empty.

Result:

Fixes #8230
2018-08-29 19:42:01 +02:00
Norman Maurer
f77891cc17
We should prefer direct buffers if we can access the cleaner even if sun.misc.Unsafe is not present. (#8233)
Motivation:

We should prefer direct buffers whenever we can use the cleaner even if sun.misc.Unsafe is not present.

Modifications:

Correctly prefer direct buffers in all cases.

Result:

More correct code.
2018-08-29 08:21:07 +02:00
Norman Maurer
8679c5ef43
CleanerJava9 should be able to do its job even with a SecurityManager installed. (#8204)
Motivation:

CleanerJava9 currently fails whever a SecurityManager is installed. We should make use of AccessController.doPrivileged(...) so the user can give it the correct rights.

Modifications:

Use doPrivileged(...) when needed.

Result:

Fixes https://github.com/netty/netty/issues/8201.
2018-08-28 16:32:29 +02:00
zhaojigang
338ef96931 Recycler will produce npe error when multiple recycled at different thread
Motivation:

Recycler may produce a NPE when the same object is recycled multiple times from different threads.

Modifications:

- Check if the id has changed or if the Stack became null and if so throw an IllegalStateException
- Add unit test

Result:

Fixes https://github.com/netty/netty/issues/8220.
2018-08-27 08:58:40 +02:00
root
a580dc7585 [maven-release-plugin] prepare for next development iteration 2018-08-24 06:36:33 +00:00
root
3fc789e83f [maven-release-plugin] prepare release netty-4.1.29.Final 2018-08-24 06:36:06 +00:00
Norman Maurer
2bb9f64e16
Try to monkey-patch library id when shading is used and we are on Mac… (#8210)
* Try to monkey-patch library id when shading is used and we are on MacOS / OSX.

Motivation:

ea4c315b45 did ensure we support using multiple versions of the same shaded native library but the user still needed to run install_name_tool -id on MacOS to ensure the ID is unique.
This is kind of error prone and also means that the shading itself would need to be done on MacOS / OSX.

This is related to https://github.com/netty/netty/issues/7272.

Modifications:

- Monkey patch the shaded native lib on MacOS to ensure the id is unique while unpacking it to the tempory location.

Result:

Easier way of using shaded native libs in netty.
2018-08-23 11:07:09 +02:00
Norman Maurer
182ffdaf6d
Only use manual safepoint polling in PlatformDependent0.copyMemory(...) when using java <= 8 (#8124)
Motivation:

Java9 and later does the safepoint polling by itself so there is not need for us to do it.

Modifications:

Check for java version before doing manual safepoint polling.

Result:

Less custom code and less overhead when using java9 and later. Fixes https://github.com/netty/netty/issues/8122.
2018-08-18 21:09:18 +02:00
Ziyan Mo
785473788f (Nio|Epoll)EventLoop.pendingTasks does not need to dispatch to the EventLoop (#8197)
Motivation:

EventLoop.pendingTasks should be (reasonably) cheap to invoke so it can be used within observability. 

Modifications:

Remove code that dispatch access to the internal taskqueue to the EventLoop when invoked as this is not needed anymore with the current MPSC queues we are using. 

See https://github.com/netty/netty/issues/8196#issuecomment-413653286.

Result:

Fixes https://github.com/netty/netty/issues/8196
2018-08-18 07:28:31 +02:00
root
fcb19cb589 [maven-release-plugin] prepare for next development iteration 2018-07-27 04:59:28 +00:00
root
ff785fbe39 [maven-release-plugin] prepare release netty-4.1.28.Final 2018-07-27 04:59:06 +00:00
Norman Maurer
0dc71cee3a
DefaultPromise.getNow() does not correctly handle DefaultPromise.setUncancellable() (#8154)
Motivation:

We do not correctly check for previous calles of setUncancellable() in getNow() which may result in ClassCastException as we incorrectly return the internally UNCANCELLABLE object and not null if setUncancellable() we as called before.

Modifications:

Correctly check for UNCANCELLABLE and add unit test.

Result:

Fixes https://github.com/netty/netty/issues/8135.
2018-07-27 01:55:21 +08:00
root
b4dbdc2036 [maven-release-plugin] prepare for next development iteration 2018-07-11 15:37:40 +00:00
root
1c16519ac8 [maven-release-plugin] prepare release netty-4.1.27.Final 2018-07-11 15:37:21 +00:00
Norman Maurer
8186c9aaea
Fix length calculation in AsciiString.indexOf(...) and so eliminate ArrayIndexOutOfBoundsException. (#8116)
Motivation:

We incorrectly calculated the length that was used for our for loop in AsciiString.indexOf(...). This lead to a possible ArrayIndexOutOfBoundsException.

Modifications:

- Not include the start in the length calculation
- Add unit test.

Result:

Fixes https://github.com/netty/netty/issues/8112.
2018-07-11 10:21:17 +01:00
Norman Maurer
93d2807ff0
Auto-detect Log4J2 for logging if on the class-path (#8109)
Motivation:

https://github.com/netty/netty/pull/5047 added Log4J2 support but missed to add code to try to auto-detect it.

Modifications:

Try to use Log4JLoggerFactory by default.

Result:

Fixes https://github.com/netty/netty/issues/8107.
2018-07-11 10:19:37 +01:00
root
7bb9e7eafe [maven-release-plugin] prepare for next development iteration 2018-07-10 05:21:24 +00:00
root
8ca5421bd2 [maven-release-plugin] prepare release netty-4.1.26.Final 2018-07-10 05:18:13 +00:00
Sebastian Utz
0920738932 Do not log explicit no unsafe, fixes helper method. (#8111)
Motivation:

Users should not see a scary log message when Netty is initialized if
Netty configuration explicitly disables unsafe. The log message that
produces this warning was previously guarded but by recent refactoring
a bug was introduced inside the guard helper method.

Modifications:

This commit brings back the guard against the scary log message if
unsafe is explicitly disabled.

Result:

No log message is produced when unsafe is unavailable because Netty was
told to not look for it.

Relates https://github.com/netty/netty/pull/5624, https://github.com/netty/netty/pull/6696
2018-07-09 15:57:35 -04:00
Norman Maurer
83710cb2e1
Replace toArray(new T[size]) with toArray(new T[0]) to eliminate zero-out and allow the VM to optimize. (#8075)
Motivation:

Using toArray(new T[0]) is usually the faster aproach these days. We should use it.

See also https://shipilev.net/blog/2016/arrays-wisdom-ancients/#_conclusion.

Modifications:

Replace toArray(new T[size]) with toArray(new T[0]).

Result:

Faster code.
2018-06-29 07:56:04 +02:00
Norman Maurer
5b1fe611a6
Remove usage of ObjectCleaner (#8064)
Motivation:

ObjectCleaner does start a Thread to handle the cleaning of resources which leaks into the users application. We should not use it in netty itself to make things more predictable.

Modifications:

- Remove usage of ObjectCleaner and use finalize as a replacement when possible.
- Clarify javadocs for FastThreadLocal.onRemoval(...) to ensure its clear that remove() is not guaranteed to be called when the Thread completees and so this method is not enough to guarantee cleanup for this case.

Result:

Fixes https://github.com/netty/netty/issues/8017.
2018-06-28 08:15:27 +02:00
nickhill
06f3574e46 Don't calculate max direct memory twice in PlatformDependent
Motivation:

I'm not sure if trivial changes like this are interesting :-) But I
noticed that the PlatformDependent.maxDirectMemory0() method is called
twice unnecessarily during static initialization (on the default path at
least).

Modifications:

Use constant MAX_DIRECT_MEMORY already set to the same value instead of
calling maxDirectMemory0() again.

Result:

A surely imperceivable reduction in operations performed at startup.
2018-06-26 13:58:54 +02:00
Roger
3e3e5155b9 Check if Log level is enabled before creating log statements (#8022)
Motivation

There is a cost to concatenating strings and calling methods that will be wasted if the Logger's level is not enabled.

Modifications

Check if Log level is enabled before producing log statement. These are just a few cases found by RegEx'ing in the code.

Result

Tiny bit more efficient code.
2018-06-13 23:21:53 -07:00
Norman Maurer
d133bf06a4
Allow to schedule tasks up to Long.MAX_VALUE (#7972)
Motivation:

We should allow to schedule tasks with a delay up to Long.MAX_VALUE as we did pre 4.1.25.Final.

Modifications:

Just ensure we not overflow and put the correct max limits in place when schedule a timer. At worse we will get a wakeup to early and then schedule a new timeout.

Result:

Fixes https://github.com/netty/netty/issues/7970.
2018-05-30 11:11:42 +02:00
Norman Maurer
8a85761500
Don't use VM.maxDirectMemory() on IBM J9 / Eclipse OpenJ9 to retrieve direct memory limit (#7966)
Motivation:

On J9 / OpenJ9 netty initializes this value with 64M, even the direct accessible memory is actually unbounded.

Modifications:

Skip usage of VM.maxDirectMemory() on J9 / OpenJ9

Result:

More correct direct memory limit detection. Fixes #7654.
2018-05-25 14:35:32 +02:00
Norman Maurer
c3d29f7b9e
Guard against calling malloc(0) when create ByteBuffer. (#7948)
Motivation:

We did not guard against the case of calling malloc(0) when creating a ByteBuffer without a Cleaner. The problem is that malloc(0) can have different behaviour, it either return a null-pointer or a valid pointer that you can pass to free.

The real problem arise if Unsafe.allocateMemory(0) returns 0 and we use it as the memoryAddress of the ByteBuffer. The problem here is that native libraries test for 0 and handle it as a null-ptr. This is for example true in SSL.bioSetByteBuffer(...) which would throw a NPE when 0 is used as memoryAddress and so produced errors during SSL usage.

Modifications:

- Always allocate 1 byte as minimum (even if we ask for an empty buffer).
- Add unit test.

Result:

No more errors possible because of malloc(0).
2018-05-17 06:55:48 +02:00
Norman Maurer
64bb279f47 [maven-release-plugin] prepare for next development iteration 2018-05-14 11:11:45 +00:00
Norman Maurer
c67a3b0507 [maven-release-plugin] prepare release netty-4.1.25.Final 2018-05-14 11:11:24 +00:00
Xiaoyan Lin
a0ed6ec06c Fix the error message in ReferenceCounted.release (#7921)
Motivation:

When a buffer is over-released, the current error message of `IllegalReferenceCountException` is `refCnt: XXX, increment: XXX`, which is confusing. The correct message should be `refCnt: XXX, decrement: XXX`.

Modifications:

Pass `-decrement` to create `IllegalReferenceCountException`.

Result:

The error message will be `refCnt: XXX, decrement: XXX` when a buffer is over-released.
2018-05-08 20:09:16 +02:00
Norman Maurer
eaf1771336
Don't use VM.maxDirectMemory() on z/OS to retrieve direct memory limit. (#7886)
Motivation:

On z/OS netty initializes this value with 64M, even the direct accessible memory is actually unbounded.

Modifications:

Skip usage of VM.maxDirectMemory() on z/OS.

Result:

More correct direct memory limit detection. Fixes https://github.com/netty/netty/issues/7654.
2018-04-25 07:33:07 +02:00
Norman Maurer
b47fb81799
EventLoop.schedule with big delay fails (#7402)
Motivation:

Using a very huge delay when calling schedule(...) may cause an Selector error when calling select(...) later on. We should gaurd against such a big value.

Modifications:

- Add guard against a very huge value.
- Added tests.

Result:

Fixes [#7365]
2018-04-24 11:15:20 +02:00
Robin Wang
010dbe3c73 Optimize space usage of FastThreadLocal (#7861)
Motivation:
A FastThreadLocal instance currently occupies 2 slots of InternalThreadLocalMap of every thread. Actually for a FastThreadLocalThread, it does not need to store cleaner flags of FastThreadLocal instances. Besides, using BitSet to store cleaner flags is also better for space usage.

Modification:
Use BitSet to optimize space usage of FastThreadLocal.

Result:
Avoid unnecessary slot occupancy. Cleaner flags are now stored into a BitSet.
2018-04-24 09:19:01 +02:00
Norman Maurer
b75f44db9a [maven-release-plugin] prepare for next development iteration 2018-04-19 11:56:07 +00:00
Norman Maurer
04fac00c8c [maven-release-plugin] prepare release netty-4.1.24.Final 2018-04-19 11:55:47 +00:00
Norman Maurer
3ec29455af
Fix eternal loop in Recycler.WeakOrderQueue.Head#run() that blocks ObjectCleaner thread (#7878)
Motivation:

When trying to cleanup WeakOrderQueue by the ObjectCleaner we end up in an endless loop which will cause the ObjectCleaner to be not able to cleanup any other resources anymore.This bug was introduced by 6eb9674bf5.

Modifications:

Correctly update link while cleanup

Result:

Fixes https://github.com/netty/netty/issues/7877
2018-04-19 09:40:38 +02:00
Scott Mitchell
3f244a94fe
AsciiString#indexOf out of bounds (#7866)
Motivation:
The bounds checking for AsciiString#indexOf and AsciiString#lastIndexOf is not correct and may lead to ArrayIndexOutOfBoundsException.

Modifications:
- Correct the bounds checking for AsciiString#indexOf and AsciiString#lastIndexOf

Result
Fixes https://github.com/netty/netty/issues/7863
2018-04-12 12:06:12 -07:00
Gustavo Fernandes
76c5f6cd03 Enable per origin Cors configuration (#7800)
Motivation:

Finer granularity when configuring CorsHandler, enabling different policies for different origins.

Modifications:

The CorsHandler has an extra constructor that accepts a List<CorsConfig> that are evaluated sequentially when processing a Cors request

Result:

The changes don't break backwards compatibility. The extra ctor can be used to provide more than one CorsConfig object.
2018-04-11 10:06:13 +02:00
Norman Maurer
5ab8342798
Add dev-tools dependency for commons (#7858)
Motivation:

We need to add a dev-tools dependecy for commons as otherwise we may fail to fetch it before we try to use it.

Modifications:

Add dependency.

Result:

Fixes https://github.com/netty/netty/issues/7842
2018-04-10 10:37:02 +02:00
Dave Moten
0f4001d598 add task before starting thread in SingleThreadEventExecutor.execute (#7841)
Motivation:

Minor performance optimisation that prevents thread from blocking due to task not having been added to queue. Discussed #7815.

Modification:

add task to the queue before starting the thread.

Result:

No additional tests.
2018-04-05 07:57:21 +02:00
root
0a61f055f5 [maven-release-plugin] prepare for next development iteration 2018-04-04 10:44:46 +00:00
root
8c549bad38 [maven-release-plugin] prepare release netty-4.1.23.Final 2018-04-04 10:44:15 +00:00
Scott Mitchell
602ee5444d NetUtil valid IP methods to accept CharSequence (#7827)
* NetUtil valid IP methods to accept CharSequence

Motivation:
NetUtil has methods to determine if a String is a valid IP address. These methods don't rely upon String specific methods and can use CharSequence instead.

Modifications:
- Use CharSequence instead of String for the IP validator methods.
- Avoid object allocation in AsciiString#indexOf(char,int) and reduce
byte code

Result:
No more copy operation required if a CharSequence exists.
2018-04-01 08:39:43 +02:00
Stephane Landelle
d60cd0231d HttpProxyHandler generates invalid CONNECT url and Host header when address is resolved
Motivation:

HttpProxyHandler uses `NetUtil#toSocketAddressString` to compute
CONNECT url and Host header.

The url is correct when the address is unresolved, as
`NetUtil#toSocketAddressString` will then use
`getHoststring`/`getHostname`. If the address is already resolved, the
url will be based on the IP instead of the hostname.

There’s an additional minor issue with the Host header: default port
443 should be omitted.

Modifications:

* Introduce NetUtil#getHostname
* Introduce HttpUtil#formatHostnameForHttp to format an
InetSocketAddress to
HTTP format
* Change url computation to favor hostname instead of IP
* Introduce HttpProxyHandler ignoreDefaultPortsInConnectHostHeader
parameter to ignore 80 and 443 ports in Host header

Result:

HttpProxyHandler performs properly when connecting to a resolved address
2018-03-27 09:43:11 +02:00
Norman Maurer
de082bf4c7 Correctly record creation stacktrace in ResourceLeakDetector.
Motivation:

We missed to correctly record the stacktrace of the creation of an ResourceLeak record. This could either have the effect to log the wrote stacktrace for creation or not log a stacktrace at all if the object was dropped on the floor after it was created.

Modifications:

Correctly create a Record on creation of the object.

Result:

Fixes https://github.com/netty/netty/issues/7781.
2018-03-16 08:24:18 +01:00
Norman Maurer
6eb9674bf5 Replace finalizer() usage in Recycler.WeakOrderQueue with ObjectCleaner usage.
Motivation:

We recently introduced ObjectCleaner which can be used to ensure some cleanup action is done once an object becomes weakable reachable. We should use this in Recycler.WeakOrderQueue to reduce the overhead of using a finalizer() (which will cause the GC to process it two times).

Modifications:

Replace finalizer() usage with ObjectCleaner

Result:

Fixes [#7343]
2018-03-09 18:45:02 -08:00
Norman Maurer
48df2f66b8 HashedWheelTimer.newTimeout(...) may overflow
Motivation:

We dont protect from overflow and so the timer may fire too early if a large timeout is used.

Modifications:

Add overflow guard and a test.

Result:

Fixes https://github.com/netty/netty/issues/7760.
2018-03-03 15:00:47 -08:00
Norman Maurer
69582c0b6c [maven-release-plugin] prepare for next development iteration 2018-02-21 12:52:33 +00:00
Norman Maurer
786f35c6c9 [maven-release-plugin] prepare release netty-4.1.22.Final 2018-02-21 12:52:19 +00:00
Carl Mastrangelo
15560530d4 Propagate full Unsafe unavailability reason in PlatformDependent
Motivation:
It is not clear why Unsafe is unavailable when it is explicitly
disabled, or when Netty thinks it is running on Android.

Modification:
Change the "has" fields and methods to be causes.  A null cause
means Unsafe is present.  This catches all possible reason why
Unsafe might not be available.

Result:
Easier to debug Netty start up when logging cannot be turned on.
2018-02-16 10:43:06 -08:00
Shohei Kamimori
73f23c5faa Fix typos in docs.
Motivation:

There are same typos in the docs.

Modifications:

Fix typos. Docs only changing.

Result:

More correct docs.
2018-02-14 08:44:07 +01:00
Scott Mitchell
4f982be91e
DefaultPromise internal state dependent on Signal
Motivation:
DefaultPromise's internal state depends upon specific Signal objects. These Signal objects can be used externally which causes the DefaultPromise object API to not function correct and state to become corrupted.

Modifications:
- DefaultPromise shouldn't depend upon Signal for its internal state

Result:
Fixes https://github.com/netty/netty/issues/7707
2018-02-12 14:24:46 -08:00