Commit Graph

260 Commits

Author SHA1 Message Date
Derek Lewis
f08e80ffd6 Remove unnecessary loop variable from AsciiString. (#8711)
Motivation:

Incrementing two variables in sync is not necessary when only one will do.

Modifications:

- Remove `j` from `for` loop and replace with `i`.
- Add more unit testing scenarios to cover changed code.

Results:

Unnecessary variable removed.
2019-01-15 08:33:48 +01:00
Nick Hill
fc0a4e15ab Harden ref-counting concurrency semantics (#8583)
Motivation

#8563 highlighted race conditions introduced by the prior optimistic
update optimization in 83a19d5650. These
were known at the time but considered acceptable given the perf
benefit in high contention scenarios.

This PR proposes a modified approach which provides roughly half the
gains but stronger concurrency semantics. Race conditions still exist
but their scope is narrowed to much less likely cases (releases
coinciding with retain overflow), and even in those
cases certain guarantees are still assured. Once release() returns true,
all subsequent release/retains are guaranteed to throw, and in
particular deallocate will be called at most once.

Modifications

- Use even numbers internally (including -ve) for live refcounts
- "Final" release changes to odd number (equivalent to refcount 0)
- Retain still uses faster getAndAdd, release uses CAS loop
- First CAS attempt uses non-volatile read
- Thread.yield() after a failed CAS provides a net gain

Result

More (though not completely) robust concurrency semantics for ref
counting; increased latency under high contention, but still roughly
twice as fast as the original logic. Bench results to follow
2018-11-29 08:32:50 +01:00
Norman Maurer
cb1090653f LocationAwareSlf4jLogger does not correctly format log message. (#8595)
Motivation:

We did miss to use MessageFormatter inside LocationAwareSlf4jLogger and so {} was not correctly replaced in log messages when using slf4j.
This regression was introduced by afe0767e9c.

Modifications:

- Make use of MessageFormatter
- Add unit test.

Result:

Fixes https://github.com/netty/netty/issues/8483.
2018-11-27 11:45:11 +01:00
Norman Maurer
afe0767e9c
Log the correct line-number when using SLF4j with netty if possible. (#8258)
* Log the correct line-number when using SLF4j with netty if possible.

Motivation:

At the moment we do not log the correct line number in many cases as it will log the line number of the logger wrapper itself. Slf4j does have an extra interface that can be used to filter out these nad make it more usable with logging wrappers.

Modifications:

Detect if the returned logger implements LocationAwareLogger and if so make use of its extra methods to be able to log the correct origin of the log request.

Result:

Better logging when using slf4j.
2018-09-07 07:34:22 +02:00
Norman Maurer
9d8846cfce
Cleanup Log4J2Logger (#8245)
Motivation:

Log4J2Logger had some code-duplication with AbstractInternalLogger

Modifications:

Reuse AbstractInternaLogger.EXCEPTION_MESSAGE in Log4J2Logger and so remove some code-duplication

Result:

Less duplicated code.
2018-08-31 17:08:38 +02:00
Terence Yim
79706357c7 Fix race condition in the NonStickyEventExecutorGroup (#8232)
Motivation:

There was a race condition between the task submitter and task executor threads such that the last Runnable submitted may not get executed. 

Modifications:

The bug was fixed by checking the task queue and state in the task executor thread after it saw the task queue was empty.

Result:

Fixes #8230
2018-08-29 19:42:01 +02:00
zhaojigang
338ef96931 Recycler will produce npe error when multiple recycled at different thread
Motivation:

Recycler may produce a NPE when the same object is recycled multiple times from different threads.

Modifications:

- Check if the id has changed or if the Stack became null and if so throw an IllegalStateException
- Add unit test

Result:

Fixes https://github.com/netty/netty/issues/8220.
2018-08-27 08:58:40 +02:00
Norman Maurer
0dc71cee3a
DefaultPromise.getNow() does not correctly handle DefaultPromise.setUncancellable() (#8154)
Motivation:

We do not correctly check for previous calles of setUncancellable() in getNow() which may result in ClassCastException as we incorrectly return the internally UNCANCELLABLE object and not null if setUncancellable() we as called before.

Modifications:

Correctly check for UNCANCELLABLE and add unit test.

Result:

Fixes https://github.com/netty/netty/issues/8135.
2018-07-27 01:55:21 +08:00
Norman Maurer
8186c9aaea
Fix length calculation in AsciiString.indexOf(...) and so eliminate ArrayIndexOutOfBoundsException. (#8116)
Motivation:

We incorrectly calculated the length that was used for our for loop in AsciiString.indexOf(...). This lead to a possible ArrayIndexOutOfBoundsException.

Modifications:

- Not include the start in the length calculation
- Add unit test.

Result:

Fixes https://github.com/netty/netty/issues/8112.
2018-07-11 10:21:17 +01:00
Norman Maurer
83710cb2e1
Replace toArray(new T[size]) with toArray(new T[0]) to eliminate zero-out and allow the VM to optimize. (#8075)
Motivation:

Using toArray(new T[0]) is usually the faster aproach these days. We should use it.

See also https://shipilev.net/blog/2016/arrays-wisdom-ancients/#_conclusion.

Modifications:

Replace toArray(new T[size]) with toArray(new T[0]).

Result:

Faster code.
2018-06-29 07:56:04 +02:00
Norman Maurer
5b1fe611a6
Remove usage of ObjectCleaner (#8064)
Motivation:

ObjectCleaner does start a Thread to handle the cleaning of resources which leaks into the users application. We should not use it in netty itself to make things more predictable.

Modifications:

- Remove usage of ObjectCleaner and use finalize as a replacement when possible.
- Clarify javadocs for FastThreadLocal.onRemoval(...) to ensure its clear that remove() is not guaranteed to be called when the Thread completees and so this method is not enough to guarantee cleanup for this case.

Result:

Fixes https://github.com/netty/netty/issues/8017.
2018-06-28 08:15:27 +02:00
Norman Maurer
d133bf06a4
Allow to schedule tasks up to Long.MAX_VALUE (#7972)
Motivation:

We should allow to schedule tasks with a delay up to Long.MAX_VALUE as we did pre 4.1.25.Final.

Modifications:

Just ensure we not overflow and put the correct max limits in place when schedule a timer. At worse we will get a wakeup to early and then schedule a new timeout.

Result:

Fixes https://github.com/netty/netty/issues/7970.
2018-05-30 11:11:42 +02:00
Norman Maurer
c3d29f7b9e
Guard against calling malloc(0) when create ByteBuffer. (#7948)
Motivation:

We did not guard against the case of calling malloc(0) when creating a ByteBuffer without a Cleaner. The problem is that malloc(0) can have different behaviour, it either return a null-pointer or a valid pointer that you can pass to free.

The real problem arise if Unsafe.allocateMemory(0) returns 0 and we use it as the memoryAddress of the ByteBuffer. The problem here is that native libraries test for 0 and handle it as a null-ptr. This is for example true in SSL.bioSetByteBuffer(...) which would throw a NPE when 0 is used as memoryAddress and so produced errors during SSL usage.

Modifications:

- Always allocate 1 byte as minimum (even if we ask for an empty buffer).
- Add unit test.

Result:

No more errors possible because of malloc(0).
2018-05-17 06:55:48 +02:00
Xiaoyan Lin
a0ed6ec06c Fix the error message in ReferenceCounted.release (#7921)
Motivation:

When a buffer is over-released, the current error message of `IllegalReferenceCountException` is `refCnt: XXX, increment: XXX`, which is confusing. The correct message should be `refCnt: XXX, decrement: XXX`.

Modifications:

Pass `-decrement` to create `IllegalReferenceCountException`.

Result:

The error message will be `refCnt: XXX, decrement: XXX` when a buffer is over-released.
2018-05-08 20:09:16 +02:00
Robin Wang
010dbe3c73 Optimize space usage of FastThreadLocal (#7861)
Motivation:
A FastThreadLocal instance currently occupies 2 slots of InternalThreadLocalMap of every thread. Actually for a FastThreadLocalThread, it does not need to store cleaner flags of FastThreadLocal instances. Besides, using BitSet to store cleaner flags is also better for space usage.

Modification:
Use BitSet to optimize space usage of FastThreadLocal.

Result:
Avoid unnecessary slot occupancy. Cleaner flags are now stored into a BitSet.
2018-04-24 09:19:01 +02:00
Scott Mitchell
3f244a94fe
AsciiString#indexOf out of bounds (#7866)
Motivation:
The bounds checking for AsciiString#indexOf and AsciiString#lastIndexOf is not correct and may lead to ArrayIndexOutOfBoundsException.

Modifications:
- Correct the bounds checking for AsciiString#indexOf and AsciiString#lastIndexOf

Result
Fixes https://github.com/netty/netty/issues/7863
2018-04-12 12:06:12 -07:00
Scott Mitchell
602ee5444d NetUtil valid IP methods to accept CharSequence (#7827)
* NetUtil valid IP methods to accept CharSequence

Motivation:
NetUtil has methods to determine if a String is a valid IP address. These methods don't rely upon String specific methods and can use CharSequence instead.

Modifications:
- Use CharSequence instead of String for the IP validator methods.
- Avoid object allocation in AsciiString#indexOf(char,int) and reduce
byte code

Result:
No more copy operation required if a CharSequence exists.
2018-04-01 08:39:43 +02:00
Norman Maurer
48df2f66b8 HashedWheelTimer.newTimeout(...) may overflow
Motivation:

We dont protect from overflow and so the timer may fire too early if a large timeout is used.

Modifications:

Add overflow guard and a test.

Result:

Fixes https://github.com/netty/netty/issues/7760.
2018-03-03 15:00:47 -08:00
Scott Mitchell
4f982be91e
DefaultPromise internal state dependent on Signal
Motivation:
DefaultPromise's internal state depends upon specific Signal objects. These Signal objects can be used externally which causes the DefaultPromise object API to not function correct and state to become corrupted.

Modifications:
- DefaultPromise shouldn't depend upon Signal for its internal state

Result:
Fixes https://github.com/netty/netty/issues/7707
2018-02-12 14:24:46 -08:00
koji lin
1e70d3092d Avoid register multiple cleaner task for same thread's FastThreadLocal index
Motivation:

Currently if user call set/remove/set/remove many times, it will create multiple cleaner task for same index. It may cause OOM due to long live thread will have more and more task in LIVE_SET.

Modification:

Add flag to avoid recreating tasks.

Result:
Only create 1 clean task. But use more space of indexedVariables.
2018-02-05 09:05:51 +01:00
Norman Maurer
e72c197aa3 Reflective setAccessible(true) will produce scary warnings on the console when using java9+, dont do it
Motivation:

Reflective setAccessible(true) will produce scary warnings on the console when using java9+, while netty still works. That said users may feel uncomfortable with these warnings, we should not try to do it by default when using java9+.

Modifications:

Add io.netty.tryReflectionSetAccessible  system property which controls if setAccessible(...) will be used. By default it will bet set to false when using java9+.

Result:

Fixes [#7254].
2018-01-30 12:18:34 +01:00
Norman Maurer
1879433ae6 ObjectCleanerThread must be a deamon thread to ensure the JVM can always terminate.
Motivation:

The ObjectCleanerThread must be a daemon thread as otherwise we may block the JVM from exit. By using a daemon thread we basically give the same garantees as the JVM when it comes to cleanup of resources (as the GC threads are also daemon threads and the CleanerImpl uses a deamon thread as well in Java9+).

Modifications:

Change ObjectCleanThread to be a daemon thread.

Result:

JVM shutdown will always be able to complete. Fixed [#7617].
2018-01-26 08:25:42 +01:00
jaymode
f0c76cacc3 Replace reflective access of Throwable#addSuppressed with version guarded access
Motivation:

In environments with a security manager, the reflective access to get the reference to
Throwable#addSuppressed can cause issues that result in Netty failing to load. The main
motivation in this pull request is to remove the use of reflection to prevent issues in
these environments.

Modifications:

ThrowableUtil no longer uses Class#getDeclaredMembers to get the Method that references
Throwable#addSuppressed and instead guards the call to Throwable#addSuppressed with a
Java version check.

Additionally, a annotation was added that suppresses the animal sniffer java16 signature
check on the given method. The benefit of the annotation is that it limits the exclusion
of Throwable to just the ThrowableUtil class and has string text indicating the reason
for suppressing the java16 signature check.

Result:

Netty no longer requires the use of Class#getDeclaredMethod for ThrowableUtil and will
work in environments restricted by a security manager without needing to grant reflection
permissions.

Fixes #7614
2018-01-25 19:56:17 +01:00
Scott Mitchell
031bad60dc ObjectCleaner should continue cleaning despite exceptions
Motivation:
ObjectCleaner inovkes a Runnable which may execute user code (FastThreadLocal#onRemoval) and therefore exceptions maybe thrown. If an exception is thrown the cleanup thread will exit prematurely and we may never finish cleaning up which will result in leaks.

Modifications:
- ObjectCleaner should suppress exceptions and continue cleaning

Result:
ObjectCleaner will reliably clean despite exceptions being thrown.
2018-01-19 20:09:20 +01:00
Norman Maurer
e004b4a354 Ensure ObjectCleaner will also be used when FastThreadLocal.set is used.
Motivation:

e329ca1 introduced the user of ObjectCleaner in FastThreadLocal but we missed the case to register our cleaner task if FastThreadLocal.set was called only.

Modifications:

- Use ObjectCleaner also when FastThreadLocal.set is used.
- Add test case.

Result:

ObjectCleaner is always used.
2017-12-22 07:11:22 +01:00
Norman Maurer
e329ca1cf3 Introduce ObjectCleaner and use it in FastThreadLocal to ensure FastThreadLocal.onRemoval(...) is called
Motivation:

There is no guarantee that FastThreadLocal.onRemoval(...) is called if the FastThreadLocal is used by "non" FastThreacLocalThreads. This can lead to all sort of problems, like for example memory leaks as direct memory is not correctly cleaned up etc.

Beside this we use ThreadDeathWatcher to check if we need to release buffers back to the pool when thread local caches are collected. In the past ThreadDeathWatcher was used which will need to "wakeup" every second to check if the registered Threads are still alive. If we can ensure FastThreadLocal.onRemoval(...) is called we do not need this anymore.

Modifications:

- Introduce ObjectCleaner and use it to ensure FastThreadLocal.onRemoval(...) is always called when a Thread is collected.
- Deprecate ThreadDeathWatcher
- Add unit tests.

Result:

Consistent way of cleanup FastThreadLocals when a Thread is collected.
2017-12-21 07:34:44 +01:00
Norman Maurer
5ad35a157c SingleThreadEventExecutor ignores startThread failures
Motivation:

When doStartThread throws an exception, e.g. due to the actual executor being depleted of threads and throwing in its rejected execution handler, the STEE ends up in started state anyway. If we try to execute another task in this executor, it will be queued but the thread won't be started anymore and the task will linger forever.

Modifications:

- Ensure we not update the internal state if the startThread() method throws.
- Add testcase

Result:

Fixes [#7483]
2017-12-14 21:38:37 +00:00
Norman Maurer
0276b6e0f6 Ensure Thread can be collected in a timely manner if Recycler.Stack holds a reference to it.
Motivation:

In our Recycler implementation we store a reference to the current Thread in the Stack that is stored in a FastThreadLocal. The Stack itself is referenced in the DefaultHandle itself. A problem can arise if a user stores a Reference to an Object that holds a reference to the DefaultHandle somewhere and either not remove the reference at all or remove it very late. In this case the Thread itself can not be collected as its still referenced in the Stack that is referenced by the DefaultHandle.

Modifications:

- Use a WeakReference to store the reference to the Thread in the Stack
- Add a test case

Result:

Ensure a Thread can be collected in a timely manner in all cases even if it used the Recycler.
2017-12-14 06:44:47 +01:00
Soner Kaya
f9cadc0a8c When System property is empty use def value.
Motivation:

When system property is empty, the default value should be used.

Modification:

- Correctly use the default value in all cases
- Add unit tests

Result:

Correct behaviour
2017-11-23 19:45:37 +01:00
Anuraag Agrawal
1f1a60ae7d Use Netty's DefaultPriorityQueue instead of JDK's PriorityQueue for scheduled tasks
Motivation:

`AbstractScheduledEventExecutor` uses a standard `java.util.PriorityQueue` to keep track of task deadlines. `ScheduledFuture.cancel` removes tasks from this `PriorityQueue`. Unfortunately, `PriorityQueue.remove` has `O(n)` performance since it must search for the item in the entire queue before removing it. This is fast when the future is at the front of the queue (e.g., already triggered) but not when it's randomly located in the queue.

Many servers will use `ScheduledFuture.cancel` on all requests, e.g., to manage a request timeout. As these cancellations will be happen in arbitrary order, when there are many scheduled futures, `PriorityQueue.remove` is a bottleneck and greatly hurts performance with many concurrent requests (>10K).

Modification:

Use netty's `DefaultPriorityQueue` for scheduling futures instead of the JDK. `DefaultPriorityQueue` is almost identical to the JDK version except it is able to remove futures without searching for them in the queue. This means `DefaultPriorityQueue.remove` has `O(log n)` performance.

Result:

Before - cancelling futures has varying performance, capped at `O(n)`
After - cancelling futures has stable performance, capped at `O(log n)`

Benchmark results

After - cancelling in order and in reverse order have similar performance within `O(log n)` bounds
```
Benchmark                                           (num)   Mode  Cnt       Score      Error  Units
ScheduledFutureTaskBenchmark.cancelInOrder            100  thrpt   20  137779.616 ± 7709.751  ops/s
ScheduledFutureTaskBenchmark.cancelInOrder           1000  thrpt   20   11049.448 ±  385.832  ops/s
ScheduledFutureTaskBenchmark.cancelInOrder          10000  thrpt   20     943.294 ±   12.391  ops/s
ScheduledFutureTaskBenchmark.cancelInOrder         100000  thrpt   20      64.210 ±    1.824  ops/s
ScheduledFutureTaskBenchmark.cancelInReverseOrder     100  thrpt   20  167531.096 ± 9187.865  ops/s
ScheduledFutureTaskBenchmark.cancelInReverseOrder    1000  thrpt   20   33019.786 ± 4737.770  ops/s
ScheduledFutureTaskBenchmark.cancelInReverseOrder   10000  thrpt   20    2976.955 ±  248.555  ops/s
ScheduledFutureTaskBenchmark.cancelInReverseOrder  100000  thrpt   20     362.654 ±   45.716  ops/s
```

Before - cancelling in order and in reverse order have significantly different performance at higher queue size, orders of magnitude worse than the new implementation.
```
Benchmark                                           (num)   Mode  Cnt       Score       Error  Units
ScheduledFutureTaskBenchmark.cancelInOrder            100  thrpt   20  139968.586 ± 12951.333  ops/s
ScheduledFutureTaskBenchmark.cancelInOrder           1000  thrpt   20   12274.420 ±   337.800  ops/s
ScheduledFutureTaskBenchmark.cancelInOrder          10000  thrpt   20     958.168 ±    15.350  ops/s
ScheduledFutureTaskBenchmark.cancelInOrder         100000  thrpt   20      53.381 ±    13.981  ops/s
ScheduledFutureTaskBenchmark.cancelInReverseOrder     100  thrpt   20  123918.829 ±  3642.517  ops/s
ScheduledFutureTaskBenchmark.cancelInReverseOrder    1000  thrpt   20    5099.810 ±   206.992  ops/s
ScheduledFutureTaskBenchmark.cancelInReverseOrder   10000  thrpt   20      72.335 ±     0.443  ops/s
ScheduledFutureTaskBenchmark.cancelInReverseOrder  100000  thrpt   20       0.743 ±     0.003  ops/s
```
2017-11-10 23:09:32 -08:00
Trask Stalnaker
58e74e9fee Support running Netty in bootstrap class loader
Motivation:

Fix NullPointerExceptions that occur when running netty-tcnative inside the bootstrap class loader.

Modifications:

- Replace loader.getResource(...) with ClassLoader.getSystemResource(...) when loader is null.
- Replace loader.loadClass(...) with Class.forName(..., false, loader) which works when loader is both null and non-null.

Result:

Support running native libs in bootstrap class loader
2017-10-29 13:13:19 +01:00
Idel Pivnitskiy
4793daa589 Make Comparators Serializable
Motivation:

Objects of java.util.TreeMap or java.util.TreeSet will become
non-Serializable if instantiated with Comparators, which are not also
 Serializable. This can result in unexpected and difficult-to-diagnose
 bugs.

Modifications:

Implements Serializable for all classes, which implements Comparator.

Result:

Proper Comparators which will not force collections to
non-Serializable mode.
2017-10-22 03:40:28 +02:00
Idel Pivnitskiy
50a067a8f7 Make methods 'static' where it possible
Motivation:

Even if it's a super micro-optimization (most JVM could optimize such
 cases in runtime), in theory (and according to some perf tests) it
 may help a bit. It also makes a code more clear and allows you to
 access such methods in the test scope directly, without instance of
 the class.

Modifications:

Add 'static' modifier for all methods, where it possible. Mostly in
test scope.

Result:

Cleaner code with proper 'static' modifiers.
2017-10-21 14:59:26 +02:00
Norman Maurer
4d5f0e7ad5 NativeLibraryLoader should check the result of ClassLoader#getResource method
Motivation:

NativeLibraryLoader uses ClassLoader#getResource method that can return nulls when the resource cannot be found. The returned url variable should be checked for nullity and fail in a more usable manner than a NullPointerException

Modifications:

Fail with a FileNotFoundException

Result:

Fixes [#7222].
2017-09-19 17:45:06 -07:00
杨浩
14189140a0 log in PatternLayout (%F:%L)%c.%M
Motivation:

When Log4j2Logger is used with PatternLayout (%F:%L)%c.%M, the log message incorrect shows:

(Log4J2Logger.java:73)io.netty.util.internal.PlatformDependent0.debug ....

Modification:

Extend AbstractLogger

Result:

Fixes [#7186].
2017-09-14 14:51:20 -07:00
Nikolay Fedorovskikh
01eb428b39 Move methods for decode hex dump into StringUtil
Motivation:

PR #6811 introduced a public utility methods to decode hex dump and its parts, but they are not visible from netty-common.

Modifications:

1. Move the `decodeHexByte`, `decodeHexDump` and `decodeHexNibble` methods into `StringUtils`.
2. Apply these methods where applicable.
3. Remove similar methods from other locations (e.g. `HpackHex` test class).

Result:

Less code duplication.
2017-06-23 18:52:42 +02:00
Nikolay Fedorovskikh
aa38b6a769 Prevent unnecessary allocations in the StringUtil#escapeCsv
Motivation:

A `StringUtil#escapeCsv` creates new `StringBuilder` on each value even if the same string is returned in the end.

Modifications:

Create new `StringBuilder` only if it really needed. Otherwise, return the original string (or just trimmed substring).

Result:

Less GC load. Up to 4x faster work for not changed strings.
2017-06-13 14:57:38 -07:00
Nikolay Fedorovskikh
d768c5e628 MessageFormatter improvements
Motivation:

`FormattingTuple.getArgArray()` is never used.
In the `MessageFormatter` it is possible to make
some improvements, e.g. replace `StringBuffer`
with `StringBuilder`, avoid redundant allocations, etc.

Modifications:

- Remove `argArray` field from the `FormattingTuple`.
- In `MessageFormatter`:
  - replace `StringBuffer` with `StringBuilder`,
  - replace `HashMap` with `HashSet` and make it lazy initialized.
  - avoid redundant allocations (`substring()`, etc.)
  - use appropriate StringBuilder's methods for the some `Number` values.
- Porting unit tests from `slf4j`.

Result:

Less GC load on logging with internal `MessageFormatter`.
2017-05-19 09:47:11 -07:00
Nikolay Fedorovskikh
e4531918a3 Optimizations in NetUtil
Motivation:

IPv4/6 validation methods use allocations, which can be avoided.
IPv4 parse method use StringTokenizer.

Modifications:

Rewriting IPv4/6 validation methods to avoid allocations.
Rewriting IPv4 parse method without use StringTokenizer.

Result:

IPv4/6 validation and IPv4 parsing faster up to 2-10x.
2017-05-18 16:42:22 -07:00
Norman Maurer
0ee49e6d66 Eliminate noisy logging when using sun.misc.Unsafe and running on pre Java9
Motivation:

We should only try to load jdk.internal.misc.Unsafe if we run on Java9+ to eliminate noise in the log.

Modifications:

- Move javaVersion() and related methods to PlatformDependent0 to be able to use these in the static initializer without creating a cycle.
- Only try to load jdk.internal.misc.Unsafe when running in Java9+

Result:

Less noise in the log when running pre java9.
2017-05-16 08:29:06 +02:00
Nikolay Fedorovskikh
5643cc6a10 IPv6 validation fixes
Motivation:

`NetUtil`'s methods `isValidIpV6Address` and `getIPv6ByName` incorrectly validate some IPv6 addresses.

Modifications:

- `getIPv6ByName`: add checks for single colon at the start or end.
- `isValidIpV6Address`: fix checks for the count of colons and use `endOffset` instead of `ipAddress.length()` for the cases with the brackets or '%'.

Result:

More correct implementation of `NetUtil#isValidIpV6Address` and `NetUtil#getIPv6ByName`.
2017-05-11 08:10:25 -07:00
Vladimir Kostyukov
ed37cf20ef Introduce HashedWheelTimer.pendingTimeouts()
Motivation:

Fixes #6681.

Modification:

For the sake of better timer observability, expose the number of pending timeouts through the new HashedWheelTimer.pendingTimeouts method .

Result:

It's now ridiculously easy to observe Netty timer's very basic and yet important metric, the number of pending tasks/timeouts.
2017-05-01 20:10:22 -07:00
Scott Mitchell
ea1cb20c90 Netutil IPv6 bugs
Motivation:
NetUtil#isValidIpV6Address and NetUtil#getIPv6ByName allowed an invalid form of mapped IPv4 addresses which lead to accepting invalid IPv6 addresses as valid.

Modifications:
- NetUtil#isValidIpV6Address and NetUtil#getIPv6ByName should only allow 7 colons for an IPv4 address if they are the first 2 characters.

Result:
More correct implementation of NetUtil#isValidIpV6Address and NetUtil#getIPv6ByName
2017-04-28 07:29:36 +02:00
Scott Mitchell
9cb858fcf6 NetUtil IPv6 bugs related to IPv4 and compression
Motivation:
NetUtil#getByName and NetUtil#isValidIpV6Address do not strictly enforce the format of IPv4 addresses that are allowed to be embedded in IPv6 addresses as specified in https://tools.ietf.org/html/rfc4291#section-2.5.5. This may lead to invalid addresses being parsed, or invalid addresses being considered valid. Compression of a single IPv6 word was also not handled correctly if there are 7 : characters.

Modifications:
- NetUtil#isValidIpV6Address should enforce the IPv4-Compatible and IPv4-Mapped are the only valid formats for including IPv4 addresses as specified in https://tools.ietf.org/html/rfc4291#section-2.5.5
- NetUtil#getByName should more stritcly parse IPv6 addresses which contain IPv4 addresses as specified in https://tools.ietf.org/html/rfc4291#section-2.5.5
- NetUtil should allow compression even if the number of : characters is 7.
- NetUtil#createByteArrayFromIpAddressString should use the same IP string to byte[] translation which is used in NetUtil#getByName

Result:
NetUtil#getByName and NetUtil#isValidIpV6Address respect the IPv6 RFC which defines the valid formats for embedding IPv4 addresses.
2017-04-25 15:10:38 -07:00
Jason Tedor
98beb777f8 Enable configuring available processors
Motivation:

In cases when an application is running in a container or is otherwise
constrained to the number of processors that it is using, the JVM
invocation Runtime#availableProcessors will not return the constrained
value but rather the number of processors available to the virtual
machine. Netty uses this number in sizing various resources.
Additionally, some applications will constrain the number of threads
that they are using independenly of the number of processors available
on the system. Thus, applications should have a way to globally
configure the number of processors.

Modifications:

Rather than invoking Runtime#availableProcessors, Netty should rely on a
method that enables configuration when the JVM is started or by the
application. This commit exposes a new class NettyRuntime for enabling
such configuraiton. This value can only be set once. Its default value
is Runtime#availableProcessors so that there is no visible change to
existing applications, but enables configuring either a system property
or configuring during application startup (e.g., based on settings used
to configure the application).

Additionally, we introduce the usage of forbidden-apis to prevent future
uses of Runtime#availableProcessors from creeping. Future work should
enable the bundled signatures and clean up uses of deprecated and
other forbidden methods.

Result:

Netty can be configured to not use the underlying number of processors,
but rather the constrained number of processors.
2017-04-23 10:31:17 +02:00
Nikolay Fedorovskikh
0692bf1b6a fix the typos 2017-04-20 04:56:09 +02:00
Norman Maurer
1b0b8f80cd AbstractScheduledEventExecutor.schedule(...) must accept delay <= 0.
Motivation:

As the javadoc of ScheduledExecutorService state:

Zero and negative delays (but not periods) are also allowed in schedule methods,and are treated as requests for immediate execution.

Modifications:

- Correctly handle delay <= 0.
- Add unit tests.

Result:

Fixes [#6627].
2017-04-19 11:35:50 +02:00
Norman Maurer
493a8135f8 Ensure test introduced in 5c1c14286d also works on Java9 2017-03-29 22:43:00 +02:00
Norman Maurer
5c1c14286d Allow negative memoryAddress when calling PlatformDependent0.newDirectBuffer(...)
Motivation:

When UNSAFE.allocateMemory is returning an address whose high bit is set we currently throw an IllegalArgumentException. This is not correct as it may return a negative number on at least sparc.

Modifications:

- Allow to pass in negative memoryAddress
- Add unit tests

Result:

Correctly validate the memoryAddress and so also work on sparc as expected. Fixes [#6574].
2017-03-29 22:33:34 +02:00
David Dossot
9c1a191696 Trim optional white space in CombinedHttpHeaders values
Motivation:

The updated HTTP/1.x RFC allows for header values to be CSV and separated by OWS [1]. CombinedHttpHeaders should remove this OWS on insertion.

[1] https://tools.ietf.org/html/rfc7230#section-7

Modification:

CombinedHttpHeaders doesn't account for the OWS and returns it back to the user as part of the value.

Result:

Fixes #6452
2017-03-19 08:17:29 -07:00
Norman Maurer
e12f504ac1 Remove deprecated usage of Mockito methods
Motivation:

We used some deprecated Mockito methods.

Modifications:

- Replace deprecated method usage
- Some cleanup

Result:

No more usage of deprecated Mockito methods. Fixes [#6482].
2017-03-09 20:59:54 +01:00
Norman Maurer
c6a3cae269 UnorderedThreadPoolEventExecutor consumes 100% CPU when idle
Motivation:

When UnorderedThreadPoolEventExecutor.execute / submit etc is called it will consume up to 100 % CPU even after the task was executed.

Modifications:

Add a special wrapper which we will be used in execute(...) to wrap the submitted Runnable. This is needed as  ScheduledThreadPoolExecutor.execute(...) will delegate to submit(...) which will then use decorateTask(...). The problem with this is that decorateTask(...) needs to ensure we only do our own decoration if we not call from execute(...) as otherwise we may end up creating an endless loop because DefaultPromise will call  EventExecutor.execute(...) when notify the listeners of the promise.

Result:

Fixes [#6507].
2017-03-09 11:12:42 +01:00
Nikolay Fedorovskikh
2993760e92 Fix misordered 'assertEquals' arguments in tests
Motivation:

Wrong argument order in some 'assertEquals' applying.

Modifications:

Flip compared arguments.

Result:

Correct `assertEquals` usage.
2017-03-08 22:48:37 -08:00
Norman Maurer
a7c0ff665c Only use Mockito for mocking.
Motivation:

We used various mocking frameworks. We should only use one...

Modifications:

Make usage of mocking framework consistent by only using Mockito.

Result:

Less dependencies and more consistent mocking usage.
2017-02-07 08:47:22 +01:00
Jason Tedor
42c0359820 Do not prefer empty MAC address
Motivation:

When comparing MAC addresses searching for the best MAC address, if
locally-administered address (e.g., from a Docker container) is compared
against an empty MAC address, the empty MAC address will be marked as
preferred. In cases this is the only available MAC address, this leaves
Netty using a random machine ID instead of using a perfectly valid
machine ID from the locally-adminstered address.

Modifications:

This commit modifies the MAC address logic so that the empty MAC address
is not preferred over a locally-administered address. This commit also
simplifies the comparison logic here.

Result:

Empty MAC addresses will not be preferred over locally-administered
addresses thus permitting the default machine ID to be the
locally-adminstered MAC address if it is the only available MAC address.
2017-02-06 12:14:27 -08:00
Scott Mitchell
3482651e0c HTTP/2 Non Active Stream RFC Corrections
Motivation:
codec-http2 couples the dependency tree state with the remainder of the stream state (Http2Stream). This makes implementing constraints where stream state and dependency tree state diverge in the RFC challenging. For example the RFC recommends retaining dependency tree state after a stream transitions to closed [1]. Dependency tree state can be exchanged on streams in IDLE. In practice clients may use stream IDs for the purpose of establishing QoS classes and therefore retaining this dependency tree state can be important to client perceived performance. It is difficult to limit the total amount of state we retain when stream state and dependency tree state is combined.

Modifications:
- Remove dependency tree, priority, and weight related items from public facing Http2Connection and Http2Stream APIs. This information is optional to track and depends on the flow controller implementation.
- Move all dependency tree, priority, and weight related code from DefaultHttp2Connection to WeightedFairQueueByteDistributor. This is currently the only place which cares about priority. We can pull out the dependency tree related code in the future if it is generally useful to expose for other implementations.
- DefaultHttp2Connection should explicitly limit the number of reserved streams now that IDLE streams are no longer created.

Result:
More compliant with the HTTP/2 RFC.
Fixes https://github.com/netty/netty/issues/6206.

[1] https://tools.ietf.org/html/rfc7540#section-5.3.4
2017-02-01 10:34:27 -08:00
Norman Maurer
eb5dc4bced Correctly handle IPV6 in HttpProxyHandler
Motivation:

The HttpProxyHandler is expected to be capable of issuing a valid CONNECT request for a tunneled connection to an IPv6 host.

Modifications:

- Correctly format the IPV6 address.
- Add unit tests

Result:

HttpProxyHandler works with IPV6 as well. Fixes [#6152].
2017-01-12 07:51:37 +01:00
Norman Maurer
31da0ddbac Revert "Disallow setting logger factory twice"
This reverts commit 3c92f2b64a which needs more thoughts and so will go into the next release.
2016-12-21 15:14:53 +01:00
Scott Mitchell
3d11334151 Fix DefaultChannelId MAC address parsing bug
Motivation:
DefaultChannelId provides a regular expression which validates if a user provided MAC address is valid. This regular expression may allow invalid MAC addresses and also not allow valid MAC addresses.

Modifications:
- Introduce a MacAddressUtil#parseMac method which can parse and validate the MAC address at the same time. The regular expression check before hand is additional overhead if we have to parse the MAC address.

Result:
Fixes https://github.com/netty/netty/issues/6132.
2016-12-20 17:06:27 -08:00
Norman Maurer
f6ac8b5d32 [#6114] Increase test timeout for test introduced in c2f4daa739
Motivation:

c2f4daa739 added a unit test but used a too small test timeout.

Modifications:

Increase timeout.

Result:

Test should have enough time to complete on the CI.
2016-12-08 13:17:42 +01:00
Jason Tedor
3c92f2b64a Disallow setting logger factory twice
Motivation:

InternalLoggerFactory either sets a default logger factory
implementation based on the logging implementations on the classpath, or
applications can set a logger factory explicitly. If applications wait
too long to set the logger factory, Netty will have already set a logger
factory leading to some objects using one logging implementation and
other objets using another logging implementation. This can happen too
if the application tries to set the logger factory twice, which is
likely a bug in the application. Yet, the Javadocs for
InternalLoggerFactory warn against this saying that
InternalLoggerFactory#setLoggerFactory "should be called as early as
possible and shouldn't be called more than once". Instead, Netty should
guard against this.

Modications:

We replace the logger factory field with an atomic reference on which we
can do CAS operations to safely guard against it being set twice. We
also add an internal holder class that captures the static interface of
InternalLoggerFactory that can aid in testing.

Result:

The logging factory can not be set twice, and applications that want to
set the logging factory must do it before any Netty classes are
initialized (or the default logger factory will be set).
2016-12-08 10:38:01 +01:00
Norman Maurer
c2f4daa739 Fix false-positives when using ResourceLeakDetector.
Motivation:

We need to ensure the tracked object can not be GC'ed before ResourceLeak.close() is called as otherwise we may get false-positives reported by the ResourceLeakDetector. This can happen as the JIT / GC may be able to figure out that we do not need the tracked object anymore and so already enqueue it for collection before we actually get a chance to close the enclosing ResourceLeak.

Modifications:

- Add ResourceLeakTracker and deprecate the old ResourceLeak
- Fix some javadocs to correctly release buffers.
- Add a unit test for ResourceLeakDetector that shows that ResourceLeakTracker has not the problems.

Result:

No more false-positives reported by ResourceLeakDetector when ResourceLeakDetector.track(...) is used.
2016-12-04 09:01:39 +01:00
Scott Mitchell
00fc239995 HashWheelTimerTest cleanup
Motivation:
HashWheelTimerTest has busy/wait and sleep statements which are not necessary. We also depend upon a com.google.common.base.Supplier which isn't necessary.

Modifications:
- Remove buys wait loops and timeouts where possible

Result:
HashWheelTimerTest more explicit in verifying conditions and less reliant on wait times.
2016-11-14 16:17:18 -08:00
Aniket Bhatnagar
3f20b8adee Added optional pending timeouts counter parameter to HashedWheelTimer constructor and ensured that pending timeouts don't exceed provided max pending timeouts.
Motivation:
If the rate at which new timeouts are created is very high and the created timeouts are not cancelled, then the JVM can crash because of out of heap space. There should be a guard in the implementation to prevent this.

Modifications:
The constructor of HashedWheelTimer now takes an optional max pending timeouts parameter beyond which it will reject new timeouts by throwing RejectedExecutionException.

Result:
After this change, if the max pending timeouts parameter is passed as constructor argument to HashedWheelTimer, then it keeps a track of pending timeouts that aren't yet expired or cancelled. When a new timeout is being created, it checks for current pending timeouts and if it's equal to or greater than provided max pending timeouts, then it throws RejectedExecutionException.
2016-11-09 10:58:35 +01:00
Norman Maurer
273778c96c Correctly add futures to list in test
Motivation:

We missed to add the futures to the list in the test.

Modifications:

Add futures to list.

Result:

More correct test.
2016-11-01 10:23:03 +01:00
radai-rosenblatt
15ac6c4a1f Clean-up unused imports
Motivation:

the build doesnt seem to enforce this, so they piled up

Modifications:

removed unused import lines

Result:

less unused imports

Signed-off-by: radai-rosenblatt <radai.rosenblatt@gmail.com>
2016-09-30 09:08:50 +02:00
Scott Mitchell
506ac2ca71 NetUtil.bytesToIpAddress bug
Motivation:
NetUtil.bytesToIpAddress does not correctly translate IPv4 address to String. Also IPv6 addresses may not follow minimization conventions when converting to a String (see rfc 5952).

Modifications:
- NetUtil.bytesToIpAddress should correctly handle negative byte values for IPv4
- NetUtil.bytesToIpAddress should leverage existing to string conversion code in NetUtil

Result:
Fixes https://github.com/netty/netty/issues/5821
2016-09-22 17:06:21 -07:00
Norman Maurer
d44017189e Remove extra conditional check in retain
Motivation:

We not need to do an extra conditional check in retain(...) as we can just check for overflow after we did the increment.

Modifications:

- Remove extra conditional check
- Add test code.

Result:

One conditional check less.
2016-08-05 13:09:26 +02:00
Norman Maurer
fc14ca31cb Add NonStickyEventExecutorGroup
Motivation:

We offer DefaultEventExecutorGroup as an EventExecutorGroup which return OrderedEventExecutor and so provide strict ordering of event execution. One limitations of this implementation is that each contained DefaultEventExecutor will always be tied to a single thread, which can lead to a very unbalanced execution as one thread may be super busy while others are idling.

Modifications:

- Add NonStickyEventExecutorGroup which can be used to wrap another EventExecutorGroup (like UnorderedThreadPoolEventExecutor) and expose ordering while not be sticky with the thread that is used for a given EventExecutor. This basically means that Threads may change between execution of tasks for an EventExecutor but ordering is still guaranteed.

Result:

Better utalization of threads in some use-cases.
2016-08-04 06:30:59 +02:00
Norman Maurer
d3dc9c9e74 Allow to limit the maximum number of WeakOrderQueue instances per Thread.
Motivation:

To better restrict resource usage we should limit the number of WeakOrderQueue instances per Thread. Once this limit is reached object that are recycled from a different Thread then the allocation Thread are dropped on the floor.

Modifications:

Add new system property io.netty.recycler.maxDelayedQueuesPerThread and constructor that allows to limit the max number of WeakOrderQueue instances per Thread for Recycler instance. The default is 2 * cores (the same as the default number of EventLoop instances per EventLoopGroup).

Result:

Better way to restrict resource / memory usage per Recycler instance.
2016-08-04 06:23:14 +02:00
Dmitriy Dumanskiy
a80ea46b8e Removed custom split method as it is not effective anymore. 2016-08-01 21:49:33 +02:00
Norman Maurer
d92c5f5f5b Introduce allocation / pooling ratio in Recycler
Motivation:

At the moment the Recyler is very sensitive to allocation bursts which means that if there is a need for X objects for only one time these will most likely end up in the Recycler and sit there forever as the normal workload only need a subset of this number.

Modifications:

Add a ratio which sets how many objects should be pooled for each new allocation. This allows to slowly increase the number of objects in the Recycler while not be to sensitive for bursts.

Result:

Less unused objects in the Recycler if allocation rate sometimes bursts.
2016-07-29 15:20:39 +02:00
Norman Maurer
d315f1b3ba [#5551] Fix sporadic GlobalEventExecutorTest.testAutomaticStartStop test failure
Motivation:

We saw some sporadic test failures for GlobalEventExecutorTest.testAutomaticStartStop test. This is caused parallel execution of tests in combination with assert checks that will be affected.

Modifications:

Remove fragile assert checks.

Result:

No more sporadic test failures
2016-07-25 08:26:30 +02:00
Jason Tedor
e00b797936 Acquire Java version simply
Motivation:

The Java version is used for platform dependent logic. Yet, the logic
for acquiring the Java version requires special permissions (the runtime
permission "getClassLoader") that some downstream projects will never
grant. As such, these projects are doomed to have Netty act is their
Java major version is six.  While there are ways to maintain the same
logic without requiring these special permissions, the logic is
needlessly complicated because it relies on loading classes that exist
in version n but not version n - 1. This complexity can be removed. As a
bonanza, the dangerous permission is no longer required.

Modifications:

Rather than attempting to load classes that exist in version n but not
in version n - 1, we can just parse the Java specification version. This
only requires a begign property (property permission
"java.specification.version") and is simple.

Result:

Acquisition of the Java version is safe and simple.
2016-07-19 18:51:25 +02:00
Jason Tedor
27520f5208 Non-sticky thread groups in DefaultThreadFactory
Motivation:

A recent change to DefaultThreadFactory modified it so that it is sticky
with respect to thread groups. In particular, this change made it so
that DefaultThreadFactory would hold on to the thread group that created
it, and then use that thread group to create threads.

This can have problematic semantics since it can lead to violations of a
tenet of thread groups that a thread can only modify threads in its own
thread group and descendant thread groups. With a sticky thread group, a
thread triggering the creation of a new thread via
DefaultThreadFactory#newThread will be modifying a thread from the
sticky thread group which will not necessarily be its own nor a
descendant thread group. When a security manager is in place that
enforces this requirement, these modifications are now impossible. This
is especially problematic in the context of Netty because certain global
singletons like GlobalEventExecutor will create a
DefaultThreadFactory. If all DefaultThreadFactory instances are sticky
about their thread groups, it means that submitting tasks to the
GlobalEventExecutor singleton can cause a thread to be created from the
DefaultThreadFactory sticky thread group, exactly the problem with
DefaultThreadFactory being sticky about thread groups. A similar problem
arises from the ThreadDeathWatcher.

Modifications:

This commit modifies DefaultThreadFactory so that a null thread group
can be set with the behavior that all threads created by such an
instance will inherit the default thread group (the thread group
provided by the security manager if there is one, otherwise the thread
group of the creating thread). The construction of the instances of
DefaultThreadFactory used by the GlobalEventExecutor singleton and
ThreadDeathWatcher are modified to use this behavior. Additionally, we
also modify the chained constructor invocations of the
DefaultThreadFactory that do not have a parameter to specify a thread
group to use the thread group from the security manager is available,
otherwise the creating thread's thread group. We also add unit tests
ensuring that all of this behavior is maintained.

Result:

It will be possible to have DefaultThreadFactory instances that are not
sticky about the thread group that led to their creation. Instead,
threads created by such a DefaultThreadFactory will inherit the default
thread group which will either be the thread group from the security
manager or the current thread's thread group.
2016-07-14 22:06:40 +02:00
Norman Maurer
afafadd3d7 [#5505] Enforce Recycler limit when recycling from different threads
Motivation:

Currently, the recycler max capacity it's only enforced on the
thread-local stack which is used when the recycling happens on the
same thread that requested the object.

When the recycling happens in a different thread, then the objects
will be queued into a linked list (where each node holds N objects,
default=16). These objects are then transfered into the stack when
new objects are requested and the stack is empty.

The problem is that the queue doesn't have a max capacity and that
can lead to bad scenarios. Eg:

- Allocate 1M object from recycler
- Recycle all of them from different thread
- Recycler WeakOrderQueue will contain 1M objects
- Reference graph will be very long to traverse and GC timeseems to be negatively impacted
- Size of the queue will never shrink after this

Modifications:

Add some shared counter which is used to manage capacity limits when recycle from different thread then the allocation thread. We modify the counter whenever we allocate a new Link to reduce the overhead of increment / decrement it.

Result:

More predictable number of objects mantained in the recycler pool.
2016-07-14 07:56:03 +02:00
Jason Tedor
cfd6db7915 Avoid missed signals on a default promise
Motivation:

Today when awaiting uninterruptibly on a default promise, a race
condition can lead to a missed signal. Quite simply, the check for
whether the condition holds is not made inside a lock before
waiting. This means that the waiting thread can enter the wait after the
promise has completed and will thus not be notified, thus missing the
signal. This leads to the waiting thread to enter a timed wait that will
only trip with the timeout elapses leading to unnecessarily long waits
(imagine a connection timeout, and the waiting thread missed the signal
that the connection is ready).

Modification:

This commit fixes this missed signal by checking the condition inside a
lock. We also add a test that reliably fails without the non-racy
condition check.

Result:

Timed uninterruptible waits on default promise will not race against the
condition and possibly wait longer than necessary.
2016-07-12 09:57:14 +02:00
Norman Maurer
ce95c50444 [#5507] SingleThreadEventExecutor should reject call invoke*() from within the EventLoop.
Motivation:

ExecutorService.invoke*(...) methods may block by API definition. This can lead to deadlocks if called from inside the EventLoop in SingleThreadEventExecutor as it only has one Thread that does all the work.

Modifications:

Throw a RejectedExectionException if someone tries to call SingleThreadEventExecutor.invoke*(...) while in the EventLoop.

Result:

No more deadlock possible.
2016-07-09 08:08:50 +02:00
Julien Viet
79c8ec4d33 DnsNameResolver search domains support
Motivation:

The current DnsNameResolver does not support search domains resolution. Search domains resolution is supported out of the box by the java.net resolver, making the DnsNameResolver not able to be a drop in replacement for io.netty.resolver.DefaultNameResolver.

Modifications:

The DnsNameResolverContext resolution has been modified to resolve a list of search path first when it is configured so. The resolve method now uses the following algorithm:

if (hostname is absolute (start with dot) || no search domains) {
 searchAsIs
} else {
  if (numDots(name) >= ndots) {
    searchAsIs
  }
  if (searchAsIs wasn't performed or failed) {
    searchWithSearchDomainsSequenciallyUntilOneSucceeds
  }
}

The DnsNameResolverBuilder provides configuration for the search domains and the ndots value. The default search domains value is configured with the OS search domains using the same native configuration the java.net resolver uses.

Result:

The DnsNameResolver performs search domains resolution when they are present.
2016-07-08 22:04:01 +02:00
墨睿
be3e6972a1 Add a test case to check sub strings' AsciiString hash code
Motivation:

AsciiString.hashCode(o) , if "o" is a subString, the hash code is not always same, when netty’s version is 4.1.1.Final and jdk’s version is 1.6.

Modifications:

Use a test to assert hash codes are equal between a new string and any sub string (a part of  a char array),If their values are equal.

Result:

Create a test method to AsciiStringCharacterTest.
2016-07-04 07:19:52 +02:00
Scott Mitchell
a7f7d9c8e0 Remove unsafe char[] access in PlatformDependent
Motivation:
PlatformDependent attempts to use reflection to get the underlying char[] (or byte[]) from String objects. This is fragile as if the String implementation does not utilize the full array, and instead uses a subset of the array, this optimization is invalid. OpenJDK6 and some earlier versions of OpenJDK7 String have the capability to use a subsection of the underlying char[].

Modifications:
- PlatformDependent should not attempt to use the underlying array from String (or other data types) via reflection

Result:
PlatformDependent hash code generation for CharSequence does not depend upon specific JDK implementation details.
2016-06-30 08:58:28 -07:00
Scott Mitchell
70651cc58d HpackUtil.equals performance improvement
Motivation:
PR #5355 modified interfaces to reduce GC related to the HPACK code. However this came with an anticipated performance regression related to HpackUtil.equals due to AsciiString's increase cost of charAt(..). We should mitigate this performance regression.

Modifications:
- Introduce an equals method in PlatformDependent which doesn't leak timing information and use this in HpcakUtil.equals

Result:
Fixes https://github.com/netty/netty/issues/5436
2016-06-27 14:37:39 -07:00
Scott Mitchell
f6ad9df8ac DefaultPromise StackOverflowError protection updates
Modifications:
DefaultPromise provides a ThreadLocal queue to protect against StackOverflowError because of executors which may immediately execute runnables instead of queue them (i.e. ImmediateEventExecutor). However this may be better addressed by fixing these executors to protect against StackOverflowError instead of just fixing for a single use case. Also the most commonly used executors already provide the desired behavior and don't need the additional overhead of a ThreadLocal queue in DefaultPromise.

Modifications:
- Remove ThreadLocal queue from DefaultPromise
- Change ImmediateEventExecutor so it maintains a queue of runnables if reentrant condition occurs

Result:
DefaultPromise StackOverflowError code is simpler, and ImmediateEventExecutor protects against StackOverflowError.
2016-06-02 09:22:47 -07:00
Scott Mitchell
b3c56d5f69 DefaultPromise StackOverflowError protection
Motivation:
f2ed3e6ce8 removed the previous mechanism for StackOverflowError because it didn't work in all cases (i.e. ImmediateExecutor). However if a chain of listeners which complete other promises is formed there is still a possibility of a StackOverflowError.

Modifications:
- Use a ThreadLocal to save any DefaultPromises which could not be notified due to the stack being too large. After the first DefaultPromise on the stack completes notification this ThreadLocal should be used to notify any DefaultPromises which have not yet been notified.

Result:
DefaultPromise has StackOverflowError protection that works with all EventExecutor types.
2016-05-24 14:59:27 -07:00
Norman Maurer
e10dca7601 Mark Recycler.recycle(...) deprecated and update usage.
Motivation:

Recycler.recycle(...) should not be used anymore and be replaced by Handle.recycle().

Modifications:

Mark it as deprecated and update usage.

Result:

Correctly document deprecated api.
2016-05-20 22:11:31 +02:00
Trustin Lee
f11a35af32 Replace DomainMappingBuilder with DomainNameMappingBuilder
Motivation:

DomainMappingBuilder should have been named as DomainNameMappingBuilder
because it builds a DomainNameMapping.

Modifications:

- Add DomainNameMappingBuilder that does the same job with
  DomainMappingBuilder
- Deprecate DomainMappingBuilder and delegate its logic to
  DomainNameMappingBuilder
- Remove the references to the deprecated methods and classes related
  with domain name mapping
- Miscellaneous:
  - Fix Javadoc of DomainNameMapping.asMap()
  - Pre-create the unmodifiable map in DomainNameMapping

Result:

- Consistent naming
- Less use of deprecated API
2016-05-18 12:03:14 +02:00
Trustin Lee
ea1f60dbf0 Replace DomainNameMapping.entries() with asMap()
Motivation:

DomainNameMapping.entries() returns Set<Map.Entry<String, V>>, which
doesn't sound very natural.

Modifications:

Replace entries() with asMap() which returns a Map<String, V> instead.

Result:

- Better looking API
- User can do a lookup because it's a Map
2016-05-18 11:18:10 +02:00
Xiaoyan Lin
a56ef03f24 Add DomainNameMapping.entries to allow retrieving the domain match lists
Motivation:

See #4200.

Modifications:

Add DomainNameMapping.entries to allow retrieving the domain match lists.

Result:

People can use DomainNameMapping.entries to retrive the match list in DomainNameMapping.
2016-05-17 09:47:45 +02:00
Scott Mitchell
88a52f7367 KObjectHashMap PrimitiveIterator NPE
Motivation:
KObjectHashMap.remove(int index) attempts to move back items which may have been displaced because their spot in the hash based array was taken by another item. If this happens the nextIndex reference in PrimitiveIterator will not be updated. At this time the PrimitiveEntry will reference the incorrect index and may result in a NPE.

Modifications:
- If KObjectHashMap.remove(int index) moves entries back then PrimitiveIterator should adjust its nextIndex

Result:
PrimitiveIterator.remove() updates its internal state to reference the new nextIndex and will not NPE.
Fixes https://github.com/netty/netty/issues/5198
2016-05-13 12:44:29 -07:00
Scott Mitchell
f2ed3e6ce8 DefaultPromise LateListener Logic Issues
Motivation:
The LateListener logic is prone to infinite loops and relies on being processed in the EventExecutor's thread for synchronization, but this EventExecutor may not be constant. An infinite loop can occur if the EventExecutor's execute method does not introduce a context switch in LateListener.run. The EventExecutor can be changed by classes which inherit from DefaultPromise. For example the DefaultChannelPromise will return w/e EventLoop the channel is registered to, but this EventLoop can change (re-registration).

Modifications:
- Remove the LateListener concept and instead use a single Object to maintain the listeners while still preserving notification order
- Make the result member variable an atomic variable so it can be outside the synchronized(this) blocks
- Cleanup/simplify existing state management code

Result:
Fixes https://github.com/netty/netty/issues/5185
2016-05-09 10:33:40 -07:00
Daniel Bevenius
ffd2450e11 Add logging configuration to pom.xml
Motivation:
Currently the default log level when running tests is debug. When
running the build on the CI server it might be nice to avoid this debug
level and allow for the level to be configured.

Modifications:
Added a logback-test.xml configuration that has been added to the
common module. This allows for the logLevel to be configured.
The default level will still be debug.

Result:
The log level can now be configured from the command line:
$ mvn test -DlogLevel=error
2016-05-09 15:18:01 +02:00
Xiaoyan Lin
bc6adab48c Make all InternalLoggerFactory implementations be singletons
Motivation:

It's better to make all InternalLoggerFactory implementations be singletons according to the discussions in #5047

Modifications:

Make all InternalLoggerFactory implementations be singletons and hide the construtors.

Result:

All InternalLoggerFactory implementations be singletons.
2016-04-08 09:39:32 +02:00
Xiaoyan Lin
105df33d8d Add Log4J2LoggerFactory and Log4J2Logger
Motivation:

See #3095

Modifications:

Add Log4J2LoggerFactory and Log4J2Logger which is an InternalLogger implementation based on log4j2.

Result:

The user can use log4j2 directly without a special slf4j binding.
2016-04-05 14:01:32 +02:00
Scott Mitchell
45849b2fa8 Deprecate PromiseAggregator
Motivation:
PromiseAggregator's API allows for the aggregate promise to complete before the user is done adding promises. In order to support this use case the API structure would need to change in a breaking manner.

Modifications:
- Deprecate PromiseAggregator and subclasses
- Introduce PromiseCombiner which corrects these issues

Result:
PromiseCombiner corrects the deficiencies in PromiseAggregator.
2016-03-14 10:53:30 -07:00
Dmitry Spikhalskiy
0d3eda38e1 Helper method to get mime-type from Content-Type header of HttpMessage 2016-03-03 15:18:39 +01:00
Scott Mitchell
050ac709ba PromiseNotifier does not propagate cancel events
Motivation:
If the Future that the PromiseNotifier is listening to is cancelled, it does not propagate the cancel to all the promises it is expected to notify.

Modifications:
- If the future is cancelled then all the promises should be cancelled
- Add a UnaryPromiseNotifier if a collection of promises is not necessary

Result:
PromiseNotifier propagates cancel events to all promises
2016-02-19 10:16:04 -08:00
Moses Nakamura
f0f0b69d90 fixed "sensative" typo to read "sensitive" 2016-02-17 08:18:11 -08:00
Xiaoyan Lin
333f55e9ce Add unescapeCsvFields to parse a CSV line and implement CombinedHttpHeaders.getAll
Motivation:

See #4855

Modifications:

Unfortunately, unescapeCsv cannot be used here because the input could be a CSV line like `"a,b",c`. Hence this patch adds unescapeCsvFields to parse a CSV line and split it into multiple fields and unescaped them. The unit tests should define the behavior of unescapeCsvFields.

Then this patch just uses unescapeCsvFields to implement `CombinedHttpHeaders.getAll`.

Result:

`CombinedHttpHeaders.getAll` will return the unescaped values of a header.
2016-02-15 15:26:15 -08:00
Xiaoyan Lin
f43dc7d551 Add unescapeCsv to StringUtil
Motivation:

See #3435

Modifications:

Add unescapeCsv to StringUtil

Result:

StringUtil has the counter part of escapeCsv: unescapeCsv
2016-02-08 14:35:30 -08:00
Xiaoyan Lin
ff11fe894d Fix the last value in AsciiString.trim
Motivation:

In AsciiString.trim, last should be `arrayOffset() + length() - 1`. See #4741.

Modifications:

Fix the last value.

Result:

AsciiString.trim works correctly.
2016-01-22 14:06:30 +01:00