Commit Graph

1264 Commits

Author SHA1 Message Date
Norman Maurer 411e786a6f Add default methods to EventExecutor / EventExecutorGroup / EventLoop / EventLoopGroup
Motivation:

We can remove some classes and duplication if we add default methods

Modifications:

- Add default methods to EventExecutor / EventExecutorGroup / EventLoop / EventLoopGroup
- Remove code duplication
- Remove AbstractEventExecutorGroup as it is not needed anymore

Result:

Cleanup and removal of code-duplication. Also makes it easier for people to implement their custom executors / groups
2021-09-02 20:12:56 +02:00
Norman Maurer cba3b4dd57
Change visibility of DefaultPromise to push users to use EventExecutor methods (#11646)
Motivation:

While we use DefaultPromise as our implementation of Promise and Future users should not really use it directly. Users should always use the EventExecutor / EventLoop to create a Promise / Future.

Modifications:

- Change static Promise methods to be package-private
- Add default implementations for Promise and Future creation to EventExecutor
- Change public constructor to protected
- Remove usage of DefaultPromise in our tests

Result:

Less likely users will depend on the actual Promise implementation
2021-09-02 19:11:01 +02:00
Chris Vest 0cb4cc4e49
Make Promise not extend Future (#11634)
Motivation:
We wish to separate these two into clearer write/read interfaces.
In particular, we don't want to be able to add listeners to promises, because it makes it easy to add them out of order.
We can't prevent it entirely, because any promise can be freely converted to a future where listeners can be added.
We can, however, discourage this in the API.

Modification:
The Promise interface no longer extends the Future interface.
Numerous changes to make the project compile and its tests run.

Result:
Clearer separation of concerns in the code.
2021-09-02 10:46:54 +02:00
Norman Maurer e04f48d802
Add cascadeTo methods to Future (#11623)
Motivation:

The need of cascade from a Future to a Promise exists. We should add some default implementation for it.

Modifications:

- Merge PromiseNotifier into Futures
- Add default cascadeTo(...) methods to Future
- Add tests to FuturesTest
- Replace usage of PromiseNotifier with Future.cascadeTo
- Use combination of map(...) and cascadeTo(...) in *Bootstrap to reduce code duplication

Result:

Provide default implementation of cascadeTo.
2021-08-29 15:44:34 +02:00
Chris Vest 445f747ce3
Add a `Futures` class with static `map()` and `flatMap()` methods (#11607)
Motivation:
Making futures easier to compose, combine, and extend is useful to have as part of the API, since implementing this correctly and efficiently can be tricky.

Modification:
Add `Future.map(Function<V,R>) -> Future<R>` and `Future.flatMap(Function<V,Future<R>>) -> Future<R>` default methods to the `Future` interface.
These methods return new Future instance, that will be completed when the original future completes, and the result will be processed through the given mapping function.
These two methods take care to propagate cancellation and exceptions correctly:
Cancellation propagates both ways between the new and original future.
Failures only propagate from the original future to the returned new Future instance.

Result:
A few convenient methods for modifying and composing futures.

This PR fixes #8523, and perhaps also #2105
2021-08-26 11:23:12 +02:00
Chris Vest edf4e28afa
Change `!future.isSuccess()` to `future.isFailed()` where it makes sense (#11616)
Motivation:
The expression "not is success" can mean that either the future failed, or it has not yet completed.
However, many places where such an expression is used is expecting the future to have completed.
Specifically, they are expecting to be able to call `cause()` on the future.
It is both more correct, and semantically clearer, to call `isFailed()` instead of `!isSuccess()`.

Modification:
Change all places that used `!isSuccess()` to  mean that the future had failed, to use `isFailed()`.
A few places are relying on `isSuccess()` returning `false` for _incomplete_ futures, and these places have been left unchanged.

Result:
Clearer code, with potentially fewer latent bugs.
2021-08-26 09:43:17 +02:00
Chris Vest b8e1341142
Future methods getNow() and cause() now throw on incomplete futures (#11594)
Motivation:
Since most futures in Netty are of the `Void` type, methods like `getNow()` and `cause()` cannot distinguish if the future has finished or not.
This can cause data race bugs which, in the case of `Void` futures, can be silent.

Modification:
The methods `getNow()` and `cause()` now throw an `IllegalStateException` if the future has not yet completed.
Most use of these methods are inside listeners, and so are not impacted.
One place in `AbstractBootstrap` was doing a racy read and has been adjusted.

Result:
Data race bugs around `getNow()` and `cause()` are no longer silent.
2021-08-24 15:47:27 +02:00
Norman Maurer 11cdf1d3cf
Use default methods in Channel (#11608)
Motivation:

We can make things easier for implementations by providing some default methods

Modifications:

- Add default methods to Channel
- Remove code from AbstractChannel

Result:

Easier to implement custom Channel
2021-08-24 09:34:50 +02:00
Chris Vest 7971a252a5
Clean up Future/Promises API (#11575)
Motivation:
The generics for the existing futures, promises, and listeners are too complicated.
This complication comes from the existence of `ChannelPromise` and `ChannelFuture`, which forces listeners to care about the particular _type_ of future being listened on.

Modification:
* Add a `FutureContextListener` which can take a context object as an additional argument. This allows our listeners to have the channel piped through to them, so they don't need to rely on the `ChannelFuture.channel()` method.
* Make the `FutureListener`, along with the `FutureContextListener` sibling, the default listener API, retiring the `GenericFutureListener` since we no longer need to abstract over the type of the future.
* Change all uses of `ChannelPromise` to `Promise<Void>`.
* Change all uses of `ChannelFuture` to `Future<Void>`.
* Change all uses of `GenericFutureListener` to either `FutureListener` or `FutureContextListener` as needed.
* Remove `ChannelFutureListener` and `GenericFutureListener`.
* Introduce a `ChannelFutureListeners` enum to house the constants that previously lived in `ChannelFutureListener`. These constants now implement `FutureContextListener` and take the `Channel` as a context.
* Remove `ChannelPromise` and `ChannelFuture` — all usages now rely on the plain `Future` and `Promise` APIs.
* Add static factory methods to `DefaultPromise` that allow us to create promises that are initialised as successful or failed.
* Remove `CompleteFuture`, `SucceededFuture`, `FailedFuture`, `CompleteChannelFuture`, `SucceededChannelFuture`, and `FailedChannelFuture`.
* Remove `ChannelPromiseNotifier`.

Result:
Cleaner generics and more straight forward code.
2021-08-20 09:55:16 +02:00
Chris Vest d27a2b3df9 Backport some fixes and cleanups for DefaultPromiseTest (#11600)
Motivation:
These cleanups were done in another PR but were not directly related to that PR.
This extracts those changes and backports them to 4.1.

Modification:
* Remove the use of mocking in DefaultPromiseTest.
* Fix a few warnings.
* Make `testStackOverFlowChainedFuturesB` test with the right listener chain.

Result:
Cleaner code.
2021-08-19 13:30:17 +02:00
Chris Vest 3e2e36eac5
Remove the deprecated ThreadDeathWatcher (#11574)
Motivation:
The deprecated ThreadDeathWatcher produces more garbage and can delay resource release, when compared to manual resource management.

Modification:
Remove the ThreadDeathWatcher and other deprecated APIs that rely on it.

Result:
Less deprecated code.
2021-08-16 14:33:58 +02:00
Chris Vest a3d5617d45
Remove deprecated stuff around ResourceLeakDetector (#11572)
Motivation:
A number of classes and APIs around the ResourceLeakDetector have been deprecated for removal in Netty 5.x, because better alternatives exist.

Modification:
Remove everything in and around ResourceLeakDetector that is deprecated, and fix the few usages that were found.

Result:
Less deprecated code.
2021-08-11 21:41:49 +02:00
Chris Vest 11fcfe1f73
Remove PromiseAggregator (#11571)
Motivation:
This class was deprecated, since a better alternative exists in `PromiseCombiner`.

Modification:
Remove `PromiseAggregator`, its Channel companion, and its test.

Result:
Less deprecated code.
2021-08-11 21:41:16 +02:00
Aayush Atharva 99bd5895dc Inline variables to make code more readable (#11565)
Motivation:
There are lots of redundant variable declarations which should be inlined to make good look better.

Modification:
Made variables inlined.

Result:
Less redundant variable and more readable code.
2021-08-11 17:07:32 +02:00
Aayush Atharva 25a0a6d425 Make variables final (#11548)
Motivation:
We should make variables `final` which are not reinstated again in code to match the code style and makes the code look better.

Modification:
Made couples of variables as `final`.

Result:
Variables marked as `final`.
2021-08-06 09:28:12 +02:00
Chris Vest ef203fa6cb
Fix a number of javadoc issues (#11544)
Motivation:
Let's have fewer warnings about broken, missing, or abuse of javadoc comments.

Modification:
Added descriptions to throws clauses that were missing them.
Remove link clauses from throws clauses - these are implied.
Turned some javadoc comments into block comments because they were not applied to APIs.
Use code clauses instead of code tags.

Result:
Fewer javadoc crimes.
2021-08-06 09:14:04 +02:00
Aayush Atharva b700793951 Remove Unused Imports (#11546)
Motivation:
There are lots of imports which are unused. We should get rid of them to make the code look better,

Modification:
Removed unused imports.

Result:
No unused imports.
2021-08-05 14:08:07 +02:00
Chris Vest 91e48902bb
Small cleanups (#11545)
Motivation:
Code that isn't used, or has better alternatives in the JDK should be removed or replaced, respectively.

Modification:
Remove dead code in DefaultPromise.
Replace the synchronised-based reachability fence in ResourceLeakDetector, with the JDK Reference.reachabilityFence.

Result:
Cleaner code
2021-08-05 10:37:52 +02:00
Chris Vest 6b11f7fbc2
All *Bootstrap methods that used to return ChannelFuture now return Future<Channel> (#11517)
Bootstrap methods now return Future<Channel> instead of ChannelFuture

Motivation:
In #8516 it was proposed to at some point remove the specialised ChannelFuture and ChannelPromise.
Or at least make them not extend Future and Promise, respectively.
One pain point encountered in this discussion is the need to get access to the channel object after it has been initialised, but without waiting for the channel registration to propagate through the pipeline.

Modification:
Add a Bootstrap.createUnregistered method, which will return a Channel directly.
All other Bootstrap methods that previously returned ChannelFuture now return Future<Channel>

Result:
It's now possible to obtain an initialised but unregistered channel from a bootstrap, without blocking.
And the other bootstrap methods now only release their channels through the result of their futures, preventing racy access to the channels.
2021-08-03 19:43:38 +02:00
Aayush Atharva 7d1de8c970 Add Increment and Decrement count positive check in ReferenceCountUtil (#11523)
Motivation:
We should have guards in place to check increment or decrement count in `ReferenceCountUtil`. Increment and Decrement counts must be a positive integer.

Modification:
Added `ObjectUtil#checkPositive` checks.

Result:
Prevent release due to invalid count.
2021-07-29 20:51:39 +02:00
Chris Vest fc922c98d8
Remove overrides of Throwable.fillInStackTrace (#11514)
Motivation:
Since Java 7, there are new constructors available that allow us to avoid initialising the stack traces of certain exceptions.

Modification:
Use these constructors instead of overriding Throwable.fillInStackTrace.

Result:
Cleaner code
2021-07-27 13:29:09 +02:00
old driver a8dc3f73d4 Optimize method io.netty.util.concurrent.DefaultPromise.await0(...) (#11504)
Motivation:

For the code pattern of `Object.wait(...)`  in `io.netty.util.concurrent.DefaultPromise.await0(...)`, it's better to follow the recommended code pattern according to [Object.wait(...)'s doc](https://docs.oracle.com/javase/7/docs/api/java/lang/Object.html#wait()):
```
synchronized (obj) {
     while (<condition does not hold>)
     obj.wait();
     ... // Perform action appropriate to condition
}
```

Modification:
Change the `Object.wait(...)`'s code pattern in `io.netty.util.concurrent.DefaultPromise.await0(...)`.

Result:

The `Object.wait(...)`'s code pattern in `io.netty.util.concurrent.DefaultPromise.await0(...)` meets the Java doc.
2021-07-26 08:33:12 +02:00
Norman Maurer 23d8fde855 Add PromiseNotifier static method which takes care of cancel propagation (#11494)
Motivation:

At the moment we not correctly propagate cancellation in some case when we use the PromiseNotifier.

Modifications:

- Add PromiseNotifier static method which takes care of cancellation
- Add unit test
- Deprecate ChannelPromiseNotifier

Result:

Correctly propagate cancellation of operation

Co-authored-by: Nitesh Kant <nitesh_kant@apple.com>
2021-07-21 13:47:43 +02:00
Aayush Atharva 55c4e2ca82 Introduce BrotliEncoder (#11256)
Motivation:
Currently, Netty only has BrotliDecoder which can decode Brotli encoded data. However, BrotliEncoder is missing which will encode normal data to Brotli encoded data.

Modification:
Added BrotliEncoder and CompressionOption

Result:
Fixes #6899.

Co-authored-by: Norman Maurer <norman_maurer@apple.com>
2021-07-08 12:01:28 +02:00
Norman Maurer 54aa4d9b68 Only run one SSL task per delegation (#11462)
Motivation:

We should only run one SSL task per delegation to allow more SSLEngines to make progress in a timely manner

Modifications:

- Only run one task per delegation to the executor
- Only create new SSL task if really needed
- Only schedule if not on the EventExecutor thread

Result:

More fair usage of resources and less allocations
2021-07-08 08:06:07 +02:00
Norman Maurer 26a0efcb93 Migrate common to junit5 (#11431) (#11438)
Motivation:

We should update to use junit5 in all modules.

Modifications:

Adjust common tests to use junit5

Result:

Part of https://github.com/netty/netty/issues/10757
2021-07-01 18:46:46 +02:00
Chris Vest 765f8989ca
Introduce alternative Buffer API (#11347)
Motivation:

In Netty 5 we wish to have a simpler, safe, future proof, and more consistent buffer API.
We developed such an API in the incubating buffer repository, and taking it through multiple rounds of review and adjustments.
This PR/commit bring the results of that work into the Netty 5 branch of the main Netty repository.

Modifications:

* `Buffer` is an interface, and all implementations are hidden behind it.
  There is no longer an inheritance hierarchy of abstract classes and implementations.
* Reference counting is gone.
  After a buffer has been allocated, calling `close` on it will deallocate it.
  It is then up to users and integrators to ensure that the life-times of buffers are managed correctly.
  This is usually not a problem as buffers tend to flow through the pipeline to be released after a terminal IO operation.
* Slice and duplicate methods are replaced with `split`.
  By removing slices, duplicate, and reference counting, there is no longer a possibility that a buffer and/or its memory can be shared and accessible through multiple routes.
  This solves the problem of data being accessed from multiple places in an uncoordinated way, and the problem of buffer memory being closed while being in use by some unsuspecting piece of code.
  Some adjustments will have to be made to other APIs, idioms, and usages, since `split` is not always a replacement for `slice` in some use cases.
* The `split` has been added which allows memory to be shared among multiple buffers, but in non-overlapping regions.
  When the memory regions don't overlap, it will not be possible for the different buffers to interfere with each other.
  An internal, and completely transparent, reference counting system ensures that the backing memory is released once the last buffer view is closed.
* A Send API has been introduced that can be used to enforce (in the type system) the transfer of buffer ownership.
  This is not expected to be used in the pipeline flow itself, but rather for other objects that wrap buffers and wish to avoid becoming "shared views" — the absence of "shared views" of memory is important for avoiding bugs in the absence of reference counting.
* A new BufferAllocator API, where the choice of implementation determines factors like on-/off-heap, pooling or not.
  How access to the different allocators will be exposed to integrators will be decided later.
  Perhaps they'll be directly accessible on the `ChannelHandlerContext`.
* The `PooledBufferAllocator` has been copied and modified to match the new allocator API.
  This includes unifying its implementation that was previously split across on-heap and off-heap.
* The `PooledBufferAllocator` implementation has also been adjusted to allocate 4 MiB chunks by default, and a few changes have been made to the implementation to make a newly created, empty allocator use significantly less heap memory.
* A `Resource` interface has been added, which defines the life-cycle methods and the `send` method.
  The `Buffer` interface extends this.
* Analogues for `ByteBufHolder` has been added in the `BufferHolder` and `BufferRef` classes.
* `ByteCursor` is added as a new way to iterate the data in buffers.
  The byte cursor API is designed to be more JIT friendly than an iterator, or the existing `ByteProcessor` interface.
* `CompositeBuffer` no longer permit the same level of access to its internal components.
  The composite buffer enforces its ownership of its components via the `Send` API, and the components can only be individually accessed with the `forEachReadable` and `forEachWritable` methods.
  This keeps the API and behavioral differences between composite and non-composite buffers to a minimum.
* Two implementations of the `Buffer` interface are provided with the API: One based on `ByteBuffer`, and one based on `sun.misc.Unsafe`.
  The `ByteBuffer` implementation is used by default.
  More implementations can be loaded from the classpath via service loading.
  The `MemorySegment` based implementation is left behind in the incubator repository.
* An extensive and highly parameterised test suite has been added, to ensure that all implementations have consistent and correct behaviour, regardless of their configuration or composition.

Result:

We have a new buffer API that is simpler, better tested, more consistent in behaviour, and safer by design, than the existing `ByteBuf` API.

The next legs of this journey will be about integrating this new API into Netty proper, and deprecate (and eventually remove) the `ByteBuf` API.

This fixes #11024, #8601, #8543, #8542, #8534, #3358, and #3306.
2021-06-28 12:06:44 +02:00
ping 79c2bb902d Recycler.WeakOrderQueue drop Object hasBeenRecycled (#11402)
Motivation:

WeakOrderQueue would drop object that has been recycled, even when it has space for it.
WeakOrderQueue#add should check DefaultHandler.hasBeenRecycler field  first

Modifications:

WeakOrderQueue test the DefaultHandler.hasBeenRecycler first

Result:

WeakOrderQueue would not drop object that has been recycled when there is space


Co-authored-by: Norman Maurer <norman_maurer@apple.com>
Co-authored-by: Trustin Lee <t@motd.kr>
2021-06-23 11:33:16 +02:00
Paulo Lopes 92b4357ea0 Update graal annotations dependencies GAV to allow license GPL2+CE (#11404)
Motivation:

We need to use a GraalVM dependency which uses GPL2 + CE.

Modifications:

- Update all graalvm dependencies to new GAV which introduces a license change from GPL2 to GPL2 + CE
- This also required a small bump on the general version from 19.2 to 19.3, which should be fine as 19.3 is an official maintained LTS version, while 19.2 wasn't.

Result:

Fixes: #11398

Signed-off-by: Paulo Lopes <pmlopes@gmail.com>
2021-06-21 16:11:57 +02:00
Norman Maurer 07baabaac5
Remove Progressive*Promise / Progressive*Future (#11374)
Motivation:

This special case implementation of Promise / Future requires the implementations responsible for completing the promise to have knowledge of this class to provide value. It also requires that the implementations are able to provide intermediate status while the work is being done. Even throughout the core of Netty it is not really supported most of the times and so just brings more complexity without real gain.

Let's remove it completely which is better then only support it sometimes.

Modifications:

Remove Progressive* API

Result:

Code cleanup.... Fixes https://github.com/netty/netty/issues/8519
2021-06-09 08:32:38 +02:00
Norman Maurer 1415938163
Remove Future.removeListener* and addListeners (#11375)
Motivation:

We didnt really have a good use-case for removeListener* and addListeners. Because of this we should just remove these methods and so make things simpler.

Modifications:

Remove methods

Result:

Cleanup
2021-06-09 08:31:02 +02:00
terrarier2111 73bafaaf9b Some minor cleanups for TypeParameterMatcher (#11286)
Motivation:

The code in TypeParamterMatcher can be simplified and cleaned up.

Modification:

Cleanup

Result:

Cleanup
2021-05-27 09:02:53 +02:00
Riley Park e0b9eeb324
Migrate common tests to JUnit 5 (#11319)
Motivation:

JUnit 5 is more expressive, extensible, and composable in many ways, and it's better able to run tests in parallel.

Modifications:

Use JUnit5 in tests

Result:

Related to https://github.com/netty/netty/issues/10757
2021-05-27 08:31:53 +02:00
Roman Fedorov 9f3f0590a2 Delete obsolete javadoc warning about pendingTasks() method expensiveness (#11287)
Co-authored-by: Roman Fedorov <roman.fedorov@spikeapp.com>
2021-05-22 09:35:37 +02:00
old driver 11de7b65c1 Change docs of method io.netty.util.internal.MathUtil#isOutOfBounds (#11281)
Motivation:

1. The docs about the 'retun value' of the method `io.netty.util.internal.MathUtil#isOutOfBounds` is not correct.
2. The capacity parameter should be checked for overflowed case.

Modification:

1. Changed the doc to:

>  @return {@code false} if the requested {@code index} and {@code length} will fit within {@code capacity}.
>  {@code true} if this would result in an index out of bounds exception.

2. Improved the bounder checking logic.

Result:
Fixes #11279 
Fixes #11280
2021-05-19 21:35:29 +02:00
old driver 7c3810fbdf make Slf4JLoggerFactory be singleton (#11253)
Motivation:

The current initialization of Slf4JLoggerFactory is not singleton.

Modification:

Use Slf4JLoggerFactory.INSTANCE to initialize Slf4JLoggerFactory.

Result:

The instance of Slf4JLoggerFactory became a singleton.
2021-05-17 09:10:35 +02:00
Violeta Georgieva 7f04b28bc7 Add DefaultHostsFileEntriesResolver#addresses to provide all hosts file's entries for a hostname (#11246)
Motivation:
DefaultHostsFileEntriesResolver should provide all hosts file's entries for a hostname when
DnsNameResolver#resolveAll as opposed to the current implementation where only the first
entry is taken into consideration

Modification:
- Add DefaultHostsFileEntriesResolver#addresses to provide all hosts file's entries for a hostname
- Add HostsFileEntriesProvider to provide all hosts file's entries for a hostname and to keep
backwards compatibility for HostsFileEntries and HostsFileParser
- DnsNameResolver#resolveAll uses the new DefaultHostsFileEntriesResolver#addresses
- BlockHound configuration: replace HostsFileParser#parse with HostsFileEntriesProvider$ParserImpl#parse
as the latter does the parsing
- Add junit tests

Result:
Fixes #10834
2021-05-14 10:01:11 +02:00
Idel Pivnitskiy b9685a63de Use `PlatformDependent#normalizedOs()` instead of reading `os.name` prop (#11239)
Motivation:

`PlatformDependent#normalizedOs()` already caches normalized variant of
the value of `os.name` system property. Instead of inconsistently
normalizing it in every case, use the utility method.

Modifications:

- `PlatformDependent`: `isWindows0()` and `isOsx0()` use `NORMALIZED_OS`;
- `PlatformDependent#normalizeOs(String)` define `darwin` as `osx`;
- `OpenSsl#loadTcNative()` does not require `equalsIgnoreCase` bcz `os`
is already normalized;
- Epoll and KQueue: `Native#loadNativeLibrary()` use `normalizedOs()`;
- Use consistent `Locale.US` for lower case conversion of `os.name`;
- `MacOSDnsServerAddressStreamProvider#loadNativeLibrary()` uses
`PlatformDependent.isOsx()`;

Result:

Consistent approach for `os.name` parsing.
2021-05-11 08:53:47 +02:00
Norman Maurer 968dfbb378 Preload classes before calling native OnLoad function to prevent clas… (#11215)
Motivation:

It turns out it is quite easy to cause a classloader deadlock in more recent java updates if you cause classloading while you are in native code. Because of this we should just workaround this issue by pre-load all the classes that needs to be accessed in the OnLoad function.

Modifications:

- Preload all classes that would otherwise be loaded by native OnLoad functions.

Result:

Workaround for https://github.com/netty/netty/issues/11209 and https://bugs.openjdk.java.net/browse/JDK-8266310
2021-05-03 10:23:57 +02:00
Boris Unckel 0a2a24f39d Utilize i.n.u.internal.ObjectUtil to assert Preconditions (commons) (#11170) (#11172)
Motivation:

NullChecks resulting in a NullPointerException or IllegalArgumentException, numeric ranges (>0, >=0) checks, not empty strings/arrays checks must never be anonymous but with the parameter or variable name which is checked. They must be specific and should not be done with an "OR-Logic" (if a == null || b == null) throw new NullPointerEx.

Modifications:

* Add some checks to ObjectUtil not present today but utilized in the code.
* Add unit test for ObjectUtil
* Update commmons internal usage with ObjectUtil

Result:

All checks needed are present, subsequent changes of usage of ObjectUtil are possible.

Fixes for https://github.com/netty/netty/issues/11170
2021-04-22 08:15:40 +02:00
Chris Vest d1b896b701 Log fewer stack traces from initialisation code (#11164)
Motivation:
We are increasingly running in environments where Unsafe, setAccessible, etc. are not available.
When debug logging is enabled, we log a complete stack trace every time one of these initialisations fail.
Seeing these stack traces can cause people unnecessary concern.
For instance, people might have alerts that are triggered by a stack trace showing up in logs, regardless of its log level.

Modification:
We continue to print debug log messages on the result of our initialisations, but now we only include the full stack trace is _trace_ logging (or FINEST, or equivalent in whatever logging framework is configured) is enabled.

Result:
We now only log these initialisation stack traces when the lowest possible log level is enabled.

Fixes #7817
2021-04-19 09:17:32 +02:00
Norman Maurer 50a83ec8a8 DefaultThreadFactory must not use Thread.currentThread() when constructed without ThreadGroup (#11119)
Motivation:

We had a bug in out DefaulThreadFactory as it always retrieved the ThreadGroup to used during creation time when now explicit ThreadGroup was given. This is problematic as the Thread may die and so the ThreadGroup is destroyed even tho the DefaultThreadFactory is still used.

This could produce exceptions like:

java.lang.IllegalThreadStateException
        at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867)
        at java.lang.Thread.init(Thread.java:405)
        at java.lang.Thread.init(Thread.java:349)
        at java.lang.Thread.<init>(Thread.java:599)
        at io.netty.util.concurrent.FastThreadLocalThread.<init>(FastThreadLocalThread.java:60)
        at io.netty.util.concurrent.DefaultThreadFactory.newThread(DefaultThreadFactory.java:122)
        at io.netty.util.concurrent.DefaultThreadFactory.newThread(DefaultThreadFactory.java:106)
        at io.netty.util.concurrent.ThreadPerTaskExecutor.execute(ThreadPerTaskExecutor.java:32)
        at io.netty.util.internal.ThreadExecutorMap$1.execute(ThreadExecutorMap.java:57)
        at io.netty.util.concurrent.SingleThreadEventExecutor.doStartThread(SingleThreadEventExecutor.java:978)
        at io.netty.util.concurrent.SingleThreadEventExecutor.startThread(SingleThreadEventExecutor.java:947)
        at io.netty.util.concurrent.SingleThreadEventExecutor.execute(SingleThreadEventExecutor.java:830)
        at io.netty.util.concurrent.SingleThreadEventExecutor.execute(SingleThreadEventExecutor.java:818)
        at io.netty.channel.AbstractChannel$AbstractUnsafe.register(AbstractChannel.java:471)
        at io.netty.channel.SingleThreadEventLoop.register(SingleThreadEventLoop.java:87)
        at io.netty.channel.SingleThreadEventLoop.register(SingleThreadEventLoop.java:81)
        at io.netty.channel.MultithreadEventLoopGroup.register(MultithreadEventLoopGroup.java:86)
        at io.netty.bootstrap.AbstractBootstrap.initAndRegister(AbstractBootstrap.java:323)
        at io.netty.bootstrap.AbstractBootstrap.doBind(AbstractBootstrap.java:272)
        at io.netty.bootstrap.AbstractBootstrap.bind(AbstractBootstrap.java:239)
        at io.netty.incubator.codec.quic.QuicTestUtils.newServer(QuicTestUtils.java:138)
        at io.netty.incubator.codec.quic.QuicTestUtils.newServer(QuicTestUtils.java:143)
        at io.netty.incubator.codec.quic.QuicTestUtils.newServer(QuicTestUtils.java:147)
        at io.netty.incubator.codec.quic.QuicStreamFrameTest.testCloseHalfClosure(QuicStreamFrameTest.java:48)
        at io.netty.incubator.codec.quic.QuicStreamFrameTest.testCloseHalfClosureUnidirectional(QuicStreamFrameTest.java:35)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
        at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
        at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
        at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
        at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:288)
        at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:282)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.lang.Thread.run(Thread.java:748)

Modifications:

- If the user dont specify a ThreadGroup we will just pass null to the constructor of FastThreadLocalThread and so let it retrieve on creation time
- Adjust tests

Result:

Don't risk to see IllegalThreadStateExceptions.
2021-03-26 18:46:55 +01:00
Chris Vest aa1b887536 Fix offset type for new PlatformDependent.put* methods
Motivation:
The offsets were accidentally typed as int, where they should have been typed as long.

Modification:
Change type of offset arguments to PlatformDependent.put*(Object,int,?) from int to long.

Result:
It is now possible to use these methods to store to memory at absolute memory addresses.
2021-03-25 16:49:43 +01:00
Chris Vest 8ae59c1e7b Expose on/off heap agnostic Unsafe accessor methods
Motivation:
These are necessary for creating a buffer implementation that uses Unsafe, and works generically for both on-heap and off-heap memory.

Modification:
PlatformDependent previously forced clients to decide if they are working on on-heap memory, or off-heap memory, by giving accessors distinct APIs for each.
What is added here, are generic accessors that work the same in either case.

Result:
We can now make an Unsafe-based buffer implementation that is agnostic to whether the memory is on- or off-heap.
2021-03-25 11:38:18 +01:00
Chris Vest 9ba653c851 Fix alignment handling for pooled direct buffers (#11106)
Motivation:
Alignment handling was broken, and basically turned into a fixed offset into each allocation address regardless of its initial value, instead of ensuring that the allocated address is either aligned or bumped to the nearest alignment offset.
The brokenness of the alignment handling extended so far, that overlapping ByteBuf instances could even be created, as was seen in #11101.

Modification:
Instead of fixing the per-allocation pointer bump, we now ensure that 1) the minimum page size is a whole multiple of the alignment, and 2) the reference memory for each chunk is bumped to the nearest aligned address, and finally 3) ensured that the reservations are whole multiples of the alignment, thus ensuring that the next allocation automatically occurs from an aligned address.

Incidentally, (3) above comes for free because the reservations are in whole pages, and in (1) we ensured that pages are sized in whole multiples of the alignment.

In order to ensure that the memory for a chunk is aligned, we introduce some new PlatformDependent infrastructure.
The PlatformDependent.alignDirectBuffer will produce a slice of the given buffer, and the slice will have an address that is aligned.
This method is plainly available on ByteBuffer in Java 9 onwards, but for pre-9 we have to use Unsafe, which means it can fail and might not be available on all platforms.
Attempts to create a PooledByteBufAllocator that uses alignment, when this is not supported, will throw an exception.
Luckily, I think use of aligned allocations are rare.

Result:
Aligned pooled byte bufs now work correctly, and never have any overlap.

Fixes #11101
2021-03-23 17:09:44 +01:00
Elliotte Rusty Harold 5451b978c4
Fix grammar in javadoc (#11090)
Motivation:

There was some grammar / spelling error in the javadocs

Modification:

Fix error

Result:

Cleanup
2021-03-19 10:02:23 +01:00
Norman Maurer 29a4dd6ea7 Test that we return correct result for Futures that are returned from UnorderedThreadPoolExecutor (#11074)
Motivation:

Add test to validate we are not affected by the regression introduced by fd8c1874b4

Modifications:

- Add unit test

Result:

Verify code works as expected
2021-03-11 08:34:00 +01:00
Chris Vest f3134080ff Fix bug in Recycler with racing calls to recycle (#11037)
Motivation:
It is possible for two separate threads to race on recycling an object.
If this happens, the object might be added to a WeakOrderQueue when it shouldn't be.
The end result of this is that an object could be acquired multiple times, without a recycle in between.
Effectively, it ends up in circulation twice.

Modification:
We fix this by making the update to the lastRecycledId field of the handle, an atomic state transition.
Only the thread that "wins" the race and succeeds in their state transition will be allowed to recycle the object.
The others will bail out on their recycling.
We use weakCompareAndSet because we only need the atomicity guarantee, and the program order within each thread is sufficient.
Also, spurious failures just means we won't recycle that particular object, which is fine.

Result:
Objects no longer risk circulating twice due to a recycle race.

This fixes #10986
2021-02-26 10:03:49 +01:00
Violeta Georgieva b1ce20c080 Allow blocking calls when parsing etcResolver/hosts files (#11009)
Motivation:

When etcResolver/hosts files are parsed, FileInputStream.read(...) is internally called by

- UnixResolverDnsServerAddressStreamProvider#parseEtcResolverSearchDomains
- UnixResolverDnsServerAddressStreamProvider#parseEtcResolverOptions
- HostsFileParser#parse

This will cause the error below when BlockHound is enabled

reactor.blockhound.BlockingOperationError: Blocking call! java.io.FileInputStream#readBytes
     	at java.io.FileInputStream.readBytes(FileInputStream.java)
     	at java.io.FileInputStream.read(FileInputStream.java:255)

Modifications:

- Add whitelist entries to BlockHound configuration
- Fix typos in UnixResolverDnsServerAddressStreamProvider
- Add tests

Result:

Fixes #11004
2021-02-11 11:05:14 +01:00
strugcoder c1b3ffafcf Simplity some code (#11000)
Motivation:

There was some code that could be simplified.

Modification:

Simplify code.

Result:

Code cleanup
2021-02-11 09:06:16 +01:00