Commit Graph

1009 Commits

Author SHA1 Message Date
Chris Vest
e9893ad021
Backport some fixes and cleanups for DefaultPromiseTest (#11600)
Motivation:
These cleanups were done in another PR but were not directly related to that PR.
This extracts those changes and backports them to 4.1.

Modification:
* Remove the use of mocking in DefaultPromiseTest.
* Fix a few warnings.
* Make `testStackOverFlowChainedFuturesB` test with the right listener chain.

Result:
Cleaner code.
2021-08-19 13:28:32 +02:00
Aayush Atharva
beb4588b4a
Inline variables to make code more readable (#11565)
Motivation:
There are lots of redundant variable declarations which should be inlined to make good look better.

Modification:
Made variables inlined.

Result:
Less redundant variable and more readable code.
2021-08-11 17:05:48 +02:00
Aayush Atharva
fe7b18ee83
Make variables final (#11548)
Motivation:
We should make variables `final` which are not reinstated again in code to match the code style and makes the code look better.

Modification:
Made couples of variables as `final`.

Result:
Variables marked as `final`.
2021-08-06 09:27:12 +02:00
Aayush Atharva
1ce28e80d1
Remove Unused Imports (#11546)
Motivation:
There are lots of imports which are unused. We should get rid of them to make the code look better,

Modification:
Removed unused imports.

Result:
No unused imports.
2021-08-05 13:54:48 +02:00
Aayush Atharva
82418dafec
Add Increment and Decrement count positive check in ReferenceCountUtil (#11523)
Motivation:
We should have guards in place to check increment or decrement count in `ReferenceCountUtil`. Increment and Decrement counts must be a positive integer.

Modification:
Added `ObjectUtil#checkPositive` checks.

Result:
Prevent release due to invalid count.
2021-07-29 20:51:01 +02:00
old driver
8f2fe8659b
Optimize method io.netty.util.concurrent.DefaultPromise.await0(...) (#11504)
Motivation:

For the code pattern of `Object.wait(...)`  in `io.netty.util.concurrent.DefaultPromise.await0(...)`, it's better to follow the recommended code pattern according to [Object.wait(...)'s doc](https://docs.oracle.com/javase/7/docs/api/java/lang/Object.html#wait()):
```
synchronized (obj) {
     while (<condition does not hold>)
     obj.wait();
     ... // Perform action appropriate to condition
}
```

Modification:
Change the `Object.wait(...)`'s code pattern in `io.netty.util.concurrent.DefaultPromise.await0(...)`.

Result:

The `Object.wait(...)`'s code pattern in `io.netty.util.concurrent.DefaultPromise.await0(...)` meets the Java doc.
2021-07-26 08:32:53 +02:00
Norman Maurer
26cdfe2cde
Add PromiseNotifier static method which takes care of cancel propagation (#11494)
Motivation:

At the moment we not correctly propagate cancellation in some case when we use the PromiseNotifier.

Modifications:

- Add PromiseNotifier static method which takes care of cancellation
- Add unit test
- Deprecate ChannelPromiseNotifier

Result:

Correctly propagate cancellation of operation

Co-authored-by: Nitesh Kant <nitesh_kant@apple.com>
2021-07-21 13:37:32 +02:00
Aayush Atharva
fef761d03e
Introduce BrotliEncoder (#11256)
Motivation:
Currently, Netty only has BrotliDecoder which can decode Brotli encoded data. However, BrotliEncoder is missing which will encode normal data to Brotli encoded data.

Modification:
Added BrotliEncoder and CompressionOption

Result:
Fixes #6899.

Co-authored-by: Norman Maurer <norman_maurer@apple.com>
2021-07-08 11:51:27 +02:00
Norman Maurer
7e39b96402
Only run one SSL task per delegation (#11462)
Motivation:

We should only run one SSL task per delegation to allow more SSLEngines to make progress in a timely manner

Modifications:

- Only run one task per delegation to the executor
- Only create new SSL task if really needed
- Only schedule if not on the EventExecutor thread

Result:

More fair usage of resources and less allocations
2021-07-08 07:56:15 +02:00
Norman Maurer
c82c17782c
Migrate common to junit5 (#11431) (#11438)
Motivation:

We should update to use junit5 in all modules.

Modifications:

Adjust common tests to use junit5

Result:

Part of https://github.com/netty/netty/issues/10757
2021-07-01 18:19:44 +02:00
ping
98a3a0c0cb
Recycler.WeakOrderQueue drop Object hasBeenRecycled (#11402)
Motivation:

WeakOrderQueue would drop object that has been recycled, even when it has space for it.
WeakOrderQueue#add should check DefaultHandler.hasBeenRecycler field  first

Modifications:

WeakOrderQueue test the DefaultHandler.hasBeenRecycler first

Result:

WeakOrderQueue would not drop object that has been recycled when there is space


Co-authored-by: Norman Maurer <norman_maurer@apple.com>
Co-authored-by: Trustin Lee <t@motd.kr>
2021-06-23 11:32:44 +02:00
terrarier2111
b646a2670a
Some minor cleanups for TypeParameterMatcher (#11286)
Motivation:

The code in TypeParamterMatcher can be simplified and cleaned up.

Modification:

Cleanup

Result:

Cleanup
2021-05-27 09:02:29 +02:00
Riley Park
3efd9642ca Migrate common tests to JUnit 5 (#11319)
Motivation:

JUnit 5 is more expressive, extensible, and composable in many ways, and it's better able to run tests in parallel.

Modifications:

Use JUnit5 in tests

Result:

Related to https://github.com/netty/netty/issues/10757
2021-05-27 08:56:23 +02:00
Roman Fedorov
bb800d6067
Delete obsolete javadoc warning about pendingTasks() method expensiveness (#11287)
Co-authored-by: Roman Fedorov <roman.fedorov@spikeapp.com>
2021-05-22 09:33:45 +02:00
old driver
9747739962
Change docs of method io.netty.util.internal.MathUtil#isOutOfBounds (#11281)
Motivation:

1. The docs about the 'retun value' of the method `io.netty.util.internal.MathUtil#isOutOfBounds` is not correct.
2. The capacity parameter should be checked for overflowed case.

Modification:

1. Changed the doc to:

>  @return {@code false} if the requested {@code index} and {@code length} will fit within {@code capacity}.
>  {@code true} if this would result in an index out of bounds exception.

2. Improved the bounder checking logic.

Result:
Fixes #11279 
Fixes #11280
2021-05-19 21:33:26 +02:00
old driver
5178b2c7a5
make Slf4JLoggerFactory be singleton (#11253)
Motivation:

The current initialization of Slf4JLoggerFactory is not singleton.

Modification:

Use Slf4JLoggerFactory.INSTANCE to initialize Slf4JLoggerFactory.

Result:

The instance of Slf4JLoggerFactory became a singleton.
2021-05-17 09:10:15 +02:00
Violeta Georgieva
fa8f7a3510
Add DefaultHostsFileEntriesResolver#addresses to provide all hosts file's entries for a hostname (#11246)
Motivation:
DefaultHostsFileEntriesResolver should provide all hosts file's entries for a hostname when
DnsNameResolver#resolveAll as opposed to the current implementation where only the first
entry is taken into consideration

Modification:
- Add DefaultHostsFileEntriesResolver#addresses to provide all hosts file's entries for a hostname
- Add HostsFileEntriesProvider to provide all hosts file's entries for a hostname and to keep
backwards compatibility for HostsFileEntries and HostsFileParser
- DnsNameResolver#resolveAll uses the new DefaultHostsFileEntriesResolver#addresses
- BlockHound configuration: replace HostsFileParser#parse with HostsFileEntriesProvider$ParserImpl#parse
as the latter does the parsing
- Add junit tests

Result:
Fixes #10834

* Address feedback
2021-05-14 09:50:42 +02:00
Idel Pivnitskiy
3ecb32357c
Use PlatformDependent#normalizedOs() instead of reading os.name prop (#11239)
Motivation:

`PlatformDependent#normalizedOs()` already caches normalized variant of
the value of `os.name` system property. Instead of inconsistently
normalizing it in every case, use the utility method.

Modifications:

- `PlatformDependent`: `isWindows0()` and `isOsx0()` use `NORMALIZED_OS`;
- `PlatformDependent#normalizeOs(String)` define `darwin` as `osx`;
- `OpenSsl#loadTcNative()` does not require `equalsIgnoreCase` bcz `os`
is already normalized;
- Epoll and KQueue: `Native#loadNativeLibrary()` use `normalizedOs()`;
- Use consistent `Locale.US` for lower case conversion of `os.name`;
- `MacOSDnsServerAddressStreamProvider#loadNativeLibrary()` uses
`PlatformDependent.isOsx()`;

Result:

Consistent approach for `os.name` parsing.
2021-05-11 08:52:29 +02:00
Norman Maurer
6b3ec62e6e
Preload classes before calling native OnLoad function to prevent clas… (#11215)
Motivation:

It turns out it is quite easy to cause a classloader deadlock in more recent java updates if you cause classloading while you are in native code. Because of this we should just workaround this issue by pre-load all the classes that needs to be accessed in the OnLoad function.

Modifications:

- Preload all classes that would otherwise be loaded by native OnLoad functions.

Result:

Workaround for https://github.com/netty/netty/issues/11209 and https://bugs.openjdk.java.net/browse/JDK-8266310
2021-05-03 10:22:22 +02:00
Norman Maurer
36dcaf0c50 Fix typo in exception message 2021-04-22 08:22:37 +02:00
Boris Unckel
2c757d11be
Utilize i.n.u.internal.ObjectUtil to assert Preconditions (commons) (#11170) (#11172)
Motivation:

NullChecks resulting in a NullPointerException or IllegalArgumentException, numeric ranges (>0, >=0) checks, not empty strings/arrays checks must never be anonymous but with the parameter or variable name which is checked. They must be specific and should not be done with an "OR-Logic" (if a == null || b == null) throw new NullPointerEx.

Modifications:

* Add some checks to ObjectUtil not present today but utilized in the code.
* Add unit test for ObjectUtil
* Update commmons internal usage with ObjectUtil

Result:

All checks needed are present, subsequent changes of usage of ObjectUtil are possible.

Fixes for https://github.com/netty/netty/issues/11170
2021-04-22 08:03:24 +02:00
Chris Vest
1a5ebfb999
Log fewer stack traces from initialisation code (#11164)
Motivation:
We are increasingly running in environments where Unsafe, setAccessible, etc. are not available.
When debug logging is enabled, we log a complete stack trace every time one of these initialisations fail.
Seeing these stack traces can cause people unnecessary concern.
For instance, people might have alerts that are triggered by a stack trace showing up in logs, regardless of its log level.

Modification:
We continue to print debug log messages on the result of our initialisations, but now we only include the full stack trace is _trace_ logging (or FINEST, or equivalent in whatever logging framework is configured) is enabled.

Result:
We now only log these initialisation stack traces when the lowest possible log level is enabled.

Fixes #7817
2021-04-19 08:53:08 +02:00
Norman Maurer
cc2b443150
DefaultThreadFactory must not use Thread.currentThread() when constructed without ThreadGroup (#11119)
Motivation:

We had a bug in out DefaulThreadFactory as it always retrieved the ThreadGroup to used during creation time when now explicit ThreadGroup was given. This is problematic as the Thread may die and so the ThreadGroup is destroyed even tho the DefaultThreadFactory is still used.

This could produce exceptions like:

java.lang.IllegalThreadStateException
        at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867)
        at java.lang.Thread.init(Thread.java:405)
        at java.lang.Thread.init(Thread.java:349)
        at java.lang.Thread.<init>(Thread.java:599)
        at io.netty.util.concurrent.FastThreadLocalThread.<init>(FastThreadLocalThread.java:60)
        at io.netty.util.concurrent.DefaultThreadFactory.newThread(DefaultThreadFactory.java:122)
        at io.netty.util.concurrent.DefaultThreadFactory.newThread(DefaultThreadFactory.java:106)
        at io.netty.util.concurrent.ThreadPerTaskExecutor.execute(ThreadPerTaskExecutor.java:32)
        at io.netty.util.internal.ThreadExecutorMap$1.execute(ThreadExecutorMap.java:57)
        at io.netty.util.concurrent.SingleThreadEventExecutor.doStartThread(SingleThreadEventExecutor.java:978)
        at io.netty.util.concurrent.SingleThreadEventExecutor.startThread(SingleThreadEventExecutor.java:947)
        at io.netty.util.concurrent.SingleThreadEventExecutor.execute(SingleThreadEventExecutor.java:830)
        at io.netty.util.concurrent.SingleThreadEventExecutor.execute(SingleThreadEventExecutor.java:818)
        at io.netty.channel.AbstractChannel$AbstractUnsafe.register(AbstractChannel.java:471)
        at io.netty.channel.SingleThreadEventLoop.register(SingleThreadEventLoop.java:87)
        at io.netty.channel.SingleThreadEventLoop.register(SingleThreadEventLoop.java:81)
        at io.netty.channel.MultithreadEventLoopGroup.register(MultithreadEventLoopGroup.java:86)
        at io.netty.bootstrap.AbstractBootstrap.initAndRegister(AbstractBootstrap.java:323)
        at io.netty.bootstrap.AbstractBootstrap.doBind(AbstractBootstrap.java:272)
        at io.netty.bootstrap.AbstractBootstrap.bind(AbstractBootstrap.java:239)
        at io.netty.incubator.codec.quic.QuicTestUtils.newServer(QuicTestUtils.java:138)
        at io.netty.incubator.codec.quic.QuicTestUtils.newServer(QuicTestUtils.java:143)
        at io.netty.incubator.codec.quic.QuicTestUtils.newServer(QuicTestUtils.java:147)
        at io.netty.incubator.codec.quic.QuicStreamFrameTest.testCloseHalfClosure(QuicStreamFrameTest.java:48)
        at io.netty.incubator.codec.quic.QuicStreamFrameTest.testCloseHalfClosureUnidirectional(QuicStreamFrameTest.java:35)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
        at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
        at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
        at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
        at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:288)
        at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:282)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.lang.Thread.run(Thread.java:748)

Modifications:

- If the user dont specify a ThreadGroup we will just pass null to the constructor of FastThreadLocalThread and so let it retrieve on creation time
- Adjust tests

Result:

Don't risk to see IllegalThreadStateExceptions.
2021-03-26 18:44:13 +01:00
Chris Vest
2071086919
Fix alignment handling for pooled direct buffers (#11106)
Motivation:
Alignment handling was broken, and basically turned into a fixed offset into each allocation address regardless of its initial value, instead of ensuring that the allocated address is either aligned or bumped to the nearest alignment offset.
The brokenness of the alignment handling extended so far, that overlapping ByteBuf instances could even be created, as was seen in #11101.

Modification:
Instead of fixing the per-allocation pointer bump, we now ensure that 1) the minimum page size is a whole multiple of the alignment, and 2) the reference memory for each chunk is bumped to the nearest aligned address, and finally 3) ensured that the reservations are whole multiples of the alignment, thus ensuring that the next allocation automatically occurs from an aligned address.

Incidentally, (3) above comes for free because the reservations are in whole pages, and in (1) we ensured that pages are sized in whole multiples of the alignment.

In order to ensure that the memory for a chunk is aligned, we introduce some new PlatformDependent infrastructure.
The PlatformDependent.alignDirectBuffer will produce a slice of the given buffer, and the slice will have an address that is aligned.
This method is plainly available on ByteBuffer in Java 9 onwards, but for pre-9 we have to use Unsafe, which means it can fail and might not be available on all platforms.
Attempts to create a PooledByteBufAllocator that uses alignment, when this is not supported, will throw an exception.
Luckily, I think use of aligned allocations are rare.

Result:
Aligned pooled byte bufs now work correctly, and never have any overlap.

Fixes #11101
2021-03-23 17:07:06 +01:00
Elliotte Rusty Harold
aea9e381a7 Fix grammar in javadoc (#11090)
Motivation:

There was some grammar / spelling error in the javadocs

Modification:

Fix error

Result:

Cleanup
2021-03-19 10:02:53 +01:00
Norman Maurer
4949d4a0ad
Return correct result for Futures that are returned from UnorderedThreadPoolExecutor (#11074)
Motivation:

Due a regression in fd8c1874b4 we did not correctly set the result for the returned Future if it was build for a Callable.

Modifications:

- Adjust code to call get() to retrive the correct result for notification of the future.
- Add unit test

Result:

Fixes https://github.com/netty/netty/issues/11072
2021-03-11 08:31:32 +01:00
Chris Vest
dde82f62f0
Fix bug in Recycler with racing calls to recycle (#11037)
Motivation:
It is possible for two separate threads to race on recycling an object.
If this happens, the object might be added to a WeakOrderQueue when it shouldn't be.
The end result of this is that an object could be acquired multiple times, without a recycle in between.
Effectively, it ends up in circulation twice.

Modification:
We fix this by making the update to the lastRecycledId field of the handle, an atomic state transition.
Only the thread that "wins" the race and succeeds in their state transition will be allowed to recycle the object.
The others will bail out on their recycling.
We use weakCompareAndSet because we only need the atomicity guarantee, and the program order within each thread is sufficient.
Also, spurious failures just means we won't recycle that particular object, which is fine.

Result:
Objects no longer risk circulating twice due to a recycle race.

This fixes #10986
2021-02-26 10:02:49 +01:00
赵延
9cac18687d
Fix link in javadoc (#11022)
Motivation:

There was an incorrect link in the javadoc

Modification:

Fix link

Result:

Doc cleanup
2021-02-12 14:40:24 +01:00
Violeta Georgieva
6808d7582a
Allow blocking calls when parsing etcResolver/hosts files (#11009)
Motivation:

When etcResolver/hosts files are parsed, FileInputStream.read(...) is internally called by

- UnixResolverDnsServerAddressStreamProvider#parseEtcResolverSearchDomains
- UnixResolverDnsServerAddressStreamProvider#parseEtcResolverOptions
- HostsFileParser#parse

This will cause the error below when BlockHound is enabled

reactor.blockhound.BlockingOperationError: Blocking call! java.io.FileInputStream#readBytes
     	at java.io.FileInputStream.readBytes(FileInputStream.java)
     	at java.io.FileInputStream.read(FileInputStream.java:255)

Modifications:

- Add whitelist entries to BlockHound configuration
- Fix typos in UnixResolverDnsServerAddressStreamProvider
- Add tests

Result:

Fixes #11004
2021-02-11 11:03:20 +01:00
strugcoder
bb9370f2a2
Simplity some code (#11000)
Motivation:

There was some code that could be simplified.

Modification:

Simplify code.

Result:

Code cleanup
2021-02-11 08:42:01 +01:00
Norman Maurer
c735357bf2 Use Files.createTempFile(...) to ensure the file is created with proper permissions
Motivation:

File.createTempFile(String, String)` will create a temporary file in the system temporary directory if the 'java.io.tmpdir'. The permissions on that file utilize the umask. In a majority of cases, this means that the file that java creates has the permissions: `-rw-r--r--`, thus, any other local user on that system can read the contents of that file.
This can be a security concern if any sensitive data is stored in this file.

This was reported by Jonathan Leitschuh <jonathan.leitschuh@gmail.com> as a security problem.

Modifications:

Use Files.createTempFile(...) which will use safe-defaults when running on java 7 and later. If running on java 6 there isnt much we can do, which is fair enough as java 6 shouldnt be considered "safe" anyway.

Result:

Create temporary files with sane permissions by default.
2021-02-08 11:44:05 +01:00
Norman Maurer
4d43f16bb9
Make native loading logging less confusing (#10944)
Motivation:

We produced a lot of noise during loading native libraries as we always included the stacktrace if we could not load by one mechanism. We should better just not include the stacktrace in the debugging logging if one mechanism fails. We will log all the stacks anyway when all of the mechanisms fail.

Modifications:

Make logging less aggressive

Result:

Less confusing behaviour for the end-user
2021-01-16 09:56:44 +01:00
Chris Vest
bba0017d6e
Update the javadocs on FastThreadLocal (#10942)
Motivation: The writing was unclear.
Modification: Clarified the documentation.
Result: No more ambiguity about what FTL.remove() does.

Fixes #10914
2021-01-15 12:03:26 +01:00
Violeta Georgieva
5b699b722c
Allow blocking calls in UnixResolverDnsServerAddressStreamProvider#parse (#10935)
Motivation:

Internally UnixResolverDnsServerAddressStreamProvider#parse calls FileInputStream.read(...)
when parsing the etcResolverFiles.
This will cause the error below when BlockHound is enabled
reactor.blockhound.BlockingOperationError: Blocking call! java.io.FileInputStream#readBytes
     	at java.io.FileInputStream.readBytes(FileInputStream.java)
     	at java.io.FileInputStream.read(FileInputStream.java:255)

Modifications:

- Add whitelist entry to BlockHound configuration
- Add test

Result:

Fixes #10925
2021-01-14 18:27:12 +01:00
Eric Anderson
4f6e62d50c
Avoid unsynchronized access to scheduledTaskQueue in GlobalEventExecutor (#10890)
Motivation:

A race detector discovered a data race in GlobalEventExecutor present in
netty 4.1.51.Final:

```
  Write of size 4 at 0x0000cea08774 by thread T103:
    #0 io.netty.util.internal.DefaultPriorityQueue.poll()Lio/netty/util/internal/PriorityQueueNode; DefaultPriorityQueue.java:113
    #1 io.netty.util.internal.DefaultPriorityQueue.poll()Ljava/lang/Object; DefaultPriorityQueue.java:31
    #2 java.util.AbstractQueue.remove()Ljava/lang/Object; AbstractQueue.java:113
    #3 io.netty.util.concurrent.AbstractScheduledEventExecutor.pollScheduledTask(J)Ljava/lang/Runnable; AbstractScheduledEventExecutor.java:133
    #4 io.netty.util.concurrent.GlobalEventExecutor.fetchFromScheduledTaskQueue()V GlobalEventExecutor.java:119
    #5 io.netty.util.concurrent.GlobalEventExecutor.takeTask()Ljava/lang/Runnable; GlobalEventExecutor.java:106
    #6 io.netty.util.concurrent.GlobalEventExecutor$TaskRunner.run()V GlobalEventExecutor.java:240
    #7 io.netty.util.internal.ThreadExecutorMap$2.run()V ThreadExecutorMap.java:74
    #8 io.netty.util.concurrent.FastThreadLocalRunnable.run()V FastThreadLocalRunnable.java:30
    #9 java.lang.Thread.run()V Thread.java:835
    #10 (Generated Stub) <null>

  Previous read of size 4 at 0x0000cea08774 by thread T110:
    #0 io.netty.util.internal.DefaultPriorityQueue.size()I DefaultPriorityQueue.java:46
    #1 io.netty.util.concurrent.GlobalEventExecutor$TaskRunner.run()V GlobalEventExecutor.java:263
    #2 io.netty.util.internal.ThreadExecutorMap$2.run()V ThreadExecutorMap.java:74
    #3 io.netty.util.concurrent.FastThreadLocalRunnable.run()V FastThreadLocalRunnable.java:30
    #4 java.lang.Thread.run()V Thread.java:835
    #5 (Generated Stub) <null>
```

The race is legit, but benign. To trigger it requires a TaskRunner to
begin exiting and set 'started' to false, more work to be scheduled
which starts a new TaskRunner, that work then needs to schedule
additional work which modifies 'scheduledTaskQueue', and then the
original TaskRunner checks 'scheduledTaskQueue'. But there is no danger
to this race as it can only produce a false negative in the condition
which causes the code to CAS 'started' which is thread-safe.

Modifications:

Delete problematic references to scheduledTaskQueue. The only way
scheduledTaskQueue could be modified since the last check is if another
TaskRunner is running, in which case the current TaskRunner doesn't
care.

Result:

Data-race free code, and a bit less code to boot.
2020-12-24 11:37:42 +01:00
Chris Vest
5c1613df81
Simplify InternalThreadLocalMap (#10872)
Motivation:
I did not see any tangible advantage to the padding.
The only other field that was guarded was a rarely changed object reference to a BitSet.
Without the padding, there is also no longer any use of the inheritance hierarchy.
The padding was also using `long`, which would not necessarily prevent the JVM from fitting the aforementioned object reference in an alignment gap.

Modification:
Move all the fields into the InternalThreadLocalMap and deprecate the stuff we'd like to remove.

Result:
Simpler code.
This resolves the discussion in https://github.com/netty/netty/issues/9284
2020-12-18 08:06:53 +01:00
Norman Maurer
8ea0d8f41a
Don't catch Throwable in InternalLoggerFactory (#10866)
Motivation:

We shouldnt catch Throwable in InternalLoggerFactory as this may hide OOME etc.

Modifications:

Only catch LinkageError and Exception

Result:

Fixes https://github.com/netty/netty/issues/10857
2020-12-16 08:45:36 +01:00
Roman Puchkovskiy
80c28a3c33
Fix native image build for the cases when io.netty.util.NetUtil is initialized at run-time (#10799)
Motivation:

In #10630, field substitutions were introduced for NetUtil.LOCALHOST4, NetUtil.LOCALHOST6 and NetUtil.LOCALHOST fields. They were required to allow a native image be built with most of Netty (including NetUtil) initialized at build time.

The substitutions created in #10630 only define getters, so the 3 fields can only be read in a native image.

But when NetUtil is initialized at run-time (this is what happens in #10797), its static initialization block is executed, and this block writes to all 3 fields. As the substitutions do not provide any setters, field stores are not valid, and such builds fail.

Modifications:

- Add netty-testsuite-native-image-client-runtime-init Maven module that builds a native image deferring NetUtil class initialization till run-time; this module is used to demonstrate the problem and verify the problem is gone with the fix
- Add no-op setters to substitutions for NetUtil.LOCALHOST4, NetUtil.LOCALHOST6 and NetUtil.LOCALHOST

Result:

A native image initializing NetUtil at run-time builds successfully.

Fixes #10797
2020-12-07 18:16:30 +01:00
Violeta Georgieva
8a7c580bf6
Add whitelist entry for SSLEngineImpl.wrap to BlockHound configuration (#10844)
Motivation:

Internally SSLEngineImpl.wrap(...) may call FileInputStream.read(...).
This will cause the error below when BlockHound is enabled
reactor.blockhound.BlockingOperationError: Blocking call! java.io.FileInputStream#readBytes
	at java.io.FileInputStream.readBytes(FileInputStream.java)
	at java.io.FileInputStream.read(FileInputStream.java:255)

Modifications:

- Add whitelist entry to BlockHound configuration
- Add test

Result:

Fixes #10837
2020-12-07 08:06:21 +01:00
Norman Maurer
44f85bba5f
Ensure we don't leak the ClassLoader in the backtrace of TrackRecord.BOTTOM (#10839)
Motivation:

We need to ensure we override fillInStacktrace() when we store exceptions in static fields to not leak the Classloader in the backtrace.

Came up in https://github.com/netty/netty/pull/10691#issuecomment-738331186. Thanks to @amir-shalem for notice this one.

Modifications:

- Add overrides of fillInStracktrace in TrackRecord.BOTTOM

Result:

Related fix to https://github.com/netty/netty/pull/10686
2020-12-05 07:01:03 +01:00
Johnny Lim
37267f95ce
Initialize Recycler.INITIAL_CAPACITY before invoking InternalLogger.debug() (#10836)
Motivation:

If Log4J2's `Filter` creates `Recycler.Stack` somehow, `Recycler.Stack()` will see uninitialized `Recycler.INITIAL_CAPACITY`. This has been raised originally in https://github.com/micrometer-metrics/micrometer/issues/2369.

Modification:

This PR changes to initialize `Recycler.INITIAL_CAPACITY` before invoking `InternalLogger.debug()` to avoid it.

Result:

Fixes the problem described in the "Motivation" section.
2020-12-04 14:57:06 +01:00
Alexandre Dutra
02cd85181a
Allow blocking calls inside SingleThreadEventExecutor.addTask (#10811)
Motivation:

GlobalEventExecutor.addTask was rightfully allowed to block by commit
09d38c8. However the same should have been done for
SingleThreadEventExecutor.addTask.

BlockHound is currently intercepting that call, and as a consequence,
it prevents SingleThreadEventExecutor from working properly, if addTask is
called from a thread that cannot block.

The interception is due to LinkedBlockingQueue.offer implementation,
which uses a ReentrantLock internally.

Modifications:

* Added one BlockHound exception to
io.netty.util.internal.Hidden.NettyBlockHoundIntegration for
SingleThreadEventExecutor.addTask.
* Also added unit tests for both SingleThreadEventExecutor.addTask
and GlobalEventExecutor.addTask.

Result:

SingleThreadEventExecutor.addTask can now be invoked from any thread
when BlockHound is activated.
2020-11-23 19:20:18 +01:00
Alexandre Dutra
2ff8973f69
Allow blocking calls inside HashedWheelTimer start() and stop() (#10810)
Motivation:

When a HashedWheelTimer instance is started or stopped, its working
thread is started or stopped. These operations block the calling
thread:
- start() calls java.util.concurrent.CountDownLatch.await() to wait
for the worker thread to finish initializing;
- stop() calls java.lang.Thread.join(long) to wait for the worker
thread to exit.
BlockHound detects these calls and as a consequence, prevents
HashedWheelTimer from working properly, if it is started or stopped
in a thread that is not allowed to block.

Modifications:

Added two more BlockHound exceptions to
io.netty.util.internal.Hidden.NettyBlockHoundIntegration: one
for HashedWheelTimer.start() and one for HashedWheelTimer.stop().

Result:

HashedWheelTimer can now be started and stopped properly when
BlockHound is activated.
2020-11-23 08:41:07 +01:00
Norman Maurer
d371b1bbaa
Rethrow NoSuchMethodError with more hints about incompatible native library versions (#10740)
Motivation:

03aafb9cff did ensure we fail while loading a natibe library which is not compatible. While this is great it is still sometimes hard for people to understand what NoSuchMethodError means in this context.

Modifications:

If possible rethrow the NoSuchMethodError and provide some more hints about multiple versions of the shared library

Result:

Easier to understand for people why loading fails
2020-10-28 19:22:08 +01:00
Norman Maurer
d4defc30b8
Add PlatformDependent* methods that are needed by io_uring (#10744)
Motivation:

ddebc10 added a few adjustments that are needed by io_uring that we will add as an incubator repository. Unfortunally we missed the changed in PlatformDependent*.

Modifications:

Add missing methods

Result:

Be able to compile io_uring code against core netty
2020-10-28 19:20:52 +01:00
Roman Puchkovskiy
4ecd78e104
Fix native image build on modern GraalVM versions for the cases when the program uses netty-dns (#10630)
Motivation:

Since GraalVM version 19.3.0, instances of java.net.InetAddress (and its subclasses Inet4Address and Inet6Address) are not allowed in native image heap (that is, they cannot be stored in static fields of classes initialized at build time or be reachable through static fields of such classes). When building a native image, it makes sense to initialize at build time as many classes as possible.
But some fields of some classes in Netty (for example, NetUtil.LOCALHOST4) contain InetAddress instances. If a program is using code path that makes it possible to reach such fields at build time initialization, it becomes impossible to build a native image initializing core Netty classes initialized at runtime. An example of such a program is a client that uses netty-dns.

Modifications:

- Add netty-testsuite-native-image-client Maven module to test that such an example program can be built after the corresponding fixes
- Add native-image.properties to resolver-dns module to move initialization of some classes to runtime (some of them are parsing configuration during initialization, so it makes no sense to initialize them at build time; for others, it's needed to avoid InetAddress reachability at build time)
- Add substitutions for NetUtil.LOCALHOST4, NetUtil.LOCALHOST6 and NetUtil.LOCALHOST to overcome the InetAddress-related prohibition
- Extract some initialization code from NetUtil to NetUtilInitializations to allow it to be used by the substitutions

Result:

A client program using netty-dns with --initialize-at-build-time=io.netty builds successfully
2020-10-26 08:34:31 +01:00
Artem Smotrakov
e5951d46fc
Enable nohttp check during the build (#10708)
Motivation:

HTTP is a plaintext protocol which means that someone may be able
to eavesdrop the data. To prevent this, HTTPS should be used whenever
possible. However, maintaining using https:// in all URLs may be
difficult. The nohttp tool can help here. The tool scans all the files
in a repository and reports where http:// is used.

Modifications:

- Added nohttp (via checkstyle) into the build process.
- Suppressed findings for the websites
  that don't support HTTPS or that are not reachable

Result:

- Prevent using HTTP in the future.
- Encourage users to use HTTPS when they follow the links they found in
  the code.
2020-10-23 14:44:18 +02:00
Norman Maurer
ffbddcd842
Replace deprecated Assert.assertThat(...) with MatcherAssert.assertThat(...) (#10699)
Motivation:

junit deprecated Assert.assertThat(...)

Modifications:

Use MatcherAssert.assertThat(...) as replacement for deprecated method

Result:

Less deprecation warnings
2020-10-18 14:30:52 +02:00
Artem Smotrakov
1ca7d5db81
Fix or suppress LGTM findings (#10689)
Motivation:

LGTM reports multiple issues. They need to be triaged,
and real ones should be fixed.

Modifications:
- Fixed multiple issues reported by LGTM, such as redundant conditions,
  resource leaks, typos, possible integer overflows.
- Suppressed false-positives.
- Added a few testcases.

Result:

Fixed several possible issues, get rid of false alarms in the LGTM report.
2020-10-17 09:49:44 +02:00
Norman Maurer
cc79f5f4ff
Fix unit tests that sometimes failed due timeouts (#10698)
Motivation:

We had two unit tests that sometimes failed due timeouts. After insepecting these I noticed these can be improved to run faster while still do the right validation

Modifications:

- Only submit one task for execution per execute
- Cleanup

Result:

No test failures due timeout
2020-10-16 21:00:26 +02:00