Commit Graph

2133 Commits

Author SHA1 Message Date
Norman Maurer
c735357bf2 Use Files.createTempFile(...) to ensure the file is created with proper permissions
Motivation:

File.createTempFile(String, String)` will create a temporary file in the system temporary directory if the 'java.io.tmpdir'. The permissions on that file utilize the umask. In a majority of cases, this means that the file that java creates has the permissions: `-rw-r--r--`, thus, any other local user on that system can read the contents of that file.
This can be a security concern if any sensitive data is stored in this file.

This was reported by Jonathan Leitschuh <jonathan.leitschuh@gmail.com> as a security problem.

Modifications:

Use Files.createTempFile(...) which will use safe-defaults when running on java 7 and later. If running on java 6 there isnt much we can do, which is fair enough as java 6 shouldnt be considered "safe" anyway.

Result:

Create temporary files with sane permissions by default.
2021-02-08 11:44:05 +01:00
Norman Maurer
1e87c711b4
Ensure we create a copy of the attributes and options when bootstrap a channel (#10965)
Motivation:

We need to ensure we copy the attributes and options when bootstrap the channel as otherwise we may change the underlying Entry.
This is similar to what was reported in https://github.com/netty/netty-incubator-codec-quic/issues/152.

Modifications:

- Do a copy and re-use methods
- Add unit tests

Result:

Don't affect attributes / options of channels that are already bootstrapped
2021-01-26 19:26:20 +01:00
root
a137ce2042 [maven-release-plugin] prepare for next development iteration 2021-01-13 10:28:54 +00:00
root
10b03e65f1 [maven-release-plugin] prepare release netty-4.1.58.Final 2021-01-13 10:27:17 +00:00
root
c6b894d03d [maven-release-plugin] prepare for next development iteration 2021-01-12 11:10:44 +00:00
root
b016568e21 [maven-release-plugin] prepare release netty-4.1.57.Final 2021-01-12 11:10:20 +00:00
root
a9ec3d86f6 [maven-release-plugin] prepare for next development iteration 2020-12-17 06:11:39 +00:00
root
1188d8320e [maven-release-plugin] prepare release netty-4.1.56.Final 2020-12-17 06:11:18 +00:00
root
f57d64f1c7 [maven-release-plugin] prepare for next development iteration 2020-12-08 11:51:39 +00:00
root
38da45ffe1 [maven-release-plugin] prepare release netty-4.1.55.Final 2020-12-08 11:51:25 +00:00
root
944a020586 [maven-release-plugin] prepare for next development iteration 2020-11-11 05:47:51 +00:00
root
715353ecd6 [maven-release-plugin] prepare release netty-4.1.54.Final 2020-11-11 05:47:37 +00:00
root
afca81a9d8 [maven-release-plugin] rollback the release of netty-4.1.54.Final 2020-11-10 12:02:24 +00:00
root
e256074e49 [maven-release-plugin] prepare for next development iteration 2020-11-10 11:12:23 +00:00
root
cea659bd8a [maven-release-plugin] prepare release netty-4.1.54.Final 2020-11-10 11:12:06 +00:00
Norman Maurer
5ffca6ef4a
Use http in xmlns URIs to make maven release plugin happy again (#10788)
Motivation:

https in xmlns URIs does not work and will let the maven release plugin fail:

```
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  1.779 s
[INFO] Finished at: 2020-11-10T07:45:21Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-release-plugin:2.5.3:prepare (default-cli) on project netty-parent: Execution default-cli of goal org.apache.maven.plugins:maven-release-plugin:2.5.3:prepare failed: The namespace xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" could not be added as a namespace to "project": The namespace prefix "xsi" collides with an additional namespace declared by the element -> [Help 1]
[ERROR]
```

See also https://issues.apache.org/jira/browse/HBASE-24014.

Modifications:

Use http for xmlns

Result:

Be able to use maven release plugin
2020-11-10 10:22:35 +01:00
Norman Maurer
c6e2934357
Revert "Allow and skip null handlers when adding a vararg list of handlers (#10751)" (#10770)
This reverts commit 3b90b536bb.
2020-11-03 21:12:29 +01:00
Bennett Lynch
3b90b536bb
Allow and skip null handlers when adding a vararg list of handlers (#10751)
Motivation:

Allowing null handlers allows for more convenient idioms in
conditionally adding handlers, e.g.,

ch.pipeline().addLast(
        new FooHandler(),
        condition ? new BarHandler() : null,
        new BazHandler()
);

Modifications:

* Change addFirst(..) and addLast(..) to skip null handlers, rather than
break or short-circuit.
* Add new unit tests.

Result:

* Makes addFirst(..) and addLast(..) behavior more consistent
* Resolves https://github.com/netty/netty/issues/10728
2020-11-03 21:11:35 +01:00
Dave Syer
d62384b227
Add reflection config for JMX runtime (#10753)
Motivation:

`DefaultChannelId` uses reflection to access the JMX runtime. This needs some extra config for GraalVM.

Modification:

Add config for GraalVM

Result:

Works when using GraalVM native image
2020-10-30 15:26:41 +01:00
Norman Maurer
ddebc1027d
Make changes to prepare for io_uring incubator repository (#10741)
Motivation:

During the last few month we did develop an io_uring based transport which shows very promising performance numbers. To give it more time to bake we will develop it outside of netty in an "incubator" module which will make it clear to users what to expect and also allow us to seperate its release cycle. While the implementation of it is very self contained there are few small adjustments that need to be made in netty itself to allow us to reuse code.

Modifications:

- AbstractChannel: Add method which can be used when a write fails and remove final from one method
- IovArray: Allow to create an IovArray from a ByteBuf instance
- FileDescriptor: Allow to reuse mark close logic via sub-class

Result:

Be able to reuse netty core classes in io_uring incubator repository
2020-10-28 15:31:02 +01:00
Norman Maurer
f5185ed73b
Release message before notify promise (#10726)
Motivation:

We should preferable always release the message before we notify the promise. Thhis has a few advantages:

 - Release memory as soon as possible
 - Listeners observe the "more correct" reference count

Modifications:

Release message before fail the promises

Result:

Faster releasing of resources. This came up in https://github.com/netty/netty/issues/10723
2020-10-26 12:59:20 +01:00
Roman Puchkovskiy
4ecd78e104
Fix native image build on modern GraalVM versions for the cases when the program uses netty-dns (#10630)
Motivation:

Since GraalVM version 19.3.0, instances of java.net.InetAddress (and its subclasses Inet4Address and Inet6Address) are not allowed in native image heap (that is, they cannot be stored in static fields of classes initialized at build time or be reachable through static fields of such classes). When building a native image, it makes sense to initialize at build time as many classes as possible.
But some fields of some classes in Netty (for example, NetUtil.LOCALHOST4) contain InetAddress instances. If a program is using code path that makes it possible to reach such fields at build time initialization, it becomes impossible to build a native image initializing core Netty classes initialized at runtime. An example of such a program is a client that uses netty-dns.

Modifications:

- Add netty-testsuite-native-image-client Maven module to test that such an example program can be built after the corresponding fixes
- Add native-image.properties to resolver-dns module to move initialization of some classes to runtime (some of them are parsing configuration during initialization, so it makes no sense to initialize them at build time; for others, it's needed to avoid InetAddress reachability at build time)
- Add substitutions for NetUtil.LOCALHOST4, NetUtil.LOCALHOST6 and NetUtil.LOCALHOST to overcome the InetAddress-related prohibition
- Extract some initialization code from NetUtil to NetUtilInitializations to allow it to be used by the substitutions

Result:

A client program using netty-dns with --initialize-at-build-time=io.netty builds successfully
2020-10-26 08:34:31 +01:00
greenjustin
090e9a7271
Allow EventLoops to rethrow Error (#10694)
Motivation:

Thread.stop() works by producing a ThreadDeath error in the target thread. EventLoops swallow all Throwables, which makes them effectively unkillable. This is effectively a memory leak, for our application. Beside this we should also just regrow all `Error` as there is almost no way to recover.

Modification:

Edit the EventLoops that swallow Throwables to instead rethrow Error.

Result:

`EventLoop` can crash if `Error` is thrown
2020-10-24 14:56:33 +02:00
Artem Smotrakov
e5951d46fc
Enable nohttp check during the build (#10708)
Motivation:

HTTP is a plaintext protocol which means that someone may be able
to eavesdrop the data. To prevent this, HTTPS should be used whenever
possible. However, maintaining using https:// in all URLs may be
difficult. The nohttp tool can help here. The tool scans all the files
in a repository and reports where http:// is used.

Modifications:

- Added nohttp (via checkstyle) into the build process.
- Suppressed findings for the websites
  that don't support HTTPS or that are not reachable

Result:

- Prevent using HTTP in the future.
- Encourage users to use HTTPS when they follow the links they found in
  the code.
2020-10-23 14:44:18 +02:00
James Yuzawa
726944146b
Use named exceptions in ChannelPool implementations (#10721)
Motivation:

I was collecting stats for failed promises with a FixedChannelPool and I was bucketing by stats using cause.getSimpleName(). After #9152 was released, the introduction of the anonymous classes make getSimpleName() return "" causing confusion.

Modification:

Use named classes in the ChannelPool implementations. I made them private, but I can change that if you think otherwise.

Result:

The SimpleChannelPool fails the promises with a ChannelPoolFullException. The FixedChannelPool fails the promises with an AcquireTimeoutException. Also AcquireTimeoutException is more specific than just a plain TimeoutException, which is also useful for troubleshooting. If you want different class names, please advise.
2020-10-23 14:29:37 +02:00
Norman Maurer
ad8fe88abd
We should have a special config that allows to configure half closure for DuplexChannel (#10701)
Motivation:

DuplexChannel allow for half-closure, we should have a special config interface for it as well.

Modifications:

Add DuplexChannelConfig which allows to configure half-closure.

Result:

More consistent types
2020-10-21 15:26:27 +02:00
Norman Maurer
ffbddcd842
Replace deprecated Assert.assertThat(...) with MatcherAssert.assertThat(...) (#10699)
Motivation:

junit deprecated Assert.assertThat(...)

Modifications:

Use MatcherAssert.assertThat(...) as replacement for deprecated method

Result:

Less deprecation warnings
2020-10-18 14:30:52 +02:00
Artem Smotrakov
1ca7d5db81
Fix or suppress LGTM findings (#10689)
Motivation:

LGTM reports multiple issues. They need to be triaged,
and real ones should be fixed.

Modifications:
- Fixed multiple issues reported by LGTM, such as redundant conditions,
  resource leaks, typos, possible integer overflows.
- Suppressed false-positives.
- Added a few testcases.

Result:

Fixed several possible issues, get rid of false alarms in the LGTM report.
2020-10-17 09:49:44 +02:00
Norman Maurer
6b613682ba
Ensure we don't leak the ClassLoader in the backtrace (#10691)
Motivation:

We have a few classes in which we store and reuse static instances of various exceptions. When doing so it is important to also override fillInStacktrace() and so prevent the leak of the ClassLoader in the internal backtrace field.

Modifications:

- Add overrides of fillInStracktrace when needed
- Move ThrowableUtil usage in the static methods

Result:

Fixes https://github.com/netty/netty/pull/10686
2020-10-15 20:41:29 +02:00
root
01b7e18632 [maven-release-plugin] prepare for next development iteration 2020-10-13 06:29:26 +00:00
root
d4a0050ef3 [maven-release-plugin] prepare release netty-4.1.53.Final 2020-10-13 06:29:02 +00:00
Norman Maurer
69f9a94f59
Increase initial buffer size in AdaptiveRecvByteBufAllocator (#10600)
Motivation:

We should use an initial buffer size with is >= 1500 (which is a common setting for MTU) to reduce the need for memory copies when a new connection is established. This is especially interesting when SSL / TLS comes into the mix.

This was ported from swiftnio:

https://github.com/apple/swift-nio/pull/1641

Modifications:

Increase the initial size from 1024 to 2048.

Result:

Possible less memory copies on new connections
2020-09-22 17:27:25 +02:00
Norman Maurer
71d034593f
Only create ConnectTimeoutException if really needed (#10595)
Motivation:

Creating exceptions is expensive so we should only do so if really needed.

Modifications:

Only create the ConnectTimeoutException if we really need it.

Result:

Less overhead
2020-09-21 21:32:20 +02:00
root
957ef746d8 [maven-release-plugin] prepare for next development iteration 2020-09-08 05:26:25 +00:00
root
ada9c38c0a [maven-release-plugin] prepare release netty-4.1.52.Final 2020-09-08 05:26:05 +00:00
Norman Maurer
4ececfb70d
Create a stackless ClosedChannelException to reduce overhead when the… (#10523)
Motivation:

In some benchmarks closing the Channel attributes to a lot of overhead due the call of fillInStackTrace(). We should reduce this overhead.

Modifications:

- Create a StacklessClosedChannelException and use it to reduce overhead.
- Only call ChannelOutboundBuffer.failFlushed(...) when there was a flushed message at all.

Result:

Less performance overhead when closing the Channel
2020-09-01 15:25:41 +02:00
Aayush Atharva
ee77fad3f8
Minor typo (#10518)
Motivation:
I was working on the transport part in Netty (ofc, solving a major issue) and I found this typo so thought to fix it.

Modification:
Fixed Typo

Result:
No more confusion between `us` and `use`.
2020-08-31 08:59:27 +02:00
Norman Maurer
830fc0d660
Remove incorrect javadocs that were not updated for new versions of netty (#10487)
Motivation:

At some point a ChannelHandlerContext did have its own AttributeMap which is not true since 4.1.x was released. Unfortunally we missed to update the javadocs and so these don't reflect reality

Modifications:

Update javadocs

Result:

Fixes https://github.com/netty/netty/issues/10477
2020-08-19 17:13:01 +02:00
Kevin Wu
54bfd21e52
Fix #10434 OutOfDirectMemoryError causes cpu load too high and socket is full (#10457)
Motivation:

When we were using the netty http protocol, OOM occurred, this problem has been in 4.1.51.Final Fix [# 10424](https://github.com/netty/netty/issues/10424), even if OOM is up, the service will still receive new connection events, will occur again OOM and eventually cause the connection not to be released. 

code `byteBuf = allocHandle.allocate(allocator);`

Modification:

I fail to create buffer when I try to receive new data, i determine if it is OOM then the close read event releases the connection.
```java
        if (close || cause instanceof OutOfMemoryError || cause instanceof IOException) {
            closeOnRead(pipeline);
        }
```

Result:

Fixes # [10434](https://github.com/netty/netty/issues/10434).
2020-08-13 10:14:19 +02:00
skyguard1
cd0203b5c7
Add default handling for switch statement (#10408)
Motivation:

When a switch statement is used we should always define a `default:` so we don't introduce bugs due fall-through.

Modification:

Add missing `default:`s

Result:

Less error-prone code
2020-07-16 10:54:50 +02:00
root
bfbeb2dec6 [maven-release-plugin] prepare for next development iteration 2020-07-09 12:27:06 +00:00
root
646934ef0a [maven-release-plugin] prepare release netty-4.1.51.Final 2020-07-09 12:26:30 +00:00
Norman Maurer
e0dc054927
Expand ChannelPipeline javadocs to cover UnorderedThreadPoolEventExecutor (#10361)
Motivation:

Seems like some users are suprised by the behaviour of DefaultEventExecutor when used within the ChannelPipeline. We should clarify the semantics and also mention UnordedThreadPoolEventExecutor which may be more inline with their expectations

Modifications:

Add javadocs section about UnorderedThreadPoolEventExecutor and expand details for DefaultEventExecutor

Result:

Clarify sematics
2020-06-23 09:28:34 +02:00
Norman Maurer
75df58a7e1
Guard against re-entrancy issues while draining AbstractCoalescingBufferQueue (#10294)
Motivation:

AbstractCoalescingBufferQueue had a bug which could lead to an empty queue while still report bytes left. This was due the fact that we decremented the pending bytes before draining the queue one-by-one. The problem here is that while the queue is drained we may notify the promise which may add again buffers to the queue for which we never decrement the bytes while we drain these

Modifications:

- Decrement the pending bytes every time we drain a buffer from the queue
- Add unit tests

Result:

Fixes https://github.com/netty/netty/issues/10286
2020-05-15 09:51:33 +02:00
root
caf51b7284 [maven-release-plugin] prepare for next development iteration 2020-05-13 06:00:23 +00:00
root
8c5b72aaf0 [maven-release-plugin] prepare release netty-4.1.50.Final 2020-05-13 05:59:55 +00:00
root
9c5008b109 [maven-release-plugin] prepare for next development iteration 2020-04-22 09:57:54 +00:00
root
d0ec961cce [maven-release-plugin] prepare release netty-4.1.49.Final 2020-04-22 09:57:26 +00:00
Norman Maurer
2c78b4c84f
Correctly propagate exceptions from inbound operations in all cases (#10176)
Motivation:

In AbstractChannelHandlerContext we had some code where we tried to guard against endless loops caused by exceptions thrown by exceptionCaught(...) that would trigger exceptionCaught again. This code was proplematic for two reasons:
 - It is quite expensive as we need to compare all the stacks
 - We may end up not notify another handlers exceptionCaught(...) if in our exeuction stack we triggered actions that will cause an exceptionCaught somewhere else in the pipeline

Modifications:

- Just remove the detection code as we already handle everything correctly when we invoke exceptionCaught(...)
- Add unit tests

Result:

Ensure we always notify correctly and also fixes performance issue reported as https://github.com/netty/netty/issues/10165
2020-04-16 08:56:45 +02:00
Cenjie Ho
2d4d64dc42
Add the unit test of ChannelOutboundBuffer (#10180)
Motivation:

Parameter maxCount needs the unit test.

Modifications:

1. Change the conditional statement to avoid the ineffective maxCount (enhance the robustness of the code merely).
2. Add the unit test for maxCount.

Result:

Enable this parameter to be tested.
2020-04-15 08:22:51 +02:00
root
14e4afeba2 [maven-release-plugin] prepare for next development iteration 2020-03-17 09:20:54 +00:00
root
c10c697e5b [maven-release-plugin] prepare release netty-4.1.48.Final 2020-03-17 09:18:28 +00:00
Bo Zhang
286f14f04a
Close Channel upon exceptions during bootstrap to avoid leaking (#10110)
Motivation:

We need to ensure we always close the Channel when we see an exception during bootstrapping it.

Modification:

Ensure we correct close the Channel if we see an exception during retrieving the Resolver from the group while bootstrapping.

Result:

Fixes #10109


Co-authored-by: Norman Maurer <norman_maurer@apple.com>
2020-03-16 13:37:29 +01:00
feijermu
427a641299
Replace oio's default thread factory with netty's DefaultThreadFactory. (#10107)
Motivation:

Netty's DefaultThreadFactory that creates FastThreadLocalThread instance is widely used in NioEventLoopGroup, EpollEventLoopGroup, etc, but not OioEventLoopGroup. Although oio is quite stale, I still think this change may be useful.

Modification:

Replace oio's default thread factory with netty's DefaultThreadFactory just like NioEventLoopGroup, EpollEventLoopGroup, etc.

Result:

Faster access to FastThreadLocal in oio.
2020-03-16 11:37:29 +01:00
root
c623a50d19 [maven-release-plugin] prepare for next development iteration 2020-03-09 12:13:56 +00:00
root
a401b2ac92 [maven-release-plugin] prepare release netty-4.1.47.Final 2020-03-09 12:13:26 +00:00
Norman Maurer
15fa45a84b
Add log level check simply before logging. (#10080)
Motivation:

In general, we will close the debug log in a product environment. However, logging without external level check may still affect performance as varargs will need to allocate an array.

Modification:

Add log level check simply before logging.

Result:

Improve performance slightly in a product environment.
2020-03-05 14:38:57 +01:00
Antony T Curtis
7fce06a0d1
Preserve order when using alternate event loops (#10069)
Motivation:

When the HttpContentCompressor is put on an alternate EventExecutor, the order of events should be
preserved so that the compressed content is correctly created.

Modifications:
- checking that the executor in the ChannelHandlerContext is the same executor as the current executor when evaluating if the handler should be skipped.
- Add unit test

Result:

Fixes https://github.com/netty/netty/issues/10067

Co-authored-by: Norman Maurer <norman_maurer@apple.com>
2020-03-03 10:42:41 +01:00
root
e0d73bca4d [maven-release-plugin] prepare for next development iteration 2020-02-28 06:37:33 +00:00
root
ebe7af5102 [maven-release-plugin] prepare release netty-4.1.46.Final 2020-02-28 06:36:45 +00:00
Norman Maurer
b1c57257af
Correctly calculate the initial size of the LinkedHashMap during DefaultChannelGroup.write* (#10055)
Motivation:

We should not include the number of ServerChannel that are part of the DefaultChannelGroup when specify the initial size of the LinkedHashMap

Modifications:

Only use the number of the non ServerChannel

Result:

Reduce memory-footprint
2020-02-24 19:29:08 +01:00
feijermu
dcd62c2598
Remove the synchronized flag of fillInStackTrace method to improve performance while creating exception instance. (#10033)
Motivation:

The method body of fillInStackTrace method of IllegalStateException class in SimpleChannelPool.java is so simple (just return this) that there is no need to be marked as synchronized.

Modification:

Remove the synchronized flag of fillInStackTrace method.

Result:

It can improve performance slightly while creating a IllegalStateException instance.
2020-02-17 09:52:36 +01:00
feijermu
862efc618c
Remove the synchronized flag of fillInStackTrace method to improve performance while throwing exception. (#10031)
Motivation:

The method body of fillInStackTrace method of TimeoutException class in FixedChannelPool.java is so simple (just return this) that there is no need to be marked as synchronized.

Modification:

Remove the synchronized flag of fillInStackTrace method.

Result:
It can improve performance slightly while throwing a TimeoutException.
2020-02-15 18:30:44 +01:00
Subba Rao Pasupuleti
0e2e17f9a2
Adding pipeline rename handler tests (#10028)
Motivation:

Current pipeline handler replace tests are replacing handler with same name.
we need test that test handler can renamed, old handlers are removed from pipeline.

Modification:

There is coverage missing related to renaming of handlers

Result:

Adds missing tests

Co-authored-by: phani254 <phani254@yahoo.com>
2020-02-14 16:53:09 +01:00
Subba Rao Pasupuleti
ef50cf5696
remove unused shutdownLock (#10010)
Motivation:

This is unused variable

Modification:

removed variable

Result:

Removes unused lock

Co-authored-by: phani254 <phani254@yahoo.com>
2020-02-11 09:57:52 +01:00
Norman Maurer
a6896eae43
NioEventLoopTest.testChannelsRegistered test failure (#10014)
Motivation:

NioEventLoopTest.testChannelsRegistered sometimes fails due a race which is related to how SelectionKey and Selector is implemented in the JDK. In the current implementation it will "lazy" remove SelectionKeys from the Set which means we may still have these included sometimes when we use size() to get the number of SelectionKeys.

Modifications:

Just retry to read the number of registered Channels if we still see 2

Result:

Fixes https://github.com/netty/netty/issues/9895
2020-02-10 20:28:53 +01:00
Norman Maurer
56055f4404
Ensure ChannelOptions are applied in the same order as configured in *Bootstrap (#9998)
Motivation:

https://github.com/netty/netty/pull/9458 changed how we handled ChannelOptions internally to use a ConcurrentHashMap. This unfortunally had the side-effect that the ordering may be affected and not stable anymore. Here the problem is that sometimes we do validation based on two different ChannelOptions (for example we validate high and low watermarks against each other). Thus even if the user specified the options in the same order we may fail to configure them.

Modifications:

- Use again a LinkedHashMap to preserve order
- Add unit test

Result:

Apply ChannelOptions in correct and expected order
2020-02-06 09:02:31 +01:00
root
9b1ea10a12 [maven-release-plugin] prepare for next development iteration 2020-01-13 09:13:53 +00:00
root
136db8680a [maven-release-plugin] prepare release netty-4.1.45.Final 2020-01-13 09:13:30 +00:00
Nick Hill
ba140ac6bc Fix SimpleChannelPoolTest#testCloseAsync() test flake (#9943)
Motivation

This test is failing intermittently and upon inspection has a race
condition.

Modification

Fix race condition by waiting for async release calls to complete prior
to closing the pool.

Result

Hopefully fixed flakey test
2020-01-11 06:03:45 +01:00
root
79d4e74019 [maven-release-plugin] prepare for next development iteration 2019-12-18 08:32:54 +00:00
root
5ddf45a2d5 [maven-release-plugin] prepare release netty-4.1.44.Final 2019-12-18 08:31:43 +00:00
Scott Mitchell
e7091c04d7 Channel Pool LocalAddress initialization and cleanup (#9879)
Motivation:
The Channel Pool tests commonly use the same fixed local address String. This
has been observed to result in test failures if a single test fails and cleanup
is not done properly, or if tests are run in parallel. Also each test should
close any channel pool objects or maps to make sure resources are reclaimed.

Modifications:
- Use a random string for the address on each test to reduce the chance of
  collision.
- close all ChannelPool and ChannelPoolMap objects at the end of each test

Result:
Less likely to observe spurious failures due to LocalAddress collisions and more
complete test cleanup.
2019-12-13 13:45:22 +01:00
Robert Mihaly
af58bfcfb8 Synchronously close pools when closing AbstractChannelPoolMap (#9857)
Motivation:

In #9830 the get/remove/close methods implementation changed to avoid
deadlocks on event loops. The change involved modifying the methods to
close the managed ChannelPools asynchronously and return immediately.
While this behavior might be fine for get/remove, it is changing what
a user expects from a close() method and after returning from close()
there might be still resources open.

Modifications:

This change is a follow-up for #9830 to preserve the synchronous
behavior of the AbstractChannelPoolMap#close() method.

Result:

AbstractChannelPoolMap#close() returns once the managed pools have
been closed.
2019-12-11 09:28:44 +01:00
梦典
66fca3b58e Update ChannelOutboundBuffer.java (#9863)
Motivation:

fix comment of ChannelOutboundBuffer.CHANNEL_OUTBOUND_BUFFER_ENTRY_OVERHEAD

Modifications:

6 (NOT 8) reference fields is right

Result:
Correct comment
2019-12-11 09:26:21 +01:00
时无两丶
0cde4d9cb4 Uniform null pointer check. (#9840)
Motivation:
Uniform null pointer check.

Modifications:

Use ObjectUtil.checkNonNull(...)

Result:
Less code, same result.
2019-12-09 09:47:35 +01:00
Norman Maurer
e69c4173ea
Replace synchronized with ConcurrentHashMap in Http2StreamChannelBootstrap (#9848)
Motivation:

97361fa2c8 replace synchronized with ConcurrentHashMap in *Bootstrap classes but missed to do the same for the Http2 variant.

Modifications:

- Use ConcurrentHashMap
- Simplify code in *Bootstrap classes

Result:

Less contention
2019-12-06 10:59:55 +01:00
Robert Mihaly
a322addb17 Asynchronously close pools in AbstractChannelPoolMap (#8238) (#9830)
Motivation:

In certain scenarios mutliple concurrent AbstractChannelPoolMap
operations might be called from event loops that handle also
ChannelPool close operations. If the map uses synchronous close
it could end up blocking the event loop and if multiple threads
are waiting for each other a deadlock might occur.

Modifications:

Previously #9226 introduced a closeAsync operation for
FixedChannelPool, which is now extended to SimpleChannelPool class.
The AbstractChannelPoolMap now uses the closeAsync operations when
closing redundant or removed SimpleChannelPool instances.

Result:

The AbstractChannelPoolMap get/remove operations will not wait
until the pools are closed as they will happen asynchronously and
avoid situations that could cause the event loop being blocked in
deadlocks.
2019-12-05 09:14:10 +01:00
Nick Hill
6d0c0e991b Small simplification to WriteTask optimization (#9807)
Motiviation

#9800 was just merged which consolidates the flush/no-flush WriteTasks
in AbstractChannelHandlerContext, but after looking at the changes again
I noticed a tiny simplification that would be good to make imo.

Modification

Remove use of conditional operator in decrementPendingOutboundBytes()

Result

Simpler code, one less branch
2019-11-27 06:55:22 +01:00
Nick Hill
e208e96f12 Clean up NioEventLoop (#9799)
Motivation

The event loop implementations had become somewhat tangled over time and
work was done recently to streamline EpollEventLoop. NioEventLoop would
benefit from the same treatment and it is more straighforward now that
we can follow the same structure as was done for epoll.

Modifications

Untangle NioEventLoop logic and mirror what's now done in EpollEventLoop
w.r.t. the volatile selector wake-up guard and scheduled task deadline
handling.

Some common refinements to EpollEventLoop have also been included - to
use constants for the "special" deadline/wakeup volatile values and to
avoid some unnecessary calls to System.nanoTime() on task-only
iterations.

Result

Hopefully cleaner, more efficient and less fragile NIO transport
implementation.
2019-11-26 08:25:59 +01:00
Nick Hill
b413464091 Optimize WriteTask recycling (#9800)
Motivation

AbstractChannelHandlerContext uses recyclable tasks when performing
writes from outside of the event loop. There's currently two distinct
classes WriteTask and WriteAndFlushTask used for executing writes versus
writeAndFlushes, and these are recycled in separate pools. However it is
straightforward to just have a single class / recycler pool, with a
flush flag.

Modifications

- Unify WriteTasks into a single class using the sign bit of the
existing size field to indicate whether a flush should be performed
- Use the new executor lazyExecute() method to lazily execute the
non-flush write tasks explicitly
- Change AbstractChannelHandlerContext#invokeWrite and
AbstractChannelHandlerContext#invokeWriteAndFlush from private to
package-private to avoid synthetic methods
- Correct the default object size estimate for WriteTask

Results

- Possibly improved reuse of recycled write tasks
- Fewer virtual method calls and shorter path lengths
- Less code
2019-11-23 20:45:35 +01:00
stroller
aa2a9931e8 Add one new constructor with threadFactory only (#9773)
Motivation:
In most cases, we want to use MultithreadEventLoopGroup such as NioEventLoopGroup without setting thread numbers but thread name only. So we need to use followed code:
NioEventLoopGroup boss = new NioEventLoopGroup(0, new DefaultThreadFactory("boss"));
It looks a bit confuse or strange for the number 0 due to we only want to set thread name. So it will be better to add new constructor for this case.

Modifications:
add new constructor into all event loop groups, for example: public NioEventLoopGroup(ThreadFactory threadFactory)

Result:
User can only set thread factory without setting the thread number to 0:
NioEventLoopGroup boss = new NioEventLoopGroup(new DefaultThreadFactory("boss"));
2019-11-18 09:42:44 +01:00
Matthew Miller
394e7f0287 Prevent memory leak when SimpleChannelPools are destroyed (#9760)
Motivation:

https://github.com/netty/netty/pull/9548 introduced a change that creates a new AttributeKey
for each SimpleChannelPool instance created. AttributeKeys are cached statically in a ConstantPool
by the AttributeKey.newInstance method. Because of this, creating a SimpleChannelPool instance will
allocate memory that will never be released, even after the SimpleChannelPool is closed.

Modifications:

This change goes back to a single AttributeKey per SimpleChannelPool, just using a more specific
name to reduce the chance of conflicts with user code.

Result:

No memory is leaked after a SimpleChannelPool instance is created and destroyed.
2019-11-07 14:48:56 +01:00
Nick Hill
a18c57ea87 Externalize lazy execution semantic for EventExecutors (#9587)
Motivation

This is already done internally for various reasons but it would make
sense i.m.o. as a top level concept: submitting a task to be run on the
event loop which doesn't need to run immediately but must still be
executed in FIFO order relative all other submitted tasks (be those
"lazy" or otherwise).

It's nice to separate this abstract "relaxed" semantic from concrete
implementations - the simplest is to just delegate to existing execute,
but for the main EL impls translates to whether a wakeup is required
after enqueuing.

Having a "global" abstraction also allows for simplification of our
internal use - for example encapsulating more of the common scheduled
future logic within AbstractScheduledEventExecutor.

Modifications

- Introduce public LazyRunnable interface and
AbstractEventExecutor#lazyExecute method (would be nice for this to be
added to EventExecutor interface in netty 5)
- Tweak existing SingleThreadEventExecutor mechanics to support these
- Replace internal use of NonWakeupRunnable (such as for pre-flush
channel writes)
- Uplift scheduling-related hooks into AbstractScheduledEventExecutor,
eliminating intermediate executeScheduledRunnable method

Result

Simpler code, cleaner and more useful/flexible abstractions - cleaner in
that they fully communicate the intent in a more general way, without
implying/exposing/restricting implementation details
2019-10-31 10:01:53 +01:00
SimpleYoung
82376fd889 Remove an unnecessary expression in NioEventLoop (#9729)
Motivation:

Remove an unnecessary expression to make the code more readable

Modification:

Remove `eventLoop == null`

Result:

Fixes #9725
2019-10-30 12:24:03 +01:00
Norman Maurer
6c061abc49
Hide Recycler implemention to allow experimenting with different implementions of an Object pool (#9715)
Motivation:

At the moment we directly extend the Recycler base class in our code which makes it hard to experiment with different Object pool implementation. It would be nice to be able to switch from one to another by using a system property in the future. This would also allow to more easily test things like https://github.com/netty/netty/pull/8052.

Modifications:

- Introduce ObjectPool class with static method that we now use internally to obtain an ObjectPool implementation.
- Wrap the Recycler into an ObjectPool and return it for now

Result:

Preparation for different ObjectPool implementations
2019-10-26 00:34:30 -07:00
root
844b82b986 [maven-release-plugin] prepare for next development iteration 2019-10-24 12:57:00 +00:00
root
d066f163d7 [maven-release-plugin] prepare release netty-4.1.43.Final 2019-10-24 12:56:30 +00:00
Norman Maurer
6c05d16967
Use @SuppressJava6Requirement for animal sniffer plugin to ensure we always guard correctly (#9655)
Motivation:

We can use the `@SuppressJava6Requirement` annotation to be more precise about when we use Java6+ APIs. This helps us to ensure we always protect these places.

Modifications:

Make use of `@SuppressJava6Requirement` explicit

Result:

Fixes https://github.com/netty/netty/issues/2509.
2019-10-14 15:54:49 +02:00
stroller
deb34a0f8b remove useless doc for 5.0 (#9546)
Fix javadoc issues

Motivation:

Remove useless doc for 5.0

Modifications:

Remove description for 5.0

Result:

Useless doc will be removed.
2019-10-12 20:17:18 +02:00
Nick Hill
99a4cecec6 Fix NioEventLoopTest#testChannelsRegistered flakiness (#9650)
Motivation

NioEventLoopTest#testChannelsRegistered() fails intermittently due to
use of SingleThreadEventLoop#channelsRegistered() which is not
threadsafe and unreliable when called from outside the event loop.

Modifications

Add static registeredChannels method to NioEventLoopTest and
AbstractSingleThreadEventLoopTest to call from the tests via event loop
instead of directly.

Result

Hopefully fewer test failures
2019-10-09 16:57:54 +04:00
Nick Hill
170e4deee6 Fix event loop shutdown timing fragility (#9616)
Motivation

The current event loop shutdown logic is quite fragile and in the
epoll/NIO cases relies on the default 1 second wait/select timeout that
applies when there are no scheduled tasks. Without this default timeout
the shutdown would hang indefinitely.

The timeout only takes effect in this case because queued scheduled
tasks are first cancelled in
SingleThreadEventExecutor#confirmShutdown(), but I _think_ even this
isn't robust, since the main task queue is subsequently serviced which
could result in some new scheduled task being queued with much later
deadline.

It also means shutdowns are unnecessarily delayed by up to 1 second.

Modifications

- Add/extend unit tests to expose the issue
- Adjust SingleThreadEventExecutor shutdown and confirmShutdown methods
to explicitly add no-op tasks to the taskQueue so that the subsequent
event loop iteration doesn't enter blocking wait (as looks like was
originally intended)

Results

Faster and more robust shutdown of event loops, allows removal of the
default wait timeout
2019-10-07 11:06:01 +04:00
Codrut Stancu
4980a6b304 Register sun.nio.ch.SelectorImpl fields for unsafe access. (#9631)
Motivation:

On JDK > 9 Netty uses Unsafe to write two internal JDK fields: sun.nio.ch.SelectorImp.selectedKeys and sun.nio.ch.SelectorImpl.publicSelectedKeys. This is done in transport/src/main/java/io/netty/channel/nio/NioEventLoop.java:225, in openSelector() method. The GraalVM analysis cannot do the Unsafe registration automatically because the object field offset computation is hidden behind two layers of calls.

Modifications:

This PR updates the Netty GraalVM configuration by registering those fields for unsafe access.
 
Result:

Improved support for Netty on GraalVM with JDK > 9.
2019-10-07 10:10:46 +04:00
bruce
b39ffed042 Fix incorrect calculation of next buffer size in AdaptiveRecvByteBufAllocator (#9555)
Motivation:

Due a bug we did not always correctly calculate the next buffer size in AdaptiveRecvByteBufAllocator.

Modification:

Fix calculation and add unit test

Result:

Correct calculation is always used.
2019-09-27 09:59:25 +02:00
root
92941cdcac [maven-release-plugin] prepare for next development iteration 2019-09-25 06:15:31 +00:00
root
bd907c3b3a [maven-release-plugin] prepare release netty-4.1.42.Final 2019-09-25 06:14:31 +00:00
Norman Maurer
3ad037470e
Correctly reset cached local and remote address when disconnect() is called (#9545)
Motivation:

We should correctly reset the cached local and remote address when a Channel.disconnect() is called and the channel has a notion of disconnect vs close (for example DatagramChannel implementations).

Modifications:

- Correctly reset cached kicak abd remote address
- Update testcase to cover it and so ensure all transports work in a consistent way

Result:

Correctly handle disconnect()
2019-09-19 08:51:10 +02:00
stroller
d01282e5f9 Remove duplicated calculation (#9565)
Motivation:

calculateMaxBytesPerGatheringWrite() contains duplicated calculation:  getSendBufferSize() << 1

Modifications:

Remove the duplicated calculation

Result:

The method will be clear and better
2019-09-15 08:05:21 +02:00
root
01d805bb76 [maven-release-plugin] prepare for next development iteration 2019-09-12 16:09:55 +00:00
root
7cf69022d4 [maven-release-plugin] prepare release netty-4.1.41.Final 2019-09-12 16:09:00 +00:00