Commit Graph

2086 Commits

Author SHA1 Message Date
Chris Vest
56adab2743
TCP Fast Open for clients (#11006)
Support TCP Fast Open for clients and make SslHandler take advantage

Motivation:
- TCP Fast Open allow us to send a small amount of data along side the initial SYN packet when establishing a TCP connection.
- The TLS Client Hello packet is small enough to fit in there, and is also idempotent (another requirement for using TCP Fast Open), so if we can save a round-trip when establishing TLS connections when using TFO.

Modification:
- Add support for client-side TCP Fast Open for Epoll, and also lowers the Linux kernel version requirements to 3.6.
- When adding the SslHandler to a pipeline, if TCP Fast Open is enabled for the channel (and the channel is not already active) then start the handshake early by writing it to the outbound buffer.
- An important detail to note here, is that the outbound buffer is not flushed at this point, like it would for normal handshakes. The flushing happens later as part of establishing the TCP connection.

Result:
- It is now possible for clients (on epoll) to open connections with TCP Fast Open.
- The SslHandler automatically detects when this is the case, and now send its Client Hello message as part of the initial data in the TCP Fast Open flow when available, saving a round-trip when establishing TLS connections.

Co-authored-by: Colin Godsey <crgodsey@gmail.com>
2021-02-15 13:13:44 +01:00
root
411f76d3ad [maven-release-plugin] prepare for next development iteration 2021-02-08 10:48:37 +00:00
root
97d044812d [maven-release-plugin] prepare release netty-4.1.59.Final 2021-02-08 10:47:46 +00:00
Norman Maurer
c735357bf2 Use Files.createTempFile(...) to ensure the file is created with proper permissions
Motivation:

File.createTempFile(String, String)` will create a temporary file in the system temporary directory if the 'java.io.tmpdir'. The permissions on that file utilize the umask. In a majority of cases, this means that the file that java creates has the permissions: `-rw-r--r--`, thus, any other local user on that system can read the contents of that file.
This can be a security concern if any sensitive data is stored in this file.

This was reported by Jonathan Leitschuh <jonathan.leitschuh@gmail.com> as a security problem.

Modifications:

Use Files.createTempFile(...) which will use safe-defaults when running on java 7 and later. If running on java 6 there isnt much we can do, which is fair enough as java 6 shouldnt be considered "safe" anyway.

Result:

Create temporary files with sane permissions by default.
2021-02-08 11:44:05 +01:00
Norman Maurer
1e87c711b4
Ensure we create a copy of the attributes and options when bootstrap a channel (#10965)
Motivation:

We need to ensure we copy the attributes and options when bootstrap the channel as otherwise we may change the underlying Entry.
This is similar to what was reported in https://github.com/netty/netty-incubator-codec-quic/issues/152.

Modifications:

- Do a copy and re-use methods
- Add unit tests

Result:

Don't affect attributes / options of channels that are already bootstrapped
2021-01-26 19:26:20 +01:00
root
a137ce2042 [maven-release-plugin] prepare for next development iteration 2021-01-13 10:28:54 +00:00
root
10b03e65f1 [maven-release-plugin] prepare release netty-4.1.58.Final 2021-01-13 10:27:17 +00:00
root
c6b894d03d [maven-release-plugin] prepare for next development iteration 2021-01-12 11:10:44 +00:00
root
b016568e21 [maven-release-plugin] prepare release netty-4.1.57.Final 2021-01-12 11:10:20 +00:00
root
a9ec3d86f6 [maven-release-plugin] prepare for next development iteration 2020-12-17 06:11:39 +00:00
root
1188d8320e [maven-release-plugin] prepare release netty-4.1.56.Final 2020-12-17 06:11:18 +00:00
root
f57d64f1c7 [maven-release-plugin] prepare for next development iteration 2020-12-08 11:51:39 +00:00
root
38da45ffe1 [maven-release-plugin] prepare release netty-4.1.55.Final 2020-12-08 11:51:25 +00:00
root
944a020586 [maven-release-plugin] prepare for next development iteration 2020-11-11 05:47:51 +00:00
root
715353ecd6 [maven-release-plugin] prepare release netty-4.1.54.Final 2020-11-11 05:47:37 +00:00
root
afca81a9d8 [maven-release-plugin] rollback the release of netty-4.1.54.Final 2020-11-10 12:02:24 +00:00
root
e256074e49 [maven-release-plugin] prepare for next development iteration 2020-11-10 11:12:23 +00:00
root
cea659bd8a [maven-release-plugin] prepare release netty-4.1.54.Final 2020-11-10 11:12:06 +00:00
Norman Maurer
5ffca6ef4a
Use http in xmlns URIs to make maven release plugin happy again (#10788)
Motivation:

https in xmlns URIs does not work and will let the maven release plugin fail:

```
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  1.779 s
[INFO] Finished at: 2020-11-10T07:45:21Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-release-plugin:2.5.3:prepare (default-cli) on project netty-parent: Execution default-cli of goal org.apache.maven.plugins:maven-release-plugin:2.5.3:prepare failed: The namespace xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" could not be added as a namespace to "project": The namespace prefix "xsi" collides with an additional namespace declared by the element -> [Help 1]
[ERROR]
```

See also https://issues.apache.org/jira/browse/HBASE-24014.

Modifications:

Use http for xmlns

Result:

Be able to use maven release plugin
2020-11-10 10:22:35 +01:00
Norman Maurer
c6e2934357
Revert "Allow and skip null handlers when adding a vararg list of handlers (#10751)" (#10770)
This reverts commit 3b90b536bb.
2020-11-03 21:12:29 +01:00
Bennett Lynch
3b90b536bb
Allow and skip null handlers when adding a vararg list of handlers (#10751)
Motivation:

Allowing null handlers allows for more convenient idioms in
conditionally adding handlers, e.g.,

ch.pipeline().addLast(
        new FooHandler(),
        condition ? new BarHandler() : null,
        new BazHandler()
);

Modifications:

* Change addFirst(..) and addLast(..) to skip null handlers, rather than
break or short-circuit.
* Add new unit tests.

Result:

* Makes addFirst(..) and addLast(..) behavior more consistent
* Resolves https://github.com/netty/netty/issues/10728
2020-11-03 21:11:35 +01:00
Dave Syer
d62384b227
Add reflection config for JMX runtime (#10753)
Motivation:

`DefaultChannelId` uses reflection to access the JMX runtime. This needs some extra config for GraalVM.

Modification:

Add config for GraalVM

Result:

Works when using GraalVM native image
2020-10-30 15:26:41 +01:00
Norman Maurer
ddebc1027d
Make changes to prepare for io_uring incubator repository (#10741)
Motivation:

During the last few month we did develop an io_uring based transport which shows very promising performance numbers. To give it more time to bake we will develop it outside of netty in an "incubator" module which will make it clear to users what to expect and also allow us to seperate its release cycle. While the implementation of it is very self contained there are few small adjustments that need to be made in netty itself to allow us to reuse code.

Modifications:

- AbstractChannel: Add method which can be used when a write fails and remove final from one method
- IovArray: Allow to create an IovArray from a ByteBuf instance
- FileDescriptor: Allow to reuse mark close logic via sub-class

Result:

Be able to reuse netty core classes in io_uring incubator repository
2020-10-28 15:31:02 +01:00
Norman Maurer
f5185ed73b
Release message before notify promise (#10726)
Motivation:

We should preferable always release the message before we notify the promise. Thhis has a few advantages:

 - Release memory as soon as possible
 - Listeners observe the "more correct" reference count

Modifications:

Release message before fail the promises

Result:

Faster releasing of resources. This came up in https://github.com/netty/netty/issues/10723
2020-10-26 12:59:20 +01:00
Roman Puchkovskiy
4ecd78e104
Fix native image build on modern GraalVM versions for the cases when the program uses netty-dns (#10630)
Motivation:

Since GraalVM version 19.3.0, instances of java.net.InetAddress (and its subclasses Inet4Address and Inet6Address) are not allowed in native image heap (that is, they cannot be stored in static fields of classes initialized at build time or be reachable through static fields of such classes). When building a native image, it makes sense to initialize at build time as many classes as possible.
But some fields of some classes in Netty (for example, NetUtil.LOCALHOST4) contain InetAddress instances. If a program is using code path that makes it possible to reach such fields at build time initialization, it becomes impossible to build a native image initializing core Netty classes initialized at runtime. An example of such a program is a client that uses netty-dns.

Modifications:

- Add netty-testsuite-native-image-client Maven module to test that such an example program can be built after the corresponding fixes
- Add native-image.properties to resolver-dns module to move initialization of some classes to runtime (some of them are parsing configuration during initialization, so it makes no sense to initialize them at build time; for others, it's needed to avoid InetAddress reachability at build time)
- Add substitutions for NetUtil.LOCALHOST4, NetUtil.LOCALHOST6 and NetUtil.LOCALHOST to overcome the InetAddress-related prohibition
- Extract some initialization code from NetUtil to NetUtilInitializations to allow it to be used by the substitutions

Result:

A client program using netty-dns with --initialize-at-build-time=io.netty builds successfully
2020-10-26 08:34:31 +01:00
greenjustin
090e9a7271
Allow EventLoops to rethrow Error (#10694)
Motivation:

Thread.stop() works by producing a ThreadDeath error in the target thread. EventLoops swallow all Throwables, which makes them effectively unkillable. This is effectively a memory leak, for our application. Beside this we should also just regrow all `Error` as there is almost no way to recover.

Modification:

Edit the EventLoops that swallow Throwables to instead rethrow Error.

Result:

`EventLoop` can crash if `Error` is thrown
2020-10-24 14:56:33 +02:00
Artem Smotrakov
e5951d46fc
Enable nohttp check during the build (#10708)
Motivation:

HTTP is a plaintext protocol which means that someone may be able
to eavesdrop the data. To prevent this, HTTPS should be used whenever
possible. However, maintaining using https:// in all URLs may be
difficult. The nohttp tool can help here. The tool scans all the files
in a repository and reports where http:// is used.

Modifications:

- Added nohttp (via checkstyle) into the build process.
- Suppressed findings for the websites
  that don't support HTTPS or that are not reachable

Result:

- Prevent using HTTP in the future.
- Encourage users to use HTTPS when they follow the links they found in
  the code.
2020-10-23 14:44:18 +02:00
James Yuzawa
726944146b
Use named exceptions in ChannelPool implementations (#10721)
Motivation:

I was collecting stats for failed promises with a FixedChannelPool and I was bucketing by stats using cause.getSimpleName(). After #9152 was released, the introduction of the anonymous classes make getSimpleName() return "" causing confusion.

Modification:

Use named classes in the ChannelPool implementations. I made them private, but I can change that if you think otherwise.

Result:

The SimpleChannelPool fails the promises with a ChannelPoolFullException. The FixedChannelPool fails the promises with an AcquireTimeoutException. Also AcquireTimeoutException is more specific than just a plain TimeoutException, which is also useful for troubleshooting. If you want different class names, please advise.
2020-10-23 14:29:37 +02:00
Norman Maurer
ad8fe88abd
We should have a special config that allows to configure half closure for DuplexChannel (#10701)
Motivation:

DuplexChannel allow for half-closure, we should have a special config interface for it as well.

Modifications:

Add DuplexChannelConfig which allows to configure half-closure.

Result:

More consistent types
2020-10-21 15:26:27 +02:00
Norman Maurer
ffbddcd842
Replace deprecated Assert.assertThat(...) with MatcherAssert.assertThat(...) (#10699)
Motivation:

junit deprecated Assert.assertThat(...)

Modifications:

Use MatcherAssert.assertThat(...) as replacement for deprecated method

Result:

Less deprecation warnings
2020-10-18 14:30:52 +02:00
Artem Smotrakov
1ca7d5db81
Fix or suppress LGTM findings (#10689)
Motivation:

LGTM reports multiple issues. They need to be triaged,
and real ones should be fixed.

Modifications:
- Fixed multiple issues reported by LGTM, such as redundant conditions,
  resource leaks, typos, possible integer overflows.
- Suppressed false-positives.
- Added a few testcases.

Result:

Fixed several possible issues, get rid of false alarms in the LGTM report.
2020-10-17 09:49:44 +02:00
Norman Maurer
6b613682ba
Ensure we don't leak the ClassLoader in the backtrace (#10691)
Motivation:

We have a few classes in which we store and reuse static instances of various exceptions. When doing so it is important to also override fillInStacktrace() and so prevent the leak of the ClassLoader in the internal backtrace field.

Modifications:

- Add overrides of fillInStracktrace when needed
- Move ThrowableUtil usage in the static methods

Result:

Fixes https://github.com/netty/netty/pull/10686
2020-10-15 20:41:29 +02:00
root
01b7e18632 [maven-release-plugin] prepare for next development iteration 2020-10-13 06:29:26 +00:00
root
d4a0050ef3 [maven-release-plugin] prepare release netty-4.1.53.Final 2020-10-13 06:29:02 +00:00
Norman Maurer
69f9a94f59
Increase initial buffer size in AdaptiveRecvByteBufAllocator (#10600)
Motivation:

We should use an initial buffer size with is >= 1500 (which is a common setting for MTU) to reduce the need for memory copies when a new connection is established. This is especially interesting when SSL / TLS comes into the mix.

This was ported from swiftnio:

https://github.com/apple/swift-nio/pull/1641

Modifications:

Increase the initial size from 1024 to 2048.

Result:

Possible less memory copies on new connections
2020-09-22 17:27:25 +02:00
Norman Maurer
71d034593f
Only create ConnectTimeoutException if really needed (#10595)
Motivation:

Creating exceptions is expensive so we should only do so if really needed.

Modifications:

Only create the ConnectTimeoutException if we really need it.

Result:

Less overhead
2020-09-21 21:32:20 +02:00
root
957ef746d8 [maven-release-plugin] prepare for next development iteration 2020-09-08 05:26:25 +00:00
root
ada9c38c0a [maven-release-plugin] prepare release netty-4.1.52.Final 2020-09-08 05:26:05 +00:00
Norman Maurer
4ececfb70d
Create a stackless ClosedChannelException to reduce overhead when the… (#10523)
Motivation:

In some benchmarks closing the Channel attributes to a lot of overhead due the call of fillInStackTrace(). We should reduce this overhead.

Modifications:

- Create a StacklessClosedChannelException and use it to reduce overhead.
- Only call ChannelOutboundBuffer.failFlushed(...) when there was a flushed message at all.

Result:

Less performance overhead when closing the Channel
2020-09-01 15:25:41 +02:00
Aayush Atharva
ee77fad3f8
Minor typo (#10518)
Motivation:
I was working on the transport part in Netty (ofc, solving a major issue) and I found this typo so thought to fix it.

Modification:
Fixed Typo

Result:
No more confusion between `us` and `use`.
2020-08-31 08:59:27 +02:00
Norman Maurer
830fc0d660
Remove incorrect javadocs that were not updated for new versions of netty (#10487)
Motivation:

At some point a ChannelHandlerContext did have its own AttributeMap which is not true since 4.1.x was released. Unfortunally we missed to update the javadocs and so these don't reflect reality

Modifications:

Update javadocs

Result:

Fixes https://github.com/netty/netty/issues/10477
2020-08-19 17:13:01 +02:00
Kevin Wu
54bfd21e52
Fix #10434 OutOfDirectMemoryError causes cpu load too high and socket is full (#10457)
Motivation:

When we were using the netty http protocol, OOM occurred, this problem has been in 4.1.51.Final Fix [# 10424](https://github.com/netty/netty/issues/10424), even if OOM is up, the service will still receive new connection events, will occur again OOM and eventually cause the connection not to be released. 

code `byteBuf = allocHandle.allocate(allocator);`

Modification:

I fail to create buffer when I try to receive new data, i determine if it is OOM then the close read event releases the connection.
```java
        if (close || cause instanceof OutOfMemoryError || cause instanceof IOException) {
            closeOnRead(pipeline);
        }
```

Result:

Fixes # [10434](https://github.com/netty/netty/issues/10434).
2020-08-13 10:14:19 +02:00
skyguard1
cd0203b5c7
Add default handling for switch statement (#10408)
Motivation:

When a switch statement is used we should always define a `default:` so we don't introduce bugs due fall-through.

Modification:

Add missing `default:`s

Result:

Less error-prone code
2020-07-16 10:54:50 +02:00
root
bfbeb2dec6 [maven-release-plugin] prepare for next development iteration 2020-07-09 12:27:06 +00:00
root
646934ef0a [maven-release-plugin] prepare release netty-4.1.51.Final 2020-07-09 12:26:30 +00:00
Norman Maurer
e0dc054927
Expand ChannelPipeline javadocs to cover UnorderedThreadPoolEventExecutor (#10361)
Motivation:

Seems like some users are suprised by the behaviour of DefaultEventExecutor when used within the ChannelPipeline. We should clarify the semantics and also mention UnordedThreadPoolEventExecutor which may be more inline with their expectations

Modifications:

Add javadocs section about UnorderedThreadPoolEventExecutor and expand details for DefaultEventExecutor

Result:

Clarify sematics
2020-06-23 09:28:34 +02:00
Norman Maurer
75df58a7e1
Guard against re-entrancy issues while draining AbstractCoalescingBufferQueue (#10294)
Motivation:

AbstractCoalescingBufferQueue had a bug which could lead to an empty queue while still report bytes left. This was due the fact that we decremented the pending bytes before draining the queue one-by-one. The problem here is that while the queue is drained we may notify the promise which may add again buffers to the queue for which we never decrement the bytes while we drain these

Modifications:

- Decrement the pending bytes every time we drain a buffer from the queue
- Add unit tests

Result:

Fixes https://github.com/netty/netty/issues/10286
2020-05-15 09:51:33 +02:00
root
caf51b7284 [maven-release-plugin] prepare for next development iteration 2020-05-13 06:00:23 +00:00
root
8c5b72aaf0 [maven-release-plugin] prepare release netty-4.1.50.Final 2020-05-13 05:59:55 +00:00
root
9c5008b109 [maven-release-plugin] prepare for next development iteration 2020-04-22 09:57:54 +00:00