Commit Graph

176 Commits

Author SHA1 Message Date
Scott Mitchell
0b0c234bed
SslHandler flushing with TCP Fast Open fix (#11077)
Motivation:
SslHandler owns the responsibility to flush non-application data
(e.g. handshake, renegotiation, etc.) to the socket. However when
TCP Fast Open is supported but the client_hello cannot be written
in the SYN the client_hello may not always be flushed. SslHandler
may not wrap/flush previously written/flushed data in the event
it was not able to be wrapped due to NEED_UNWRAP state being
encountered in wrap (e.g. peer initiated renegotiation).

Modifications:
- SslHandler to flush in channelActive() if TFO is enabled and
  the client_hello cannot be written in the SYN.
- SslHandler to wrap application data after non-application data
  wrap and handshake status is FINISHED.
- SocketSslEchoTest only flushes when writes are done, and waits
  for the handshake to complete before writing.

Result:
SslHandler flushes handshake data for TFO, and previously flushed
application data after peer initiated renegotiation finishes.
2021-03-14 14:18:27 +01:00
Netty Project Bot
e83132fcf2 [maven-release-plugin] prepare for next development iteration 2021-03-09 08:18:31 +00:00
Netty Project Bot
eef26e8bb5 [maven-release-plugin] prepare release netty-4.1.60.Final 2021-03-09 08:18:26 +00:00
Netty Project Bot
b8d843757b [maven-release-plugin] rollback the release of netty-4.1.60.Final 2021-03-09 07:39:13 +00:00
Netty Project Bot
4f099dd11f [maven-release-plugin] prepare for next development iteration 2021-03-09 07:21:55 +00:00
Netty Project Bot
44e392e851 [maven-release-plugin] prepare release netty-4.1.60.Final 2021-03-09 07:21:51 +00:00
Chris Vest
56adab2743
TCP Fast Open for clients (#11006)
Support TCP Fast Open for clients and make SslHandler take advantage

Motivation:
- TCP Fast Open allow us to send a small amount of data along side the initial SYN packet when establishing a TCP connection.
- The TLS Client Hello packet is small enough to fit in there, and is also idempotent (another requirement for using TCP Fast Open), so if we can save a round-trip when establishing TLS connections when using TFO.

Modification:
- Add support for client-side TCP Fast Open for Epoll, and also lowers the Linux kernel version requirements to 3.6.
- When adding the SslHandler to a pipeline, if TCP Fast Open is enabled for the channel (and the channel is not already active) then start the handshake early by writing it to the outbound buffer.
- An important detail to note here, is that the outbound buffer is not flushed at this point, like it would for normal handshakes. The flushing happens later as part of establishing the TCP connection.

Result:
- It is now possible for clients (on epoll) to open connections with TCP Fast Open.
- The SslHandler automatically detects when this is the case, and now send its Client Hello message as part of the initial data in the TCP Fast Open flow when available, saving a round-trip when establishing TLS connections.

Co-authored-by: Colin Godsey <crgodsey@gmail.com>
2021-02-15 13:13:44 +01:00
root
411f76d3ad [maven-release-plugin] prepare for next development iteration 2021-02-08 10:48:37 +00:00
root
97d044812d [maven-release-plugin] prepare release netty-4.1.59.Final 2021-02-08 10:47:46 +00:00
Norman Maurer
5b41f3d25b
Ensure native methods for unix-native-common are only registered once. (#10932)
Motiviation:

We need to ensure we only register the methods for unix-native-common once as otherwise it may have strange side-effects.

Modifications:

- Add extra method that should be called to signal that we need to register the methods. The registration will only happen once.
- Adjust code to make use of it.

Result:

No more problems due incorrect registration of these methods.
2021-01-14 17:52:04 +01:00
root
a137ce2042 [maven-release-plugin] prepare for next development iteration 2021-01-13 10:28:54 +00:00
root
10b03e65f1 [maven-release-plugin] prepare release netty-4.1.58.Final 2021-01-13 10:27:17 +00:00
Norman Maurer
d58ce7a151
Revert "Ensure we only register native methods once (#10876)" (#10928)
Motivation:

This reverts commit 7fb62a93b8 as it broke native loading in some cases due maven dependencies.

Modification:

Revert the commit.

Result:

Native loading works again
2021-01-13 11:07:39 +01:00
root
c6b894d03d [maven-release-plugin] prepare for next development iteration 2021-01-12 11:10:44 +00:00
root
b016568e21 [maven-release-plugin] prepare release netty-4.1.57.Final 2021-01-12 11:10:20 +00:00
Norman Maurer
3e8e52725b
Use netty-jni-util via maven central (#10877)
Motivation:

netty-jni-util is now also hosted on maven central. Let's use it

Modifications:

Adjust plugins to just unpack netty-jni-util and use it

Result:

Be able to use what is in the maven cache for netty-jni-util
2020-12-19 10:37:32 +01:00
Norman Maurer
7fb62a93b8
Ensure we only register native methods once (#10876)
Motivation:

We need to ensure we only register native methods once as otherwise we may end up in an "invalid" state. The problem here was that before it was basically the responsibility the user of transport-native-unix-common to register the methods. This is error prone as there may be multiple users of these on the classpath at the same time.

Modifications:

- Provide a way to init native lib without register the native methods of the provided classes. This is needed to be able to re-use functionality which is exposed to our internal native code
- Use flatten plugin to correctly resolve classifier and so have the correct dependency
- Call Unix.* method to ensure we register the methods correctly once
- Include native lib as well in the native jars of unix-common

Result:

Be able to have multiple artifacts of the classpath that depends on the unix-common. Related to https://github.com/netty/netty-incubator-transport-io_uring/issues/15
2020-12-18 10:37:49 +01:00
root
a9ec3d86f6 [maven-release-plugin] prepare for next development iteration 2020-12-17 06:11:39 +00:00
root
1188d8320e [maven-release-plugin] prepare release netty-4.1.56.Final 2020-12-17 06:11:18 +00:00
Norman Maurer
882c111568
Ensure IovArray is usuable without sun.misc.Unsafe (#10870)
Motivation:

https://github.com/netty/netty/pull/10814 did fix a bug where we did try to call memoryAddress() even tho this is not supported. Unfortunally this fix was only applied for one method and so we missed another method which then could throw an exception when we called memoryAddress()

Modifications:

- Also fix the memoryAddress(offset) method.
_ Adjust unit test to also test this.

Result:

Fixes https://github.com/netty/netty/issues/10813 completely.
2020-12-16 20:46:19 +01:00
root
f57d64f1c7 [maven-release-plugin] prepare for next development iteration 2020-12-08 11:51:39 +00:00
root
38da45ffe1 [maven-release-plugin] prepare release netty-4.1.55.Final 2020-12-08 11:51:25 +00:00
Norman Maurer
3ac9685580
Use aarch_64 in a consistent way (#10845)
Motivation:

We should use aarch_64 in our classifier / jni libname on aarch64 as  os.detected.arch uses the name. Being non consistent (especially across our different projects) already gave us a lot of trouble in the past.
Let's fix this once for all.

Modifications:

Use aarch_64

Result:

More consistent classifier usage on aarch64
2020-12-07 12:14:23 +01:00
Norman Maurer
ba83a8840f
IovArray should support when there is no unsafe present (#10814)
Motivation:

In some enviroments sun.misc.Unsafe is not present. We should support these as well.

Modifications:

Fallback to JNI if we can't directly access the memoryAddress of the buffer.

Result:

Fixes https://github.com/netty/netty/issues/10813
2020-11-23 14:03:32 +01:00
root
944a020586 [maven-release-plugin] prepare for next development iteration 2020-11-11 05:47:51 +00:00
root
715353ecd6 [maven-release-plugin] prepare release netty-4.1.54.Final 2020-11-11 05:47:37 +00:00
root
afca81a9d8 [maven-release-plugin] rollback the release of netty-4.1.54.Final 2020-11-10 12:02:24 +00:00
root
e256074e49 [maven-release-plugin] prepare for next development iteration 2020-11-10 11:12:23 +00:00
root
cea659bd8a [maven-release-plugin] prepare release netty-4.1.54.Final 2020-11-10 11:12:06 +00:00
Norman Maurer
5ffca6ef4a
Use http in xmlns URIs to make maven release plugin happy again (#10788)
Motivation:

https in xmlns URIs does not work and will let the maven release plugin fail:

```
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  1.779 s
[INFO] Finished at: 2020-11-10T07:45:21Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-release-plugin:2.5.3:prepare (default-cli) on project netty-parent: Execution default-cli of goal org.apache.maven.plugins:maven-release-plugin:2.5.3:prepare failed: The namespace xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" could not be added as a namespace to "project": The namespace prefix "xsi" collides with an additional namespace declared by the element -> [Help 1]
[ERROR]
```

See also https://issues.apache.org/jira/browse/HBASE-24014.

Modifications:

Use http for xmlns

Result:

Be able to use maven release plugin
2020-11-10 10:22:35 +01:00
Norman Maurer
a63faa4fa1
Use netty-jni-util and so remove a lot of duplication (#10735)
Motivation:

We had a lot of duplication in our jni code which was mostly due macros but also related to how we support shading. By using netty-jni-util we can share all the code between netty and netty-tcnative ( and possible other jni based netty projects in the future).

Modifications:

- Use netty-jni-util and re-use its macros / functions
- Remove duplicated code
- Adjust build files

Result:

Less code duplication for JNI
2020-10-29 16:36:07 +01:00
Norman Maurer
b4edf21bd3
Errors.throwConnectException(...) should be public (#10745)
Motivation:

ddebc1027d missed to make Errors.throwConnectException(...) public

Modifications:

Make method public

Result:

Be able to use Errors.throwConnectException(...) from other module
2020-10-29 07:44:49 +01:00
Norman Maurer
ddebc1027d
Make changes to prepare for io_uring incubator repository (#10741)
Motivation:

During the last few month we did develop an io_uring based transport which shows very promising performance numbers. To give it more time to bake we will develop it outside of netty in an "incubator" module which will make it clear to users what to expect and also allow us to seperate its release cycle. While the implementation of it is very self contained there are few small adjustments that need to be made in netty itself to allow us to reuse code.

Modifications:

- AbstractChannel: Add method which can be used when a write fails and remove final from one method
- IovArray: Allow to create an IovArray from a ByteBuf instance
- FileDescriptor: Allow to reuse mark close logic via sub-class

Result:

Be able to reuse netty core classes in io_uring incubator repository
2020-10-28 15:31:02 +01:00
Norman Maurer
03aafb9cff
Unregister all previous registered native methods if loading of native code fails… (#10719)
Motivation:

It's important to unload all previous registered native methods when there is a failure during loading the native lib. Failing to do so may lead to an "invalid state" and so may segfault the JVM when trying to call a native method that was previous loaded.

This was observed when two versions of netty-tcnative were on the classpath which had different requirements in terms of linking.

Something like this was reported in he hs log:

```
Instructions: (pc=0x0000000116413bf0)
0x0000000116413bd0:
[error occurred during error reporting (printing registers, top of stack, instructions near pc), id 0xb]

Register to memory mapping:

RAX=0x0000000116413bf0 is an unknown value
RBX={method} {0x000000011422e708} 'aprMajorVersion' '()I' in 'io/netty/internal/tcnative/Library'
RCX=0x000000000000000a is an unknown value
RDX=0x000000000000000a is an unknown value
```

Modifications:

- Unregister previous registered native methods on failure
- Unregister previous registered native methods on on unload of the native lib

Result:

No more segfault caused by invalid state when loading of the native lib fails in between. In this case the user will receive an error now like:
2020-10-26 14:15:04 +01:00
Artem Smotrakov
e5951d46fc
Enable nohttp check during the build (#10708)
Motivation:

HTTP is a plaintext protocol which means that someone may be able
to eavesdrop the data. To prevent this, HTTPS should be used whenever
possible. However, maintaining using https:// in all URLs may be
difficult. The nohttp tool can help here. The tool scans all the files
in a repository and reports where http:// is used.

Modifications:

- Added nohttp (via checkstyle) into the build process.
- Suppressed findings for the websites
  that don't support HTTPS or that are not reachable

Result:

- Prevent using HTTP in the future.
- Encourage users to use HTTPS when they follow the links they found in
  the code.
2020-10-23 14:44:18 +02:00
Norman Maurer
24e6f37745
Add NULL checks to fix possible undefined behavior (#10718)
Motivation:

In some situations we could have end up calling some functions with NULL parameters which in this case could lead to undefined behavior. All of this would have happened during loading of the native lib.

Modifications:

Add NULL check as guards and return early

Result:

Fix some possible undefined behavior
2020-10-23 14:39:27 +02:00
root
01b7e18632 [maven-release-plugin] prepare for next development iteration 2020-10-13 06:29:26 +00:00
root
d4a0050ef3 [maven-release-plugin] prepare release netty-4.1.53.Final 2020-10-13 06:29:02 +00:00
root
957ef746d8 [maven-release-plugin] prepare for next development iteration 2020-09-08 05:26:25 +00:00
root
ada9c38c0a [maven-release-plugin] prepare release netty-4.1.52.Final 2020-09-08 05:26:05 +00:00
Chris Vest
f58223982c
Fix netty_unix_errors.c compilation on MacOS (#10469)
Motivation:
 The feature test macros did not work as expected on MacOS, so we ended up compiling for the GNU
 variant which resulted in compilation errors.
Modification:
 Add `__APPLE__` as another indicator to use the XSI variant of `strerror_r`.
Result:
 The project now once again compiles on MacOS.
2020-08-11 14:00:08 +02:00
Chris Vest
8007f907dd
Use strerror_r for JNI error messages. (#10463)
Motivation:
We previously relied on `strerror`, but this function is unfortunately not thread-safe.

Modification:
The use of `strerror` has been changed to `strerror_r`, which is thread-safe.
This function has a more complicated API, and has portability concerns that needs to be handled.
This accounts for the relatively large increase in lines of code.

Result:
Error messages from JNI are now always generated in a thread-safe way.
2020-08-11 09:58:56 +02:00
Norman Maurer
4b7dba14c4
If user explicit ask to use an Inet6Address we should try to do so in… (#10415)
Motivation:

Even if the system does not support ipv6 we should try to use it if the user explicit pass an Inet6Address. This way we ensure we fail and not try to convert this to an ipv4 address internally.

This incorrect behavior was introduced by 70731bfa7e

Modifications:

If the user explicit passed an Inet6Address we force the usage of ipv6

Result:

Fixes https://github.com/netty/netty/issues/10402
2020-08-10 16:29:09 +02:00
root
bfbeb2dec6 [maven-release-plugin] prepare for next development iteration 2020-07-09 12:27:06 +00:00
root
646934ef0a [maven-release-plugin] prepare release netty-4.1.51.Final 2020-07-09 12:26:30 +00:00
root
caf51b7284 [maven-release-plugin] prepare for next development iteration 2020-05-13 06:00:23 +00:00
root
8c5b72aaf0 [maven-release-plugin] prepare release netty-4.1.50.Final 2020-05-13 05:59:55 +00:00
wangxiyuan
660611c450
Add epoll aarch64 maven config and Dockerfile (#9804)
Motivation:

`transport-native-epoll` doesn't have ARM release package. 

Modification:

This PR added cross compile profile for epoll. Then we can easily build aarch64 package on X86 machine. 

Result:
Fixes #8279
2020-04-23 13:58:08 +02:00
root
9c5008b109 [maven-release-plugin] prepare for next development iteration 2020-04-22 09:57:54 +00:00
root
d0ec961cce [maven-release-plugin] prepare release netty-4.1.49.Final 2020-04-22 09:57:26 +00:00