Commit Graph

10451 Commits

Author SHA1 Message Date
Norman Maurer
775b36ee66
Verify we can load native modules and add job that verifies on aarch64 as well (#10933)
Motivation:

As shown in the past we need to verify we actually can load the native as otherwise we may introduce regressions.

Modifications:

- Add new maven module which tests loading of native modules
- Add job that will also test loading on aarch64

Result:

Less likely to introduce regressions related to loading native code in the future
2021-01-14 17:11:19 +01:00
root
a137ce2042 [maven-release-plugin] prepare for next development iteration 2021-01-13 10:28:54 +00:00
root
10b03e65f1 [maven-release-plugin] prepare release netty-4.1.58.Final 2021-01-13 10:27:17 +00:00
Norman Maurer
d58ce7a151
Revert "Ensure we only register native methods once (#10876)" (#10928)
Motivation:

This reverts commit 7fb62a93b8 as it broke native loading in some cases due maven dependencies.

Modification:

Revert the commit.

Result:

Native loading works again
2021-01-13 11:07:39 +01:00
root
c6b894d03d [maven-release-plugin] prepare for next development iteration 2021-01-12 11:10:44 +00:00
root
b016568e21 [maven-release-plugin] prepare release netty-4.1.57.Final 2021-01-12 11:10:20 +00:00
Norman Maurer
e01f80ec0a
Update to netty-tcnative 2.0.36.Final (#10923)
Motivation:

Update to latest netty-tcnative release

Modifications:

Update version

Result:

Use latest version
2021-01-12 12:05:57 +01:00
Norman Maurer
cbebad8417
We need to ensure we always drain the error stack when a callback throws (#10920)
Motivation:

We need to ensure we always drain the error stack when a callback throws as otherwise we may pick up the error on a different SSL instance which uses the same thread.

Modifications:

- Correctly drain the error stack if native method throws
- Add a unit test which failed before the change

Result:

Always drain the error stack
2021-01-11 20:56:52 +01:00
Aki
3f04385919
Fix url in README.md (#10915)
Motivation:

We had a typo in the markdown

Modification:

Fix typo

Result:

Corect markdown
2021-01-11 07:48:58 +01:00
Aayush Atharva
0dc246eb12
Update BouncyCastle Libraries (#10906)
Motivation:
New versions of `Bouncy Castle` libraries are out and we should upgrade to them.

Modification:
Upgraded all `Bouncy Castle` libraries to the latest version.

Result:
The latest versions of `Bouncy Castle` libraries.

Fixes #10905.
2021-01-01 19:31:44 +01:00
YoungTae Seok
3043ebb21c
Ensure Checkstyle suppression for dependency-reduced-pom.xml on Windows (#10899)
Motivation:

`mvn package` on Windows fails if there are some `dependency-reduced-pom.xml` files generated by the previous build.
`mvn clean package` does not help because it does not remove those files at the clean phase.

Modifications:

Use the correct file pattern for the suppression filters.

Result:

Following `mvn package` runs well on Windows.
2021-01-01 19:30:13 +01:00
Norman Maurer
249392fe12
Update action-docker-layer-caching (#10900)
Motivation:

We are three releases behind.

Modifications:

Update to latest version

Result:

Use up-to-date action-docker-layer-caching version
2020-12-28 12:54:12 +01:00
Aayush Atharva
08681a817d
Add PushPromise and Priority Frame support in Http2FrameCodec (#10765)
Motivation:
Right now, we don't have to handle Push Promise Read in `Http2FrameCodec`. Push Promise is one of the key features of HTTP/2 and we should support it in our `Http2FrameCodec`.

Modification:
Added `Http2PushPromiseFrame` and `Http2PushPromiseFrame` to handle Push Promise and Promise Frame.

Result:
Fixes #10748
2020-12-27 14:30:24 +01:00
Norman Maurer
c5ffa293ac
Update README
Motivation:

We can add some status badge and also should clarify requirements.

Modifictations:

- Add status badge
- Clarify requirements

Result:

Cleanup docs
2020-12-27 10:14:08 +01:00
Norman Maurer
3a058d2172 We run all combinations when validate the PR, let's just use one type for normal push
Motivation:

Let us just only use one build config when building the 4.1 branch.

Modifications:

As we already do a full validation when doing the PR builds we can just only use one build config for pushes to the "main" branches

Result:

Faster build times
2020-12-25 10:20:12 +01:00
Norman Maurer
3bfe9d7242 Dont use cron for PRs
Motivation:

It doesnt make sense to use cron for PRs

Modifications:

Remove cron config

Result:

Cleanup
2020-12-24 17:13:44 +01:00
Norman Maurer
42145902a7
Correctly deploy artifacts that are build on different archs (#10893)
Motivation:

We need to take special care when deploying snapshots as we need to generate the jars in multiple steps

Modifications:

- Use the nexus staging pluging to stage jars locally in multiple steps
- Add extra job that will merge these staged jars and deploy these

Result:

Fixes https://github.com/netty/netty/issues/10887
2020-12-24 12:00:08 +01:00
Eric Anderson
4f6e62d50c
Avoid unsynchronized access to scheduledTaskQueue in GlobalEventExecutor (#10890)
Motivation:

A race detector discovered a data race in GlobalEventExecutor present in
netty 4.1.51.Final:

```
  Write of size 4 at 0x0000cea08774 by thread T103:
    #0 io.netty.util.internal.DefaultPriorityQueue.poll()Lio/netty/util/internal/PriorityQueueNode; DefaultPriorityQueue.java:113
    #1 io.netty.util.internal.DefaultPriorityQueue.poll()Ljava/lang/Object; DefaultPriorityQueue.java:31
    #2 java.util.AbstractQueue.remove()Ljava/lang/Object; AbstractQueue.java:113
    #3 io.netty.util.concurrent.AbstractScheduledEventExecutor.pollScheduledTask(J)Ljava/lang/Runnable; AbstractScheduledEventExecutor.java:133
    #4 io.netty.util.concurrent.GlobalEventExecutor.fetchFromScheduledTaskQueue()V GlobalEventExecutor.java:119
    #5 io.netty.util.concurrent.GlobalEventExecutor.takeTask()Ljava/lang/Runnable; GlobalEventExecutor.java:106
    #6 io.netty.util.concurrent.GlobalEventExecutor$TaskRunner.run()V GlobalEventExecutor.java:240
    #7 io.netty.util.internal.ThreadExecutorMap$2.run()V ThreadExecutorMap.java:74
    #8 io.netty.util.concurrent.FastThreadLocalRunnable.run()V FastThreadLocalRunnable.java:30
    #9 java.lang.Thread.run()V Thread.java:835
    #10 (Generated Stub) <null>

  Previous read of size 4 at 0x0000cea08774 by thread T110:
    #0 io.netty.util.internal.DefaultPriorityQueue.size()I DefaultPriorityQueue.java:46
    #1 io.netty.util.concurrent.GlobalEventExecutor$TaskRunner.run()V GlobalEventExecutor.java:263
    #2 io.netty.util.internal.ThreadExecutorMap$2.run()V ThreadExecutorMap.java:74
    #3 io.netty.util.concurrent.FastThreadLocalRunnable.run()V FastThreadLocalRunnable.java:30
    #4 java.lang.Thread.run()V Thread.java:835
    #5 (Generated Stub) <null>
```

The race is legit, but benign. To trigger it requires a TaskRunner to
begin exiting and set 'started' to false, more work to be scheduled
which starts a new TaskRunner, that work then needs to schedule
additional work which modifies 'scheduledTaskQueue', and then the
original TaskRunner checks 'scheduledTaskQueue'. But there is no danger
to this race as it can only produce a false negative in the condition
which causes the code to CAS 'started' which is thread-safe.

Modifications:

Delete problematic references to scheduledTaskQueue. The only way
scheduledTaskQueue could be modified since the last check is if another
TaskRunner is running, in which case the current TaskRunner doesn't
care.

Result:

Data-race free code, and a bit less code to boot.
2020-12-24 11:37:42 +01:00
Norman Maurer
78fcd5cf8f
Use matrix for job configurations (#10889)
Motivation:

We can use the matrix feature to define our jobs. This reduces a lot of config

Modification:

Use job matrix

Result:

Easier to maintain
2020-12-23 20:44:57 +01:00
Norman Maurer
1a708ac172
Also include one PR job that uses boringssl (#10886)
Motivation:

When validating PRs we should also at least run one job that uses boringssl

Modifications:

- Add job that uses boringssl
- Cleanup docker compose files
- Fix buffer leak in test

Result:

Also run with boringssl when PRs are validated
2020-12-23 19:11:04 +01:00
Norman Maurer
9145f28586
Use maven cachen when running analyze job (#10888)
Motivation:

To prevent failures to problems while downloading dependencies we shoud cache these

Modifications:

Add maven cache

Result:

No more failures due problems while downloading dependencies
2020-12-23 19:01:21 +01:00
Norman Maurer
c10606ef2a Only deploy snapshots for one arch
Motivation:

We need to find a way to deploy SNAPSHOTS for different arch with the same timestamp. Otherwise it will cause problems.

See https://github.com/netty/netty/issues/10887

Modification:

Skip all other deploys then x86_64

Result:

Users are able to use SNAPSHOTS for x86_6
2020-12-23 17:36:30 +01:00
Andrea Selva
01d44ff592
Add ConnAck message builder method to handle the creation of related properties. (#10812)
Motivation:
The CONNACK message builder `ConnAckBuilder` doesn't provide a smooth way to assign the message properties. This PR try to provide an simpler way to create them, in a lazy way.

Modification:
This PR permit to store properties in the ConnAck message, collecting them and inserting during the build phase. The syntax this PR introduces is:
```java
 MqttMessageBuilders.connAck().properties(new MqttMessageBuilders.PropertiesInitializer<MqttMessageBuilders.ConnAckPropertiesBuilder>() {
        @Override
        public void apply(MqttMessageBuilders.ConnAckPropertiesBuilder builder) {
            builder.assignedClientId("client1234");
            builder.userProperty("custom_property", "value");
         }
 }).build()
```
The name of the properties are defined in the `ConnAckPropertiesBuilder` so that is can be easily used by autocompletion tools.

Result:
This PR adds the builder class `ConnAckPropertiesBuilder`which is used by newly introduced method `properties` inside the message builder class `ConnAckBuilder`.
2020-12-23 10:50:09 +01:00
Norman Maurer
9da336f2fc Split pr and normal build (#10884)
Motivation:

We should better use seperate workflows for PR and normal builds

Modifications:

- Split workflows
- Better cache reuse

Result:

Cleanup
2020-12-23 09:32:04 +01:00
Norman Maurer
33833dec26 Use correct docker-compose task to deploy for cross compiled artifacts 2020-12-22 21:21:39 +01:00
Norman Maurer
900a85c688 Use correct docker-compose file to deploy cross compiled artifacts 2020-12-22 21:01:25 +01:00
Norman Maurer
0af6031110
Add deploy job for cross compiled aarch64 (#10883)
Motivation:

We should also deploy snapshots for our cross compiled native jars.

Modifications:

- Add job and docker files for deploying cross compiled native jars
- Ensure we map the maven cache into our docker containers

Result:

Deploy aarch64 jars and re-use cache
2020-12-22 20:43:35 +01:00
Norman Maurer
11d2ce7614
Add fallback for android when trying to access the filedescriptor via JNI (#10882)
Motivation:

Android seems to use a different field name so we should also try to access it with the name used by android.

Modifications:

Try first fd and if this fails try descriptor as field name

Result:

Workaround for android.
2020-12-22 20:42:52 +01:00
Norman Maurer
cd33d56f45 Only use build and deploy workflows for 4.1 for now 2020-12-22 19:30:57 +01:00
Aayush Atharva
2b1785458b
Change switch to if (#10880)
Motivation:
switch is used when we have a good amount of cases because switch is faster than if-else. However, we're using only 1 case in switch which can affect performance.

Modification:
Changed switch to if.

Result:
Good code.
2020-12-22 19:25:33 +01:00
Norman Maurer
87f5cc5fa2 Fix non leak build condition 2020-12-22 19:16:20 +01:00
Norman Maurer
7bf2ffac35 Fix deploy config 2020-12-22 18:48:22 +01:00
Norman Maurer
95b96126c3
Use Github workflows for CI (#10879)
Motivation:

We should just use GitHub Actions for the CI

Modifications:

- Adjust docker / docker compose files 
- Add different workflows and jobs to deploy and build the project

Result:

Don't depend on external CI services
2020-12-22 18:33:14 +01:00
Oleksii Kachaiev
ab8c4f22c6
Improve performance of HPACK static table lookup (#10840)
Motivation:

HPACK static table is organized in a way that fields with the same
name are sequential. Which means when doing sequential scan we can
short-circuit scan on name mismatch.

Modifications:

* `HpackStaticTable.getIndexIndensitive` returns -1 on name mismatch
rather than keep scanning.
* `HpackStaticTable` statically defined max position in the array
where name duplication is possible (after the given index there's
no need to check for other fields with the same name)
* Benchmark for different lookup patterns

Result:

Better HPACK static table lookup performance.

Co-authored-by: Norman Maurer <norman_maurer@apple.com>
2020-12-21 15:34:04 +01:00
Norman Maurer
3e8e52725b
Use netty-jni-util via maven central (#10877)
Motivation:

netty-jni-util is now also hosted on maven central. Let's use it

Modifications:

Adjust plugins to just unpack netty-jni-util and use it

Result:

Be able to use what is in the maven cache for netty-jni-util
2020-12-19 10:37:32 +01:00
Paul Lysak
9c0c996729
MQTT: foolproof SUBSCRIBE QoS encoding (#10874)
Motivation:

If the MQTT client specifies Subscribe Options parameters only available in MQTT v5 and tries to encode the message as MQTT v3 then an invalid QoS value is encoded

Modification:

Check MQTT version when encoding SUBSCRIBE message options, if it's 3.1 or 3.1.1 - only encode QoS, skip other options.

Result:

MqttEncoder produces a valid SUBSCRIBE message even if the client has specified options not available in the current MQTT version.
2020-12-18 10:40:15 +01:00
Norman Maurer
7fb62a93b8
Ensure we only register native methods once (#10876)
Motivation:

We need to ensure we only register native methods once as otherwise we may end up in an "invalid" state. The problem here was that before it was basically the responsibility the user of transport-native-unix-common to register the methods. This is error prone as there may be multiple users of these on the classpath at the same time.

Modifications:

- Provide a way to init native lib without register the native methods of the provided classes. This is needed to be able to re-use functionality which is exposed to our internal native code
- Use flatten plugin to correctly resolve classifier and so have the correct dependency
- Call Unix.* method to ensure we register the methods correctly once
- Include native lib as well in the native jars of unix-common

Result:

Be able to have multiple artifacts of the classpath that depends on the unix-common. Related to https://github.com/netty/netty-incubator-transport-io_uring/issues/15
2020-12-18 10:37:49 +01:00
Chris Vest
5c1613df81
Simplify InternalThreadLocalMap (#10872)
Motivation:
I did not see any tangible advantage to the padding.
The only other field that was guarded was a rarely changed object reference to a BitSet.
Without the padding, there is also no longer any use of the inheritance hierarchy.
The padding was also using `long`, which would not necessarily prevent the JVM from fitting the aforementioned object reference in an alignment gap.

Modification:
Move all the fields into the InternalThreadLocalMap and deprecate the stuff we'd like to remove.

Result:
Simpler code.
This resolves the discussion in https://github.com/netty/netty/issues/9284
2020-12-18 08:06:53 +01:00
root
a9ec3d86f6 [maven-release-plugin] prepare for next development iteration 2020-12-17 06:11:39 +00:00
root
1188d8320e [maven-release-plugin] prepare release netty-4.1.56.Final 2020-12-17 06:11:18 +00:00
Norman Maurer
882c111568
Ensure IovArray is usuable without sun.misc.Unsafe (#10870)
Motivation:

https://github.com/netty/netty/pull/10814 did fix a bug where we did try to call memoryAddress() even tho this is not supported. Unfortunally this fix was only applied for one method and so we missed another method which then could throw an exception when we called memoryAddress()

Modifications:

- Also fix the memoryAddress(offset) method.
_ Adjust unit test to also test this.

Result:

Fixes https://github.com/netty/netty/issues/10813 completely.
2020-12-16 20:46:19 +01:00
Norman Maurer
8ea0d8f41a
Don't catch Throwable in InternalLoggerFactory (#10866)
Motivation:

We shouldnt catch Throwable in InternalLoggerFactory as this may hide OOME etc.

Modifications:

Only catch LinkageError and Exception

Result:

Fixes https://github.com/netty/netty/issues/10857
2020-12-16 08:45:36 +01:00
Norman Maurer
e8b5aeddef
Fix Bundle-NativeCode in MANIFEST file (#10867)
Motivation:

We used , but should have used ;

Modifications:

Replace , by ;

Result:

Correct manifest
2020-12-15 18:08:14 +01:00
Norman Maurer
286176be65
Fix Bundle-NativeCode in MANIFEST file on macOS (#10865)
Motivation:

We did include some extra " in the Bundle-NativeCode line on macOS.

Modifications:

Remove "

Result:

Fixes https://github.com/netty/netty/issues/10856
2020-12-15 12:47:51 +01:00
terrarier2111
9560241e1b
Removed redundant local variable (#10858)
Motivation:

Found a redundant local variable.

Modification:

Removed the local variable.

Result:
Minor performance improvement.
2020-12-15 08:06:37 +01:00
Norman Maurer
dd6ef486f4
Workaround possible JDK bug in SSLEngineImpl when using TLSv1.3 that lead to multiple notifications (#10860)
Motivation:

When using the JDKs SSLEngineImpl with TLSv1.3 it sometimes returns HandshakeResult.FINISHED multiple times. This can lead to have SslHandshakeCompletionEvents to be fired multiple times.

Modifications:

- Keep track of if we notified before and if so not do so again if we use TLSv1.3
- Add unit test

Result:

Consistent usage of events
2020-12-15 07:53:15 +01:00
James Kleeh
c0674cff29
Fix infinite loop (#10855)
Motivation:

To fix the infinite loop parsing a multipart body.

Modifications:

Modified the loop to use the correct variable.

Result:

Multipart bodies will be parsed correctly again.
2020-12-11 14:15:23 +01:00
Idel Pivnitskiy
b4479353e2
Improve logging in DnsServerAddressStreamProviders (#10848)
Motivation:

`DnsServerAddressStreamProviders` tries to load `MacOSDnsServerAddressStreamProvider`
on macOS. However, it doesn't warn users when `MacOSDnsServerAddressStreamProvider`
is not awailable, which may cause incorrect results for DNS resolutions.

Modifications:

- Log at warn level if `MacOSDnsServerAddressStreamProvider` is not found on macOS;
- Log at debug level when `MacOSDnsServerAddressStreamProvider` is loaded and available;

Result:

macOS users are notified when `MacOSDnsServerAddressStreamProvider` is not available.
2020-12-10 14:19:39 +01:00
terrarier2111
0537170ab5
Fixed a comment in UnpooledDirectByteBuf (#10854)
Motivation:

Found an invalid comment in UnpooledDirectByteBuf.

Modification:

Fixed a comment in UnpooledDirectByteBuf.

Result:

Fixed a comment in UnpooledDirectByteBuf.
2020-12-10 10:31:15 +01:00
terrarier2111
aba3cc49db
Fixed JavaDoc of LimitingByteInput (#10852)
Motivation:

I found a typo in LimitingByteInput.

Modification:

Fixed the typo in LimitingByteInput

Result:

No typo anymore in LimitingByteInput.
2020-12-10 10:30:25 +01:00