Commit Graph

10182 Commits

Author SHA1 Message Date
Zxy
d0bcf44445
Fix memory release failure when "maxNumElems == 1" of PoolSubpage (#10988)
Motivation:

when customer need large of 'byteBuf.capacity' in [7168, 8192], the size of 'chunk.subpages' may be inflated when large of byteBuf be released, not consistent with other 'byteBuf.capacity'

Modification:

when maxNumElems == 1 need consider remove from pool

Result:

Fixes #10896. 

Co-authored-by: zxingy <zxingy@servyou.com.cn>
2021-02-05 14:53:58 +01:00
dependabot[bot]
5460ae7d54
Bump ant from 1.9.15 to 1.10.9 (#10990)
Bumps ant from 1.9.15 to 1.10.9.

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-02-04 18:21:13 +01:00
Norman Maurer
6ce15414ff
Revert HttpPostMultipartRequestDecoder and HttpPostStandardRequestDecoder to e5951d46fc (#10989)
Motivation:

The changes introduced in 1c230405fd did cause various issues while the fix itself is not considered critical. For now it is considered the best to just rollback and investigate more.

Modifications:

- Revert changes done in 1c230405fd (and later) for
the post decoders.
- Ensure we give memory back to the system as soon as possible in a safe manner

Result:

Fixes https://github.com/netty/netty/issues/10973
2021-02-03 20:40:29 +01:00
Norman Maurer
ab8b8ae81b
Introduce SslContextOption which can be used for "optional" features … (#10981)
Motivation:

Some of the features we want to support can only be supported by some of the SslContext implementations. We should allow to configure these in a consistent way the same way as we do it with Channel / ChannelOption

Modifications:

- Add SslContextOption and add builder methods that take these
- Add OpenSslContextOption and define two options there which are specific to openssl

Result:

More flexible configuration and implementation of SslContext
2021-02-02 08:29:21 +01:00
Norman Maurer
a9a3e2cef1
Clarify who is responsible closing the InputStream once SslContext w… (#10983)
Motivation:

It was not 100% clear who is responsible calling close() on the InputStream.

Modifications:

Clarify javadocs.

Result:

Related to https://github.com/netty/netty/issues/10974

Co-authored-by: Chris Vest <christianvest_hansen@apple.com>
2021-02-01 11:50:26 +01:00
Norman Maurer
3353bc1152
Revert "Enable SSL_MODE_ENABLE_FALSE_START if jdkCompatibilityMode is false (#10407)" (#10980)
Motivation:

TLS_FALSE_START slightly changes the "flow" during handshake which may cause suprises for the end-user. We should better disable it by default again and later add a way to enable it for the user.

Modification:

This reverts commit 514d349e1f.

Result:

Restore "old flow" during TLS handshakes.
2021-01-29 19:55:56 +01:00
Norman Maurer
5c522916f7
Fix possible SEGV when using native dns resolver on macos (#10971)
Motivation:

We should add some more NULL checks to ensure we not SEGV in some cases

Modifications:

Add more NULL checks

Result:

More robust implementation
2021-01-28 15:15:21 +01:00
Norman Maurer
df10dfbb49
Correctly filter out TLSv1.3 ciphers if TLSv1.3 is not enabled (#10919)
Motivation:

We didnt correctly filter out TLSv1.3 ciphers when TLSv1.3 is not enabled.

Modifications:

- Filter out ciphers that are not supported due the selected TLS version
- Add unit test

Result:

Fixes https://github.com/netty/netty/issues/10911

Co-authored-by: Bryce Anderson <banderson@twitter.com>
2021-01-28 11:00:51 +01:00
Norman Maurer
c932e1d652
Ignore SocketShutdownOutputBySelfTest.testWriteAfterShutdownOutputNoWritabilityChange (#10970)
Motivation:

The testWriteAfterShutdownOutputNoWritabilityChange() failed a few times on the CI randomly. Let's skip it for now while we investigate and see if there is anything we can do to make the test less flaky on the CI.

Modifications:

Add @Ignore on the testWriteAfterShutdownOutputNoWritabilityChange method

Result:

Less flaky CI
2021-01-28 07:27:06 +01:00
Norman Maurer
147ccb6213
Use custom script to check for build failures (#10968)
Motivation:

It turns out we can't use the action to check for build failures as it can't be used when a PR is done from a fork. Let's just use our simple script.

Modifications:

- Replace action with custom script

Result:

Builds for PRs that are done via forks work again.
2021-01-27 12:02:24 +01:00
Norman Maurer
b743e3ab42
Update dependency declaration in all pom.xml (#10967)
Motivation:

We did have the architecture hardcoded in the dependency which is not correct as this will let the build fail on Applie Silicion (m1). Also we did miss some dependencies on other BSDs

Modifications:

- Fix classifier
- Add missing dependencies

Result:

Be able to build on Apple Silicon (m1)
2021-01-27 10:25:08 +01:00
Norman Maurer
1e87c711b4
Ensure we create a copy of the attributes and options when bootstrap a channel (#10965)
Motivation:

We need to ensure we copy the attributes and options when bootstrap the channel as otherwise we may change the underlying Entry.
This is similar to what was reported in https://github.com/netty/netty-incubator-codec-quic/issues/152.

Modifications:

- Do a copy and re-use methods
- Add unit tests

Result:

Don't affect attributes / options of channels that are already bootstrapped
2021-01-26 19:26:20 +01:00
Norman Maurer
305cb1719a
Use action to report unit test errors (#10966)
Motivation:

To make it easier to understand why the build fails lets use an action that will report which unit test failed

Modifications:

- Replace custom script with action-surefire-report

Result:

Easier to understand test failures
2021-01-26 15:39:44 +01:00
Carl Mastrangelo
16f2a5e49f
Don't use Fixed ports for UDP test (#10961)
Motivation:
If the given port is already bound, the PcapWriteHandlerTest will sometimes fail.

Modification:
Use a dynamic port using `0`, which is more reliable

Result:
Less Flaky
2021-01-26 08:30:58 +01:00
Stephane Landelle
ccd01934f5
Merge WebSocket extensions, close #10792 (#10956)
Motivation:

We currently append extensions to the user defined "sec-websocket-extensions" headers. This can cause duplicated entries.

Modifications:

* Replace existing `WebSocketExtensionUtil#appendExtension` private helper with a new `computeMergeExtensionsHeaderValue`. User defined parameters have higher precedence.
* Add tests (existing method wasn't tested)
* Reuse code for both client and server side (code was duplicated).

Result:

No more duplicated entries when user defined extensions overlap with the ones Netty generated.
2021-01-22 08:15:56 +01:00
Duke Bartholomew
f54bf7aeb0
Allow to use int while build MqttMessageIdVariableHeader (#10930)
Motivation:

Exception occurs in the MQTT message builder class (`io.netty.handler.codec.mqtt.MqttMessageBuilders`) when trying to create a message with packetId > 32767

Modification:

-add method that takes int
- deprecate old methods that take short. 

Result:


Fixes #10929 .
2021-01-21 14:40:45 +01:00
Norman Maurer
60780c647e Revert "Merge WebSocket extensions, close #10792 (#10951)"
This reverts commit 4fbbcf8702.
2021-01-21 14:22:59 +01:00
Stephane Landelle
4fbbcf8702
Merge WebSocket extensions, close #10792 (#10951)
Motivation:

We currently append extensions to the user defined "sec-websocket-extensions" headers. This can cause duplicated entries.

Modifications:

* Replace existing `WebSocketExtensionUtil#appendExtension` private helper with a new `computeMergeExtensionsHeaderValue`. User defined parameters have higher precedence.
* Add tests (existing method wasn't tested)
* Reuse code for both client and server side (code was duplicated).

Result:

No more duplicated entries when user defined extensions overlap with the ones Netty generated.
2021-01-21 13:58:52 +01:00
Norman Maurer
41e79835f2
Move OpenSsl*X509Certificate to util package and rename it (#10955)
Motivation:

Creating certificates from a byte[] while lazy parse it is general useful and is also needed by https://github.com/netty/netty-incubator-codec-quic/pull/141

Modifications:

Move classes, rename these and make them public

Result:

Be able to reuse code
2021-01-21 11:12:54 +01:00
Norman Maurer
42ba433576
Override ALPN methods on ReferenceCountedOpenSslEngine (#10954)
Motivation:

We should override the get*ApplicationProtocol() methods in ReferenceCountedOpenSslEngine to make it easier for users to obtain the selected application protocol

Modifications:

Add missing overrides

Result:

Easier for the user to get the selected application protocol (if any)
2021-01-21 10:55:16 +01:00
Norman Maurer
ae3eebf11b
Mark some methods as protected to make it easier to write own SslContext implementations (#10953)
Motivation:

We should expose some methods as protected to make it easier to write custom SslContext implementations.
This will be reused by the code for https://github.com/netty/netty-incubator-codec-quic/issues/97

Modifications:

- Add protected to some static methods which are useful for sub-classes
- Remove some unused methods
- Move *Wrapper classes to util package and make these public

Result:

Easier to write custom SslContext implementations
2021-01-21 10:41:33 +01:00
Norman Maurer
8cae1ed959
Ignore priority frames for non existing streams and so prevent a NPE (#10943)
Motivation:

https://github.com/netty/netty/pull/10765 added support for push promise and priority frames when using the Http2FrameCodec. Unfortunally it didnt correctly guard against the possibility to receive a priority frame for an non-existing stream, which resulted in a NPE

Modifications:

- Ignore priority frame for non existing stream
- Correctly implement equals / hashcode for DefaultHttp2PriorityFrame
- Add unit tests

Result:

Fixes https://github.com/netty/netty/issues/10941
2021-01-18 14:11:07 +01:00
Norman Maurer
59b3831bbc
Correctly handle fragmentation in JdkZlibDecoder (#10948)
Motivation:

We had multiple bugs in JdkZlibDecoder which could lead to decoding errors when the data was received in a fragmentated manner.

Modifications:

- Correctly handle skipping of comments
- Correctly handle footer / header decoding
- Add unit test that verifies the correct handling of fragmentation

Result:

Fixes https://github.com/netty/netty/issues/10875
2021-01-18 14:02:37 +01:00
Norman Maurer
4d43f16bb9
Make native loading logging less confusing (#10944)
Motivation:

We produced a lot of noise during loading native libraries as we always included the stacktrace if we could not load by one mechanism. We should better just not include the stacktrace in the debugging logging if one mechanism fails. We will log all the stacks anyway when all of the mechanisms fail.

Modifications:

Make logging less aggressive

Result:

Less confusing behaviour for the end-user
2021-01-16 09:56:44 +01:00
Norman Maurer
94646ac546
Use GracefulShutdown when stream space is exhausted (#10946)
Motivation:

We should use GracefulShutdown when we try to create a stream and fail it because the stream space is exhausted as we may still want to process the active streams.

Modifications:

- Use graceful shutdown
- Add unit test

Result:

More graceful handling of stream creation failure due stream space exhaustation
2021-01-16 09:55:26 +01:00
Norman Maurer
342f52e813
Let script fail if one command fail (#10945)
Motivation:

We should use `set -e` to ensure we fail the script if one command fails.

Modifications:

Add set -e to script

Result:

Fail fast
2021-01-15 16:47:49 +01:00
Francesco Nigro
9a02832fdb
Implement SWAR indexOf byte search (#10737)
Motivation:

Faster indexOf

Modification:

Create generic SWAR indexOf that any ByteBuf implementation can use

Result:

Fixes #10731
2021-01-15 15:09:27 +01:00
Chris Vest
bba0017d6e
Update the javadocs on FastThreadLocal (#10942)
Motivation: The writing was unclear.
Modification: Clarified the documentation.
Result: No more ambiguity about what FTL.remove() does.

Fixes #10914
2021-01-15 12:03:26 +01:00
Violeta Georgieva
5b699b722c
Allow blocking calls in UnixResolverDnsServerAddressStreamProvider#parse (#10935)
Motivation:

Internally UnixResolverDnsServerAddressStreamProvider#parse calls FileInputStream.read(...)
when parsing the etcResolverFiles.
This will cause the error below when BlockHound is enabled
reactor.blockhound.BlockingOperationError: Blocking call! java.io.FileInputStream#readBytes
     	at java.io.FileInputStream.readBytes(FileInputStream.java)
     	at java.io.FileInputStream.read(FileInputStream.java:255)

Modifications:

- Add whitelist entry to BlockHound configuration
- Add test

Result:

Fixes #10925
2021-01-14 18:27:12 +01:00
Norman Maurer
c509a477ef
Ensure we also build dependent modules for deployments (#10936)
Motivation:

We need to ensure we also build the modules we depend on as otherwise we may not be able to resolve the dependencies correctly

Modifications:

- Add -am when calling maven

Result:

Deployment works all the time
2021-01-14 18:08:44 +01:00
Norman Maurer
5b41f3d25b
Ensure native methods for unix-native-common are only registered once. (#10932)
Motiviation:

We need to ensure we only register the methods for unix-native-common once as otherwise it may have strange side-effects.

Modifications:

- Add extra method that should be called to signal that we need to register the methods. The registration will only happen once.
- Adjust code to make use of it.

Result:

No more problems due incorrect registration of these methods.
2021-01-14 17:52:04 +01:00
Norman Maurer
775b36ee66
Verify we can load native modules and add job that verifies on aarch64 as well (#10933)
Motivation:

As shown in the past we need to verify we actually can load the native as otherwise we may introduce regressions.

Modifications:

- Add new maven module which tests loading of native modules
- Add job that will also test loading on aarch64

Result:

Less likely to introduce regressions related to loading native code in the future
2021-01-14 17:11:19 +01:00
root
a137ce2042 [maven-release-plugin] prepare for next development iteration 2021-01-13 10:28:54 +00:00
root
10b03e65f1 [maven-release-plugin] prepare release netty-4.1.58.Final 2021-01-13 10:27:17 +00:00
Norman Maurer
d58ce7a151
Revert "Ensure we only register native methods once (#10876)" (#10928)
Motivation:

This reverts commit 7fb62a93b8 as it broke native loading in some cases due maven dependencies.

Modification:

Revert the commit.

Result:

Native loading works again
2021-01-13 11:07:39 +01:00
root
c6b894d03d [maven-release-plugin] prepare for next development iteration 2021-01-12 11:10:44 +00:00
root
b016568e21 [maven-release-plugin] prepare release netty-4.1.57.Final 2021-01-12 11:10:20 +00:00
Norman Maurer
e01f80ec0a
Update to netty-tcnative 2.0.36.Final (#10923)
Motivation:

Update to latest netty-tcnative release

Modifications:

Update version

Result:

Use latest version
2021-01-12 12:05:57 +01:00
Norman Maurer
cbebad8417
We need to ensure we always drain the error stack when a callback throws (#10920)
Motivation:

We need to ensure we always drain the error stack when a callback throws as otherwise we may pick up the error on a different SSL instance which uses the same thread.

Modifications:

- Correctly drain the error stack if native method throws
- Add a unit test which failed before the change

Result:

Always drain the error stack
2021-01-11 20:56:52 +01:00
Aki
3f04385919
Fix url in README.md (#10915)
Motivation:

We had a typo in the markdown

Modification:

Fix typo

Result:

Corect markdown
2021-01-11 07:48:58 +01:00
Aayush Atharva
0dc246eb12
Update BouncyCastle Libraries (#10906)
Motivation:
New versions of `Bouncy Castle` libraries are out and we should upgrade to them.

Modification:
Upgraded all `Bouncy Castle` libraries to the latest version.

Result:
The latest versions of `Bouncy Castle` libraries.

Fixes #10905.
2021-01-01 19:31:44 +01:00
YoungTae Seok
3043ebb21c
Ensure Checkstyle suppression for dependency-reduced-pom.xml on Windows (#10899)
Motivation:

`mvn package` on Windows fails if there are some `dependency-reduced-pom.xml` files generated by the previous build.
`mvn clean package` does not help because it does not remove those files at the clean phase.

Modifications:

Use the correct file pattern for the suppression filters.

Result:

Following `mvn package` runs well on Windows.
2021-01-01 19:30:13 +01:00
Norman Maurer
249392fe12
Update action-docker-layer-caching (#10900)
Motivation:

We are three releases behind.

Modifications:

Update to latest version

Result:

Use up-to-date action-docker-layer-caching version
2020-12-28 12:54:12 +01:00
Aayush Atharva
08681a817d
Add PushPromise and Priority Frame support in Http2FrameCodec (#10765)
Motivation:
Right now, we don't have to handle Push Promise Read in `Http2FrameCodec`. Push Promise is one of the key features of HTTP/2 and we should support it in our `Http2FrameCodec`.

Modification:
Added `Http2PushPromiseFrame` and `Http2PushPromiseFrame` to handle Push Promise and Promise Frame.

Result:
Fixes #10748
2020-12-27 14:30:24 +01:00
Norman Maurer
c5ffa293ac
Update README
Motivation:

We can add some status badge and also should clarify requirements.

Modifictations:

- Add status badge
- Clarify requirements

Result:

Cleanup docs
2020-12-27 10:14:08 +01:00
Norman Maurer
3a058d2172 We run all combinations when validate the PR, let's just use one type for normal push
Motivation:

Let us just only use one build config when building the 4.1 branch.

Modifications:

As we already do a full validation when doing the PR builds we can just only use one build config for pushes to the "main" branches

Result:

Faster build times
2020-12-25 10:20:12 +01:00
Norman Maurer
3bfe9d7242 Dont use cron for PRs
Motivation:

It doesnt make sense to use cron for PRs

Modifications:

Remove cron config

Result:

Cleanup
2020-12-24 17:13:44 +01:00
Norman Maurer
42145902a7
Correctly deploy artifacts that are build on different archs (#10893)
Motivation:

We need to take special care when deploying snapshots as we need to generate the jars in multiple steps

Modifications:

- Use the nexus staging pluging to stage jars locally in multiple steps
- Add extra job that will merge these staged jars and deploy these

Result:

Fixes https://github.com/netty/netty/issues/10887
2020-12-24 12:00:08 +01:00
Eric Anderson
4f6e62d50c
Avoid unsynchronized access to scheduledTaskQueue in GlobalEventExecutor (#10890)
Motivation:

A race detector discovered a data race in GlobalEventExecutor present in
netty 4.1.51.Final:

```
  Write of size 4 at 0x0000cea08774 by thread T103:
    #0 io.netty.util.internal.DefaultPriorityQueue.poll()Lio/netty/util/internal/PriorityQueueNode; DefaultPriorityQueue.java:113
    #1 io.netty.util.internal.DefaultPriorityQueue.poll()Ljava/lang/Object; DefaultPriorityQueue.java:31
    #2 java.util.AbstractQueue.remove()Ljava/lang/Object; AbstractQueue.java:113
    #3 io.netty.util.concurrent.AbstractScheduledEventExecutor.pollScheduledTask(J)Ljava/lang/Runnable; AbstractScheduledEventExecutor.java:133
    #4 io.netty.util.concurrent.GlobalEventExecutor.fetchFromScheduledTaskQueue()V GlobalEventExecutor.java:119
    #5 io.netty.util.concurrent.GlobalEventExecutor.takeTask()Ljava/lang/Runnable; GlobalEventExecutor.java:106
    #6 io.netty.util.concurrent.GlobalEventExecutor$TaskRunner.run()V GlobalEventExecutor.java:240
    #7 io.netty.util.internal.ThreadExecutorMap$2.run()V ThreadExecutorMap.java:74
    #8 io.netty.util.concurrent.FastThreadLocalRunnable.run()V FastThreadLocalRunnable.java:30
    #9 java.lang.Thread.run()V Thread.java:835
    #10 (Generated Stub) <null>

  Previous read of size 4 at 0x0000cea08774 by thread T110:
    #0 io.netty.util.internal.DefaultPriorityQueue.size()I DefaultPriorityQueue.java:46
    #1 io.netty.util.concurrent.GlobalEventExecutor$TaskRunner.run()V GlobalEventExecutor.java:263
    #2 io.netty.util.internal.ThreadExecutorMap$2.run()V ThreadExecutorMap.java:74
    #3 io.netty.util.concurrent.FastThreadLocalRunnable.run()V FastThreadLocalRunnable.java:30
    #4 java.lang.Thread.run()V Thread.java:835
    #5 (Generated Stub) <null>
```

The race is legit, but benign. To trigger it requires a TaskRunner to
begin exiting and set 'started' to false, more work to be scheduled
which starts a new TaskRunner, that work then needs to schedule
additional work which modifies 'scheduledTaskQueue', and then the
original TaskRunner checks 'scheduledTaskQueue'. But there is no danger
to this race as it can only produce a false negative in the condition
which causes the code to CAS 'started' which is thread-safe.

Modifications:

Delete problematic references to scheduledTaskQueue. The only way
scheduledTaskQueue could be modified since the last check is if another
TaskRunner is running, in which case the current TaskRunner doesn't
care.

Result:

Data-race free code, and a bit less code to boot.
2020-12-24 11:37:42 +01:00
Norman Maurer
78fcd5cf8f
Use matrix for job configurations (#10889)
Motivation:

We can use the matrix feature to define our jobs. This reduces a lot of config

Modification:

Use job matrix

Result:

Easier to maintain
2020-12-23 20:44:57 +01:00