Commit Graph

8025 Commits

Author SHA1 Message Date
Norman Maurer
461f9a1212 Allow to obtain informations of used direct and heap memory for ByteBufAllocator implementations
Motivation:

Often its useful for the user to be able to get some stats about the memory allocated via an allocator.

Modifications:

- Allow to obtain the used heap and direct memory for an allocator
- Add test case

Result:

Fixes [#6341]
2017-03-01 18:53:43 +01:00
Scott Mitchell
93f5f62a20 Cleanup EPOLL native exceptions
Motivation:
Exceptions generated from transport-native-epoll may include duplicate error string description or inconsistent usage of the method name in the string description.

Modifications:
- Ensure the method name from static exceptions and dynamic exceptions is of the same format
- Remove duplicate string rational from the exception messages

Result:
More consistent error messages with no duplicate error description.
2017-03-01 05:42:48 -08:00
Johno Crawford
df6796153b Add a 'bill of materials' project for Maven users
Motivation:

Projects may import multiple libraries which use different versions of Netty.

Modifications:

Add 'netty-bom' meta-project that contains the other projects in a dependencyManagement section.

Result:

Developers can import the BOM to enforce specific version of Netty.
2017-03-01 10:50:57 +01:00
Dmitriy Dumanskiy
d1b0225724 Improvement : WebSocketServerProtocolHandshakeHandler.getWebSocketLocation now applies concat optimization and WebSocket path check moved to separated method.
Motivation:

Make code easier to read, make WebSocketServerProtocolHandshakeHandler.getWebSocketLocation method faster.

Modification:

WebSocket path check moved to separate method. Get header operation moved out from concat operation.

Result:

WebSocketServerProtocolHandshakeHandler.getWebSocketLocation is faster as OptimizeStringConcat could be applied. Code easier to read.
2017-03-01 10:47:20 +01:00
Stephane Landelle
79e24d1a17 Add javadoc warning on SslContext#newHandler client-side
Motivation:

SslContext#newHandler currently creates underlying SSLEngine without
enabling HTTPS endpointIdentificationAlgorithm. This behavior in
unsecured when used on the client side.

We can’t harden the behavior for now, as it would break existing
behavior, for example tests using self signed certificates.

Proper hardening will happen in a future major version when we can
break behavior.

Modifications:

Add javadoc warnings with code snippets.

Result:

Existing unsafe behavior and workaround documented.
2017-03-01 10:44:44 +01:00
Norman Maurer
d675febf07 Optimize / Cleanup QueryStringDecoder / QueryStringEncoder
Motivation:

QueryStringDecoder and QueryStringEncoder contained some code that could either cleaned-up or optimized.

Modifications:

- Fix typos in exception messages and javadocs
- Precompile Pattern
- Make use of StringUtil.EMPTY_STRING

Result:

Faster and cleaner code.
2017-03-01 06:46:54 +01:00
Norman Maurer
ca2c349a4a Correctly have exceptions thrown from decode(...) method be wrapped with DecodingException
Motivation:

Normally if a decoder produces an exception its wrapped with DecodingException. This is not the cause for NotSslRecordException in SslHandler and SniHandler.

Modifications:

Just throw the NotSslRecordException exception for decode(...) and so ensure its correctly wrapped in a DecodingException before its passed through the pipeline.

Result:

Consist behavior.
2017-03-01 06:41:30 +01:00
Norman Maurer
a7fe6c0153 Metrics exposed by PooledByteBufAllocator needs to be correctly synchronized
Motivation:

As we may access the metrics exposed of PooledByteBufAllocator from another thread then the allocations happen we need to ensure we synchronize on the PoolArena to ensure correct visibility.

Modifications:

Synchronize on the PoolArena to ensure correct visibility.

Result:

Fix multi-thread issues on the metrics
2017-03-01 06:26:08 +01:00
Norman Maurer
4734ef61a5 Mark transport udt as deprecated as barchart-udt is not maintained anymore
Motivation:

barchart-udt is not maintained anymore so there is not way for us to get fixes for udt. Because of this we should mark the transport as deprecated.

Modifications:

Deprecate all udt classes.

Result:

transport udt is deprecated and so the user knows it will be removed in the future.
2017-03-01 06:20:39 +01:00
Scott Mitchell
70bcb40855 OpenSslEngine may use networkBIO after calling shutdown
Motivation:
To ensure that all bytes queued in OpenSSL/tcnative internal buffers we invoke SSL_shutdown again to stimulate OpenSSL to write any pending bytes. If this call fails we may call SSL_free and the associated shutdown method to free resources. At this time we may attempt to use the networkBIO which has already been freed and get a NPE.

Modifications:
- Don't call bioLengthByteBuffer(networkBIO) if we have called shutdown() in ReferenceCountedOpenSslEngine

Result:
Fixes https://github.com/netty/netty/issues/6466
2017-02-28 12:01:12 -08:00
Norman Maurer
90a61046c7 Add benchmarks for UnpooledUnsafeNoCleanerDirectByteBuf vs UnpooledUnsafeDirectByteBuf
Motivation:

Issue [#6349] brought up the idea to not use UnpooledUnsafeNoCleanerDirectByteBuf by default. To decide what to do a benchmark is needed.

Modifications:

Add benchmarks for UnpooledUnsafeNoCleanerDirectByteBuf vs UnpooledUnsafeDirectB
yteBuf

Result:

Better idea about impact of using UnpooledUnsafeNoCleanerDirectByteBuf.
2017-02-27 20:04:09 +01:00
Nikolay Fedorovskikh
c0cd3db5b1 Fix class references of its subclass issue
Motivation:

Realization of `AbstractTrafficShapingHandler.userDefinedWritabilityIndex()` has references to subclasses.
In addition, one of the subclasses overriding it, but the other does not.

Modifications:

Add overriding to the second subclass. Remove references to subclasses from parent class.

Result:

More consistent and clean code (OOP-stylish).
2017-02-27 07:48:51 +01:00
Norman Maurer
22ccf6c7b1 Fix test-failures introduced 325cc84a2e on Java7
Motivation:

325cc84a2e introduced new tests which uses classes only provided by Java8+. We need to ensure we only try to load classes needed for these when we run the tests on Java8+ so we still can run the testsuite with Java7.

Modifications:

Add extra class which only gets loaded when Java8+ is used and move code there.

Result:

No more class-loader issue when running tests with Java7.
2017-02-24 10:36:30 +01:00
Norman Maurer
d73477c7bd Add benchmarks for SSLEngine implementations
Motivation:

As we provide our own SSLEngine implementation we should have benchmarks to compare it against JDK impl.

Modifications:

Add benchmarks for wrap / unwrap and handshake performance.

Result:

Benchmarks FTW.
2017-02-24 08:02:10 +01:00
Norman Maurer
7aff6b0330 Increase timeouts in Http2ConnectionRoundtripTest
Motivation:

The timeouts used in the Http2ConnectionRoundtripTest seems to be too low when leak-detection is enabled so we sometimes get failed tests due timeout.

Modifications:

Increase timeouts.

Result:

Fixes [#6442].
2017-02-24 07:58:59 +01:00
Norman Maurer
4f78bae2eb DatagramPacketEncoder|Decoder should take into account if wrapped handler is sharable
Motivation:

DatagramPacketEncoder|Decoder should respect if the wrapped handler is sharable or not and depending on that be sharable or not.

Modifications:

- Delegate isSharable() to wrapped handler
- Add test-cases

Result:

Correct behavior
2017-02-23 20:22:34 +01:00
Norman Maurer
325cc84a2e Throw if SSLParameters contains settings that are not supported by ReferenceCountedOpenSslEngine
Motivation:

We not support all SSLParameters settings so we should better throw if a user try to use them.

Modifications:

- Check for unsupported parameters
- Add unit test

Result:

Less surprising behavior.
2017-02-23 20:00:40 +01:00
Norman Maurer
a80d3411ee Move all the microbenchmark code into one directory.
Motivation:

Allmost all our benchmarks are in src/main/java but a few are in src/test/java. We should make it consistent.

Modifications:

Move everything to src/main/java

Result:

Consistent code base.
2017-02-23 19:59:09 +01:00
Norman Maurer
deb90923a2 Ensure PooledByteBuf.initUnpooled(...) correctly set the allocator
Motivation:

Commit 8dda984afe introduced a regression which lead to the situation that the allocator is not set when PooledByteBuf.initUnpooled(...) is called. Thus it was possible that PooledByteBuf.alloc() returns null or the wrong allocator if multiple PooledByteBufAllocator are used in an application.

Modifications:

- Correctly set the allocator
- Add test-case

Result:

Fixes [#6436].
2017-02-23 19:53:17 +01:00
Nikolay Fedorovskikh
943f4ec7ff Make methods 'static' where it missed
Motivation:

Calling a static method is faster then dynamic

Modifications:

Add 'static' keyword for methods where it missed

Result:

A bit faster method calls
2017-02-23 11:01:57 +01:00
Norman Maurer
7d08b4fc35 Remove optional dependency on javassist
Motivation:

We shipped a javassist based implementation for typematching and logged a confusing debug message about missing javassist. We never were able to prove it really gives any perf improvements so we should just remove it.

Modifications:

- Remove javassist dependency and impl
- Fix possible classloader deadlock as reported by intellij

Result:

Less code to maintain and less confusing log message.
2017-02-23 07:54:42 +01:00
Scott Mitchell
77e65fe6bb Base64 reduce byte manipulation operations
Motivation:
Base64#decode4to3 generally calculates an int value where the contents of the decodabet straddle bytes, and then uses a byte shifting or a full byte swapping operation to get the resulting contents. We can directly calculate the contents and avoid any intermediate int values and full byte swap operations. This will reduce the number of operations required during the decode operation.

Modifications:
- remove the intermediate int in the Base64#decond4to3 method.
- manually do the byte shifting since we are already doing bit/byte manipulations here anyways.

Result:
Base64#decode4to3 requires less operations to compute the end result.
2017-02-22 21:32:44 -08:00
Norman Maurer
7feb92959e Improve performance of Base64.decode and encode methods.
Motivation:

The decode and encode method uses getByte(...) and setByte(...) in loops which can be very expensive because of bounds / reference-count checking. Beside this it also slows-down a lot when paranoid leak-detection is enabled as it will track each access.

Modifications:

- Pack bytes into int / short and so reduce operations on the ByteBuf
- Use ByteProcessor to reduce getByte calls.

Result:

Better performance in general. Also when you run the build with -Pleak the handler module will build in 1/4 of the time it took before.
2017-02-22 20:12:12 +01:00
Nikolay Fedorovskikh
0623c6c533 Fix javadoc issues
Motivation:

Invalid javadoc in project

Modifications:

Fix it

Result:

More correct javadoc
2017-02-22 07:31:07 +01:00
Nikolay Fedorovskikh
634a8afa53 Fix some warnings at generics usage
Motivation:

Existing warnings from java compiler

Modifications:

Add/fix type parameters

Result:

Less warnings
2017-02-22 07:29:59 +01:00
Norman Maurer
ce222308a6 Update to netty-tcnative 2.0.0.Beta6
Motivation:

New version of netty-tcnative was released.

Modifications:

Bump up version number

Result:

Using latest netty-tcnative
2017-02-21 15:55:57 +01:00
Norman Maurer
2a72c87673 Move AutobahnTestsuite to extra module
Motivation:
We should move the AutobahnTestsuite to an extra module. This allows easier to run only the testsuite or only the autobahntestsuite

Modifications:

Create a new module (testsuite-autobahn)

Result:

Better project structure.
2017-02-21 10:13:31 +01:00
chhsiao90
e148b53c67 Add unit test for DefaultHttp2FrameWriter
Motivation:

DefaultHttp2FrameWriter contains important logic for how http2
frame encode into binary, currently, netty had no unit test
just for DefaultHttp2FrameWriter.

Modifications:

Write DefaultHttp2FrameWriterTest to test DefaultHttp2FrameWriter

Result:

The coverage of DefaultHttp2FrameWriter is increased, and the
class was more reliable
2017-02-21 09:57:16 +01:00
chhsiao90
b1436e80ef Cleanup DefaultHttp2FrameReader about verifyUnknownFrame
Motivation:

In previous PR about handling unknwon frame in the middle of header
block, I didn't notice and re-use about checking is processing header
. And I added a redundant method for same functionality.
I think that the redundant method would lead to some misleading
situation.

Modifications:

Removed redundant code on DefaultHttp2FrameReader

Result:

The code is more readable
2017-02-20 22:12:20 -08:00
Scott Mitchell
e08a3b1f35 Fix SSLException check for JDK work around missed by 2dffc2f9fb 2017-02-20 21:38:04 -08:00
Scott Mitchell
08e0c612cf HTTP/2 Unit Test LocalChannel Leaks
Motivation:
Some unit HTTP/2 unit tests use LocalChannel. LocalChannel's doClose method will ensure any pending items in the queue will be released, but it may execute a Runnable on the peer's EventLoop to ensure the peer's queue is also cleaned up. The HTTP/2 unit tests close the event loop groups with no wait time so that unit tests will execute quickly, but if the doClose Runnable is in the EventLoop's queue it will not run and thus the items in the queue will not be released.

Modifications:
- Ensure all HTTP/2 unit tests which use LocalChannel wait for both client and server channels to be closed before closing the EventLoop.

Result:
Related to https://github.com/netty/netty/issues/5850.
2017-02-20 13:54:30 -08:00
Scott Mitchell
2dffc2f9fb SSLEngineTest issue introduced by d8e6fbb9c3
Motivation:
d8e6fbb9c3 attempted to account for the JDK not throwing the expected SSLHandshakeException by allowing a SSLException to also pass the test. However in some situations the SSLException will not be the top level exception and the Throwable must be unwrapped to see if the root cause is an SSLException.

Modifications:
- Unwrap exceptions thrown by the JDK's SSLEngine to check for SSLException.

Result:
SSLEngineTest (and derived classes) are more reliable.
2017-02-20 12:19:41 -08:00
Norman Maurer
576baf8e6c Update to autobahntestsuite-maven-plugin 0.1.4 to support Java9
Motivation:

autobahntestsuite-maven-plugin 0.1.4 was released and supports Java9.

Modifications:

Update plugin to be able to run tests on Java9

Result:

Autobahntestsuite can also be run on Java9.
2017-02-20 20:54:14 +01:00
Norman Maurer
c57a1bdb2d Log used native library by netty-tcnative
Motivation:

As netty-tcnative can be build against different native libraries and versions we should log the used one.

Modifications:

Log the used native library after netty-tcnative was loaded.

Result:

Easier to understand what native SSL library was used.
2017-02-20 20:52:22 +01:00
Johno Crawford
007e8969c1 Missing release modules in netty-all project
Motivation:

codec-redis and codec-xml are release modules and should be included in netty-all.

Modifications:

Add codec-redis and codec-xml modules to netty-all pom.

Result:

codec-redis and codec-xml can be used with the netty-all artifact.
2017-02-20 13:44:36 +01:00
Hanson
d0a3877535 Ensure trying to recover from exceptionCaught on the ServerChannel works as expected
Motivation:

When "Too many open files" happens,the URLClassLoader cannot do any classloading because URLClassLoader need a FD  for findClass. Because of this the anonymous inner class that is created to re-enable auto read may cause a problem.

Modification:

Pre-create Runnable that is scheduled and so ensure it is not lazy loaded.

Result:

No more problems when try to recover.
2017-02-19 13:41:11 +01:00
andsel
ad51cda2cd Introduced MqttMessageBuilders to fluently create MQTT messages 2017-02-19 13:39:59 +01:00
Scott Mitchell
d8e6fbb9c3 OpenSslEngine should respect hostname verification
Motivation:
OpenSSL doesn't automatically verify hostnames and requires extract method calls to enable this feature [1]. We should allow this to be configured.

Modifications:
- SSLParamaters#getEndpointIdentificationAlgorithm() should be respected and configured via tcnative interfaces.

Result:
OpenSslEngine respects hostname verification.

[1] https://wiki.openssl.org/index.php/Hostname_validation
2017-02-17 13:21:29 -08:00
Norman Maurer
67be7c5b9f Log why it was not possible to use ByteBuffer.cleaner
Motivation:

We should log why we can not use ByteBuffer.cleaner and so maybe allow the user to fix it.

Modifications:

- Use Unsafe to access the field
- Log the exception when we can not use ByteBuffer.cleaner

Result:

Easier to debug why using cleaner is not possible.
2017-02-17 07:34:34 +01:00
Scott Mitchell
5de4029b43 Checkstyle fix from 56694eb 2017-02-16 22:28:56 -08:00
Scott Mitchell
56694ebc0f Cleanup from fbf0e5f4dd
Motivation:
ThreadLocalInsecureRandom still referenced ThreadLocalRandom directly, but shouldn't.

Modifications:
ThreadLocalInsecureRandom should reference PlatformDependent#threadLocalRandom() in comments

Result:
Less usage of internal.ThreadLocalRandom.
2017-02-16 15:56:23 -08:00
Norman Maurer
fbf0e5f4dd Prefer JDK ThreadLocalRandom implementation over ours.
Motivation:

We have our own ThreadLocalRandom implementation to support older JDKs . That said we should prefer the JDK provided when running on JDK >= 7

Modification:

Using ThreadLocalRandom implementation of the JDK when possible.

Result:

Make use of JDK implementations when possible.
2017-02-16 15:44:00 -08:00
Norman Maurer
38496a23da Update to netty-tcnative 2.0.0.Beta4 2017-02-16 15:42:19 -08:00
Scott Mitchell
795f318c3c Use a single array in SelectedSelectionKeySet
Motivation:
SelectedSelectionKeySet currently uses 2 arrays internally and users are expected to call flip() to access the underlying array and switch the active array. However we do not concurrently use 2 arrays at the same time and we can get away with using a single array if we are careful about when we reset the elements of the array.

Modifications:
- Introduce SelectedSelectionKeySetSelector which wraps a Selector and ensures we reset the underlying SelectedSelectionKeySet data structures before we select
- The loop bounds in NioEventLoop#processSelectedKeysOptimized can be defined more precisely because we know the real size of the underlying array

Result:
Fixes https://github.com/netty/netty/issues/6058
2017-02-16 15:10:37 -08:00
Scott Mitchell
4d7d478a3d Update JCTools to 2.0.1 2017-02-16 15:09:35 -08:00
Norman Maurer
6ac5f35077 Use Unsafe to read ByteBuffer.address field to make it work on Java9 as well.
Motivation:

Java9 does not allow changing access level via reflection by default. This lead to the situation that netty disabled Unsafe completely as ByteBuffer.address could not be read.

Modification:

Use Unsafe to read the address field as this works on all Java versions.

Result:

Again be able to use Unsafe optimisations when using Netty with Java9
2017-02-16 20:40:59 +01:00
Norman Maurer
c5324ea48c Make netty build work on Java9
Motivation:

We missed some stuff in 5728e0eb2c and so the build failed on java9

Modifications:

- Add extra cmdline args when needed
- skip the autobahntestsuite as jython not works with java9
- skip the osgi testsuite as the maven plugin not works with java9

Result:

Build finally passed on java9
2017-02-16 20:26:30 +01:00
Norman Maurer
ddd0e2f184 Enforce java8 for compilation
Motivation:

Commit 591293bfb4 changed the build to need java8 but missed to adjust the enforce rule as well.

Modifications:

Enforce java8+

Result:

Quickly fail when user tries to compile with pre java8
2017-02-16 20:24:54 +01:00
Scott Mitchell
0befcd8431 EpollDatagramChannel avoid getOption
Motivation:
EpollDatagramChannel uses getOption in the isActive method. getOption is backed by a relatively large conditional if/else if block and this conditional checking can be avoided in the epoll transport.

Modifications:
- Add EpollDatagramChannelConfig#getActiveOnOpen and use this in EpollDatagramChannel

Result:
Conditional checking due to getOption is removed from EpollDatagramChannel.
2017-02-16 11:04:37 -08:00
Scott Mitchell
f3dd410142 EPOLLRDHUP prematurely marking input as shutdown
Motivation:
When the EPOLLRDHUP event is received we assume that the read side of the FD is no longer functional and force the input state to be shutdown. However if the channel is still active we should rely upon EPOLLIN and read to indicate there is no more data before we update the shutdown state. If we do not do this we may not read all pending data in the FD if the RecvByteBufAllocator doesn't want to consume it all in a single read operation.

Modifications:
- AbstractEpollChannel#epollRdHupReady() shouldn't force shutdown the input if the channel is active

Result:
All data can be read even if the RecvByteBufAllocator doesn't read it all in the current read loop.
Fixes https://github.com/netty/netty/issues/6303
2017-02-16 08:40:32 -08:00