Commit Graph

756 Commits

Author SHA1 Message Date
Scott Mitchell
24d0f3d45d SslHandlerTest ByteBuf leak
Motivation:
We are now more careful to flush alerts that are generated when errors occur. We should also be more careful in unit tests to release any buffers that may be queued due to potential errors wich result in alerts.

Modifications:
- When SslHandlerTest uses EmbeddedChannel we should always call finishAndReleaseAll

Results:
Fixes https://github.com/netty/netty/issues/6057
2016-12-01 06:51:14 +01:00
Norman Maurer
fffc1ba872 Ensure alert is send when SSLException happens during calling SslHandler.unwrap(...)
Motivation:

When the SslHandler.unwrap(...) (which is called via decode(...)) method did produce an SSLException it was possible that the produced alert was not send to the remote peer. This could lead to staling connections if the remote peer did wait for such an alert and the connection was not closed.

Modifications:

- Ensure we try to flush any pending data when a SSLException is thrown during unwrapping.
- Fix SniHandlerTest to correct test this
- Add explicit new test in SslHandlerTest to verify behaviour with all SslProviders.

Result:

The alert is correctly send to the remote peer in all cases.
2016-11-21 20:55:57 +01:00
Norman Maurer
139af2aac4 Use the correct alert depending on the CertificateException when using OpenSslEngine
Motivation:

We tried to detect the correct alert to use depending on the CertificateException that is thrown by the TrustManager. This not worked all the time as depending on the TrustManager implementation it may also wrap a CertPathValidatorException.

Modification:

- Try to unwrap the CertificateException if needed and detect the right alert via the CertPathValidatorException.
- Add unit to verify

Result:

Send the correct alert depending on the CertificateException when using OpenSslEngine.
2016-11-21 07:51:34 +01:00
Norman Maurer
eff8413d9d Eliminate usage of releaseLater(...) to reduce memory usage during tests
Motiviation:

We used ReferenceCountUtil.releaseLater(...) in our tests which simplifies a bit the releasing of ReferenceCounted objects. The problem with this is that while it simplifies stuff it increase memory usage a lot as memory may not be freed up in a timely manner.

Modifications:

- Deprecate releaseLater(...)
- Remove usage of releaseLater(...) in tests.

Result:

Less memory needed to build netty while running the tests.
2016-11-18 11:21:23 +00:00
nmittler
5a8a365d17 Restructuring SslHandler to support new engines
Motivation:

In preparation for support of Conscrypt, I'm consolidating all of the engine-specific details so that it's easier to add new engine types that affect the behavior of SslHandler.

Modifications:

Added an enum SslEngineType that provides SSL engine-specific details.

Result:

SslHandler is more extensible for other engine types.
2016-11-17 20:13:16 +01:00
Norman Maurer
d80bd8fb4a [#5874] [#5971] Ensure SniHandlerTest.testServerNameParsing not fails with SslProvider.JDK
Motivation:

The SniHandlerTest.testServerNameParsing did fail when SslProvider.JDK was used as it the JDK SSLEngineImpl does not send an alert.

Modifications:

Ensure tests pass with JDK and OPENSSL ssl implementations.

Result:

SniHandlerTest will run with all SslProvider and not fail when SslProvider.JDK is used.
2016-11-16 08:06:13 +01:00
Norman Maurer
595b2676be [#5976] Ensure we only consume as much data on wrap(...) as we can handle.
Motiviation:

We need to ensure we only consume as much da as we can maximal put in one ssl record to not produce a BUFFER_OVERFLOW when calling wrap(...).

Modification:

- Limit the amount of data that we consume based on the maximal plain text size that can be put in one ssl record
- Add testcase to verify the fix
- Tighten up testcases to ensure the amount of produced and consumed data in SslEngineResult matches the buffers. If not the tests will fail now.

Result:

Correct and conform behavior of OpenSslEngine.wrap(...) and better test coverage during handshaking in general.
2016-11-15 09:39:38 +01:00
Scott Mitchell
2ec06e2c34 ByteBuf Input Stream Reference Count Ownership
Motivation:
Netty provides a adaptor from ByteBuf to Java's InputStream interface. The JDK Stream interfaces have an explicit lifetime because they implement the Closable interface. This lifetime may be differnt than the ByteBuf which is wrapped, and controlled by the interface which accepts the JDK Stream. However Netty's ByteBufInputStream currently does not take reference count ownership of the underlying ByteBuf. There may be no way for existing classes which only accept the InputStream interface to communicate when they are done with the stream, other than calling close(). This means that when the stream is closed it may be appropriate to release the underlying ByteBuf, as the ownership of the underlying ByteBuf resource may be transferred to the Java Stream.

Motivation:
- ByteBufInputStream.close() supports taking reference count ownership of the underyling ByteBuf

Result:
ByteBufInputStream can assume reference count ownership so the underlying ByteBuf can be cleaned up when the stream is closed.
2016-11-14 16:38:49 -08:00
Norman Maurer
f87d0698f3 Let OpenSslEngine.wrap(...) / OpenSslEngine.unwrap(...) behave like stated in the javadocs.
Motivation:

OpenSslEngine.wrap(...) and OpenSslEngie.unwrap(...) may consume bytes even if an BUFFER_OVERFLOW / BUFFER_UNDERFLOW is detected. This is not correct as it should only consume bytes if it can process them without storing data between unwrap(...) / wrap(...) calls. Beside this it also should only process one record at a time.

Modifications:

- Correctly detect BUFFER_OVERFLOW / BUFFER_UNDERFLOW and only consume bytes if non of them is detected.
- Only process one record per call.

Result:

OpenSslEngine behaves like stated in the javadocs of SSLEngine.
2016-11-11 20:22:07 +01:00
Norman Maurer
de4692e6e0 Not use InternalThreadLocalMap where access may be done from outside the EventLoop.
Motivation:

We should not use the InternalThreadLocalMap where access may be done from outside the EventLoop as this may create a lot of memory usage while not be reused anyway.

Modifications:

Not use InternalThreadLocalMap in places where the code-path will likely be executed from outside the EventLoop.

Result:

Less memory bloat.
2016-11-10 14:42:26 +01:00
Evgeny Slutsky
a7662db470 OpenSslSession#initPeerCerts creates too long almost empty arrays.
Motivation:

https://github.com/netty/netty/issues/5945

Modifications:

Refactored initialization of arrays. Fixed arrays length

Result:

Cert arrays have proper length. Testing added
2016-11-03 12:11:01 +01:00
Norman Maurer
50a11d964d [maven-release-plugin] prepare for next development iteration 2016-10-14 14:32:28 +02:00
Norman Maurer
73306e017d [maven-release-plugin] prepare release netty-4.0.42.Final 2016-10-14 14:31:27 +02:00
Trustin Lee
bcea25cc52 Convert X509TrustManager into X509ExtendedTrustManager for Java 7+
Motivation:

Since Java 7, X509TrustManager implementation is wrapped by a JDK class
called AbstractTrustManagerWrapper, which performs an additional
certificate validation for Socket or SSLEngine-backed connections.

This makes the TrustManager implementations provided by
InsecureTrustManagerFactory and FingerprintTrustManagerFactory not
insecure enough, where their certificate validation fails even when it
should pass.

Modifications:

- Add X509TrustManagerWrapper which adapts an X509TrustManager into an
  X509ExtendedTrustManager
- Make SimpleTrustManagerFactory wrap an X509TrustManager with
  X509TrustManagerWrapper is the provided TrustManager does not extend
  X509ExtendedTrustManager

Result:

- InsecureTrustManagerFactory and FingerprintTrustManagerFactory are now
  insecure as expected.
- Fixes #5910
2016-10-12 13:46:27 +02:00
Scott Mitchell
fd30c22665 Update Default Cipher List
Motivation:
Our default cipher list has not been updated in a while. We current support some older ciphers not commonly in use and we don't support some newer ciphers which are more commonly used.

Modifications:
- Update the default list of ciphers for JDK and OpenSSL.

Result:
Default cipher list is more likely to connect to peers.
Fixes https://github.com/netty/netty/issues/5859
2016-10-11 07:50:39 -07:00
Norman Maurer
4920541cce [#5860] Ensure removal of SslHandler not produce IllegalReferenceCountException
Motivation:

If the user removes the SslHandler while still in the processing loop we will produce an IllegalReferenceCountException. We should stop looping when the handlerwas removed.

Modifications:

Ensure we stop looping when the handler is removed.

Result:

No more IllegalReferenceCountException.
2016-10-10 11:06:32 +02:00
Norman Maurer
418ca9a8f1 [#5841] Add test-case for mutual auth when using certificate chain
Motivation:

Add test-case for doing mutal auth with a certificate chain that holds more then one certificate.

Modifications:

Add test case

Result:

more tests.
2016-09-30 11:50:23 +02:00
olim7t
11f09bb5a6 Consolidate flushes even when no read in progress
Motivation:

Currently FlushConsolidationHandler only consolidates if a read loop is
active for a Channel, otherwise each writeAndFlush(...) call will still
be flushed individually. When these calls are close enough, it can be
beneficial to consolidate them even outside of a read loop.

Modifications:

When we allow a flush to "go through", don't perform it immediately, but
submit it on the channel's executor. Under high pressure, this gives
other writes a chance to enqueue before the task gets executed, and so
we flush multiple writes at once.

Result:

Lower CPU usage and less context switching.
2016-09-23 15:23:36 -07:00
Roger Kapsi
262fce7662 Catch+fire Exceptions thrown from SniHandler's select() method
Motivation

Give the user the ability to back out from SNI negoations.

Modifications

Put a try-catch around the select() call and re-fire any caught Exceptions.

Result

Fixes #5787
2016-09-19 16:10:53 -07:00
Scott Mitchell
b8cc72645d IdleStateHandler volatile member variables
Motivation:
IdleStateHandler has a number of volatile member variables which are only accessed from the EventLoop thread. These do not have to be volatile. The accessibility of these member variables are not consistent between private and package private. The state variable can also use a byte instead of an int.

Modifications:
- Remove volatile from member variables
- Change access to private for member variables
- Change state from int to byte

Result:
IdleStateHandler member variables cleaned up.
2016-09-15 10:27:52 -07:00
Norman Maurer
1a457ec269 Only set lastReadTime if an read actually happened before in IdleStateHandler.
Motivation:

IdleStateHandler and ReadTimeoutHandler could mistakely not fire an event even if no channelRead(...) call happened.

Modifications:

Only set lastReadTime if a read happened before.

Result:

More correct IdleStateHandler / ReadTimeoutHandler.
2016-09-13 16:57:37 -07:00
Victor Noël
e64f2acdf8 Add startTls parameter to SslContextBuilder
Motivation:

There is an incoherence in terms of API when one wants to use
startTls: without startTls one can use the SslContextBuilder's
method newHandler, but with startTls, the developper is forced
to call directly the SslHandler constructor.

Modifications:

Introduce startTls as a SslContextBuilder parameter as well as a
member in SslContext (and thus Jdk and OpenSsl implementations!).
Always use this information to call the SslHandler constructor.
Use false by default, in particular in deprecated constructors of
the SSL implementations.
The client Context use false by default

Results:

Fixes #5170 and more generally homogenise the API so that
everything can be done via SslContextBuilder.
2016-09-06 11:33:52 +02:00
Norman Maurer
bd559d106f Ensure SSLEngineTest works on different jvm versions.
Motivation:

af632278d2 introduced a test which only worked on some jvm versions and specific os'es.

Modifications:

Fix test to work on different java versions and os'es

Result:

No flacky test.
2016-09-05 21:07:58 +02:00
Norman Maurer
4fee26d7be Ensure we not sent duplicate certificates when using OpenSslEngine
Motivation:

We need to ensure we not set duplicated certificates when using OpenSslEngine.

Modifications:

- Skip first cert in chain when set the chain itself and so not send duplicated certificates
- Add interopt unit tests to ensure no duplicates are send.

Result:

No more duplicates.
2016-09-05 15:05:53 +02:00
Norman Maurer
a8011dd97c [#5712] Allow clients to override userDefinedWritabilityIndex from AbstractTrafficShapingHandler
Motivation:

AbstractTrafficShapingHandler has a package-private method called "userDefinedWritabilityIndex()" which a user may need to override if two sub-classes wants to be used in the ChannelPipeline.

Modifications:

Mark method protected.

Result:

Easier to extend AbstractTrafficShapingHandler.
2016-09-05 12:29:50 +02:00
Norman Maurer
26bdae9b47 Cleanup and simplify SslHandler
Motivation:

SslHandler can be cleaned up a bit in terms of naming and duplicated code.

Modifications:

- Fix naming of arguments
- Not schedule timeout event if not really needed
- share some code and simplify

Result:

Cleaner code.
2016-09-01 08:32:55 +02:00
Norman Maurer
2fd24b1867 Ensure accessing System property is done via AccessController.
Motivation:

When a SecurityManager is in place it may dissallow accessing the property which will lead to not be able to load the application.

Modifications:

Use AccessController.doPrivileged(...)

Result:

No more problems with SecurityManager.
2016-08-31 14:01:18 +02:00
Norman Maurer
3b86867992 [maven-release-plugin] prepare for next development iteration 2016-08-26 08:36:54 +02:00
Norman Maurer
8bdfc9ce39 [maven-release-plugin] prepare release netty-4.0.41.Final 2016-08-26 06:51:15 +02:00
Norman Maurer
53d4950704 Update CertificateRequestCallback usage to match new tcnative
Motiviation:

Previously the way how CertificateRequestCallback was working had some issues which could cause memory leaks and segfaults. Due of this tcnative code was updated to change the signature of the method provided by the interface.

Modifications:

Update CertificateRequestCallback implementations to match new interface signature.

Result:

No more segfaults / memory leaks when using boringssl or openssl >= 1.1.0
2016-08-28 20:27:02 +02:00
Roger Kapsi
ea45f89724 Unit Test for SslHandler's handlerRemoved()
Motivation

SslHandler's handlerRemoved() is supposed to release the SSLEngine (which it does) but there is no Test for it to make sure it really happens and doesn't unexpectedly change in the future.

Modifications

Add a Unit Test that makes sure that SslHandler releases the SSLEngine when the Channel gets closed.

Result

Assurance that SslHandler will not leak (ReferenceCounted) SSLEngines.
2016-08-27 20:50:52 +02:00
Scott Mitchell
aef16549ef SniHandler reference count leak if pipeline replace fails
Motivation:
The SniHandler attempts to generate a new SslHandler from the selected SslContext in a and insert that SslHandler into the pipeline. However if the underlying channel has been closed or the pipeline has been modified the pipeline.replace(..) operation may fail. Creating the SslHandler may also create a SSLEngine which is of type ReferenceCounted. The SslHandler states that if it is not inserted into a pipeline that it will not take reference count ownership of the SSLEngine. Under these conditions we will leak the SSLEngine if it is reference counted.

Modifications:
- If the pipeline.replace(..) operation fails we should release the SSLEngine object.

Result:
Fixes https://github.com/netty/netty/issues/5678
2016-08-25 13:13:43 -07:00
Norman Maurer
d08afec058 Ensure SslHandler.close(...) will not throw exception if flush of pending messages fails
Motivation:

When SslHandler.close(...) is called (as part of Channel.close()). it will also try to flush pending messages. This may fail for various reasons, but we still should propergate the close operation

Modifications:

- Ensure flush(...) itself will not throw an Exception if we was able to at least fail one pending promise (which should always be the case).
- If flush(...) fails as part of close ensure we still close the channel and then rethrow.

Result:

No more lost close operations possible if an exception is thrown during close
2016-08-24 07:33:40 +02:00
Scott Mitchell
0a3e6999c1 ReferenceCountedOpenSslEngineTest cleanup sequencing bug
Motivation:
ReferenceCountedOpenSslEngine depends upon the the SslContext to cleanup JNI resources. If we don't wait until the ReferenceCountedOpenSslEngine is done with cleanup before cleaning up the SslContext we may crash the JVM.

Modifications:
- Wait for the channels to close (and thus the ReferenceCountedOpenSslEngine to be cleaned up) before cleaning up the associated SslContext.

Result:
Cleanup sequencing is correct and no more JVM crash.
Fixes https://github.com/netty/netty/issues/5692
2016-08-18 21:04:10 +02:00
Norman Maurer
2e15818a2a Correctly fail all promises SSLENGINE_CLOSED once the engine is closed
Motivation:

We should fail all promises with the correct SSLENGINE_CLOSED exception one the engine is closed. We did not fail the current promise with this exception if the ByteBuf was not readable.

Modifications:

Correctly fail promises.

Result:

More correct handling of promises if the SSLEngine is closed.
2016-08-17 21:47:15 +02:00
Norman Maurer
44b87a4b1c Only run PemEncodedTest if OpenSsl.useKeyManagerFactory() returns false.
Motivation:

Commit b963595988 added a unit that will not work when KeyManagerFactory is used.

Modifications:

Only run the test if OpenSsl.useKeyManagerFactory() returns false.

Result:

Builds with boringssl
2016-08-16 13:44:03 +02:00
Roger Kapsi
4c9bd688a3 Fix handling of non direct backed PemEncoded.
Motivation:

The private key and certificate that are passed into #serKeyMaterial() could be PemEncoded in which case the #toPEM() methods return the identity of the value.

That in turn will fail in the #toBIO() step because the underlying ByteBuf is not necessarily direct.

Modifications:

- Use toBIO(...) which also works with non direct PemEncoded values
- Add unit test.

Result:

Correct handling of PemEncoded.
2016-08-16 09:13:11 +02:00
Norman Maurer
b618848a72 Remove missleading javadoc
Motivation:

Its completely fine to start writing before the handshake completes when using SslHandler. The writes will be just queued.

Modifications:

Remove the missleading and incorrect javadoc.

Result:

Correct javadoc.
2016-08-14 13:21:45 +02:00
Norman Maurer
93702ac877 Fix compilation error introduced by abf5e9fda5 2016-08-12 22:19:01 +02:00
Norman Maurer
abf5e9fda5 Ensure BIO is only used one time
Motivation:

Its not safe to reuse a BIO, we need to ensure we only use it once.

Modifications:

Only use the BIO once.

Result:

Correctly usage of BIO
2016-08-12 14:41:59 +02:00
Norman Maurer
a03f305f79 [#5648] Detect if netty-tcnative is in classpath or just tcnative
Motivation:

If netty is used in a tomcat container tomcat itself may ship tcnative. Because of this we will try to use OpenSsl in netty and fail because it is different to netty-tcnative.

Modifications:

Ensure if we find tcnative it is really netty-tcnative before using it.

Result:

No more problems when using netty in a tomcat container that also has tcnative installed.
2016-08-11 14:13:33 +02:00
Norman Maurer
6f8b2072e7 Ensure we only call ReferenceCountUtil.safeRelease(...) in finalize() if the refCnt() > 0
Motivation:

We need to ensure we only call ReferenceCountUtil.safeRelease(...) in finalize() if the refCnt() > 0 as otherwise we will log a message about IllegalReferenceCountException.

Modification:

Check for a refCnt() > 0 before try to release

Result:

No more IllegalReferenceCountException produced when run finalize() on OpenSsl* objects that where explicit released before.
2016-08-11 08:56:23 +02:00
Scott Mitchell
ede1a363be Update for netty-tcnative API changes
Motivation:
netty-tcnative API has changed to remove a feature that contributed to a memory leak.

Modifications:
- Update to use the modified netty-tcnative API

Result:
Netty can use the latest netty-tcnative.
2016-08-10 21:56:35 -07:00
Norman Maurer
35318cadbd Remove unused imports and not needed throws declarations.
Motivation:

In latest refeactoring we failed to cleanup imports and also there are some throws declarations which are not needed.

Modifications:

Cleanup imports and throws declarations

Result:

Cleaner code.
2016-08-10 11:47:18 +02:00
Scott Mitchell
c1c17ddb11 OpenSslEngine with no finalizer
Motivation:
OpenSslEngine and OpenSslContext currently rely on finalizers to ensure that native resources are cleaned up. Finalizers require the GC to do extra work, and this extra work can be avoided if the user instead takes responsibility of releasing the native resources.

Modifications:
- Make a base class for OpenSslENgine and OpenSslContext which does not have a finalizer but instead implements ReferenceCounted. If this engine is inserted into the pipeline it will be released by the SslHandler
- Add a new SslProvider which can be used to enable this new feature

Result:
Users can opt-in to a finalizer free OpenSslEngine and OpenSslContext.
Fixes https://github.com/netty/netty/issues/4958
2016-08-05 01:07:26 -07:00
Norman Maurer
d98b0236cd Allow to explicit disable usage of KeyManagerFactory when using OpenSsl
Motivation:

Sometimes it may be useful to explicit disable the usage of the KeyManagerFactory when using OpenSsl.

Modifications:

Add io.netty.handler.ssl.openssl.useKeyManagerFactory which can be used to explicit disable KeyManagerFactory usage.

Result:

More flexible usage.
2016-08-05 07:16:02 +02:00
Norman Maurer
4182971671 Take readerIndex into account when write to BIO.
Motivation:

We should take the readerIndex into account whe write into the BIO. Its currently not a problem as we slice before and so the readerIndex is always 0 but we should better not depend on this as this will break easily if we ever refactor the code and not slice anymore.

Modifications:

Take readerIndex into acount.

Result:

More safe and correct use.
2016-08-05 07:14:34 +02:00
Norman Maurer
6dca29e1fc Servers should not send duplicate intermediate certificates.
Motivation:
Servers sometimes send duplicate intermediate certificates.

Modifications:
OpenSslKeyMaterialManager.setKeyMaterial() dedups aliases before calling SSL.setCertificateChainBio().

Result:
Servers no longer send duplicate itermediate certificates.
2016-08-01 10:53:09 +02:00
Norman Maurer
a6e1f6290c [#5598] Ensure SslHandler not log false-positives when try to close the channel due timeout.
Motivation:

When we try to close the Channel due a timeout we need to ensure we not log if the notification of the promise fails as it may be completed in the meantime.

Modifications:

Add another constructor to ChannelPromiseNotifier and PromiseNotifier which allows to log on notification failure.

Result:

No more miss-leading logs.
2016-07-30 21:15:41 +02:00
Norman Maurer
e015dfaea2 [maven-release-plugin] prepare for next development iteration 2016-07-27 10:47:03 +02:00
Norman Maurer
837d9947ec [maven-release-plugin] prepare release netty-4.0.40.Final 2016-07-27 10:30:08 +02:00
Scott Mitchell
44ad7ec109 FlushConsolidationHandler remove conditional
Motivation:
FlushConsolidationHandler#flushIfNeeded has a conditional which is fixed based upon code path. This conditional can be removed and instead just manually set in each fixed code path.

Modifications:
- Remove boolean parameter on FlushConsolidationHandler#flushIfNeeded and set readInprogess to false manually when necessary

Result:
Less conditionals in FlushConsolidationHandler
2016-07-27 07:12:08 +02:00
Ian Haken
6794661926 Elliminated some buggy behavior when using a KeyManagerFactory with OpenSslServerContext.
Motivation:

PR #5493 added support for KeyManagerFactories when using the OpenSsl context. This commit corrects a bug causing a NullPointerException that occurs when using a KeyManagerFactory without a certificate chain and private key.

Modifications:

Removes assertNotNull() assertions which were causing a certificate chain and private key to be required even when using a KeyManagerFactory. Also removed a redundant call to buildKeyManagerFactory() which was also causing a exception when a KeyManagerFactory is provided but a certificate chain and private key is not.

Result:

A KeyManagerFactory can now be used in the OpenSslServerContext without an independent certificate chain and private key.
2016-07-22 09:15:12 +02:00
Norman Maurer
45f9d29fc1 [maven-release-plugin] prepare for next development iteration 2016-07-15 07:10:09 +02:00
Norman Maurer
38bdf86ba1 [maven-release-plugin] prepare release netty-4.0.39.Final 2016-07-15 07:08:29 +02:00
Norman Maurer
0616f5bac7 Add FlushConsolidationHandler which consolidates flush operations as these are expensive
Motivation:

Calling flush() and writeAndFlush(...) are expensive operations in the sense as both will produce a write(...) or writev(...) system call if there are any pending writes in the ChannelOutboundBuffer. Often we can consolidate multiple flush operations into one if currently a read loop is active for a Channel, as we can just flush when channelReadComplete is triggered. Consolidating flushes can give a huge performance win depending on how often is flush is called. The only "downside" may be a bit higher latency in the case of where only one flush is triggered by the user.

Modifications:

Add a FlushConsolidationHandler which will consolidate flushes and so improve the throughput.

Result:

Better performance (throughput). This is especially true for protocols that use some sort of PIPELINING.
2016-07-07 06:53:45 +02:00
Norman Maurer
9b555b3af4 Share code between ReadTimeoutHandler and IdleStateHandler
Motivation:

ReadTimeoutHandler and IdleStateHandler have duplicated code, we should share whatever possible.

Modifications:

Let ReadTimeoutHandler extend IdleStateHandler.

Result:

Remove code duplication.
2016-07-05 20:12:52 +02:00
Scott Mitchell
c417e4fa8f OpenSslEngine writePlaintextData WANT_READ with no data in BIO buffer unit test
Motivation:
Unit test for the OpenSslEngine "OpenSslEngine writePlaintextData WANT_READ with no data in BIO buffer" issue.

Modifications:
- Update SslEngine test to include renegotiation

Result:
More test coverage in OpenSslEngine.
2016-07-01 12:00:35 -07:00
Norman Maurer
4329e97455 [maven-release-plugin] prepare for next development iteration 2016-07-01 07:59:55 +02:00
Norman Maurer
8642f16f35 [maven-release-plugin] prepare release netty-4.0.38.Final 2016-07-01 07:59:37 +02:00
buchgr
a96a97356f Fix NPE in OpenSslEngine
Motivation:

The gRPC interop tests fail due to a NPE in OpenSslEngine.

Caused by: java.lang.NullPointerException
at io.netty.handler.ssl.OpenSslEngine.setSSLParameters(OpenSslEngine.java:1473)

Modifications:

Add a null check

Result:

No more NPE exceptions :-)
2016-06-30 11:13:22 +02:00
Norman Maurer
352e81c41d Add support for KeyManagerFactory when using SslProvider.OpenSsl.
Motivation:

To be able to use SslProvider.OpenSsl with existing java apps that use the JDK SSL API we need to also provide a way to use it with an existing KeyManagerFactory.

Modification:

Make use of new tcnative apis and so hook in KeyManagerFactory.

Result:

SslProvider.OpenSsl can be used with KeyManagerFactory as well.
2016-06-28 09:35:47 +02:00
Norman Maurer
707784081e [#5401] Support -Djdk.tls.ephemeralDHKeySize=num when using OpenSslContext
Motivation:

Java8+ adds support set a DH key size via a System property (jdk.tls.ephemeralDHKeySize). We should respect this when using OpenSSL.

Modifications:

Respect system property.

Result:

More consistent SSL implementation.
2016-06-28 07:10:23 +02:00
Norman Maurer
b0f0922834 Expose session ticket statistics.
Motivation:

We recently added support for session ticket statistics which we can expose now.

Modifications:

Expose the statistics.

Result:

Be able to obtain session ticket statistics.
2016-06-27 19:36:55 +02:00
Norman Maurer
fc85eb34ce Correctly implement SSLSession.getLastAccessedTime() for OpenSSLEngine
Motivation:

We need to return a correct time for SSLSession.getLastAccessedTime() so it reflect when the handshake was done when the session was reused.

Modifications:

Correctly reflect handshake time in getLastAccessedTime().

Result:

More conform SSLSession implementation.
2016-06-23 11:31:12 +02:00
Norman Maurer
f982b0bda9 Allow to wrap another SslContext implementation and do extra init steps on the SSLEngine.
Motivation:

Sometimes its needed to customize the SSLEngine (like setting protocols etc). For this it would be useful if the user could wrap an SslContext and do init steps on the SSLEngine.

Modifications:

Add new SslContext implementation which can wrap another one and allow to customize the SSLEngine

Result:

More flexible usage of SslContext.
2016-06-23 11:29:04 +02:00
Norman Maurer
77350b7010 OpenSslEngine.getSupportedCipherSuites() must return java names as well.
Motivation:

At the moment OpenSslEngine.getSupportedCipherSuites() only return the original openssl cipher names and not the java names. We need also include the java names.

Modifications:

Correctly return the java names as well.

Result:

Correct implementation of OpenSslEngine.getSupportedCipherSuites()
2016-06-23 11:27:36 +02:00
Norman Maurer
836b2d22c1 Move validation of arguments out of synchronized block
Motivation:

There is no need already use synchronized when validate the args of the methods.

Modifications:

First validate arguments and then use synchronized

Result:

Less code executed in synchronized block.
2016-06-20 14:24:22 +02:00
Norman Maurer
3719b5f919 Set some StackTraceElement on pre-instantiated static exceptions
Motivation:

We use pre-instantiated exceptions in various places for performance reasons. These exceptions don't include a stacktrace which makes it hard to know where the exception was thrown. This is especially true as we use the same exception type (for example ChannelClosedException) in different places. Setting some StackTraceElements will provide more context as to where these exceptions original and make debugging easier.

Modifications:

Set a generated StackTraceElement on these pre-instantiated exceptions which at least contains the origin class and method name. The filename and linenumber are specified as unkown (as stated in the javadocs of StackTraceElement).

Result:

Easier to find the origin of a pre-instantiated exception.
2016-06-20 11:37:19 +02:00
Norman Maurer
f68517d7fb Correctly support SSLSession.getId() when using OpenSslEngine
Motivation:

At the moment SSLSession.getId() may always return an empty byte array when OpenSSLEngine is used. This is as we not set SSL_OP_NO_TICKET on the SSLContext and so SSL_SESSION_get_id(...) will return an session id with length of 0 if tickets are not used.

Modifications:

- Set SSL_OP_NO_TICKET by default and only clear it if the user requests the usage of session tickets.
- Add unit test

Result:

Ensure consistent behavior between different SSLEngine implementations.
2016-06-20 09:35:06 +02:00
Norman Maurer
b496fd364a Add support for SSLParameters.setCipherSuiteOrder() when using Java8+
Motivation:

When using java8+ we should support SSLParameters.setCipherSuiteOrder()

Modifications:

Add support of SLParameters.setCipherSuiteOrder() by using reflection, so we can compile with java7 but still support it.

Result:

Users that use java8+ can use SSLParameters.setCipherSuiteOrder()
2016-06-20 09:34:06 +02:00
Norman Maurer
51bb7f39b6 Add support for SNIHostName when using Java8+
Motivation:

Java8 added support for using SNIHostName with SSLParameters. We currently ignore it in OpenSslEngine.

Modifications:

Use reflection to support SNIHostName.

Result:

People using Java8 can use SNIHostName even when OpenSslEngine is used.
2016-06-20 09:25:26 +02:00
Norman Maurer
1929aa8256 Correctly skip OpenSsl* tests if OpenSsl.isAvailable() is false.
Motivation:

We missed to skip some tests for OpenSsl when OpenSsl.isAvailable() is false.

Modifications:

- Correctly skip tests when OpenSsl.isAvailable() is false.
- Simplify some code by using @BeforeClass.

Result:

Be able to compile netty even when OpenSsl is not present on the system.
2016-06-17 08:36:08 +02:00
Norman Maurer
afd6094c53 Guard against possible segfault when OpenSslContext is gc'ed and user still hold reference to OpenSslSessionContext / OpenSslSessionStats
Motivation:

When the OpenSslContext is gc'ed and the user still hold a reference to OpenSslSessionContext / OpenSslSessionStats it is possible to produce a segfault when calling
a method on any of these that tries to pass down the ctx pointer to the native methods. This is because the OpenSslContext finalizer will free the native pointer.

Modifications:

Change OpenSslSessionContext / OpenSslSessionContext to store a reference to OpenSslContext and so prevent the GC to collect it as long as the user has a reference to OpenSslSessionContext / OpenSslSessionContext.

Result:

No more sefault possible.
2016-06-17 08:33:48 +02:00
Roger Kapsi
5fa5fcc9d9 Fix for a newly intrduced bug in #5377
Motivation

This bug was introduced with #5377 and affects only users who'd like to share/cache/re-use `PemPrivateKey` and `PemX509Certificate` instances.

Modifications

Use `ByteBuf#writeBytes(src, readerIndex, length)` so that the src's readerIndex doesn't change and can consequently be used more than once.

Result

It's possible to share/cache/re-use `PemPrivateKey` and `PemX509Certificate` instances as long as their refCnt remains >= 1.
2016-06-13 20:33:17 +02:00
Norman Maurer
9558d0b5e0 Fix compile error produced by bad cherry-pick of 7ed105adbf 2016-06-10 19:29:04 +02:00
Roger Kapsi
7ed105adbf Let OpenSslContext take pre-encoded pkcs#8 private key/cert bytes
Motivation

OpenSslContext is expecting Java's PrivateKey and X509Certificate objects as input
(for JdkSslContext API compatibility reasons) but doesn't really use them beyond
turning them into PEM/PKCS#8 strings.

This conversion can be entirely skipped if the user can pass in private keys and
certificates in a format that Netty's OpenSSL code can digest.

Modifications

Two new classes have been added that act as a wrapper around the pre-encoded byte[]
and also retain API compatibility to JdkSslContext.

Result

It's possible to pass PEM encoded bytes straight into OpenSSL without having to
parse them (e.g. File to Java's PrivateKey) and then encode them (i.e. PrivateKey
into PEM/PKCS#8).

File pemPrivateKeyFile;
byte[] pemBytes = readBytes(pemPrivateKeyFile);
PemPrivateKey pemPrivateKey = PemPrivateKey.valueOf(pemBytes);

SslContextBuilder.forServer(pemPrivateKey)
    .sslProvider(SslProvider.OPENSSL)
2016-06-10 18:08:09 +02:00
Guido Medina
c784271620 Use shaded dependency on JCTools instead of copy and paste
Motivation:
JCTools supports both non-unsafe, unsafe versions of queues and JDK6 which allows us to shade the library in netty-common allowing it to stay "zero dependency".

Modifications:
- Remove copy paste JCTools code and shade the library (dependencies that are shaded should be removed from the <dependencies> section of the generated POM).
- Remove usage of OneTimeTask and remove it all together.

Result:
Less code to maintain and easier to update JCTools and less GC pressure as the queue implementation nt creates so much garbage
2016-06-10 13:53:28 +02:00
Norman Maurer
c5d70ec7dc [#5372] Ensure OpenSslClientContext / OpenSslServerContext can be garbage collected
Motivation:

OpenSslClientContext / OpenSslServerContext can never be garbage collected as both are part of a reference to a callback that is stored as global reference in jni code.

Modifications:

Ensure the callbacks are static and so not hold the reference.

Result:

No more leak due not collectable OpenSslClientContext / OpenSslServerContext
2016-06-09 22:39:33 +02:00
Scott Mitchell
8ed9953f36 OpenSslEngine encrypt more data per wrap call
Motivation:
OpenSslEngine.wrap will only encrypt at most 1 buffer per call. We may be able to encrypt multiple buffers per call.

Modifications:
- OpensslEngine.wrap should continue encrypting data until there is an error, no more data, or until the destination buffer would be overflowed.

Result:
More encryption is done per OpenSslEngine.wrap call
2016-06-08 12:23:33 -07:00
Norman Maurer
2919145072 [maven-release-plugin] prepare for next development iteration 2016-06-07 20:00:14 +02:00
Norman Maurer
4169779352 [maven-release-plugin] prepare release netty-4.0.37.Final 2016-06-07 19:57:15 +02:00
Scott Mitchell
524156f164 OpenSslEngine writePlaintextData WANT_READ with no data in BIO buffer
Motivation:
CVE-2016-4970

OpenSslEngine.wrap calls SSL_write which may return SSL_ERROR_WANT_READ, and if in this condition there is nothing to read from the BIO the OpenSslEngine and SslHandler will enter an infinite loop.

Modifications:
- Use the error code provided by OpenSSL and go back to the EventLoop selector to detect if the socket is closed

Result:
OpenSslEngine correctly handles the return codes from OpenSSL and does not enter an infinite loop.
2016-06-07 08:59:36 -07:00
Scott Mitchell
b3160bd696 OpenSslEngine remove unecessary rejectRemoteInitiatedRenegation call
Motivation:
OpenSslEngine calls rejectRemoteInitiatedRenegation in a scenario where the number of handshakes has not been observed to change. The number of handshakes has only been observed to change after readPlaintextData is called.

Modifications:
- Remove the call to rejectRemoteInitiatedRenegation before calls to readPlaintextData

Result:
Less code.
2016-06-03 13:02:22 -07:00
floragunn
87efddcda7 Set the session id context properly to make client authentication work with open ssl provider.
Motivation:

When netty is used with open ssl provider and client authentication the following errors can occur:
error:140D9115:SSL routines:ssl_get_prev_session:session id context uninitialized
error:140A1175:SSL routines:ssl_bytes_to_cipher_list:inappropriate fallback
error:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol

Modifications:

Set the session id context in OpenSslServerContext so that sessions which use client authentication
which are cached have the same context id value.

Result:

Client authentication now works with open ssl provider.
2016-05-28 21:29:19 +02:00
Trustin Lee
6a7ee52cd4 Replace DomainMappingBuilder with DomainNameMappingBuilder
Motivation:

DomainMappingBuilder should have been named as DomainNameMappingBuilder
because it builds a DomainNameMapping.

Modifications:

- Add DomainNameMappingBuilder that does the same job with
  DomainMappingBuilder
- Deprecate DomainMappingBuilder and delegate its logic to
  DomainNameMappingBuilder
- Remove the references to the deprecated methods and classes related
  with domain name mapping
- Miscellaneous:
  - Fix Javadoc of DomainNameMapping.asMap()
  - Pre-create the unmodifiable map in DomainNameMapping

Result:

- Consistent naming
- Less use of deprecated API
2016-05-18 12:03:51 +02:00
Norman Maurer
219bc6474e Allow to extend IdleStateHandler and so provide more details for IdleStateEvents
Motivation:

Sometimes it is useful to include more details in the IdleStateEvents that are produced by the IdleStateHandler. For this users should be able to create their own IdleStateEvents that encapsulate more informations.

Modifications:

- Make IdleStateEvent constructor protected and the class non-final
- Add protected method to IdleStateHandler that users can override and so create their own IdleStateEvents.

Result:

More flexible and customizable IdleStateEvents / IdleStateHandler
2016-05-14 07:19:39 +02:00
Norman Maurer
e4501a5de0 [#5218] Zero out private key copied to ByteBuf before release.
Motivation:

We should zero-out the private key as soon as possible when we not need it anymore.

Modifications:

zero out the private key before release the buffer.

Result:

Limit the time the private key resist in memory.
2016-05-09 10:00:19 +02:00
Fabian Lange
5aa87774a0 Rewrite misleading Note in FingerprintTrustManagerFactory javadoc
Motivation:

The current note reads as if this class is dangerous and advises the reader to "understand what this class does".

Modifications:

Rewrite the Javadoc note to describe what fingerprint checks are and what problems remain.

Result:

Clearer description which no longer causes the impression this class is dangerous.
2016-05-03 07:39:52 +02:00
Renjie Sun
ad27387646 Improve client SNI in testSniWithApnHandler
Motivations
The test SniHandlerTest#testSniWithApnHandler() does not actually
involve SNI: given the client setup, the ClientHello in the form of hex
strings is not actually written to the wire, so the server never receives that.
We may need to write in somewhere else (e.g., channelActive()) instead of in
initChannel() in order for the hex strings to reach the server. So here
what's actually going on is an ordinary TLS C/S communication without SNI.

Modifications
The client part is modified to enable SNI by using an SslHandler with an
SSLEngine created by io.netty.handler.ssl.SslContext#newEngine(), where
the server hostname is specified. Also, more clauses are added to verify that
the SNI is indeed successful.

Results
Now the test verifies that both SNI and APN actually happen and succeed.
2016-05-01 20:23:45 +02:00
nmittler
6fc05772d0 Support preloading of tcnative share lib
Motivation:

Some applications may use alternative methods of loading the tcnative JNI symbols. We should support this use case.

Modifications:

Separate the loading and initialzation of the tcnative library so that each can fail independently.

Result:

Fixes #5043
2016-04-12 20:21:19 +02:00
Norman Maurer
bfdfb50df6 Allow to create a JdkSslContext from an existing JDK SSLContext. Related to [#5095] and [#4929]
Motivation:

Sometimes a user only has access to a preconfigured SSLContext but still would like to use our ssl sub-system. For this situations it would be very useful if the user could create a JdkSslContext instance from an existing SSLContext.

Modifications:

- Create new public constructors in JdkSslContext which allow to wrap an existing SSLContext and make the class non-abstract
- Mark JdkSslServerContext and JdkSslClientContext as deprecated as the user should not directly use these.

Result:

It's now possible to create an JdkSslContext from an existing SSLContext.
2016-04-09 19:08:32 +02:00
Scott Mitchell
4506bbd27b fcbeebf6df unit test bug
Motivation:
fcbeebf6df introduced a unit test to verify ApplicationProtocolNegotiationHandler is compatible with SniHandler. However only the server attempts ALPN and verifies that it completes and the client doesn't verify the handshake is completed. This can lead to the client side SSL engine to prematurely close and throw an exception.

Modifications:
- The client should wait for the SSL handshake and ALPN to complete before the test exits.

Result:
SniHandlerTest.testSniWithApnHandler is more reliable.
2016-04-06 00:12:54 -07:00
Scott Mitchell
2263d9b46b ApplicationProtocolNegotiationHandler doesn't work with SniHandler
Motivation:
ApplicationProtocolNegotiationHandler attempts to get a reference to an SslHandler in handlerAdded, but when SNI is in use the actual SslHandler will be added to the pipeline dynamically at some later time. When the handshake completes ApplicationProtocolNegotiationHandler throws an IllegalStateException because its reference to SslHandler is null.

Modifications:
- Instead of saving a reference to SslHandler in handlerAdded just search the pipeline when the SslHandler is needed

Result:
ApplicationProtocolNegotiationHandler support SniHandler.
Fixes https://github.com/netty/netty/issues/5066
2016-04-05 09:03:07 +02:00
Norman Maurer
4b6b167839 [maven-release-plugin] prepare for next development iteration 2016-04-04 16:53:40 +02:00
Norman Maurer
e8fa848f43 [maven-release-plugin] prepare release netty-4.0.36.Final 2016-04-04 16:52:53 +02:00
Norman Maurer
47baeb5ded [maven-release-plugin] rollback the release of netty-4.0.36.Final 2016-03-29 22:58:32 +02:00
Norman Maurer
29a4f3e363 [maven-release-plugin] prepare release netty-4.0.36.Final 2016-03-29 21:30:50 +02:00
Vladimir Kostyukov
ea4a8e339c Read if needed on NEED_UNWRAP
Motivation:

There are some use cases when a client may only be willing to read from a channel once
its previous write is finished (eg: serial dispatchers in Finagle). In this case, a
connection with SslHandler installed and ctx.channel().config().isAutoRead() == false
will stall in 100% of cases no matter what order of "channel active", "write", "flush"
events was.

The use case is following (how Finagle serial dispatchers work):

1. Client writeAndFlushes and waits on a write-promise to perform read() once it's satisfied.
2. A write-promise will only be satisfied once SslHandler finishes with handshaking and
   sends the unencrypted queued message.
3. The handshaking process itself requires a number of read()s done by a client but the
   SslHandler doesn't request them explicitly assuming that either auto-read is enabled
   or client requested at least one read() already.
4. At this point a client will stall with NEED_UNWRAP status returned from underlying engine.

Modifiations:

Always request a read() on NEED_UNWRAP returned from engine if

a) it's handshaking and
b) auto read is disabled and
c) it wasn't requested already.

Result:

SslHandler is now completely tolerant of whether or not auto-read is enabled and client
is explicitly reading a channel.
2016-03-29 08:48:08 +02:00
Norman Maurer
e537c6b472 [#4637] More helpful exception message when a non PKCS#8 key is used.
Motivation:

We should throw a more helpful exception when a non PKCS#8 key is used by the user.

Modifications:

Change exception message to give a hint what is wrong.

Result:

Easier for user to understand whats wrong with their used key.
2016-03-27 20:19:54 +02:00