Commit Graph

510 Commits

Author SHA1 Message Date
Trustin Lee
2056aed50d Implement OpenSslEngine.getSupportedCipherSuites() and get/setEnabledCipherSuites()
Motivation:

To make OpenSslEngine a full drop-in replacement, we need to implement
getSupportedCipherSuites() and get/setEnabledCipherSuites().

Modifications:

- Retrieve the list of the available cipher suites when initializing
  OpenSsl.
- Improve CipherSuiteConverter to understand SRP
- Add more test data to CipherSuiteConverterTest
- Add bulk-conversion method to CipherSuiteConverter

Result:

OpenSslEngine should now be a drop-in replacement for JDK SSLEngineImpl
for most cases.
2014-12-30 19:02:32 +09:00
Trustin Lee
827a4cbf36 Cipher suite conversion between Java and OpenSSL
Related: #3285

Motivation:

When a user attempts to switch from JdkSslContext to OpenSslContext, he
or she will see the initialization failure if he or she specified custom
cipher suites.

Modifications:

- Provide a utility class that converts between Java cipher suite string
  and OpenSSL cipher suite string
- Attempt to convert the cipher suite so that a user can use the cipher
  suite string format of Java regardless of the chosen SslContext impl

Result:

- It is possible to convert all known cipher suite strings.
- It is possible to switch from JdkSslContext and OpenSslContext and
  vice versa without any configuration changes
2014-12-30 17:26:50 +09:00
Trustin Lee
6201bbd2e1 Clean-up 2014-12-29 15:56:32 +09:00
Frederic Bregier
b886c056bc Fix big transfer and Write traffic shaping issues
Motivation:

Several issues were shown by various ticket (#2900 #2956).
Also use the improvement on writability user management from #3036.
And finally add a mixte handler, both for Global and Channels, with
the advantages of being uniquely created and using less memory and
less shaping.

Issue #2900

When a huge amount of data are written, the current behavior of the
TrafficShaping handler is to limit the delay to 15s, whatever the delay
the previous write has. This is wrong, and when a huge amount of writes
are done in a short time, the traffic is not correctly shapened.

Moreover, there is a high risk of OOM if one is not using in his/her own
handler for instance ChannelFuture.addListener() to handle the write
bufferisation in the TrafficShapingHandler.

This fix use the "user-defined writability flags" from #3036 to
allow the TrafficShapingHandlers to "user-defined" managed writability
directly, as for reading, thus using the default isWritable() and
channelWritabilityChanged().
This allows for instance HttpChunkedInput to be fully compatible.

The "bandwidth" compute on write is only on "acquired" write orders, not
on "real" write orders, which is wrong from statistic point of view.

Issue #2956

When using GlobalTrafficShaping, every write (and read) are
synchronized, thus leading to a drop of performance.
ChannelTrafficShaping is not touched by this issue since synchronized is
then correct (handler is per channel, so the synchronized).

Modifications:
The current write delay computation takes into account the previous
write delay and time to check is the 15s delay (maxTime) is really
exceeded or not (using last scheduled write time). The algorithm is
simplified and in the same time more accurate.

This proposal uses the #3036 improvement on user-defined writability
flags.

When the real write occurs, the statistics are update accordingly on a
new attribute (getRealWriteThroughput()).

To limit the synchronisations, all synchronized on
GlobalTrafficShapingHandler on submitWrite were removed. They are
replaced with a lock per channel (since synchronization is still needed
to prevent unordered write per channel), as in the sendAllValid method
for the very same reason.
Also all synchronized on TrafficCounter on read/writeTimeToWait() are
removed as they are unnecessary since already locked before by the
caller.
Still the creation and remove operations on lock per channel (PerChannel
object) are synchronized to prevent concurrency issue on this critical
part, but then limited.

Additionnal changes:
1) Use System.nanoTime() instead of System.currentTimeMillis() and
minimize calls
2) Remove / 10 ° 10 since no more sleep usage
3) Use nanoTime instead of currentTime such that time spend is computed,
not real time clock. Therefore the "now" relative time (nanoTime based)
is passed on all sub methods.
4) Take care of removal of the handler to force write all pending writes
and release read too
8) Review Javadoc to explicit:

- recommandations to take into account isWritable

- recommandations to provide reasonable message size according to
traffic shaping limit

- explicit "best effort" traffic shaping behavior when changing
configuration dynamically

Add a MixteGlobalChannelTrafficShapingHandler which allows to use only one
handler for mixing Global and Channel TSH. I enables to save more memory and
tries to optimize the traffic among various channels.

Result:
The traffic shaping is more stable, even with a huge number of writes in
short time by taking into consideration last scheduled write time.

The current implementation of TrafficShapingHandler using user-defined
writability flags and default isWritable() and
fireChannelWritabilityChanged works as expected.

The statistics are more valuable (asked write vs real write).

The Global TrafficShapingHandler should now have less "global"
synchronization, hoping to the minimum, but still per Channel as needed.

The GlobalChannel TrafficShapingHandler allows to have only one handler for all channels while still offering per channel in addition to global traffic shaping.

And finally maintain backward compatibility.
2014-12-29 15:36:02 +09:00
Norman Maurer
b77e338cb7 Allow to set the context for which sessions can be used.
Motivation:

Openssl supports the SSL_CTX_set_session_id_context function to limit for which context a session can be used. We should support this.

Modifications:

Add OpenSslServerSessionContext that exposes a setSessionIdContext(...) method now.

Result:

It's now possible to use SSL_CTX_set_session_id_context.
2014-12-26 15:02:47 +01:00
Norman Maurer
2898340c39 Explicit allow to enable / disable session cache
Motivation:

It is sometimes useful to enable / disable the session cache.

Modifications:

* Add OpenSslSessionContext.setSessionCacheEnabled(...) and isSessionCacheEnabled()

Result:

It is now possible to enable / disable cache on the fly
2014-12-26 14:56:22 +01:00
Norman Maurer
eabfb91686 Allow to enable/disable protocols on the OpenSslEngine
Motivation:

To be compatible with SSLEngine we need to support enable / disable procols on the OpenSslEngine

Modifications:

Implement OpenSslEngine.getSupportedProtocols() , getEnabledProtocols() and setEnabledProtocols(...)

Result:

Better compability with SSLEngine
2014-12-26 09:33:45 +01:00
Norman Maurer
6bfc68c785 Add proper Openssl.SSLSession.getId() implementation
Motivation:

The current implementation not returns the real session as byte[] representation.

Modifications:

Create a proper Openssl.SSLSession.get() implementation which returns the real session as byte[].

Result:

More correct implementation
2014-12-26 09:29:55 +01:00
Norman Maurer
8282b12b6a Allow to enable session cache when using OpenSsl
Motivation:

At the moment it is not possible to make use of the session cache when OpenSsl is used. This should be possible when server mode is used.

Modifications:

- Add OpenSslSessionContext (implements SSLSessionContext) which exposes all the methods to modify the session cache.
- Add various extra methods to OpenSslSessionContext for extra functionality
- Return OpenSslSessionContext when OpenSslEngine.getSession().getContext() is called.
- Add sessionContext() to SslContext
- Move OpenSsl specific session operations to OpenSslSessionContext and mark the old methods @deprecated

Result:

It's now possible to use session cache with OpenSsl
2014-12-26 09:18:25 +01:00
Trustin Lee
54a27ef07e Fix NoClassDefFoundError when netty-tcnative is unavailable
Motivation:

ProxyHandlerTest fails with NoClassDefFoundError raised by
SslContext.newClientContext().

Modifications:

Fix a missing 'return' statement that makes the switch-case block fall
through unncecessarily

Result:

- ProxyHandlerTest does not fail anymore.
- SslContext.newClientContext() does not raise NoClassDefFoundError
  anymore.
2014-12-26 15:47:35 +09:00
Norman Maurer
eeb5e2c0a9 Check for errors without object allocation
Motivation:

At the moment we use SSL.getLastError() in unwrap(...) to check for error. This is very inefficient as it creates a new String for each check and we also use a String.startsWith(...) to detect if there was an error we need to handle.

Modifications:

Use SSL.getLastErrorNumber() to detect if we need to handle an error, as this only returns a long and so no String creation happens. Also the detection is much cheaper as we can now only compare longs. Once an error is detected the lately SSL.getErrorString(long) is used to conver the error number to a String and include it in log and exception message.

Result:

Performance improvements in OpenSslEngine.unwrap(...) due less object allocation and also faster comparations.
2014-12-22 21:09:18 +01:00
Norman Maurer
f13a3fbefb Use OpenSslClientContext as default if openssl is avaible
Motivation:

As we now support OpenSslEngine for client side, we should use it when avaible.

Modifications:

Use SslProvider.OPENSSL when openssl can be found

Result:

OpenSslEngine is used whenever possible
2014-12-22 21:07:28 +01:00
Norman Maurer
a86a2f6478 Allow to use custom TrustManagerFactory for JdkSslServerContent and OpenSslServerContext
Motivation:

When using client auth it is sometimes needed to use a custom TrustManagerFactory.

Modifications:

Allow to pass in TrustManagerFactory

Result:

It's now possible to use custom TrustManagerFactories for JdkSslServerContext and OpenSslServerContext
2014-12-22 20:55:55 +01:00
Norman Maurer
16f69985d4 Use TrustManager for certificate verification
Motivation:

To make OpenSsl*Context a drop in replacement for JdkSsl*Context we need to use TrustManager.

Modifications:

Correctly hook in the TrustManager

Result:

Better compatibility
2014-12-22 20:26:30 +01:00
Norman Maurer
d0b18db383 Allow to enable client authentication on the OpenSslEngine
Motivation:

At the moment there is no way to enable client authentication when using OpenSslEngine. This limits the uses of OpenSslEngine.

Modifications:

Add support for different authentication modes.

Result:

OpenSslEngine can now also be used when client authenticiation is needed.
2014-12-22 20:16:02 +01:00
Norman Maurer
7088235cac More complete OpenSslEngine SSLSession implementation
Motivation:

The current SSLSession implementation used by OpenSslEngine does not support various operations and so may not be a good replacement by the SSLEngine provided by the JDK implementation.

Modifications:

- Add SSLSession.getCreationTime()
- Add SSLSession.getLastAccessedTime()
- Add SSLSession.putValue(...), getValue(...), removeValue(...), getValueNames()
- Add correct SSLSession.getProtocol()
- Ensure OpenSSLEngine.getSession() is thread-safe
- Use optimized AtomicIntegerFieldUpdater when possible

Result:

More complete OpenSslEngine SSLSession implementation
2014-12-22 20:12:34 +01:00
Norman Maurer
60f24ac2a1 Add OpenSslClientContext to allow creating SslEngine for client side
Motivation:

We only support openssl for server side at the moment but it would be also useful for client side.

Modification:

* Upgrade to new netty-tcnative snapshot to support client side openssl support
* Add OpenSslClientContext which can be used to create SslEngine for client side usage
* Factor out common logic between OpenSslClientContext and OpenSslServerContent into new abstract base class called OpenSslContext
* Correctly detect handshake failures as soon as possible
* Guard against segfault caused by multiple calls to destroyPools(). This can happen if OpenSslContext throws an exception in the constructor and the finalize() method is called later during GC

Result:

openssl can be used for client and servers now.
2014-12-22 19:37:48 +01:00
Norman Maurer
e266cfcf2b Efficiently handle writing ( wrap(...) ) of CompositeByteBuf when using SslHandler
Motivation:

SslHandler.wrap(...) does a poor job when handling CompositeByteBuf as it always call ByteBuf.nioBuffer() which will do a memory copy when a CompositeByteBuf is used that is backed by multiple ByteBuf.

Modifications:

- Use SslEngine.wrap(ByteBuffer[]...) to allow wrap CompositeByteBuf in an efficient manner
- Reduce object allocation in unwrapNonAppData(...)

Result:

Performance improvement when a CompositeByteBuf is written and the SslHandler is in the ChannelPipeline.
2014-12-22 12:14:14 +01:00
Norman Maurer
17d1fa093e Fix memory leak in OpenSslEngine
Motivation:

When a remote peer did open a connection and only do the handshake without sending any data and then directly close the connection we did not call shutdown() in the OpenSslEngine. This leads to a native memory leak. Beside this it also was not fireed when a OpenSslEngine was created but never used.

Modifications:

- Make sure shutdown() is called in all cases when closeInbound() is called
- Call shutdown() also in the finalize() method to ensure we release native memory when the OpenSslEngine is GC'ed

Result:

No more memory leak when using OpenSslEngine
2014-12-21 09:26:38 +01:00
Trustin Lee
689d59b04b Make sure to notify handshake success even if SSLEngine is closed
Related:

e9685ea45a

Motivation:

SslHandler.unwrap() does not evaluate the handshake status of
SSLEngine.unwrap() when the status of SSLEngine.unwrap() is CLOSED.

It is not correct because the status does not reflect the state of the
handshake currently in progress, accoding to the API documentation of
SSLEngineResult.Status.

Also, sslCloseFuture can be notified earlier than handshake notification
because we call sslCloseFuture.trySuccess() before evaluating handshake
status.

Modifications:

- Notify sslCloseFuture after the unwrap loop is finished
- Add more assertions to SocketSslEchoTest

Result:

Potentially fix the regression caused by:
- e9685ea45a
2014-12-12 11:47:52 +09:00
Trustin Lee
e9685ea45a Add SslHandler.renegotiate()
Related: #3125

Motivation:

We did not expose a way to initiate TLS renegotiation and to get
notified when the renegotiation is done.

Modifications:

- Add SslHandler.renegotiate() so that a user can initiate TLS
  renegotiation and get the future that's notified on completion
- Make SslHandler.handshakeFuture() return the future for the most
  recent handshake so that a user can get the future of the last
  renegotiation
- Add the test for renegotiation to SocketSslEchoTest

Result:

Both client-initiated and server-initiated renegotiations are now
supported properly.
2014-12-11 18:06:16 +09:00
Trustin Lee
b653491a9f Make SslHandler work when autoRead is turned off
Related: #2958

Motivation:

SslHandler currently does not issue a read() request when it is
handshaking. It makes a connection with autoRead off stall, because a
user's read() request can be used to read the handshake response which
is invisible to the user.

Modifications:

- SslHandler now issues a read() request when:
  - the current handshake is in progress and channelReadComplete() is
    invoked
  - the current handshake is complete and a user issued a read() request
    during handshake
- Rename flushedBeforeHandshakeDone to flushedBeforeHandshake for
  consistency with the new variable 'readDuringHandshake'

Result:

SslHandler should work regardless whether autoRead is on or off.
2014-12-11 17:58:49 +09:00
Scott Mitchell
3c2b5c96fd Fix build issue from 5edb708
Motivation:
Commit 5edb708 introduced a build failure.

Modifications:
Add missing imports so build can complete.

Result:
Compilation error no more.
2014-12-09 13:29:37 -05:00
Trustin Lee
5edb7083b6 Call ctx.flush() at least once in ChunkedWriteHandler.flush()
Related: #3219

Motivation:

ChunkedWriteHandler.flush() does not call ctx.flush() when channel is
not writable. This can be a problem when other handler / non-Netty
thread writes messages simultaneously, because
ChunkedWriteHandler.flush() might have no chance to observe
channel.isWritable() returns true and thus the channel is never flushed.

Modifications:

- Ensure that ChunkedWriteHandler.flush() calls ctx.flush() at least
  once.

Result:

A stall connection issue, that occurs when certain combination of
handlers exist in a pipeline, has been fixed. (e.g. SslHandler and
ChunkedWriteHandler)
2014-12-09 18:17:56 +09:00
Trustin Lee
0f8ccbb1ac Overall clean-up of the initial SniHandler/DomainNameMapping work
- Parameterize DomainNameMapping to make it useful for other use cases
  than just mapping to SslContext
- Move DomainNameMapping to io.netty.util
- Clean-up the API documentation
- Make SniHandler.hostname and sslContext volatile because they can be
  accessed by non-I/O threads
2014-12-04 18:23:24 +09:00
Trustin Lee
7d8873a490 Fix dependency issues with hamcrest
Motivation:

We use 3 (!) libraries to build mock objects - easymock, mockito, jmock.
Mockito and jMock pulls in the different versions of Hamcrest, and it
conflicts with the version pulled by jUnit.

Modifications:

- Replace mockito-all with mockito-core to avoid pulling in outdated
  jUnit and Hamcrest
- Exclude junit-dep when pulling in jmock-junit4, because it pulls an
  outdated Hamcrest version
- Pull in the hamcrest-library version used by jUnit explicitly

Result:

No more dependency hell that results in NoSuchMethodError during the
tests
2014-12-04 18:00:02 +09:00
Sun Ning
b438ea4758 Added support for SSL Server Name Indication.
Motivation:

When we need to host multiple server name with a single IP, it requires
the server to support Server Name Indication extension to serve clients
with proper certificate. So the SniHandler will host multiple
SslContext(s) and append SslHandler for requested hostname.

Modification:

* Added SniHandler to host multiple certifications in a single server
* Test case

Result:

User could use SniHandler to host multiple certifcates at a time.
It's server-side only.
2014-12-03 10:21:54 +01:00
Ronald Chen
ae4017e76a replaced broken &lt with < and same for gt 2014-11-29 19:36:30 +01:00
Graham Edgecombe
e7ad9fed77 Use Triple DES in JdkSslContext cipher suite list.
Motivation:

JdkSslContext used SSL_RSA_WITH_DES_CBC_SHA in its cipher suite list.
OpenSslServerContext used DES-CBC3-SHA in the same place in its cipher suite
list, which is equivalent to SSL_RSA_WITH_3DES_EDE_CBC_SHA.

This means the lists were out of sync. Furthermore, using
SSL_RSA_WITH_DES_CBC_SHA is not desirable as it uses DES, a weak cipher. Triple
DES should be used instead.

Modifications:

Replace SSL_RSA_WITH_DES_CBC_SHA with SSL_RSA_WITH_3DES_EDE_CBC_SHA in
JdkSslContext.

Result:

The JdkSslContext and OpenSslServerContext cipher suite lists are now in sync.
Triple DES is used instead of DES, which is stronger.
2014-11-27 08:15:47 +01:00
Trustin Lee
3fe5ba1dd4 Remove or de-prioritize RC4 from default cipher suites
Motivation:

RC4 is not a recommended cipher suite anymore, as the recent research
reveals, such as:

- http://www.isg.rhul.ac.uk/tls/

Modifications:

- Remove most RC4 cipher suites from the default cipher suites
- For backward compatibility, leave RC4-SHA, while de-prioritizing it

Result:

Potentially safer default
2014-11-25 17:17:28 +09:00
Idel Pivnitskiy
3d200085a4 Small performance improvements
Motivation:

Found performance issues via FindBugs and PMD.

Modifications:

- Removed unnecessary boxing/unboxing operations in DefaultTextHeaders.convertToInt(CharSequence) and DefaultTextHeaders.convertToLong(CharSequence). A boxed primitive is created from a string, just to extract the unboxed primitive value.
- Added a static modifier for DefaultHttp2Connection.ParentChangedEvent class. This class is an inner class, but does not use its embedded reference to the object which created it. This reference makes the instances of the class larger, and may keep the reference to the creator object alive longer than necessary.
- Added a static compiled Pattern to avoid compile it each time it is used when we need to replace some part of authority.
- Improved using of StringBuilders.

Result:

Performance improvements.
2014-11-20 00:58:35 -05:00
Trustin Lee
3ddac6adff Add ApplicationProtocolConfig.DISABLED
Motivation:

When ALPN/NPN is disabled, a user has to instantiate a new
ApplicationProtocolConfig with meaningless parameters.

Modifications:

- Add ApplicationProtocolConfig.DISABLED, the singleton instance
- Reject the constructor calls with Protocol.NONE, which doesn't make
  much sense because a user should use DISABLED instead.

Result:

More user-friendly API when ALPN/NPN is not needed by a user.
2014-10-31 14:15:43 +09:00
Trustin Lee
a6a42d2f19 Add back the removed deprecated methods in SslContext
Motivation:

Previous backport removed the old methods and constructors. They should
not be removed in 4.x but just deprecated in favor of the new methods
and constructors.

Modifications:

Add back the removed methods and constructors in SslContext and its
subtypes for backward compatibility.

Result:

Backward compatibility issues fixed.
2014-10-31 14:15:33 +09:00
Trustin Lee
8f3904f6dc Code clean-up
- Fix the inspector warnings
- Fix the infinite recursion in SslContext.newClientContext()
- Fix Javadoc errors
2014-10-31 14:15:25 +09:00
Scott Mitchell
56b8bb30b2 Backport ALPN and Mutual Auth SSL
Motivation:

Improvements were made on the main line to support ALPN and mutual
authentication for TLS. These should be backported.

Modifications:

- Backport commits from the master branch
  - f8af84d599
  - e74c8edba3

Result:

Support for ALPN and mutual authentication.
2014-10-31 14:15:12 +09:00
Scott Mitchell
e73f32b52d SslHander wrap conditional direct buffer allocation
Motivation:
The SslHandler currently forces the use of a direct buffer for the input to the SSLEngine.wrap(..) operation. This allocation may not always be desired and should be conditionally done.

Modifications:
- Use the pre-existing wantsDirectBuffer variable as the condition to do the conversion.

Result:
- An allocation of a direct byte buffer and a copy of data is now not required for every SslHandler wrap operation.
2014-10-30 09:26:15 +01:00
Norman Maurer
1914b77c71 [maven-release-plugin] prepare for next development iteration 2014-10-29 11:48:40 +01:00
Norman Maurer
c170e7df3f [maven-release-plugin] prepare release netty-4.0.24.Final 2014-10-29 11:47:19 +01:00
Scott Mitchell
8db8aca1e7 SslHandler wrap memory leak
Motivation:
The SslHandler wrap method requires that a direct buffer be passed to the SSLEngine.wrap() call. If the ByteBuf parameter does not have an underlying direct buffer then one is allocated in this method, but it is not released.

Modifications:
- Release the direct ByteBuffer only accessible in the scope of SslHandler.wrap

Result:
Memory leak in SslHandler.wrap is fixed.
2014-10-28 06:13:06 +01:00
Trustin Lee
6150de5eb2 Disable SSLv3 to avoid POODLE vulnerability
Related: #3031

Motivation:

The only way to protect ourselves from POODLE vulnerability in Java for
now is to disable SSLv3.

- http://en.wikipedia.org/wiki/POODLE
- https://blogs.oracle.com/security/entry/information_about_ssl_poodle_vulnerability

Modifivation:

Disable SSLv3 in SslContext implementations

Result:

Prevent POODLE vulnerability when a user used SslContext with the
default configuration
2014-10-21 14:01:04 +09:00
kerr
a57408ebfc Change the operator order of TrafficCounter to calculate the throughput to get the correct result
Motivation:
Currently the last read/write throughput is calculated by first division,this will be 0 if the last read/write bytes < interval,change the order will get the correct result

Modifications:
Change the operator order from first do division to multiplication

Result:
Get the correct result instead of 0 when bytes are smaller than interval
2014-10-01 06:45:17 +02:00
Frederic Bregier
9479cdd0d4 Fix overriden method not called in TrafficShaping
Motivation:
handlerAdded and handlerRemoved were overriden but super was never
called, while it should.
Also add one missing information in the toString method.

Modifications:
Add the super corresponding call, and add checkInterval to the
toString() method

Result;
super method calls are correctly passed to the super implementation
part.
2014-09-20 20:41:00 +02:00
Wolfgang Profer
aa376b51de Fixed FingerprintTrustManagerFactory constructor
Motivation:

When constructing a FingerprintTrustManagerFactory from an Iterable of Strings, the fingerprints were correctly parsed but never added to the result array. The constructed FingerprintTrustManagerFactory consequently fails to validate any certificate.

Modifications:

I added a line to add each converted SHA-1 certificate fingerprint to the result array which then gets passed on to the next constructor.

Result:

Certificate fingerprints passed to the constructor are now correctly added to the array of valid fingerprints. The resulting FingerprintTrustManagerFactory object correctly validates certificates against the list of specified fingerprints.
2014-09-19 19:49:19 +02:00
Brad Fritz
17e55c7d7b Correct javadoc typo in IdleStateHandler example code 2014-09-10 20:55:25 +02:00
Trustin Lee
7710e7da44 [maven-release-plugin] prepare for next development iteration 2014-08-16 03:02:02 +09:00
Trustin Lee
208198c0cb [maven-release-plugin] prepare release netty-4.0.23.Final 2014-08-16 03:01:57 +09:00
Trustin Lee
a2d508711d [maven-release-plugin] prepare for next development iteration 2014-08-14 09:41:33 +09:00
Trustin Lee
3051db9d59 [maven-release-plugin] prepare release netty-4.0.22.Final 2014-08-14 09:41:28 +09:00
fredericBregier
ef3c030013 [#2722] Improve Traffic Shaping Handling
Motivation:
Currently Traffic Shaping is using 1 timer only and could lead to
"partial" wrong bandwidth computation when "short" time occurs between
adding used bytes and when the TrafficCounter updates itself and finally
when the traffic is computed.
Indeed, the TrafficCounter is updated every x delay and it is at the
same time saved into "lastXxxxBytes" and set to 0. Therefore, when one
request the counter, it first updates the TrafficCounter with the added
used bytes. If this value is set just before the TrafficCounter is
updated, then the bandwidth computation will use the TrafficCounter with
a "0" value (this value being reset once the delay occurs). Therefore,
the traffic shaping computation is wrong in rare cases.

Secondly the traffic shapping should avoid if possible the "Timeout"
effect by not stopping reading or writing more than a maxTime, this
maxTime being less than the TimeOut limit.

Thirdly the traffic shapping in read had an issue since the readOp was
not set but should, turning in no read blocking from socket point of
view. (see #2696)

Take into account setAutoRead(boolean) setting directly
by the user in the program external to this handler.

Modifications:
The TrafficCounter has 2 new methods that compute the time to wait
according to read or write) using in priority the currentXxxxBytes (as
before), but could used (if current is at 0) the lastXxxxxBytes, and
therefore having more chance to take into account the real traffic.

Moreover the Handler could change the default "max time to wait", which
is by default set to half of "standard" Time Out (30s:2 = 15s).

Finally we add the setAutoRead(boolean) accordingly to the situation, as
proposed in #2696 (the original pull request is in error for unknown
reason so this merge).

Result:
The Traffic Shaping is better take into account (no 0 value when it
shouldn't) and it tries to not block traffic more than Time Out event.

Moreover the read is really stopped from socket point of view.

This version is similar to #2388 and #2450.
This version is for V4.0, and includes the #2696 pull request to ease
the merge process.

The test minimizes time check by reducing to 66ms steps (50s total).
2014-08-13 01:29:58 +02:00
Norman Maurer
8f019ae4fa [#2752] Add PendingWriteQueue for queue up writes
Motivation:

Sometimes ChannelHandler need to queue writes to some point and then process these. We currently have no datastructure for this so the user will use an Queue or something like this. The problem is with this Channel.isWritable() will not work as expected and so the user risk to write to fast. That's exactly what happened in our SslHandler. For this purpose we need to add a special datastructure which will also take care of update the Channel and so be sure that Channel.isWritable() works as expected.

Modifications:

- Add PendingWriteQueue which can be used for this purpose
- Make use of PendingWriteQueue in SslHandler

Result:

It is now possible to queue writes in a ChannelHandler and still have Channel.isWritable() working as expected. This also fixes #2752.
2014-08-12 06:38:10 +02:00
Trustin Lee
d0e4a85830 Clean-up d9cccccbb3
- Revert irrelevant formatting changes
- Rename resource files
  - Add .pem
  - Remove 'netty' from names
2014-08-04 10:53:39 -07:00
Peter Schulz
d9cccccbb3 [#2718] Added private key decryption to JDK SSL server context.
Motivation:

Currently it is not possible to load an encrypted private key when
creating a JDK based SSL server context.

Modifications:

- Added static method to JdkSslServerContext which handles key spec generation for (encrypted) private keys and make use of it.
-Added tests for creating a SSL server context based on a (encrypted)
private key.

Result:

It is now possible to create a JDK based SSL server context with an
encrypted (password protected) private key.
2014-08-04 14:19:48 +02:00
Norman Maurer
e25891964c [#2675] Replace synchronization performed on util.concurrent instance in TrafficCounter
Motivation:

Message from FindBugs:
This method performs synchronization an object that is an instance of a class from the java.util.concurrent package (or its subclasses). Instances of these classes have their own concurrency control mechanisms that are orthogonal to the synchronization provided by the Java keyword synchronized. For example, synchronizing on an AtomicBoolean will not prevent other threads from modifying the AtomicBoolean.
Such code may be correct, but should be carefully reviewed and documented, and may confuse people who have to maintain the code at a later date.

Modification:

Use synchronized(this)

Result:

Less confusing code
2014-07-21 08:21:27 +02:00
Idel Pivnitskiy
dd026eb60a Fix NPE problems
Motivation:

Now Netty has a few problems with null values.

Modifications:

- Check File in DiskFileUpload.toString().
If File is null we will get NPE when calling toString() method.
- Check Result<String> in MqttDecoder.decodeConnectionPayload(...).
- Check Unsafe before calling unsafe.getClass() in PlatformDependent0 static block.
- Removed unnecessary null check in WebSocket08FrameEncoder.encode(...).
Because msg.content() can not return null.
- Removed unnecessary null checks in ConcurrentHashMapV8.removeTreeNode(TreeNode<K,V>).
- Removed unnecessary null check in OioDatagramChannel.doReadMessages(List<Object>).
Because tmpPacket.getSocketAddress() always returns new SocketAddress instance.
- Removed unnecessary null check in OioServerSocketChannel.doReadMessages(List<Object>).
Because socket.accept() always returns new Socket instance.
- Pass Unpooled.buffer(0) instead of null inside CloseWebSocketFrame(boolean, int) constructor.
If we will pass null we will get NPE in super class constructor.
- Added throw new IllegalStateException in GlobalEventExecutor.awaitInactivity(long, TimeUnit) if it will be called before GlobalEventExecutor.execute(Runnable).
Because now we will get NPE. IllegalStateException will be better in this case.
- Fixed null check in OpenSslServerContext.setTicketKeys(byte[]).
Now we throw new NPE if byte[] is not null.

Result:

Added new null checks when it is necessary, removed unnecessary null checks and fixed some NPE problems.
2014-07-20 12:56:21 +02:00
Trustin Lee
0a8ff3b52d Fix most inspector warnings
Motivation:

It's good to minimize potentially broken windows.

Modifications:

Fix most inspector warnings from our profile
Update IntObjectHashMap

Result:

Cleaner code
2014-07-02 20:21:30 +09:00
Norman Maurer
e8f4def2a3 [maven-release-plugin] prepare for next development iteration 2014-06-30 14:31:08 +02:00
Norman Maurer
25e3c8ce3d [maven-release-plugin] prepare release netty-4.0.21.Final 2014-06-30 14:29:15 +02:00
Trustin Lee
fb538ea532 Refactor FastThreadLocal to simplify TLV management
Motivation:

When Netty runs in a managed environment such as web application server,
Netty needs to provide an explicit way to remove the thread-local
variables it created to prevent class loader leaks.

FastThreadLocal uses different execution paths for storing a
thread-local variable depending on the type of the current thread.
It increases the complexity of thread-local removal.

Modifications:

- Moved FastThreadLocal and FastThreadLocalThread out of the internal
  package so that a user can use it.
- FastThreadLocal now keeps track of all thread local variables it has
  initialized, and calling FastThreadLocal.removeAll() will remove all
  thread-local variables of the caller thread.
- Added FastThreadLocal.size() for diagnostics and tests
- Introduce InternalThreadLocalMap which is a mixture of hard-wired
  thread local variable fields and extensible indexed variables
- FastThreadLocal now uses InternalThreadLocalMap to implement a
  thread-local variable.
- Added ThreadDeathWatcher.unwatch() so that PooledByteBufAllocator
  tells it to stop watching when its thread-local cache has been freed
  by FastThreadLocal.removeAll().
- Added FastThreadLocalTest to ensure that removeAll() works
- Added microbenchmark for FastThreadLocal and JDK ThreadLocal
- Upgraded to JMH 0.9

Result:

- A user can remove all thread-local variables Netty created, as long as
  he or she did not exit from the current thread. (Note that there's no
  way to remove a thread-local variable from outside of the thread.)
- FastThreadLocal exposes more useful operations such as isSet() because
  we always implement a thread local variable via InternalThreadLocalMap
  instead of falling back to JDK ThreadLocal.
- FastThreadLocalBenchmark shows that this change improves the
  performance of FastThreadLocal even more.
2014-06-19 21:08:16 +09:00
Norman Maurer
b737d631f1 [maven-release-plugin] prepare for next development iteration 2014-06-12 16:20:52 +02:00
Norman Maurer
1709113a1f [maven-release-plugin] prepare release netty-4.0.20.Final 2014-06-12 16:14:48 +02:00
Norman Maurer
76043bc8c8 Make use of an array to store FastThreadLocals and so allow to also use it in PooledByteBufAllocator that is instanced by users.
Motivation:
Allow to make use of our new FastThreadLocal whereever possible

Modification:
Make use of an array to store FastThreadLocals and so allow to also use it in PooledByteBufAllocator that is instanced by users.
The maximal size of the array is configurable per system property to allow to tune it if needed. As default we use 64 entries which should be good enough.

Result:
More flexible usage of FastThreadLocal
2014-06-12 15:43:20 +02:00
DhanaRaj Durairaj
f3b1816db8 [#2494] Fix data curruption by ChannelTrafficShapingHandler
Motivation:
ChannelTrafficShapingHandler may corrupt inbound data stream by
scheduling the fireChannelRead event.

Modification:
Always call fireChannelRead(...) and only suspend reads after it

Result:
No more data corruption
2014-06-03 08:38:23 +02:00
Trustin Lee
10a5270e79 Fix NPE 2014-05-22 10:29:31 +09:00
Trustin Lee
a4b1e68b9d Future compatibility with TLS ALPN
Motivation:

According to TLS ALPN draft-05, a client sends the list of the supported
protocols and a server responds with the selected protocol, which is
different from NPN.  Therefore, ApplicationProtocolSelector won't work
with ALPN

Modifications:

- Use Iterable<String> to list the supported protocols on the client
  side, rather than using ApplicationProtocolSelector
- Remove ApplicationProtocolSelector

Result:

Future compatibility with TLS ALPN
2014-05-22 10:03:16 +09:00
Trustin Lee
9afd8f0c26 Fix NPE in OpenSslEngine 2014-05-21 20:02:49 +09:00
Trustin Lee
b436edf114 Escape a colon in protocol names 2014-05-21 17:44:47 +09:00
Trustin Lee
8953f5a6cb Add unified NextProtoNego extension support to SslContext
Motivation:

- OpenSslEngine and JDK SSLEngine (+ Jetty NPN) have different APIs to
  support NextProtoNego extension.
  - It is impossible to configure NPN with SslContext when the provider
    type is JDK.

Modification:

- Implement NextProtoNego extension by overriding the behavior of
  SSLSession.getProtocol() for both OpenSSLEngine and JDK SSLEngine.
  - SSLEngine.getProtocol() returns a string delimited by a colon (':')
    where the first component is the transport protosol (e.g. TLSv1.2)
    and the second component is the name of the application protocol
- Remove the direct reference of Jetty NPN classes from the examples
- Add SslContext.newApplicationProtocolSelector

Result:

- A user can now use both JDK SSLEngine and OpenSslEngine for NPN-based
  protocols such as HTTP2 and SPDY
2014-05-21 17:25:55 +09:00
Trustin Lee
a6541e40d4 Work around the JVM crash that occurs when cipher suite uses GCM
Motivation:

For an unknown reason, JVM of JDK8 crashes intermittently when
SslHandler feeds a direct buffer to SSLEngine.unwrap() *and* the current
cipher suite has GCM (Galois/Counter Mode) enabled.

Modifications:

Convert the inbound network buffer to a heap buffer when the current
cipher suite is using GCM.

Result:

JVM does not crash anymore.
2014-05-19 11:47:39 +09:00
Trustin Lee
927e28cb20 Reduce memory usage of SslHandler when OpenSslEngine is in use
Motivation:

JDK's SSLEngine.wrap() requires the output buffer to be always as large as MAX_ENCRYPTED_PACKET_LENGTH even if the input buffer contains small number of bytes.  Our OpenSslEngine implementation does not have such wasteful behaviot.

Modifications:

If the current SSLEngine is OpenSslEngine, allocate as much as only needed.

Result:

Less peak memory usage.
2014-05-18 05:10:58 +09:00
Trustin Lee
d6262a3542 Fix buffer leaks during PEM to KeyStore conversion 2014-05-18 04:30:05 +09:00
Trustin Lee
79f2f332ec Fix JDK 8 compatibility issue with OpenJdkSelfSignedCertGenerator
- X509CertInfo.setSubject/setIssuer() requires X500Name instead of
  CertificateSubjectName/CertificateIssuerName.
2014-05-18 03:48:07 +09:00
Trustin Lee
097ea8b5b5 Optimize SslHandler in an OpenSslEngine-friendly way
Motivation:

Previous fix for the OpenSslEngine compatibility issue (#2216 and
18b0e95659) was to feed SSL records one by
one to OpenSslEngine.unwrap().  It is not optimal because it will result
in more JNI calls.

Modifications:

- Do not feed SSL records one by one.
- Feed as many records as possible up to MAX_ENCRYPTED_PACKET_LENGTH
- Deduplicate MAX_ENCRYPTED_PACKET_LENGTH definitions

Result:

- No allocation of intemediary arrays
- Reduced number of calls to SSLEngine and thus its underlying JNI calls
- A tad bit increase in throughput, probably reverting the tiny drop
  caused by 18b0e95659
2014-05-18 03:32:35 +09:00
Trustin Lee
c58f28dfdd Add an OpenSslEngine and the universal API for enabling SSL
Motivation:

Some users already use an SSLEngine implementation in finagle-native. It
wraps OpenSSL to get higher SSL performance.  However, to take advantage
of it, finagle-native must be compiled manually, and it means we cannot
pull it in as a dependency and thus we cannot test our SslHandler
against the OpenSSL-based SSLEngine.  For an instance, we had #2216.

Because the construction procedures of JDK SSLEngine and OpenSslEngine
are very different from each other, we also need to provide a universal
way to enable SSL in a Netty application.

Modifications:

- Pull netty-tcnative in as an optional dependency.
  http://netty.io/wiki/forked-tomcat-native.html
- Backport NativeLibraryLoader from 4.0
- Move OpenSSL-based SSLEngine implementation into our code base.
  - Copied from finagle-native; originally written by @jpinner et al.
  - Overall cleanup by @trustin.
- Run all SslHandler tests with both default SSLEngine and OpenSslEngine
- Add a unified API for creating an SSL context
  - SslContext allows you to create a new SSLEngine or a new SslHandler
    with your PKCS#8 key and X.509 certificate chain.
  - Add JdkSslContext and its subclasses
  - Add OpenSslServerContext
- Add ApplicationProtocolSelector to ensure the future support for NPN
  (NextProtoNego) and ALPN (Application Layer Protocol Negotiation) on
  the client-side.
- Add SimpleTrustManagerFactory to help a user write a
  TrustManagerFactory easily, which should be useful for those who need
  to write an alternative verification mechanism. For example, we can
  use it to implement an unsafe TrustManagerFactory that accepts
  self-signed certificates for testing purposes.
- Add InsecureTrustManagerFactory and FingerprintTrustManager for quick
  and dirty testing
- Add SelfSignedCertificate class which generates a self-signed X.509
  certificate very easily.
- Update all our examples to use SslContext.newClient/ServerContext()
- SslHandler now logs the chosen cipher suite when handshake is
  finished.

Result:

- Cleaner unified API for configuring an SSL client and an SSL server
  regardless of its internal implementation.
- When native libraries are available, OpenSSL-based SSLEngine
  implementation is selected automatically to take advantage of its
  performance benefit.
- Examples take advantage of this modification and thus are cleaner.
2014-05-18 02:54:23 +09:00
Trustin Lee
bb787f57b5 Preparation for porting OpenSSL support in 3.10
- Add licenses and dependencies
2014-05-17 20:01:47 +09:00
Norman Maurer
a597087a9f [maven-release-plugin] prepare for next development iteration 2014-04-30 15:40:54 +02:00
Norman Maurer
b562148e2d [maven-release-plugin] prepare release netty-4.0.19.Final 2014-04-30 15:40:31 +02:00
Norman Maurer
76355a28b0 Correctly write pending data after ssl handshake completes. Related to [#2437]
Motivation:
When writing data from a server before the ssl handshake completes may not be written at all to the remote peer
if nothing else is written after the handshake was done.

Modification:
Correctly try to write pending data after the handshake was complete

Result:
Correctly write out all pending data
2014-04-30 14:23:03 +02:00
Trustin Lee
bdab831ba5 Undeprecate deregister() and chanelUnregistered()
Motivation:

As discussed in #2250, it will become much less complicated to implement
deregistration and reregistration of a channel once #2250 is resolved.
Therefore, there's no need to deprecate deregister() and
channelUnregistered().

Modification:

- Undeprecate deregister() and channelUnregistered()
- Remove SuppressWarnings annotations where applicable

Result:

We (including @jakobbuchgraber) are now ready to play with #2250 at
master
2014-04-25 16:53:59 +09:00
Trustin Lee
f175ce0653 Feed only a single SSL record to SSLEngine.unwrap()
Motivation:

Some SSLEngine implementations violate the contract and raises an
exception when SslHandler feeds an input buffer that contains multiple
SSL records to SSLEngine.unwrap(), while the expected behavior is to
decode the first record and return.

Modification:

- Modify SslHandler.decode() to keep the lengths of each record and feed
  SSLEngine.unwrap() record by record to work around the forementioned
  issue.
- Rename unwrap() to unwrapMultiple() and unwrapNonApp()
- Rename unwrap0() to unwrapSingle()

Result:

SslHandler now works OpenSSLEngine from finagle-native.  Performance
impact remains unnoticeable.  Slightly better readability. Fixes #2116.
2014-04-20 17:33:36 +09:00
Trustin Lee
8008530498 Work around an Android SSLEngine issue
Motivation:

Some Android SSLEngine implementations skip FINISHED handshake status
and go straightly into NOT_HANDSHAKING.  This behavior blocks SslHandler
from notifying its handshakeFuture, because we do the notification when
SSLEngine enters the FINISHED state.

Modification:

When the current handshake state is NOT_HANDSHAKING and the
handshakeFuture is not fulfilled yet, treat NOT_HANDSHAKING as FINISHED.

Result:

Better Android compatibility - fixes #1823
2014-04-18 18:01:40 +09:00
Norman Maurer
012166803a [#2353] Use a privileged block to get ClassLoader and System property if needed
Motivation:
When using System.getProperty(...) and various methods to get a ClassLoader it will fail when a SecurityManager is in place.

Modifications:
Use a priveled block if needed. This work is based in the PR #2353 done by @anilsaldhana .

Result:
Code works also when SecurityManager is present
2014-04-08 13:59:03 +02:00
Norman Maurer
7966c42636 [#2358] SslHandler.safeClose(...) may not notify the ChannelPromise
Motivation:
In SslHandler.safeClose(...) we attach a ChannelFutureListener to the flushFuture and will notify the ChannelPromise which was used for close(...) in it. The problem here is that we only call ChannelHandlerContext.close(ChannelPromise) if Channel.isActive() is true and otherwise not notify it at all. We should just call ChannelHandlerContext.close(ChannelPromise) in all cases.

Modifications:
Always call ChannelHandlerContext.close(ChannelPromise) in the ChannelFutureListeiner

Result:
ChannelPromise used for close the Channel is notified in all cases
2014-04-03 13:29:54 +02:00
Norman Maurer
816165c96a [maven-release-plugin] prepare for next development iteration 2014-04-01 07:21:40 +02:00
Norman Maurer
1512a4dcca [maven-release-plugin] prepare release netty-4.0.18.Final 2014-04-01 07:20:16 +02:00
Ian Barfield
5f9993ee1f Deleting redundant needsFlush boolean
Motivation:

In ChunkedWriteHandler, there is a redundant variable that servers
no purpose. It implies that under some conditions you might not want
to flush.

Modifications:

Removed the variable and the if condition that read it. The boolean
was always true so just removing the if statement was fine.

Result:

Slightly less misleading code.
2014-03-29 20:21:33 +01:00
Norman Maurer
cf2c8b40ae Replace usage of System.currentTimeMillis() with System.nanoTime()
Motivation:

Currently we use System.currentTimeMillis() in our timeout handlers this is bad
for various reasons like when the clock adjusts etc.

Modifications:

Replace System.currentTimeMillis() with System.nanoTime()

Result:

More robust timeout handling
2014-03-18 16:05:51 +09:00
Norman Maurer
ef69c4592b Revert "ipfilter implementation for netty 4/5 [#2129]"
Motivation:
ipfilter implementation may need a bit more time to stabilize.

Modifications:
Revert the commit that added it to 4.0 branch (it is still included in 4.1 and master)

Result:
ipfilter will not be included in the next 4.0.x release (just as before9
2014-03-12 06:26:50 +01:00
Jakob Buchgraber
ae20f278ff ipfilter implementation for netty 4/5 [#2129] 2014-03-10 20:46:05 +01:00
Norman Maurer
6513859a79 [#2261] Correct javadoc of ChunkedInput 2014-03-03 07:03:58 +01:00
Norman Maurer
ccd135df01 [maven-release-plugin] prepare for next development iteration 2014-02-24 15:39:26 +01:00
Norman Maurer
33587eb183 [maven-release-plugin] prepare release netty-4.0.17.Final 2014-02-24 15:37:31 +01:00
Trustin Lee
5c27273885 Avoid unnecessary IllegalStateException in ChunkedWriteHandler
Motivation:
ChunkedWriteHandler can sometimes fail to write the last chunk of a ChunkedInput due to an I/O error.  Subsequently, the ChunkedInput's associated promise is marked as failure and the connection is closed.  When the connection is closed, ChunkedWriteHandler attempts to clean up its message queue and to mark their promises as success or failure.  However, because the promise of the ChunkedInput, which was consumed completely yet failed to be written, is already marked as failure, the attempt to mark it as success fails, leading a WARN level log.

Modification:
Use trySuccess() instead of setSuccess() so that the attempt to mark a ChunkedInput as success does not raise an exception even if the promise is already done.

Result:
Fixes #2249
2014-02-20 17:14:53 -08:00
Norman Maurer
66e2bb1e75 [maven-release-plugin] prepare for next development iteration 2014-02-19 03:41:24 +01:00
Norman Maurer
c466bb803d [maven-release-plugin] prepare release netty-4.0.16.Final 2014-02-19 03:36:54 +01:00
Trustin Lee
1ddf91828d Do not use String.format() for log message generation
- It's slow.
2014-02-13 19:33:36 -08:00
Trustin Lee
c4c71e6d28 Fix the potential copyright issue in SocksCommonUtils
- Add StringUtil.toHexString() methods which are based on LoggingHandler's lookup table implementation, and use it wherever possible
2014-02-06 15:00:06 -08:00
Norman Maurer
37e6588845 [#2159] Not fail the ChannelPromise with WriteTimeoutException to prevent warning 2014-01-30 07:02:06 +01:00
Norman Maurer
d67184b488 [maven-release-plugin] prepare for next development iteration 2014-01-21 08:18:32 +01:00
Norman Maurer
287515210d [maven-release-plugin] prepare release netty-4.0.15.Final 2014-01-21 08:18:26 +01:00
Trustin Lee
bc21443ea9 Fix a regression in SslHandler where delegated tasks run in a different executor makes the session hang
- Fixes #2098
- Deprecate specifying an alternative Executor for delegated tasks for SslHandler
2014-01-09 18:08:05 +09:00