Commit Graph

417 Commits

Author SHA1 Message Date
Norman Maurer
02e7e53cbb [#2752] Add PendingWriteQueue for queue up writes
Motivation:

Sometimes ChannelHandler need to queue writes to some point and then process these. We currently have no datastructure for this so the user will use an Queue or something like this. The problem is with this Channel.isWritable() will not work as expected and so the user risk to write to fast. That's exactly what happened in our SslHandler. For this purpose we need to add a special datastructure which will also take care of update the Channel and so be sure that Channel.isWritable() works as expected.

Modifications:

- Add PendingWriteQueue which can be used for this purpose
- Make use of PendingWriteQueue in SslHandler

Result:

It is now possible to queue writes in a ChannelHandler and still have Channel.isWritable() working as expected. This also fixes #2752.
2014-08-12 06:38:22 +02:00
Trustin Lee
8263a62014 Clean-up d9cccccbb3
- Revert irrelevant formatting changes
- Rename resource files
  - Add .pem
  - Remove 'netty' from names
2014-08-04 10:54:05 -07:00
Peter Schulz
20dffa8ee6 [#2718] Added private key decryption to JDK SSL server context.
Motivation:

Currently it is not possible to load an encrypted private key when
creating a JDK based SSL server context.

Modifications:

- Added static method to JdkSslServerContext which handles key spec generation for (encrypted) private keys and make use of it.
-Added tests for creating a SSL server context based on a (encrypted)
private key.

Result:

It is now possible to create a JDK based SSL server context with an
encrypted (password protected) private key.
2014-08-04 14:19:40 +02:00
Norman Maurer
c85319213a [#2675] Replace synchronization performed on util.concurrent instance in TrafficCounter
Motivation:

Message from FindBugs:
This method performs synchronization an object that is an instance of a class from the java.util.concurrent package (or its subclasses). Instances of these classes have their own concurrency control mechanisms that are orthogonal to the synchronization provided by the Java keyword synchronized. For example, synchronizing on an AtomicBoolean will not prevent other threads from modifying the AtomicBoolean.
Such code may be correct, but should be carefully reviewed and documented, and may confuse people who have to maintain the code at a later date.

Modification:

Use synchronized(this)

Result:

Less confusing code
2014-07-21 08:23:24 +02:00
Idel Pivnitskiy
b83df4c6b3 Fix NPE problems
Motivation:

Now Netty has a few problems with null values.

Modifications:

- Check HAProxyProxiedProtocol in HAProxyMessage constructor and throw NPE if it is null.
If HAProxyProxiedProtocol is null we will set AddressFamily as null. So we will get NPE inside checkAddress(String, AddressFamily) and it won't be easy to understand why addrFamily is null.
- Check File in DiskFileUpload.toString().
If File is null we will get NPE when calling toString() method.
- Check Result<String> in MqttDecoder.decodeConnectionPayload(...).
If !mqttConnectVariableHeader.isWillFlag() || !mqttConnectVariableHeader.hasUserName() || !mqttConnectVariableHeader.hasPassword() we will get NPE when we will try to create new instance of MqttConnectPayload.
- Check Unsafe before calling unsafe.getClass() in PlatformDependent0 static block.
- Removed unnecessary null check in WebSocket08FrameEncoder.encode(...).
Because msg.content() can not return null.
- Removed unnecessary null check in DefaultStompFrame(StompCommand) constructor.
Because we have this check in the super class.
- Removed unnecessary null checks in ConcurrentHashMapV8.removeTreeNode(TreeNode<K,V>).
- Removed unnecessary null check in OioDatagramChannel.doReadMessages(List<Object>).
Because tmpPacket.getSocketAddress() always returns new SocketAddress instance.
- Removed unnecessary null check in OioServerSocketChannel.doReadMessages(List<Object>).
Because socket.accept() always returns new Socket instance.
- Pass Unpooled.buffer(0) instead of null inside CloseWebSocketFrame(boolean, int) constructor.
If we will pass null we will get NPE in super class constructor.
- Added throw new IllegalStateException in GlobalEventExecutor.awaitInactivity(long, TimeUnit) if it will be called before GlobalEventExecutor.execute(Runnable).
Because now we will get NPE. IllegalStateException will be better in this case.
- Fixed null check in OpenSslServerContext.setTicketKeys(byte[]).
Now we throw new NPE if byte[] is not null.

Result:

Added new null checks when it is necessary, removed unnecessary null checks and fixed some NPE problems.
2014-07-20 12:55:22 +02:00
Trustin Lee
e167b02d52 [maven-release-plugin] prepare for next development iteration 2014-07-04 17:26:02 +09:00
Trustin Lee
ba50cb829b [maven-release-plugin] prepare release netty-4.1.0.Beta1 2014-07-04 17:25:54 +09:00
Trustin Lee
787663a644 [maven-release-plugin] rollback the release of netty-4.1.0.Beta1 2014-07-04 17:11:14 +09:00
Trustin Lee
83eae705e1 [maven-release-plugin] prepare release netty-4.1.0.Beta1 2014-07-04 17:02:17 +09:00
Norman Maurer
217fb0de05 [#2618] Introduce ChannelPromise.unvoid() and ChannelFuture.isVoid()
Motivation:

There is no way for a ChannelHandler to check if the passed in ChannelPromise for a write(...) call is a VoidChannelPromise. This is a problem as some handlers need to add listeners to the ChannelPromise which is not possible in the case of a VoidChannelPromise.

Modification:

- Introduce ChannelFuture.isVoid() which will return true if it is not possible to add listeners or wait on the result.
- Add ChannelPromise.unvoid() which allows to create a ChannelFuture out of a void ChannelFuture which supports all the operations.

Result:

It's now easy to write ChannelHandler implementations which also works when a void ChannelPromise is used.
2014-07-03 14:17:12 +02:00
Trustin Lee
d0912f2709 Fix most inspector warnings
Motivation:

It's good to minimize potentially broken windows.

Modifications:

Fix most inspector warnings from our profile
Update IntObjectHashMap

Result:

Cleaner code
2014-07-02 19:55:07 +09:00
Trustin Lee
085a61a310 Refactor FastThreadLocal to simplify TLV management
Motivation:

When Netty runs in a managed environment such as web application server,
Netty needs to provide an explicit way to remove the thread-local
variables it created to prevent class loader leaks.

FastThreadLocal uses different execution paths for storing a
thread-local variable depending on the type of the current thread.
It increases the complexity of thread-local removal.

Modifications:

- Moved FastThreadLocal and FastThreadLocalThread out of the internal
  package so that a user can use it.
- FastThreadLocal now keeps track of all thread local variables it has
  initialized, and calling FastThreadLocal.removeAll() will remove all
  thread-local variables of the caller thread.
- Added FastThreadLocal.size() for diagnostics and tests
- Introduce InternalThreadLocalMap which is a mixture of hard-wired
  thread local variable fields and extensible indexed variables
- FastThreadLocal now uses InternalThreadLocalMap to implement a
  thread-local variable.
- Added ThreadDeathWatcher.unwatch() so that PooledByteBufAllocator
  tells it to stop watching when its thread-local cache has been freed
  by FastThreadLocal.removeAll().
- Added FastThreadLocalTest to ensure that removeAll() works
- Added microbenchmark for FastThreadLocal and JDK ThreadLocal
- Upgraded to JMH 0.9

Result:

- A user can remove all thread-local variables Netty created, as long as
  he or she did not exit from the current thread. (Note that there's no
  way to remove a thread-local variable from outside of the thread.)
- FastThreadLocal exposes more useful operations such as isSet() because
  we always implement a thread local variable via InternalThreadLocalMap
  instead of falling back to JDK ThreadLocal.
- FastThreadLocalBenchmark shows that this change improves the
  performance of FastThreadLocal even more.
2014-06-19 21:13:55 +09:00
belliottsmith
2a2a21ec59 Introduce FastThreadLocal which uses an EnumMap and a predefined fixed set of possible thread locals
Motivation:
Provide a faster ThreadLocal implementation

Modification:
Add a "FastThreadLocal" which uses an EnumMap and a predefined fixed set of possible thread locals (all of the static instances created by netty) that is around 10-20% faster than standard ThreadLocal in my benchmarks (and can be seen having an effect in the direct PooledByteBufAllocator benchmark that uses the DEFAULT ByteBufAllocator which uses this FastThreadLocal, as opposed to normal instantiations that do not, and in the new RecyclableArrayList benchmark);

Result:
Improved performance
2014-06-13 10:56:18 +02:00
DhanaRaj Durairaj
eb27cd279c [#2494] Fix data curruption by ChannelTrafficShapingHandler
Motivation:
ChannelTrafficShapingHandler may corrupt inbound data stream by
scheduling the fireChannelRead event.

Modification:
Always call fireChannelRead(...) and only suspend reads after it

Result:
No more data corruption
2014-06-03 08:38:05 +02:00
Trustin Lee
2f8c13e85a Fix NPE 2014-05-22 10:29:24 +09:00
Trustin Lee
5ce410c69e Future compatibility with TLS ALPN
Motivation:

According to TLS ALPN draft-05, a client sends the list of the supported
protocols and a server responds with the selected protocol, which is
different from NPN.  Therefore, ApplicationProtocolSelector won't work
with ALPN

Modifications:

- Use Iterable<String> to list the supported protocols on the client
  side, rather than using ApplicationProtocolSelector
- Remove ApplicationProtocolSelector

Result:

Future compatibility with TLS ALPN
2014-05-22 10:03:02 +09:00
Trustin Lee
27b67e981e Fix NPE in OpenSslEngine 2014-05-21 20:02:40 +09:00
Trustin Lee
ac968fbf7b Escape a colon in protocol names 2014-05-21 17:45:18 +09:00
Trustin Lee
861ed1e7ad Add unified NextProtoNego extension support to SslContext
Motivation:

- OpenSslEngine and JDK SSLEngine (+ Jetty NPN) have different APIs to
  support NextProtoNego extension.
  - It is impossible to configure NPN with SslContext when the provider
    type is JDK.

Modification:

- Implement NextProtoNego extension by overriding the behavior of
  SSLSession.getProtocol() for both OpenSSLEngine and JDK SSLEngine.
  - SSLEngine.getProtocol() returns a string delimited by a colon (':')
    where the first component is the transport protosol (e.g. TLSv1.2)
    and the second component is the name of the application protocol
- Remove the direct reference of Jetty NPN classes from the examples
- Add SslContext.newApplicationProtocolSelector

Result:

- A user can now use both JDK SSLEngine and OpenSslEngine for NPN-based
  protocols such as HTTP2 and SPDY
2014-05-21 17:24:52 +09:00
Trustin Lee
4882377c27 Work around the JVM crash that occurs when cipher suite uses GCM
Motivation:

For an unknown reason, JVM of JDK8 crashes intermittently when
SslHandler feeds a direct buffer to SSLEngine.unwrap() *and* the current
cipher suite has GCM (Galois/Counter Mode) enabled.

Modifications:

Convert the inbound network buffer to a heap buffer when the current
cipher suite is using GCM.

Result:

JVM does not crash anymore.
2014-05-19 11:47:20 +09:00
Trustin Lee
ae61b12b9e Reduce memory usage of SslHandler when OpenSslEngine is in use
Motivation:

JDK's SSLEngine.wrap() requires the output buffer to be always as large as MAX_ENCRYPTED_PACKET_LENGTH even if the input buffer contains small number of bytes.  Our OpenSslEngine implementation does not have such wasteful behaviot.

Modifications:

If the current SSLEngine is OpenSslEngine, allocate as much as only needed.

Result:

Less peak memory usage.
2014-05-18 05:10:41 +09:00
Trustin Lee
28c390d86d Fix buffer leaks during PEM to KeyStore conversion 2014-05-18 04:29:50 +09:00
Trustin Lee
167a685a5f Fix JDK 8 compatibility issue with OpenJdkSelfSignedCertGenerator
- X509CertInfo.setSubject/setIssuer() requires X500Name instead of
  CertificateSubjectName/CertificateIssuerName.
2014-05-18 03:47:55 +09:00
Trustin Lee
9125060752 Optimize SslHandler in an OpenSslEngine-friendly way
Motivation:

Previous fix for the OpenSslEngine compatibility issue (#2216 and
18b0e95659) was to feed SSL records one by
one to OpenSslEngine.unwrap().  It is not optimal because it will result
in more JNI calls.

Modifications:

- Do not feed SSL records one by one.
- Feed as many records as possible up to MAX_ENCRYPTED_PACKET_LENGTH
- Deduplicate MAX_ENCRYPTED_PACKET_LENGTH definitions

Result:

- No allocation of intemediary arrays
- Reduced number of calls to SSLEngine and thus its underlying JNI calls
- A tad bit increase in throughput, probably reverting the tiny drop
  caused by 18b0e95659
2014-05-18 03:33:00 +09:00
Trustin Lee
b6c0c0c95f Add an OpenSslEngine and the universal API for enabling SSL
Motivation:

Some users already use an SSLEngine implementation in finagle-native. It
wraps OpenSSL to get higher SSL performance.  However, to take advantage
of it, finagle-native must be compiled manually, and it means we cannot
pull it in as a dependency and thus we cannot test our SslHandler
against the OpenSSL-based SSLEngine.  For an instance, we had #2216.

Because the construction procedures of JDK SSLEngine and OpenSslEngine
are very different from each other, we also need to provide a universal
way to enable SSL in a Netty application.

Modifications:

- Pull netty-tcnative in as an optional dependency.
  http://netty.io/wiki/forked-tomcat-native.html
- Backport NativeLibraryLoader from 4.0
- Move OpenSSL-based SSLEngine implementation into our code base.
  - Copied from finagle-native; originally written by @jpinner et al.
  - Overall cleanup by @trustin.
- Run all SslHandler tests with both default SSLEngine and OpenSslEngine
- Add a unified API for creating an SSL context
  - SslContext allows you to create a new SSLEngine or a new SslHandler
    with your PKCS#8 key and X.509 certificate chain.
  - Add JdkSslContext and its subclasses
  - Add OpenSslServerContext
- Add ApplicationProtocolSelector to ensure the future support for NPN
  (NextProtoNego) and ALPN (Application Layer Protocol Negotiation) on
  the client-side.
- Add SimpleTrustManagerFactory to help a user write a
  TrustManagerFactory easily, which should be useful for those who need
  to write an alternative verification mechanism. For example, we can
  use it to implement an unsafe TrustManagerFactory that accepts
  self-signed certificates for testing purposes.
- Add InsecureTrustManagerFactory and FingerprintTrustManager for quick
  and dirty testing
- Add SelfSignedCertificate class which generates a self-signed X.509
  certificate very easily.
- Update all our examples to use SslContext.newClient/ServerContext()
- SslHandler now logs the chosen cipher suite when handshake is
  finished.

Result:

- Cleaner unified API for configuring an SSL client and an SSL server
  regardless of its internal implementation.
- When native libraries are available, OpenSSL-based SSLEngine
  implementation is selected automatically to take advantage of its
  performance benefit.
- Examples take advantage of this modification and thus are cleaner.
2014-05-18 02:54:06 +09:00
Trustin Lee
2a6ed67efc Preparation for porting OpenSSL support in 3.10
- Add licenses and dependencies
2014-05-17 20:01:30 +09:00
Norman Maurer
0aea9eaed5 Correctly write pending data after ssl handshake completes. Related to [#2437]
Motivation:
When writing data from a server before the ssl handshake completes may not be written at all to the remote peer
if nothing else is written after the handshake was done.

Modification:
Correctly try to write pending data after the handshake was complete

Result:
Correctly write out all pending data
2014-04-30 14:23:18 +02:00
Trustin Lee
0fceef8ab6 Undeprecate deregister() and chanelUnregistered()
Motivation:

As discussed in #2250, it will become much less complicated to implement
deregistration and reregistration of a channel once #2250 is resolved.
Therefore, there's no need to deprecate deregister() and
channelUnregistered().

Modification:

- Undeprecate deregister() and channelUnregistered()
- Remove SuppressWarnings annotations where applicable

Result:

We (including @jakobbuchgraber) are now ready to play with #2250 at
master
2014-04-25 16:40:42 +09:00
Trustin Lee
d765f6b870 Synchronized between 4.1 and master (part 3)
Motivation:

4 and 5 were diverged long time ago and we recently reverted some of the
early commits in master.  We must make sure 4.1 and master are not very
different now.

Modification:

Fix found differences

Result:

4.1 and master got closer.
2014-04-25 16:17:59 +09:00
Trustin Lee
b9039eaa82 Synchronized between 4.1 and master again (part 2)
Motivation:
4 and 5 were diverged long time ago and we recently reverted some of the
early commits in master.  We must make sure 4.1 and master are not very
different now.

Modification:
Remove ChannelHandlerInvoker.writeAndFlush(...) and the related
implementations.

Result:
4.1 and master got closer.
2014-04-25 15:06:26 +09:00
Trustin Lee
db3709e652 Synchronized between 4.1 and master
Motivation:

4 and 5 were diverged long time ago and we recently reverted some of the
early commits in master.  We must make sure 4.1 and master are not very
different now.

Modification:

Fix found differences

Result:

4.1 and master got closer.
2014-04-25 00:38:02 +09:00
Trustin Lee
af9ab8b370 Feed only a single SSL record to SSLEngine.unwrap()
Motivation:

Some SSLEngine implementations violate the contract and raises an
exception when SslHandler feeds an input buffer that contains multiple
SSL records to SSLEngine.unwrap(), while the expected behavior is to
decode the first record and return.

Modification:

- Modify SslHandler.decode() to keep the lengths of each record and feed
  SSLEngine.unwrap() record by record to work around the forementioned
  issue.
- Rename unwrap() to unwrapMultiple() and unwrapNonApp()
- Rename unwrap0() to unwrapSingle()

Result:

SslHandler now works OpenSSLEngine from finagle-native.  Performance
impact remains unnoticeable.  Slightly better readability. Fixes #2116.
2014-04-20 17:33:04 +09:00
Trustin Lee
e9161147a5 Work around an Android SSLEngine issue
Motivation:

Some Android SSLEngine implementations skip FINISHED handshake status
and go straightly into NOT_HANDSHAKING.  This behavior blocks SslHandler
from notifying its handshakeFuture, because we do the notification when
SSLEngine enters the FINISHED state.

Modification:

When the current handshake state is NOT_HANDSHAKING and the
handshakeFuture is not fulfilled yet, treat NOT_HANDSHAKING as FINISHED.

Result:

Better Android compatibility - fixes #1823
2014-04-18 17:59:48 +09:00
Norman Maurer
88481131be [#2353] Use a privileged block to get ClassLoader and System property if needed
Motivation:
When using System.getProperty(...) and various methods to get a ClassLoader it will fail when a SecurityManager is in place.

Modifications:
Use a priveled block if needed. This work is based in the PR #2353 done by @anilsaldhana .

Result:
Code works also when SecurityManager is present
2014-04-08 14:12:25 +02:00
Norman Maurer
3eec26b0a2 [#2358] SslHandler.safeClose(...) may not notify the ChannelPromise
Motivation:
In SslHandler.safeClose(...) we attach a ChannelFutureListener to the flushFuture and will notify the ChannelPromise which was used for close(...) in it. The problem here is that we only call ChannelHandlerContext.close(ChannelPromise) if Channel.isActive() is true and otherwise not notify it at all. We should just call ChannelHandlerContext.close(ChannelPromise) in all cases.

Modifications:
Always call ChannelHandlerContext.close(ChannelPromise) in the ChannelFutureListeiner

Result:
ChannelPromise used for close the Channel is notified in all cases
2014-04-03 13:27:33 +02:00
Ian Barfield
cf9c1f946a Deleting redundant needsFlush boolean
Motivation:

In ChunkedWriteHandler, there is a redundant variable that servers
no purpose. It implies that under some conditions you might not want
to flush.

Modifications:

Removed the variable and the if condition that read it. The boolean
was always true so just removing the if statement was fine.

Result:

Slightly less misleading code.
2014-03-29 20:21:19 +01:00
Norman Maurer
9856563144 Replace usage of System.currentTimeMillis() with System.nanoTime()
Motivation:

Currently we use System.currentTimeMillis() in our timeout handlers this is bad
for various reasons like when the clock adjusts etc.

Modifications:

Replace System.currentTimeMillis() with System.nanoTime()

Result:

More robust timeout handling
2014-03-18 16:06:16 +09:00
Trustin Lee
1e4c22453c Do not use finally to propagate events in AbstractRemoteAddressFilter
Motivation:

We don't really need to propagate an event when handling the event fails.

Modifications:

Do not use finally block in AbstractRemoteAddressFilter

Result:

AbstractRemoteaddressFilter does not forward an event in case of failure.
2014-03-12 16:17:40 +09:00
Trustin Lee
cf275237c9 Overall clean-up of ipfilter package
Motivation:

Recently merged ipfilter package has the following problems:
* AbstractIpFilterHandler could be improved to support any SocketAddress types rather than only InetSocketAddress.
* AbstractIpFilterHandler can be removed immediately after decision is made rather than keeping the outcome of the decision as an attribute.
* AbstractIpFilterHandler doesn't have a hook for the accepted addresses.
* The hook method (reject()) needs to be named in line with other handler methods (i.e. channelRejected())
* IpFilterRuleHandler should allow accepting zero rules - it's particularly useful for machine-configured setup (i.e. specifying zero rules disables ipfilter).
* IpFilterRuleType.ALLOW/DENY should be ACCEPT/REJECT for consistency.

Modifications:

* AbstractIpFilterHandler has been renamed to AbstractRemoteAddressFilter and now uses type parameter.
* Added channelAccepted() and renamed reject() to channelRejected()
* Added ChannelHandlerContext as a parameter of accept() so that accept() can add a listener to the closeFuture() of the channel. This way, UniqueIpFilter continue working even if we remove the filtering handler early.
* Various renames
  * IpFilterRuleHandler -> RuleBasedIpFilter
  * UniqueIpFilterHandler -> UniqueIpFilter

Result:

* Much cleaner API with more extensibility
2014-03-12 16:06:04 +09:00
Jakob Buchgraber
386cc2cb73 ipfilter implementation for netty 4/5 [#2129] 2014-03-10 20:43:39 +01:00
Norman Maurer
918dd54a72 [#2261] Correct javadoc of ChunkedInput 2014-03-03 07:04:11 +01:00
Trustin Lee
1884a5697c Avoid unnecessary IllegalStateException in ChunkedWriteHandler
Motivation:
ChunkedWriteHandler can sometimes fail to write the last chunk of a ChunkedInput due to an I/O error.  Subsequently, the ChunkedInput's associated promise is marked as failure and the connection is closed.  When the connection is closed, ChunkedWriteHandler attempts to clean up its message queue and to mark their promises as success or failure.  However, because the promise of the ChunkedInput, which was consumed completely yet failed to be written, is already marked as failure, the attempt to mark it as success fails, leading a WARN level log.

Modification:
Use trySuccess() instead of setSuccess() so that the attempt to mark a ChunkedInput as success does not raise an exception even if the promise is already done.

Result:
Fixes #2249
2014-02-20 17:14:25 -08:00
Trustin Lee
abcb39b638 Do not use String.format() for log message generation
- It's slow.
2014-02-13 19:31:17 -08:00
Trustin Lee
ac70dc4546 Update the version to 4.1.0.Alpha1-SNAPSHOT 2014-02-13 18:32:26 -08:00
Trustin Lee
df346a023b Change the return type of EmbeddedChannel.read*() from Object to an ad-hoc type parameter
.. so that there's no need to explicitly down-cast.

Fixes #2067
2014-02-13 17:19:26 -08:00
Trustin Lee
a8bc720977 Fixed buffer leaks in LoggingHandlerTest 2014-02-13 17:03:21 -08:00
Trustin Lee
3d874f0bd7 Improve documentation for the two-args formatter in LoggingHandler 2014-02-13 16:51:26 -08:00
Trustin Lee
fbed62249c Rename formatUserMessage to formatSimple and use it in two-args formatter in LoggingHandler 2014-02-13 16:51:15 -08:00
Trustin Lee
a9150ee7d6 Simplify two-args message formatter in LoggingHandler 2014-02-13 16:51:06 -08:00
Trustin Lee
2e58497160 Make LoggingHandler.appendHexDump(..) protected for the subclasses 2014-02-13 16:50:50 -08:00