Commit Graph

5850 Commits

Author SHA1 Message Date
Norman Maurer
2d9e0f53a5 Better implementation of AttributeMap and also add hasAttr(...). See [#2439]
Motivation:
The old DefaultAttributeMap impl did more synchronization then needed and also did not expose a efficient way to check if an attribute exists with a specific key.

Modifications:
* Rewrite DefaultAttributeMap to not use IdentityHashMap and synchronization on the map directly. The new impl uses a combination of AtomicReferenceArray and synchronization per chain (linked-list). Also access the first Attribute per bucket can be done without any synchronization at all and just uses atomic operations. This should fit for most use-cases pretty weel.
* Add hasAttr(...) implementation

Result:
It's now possible to check for the existence of a attribute without create one. Synchronization is per linked-list and the first entry can even be added via atomic operation.
2014-05-15 06:47:42 +02:00
Jake Luciani
d547b5d51d Fix capacity check bug affecting offheap buffers 2014-05-13 07:25:15 +02:00
Trustin Lee
e29f8231aa Fix the life cycle mapping problem with m2e
Motivation:

It should be frictionless to import our project into Eclipse

Modifications:

Exclude the plugins with missing life cycle mapping. They are not useful
for use with IDE anyway.

Result:

Fixes #2488
Netty is imported into Eclipse without a problem.
2014-05-12 21:38:09 +09:00
Norman Maurer
2f7d60f234 Minimize memory footprint of HashedWheelTimer and context-switching
Motivation:
At the moment there are two issues with HashedWheelTimer:
* the memory footprint of it is pretty heavy (250kb fon an empty instance)
* the way how added Timeouts are handled is inefficient in terms of how locks etc are used and so a lot of context-switching / condition can happen.

Modification:
Rewrite HashedWheelTimer to use an optimized bucket implementation to store the submitted Timeouts and a MPSC queue to handover the timeouts.  So volatile writes are reduced to a minimum and also the memory foot-print of the buckets itself is reduced a lot as the bucket uses a double-linked-list. Beside this we use Atomic*FieldUpdater where-ever possible to improve the memory foot-print and performance.

Result:
Lower memory-footprint and better performance
2014-05-11 15:12:29 +02:00
Trustin Lee
d69ad2f85c More robust native library discovery in Mac OS X
Motivation:

Some JDK versions of Mac OS X generates a JNI dynamic library with '.jnilib' extension rather than with '.dynlib' extension.  However, System.mapLibraryName() always returns 'lib<name>.dynlib'. As a result, NativeLibraryLoader fails to load the native library whose extension is .jnilib.

Modification:

Try to find both '.jnilib' and '.dynlib' resources on OS X.

Result:

Dynamic libraries are loaded correctly in Mac OS X, and thus we can continue the OpenSslEngine work.
2014-05-11 18:53:43 +09:00
Norman Maurer
880acbca72 Use ByteBuf.readSlice(...).retain() to minimize memory copies.
Motivation:
At the moment we call ByteBuf.readBytes(...) in these handlers but with optimizations done as part of 25e0d9d we can just use readSlice(...).retain() and eliminate the memory copy.

Modifications:
Replace ByteBuf.readBytes(...) usage with readSlice(...).retain().

Result:
Less memory copies.
2014-05-10 17:20:53 +02:00
Norman Maurer
59d92ad6cf [#2485] Use RecvByteBufAllocator for all allocations related to read from Channel
Motivation:
At the moment we sometimes use only RecvByteBufAllocator.guess() to guess the next size and the use the ByteBufAllocator.* directly to allocate the buffer. We should always use RecvByteBufAllocator.allocate(...) all the time as this makes the behavior easier to adjust.

Modifications:
Change the read() implementations to make use of RecvByteBufAllocator.

Result:
Behavior is more consistent.
2014-05-10 15:27:43 +02:00
Trustin Lee
69c72220aa Update netty-build to version 20
- Preparation for merging OpenSslEngine work
2014-05-08 10:00:16 +09:00
Norman Maurer
c5e4c2294e [#2469] Correctly update indices after incomplete gathering write
Motivation:
 When doing a gathering write we need to update the indices after the write partial completes. In the current code-base we use the wrong value when compare the expected written bytes and the actual written bytes.

 Modifications:
 Use the correct value when compare.

Result:
Indices are updated correctly and so no corruption can happen when resume writing after data was only partial written before.
2014-05-07 07:13:39 +02:00
Daniel Bevenius
1a22186645 Adding short-curcuit option for CORS
Motivation:
CORS request are currently processed, and potentially failed, after the
target ChannelHandler(s) have been invoked. This might not be desired, for
example a HTTP PUT or POST might have been performed.

Modifications:
Added a shortCurcuit option to CorsConfig which when set will
cause a validation of the HTTP request's 'Origin' header and verify that
it is valid according to the configuration. If found invalid an 403
"Forbidden" response will be returned and not further processing will
take place.

This is indeed no help for non browser request, like using curl, which
can set the 'Origin' header.

Result:
Users can now configure if the 'Origin' header should be validated
upfront and have the request rejected before any further processing
takes place.
2014-05-06 12:16:57 +02:00
Norman Maurer
8a39b9fedc Fix buffer leak introduced by #2462
Motivation:
Because of not correctly release a buffer before null out the reference a memory leak shows up.

Modifications:
Correct call buffer.release() before null out reference.

Result:
No more leak
2014-05-06 10:03:24 +02:00
Jeff Pinner
1351349243 SPDY: ensure SpdyHeaderBlockRawDecoder always reads entire input 2014-05-05 07:33:05 +02:00
Jeff Pinner
326efe5b42 SPDY: release headerBlock in SpdyFrameCodec 2014-05-05 07:26:21 +02:00
Jeff Pinner
1d9c431604 SPDY: remove unused code in SpdyHeaderBlockRawEncoder 2014-05-05 07:01:56 +02:00
Sun Ning
d713f015da Made websocket maxFramePayloadSize configurable from WebSocketServerProtocolHandler.
Motivation:

Currently there's no way to configure maxFramePayloadSize from
WebSocketServerProtocolHandler, which is the most used entry point of
WebSocket server.

Modifications:

Added another constructor for maxFramePayloadSize.

Result:

We can configure max frame size for websocket packet in
WebSocketServerProtocolHandler. It will also keep backward compatibility
with default max size: 65536. (65536 is hard-coded max size in previous
version of Netty)
2014-05-05 06:24:28 +02:00
Norman Maurer
9752f37fa0 [#2454] Correctly return null when DefaultChannelPipeline.firstContext() is called on empty pipeline
Motivation:
DefaultChannelPipeline.firstContext() should return null when the ipeline is empty. This is not the case atm.

Modification:
Fix incorrect check in DefaultChannelPipeline.firstContext() and add unit tests.

Result:
Correctly return null when DefaultChannelPipeline.firstContext() is called on empty pipeline.
2014-05-04 21:13:17 +02:00
Jeff Pinner
9dbcf1af49 SPDY: allow subclassing of SpdyFrameCodec 2014-05-03 14:41:51 +02:00
Trustin Lee
296db210ab Generate the default JAR to make oss.sonatype.org happy
Motivation:

oss.sonatype.org refuses to promote an artifact if it doesn't have the
default JAR (the JAR without classifier.)

Modifications:

- Generate both the default JAR and the native JAR to make
  oss.sonatype.org happy
- Rename the profile 'release' to 'restricted-release' which reflects
  what it really does better
- Remove the redundant <quickbuild>true</quickbuild> in all/pom.xml
  We specify the profile 'full' that triggers that property already
  in maven-release-plugin configuration.

Result:

oss.sonatype.org is happy.  Simpler pom.xml
2014-05-03 17:32:45 +09:00
Trustin Lee
a566164b98 Enforce the release is performed only from RHEL 6.5 or equivalent
Motivation:

Netty must be released from RHEL 6.5 x86_64 or compatible so that:

1) we ship x86_64 version of epoll transport officially, and
2) we ensure the ABI compatibility with older GLIBC versions.
   The shared library built on a distribution with newer GLIBC will not
   run on older distributions.

Modifications:

- When 'release' profile is active, perform an additional check using
  maven-enforcer-plugin so that 'mvn release:*' fails when running on
  non-RHEL6.5.  This rule is active only when releasing, so a user
  should not be affected.
- Simplify maven-release-plugin configuration by removing redundant
  profiles such as 'linux'.  'linux' is automatically activated when
  releasing because we now enforce the release occurs on linux-x86_64.
- Remove the no-osgi profile, which is unused
- Remove the reference to 'sonatype-oss-release' profile in all/pom.xml,
  because we always specify 'release' profile when releasing
- Rename the profile 'linux-native' to 'linux' for brevity
- Upgrade oss-parent and maven-enforcer-plugin

Result:

No one can make a mistake to release Netty on an environment that can
produce incompatible or missing native library.
2014-05-02 21:20:34 +09:00
Trustin Lee
df93b35027 Upgrade os-maven-plugin to fix a build issue on windows-x86_32 2014-05-02 21:15:30 +09:00
Trustin Lee
2ff06676ca Fix a build failure due to missing classifier 2014-05-02 14:36:02 +09:00
Trustin Lee
6c1af9036f Simplify native library resolution using os-maven-plugin
Motivation:

So far, we used a very simple platform string such as linux64 and
linux32.  However, this is far from perfection because it does not
include anything about the CPU architecture.

Also, the current build tries to put multiple versions of .so files into
a single JAR.  This doesn't work very well when we have to ship for many
different platforms.  Think about shipping .so/.dynlib files for both
Linux and Mac OS X.

Modification:

- Use os-maven-plugin as an extension to determine the current OS and
  CPU architecture reliable at build time
- Use Maven classifier instead of trying to put all shared libraries
  into a single JAR
- NativeLibraryLoader does not guess the OS and bit mode anymore and it
  always looks for the same location regardless of platform, because the
  Maven classifier does the job instead.

Result:

Better scalable native library deployment and retrieval
2014-05-02 04:21:47 +09:00
Norman Maurer
93fab1d5a3 Remove ContinuationWebSocketFrame.aggregatedText()
Motivation:
Before we aggregated the full text in the WebSocket08FrameDecoder just to fill in the ContinuationWebSocketFrame.aggregatedText(). The problem was that there was no upper-limit and so it would be possible to see an OOME if the remote peer sends a TextWebSocketFrame + a never ending stream of ContinuationWebSocketFrames. Furthermore the aggregation does not really belong in the WebSocket08FrameDecoder, as we provide an extra ChannelHandler for this anyway (WebSocketFrameAggregator).

Modification:
Remove the ContinuationWebSocketFrame.aggregatedText() method and corresponding constructor. Also refactored WebSocket08FrameDecoder a bit to me more efficient which is now possible as we not need to aggregate here.

Result:
No more risk of OOME because of frames.
2014-04-30 14:49:17 +02:00
Norman Maurer
0aea9eaed5 Correctly write pending data after ssl handshake completes. Related to [#2437]
Motivation:
When writing data from a server before the ssl handshake completes may not be written at all to the remote peer
if nothing else is written after the handshake was done.

Modification:
Correctly try to write pending data after the handshake was complete

Result:
Correctly write out all pending data
2014-04-30 14:23:18 +02:00
Norman Maurer
3ba856a5f4 Correctly handle SocksCmdResponse. Related to #2428
Motivation:
Ports range check is not correct

Modification:
Allow port between 0 and 65535. 0 is wildcard / unknown port here

Result:
Correct validation
2014-04-30 10:44:17 +02:00
Norman Maurer
80f1dfaec0 [#2428] Proper fix of incorrect port range check
Motivation:

In the Internet Protocol, the valid port number range is from 1 to 65535
(inclusive on the both side.)  However, SocksCmdRequest and SocksCmdResponse
refuses to construct itself when the port number 65535 is specified. Beside
this it excepts 0 as port number which should not allowed.

    Modification:

* Not raise an exception when the specified port number is 65535.
* Raise an exception when the specified port number is 0

Result:

Fixes #2428
2014-04-30 08:17:40 +02:00
Trustin Lee
01ef614e77 Fix incorrect port range check
Motivation:

In the Internet Protocol, the valid port number range is from 1 to 65535
(inclusive on the both side.)  However, SocksCmdRequest refuses to
construct itself when the port number 65535 is specified.

Modification:

Do not raise an exception when the specified port number is 65535.

Result:

Fixes #2428
2014-04-29 17:58:08 +09:00
Norman Maurer
e8b5f19b82 Not cause busy loop when interrupt Thread of NioEventLoop
Motivation:
Because Thread.currentThread().interrupt() will unblock Selector.select() we need to take special care when check if we need to rebuild the Selector. If the unblock was caused by the interrupt() we will clear it and move on as this is most likely a bug in a custom ChannelHandler or a library the user makes use of.

Modification:
Clear the interrupt state of the Thread if the Selector was unblock because of an interrupt and the returned keys was 0.

Result:
No more busy loop caused by Thread.currentThread().interrupt()
2014-04-28 08:18:57 +02:00
Daniel Bevenius
2f082cf9a2 Adding support for echoing the request origin for CORS.
Motivation:
When CORS has been configured to allow "*" origin, and at the same time
is allowing credentials/cookies, this causes an error from the browser
because when the response 'Access-Control-Allow-Credentials' header
is true, the 'Access-Control-Allow-Origin' must be an actual origin.

Modifications:
Changed CorsHandler setOrigin method to check for the combination of "*"
origin and allowCredentials, and if the check matches echo the CORS
request's 'Origin' value.

Result:
This addition enables the echoing of the request 'Origin' value as the
'Access-Control-Allow-Origin' value when the server has been configured
to allow any origin in combination with allowCredentials.

This allows client requests to succeed when expecting the server to
be able to handle "*" origin and at the same time be able to send cookies
by setting 'xhr.withCredentials=true'. A concrete example of this is
the SockJS protocol which expects behaviour.
2014-04-25 18:20:02 +02:00
Norman Maurer
d8d822fddd Eliminate unnecessary extra synchronization in DefaultChannelPipeline
Motivation:
At the moment whenever we add/remove a ChannelHandler with an EventExecutorGroup we have two synchronization points in the execution path. One to find the childInvoker and one for add/remove itself. We can eliminate the former by call findIInvoker in the synchronization block, as we need to synchronize anyway.

Modification:
Remove the usage of AtomicFieldUpdater and the extra synchronization in findInvoker by moving the call of the method in the synchronized(this) block.

Result:
Less synchronization points and volatile reads/writes
2014-04-25 14:15:23 +02:00
Trustin Lee
1492b32da7 Code clean-up
Motivation:

It is less confusing not to spread Thread.interrupt() calls.

Modification:

- Comments
- Move generatorThread.interrupt() to where currentThread.interrupt() is
  triggered

Result:

Code that is easier to read
2014-04-25 17:44:04 +09:00
Trustin Lee
0fceef8ab6 Undeprecate deregister() and chanelUnregistered()
Motivation:

As discussed in #2250, it will become much less complicated to implement
deregistration and reregistration of a channel once #2250 is resolved.
Therefore, there's no need to deprecate deregister() and
channelUnregistered().

Modification:

- Undeprecate deregister() and channelUnregistered()
- Remove SuppressWarnings annotations where applicable

Result:

We (including @jakobbuchgraber) are now ready to play with #2250 at
master
2014-04-25 16:40:42 +09:00
Trustin Lee
d765f6b870 Synchronized between 4.1 and master (part 3)
Motivation:

4 and 5 were diverged long time ago and we recently reverted some of the
early commits in master.  We must make sure 4.1 and master are not very
different now.

Modification:

Fix found differences

Result:

4.1 and master got closer.
2014-04-25 16:17:59 +09:00
Trustin Lee
b9039eaa82 Synchronized between 4.1 and master again (part 2)
Motivation:
4 and 5 were diverged long time ago and we recently reverted some of the
early commits in master.  We must make sure 4.1 and master are not very
different now.

Modification:
Remove ChannelHandlerInvoker.writeAndFlush(...) and the related
implementations.

Result:
4.1 and master got closer.
2014-04-25 15:06:26 +09:00
Norman Maurer
8c3eaf3b56 Synchronized between 4.1 and master again
Motivation:
4 and 5 were diverged long time ago and we recently reverted some of the
early commits in master.  We must make sure 4.1 and master are not very
different now.

Modification:
Remove ChannelHandlerInvoker.writeAndFlush(...) and the related implementations.

Result:
4.1 and master got closer.
2014-04-24 21:28:24 +02:00
Norman Maurer
a2f1f21612 Synchronized between 4.1 and master
Motivation:

4 and 5 were diverged long time ago and we recently reverted some of the
early commits in master.  We must make sure 4.1 and master are not very
different now.

Modification:

Small adjustments to match up branches

Result:

4.1 and master got closer.
2014-04-24 21:04:20 +02:00
Trustin Lee
db3709e652 Synchronized between 4.1 and master
Motivation:

4 and 5 were diverged long time ago and we recently reverted some of the
early commits in master.  We must make sure 4.1 and master are not very
different now.

Modification:

Fix found differences

Result:

4.1 and master got closer.
2014-04-25 00:38:02 +09:00
Michael Nitschinger
65ee10fbd0 [codec-memcache] Avoid NPE on channelInactive.
Motivation:
When no currentMessage has been set and the channel is inactive, a NPE is raised.

Modification:
Make sure that a currentMessage is available before checking the extras.

Result:
No more NPE raised potentially.
2014-04-23 11:41:44 +02:00
Jens Reimann
8fd2f1c880 Mark sun.nio.ch as optional
Motivation:

If sun.nio.ch is not optional this will cause troubles in the
OSGi world. The package is not exposed by default in OSGi, so
actually the whole netty framework cannot be used directly.

There are workarounds, but workarounds are ugly. Especially since
the use of sun.nio.ch is optional. So the requirement on the
package should be optional as well.

Modifications:

Make the import of sun.nio.ch optional.

Result:

If the package cannot be imported it will behave as if the package
sun.nio.ch is not present (like with other JVMs). If the package is
exposed in OSGi (e.g. bootclassloader delegation, extension fragment)
it will be used.
2014-04-23 10:37:14 +02:00
Norman Maurer
81b0e2af02 Make NioSocketChannelTest more bullet-proof
Motivation:
I had the NioSocketChannelTest.testFlushCloseReentrance() fail sometimes on one of my linux installation. This change let it pass all the time.

Modification:
Set the SO_SNDBUF to a small value to force split writes

Result:
Test is passing all the time where it was sometimes fail before.
2014-04-23 09:28:20 +02:00
Norman Maurer
9afb56c3c6 [#2414] Fix IllegalStateException when try to configure AbstractEpollChannel once it is deregistered
Motivation:
AbstractEpollChannel.clearEpollIn() throws an IllegalStateException if a user tries to change the autoRead configuration for the Channel and the Channel is not registered on an EventLoop yet. This makes it for example impossible to set AUTO_READ to false via the ServerBootstrap as the configuration is modifed before the Channel is registered.

Modification:
Check if the Channel is registered and if not just modify the flags directly so they are respected once the Channel is registered

Result:
It is possible now to configure AUTO_READ via the ServerBootstrap
2014-04-22 10:18:17 +02:00
Norman Maurer
ff4a89b5ae [#2414] Fix RuntimeException during modify events via EpollEventLoop
Motivation:
We are currently try to modify the events via EpollEventLoop even when the channel was closed before and so the fd was set to -1. This fails with a RuntimeException in this case.

Modification:
Always check if the Channel is still open before try to modify the events.

Result:
No more RuntimeException because of a not open channel
2014-04-21 11:19:08 +02:00
Norman Maurer
7ae40ace32 [#2400] Not close LocalChannel during deregister() to allow register to other EventLoop
Motivation:
At the moment it is not possible to deregister a LocalChannel from its EventLoop and register it to another one as the LocalChannel is closed during the deregister.

Modification:
Not close the LocalChannel during dergister

Result:
It is now possible to deregister a LocalChannel and register it to another EventLoop
2014-04-21 10:07:42 +02:00
Norman Maurer
b5ab5150be Use correct generics for TCP_KEEPIDLE, TCP_KEEPINTVL and TCP_KEEPCNT in EpollChannelOption. Part of [#2396]
Motivation:
Currently the generics used for TCP_KEEPIDLE, TCP_KEEPINTVL and TCP_KEEPCNT are incorrect.

Modifications:
Use Integer as type

Result:
User can use TCP_KEEPIDLE, TCP_KEEPINTVL and TCP_KEEPCNT as expected
2014-04-21 10:00:26 +02:00
Trustin Lee
a31f36d933 Stop ThreadLocalRandom's initial seed generation immediately on interruption
Motivation:

ThreadLocalRandomTest reveals that ThreadLocalRandom's initial seed generation loop becomes tight if the thread is interrupted.
We currently interrupt ourselves inside the wait loop, which will raise an InterruptedException again in the next iteration, resulting in infinite (up to 3 seconds) exception construction and thread interruptions.

Modification:

- When the initial seed generator thread is interrupted, break out of the wait loop immediately.
- Log properly when the initial seed generation failed due to interruption.
- When failed to generate the initial seed, interrupt the generator thread just in case the SecureRandom implementation handles it properly.
- Make the initial seed generator thread daemon and handle potential exceptions raised due to the interruption.

Result:

No more tight loop on interruption.  More robust generator thread termination. Fixes #2412
2014-04-20 17:55:25 +09:00
Trustin Lee
af9ab8b370 Feed only a single SSL record to SSLEngine.unwrap()
Motivation:

Some SSLEngine implementations violate the contract and raises an
exception when SslHandler feeds an input buffer that contains multiple
SSL records to SSLEngine.unwrap(), while the expected behavior is to
decode the first record and return.

Modification:

- Modify SslHandler.decode() to keep the lengths of each record and feed
  SSLEngine.unwrap() record by record to work around the forementioned
  issue.
- Rename unwrap() to unwrapMultiple() and unwrapNonApp()
- Rename unwrap0() to unwrapSingle()

Result:

SslHandler now works OpenSSLEngine from finagle-native.  Performance
impact remains unnoticeable.  Slightly better readability. Fixes #2116.
2014-04-20 17:33:04 +09:00
Martin Krüger
d854d3a617 Fix chunk type for stream identifier
Motivation:
The problem with the current snappy implementation is that it does
not comply with framing format definition found on
https://code.google.com/p/snappy/source/browse/trunk/framing_format.txt

The document describes that chunk type of the stream identifier is defined
as 0xff. The current implentation uses 0x80.

Modifications:
This patch replaces the first byte of the chunk type of the stream identifier
with 0xff.

Result:
After this modification the snappy implementation is compliant to the
framing format described at
https://code.google.com/p/snappy/source/browse/trunk/framing_format.txt.
This results in a better compatibility with other implementations.
2014-04-19 21:06:28 +02:00
Norman Maurer
d6ec44c871 Fix buffer leak in EpollDatagramChannel
Motivation:
EpollDatagramChannel produced buffer leaks when tried to read from the channel and nothing was ready to be read.

Modifications:
Correctly release buffer if nothing was read

Result:
No buffer leak
2014-04-18 20:37:52 +02:00
Norman Maurer
97553069b6 [#2396] Allow to set TCP_KEEPIDLE, TCP_KEEPINTVL and TCP_KEEPCNT in native transport
Motivation:
Allow to set TCP_KEEPIDLE, TCP_KEEPINTVL and TCP_KEEPCNT in native transport to offer the user with more flexibility.

Modifications:
Expose methods to set these options and write the JNI implementation.

Result:
User can now use TCP_KEEPIDLE, TCP_KEEPINTVL and TCP_KEEPCNT.
2014-04-18 11:32:17 +02:00
Trustin Lee
e9161147a5 Work around an Android SSLEngine issue
Motivation:

Some Android SSLEngine implementations skip FINISHED handshake status
and go straightly into NOT_HANDSHAKING.  This behavior blocks SslHandler
from notifying its handshakeFuture, because we do the notification when
SSLEngine enters the FINISHED state.

Modification:

When the current handshake state is NOT_HANDSHAKING and the
handshakeFuture is not fulfilled yet, treat NOT_HANDSHAKING as FINISHED.

Result:

Better Android compatibility - fixes #1823
2014-04-18 17:59:48 +09:00