Commit Graph

5924 Commits

Author SHA1 Message Date
Frederic Bregier
89ed1517f5 [#2542] HTTP post request decoder does not support quoted boundaries
Motivation:
According to RFC2616 section 19, boundary string could be quoted, but
currently the PostRequestDecoder does not support it while it should.

Modifications:
Once the boundary is found, one check is made to verify if the boundary
is "quoted", and if so, it is "unqoted".

Note: in following usage of this boundary (as delimiter), quote seems no
more allowed according to the same RFC, so the reason that only the
boundary definition is corrected.

Result:
Now the boundary could be whatever quoted or not. A Junit test case
checks it.
2014-06-08 12:07:44 +02:00
Frederic Bregier
c16de8fcbc [#2544] Correctly parse Multipart-mixed POST HTTP request in case of entity ends with odd number of 0x0D
Motivation:
When an attribute is ending with an odd number of CR (0x0D), the decoder
add an extra CR in the decoded attribute and should not.

Modifications:
Each time a CR is detected, the next byte was tested to be LF or not. If
not, in a number of places, the CR byte was lost while it should not be.
When a CR is detected, if the next byte is not LF, the CR byte should be
saved as the position point to the next byte (not LF). When a CR is
detected, if there is not yet other available bytes, the position is
reset to the position of CR (since a LF could follow).

A new Junit test case is added, using DECODER and variable number of CR
in the final attribute (testMultipartCodecWithCRasEndOfAttribute).

Result:
The attribute is now correctly decoded with the right number of CR
ending bytes.
2014-06-08 11:22:51 +02:00
nmittler
bcb180b08d Reducing memory usage for HTTP/2
Motivation:

The connection, priority tree, and inbound/outbound flow controllers
each maintain a separate map for stream information. This is wasteful
and complicates the design since as streams are added/removed, multiple
structures have to be updated.

Modifications:

- Merging the priority tree into Http2Connection. Then we can use
Http2Connection as the central stream repository.

- Adding observer pattern to Http2Connection so flow controllers can be
told when a new stream is created, closed, etc.

- Adding properties for inboundFlow/outboundFlow state to Http2Stream.
This allows the controller to access flow control state directly from
the stream without requiring additional structures.

- Separate out the StreamRemovalPolicy and created a "default"
implementation that runs periodic garbage collection.  This used to be
internal to the outbound flow controller, but I think it is more general
than that.

Result:

HTTP/2 classes will require less storage for new streams.
2014-06-06 20:52:51 +02:00
Norman Maurer
405d573715 [#2436] Unsafe*ByteBuf implementation should only invert bytes if ByteOrder differ from native ByteOrder
Motivation:
Our Unsafe*ByteBuf implementation always invert bytes when the native ByteOrder is LITTLE_ENDIAN (this is true on intel), even when the user calls order(ByteOrder.LITTLE_ENDIAN). This is not optimal for performance reasons as the user should be able to set the ByteOrder to LITTLE_ENDIAN and so write bytes without the extra inverting.

Modification:
- Introduce a new special SwappedByteBuf (called UnsafeDirectSwappedByteBuf) that is used by all the Unsafe*ByteBuf implementation and allows to write without inverting the bytes.
- Add benchmark
- Upgrade jmh to 0.8

Result:
The user is be able to get the max performance even on servers that have ByteOrder.LITTLE_ENDIAN as their native ByteOrder.
2014-06-05 10:59:03 +02:00
Trustin Lee
84fc63f927 Fix StompSubframeEncoderTest failure
Motivation:

StompSubframeEncoderTest fails because StompHeaders does not respect the order of the headers set.

Modifications:

Use LinkedHashMap instead of HashMap

Result:

Fixes test failures
2014-06-05 17:05:12 +09:00
Trustin Lee
2e98566916 Introduce MessageAggregator and DecoderResultProvider
Motivation:

We have different message aggregator implementations for different
protocols, but they are very similar with each other.  They all stems
from HttpObjectAggregator.  If we provide an abstract class that provide
generic message aggregation functionality, we will remove their code
duplication.

Modifications:

- Add MessageAggregator which provides generic message aggregation
- Reimplement all existing aggregators using MessageAggregator
- Add DecoderResultProvider interface and extend it wherever possible so
  that MessageAggregator respects the state of the decoded message

Result:

Less code duplication
2014-06-05 16:39:59 +09:00
Trustin Lee
25e2f5d288 Overall refactoring of the STOMP codec
- StompObject -> StompSubframe
- StompFrame -> StompHeadersSubframe
- StompContent -> StompContntSubframe
- FullStompFrame -> StompFrame
- StompEncoder/Decoder -> StompSubframeEncoder/Decoder
- StompAggregator -> StompSubframeAggregator
- Simplify the example
- Update Javadoc
- Miscellaneous cleanup
2014-06-04 17:09:11 +09:00
Sergey Skachkov
d19d12511a Initial STOMP protocol work from @sskachkov 2014-06-04 17:09:11 +09:00
Trustin Lee
d0640a6686 Clean up MpscLinkedQueue, fix its leak, and make it work without Unsafe
Motivation:

MpscLinkedQueue has various issues:
- It does not work without sun.misc.Unsafe.
- Some field names are confusing.
  - Node.tail does not refer to the tail node really.
  - The tail node is the starting point of iteration. I think the tail
    node should be the head node and vice versa to reduce confusion.
- Some important methods are not implemented (e.g. iterator())
- Not serializable
- Potential false cache sharing problem due to lack of padding
- MpscLinkedQueue extends AtomicReference and thus exposes various
  operations that mutates the internal state of the queue directly.

Modifications:

- Use AtomicReferenceFieldUpdater wherever possible so that we do not
  use Unsafe directly. (e.g. use lazySet() instead of putOrderedObject)
- Extend AbstractQueue to implement most operations
- Implement serialization and iterator()
- Rename tail to head and head to tail to reduce confusion.
- Rename Node.tail to Node.next.
- Fix a leak where the references in the removed head are not cleared
  properly.
- Add Node.clearMaybe() method so that the value of the new head node
  is cleared if possible.
- Add some comments for my own educational purposes
- Add padding to the head node
  - Add FullyPaddedReference and RightPaddedReference for future reuse
- Make MpscLinkedQueue package-local so that a user cannot access the
  dangerous yet public operations exposed by the superclass.
  - MpscLinkedQueue.Node becomes MpscLinkedQueueNode, a top level class

Result:

- It's more like a drop-in replacement of ConcurrentLinkedQueue for the
  MPSC case.
- Works without sun.misc.Unsafe
- Code potentially easier to understand
- Fixed leak (related: #2372)
2014-06-04 03:23:44 +09:00
Trustin Lee
5be069f92d Require Maven 3.1.1 or above
.. because the build fails with an older Maven version due to Eclipse
Aether issues
2014-06-04 03:12:28 +09:00
Norman Maurer
4666ac08d0 ChannelFlushPromiseNotifier should allow long value for pendingDataSize
Motivation:
At the moment ChannelFlushPromiseNotifier.add(....) takes an int value for pendingDataSize, which may be too small as a user may need to use a long. This can for example be useful when a user writes a FileRegion etc. Beside this the notify* method names are kind of missleading as these should not contain *Future* because it is about ChannelPromises.

Modification:
Alter add(...) method to take a long for pendingDataSize. Rename all *Future* methods to have *Promise* in the method name to better reflect usage.

Result:
ChannelFlushPromiseNotifier can be used with bigger data.
2014-06-03 17:39:41 +02:00
Daniel Bevenius
ac4c1c88a7 OkResponseHandler should return a FullHttpResponse.
Motivation:
Currently OkResponseHandler returns a DefaultHttpResponse which is not
correct and it should be returning complete http response.

Modifications:
Updated OkResponseHandler to return an instance of
DefaultFullHttpResponse.

Result:
It is not possible to add compression to the example without getting any
errors.
2014-06-03 09:21:03 +02:00
DhanaRaj Durairaj
dfb5efb7d0 [#2494] Fix data curruption by ChannelTrafficShapingHandler
Motivation:
ChannelTrafficShapingHandler may corrupt inbound data stream by
scheduling the fireChannelRead event.

Modification:
Always call fireChannelRead(...) and only suspend reads after it

Result:
No more data corruption
2014-06-03 08:36:14 +02:00
Trustin Lee
fc45635af3 Fix checkstyle 2014-06-02 19:27:26 +09:00
Trustin Lee
c827c6da37 Add awaitInactivity() to GlobalEventExecutor and ThreadDeathWatcher
Motivation:

When running Netty on a container environment, the container will often
complain about the lingering threads such as the worker threads of
ThreadDeathWatcher and GlobalEventExecutor.  We should provide an
operation that allows a use to wait until such threads are terminated.

Modifications:

- Add awaitInactivity()
- (misc) Fix typo in GlobalEventExecutorTest
- (misc) Port ThreadDeathWatch's CAS-based thread life cycle management
  to GlobalEventExecutor

Result:

- Fixes #2084
- Less overhead on task submission of GlobalEventExecutor
2014-06-02 19:23:50 +09:00
Trustin Lee
9f4543fb39 Fix checkstyle 2014-06-02 18:26:54 +09:00
Trustin Lee
ddb6441212 Use Java 5 foreach for arrays for brevity at no cost 2014-06-02 18:25:49 +09:00
Trustin Lee
642f4bb3b1 Introduce ThreadDeathWatcher
Motivation:

PooledByteBufAllocator's thread local cache and
ReferenceCountUtil.releaseLater() are in need of a way to run an
arbitrary logic when a certain thread is terminated.

Modifications:

- Add ThreadDeathWatcher, which spawns a low-priority daemon thread
  that watches a list of threads periodically (every second) and
  invokes the specified tasks when the associated threads are not alive
  anymore
  - Start-stop logic based on CAS operation proposed by @tea-dragon
- Add debug-level log messages to see if ThreadDeathWatcher works

Result:

- Fixes #2519 because we don't use GlobalEventExecutor anymore
- Cleaner code
2014-06-02 18:14:23 +09:00
Norman Maurer
35cc23ee8b [#2525] Use VoidChannelPromise in MessageToMessageEncoder when possible
Motivation:
At the moment MessageToMessageEncoder uses ctx.write(msg) when have more then one message was produced. This may produce more GC pressure then necessary as when the original ChannelPromise is a VoidChannelPromise we can safely also use one when write messages.

Modifications:
Use VoidChannelPromise when the original ChannelPromise was of this type

Result:
Less object creation and GC pressure
2014-06-01 19:24:30 +02:00
Korotaev Boris
cde667f668 Fix broken CompositeMatcher
Motivation:

ChannelMatchers#CompositeMatcher inverts matches result.

Modifications:

Switched return values.

Result:

ChannelMatchers#CompositeMatcher will return correct results.
2014-06-01 13:14:01 +02:00
Norman Maurer
1347e7f312 [#2523] Fix infinite-loop when remove attribute and create the same attribute again
Motivation:
The current DefaultAttributeMap cause an infinite-loop when the user removes an attribute and create the same attribute again. This regression was introduced by c3bd7a8ff1.

Modification:
Correctly break out loop

Result:
No infinite-loop anymore.
2014-06-01 13:06:50 +02:00
Josh Hoyt
2c38e6b5d9 codec-http: Document the semantics of HttpResponseStatus equality and comparison 2014-05-30 07:52:58 +02:00
Frédéric Brégier
e1abfbb6c1 Correction on StringBuilder default init size
Motivation:
The default StringBuilder size is too small (data.length + 4) while it will be 2*data.length (byte to Hex) + 5 "-" char (since 5 peaces appended).

Modification:
Changing initial size to the correct one

Result:
Allocation of the correct final size from the beginning for this StringBuilder.
2014-05-30 07:33:38 +02:00
Trustin Lee
ea0eb4fdab Do not use a pseudo random for tree traversal
Motivation:

If we make allocateRun/SubpageSimple() always try the left node first and make allocateRun/Subpage() always tries the right node first,  it is more likely that allocateRun/Subpage() will find a node with ST_UNUSED sooner.

Modifications:

- Make allocateRunSimple() and allocateSubpageSimple() always try the left node first.
- Make allocateRun() and allocateSubpage() always try the right node first.
- Remove randome

Result:

We get the same performance without using random numbers.
2014-05-30 11:23:46 +09:00
Trustin Lee
1d0a79e11e Optimize PooledByteBufAllocator
Motivation:

We still have a room for improvement in PoolChunk.allocateRun() and
Subpage.allocate().

Modifications:

- Unroll the recursion in PoolChunk.allocateRun()
- Subpage.allocate() makes use of the 'nextAvail' value set by previous
  free().

Result:

- PoolChunk.allocateRun() optimization yields 10%+ improvements in
  allocation throughput for non-subpage allocations.
- Subpage.allocate() optimization makes the subpage allocations for
  tiny buffers as fast as non-tiny buffers even when the pageSize is
  huge (e.g. 1048576) because it doesn't need to perform a linear search
  in most cases.
2014-05-30 10:50:23 +09:00
Trustin Lee
b50f91f6d0 More realistic ByteBuf allocation benchmark
Motivation:

Allocating a single buffer and releasing it repetitively for a benchmark will not involve the realistic execution path of the allocators.

Modifications:

Keep the last 8192 allocations and release them randomly.

Result:

We are now getting the result close to what we got with caliper.
2014-05-29 19:50:43 +09:00
Trustin Lee
d54030266b Upgrade to netty-tcnative 1.1.30.Fork2 to support Windows 2014-05-28 10:55:52 +09:00
Trustin Lee
d90207b3ac Work around the system configuration issue that causes NioSocketChannelTest to fail
Motivation:

On some ill-configured systems, InetAddress.getLocalHost() fails.  NioSocketChannelTest calls java.net.Socket.connect() and it internally invoked InetAddress.getLocalHost(), which causes the test failures in NioSocketChannelTes on such an ill-configured system.

Modifications:

Use NetUtil.LOCALHOST explicitly.

Result:

NioSocketChannelTest should not fail anymore.
2014-05-28 09:38:42 +09:00
Trustin Lee
e044fc3e56 Upgrade os-maven-plugin to fix an issue with IntelliJ IDEA on Windows 2014-05-27 04:41:58 +09:00
Trustin Lee
7667a4a32e Use a forked exec-maven-plugin instead of maven-antrun-plugin
Motivation:

maven-antrun-plugin does not redirect stdin, and thus it's impossible to
run interactive examples such as securechat-client and telnet-client.
org.codehaus.mojo:exec-maven-plugin redirects stdin, but it buffers
stdout and stderr, and thus an application output is not flushed timely.

Modifications:

Deploy a forked version of exec-maven-plugin which flushes output
buffers in a timely manner.

Result:

Interactive examples work.  Launches faster than maven-antrun-plugin.
2014-05-23 20:03:14 +09:00
Trustin Lee
3f8194557b Clean up the examples
Motivation:

The examples have not been updated since long time ago, showing various
issues fixed in this commit.

Modifications:

- Overall simplification to reduce LoC
  - Use system properties to get options instead of parsing args.
  - Minimize option validation
  - Just use System.out/err instead of Logger
  - Do not pass config as parameters - just access it directly
  - Move the main logic to main(String[]) instead of creating a new
    instance meaninglessly
    - Update netty-build-21 to make checkstyle not complain
  - Remove 'throws Exception' clause if possible
- Line wrap at 120 (previously at 80)
- Add an option to enable SSL for most examples
- Use ChannelFuture.sync() instead of await()
- Use System.out for the actual result. Use System.err otherwise.
- Delete examples that are not very useful:
  - applet
  - websocket/html5
  - websocketx/sslserver
  - localecho/multithreaded
- Add run-example.sh which simplifies launching an example from command
  line
- Rewrite FileServer example

Result:

Shorter and simpler examples.  A user can focus more on what it actually
does than miscellaneous stuff.  A user can launch an example very
easily.
2014-05-23 16:58:47 +09:00
Trustin Lee
47f961b6b9 Fix a bug in DefaultPromise.notifyLateListener() where the listener is not notified
Motivation:

When (listeners == null && lateListeners == null) and (stackDepth >= MAX_LISTENER_STACK_DEPTH), the listener is not notified at all. The discard client does not work.

Modification:

Make sure to submit the notification task.

Result:

The discard client works again and all listeners are notified.
2014-05-23 09:47:22 +09:00
Trustin Lee
53e35480f0 Fix a regression introduced by 00853d9453
.. by fixing incorrect bitwise operation for pipeline traversal
2014-05-23 01:15:41 +09:00
Trustin Lee
e23a688c63 Fix packagine issues in all-in-one JAR
Motivation:

- dependencyVersionsDir property is not resolved during the build
  process. The build doesn't fail because of this, but it creates an
  ugly directory.
- All-in-one JAR contains libnetty-tcnative.so, which is not part of the
  all-in-one JAR.

Modifications:

- Fix an incorrect property name
  (dependencyVersionDir -> dependencyVersionsDir)
- Exclude libnetty-tcnative.so
- Remove unnecessary includes in source expanding configuration

Result:

- Cleaner pom.xml
- We do not ship libnetty-tcnative.so in all-in-one JAR anymore, which
  is correct, because strictly speaking the native library belongs to
  org.apache.tomcat.jni package.
2014-05-22 19:11:01 +09:00
Trustin Lee
3e8c80a26f Use maven-antrun-plugin instead of exec-maven-plugin
Motivation:

exec-maven-plugin does not flush stdout and stderr, making the console
output from the examples invisible to users

Modification:

Use maven-antrun-plugin instead

Result:

A user sees the output from the examples immediately.
2014-05-22 18:53:09 +09:00
Trustin Lee
b58737b898 Fix an incorrect path problem in run-example.sh 2014-05-22 17:58:08 +09:00
Trustin Lee
a99371a4a0 Display example names in 2 columns
- ported from ba28679775
2014-05-22 17:54:31 +09:00
Trustin Lee
00853d9453 Fix an event ordering issue
Motivation:

In the early days of 5.0, we merged ChannelInboundHandler and
ChannelOutboundHandler into ChannelHandler, and introduced the
annotation called 'Skip'.  The annotation 'Skip' was introduced to
determine which handler methods are no-op (i.e. simply forwarding the
event to the next handler) so that DefaultChannelHandlerContext doesn't
even need to submit an event-invoking task to an EventExecutor,
significantly reducing the context switches.

However, this introduced a regression for the handlers which implemented
write() but not flush(). Because flush() was skippable for such
handlers, flush() event went through to the next handler before write() does.

To address this problem, we came up with a naive workaround that sets
MASK_FLUSH when MASK_WRITE is set.

Although the previous workaround works fine for many cases, we still
seem to have an event ordering problem.  We keep seeing the intermittant
failure of LocalTransportThreadModelTest.testStagedExecution(), because
other handler methods are still skipped.

Modifications:

We do not skip the execution of handler methods annotated with 'Skip'
unless all inbound methods (or all outbound methods) are marked with
'Skip'.

Result:

This change Brings back the event ordering behavior of 4.x, making
LocalTransportThreadModelTest.testStagedExecution() pass.
2014-05-22 11:14:36 +09:00
Trustin Lee
d803a75e2f Fix NPE 2014-05-22 10:28:38 +09:00
Trustin Lee
af3b358892 Checkstyle 2014-05-22 10:14:42 +09:00
Trustin Lee
cb66866730 Future compatibility with TLS ALPN
Motivation:

According to TLS ALPN draft-05, a client sends the list of the supported
protocols and a server responds with the selected protocol, which is
different from NPN.  Therefore, ApplicationProtocolSelector won't work
with ALPN

Modifications:

- Use Iterable<String> to list the supported protocols on the client
  side, rather than using ApplicationProtocolSelector
- Remove ApplicationProtocolSelector

Result:

Future compatibility with TLS ALPN
2014-05-22 09:59:58 +09:00
Nathan Mittler
2fb7af0163 Merge pull request #2496 from nmittler/http2
Adding a general handler for upgrading protocols
2014-05-21 09:58:21 -07:00
nmittler
054e4c5233 Merge branch 'master' of https://github.com/netty/netty into http2
Conflicts:
	example/src/main/java/io/netty/example/http2/client/Http2Client.java
	example/src/main/java/io/netty/example/http2/client/Http2ClientInitializer.java
	example/src/main/java/io/netty/example/http2/server/Http2Server.java
	example/src/main/java/io/netty/example/http2/server/Http2ServerInitializer.java
2014-05-21 07:44:18 -07:00
Trustin Lee
46ed3d4c97 Fix NPE in OpenSslEngine 2014-05-21 20:02:15 +09:00
Trustin Lee
cf3e4e7043 Escape a colon in protocol names 2014-05-21 17:45:28 +09:00
Trustin Lee
ddb59fbc01 Update the instruction for running SPDY examples 2014-05-21 17:29:50 +09:00
Trustin Lee
e167ec51eb Add unified NextProtoNego extension support to SslContext
Motivation:

- OpenSslEngine and JDK SSLEngine (+ Jetty NPN) have different APIs to
  support NextProtoNego extension.
  - It is impossible to configure NPN with SslContext when the provider
    type is JDK.

Modification:

- Implement NextProtoNego extension by overriding the behavior of
  SSLSession.getProtocol() for both OpenSSLEngine and JDK SSLEngine.
  - SSLEngine.getProtocol() returns a string delimited by a colon (':')
    where the first component is the transport protosol (e.g. TLSv1.2)
    and the second component is the name of the application protocol
- Remove the direct reference of Jetty NPN classes from the examples
- Add SslContext.newApplicationProtocolSelector

Result:

- A user can now use both JDK SSLEngine and OpenSslEngine for NPN-based
  protocols such as HTTP2 and SPDY
2014-05-21 17:21:18 +09:00
Trustin Lee
3051c1ee0f Bash 3 compatibility
Motivation:

Mac OS X ships Bash 3, and it does not have an associative array
(declare -A).

Modifications:

Do not use an associative array.

Result:

Can run examples on Mac OS X using run-example.sh
2014-05-21 13:35:28 +09:00
Trustin Lee
579973e35a Fix resource leak in DefaultHttp2FrameIOTest 2014-05-21 12:54:36 +09:00
Trustin Lee
50ae950203 Improve run-example.sh
- More usage example
- Newlines for prettier output
2014-05-20 23:39:43 +09:00