Commit Graph

6034 Commits

Author SHA1 Message Date
Norman Maurer
639db72839 Fix buffer leak in test introduced by 282d6e73b8 2014-07-10 06:43:09 +02:00
fboucher38
282d6e73b8 Add WebSocket Extension handlers for client and server.
Add permessage-deflate and deflate-frame WebSocket extension
implementations.

Motivation:

Need to compress HTTP WebSocket frames payload.

Modifications:

- Move UTF8 checking of WebSocketFrames from frame decoder to and
external handler.
- Change ZlibCodecFactory in order to use ZLib implentation instead of
JDK if windowSize or memLevel is different from the default one.
- Add WebSocketServerExtensionHandler and
WebSocketClientExtensionHandler to handle WebSocket Extension headers.
- Add DeflateFrame and PermessageDeflate extension implementations.
2014-07-09 16:28:40 +02:00
Daniel Bevenius
cbe20faf6c Adding payload to the Http2Client example.
Modifications:
When trying out the Http2Client example I noticed that adding a request
payload would not cause a data frame to written. This seems to be
because writeHeaders completes the promise, and then the writeData
call ends up in FlowControlWriter.writeFrame, isDone is true and
the data released and the call aborted and returned.
Adding a new promise for the writeData method allows a data frame to be
written.

Result:
A body/payload can now be sent to the server. The example was updated to
simply echo the payload received back to the calling client.
2014-07-09 13:53:01 +02:00
Idel Pivnitskiy
9d08a34c34 Moved bit-level read operations from Bzip2Decoder to the new Bzip2BitReader
Motivation:

Collect all bit-level read operations in one class is better. And now it's easy to use not only in Bzip2Decoder. For example, in Bzip2HuffmanStageDecoder.

Modifications:

Created a new class - Bzip2BitReader which provides bit-level reads.
Removed bit-level read operations from Bzip2Decoder.
Improved javadoc.

Result:

Bzip2BitReader allows the reading of single bit booleans, bit strings of arbitrary length (up to 24 bits), and bit aligned 32-bit integers.
2014-07-09 13:48:29 +02:00
Norman Maurer
6b71089373 Fix JVM segfault during JNI call. Part of [#2647]
Motivation:
Currently when Native.writev(...) is used it is possible to see a JVM segfault because the offset is updated to early.

Modification:
Only update the offset once it is safe to do so.

Result:
No more segfault
2014-07-09 13:36:15 +02:00
Norman Maurer
578068c7e6 [#2647] Respect IOV_MAX when call writev in native transport
Motivation:

epoll transport fails on gathering write of more then 1024 buffers. As linux supports max. 1024 iov entries when calling writev(...) the epoll transport throws an exception.

Thanks again to @blucas to provide me with a reproducer and so helped me to understand what the issue is.

Modifications:

Make sure we break down the writes if to many buffers are uses for gathering writes.

Result:

Gathering writes work with any number of buffers
2014-07-09 12:18:48 +02:00
nmittler
67159e7119 Upgrade to HTTP/2 draft 13
Motivation:

Need to upgrade HTTP/2 implementation to latest draft.

Modifications:

Various changes to support draft 13.

Result:

Support for HTTP/2 draft 13.
2014-07-08 21:10:46 +02:00
Brendt Lucas
58cc16d106 [#2642] CompositeByteBuf.deallocate memory/GC improvement
Motivation:

CompositeByteBuf.deallocate generates unnecessary GC pressure when using the 'foreach' loop, as a 'foreach' loop creates an iterator when looping.

Modification:

Convert 'foreach' loop into regular 'for' loop.

Result:

Less GC pressure (and possibly more throughput) as the 'for' loop does not create an iterator
2014-07-08 21:04:55 +02:00
Norman Maurer
696e355a70 Move generic code to HttpOrSpdyChooser to simplify implementations
Motivation:

HttpOrSpdyChooser can be simplified so the user not need to implement getProtocol(...) method.

Modification:

Add implementation for the method. The user can override it if necessary.

Result:

Easier usage of HttpOrSpdyChooser.
2014-07-07 09:44:50 +02:00
Trustin Lee
93a265ed0a Add missing m2eclipse life cycle mapping
Also, use ignore instead of execution because those plugins are not
really useful when building from Eclipse anyway.
2014-07-06 16:53:42 +09:00
Trustin Lee
65199873c2 Fix another buffer leaks in JsonObjectDecoderTest 2014-07-04 16:11:38 +09:00
Trustin Lee
3921f7c88a Reduce the perceived time taken to retrieve initialSeedUniquifier
Motivation:

When system is in short of entrophy, the initialization of
ThreadLocalRandom can take at most 3 seconds.  The initialization occurs
when ThreadLocalRandom.current() is invoked first time, which might be
much later than the moment when the application has started.  If we
start the initialization of ThreadLocalRandom as early as possible, we
can reduce the perceived time taken for the retrieval.

Modification:

Begin the initialization of ThreadLocalRandom in InternalLoggerFactory,
potentially one of the firstly initialized class in a Netty application.

Make DefaultChannelId retrieve the current process ID before retrieving
the current machine ID, because retrieval of a machine ID is more likely
to use ThreadLocalRandom.current().

Use a dummy channel ID for EmbeddedChannel, which prevents many unit
tests from creating a ThreadLocalRandom instance.

Result:

We gain extra 100ms at minimum for initialSeedUniquifier generation.  If
an application has its own initialization that takes long enough time
and generates good amount of entrophy, it is very likely that we will
gain a lot more.
2014-07-04 16:04:37 +09:00
Trustin Lee
b4c2ae5e59 Log the time taken for generating the initialSeedUniquifier
- Sometimes useful to know it how long it takes from the log, to make
  sure it's not something else that is blocking.
2014-07-04 13:25:26 +09:00
Trustin Lee
13a0a36db3 Make JsonObjectDecoder discard everything after stream corruption
Motivation:

There's no way to recover from a corrupted JSON stream. The current
implementation will raise an infinite exception storm when a peer sends
a large corrupted stream.

Modification:

Discard everything once stream corruption is detected.

Result:

Fixes a buffer leak
Fixes exception storm
2014-07-04 11:15:43 +09:00
Norman Maurer
9c4a733471 [#2586] Use correct EventLoop to notify delayed bind failures
Motivation:

When a bind fails AbstractBootstrap will use the GlobalEventExecutor to notify the ChannelPromise. We should use the EventLoop of the Channel if possible.

Modification:

Use EventLoop of the Channel if possible to use the correct Thread to notify and so guaranteer the right order of events.

Result:

Use the correct EventLoop for notification
2014-07-03 21:31:27 +02:00
Norman Maurer
2438ec5e24 [#2618] Introduce ChannelPromise.unvoid() and ChannelFuture.isVoid()
Motivation:

There is no way for a ChannelHandler to check if the passed in ChannelPromise for a write(...) call is a VoidChannelPromise. This is a problem as some handlers need to add listeners to the ChannelPromise which is not possible in the case of a VoidChannelPromise.

Modification:

- Introduce ChannelFuture.isVoid() which will return true if it is not possible to add listeners or wait on the result.
- Add ChannelPromise.unvoid() which allows to create a ChannelFuture out of a void ChannelFuture which supports all the operations.

Result:

It's now easy to write ChannelHandler implementations which also works when a void ChannelPromise is used.
2014-07-03 14:16:46 +02:00
Trustin Lee
9c4970363e Fix an inspector warning in JsonObjectDecoder 2014-07-03 20:01:23 +09:00
Trustin Lee
74d0c693fb Fix a buffer leak in JsonObjectDecoderTest 2014-07-03 19:58:31 +09:00
Jakob Buchgraber
c63378cd30 Split a JSON byte stream into JSON objects/arrays. Fixes #2536
Motivation:

See GitHub Issue #2536.

Modifications:

Introduce the class JsonObjectDecoder to split a JSON byte stream
into individual JSON objets/arrays.

Result:

A Netty application can now handle a byte stream where multiple JSON
documents follow eachother as opposed to only a single JSON document
per request.
2014-07-03 18:33:48 +09:00
Trustin Lee
8784fd627b Let OkResponseHandler extend SimpleChannelInboundHandler
Motivation:

OkResponseHandler is the last handler in the pipeline of the HTTP CORS
example.  It is responsible for releasing all messages it handled.

Modification:

Extend SimpleChannelInboundHandler instead of ChannelHandlerAdapter

Result:

Fixed a leak
2014-07-03 18:15:23 +09:00
Trustin Lee
71737a2a65 Fix the build timeout when 'leak' profile is active
Motivation:

AbstractByteBufTest.testInternalBuffer() uses writeByte() operations to
populate the sample data.  Usually, this isn't a problem, but it starts
to take a lot of time when the resource leak detection level gets
higher.

In our CI machine, testInternalBuffer() takes more than 30 minutes,
causing the build timeout when the 'leak' profile is active (paranoid
level resource detection.)

Modification:

Populate the sample data using ThreadLocalRandom.nextBytes() instead of
using millions of writeByte() operations.

Result:

Test runs much faster when leak detection level is high.
2014-07-03 17:51:15 +09:00
Daniel Bevenius
43bfc244cf Changed Http2ClientConnectionHandler to print out the aggregated buffers content
Motivation:
When running the Http2Client the data returned from the server, the
"Hello World" string, is supposed to be printed but instead the following
is displayed:
Received message:

Modifications:
Use the aggregated buffer to print out.

Result:
The example now logs the correct data sent from the server:
Received message: Hello World
2014-07-03 07:06:19 +02:00
nmittler
812e1719d7 Removing unused Http2PrefaceHandler.
Motivation:

All of the functionality has been merged into
AbstractHttp2ConnectionHandler.

Modifications:

Removed the Http2PrefaceHandler/Test and all uses.

Result:

Code cleaned up a bit.
2014-07-02 07:10:55 -07:00
Trustin Lee
ba81831e0a Fix the cruft produced from the refactoring
.. in 330404da07
2014-07-02 20:30:45 +09:00
Trustin Lee
330404da07 Fix most inspector warnings
Motivation:

It's good to minimize potentially broken windows.

Modifications:

Fix most inspector warnings from our profile

Result:

Cleaner code
2014-07-02 19:04:11 +09:00
Trustin Lee
cea3b6b2ab Clean up HTTP/2 integration tests
Related pull request: #2481 written by @nmittler

Motivation:

Some tests were still sending requests from the test thread rather than
from the event loop.

Modifications:

- Modified the roundtrip tests to use a new utility Http2TestUtil to
  perform the write operations in the event loop thread.
- Modified the Http2Client under examples to write all requests in the
  event loop thread.
- Added HttpPrefaceHandler and its test which were missing.
- Fixed various inspector warnings

Result:

Tests and examples now send requests in the correct thread context.
2014-07-02 16:58:24 +09:00
Trustin Lee
66d8d8219a Fix inspector warnings
- Move the methods used by an inner class to the inner class
- Removal of various redundant things (throws, parens, ..)
2014-07-02 16:47:48 +09:00
Norman Maurer
750f023ced [#2558] Define SO_REUSEPORT if not defined
Motivation:

Currently it is impossible to build netty on linux system that not define SO_REUSEPORT even if it is supported.

Modification:

Define SO_REUSEPORT if not defined.

Result:

Possible to build on more linux dists.
2014-07-02 09:40:36 +02:00
Norman Maurer
887838a06b Correctly return from selector loop one a scheduled task is ready for processing
Motivation:

We use the nanoTime of the scheduledTasks to calculate the milli-seconds to wait for a select operation to select something. Once these elapsed we check if there was something selected or some task is ready for processing. Unfortunally we not take into account scheduled tasks here so the selection loop will continue if only scheduled tasks are ready for processing. This will delay the execution of these tasks.

Modification:

- Check if a scheduled task is ready after selecting
- also make a tiny change in NioEventLoop to not trigger a rebuild if nothing was selected because the timeout was reached a few times in a row.

Result:

Execute scheduled tasks on time.
2014-07-02 09:10:49 +02:00
Norman Maurer
5a6719406c [#2623] Release local references to guard against StackOverflow in JNI
Motivation:

When we do a (env*)->GetObjectArrayElement(...) call we may created many local references which will only be cleaned up once we exist the native method. Thus a lot of memory can be used and so a StackOverFlow may be triggered. Beside this the JNI specification only say that an implementation must cope with 16 local references.

Modification:

Call (env*)->ReleaseLocalRef(...) to release the resource once not needed anymore.

Result:

Less memory usage and guard against StackOverflow
2014-06-30 09:53:13 +02:00
Norman Maurer
85b39d894f [#2622] Correctly check reference count before try to work on the underlying memory
Motivation:

Because of how we use reference counting we need to check for the reference count before each operation that touches the underlying memory. This is especially true as we use sun.misc.Cleaner.clean() to release the memory ASAP when possible. Because of this the user may cause a SEGFAULT if an operation is called that tries to access the backing memory after it was released.

Modification:

Correctly check the reference count on all methods that access the underlying memory or expose it via a ByteBuffer.

Result:

Safer usage of ByteBuf
2014-06-30 07:14:37 +02:00
Norman Maurer
bebcf81687 Fix compile error 2014-06-28 22:27:15 +02:00
Jay
db3c592e6a Maintain decoder result in HttpObjectAggregator
Motivation:
DecodeResult is dropped when aggregate HTTP messages.

Modification:

Make sure we not drop the DecodeResult while aggregate HTTP messages.

Result:

Correctly include the DecodeResult for later processing.
2014-06-28 22:09:46 +02:00
Michael Klishin
c8b170e9eb Correct a JavaDoc typo 2014-06-28 21:44:48 +02:00
xmxsuperstar
85993e10c1 fix example missing break statement in ReplayingDecoder 2014-06-28 21:41:49 +02:00
Norman Maurer
0ee7f9764e Cleanup comment / code 2014-06-27 21:37:21 +02:00
Norman Maurer
773be9787d Let EpollReuseAddrTest also work with kernel versions that not have bugfix release part 2014-06-27 17:55:55 +02:00
Norman Maurer
dbd041efc1 [#2615] Correctly update SelectionKey after selector rebuild
Motivation:

When a select rebuild was triggered the reference to the SelectionKey is not updated in AbstractNioChannel. This will cause a CancelledKeyException later.

Modification:

Correctly update SelectionKey reference after rebuild

Result:

Fix exception
2014-06-27 17:21:48 +02:00
Trustin Lee
bb9ebb4bd0 Upgrade JACOCO to the latest version
.. to fix the build errors triggered by the old JACOCO release
2014-06-27 17:24:05 +09:00
Trustin Lee
eff05c439e Enable verbose:gc only when 'leak' profile is disabled
The leak profile generates a lot of garbage which makes the verbose:gc
output too large.
2014-06-27 17:09:18 +09:00
Trustin Lee
6071c9c267 Remove the deprecated methods in SPDY and HTTP
No need to keep ABI compatibility since we are working on a
backward-incompatible branch
2014-06-27 16:55:23 +09:00
Norman Maurer
20adbb325e [#2604] Not try to use sun.misc.Cleaner when on android
Motivation:

When a user tries to use netty on android it currently fails with "Could not find class 'sun.misc.Cleaner'"

Modification:

Encapsulate sun.misc.Cleaner usage in extra class to workaround this isssue.

Result:
Netty can be used on android again
2014-06-27 08:25:56 +02:00
Zhihui Jiao
daa4fbb54a Fix inconsistent code in the doc 2014-06-27 06:42:49 +02:00
Scott Mitchell
761d50d86a The temporary variable is no longer used in an attempt to accumulate data over onDataRead method class 2014-06-27 06:39:35 +02:00
Norman Maurer
af042bb7d2 Add missing @Test annotation 2014-06-26 12:24:44 +02:00
Norman Maurer
f08848da7c [#2598] Add Epoll.isAvailable() which allows to check if epoll can be used.
Motivation:

At the moment there is no simple way for a user to check if the native epoll transport can be used on the running platform. Thus the user can only try to instance it and catch any exception and fallback to nio transport.

Modification:

Add Epoll.isAvailable() which allows to check if epoll can be used.

Result:

User can easily check if epoll transport can be used or not
2014-06-26 12:20:47 +02:00
Norman Maurer
d3dbeace15 [#2605] Use SO_REUSEADDR on EpollServerSocketChannel to match defaults of java.nio.ServerSocketChannel impl
Motivation:

When using openjdk and oracle jdk's nio (while using the nio transport) the ServerSocketChannel uses SO_REUSEADDR by default. Our native transport should do the same to make it easier to switch between the different implementations and get the expected result.

Modification:

Change EpollServerSocketChannelConfig to set SO_REUSEADDR on the created socket.

Result:

SO_REUSEADDR is used by default on servers.
2014-06-26 11:57:51 +02:00
Trustin Lee
bfd7e510ac Fix incorrect bytesBefore/indexOf() in ReplayingDecoderBuffer
Motivation:

bytesBefore(length, ...), bytesBefore(index, length, ...), and
indexOf(fromIndex, toIndex,...) in ReplayingDecoderBuffer are buggy.
They trigger 'REPLAY even when they don't need to.

Modification:

Implement the buggy methods properly so that REPLAYs are not triggered
unnecessarily.

Result:

Correct behvaior
2014-06-26 18:56:10 +09:00
Norman Maurer
c4d94a52cd Reduce the memory copies in JdkZlibEncoder
Motivation:

At the moment we use a lot of unnecessary memory copies in JdkZlibEncoder. This is caused by either allocate a to small ByteBuf and expand it later or using a temporary byte array.
Beside this the memory footprint of JdkZlibEncoder is pretty high because of the byte[] used for compressing.

Modification:

- Override allocateBuffer(...) and calculate the estimatedsize in there, this reduce expanding of the ByteBuf later
- Not use byte[] in the instance itself but allocate a heap ByteBuf and write directly into the byte array

Result:

Less memory copies and smaller memory footprint
2014-06-26 11:12:30 +02:00
Trustin Lee
07a1983f27 Checkstyle 2014-06-26 17:48:11 +09:00