Commit Graph

6386 Commits

Author SHA1 Message Date
Norman Maurer
a765753ae6 [#2847] Correctly encode HTTP to SPDY if X-SPDY-Associated-To-Stream-ID is not present
Motivation:

Because of a bug a NPE was thrown when tried to encode HTTP to SPDY and no X-SPDY-Associated-To-Stream-ID was present.

Modifications:

Use 0 as default value when X-SPDY-Associated-To-Stream-ID not present.

Result:

No NPE anymore.
2014-08-31 21:08:25 +02:00
Amir Szekely
bdd950dff8 Don't ignore maxCapacity if it's not a power of 2
Motivation:

This fixes bug #2848 which caused Recycler to become unbounded and cache infinite number of objects with maxCapacity that's not a power of two. This can result in general sluggishness of the application and OutOfMemoryError.

Modifications:

The test for maxCapacity has been moved out of test to check if the buffer has filled. The buffer is now also capped at maxCapacity and cannot grow over it as it jumps from one power of two to the other.

Additionally, a unit test was added to verify maxCapacity is honored even when it's not a power of two.

Result:

With these changes the user is able to use a custom maxCapacity number and not have it ignored. The unit test assures this bug will not repeat itself.
2014-08-31 09:06:10 +02:00
Norman Maurer
cc55a71b14 Correctly release buffers on protocol errors
Motivation:

We failed to release buffers on protocolErrors which caused buffer leaks when using HTTP/2

Modifications:

Release buffer on protocol errors

Result:

No more buffer leaks
2014-08-30 20:53:07 +02:00
Norman Maurer
ce8230f455 Correctly release buffer in DelegatingHttp2ConnectionHandlerTest
Motivation:

Because we did not release the buffer correctly in DelegatingHttp2ConnectionHandlerTest the CI failed because of a buffer leak.
See https://secure.motd.kr/jenkins/job/pr_2849_kichik_netty_master/4/consoleFull

Modifications:

Correctly release buffer

Result:

No more leak error.
2014-08-30 20:43:23 +02:00
Scott Mitchell
5f232b2220 ALPN java implementation
Motivation:

Netty only supports a java NPN implementation provided by npn-api and npn-boot.
There is no java implementation for ALPN.
ALPN is needed to be compliant with the HTTP/2 spec.

Modifications:
-SslContext and JdkSslContext to support ALPN
-JettyNpn* class restructure for NPN and ALPN common aspects
-Pull in alpn-api and alpn-boot optional dependencies for ALPN java implementation

Result:

-Netty provides access to a java implementation of APLN
2014-08-30 13:41:07 -04:00
Jakob Buchgraber
0d549706f1 Reduce Memory Usage of SingleThreadEventLoopTest
Motivation:

The SingleThreadEventLoopTest allocated up to half a gigabyte of memory per run,
causing up to 50 GC runs in ~20 seconds.

Modification:

Charlie Hunt identified TLAB allocations to be the root cause of this excessive
memory usage [1]. By reusing the Executor in every test we can reduce the
memory usage by >50%.

Result:

Lower Memory Usage and fewer Garbage Collector runs.
Helps to resolve GitHub issue #2841.

[1] https://twitter.com/charlesjhunt/status/505351389317722112
2014-08-29 19:31:31 +02:00
Norman Maurer
3b114a51df Cleanup test
Motivation:

Saw a lot of inspector warnings

Modifications:

Fix inspector warnings

Result:

Cleaner code
2014-08-29 16:01:54 +02:00
Norman Maurer
5b0f60cff9 [#2841] Fix SingleThreadEventLoopTest that was failing because of GC pressure
Motivation:

Sometimes the SingleThreadEventLoopTest did fail on our CI. This was because of GC pressure produced by Thread.sleep(...) when interrupted as it creates a new InterruptedException all the time (and need to fill it).

Modifications:

Replace Thread.sleep(...) with LockSupport.parkNanos(...) to eliminate exception overhead.

Result:

SingleThreadEventLoopTest produce a lot less garbage.
2014-08-29 14:57:48 +02:00
nmittler
4599b7a0ba Fixing HTTP/2 data write completion bug
Motivation:

A recent refactoring of the outbound flow controller interface
introduced a bug when writing data. We're no longer properly handling
the completion of the write (i.e. updating stream state/handling error).

Modifications:

Updated AbstractHttp2ConnectionHandler.writeData to properly handle the
completion of the write future.

Result:

DATA writes now perform post-write cleanup.
2014-08-28 09:58:00 -07:00
Scott Mitchell
2128753484 Exclude bin directory from git Motivation:
The 'bin/' directory is currently not in the .gitignore file.
Eclipse creates this directory to support IDE operations.
These directory is generated and not related to source code and therefore does not belong in git.

Modifications:
-Add 'bin/' to .gitignore

Result:
-No more 'bin/' directory confusion for eclipse users.
2014-08-27 06:33:22 +02:00
nmittler
102e781715 Renaming "Observer" to "Listener" for consistency in HTTP/2
Motivation:

We currently have a mix of "Observer" and "Listener" for interface
names. We should make them all "Listener" to be consistent.

Modifications:

Renamed Http2DataObserver->Http2DataListener and
Http2FrameObserver->Http2FrameListener.

Result:

Listener interface names are consistent.
2014-08-26 18:52:05 -07:00
Scott Mitchell
63d5925d6e HTTP/2 to HTTP priority translation
Motivation:

The priority information reported by the HTTP/2 to HTTP tranlsation layer is not correct in all situations.
The HTTP translation layer is not using the Http2Connection.Listener interface to track tree restructures.
This incorrect information is being sent up to clients and is misleading.

Modifications:
-Restructure InboundHttp2ToHttpAdapter to allow a default data/header mode
-Extend this interface to provide an optional priority translation layer

Result:
-Priority information being correctly reported in HTTP/2 to HTTP translation layer
-Cleaner code with seperation of concerns (optional priority conversion).
2014-08-26 14:06:02 -07:00
nmittler
6409a5a1d5 Adding window maintenance flag to HTTP/2 inbound flow control
Motivation:

Currently, window maintenance is automatically performed when a flow
control window drops below half its initial size. We should provide a
way for advanced applications to determine whether or not this should be
done on a per-stream basis.

Modifications:

Modifying DefaultHttp2InboundFlowController to allow enabling/disabling
of window maintenance per stream.

Result:

Inbound flow control window maintenance will be dynamically
configurable.
2014-08-26 12:02:05 -07:00
Norman Maurer
e166780f0b [#2823] Writing DefaultFileRegion with EpollSocketChannel may cause hang
Motivation:

In EpollSocketchannel.doWriteFileRegion(...) we need to make sure we write until sendFile(...) returns either 0 or all is written. Otherwise we may not get notified once the Channel is writable again.

This is the case as we use EPOLL_ET.

Modifications:

Always write until either sendFile returns 0 or all is written.

Result:

No more hangs when writing DefaultFileRegion can happen.
2014-08-26 15:09:27 +02:00
Norman Maurer
0b5df22aec [#2797] Let EventExecutorGroup extend AutoCloseable
Motivation:

To make it more easy to shutdown an EventExecutorGroup / EventLoopGroup we should let both of them extend AutoCloseable.

Modifications:

Let EventExecutorGroup extend AutoCloseable and impement it.

Result:

Easier shutdown of EventExecutorGroup and EventLoopGroup
2014-08-26 14:04:27 +02:00
Jakob Buchgraber
0381fa67a8 Undo AbstractChannelHandlerContext extend PausableChannelEventExecutor. Fixes #2814
Motivation:
Our last-minute change that made AbstractChannelHandlerContext implement PausableChannelEventExecutor broke the socksproxy example.

Modifications:
AbstractChannelHandlerContext does not inherit from PausableChannelEventExecutor anymore. Instead we'll allocate an extra object on demand.

Result:

AbstractChannelHandlerContext.executor().newPromise() returns the correct type.
2014-08-26 13:50:40 +02:00
Trustin Lee
de6e73e3ea Check noJdkZlibEncoder before comparing other parameters
.. for consistency
2014-08-26 16:12:38 +09:00
Trustin Lee
239371abaf Add io.netty.noJdkZlibEncoder system property
Related issue: #2821

Motivation:

There's no way for a user to change the default ZlibEncoder
implementation.

It is already possible to change the default ZlibDecoder implementation.

Modification:

Add a new system property 'io.netty.noJdkZlibEncoder'.

Result:

A user can disable JDK ZlibEncoder, just like he or she can disable JDK
ZlibDecoder.
2014-08-26 15:49:29 +09:00
Scott Mitchell
892944eba8 HTTP/2 draft 14 HTTP message flow
Motivation:

HTTP/2 draft 14 came out a couple of weeks ago and we need to keep up
with the spec.

Modifications:
-Revert back to dispatching FullHttpMessage objects instead of individual HttpObjects
-Corrections to HttpObject comparitors to support test cases
-New test cases to support sending headers immediatley
-Bug fixes cleaned up to ensure the message flow is terminated properly

Result:
Netty HTTP/2 to HTTP/1.x translation layer will support the HTTP/2 draft message flow.
2014-08-25 11:34:15 -07:00
nmittler
5b1d50fa7c Changing HTTP/2 inbound flow control to use Http2FrameWriter
Motivation:

This is just some general cleanup to get rid of the FrameWriter inner
interface withing Http2InboundFlowController.  It's not necessary since
the flow controller can just use the Http2FrameWriter to send
WINDOW_UPDATE frames.

Modifications:

Updated DefaultHttp2InboundFlowController to use Http2FrameWriter.

Result:

The inbound flow control code is somewhat less smelly :).
2014-08-25 07:28:07 -07:00
Norman Maurer
8785ff5256 Move duplicated code into CodecUtil
Motivation:

We have some duplicated code that can be reused.

Modifications:

Create package private class called CodecUtil that now contains the shared code / helper method.

Result:

Less code-duplication
2014-08-25 07:37:35 +02:00
Norman Maurer
488f36d009 [#2812] Ensure we call checkForSharableAnnotation in all constructors of ByteToMessageCodec
Motivation:

ByteToMessageCodec miss to check for @Sharable annotation in one of its constructors.

Modifications:

Ensure we call checkForSharableAnnotation in all constructors.

Result:

After your change, what will change.
2014-08-23 21:03:57 +02:00
nmittler
3ffd205f57 Adding padding to HTTP/2 flow control.
Motivation:

We're currently out-of-spec with HTTP/2 in that we don't include padding
in the flow control logic.

Modifications:

Modified both DefaultHttp2InboundFlowController and
DefaultHttp2OutboundFlowController to properly take padding into
account. Outbound is more complicated since padding has to be properly
accounted for when splitting frames.

Result:

HTTP/2 codec properly flow controls padding.
2014-08-22 13:22:21 -07:00
Scott Mitchell
d3538dee2e HTTP/2 Priority Tree Restructure
Motivation:

The current interface and implementation for HTTP/2 priority tree events does not notify listeners of all parent change events. As a result operations which depend upon knowing about parent change events may be missing events and result in stale data. This interface also allows new listeners to easily consume priority tree change events.

Modifications:

-Http2Connection.Listener interface will change to support notifications on every node after the priority has changed and tree events have settled
-This will affect the outbound flow controller, DefaultHttp2Connection, and other listeners using the old interface

Result:
A modified (hopefully simplified and correct) Listener interface to get priority tree change event notification
2014-08-22 13:07:22 -07:00
nmittler
21bc279700 Refactoring outbound flow controller to use frame writer.
Motivation:

This is addressing a TODO in the outbound flow controller. We currently
have a separate writer interface passed into the outbound flow
controller. This is confusing and limiting as to how the flow controller
can perform its writes (e.g. no control over flushing). Instead it would
be better to just let the flow controller use the Http2FrameWriter
directly.

Modifications:

- Added a new Http2DataWriter interface, which is extended by
Http2FrameWriter and Http2OutboundFlowController.

- Removed automatic flushing from Http2DataWriter in order to facilitate
optimizing the case where there are multiple writes.

- Updated DefaultHttp2OutboundFlowController to properly optimize
flushing of the ChannelHandlerContext when multiple writes occur.

Result:

Code is greatly simplified WRT outbound flow control and flushes are
optimized for flow-controlled DATA frames.
2014-08-21 11:49:12 -07:00
Norman Maurer
d332c00e2f Reduce memory copies in spdy compression implementation.
Motivation:

Currently we do more memory copies then needed.

Modification:

- Directly use heap buffers to reduce memory copy
- Correctly release buffers to fix buffer leak

Result:

Less memory copies and no leaks
2014-08-21 11:32:36 +02:00
Norman Maurer
ee198d7cfc Allow efficient writing of CompositeByteBuf when using native epoll transport.
Motivation:

There were no way to efficient write a CompositeByteBuf as we always did a memory copy to a direct buffer in this case. This is not needed as we can just write a CompositeByteBuf as long as all the components are buffers with a memory address.

Modifications:

- Write CompositeByteBuf which contains only direct buffers without memory copy
- Also handle CompositeByteBuf that have more components then 1024.

Result:

More efficient writing of CompositeByteBuf.
2014-08-21 10:58:42 +02:00
Norman Maurer
a32652197b Avoid redundant reads of head in peakNode
Motivation:

There is not need todo redunant reads of head in peakNode as we can just spin on next() until it becomes visible.

Modifications:

Remove redundant reads of head in peakNode. This is based on @nitsanw's patch for akka.
See https://github.com/akka/akka/pull/15596

Result:

Less volatile access.
2014-08-21 09:01:34 +02:00
Norman Maurer
16a8f8b5aa Code-inspection fixes
Motivation:

Saw some code-inspection warnings

Modifications:

Fix warnings

Result:

Less code-inspection warnings
2014-08-21 07:20:00 +02:00
Norman Maurer
ceaab70153 [#2800] Fix NPE in SingleThreadEventExecutor on Android
Motivation:

The recent changes f8bee2e94c to use a FJP introduced a regression on Android that cause a NPE.

Modifications:

- Use AtomicReferenceFieldUpdater so it works also on Android
- Fix inspection warnings

Result:

Netty works again on Android too.
2014-08-21 06:31:56 +02:00
Norman Maurer
62e26a2818 [#2586] Use correct EventExecutor to notify for bind failures on late registration
Motivation:

We used the wrong EventExecutor to notify for bind failures if a late registration was done.

Modifications:

Use the correct EventExecutor to notify and only use the GlobelEventExecutor if the registration fails itself.

Result:

The correct Thread will do the notification.
2014-08-20 16:35:06 +02:00
Norman Maurer
5386c851c8 codec-socks test cleanup
Motivation:

The unit tests in codec-socks contained redundant casts and empty test classes.

Modifications:

- Remove redundant casts
- Delete empty test classes

Result:

Cleanup
2014-08-20 06:54:55 +02:00
Scott Mitchell
075190f711 Add HTTP/2 codec to netty-all maven dependency Motivation:
The HTTP/2 codec is missing from the netty-all maven dependency.
This means third party libraries have to pull in additional dependencies
above netty-all to use this codec.

Modifications:

Add HTTP/2 codec to netty-all

Result:

Third party libraries don't need to pull in additional dependencies
to use HTTP/2 codec.
2014-08-18 19:07:49 +02:00
nmittler
0c817d61b3 Upgrading to HTTP/2 draft 14 framing
Motivation:

HTTP/2 draft 14 came out a couple of weeks ago and we need to keep up
with the spec.

Modifications:

- Removed use of segment throughout.
- Added new setting for MAX_FRAME_SIZE. Used by the frame reader/writer
rather than a constant.
- Added new setting for MAX_HEADER_LIST_SIZE. This is currently unused.
- Expanded the header size to 9 bytes. The frame length field is now 3
bytes and added logic for checking that it falls within the valid range.

Result:

Netty will support HTTP/2 draft 14 framing. There will still be some
work to do to be compliant with the HTTP adaptation layer.
2014-08-18 07:39:36 -07:00
fredericBregier
26116541ed Fix example for Http Upload
Motivation:
The example mis handle two elements:
1) Last message is a LastHttpContent and is not taken into account by
the server handler
2) The client makes a sync on last write (chunked) but there is no flush
before, therefore the sync is waiting forever.

Modifications:
1) Take into account the message LastHttpContent in simple Get.
2) Removes sync but add flush for each post and multipost parts

Results:
Example is no more blocked after get test.

Should be done also in 4.0 and Master (similar changes)
2014-08-18 10:48:17 +02:00
Norman Maurer
d150ee7886 Document the correct default value of SOMAXCONN
Motivation:

Recently we changed the default value of SOMAXCONN that is used when we can not determine it by reading /proc/sys/net/core/somaxconn. While doing this we missed to update the javadocs to reflect the new default value that is used.

Modifications:

List correct default value in the javadocs of SOMAXCONN.

Result:

Correct javadocs.
2014-08-18 05:57:35 +02:00
Jakob Buchgraber
d7734861c8 Support IPv4 default route in ipfilter.
Motivation:

In GitHub issue #2767 a bug was reported that the IPv4
default route leads to the ipfilter package denying
instead of accepting all addresses.

While the issue was reported for Netty 3.9, this bug
also applies to Netty 4 and higher.

Modifications:

When computing the subnet address from the CIDR prefix,
correctly handle the case where the prefix is set to zero.

Result:

Ipfilter accepts all addresses when passed the
IPv4 default route.
2014-08-17 14:32:57 +02:00
Frédéric Brégier
5aa6bcde9e Better fix for TrafficShapingHandlerTest
Motivation:
The test procedure is unstable when testing quick time (factor less or equal to 1). Changing to default 10ms in this case will force time to be correct and time to be checked only when factor is >= 2.

Modifications:
When factor is <= 1, minimalWaitBetween is 10ms

Result:
Hoping this version is finally stable.
2014-08-16 18:08:43 +02:00
Frédéric Brégier
4c30067b58 Better fix for TrafficShapingHandlerTest
Motivation:
It seems that in certain conditions, the write back from the server is so quick that the handler has no time to compute traffic shaping. So 10ms of wait before acknowledging is added in server side.

Modifications:
Add 10ms waiting before server ackonwledge the client.

Result:
The timing is now suppsed to be stable.
2014-08-16 10:54:00 +02:00
Frédéric Brégier
4d20f03b6d Update TrafficShapingHandlerTest for Master
Motivation:
Test seems not yet stable (20ms instead of 40ms), so this small fix provides a "lightest" check.
If this is not passing again in master, one could deactivate this test by commenting the assertTrue in line 431.

Modification:
Divide by 2 the to be compared elapse time.

Result:
Locally this test is now passing.
2014-08-16 10:44:20 +02:00
Trustin Lee
138b366b00 Fix checkstyle 2014-08-15 11:09:42 -07:00
Trustin Lee
98dfc91471 Fix for issue #2765 relative to unstable trafficshaping test procedure
Motivation:

The test procedure is unstable due to not enough precise timestamping
during the check.

Modifications:

Reducing the test cases and cibling "stable" test ("timestamp-able")
bring more stability to the tests.

Result:

Tests for TrafficShapingHandler seem more stable (whatever using JVM 6,
7 or 8).
2014-08-15 10:21:11 -07:00
Trustin Lee
44d9163169 Fix a bug where ChannelOutboundBuffer.removeBytes() throws ClassCastException
When a ChannelOutboundBuffer contains ByteBufs followed by a FileRegion,
removeBytes() will fail with a ClassCastException.  It should break the
loop instead.
2014-08-15 09:57:12 -07:00
Trustin Lee
23d15c71ee Fix the regression caused by f31c630c8c
f31c630c8c was causing
SocketGatheringWriteTest to fail because it does not take the case where
an empty buffer exists in a gathering write.

When there is an empty buffer in a gathering write, the number of
buffers returned by ChannelOutboundBuffer.nioBuffer() and the actual
number of write attemps can differ.

To remove the write requests correctly, a byte transport must use
ChannelOutboundBuffer.removeBytes()
2014-08-15 09:44:05 -07:00
Norman Maurer
cb666c1169 [#2771] Correctly handle constructing of EmbeddedChannel
Motivation:

Because of an incorrect logic in teh EmbeddedChannel constructor it is not possible to use EmbeddedChannel with a ChannelInitializer as constructor argument. This is because it adds the internal LastInboundHandler to its ChannelPipeline before it register itself to the EventLoop.

Modifications:

First register self to EventLoop before add LastInboundHandler to the ChannelPipeline.

Result:

It's now possible to use EmbeddedChannel with ChannelInitializer.
2014-08-15 12:59:25 +02:00
Norman Maurer
050e7ed933 [#2769] Fix regression when writing different message types
Motivation:

Due a regression NioSocketChannel.doWrite(...) will throw a ClassCastException if you do something like:

channel.write(bytebuf);
channel.write(fileregion);
channel.flush();

Modifications:

Correctly handle writing of different message types by using the correct message count while loop over them.

Result:

No more ClassCastException
2014-08-15 11:56:27 +02:00
plucury
a2416481e3 Allow ChunkedInput to provide the progress of its transfer
Related issue: #2741 and #2151

Motivation:

There is no way for ChunkedWriteHandler to know the progress of the
transfer of a ChannelInput. Therefore, ChannelProgressiveFutureListener
cannot get exact information about the progress of the transfer.

If you add a few methods that optionally provides the transfer progress
to ChannelInput, it becomes possible for ChunkedWriteHandler to notify
ChannelProgressiveFutureListeners.

If the input has no definite length, we can still use the progress so
far, and consider the length of the input as 'undefined'.

Modifications:

- Add ChunkedInput.progress() and ChunkedInput.length()
- Modify ChunkedWriteHandler to use progress() and length() to notify
  the transfer progress

Result:

ChunkedWriteHandler now notifies ChannelProgressiveFutureListener.
2014-08-14 16:55:35 -07:00
Trustin Lee
d92875402d Overall clean-up on socksx package
- SocksV[45] -> Socks[45]
- Make encodeAsByteBuf package private with some hassle
- Split SocksMessageEncoder into Socks4MessageEncoder and
  Socks5MessageEncoder, and remove the original
- Remove lazy singleton instantiation; we don't need it.
- Remove the deprecated methods
- Fix Javadoc errors
2014-08-14 16:53:45 -07:00
bk1te
a242224646 Add socksx package which supports SOCKS 4/4a/5
Motivation:

SOCKS 4 and 5 are very different protocols although they share the same
name.  It is not possible to incorporate the two protocol versions into
a single package.

Modifications:

- Add a new package called 'socksx' to supercede 'socks' package.
- Add SOCKS 4/4a support to the 'socksx' package

Result:

codec-socks now supports all SOCKS versions
2014-08-14 16:52:43 -07:00
Trustin Lee
951ff902bd Reduce the fallback SOMAXCONN value
Related issue: #2407

Motivation:

The current fallback SOMAXCONN value is 3072.  It is way too large
comparing to the default SOMAXCONN value of popular OSes.

Modifications:

Decrease the fallback SOMAXCONN value to 128 or 200 depending on the
current OS

Result:

Saner fallback value
2014-08-14 15:40:18 -07:00