Commit Graph

869 Commits

Author SHA1 Message Date
Norman Maurer
d332c00e2f Reduce memory copies in spdy compression implementation.
Motivation:

Currently we do more memory copies then needed.

Modification:

- Directly use heap buffers to reduce memory copy
- Correctly release buffers to fix buffer leak

Result:

Less memory copies and no leaks
2014-08-21 11:32:36 +02:00
plucury
a2416481e3 Allow ChunkedInput to provide the progress of its transfer
Related issue: #2741 and #2151

Motivation:

There is no way for ChunkedWriteHandler to know the progress of the
transfer of a ChannelInput. Therefore, ChannelProgressiveFutureListener
cannot get exact information about the progress of the transfer.

If you add a few methods that optionally provides the transfer progress
to ChannelInput, it becomes possible for ChunkedWriteHandler to notify
ChannelProgressiveFutureListeners.

If the input has no definite length, we can still use the progress so
far, and consider the length of the input as 'undefined'.

Modifications:

- Add ChunkedInput.progress() and ChunkedInput.length()
- Modify ChunkedWriteHandler to use progress() and length() to notify
  the transfer progress

Result:

ChunkedWriteHandler now notifies ChannelProgressiveFutureListener.
2014-08-14 16:55:35 -07:00
Norman Maurer
3f10ed3082 [#2768] Correctly duplicate buffer for CloseWebSocketFrames
Motivation:

The _0XFF_0X00 buffer is not duplicated and empty after the first usage preventing the connection close to happen on subsequent close frames.

Modifications:

Correctly duplicate the buffer.

Result:

Multiple CloseWebSocketFrames are handled correctly.
2014-08-14 09:54:52 +02:00
Scott Mitchell
b41b11c53d Introduced HTTP/2 frame to HTTP/1.x translation layer.
Motivation:

The HTTP/2 codec currently provides direct callbacks to access stream events/data. The HTTP/2 codec provides the protocol support for HTTP/2 but it does not pass messages up the context pipeline. It would be nice to have a decoder which could collect the data framed by HTTP/2 and translate this into traditional HTTP type objects. This would allow the traditional Netty context pipeline to be used to separate processing concerns (i.e. HttpContentDecompressor).  It would also be good to have a layer which can translate FullHttp[Request|Response] objects into HTTP/2 frame outbound events.

Modifications:

Introduce a new InboundHttp2ToHttpAdapter and supporting classes which will translate HTTP/2 stream events/data into HttpObject objects. Introduce a new DelegatingHttp2HttpConnectionHandler which will translate FullHttp[Request|Response] objects to HTTP/2 frame events.

Result:

Introduced HTTP/2 frame events to HttpObject layer.
Introduced FullHttp[Request|Response] to HTTP/2 frame events.
Introduced new unit tests to support new code.
Updated HTTP/2 client example to use new code.
Miscelaneous updates and bug fixes made to support new code.
2014-08-12 12:31:26 -07:00
Jeff Pinner
f7e183f8e4 SPDY: fix SpdySessionHandler::updateSendWindowSize
In Netty 3, downstream writes of SPDY data frames and upstream reads of
SPDY window udpate frames occur on different threads.

When receiving a window update frame, we synchronize on a java object
(SpdySessionHandler::flowControlLock) while sending any pending writes
that are now able to complete.

When writing a data frame, we check the send window size to see if we
are allowed to write it to the socket, or if we have to enqueue it as a
pending write. To prevent races with the window update frame, this is
also synchronized on the same SpdySessionHandler::flowControlLock.

In Netty 4, upstream and downstream operations on any given channel now
occur on the same thread. Since java locks are re-entrant, this now
allows downstream writes to occur while processing window update frames.

In particular, when we receive a window update frame that unblocks a
pending write, this write completes which triggers an event notification
on the response, which in turn triggers a write of a data frame. Since
this is on the same thread it re-enters the lock and modifies the send
window. When the write completes, we continue processing pending writes
without knowledge that the window size has been decremented.
2014-08-11 11:13:22 +02:00
Trustin Lee
168e81e9cd Fix a bug where SpdySession.getActiveStreams() returns incorrect set
Related issue: #2743

Motivation:

When there are more than one stream with the same priority, the set
returned by SpdySession.getActiveStream() will not include all of them,
because it uses TreeSet and only compares the priority of streams. If
two different streams have the same priority, one of them will be
discarded by TreeSet.

Modification:

- Rename getActiveStreams() to activeStreams()
- Replace PriorityComparator with StreamComparator

Result:

Two different streams with the same priority are compared correctly.
2014-08-05 17:46:13 -07:00
Trustin Lee
6ecca2722e Add test cases for HttpContentCompressor
- Ported from 386a06dbfa
2014-08-05 15:48:14 -07:00
Norman Maurer
2258b32549 [#2732] HttpRequestEncoder may produce invalid uri if uri parameters are included.
Motivation:

If the requests contains uri parameters but not path the HttpRequestEncoder does produce an invalid uri while try to add the missing path.

Modifications:

Correctly handle the case of uri with paramaters but no path.

Result:

HttpRequestEncoder produce correct uri in all cases.
2014-08-05 10:14:06 +02:00
Trustin Lee
c5ae3e3dcb Use our own URL shortener wherever possible 2014-07-31 17:06:30 -07:00
Jeff Pinner
6935c07ce1 SPDY: fix pushed response NullPointerException 2014-07-21 07:50:22 +02:00
Idel Pivnitskiy
6860019f41 Fix NPE problems
Motivation:

Now Netty has a few problems with null values.

Modifications:

- Check HAProxyProxiedProtocol in HAProxyMessage constructor and throw NPE if it is null.
If HAProxyProxiedProtocol is null we will set AddressFamily as null. So we will get NPE inside checkAddress(String, AddressFamily) and it won't be easy to understand why addrFamily is null.
- Check File in DiskFileUpload.toString().
If File is null we will get NPE when calling toString() method.
- Check Result<String> in MqttDecoder.decodeConnectionPayload(...).
If !mqttConnectVariableHeader.isWillFlag() || !mqttConnectVariableHeader.hasUserName() || !mqttConnectVariableHeader.hasPassword() we will get NPE when we will try to create new instance of MqttConnectPayload.
- Check Unsafe before calling unsafe.getClass() in PlatformDependent0 static block.
- Removed unnecessary null check in WebSocket08FrameEncoder.encode(...).
Because msg.content() can not return null.
- Removed unnecessary null check in DefaultStompFrame(StompCommand) constructor.
Because we have this check in the super class.
- Removed unnecessary null checks in ConcurrentHashMapV8.removeTreeNode(TreeNode<K,V>).
- Removed unnecessary null check in OioDatagramChannel.doReadMessages(List<Object>).
Because tmpPacket.getSocketAddress() always returns new SocketAddress instance.
- Removed unnecessary null check in OioServerSocketChannel.doReadMessages(List<Object>).
Because socket.accept() always returns new Socket instance.
- Pass Unpooled.buffer(0) instead of null inside CloseWebSocketFrame(boolean, int) constructor.
If we will pass null we will get NPE in super class constructor.
- Added throw new IllegalStateException in GlobalEventExecutor.awaitInactivity(long, TimeUnit) if it will be called before GlobalEventExecutor.execute(Runnable).
Because now we will get NPE. IllegalStateException will be better in this case.
- Fixed null check in OpenSslServerContext.setTicketKeys(byte[]).
Now we throw new NPE if byte[] is not null.

Result:

Added new null checks when it is necessary, removed unnecessary null checks and fixed some NPE problems.
2014-07-20 12:55:08 +02:00
Idel Pivnitskiy
dd26d47adc Small performance improvements
Modifications:

- Added a static modifier for CompositeByteBuf.Component.
This class is an inner class, but does not use its embedded reference to the object which created it. This reference makes the instances of the class larger, and may keep the reference to the creator object alive longer than necessary.
- Removed unnecessary boxing/unboxing operations in HttpResponseDecoder, RtspResponseDecoder, PerMessageDeflateClientExtensionHandshaker and PerMessageDeflateServerExtensionHandshaker
A boxed primitive is created from a String, just to extract the unboxed primitive value.
- Removed unnecessary 3 times calculations in DiskAttribute.addContent(...).
- Removed unnecessary checks if file exists before call mkdirs() in NativeLibraryLoader and PlatformDependent.
Because the method mkdirs() has this check inside.
- Removed unnecessary `instanceof AsciiString` check in StompSubframeAggregator.contentLength(StompHeadersSubframe) and StompSubframeDecoder.getContentLength(StompHeaders, long).
Because StompHeaders.get(CharSequence) always returns java.lang.String.
2014-07-20 09:24:56 +02:00
Gernot Pansy
d8a78424b3 Fixed NPE in WebSocket00FrameDecoder if end couldn't be found in text frame
Motivation:
When we receive an incomplete WebSocketFrame we need to make sure to wait for more data. Because we not did this we could produce a NPE.

Modification:
Make sure we not try to add null into the RecyclableArrayList

Result:
no more NPE on incomplete frames.
2014-07-16 20:04:21 +02:00
Norman Maurer
edc7abb461 [#2650] Allow to disable http header validation in SpdyHttpDecoder and SpdyHttpCodec
Motivation:

HTTP header validation can be expensive so we should allow to disable it like we do in HttpObjectDecoder.

Modification:

Add constructor argument to disable validation.

Result:
Performance improvement
2014-07-11 08:31:39 +02:00
Norman Maurer
dfa5a88978 Reuse previous created HttpHeaders by HttpObjectAggregator
Motivation:

HttpObjectAggregator currently creates a new FullHttpResponse / FullHttpRequest for each message it needs to aggregate. While doing so it also creates 2 DefaultHttpHeader instances (one for the headers and one for the trailing headers). This is bad for two reasons:
  - More objects are created then needed and also populate the headers is not for free
  - Headers may get validated even if the validation was disabled in the decoder

Modification:

- Wrap the previous created HttpResponse / HttpRequest and so reuse the original HttpHeaders
- Reuse the previous created trailing HttpHeader.
- Fix a bug where the trailing HttpHeader was incorrectly mixed in the headers.

Result:

- Less GC
- Faster HttpObjectAggregator implementation
2014-07-11 07:23:58 +02:00
Norman Maurer
639db72839 Fix buffer leak in test introduced by 282d6e73b8 2014-07-10 06:43:09 +02:00
fboucher38
282d6e73b8 Add WebSocket Extension handlers for client and server.
Add permessage-deflate and deflate-frame WebSocket extension
implementations.

Motivation:

Need to compress HTTP WebSocket frames payload.

Modifications:

- Move UTF8 checking of WebSocketFrames from frame decoder to and
external handler.
- Change ZlibCodecFactory in order to use ZLib implentation instead of
JDK if windowSize or memLevel is different from the default one.
- Add WebSocketServerExtensionHandler and
WebSocketClientExtensionHandler to handle WebSocket Extension headers.
- Add DeflateFrame and PermessageDeflate extension implementations.
2014-07-09 16:28:40 +02:00
Norman Maurer
696e355a70 Move generic code to HttpOrSpdyChooser to simplify implementations
Motivation:

HttpOrSpdyChooser can be simplified so the user not need to implement getProtocol(...) method.

Modification:

Add implementation for the method. The user can override it if necessary.

Result:

Easier usage of HttpOrSpdyChooser.
2014-07-07 09:44:50 +02:00
Trustin Lee
330404da07 Fix most inspector warnings
Motivation:

It's good to minimize potentially broken windows.

Modifications:

Fix most inspector warnings from our profile

Result:

Cleaner code
2014-07-02 19:04:11 +09:00
Norman Maurer
bebcf81687 Fix compile error 2014-06-28 22:27:15 +02:00
Jay
db3c592e6a Maintain decoder result in HttpObjectAggregator
Motivation:
DecodeResult is dropped when aggregate HTTP messages.

Modification:

Make sure we not drop the DecodeResult while aggregate HTTP messages.

Result:

Correctly include the DecodeResult for later processing.
2014-06-28 22:09:46 +02:00
Trustin Lee
6071c9c267 Remove the deprecated methods in SPDY and HTTP
No need to keep ABI compatibility since we are working on a
backward-incompatible branch
2014-06-27 16:55:23 +09:00
Alexey Parfenov
087d329d07 Fix integer overflow in HttpObjectEncoder when handling chunked encoding and FileRegion > Integer.MAX_VALUE
Motivation:

Due to integer overflow bug, writes of FileRegions to http server pipeline (eg like one from HttpStaticFileServer example) with length greater than Integer.MAX_VALUE are ignored in 1/2 of cases (ie no data gets sent to client)

Modification:

Correctly handle chunk sized > Integer.MAX_VALUE

Result:

Be able to use FileRegion > Integer.MAX_VALUE when using chunked encoding.
2014-06-24 12:01:27 +02:00
Trustin Lee
24e17d1832 Remove 'get' prefix from all HTTP/SPDY messages
Motivation:

Persuit for the consistency in method naming

Modifications:

- Remove the 'get' prefix from all HTTP/SPDY message classes
- Fix some inspector warnings

Result:

Consistency
2014-06-24 18:41:27 +09:00
Trustin Lee
760bbc7ea6 Refactor FastThreadLocal to simplify TLV management
Motivation:

When Netty runs in a managed environment such as web application server,
Netty needs to provide an explicit way to remove the thread-local
variables it created to prevent class loader leaks.

FastThreadLocal uses different execution paths for storing a
thread-local variable depending on the type of the current thread.
It increases the complexity of thread-local removal.

Modifications:

- Moved FastThreadLocal and FastThreadLocalThread out of the internal
  package so that a user can use it.
- FastThreadLocal now keeps track of all thread local variables it has
  initialized, and calling FastThreadLocal.removeAll() will remove all
  thread-local variables of the caller thread.
- Added FastThreadLocal.size() for diagnostics and tests
- Introduce InternalThreadLocalMap which is a mixture of hard-wired
  thread local variable fields and extensible indexed variables
- FastThreadLocal now uses InternalThreadLocalMap to implement a
  thread-local variable.
- Added ThreadDeathWatcher.unwatch() so that PooledByteBufAllocator
  tells it to stop watching when its thread-local cache has been freed
  by FastThreadLocal.removeAll().
- Added FastThreadLocalTest to ensure that removeAll() works
- Added microbenchmark for FastThreadLocal and JDK ThreadLocal
- Upgraded to JMH 0.9

Result:

- A user can remove all thread-local variables Netty created, as long as
  he or she did not exit from the current thread. (Note that there's no
  way to remove a thread-local variable from outside of the thread.)
- FastThreadLocal exposes more useful operations such as isSet() because
  we always implement a thread local variable via InternalThreadLocalMap
  instead of falling back to JDK ThreadLocal.
- FastThreadLocalBenchmark shows that this change improves the
  performance of FastThreadLocal even more.
2014-06-19 21:17:46 +09:00
Trustin Lee
de2872f7f7 Introduce TextHeaders and AsciiString
Motivation:

We have quite a bit of code duplication between HTTP/1, HTTP/2, SPDY,
and STOMP codec, because they all have a notion of 'headers', which is a
multimap of string names and values.

Modifications:

- Add TextHeaders and its default implementation
- Add AsciiString to replace HttpHeaderEntity
  - Borrowed some portion from Apache Harmony's java.lang.String.
- Reimplement HttpHeaders, SpdyHeaders, and StompHeaders using
  TextHeaders
- Add AsciiHeadersEncoder to reuse the encoding a TextHeaders
  - Used a dedicated encoder for HTTP headers for better performance
    though
- Remove shortcut methods in SpdyHeaders
- Remove shortcut methods in SpdyHttpHeaders
- Replace SpdyHeaders.getStatus() with HttpResponseStatus.parseLine()

Result:

- Removed quite a bit of code duplication in the header implementations.
- Slightly better performance thanks to improved header validation and
  hash code calculation
2014-06-14 17:14:30 +09:00
belliottsmith
7d37af5dfb Introduce FastThreadLocal which uses an EnumMap and a predefined fixed set of possible thread locals
Motivation:
Provide a faster ThreadLocal implementation

Modification:
Add a "FastThreadLocal" which uses an EnumMap and a predefined fixed set of possible thread locals (all of the static instances created by netty) that is around 10-20% faster than standard ThreadLocal in my benchmarks (and can be seen having an effect in the direct PooledByteBufAllocator benchmark that uses the DEFAULT ByteBufAllocator which uses this FastThreadLocal, as opposed to normal instantiations that do not, and in the new RecyclableArrayList benchmark);

Result:
Improved performance
2014-06-13 11:02:16 +02:00
Frederic Bregier
89ed1517f5 [#2542] HTTP post request decoder does not support quoted boundaries
Motivation:
According to RFC2616 section 19, boundary string could be quoted, but
currently the PostRequestDecoder does not support it while it should.

Modifications:
Once the boundary is found, one check is made to verify if the boundary
is "quoted", and if so, it is "unqoted".

Note: in following usage of this boundary (as delimiter), quote seems no
more allowed according to the same RFC, so the reason that only the
boundary definition is corrected.

Result:
Now the boundary could be whatever quoted or not. A Junit test case
checks it.
2014-06-08 12:07:44 +02:00
Frederic Bregier
c16de8fcbc [#2544] Correctly parse Multipart-mixed POST HTTP request in case of entity ends with odd number of 0x0D
Motivation:
When an attribute is ending with an odd number of CR (0x0D), the decoder
add an extra CR in the decoded attribute and should not.

Modifications:
Each time a CR is detected, the next byte was tested to be LF or not. If
not, in a number of places, the CR byte was lost while it should not be.
When a CR is detected, if the next byte is not LF, the CR byte should be
saved as the position point to the next byte (not LF). When a CR is
detected, if there is not yet other available bytes, the position is
reset to the position of CR (since a LF could follow).

A new Junit test case is added, using DECODER and variable number of CR
in the final attribute (testMultipartCodecWithCRasEndOfAttribute).

Result:
The attribute is now correctly decoded with the right number of CR
ending bytes.
2014-06-08 11:22:51 +02:00
Trustin Lee
2e98566916 Introduce MessageAggregator and DecoderResultProvider
Motivation:

We have different message aggregator implementations for different
protocols, but they are very similar with each other.  They all stems
from HttpObjectAggregator.  If we provide an abstract class that provide
generic message aggregation functionality, we will remove their code
duplication.

Modifications:

- Add MessageAggregator which provides generic message aggregation
- Reimplement all existing aggregators using MessageAggregator
- Add DecoderResultProvider interface and extend it wherever possible so
  that MessageAggregator respects the state of the decoded message

Result:

Less code duplication
2014-06-05 16:39:59 +09:00
Josh Hoyt
2c38e6b5d9 codec-http: Document the semantics of HttpResponseStatus equality and comparison 2014-05-30 07:52:58 +02:00
nmittler
086dc9140a Adding a general handler for upgrading protocols
Adding full implementation for cleartext upgrade from HTTP to HTTP/2.
2014-05-19 07:30:18 -07:00
Daniel Bevenius
b1a51f0669 Adding short-curcuit option for CORS
Motivation:
CORS request are currently processed, and potentially failed, after the
target ChannelHandler(s) have been invoked. This might not be desired, for
example a HTTP PUT or POST might have been performed.

Modifications:
Added a shortCurcuit option to CorsConfig which when set will
cause a validation of the HTTP request's 'Origin' header and verify that
it is valid according to the configuration. If found invalid an 403
"Forbidden" response will be returned and not further processing will
take place.

This is indeed no help for non browser request, like using curl, which
can set the 'Origin' header.

Result:
Users can now configure if the 'Origin' header should be validated
upfront and have the request rejected before any further processing
takes place.
2014-05-06 12:06:34 +02:00
Norman Maurer
e1becd2147 Fix buffer leak introduced by #2462
Motivation:
Because of not correctly release a buffer before null out the reference a memory leak shows up.

Modifications:
Correct call buffer.release() before null out reference.

Result:
No more leak
2014-05-06 10:06:19 +02:00
Jeff Pinner
1aca3863eb SPDY: ensure SpdyHeaderBlockRawDecoder always reads entire input 2014-05-05 07:32:37 +02:00
Jeff Pinner
3f50a9cde3 SPDY: release headerBlock in SpdyFrameCodec 2014-05-05 07:26:39 +02:00
Jeff Pinner
78963635ba SPDY: remove unused code in SpdyHeaderBlockRawEncoder 2014-05-05 07:01:23 +02:00
Sun Ning
31b01c87e4 Made websocket maxFramePayloadSize configurable from WebSocketServerProtocolHandler.
Motivation:

Currently there's no way to configure maxFramePayloadSize from
WebSocketServerProtocolHandler, which is the most used entry point of
WebSocket server.

Modifications:

Added another constructor for maxFramePayloadSize.

Result:

We can configure max frame size for websocket packet in
WebSocketServerProtocolHandler. It will also keep backward compatibility
with default max size: 65536. (65536 is hard-coded max size in previous
version of Netty)
2014-05-05 06:24:40 +02:00
Jeff Pinner
c09d85f699 SPDY: allow subclassing of SpdyFrameCodec 2014-05-03 14:44:08 +02:00
Norman Maurer
8599ab5bdb Remove ContinuationWebSocketFrame.aggregatedText()
Motivation:
Before we aggregated the full text in the WebSocket08FrameDecoder just to fill in the ContinuationWebSocketFrame.aggregatedText(). The problem was that there was no upper-limit and so it would be possible to see an OOME if the remote peer sends a TextWebSocketFrame + a never ending stream of ContinuationWebSocketFrames. Furthermore the aggregation does not really belong in the WebSocket08FrameDecoder, as we provide an extra ChannelHandler for this anyway (WebSocketFrameAggregator).

Modification:
Remove the ContinuationWebSocketFrame.aggregatedText() method and corresponding constructor. Also refactored WebSocket08FrameDecoder a bit to me more efficient which is now possible as we not need to aggregate here.

Result:
No more risk of OOME because of frames.
2014-04-30 14:49:37 +02:00
Daniel Bevenius
32e0170041 Adding support for echoing the request origin for CORS.
Motivation:
When CORS has been configured to allow "*" origin, and at the same time
is allowing credentials/cookies, this causes an error from the browser
because when the response 'Access-Control-Allow-Credentials' header
is true, the 'Access-Control-Allow-Origin' must be an actual origin.

Modifications:
Changed CorsHandler setOrigin method to check for the combination of "*"
origin and allowCredentials, and if the check matches echo the CORS
request's 'Origin' value.

Result:
This addition enables the echoing of the request 'Origin' value as the
'Access-Control-Allow-Origin' value when the server has been configured
to allow any origin in combination with allowCredentials.

This allows client requests to succeed when expecting the server to
be able to handle "*" origin and at the same time be able to send cookies
by setting 'xhr.withCredentials=true'. A concrete example of this is
the SockJS protocol which expects behaviour.
2014-04-25 18:20:18 +02:00
Trustin Lee
872d4c5bc1 Synchronized between 4.1 and master again (part 2)
Motivation:
4 and 5 were diverged long time ago and we recently reverted some of the
early commits in master.  We must make sure 4.1 and master are not very
different now.

Modification:
Remove ChannelHandlerInvoker.writeAndFlush(...) and the related
implementations.

Result:
4.1 and master got closer.
2014-04-25 15:07:12 +09:00
Trustin Lee
d2614cfc01 Synchronized between 4.1 and master
Motivation:

4 and 5 were diverged long time ago and we recently reverted some of the
early commits in master.  We must make sure 4.1 and master are not very
different now.

Modification:

Fix found differences

Result:

4.1 and master got closer.
2014-04-25 00:36:01 +09:00
Norman Maurer
cc320a4be6 [#2401] Improve documentation of HttpObjectAggregator
Motivation:
Make it more clear what the output of HttpObjectAggregator is and that it need to come after the encoder in the pipeline.

Modifications:
Change javadocs to make things more clear.

Result:
Better docs
2014-04-17 15:42:47 +02:00
Norman Maurer
7a55436aa4 Fix missed buffer leaks in SpdyFrameDecoderTest
Motivation:
 Fix leaks reported during running SpdyFrameDecoderTest

Modifications:
Make sure the produced buffers of SpdyFrameDecoder and SpdyFrameDecoderTest are released

Result:

No more leak reports during run the tests.
2014-04-16 14:03:01 +02:00
Norman Maurer
f9884ded7c Fix buffer leaks in SpdyFrameDecoderTest
Motivation:

Fix leaks reported during running SpdyFrameDecoderTest

Modifications:
Make sure the produced buffer of SpdyFrameDecoder is released

Result:

No more leak reports during run the tests.
2014-04-16 10:48:12 +02:00
Norman Maurer
f443754af6 Fix buffer leaks in SPDY test
Motivation:

Fix leaks reported during SPDY test.

Modifications:

Use ReferenceCountUtil.releaseLater(...) to make sure everything is released once the tests are done.

Result:

No more leak reports during run the tests.
2014-04-16 06:49:25 +02:00
Norman Maurer
58423a448a Add missing override annotations 2014-04-15 20:15:30 +02:00
Jeff Pinner
350c8ef42a SPDY: refactor frame codec implementation
Motivation:

Currently, the SPDY frame encoding and decoding code is based upon
the ChannelHandler abstraction. This requires maintaining multiple
versions for 3.x and 4.x (and possibly 5.x moving forward).

Modifications:

The SPDY frame encoding and decoding code is separated from the
ChannelHandler and SpdyFrame abstractions. Also test coverage is
improved.

Result:

SpdyFrameCodec now implements the ChannelHandler abstraction and is
responsible for creating and handling SpdyFrame objects.
2014-04-15 18:42:08 +02:00
Daniel Bevenius
7c6f862d98 Adding origins whitelist support for CORS
Motivation:
Currently the CORS support only handles a single origin, or a wildcard
origin. This task should enhance Netty's CORS support to allow multiple
origins to be specified. Just being allowed to specify one origin is
particulary limiting when a site support both http and https for
example.

Modifications:
- Updated CorsConfig and its Builder to accept multiple origins.

Result:
Users are now able to configure multiple origins for CORS.

[https://github.com/netty/netty/issues/2346]
2014-03-30 19:40:59 +02:00
Brendt Lucas
98bc7b7859 [#2234] Use QueryStringDecoder.decodeComponent to decode url-encoded data instead of Java's URLDecoder.
Motivation:
Previously, we used URLDecoder.decode(...) to decode url-encoded data. This generates a lot of garbage and takes a considerable amount of time.

Modifications:
Replace URLDecoder.decode(...) with QueryStringDecoder.decodeComponent(...)

Result:
Less garbage to GC and faster decode processing.
2014-03-22 14:02:35 +01:00
Daniel Bevenius
27671e0dbd Fixing CorsConfigTest failure under Java 8.
Motivation:
When running the build with Java 8 the following error occurred:

java: reference to preflightResponseHeader is ambiguous
  both method
  <T>preflightResponseHeader(java.lang.CharSequence,java.lang.Iterable<T>)
  in io.netty.handler.codec.http.cors.CorsConfig.Builder and method
  <T>preflightResponseHeader(java.lang.String,java.util.concurrent.Callable<T>)
  in io.netty.handler.codec.http.cors.CorsConfig.Builder match

The offending class was CorsConfigTest and its shouldThrowIfValueIsNull
which contained the following line:
withOrigin("*").preflightResponseHeader("HeaderName", null).build();

Modifications:
Updated the offending method with to supply a type, and object array, to
avoid the error.

Result:
After this I was able to build with Java 7 and Java 8
2014-03-22 07:20:10 +01:00
Daniel Bevenius
321e770c9b Adding support for specifying preflight response headers.
Motivation:

An intermediary like a load balancer might require that a Cross Origin
Resource Sharing (CORS) preflight request have certain headers set.
As a concrete example the Elastic Load Balancer (ELB) requires the
'Date' and 'Content-Length' header to be set or it will fail with a 502
error code.

This works is an enhancement of https://github.com/netty/netty/pull/2290

Modifications:

CorsConfig has been extended to make additional HTTP response headers
configurable for preflight responses. Since some headers, like the
'Date' header need to be generated each time, m0wfo suggested using a
Callable.

Result:

By default, the 'Date' and 'Content-Lenght' headers will be sent in a
preflight response. This can be overriden and users can specify
any headers that might be required by different intermediaries.
2014-03-21 15:05:29 +01:00
Frederic Bregier
ef53227836 [#2305] Fix issue related to decoding post request raized an exception due to a split of information by chunk not correctly taken into account by the decoder
Motivation:

If the last item analyzed in a previous received HttpChunk/HttpContent was a part of an attribute's name, the read index was not set to the new right place and therefore raizing an exception in some case (since the "new" name analyzed is empty, which is not allowed so the exception).

What appears there is that the read index should be reset to the last valid position encountered whatever the case. Currently it was set when only when there is an attribute not already finished (name is ok, but content is possibly not).

Therefore the issue is that elements could be rescanned multiple times (including completed elements) and moreover some bad decoding can occur such as when in a middle of an attribute's name.

Modifications:

To fix this issue, since "firstpos" contains the last "valid" read index of the decoding (when finding a '&', '=', 'CR/LF'), we should add the setting of the read index for the following cases:

'lastchunk' encountered, therefore finishing the current buffer
any other cases than current attribute is not finished (name not found yet in particular)
So adding for this 2 cases:

undecodedChunk.readerIndex(firstpos);

Result:

Now the decoding is done once, content is added from chunk/content to chunk/content, name is decoded correctly even if in the middle of 2 chunks/contents.
A Junit test code was added: testChunkCorrect that should not raized any exception.
2014-03-14 09:46:58 +01:00
David Dossot
c31e8f59ba added support for empty query parameters 2014-03-10 06:29:45 +01:00
Chris Mowforth
59bfd69c9b Add content length and date headers to CORS response 2014-03-06 20:59:53 +01:00
Dao Ngoc
94bcbba324 Fix #2275 %s WebSocket version %s server handshake 2014-02-28 17:18:58 -08:00
Trustin Lee
b6aa032376 Fix checkstyle 2014-02-26 16:18:39 -08:00
Trustin Lee
c42570576e Fix a bug where HttpObjectDecoder generates two LastHttpContent consecutively
Motivation:
When an HttpResponseDecoder decodes an invalid chunk, a LastHttpContent instance is produced and the decoder enters the 'BAD_MESSAGE' state, which is not supposed to produce a message any further.  However, because HttpObjectDecoder.invalidChunk() did not clear this.message out to null, decodeLast() will produce another LastHttpContent message on a certain situation.

Modification:
Do not forget to null out HttpObjectDecoder.message in invalidChunk(), and add a test case for it.

Result:
No more consecutive LastHttpContent messages produced by HttpObjectDecoder.
2014-02-26 15:54:45 -08:00
Jeff Pinner
97f120330b SPDY: remove SPDY/3 support 2014-02-20 15:02:16 -08:00
Trustin Lee
ff8df41541 Fix another leak in HttpObjectAggregatorTest by removing redundant code 2014-02-20 14:30:42 -08:00
Trustin Lee
2e6918c21a Fix a memory leak in HttpObjectAggregatorTest 2014-02-20 13:49:56 -08:00
Trustin Lee
f4b9aaaf18 Add an operation that resets the state of HttpObjectDecoder
Motivation:

Currently, it is impossible to give a user the full control over what to do in response to the request with 'Expect: 100-continue' header.  Currently, a user have to do one of the following:

- Accept the request and respond with 100 Continue, or
- Send the reject response and close the connection.

.. which means it is impossible to send the reject response and keep the connection alive so that the client sends additional requests.

Modification:

Added a public method called 'reset()' to HttpObjectDecoder so that a user can reset the state of the decoder easily.  Once called, the decoder will assume the next input will be the beginning of a new request.

HttpObjectAggregator now calls `reset()`right after calling 'handleOversizedMessage()' so that the decoder can continue to decode the subsequent request even after the request with 'Expect: 100-continue' header is rejected.

Added relevant unit tests / Minor clean-up

Result:

This commit completes the fix of #2211
2014-02-20 13:42:58 -08:00
Trustin Lee
7d238aaf45 Added a test for oversized HTTP responses 2014-02-20 11:47:19 -08:00
Trustin Lee
5f62821b84 Remove the deprecated constant field
Exposing the constant as a public static final field is usually not a
good idea because it gives less flexibility to us when we happen to
change our internal implementation.
2014-02-20 11:38:49 -08:00
Trustin Lee
b2174fccc1 Overall clean-up of HttpObjectAggregator / Handle oversized response differently
- Related: #2211
2014-02-20 11:37:35 -08:00
Chris Mowforth
84946b617b Expose a callback in HttpObjectAggregator to handle oversized messages
- Related: #2211
2014-02-20 11:37:35 -08:00
Trustin Lee
97662a6c94 Fix a bug where HttpObjectAggregator doesn't always produce FullHttpMessage
- Fixes #2182
- Always convert an unfull invalid message to a full message
2014-02-19 15:33:53 -08:00
Trustin Lee
acc781ba1b Ensure that the last bad chunk is LastHttpChunk 2014-02-19 15:20:58 -08:00
Trustin Lee
18d91b8bf6 HttpObjectDecoder.invalidChunk() should return LastHttpChunk
because the decoder will enter 'BAD_MESSAGE' state and will not produce any chunk since then.
2014-02-19 14:59:26 -08:00
Trustin Lee
c73e1e3c1e Fix inefficient loop in QueryStringDecoder.decodeComponent()
- Fixes #2228
- There's no point of looping until the end of the component if at least one '%' or '+' is found.
2014-02-14 13:27:08 -08:00
Trustin Lee
ce9403cc3a Do not use String.format() for log message generation
- It's slow.
2014-02-13 19:31:45 -08:00
Norman Maurer
b07acfc28b [#2219] Fix javadocs of HttpObjectAggregator to show correct handler order in pipeline 2014-02-09 17:10:38 +01:00
Trustin Lee
ad121c73b0 Prefer interface to implementation in type declaration
This fixes the build failure with JDK 8 due to animal sniffer error
2014-02-08 08:50:41 -08:00
Daniel Bevenius
96216a9657 Fixing spelling of 'request' for handler names. 2014-02-07 13:51:02 +01:00
Trustin Lee
2b769c6daf Fix resource leaks in WebSocketServerProtocolHandler
- Related: #1975
2014-02-06 21:23:10 -08:00
Trustin Lee
8738bc4ae7 Clean up HttpObjectAggregator 2014-02-06 21:00:24 -08:00
Trustin Lee
c01f08d306 Fix a leak in WebSocketServerProtocolHandshakeHandler
- Related: #1975
2014-02-06 20:57:55 -08:00
Norman Maurer
4f6ccbbb78 [#2173] Fix regression that let HttpRequestDecoder fail if the websocket response and a websocketframe are send in one go 2014-02-06 10:48:48 +01:00
Trustin Lee
0f1b1be0aa Enable a user specify an arbitrary information with ReferenceCounted.touch()
- Related: #2163
- Add ResourceLeakHint to allow a user to provide a meaningful information about the leak when touching it
- DefaultChannelHandlerContext now implements ResourceLeakHint to tell where the message is going.
- Cleaner resource leak report by excluding noisy stack trace elements
2014-01-29 11:44:59 +09:00
Trustin Lee
b887e35ac2 Add ReferenceCounted.touch() / Add missing retain() overrides
- Fixes #2163
- Inspector warnings
2014-01-28 20:06:55 +09:00
Norman Maurer
f122118bf7 Not throw an exception if subprotocol is not supported but just drop the header as stated in the RFC's 2014-01-26 11:00:40 +01:00
Trustin Lee
714ed77e17 Add a log message on deletion failure 2014-01-24 16:45:20 +09:00
Frederic Bregier
19f9bcfe8b Fix fileChannel not closed, preventing delete to occur correctly 2014-01-24 16:45:20 +09:00
Frederic Bregier
ce39e6409e Fix try to delete file while it does not exist anymore (see #2102 and #1713) in reference to this issue 2014-01-21 14:16:59 +09:00
Trustin Lee
a69c4f2371 Fix IllegalReferenceCountException which occurs when multipart data is stored in memory
- Fixes #2102
2014-01-19 17:51:20 +09:00
Trustin Lee
a4e587c46b Fix resource leak in HttpPostRequestEncoderTest / Typo 2014-01-19 16:52:13 +09:00
Trustin Lee
5f9854d8ac Fix resource leak in InternalAttribute.setValue() 2014-01-19 16:51:31 +09:00
Trustin Lee
eb4ef433c6 Fix a test failure in Windows due to newline differences 2014-01-19 16:13:49 +09:00
Trustin Lee
f97f6b938e Use package-local fields instead of reflection / Reapply 4dc78c10ee / Inspector warnings 2014-01-19 14:58:48 +09:00
IonuÈ› G. Stan
709be30442 Add an HTML5 encoder mode for HttpPostRequestEncoder 2014-01-19 14:58:28 +09:00
Trustin Lee
a0714d1da3 Revert "+= won't let compile optimize String concats"
This reverts commit 4dc78c10ee.
2014-01-19 14:39:29 +09:00
Norman Maurer
1f049367a4 Make use of ByteBufProcessor for extract initial line and headers
This gives some nice performance boost as readByte() is quite expensive because of the index / replay checks.
2014-01-16 20:14:28 +01:00
Stephane Landelle
4dc78c10ee += won't let compile optimize String concats 2014-01-16 20:09:11 +01:00
Michael Nitschinger
75cb91ed0c Rename HttpDataFactory.*datas to *data. Fixes #2094 2014-01-16 09:52:25 +01:00
Daniel Bevenius
6a954d5b47 Adding a Cross Origin Resource Sharing (CORS) handler. 2014-01-16 07:58:32 +01:00
Norman Maurer
29484a44d5 Fix inspector warnings 2014-01-15 17:02:04 +01:00
Veebs
4c0be9b57c Add HttpChunkedInput for easier streaming of chunked content 2014-01-15 15:33:29 +09:00
Trustin Lee
2f9ad15d3f Use the source frame's allocator instead of the unpooled allocator 2014-01-14 14:22:06 +09:00
Trustin Lee
ae6aea794c Fix checkstyle 2014-01-13 23:35:52 +09:00
Trustin Lee
1a54ff843c Port the changes in SpdyHeaderBlockRawDecoder to master & 4.0
Somehow it slipped away.
2014-01-13 23:07:51 +09:00
Trustin Lee
ad70e69e47 Port the changes in SpdyHeaderBlockZlibEncoder to master & 4.0
Somehow it slipped away.
2014-01-13 23:01:07 +09:00
Trustin Lee
e1ab46f56a Fix a bug where SpdyHeaderBlockZlibDecoder fails to decompress
- Forward-port 4c35b593c1, originally written by @wgallagher
- Fixes #2077
2014-01-13 22:40:34 +09:00
Veebs
a389d69ba8 Replaced deprecated 'sendUnsupportedWebSocketVersionResponse()' with 'sendUnsupportedVersionResponse()' 2014-01-13 06:33:25 +01:00
Norman Maurer
4ee81d9235 Cleanup and remove get prefix of method to be more inline with our API 2014-01-11 14:19:43 +01:00
Leonardo Freitas Gomes
7324ce6fde Improves SelectedProtocol enum to avoid boilerplate on subclasses
NPN server providers return a String version of the negotiated protocol
and the getProtocolByName method allows to easily get an instance of
the SelectedProtocol enum and avoid the need for a switch statement in
each subclass to match the String against the enum value.
2014-01-11 13:54:46 +01:00
Trustin Lee
0244e35fa6 Fix a bug where HttpObjectDecoder produces LastHttpContent after entering BAD_MESSAGE state
- Fixes #2103
- Added LastHttpContent.EMPTY_LAST_CONTENT.toString() for athestic reasons.
2014-01-10 16:16:02 +09:00
Norman Maurer
2670bb4aae [#2090] Correctly format DefaultHttpRequest#toString() 2014-01-06 17:34:19 +01:00
Norman Maurer
906ac233a2 [#2088] Introduce sendUnsupportedVersionResponse(...) methods which allows to use the ChannelFuture/ChannelPromise to get notified once the response was send.
Also mark the old method as deprecated.
2014-01-03 11:13:05 +01:00
Trustin Lee
f3a842ecca [maven-release-plugin] prepare for next development iteration 2013-12-22 22:06:15 +09:00
Trustin Lee
888dfba76f [maven-release-plugin] prepare release netty-5.0.0.Alpha1 2013-12-22 22:06:06 +09:00
Trustin Lee
ddacf784c1 Use LinkedHashSet for HttpHeaders.names() and add Javadoc about the performance of names() and entries() 2013-12-22 19:18:27 +09:00
Trustin Lee
906e2d2b37 Fix a leak in HttpResponseDecoderTest 2013-12-19 01:13:02 +09:00
Norman Maurer
ca1c82642a Limit the access of the HttpHeaders for performance reasons 2013-12-18 23:08:39 +09:00
Trustin Lee
d35c2010c4 Stop decoding after protocol upgrade / Do not use singleDecode option 2013-12-18 23:08:32 +09:00
Trustin Lee
fa336db891 Remove unnecessary code from HttpObjectDecoder and re-enable all HTTP tests
- Since Netty 4, HTTP decoder does not generate a full message at all.  Therefore, there's no need to keep separate states for the content smaller than maxChunkSize.
- maxChunkSize must be greater than 0. Setting it to 0 should not disable chunked encoding. We have a dedicated flag for that.
- Uncommented the tests that were commented out for an unknown reason, with some fixes.
- Added more tests for HTTP decoder.
- Removed the Ignore annotation on some tests.
2013-12-18 23:08:26 +09:00
Trustin Lee
94d6e44bba Change the return type of EmbeddedChannel.read*() from Object to an ad-hoc type parameter
.. so that there's no need to explicitly down-cast.

Fixes #2067
2013-12-16 22:22:47 +09:00
Trustin Lee
61ed9476ae Improve the unit test for #1742 2013-12-16 21:55:14 +09:00
Trustin Lee
af07cc95fa Fix a bug where HttpObjectAggregator fails to send a '100 Continue' response
- Fixes #1742
2013-12-16 21:45:10 +09:00
Frederic Bregier
d7faf042a1 Add support for 'charset' property for multipart boundaries
- Fixes #2004
2013-12-16 14:30:10 +09:00
Jeff Pinner
f41e2f830d Merge pull request #2064 from jpinner/spdy_minor_version
SPDY: add SETTINGS_MINOR_VERSION
2013-12-15 21:10:10 -08:00
Jeff Pinner
78f3ee3396 SPDY: add SETTINGS_MINOR_VERSION 2013-12-14 10:27:14 -08:00
Norman Maurer
48b416f112 Optimize encoding of websocket frames by merge if possible 2013-12-13 11:39:39 +01:00
Trustin Lee
b6ac202c15 Fix buffer leak in AbstractMemoryHttpData 2013-12-07 11:13:50 +09:00
Norman Maurer
643ce2f8c0 Fix all leaks reported during tests
- One notable leak is from WebSocketFrameAggregator
- All other leaks are from tests
2013-12-07 00:47:30 +09:00
Trustin Lee
f2bb5f1b4c Allow specifying a custom charset for a multipart Attribute
- Fixed #2025
- Adapted from @BabyDuncan's pull request: #2031
- Overall ugliness clean-up
2013-12-05 11:23:23 +09:00
Norman Maurer
ec8967f9ff Replace usage of StringBuilder by AppendableCharSequence for performance reasons 2013-12-03 14:47:11 +01:00
Norman Maurer
5106382b44 Port the HTTP / RTSP encoder and decoder from 4.0 branch as those are just faster. 2013-12-02 20:53:39 +01:00
Norman Maurer
4a680a3b49 Remove the use of ThreadLocal for performance reasons 2013-11-29 19:18:27 +01:00
Norman Maurer
4ea39aa9df Fix test 2013-11-28 13:50:08 +01:00
Norman Maurer
dab954dfb5 Merge buffers for performance reasons if possible when encode http requests/responses. 2013-11-28 13:48:34 +01:00
Trustin Lee
387f6917af Add test case for HttpHeaders.equalsIgnoreCase(...) 2013-11-28 18:24:03 +09:00
Trustin Lee
d4fd7ff458 Fix NPE raised by HttpHeaders.equalsIgnoreCases() when one of its arguments is null 2013-11-28 15:51:13 +09:00
Norman Maurer
b067566a7c Optimize the decoding and encoding of HTTP for better performance
* No need to calculate the hash of known header names multiple times
* Optimize header encoding
* Also optimize the encoding of known header values
* Optimize encoding of HTTP method, version and status
2013-11-27 12:35:58 +01:00
Trustin Lee
110745b0eb Remove the distinction of inbound handlers and outbound handlers
- Fixes #1808
- Move all methods in ChannelInboundHandler and ChannelOutboundHandler up to ChannelHandler
- Remove ChannelInboundHandler and ChannelOutboundHandler
- Deprecate ChannelInboundHandlerAdapter, ChannelOutboundHandlerAdapter, and ChannelDuplexHandler
- Replace CombinedChannelDuplexHandler with ChannelHandlerAppender
  because it's not possible to combine two handlers into one easily now
- Introduce 'Skip' annotation to pass events through efficiently
- Remove all references to the deprecated types and update Javadoc
2013-11-27 17:31:28 +09:00
Trustin Lee
807d96ed6c Simplify bundle generation / Add io.netty.versions.properties to all JARs
- Fixes #2003 properly
- Instead of using 'bundle' packaging, use 'jar' packaging.  This is
  more robust because some strict build tools fail to retrieve the
  artifacts from a Maven repository unless their packaging is not 'jar'.
- All artifacts now contain META-INF/io.netty.version.properties, which
  provides the detailed information about the build and repository.
- Removed OSGi testsuite temporarily because it gives false errors
  during split package test and examination.
- Add io.netty.util.Version for easy retrieval of version information
2013-11-26 22:00:14 +09:00
Norman Maurer
10e3651bf8 [#2006] Correctly handle HttpMessage / HttpContent implementations 2013-11-26 10:57:50 +01:00
Norman Maurer
98845cb392 [#2005] Allow to use HttpContentDecompressor in strict mode 2013-11-26 08:20:56 +01:00
Norman Maurer
daeaaf25a9 Respect validateHeaders when duplicate/copy the response/request 2013-11-26 08:12:58 +01:00
Norman Maurer
279e2e3cd2 Optimize DefaultHttpHeaders.contains(...) 2013-11-24 14:42:33 +01:00
Norman Maurer
8eb88b7889 Optimize check for equals header name and value 2013-11-24 14:32:55 +01:00
Norman Maurer
a01f246091 Fix compile errors caused by last commit 2013-11-24 10:32:05 +01:00
Norman Maurer
f84a50d3cb Use the same HttpHeaders optimisations in master as in 4.0 2013-11-24 10:19:15 +01:00
Norman Maurer
0fa6dcefc5 [#1682] HttpVersion.valueOf(...) should not convert to uppercase 2013-11-23 20:35:47 +01:00
Trustin Lee
132af3a485 Introduce ChannelHandlerInvoker, dedeciated for invoking event handler methods, and move most handler invocation code in ChannelHandlerContext to the default ChannelHandlerInvoker implementation
- Fixes #1912
- Add ChannelHandlerInvoker and its default implementation
- Add pipeline manipulation methods that accept ChannelHandlerInvoker
- Rename Channel(Inbound|Outbound)Invoker to
  Channel(Inbound|Outbound)Ops to avoid confusion
- Remove the Javadoc references to the package-private interfaces
2013-11-21 14:14:23 +09:00
Daniel Bevenius
624573971f Support for empty HTTP request header values. 2013-11-17 15:49:14 +01:00
Norman Maurer
6306b83e2d [#1982] Limit the usage of ThreadLocal.get() for performance reasons 2013-11-14 09:48:33 +01:00
Norman Maurer
41ccbfd388 Also allow to disable header validation via HttpServerCodec and HttpClientCodec. Related to [#1981] 2013-11-14 09:37:57 +01:00
Daniel Bevenius
7cc2507076 Adding check for zero readable bytes in WebSocket08FrameDecoder decode. 2013-11-13 07:11:05 +01:00
Trustin Lee
1f0a73c238 Fix checkstyle 2013-11-04 19:52:10 +09:00