Commit Graph

429 Commits

Author SHA1 Message Date
Norman Maurer
398efb1f71 Ensure valid message sequence if channel is closed before receive headers.
Motivation:

When the channel is closed while we still decode the headers we currently not preserve correct message sequence. In this case we should generate an invalid message with a current cause.

Modifications:

Create an invalid message with a PrematureChannelClosureException as cause when the channel is closed while we decode the headers.

Result:

Correct message sequence preserved and correct DecoderResult if the channel is closed while decode headers.
2016-06-09 22:42:46 +02:00
Norman Maurer
7b25402e80 Add CompositeByteBuf.addComponent(boolean ...) method to simplify usage
Motivation:

At the moment the user is responsible to increase the writer index of the composite buffer when a new component is added. We should add some methods that handle this for the user as this is the most popular usage of the composite buffer.

Modifications:

Add new methods that autoamtically increase the writerIndex when buffers are added.

Result:

Easier usage of CompositeByteBuf.
2016-05-21 19:52:16 +02:00
Scott Mitchell
1cb706ac93 HTTP/2 HPACK Header Name Validation and Trailing Padding
Motivation:
The HPACK code currently disallows empty header names. This is not explicitly forbidden by the HPACK RFC https://tools.ietf.org/html/rfc7541. However the HTTP/1.x RFC https://tools.ietf.org/html/rfc7230#section-3.2 and thus HTTP/2 both disallow empty header names, and so this precondition check should be moved from the HPACK code to the protocol level.
HPACK also requires that string literals which are huffman encoded must be treated as an encoding error if the string has more than 7 trailing padding bits https://tools.ietf.org/html/rfc7541#section-5.2, but this is currently not enforced.

Result:
- HPACK to allow empty header names
- HTTP/1.x and HTTP/2 header validation should not allow empty header names
- Enforce max of 7 trailing padding bits

Result:
Code is more compliant with the above mentioned RFCs
Fixes https://github.com/netty/netty/issues/5228
2016-05-17 13:42:16 -07:00
Trustin Lee
3a9f472161 Make retained derived buffers recyclable
Related: #4333 #4421 #5128

Motivation:

slice(), duplicate() and readSlice() currently create a non-recyclable
derived buffer instance. Under heavy load, an application that creates a
lot of derived buffers can put the garbage collector under pressure.

Modifications:

- Add the following methods which creates a non-recyclable derived buffer
  - retainedSlice()
  - retainedDuplicate()
  - readRetainedSlice()
- Add the new recyclable derived buffer implementations, which has its
  own reference count value
- Add ByteBufHolder.retainedDuplicate()
- Add ByteBufHolder.replace(ByteBuf) so that..
  - a user can replace the content of the holder in a consistent way
  - copy/duplicate/retainedDuplicate() can delegate the holder
    construction to replace(ByteBuf)
- Use retainedDuplicate() and retainedSlice() wherever possible
- Miscellaneous:
  - Rename DuplicateByteBufTest to DuplicatedByteBufTest (missing 'D')
  - Make ReplayingDecoderByteBuf.reject() return an exception instead of
    throwing it so that its callers don't need to add dummy return
    statement

Result:

Derived buffers are now recycled when created via retainedSlice() and
retainedDuplicate() and derived from a pooled buffer
2016-05-17 11:16:13 +02:00
Norman Maurer
ef13d19b8b [#5202] Correctly throw ErrorDataDecoderException when invalid encoded form parameters are present.
Motivation:

At the moment we let the IllegalArgumentException escape when parsing form parameters. This is not expected.

Modifications:

Correctly catch IllegalArgumentException and rethrow as ErrorDataDecoderException.

Result:

Throw correct exception.
2016-05-04 21:14:53 +02:00
Daniel Bevenius
0557927b65 Updating allowNullOrigin to return 'null' instead of '*'.
Motivation:
Currently the way a 'null' origin, a request that most often indicated
that the request is coming from a file on the local file system, is
handled is incorrect. We are currently returning a wildcard origin '*'
but should be returning 'null' for the 'Access-Control-Allow-Origin'
which is valid according to the specification [1].

Modifications:
Updated CorsHandler to add a 'null' origin instead of the '*' origin in
the case the request origin is 'null.

Result:
All test pass and the CORS example as does the cors.html example if you
try to serve it by opening the file directly in a web browser.

[1]
https://www.w3.org/TR/cors/#access-control-allow-origin-response-header
2016-05-03 08:39:38 +02:00
Norman Maurer
718bf2fa45 Fix resource-leak which was reported as a result of commit 69070c37ba 2016-04-12 16:27:02 +02:00
Norman Maurer
4652223dec Fix resource leak in test introduced by 69070c37ba 2016-04-10 08:04:57 +02:00
Norman Maurer
f46cfbc590 [#5059] Deprecate method with typo and introduce a new one without typo
Motivation:

There is a spelling error in FileRegion.transfered() as it should be transferred().

Modifications:

Deprecate old method and add a new one.

Result:

Fix typo and can remove the old method later.
2016-04-05 15:06:46 +02:00
Stephane Landelle
881ff3cd98 Drop broken DefaultCookie name validation, close #4999
Motivation:

DefaultCookie constructor performs a name validation that doesn’t match
RFC6265. Moreover, such validation is already performed in strict
encoders and decoders.

Modifications:

Drop DefaultCookie name validation, rely on encoders and decoders.

Result:

no more duplicate broken validation
2016-03-22 12:32:09 +01:00
Stephane Landelle
d747438366 Add ! to allowed cookie value chars
Motivation:

! is missing from allowed cookie value chars, as per https://tools.ietf.org/html/rfc6265#section-4.1.1.
Issue was originally reported on Play!, see https://github.com/playframework/playframework/issues/4460#issuecomment-198177302.

Modifications:

Stick to RFC6265 ranges.

Result:

RFC6265 compliance, ! is supported
2016-03-18 16:58:54 +01:00
Julien Viet
3d7cec6376 Bug fix for HttpPostMultipartRequestDecoder part decoding with an invalid charset not reported as an ErrorDataDecoderException
Motivation:

The current HttpPostMultipartRequestDecoder can decode multipart/form-data parts with a Content-Type that specifies a charset. When this charset is invalid the Charset.forName() throws an unchecked UnsupportedCharsetException. This exception is not catched by the decoder. It should actually be rethrown as an ErrorDataDecoderException, because the developer using the API would expect this validation failure to be reported as such.

Modifications:

Add a catch block for UnsupportedCharsetException and rethrow it as an ErrorDataDecoderException.

Result:

UnsupportedCharsetException are now rethrown as ErrorDataDecoderException.
2016-03-10 18:33:06 +01:00
Dmitry Spikhalskiy
0d3eda38e1 Helper method to get mime-type from Content-Type header of HttpMessage 2016-03-03 15:18:39 +01:00
Xiaoyan Lin
333f55e9ce Add unescapeCsvFields to parse a CSV line and implement CombinedHttpHeaders.getAll
Motivation:

See #4855

Modifications:

Unfortunately, unescapeCsv cannot be used here because the input could be a CSV line like `"a,b",c`. Hence this patch adds unescapeCsvFields to parse a CSV line and split it into multiple fields and unescaped them. The unit tests should define the behavior of unescapeCsvFields.

Then this patch just uses unescapeCsvFields to implement `CombinedHttpHeaders.getAll`.

Result:

`CombinedHttpHeaders.getAll` will return the unescaped values of a header.
2016-02-15 15:26:15 -08:00
Norman Maurer
7ef6db3ffd [#4754] Correctly detect websocket upgrade
Motivation:

If the Connection header contains multiple values (which is valid) we fail to detect a websocket upgrade

Modification:

- Add new method which allows to check if a header field contains a specific value (and also respect multiple header values)
- Use this method to detect handshake

Result:

Correct detect handshake if Connection header contains multiple values (seperated by ',').
2016-02-04 14:03:08 +01:00
Norman Maurer
a0758e7e60 [#4794] Support window size flag by default if ZlibCodecFactory supports it.
Motivation:

If the ZlibCodecFactory can support using a custom window size we should support it by default in the websocket extensions as well.

Modifications:

Detect if a custom window size can be handled by the ZlibCodecFactory and if so enable it by default for PerMessageDeflate*ExtensionHandshaker.

Result:

Support window size flag by default in most installations.
2016-02-04 14:01:40 +01:00
Norman Maurer
7a562943ad [#4533] Ensure replacement of decoder is delayed after finishHandshake() is called
Motivation:

If the user calls handshake.finishHandshake() we need to ensure that the user has the chance to setup the pipeline before any WebSocketFrames are read. Because of this we need
to delay the removal of the HttpRequestDecoder.

Modifications:

- Remove the HttpRequestDecoder via the EventLoop and so delay it which gives the user a chance to setup the pipeline after finishHandshake() completes
- Add unit test for this.

Result:

Less surpising and correct behaviour even if the http response and websocket frame are received in one read operation.
2016-02-04 13:57:35 +01:00
houdejun214
a6fd8a96bf Set default CONTENT_TYPE when it is absent in multipart request body
Motivation:

I am use netty as a http server, it fail to decode some POST request when the request absent Content-Type in the multipart/form-data body.

Modifications:

Set content_type with default application/octet-stream to parse the uploaded file data when the Content-Type is absent in multipart request body

Result:

Can decode the http request as normal.
2016-01-26 10:47:11 +01:00
Fabian Lange
619d82b56f Removed unused imports
Motivation:

Warnings in IDE, unclean code, negligible performance impact.

Modification:

Deletion of unused imports

Result:

No more warnings in IDE, cleaner code, negligible performance improvement.
2016-01-04 14:32:29 +01:00
Norman Maurer
79bc90be32 Fix buffer leak introduced by 693633eeff
Motivation:

As we not used Unpooled anymore for allocate buffers in Base64.* methods we need to ensure we realease all the buffers.

Modifications:

Correctly release buffers

Result:

No more buffer leaks
2015-12-29 17:13:07 +01:00
Scott Mitchell
fd5316ed6f ChunkedInput.readChunk parameter of type ByteBufAllocator
Motivation:
ChunkedInput.readChunk currently takes a ChannelHandlerContext object as a parameters. All current implementations of this interface only use this object to get the ByteBufAllocator object. Thus taking a ChannelHandlerContext as a parameter is more restrictive for users of this API than necessary.

Modifications:
- Add a new method readChunk(ByteBufAllocator)
- Deprecate readChunk(ChannelHandlerContext) and updates all implementations to call readChunk(ByteBufAllocator)

Result:
API that only requires ByteBufAllocator to use ChunkedInput.
2015-12-24 12:46:40 -08:00
Norman Maurer
1a2162ec35 Fix broken tests introduced by dc615ecaaf 2015-12-18 10:16:07 +01:00
Norman Maurer
dc615ecaaf [#4212] Backport WebSocket Extension handlers for client and server.
Motivation:

We have websocket extension support (with compression) in old master. We should port this to 4.1

Modifications:

Backport relevant code.

Result:

websocket extension support (with compression) is now in 4.1.
2015-12-18 09:48:10 +01:00
Trustin Lee
412f719aa8 Extract the builder of CorsConfig to top level
Motivation:

Consistency in API design

Modifications:

- Deprecate CorsConfig.Builder and its factory methods
- Deprecate CorsConfig.DateValueGenerator
- Add CorsConfigBuilder and its factory methods
- Fix typo (curcuit -> circuit)

Result:

Consistency with other builder APIs such as SslContextBuilder and
Http2ConnectionHandlerBuilder
2015-12-18 12:38:44 +09:00
Norman Maurer
f31be51774 [#4505] Correctly handle whitespaces in websocket uri's.
Motivation:

If a uri contains whitespaces we need to ensure we correctly escape these when creating the request for the handshake.

Modifications:

- Correctly encode path for uri
- Add tests

Result:

Correctly handle whitespaces when doing websocket upgrade requests.
2015-12-10 13:52:42 +01:00
Luke Hutchison
4978266d52 Make cookie encoding conform better to RFC 6265 in STRICT mode.
Motivation:

- On the client, cookies should be sorted in decreasing order of path
  length. From RFC 6265:

      5.4.2. The user agent SHOULD sort the cookie-list in the following
      order:

        *  Cookies with longer paths are listed before cookies with
           shorter paths.

        *  Among cookies that have equal-length path fields, cookies with
           earlier creation-times are listed before cookies with later
           creation-times.

      NOTE: Not all user agents sort the cookie-list in this order, but
      this order reflects common practice when this document was
      written, and, historically, there have been servers that
      (erroneously) depended on this order.

  Note that the RFC does not define the path length of cookies without a
  path. We sort pathless cookies before cookies with the longest path,
  since pathless cookies inherit the request path (and setting a path
  that is longer than the request path is of limited use, since it cannot
  be read from the context in which it is written).

- On the server, if there are multiple cookies of the same name, only one
  of them should be encoded. RFC 6265 says:

      Servers SHOULD NOT include more than one Set-Cookie header field in
      the same response with the same cookie-name.

  Note that the RFC does not define which cookie should be set in the case
  of multiple cookies with the same name; we arbitrarily pick the last one.

Modifications:

- Changed the visibility of the 'strict' field to 'protected' in
  CookieEncoder.

- Modified ClientCookieEncoder to sort cookies in decreasing order of path
  length when in strict mode.

- Modified ServerCookieEncoder to return only the last cookie of a given
  name when in strict mode.

- Added a fast path for both strict mode in both client and server code
  for cases with only one cookie, in order avoid the overhead of sorting
  and memory allocation.

- Added unit tests for the new cases.

Result:

- Cookie generation on client and server is now more conformant to RFC 6265.
2015-11-26 21:41:58 +01:00
Dmitry Spikhalskiy
2a65ae256e [#4331] Helper methods to get charset from Content-Type header of HttpMessage
Motivation:

HttpHeaders already has specific methods for such popular and simple headers like "Host", but if I need to convert POST raw body to string I need to parse complex ContentType header in my code.

Modifications:

Add getCharset and getCharsetAsString methods to parse charset from Content-Length header.

Result:

Easy to use utility method.
2015-11-19 15:59:34 -08:00
Louis Ryan
6e108cb96a Improve the performance of copying header sets when hashing and name validation are equivalent.
Motivation:
Headers and groups of headers are frequently copied and the current mechanism is slower than it needs to be.

Modifications:
Skip name validation and hash computation when they are not necessary.
Fix emergent bug in CombinedHttpHeaders identified with better testing
Fix memory leak in DefaultHttp2Headers when clearing
Added benchmarks

Result:
Faster header copying and some collateral bug fixes
2015-11-07 08:53:10 -08:00
Louis Ryan
3eb65797ed Make headers.set(self) a no-op instead of throwing. Makes it consistent with setAll
Motivation:

Makes the API contract of headers more consistent and simpler.

Modifications:

If self is passed to set then simply return

Result:

set and setAll will be consistent
2015-11-06 07:00:54 -08:00
Sverker Abrahamsson
e121c68e0f Created RTSPEncoder and RTSPDecoder which are now common for both requests and responses to be able to handle both types of messages on the same channel.
Keep RTSPRequestEncoder, RTSPRequestDecoder, RTSPResponseEncoder and
RTSPResponseDecoder for backwards compatibility but they now just extends
the generic encoder/decoder and are markes as deprecated.

Renamed the decoder test, because the decoder is now generic. Added
testcase for when ANNOUNCE request is received from server.

Created testcases for encoder.

Mark abstract base classes RTSPObjectEncoder and RTSPObjectDecoder as
deprecated, that functionality is now in RTSPEncoder and RTSPDecoder.

Added annotation in RtspHeaders to suppress warnings about deprecation, no need when
whole class is deprecated.
2015-10-27 14:01:20 +01:00
Norman Maurer
ca44436ce6 [#4265] Not allow to add/set DefaultHttpHeaders to itself.
Motivation:

We should prevent to add/set DefaultHttpHeaders to itself to prevent unexpected side-effects.

Modifications:

Throw IllegalArgumentException if user tries to pass the same instance to set/add.

Result:

No surprising side-effects.
2015-09-30 08:57:53 +02:00
Norman Maurer
2dde3a386b [#3687] Correctly store WebSocketServerHandshaker in Channel attributes
Motivation:

As we stored the WebSocketServerHandshaker in the ChannelHandlerContext it was always null and so no close frame was send if WebSocketServerProtocolHandler was used.

Modifications:

Store WebSocketServerHAndshaker in the Channel attributes and so make it visibile between different handlers.

Result:

Correctly send close frame.
2015-09-15 09:48:04 +02:00
Scott Mitchell
f89dfb0bd5 Deprecation cleanup for HTTP headers
Motivaion:
The HttpHeaders and DefaultHttpHeaders have methods deprecated due to being removed in future releases, but no replacement method to use in the current release. The deprecation policy should not be so aggressive as to not provide any non-deprecated method to use.

Modifications:
- Remove deprecated annotations and javadocs from methods which are the best we can do in terms of matching the master's api for 4.1

Result:
There should be non-deprecated methods available for HttpHeaders in 4.1.
2015-09-09 14:30:21 -07:00
Sivasubramaniam S
5987cddf7c Fixed a typo [testEquansIgnoreCase() --> testEqualsIgnoreCase()] 2015-08-28 20:49:57 +09:00
Scott Mitchell
b6a4f5de9d Refactor of HttpUtil and HttpHeaderUtil
Motivation:
There currently exists http.HttpUtil, http2.HttpUtil, and http.HttpHeaderUtil. Having 2 HttpUtil methods can be confusing and the utilty methods in the http package could be consolidated.

Modifications:
- Rename http2.HttpUtil to http2.HttpConversionUtil
- Move http.HttpHeaderUtil methods into http.HttpUtil

Result:
Consolidated utilities whose names don't overlap.
Fixes https://github.com/netty/netty/issues/4120
2015-08-27 08:49:58 -07:00
Norman Maurer
e59ae12b42 [#4079] Fix IllegalStateException when HttpContentEncoder is used and 100 Continue response is used.
Motivation:

Whe a 100 Continue response was written an IllegalStateException was produced as soon as the user wrote the following response. This regression was introduced by 41b0080fcc.

Modifications:

- Special handle 100 Continue responses
- Added unit tests

Result:

Fixed regression.
2015-08-21 08:02:43 +02:00
Scott Mitchell
a7135e8677 HttpObjectAggregator doesn't check content-length header
Motivation:
The HttpObjectAggregator always responds with a 100-continue response. It should check the Content-Length header to see if the content length is OK, and if not responds with a 417.

Modifications:
- HttpObjectAggregator checks the Content-Length header in the case of a 100-continue.

Result:
HttpObjectAggregator responds with 417 if content is known to be too big.
2015-08-17 09:26:50 -07:00
Brendt Lucas
e796c99b23 Add unit tests for HTTP and SPDY headers
Motivation:

When attempting to retrieve a SPDY header using an AsciiString key, if the header was inserted using a String based key, the lookup would fail. Similarly, the lookup would fail if the header was inserted with an AsciiString key, and retrieved using a String key. This has been fixed with the header simplification commit (1a43923aa8).

Extra unit tests have been added to protect against this issue occurring in the future.  The tests check that a header added using String or AsciiString can be retrieved using AsciiString or String respectively.

Modifications:

Added more unit tests

Result:

Protect against issue #4053 happening again.
2015-08-17 08:51:14 -07:00
Scott Mitchell
ba6ce5449e Headers Performance Boost and Interface Simplification
Motivation:
A degradation in performance has been observed from the 4.0 branch as documented in https://github.com/netty/netty/issues/3962.

Modifications:
- Simplify Headers class hierarchy.
- Restore the DefaultHeaders to be based upon DefaultHttpHeaders from 4.0.
- Make various other modifications that are causing hot spots.

Result:
Performance is now on par with 4.0.
2015-08-17 08:50:11 -07:00
Norman Maurer
fd27c403d3 [#4010] Correctly handle whitespaces in HttpPostMultipartRequestDecoder
Motivation:

Due not using a cast we insert 32 and not a whitespace into the String.

Modifications:

Correclty cast to char.

Result:

Correct handling of whitespaces.
2015-08-14 21:16:42 +02:00
Scott Mitchell
b714297a44 HttpObjectDecoder half close behavior
Motivation:
In the event an HTTP message does not include either a content-length or a transfer-encoding header [RFC 7230](https://tools.ietf.org/html/rfc7230#section-3.3.3) states the behavior must be treated differently for requests and responses. If the channel is half closed then the HttpObjectDecoder is not invoking decodeLast and thus not checking if messages should be sent up the pipeline.

Modifications:
- Add comments to clarify regular decode default case.
- Handle the ChannelInputShutdownEvent in the HttpObjectDecoder and evaluate if messages need to be generated.

Result:
Messages are generated on half closed, and comments clarify existing logic.
2015-08-05 09:04:59 -07:00
Jakob Buchgraber
6fd0a0c55f Faster and more memory efficient headers for HTTP, HTTP/2, STOMP and SPYD. Fixes #3600
Motivation:

We noticed that the headers implementation in Netty for HTTP/2 uses quite a lot of memory
and that also at least the performance of randomly accessing a header is quite poor. The main
concern however was memory usage, as profiling has shown that a DefaultHttp2Headers
not only use a lot of memory it also wastes a lot due to the underlying hashmaps having
to be resized potentially several times as new headers are being inserted.

This is tracked as issue #3600.

Modifications:
We redesigned the DefaultHeaders to simply take a Map object in its constructor and
reimplemented the class using only the Map primitives. That way the implementation
is very concise and hopefully easy to understand and it allows each concrete headers
implementation to provide its own map or to even use a different headers implementation
for processing requests and writing responses i.e. incoming headers need to provide
fast random access while outgoing headers need fast insertion and fast iteration. The
new implementation can support this with hardly any code changes. It also comes
with the advantage that if the Netty project decides to add a third party collections library
as a dependency, one can simply plug in one of those very fast and memory efficient map
implementations and get faster and smaller headers for free.

For now, we are using the JDK's TreeMap for HTTP and HTTP/2 default headers.

Result:

- Significantly fewer lines of code in the implementation. While the total commit is still
  roughly 400 lines less, the actual implementation is a lot less. I just added some more
  tests and microbenchmarks.

- Overall performance is up. The current implementation should be significantly faster
  for insertion and retrieval. However, it is slower when it comes to iteration. There is simply
  no way a TreeMap can have the same iteration performance as a linked list (as used in the
  current headers implementation). That's totally fine though, because when looking at the
  benchmark results @ejona86 pointed out that the performance of the headers is completely
  dominated by insertion, that is insertion is so significantly faster in the new implementation
  that it does make up for several times the iteration speed. You can't iterate what you haven't
  inserted. I am demonstrating that in this spreadsheet [1]. (Actually, iteration performance is
  only down for HTTP, it's significantly improved for HTTP/2).

- Memory is down. The implementation with TreeMap uses on avg ~30% less memory. It also does not
  produce any garbage while being resized. In load tests for GRPC we have seen a memory reduction
  of up to 1.2KB per RPC. I summarized the memory improvements in this spreadsheet [1]. The data
  was generated by [2] using JOL.

- While it was my original intend to only improve the memory usage for HTTP/2, it should be similarly
  improved for HTTP, SPDY and STOMP as they all share a common implementation.

[1] https://docs.google.com/spreadsheets/d/1ck3RQklyzEcCLlyJoqDXPCWRGVUuS-ArZf0etSXLVDQ/edit#gid=0
[2] https://gist.github.com/buchgr/4458a8bdb51dd58c82b4
2015-08-04 17:12:24 -07:00
James Roper
b958263853 Send full response for unsupported websocket versions
Motivation:

WebSocketServerHandshakerFactory.sendUnsupportedVersionResponse does not
send a LastHttpContent, nor does it flush, and it doesn't send a content
length.

Modifications:

Changed sendUnsupportedVersionResponse to send FullHttpResponse, to
writeAndFlush, and to set a content length of 0. Also added a test for
this method.

Result:

Upstream handlers will be able to determine the end of the response, the
response will actually get written to the client, and the client will be
able to determine the end of the response.
2015-07-17 10:56:59 +02:00
Frederic Bregier
caa1505020 Get uploaded size while upload is in progress
Proposal to fix issue #3636

Motivations:
Currently, while adding the next buffers to the decoder
(`decoder.offer()`), there is no way to access to the current HTTP
object being decoded since it can only be available currently once fully
decoded by `decoder.hasNext()`.
Some could want to know the progression on the overall transfer but also
per HTTP object.
While overall progression could be done using (if available) the global
Content-Length of the request and taking into account each HttpContent
size, the per HttpData object progression is unknown.

Modifications:
1) For HTTP object, `AbstractHttpData` has 2 protected properties named
`definedSize` and `size`, respectively the supposely final size and the
current (decoded until now) size.
This provides a new method `definedSize()` to get the current value for
`definedSize`. The `size` attribute is reachable by the `length()`
method.

Note however there are 2 different ways that currently managed the
`definedSize`:
a) `Attribute`: it is reset each time the value is less than actual
(when a buffer is added, the value is increased) since the final length
is not known (no Content-Length)
b) `FileUpload`: it is set at startup from the lengh provided

So these differences could lead in wrong perception;
a) `Attribute`: definedSize = size always
b) `FileUpload`: definedSize >= size always

Therefore the comment tries to explain clearly the different behaviors.

2) In the InterfaceHttpPostRequestDecoder (and the derived classes), I
add a new method: `decoder.currentPartialHttpData()` which will return a
`InterfaceHttpData` (if any) as the current `Attribute` or `FileUpload`
(the 2 generic types), which will allow then the programmer to check
according to the real type (instance of) the 2 methods `definedSize()`
and `length()`.

This method check if currentFileUpload or currentAttribute are null and
returns the one (only one could be not null) that is not null.

Note that if this method returns null, it might mean 2 situations:
a) the last `HttpData` (whatever attribute or file upload) is already
finished and therefore accessible through `next()`
b) there is not yet any `HttpData` in decoding (body not yet parsed for
instance)

Result:
The developper has more access and therefore control on the current
upload.
The coding from developper side could looks like in the example in
HttpUloadServerHandler.
2015-06-12 14:16:07 +02:00
Trustin Lee
b169a76d46 Fix the failing HttpObjectAggregatorTest.testInvalidConstructorUsage()
Related: 950da2eae1
2015-06-10 12:20:50 +09:00
Norman Maurer
e9a2cac16d [#3869] Add unit test to ensure adding null header values is not allowed.
Motivation:

We need to ensure we never allow to have null values set on headers, otherwise we will see a NPE during encoding them.

Modifications:

Add unit test that shows we correctly handle null values.

Result:

Verify correct implementation.
2015-06-08 09:59:44 +02:00
Roelof Naude
757671b7cc Support empty http responses when using compression
Motivation:

Found a bug in that netty would generate a 20 byte body when returing a response
to an HTTP HEAD. the 20 bytes seems to be related to the compression footer.

RFC2616, section 9.4 states that responses to an HTTP HEAD MUST not return a message
body in the response.

Netty's own client implementation expected an empty response. The extra bytes lead to a
2nd response with an error decoder result:
java.lang.IllegalArgumentException: invalid version format: 14

Modifications:

Track the HTTP request method. When processing the response we determine if the response
is passthru unnchanged. This decision now takes into account the request method and passthru
responses related to HTTP HEAD requests.

Result:

Netty's http client works and better RFC conformance.
2015-05-26 09:55:34 +02:00
Stephane Landelle
f6299d942c Minor ClientCookieDecoder improvements
Motivation:

* Path attribute should be null, not empty String, if it's passed as "Path=".
* Only extract attribute value when the name is recognized.
* Only extract Expires attribute value String if MaxAge is undefined as it has precedence.

Modification:

Modify ClientCookieDecoder.
Add "testIgnoreEmptyPath" test in ClientCookieDecoderTest.

Result:

More idyomatic Path behavior (like Domain).
Minor performance improvement in some corner cases.
2015-05-12 11:25:28 +02:00
Stephane Landelle
97d871a755 Validate cookie name and value characters Motivation:
RFC6265 specifies which characters are allowed in a cookie name and value.

Netty is currently too lax, which can used for HttpOnly escaping.

Modification:

In ServerCookieDecoder: discard cookie key-value pairs that contain invalid characters.
In ClientCookieEncoder: throw an exception when trying to encode cookies with invalid characters.

Result:

The problem described in the motivation section is fixed.
2015-05-07 06:33:36 +02:00
Scott Mitchell
9a7a85dbe5 ByteString introduced as AsciiString super class
Motivation:
The usage and code within AsciiString has exceeded the original design scope for this class. Its usage as a binary string is confusing and on the verge of violating interface assumptions in some spots.

Modifications:
- ByteString will be created as a base class to AsciiString. All of the generic byte handling processing will live in ByteString and all the special character encoding will live in AsciiString.

Results:
The AsciiString interface will be clarified. Users of AsciiString can now be clear of the limitations the class imposes while users of the ByteString class don't have to live with those limitations.
2015-04-14 16:35:17 -07:00
Trustin Lee
6d5c38897e Fix header and initial line length counting
Related: #3445

Motivation:

HttpObjectDecoder.HeaderParser does not reset its counter (the size
field) when it failed to find the end of line.  If a header is split
into multiple fragments, the counter is increased as many times as the
number of fragments, resulting an unexpected TooLongFrameException.

Modifications:

- Add test cases that reproduces the problem
- Reset the HeaderParser.size field when no EOL is found.

Result:

One less bug
2015-03-04 17:24:22 +09:00
Daniel Bevenius
5b1b334f01 When null origin is supported then credentials header must not be set.
Motivation:
Currently CORS can be configured to support a 'null' origin, which can
be set by a browser if a resources is loaded from the local file system.
When this is done 'Access-Control-Allow-Origin' will be set to "*" (any
origin). There is also a configuration option to allow credentials being
sent from the client (cookies, basic HTTP Authentication, client side
SSL). This is indicated by the response header
'Access-Control-Allow-Credentials' being set to true. When this is set
to true, the "*" origin is not valid as the value of
'Access-Control-Allow-Origin' and a browser will reject the request:
http://www.w3.org/TR/cors/#resource-requests

Modifications:
Updated CorsHandler's setAllowCredentials to check the origin and if it
is "*" then it will not add the 'Access-Control-Allow-Credentials'
header.

Result:
Is is possible to have a client send a 'null' origin, and at the same
time have configured the CORS to support that and to allow credentials
in that combination.
2015-02-18 16:06:30 +01:00
Daniel Bevenius
c53b8d5a85 Suggestion for supporting single header fields.
Motivation:
At the moment if you want to return a HTTP header containing multiple
values you have to set/add that header once with the values wanted. If
you used set/add with an array/iterable multiple HTTP header fields will
be returned in the response.

Note, that this is indeed a suggestion and additional work and tests
should be added. This is mainly to bring up a discussion.

Modifications:
Added a flag to specify that when multiple values exist for a single
HTTP header then add them as a comma separated string.
In addition added a method to StringUtil to help escape comma separated
value charsequences.

Result:
Allows for responses to be smaller.
2015-02-18 10:54:15 +01:00
Trustin Lee
3c6cbd40e2 Fix an sporadic failure in ServerCookieEncoderTest
In testEncodingSingleCookieV0():

Let's assume we encoded a cookie with MaxAge=50 when currentTimeMillis
is 10999.

Because the encoder will not encode the millisecond part for Expires,
the timeMillis value of the encoded Expires field will be 60000. (If we
did not dropped the millisecond part, it would be 60999.)

Encoding a cookie will take some time, so currentTimeMillis will
increase slightly, such as to 11001.

  diff = (60000 - 11001) / 1000 = 48999 / 1000 = 48
  maxAge - diff = 50 - 48 = 2

Due to losing millisecond part twice, we end up with the precision
problem illustrated above, and thus we should increase the tolerance
from 1 second to 2 seconds.

/cc @slandelle
2015-02-02 16:19:27 +09:00
Stephane Landelle
8614b88c18 Generate Expires attribute along MaxAge one so IE can honor it, close #1466
Motivation:

Internet Explorer doesn't honor Set-Cookie header Max-Age attribute. It only honors the Expires one.

Modification:

Always generate an Expires attribute along the Max-Age one.

Result:

Internet Explorer compatible expiring cookies. Close #1466.
2015-01-25 16:55:56 +01:00
igariev
ed10513238 Fixed several issues with HttpContentDecoder
Motivation:

HttpContentDecoder had the following issues:
- For chunked content, the decoder set invalid "Content-Length" header
	with length of the first decoded chunk.
- Decoding of FullHttpRequests put both the original conent and decoded
	content into output. As result, using HttpObjectAggregator before the
	decoder lead to errors.
- Requests with "Expect: 100-continue" header were not acknowleged:
	the decoder didn't pass the header message down the handler's chain
	until content is received. If client expected "100 Continue" response,
	deadlock happened.

Modification:

- Invalid "Content-Length" header is removed; handlers down the chain can either
	rely on LastHttpContent message or ask HttpObjectAggregator to add the header.
- FullHttpRequest is split into HttpRequest and HttpContent (decoded) parts.
- Header (HttpRequest) part of request is sent down the chain as soon as it's received.

Result:

The issues are fixed, unittest is added.
2015-01-23 11:14:20 +01:00
Trustin Lee
7d102084c1 Remove Rfc6265 prefix from cookie encoders and decoders
Motivation:

Rfc6265Client/ServerCookieEncoder is a better replacement of the old
Client/ServerCookieEncoder, and thus there's no point of keeping both.

Modifications:

- Remove the old Client/ServerCookieEncoder
- Remove the 'Rfc6265' prefix from the new cookie encoder/decoder
  classes
- Deprecate CookieDecoder

Result:

We have much better cookie encoder/decoder implementation now.
2015-01-21 22:27:50 +09:00
Stephane Landelle
c298230128 RFC6265 cookies support
Motivation:

Currently Netty supports a weird implementation of RFC 2965.
First, this RFC has been deprecated by RFC 6265 and nobody on the
internet use this format.

Then, there's a confusion between client side and server side encoding
and decoding.

Typically, clients should only send name=value pairs.

This PR introduces RFC 6265 support, but keeps on supporting RFC 2965 in
the sense that old unused fields are simply ignored, and Cookie fields
won't be populated. Deprecated fields are comment, commentUrl, version,
discard and ports.

It also provides a mechanism for safe server-client-server roundtrip, as
User-Agents are not supposed to interpret cookie values but return them
as-is (e.g. if Set-Cookie contained a quoted value, it should be sent
back in the Cookie header in quoted form too).

Also, there are performance gains to be obtained by not allocating the
attribute name Strings, as we only want to match them to find which POJO
field to populate.

Modifications:

- New RFC6265ClientCookieEncoder/Decoder and
  RFC6265ServerCookieEncoder/Decoder pairs that live alongside old
  CookieEncoder/Decoder pair to not break backward compatibility.
- New Cookie.rawValue field, used for lossless server-client-server
  roundtrip.

Result:

RFC 6265 support.
Clean separation of client and server side.

Decoder performance gain:

Benchmark                     Mode  Samples        Score        Error
Units
parseOldClientDecoder        thrpt       20  2070169,228 ± 105044,970
ops/s
parseRFC6265ClientDecoder    thrpt       20  2954015,476 ± 126670,633
ops/s

This commit closes #3221 and #1406.
2015-01-21 19:07:13 +09:00
Norman Maurer
e1a53e61d0 Fix compilation error introduced by 7f907e8c2a 2015-01-16 16:47:51 +01:00
Frederic Bregier
7f907e8c2a Accept ';' '\\"' in the filename of HTTP Content-Disposition header
Motivation:
HttpPostMultipartRequestDecoder threw an ArrayIndexOutOfBoundsException
when trying to decode Content-Disposition header with filename
containing ';' or protected \\".
See issue #3326 and #3327.

Modifications:
Added splitMultipartHeaderValues method which cares about quotes, and
use it in splitMultipartHeader method, instead of StringUtils.split.

Result:
Filenames can contain semicolons and protected \\".
2015-01-16 13:54:32 +01:00
Frederic Bregier
85fecba770 Fix for Issue #3308 related to slice missing retain
Motivations:
It seems that slicing a buffer and using this slice to write to CTX will
decrease the initial refCnt to 0, while the original buffer is not yet
fully used (not empty).

Modifications:
As suggested in the ticket and tested, when the currentBuffer is sliced
since it will still be used later on, the currentBuffer is retained.

Add a test case for this issue.

Result:
The currentBuffer still has its correct refCnt when reaching the last
write (not sliced) of 1 and therefore will be released correctly.
The exception does no more occur.

This fix should be applied to all branches >= 4.0.
2015-01-06 21:05:33 +01:00
zcourts
bd6d0f3fd5 ensure getRawQuery is not null before appending
Motivation:

without this check then given a URI with path /path the resulting URL will be /path?null=

Modifications:

check that getRawQuery doesn't return null and only append if not

Result:

urls of the form /path will not have a null?= appended
2014-12-16 06:53:29 +01:00
Scott Mitchell
72a611a28f HTTP Content Encoder allow EmptyLastHttpContent
Motiviation:
The HttpContentEncoder does not account for a EmptyLastHttpContent being provided as input.  This is useful in situations where the client is unable to determine if the current content chunk is the last content chunk (i.e. a proxy forwarding content when transfer encoding is chunked).

Modifications:
- HttpContentEncoder should not attempt to compress empty HttpContent objects

Result:
HttpContentEncoder supports a EmptyLastHttpContent to terminate the response.
2014-11-05 23:31:27 -05:00
Trustin Lee
4ce994dd4f Fix backward compatibility from the previous backport
Motivation:

The commit 50e06442c3 changed the type of
the constants in HttpHeaders.Names and HttpHeaders.Values, making 4.1
backward-incompatible with 4.0.

It also introduces newer utility classes such as HttpHeaderUtil, which
deprecates most static methods in HttpHeaders.  To ease the migration
between 4.1 and 5.0, we should deprecate all static methods that are
non-existent in 5.0, and provide proper counterpart.

Modification:

- Revert the changes in HttpHeaders.Names and Values
- Deprecate all static methods in HttpHeaders in favor of:
  - HttpHeaderUtil
  - the member methods of HttpHeaders
  - AsciiString
- Add integer and date access methods to HttpHeaders for easier future
  migration to 5.0
- Add HttpHeaderNames and HttpHeaderValues which provide standard HTTP
  constants in AsciiString
  - Deprecate HttpHeaders.Names and Values
  - Make HttpHeaderValues.WEBSOCKET lowercased because it's actually
    lowercased in all WebSocket versions but the oldest one
- Add RtspHeaderNames and RtspHeaderValues which provide standard RTSP
  constants in AsciiString
  - Deprecate RtspHeaders.*
- Do not use AsciiString.equalsIgnoreCase(CharSeq, CharSeq) if one of
  the parameters are AsciiString
- Avoid using AsciiString.toString() repetitively
  - Change the parameter type of some methods from String to
    CharSequence

Result:

Backward compatibility is recovered.  New classes and methods will make
the migration to 5.0 easier, once (Http|Rtsp)Header(Names|Values) are
ported to master.
2014-11-01 01:00:25 +09:00
Scott Mitchell
50e06442c3 Backport header improvements from 5.0
Motivation:
The header class hierarchy and algorithm was improved on the master branch for versions 5.x. These improvments should be backported to the 4.1 baseline.

Modifications:
- cherry-pick the following commits from the master branch: 2374e17, 36b4157, 222d258

Result:
Header improvements in master branch are available in 4.1 branch.
2014-11-01 00:59:57 +09:00
Matthias Einwag
7fbd66f814 Added an option to use websockets without masking
Motivation:

The requirement for the masking of frames and for checks of correct
masking in the websocket specifiation have a large impact on performance.
While it is mandatory for browsers to use masking there are other
applications (like IPC protocols) that want to user websocket framing and proxy-traversing
characteristics without the overhead of masking. The websocket standard
also mentions that the requirement for mask verification on server side
might be dropped in future.

Modifications:

Added an optional parameter allowMaskMismatch for the websocket decoder
that allows a server to also accept unmasked frames (and clients to accept
masked frames).
Allowed to set this option through the websocket handshaker
constructors as well as the websocket client and server handlers.
The public API for existing components doesn't change, it will be
forwarded to functions which implicetly set masking as required in the
specification.
For websocket clients an additional parameter is added that allows to
disable the masking of frames that are sent by the client.

Result:

This update gives netty users the ability to create and use completely
unmasked websocket connections in addition to the normal masked channels
that the standard describes.
2014-10-25 22:18:43 +09:00
George Agnelli
0666924e8c Don't close the connection whenever Expect: 100-continue is missing.
Motivation:

The 4.1.0-Beta3 implementation of HttpObjectAggregator.handleOversizedMessage closes the
connection if the client sent oversized chunked data with no Expect:
100-continue header. This causes a broken pipe or "connection reset by
peer" error in some clients (tested on Firefox 31 OS X 10.9.5,
async-http-client 1.8.14).

This part of the HTTP 1.1 spec (below) seems to say that in this scenario the connection
should not be closed (unless the intention is to be very strict about
how data should be sent).

http://www.w3.org/Protocols/rfc2616/rfc2616-sec8.html

"If an origin server receives a request that does not include an
Expect request-header field with the "100-continue" expectation,
the request includes a request body, and the server responds
with a final status code before reading the entire request body
from the transport connection, then the server SHOULD NOT close
the transport connection until it has read the entire request,
or until the client closes the connection. Otherwise, the client
might not reliably receive the response message. However, this
requirement is not be construed as preventing a server from
defending itself against denial-of-service attacks, or from
badly broken client implementations."

Modifications:

Change HttpObjectAggregator.handleOversizedMessage to close the
connection only if keep-alive is off and Expect: 100-continue is
missing. Update test to reflect the change.

Result:

Broken pipe and connection reset errors on the client are avoided when
oversized data is sent.
2014-10-24 21:35:17 +02:00
Trustin Lee
789e323b79 Handle an empty ByteBuf specially in HttpObjectEncoder
Related: #2983

Motivation:

It is a well known idiom to write an empty buffer and add a listener to
its future to close a channel when the last byte has been written out:

  ChannelFuture f = channel.writeAndFlush(Unpooled.EMPTY_BUFFER);
  f.addListener(ChannelFutureListener.CLOSE);

When HttpObjectEncoder is in the pipeline, this still works, but it
silently raises an IllegalStateException, because HttpObjectEncoder does
not allow writing a ByteBuf when it is expecting an HttpMessage.

Modifications:

- Handle an empty ByteBuf specially in HttpObjectEncoder, so that
  writing an empty buffer does not fail even if the pipeline contains an
  HttpObjectEncoder
- Add a test

Result:

An exception is not triggered anymore by HttpObjectEncoder, when a user
attempts to write an empty buffer.
2014-10-22 14:46:22 +09:00
Daniel Bevenius
67c68ef8ba CorsHandler should release HttpRequest after processing preflight/error.
Motivation:
Currently, when the CorsHandler processes a preflight request, or
respondes with an 403 Forbidden using the short-curcuit option, the
HttpRequest is not released which leads to a buffer leak.

Modifications:
Releasing the HttpRequest when done processing a preflight request or
responding with an 403.

Result:
Using the CorsHandler will not cause buffer leaks.
2014-10-22 06:37:34 +02:00
Matthias Einwag
80ae2c9180 Add a test for handover from HTTP to Websocket
Motivation:
I was not fully reassured that whether everything works correctly when a websocket client receives the websocket handshake HTTP response and a websocket frame in a single ByteBuf (which can happen when the server sends a response directly or shortly after the connect). In this case some parts of the ByteBuf must be processed by HTTP decoder and the remaining by the websocket decoder.

Modification:
Adding a test that verifies that in this scenaria the handshake and the message are correctly interpreted and delivered by Netty.

Result:
One more test for Netty.
The test succeeds - No problems
2014-10-13 07:23:10 +02:00
Matthias Einwag
4eb1529d2c Improve WebSocket performance
Motivation:

Websocket performance is to a large account determined through the masking
and unmasking of frames. The current behavior of this in Netty can be
improved.

Modifications:

Perform the XOR operation not bytewise but in int blocks as long as
possible. This reduces the number of necessary operations by 4. Also don't
read the writerIndex in each iteration.
Added a unit test for websocket decoding and encoding for verifiation.

Result:

A large performance gain (up to 50%) in websocket throughput.
2014-10-12 19:48:49 +02:00
Norman Maurer
65686b6c83 Reduce memory copies in spdy compression implementation.
Motivation:

Currently we do more memory copies then needed.

Modification:

- Directly use heap buffers to reduce memory copy
- Correctly release buffers to fix buffer leak

Result:

Less memory copies and no leaks
2014-08-21 11:27:32 +02:00
Norman Maurer
d0b5fb9548 [#2768] Correctly duplicate buffer for CloseWebSocketFrames
Motivation:

The _0XFF_0X00 buffer is not duplicated and empty after the first usage preventing the connection close to happen on subsequent close frames.

Modifications:

Correctly duplicate the buffer.

Result:

Multiple CloseWebSocketFrames are handled correctly.
2014-08-14 09:54:35 +02:00
Trustin Lee
fd813f76fa Add test cases for HttpContentCompressor
- Ported from 386a06dbfa
2014-08-05 15:48:43 -07:00
Norman Maurer
e33f12f5b8 [#2732] HttpRequestEncoder may produce invalid uri if uri parameters are included.
Motivation:

If the requests contains uri parameters but not path the HttpRequestEncoder does produce an invalid uri while try to add the missing path.

Modifications:

Correctly handle the case of uri with paramaters but no path.

Result:

HttpRequestEncoder produce correct uri in all cases.
2014-08-05 10:13:53 +02:00
Norman Maurer
909d1f2c0f Reuse previous created HttpHeaders by HttpObjectAggregator
Motivation:

HttpObjectAggregator currently creates a new FullHttpResponse / FullHttpRequest for each message it needs to aggregate. While doing so it also creates 2 DefaultHttpHeader instances (one for the headers and one for the trailing headers). This is bad for two reasons:
  - More objects are created then needed and also populate the headers is not for free
  - Headers may get validated even if the validation was disabled in the decoder

Modification:

- Wrap the previous created HttpResponse / HttpRequest and so reuse the original HttpHeaders
- Reuse the previous created trailing HttpHeader.
- Fix a bug where the trailing HttpHeader was incorrectly mixed in the headers.

Result:

- Less GC
- Faster HttpObjectAggregator implementation
2014-07-11 07:08:12 +02:00
Trustin Lee
d0912f2709 Fix most inspector warnings
Motivation:

It's good to minimize potentially broken windows.

Modifications:

Fix most inspector warnings from our profile
Update IntObjectHashMap

Result:

Cleaner code
2014-07-02 19:55:07 +09:00
Jay
461d3c876f Maintain decoder result in HttpObjectAggregator
Motivation:
DecodeResult is dropped when aggregate HTTP messages.

Modification:

Make sure we not drop the DecodeResult while aggregate HTTP messages.

Result:

Correctly include the DecodeResult for later processing.
2014-06-28 22:08:53 +02:00
Alexey Parfenov
4b4f4e619e Fix integer overflow in HttpObjectEncoder when handling chunked encoding and FileRegion > Integer.MAX_VALUE
Motivation:

Due to integer overflow bug, writes of FileRegions to http server pipeline (eg like one from HttpStaticFileServer example) with length greater than Integer.MAX_VALUE are ignored in 1/2 of cases (ie no data gets sent to client)

Modification:

Correctly handle chunk sized > Integer.MAX_VALUE

Result:

Be able to use FileRegion > Integer.MAX_VALUE when using chunked encoding.
2014-06-24 12:03:48 +02:00
Trustin Lee
41d44a8161 Remove 'get' prefix from all HTTP/SPDY messages
Motivation:

Persuit for the consistency in method naming

Modifications:

- Remove the 'get' prefix from all HTTP/SPDY message classes
- Fix some inspector warnings

Result:

Consistency
2014-06-24 18:03:33 +09:00
Trustin Lee
c076c33901 Backport the additional AsciiString/TextHeader changes from master
- Add useful static methods to AsciiString
- Add more getters in TextHeaders
- Remove unnecessary utility methods in SpdyHttpHeaders
2014-06-14 17:33:34 +09:00
Trustin Lee
681d460938 Introduce TextHeaders and AsciiString
Motivation:

We have quite a bit of code duplication between HTTP/1, HTTP/2, SPDY,
and STOMP codec, because they all have a notion of 'headers', which is a
multimap of string names and values.

Modifications:

- Add TextHeaders and its default implementation
- Add AsciiString to replace HttpHeaderEntity
  - Borrowed some portion from Apache Harmony's java.lang.String.
- Reimplement HttpHeaders, SpdyHeaders, and StompHeaders using
  TextHeaders
- Add AsciiHeadersEncoder to reuse the encoding a TextHeaders
  - Used a dedicated encoder for HTTP headers for better performance
    though
- Remove shortcut methods in SpdyHeaders
- Replace SpdyHeaders.getStatus() with HttpResponseStatus.parseLine()

Result:

- Removed quite a bit of code duplication in the header implementations.
- Slightly better performance thanks to improved header validation and
  hash code calculation
2014-06-14 15:36:19 +09:00
Frederic Bregier
8074b5c6ee [#2542] HTTP post request decoder does not support quoted boundaries
Motivation:
According to RFC2616 section 19, boundary string could be quoted, but
currently the PostRequestDecoder does not support it while it should.

Modifications:
Once the boundary is found, one check is made to verify if the boundary
is "quoted", and if so, it is "unqoted".

Note: in following usage of this boundary (as delimiter), quote seems no
more allowed according to the same RFC, so the reason that only the
boundary definition is corrected.

Result:
Now the boundary could be whatever quoted or not. A Junit test case
checks it.
2014-06-08 12:08:03 +02:00
Frederic Bregier
f402350d76 [#2544] Correctly parse Multipart-mixed POST HTTP request in case of entity ends with odd number of 0x0D
Motivation:
When an attribute is ending with an odd number of CR (0x0D), the decoder
add an extra CR in the decoded attribute and should not.

Modifications:
Each time a CR is detected, the next byte was tested to be LF or not. If
not, in a number of places, the CR byte was lost while it should not be.
When a CR is detected, if the next byte is not LF, the CR byte should be
saved as the position point to the next byte (not LF). When a CR is
detected, if there is not yet other available bytes, the position is
reset to the position of CR (since a LF could follow).

A new Junit test case is added, using DECODER and variable number of CR
in the final attribute (testMultipartCodecWithCRasEndOfAttribute).

Result:
The attribute is now correctly decoded with the right number of CR
ending bytes.
2014-06-08 11:24:18 +02:00
Trustin Lee
8b0a0f9a8f Introduce MessageAggregator and DecoderResultProvider
Motivation:

We have different message aggregator implementations for different
protocols, but they are very similar with each other.  They all stems
from HttpObjectAggregator.  If we provide an abstract class that provide
generic message aggregation functionality, we will remove their code
duplication.

Modifications:

- Add MessageAggregator which provides generic message aggregation
- Reimplement all existing aggregators using MessageAggregator
- Add DecoderResultProvider interface and extend it wherever possible so
  that MessageAggregator respects the state of the decoded message

Result:

Less code duplication
2014-06-05 16:51:14 +09:00
Daniel Bevenius
1a22186645 Adding short-curcuit option for CORS
Motivation:
CORS request are currently processed, and potentially failed, after the
target ChannelHandler(s) have been invoked. This might not be desired, for
example a HTTP PUT or POST might have been performed.

Modifications:
Added a shortCurcuit option to CorsConfig which when set will
cause a validation of the HTTP request's 'Origin' header and verify that
it is valid according to the configuration. If found invalid an 403
"Forbidden" response will be returned and not further processing will
take place.

This is indeed no help for non browser request, like using curl, which
can set the 'Origin' header.

Result:
Users can now configure if the 'Origin' header should be validated
upfront and have the request rejected before any further processing
takes place.
2014-05-06 12:16:57 +02:00
Norman Maurer
8a39b9fedc Fix buffer leak introduced by #2462
Motivation:
Because of not correctly release a buffer before null out the reference a memory leak shows up.

Modifications:
Correct call buffer.release() before null out reference.

Result:
No more leak
2014-05-06 10:03:24 +02:00
Jeff Pinner
1351349243 SPDY: ensure SpdyHeaderBlockRawDecoder always reads entire input 2014-05-05 07:33:05 +02:00
Daniel Bevenius
2f082cf9a2 Adding support for echoing the request origin for CORS.
Motivation:
When CORS has been configured to allow "*" origin, and at the same time
is allowing credentials/cookies, this causes an error from the browser
because when the response 'Access-Control-Allow-Credentials' header
is true, the 'Access-Control-Allow-Origin' must be an actual origin.

Modifications:
Changed CorsHandler setOrigin method to check for the combination of "*"
origin and allowCredentials, and if the check matches echo the CORS
request's 'Origin' value.

Result:
This addition enables the echoing of the request 'Origin' value as the
'Access-Control-Allow-Origin' value when the server has been configured
to allow any origin in combination with allowCredentials.

This allows client requests to succeed when expecting the server to
be able to handle "*" origin and at the same time be able to send cookies
by setting 'xhr.withCredentials=true'. A concrete example of this is
the SockJS protocol which expects behaviour.
2014-04-25 18:20:02 +02:00
Trustin Lee
db3709e652 Synchronized between 4.1 and master
Motivation:

4 and 5 were diverged long time ago and we recently reverted some of the
early commits in master.  We must make sure 4.1 and master are not very
different now.

Modification:

Fix found differences

Result:

4.1 and master got closer.
2014-04-25 00:38:02 +09:00
Norman Maurer
40bcb17bf9 Fix missed buffer leaks in SpdyFrameDecoderTest
Motivation:
 Fix leaks reported during running SpdyFrameDecoderTest

Modifications:
Make sure the produced buffers of SpdyFrameDecoder and SpdyFrameDecoderTest are released

Result:

No more leak reports during run the tests.
2014-04-16 14:01:50 +02:00
Norman Maurer
2e8e7e486a Fix buffer leaks in SpdyFrameDecoderTest
Motivation:

Fix leaks reported during running SpdyFrameDecoderTest

Modifications:
Make sure the produced buffer of SpdyFrameDecoder is released

Result:

No more leak reports during run the tests.
2014-04-16 10:48:23 +02:00
Norman Maurer
71f5bb84a6 Fix buffer leaks in SPDY test
Motivation:

Fix leaks reported during SPDY test.

Modifications:

Use ReferenceCountUtil.releaseLater(...) to make sure everything is released once the tests are done.

Result:

No more leak reports during run the tests.
2014-04-16 06:51:26 +02:00
Jeff Pinner
7808b9926d SPDY: refactor frame codec implementation
Motivation:

Currently, the SPDY frame encoding and decoding code is based upon
the ChannelHandler abstraction. This requires maintaining multiple
versions for 3.x and 4.x (and possibly 5.x moving forward).

Modifications:

The SPDY frame encoding and decoding code is separated from the
ChannelHandler and SpdyFrame abstractions. Also test coverage is
improved.

Result:

SpdyFrameCodec now implements the ChannelHandler abstraction and is
responsible for creating and handling SpdyFrame objects.
2014-04-15 19:57:38 +02:00
Daniel Bevenius
4fc9afa102 Adding origins whitelist support for CORS
Motivation:
Currently the CORS support only handles a single origin, or a wildcard
origin. This task should enhance Netty's CORS support to allow multiple
origins to be specified. Just being allowed to specify one origin is
particulary limiting when a site support both http and https for
example.

Modifications:
- Updated CorsConfig and its Builder to accept multiple origins.

Result:
Users are now able to configure multiple origins for CORS.

[https://github.com/netty/netty/issues/2346]
2014-03-30 19:40:48 +02:00
Daniel Bevenius
110878fe2c Fixing CorsConfigTest failure under Java 8.
Motivation:
When running the build with Java 8 the following error occurred:

java: reference to preflightResponseHeader is ambiguous
  both method
  <T>preflightResponseHeader(java.lang.CharSequence,java.lang.Iterable<T>)
  in io.netty.handler.codec.http.cors.CorsConfig.Builder and method
  <T>preflightResponseHeader(java.lang.String,java.util.concurrent.Callable<T>)
  in io.netty.handler.codec.http.cors.CorsConfig.Builder match

The offending class was CorsConfigTest and its shouldThrowIfValueIsNull
which contained the following line:
withOrigin("*").preflightResponseHeader("HeaderName", null).build();

Modifications:
Updated the offending method with to supply a type, and object array, to
avoid the error.

Result:
After this I was able to build with Java 7 and Java 8
2014-03-22 07:44:54 +01:00
Daniel Bevenius
74f418bace Adding support for specifying preflight response headers.
Motivation:

An intermediary like a load balancer might require that a Cross Origin
Resource Sharing (CORS) preflight request have certain headers set.
As a concrete example the Elastic Load Balancer (ELB) requires the
'Date' and 'Content-Length' header to be set or it will fail with a 502
error code.

This works is an enhancement of https://github.com/netty/netty/pull/2290

Modifications:

CorsConfig has been extended to make additional HTTP response headers
configurable for preflight responses. Since some headers, like the
'Date' header need to be generated each time, m0wfo suggested using a
Callable.

Result:

By default, the 'Date' and 'Content-Lenght' headers will be sent in a
preflight response. This can be overriden and users can specify
any headers that might be required by different intermediaries.
2014-03-21 15:40:59 +01:00
Frederic Bregier
397f81f253 [#2305] Fix issue related to decoding post request raized an exception due to a split of information by chunk not correctly taken into account by the decoder
Motivation:

If the last item analyzed in a previous received HttpChunk/HttpContent was a part of an attribute's name, the read index was not set to the new right place and therefore raizing an exception in some case (since the "new" name analyzed is empty, which is not allowed so the exception).

What appears there is that the read index should be reset to the last valid position encountered whatever the case. Currently it was set when only when there is an attribute not already finished (name is ok, but content is possibly not).

Therefore the issue is that elements could be rescanned multiple times (including completed elements) and moreover some bad decoding can occur such as when in a middle of an attribute's name.

Modifications:

To fix this issue, since "firstpos" contains the last "valid" read index of the decoding (when finding a '&', '=', 'CR/LF'), we should add the setting of the read index for the following cases:

'lastchunk' encountered, therefore finishing the current buffer
any other cases than current attribute is not finished (name not found yet in particular)
So adding for this 2 cases:

undecodedChunk.readerIndex(firstpos);

Result:

Now the decoding is done once, content is added from chunk/content to chunk/content, name is decoded correctly even if in the middle of 2 chunks/contents.
A Junit test code was added: testChunkCorrect that should not raized any exception.
2014-03-14 09:45:40 +01:00
David Dossot
4f069e5c36 added support for empty query parameters 2014-03-10 06:46:08 +01:00
Trustin Lee
ac5592fc05 Fix checkstyle 2014-02-26 16:18:19 -08:00
Trustin Lee
24fc1b9b37 Fix a bug where HttpObjectDecoder generates two LastHttpContent consecutively
Motivation:
When an HttpResponseDecoder decodes an invalid chunk, a LastHttpContent instance is produced and the decoder enters the 'BAD_MESSAGE' state, which is not supposed to produce a message any further.  However, because HttpObjectDecoder.invalidChunk() did not clear this.message out to null, decodeLast() will produce another LastHttpContent message on a certain situation.

Modification:
Do not forget to null out HttpObjectDecoder.message in invalidChunk(), and add a test case for it.

Result:
No more consecutive LastHttpContent messages produced by HttpObjectDecoder.
2014-02-26 15:53:53 -08:00
Jeff Pinner
b02531f0aa SPDY: remove SPDY/3 support 2014-02-20 15:00:32 -08:00
Trustin Lee
738d3f77b9 Fix another leak in HttpObjectAggregatorTest by removing redundant code 2014-02-20 14:30:25 -08:00
Trustin Lee
874a66c8fb Fix a memory leak in HttpObjectAggregatorTest 2014-02-20 13:49:35 -08:00
Trustin Lee
2c4aff13c7 Add an operation that resets the state of HttpObjectDecoder
Motivation:

Currently, it is impossible to give a user the full control over what to do in response to the request with 'Expect: 100-continue' header.  Currently, a user have to do one of the following:

- Accept the request and respond with 100 Continue, or
- Send the reject response and close the connection.

.. which means it is impossible to send the reject response and keep the connection alive so that the client sends additional requests.

Modification:

Added a public method called 'reset()' to HttpObjectDecoder so that a user can reset the state of the decoder easily.  Once called, the decoder will assume the next input will be the beginning of a new request.

HttpObjectAggregator now calls `reset()`right after calling 'handleOversizedMessage()' so that the decoder can continue to decode the subsequent request even after the request with 'Expect: 100-continue' header is rejected.

Added relevant unit tests / Minor clean-up

Result:

This commit completes the fix of #2211
2014-02-20 13:41:54 -08:00
Trustin Lee
e278b57489 Added a test for oversized HTTP responses 2014-02-20 11:46:41 -08:00
Trustin Lee
fcc41a62bd Overall clean-up of HttpObjectAggregator / Handle oversized response differently
- Related: #2211
2014-02-20 11:36:56 -08:00
Chris Mowforth
91376263d7 Expose a callback in HttpObjectAggregator to handle oversized messages
- Related: #2211
2014-02-20 11:36:56 -08:00
Trustin Lee
ddb6e3672c Fix a bug where HttpObjectAggregator doesn't always produce FullHttpMessage
- Fixes #2182
- Always convert an unfull invalid message to a full message
2014-02-19 15:31:53 -08:00
Trustin Lee
396a81c811 Ensure that the last bad chunk is LastHttpChunk 2014-02-19 15:19:35 -08:00
Trustin Lee
45e70d9935 Add ReferenceCounted.touch() / Add missing retain() overrides
- Fixes #2163
- Inspector warnings
2014-02-13 18:10:11 -08:00
Trustin Lee
77d1f97b57 Add an HTML5 encoder mode for HttpPostRequestEncoder
- backported from 709be30442 by @igstan
2014-02-13 17:55:07 -08:00
Trustin Lee
df346a023b Change the return type of EmbeddedChannel.read*() from Object to an ad-hoc type parameter
.. so that there's no need to explicitly down-cast.

Fixes #2067
2014-02-13 17:19:26 -08:00
Daniel Bevenius
fa33529ca5 Adding check for zero readable bytes in WebSocket08FrameDecoder decode. 2014-02-13 15:49:44 -08:00
Norman Maurer
dddfb149a5 [#2173] Fix regression that let HttpRequestDecoder fail if the websocket response and a websocketframe are send in one go 2014-02-06 10:34:05 +01:00
Norman Maurer
407e12d99b Not throw an exception if subprotocol is not supported but just drop the header as stated in the RFC's 2014-01-26 11:01:00 +01:00
Trustin Lee
149c7e82a2 Add HttpPostrequestEncoderTest by @igstan 2014-01-19 22:22:08 +09:00
Trustin Lee
95bcbe5e08 Backport the HttpPostRequestEncoder fixes by @igstan 2014-01-19 22:21:24 +09:00
Norman Maurer
faf8becf2e Make use of ByteBufProcessor for extract initial line and headers
This gives some nice performance boost as readByte() is quite expensive because of the index / replay checks.
2014-01-16 20:14:08 +01:00
Daniel Bevenius
75b0360867 Adding a Cross Origin Resource Sharing (CORS) handler. 2014-01-16 08:18:16 +01:00
Veebs
2df690b0e4 Add HttpChunkedInput for easier streaming of chunked content 2014-01-15 17:18:19 +01:00
Trustin Lee
cead74f84d Fix checkstyle 2014-01-13 22:57:34 +09:00
Trustin Lee
06cbac922a Fix a bug where SpdyHeaderBlockZlibDecoder fails to decompress
- Forward-port 4c35b593c1, originally written by @wgallagher
- Fixes #2077
2014-01-13 22:42:24 +09:00
Trustin Lee
8d431679d7 Fix a compilation error 2014-01-10 16:36:48 +09:00
Trustin Lee
b24494abeb Fix a bug where HttpObjectDecoder produces LastHttpContent after entering BAD_MESSAGE state
- Fixes #2103
- Added LastHttpContent.EMPTY_LAST_CONTENT.toString() for athestic reasons.
2014-01-10 16:17:08 +09:00
Trustin Lee
47d0a85720 Fix a leak in HttpResponseDecoderTest 2013-12-19 01:13:30 +09:00
Trustin Lee
0f76b3c357 Remove unnecessary code from HttpObjectDecoder and re-enable all HTTP tests
- Since Netty 4, HTTP decoder does not generate a full message at all.  Therefore, there's no need to keep separate states for the content smaller than maxChunkSize.
- maxChunkSize must be greater than 0. Setting it to 0 should not disable chunked encoding. We have a dedicated flag for that.
- Uncommented the tests that were commented out for an unknown reason, with some fixes.
- Added more tests for HTTP decoder.
- Removed the Ignore annotation on some tests.
2013-12-18 15:12:55 +01:00
Trustin Lee
2b09d92c37 Improve the unit test for #1742 2013-12-16 21:55:59 +09:00
Trustin Lee
3444c06654 Fix a bug where HttpObjectAggregator fails to send a '100 Continue' response
- Fixes #1742
2013-12-16 21:44:44 +09:00
Jeff Pinner
db59b38db3 SPDY: remove SPDY/2 support 2013-12-16 14:18:01 +09:00
Jeff Pinner
39ae2dd3f1 SPDY: add SPDY/3.1 support
- with Michael Schore <mschore@twitter.com>
2013-12-16 14:11:52 +09:00
Norman Maurer
b3d8c81557 Fix all leaks reported during tests
- One notable leak is from WebSocketFrameAggregator
- All other leaks are from tests
2013-12-07 00:44:56 +09:00
Norman Maurer
0918f3edb2 Fix test 2013-11-28 13:49:39 +01:00
Trustin Lee
14327706a3 Add test case for HttpHeaders.equalsIgnoreCase(...) 2013-11-28 18:24:22 +09:00
Michael Grove
951dcc6c10 copy all sliced buffers, fixes #1848 2013-10-12 19:54:43 +02:00
Bill Gallagher
b5f5175338 minor gc optimization: better DefaultSpdyHeaders.iterator() 2013-10-12 10:05:31 +02:00
Norman Maurer
ee192f0321 [#1880] Use ByteBufAllocator when read bytes into new chunks 2013-10-01 10:10:43 +02:00
Arron Norwell
ce58e76e13 HttpRequestEncoder should append '/' to absolute path requests only when needed 2013-09-22 13:41:35 +02:00
Norman Maurer
3957ee32bb [#1833] Add testcase for fix 2013-09-14 12:00:05 +02:00
Norman Maurer
0065006824 [#1818] Pass through message as they are when no compression is needed 2013-09-09 11:32:37 +02:00
Mike Schore
005d33a761 SPDY: allow MAX_CONCURRENT_STREAMS to be set to 0 2013-08-24 16:55:14 +02:00
Norman Maurer
3b94154893 [#1690] Correctly return first header value on DefaultHttpHeaders.get(..) 2013-08-06 07:12:45 +02:00
Norman Maurer
e09aea5902 Rename methods of EmbeddedChannel and mark old ones as @deprecated 2013-07-31 16:04:08 +09:00
Norman Maurer
32b671f4dc [1575] Correctly parse Content-Type value 2013-07-14 15:48:00 +02:00
Norman Maurer
b57d9f307f Allow per-write promises and disallow promises on flush()
- write() now accepts a ChannelPromise and returns ChannelFuture as most
  users expected.  It makes the user's life much easier because it is
  now much easier to get notified when a specific message has been
  written.
- flush() does not create a ChannelPromise nor returns ChannelFuture.
  It is now similar to what read() looks like.
2013-07-11 00:49:48 +09:00
Trustin Lee
26e9d70457 Remove channelReadSuspended / Rename messageReceived(Last) to channelRead(Complete)
- Remove channelReadSuspended because it's actually same with messageReceivedLast
- Rename messageReceived to channelRead
- Rename messageReceivedLast to channelReadComplete

We renamed messageReceivedLast to channelReadComplete because it
reflects what it really is for.  Also, we renamed messageReceived to
channelRead for consistency in method names.
2013-07-09 23:58:51 +09:00
Trustin Lee
cbd8817905 Remove MessageList from public API and change ChannelInbound/OutboundHandler accordingly
I must admit MesageList was pain in the ass.  Instead of forcing a
handler always loop over the list of messages, this commit splits
messageReceived(ctx, list) into two event handlers:

- messageReceived(ctx, msg)
- mmessageReceivedLast(ctx)

When Netty reads one or more messages, messageReceived(ctx, msg) event
is triggered for each message.  Once the current read operation is
finished, messageReceivedLast() is triggered to tell the handler that
the last messageReceived() was the last message in the current batch.

Similarly, for outbound, write(ctx, list) has been split into two:

- write(ctx, msg)
- flush(ctx, promise)

Instead of writing a list of message with a promise, a user is now
supposed to call write(msg) multiple times and then call flush() to
actually flush the buffered messages.

Please note that write() doesn't have a promise with it.  You must call
flush() to get notified on completion. (or you can use writeAndFlush())

Other changes:

- Because MessageList is completely hidden, codec framework uses
  List<Object> instead of MessageList as an output parameter.
2013-07-09 23:51:48 +09:00
Norman Maurer
086ae3536c [#1533] Introduce ByteBufHolder.duplicate() and make use of it in DefaultChannelGroup.write(...) 2013-07-06 21:17:51 +02:00
Norman Maurer
b26e61d689 Fix broken tests 2013-07-05 07:18:33 +02:00
Trustin Lee
e0805ecea9 SPDY: handle too large header blocks
- Forward-ported 22b8a96e044b77e5fadc5a1217080a1f9c69aa9c
2013-06-25 11:07:15 +09:00
Trustin Lee
a969613540 Merge ChannelInboundConsumingHandler into SimpleChannelInboundHandler
- SimpleChannelInboundHandler now has a constructor parameter to let a
  user decide to enable automatic message release. (the default is to
  enable), which makes ChannelInboundConsumingHandler of less value.
2013-06-25 11:07:14 +09:00
Norman Maurer
bfc9c6d80d Add ChannelInboundConsumingHandler
..which is useful when the handler is placed at the last position of the
pipeline because it releases the received messages automatically.
2013-06-25 11:07:14 +09:00
Trustin Lee
96380e756c Fix test failures introduced by 78d8f05c21 2013-06-13 11:51:03 +09:00
Norman Maurer
a403da3042 Rewrite HTTP encoder to use gathering writes 2013-06-13 11:02:31 +09:00
Trustin Lee
78d8f05c21 Make sure that HttpObjectDecoder decodes the last HTTP message without 'Content-Length' header
- Fixes #1410
- Revert 1e5f266a3c and provide a proper fix with a test
2013-06-13 10:57:06 +09:00
Jeff Pinner
c8ca329932 SPDY: update object hierarchy 2013-06-10 16:50:38 +02:00
Trustin Lee
14158070bf Revamp the core API to reduce memory footprint and consumption
The API changes made so far turned out to increase the memory footprint
and consumption while our intention was actually decreasing them.

Memory consumption issue:

When there are many connections which does not exchange data frequently,
the old Netty 4 API spent a lot more memory than 3 because it always
allocates per-handler buffer for each connection unless otherwise
explicitly stated by a user.  In a usual real world load, a client
doesn't always send requests without pausing, so the idea of having a
buffer whose life cycle if bound to the life cycle of a connection
didn't work as expected.

Memory footprint issue:

The old Netty 4 API decreased overall memory footprint by a great deal
in many cases.  It was mainly because the old Netty 4 API did not
allocate a new buffer and event object for each read.  Instead, it
created a new buffer for each handler in a pipeline.  This works pretty
well as long as the number of handlers in a pipeline is only a few.
However, for a highly modular application with many handlers which
handles connections which lasts for relatively short period, it actually
makes the memory footprint issue much worse.

Changes:

All in all, this is about retaining all the good changes we made in 4 so
far such as better thread model and going back to the way how we dealt
with message events in 3.

To fix the memory consumption/footprint issue mentioned above, we made a
hard decision to break the backward compatibility again with the
following changes:

- Remove MessageBuf
- Merge Buf into ByteBuf
- Merge ChannelInboundByte/MessageHandler and ChannelStateHandler into ChannelInboundHandler
  - Similar changes were made to the adapter classes
- Merge ChannelOutboundByte/MessageHandler and ChannelOperationHandler into ChannelOutboundHandler
  - Similar changes were made to the adapter classes
- Introduce MessageList which is similar to `MessageEvent` in Netty 3
- Replace inboundBufferUpdated(ctx) with messageReceived(ctx, MessageList)
- Replace flush(ctx, promise) with write(ctx, MessageList, promise)
- Remove ByteToByteEncoder/Decoder/Codec
  - Replaced by MessageToByteEncoder<ByteBuf>, ByteToMessageDecoder<ByteBuf>, and ByteMessageCodec<ByteBuf>
- Merge EmbeddedByteChannel and EmbeddedMessageChannel into EmbeddedChannel
- Add SimpleChannelInboundHandler which is sometimes more useful than
  ChannelInboundHandlerAdapter
- Bring back Channel.isWritable() from Netty 3
- Add ChannelInboundHandler.channelWritabilityChanges() event
- Add RecvByteBufAllocator configuration property
  - Similar to ReceiveBufferSizePredictor in Netty 3
  - Some existing configuration properties such as
    DatagramChannelConfig.receivePacketSize is gone now.
- Remove suspend/resumeIntermediaryDeallocation() in ByteBuf

This change would have been impossible without @normanmaurer's help. He
fixed, ported, and improved many parts of the changes.
2013-06-10 16:10:39 +09:00
Norman Maurer
83dcf829d6 [#1384] Cache HttpHeaderDateFormat in ThreadLocal 2013-05-23 11:39:50 +02:00
Norman Maurer
3268d6fc2e [#1352] WebSocketFrameAggregator should only throw TooLongFrameException once per complete frame 2013-05-09 21:12:36 +02:00
Norman Maurer
c8de4f03f1 [#1007] Make sure the current message is only reset on LastHttpContent 2013-05-09 20:11:34 +02:00
Norman Maurer
268b059ebb [#1007] HttpObjectAggregator should only throw one TooLongFrameException per full HTTP message 2013-05-09 19:44:39 +02:00
Trustin Lee
1e0c83db23 Introduce AddressedEnvelope message type for generic representation of an addressed message
- Fixes #1282 (not perfectly, but to the extent it's possible with the current API)
- Add AddressedEnvelope and DefaultAddressedEnvelope
- Make DatagramPacket extend DefaultAddressedEnvelope<ByteBuf, InetSocketAddress>
- Rename ByteBufHolder.data() to content() so that a message can implement both AddressedEnvelope and ByteBufHolder (DatagramPacket does) without introducing two getter methods for the content
- Datagram channel implementations now understand ByteBuf and ByteBufHolder as a message with unspecified remote address.
2013-05-01 17:04:43 +09:00
Norman Maurer
9128d4f16a Fix checkstyle 2013-04-30 21:11:41 +02:00
Norman Maurer
a48ed71e86 Fix type in class name 2013-04-30 20:41:50 +02:00
Norman Maurer
57a8a9fd98 [#1321] HttpHeaders.removeTransferEncodingChunked(..) must handle the value in case insensitive manner 2013-04-30 20:41:20 +02:00
Trustin Lee
d92bcff1b6 Fix a bug where HttpContentEncoder generates an empty chunk even if it's not the last chunk
- Fixes #1312
- Added more test cases to ensure the fix
2013-04-27 15:38:28 +09:00
Trustin Lee
7884574c7b Remove freeInboundBuffer() and freeOutboundBuffer() which has no value
- Fixes #1308

freeInboundBuffer() and freeOutboundBuffer() were introduced in the early days of the new API when we did not have reference counting mechanism in the buffer. A user did not want Netty to free the handler buffers had to override these methods.

However, now that we have reference counting mechanism built into the buffer, a user who wants to retain the buffers beyond handler's life cycle can simply return the buffer whose reference count is greater than 1 in newInbound/OutboundBuffer().
2013-04-25 09:15:55 +09:00
Norman Maurer
ca5554dfe7 [#1236] Fix problem where adding a new ChannelHandler could block the eventloop
This change also introduce a few other changes which was needed:
 * ChannelHandler.beforeAdd(...) and ChannelHandler.beforeRemove(...) were removed
 * ChannelHandler.afterAdd(...) -> handlerAdded(...)
 * ChannelHandler.afterRemoved(...) -> handlerRemoved(...)
 * SslHandler.handshake() -> SslHandler.hanshakeFuture() as the handshake is triggered automatically after
   the Channel becomes active
2013-04-19 07:00:50 +02:00
Trustin Lee
70eee55a48 Revamp HttpContentEncoder
- Use state machine to simplify the code
- Always produce a chunked response for simplicity
- Change the signature of beginEncode() - HttpContent was simply unnecessary.
- Add more test cases
- Fixes #1280
2013-04-17 12:51:22 +09:00
Trustin Lee
4b1df148a6 Fix a bug where HttpContentEncoder does not update 'Content-Length' and 'Transfer-Encoding' correctly
- Add 'Content-Length' and remove 'Transfer-Encoding' in case of FullHttpMessage
- Remove 'Content-Length' and add 'Transfer-Encoding: chunked' in case of non-full HttpMessage
- Fixes #1275
2013-04-16 11:49:47 +09:00
Andrei Pozolotin
a3e760a003 fix #1234 - duplicate package-info.java errors in eclipse requires release of netty-build v 19 and netty-parent update. 2013-04-05 05:38:05 +09:00
Prajwal Tuladhar
05850da863 enable checkstyle for test source directory and fix checkstyle errors 2013-03-30 13:18:57 +01:00
Norman Maurer
4a9ab4f57c Add WebSocketFrameAggregator which takes care to aggregate fragmented websocket frames 2013-03-27 07:46:11 +01:00
Trustin Lee
d3327e7f46 Fix inspector warnings 2013-03-18 13:10:55 +09:00
ursa
ecaba68348 Fix bug in memory-based HTTP data content initialization with input stream, add test (port from branch 3).
- Fixes #1170
2013-03-18 13:05:40 +09:00
Trustin Lee
6e0e38f09f Fix test failures 2013-03-12 14:43:38 +09:00
Trustin Lee
c660002b4e Add DecoderResult.UNFINISHED to represent the case where a decoder generated a message that was not decoded completely / Remove partial failure in DecoderResult which is not very useful but confusing 2013-03-12 13:04:53 +09:00
Trustin Lee
f67441354a Move logging classes from internal to internal.logging
.. because internal is crowded
2013-02-26 14:54:25 -08:00
Norman Maurer
eb2aa7f6d4 Cleanup 2013-02-26 08:01:18 +01:00
Norman Maurer
30e7ab2f7d [#1089] Correctly offer FullHttpContent if it is used to construct HttpPostRequestDecoder 2013-02-26 07:33:04 +01:00
Trustin Lee
b4f4b95739 Move io.netty.logging to io.netty.internal / Move Signal out of internal because we use it in Channel*MessageAdapters 2013-02-11 20:08:18 +09:00
Trustin Lee
bf0bfe9a69 Fix inspector warnings 2013-02-11 16:52:43 +09:00
Trustin Lee
b9996908b1 Implement reference counting
- Related: #1029
- Replace Freeable with ReferenceCounted
- Add AbstractReferenceCounted
- Add AbstractReferenceCountedByteBuf
- Add AbstractDerivedByteBuf
- Add EmptyByteBuf
2013-02-10 13:10:09 +09:00
Trustin Lee
2ac7983471 Do not trigger inboundBufferUpdated unnecessarily / Fix a bug in WebSocketServerProtocolHandlerTest / Fix a bug in forbiddenHttpRequestResponder() 2013-02-09 20:32:49 +09:00
Trustin Lee
76eb40a4d2 Make ChannelOutboundMessageHandlerAdapter similar to ChannelInboundMessageHandlerAdapter 2013-02-08 17:07:01 +09:00
ursa
f09b85f4e7 Add test for HttpPostRequestDecoder: validate '\r' symbols in the end of binary stream are parsed correctly. 2013-02-06 20:21:56 +01:00
Norman Maurer
3843cfd702 [SPDY] Factor out headers into extra class like we did in http and support method chaining where possible 2013-01-30 08:23:07 +01:00
Norman Maurer
238e03f75b Add setters and getters back to Http Objects
* This is done because we noticed that the previous change limit the usage more then it gave us any benefit. Now it is possible
  again to rewrite the url on the fly or reuse the objects when writing a proxy and so limit the GC pressure.
* Fixes also #979
2013-01-30 07:42:18 +01:00
Trustin Lee
4472fe9795 Remove 'get' prefix 2013-01-17 15:06:46 +09:00
Trustin Lee
eacc474cda Remove 'get' prefix 2013-01-17 14:48:03 +09:00
Norman Maurer
dfbecb796c [#910] Make use of ByteBufHolder in SPDY, HTTP multipart and WebSockets to allow for buffer pooling 2013-01-17 06:34:52 +01:00
Trustin Lee
4ee11cd36f Fix bad test name 2013-01-17 00:42:51 +09:00
Trustin Lee
3b79008eda Change the WebSocket API to use HttpHeaders instead of Map<String, String> for custom headers / Cleanup 2013-01-17 00:33:40 +09:00
Trustin Lee
34820511ff Second HTTP overhaul
- Rename message types for clarity
  - HttpMessage -> FullHttpMessage
  - HttpHeader -> HttpMessage
  - HttpRequest -> FullHttpRequest
  - HttpResponse -> FulllHttpResponse
  - HttpRequestHeader -> HttpRequest
  - HttpResponseHeader -> HttpResponse
- HttpContent now extends ByteBufHolder; no more content() method
- Make HttpHeaders abstract, make its header access methods public, and
  add DefaultHttpHeaders
- Header accessor methods in HttpMessage and LastHttpContent are
  replaced with HttpMessage.headers() and
  LastHttpContent.trailingHeaders(). Both methods return HttpHeaders.
- Remove setters wherever possible and remove 'get' prefix
- Instead of calling setContent(), a user can either specify the content
  when constructing a message or write content into the buffer.
  (e.g. m.content().writeBytes(...))
- Overall cleanup & fixes
2013-01-16 23:46:02 +09:00
Norman Maurer
b7de868003 [#677] Overhaul HTTP codec
This commit tries to simplify the handling of Http easier and more consistent. This has a effect of many channges. Including:
 - HttpMessage was renamed to HttpHeader and the setContent and getContent methods were removed
 - HttpChunk was renamed to HttpContent
 - HttpChunkTrailer was renamed to LastHttpContent
 - HttpCodecUtil was merged into HttpHeaders

Now a "complete" Http message (request or response) contains of the following parts:
 - HttpHeader (HttpRequestHeader or HttpResponseHeader)
 - 0 - n HttpContent objects which contains parts of the content of the message
 - 1 LastHttpContent which marks the end of the message and contains the remaining data of the content

I also changed the sematic of HttpResponse and HttpRequest, these now represent a "complete" message which contains the HttpHeader and the HttpLastContent, and so can be used to eeasily send requests. The HttpMessageAggregator was renamed to HttpObjectAggregator and produce HttpResponse / HttpRequest message.
2013-01-15 17:51:12 +01:00
Trustin Lee
eb337ff5a7 Fix various inspection warnings 2013-01-10 15:23:58 +09:00
Norman Maurer
4e77bacdf7 [#873] [#868] Split ChannelFuture into ChannelFuture and ChannelPromise 2012-12-31 23:27:16 +09:00
Trustin Lee
310a87a51d Fix #814 - Prevent IllegalBufferAccessException on write() and flush()
- Also fixed a incorrect port of SpdySessionHandler
  - Previously, it closed the connection too early when sending a GOAWAY frame
  - After this fix, SpdySessionHandlerTest now passes again without the previous fix
2012-12-18 04:53:37 +09:00
Trustin Lee
e59ac8e79b Do not call inbound event methods directly
- Fixes #831

This commit ensures the following events are never triggered as a direct
invocation if they are triggered via ChannelPipeline.fire*():

- channelInactive
- channelUnregistered
- exceptionCaught

This commit also fixes the following issues surfaced by this fix:

- Embedded channel implementations run scheduled tasks too early
- SpdySessionHandlerTest tries to generate inbound data even after the
  channel is closed.
- AioSocketChannel enters into an infinite loop on I/O error.
2012-12-18 03:04:26 +09:00
Trustin Lee
36c8eb02e8 Fix parameter namings + some more 2012-11-12 12:59:37 +09:00