Commit Graph

1254 Commits

Author SHA1 Message Date
Dmitriy Dumanskiy
c95517f759 Cleanup : removed unnecessary 'continue', explicit array creation, unwrapping 2017-02-10 12:25:01 +01:00
Stephane Landelle
9d45f514a4 Add a constant for Cookie "undefined maxAge"
Motivation:

DefaultCookie currently used an undocumented magic value for undefined
maxAge.

Clients need to be able to identify such value so they can implement a
proper CookieJar.

Ideally, we should add a `Cookie::isMaxAgeDefined` method but I guess
we can’t add a new method without breaking API :(

Modifications:

Add a new constant on `Cookie` interface so clients can use it to
compare with value return by `Cookie.maxAge` and decide if `maxAge` was
actually defined.

Result:

Clients have a better documented way to check if the maxAge attribute
was defined.
2017-02-09 10:14:29 +01:00
Norman Maurer
a7c0ff665c Only use Mockito for mocking.
Motivation:

We used various mocking frameworks. We should only use one...

Modifications:

Make usage of mocking framework consistent by only using Mockito.

Result:

Less dependencies and more consistent mocking usage.
2017-02-07 08:47:22 +01:00
Vladimir Kostyukov
0f9b739508 AggregatedFullHttpMessage.replace should also copy a decoder result
Motivation:

HttpObjectAggregator yields full HTTP messgaes (AggregatedFullHttpMessages) that don't respect decoder result when copied/replaced.

Modifications:

Copy the decoding result over to a new instance produced by AggregatedFullHttpRequest.replace or AggregatedFullHttpResponse.replace .

Result:

DecoderResult is now copied over when an original AggregatedFullHttpMessage is being replaced (i.e., AggregatedFullHttpRequest.replace or AggregatedFullHttpResponse.replace is being called).

New unit tests are passing on this branch but are failing on master.
2017-02-06 07:49:53 +01:00
Norman Maurer
735d6dd636 [maven-release-plugin] prepare for next development iteration 2017-01-30 15:14:02 +01:00
Norman Maurer
76e22e63f3 [maven-release-plugin] prepare release netty-4.1.8.Final 2017-01-30 15:12:36 +01:00
Chris Conroy
9bec25a6eb Set the Transfer-Encoding header instead of adding
Motivation:

HttpUtil.setTransferEncodingChunked could add a second Transfer-Encoding
header if one was already present. While this is technically valid, it
does not appear to be the intent of the method.

Result:

Only one Transfer-Encoding header is present after calling this method.
2017-01-25 07:53:53 +01:00
Dmitriy Dumanskiy
2d11331591 Typo fix in post encoder and replaced static hashmap with array. 2017-01-19 10:44:27 -08:00
Norman Maurer
7f01da8d0f [maven-release-plugin] prepare for next development iteration 2017-01-12 11:36:51 +01:00
Norman Maurer
7a21eb1178 [maven-release-plugin] prepare release netty-4.1.7.Final 2017-01-12 11:35:58 +01:00
Frederic BREGIER
56ddc47f23 Extends HttpPostRequestEncoder to support all methods except TRACE
Motivation:

In Netty, currently, the HttpPostRequestEncoder only supports POST, PUT, PATCH and OPTIONS, while the RFC 7231 allows with a warning that GET, HEAD, DELETE and CONNECT use a body too (but not TRACE where it is explicitely not allowed).
The RFC in chapter 4.3 says:
"A payload within a XXX request message has no defined semantics;
sending a payload body on a XXX request might cause some existing
implementations to reject the request."
where XXX can be replaced by one of GET, HEAD, DELETE or CONNECT.

Current usages, on particular in REST mode, tend to use those extra HttpMethods for such queries.

So this PR proposes to remove the current restrictions, leaving only TRACE as explicitely not supported.

Modification:

In the constructor, where the test is done, replacing all by checking only against TRACE, and adding one test to check that all methods are supported or not.

Result:

Fixes #6138.
2016-12-30 12:00:21 -08:00
Norman Maurer
0eeeb76439 Fix handling of FullHttpResponse when respond to HEAD in HttpServerCodec
Motivation:

cb139043f3 introduced special handling of response to HEAD requests. Due a bug we failed to handle FullHttpResponse correctly.

Modifications:

Correctly handle FullHttpResponse for HEAD requests.

Result:

Works as expected.
2016-12-21 20:53:02 +01:00
Scott Mitchell
3f82b53bae Add unit test for HttpObjectDecoder with message split on buffer boundaries
Motivation:
We should have a unit test which explicitly tests a HTTP message being split between multiple ByteBuf objects.

Modifications:
- Add a unit test to HttpRequestDecoderTest which splits a request between 2 ByteBuf objects

Result:
More unit test coverage for HttpObjectDecoder.
2016-12-20 12:59:00 -08:00
Malik Baktiyarov
16ddf460a6 Added checkStartsWith option for WebSocketServerProtocolHandler
Motivation:

Enables optional .startsWith() matching of req.uri() with websocketPath.

Modifications:

New checkStartsWith boolean option with default false value added to both WebSocketServerProtocolHandler and WebSocketServerProtocolHandshakeHandler. req.uri() matching is based on this option.

Result:

By default old behavior matching via .equal() is preserved. To use checkStartsWith use constructor shortcut: new WebSocketServerProtocolHandler(websocketPath, true) or fill this flag on full form of constructor among other options.
2016-12-20 10:53:33 +01:00
Norman Maurer
cb139043f3 [#5831] HttpServerCodec cannot encode a respons e to HEAD
request with a 'content-encoding: chunked' header

Motivation:

It is valid to send a response to a HEAD request that contains a transfer-encoding: chunked header, but it is not valid to include a body, and there is no way to do this using the netty4 HttpServerCodec.

The root cause is that the netty4 HttpObjectEncoder will transition to the state ST_CONTENT_CHUNK and the only way to transition back to ST_INIT is through the encodeChunkedContent method which will write the terminating length (0\r\n\r\n\r\n), a protocol error when responding to a HEAD request

Modifications:

- Keep track of the method of the request and depending on it handle the response differently when encoding it.
- Added a unit test.

Result:

Correclty handle HEAD responses that are chunked.
2016-12-15 07:54:51 +00:00
Stephane Maldini
ea0ddc0ea2 fix #6066 Support optional filename in HttpPostRequestEncoder
Motivation:

According to https://www.ietf.org/rfc/rfc2388.txt 4.4, filename after "content-disposition" is optional and arbitrary (does not need to match a real filename).

Modifications:

This change supports an extra addBodyFileUpload overload to precise the filename (default to File.getName). If empty or null this argument should be ignored during encoding.

Result:
- A backward-compatible addBodyFileUpload(String, File, String, boolean) to use file.getName() as filename.
- A new addBodyFileUpload(String, String, File, String, boolean) overload to precise filename
- Couple of tests for the empty use case
2016-12-01 06:54:51 +01:00
Stephane Landelle
ba95c401a7 Misc clean up
Motivation:
IntelliJ issues several warnings.

Modifications:

* `ClientCookieDecoder` and `ServerCookieDecoder`:
  * `nameEnd`, `valueBegin` and `valueEnd` don't need to be initialized
  * `keyValLoop` loop doesn't been to be labelled, as it's the most inner one (same thing for labelled breaks)
  * Remove `if (i != headerLen)` as condition is always true
* `ClientCookieEncoder` javadoc still mention old logic
* `DefaultCookie`, `ServerCookieEncoder` and `DefaultHttpHeaders` use ternary ops that can be turned into simple boolean ones
* `DefaultHeaders` uses a for(int) loop over an array. It can be turned into a foreach one as javac doesn't allocate an iterator to iterate over arrays
* `DefaultHttp2Headers` and `AbstractByteBuf` `equal` can be turned into a single boolean statement
Result:

Cleaner code
2016-11-22 15:17:05 -08:00
Stephane Landelle
f755e58463 Clean up following #6016
Motivation:

* DefaultHeaders from netty-codec has some duplicated logic for header date parsing
* Several classes keep on using deprecated HttpHeaderDateFormat

Modifications:

* Move HttpHeaderDateFormatter to netty-codec and rename it into HeaderDateFormatter
* Make DefaultHeaders use HeaderDateFormatter
* Replace HttpHeaderDateFormat usage with HeaderDateFormatter

Result:

Faster and more consistent code
2016-11-21 12:35:40 -08:00
radai-rosenblatt
886a7aae46 Fix timestamp parsing in HttpHeaderDateFormatter
Motivation:
code assumes a numeric value of 0 means no digits were read between separators, which fails for timestamps like 00:00:00.
also code accepts invalid timestamps like 0:0:000

Modifications:
explicitly check for number of digits between separators instead of relying on the numeric value.
also add tests.

Result:
timestamps with 00 successfully parse, timestamps with 000 no longer

Signed-off-by: radai-rosenblatt <radai.rosenblatt@gmail.com>
2016-11-21 10:17:54 +01:00
Norman Maurer
0b3122d8ff Deprecate HttpUtil.getCharsetAsString(...) and introduce HttpUtil.getCharsetAsSequence(...).
Motivation:

The method HttpUtil.getCharsetAsString(...) is missleading as its return type is CharSequence and not String.

Modifications:

Deprecate HttpUtil.getCharsetAsString(...) and introduce HttpUtil.getCharsetAsSe
quence(...).

Result:

Less confusing method name.
2016-11-21 07:47:20 +01:00
Stephane Landelle
edc4842309 Fix cookie date parsing, close #6016
Motivation:
* RFC6265 defines its own parser which is different from RFC1123 (it accepts RFC1123 format but also other ones). Basically, it's very lax on delimiters, ignores day of week and timezone. Currently, ClientCookieDecoder uses HttpHeaderDateFormat underneath, and can't parse valid cookies such as Github ones whose expires attribute looks like "Sun, 27 Nov 2016 19:37:15 -0000"
* ServerSideCookieEncoder currently uses HttpHeaderDateFormat underneath for formatting expires field, and it's slow.

Modifications:
* Introduce HttpHeaderDateFormatter that correctly implement RFC6265
* Use HttpHeaderDateFormatter in ClientCookieDecoder and ServerCookieEncoder
* Deprecate HttpHeaderDateFormat

Result:
* Proper RFC6265 dates support
* Faster ServerCookieEncoder and ClientCookieDecoder
* Faster tool for handling headers such as "Expires" and "Date"
2016-11-18 11:22:21 +00:00
Norman Maurer
0bc30a123e Eliminate usage of releaseLater(...) to reduce memory usage during tests
Motiviation:

We used ReferenceCountUtil.releaseLater(...) in our tests which simplifies a bit the releasing of ReferenceCounted objects. The problem with this is that while it simplifies stuff it increase memory usage a lot as memory may not be freed up in a timely manner.

Modifications:

- Deprecate releaseLater(...)
- Remove usage of releaseLater(...) in tests.

Result:

Less memory needed to build netty while running the tests.
2016-11-18 09:34:11 +01:00
Adrian Gonzalez
baac352f74 WebSocketClientHandshaker.rawPath(URI) should use the raw query
Motivation:

If the wsURL contains an encoded query, it will be decoded when generating the raw path.  For example if the wsURL is http://test.org/path?a=1%3A5, the returned raw path would be /path?a=1:5

Modifications:

Use wsURL.getRawQuery() rather than wsURL.getQuery()

Result:

rawPath will now return /path?a=1%3A5
2016-11-14 08:45:27 +01:00
Stephane Landelle
75728faf9b Add more HttpHeaders values
Motivation:
Some commons values are missing from HttpHeader values constants.

Modifications:
- Add constants for "application/json" Content-Type
- Add constants for "gzip,deflate" Content-Encoding

Result:
More HttpHeader values constants available, both in
`HttpHeaders.Values` and `HttpHeaderValues`.
2016-11-12 15:54:02 +01:00
Bryce Anderson
f0f0edbf78 HttpObjectAggregator adds 'Connection: close' header if necessary
Motivation:

The HttpObjectAggregator never appends a 'Connection: close' header to
the response of oversized messages even though in the majority of cases
its going to close the connection.

Modification:

This PR addresses that by ensuring the requisite header is present when
the connection is going to be closed.

Result:

Gracefully signal that we are about to close the connection.
2016-11-08 08:43:30 +01:00
Norman Maurer
8269e0f046 [#5892] Correct handle HttpMessage that is EOF terminated
Motivation:

We need to ensure we not add the Transfer-Encoding header if the HttpMessage is EOF terminated.

Modifications:

Only add the Transfer-Encoding header if an Content-Length header is present.

Result:

Correctly handle HttpMessage that is EOF terminated.
2016-11-01 11:13:44 +01:00
Moses Nakamura
bff951ca07 codec-http: HttpClientUpgradeHandler can handle streamed responses
Motivation:

We want to reject the upgrade as quickly as possible, so that we can
support streamed responses.

Modifications:

Reject the upgrade as soon as we inspect the headers if they're wrong,
instead of waiting for the entire response body.

Result:

If a remote server doesn't know how to use the http upgrade and tries to
responsd with a streaming response that never ends, the client doesn't
buffer forever, but can instead pass it along.  Fixes #5954
2016-11-01 06:32:41 +01:00
Norman Maurer
5f533b7358 [maven-release-plugin] prepare for next development iteration 2016-10-14 13:20:41 +02:00
Norman Maurer
35fb0babe2 [maven-release-plugin] prepare release netty-4.1.6.Final 2016-10-14 12:47:19 +02:00
radai-rosenblatt
15ac6c4a1f Clean-up unused imports
Motivation:

the build doesnt seem to enforce this, so they piled up

Modifications:

removed unused import lines

Result:

less unused imports

Signed-off-by: radai-rosenblatt <radai.rosenblatt@gmail.com>
2016-09-30 09:08:50 +02:00
Norman Maurer
cf8f6e3e2f [#5861] HttpUtil.getContentLength(HttpMessage, long) throws unexpected NumberFormatException
Motivation:

The Javadocs of HttpUtil.getContentLength(HttpMessage, long) and its int overload state that the provided default value is returned if the Content-Length value is not a number. NumberFormatException is thrown instead.

Modifications:

Correctly handle when the value is not a number.

Result:

API works as stated in javadocs.
2016-09-29 21:32:07 +02:00
Scott Mitchell
dd1ba2a252 HttpObjectDecoder resetRequested not updated after reset
Motivation:
HttpObjectDecoder maintains a resetRequested flag which is used to determine if internal state should be reset when a decode occurs. However after a reset is done the resetRequested flag is not set to false. This leads to all data after this point being discarded.

Modifications:
- Set resetRequested to false when a reset is done

Result:
HttpObjectDecoder can still function after a reset.
2016-09-22 10:58:44 -07:00
Christopher O'Toole
c57d4bed91 Add HttpServerKeepAliveHandler
Motivation:

As discussed in #5738, developers need to concern themselves with setting
connection: keep-alive on the response as well as whether to close a
connection or not after writing a response.  This leads to special keep-alive
handling logic in many different places.  The purpose of the HttpServerKeepAliveHandler
is to allow developers to add this handler to their pipeline and therefore
free themselves of having to worry about the details of how Keep-Alive works.

Modifications:

Added HttpServerKeepAliveHandler to the io.netty.handler.codec.http package.

Result:

Developers can start using HttpServerKeepAliveHandler in their pipeline instead
of worrying about when to close a connection for keep-alive.
2016-09-15 15:59:21 -07:00
Gaston Tonietti
245fb52c90 Provide extra info together with handshake complete event.
Motivation:

As described in #5734

Before this change, if the server had to do some sort of setup after a
handshake was completed based on handshake's information, the only way
available was to wait (in a separate thread) for the handshaker to be
added as an attribute to the channel. Too much hassle.

Modifications:

Handshake completed event need to be stateful now, so I've added a tiny
class holding just the HTTP upgrade request and the selected subprotocol
which is fired as an event after the handshake has finished.
I've also deprecated the old enum used as stateless event and I left the
code that fires it for backward compatibility. It should be removed in
the next mayor release.

Result:

It should be much simpler now to do initialization stuff based on
subprotocol or request headers on handshake completion. No asynchronous
waiting needed anymore.
2016-09-11 17:52:07 +02:00
William Blackie
e3aca1f3d6 CorsHandler to respect http connection (keep-alive) header.
Motivation:

The CorsHandler currently closes the channel when it responds to a preflight (OPTIONS)
request or in the event of a short circuit due to failed validation.

Especially in an environment where there's a proxy in front of the service this causes
unnecessary connection churn.

Modifications:

CorsHandler now uses HttpUtil to determine if the connection should be closed
after responding and to set the Connection header on the response.

Result:

Channel will stay open when the CorsHandler responds unless the client specifies otherwise
or the protocol version is HTTP/1.0
2016-09-06 07:18:53 +02:00
Norman Maurer
54b1a100f4 [maven-release-plugin] prepare for next development iteration 2016-08-26 10:06:32 +02:00
Christopher O'Toole
f6c16f4897 Attempt at improving docs in HttpObjectAggregator
Motivation:

Documentation was added in #2401 to aid developers in understanding
how HttpObjectAggregator works and that it needs an encoder before it.

In #2471 it was pointed out that the documentation added can actually
add to the confusion and that it might have a typo.

This is an attempt at clearing up that confusion.  Feedback is welcome.

Modifications:

- Adjust class level javadoc for HttpObjectAggregator
  * Remove reference to HttpRequestEncoder
  * Point out when HttpResponseEncoder is needed
  * Point out that either HttpRequestDecoder or HttpResponseDecoder is needed
  * Make clear everything must be added before HttpObjectAggregator
  * Mention HttpServerCodec

Result:

Avoid confusion about dependencies for HttpObjectAggregator on the pipeline.
2016-08-30 22:10:51 +02:00
Norman Maurer
1208b90f57 [maven-release-plugin] prepare release netty-4.1.5.Final 2016-08-26 04:59:35 +02:00
Norman Maurer
a8b8553ad1 Revert "CorsHandler to respect http connection (keep-alive) header."
This reverts commit ecd6e5ce6d.
2016-08-24 08:54:29 +02:00
William Blackie
ecd6e5ce6d CorsHandler to respect http connection (keep-alive) header.
Motivation:

The CorsHandler currently closes the channel when it responds to a preflight (OPTIONS)
request or in the event of a short circuit due to failed validation.

Especially in an environment where there's a proxy in front of the service this causes
unnecessary connection churn.

Modifications:

CorsHandler now uses HttpUtil to determine if the connection should be closed
after responding

Result:

Channel will stay open when the CorsHandler responds unless the client specifies otherwise
or the protocol version is HTTP/1.0
2016-08-24 08:50:29 +02:00
Sergey Polovko
3451b3cbb3 Cookie name must be case sensitive
Motivation:

RFC 6265 does not state that cookie names must be case insensitive.

Modifications:

Fix io.netty.handler.codec.http.cookie.DefaultCookie#equals() method to
use case sensitive String#equals() and String#compareTo().

Result:

It is possible to parse several cookies with same names but with
different cases.
2016-08-23 09:44:38 +02:00
Akhil
8d043cc4dd Do not return Access-Control-Allow-Headers on Non-Preflight Cors requests
Motivation:

The CorsHandler currently returns the Access-Control-Allow-Headers
header as on a Non-Preflight CORS request (Simple request).
As per the CORS specification the Access-Control-Allow-Headers header
should only be returned on Preflight requests. (not on simple requests).

https://www.w3.org/TR/2014/REC-cors-20140116/#access-control-allow-headers-response-header

http://www.html5rocks.com/static/images/cors_server_flowchart.png

Modifications:

Modified CorsHandler.java to not add the Access-Control-Allow-Headers
header when responding to Non-preflight CORS request.

Result:

Access-Control-Allow-Headers header will not be returned on a Simple
request (Non-preflight CORS request).
2016-08-16 13:45:04 +02:00
Dmitriy Dumanskiy
1bcc070943 cleanup, duplicated static final fields
Motivation:

There are few duplicated byte[] CRLF fields in code.

Modifications:

Removed duplicated fields as they could be inherited from parent encoder.

Result:

Less static fields.
2016-08-10 11:13:26 +02:00
Dmitriy Dumanskiy
a80ea46b8e Removed custom split method as it is not effective anymore. 2016-08-01 21:49:33 +02:00
Dmitriy Dumanskiy
a4d8f930af small performance fixes : unnecessary unboxing operations removed
Motivation :

Unboxing operations allocate unnecessary objects when it could be avoided.

Modifications:
Replaced Float.valueOf with Number.parseFloat where possible.

Result:

Less unnecessary objects allocations.
2016-08-01 07:10:25 +02:00
Scott Mitchell
82b617dfe9 retainSlice() unwrap ByteBuf
Motivation:
retainSlice() currently does not unwrap the ByteBuf when creating the ByteBuf wrapper. This effectivley forms a linked list of ByteBuf when it is only necessary to maintain a reference to the unwrapped ByteBuf.

Modifications:
- retainSlice() and retainDuplicate() variants should only maintain a reference to the unwrapped ByteBuf
- create new unit tests which generally verify the retainSlice() behavior
- Remove unecessary generic arguments from AbstractPooledDerivedByteBuf
- Remove unecessary int length member variable from the unpooled sliced ByteBuf implementation
- Rename the unpooled sliced/derived ByteBuf to include Unpooled in their name to be more consistent with the Pooled variants

Result:
Fixes https://github.com/netty/netty/issues/5582
2016-07-29 11:16:44 -07:00
Norman Maurer
cb7cf4491c [maven-release-plugin] prepare for next development iteration 2016-07-27 13:29:56 +02:00
Norman Maurer
9466b32d05 [maven-release-plugin] prepare release netty-4.1.4.Final 2016-07-27 13:16:59 +02:00
Ngoc Dao
835f901d5f Fix #5590 QueryStringDecoder#path should decode the path info
Motivation:

Currently, QueryStringDecoder#path simply returns the path info as is, without decoding it as the Javadoc states.

Modifications:

* Make QueryStringDecoder#path decode the path info.
* Add tests to QueryStringDecoderTest.

Result:

QueryStringDecoder#path now decodes the path info as expected.
2016-07-27 09:29:54 +02:00
Norman Maurer
047f6aed28 [maven-release-plugin] prepare for next development iteration 2016-07-15 09:09:13 +02:00
Norman Maurer
b2adea87a0 [maven-release-plugin] prepare release netty-4.1.3.Final 2016-07-15 09:08:53 +02:00
Norman Maurer
c735b3e147 [#5514] Fix DiskFileUpload and MemoryFileUpload equals(...) method.
Motivation:

DiskFileUpload and MemoryFileUpload.equals(...) are broken.

Modifications:

Fix implementation and add unit test.

Result:

Equals method are correct now.
2016-07-14 09:09:16 +02:00
Masaru Nomura
009680488e Fix set100ContinueExpected(...) jsvadoc
Motivation:

We don't have an argument named {@code value} but have {@code set} and
{@code expected} in HttpHeaders and HttpUtil respectively.

Modifications:

I replaced {@code value} to {@code set} and {@code expected} in HttpHeaders
and HttpUtil respectively.

Result:

Now javadoc says;
If {@code set} is {@code true}, the {@code "Expect: 100-continue"} header is
set and all other previous {@code "Expect"} headers are removed. Otherwise,
all {@code "Expect"} headers are removed completely. in HttpHeaders

If {@code expected} is {@code true}, the {@code "Expect: 100-continue"} header
is set and all other previous {@code "Expect"} headers are removed. Otherwise,
all {@code "Expect"} headers are removed completely. in HttpUtil
2016-07-04 19:03:14 +02:00
Norman Maurer
4676a2271c [maven-release-plugin] prepare for next development iteration 2016-07-01 10:33:32 +02:00
Norman Maurer
ad270c02b9 [maven-release-plugin] prepare release netty-4.1.2.Final 2016-07-01 09:07:40 +02:00
Tim Brooks
d964bf6f18 Remove usages of deprecated methods group() and childGroup().
Motivation:

These methods were recently deprecated. However, they remained in use in several locations in Netty's codebase.

Modifications:

Netty's code will now access the bootstrap config to get the group or child group.

Result:

No impact on functionality.
2016-06-21 14:06:57 +02:00
Norman Maurer
e845670043 Set some StackTraceElement on pre-instantiated static exceptions
Motivation:

We use pre-instantiated exceptions in various places for performance reasons. These exceptions don't include a stacktrace which makes it hard to know where the exception was thrown. This is especially true as we use the same exception type (for example ChannelClosedException) in different places. Setting some StackTraceElements will provide more context as to where these exceptions original and make debugging easier.

Modifications:

Set a generated StackTraceElement on these pre-instantiated exceptions which at least contains the origin class and method name. The filename and linenumber are specified as unkown (as stated in the javadocs of StackTraceElement).

Result:

Easier to find the origin of a pre-instantiated exception.
2016-06-20 11:33:05 +02:00
Norman Maurer
16be36a55f [#5402] sec-websocket-origin should mention HTTPS
Motivation:

When HTTPS is used we should use https in the sec-websocket-origin / origin header

Modifications:

- Correctly generate the sec-websocket-origin / origin header
- Add unit tests.

Result:

Generate correct header.
2016-06-20 11:22:09 +02:00
Nitesh Kant
ee0897a1d9 HttpContentDecompressor should change decompressed requests to chunked encoding. Fixes issue #5428
`HttpContentDecoder` was removing `Content-Length` header but not adding a `Transfer-Encoding` header which goes against the HTTP spec.

Added `Transfer-Encoding` header with value `chunked` when `Content-Length` is removed.
Modified existing unit test to also check for this condition.

Compliance with HTTP spec.
2016-06-20 07:43:06 +02:00
Norman Maurer
4a1e0ceb4d [5382] HttpContentEncoder should not set chunked transfer-encoding for HTTP/1.0
Motivation:

When using HttpContentCompressor and the HttpResponse is protocol version 1.0, HttpContentEncoder.encode() should not set the transfer-encoding header to chunked. Chunked transfer-encoding is not valid for HTTP 1.0 - this causes ERR_CONTENT_DECODING_FAILED errors in chrome and similar failures in IE.

Modifications:

Skip HTTP/1.0 messages

Result:

Be able to serve HTTP/1.0 as well when HttpContentEncoder is in the pipeline.
2016-06-17 06:35:33 +02:00
Norman Maurer
f5eea4698d Fix possible NPE in HttpCunkedInput if wrapped ChunkedInput.readChunk(...) return null.
Motivation:

Its completly fine for ChunkedInput.readChunk(...) to return null to indicate there is currently not any data to read. We need to handle this in HttpChunkedInput to not produce a NPE when constructing the HttpContent.

Modifications:

If readChunk(...) return null just return null as well.

Result:

No more NPE.
2016-06-17 06:27:04 +02:00
Scott Mitchell
328a1ec01b cleanup from 819b26b
Motivation:
I cherry-picked 819b26b too soon. There were entries added to a deprecated class which should only go into the non-deprecated version of the class.

Modifications:
- Remove the static final variables that were added as duplicates to the deprecated class

Result:
Deprecated code does not grown in volume without need.
2016-06-14 09:28:35 -07:00
Stephane Landelle
819b26b4bc Add more HttpHeaders values
Motivation:
Some commons values are missing from HttpHeader values constants.

Modifications:
- Add constants for "application/json" Content-Type
- Add constants for "gzip,deflate" Content-Encoding

Result:
More HttpHeader values constants available, both in
`HttpHeaders.Values` and `HttpHeaderValues`.
2016-06-14 09:23:21 -07:00
Sina Tadayon
eb1d932466 Support WebSocket data chunked transfer
Motivation:
Support fetches data chunk by chunk for use with WebSocket chunked transfers.

Modifications:
Create a WebSocketChunkedInput.java that add to io.netty.handler.codec.http.websocketx package

Result:
The WebSocket transfers/fetches data chunk by chunk.
2016-06-13 08:55:07 +02:00
Guido Medina
c3abb9146e Use shaded dependency on JCTools instead of copy and paste
Motivation:
JCTools supports both non-unsafe, unsafe versions of queues and JDK6 which allows us to shade the library in netty-common allowing it to stay "zero dependency".

Modifications:
- Remove copy paste JCTools code and shade the library (dependencies that are shaded should be removed from the <dependencies> section of the generated POM).
- Remove usage of OneTimeTask and remove it all together.

Result:
Less code to maintain and easier to update JCTools and less GC pressure as the queue implementation nt creates so much garbage
2016-06-10 13:19:45 +02:00
Norman Maurer
398efb1f71 Ensure valid message sequence if channel is closed before receive headers.
Motivation:

When the channel is closed while we still decode the headers we currently not preserve correct message sequence. In this case we should generate an invalid message with a current cause.

Modifications:

Create an invalid message with a PrematureChannelClosureException as cause when the channel is closed while we decode the headers.

Result:

Correct message sequence preserved and correct DecoderResult if the channel is closed while decode headers.
2016-06-09 22:42:46 +02:00
Norman Maurer
4dec7f11b7 [maven-release-plugin] prepare for next development iteration 2016-06-07 18:52:34 +02:00
Norman Maurer
cf670fab75 [maven-release-plugin] prepare release netty-4.1.1.Final 2016-06-07 18:52:22 +02:00
Norman Maurer
844976a0a2 Ensure the same ByteBufAllocator is used in the EmbeddedChannel when compress / decompress. Related to [#5294]
Motivation:

The user may specify to use a different allocator then the default. In this case we need to ensure it is shared when creating the EmbeddedChannel inside of a ChannelHandler

Modifications:

Use the config of the "original" Channel in the EmbeddedChannel and so share the same allocator etc.

Result:

Same type of buffers are used.
2016-05-31 09:08:33 +02:00
Norman Maurer
6ca49d1336 [maven-release-plugin] prepare for next development iteration 2016-05-25 19:16:44 +02:00
Norman Maurer
446b38db52 [maven-release-plugin] prepare release netty-4.1.0.Final 2016-05-25 19:14:15 +02:00
Norman Maurer
7b25402e80 Add CompositeByteBuf.addComponent(boolean ...) method to simplify usage
Motivation:

At the moment the user is responsible to increase the writer index of the composite buffer when a new component is added. We should add some methods that handle this for the user as this is the most popular usage of the composite buffer.

Modifications:

Add new methods that autoamtically increase the writerIndex when buffers are added.

Result:

Easier usage of CompositeByteBuf.
2016-05-21 19:52:16 +02:00
Scott Mitchell
1cb706ac93 HTTP/2 HPACK Header Name Validation and Trailing Padding
Motivation:
The HPACK code currently disallows empty header names. This is not explicitly forbidden by the HPACK RFC https://tools.ietf.org/html/rfc7541. However the HTTP/1.x RFC https://tools.ietf.org/html/rfc7230#section-3.2 and thus HTTP/2 both disallow empty header names, and so this precondition check should be moved from the HPACK code to the protocol level.
HPACK also requires that string literals which are huffman encoded must be treated as an encoding error if the string has more than 7 trailing padding bits https://tools.ietf.org/html/rfc7541#section-5.2, but this is currently not enforced.

Result:
- HPACK to allow empty header names
- HTTP/1.x and HTTP/2 header validation should not allow empty header names
- Enforce max of 7 trailing padding bits

Result:
Code is more compliant with the above mentioned RFCs
Fixes https://github.com/netty/netty/issues/5228
2016-05-17 13:42:16 -07:00
Trustin Lee
3a9f472161 Make retained derived buffers recyclable
Related: #4333 #4421 #5128

Motivation:

slice(), duplicate() and readSlice() currently create a non-recyclable
derived buffer instance. Under heavy load, an application that creates a
lot of derived buffers can put the garbage collector under pressure.

Modifications:

- Add the following methods which creates a non-recyclable derived buffer
  - retainedSlice()
  - retainedDuplicate()
  - readRetainedSlice()
- Add the new recyclable derived buffer implementations, which has its
  own reference count value
- Add ByteBufHolder.retainedDuplicate()
- Add ByteBufHolder.replace(ByteBuf) so that..
  - a user can replace the content of the holder in a consistent way
  - copy/duplicate/retainedDuplicate() can delegate the holder
    construction to replace(ByteBuf)
- Use retainedDuplicate() and retainedSlice() wherever possible
- Miscellaneous:
  - Rename DuplicateByteBufTest to DuplicatedByteBufTest (missing 'D')
  - Make ReplayingDecoderByteBuf.reject() return an exception instead of
    throwing it so that its callers don't need to add dummy return
    statement

Result:

Derived buffers are now recycled when created via retainedSlice() and
retainedDuplicate() and derived from a pooled buffer
2016-05-17 11:16:13 +02:00
Norman Maurer
ef13d19b8b [#5202] Correctly throw ErrorDataDecoderException when invalid encoded form parameters are present.
Motivation:

At the moment we let the IllegalArgumentException escape when parsing form parameters. This is not expected.

Modifications:

Correctly catch IllegalArgumentException and rethrow as ErrorDataDecoderException.

Result:

Throw correct exception.
2016-05-04 21:14:53 +02:00
Daniel Bevenius
d0cfe24972 Clarifying the that a null String is returned by using @{code} 2016-05-03 08:39:38 +02:00
Daniel Bevenius
0557927b65 Updating allowNullOrigin to return 'null' instead of '*'.
Motivation:
Currently the way a 'null' origin, a request that most often indicated
that the request is coming from a file on the local file system, is
handled is incorrect. We are currently returning a wildcard origin '*'
but should be returning 'null' for the 'Access-Control-Allow-Origin'
which is valid according to the specification [1].

Modifications:
Updated CorsHandler to add a 'null' origin instead of the '*' origin in
the case the request origin is 'null.

Result:
All test pass and the CORS example as does the cors.html example if you
try to serve it by opening the file directly in a web browser.

[1]
https://www.w3.org/TR/cors/#access-control-allow-origin-response-header
2016-05-03 08:39:38 +02:00
Guido Medina
5b59250657 TreeMap extra get operation removed.
Motivation:
Checking if a key exists on a TreeMap has a Big O of "log 2 N",
doing it twice is not cheap.

Modifications:
Get the key instead which has the same cost and check if it is null.

Result:
Faster code due to one expensive operation removed.
2016-04-25 09:49:05 -07:00
Norman Maurer
0035630bd0 We need to ensure we correct reset decoder in decodeLast() to not produce multiple LastHttpContent instances.
Motivation:

We missed to reset the decoder when asked for it in HttpObjectDecoder and so sometimes could produce more then one LastHttpContent in a sequence during channelInactive.

This did show up as AssertionError:

22:22:35.499 [nioEventLoopGroup-3-1] WARN  i.n.channel.DefaultChannelPipeline - An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.
java.lang.AssertionError: null
	at io.netty.handler.codec.http.HttpObjectAggregator.decode(HttpObjectAggregator.java:205) ~[classes/:na]
	at io.netty.handler.codec.http.HttpObjectAggregator.decode(HttpObjectAggregator.java:57) ~[classes/:na]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:89) ~[classes/:na]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:292) [classes/:na]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:278) [classes/:na]
	at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:428) [classes/:na]
	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:277) [classes/:na]
	at io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:343) [classes/:na]
	at io.netty.handler.codec.ByteToMessageDecoder.channelInactive(ByteToMessageDecoder.java:309) [classes/:na]
	at io.netty.handler.codec.http.HttpClientCodec$Decoder.channelInactive(HttpClientCodec.java:228) [classes/:na]
	at io.netty.channel.CombinedChannelDuplexHandler.channelInactive(CombinedChannelDuplexHandler.java:213) [classes/:na]
        ...

Modifications:

Correctly reset decoder.

Result:

Correctly only produce one LastHttpContent per sequence.
2016-04-14 09:33:23 +02:00
Norman Maurer
718bf2fa45 Fix resource-leak which was reported as a result of commit 69070c37ba 2016-04-12 16:27:02 +02:00
Norman Maurer
572bdfb494 [maven-release-plugin] prepare for next development iteration 2016-04-10 08:37:18 +02:00
Norman Maurer
c6121a6f49 [maven-release-plugin] prepare release netty-4.1.0.CR7 2016-04-10 08:36:56 +02:00
Norman Maurer
4652223dec Fix resource leak in test introduced by 69070c37ba 2016-04-10 08:04:57 +02:00
Norman Maurer
6e919f70f8 [maven-release-plugin] rollback the release of netty-4.1.0.CR7 2016-04-09 22:13:44 +02:00
Norman Maurer
4cdd51509a [maven-release-plugin] prepare release netty-4.1.0.CR7 2016-04-09 22:05:34 +02:00
Hyangtack Lee
24254b159f Propagate h2c upgrade success event to the next handler before removing source codec
Motivation:

When upgrading h2c, I found that sometimes both of http2 settings frame and http response message was arrived before receiving upgrade success event. It was because ByteToMessageDecoder propagated its internally buffered message to the next handler when removing itself from pipeline.(refer to ByteToMessageDecoder#handlerRemoved)
I think it's better to propagate upgrade success event when handling 101 switching protocol response.

Modifications:

Upgrade success event will be propagated before removing source codec.

Result:

It guarantees that upgrade success event will be arrived first at the next handler.
2016-04-07 17:41:46 +02:00
Norman Maurer
f46cfbc590 [#5059] Deprecate method with typo and introduce a new one without typo
Motivation:

There is a spelling error in FileRegion.transfered() as it should be transferred().

Modifications:

Deprecate old method and add a new one.

Result:

Fix typo and can remove the old method later.
2016-04-05 15:06:46 +02:00
Trustin Lee
3b941c2a7c [maven-release-plugin] prepare for next development iteration 2016-04-02 01:25:05 -04:00
Trustin Lee
7368ccc539 [maven-release-plugin] prepare release netty-4.1.0.CR6 2016-04-02 01:24:55 -04:00
Norman Maurer
cee38ed2b6 [maven-release-plugin] prepare for next development iteration 2016-03-29 16:45:13 +02:00
Norman Maurer
9cd9e7daeb [maven-release-plugin] prepare release netty-4.1.0.CR5 2016-03-29 16:44:33 +02:00
Stephane Landelle
881ff3cd98 Drop broken DefaultCookie name validation, close #4999
Motivation:

DefaultCookie constructor performs a name validation that doesn’t match
RFC6265. Moreover, such validation is already performed in strict
encoders and decoders.

Modifications:

Drop DefaultCookie name validation, rely on encoders and decoders.

Result:

no more duplicate broken validation
2016-03-22 12:32:09 +01:00
Norman Maurer
28d03adbfe [maven-release-plugin] prepare for next development iteration 2016-03-21 11:51:50 +01:00
Norman Maurer
4653dc1d05 [maven-release-plugin] prepare release netty-4.1.0.CR4 2016-03-21 11:51:12 +01:00
Norman Maurer
4e3a413047 Correctly handle UpgradeEvent.release(decrement).
Motivation:

We missed to pass the decrement value to the wrapped FullHttpRequest and so missed to decrement the reference count in the correct way.

Modifications:

Correctly pass the decrement value to the wrapped request.

Result:

UpgradeEvent.release(decrement) works as expected.
2016-03-20 09:34:12 +01:00
Norman Maurer
8ec594c6eb Change HttpServerUpgradeHandler.UpgradeCodec to allow aborting upgrade
Motivation:

HttpServerUpgradeHandler.UpgradeCodec.prepareUpgradeResponse should allow to abort the upgrade and so just continue with using HTTP. Beside this we should only pass in the response HttpHeaders as this is inline with the docs.

Modifications:

- UpgradeCodec.prepareUpgradeResponse now allows to return a boolean and so allows to specifiy if the upgrade should take place.
- Change the param from FullHttpResponse to HttpHeaders to be inline with the javadocs.

Result:

More flexible and correct handling of upgrades.
2016-03-18 17:01:59 +01:00
Stephane Landelle
d747438366 Add ! to allowed cookie value chars
Motivation:

! is missing from allowed cookie value chars, as per https://tools.ietf.org/html/rfc6265#section-4.1.1.
Issue was originally reported on Play!, see https://github.com/playframework/playframework/issues/4460#issuecomment-198177302.

Modifications:

Stick to RFC6265 ranges.

Result:

RFC6265 compliance, ! is supported
2016-03-18 16:58:54 +01:00
Norman Maurer
ed9d6c79bc [#4972] Remove misleading argument from HttpServerUpgradeHandler.UpgradeCodec.upgradeTo
Motivation:

upgradeTo(...) takes the response as paramater, but the respone itself was already written to the Channel. This gives the user the impression the response can be changed or even act on it which may not be safe anymore once it was written and has been released.

Modifications:

Remove the response param from the method.

Result:

Less confusion and safer usage.
2016-03-17 10:50:07 +01:00
Julien Viet
3d7cec6376 Bug fix for HttpPostMultipartRequestDecoder part decoding with an invalid charset not reported as an ErrorDataDecoderException
Motivation:

The current HttpPostMultipartRequestDecoder can decode multipart/form-data parts with a Content-Type that specifies a charset. When this charset is invalid the Charset.forName() throws an unchecked UnsupportedCharsetException. This exception is not catched by the decoder. It should actually be rethrown as an ErrorDataDecoderException, because the developer using the API would expect this validation failure to be reported as such.

Modifications:

Add a catch block for UnsupportedCharsetException and rethrow it as an ErrorDataDecoderException.

Result:

UnsupportedCharsetException are now rethrown as ErrorDataDecoderException.
2016-03-10 18:33:06 +01:00
Sergey Polovko
68bbd4e966 Handle only those http requests that equal to adjusted websocket path
Motivation:

It will be easier to support websockets in server application by using WebSocketServerProtocolHandshakeHandler class and not reinvent its functionality. But currently it handles all http requests as if they were websocket handshake requests.

Modifications:

Check if http request path is equals to adjusted websocket path.
Fixed example of websocket server implementation.

Result:

WebSocketServerProtocolHandshakeHandler handles only websocket handshake requests.
2016-03-04 08:36:14 +01:00
Dmitry Spikhalskiy
0d3eda38e1 Helper method to get mime-type from Content-Type header of HttpMessage 2016-03-03 15:18:39 +01:00
Norman Maurer
9aac6dac2e [#4386] ByteToMessage.decodeLast(...) should not call decode(...) if buffer is empty.
Motivation:

If the input buffer is empty we should not have decodeLast(...) call decode(...) as the user may not expect this.

Modifications:

- Not call decode(...) in decodeLast(...) if the input buffer is empty.
- Add testcases.

Result:

decodeLast(...) will not call decode(...) if input buffer is empty.
2016-03-01 08:42:26 +01:00
Norman Maurer
ca443e42e0 [maven-release-plugin] prepare for next development iteration 2016-02-19 23:00:11 +01:00
Norman Maurer
f39eb9a6b2 [maven-release-plugin] prepare release netty-4.1.0.CR3 2016-02-19 22:59:52 +01:00
Brendt Lucas
41d0a81691 Use ByteBufAllocator to allocate ByteBuf for FullHttpMessage Motivation: When converting SPDY or HTTP/2 frames to HTTP/1.x, netty always used an unpooled heap ByteBuf.
Modifications:
When constructing the FullHttpMessage pass in the ByteBuf to use via the ByteBufAllocator assigned via the context.

Result:
The ByteBuf assigned to the FullHttpMessage can now be configured as a pooled/unpooled, direct/heap based ByteBuf via the ByteBufAllocator used.
2016-02-17 19:55:52 -08:00
Xiaoyan Lin
333f55e9ce Add unescapeCsvFields to parse a CSV line and implement CombinedHttpHeaders.getAll
Motivation:

See #4855

Modifications:

Unfortunately, unescapeCsv cannot be used here because the input could be a CSV line like `"a,b",c`. Hence this patch adds unescapeCsvFields to parse a CSV line and split it into multiple fields and unescaped them. The unit tests should define the behavior of unescapeCsvFields.

Then this patch just uses unescapeCsvFields to implement `CombinedHttpHeaders.getAll`.

Result:

`CombinedHttpHeaders.getAll` will return the unescaped values of a header.
2016-02-15 15:26:15 -08:00
Scott Mitchell
b112673554 ByteToMessageDecoder ChannelInputShutdownEvent support
Motivation:
b714297a44 introduced ChannelInputShutdownEvent support for HttpObjectDecoder. However this should have been added to the super class ByteToMessageDecoder, and ByteToMessageDecoder should not propegate a channelInactive event through the pipeline in this case.

Modifications:
- Move the ChannelInputShutdownEvent handling from HttpObjectDecoder to ByteToMessageDecoder
- ByteToMessageDecoder doesn't call ctx.fireChannelInactive() on ChannelInputShutdownEvent

Result:
Half closed events are treated more generically, and don't get translated into a channelInactive pipeline event.
2016-02-12 16:15:17 -08:00
Scott Mitchell
a15ff32608 HttpObjectDecoder configurable initial buffer size
Motivation:
The initial buffer size used to decode HTTP objects is currently fixed at 128. This may be too small for some use cases and create a high amount of overhead associated with resizing/copying. The user should be able to configure the initial size as they please.

Modifications:
- Make HttpObjectDecoder's AppendableCharSequence initial size configurable

Result:
Users can more finely tune initial buffer size for increased performance or to save memory.
Fixes https://github.com/netty/netty/issues/4807
2016-02-07 21:23:29 -08:00
Xiaoyan Lin
f59392d9f5 Make "CorsConfigBuilder.allowNullOrigin()" public
Motivation:

"CorsConfigBuilder.allowNullOrigin()" should be public otherwise people can not set it. See #4835

Modifications:

Make "CorsConfigBuilder.allowNullOrigin()" public.

Result:

The user can call "CorsConfigBuilder.allowNullOrigin()" now.
2016-02-07 10:23:24 -08:00
Norman Maurer
75a2ddd61c [maven-release-plugin] prepare for next development iteration 2016-02-04 16:51:44 +01:00
Norman Maurer
7eb3a60dba [maven-release-plugin] prepare release netty-4.1.0.CR2 2016-02-04 16:37:06 +01:00
Norman Maurer
7ef6db3ffd [#4754] Correctly detect websocket upgrade
Motivation:

If the Connection header contains multiple values (which is valid) we fail to detect a websocket upgrade

Modification:

- Add new method which allows to check if a header field contains a specific value (and also respect multiple header values)
- Use this method to detect handshake

Result:

Correct detect handshake if Connection header contains multiple values (seperated by ',').
2016-02-04 14:03:08 +01:00
Norman Maurer
a0758e7e60 [#4794] Support window size flag by default if ZlibCodecFactory supports it.
Motivation:

If the ZlibCodecFactory can support using a custom window size we should support it by default in the websocket extensions as well.

Modifications:

Detect if a custom window size can be handled by the ZlibCodecFactory and if so enable it by default for PerMessageDeflate*ExtensionHandshaker.

Result:

Support window size flag by default in most installations.
2016-02-04 14:01:40 +01:00
Norman Maurer
7a562943ad [#4533] Ensure replacement of decoder is delayed after finishHandshake() is called
Motivation:

If the user calls handshake.finishHandshake() we need to ensure that the user has the chance to setup the pipeline before any WebSocketFrames are read. Because of this we need
to delay the removal of the HttpRequestDecoder.

Modifications:

- Remove the HttpRequestDecoder via the EventLoop and so delay it which gives the user a chance to setup the pipeline after finishHandshake() completes
- Add unit test for this.

Result:

Less surpising and correct behaviour even if the http response and websocket frame are received in one read operation.
2016-02-04 13:57:35 +01:00
Luke Daley
d97f17060f Support non chunked HTTP request bodies larger than Integer.MAX_VALUE.
Motivation:

Request bodies can easily be larger than Integer.MAX_VALUE in practice.
There's no reason, or intention, for Netty to impose this artificial constraint.

Worse, it currently does not fail if the body is larger than this value;
it just silently only reads the first Integer.MAX_VALUE bytes and discards the rest.

This restriction doesn't effect chunked transfers, with no Content-Length header.

Modifications:

Force the use of `long HttpUtil.getContentLength(HttpMessage, long)` instead of
`long HttpUtil.getContentLength(HttpMessage, long)`.

Result:

Netty will support HTTP request bodies of up to Long.MAX_VALUE length.
2016-02-02 08:28:27 +01:00
Trustin Lee
4d6ab1d30d Fix missing trailing data on HTTP client upgrade
Motivation:

When HttpClientUpgradeHandler upgrades from HTTP/1 to another protocol,
it performs a two-step opertion:

1. Remove the SourceCodec (HttpClientCodec)
2. Add the UpgradeCodec

When HttpClientCodec is removed from the pipeline, the decoder being
removed triggers channelRead() event with the data left in its
cumulation buffer. However, this is not received by the UpgradeCodec
becuase it's not added yet. e.g. HTTP/2 SETTINGS frame sent by the
server can be missed out.

To fix the problem, we need to reverse the steps:

1. Add the UpgradeCodec
2. Remove the SourceCodec

However, this does not work as expected either, because UpgradeCodec can
send a greeting message such as HTTP/2 Preface. Such a greeting message
will be handled by the SourceCodec and will trigger an 'unsupported
message type' exception.

To fix the problem really, we need to make the upgrade process 3-step:

1. Remove/disable the encoder of SourceCodec
2. Add the UpgradeCodec
3. Remove the SourceCodec

Modifications:

- Add SourceCodec.prepareUpgradeFrom() so that SourceCodec can remove or
  disable its encoder
- Implement HttpClientCodec.prepareUpgradeFrom() properly
- Miscellaneous:
  - Log the related channel as well When logging the failure to send a
    GOAWAY

Result:

Cleartext HTTP/1-to-HTTP/2 upgrade works again.
2016-02-01 15:52:37 +01:00
Xiaoyan Lin
b7415a3307 Add a reusable ArrayList to InternalThreadLocalMap
Motivation:

See #3411. A reusable ArrayList in InternalThreadLocalMap can avoid allocations in the following pattern:

```
List<...> list = new ArrayList<...>();

add something to list but never use InternalThreadLocalMap

return list.toArray(new ...[list.size()]);

```

Modifications:

Add a reusable ArrayList to InternalThreadLocalMap and update codes to use it.

Result:

Reuse a thread local ArrayList to avoid allocations.
2016-02-01 15:49:28 +01:00
liuzhengyang
b354868dd8 Fix spelling in javadocs and field name.
Motivation:

Fix a spell mistake.

Modifications:

Change 'treshold' to 'threshold'

Result:

The spellchecker warnings of the IDE disappeared.
2016-02-01 12:03:14 +01:00
liuzhengyang
2a9d392a31 Motivation:
Fix a spell mistake.

Modifications:

Change 'treshold' to 'threshold'

Result:

The spellchecker warnings of the IDE disappeared.
2016-02-01 12:03:06 +01:00
Norman Maurer
a2732c6542 [#4755] Make WebSocketClientCompressionHandler @Sharable
Motivation:

WebSocketClientCompressionHandler is stateless so it should be @Sharable.

Modifications:

Add @Sharable annotation to WebSocketClientCompressionHandler, make constructor private and add static field to get the instance.

Result:

Less object creation.
2016-01-28 10:28:09 +01:00
houdejun214
a6fd8a96bf Set default CONTENT_TYPE when it is absent in multipart request body
Motivation:

I am use netty as a http server, it fail to decode some POST request when the request absent Content-Type in the multipart/form-data body.

Modifications:

Set content_type with default application/octet-stream to parse the uploaded file data when the Content-Type is absent in multipart request body

Result:

Can decode the http request as normal.
2016-01-26 10:47:11 +01:00
Norman Maurer
1c417e5f82 [maven-release-plugin] prepare for next development iteration 2016-01-21 15:35:55 +01:00
Norman Maurer
c681a40a78 [maven-release-plugin] prepare release netty-4.1.0.CR1 2016-01-21 15:28:21 +01:00
Norman Maurer
e969b6917c Let CombinedChannelDuplexHandler correctly handle exceptionCaught. Related to [#4528]
Motivation:

ChannelInboundHandler and ChannelOutboundHandler both can implement exceptionCaught(...) method and so we need to dispatch to both of them.

Modifications:

- Correctly first dispatch exceptionCaught to the ChannelInboundHandler but also make sure the next handler it will be dispatched to will be the ChannelOutboundHandler
- Add removeInboundHandler() and removeOutboundHandler() which allows to remove one of the combined handlers
- Let *Codec extends it and not ChannelHandlerAppender
- Remove ChannelHandlerAppender

Result:

Correctly handle events and also have same behavior as in 4.0
2016-01-18 09:54:48 +01:00
Xiaoyan Lin
9ae155d257 Fix InternalAttribute.equals
Motivation:

InternalAttribute doesn't extend Attribute, but its equals only returns true when it compares with an Attribute. So it will return false when comparing with itself.

Modifications:

Make sure InternalAttribute return false for non InternalAttribute objects.

Result:

InternalAttribute's equals works correctly.
2016-01-11 09:25:17 +01:00
Xiaoyan Lin
751ed6cc94 Avoid unnecessary boxing/unboxing
Motivation:

Boxing/unboxing can be avoided.

Modifications:

Use parseInt/parseLong to avoid unnecessary boxing/unboxing.

Result:

Remove unnecessary boxing/unboxing.
2016-01-08 17:38:20 +01:00
Fabian Lange
619d82b56f Removed unused imports
Motivation:

Warnings in IDE, unclean code, negligible performance impact.

Modification:

Deletion of unused imports

Result:

No more warnings in IDE, cleaner code, negligible performance improvement.
2016-01-04 14:32:29 +01:00
Norman Maurer
8dc164ace6 Correctly reset MessageDigest before reusing it.
Motivation:

I missed to reset the MessageDigest before reusing it. This bug was introduced by 79634e661b.

Modifications:

Call reset() on the MessageDigest.

Result:

Correctly reset MessageDigest before re-using
2016-01-04 14:29:21 +01:00
Xiaoyan Lin
1b0adb334b Fix incorrect Serializable
Motivation:

SpdySession.StreamComparator should not be Serializable since SpdySession is not Serializable

Modifications:

Remove Serializable fom SpdySession.StreamComparator

Result:

StreamComparator is not Serializable any more
2015-12-31 22:28:32 +01:00
Alex Petrov
78c4bd474e IllealRefCountException should be IllegalReferenceCountException, fix typos
Motivation:

Typos in javadoc, in "combine" and "recommendations", IllegalReferenceCountException

Modification:

Rename incorrect reference, typos are modified

Result:

Reference is correct, typos are fixed
2015-12-31 19:03:27 +01:00
Norman Maurer
8716b9d4bd Revert "Fix unnecessary boxing and incorrect Serializable"
This reverts commit 0ae6f17285.
2015-12-31 14:48:10 +01:00
Norman Maurer
79634e661b Obtain MessageDigest via FastThreadLocal
Motivation:

Creating a new MessageDigest every time is wasteful, we should store them in FastThreadLocal.

Modifications:

Change WebSocketUtil to store MD5 and SHA1 MessageDigest in FastThreadLocal and use these.

Result:

Less overhead and less GC.
2015-12-31 11:30:47 +01:00
Xiaoyan Lin
0ae6f17285 Fix unnecessary boxing and incorrect Serializable
Motivation:

- AbstractHttp2ConnectionHandlerBuilder.encoderEnforceMaxConcurrentStreams can be the primitive boolean
- SpdySession.StreamComparator should not be Serializable since SpdySession is not Serializable

Modifications:

Use boolean instead and remove Serializable

Result:

- Minor improvement for AbstractHttp2ConnectionHandlerBuilder
- StreamComparator is not Serializable any more
2015-12-31 10:45:24 +01:00
Alex Petrov
0b16c3c513 Add a possibility to create HttpMessage instances with pre-existing Headers
Motivation:

Allow passing HttpHeaders instance to DefaultHttpMessage
in order to avoid eager creation of Headers to
allow users reuse their Headers instance.

Modifications:

Added a constructor with HttpHeaders to DefaultHttpMessage,
Modified DefaultHttpResponse and DefaultHttpRequest
to receive HttpHeaders instances.
Modified DefaultFullHttpReqest and DefaultFullHttpResponse
to receive HttpHeaders, and updated `duplicate` and
`copy` to use new constructors.

Result:

Users can now pass HttpHeaders instance when
constructing Http Requests and Responses.
2015-12-31 08:52:30 +01:00
Norman Maurer
79bc90be32 Fix buffer leak introduced by 693633eeff
Motivation:

As we not used Unpooled anymore for allocate buffers in Base64.* methods we need to ensure we realease all the buffers.

Modifications:

Correctly release buffers

Result:

No more buffer leaks
2015-12-29 17:13:07 +01:00
Xiaoyan Lin
475d901131 Fix errors reported by javadoc
Motivation:

Javadoc reports errors about invalid docs.

Modifications:

Fix some errors reported by javadoc.

Result:

A lot of javadoc errors are fixed by this patch.
2015-12-27 08:36:45 +01:00
Xiaoyan Lin
a96d52fe66 Fix javadoc links and tags
Motivation:

There are some wrong links and tags in javadoc.

Modifications:

Fix the wrong links and tags in javadoc.

Result:

These links will work correctly in javadoc.
2015-12-26 08:34:31 +01:00
Scott Mitchell
fd5316ed6f ChunkedInput.readChunk parameter of type ByteBufAllocator
Motivation:
ChunkedInput.readChunk currently takes a ChannelHandlerContext object as a parameters. All current implementations of this interface only use this object to get the ByteBufAllocator object. Thus taking a ChannelHandlerContext as a parameter is more restrictive for users of this API than necessary.

Modifications:
- Add a new method readChunk(ByteBufAllocator)
- Deprecate readChunk(ChannelHandlerContext) and updates all implementations to call readChunk(ByteBufAllocator)

Result:
API that only requires ByteBufAllocator to use ChunkedInput.
2015-12-24 12:46:40 -08:00
Shixiong Zhu
b5d90388ea Fix HttpHeaderValues.IDENTITY equals usage
Motivation:

HttpHeaderValues.IDENTITY is an AsciiString, but was compared using equals to a String.

Modifications:

Use contentEquals instead.

Result:
Correct comparison.
2015-12-22 09:05:27 +01:00
Norman Maurer
1a2162ec35 Fix broken tests introduced by dc615ecaaf 2015-12-18 10:16:07 +01:00
Norman Maurer
dc615ecaaf [#4212] Backport WebSocket Extension handlers for client and server.
Motivation:

We have websocket extension support (with compression) in old master. We should port this to 4.1

Modifications:

Backport relevant code.

Result:

websocket extension support (with compression) is now in 4.1.
2015-12-18 09:48:10 +01:00
Trustin Lee
412f719aa8 Extract the builder of CorsConfig to top level
Motivation:

Consistency in API design

Modifications:

- Deprecate CorsConfig.Builder and its factory methods
- Deprecate CorsConfig.DateValueGenerator
- Add CorsConfigBuilder and its factory methods
- Fix typo (curcuit -> circuit)

Result:

Consistency with other builder APIs such as SslContextBuilder and
Http2ConnectionHandlerBuilder
2015-12-18 12:38:44 +09:00
Norman Maurer
f31be51774 [#4505] Correctly handle whitespaces in websocket uri's.
Motivation:

If a uri contains whitespaces we need to ensure we correctly escape these when creating the request for the handshake.

Modifications:

- Correctly encode path for uri
- Add tests

Result:

Correctly handle whitespaces when doing websocket upgrade requests.
2015-12-10 13:52:42 +01:00
Trustin Lee
c1f3200c87 Fix the incorrect usage/value of 'Connection: upgrade'
Motivation:

HttpClientUpgradeHandler uses HttpHeaderNames.UPGRADE as the value of
the 'Connection' header, which is incorrect. It should use
HttpHeaderValues.UPGRADE instead (note Names vs Values.)

Also, HttpHeaderValues.UPGRADE should be 'upgrade' rather than
'Upgrade', as defined in:

- https://tools.ietf.org/html/rfc7230#section-6.7

Modifications:

- Use HttpHeaderValues.UPGRADE for a 'Connection' header
- Lowercase the value of HttpHeaderValues.UPGRADE

Result:

- Fixes #4508
- Correct behavior
2015-11-29 07:22:53 +01:00
Trustin Lee
9dd68d0c3e Fix IllegalReferenceCountException caused by HttpClientCodec.upgradeFrom()
Motivation:

On a successful protocol upgrade in HTTP, HttpClientUpgradeHandler calls
HttpClientCodec.upgradeFrom(), which removed both the HTTP encoder and
decoder from the pipeline immediately.

However, because the decoder is in the middle of the decode loop,
removing it from the pipeline immediately will cause the cumulation
buffer to be released prematurely.

This often leads to an IllegalReferenceCountException or missing first
response after the upgrade response.

Modifications:

- Remove the decoder *after* the decode loop is done

Result:

Fixes #4504
2015-11-29 07:21:08 +01:00
Trustin Lee
ef3a9b0acd Relax the sanity check in HttpClientUpgradeHandler
Motivation:

HttpClientUpgradeHandler currently throws an IllegalStateException when
the server sends a '101 Switching Protocols' response that has no
'Upgrade' header.

Some servers do not send the 'Upgrade' header on a successful protocol
upgrade and we could safely assume that the server accepted the
requested protocol upgrade in such a case, looking from the response
status code (101)

Modifications:

- Do not throw an IllegalStateException when the server responded 101
  without a 'Upgrade' header
- Note that we still check the equality of the 'Upgrade' header when it
  is present.

Result:

- Fixes #4523
- Better interoperability
2015-11-29 07:20:08 +01:00
Luke Hutchison
4978266d52 Make cookie encoding conform better to RFC 6265 in STRICT mode.
Motivation:

- On the client, cookies should be sorted in decreasing order of path
  length. From RFC 6265:

      5.4.2. The user agent SHOULD sort the cookie-list in the following
      order:

        *  Cookies with longer paths are listed before cookies with
           shorter paths.

        *  Among cookies that have equal-length path fields, cookies with
           earlier creation-times are listed before cookies with later
           creation-times.

      NOTE: Not all user agents sort the cookie-list in this order, but
      this order reflects common practice when this document was
      written, and, historically, there have been servers that
      (erroneously) depended on this order.

  Note that the RFC does not define the path length of cookies without a
  path. We sort pathless cookies before cookies with the longest path,
  since pathless cookies inherit the request path (and setting a path
  that is longer than the request path is of limited use, since it cannot
  be read from the context in which it is written).

- On the server, if there are multiple cookies of the same name, only one
  of them should be encoded. RFC 6265 says:

      Servers SHOULD NOT include more than one Set-Cookie header field in
      the same response with the same cookie-name.

  Note that the RFC does not define which cookie should be set in the case
  of multiple cookies with the same name; we arbitrarily pick the last one.

Modifications:

- Changed the visibility of the 'strict' field to 'protected' in
  CookieEncoder.

- Modified ClientCookieEncoder to sort cookies in decreasing order of path
  length when in strict mode.

- Modified ServerCookieEncoder to return only the last cookie of a given
  name when in strict mode.

- Added a fast path for both strict mode in both client and server code
  for cases with only one cookie, in order avoid the overhead of sorting
  and memory allocation.

- Added unit tests for the new cases.

Result:

- Cookie generation on client and server is now more conformant to RFC 6265.
2015-11-26 21:41:58 +01:00
Dmitry Spikhalskiy
2a65ae256e [#4331] Helper methods to get charset from Content-Type header of HttpMessage
Motivation:

HttpHeaders already has specific methods for such popular and simple headers like "Host", but if I need to convert POST raw body to string I need to parse complex ContentType header in my code.

Modifications:

Add getCharset and getCharsetAsString methods to parse charset from Content-Length header.

Result:

Easy to use utility method.
2015-11-19 15:59:34 -08:00
Norman Maurer
2ecce8fa56 [maven-release-plugin] prepare for next development iteration 2015-11-10 22:59:33 +01:00
Norman Maurer
6a93f331d3 [maven-release-plugin] prepare release netty-4.1.0.Beta8 2015-11-10 22:50:57 +01:00
Scott Mitchell
0d71744d5b IllegalRefCountException in FullHttp[Request|Response].hashCode()
Motivation:
FullHttp[Request|Response].hashCode() uses a releasable object and in vulnerable to a IllegalRefCountException if that object has been released.

Modifications:
- Ensure the released object is not used.

Result:
No more IllegalRefCountException.
2015-11-09 16:47:29 -08:00
Louis Ryan
6e108cb96a Improve the performance of copying header sets when hashing and name validation are equivalent.
Motivation:
Headers and groups of headers are frequently copied and the current mechanism is slower than it needs to be.

Modifications:
Skip name validation and hash computation when they are not necessary.
Fix emergent bug in CombinedHttpHeaders identified with better testing
Fix memory leak in DefaultHttp2Headers when clearing
Added benchmarks

Result:
Faster header copying and some collateral bug fixes
2015-11-07 08:53:10 -08:00
Louis Ryan
3eb65797ed Make headers.set(self) a no-op instead of throwing. Makes it consistent with setAll
Motivation:

Makes the API contract of headers more consistent and simpler.

Modifications:

If self is passed to set then simply return

Result:

set and setAll will be consistent
2015-11-06 07:00:54 -08:00
Scott Mitchell
19658e9cd8 HTTP/2 Headers Type Updates
Motivation:
The HTTP/2 RFC (https://tools.ietf.org/html/rfc7540#section-8.1.2) indicates that header names consist of ASCII characters. We currently use ByteString to represent HTTP/2 header names. The HTTP/2 RFC (https://tools.ietf.org/html/rfc7540#section-10.3) also eludes to header values inheriting the same validity characteristics as HTTP/1.x. Using AsciiString for the value type of HTTP/2 headers would allow for re-use of predefined HTTP/1.x values, and make comparisons more intuitive. The Headers<T> interface could also be expanded to allow for easier use of header types which do not have the same Key and Value type.

Motivation:
- Change Headers<T> to Headers<K, V>
- Change Http2Headers<ByteString> to Http2Headers<CharSequence, CharSequence>
- Remove ByteString. Having AsciiString extend ByteString complicates equality comparisons when the hash code algorithm is no longer shared.

Result:
Http2Header types are more representative of the HTTP/2 RFC, and relationship between HTTP/2 header name/values more directly relates to HTTP/1.x header names/values.
2015-10-30 15:29:44 -07:00
Sverker Abrahamsson
e121c68e0f Created RTSPEncoder and RTSPDecoder which are now common for both requests and responses to be able to handle both types of messages on the same channel.
Keep RTSPRequestEncoder, RTSPRequestDecoder, RTSPResponseEncoder and
RTSPResponseDecoder for backwards compatibility but they now just extends
the generic encoder/decoder and are markes as deprecated.

Renamed the decoder test, because the decoder is now generic. Added
testcase for when ANNOUNCE request is received from server.

Created testcases for encoder.

Mark abstract base classes RTSPObjectEncoder and RTSPObjectDecoder as
deprecated, that functionality is now in RTSPEncoder and RTSPDecoder.

Added annotation in RtspHeaders to suppress warnings about deprecation, no need when
whole class is deprecated.
2015-10-27 14:01:20 +01:00
Scott Mitchell
0555b0aefd HTTP Headers Over Deprecation
Motivation:
As part of recent efforts to rectify performance and make 4.1 headers more similar to 5.0 some methods were deprecated. Some of these methods were deprecated because they used String instead of CharSequence in the signature, which may require casting at the user level. Some of the deprecated methods have no direct alternatives and were done to inform a user the method will go away in future releases.

Modifications:
- Remove the deprecated qualifier from methods where no direct replacement exists

Result:
Less warnings in user code.
2015-10-15 10:23:35 +02:00
Norman Maurer
99b11c95b4 [#4327] Ensure toString() will not throw IllegalReferenceCountException
Motivation:

As toString() is often used while logging we need to ensure this produces no exception.

Modifications:

Ensure we never throw an IllegalReferenceCountException.

Result:

Be able to log without produce exceptions.
2015-10-10 20:12:43 +02:00
Norman Maurer
2ff2806ada [maven-release-plugin] prepare for next development iteration 2015-10-02 09:03:29 +02:00
Norman Maurer
5a43de10f7 [maven-release-plugin] prepare release netty-4.1.0.Beta7 2015-10-02 09:02:58 +02:00
Norman Maurer
ca44436ce6 [#4265] Not allow to add/set DefaultHttpHeaders to itself.
Motivation:

We should prevent to add/set DefaultHttpHeaders to itself to prevent unexpected side-effects.

Modifications:

Throw IllegalArgumentException if user tries to pass the same instance to set/add.

Result:

No surprising side-effects.
2015-09-30 08:57:53 +02:00
Scott Mitchell
c7e3f6c6fd HTTP/2 defines using String instead of CharSequence
Motivation:
Http2CodecUtils has some static variables which are defined as Strings instead of CharSequence. One of these defines is used as a header name and should be AsciiString.

Modifications:
- Change the String defines in Http2CodecUtils to CharSequence

Result:
Types are more consistently using CharSequence and adding the upgrade header will require less work.
2015-09-16 14:55:33 -07:00
Scott Mitchell
2a4276e1ff SpdyHttpHeaders are not lowercase
Motivation:
According to the SPDY spec https://www.chromium.org/spdy/spdy-protocol/spdy-protocol-draft3-1#TOC-3.2.1-Request header names must be lowercase. Our predefined SPDY extension headers are not lowercase.

Modifications
- SpdyHttpHeaders should define header names in lower case

Result:
Compliant with SPDY spec, and header validation code does not detect errors for our own header names.
2015-09-16 11:32:01 -07:00
Scott Mitchell
59600f1812 HTTP/2 to HTTP/1.x headers conversion more accessible
Motivation:
Currently there is a HttpConversionUtil.addHttp2ToHttpHeaders which requires a FullHttpMessage, but this may not always be available. There is no interface that can be used with just Http2Headers and HttpHeaders.

Modifications:
- add an overload for HttpConversionUtil.addHttp2ToHttpHeaders which does not take FullHttpMessage

Result:
An overload for HttpConversionUtil.addHttp2ToHttpHeaders exists which does not require FullHttpMessage.
2015-09-16 10:02:48 -07:00
Norman Maurer
2dde3a386b [#3687] Correctly store WebSocketServerHandshaker in Channel attributes
Motivation:

As we stored the WebSocketServerHandshaker in the ChannelHandlerContext it was always null and so no close frame was send if WebSocketServerProtocolHandler was used.

Modifications:

Store WebSocketServerHAndshaker in the Channel attributes and so make it visibile between different handlers.

Result:

Correctly send close frame.
2015-09-15 09:48:04 +02:00
Scott Mitchell
f89dfb0bd5 Deprecation cleanup for HTTP headers
Motivaion:
The HttpHeaders and DefaultHttpHeaders have methods deprecated due to being removed in future releases, but no replacement method to use in the current release. The deprecation policy should not be so aggressive as to not provide any non-deprecated method to use.

Modifications:
- Remove deprecated annotations and javadocs from methods which are the best we can do in terms of matching the master's api for 4.1

Result:
There should be non-deprecated methods available for HttpHeaders in 4.1.
2015-09-09 14:30:21 -07:00
Brendt Lucas
7049d8debb Add validateHeaders and headersToLowerCase options for SPDY
Motivation:

Related to issue #4185.

HTTP has the option to disable header validation for optimisation purposes.  Introduce the same option for SPDY headers.
Also, optimise SpdyHttpEncoder by allowing the user to specify whether or not the encoder needs to convert header names to lowercase.

Modifications:

Added flags for validation and conversion.

Result:

SpdyHeader validation and conversion can be disabled.
2015-09-08 08:38:45 +02:00
Brendt Lucas
070f1470e8 [#4185] SpdyHttpEncoder fails to convert HttpResponse to SpdyFrame
Motivation:

When SpdyHttpEncoder attempts to create an SpdyHeadersFrame from a HttpResponse an IllegalArgumentException is thrown if the original HttpResponse contains a header that includes uppercase characters. The IllegalArgumentException is thrown due to the additional validation check introduced by #4047.

Previous versions of the SPDY codec would handle this by converting the HTTP header name to lowercase before adding the header to the SpdyHeadersFrame.

Modifications:

Convert the header name to lowercase before adding it to SpdyHeaders

Result:

SpdyHttpEncoder can now convert a valid HttpResponse into a valid SpdyFrame
2015-09-04 13:06:04 -07:00
Norman Maurer
34de2667c7 [maven-release-plugin] prepare for next development iteration 2015-09-02 11:45:20 +02:00
Norman Maurer
2eb444ec1d [maven-release-plugin] prepare release netty-4.1.0.Beta6 2015-09-02 11:36:11 +02:00
Norman Maurer
d9d488e477 [#2677] Remove unnessary synchronized in SpdySessionHandler
Motivation:

As all methods in the ChannelHandler are executed by the same thread there is no need to use synchronized.

Modifications:

Remove synchronized keyword.

Result:

No more unnessary synchronized in SpdySessionHandler.
2015-08-28 23:14:13 +02:00
Sivasubramaniam S
5987cddf7c Fixed a typo [testEquansIgnoreCase() --> testEqualsIgnoreCase()] 2015-08-28 20:49:57 +09:00
Scott Mitchell
5498fd12dc Fix and add comments to HttpUtil
Motivation:
The comments in HttpUtil need some love.

Modifications
- Update comments in HttpUtil

Result:
Comments are cleaner in HttpUtil.
2015-08-27 09:24:45 -07:00
Scott Mitchell
b6a4f5de9d Refactor of HttpUtil and HttpHeaderUtil
Motivation:
There currently exists http.HttpUtil, http2.HttpUtil, and http.HttpHeaderUtil. Having 2 HttpUtil methods can be confusing and the utilty methods in the http package could be consolidated.

Modifications:
- Rename http2.HttpUtil to http2.HttpConversionUtil
- Move http.HttpHeaderUtil methods into http.HttpUtil

Result:
Consolidated utilities whose names don't overlap.
Fixes https://github.com/netty/netty/issues/4120
2015-08-27 08:49:58 -07:00
Scott Mitchell
6046adef2b HttpRequestEncoder consistency with master branch
Motivation:
The HttpRequestEncoder.encodeInitialLine can now be consistent with the master branch after 85c79dbbe4

Modifications:
- Use the AsciiString and ByteBufUtil.copy methods

Result:
Consistent behavior/code between 4.1 and master branches.
2015-08-21 11:49:46 -07:00
Scott Mitchell
85c79dbbe4 HTTP to HTTP/2 tranlation errors
Motivation:
HttpUtil.toHttp2Headers is currently not translating HTTP request headers to HTTP/2 request headers correctly.  The path, scheme, and authority are tranlation process are not respecting the HTTP/2 RFC https://tools.ietf.org/html/rfc7540#section-8.1.2.3 and HTTP RFC https://tools.ietf.org/html/rfc7230#section-5.3.

Modifications:
- path, scheme, authority must be set according to rules defined in https://tools.ietf.org/html/rfc7540#section-8.1.2.3
- HTTP/1.x URIs must be handled as defined in https://tools.ietf.org/html/rfc7230#section-5.3

Result:
More correct translation from HTTP/1.x requests to HTTP/2 requests.
2015-08-21 11:33:10 -07:00
Norman Maurer
e59ae12b42 [#4079] Fix IllegalStateException when HttpContentEncoder is used and 100 Continue response is used.
Motivation:

Whe a 100 Continue response was written an IllegalStateException was produced as soon as the user wrote the following response. This regression was introduced by 41b0080fcc.

Modifications:

- Special handle 100 Continue responses
- Added unit tests

Result:

Fixed regression.
2015-08-21 08:02:43 +02:00
Norman Maurer
50b9768928 [#4095] Correctly handle Upgrade responses with special handling of Hixie 76
Motivation:

Hixie 76 needs special handling compared to other connection upgrade responses. Our detection code of non websocket responses did actually always use the special handling that only should be used for Hixie 76 responses.

Modifications:

Correctly detect connection upgrade responses which are not for websockets.

Result:

Be able to upgrade connections for other protocols then websockets.
2015-08-21 07:28:05 +02:00
Scott Mitchell
559f1b110a HttpScheme class
Motivation:
The HTTP schemes defined by https://tools.ietf.org/html/rfc7230 don't have a common representation in Netty.

Modifications:
- Add a class to represent HttpScheme

Result:
The HTTP Scheme is now defined in 1 common location.
2015-08-20 09:59:23 -07:00
Scott Mitchell
c2d5a53104 HttpUtil class for Http specific utilities
Motivation:
The HTTP specification defines specific request-targets in https://tools.ietf.org/html/rfc7230#section-5.3. Netty does not have a way to distinguish between these differnt types, and there is currently no obvious location where these types of methods would live.

Modifications:
- Add methods to distinguish request-targets as defined in https://tools.ietf.org/html/rfc7230#section-5.3

Result:
Common utitlity methods exist to inpsect request-targets.
2015-08-20 09:57:47 -07:00
Scott Mitchell
a45e844395 HttpResponseStatus reasonPhrase equals usage
Motivation:
HttpResponseStatus.reasonPhrase returns an AsciiString, but was compared using equals to a String. Other usages of the reasonPhrase also use the toString() method when not necessary.

Modifications:
- Use the contentEquals method

Result:
Correct comparison, and no toString() when not needed.
2015-08-19 13:28:44 -07:00
Scott Mitchell
a7135e8677 HttpObjectAggregator doesn't check content-length header
Motivation:
The HttpObjectAggregator always responds with a 100-continue response. It should check the Content-Length header to see if the content length is OK, and if not responds with a 417.

Modifications:
- HttpObjectAggregator checks the Content-Length header in the case of a 100-continue.

Result:
HttpObjectAggregator responds with 417 if content is known to be too big.
2015-08-17 09:26:50 -07:00
Brendt Lucas
e796c99b23 Add unit tests for HTTP and SPDY headers
Motivation:

When attempting to retrieve a SPDY header using an AsciiString key, if the header was inserted using a String based key, the lookup would fail. Similarly, the lookup would fail if the header was inserted with an AsciiString key, and retrieved using a String key. This has been fixed with the header simplification commit (1a43923aa8).

Extra unit tests have been added to protect against this issue occurring in the future.  The tests check that a header added using String or AsciiString can be retrieved using AsciiString or String respectively.

Modifications:

Added more unit tests

Result:

Protect against issue #4053 happening again.
2015-08-17 08:51:14 -07:00
Scott Mitchell
ba6ce5449e Headers Performance Boost and Interface Simplification
Motivation:
A degradation in performance has been observed from the 4.0 branch as documented in https://github.com/netty/netty/issues/3962.

Modifications:
- Simplify Headers class hierarchy.
- Restore the DefaultHeaders to be based upon DefaultHttpHeaders from 4.0.
- Make various other modifications that are causing hot spots.

Result:
Performance is now on par with 4.0.
2015-08-17 08:50:11 -07:00
Norman Maurer
fd27c403d3 [#4010] Correctly handle whitespaces in HttpPostMultipartRequestDecoder
Motivation:

Due not using a cast we insert 32 and not a whitespace into the String.

Modifications:

Correclty cast to char.

Result:

Correct handling of whitespaces.
2015-08-14 21:16:42 +02:00
Scott Mitchell
b714297a44 HttpObjectDecoder half close behavior
Motivation:
In the event an HTTP message does not include either a content-length or a transfer-encoding header [RFC 7230](https://tools.ietf.org/html/rfc7230#section-3.3.3) states the behavior must be treated differently for requests and responses. If the channel is half closed then the HttpObjectDecoder is not invoking decodeLast and thus not checking if messages should be sent up the pipeline.

Modifications:
- Add comments to clarify regular decode default case.
- Handle the ChannelInputShutdownEvent in the HttpObjectDecoder and evaluate if messages need to be generated.

Result:
Messages are generated on half closed, and comments clarify existing logic.
2015-08-05 09:04:59 -07:00
Jakob Buchgraber
6fd0a0c55f Faster and more memory efficient headers for HTTP, HTTP/2, STOMP and SPYD. Fixes #3600
Motivation:

We noticed that the headers implementation in Netty for HTTP/2 uses quite a lot of memory
and that also at least the performance of randomly accessing a header is quite poor. The main
concern however was memory usage, as profiling has shown that a DefaultHttp2Headers
not only use a lot of memory it also wastes a lot due to the underlying hashmaps having
to be resized potentially several times as new headers are being inserted.

This is tracked as issue #3600.

Modifications:
We redesigned the DefaultHeaders to simply take a Map object in its constructor and
reimplemented the class using only the Map primitives. That way the implementation
is very concise and hopefully easy to understand and it allows each concrete headers
implementation to provide its own map or to even use a different headers implementation
for processing requests and writing responses i.e. incoming headers need to provide
fast random access while outgoing headers need fast insertion and fast iteration. The
new implementation can support this with hardly any code changes. It also comes
with the advantage that if the Netty project decides to add a third party collections library
as a dependency, one can simply plug in one of those very fast and memory efficient map
implementations and get faster and smaller headers for free.

For now, we are using the JDK's TreeMap for HTTP and HTTP/2 default headers.

Result:

- Significantly fewer lines of code in the implementation. While the total commit is still
  roughly 400 lines less, the actual implementation is a lot less. I just added some more
  tests and microbenchmarks.

- Overall performance is up. The current implementation should be significantly faster
  for insertion and retrieval. However, it is slower when it comes to iteration. There is simply
  no way a TreeMap can have the same iteration performance as a linked list (as used in the
  current headers implementation). That's totally fine though, because when looking at the
  benchmark results @ejona86 pointed out that the performance of the headers is completely
  dominated by insertion, that is insertion is so significantly faster in the new implementation
  that it does make up for several times the iteration speed. You can't iterate what you haven't
  inserted. I am demonstrating that in this spreadsheet [1]. (Actually, iteration performance is
  only down for HTTP, it's significantly improved for HTTP/2).

- Memory is down. The implementation with TreeMap uses on avg ~30% less memory. It also does not
  produce any garbage while being resized. In load tests for GRPC we have seen a memory reduction
  of up to 1.2KB per RPC. I summarized the memory improvements in this spreadsheet [1]. The data
  was generated by [2] using JOL.

- While it was my original intend to only improve the memory usage for HTTP/2, it should be similarly
  improved for HTTP, SPDY and STOMP as they all share a common implementation.

[1] https://docs.google.com/spreadsheets/d/1ck3RQklyzEcCLlyJoqDXPCWRGVUuS-ArZf0etSXLVDQ/edit#gid=0
[2] https://gist.github.com/buchgr/4458a8bdb51dd58c82b4
2015-08-04 17:12:24 -07:00
Scott Mitchell
209aa28573 SPDY codec must check headers are lower case
Motivation:
The SPDY spec requires that all header names be lowercase (see https://www.chromium.org/spdy/spdy-protocol/spdy-protocol-draft3-1#TOC-3.2-HTTP-Request-Response). The SPDY codec header name validator does not enforce this requirement.

Modifications:
- SpdyCodecUtil.validateHeaderName should check for upper case characters and throw an error if any are found.

Result:
SPDY codec header validation enforces specification requirement.
2015-07-31 11:26:18 -07:00
Scott Mitchell
a7713069a1 HttpObjectDecoder performance improvements
Motivation:
The HttpObjectDecoder is on the hot code path for the http codec. There are a few hot methods which can be modified to improve performance.

Modifications:
- Modify AppendableCharSequence to provide unsafe methods which don't need to re-check bounds for every call.
- Update HttpObjectDecoder methods to take advantage of new AppendableCharSequence methods.

Result:
Peformance boost for decoding http objects.
2015-07-29 23:26:26 -07:00
James Roper
b958263853 Send full response for unsupported websocket versions
Motivation:

WebSocketServerHandshakerFactory.sendUnsupportedVersionResponse does not
send a LastHttpContent, nor does it flush, and it doesn't send a content
length.

Modifications:

Changed sendUnsupportedVersionResponse to send FullHttpResponse, to
writeAndFlush, and to set a content length of 0. Also added a test for
this method.

Result:

Upstream handlers will be able to determine the end of the response, the
response will actually get written to the client, and the client will be
able to determine the end of the response.
2015-07-17 10:56:59 +02:00
Frederic Bregier
caa1505020 Get uploaded size while upload is in progress
Proposal to fix issue #3636

Motivations:
Currently, while adding the next buffers to the decoder
(`decoder.offer()`), there is no way to access to the current HTTP
object being decoded since it can only be available currently once fully
decoded by `decoder.hasNext()`.
Some could want to know the progression on the overall transfer but also
per HTTP object.
While overall progression could be done using (if available) the global
Content-Length of the request and taking into account each HttpContent
size, the per HttpData object progression is unknown.

Modifications:
1) For HTTP object, `AbstractHttpData` has 2 protected properties named
`definedSize` and `size`, respectively the supposely final size and the
current (decoded until now) size.
This provides a new method `definedSize()` to get the current value for
`definedSize`. The `size` attribute is reachable by the `length()`
method.

Note however there are 2 different ways that currently managed the
`definedSize`:
a) `Attribute`: it is reset each time the value is less than actual
(when a buffer is added, the value is increased) since the final length
is not known (no Content-Length)
b) `FileUpload`: it is set at startup from the lengh provided

So these differences could lead in wrong perception;
a) `Attribute`: definedSize = size always
b) `FileUpload`: definedSize >= size always

Therefore the comment tries to explain clearly the different behaviors.

2) In the InterfaceHttpPostRequestDecoder (and the derived classes), I
add a new method: `decoder.currentPartialHttpData()` which will return a
`InterfaceHttpData` (if any) as the current `Attribute` or `FileUpload`
(the 2 generic types), which will allow then the programmer to check
according to the real type (instance of) the 2 methods `definedSize()`
and `length()`.

This method check if currentFileUpload or currentAttribute are null and
returns the one (only one could be not null) that is not null.

Note that if this method returns null, it might mean 2 situations:
a) the last `HttpData` (whatever attribute or file upload) is already
finished and therefore accessible through `next()`
b) there is not yet any `HttpData` in decoding (body not yet parsed for
instance)

Result:
The developper has more access and therefore control on the current
upload.
The coding from developper side could looks like in the example in
HttpUloadServerHandler.
2015-06-12 14:16:07 +02:00
Trustin Lee
b169a76d46 Fix the failing HttpObjectAggregatorTest.testInvalidConstructorUsage()
Related: 950da2eae1
2015-06-10 12:20:50 +09:00
Trustin Lee
0ca65f1373 Lazily instantiate HttpServerUpgradeHandler.UpgradeCodec
Related: #3814

Motivation:

To implement the support for an upgrade from cleartext HTTP/1.1
connection to cleartext HTTP/2 (h2c) connection, a user usually uses
HttpServerUpgradeHandler.

It does its job, but it requires a user to instantiate the UpgradeCodecs
for all supported protocols upfront. It means redundancy for the
connections that are not upgraded.

Modifications:

- Change the constructor of HttpServerUpgradeHandler
  - Accept UpgraceCodecFactory instead of UpgradeCodecs
- The default constructor of HttpServerUpgradeHandler sets the
  maxContentLength to 0 now, which shouldn't be a problem because a
  usual upgrade request is a GET.
- Update the examples accordingly

Result:

A user can instantiate Http2ServerUpgradeCodec and its related objects
(Http2Connection, Http2FrameReader/Writer, Http2FrameListener, etc) only
when necessary.
2015-06-10 12:06:27 +09:00
Norman Maurer
e9a2cac16d [#3869] Add unit test to ensure adding null header values is not allowed.
Motivation:

We need to ensure we never allow to have null values set on headers, otherwise we will see a NPE during encoding them.

Modifications:

Add unit test that shows we correctly handle null values.

Result:

Verify correct implementation.
2015-06-08 09:59:44 +02:00
Trustin Lee
0775089496 Replace SpdyOrHttpChooser and Http2OrHttpChooser with ApplicationProtocolNegotiationHandler
Motivation:

SpdyOrHttpChooser and Http2OrHttpChooser duplicate fair amount code with each other.

Modification:

- Replace SpdyOrHttpChooser and Http2OrHttpChooser with ApplicationProtocolNegotiationHandler
- Add ApplicationProtocolNames to define the known application-level protocol names

Result:

- Less code duplication
- A user can perform dynamic pipeline configuration that follows ALPN/NPN for any protocols.
2015-06-05 11:58:20 +09:00
Trustin Lee
afb46b926f Improve the API design of Http2OrHttpChooser and SpdyOrHttpChooser
Related: #3641 and #3813

Motivation:

When setting up an HTTP/1 or HTTP/2 (or SPDY) pipeline, a user usually
ends up with adding arbitrary set of handlers.

Http2OrHttpChooser and SpdyOrHttpChooser have two abstract methods
(create*Handler()) that expect a user to return a single handler, and
also have add*Handlers() methods that add the handler returned by
create*Handler() to the pipeline as well as the pre-defined set of
handlers.

The problem is, some users (read: I) don't need all of them or the
user wants to add more than one handler. For example, take a look at
io.netty.example.http2.tiles.Http2OrHttpHandler, which works around
this issue by overriding addHttp2Handlers() and making
createHttp2RequestHandler() a no-op.

Modifications:

- Replace add*Handlers() and create*Handler() with configure*()
- Rename getProtocol() to selectProtocol() to make what it does clear
- Provide the default implementation of selectProtocol()
- Remove SelectedProtocol.UNKNOWN and use null instead, because
  'UNKNOWN' is not a protocol
- Proper exception handling in the *OrHttpChooser so that the
  exception is logged and the connection is closed when failed to
  select a protocol
- Make SpdyClient example always use SSL. It was always using SSL
  anyway.
- Implement SslHandshakeCompletionEvent.toString() for debuggability
- Remove an orphaned class: JettyNpnSslSession
- Add SslHandler.applicationProtocol() to get the name of the
  application protocol
  - SSLSession.getProtocol() now returns transport-layer protocol name
    only, so that it conforms to its contract.

Result:

- *OrHttpChooser have better API.
- *OrHttpChooser handle protocol selection failure properly.
- SSLSession.getProtocol() now conforms to its contract.
- SpdyClient example works with SpdyServer example out of the box
2015-06-05 11:58:19 +09:00
Roelof Naude
757671b7cc Support empty http responses when using compression
Motivation:

Found a bug in that netty would generate a 20 byte body when returing a response
to an HTTP HEAD. the 20 bytes seems to be related to the compression footer.

RFC2616, section 9.4 states that responses to an HTTP HEAD MUST not return a message
body in the response.

Netty's own client implementation expected an empty response. The extra bytes lead to a
2nd response with an error decoder result:
java.lang.IllegalArgumentException: invalid version format: 14

Modifications:

Track the HTTP request method. When processing the response we determine if the response
is passthru unnchanged. This decision now takes into account the request method and passthru
responses related to HTTP HEAD requests.

Result:

Netty's http client works and better RFC conformance.
2015-05-26 09:55:34 +02:00
Ruslan Sennov
1a5254b597 QueryStringDecoder's javadoc fix 2015-05-21 11:45:36 +02:00
Stephane Landelle
f6299d942c Minor ClientCookieDecoder improvements
Motivation:

* Path attribute should be null, not empty String, if it's passed as "Path=".
* Only extract attribute value when the name is recognized.
* Only extract Expires attribute value String if MaxAge is undefined as it has precedence.

Modification:

Modify ClientCookieDecoder.
Add "testIgnoreEmptyPath" test in ClientCookieDecoderTest.

Result:

More idyomatic Path behavior (like Domain).
Minor performance improvement in some corner cases.
2015-05-12 11:25:28 +02:00
Frederic Bregier
61dced3dcd Proposal to fix issue #3768 (3.10)
Motivations:
When using HttpPostRequestEncoder and trying to set an attribute if a
charset is defined, currenlty implicit Charset.toStrng() is used, given
wrong format.
As in Android for UTF-16 = "com.ibm.icu4jni.charset.CharsetICU[UTF-16]".

Modifications:
Each time charset is used to be printed as its name, charset.name() is
used to get the canonical name.

Result:
Now get "UTF-16" instead.
(3.10 version)
2015-05-11 06:36:27 +02:00
Norman Maurer
f23b7b4efd [maven-release-plugin] prepare for next development iteration 2015-05-07 14:21:08 -04:00
Norman Maurer
871ce43b1f [maven-release-plugin] prepare release netty-4.1.0.Beta5 2015-05-07 14:20:38 -04:00
Stephane Landelle
97d871a755 Validate cookie name and value characters Motivation:
RFC6265 specifies which characters are allowed in a cookie name and value.

Netty is currently too lax, which can used for HttpOnly escaping.

Modification:

In ServerCookieDecoder: discard cookie key-value pairs that contain invalid characters.
In ClientCookieEncoder: throw an exception when trying to encode cookies with invalid characters.

Result:

The problem described in the motivation section is fixed.
2015-05-07 06:33:36 +02:00
Norman Maurer
62057f73d6 Fix handling of non-auto read for ByteToMessageDecoder and SslHandler
Motivation:

Our automatically handling of non-auto-read failed because it not detected the need of calling read again by itself if nothing was decoded. Beside this handling of non-auto-read never worked for SslHandler as it always triggered a read even if it decoded a message and auto-read was false.

This fixes [#3529] and [#3587].

Modifications:

- Implement handling of calling read when nothing was decoded (with non-auto-read) to ByteToMessageDecoder again
- Correctly respect non-auto-read by SslHandler

Result:

No more stales and correctly respecting of non-auto-read by SslHandler.
2015-04-20 09:11:02 +02:00
Derek Troy-West
854859ba69 Change AggregatedFullHttpMessage to contain a content ByteBuf
Motivation:

Other implementations of FullHttpMessage allow .toString to be called after the Message has been released
This brings AggregatedFullHttpMessage into line with those impls.

Modifications:

- Changed AggregatedFullHttpMessage to no longer be a sub-class of DefaultByteBufHolder
- Changes AggregatedFullHttpMessage to implement ByteBufHolder
- Hold the content buffer internally to AggregatedFullHttpMessage
- Implement the required content() and release() methods that were missing
- Do not check refcnt when accessing content() (similar to DefaultFullHttpMessage)

Result:

A released AggregatedFullHttpMessage can have .toString called without throwing an exception
2015-04-16 14:43:50 +02:00
Scott Mitchell
e36c1436b8 ByteString misses encountered during forward port
Motivation:
While forward porting https://github.com/netty/netty/pull/3579 there were a few areas that had not been previously back ported.

Modifications:
Backport the missed areas to ensure consistency.

Result:
More consistent 4.1 and master branches.
2015-04-14 17:11:09 -07:00
Scott Mitchell
9a7a85dbe5 ByteString introduced as AsciiString super class
Motivation:
The usage and code within AsciiString has exceeded the original design scope for this class. Its usage as a binary string is confusing and on the verge of violating interface assumptions in some spots.

Modifications:
- ByteString will be created as a base class to AsciiString. All of the generic byte handling processing will live in ByteString and all the special character encoding will live in AsciiString.

Results:
The AsciiString interface will be clarified. Users of AsciiString can now be clear of the limitations the class imposes while users of the ByteString class don't have to live with those limitations.
2015-04-14 16:35:17 -07:00
David Dossot
18abc6d893 Fix example in CookieDecoder Javadoc
- CookieDecoder.decode() is a static method.
2015-03-26 11:48:51 +09:00
Trustin Lee
61859eefa0 Safely encode Strings to ASCII
(Ported @luciferous's changes against 3.10)

Motivation:

The current implementation of the encoder writes each character of the
String as a single byte to the buffer, however not all characters are
mappable to a single byte.

Modifications:

If a character is outside the ASCII range, it's converted to '?'.

Result:

A safer encoder for String to ASCII, which substitutes unmappable
characters with'?'.
2015-03-18 15:55:35 +09:00
Leo Gomes
944e11c065 Add unit to maxContentLength javadoc of HttpObjectAggregator
Motivation:
Not knowing which unit is used for the maxContentLength of the HttpObjectAggregator when reading the Javadoc is annoying and can be a source of bugs.

Modifications:
Added the mention "in bytes"

Result:
Javadoc is clear.
2015-03-05 20:54:01 +01:00
Trustin Lee
6d5c38897e Fix header and initial line length counting
Related: #3445

Motivation:

HttpObjectDecoder.HeaderParser does not reset its counter (the size
field) when it failed to find the end of line.  If a header is split
into multiple fragments, the counter is increased as many times as the
number of fragments, resulting an unexpected TooLongFrameException.

Modifications:

- Add test cases that reproduces the problem
- Reset the HeaderParser.size field when no EOL is found.

Result:

One less bug
2015-03-04 17:24:22 +09:00
Norman Maurer
fce0989844 [maven-release-plugin] prepare for next development iteration 2015-03-03 02:06:47 -05:00
Norman Maurer
ca3b1bc4b7 [maven-release-plugin] prepare release netty-4.1.0.Beta4 2015-03-03 02:05:52 -05:00
Daniel Bevenius
5b1b334f01 When null origin is supported then credentials header must not be set.
Motivation:
Currently CORS can be configured to support a 'null' origin, which can
be set by a browser if a resources is loaded from the local file system.
When this is done 'Access-Control-Allow-Origin' will be set to "*" (any
origin). There is also a configuration option to allow credentials being
sent from the client (cookies, basic HTTP Authentication, client side
SSL). This is indicated by the response header
'Access-Control-Allow-Credentials' being set to true. When this is set
to true, the "*" origin is not valid as the value of
'Access-Control-Allow-Origin' and a browser will reject the request:
http://www.w3.org/TR/cors/#resource-requests

Modifications:
Updated CorsHandler's setAllowCredentials to check the origin and if it
is "*" then it will not add the 'Access-Control-Allow-Credentials'
header.

Result:
Is is possible to have a client send a 'null' origin, and at the same
time have configured the CORS to support that and to allow credentials
in that combination.
2015-02-18 16:06:30 +01:00
Daniel Bevenius
c53b8d5a85 Suggestion for supporting single header fields.
Motivation:
At the moment if you want to return a HTTP header containing multiple
values you have to set/add that header once with the values wanted. If
you used set/add with an array/iterable multiple HTTP header fields will
be returned in the response.

Note, that this is indeed a suggestion and additional work and tests
should be added. This is mainly to bring up a discussion.

Modifications:
Added a flag to specify that when multiple values exist for a single
HTTP header then add them as a comma separated string.
In addition added a method to StringUtil to help escape comma separated
value charsequences.

Result:
Allows for responses to be smaller.
2015-02-18 10:54:15 +01:00
Norman Maurer
afa9e71ed3 Allow to use WebSocketClientHandshaker and WebSocketServerHandshaker with HttpResponse / HttpRequest
Motivation:

To use WebSocketClientHandshaker / WebSocketServerHandshaker it's currently a requirement of having a HttpObjectAggregator in the ChannelPipeline. This is not a big deal when a user only wants to server WebSockets but is a limitation if the server serves WebSockets and normal HTTP traffic.

Modifications:

Allow to use WebSocketClientHandshaker and WebSocketServerHandshaker without HttpObjectAggregator in the ChannelPipeline.

Result:

More flexibility
2015-02-06 10:46:07 +01:00
scottmitch
86cb41bf95 Possible leak in AbstractDiskHttpData
Motivation:
SonarQube (clinker.netty.io/sonar) reported a resource which may not have been properly closed in all situations in AbstractDiskHttpData.

Modifications:
- Ensure file channels are closed in the presence of exceptions.
- Correct instances where local channels were created but potentially not closed.

Result:
Less leaks. Less SonarQube vulnerabilities.
2015-02-03 20:10:45 +01:00
Nitesh Kant
b19a12b952 Fixes #3362 (Possible wrong behavior in HttpResponseDecoder/HttpRequestDecoder for large header/initline/content)
Motivation:

`HttpResponseDecoder` and `HttpRequestDecoder` in the event when the max configured sizes for HTTP initial line, headers or content is breached, sends a `DefaultHttpResponse` and `DefaultHttpRequest` respectively. After this `HttpObjectDecoder` gets into `BAD_MESSAGE` state and ignores any other data received on this connection.
The combination of the above two behaviors, means that the decoded response/request are not complete (absence of sending `LastHTTPContent`). So, any code, waiting for a complete message will have to additionally check for decoder result to follow the correct semantics of HTTP.

If `HttpResponseDecoder` and `HttpRequestDecoder` creates a Full* invalid message then the request/response is a complete HTTP message and hence obeys the HTTP contract.

Modification:

Modified `HttpRequestDecoder`, `HttpResponseDecoder`, `RtspRequestDecoder` and  `RtspResponseDecoder` to return Full* messages from `createInvalidMessage()`

Result:

Fixes the wrong behavior of sending incomplete messages from these codecs
2015-02-02 17:03:32 +09:00
Trustin Lee
3c6cbd40e2 Fix an sporadic failure in ServerCookieEncoderTest
In testEncodingSingleCookieV0():

Let's assume we encoded a cookie with MaxAge=50 when currentTimeMillis
is 10999.

Because the encoder will not encode the millisecond part for Expires,
the timeMillis value of the encoded Expires field will be 60000. (If we
did not dropped the millisecond part, it would be 60999.)

Encoding a cookie will take some time, so currentTimeMillis will
increase slightly, such as to 11001.

  diff = (60000 - 11001) / 1000 = 48999 / 1000 = 48
  maxAge - diff = 50 - 48 = 2

Due to losing millisecond part twice, we end up with the precision
problem illustrated above, and thus we should increase the tolerance
from 1 second to 2 seconds.

/cc @slandelle
2015-02-02 16:19:27 +09:00
Stephane Landelle
8614b88c18 Generate Expires attribute along MaxAge one so IE can honor it, close #1466
Motivation:

Internet Explorer doesn't honor Set-Cookie header Max-Age attribute. It only honors the Expires one.

Modification:

Always generate an Expires attribute along the Max-Age one.

Result:

Internet Explorer compatible expiring cookies. Close #1466.
2015-01-25 16:55:56 +01:00
Nitesh Kant
2d24e1f27d Back port HTTP/2 codec from master to 4.1
Motivation:

HTTP/2 codec was implemented in master branch.
Since, master is not yet stable and will be some time before it gets released, backporting it to 4.1, enables people to use the codec with a stable netty version.

Modification:

The code has been copied from master branch as is, with minor modifications to suit the `ChannelHandler` API in 4.x.
Apart from that change, there are two backward incompatible API changes included, namely,

- Added an abstract method:

  `public abstract Map.Entry<CharSequence, CharSequence> forEachEntry(EntryVisitor<CharSequence> visitor)
            throws Exception;`

to `HttpHeaders` and implemented the same in `DefaultHttpHeaders` as a delegate to the internal `TextHeader` instance.

- Added a method:

`FullHttpMessage copy(ByteBuf newContent);`

in `FullHttpMessage` with the implementations copied from relevant places in the master branch.

- Added missing abstract method related to setting/adding short values to `HttpHeaders`

Result:

HTTP/2 codec can be used with netty 4.1
2015-01-23 11:06:11 -05:00
igariev
ed10513238 Fixed several issues with HttpContentDecoder
Motivation:

HttpContentDecoder had the following issues:
- For chunked content, the decoder set invalid "Content-Length" header
	with length of the first decoded chunk.
- Decoding of FullHttpRequests put both the original conent and decoded
	content into output. As result, using HttpObjectAggregator before the
	decoder lead to errors.
- Requests with "Expect: 100-continue" header were not acknowleged:
	the decoder didn't pass the header message down the handler's chain
	until content is received. If client expected "100 Continue" response,
	deadlock happened.

Modification:

- Invalid "Content-Length" header is removed; handlers down the chain can either
	rely on LastHttpContent message or ask HttpObjectAggregator to add the header.
- FullHttpRequest is split into HttpRequest and HttpContent (decoded) parts.
- Header (HttpRequest) part of request is sent down the chain as soon as it's received.

Result:

The issues are fixed, unittest is added.
2015-01-23 11:14:20 +01:00
Stephane Landelle
f0181a35ef Drop first flag that's no longer used
Motivation:

Pull request for RFC6265 support had some unused flag first in ClientCookieDecoder.

Modification:

Remove unused flag first.

Result:

Cleaner code.
2015-01-23 07:11:16 +01:00
Trustin Lee
0fc097cb20 Remove the references to the deprecated CookieDecoder 2015-01-21 22:31:06 +09:00
Trustin Lee
7d102084c1 Remove Rfc6265 prefix from cookie encoders and decoders
Motivation:

Rfc6265Client/ServerCookieEncoder is a better replacement of the old
Client/ServerCookieEncoder, and thus there's no point of keeping both.

Modifications:

- Remove the old Client/ServerCookieEncoder
- Remove the 'Rfc6265' prefix from the new cookie encoder/decoder
  classes
- Deprecate CookieDecoder

Result:

We have much better cookie encoder/decoder implementation now.
2015-01-21 22:27:50 +09:00
Trustin Lee
0ba4d32040 Fix Javadoc 2015-01-21 19:13:51 +09:00
Stephane Landelle
c298230128 RFC6265 cookies support
Motivation:

Currently Netty supports a weird implementation of RFC 2965.
First, this RFC has been deprecated by RFC 6265 and nobody on the
internet use this format.

Then, there's a confusion between client side and server side encoding
and decoding.

Typically, clients should only send name=value pairs.

This PR introduces RFC 6265 support, but keeps on supporting RFC 2965 in
the sense that old unused fields are simply ignored, and Cookie fields
won't be populated. Deprecated fields are comment, commentUrl, version,
discard and ports.

It also provides a mechanism for safe server-client-server roundtrip, as
User-Agents are not supposed to interpret cookie values but return them
as-is (e.g. if Set-Cookie contained a quoted value, it should be sent
back in the Cookie header in quoted form too).

Also, there are performance gains to be obtained by not allocating the
attribute name Strings, as we only want to match them to find which POJO
field to populate.

Modifications:

- New RFC6265ClientCookieEncoder/Decoder and
  RFC6265ServerCookieEncoder/Decoder pairs that live alongside old
  CookieEncoder/Decoder pair to not break backward compatibility.
- New Cookie.rawValue field, used for lossless server-client-server
  roundtrip.

Result:

RFC 6265 support.
Clean separation of client and server side.

Decoder performance gain:

Benchmark                     Mode  Samples        Score        Error
Units
parseOldClientDecoder        thrpt       20  2070169,228 ± 105044,970
ops/s
parseRFC6265ClientDecoder    thrpt       20  2954015,476 ± 126670,633
ops/s

This commit closes #3221 and #1406.
2015-01-21 19:07:13 +09:00
Norman Maurer
e1a53e61d0 Fix compilation error introduced by 7f907e8c2a 2015-01-16 16:47:51 +01:00
Frederic Bregier
7f907e8c2a Accept ';' '\\"' in the filename of HTTP Content-Disposition header
Motivation:
HttpPostMultipartRequestDecoder threw an ArrayIndexOutOfBoundsException
when trying to decode Content-Disposition header with filename
containing ';' or protected \\".
See issue #3326 and #3327.

Modifications:
Added splitMultipartHeaderValues method which cares about quotes, and
use it in splitMultipartHeader method, instead of StringUtils.split.

Result:
Filenames can contain semicolons and protected \\".
2015-01-16 13:54:32 +01:00
Trustin Lee
fbc0ce4784 Avoid unnecessary string conversion where possible
Motivation:

HttpResponseStaus, HttpMethod and HttpVersion have methods that return
AsciiString.  There's no need for object-to-string conversion.

Modifications:

Use codeAsText(), name(), text() instead of setInt() and setObject()

Result:

Efficiency
2015-01-11 12:48:14 +09:00
Trustin Lee
edb93a6fcc Remove static imports and inner class imports for disambiguation 2015-01-11 12:43:38 +09:00
Jeff Pinner
04dd885421 SPDY: fix support for pushed resources in SpdyHttpEncoder
Motivation:

The SpdyHttpDecoder was modified to support pushed resources that are
divided into multiple frames. The decoder accepts a pushed
SpdySynStreamFrame containing the request headers, followed by a
SpdyHeadersFrame containing the response headers.

Modifications:

This commit modifies the SpdyHttpEncoder so that it encodes pushed
resources in a format that the SpdyHttpDecoder can decode. The encoder
will accept an HttpRequest object containing the request headers,
followed by an HttpResponse object containing the response headers.

Result:

The SpdyHttpEncoder will create a SpdySynStreamFrame followed by a
SpdyHeadersFrame when sending pushed resources.
2015-01-11 12:18:11 +09:00
Frederic Bregier
85fecba770 Fix for Issue #3308 related to slice missing retain
Motivations:
It seems that slicing a buffer and using this slice to write to CTX will
decrease the initial refCnt to 0, while the original buffer is not yet
fully used (not empty).

Modifications:
As suggested in the ticket and tested, when the currentBuffer is sliced
since it will still be used later on, the currentBuffer is retained.

Add a test case for this issue.

Result:
The currentBuffer still has its correct refCnt when reaching the last
write (not sliced) of 1 and therefore will be released correctly.
The exception does no more occur.

This fix should be applied to all branches >= 4.0.
2015-01-06 21:05:33 +01:00
Trustin Lee
ecfa241768 Make sure AggregatedFullHttpMessage.trailingHeaders() return non-null
When handling an oversized message, HttpObjectAggregator does not wait
until the last chunk is received to produce the failed message, making
AggregatedFullHttpMessage.trailingHeaders() return null.
2014-12-31 19:24:53 +09:00
Trustin Lee
20d818ccec Implement toString() for all HttpMessage implementations
Related: #3019

Motivation:

We have multiple (Full)HttpRequest/Response implementations and only
some of them implements toString() properly.

Modifications:

- Add the reusable string converter for HttpMessages to HttpMessageUtil
- Implement toString() of (Full)HttpRequest/Response implementations
  properly using HttpMessageUtil

Result:

Prettier string representation is returned by HttpMessage
implementations.
2014-12-31 18:39:00 +09:00
Norman Maurer
f0e306c2fd Allow to override how headers are encoded
Motivation:

Even if its against the HTTP RFC there are situations where it may be useful to use other chars then US_ASCII in the headers. We should allow to make it possible by allow the user to override the how headers are encoded.

Modifications:

- Add encodeHeaders(...) method and so allow to override it.

Result:

It's now possible to encode headers with other charset then US_ASCII by just extend the encoder and override the encodeHeaders(...) method.
2014-12-26 15:11:39 +01:00
Jeff Beck
5ba4fdf3ba HttpObjectAggregator only set Content-Length is not already set.
Motivation:

HEAD requests will have a Content-Length set that doesn't match the
actual length. So we only want to set Content-Length header if it isn't
already set.

Modifications:

If check around setting the Content-Length.

Result:

A HEAD request will no correctly return the specified Content-Length
instead of the body length.
2014-12-25 20:44:10 +01:00
Leonardo Freitas Gomes
9ee75126eb Motivation: Sonar points out an equals comparison, where the types compared are different and don't share any common parent http://clinker.netty.io/sonar/drilldown/issues/io.netty:netty-parent:master?severity=CRITICAL#
Modifications:
Converted AsciiString into a String by calling toString() method before comparing with equals(). Also added a unit-test to show that it works.

Result:
Major violation is gone. Code is correct.
2014-12-16 07:17:31 +01:00
zcourts
bd6d0f3fd5 ensure getRawQuery is not null before appending
Motivation:

without this check then given a URI with path /path the resulting URL will be /path?null=

Modifications:

check that getRawQuery doesn't return null and only append if not

Result:

urls of the form /path will not have a null?= appended
2014-12-16 06:53:29 +01:00
Frederic Bregier
9b2fede68e Fix AbstractDiskHttpData int conversion from long
Motivations:
The chunkSize might be oversized after comparison (size being > of int
capacity) if file size is bigger than an integer.

Modifications:
Change it to long.

Result:
There is no more int oversized.

Same fix for 4.1 and Master
2014-12-08 07:17:46 +01:00
Scott Mitchell
8206cc6e14 Headers set/add/contains timeMillis methods
Motivation:
The new Headers interface contains methods to getTimeMillis but no add/set/contains variants.  These should be added for consistency.

Modifications:
- Add three new methods: addTimeMillis, setTimeMillis, containsTimeMillis to the Headers interface.
- Add a new method to the Headers.ValueConverter interface: T convertTimeMillis(long)
- Bring these new interfaces up the class hierarchy

Result:
All Headers classes have setters/getters for timeMillis.
2014-12-06 22:40:28 +09:00
Trustin Lee
948eafdce2 Add HttpStatusClass
Related: #3157

Motivation:

It should be convenient to have an easy way to classify an
HttpResponseStatus based on the first digit of the HTTP status code, as
defined in the RFC 2616:

- Information 1xx
- Success 2xx
- Redirection 3xx
- Client Error 4xx
- Server Error 5xx

Modification:

- Add HttpStatusClass
- Add HttpResponseStatus.codeClass() that returns the class of the HTTP
  status code

Result:

It's easier to determine the class of an HTTP status
2014-11-21 10:52:28 +09:00
Trustin Lee
ef11a31a06 Clean up 000d3a55c5
- Rename httpResponseStatus() to newStatus()
  - Move newStatus up so that static methods are grouped together
- Rename codeAsString to codeAsText
2014-11-20 19:12:01 +09:00
Daniel Bevenius
3ebc1ab321 Adding codeAsText to HttpResponseStatus.
Motivation:

I found myself writing AsciiString constants in my code for
response statuses and thought that perhaps it might be nice to have
them defined by Netty instead.

Modifications:

Adding codeAsText to HttpResponseStatus that returns the status code as
AsciiText.

In addition, added the 421 Misdirected Request response code from
https://tools.ietf.org/html/draft-ietf-httpbis-http2-15#section-9.1.2

This response header was renamed in draft 15:
https://tools.ietf.org/html/draft-ietf-httpbis-http2-15#appendix-A.1
But the code itself was not changed, and I thought using the latest would
be better.

Result:

It is now possible to specify a status like this:
new DefaultHttp2Headers().status(HttpResponseStatus.OK.codeAsText());
2014-11-20 19:12:01 +09:00
Idel Pivnitskiy
35db3c6710 Small performance improvements
Motivation:

Found performance issues via FindBugs and PMD.

Modifications:

- Removed unnecessary boxing/unboxing operations in DefaultTextHeaders.convertToInt(CharSequence) and DefaultTextHeaders.convertToLong(CharSequence). A boxed primitive is created from a string, just to extract the unboxed primitive value.
- Added a static modifier for DefaultHttp2Connection.ParentChangedEvent class. This class is an inner class, but does not use its embedded reference to the object which created it. This reference makes the instances of the class larger, and may keep the reference to the creator object alive longer than necessary.
- Added a static compiled Pattern to avoid compile it each time it is used when we need to replace some part of authority.
- Improved using of StringBuilders.

Result:

Performance improvements.
2014-11-20 00:10:06 -05:00
Jeff Pinner
6f80fdcac4 SPDY: add support for pushed resources in SpdyHttpDecoder
Motivation:

The SPDY/3.1 spec does not adequate describe how to push resources
from the server. This was solidified in the HTTP/2 drafts by dividing
the push into two frames, a PushPromise containing the request,
followed by a Headers frame containing the response.

Modifications:

This commit modifies the SpdyHttpDecoder to support pushed resources
that are divided into multiple frames. The decoder will accept a
pushed SpdySynStreamFrame containing the request headers, followed by
a SpdyHeadersFrame containing the response headers.

Result:

The SpdyHttpDecoder will create an HttpRequest object followed by an
HttpResponse object when receiving pushed resources.
2014-11-15 21:55:56 +01:00
Roelof Naude
1db8b83d22 Cater for empty response bodies when performing response compression.
Motivation:
RFC 2616, 4.3 Message Body states that:
All 1xx (informational), 204 (no content), and 304 (not modified) responses MUST NOT include a
message-body. All other responses do include a message-body, although it MAY be of zero length.

Modifications:
HttpContentEncoder was previously modified to cater for HTTP 100 responses. This check is enhanced to
include HTTP 204 and 304 responses.

Result:
Empty response bodies will not be modified to include the compression footer. This footer messed with Chrome's
response parsing leading to "hanging" requests.
2014-11-13 08:10:30 +01:00
Idel Pivnitskiy
fda8808210 Rewrite HttpObjectDecoder to make use of proper state machine
Motivation:

HttpObjectDecoder extended ReplayDecoder which is slightly slower then ByteToMessageDecoder.

Modifications:

- Changed super class of HttpObjectDecoder from ReplayDecoder to ByteToMessageDecoder.
- Rewrote decode() method of HttpObjectDecoder to use proper state machine.
- Changed private methods HeaderParser.parse(ByteBuf), readHeaders(ByteBuf) and readTrailingHeaders(ByteBuf), skipControlCharacters(ByteBuf) to consider available bytes.
- Set HeaderParser and LineParser as static inner classes.
- Replaced not safe actualReadableBytes() with buffer.readableBytes().

Result:

Improved performance of HttpObjectDecoder by approximately 177%.
2014-11-12 14:29:05 +01:00
Scott Mitchell
72a611a28f HTTP Content Encoder allow EmptyLastHttpContent
Motiviation:
The HttpContentEncoder does not account for a EmptyLastHttpContent being provided as input.  This is useful in situations where the client is unable to determine if the current content chunk is the last content chunk (i.e. a proxy forwarding content when transfer encoding is chunked).

Modifications:
- HttpContentEncoder should not attempt to compress empty HttpContent objects

Result:
HttpContentEncoder supports a EmptyLastHttpContent to terminate the response.
2014-11-05 23:31:27 -05:00
Trustin Lee
e82595502b Replace HttpHeaders.getDate() with getTimeMillis()
Motivation:

Headers has getTimeMillis(), not getDate()

Modification:

- Replace HttpHeaders.getDate() with getTimeMillis() so that migration
  is smoother

Result:

User code which accesses a date header is easier to migrate
2014-11-01 03:08:59 +09:00
Trustin Lee
4ce994dd4f Fix backward compatibility from the previous backport
Motivation:

The commit 50e06442c3 changed the type of
the constants in HttpHeaders.Names and HttpHeaders.Values, making 4.1
backward-incompatible with 4.0.

It also introduces newer utility classes such as HttpHeaderUtil, which
deprecates most static methods in HttpHeaders.  To ease the migration
between 4.1 and 5.0, we should deprecate all static methods that are
non-existent in 5.0, and provide proper counterpart.

Modification:

- Revert the changes in HttpHeaders.Names and Values
- Deprecate all static methods in HttpHeaders in favor of:
  - HttpHeaderUtil
  - the member methods of HttpHeaders
  - AsciiString
- Add integer and date access methods to HttpHeaders for easier future
  migration to 5.0
- Add HttpHeaderNames and HttpHeaderValues which provide standard HTTP
  constants in AsciiString
  - Deprecate HttpHeaders.Names and Values
  - Make HttpHeaderValues.WEBSOCKET lowercased because it's actually
    lowercased in all WebSocket versions but the oldest one
- Add RtspHeaderNames and RtspHeaderValues which provide standard RTSP
  constants in AsciiString
  - Deprecate RtspHeaders.*
- Do not use AsciiString.equalsIgnoreCase(CharSeq, CharSeq) if one of
  the parameters are AsciiString
- Avoid using AsciiString.toString() repetitively
  - Change the parameter type of some methods from String to
    CharSequence

Result:

Backward compatibility is recovered.  New classes and methods will make
the migration to 5.0 easier, once (Http|Rtsp)Header(Names|Values) are
ported to master.
2014-11-01 01:00:25 +09:00