Commit Graph

1240 Commits

Author SHA1 Message Date
Dmitry Minkovsky
8aeba78ecc HttpPostMultipartRequestDecoder should decode header field parameters
Motivation:

I am receiving a multipart/form_data upload from a Mailgun webhook. This webhook used to send parts like this:

--74e78d11b0214bdcbc2f86491eeb4902
Content-Disposition: form-data; name="attachment-2"; filename="attached_�айл.txt"
Content-Type: text/plain
Content-Length: 32

This is the content of the file

--74e78d11b0214bdcbc2f86491eeb4902--
but now it posts parts like this:

--74e78d11b0214bdcbc2f86491eeb4902
Content-Disposition: form-data; name="attachment-2"; filename*=utf-8''attached_%D1%84%D0%B0%D0%B9%D0%BB.txt

This is the content of the file

--74e78d11b0214bdcbc2f86491eeb4902--
This new format uses field parameter encoding described in RFC 5987. More about this encoding can be found here.

Netty does not parse this format. The result is the filename is not decoded and the part is not parsed into a FileUpload.

Modification:

Added failing test in HttpPostRequestDecoderTest.java and updated HttpPostMultipartRequestDecoder.java
Refactored to please Netkins
Result:

Fixes:

HttpPostMultipartRequestDecoder identifies the RFC 5987 format and parses it.
Previous functionality is retained.
2017-10-24 19:30:59 +02:00
Ned Twigg
dcbbae7f90 Added QueryStringDecoder.rawPath() and rawQuery()
Motivation:

Before this commit, it is impossible to access the path component of the
URI before it has been decoded.  This makes it impossible to distinguish
between the following URIs:

/user/title?key=value
/user%2Ftitle?key=value

The user could already access the raw uri value, but they had to calculate
pathEndIdx themselves, even though it might already be cached inside
QueryStringDecoder.

Result:

The user can easily and efficiently access the undecoded path and query.
2017-10-24 09:32:06 +02:00
Nikolay Fedorovskikh
dc98eae5a5 Correct filling an origin header for WS client
Motivation:
An `origin`/`sec-websocket-origin` header value in websocket client is filling incorrect in some cases:
- Hostname is not converting to lower-case as prescribed by RFC 6354 (see [1]).
- Selecting a `http` scheme when source URI has `wss`/`https` scheme and non-standard port.

Modifications:
- Convert uri-host to lower-case.
- Use a `https` scheme if source URI scheme is `wss`/`https`, or if source scheme is null and port == 443.

Result:
Correct filling an `origin` header for WS client.

[1] https://tools.ietf.org/html/rfc6454#section-4
2017-10-23 11:38:34 +02:00
Idel Pivnitskiy
50a067a8f7 Make methods 'static' where it possible
Motivation:

Even if it's a super micro-optimization (most JVM could optimize such
 cases in runtime), in theory (and according to some perf tests) it
 may help a bit. It also makes a code more clear and allows you to
 access such methods in the test scope directly, without instance of
 the class.

Modifications:

Add 'static' modifier for all methods, where it possible. Mostly in
test scope.

Result:

Cleaner code with proper 'static' modifiers.
2017-10-21 14:59:26 +02:00
Cory Benfield
1b0a545921 Do not send Content-Length: 0 on 101 responses.
Motivation:

During code read of the Netty codebase I noticed that the Netty
HttpServerUpgradeHandler unconditionally sets a Content-Length: 0
header on 101 Switching Protocols responses. This explicitly
contravenes RFC 7230 Section 3.3.2 (Content-Length), which notes
that:

    A server MUST NOT send a Content-Length header field in any
    response with a status code of 1xx (Informational) or 204
    (No Content).

While it is unlikely that any client will ever be confused by
this behaviour, there is no reason to contravene this part of the
specification.

Modifications:

Removed the line of code setting the header field and changed the
only test that expected it to be there.

Result:

When performing the server portion of HTTP upgrade, the 101
Switching Protocols response will no longer contain a
Content-Length: 0 header field.
2017-10-21 14:36:19 +02:00
Stephane Landelle
940f15f0d2 Use predefined HttpResponseStatus constant instead of hardcoded 101 code
Motivation:

#7269 removed an unnecessary instanciation for verifying WebSocket
handshake status code.
But it uses a hardcoded status code value for 101 instead of using the
intended `HttpResponseStatus#SWITCHING_PROTOCOLS` constant.

Modidication:

Compare actual `HttpResponseStatus` against predefined constant. Note
that `HttpResponseStatus#equals` is implemented in respect with the RFC
(only honor code, not text) so it’s intended to be used this way.

Result:

Cleaner code, use intended constant instead of hard coded value.
2017-10-02 18:58:12 +02:00
Matt Belisle
ad548a6a0a Fixing CorsHandler response Content-Length
Motivation:

https://github.com/netty/netty/issues/7253

Modifications:

Adding `Content-Length: 0` to `CorsHandler.forbidden()` and `CorsHandler.handlePreflight()`

Result:

Contexts that are terminated by the CorsHandler will always include a Content-Length header
2017-10-02 08:20:15 +02:00
Nikolay Fedorovskikh
78786e6052 Remove unnecessary instantiation of HttpResponseStatus
Motivation:
- In the `HttpResponseStatus#equals` checks only status code. No need to create new instance of `HttpResponseStatus` for comparison with response status.
- The RFC says: `the HTTP version and reason phrase aren't important` [1].

Modifications:
Use comparison by status code without creating new `HttpResponseStatus`.

Result:
Less allocations, more clear code.

[1] https://tools.ietf.org/html/draft-ietf-hybi-thewebsocketprotocol-00
2017-10-02 08:08:34 +02:00
Norman Maurer
625a7426cd [maven-release-plugin] prepare for next development iteration 2017-09-25 06:12:32 +02:00
Norman Maurer
f57d8f00e1 [maven-release-plugin] prepare release netty-4.1.16.Final 2017-09-25 06:12:16 +02:00
Norman Maurer
70c5c48eab Correctly not write any body when 1xx, 204 or 304 is used as response status code.
Motivation:

We need to ensure we not write any body when a response with status code of 1xx, 204 or 304 is used as stated in rfc:
https://tools.ietf.org/html/rfc7230#section-3.3.3

Modifications:

- Correctly handle status codes
- Add unit tests

Result:

Correctly handle responses with 1xx, 204, 304 status codes.
2017-09-20 07:41:13 -07:00
Scott Mitchell
9bd6d8129e HttpObjectEncoder buffer size estimation
Motivation:
HttpObjectEncoder allocates a new buffer when encoding the initial line and headers, and also allocates a buffer when encoding the trailers. The allocation always uses the default size of 256. This may lead to consistent under allocation and require a few resize/copy operations which can cause GC/memory pressure.

Modifications:
- Introduce a weighted average which tracks the historical size of encoded data and uses this as an estimate for future buffer allocations

Result:
Better approximation of buffer sizes.
2017-08-31 01:40:53 -07:00
Stephane Landelle
d9d0e633dc Fix ServerCookieEncoder javadoc, close #7115
Motivation:

ServerCookieEncoder’s javadoc contains some invalid copy-pasting from
ClientCookieEncoder.

Modifications:
* As per RFC6265, multiple cookies are sent as separate Set-Cookie
response headers.
* Fix code sample

Result:

Proper javadoc
2017-08-28 20:21:57 +02:00
Norman Maurer
b967805f32 [maven-release-plugin] prepare for next development iteration 2017-08-24 15:38:22 +02:00
Norman Maurer
da8e010a42 [maven-release-plugin] prepare release netty-4.1.15.Final 2017-08-24 15:37:59 +02:00
Norman Maurer
123e07ca80 Revert "Only call ctx.fireChannelReadComplete() if ByteToMessageDecoder decoded at least one message."
This reverts commit d63bb4811e as this not covered correctly all cases and so could lead to missing fireChannelReadComplete() calls. We will re-evalute d63bb4811e and resbumit a pr once we are sure all is handled correctly
2017-08-18 09:06:37 +02:00
Nikolay Fedorovskikh
4875a2aad4 Immediate caching the strings wrapped to AsciiString
Motivation:
The `AsciiString#toString` method calculate string value and cache it into field. If an `AsciiString` created from the `String` value, we can avoid rebuilding strings if we cache them immediately when creating `AsciiString`. It would be useful for constants strings, which already stored in the JVMs string table, or in cases where an unavoidable `#toString `method call is assumed.

Modifications:
- Add new static method `AsciiString#cache(String)` which save string value into cache field.
- Apply a "benign" data race in the `#hashCode` and `#toString` methods.

Result:
Less memory usage in some `AsciiString` use cases.
2017-08-15 06:22:14 +02:00
Daniel Schobel
52e19d5c63 Strip http 'expect' headers when expectation response is produced
Motivation:

HttpObjectAggregator differs from HttpServerExpectContinueHandler's handling
of expect headers by not stripping the 'expect' header when a response
is generated.

Modifications:

HttpObjectAggregator now removes the 'expect' header in cases where it generates
a response.

Result:

Consistent and correct behavior between HttpObjectAggregator and HttpServerExpectContinueHandler.
2017-08-12 17:18:27 +02:00
Norman Maurer
918fde82f8 Add testcases to prove HttpResponseEncoder correctly handles empty content
Motivation:

Issue #6695 states that there is an issue when writing empty content via HttpResponseEncoder.

Modifications:

Add two test-cases.

Result:

Verified that all works as expected.
2017-08-07 07:25:33 +02:00
Norman Maurer
d63bb4811e Only call ctx.fireChannelReadComplete() if ByteToMessageDecoder decoded at least one message.
Motivation:

Its wasteful and also confusing that channelReadComplete() is called even if there was no message forwarded to the next handler.

Modifications:

- Only call ctx.fireChannelReadComplete() if at least one message was decoded
- Add unit test

Result:

Less confusing behavior. Fixes [#4312].
2017-08-04 10:54:56 +02:00
Norman Maurer
52f384b37f [maven-release-plugin] prepare for next development iteration 2017-08-02 12:55:10 +00:00
Norman Maurer
8cc1071881 [maven-release-plugin] prepare release netty-4.1.14.Final 2017-08-02 12:54:51 +00:00
Norman Maurer
d141ba11bf Fix flacky multipart test introduced by 08748344d8.
Motivation:

08748344d8 introduced two new tests which did not take into account that the multipart delimiter can be between 2 and 16 bytes long.

Modifications:

Take the multipart delimiter length into account.

Result:

Fixes [#7001]
2017-07-21 14:28:32 +02:00
Scott Mitchell
a91df58ca1 HTTP/2 enforce HTTP message flow
Motivation:
codec-http2 currently does not strictly enforce the HTTP/1.x semantics with respect to the number of headers defined in RFC 7540 Section 8.1 [1]. We currently don't validate the number of headers nor do we validate that the trailing headers should indicate EOS.

[1] https://tools.ietf.org/html/rfc7540#section-8.1

Modifications:
- DefaultHttp2ConnectionDecoder should only allow decoding of a single headers and a single trailers
- DefaultHttp2ConnectionEncoder should only allow encoding of a single headers and optionally a single trailers

Result:
Constraints of RFC 7540 restricting the number of headers/trailers is enforced.
2017-07-19 13:37:23 -07:00
Violeta Georgieva
08748344d8 Fix NPEs in HttpPostRequestEncoder#nextChunk
Motivation:

HttpPostRequestEncoder maintains an internal buffer that holds the
current encoded data. There are use cases when this internal buffer
becomes null, the next chunk processing implementation should take
this into consideration.

Modifications:

- When preparing the last chunk if currentBuffer is null, mark
isLastChunkSent as true and send LastHttpContent.EMPTY_LAST_CONTENT
- When calculating the remaining size take into consideration that the
currentBuffer might be null
- Tests are based on those provided in the issue by @nebhale and @bfiorini

Result:

Fixes #5478
2017-07-19 14:35:51 +02:00
Nikolay Fedorovskikh
3e9f617504 Deduplicate and simplify code in HttpPostMultipartRequestDecoder
Motivation:

- A `HttpPostMultipartRequestDecoder` contains two pairs of the same methods: `readFileUploadByteMultipartStandard`+`readFileUploadByteMultipart` and `loadFieldMultipartStandard`+`loadFieldMultipart`.
- These methods use `NotEnoughDataDecoderException` to detecting not last data chunk (exception handling is very expensive).
- These methods can be greatly simplified.
- Methods `loadFieldMultipart` and `loadFieldMultipartStandard` has an unnecessary catching for the `IndexOutOfBoundsException`.

Modifications:

- Remove duplicate methods.
- Replace handling `NotEnoughDataDecoderException` by the return of a boolean result.
- Simplify code.

Result:

The code is cleaner and easier to support. Less exception handling logic.
2017-07-18 13:25:12 +02:00
Norman Maurer
f1e14d0cb2 Only add / to uri if really needed.
Motivation:

We not need to include the start index in the check. See https://github.com/netty/netty/pull/6924#discussion_r125263918

Modifications:

Change <= to <

Result:

More correct code.
2017-07-18 09:24:19 +02:00
Nikolay Fedorovskikh
df568c739e Use ByteBuf#writeShort/writeMedium instead of writeBytes
Motivation:

1. Some encoders used a `ByteBuf#writeBytes` to write short constant byte array (2-3 bytes). This can be replaced with more faster `ByteBuf#writeShort` or `ByteBuf#writeMedium` which do not access the memory.
2. Two chained calls of the `ByteBuf#setByte` with constants can be replaced with one `ByteBuf#setShort` to reduce index checks.
3. The signature of method `HttpHeadersEncoder#encoderHeader` has an unnecessary `throws`.

Modifications:

1. Use `ByteBuf#writeShort` or `ByteBuf#writeMedium` instead of `ByteBuf#writeBytes` for the constants.
2. Use `ByteBuf#setShort` instead of chained call of the `ByteBuf#setByte` with constants.
3. Remove an unnecessary `throws` from `HttpHeadersEncoder#encoderHeader`.

Result:

A bit faster writes constants into buffers.
2017-07-10 14:37:41 +02:00
Norman Maurer
2a376eeb1b [maven-release-plugin] prepare for next development iteration 2017-07-06 13:24:06 +02:00
Norman Maurer
c7f8168324 [maven-release-plugin] prepare release netty-4.1.13.Final 2017-07-06 13:23:51 +02:00
Dmitriy Dumanskiy
dd69a813d4 Performance improvement for HttpRequestEncoder. Insert char into the string optimized.
Motivation:

Right now HttpRequestEncoder does insertion of slash for url like http://localhost?pararm=1 before the question mark. It is done not effectively.

Modification:

Code:

new StringBuilder(len + 1)
                .append(uri, 0, index)
                .append(SLASH)
                .append(uri, index, len)
                .toString();
Replaced with:

new StringBuilder(uri)
                .insert(index, SLASH)
                .toString();
Result:

Faster HttpRequestEncoder. Additional small test. Attached benchmark in PR.

Benchmark                                      Mode  Cnt        Score        Error  Units
HttpRequestEncoderInsertBenchmark.newEncoder  thrpt   40  3704843.303 ±  98950.919  ops/s
HttpRequestEncoderInsertBenchmark.oldEncoder  thrpt   40  3284236.960 ± 134433.217  ops/s
2017-06-27 10:53:43 +02:00
Norman Maurer
3b5dd4e9dc Change type of X_FRAME_OPTIONS and CONTENT_SECURITY_POLICY to AsciiString as a followup to cc1a209185 2017-06-27 08:37:46 +02:00
Nikolay Fedorovskikh
ba3616da3e Apply appropriate methods for writing CharSequence into ByteBuf
Motivation:

1. `ByteBuf` contains methods to writing `CharSequence` which optimized for UTF-8 and ASCII encodings. We can also apply optimization for ISO-8859-1.
2. In many places appropriate methods are not used.

Modifications:

1. Apply optimization for ISO-8859-1 encoding in the `ByteBuf#setCharSequence` realizations.
2. Apply appropriate methods for writing `CharSequences` into buffers.

Result:

Reduce overhead from string-to-bytes conversion.
2017-06-27 07:58:39 +02:00
Norman Maurer
cc1a209185 Add content-security-policy and x-frame-options to HttpHeaderNames
Motivation:

These headers can be used to prevent clickjacking.

Modifications:

Add static fields for content-security-policy and x-frame-options

Result:

Expose general useful names
2017-06-27 07:55:25 +02:00
Nikolay Fedorovskikh
d672a5a483 Optimizations in QueryStringEncoder
Motivation:

A life cycle of QueryStringEncoder is simple: create, append params, convert to String. Current realization collect params in the list, and calculate an URI string in `toString` method. We can simplify this: don't store params to the list, and immediately append parameters to the `StringBuilder`.

Modifications:

- Remove list for params and remove a tuple class `Param`.
- Use one common `StringBuilder` and append parameters into it.
- Resolve `TODO` in the `encodeParam` method.

Result:

Less allocations (no `ArrayList`, no `Param` tuples). Second `toString` call is faster.
2017-06-23 14:03:32 -07:00
Nikolay Fedorovskikh
01eb428b39 Move methods for decode hex dump into StringUtil
Motivation:

PR #6811 introduced a public utility methods to decode hex dump and its parts, but they are not visible from netty-common.

Modifications:

1. Move the `decodeHexByte`, `decodeHexDump` and `decodeHexNibble` methods into `StringUtils`.
2. Apply these methods where applicable.
3. Remove similar methods from other locations (e.g. `HpackHex` test class).

Result:

Less code duplication.
2017-06-23 18:52:42 +02:00
Jason Tedor
9ad74e72e6 Remove content-length header leniency
Motivation:

If the content-length does not parse as a number, leniency causes this
to instead be parsed as the default value. This leads to bodies being
silently ignored on requests which can be incredibly dangerous. Instead,
if the content-length header is invalid, an exception should be thrown
for upstream handling.

Modifications:

This commit removes the leniency in parsing the content-length header by
allowing a number format exception, if thrown, to escape from the method
rather than falling back to the default value.

Result:

In invalid content-length header will not be silently ignored.
2017-06-22 09:20:11 -07:00
Dmitriy Dumanskiy
81f9434ec7 Added test for multi header, HttpObjectDecoder performance improvement for multi header, removed empty else block.
Motivation:

For multi-line headers HttpObjectDecoder uses StringBuilder.append(a).append(b) pattern that could be easily replaced with regular a + b. Also oparations with a and b moved out from concat operation to make it friendly for StringOptimizeConcat optimization and thus - faster.

Modification:

StringBuilder.append(a).append(b) reaplced with a + b. Operations with a and b moved out from concat oparation.

Result:
Code simpler to read and faster.
2017-06-20 07:11:02 +02:00
Nikolay Fedorovskikh
b8a418d53d Remove redundant code block in HttpPostRequestEncoder and make some cleanup
Motivation:

The class `HttpPostRequestEncoder` has minor issues:
- The `encodeNextChunkMultipart()` method contains two identical blocks of code with a difference only in the cast interfaces: `Attribute` vs `HttpData`. Because the `Attribute` is extended by `HttpData`, the block with the `Attribute` can be safely deleted.
- The `getNewMultipartDelimiter()` method contains a redundant `toLowerCase()`.
- The `addBodyFileUploads()` method throws `NPE` instead of `IllegalArgumentException`.

Modifications:

- Remove duplicated code block from `encodeNextChunkMultipart()`.
- Remove redundant `toLowerCase()` from `getNewMultipartDelimiter()`.
- Replace `NPE` with `IllegalArgumentException` in `addBodyFileUploads()`.
- Use `ObjectUtil#checkNotNull` where possible.

Result:

More correct and clean code.
2017-06-14 06:49:20 +02:00
Norman Maurer
fd67a2354d [maven-release-plugin] prepare for next development iteration 2017-06-08 21:06:24 +02:00
Norman Maurer
3acd5c68ea [maven-release-plugin] prepare release netty-4.1.12.Final 2017-06-08 21:06:01 +02:00
Renjie Sun
629b83e0a5 Move QueryStringDecoder.decodeHexByte into ByteBufUtil
Motivations:
1. There are duplicated implementations of decoding hex strings. #6797
2. ByteBufUtil.HexUtil.decodeHexDump does not handle substring start
index properly and does not decode hex byte rigorously.

Modifications:
1. Function decodeHexByte is moved from QueryStringDecoder into ByteBufUtil.
2. ByteBufUtil.HexUtil.decodeHexDump is changed to use decodeHexByte.
3. Tests are Updated accordingly.

Result:
Fixed #6797 and made hex decoding functions more robust.
2017-06-07 09:27:36 -07:00
Nikolay Fedorovskikh
b03b0f22d1 Removing a SeekAheadNoBackArrayException to avoid exception handling
Motivation:

A `SeekAheadNoBackArrayException` used as check for `ByteBuf#hasArray`. The catch of exceptions carries a large overhead on stack trace filling, and this should be avoided.

Modifications:

- Remove the class `SeekAheadNoBackArrayException` and replace its usage with `if` statements.
- Use methods from `ObjectUtils` for better readability.
- Make private methods static where it make sense.
- Remove unused private methods.

Result:

Less of exception handling logic, better performance.
2017-06-06 19:30:04 -07:00
Bryce Anderson
9fa3e556f3 Adjust Content-Length header when encoding Full Responses
Motivation:
If a full HttpResponse with a Content-Length header is encoded by the HttpContentEncoder subtypes the Content-Length header is removed and the message is set to Transfer-Encoder: chunked. This is an unnecessary loss of information about the message content.

Modifications:
- If a full HttpResponse has a Content-Length header, the header is adjusted after encoding.

Result:
Complete messages continue to have the Content-Length header after encoding.
2017-06-06 22:07:29 +02:00
Nikolay Fedorovskikh
270e9d66c5 Fixes in QueryStringDecoder
Motivation:

QueryStringDecoder has several problems:
- doesn't decode correctly path part with `+` (plus) sign in it,
- doesn't cut a `fragment` (after `#`) from query string (see RFC 3986),
- doesn't work correctly with encoding,
- treat `%%` as a percent character escaping (it's don't described in RFC).

Modifications:

- leave `+` chars in a `path` part of uri string,
- ignore `fragment` part (after `#`),
- correctly work with encoding.
- don't treat `%%` as escaping for the `%`.

Result:

Fixed issues from #6745.
2017-05-31 13:54:56 -07:00
Julien Viet
eee0ec3902 HttpObjectEncoder#isContentAlwaysEmpty cannot be overridden by subclasses
Motivation:

Allow subclasses of HttpObjectEncoder other than HttpServerCodec to override the isContentAlwaysEmpty method

Modification:

Change the method visibility from package private to protected

Result:

Fixes #6761
2017-05-31 07:36:12 +02:00
Nolan O'Brien
d56a7560ea Netty force encodes already encoded responses
Motivation:

Fix the regression recently introduced that causes already encoded responses to be encoded again as gzip

Modification:

instead of just looking for IDENTITY, anything set for Content-Encoding should be respected and left as-is

added unit tests to capture this use case

Result:

Fixes #6784
2017-05-27 08:26:14 +02:00
Roger Kapsi
b419bd152a Handle the possibility of HTTP/0.9 with a better error message
Motivation

RFC 1945 (see section 3.1) says that request lines may not have a version in which case the request is assumed to be HTTP/0.9. We don't necessarily want to support that but the existing Exception should indicate the possibility of the request being HTTP/0.9 and give the user a chance to track it down.

Modifications

Indicate in the Exception's message that the request is possibly HTTP/0.9.

Result

Fixes #6739
2017-05-26 07:51:49 +02:00
Cédric L
61efd81952 Add HTTP Status code 308 in HttpResponseStatus.java
Motivation:

The status 308 is defined by RFC7538.
This RFC has currently the state Proposed Standard since 2 years, but the status code is already handle by all browsers (Chrome, Firefox, Edge, Safari, …).
To let developer handles easily this status code, it is added into this list.

Modifications:

Added this status code in the list of all status codes and changed the valudOf() method

Result:

Status code 308 included
2017-05-24 16:26:32 +02:00
Anuraag Agrawal
ba5d1880bc Make HttpHeadersEncoder.encodeHeader package private to match class visibility. 2017-05-19 11:17:31 +02:00
Norman Maurer
0db2901f4d [maven-release-plugin] prepare for next development iteration 2017-05-11 16:00:55 +02:00
Norman Maurer
f7a19d330c [maven-release-plugin] prepare release netty-4.1.11.Final 2017-05-11 16:00:16 +02:00
Nitesh Kant
a093b89bfe Allow HTTP decoding post CONNECT in HttpClientCode
__Motivation__

`HttpClientCodec` skips HTTP decoding on the connection after a successful HTTP CONNECT response is received.
 This behavior follows the spec for a client but pragmatically, if one creates a client to use a proxy transparently, the codec becomes useless after HTTP CONNECT.
 Ideally, one should be able to configure whether HTTP CONNECT should result in pass-through or not. This will enable client writers to continue using HTTP decoding even after HTTP CONNECT.

 __Modification__

 Added overloaded constructors to accept `parseHttpPostConnect`. If this parameter is `true` then the codec continues decoding even after a successful HTTP CONNECT.

 Also fixed a bug in the codec that was incrementing request count post HTTP CONNECT but not decrementing it on response. Now, the request count is only incremented if the codec is not `done`.

 __Result__

 Easier usage by HTTP client writers who wants to connect to a proxy but still decode HTTP for their users for subsequent requests.
2017-05-11 16:48:16 +02:00
Nikolay Fedorovskikh
94e9448ae3 Simplify JUnit assertions
Motivation:

Some JUnit assert calls can be replaced by simpler.

Modifications:

Replacement with a more suitable methods.

Result:

More informative JUnit reports.
2017-05-09 20:19:10 +02:00
Dmitriy Dumanskiy
174f4ea005 HttpServerKeepAliveHandler doesn't correctly handle VoidChannelPromise
Motivation:

HttpServerKeepAliveHandler throws unexpected error when I do ctx.writeAndFlush(msg, ctx.voidPromise()); where msg is with header "Connection:close".

Modification:

HttpServerKeepAliveHandler does promise.unvoid() before adding close listener.

Result:

No error for VoidChannelPromise with HttpServerKeepAliveHandler. Fixes [#6698].
2017-05-04 14:08:18 -07:00
Dmitry Spikhalskiy
464ae9fb7a Expose CharSequence version of HttpUtil#getMimeType and HttpUtil#getCharset
Motivation:

It would be more flexible to make getCharset and getMimeType code usable not only for HttpMessage entity but just for any CharSequence. This will improve usability in general purpose code and will help to avoid multiple fetching of ContentType header from a message. It could be done in an external code once and CharSequence method versions could be applied.

Modification:
Expose HttpUtil#getMimeType, HttpUtil#getCharsetAsString, HttpUtil#getCharset versions which works with CharSequence. New methods are reused in the old ones which work with HttpMessage entity.

Result:

More flexible methods set with a good code reusing.
2017-05-03 14:14:39 -07:00
Michael K. Werle
e70fbe316d Fire exceptionCaught before exception-caused close for WebSockets.
Motivation:

WebSocket decoding throws exceptions on failure that should cause the
pipline to close.  These are currently ignored in the
`WebSocketProtocolHandler` and `WebSocketServerProtocolHandler`.  In
particular, this means that messages exceding the max message size will
cause the channel to close with no reported failure.

Modifications:

Re-fire the event just before closing the socket to allow it to be
handled appropriately.

Result:

Closes [#3063].
2017-05-03 13:27:11 -07:00
Norman Maurer
a3e496a521 Not try to compresses HttpMessage if IDENTITY header value is set.
Motivation:

If Content-Encoding: IDENTITY is used we should not try to compress the http message but just let it pass-through.

Modifications:

Remove "!"

Result:

Fixes [#6689]
2017-05-03 10:55:13 -07:00
Norman Maurer
6915ec3bb9 [maven-release-plugin] prepare for next development iteration 2017-04-29 14:10:00 +02:00
Norman Maurer
f30f242fee [maven-release-plugin] prepare release netty-4.1.10.Final 2017-04-29 14:09:32 +02:00
Daniel Schobel
b1cb059540 Motivation:
It is generally useful to have origin http servers respond to
"expect: continue-100" as soon as possible but applications without a
HttpObjectAggregator in their pipelines must use boiler plate to do so.

Modifications:

Introduce the HttpServerExpectContinueHandler handler to make it easier.

Result:

Less boiler plate for http application authors.
2017-04-27 16:20:29 -07:00
Aron Wieck
ffd6911586 Use constant string instead of user provided file name for DiskFileUpload temp file names.
Motivation:

DiskFileUpload creates temporary files for storing user uploads containing the user provided file name as part of the temporary file name. While most security problems are prevented by using "new File(userFileName).getName()" a small risk for bugs or security issues remains.

Modifications:

Use a constant string as file name and rely on the callers use of File.createTemp to ensure unique disk file names.

Result:

A slight security improvement at the cost of a little more obfuscated temp file names.
2017-04-27 16:02:41 -07:00
Dmitriy Dumanskiy
a45f9d7939 Improvement : allocate less object during multipart form parsing. 2017-04-25 14:06:10 +02:00
Nikolay Fedorovskikh
0692bf1b6a fix the typos 2017-04-20 04:56:09 +02:00
Norman Maurer
fb113dce3a HttpPostRequestEncoder may return a slice which is not retained.
Motivation:

We miss to retain a slice before return it to the user and so an reference count error may accour later on.

Modifications:

Use readRetainedSlice(...) and so ensure we retain the buffer before hand it of to the user.

Result:

Fixes [#6626].
2017-04-19 11:40:38 +02:00
Brendt Lucas
dcd322dda2 Fix regression in QueryStringEncoder
Motivation:

Commit #d675febf07d14d4dff82471829f974369705655a introduced a regression in QueryStringEncoder, resulting in whitespace being converted into a literal `+` sign instead of `%20`.

Modification:

Modify `encodeComponent` to pattern match and replace on the result of the call to `URLEncoder#encode`

Result:

Fixes regression
2017-04-17 19:32:50 +02:00
Scott Mitchell
b041f1a7a9 HttpServerKeepAliveHandler 204 response with no Content-Length should keepalive
Motivation:
https://tools.ietf.org/html/rfc7230#section-3.3.2 states that a 204 response MUST NOT include a Content-Length header. If the HTTP version permits keep alive these responses should be treated as keeping the connection alive even if there is no Content-Length header.

Modifications:
- HttpServerKeepAliveHandler#isSelfDefinedMessageLength should account for 204 respones

Result:
Fixes https://github.com/netty/netty/issues/6549.
2017-03-31 17:41:10 -07:00
cdn
71b338ce17 Non-latin character broken on HttpHeader by HttpObjectDecoder.
Motivation:

Currently netty is receiving HTTP request by ByteBuf and store it as "CharSequence" on HttpObjectDecoder. During this operation, all character on ByteBuf is moving to char[] without breaking encoding.
But in process() function, type casting from byte to char does not consider msb (sign-bit). So the value over 127 can be casted wrong value. (ex : 0xec in byte -> 0xffec in char). This is type casting bug.

Modification:

Fix type casting

Result:

Non-latin characters work.
2017-03-28 11:58:30 +02:00
David Dossot
9c1a191696 Trim optional white space in CombinedHttpHeaders values
Motivation:

The updated HTTP/1.x RFC allows for header values to be CSV and separated by OWS [1]. CombinedHttpHeaders should remove this OWS on insertion.

[1] https://tools.ietf.org/html/rfc7230#section-7

Modification:

CombinedHttpHeaders doesn't account for the OWS and returns it back to the user as part of the value.

Result:

Fixes #6452
2017-03-19 08:17:29 -07:00
Norman Maurer
2b8c8e0805 [maven-release-plugin] prepare for next development iteration 2017-03-10 07:46:17 +01:00
Norman Maurer
1db58ea980 [maven-release-plugin] prepare release netty-4.1.9.Final 2017-03-10 07:45:28 +01:00
Norman Maurer
e12f504ac1 Remove deprecated usage of Mockito methods
Motivation:

We used some deprecated Mockito methods.

Modifications:

- Replace deprecated method usage
- Some cleanup

Result:

No more usage of deprecated Mockito methods. Fixes [#6482].
2017-03-09 20:59:54 +01:00
Nikolay Fedorovskikh
2993760e92 Fix misordered 'assertEquals' arguments in tests
Motivation:

Wrong argument order in some 'assertEquals' applying.

Modifications:

Flip compared arguments.

Result:

Correct `assertEquals` usage.
2017-03-08 22:48:37 -08:00
Nikolay Fedorovskikh
f49bf4b201 Convert fields to the local variable when possible
Motivation:

Some classes have fields which can be local.

Modifications:

Convert fields to the local variable when possible.

Result:

Clean up. More chances for young generation or scalar replacement.
2017-03-08 17:09:17 -08:00
Nikolay Fedorovskikh
d702c47cab TODO for the method with typo in name 2017-03-08 09:28:33 -08:00
Norman Maurer
1392bc351f Correctly build socketaddress string, followup of 8b2badf44f 2017-03-01 20:05:20 +01:00
Norman Maurer
0514b0c61b Only add port to HOST header value if needed
Motivation:

We only need to add the port to the HOST header value if its not a standard port.

Modifications:

- Only add port if needed.
- Fix parsing of ipv6 address which is enclosed by [].

Result:

Fixes [#6426].
2017-03-01 19:08:19 +01:00
Dmitriy Dumanskiy
d1b0225724 Improvement : WebSocketServerProtocolHandshakeHandler.getWebSocketLocation now applies concat optimization and WebSocket path check moved to separated method.
Motivation:

Make code easier to read, make WebSocketServerProtocolHandshakeHandler.getWebSocketLocation method faster.

Modification:

WebSocket path check moved to separate method. Get header operation moved out from concat operation.

Result:

WebSocketServerProtocolHandshakeHandler.getWebSocketLocation is faster as OptimizeStringConcat could be applied. Code easier to read.
2017-03-01 10:47:20 +01:00
Norman Maurer
d675febf07 Optimize / Cleanup QueryStringDecoder / QueryStringEncoder
Motivation:

QueryStringDecoder and QueryStringEncoder contained some code that could either cleaned-up or optimized.

Modifications:

- Fix typos in exception messages and javadocs
- Precompile Pattern
- Make use of StringUtil.EMPTY_STRING

Result:

Faster and cleaner code.
2017-03-01 06:46:54 +01:00
Nikolay Fedorovskikh
943f4ec7ff Make methods 'static' where it missed
Motivation:

Calling a static method is faster then dynamic

Modifications:

Add 'static' keyword for methods where it missed

Result:

A bit faster method calls
2017-02-23 11:01:57 +01:00
Nikolay Fedorovskikh
0623c6c533 Fix javadoc issues
Motivation:

Invalid javadoc in project

Modifications:

Fix it

Result:

More correct javadoc
2017-02-22 07:31:07 +01:00
Norman Maurer
fbf0e5f4dd Prefer JDK ThreadLocalRandom implementation over ours.
Motivation:

We have our own ThreadLocalRandom implementation to support older JDKs . That said we should prefer the JDK provided when running on JDK >= 7

Modification:

Using ThreadLocalRandom implementation of the JDK when possible.

Result:

Make use of JDK implementations when possible.
2017-02-16 15:44:00 -08:00
Stephen E. Baker
9ee4cc0ada Correct comment for allowMaskMismatch parameter
Motivation:

The allowMaskMismatch parameter used throughout websocketx allows frames
with noncompliant masks when set to true, not false.

Modification:

Changed the javadoc comment everywhere it appears.

Result:

Fixes #6387
2017-02-16 17:03:55 +01:00
Jason Tedor
c92565d5c7 Correct expect header handling
Motivation:

Today, the HTTP codec in Netty responds to HTTP/1.1 requests containing
an "expect: 100-continue" header and a content-length that exceeds the
max content length for the server with a 417 status (Expectation
Failed). This is a violation of the HTTP specification. The purpose of
this commit is to address this situation by modifying the HTTP codec to
respond in this situation with a 413 status (Request Entity Too
Large). Additionally, the HTTP codec ignores expectations in the expect
header that are currently unsupported. This commit also addresses this
situation by responding with a 417 status.

Handling the expect header is tricky business as the specification (RFC
2616) is more complicated than it needs to be. The specification defines
the legitimate values for this header as "100-continue" and defines the
notion of expectatation extensions. Further, the specification defines a
417 status (Expectation Failed) and this is where implementations go
astray. The intent of the specification was for servers to respond with
417 status when they do not support the expectation in the expect
header.

The key sentence from the specification follows:

    The server MUST respond with a 417 (Expectation Failed) status if
    any of the expectations cannot be met or, if there are other
    problems with the request, some other 4xx status.

That is, a server should respond with a 417 status if and only if there
is an expectation that the server does not support (whether it be
100-continue, or another expectation extension), and should respond with
another 4xx status code if the expectation is supported but there is
something else wrong with the request.

Modifications:

This commit modifies the HTTP codec by changing the handling for the
expect header in the HTTP object aggregator. In particular, the codec
will now respond with 417 status if any expectation other than
100-continue is present in the expect header, the codec will respond
with 413 status if the 100-continue expectation is present in the expect
header and the content-length is larger than the max content length for
the aggregator, and otherwise the codec will respond with 100 status.

Result:

The HTTP codec can now be used to correctly reply to clients that send a
100-continue expectation with a content-length that is too large for the
server with a 413 status, and servers that use the HTTP codec will now
no longer ignore expectations that are not supported (any value other
than 100-continue).
2017-02-15 14:39:14 +01:00
Dmitriy Dumanskiy
506f0d8f8c Cleanup : String.length() == 0 replaced with String.isEmpty, removed unnecessary assert, class cast 2017-02-14 15:36:42 +01:00
Stephane Landelle
64abef5f5b Add exposeHeaders and allowedRequestHeaders that accept CharSequence, close #6328
Motivation:

Netty 4.1 introduced AsciiString and defines HttpHeaderNames constants
as such.

It would be convenient to be able to pass them to `exposeHeaders` and
`allowedRequestHeaders` directly without having to call `toString`.

Modifications:

Add `exposeHeaders` and `allowedRequestHeaders` overloads that take a
`CharSequence`.

Result:

More convenient API
2017-02-10 14:31:00 +01:00
Dmitriy Dumanskiy
c95517f759 Cleanup : removed unnecessary 'continue', explicit array creation, unwrapping 2017-02-10 12:25:01 +01:00
Stephane Landelle
9d45f514a4 Add a constant for Cookie "undefined maxAge"
Motivation:

DefaultCookie currently used an undocumented magic value for undefined
maxAge.

Clients need to be able to identify such value so they can implement a
proper CookieJar.

Ideally, we should add a `Cookie::isMaxAgeDefined` method but I guess
we can’t add a new method without breaking API :(

Modifications:

Add a new constant on `Cookie` interface so clients can use it to
compare with value return by `Cookie.maxAge` and decide if `maxAge` was
actually defined.

Result:

Clients have a better documented way to check if the maxAge attribute
was defined.
2017-02-09 10:14:29 +01:00
Norman Maurer
a7c0ff665c Only use Mockito for mocking.
Motivation:

We used various mocking frameworks. We should only use one...

Modifications:

Make usage of mocking framework consistent by only using Mockito.

Result:

Less dependencies and more consistent mocking usage.
2017-02-07 08:47:22 +01:00
Vladimir Kostyukov
0f9b739508 AggregatedFullHttpMessage.replace should also copy a decoder result
Motivation:

HttpObjectAggregator yields full HTTP messgaes (AggregatedFullHttpMessages) that don't respect decoder result when copied/replaced.

Modifications:

Copy the decoding result over to a new instance produced by AggregatedFullHttpRequest.replace or AggregatedFullHttpResponse.replace .

Result:

DecoderResult is now copied over when an original AggregatedFullHttpMessage is being replaced (i.e., AggregatedFullHttpRequest.replace or AggregatedFullHttpResponse.replace is being called).

New unit tests are passing on this branch but are failing on master.
2017-02-06 07:49:53 +01:00
Norman Maurer
735d6dd636 [maven-release-plugin] prepare for next development iteration 2017-01-30 15:14:02 +01:00
Norman Maurer
76e22e63f3 [maven-release-plugin] prepare release netty-4.1.8.Final 2017-01-30 15:12:36 +01:00
Chris Conroy
9bec25a6eb Set the Transfer-Encoding header instead of adding
Motivation:

HttpUtil.setTransferEncodingChunked could add a second Transfer-Encoding
header if one was already present. While this is technically valid, it
does not appear to be the intent of the method.

Result:

Only one Transfer-Encoding header is present after calling this method.
2017-01-25 07:53:53 +01:00
Dmitriy Dumanskiy
2d11331591 Typo fix in post encoder and replaced static hashmap with array. 2017-01-19 10:44:27 -08:00
Norman Maurer
7f01da8d0f [maven-release-plugin] prepare for next development iteration 2017-01-12 11:36:51 +01:00
Norman Maurer
7a21eb1178 [maven-release-plugin] prepare release netty-4.1.7.Final 2017-01-12 11:35:58 +01:00
Frederic BREGIER
56ddc47f23 Extends HttpPostRequestEncoder to support all methods except TRACE
Motivation:

In Netty, currently, the HttpPostRequestEncoder only supports POST, PUT, PATCH and OPTIONS, while the RFC 7231 allows with a warning that GET, HEAD, DELETE and CONNECT use a body too (but not TRACE where it is explicitely not allowed).
The RFC in chapter 4.3 says:
"A payload within a XXX request message has no defined semantics;
sending a payload body on a XXX request might cause some existing
implementations to reject the request."
where XXX can be replaced by one of GET, HEAD, DELETE or CONNECT.

Current usages, on particular in REST mode, tend to use those extra HttpMethods for such queries.

So this PR proposes to remove the current restrictions, leaving only TRACE as explicitely not supported.

Modification:

In the constructor, where the test is done, replacing all by checking only against TRACE, and adding one test to check that all methods are supported or not.

Result:

Fixes #6138.
2016-12-30 12:00:21 -08:00
Norman Maurer
0eeeb76439 Fix handling of FullHttpResponse when respond to HEAD in HttpServerCodec
Motivation:

cb139043f3 introduced special handling of response to HEAD requests. Due a bug we failed to handle FullHttpResponse correctly.

Modifications:

Correctly handle FullHttpResponse for HEAD requests.

Result:

Works as expected.
2016-12-21 20:53:02 +01:00
Scott Mitchell
3f82b53bae Add unit test for HttpObjectDecoder with message split on buffer boundaries
Motivation:
We should have a unit test which explicitly tests a HTTP message being split between multiple ByteBuf objects.

Modifications:
- Add a unit test to HttpRequestDecoderTest which splits a request between 2 ByteBuf objects

Result:
More unit test coverage for HttpObjectDecoder.
2016-12-20 12:59:00 -08:00
Malik Baktiyarov
16ddf460a6 Added checkStartsWith option for WebSocketServerProtocolHandler
Motivation:

Enables optional .startsWith() matching of req.uri() with websocketPath.

Modifications:

New checkStartsWith boolean option with default false value added to both WebSocketServerProtocolHandler and WebSocketServerProtocolHandshakeHandler. req.uri() matching is based on this option.

Result:

By default old behavior matching via .equal() is preserved. To use checkStartsWith use constructor shortcut: new WebSocketServerProtocolHandler(websocketPath, true) or fill this flag on full form of constructor among other options.
2016-12-20 10:53:33 +01:00
Norman Maurer
cb139043f3 [#5831] HttpServerCodec cannot encode a respons e to HEAD
request with a 'content-encoding: chunked' header

Motivation:

It is valid to send a response to a HEAD request that contains a transfer-encoding: chunked header, but it is not valid to include a body, and there is no way to do this using the netty4 HttpServerCodec.

The root cause is that the netty4 HttpObjectEncoder will transition to the state ST_CONTENT_CHUNK and the only way to transition back to ST_INIT is through the encodeChunkedContent method which will write the terminating length (0\r\n\r\n\r\n), a protocol error when responding to a HEAD request

Modifications:

- Keep track of the method of the request and depending on it handle the response differently when encoding it.
- Added a unit test.

Result:

Correclty handle HEAD responses that are chunked.
2016-12-15 07:54:51 +00:00
Stephane Maldini
ea0ddc0ea2 fix #6066 Support optional filename in HttpPostRequestEncoder
Motivation:

According to https://www.ietf.org/rfc/rfc2388.txt 4.4, filename after "content-disposition" is optional and arbitrary (does not need to match a real filename).

Modifications:

This change supports an extra addBodyFileUpload overload to precise the filename (default to File.getName). If empty or null this argument should be ignored during encoding.

Result:
- A backward-compatible addBodyFileUpload(String, File, String, boolean) to use file.getName() as filename.
- A new addBodyFileUpload(String, String, File, String, boolean) overload to precise filename
- Couple of tests for the empty use case
2016-12-01 06:54:51 +01:00
Stephane Landelle
ba95c401a7 Misc clean up
Motivation:
IntelliJ issues several warnings.

Modifications:

* `ClientCookieDecoder` and `ServerCookieDecoder`:
  * `nameEnd`, `valueBegin` and `valueEnd` don't need to be initialized
  * `keyValLoop` loop doesn't been to be labelled, as it's the most inner one (same thing for labelled breaks)
  * Remove `if (i != headerLen)` as condition is always true
* `ClientCookieEncoder` javadoc still mention old logic
* `DefaultCookie`, `ServerCookieEncoder` and `DefaultHttpHeaders` use ternary ops that can be turned into simple boolean ones
* `DefaultHeaders` uses a for(int) loop over an array. It can be turned into a foreach one as javac doesn't allocate an iterator to iterate over arrays
* `DefaultHttp2Headers` and `AbstractByteBuf` `equal` can be turned into a single boolean statement
Result:

Cleaner code
2016-11-22 15:17:05 -08:00
Stephane Landelle
f755e58463 Clean up following #6016
Motivation:

* DefaultHeaders from netty-codec has some duplicated logic for header date parsing
* Several classes keep on using deprecated HttpHeaderDateFormat

Modifications:

* Move HttpHeaderDateFormatter to netty-codec and rename it into HeaderDateFormatter
* Make DefaultHeaders use HeaderDateFormatter
* Replace HttpHeaderDateFormat usage with HeaderDateFormatter

Result:

Faster and more consistent code
2016-11-21 12:35:40 -08:00
radai-rosenblatt
886a7aae46 Fix timestamp parsing in HttpHeaderDateFormatter
Motivation:
code assumes a numeric value of 0 means no digits were read between separators, which fails for timestamps like 00:00:00.
also code accepts invalid timestamps like 0:0:000

Modifications:
explicitly check for number of digits between separators instead of relying on the numeric value.
also add tests.

Result:
timestamps with 00 successfully parse, timestamps with 000 no longer

Signed-off-by: radai-rosenblatt <radai.rosenblatt@gmail.com>
2016-11-21 10:17:54 +01:00
Norman Maurer
0b3122d8ff Deprecate HttpUtil.getCharsetAsString(...) and introduce HttpUtil.getCharsetAsSequence(...).
Motivation:

The method HttpUtil.getCharsetAsString(...) is missleading as its return type is CharSequence and not String.

Modifications:

Deprecate HttpUtil.getCharsetAsString(...) and introduce HttpUtil.getCharsetAsSe
quence(...).

Result:

Less confusing method name.
2016-11-21 07:47:20 +01:00
Stephane Landelle
edc4842309 Fix cookie date parsing, close #6016
Motivation:
* RFC6265 defines its own parser which is different from RFC1123 (it accepts RFC1123 format but also other ones). Basically, it's very lax on delimiters, ignores day of week and timezone. Currently, ClientCookieDecoder uses HttpHeaderDateFormat underneath, and can't parse valid cookies such as Github ones whose expires attribute looks like "Sun, 27 Nov 2016 19:37:15 -0000"
* ServerSideCookieEncoder currently uses HttpHeaderDateFormat underneath for formatting expires field, and it's slow.

Modifications:
* Introduce HttpHeaderDateFormatter that correctly implement RFC6265
* Use HttpHeaderDateFormatter in ClientCookieDecoder and ServerCookieEncoder
* Deprecate HttpHeaderDateFormat

Result:
* Proper RFC6265 dates support
* Faster ServerCookieEncoder and ClientCookieDecoder
* Faster tool for handling headers such as "Expires" and "Date"
2016-11-18 11:22:21 +00:00
Norman Maurer
0bc30a123e Eliminate usage of releaseLater(...) to reduce memory usage during tests
Motiviation:

We used ReferenceCountUtil.releaseLater(...) in our tests which simplifies a bit the releasing of ReferenceCounted objects. The problem with this is that while it simplifies stuff it increase memory usage a lot as memory may not be freed up in a timely manner.

Modifications:

- Deprecate releaseLater(...)
- Remove usage of releaseLater(...) in tests.

Result:

Less memory needed to build netty while running the tests.
2016-11-18 09:34:11 +01:00
Adrian Gonzalez
baac352f74 WebSocketClientHandshaker.rawPath(URI) should use the raw query
Motivation:

If the wsURL contains an encoded query, it will be decoded when generating the raw path.  For example if the wsURL is http://test.org/path?a=1%3A5, the returned raw path would be /path?a=1:5

Modifications:

Use wsURL.getRawQuery() rather than wsURL.getQuery()

Result:

rawPath will now return /path?a=1%3A5
2016-11-14 08:45:27 +01:00
Stephane Landelle
75728faf9b Add more HttpHeaders values
Motivation:
Some commons values are missing from HttpHeader values constants.

Modifications:
- Add constants for "application/json" Content-Type
- Add constants for "gzip,deflate" Content-Encoding

Result:
More HttpHeader values constants available, both in
`HttpHeaders.Values` and `HttpHeaderValues`.
2016-11-12 15:54:02 +01:00
Bryce Anderson
f0f0edbf78 HttpObjectAggregator adds 'Connection: close' header if necessary
Motivation:

The HttpObjectAggregator never appends a 'Connection: close' header to
the response of oversized messages even though in the majority of cases
its going to close the connection.

Modification:

This PR addresses that by ensuring the requisite header is present when
the connection is going to be closed.

Result:

Gracefully signal that we are about to close the connection.
2016-11-08 08:43:30 +01:00
Norman Maurer
8269e0f046 [#5892] Correct handle HttpMessage that is EOF terminated
Motivation:

We need to ensure we not add the Transfer-Encoding header if the HttpMessage is EOF terminated.

Modifications:

Only add the Transfer-Encoding header if an Content-Length header is present.

Result:

Correctly handle HttpMessage that is EOF terminated.
2016-11-01 11:13:44 +01:00
Moses Nakamura
bff951ca07 codec-http: HttpClientUpgradeHandler can handle streamed responses
Motivation:

We want to reject the upgrade as quickly as possible, so that we can
support streamed responses.

Modifications:

Reject the upgrade as soon as we inspect the headers if they're wrong,
instead of waiting for the entire response body.

Result:

If a remote server doesn't know how to use the http upgrade and tries to
responsd with a streaming response that never ends, the client doesn't
buffer forever, but can instead pass it along.  Fixes #5954
2016-11-01 06:32:41 +01:00
Norman Maurer
5f533b7358 [maven-release-plugin] prepare for next development iteration 2016-10-14 13:20:41 +02:00
Norman Maurer
35fb0babe2 [maven-release-plugin] prepare release netty-4.1.6.Final 2016-10-14 12:47:19 +02:00
radai-rosenblatt
15ac6c4a1f Clean-up unused imports
Motivation:

the build doesnt seem to enforce this, so they piled up

Modifications:

removed unused import lines

Result:

less unused imports

Signed-off-by: radai-rosenblatt <radai.rosenblatt@gmail.com>
2016-09-30 09:08:50 +02:00
Norman Maurer
cf8f6e3e2f [#5861] HttpUtil.getContentLength(HttpMessage, long) throws unexpected NumberFormatException
Motivation:

The Javadocs of HttpUtil.getContentLength(HttpMessage, long) and its int overload state that the provided default value is returned if the Content-Length value is not a number. NumberFormatException is thrown instead.

Modifications:

Correctly handle when the value is not a number.

Result:

API works as stated in javadocs.
2016-09-29 21:32:07 +02:00
Scott Mitchell
dd1ba2a252 HttpObjectDecoder resetRequested not updated after reset
Motivation:
HttpObjectDecoder maintains a resetRequested flag which is used to determine if internal state should be reset when a decode occurs. However after a reset is done the resetRequested flag is not set to false. This leads to all data after this point being discarded.

Modifications:
- Set resetRequested to false when a reset is done

Result:
HttpObjectDecoder can still function after a reset.
2016-09-22 10:58:44 -07:00
Christopher O'Toole
c57d4bed91 Add HttpServerKeepAliveHandler
Motivation:

As discussed in #5738, developers need to concern themselves with setting
connection: keep-alive on the response as well as whether to close a
connection or not after writing a response.  This leads to special keep-alive
handling logic in many different places.  The purpose of the HttpServerKeepAliveHandler
is to allow developers to add this handler to their pipeline and therefore
free themselves of having to worry about the details of how Keep-Alive works.

Modifications:

Added HttpServerKeepAliveHandler to the io.netty.handler.codec.http package.

Result:

Developers can start using HttpServerKeepAliveHandler in their pipeline instead
of worrying about when to close a connection for keep-alive.
2016-09-15 15:59:21 -07:00
Gaston Tonietti
245fb52c90 Provide extra info together with handshake complete event.
Motivation:

As described in #5734

Before this change, if the server had to do some sort of setup after a
handshake was completed based on handshake's information, the only way
available was to wait (in a separate thread) for the handshaker to be
added as an attribute to the channel. Too much hassle.

Modifications:

Handshake completed event need to be stateful now, so I've added a tiny
class holding just the HTTP upgrade request and the selected subprotocol
which is fired as an event after the handshake has finished.
I've also deprecated the old enum used as stateless event and I left the
code that fires it for backward compatibility. It should be removed in
the next mayor release.

Result:

It should be much simpler now to do initialization stuff based on
subprotocol or request headers on handshake completion. No asynchronous
waiting needed anymore.
2016-09-11 17:52:07 +02:00
William Blackie
e3aca1f3d6 CorsHandler to respect http connection (keep-alive) header.
Motivation:

The CorsHandler currently closes the channel when it responds to a preflight (OPTIONS)
request or in the event of a short circuit due to failed validation.

Especially in an environment where there's a proxy in front of the service this causes
unnecessary connection churn.

Modifications:

CorsHandler now uses HttpUtil to determine if the connection should be closed
after responding and to set the Connection header on the response.

Result:

Channel will stay open when the CorsHandler responds unless the client specifies otherwise
or the protocol version is HTTP/1.0
2016-09-06 07:18:53 +02:00
Norman Maurer
54b1a100f4 [maven-release-plugin] prepare for next development iteration 2016-08-26 10:06:32 +02:00
Christopher O'Toole
f6c16f4897 Attempt at improving docs in HttpObjectAggregator
Motivation:

Documentation was added in #2401 to aid developers in understanding
how HttpObjectAggregator works and that it needs an encoder before it.

In #2471 it was pointed out that the documentation added can actually
add to the confusion and that it might have a typo.

This is an attempt at clearing up that confusion.  Feedback is welcome.

Modifications:

- Adjust class level javadoc for HttpObjectAggregator
  * Remove reference to HttpRequestEncoder
  * Point out when HttpResponseEncoder is needed
  * Point out that either HttpRequestDecoder or HttpResponseDecoder is needed
  * Make clear everything must be added before HttpObjectAggregator
  * Mention HttpServerCodec

Result:

Avoid confusion about dependencies for HttpObjectAggregator on the pipeline.
2016-08-30 22:10:51 +02:00
Norman Maurer
1208b90f57 [maven-release-plugin] prepare release netty-4.1.5.Final 2016-08-26 04:59:35 +02:00
Norman Maurer
a8b8553ad1 Revert "CorsHandler to respect http connection (keep-alive) header."
This reverts commit ecd6e5ce6d.
2016-08-24 08:54:29 +02:00
William Blackie
ecd6e5ce6d CorsHandler to respect http connection (keep-alive) header.
Motivation:

The CorsHandler currently closes the channel when it responds to a preflight (OPTIONS)
request or in the event of a short circuit due to failed validation.

Especially in an environment where there's a proxy in front of the service this causes
unnecessary connection churn.

Modifications:

CorsHandler now uses HttpUtil to determine if the connection should be closed
after responding

Result:

Channel will stay open when the CorsHandler responds unless the client specifies otherwise
or the protocol version is HTTP/1.0
2016-08-24 08:50:29 +02:00
Sergey Polovko
3451b3cbb3 Cookie name must be case sensitive
Motivation:

RFC 6265 does not state that cookie names must be case insensitive.

Modifications:

Fix io.netty.handler.codec.http.cookie.DefaultCookie#equals() method to
use case sensitive String#equals() and String#compareTo().

Result:

It is possible to parse several cookies with same names but with
different cases.
2016-08-23 09:44:38 +02:00
Akhil
8d043cc4dd Do not return Access-Control-Allow-Headers on Non-Preflight Cors requests
Motivation:

The CorsHandler currently returns the Access-Control-Allow-Headers
header as on a Non-Preflight CORS request (Simple request).
As per the CORS specification the Access-Control-Allow-Headers header
should only be returned on Preflight requests. (not on simple requests).

https://www.w3.org/TR/2014/REC-cors-20140116/#access-control-allow-headers-response-header

http://www.html5rocks.com/static/images/cors_server_flowchart.png

Modifications:

Modified CorsHandler.java to not add the Access-Control-Allow-Headers
header when responding to Non-preflight CORS request.

Result:

Access-Control-Allow-Headers header will not be returned on a Simple
request (Non-preflight CORS request).
2016-08-16 13:45:04 +02:00
Dmitriy Dumanskiy
1bcc070943 cleanup, duplicated static final fields
Motivation:

There are few duplicated byte[] CRLF fields in code.

Modifications:

Removed duplicated fields as they could be inherited from parent encoder.

Result:

Less static fields.
2016-08-10 11:13:26 +02:00
Dmitriy Dumanskiy
a80ea46b8e Removed custom split method as it is not effective anymore. 2016-08-01 21:49:33 +02:00
Dmitriy Dumanskiy
a4d8f930af small performance fixes : unnecessary unboxing operations removed
Motivation :

Unboxing operations allocate unnecessary objects when it could be avoided.

Modifications:
Replaced Float.valueOf with Number.parseFloat where possible.

Result:

Less unnecessary objects allocations.
2016-08-01 07:10:25 +02:00
Scott Mitchell
82b617dfe9 retainSlice() unwrap ByteBuf
Motivation:
retainSlice() currently does not unwrap the ByteBuf when creating the ByteBuf wrapper. This effectivley forms a linked list of ByteBuf when it is only necessary to maintain a reference to the unwrapped ByteBuf.

Modifications:
- retainSlice() and retainDuplicate() variants should only maintain a reference to the unwrapped ByteBuf
- create new unit tests which generally verify the retainSlice() behavior
- Remove unecessary generic arguments from AbstractPooledDerivedByteBuf
- Remove unecessary int length member variable from the unpooled sliced ByteBuf implementation
- Rename the unpooled sliced/derived ByteBuf to include Unpooled in their name to be more consistent with the Pooled variants

Result:
Fixes https://github.com/netty/netty/issues/5582
2016-07-29 11:16:44 -07:00
Norman Maurer
cb7cf4491c [maven-release-plugin] prepare for next development iteration 2016-07-27 13:29:56 +02:00
Norman Maurer
9466b32d05 [maven-release-plugin] prepare release netty-4.1.4.Final 2016-07-27 13:16:59 +02:00
Ngoc Dao
835f901d5f Fix #5590 QueryStringDecoder#path should decode the path info
Motivation:

Currently, QueryStringDecoder#path simply returns the path info as is, without decoding it as the Javadoc states.

Modifications:

* Make QueryStringDecoder#path decode the path info.
* Add tests to QueryStringDecoderTest.

Result:

QueryStringDecoder#path now decodes the path info as expected.
2016-07-27 09:29:54 +02:00
Norman Maurer
047f6aed28 [maven-release-plugin] prepare for next development iteration 2016-07-15 09:09:13 +02:00
Norman Maurer
b2adea87a0 [maven-release-plugin] prepare release netty-4.1.3.Final 2016-07-15 09:08:53 +02:00
Norman Maurer
c735b3e147 [#5514] Fix DiskFileUpload and MemoryFileUpload equals(...) method.
Motivation:

DiskFileUpload and MemoryFileUpload.equals(...) are broken.

Modifications:

Fix implementation and add unit test.

Result:

Equals method are correct now.
2016-07-14 09:09:16 +02:00
Masaru Nomura
009680488e Fix set100ContinueExpected(...) jsvadoc
Motivation:

We don't have an argument named {@code value} but have {@code set} and
{@code expected} in HttpHeaders and HttpUtil respectively.

Modifications:

I replaced {@code value} to {@code set} and {@code expected} in HttpHeaders
and HttpUtil respectively.

Result:

Now javadoc says;
If {@code set} is {@code true}, the {@code "Expect: 100-continue"} header is
set and all other previous {@code "Expect"} headers are removed. Otherwise,
all {@code "Expect"} headers are removed completely. in HttpHeaders

If {@code expected} is {@code true}, the {@code "Expect: 100-continue"} header
is set and all other previous {@code "Expect"} headers are removed. Otherwise,
all {@code "Expect"} headers are removed completely. in HttpUtil
2016-07-04 19:03:14 +02:00
Norman Maurer
4676a2271c [maven-release-plugin] prepare for next development iteration 2016-07-01 10:33:32 +02:00
Norman Maurer
ad270c02b9 [maven-release-plugin] prepare release netty-4.1.2.Final 2016-07-01 09:07:40 +02:00
Tim Brooks
d964bf6f18 Remove usages of deprecated methods group() and childGroup().
Motivation:

These methods were recently deprecated. However, they remained in use in several locations in Netty's codebase.

Modifications:

Netty's code will now access the bootstrap config to get the group or child group.

Result:

No impact on functionality.
2016-06-21 14:06:57 +02:00
Norman Maurer
e845670043 Set some StackTraceElement on pre-instantiated static exceptions
Motivation:

We use pre-instantiated exceptions in various places for performance reasons. These exceptions don't include a stacktrace which makes it hard to know where the exception was thrown. This is especially true as we use the same exception type (for example ChannelClosedException) in different places. Setting some StackTraceElements will provide more context as to where these exceptions original and make debugging easier.

Modifications:

Set a generated StackTraceElement on these pre-instantiated exceptions which at least contains the origin class and method name. The filename and linenumber are specified as unkown (as stated in the javadocs of StackTraceElement).

Result:

Easier to find the origin of a pre-instantiated exception.
2016-06-20 11:33:05 +02:00
Norman Maurer
16be36a55f [#5402] sec-websocket-origin should mention HTTPS
Motivation:

When HTTPS is used we should use https in the sec-websocket-origin / origin header

Modifications:

- Correctly generate the sec-websocket-origin / origin header
- Add unit tests.

Result:

Generate correct header.
2016-06-20 11:22:09 +02:00
Nitesh Kant
ee0897a1d9 HttpContentDecompressor should change decompressed requests to chunked encoding. Fixes issue #5428
`HttpContentDecoder` was removing `Content-Length` header but not adding a `Transfer-Encoding` header which goes against the HTTP spec.

Added `Transfer-Encoding` header with value `chunked` when `Content-Length` is removed.
Modified existing unit test to also check for this condition.

Compliance with HTTP spec.
2016-06-20 07:43:06 +02:00
Norman Maurer
4a1e0ceb4d [5382] HttpContentEncoder should not set chunked transfer-encoding for HTTP/1.0
Motivation:

When using HttpContentCompressor and the HttpResponse is protocol version 1.0, HttpContentEncoder.encode() should not set the transfer-encoding header to chunked. Chunked transfer-encoding is not valid for HTTP 1.0 - this causes ERR_CONTENT_DECODING_FAILED errors in chrome and similar failures in IE.

Modifications:

Skip HTTP/1.0 messages

Result:

Be able to serve HTTP/1.0 as well when HttpContentEncoder is in the pipeline.
2016-06-17 06:35:33 +02:00
Norman Maurer
f5eea4698d Fix possible NPE in HttpCunkedInput if wrapped ChunkedInput.readChunk(...) return null.
Motivation:

Its completly fine for ChunkedInput.readChunk(...) to return null to indicate there is currently not any data to read. We need to handle this in HttpChunkedInput to not produce a NPE when constructing the HttpContent.

Modifications:

If readChunk(...) return null just return null as well.

Result:

No more NPE.
2016-06-17 06:27:04 +02:00
Scott Mitchell
328a1ec01b cleanup from 819b26b
Motivation:
I cherry-picked 819b26b too soon. There were entries added to a deprecated class which should only go into the non-deprecated version of the class.

Modifications:
- Remove the static final variables that were added as duplicates to the deprecated class

Result:
Deprecated code does not grown in volume without need.
2016-06-14 09:28:35 -07:00
Stephane Landelle
819b26b4bc Add more HttpHeaders values
Motivation:
Some commons values are missing from HttpHeader values constants.

Modifications:
- Add constants for "application/json" Content-Type
- Add constants for "gzip,deflate" Content-Encoding

Result:
More HttpHeader values constants available, both in
`HttpHeaders.Values` and `HttpHeaderValues`.
2016-06-14 09:23:21 -07:00
Sina Tadayon
eb1d932466 Support WebSocket data chunked transfer
Motivation:
Support fetches data chunk by chunk for use with WebSocket chunked transfers.

Modifications:
Create a WebSocketChunkedInput.java that add to io.netty.handler.codec.http.websocketx package

Result:
The WebSocket transfers/fetches data chunk by chunk.
2016-06-13 08:55:07 +02:00