Commit Graph

6073 Commits

Author SHA1 Message Date
Trustin Lee
27b67e981e Fix NPE in OpenSslEngine 2014-05-21 20:02:40 +09:00
Trustin Lee
ac968fbf7b Escape a colon in protocol names 2014-05-21 17:45:18 +09:00
Trustin Lee
d318f526a2 Update the instruction for running SPDY examples 2014-05-21 17:30:07 +09:00
Trustin Lee
861ed1e7ad Add unified NextProtoNego extension support to SslContext
Motivation:

- OpenSslEngine and JDK SSLEngine (+ Jetty NPN) have different APIs to
  support NextProtoNego extension.
  - It is impossible to configure NPN with SslContext when the provider
    type is JDK.

Modification:

- Implement NextProtoNego extension by overriding the behavior of
  SSLSession.getProtocol() for both OpenSSLEngine and JDK SSLEngine.
  - SSLEngine.getProtocol() returns a string delimited by a colon (':')
    where the first component is the transport protosol (e.g. TLSv1.2)
    and the second component is the name of the application protocol
- Remove the direct reference of Jetty NPN classes from the examples
- Add SslContext.newApplicationProtocolSelector

Result:

- A user can now use both JDK SSLEngine and OpenSslEngine for NPN-based
  protocols such as HTTP2 and SPDY
2014-05-21 17:24:52 +09:00
Trustin Lee
d4f2488eac Bash 3 compatibility
Motivation:

Mac OS X ships Bash 3, and it does not have an associative array
(declare -A).

Modifications:

Do not use an associative array.

Result:

Can run examples on Mac OS X using run-example.sh
2014-05-21 13:37:15 +09:00
Trustin Lee
e61ad7f80b Improve run-example.sh
- More usage example
- Newlines for prettier output
2014-05-20 23:39:32 +09:00
Trustin Lee
4436870b28 Clean up the execution mechanism of examples
Motivation:

- There's no way to pass an argument to an example.
- Assigning a Maven profile for each example is an overkill.
  It makes the pom.xml crowded.

Modifications:

- Remove example profiles from example/pom.xml
- Keep the list of examples in run-example.sh
- run-example.sh passes all options to exec-maven-plugin.
  For example, we can now do this:

    ./run-example.sh -Dssl -Dport=443 http-server

Result:

- It's much easier to add a new example and provide an easy way to
  launch it.
- We can still pass an arbitrary argument to the example being launched.
  (I'll update all examples to make them get their options from system
  properties rather than from args[].
2014-05-20 23:33:45 +09:00
Trustin Lee
5354ccaa8f Fix a build problem with JDK 8
Motivation:

Build fails with JDK 8 because npn-boot does not work with JDK 8

Modifications:

Do not specify bootclasspath when on JDK 8

Result:

Build is green again.
2014-05-20 22:40:11 +09:00
Trustin Lee
d942c13c8e Fix a problem where all classes are compiled again
Motivation:

Due to a known problem[1] of maven-compiler-plugin, our build always
compiles everything from scratch, which is waste of time.

Modifications:

Exclude package-info.java from the source list.

Result:

Much shorter build time.

[1]: https://jira.codehaus.org/browse/MCOMPILER-205
2014-05-20 20:09:47 +09:00
Trustin Lee
c77375cf54 Clean up example execution
Motivation:

- example/pom.xml has quite a bit of duplication.
- We expect that we depend on npn-boot in more than one module in the
  near future. (e.g. handler, codec-http, and codec-http2)

Modification:

- Deduplicate the profiles in example/pom.xml
- Move the build configuration related with npn-boot to the parent pom.
- Add run-example.sh that helps a user launch an example easily

Result:

- Cleaner build files
- Easier to add a new example
- Easier to launch an example
- Easier to run the tests that relies on npn-boot in the future
2014-05-20 20:09:05 +09:00
Trustin Lee
c4fc6b7043 Checkstyle 2014-05-20 17:39:14 +09:00
Trustin Lee
7252934f9b Limit the number of bytes to copy per Unsafe.copyMemory()
Motivation:

During a large memory copy, safepoint polling is diabled, hindering
accurate profiling.

Modifications:

Only copy up to 1 MiB per Unsafe.copyMemory()

Result:

Potentially more reliable performance
2014-05-20 17:19:51 +09:00
Trustin Lee
4882377c27 Work around the JVM crash that occurs when cipher suite uses GCM
Motivation:

For an unknown reason, JVM of JDK8 crashes intermittently when
SslHandler feeds a direct buffer to SSLEngine.unwrap() *and* the current
cipher suite has GCM (Galois/Counter Mode) enabled.

Modifications:

Convert the inbound network buffer to a heap buffer when the current
cipher suite is using GCM.

Result:

JVM does not crash anymore.
2014-05-19 11:47:20 +09:00
Trustin Lee
66851d222b Update os-maven-plugin again to address an IDEA integration issue 2014-05-19 01:32:49 +09:00
Trustin Lee
ccbc19a45f Add JVM crash logs to .gitignore 2014-05-18 21:36:54 +09:00
Trustin Lee
2a30d0c22a Upgrade os-maven-plugin to fix the build issue 2014-05-18 17:31:08 +09:00
Trustin Lee
ae61b12b9e Reduce memory usage of SslHandler when OpenSslEngine is in use
Motivation:

JDK's SSLEngine.wrap() requires the output buffer to be always as large as MAX_ENCRYPTED_PACKET_LENGTH even if the input buffer contains small number of bytes.  Our OpenSslEngine implementation does not have such wasteful behaviot.

Modifications:

If the current SSLEngine is OpenSslEngine, allocate as much as only needed.

Result:

Less peak memory usage.
2014-05-18 05:10:41 +09:00
Trustin Lee
ef59c4ce61 Add netty-tcnative dependency to netty-example
Motivation:

It's useful to have netty-tcnative dependency in netty-example because
we can play with OpenSslEngine from our IDE.

Modifications:

Add netty-tcnative to example/pom.xml
2014-05-18 04:29:50 +09:00
Trustin Lee
28c390d86d Fix buffer leaks during PEM to KeyStore conversion 2014-05-18 04:29:50 +09:00
Trustin Lee
167a685a5f Fix JDK 8 compatibility issue with OpenJdkSelfSignedCertGenerator
- X509CertInfo.setSubject/setIssuer() requires X500Name instead of
  CertificateSubjectName/CertificateIssuerName.
2014-05-18 03:47:55 +09:00
Trustin Lee
9125060752 Optimize SslHandler in an OpenSslEngine-friendly way
Motivation:

Previous fix for the OpenSslEngine compatibility issue (#2216 and
18b0e95659) was to feed SSL records one by
one to OpenSslEngine.unwrap().  It is not optimal because it will result
in more JNI calls.

Modifications:

- Do not feed SSL records one by one.
- Feed as many records as possible up to MAX_ENCRYPTED_PACKET_LENGTH
- Deduplicate MAX_ENCRYPTED_PACKET_LENGTH definitions

Result:

- No allocation of intemediary arrays
- Reduced number of calls to SSLEngine and thus its underlying JNI calls
- A tad bit increase in throughput, probably reverting the tiny drop
  caused by 18b0e95659
2014-05-18 03:33:00 +09:00
Trustin Lee
b6c0c0c95f Add an OpenSslEngine and the universal API for enabling SSL
Motivation:

Some users already use an SSLEngine implementation in finagle-native. It
wraps OpenSSL to get higher SSL performance.  However, to take advantage
of it, finagle-native must be compiled manually, and it means we cannot
pull it in as a dependency and thus we cannot test our SslHandler
against the OpenSSL-based SSLEngine.  For an instance, we had #2216.

Because the construction procedures of JDK SSLEngine and OpenSslEngine
are very different from each other, we also need to provide a universal
way to enable SSL in a Netty application.

Modifications:

- Pull netty-tcnative in as an optional dependency.
  http://netty.io/wiki/forked-tomcat-native.html
- Backport NativeLibraryLoader from 4.0
- Move OpenSSL-based SSLEngine implementation into our code base.
  - Copied from finagle-native; originally written by @jpinner et al.
  - Overall cleanup by @trustin.
- Run all SslHandler tests with both default SSLEngine and OpenSslEngine
- Add a unified API for creating an SSL context
  - SslContext allows you to create a new SSLEngine or a new SslHandler
    with your PKCS#8 key and X.509 certificate chain.
  - Add JdkSslContext and its subclasses
  - Add OpenSslServerContext
- Add ApplicationProtocolSelector to ensure the future support for NPN
  (NextProtoNego) and ALPN (Application Layer Protocol Negotiation) on
  the client-side.
- Add SimpleTrustManagerFactory to help a user write a
  TrustManagerFactory easily, which should be useful for those who need
  to write an alternative verification mechanism. For example, we can
  use it to implement an unsafe TrustManagerFactory that accepts
  self-signed certificates for testing purposes.
- Add InsecureTrustManagerFactory and FingerprintTrustManager for quick
  and dirty testing
- Add SelfSignedCertificate class which generates a self-signed X.509
  certificate very easily.
- Update all our examples to use SslContext.newClient/ServerContext()
- SslHandler now logs the chosen cipher suite when handshake is
  finished.

Result:

- Cleaner unified API for configuring an SSL client and an SSL server
  regardless of its internal implementation.
- When native libraries are available, OpenSSL-based SSLEngine
  implementation is selected automatically to take advantage of its
  performance benefit.
- Examples take advantage of this modification and thus are cleaner.
2014-05-18 02:54:06 +09:00
Trustin Lee
2a6ed67efc Preparation for porting OpenSSL support in 3.10
- Add licenses and dependencies
2014-05-17 20:01:30 +09:00
Norman Maurer
2d9e0f53a5 Better implementation of AttributeMap and also add hasAttr(...). See [#2439]
Motivation:
The old DefaultAttributeMap impl did more synchronization then needed and also did not expose a efficient way to check if an attribute exists with a specific key.

Modifications:
* Rewrite DefaultAttributeMap to not use IdentityHashMap and synchronization on the map directly. The new impl uses a combination of AtomicReferenceArray and synchronization per chain (linked-list). Also access the first Attribute per bucket can be done without any synchronization at all and just uses atomic operations. This should fit for most use-cases pretty weel.
* Add hasAttr(...) implementation

Result:
It's now possible to check for the existence of a attribute without create one. Synchronization is per linked-list and the first entry can even be added via atomic operation.
2014-05-15 06:47:42 +02:00
Jake Luciani
d547b5d51d Fix capacity check bug affecting offheap buffers 2014-05-13 07:25:15 +02:00
Trustin Lee
e29f8231aa Fix the life cycle mapping problem with m2e
Motivation:

It should be frictionless to import our project into Eclipse

Modifications:

Exclude the plugins with missing life cycle mapping. They are not useful
for use with IDE anyway.

Result:

Fixes #2488
Netty is imported into Eclipse without a problem.
2014-05-12 21:38:09 +09:00
Norman Maurer
2f7d60f234 Minimize memory footprint of HashedWheelTimer and context-switching
Motivation:
At the moment there are two issues with HashedWheelTimer:
* the memory footprint of it is pretty heavy (250kb fon an empty instance)
* the way how added Timeouts are handled is inefficient in terms of how locks etc are used and so a lot of context-switching / condition can happen.

Modification:
Rewrite HashedWheelTimer to use an optimized bucket implementation to store the submitted Timeouts and a MPSC queue to handover the timeouts.  So volatile writes are reduced to a minimum and also the memory foot-print of the buckets itself is reduced a lot as the bucket uses a double-linked-list. Beside this we use Atomic*FieldUpdater where-ever possible to improve the memory foot-print and performance.

Result:
Lower memory-footprint and better performance
2014-05-11 15:12:29 +02:00
Trustin Lee
d69ad2f85c More robust native library discovery in Mac OS X
Motivation:

Some JDK versions of Mac OS X generates a JNI dynamic library with '.jnilib' extension rather than with '.dynlib' extension.  However, System.mapLibraryName() always returns 'lib<name>.dynlib'. As a result, NativeLibraryLoader fails to load the native library whose extension is .jnilib.

Modification:

Try to find both '.jnilib' and '.dynlib' resources on OS X.

Result:

Dynamic libraries are loaded correctly in Mac OS X, and thus we can continue the OpenSslEngine work.
2014-05-11 18:53:43 +09:00
Norman Maurer
880acbca72 Use ByteBuf.readSlice(...).retain() to minimize memory copies.
Motivation:
At the moment we call ByteBuf.readBytes(...) in these handlers but with optimizations done as part of 25e0d9d we can just use readSlice(...).retain() and eliminate the memory copy.

Modifications:
Replace ByteBuf.readBytes(...) usage with readSlice(...).retain().

Result:
Less memory copies.
2014-05-10 17:20:53 +02:00
Norman Maurer
59d92ad6cf [#2485] Use RecvByteBufAllocator for all allocations related to read from Channel
Motivation:
At the moment we sometimes use only RecvByteBufAllocator.guess() to guess the next size and the use the ByteBufAllocator.* directly to allocate the buffer. We should always use RecvByteBufAllocator.allocate(...) all the time as this makes the behavior easier to adjust.

Modifications:
Change the read() implementations to make use of RecvByteBufAllocator.

Result:
Behavior is more consistent.
2014-05-10 15:27:43 +02:00
Trustin Lee
69c72220aa Update netty-build to version 20
- Preparation for merging OpenSslEngine work
2014-05-08 10:00:16 +09:00
Norman Maurer
c5e4c2294e [#2469] Correctly update indices after incomplete gathering write
Motivation:
 When doing a gathering write we need to update the indices after the write partial completes. In the current code-base we use the wrong value when compare the expected written bytes and the actual written bytes.

 Modifications:
 Use the correct value when compare.

Result:
Indices are updated correctly and so no corruption can happen when resume writing after data was only partial written before.
2014-05-07 07:13:39 +02:00
Daniel Bevenius
1a22186645 Adding short-curcuit option for CORS
Motivation:
CORS request are currently processed, and potentially failed, after the
target ChannelHandler(s) have been invoked. This might not be desired, for
example a HTTP PUT or POST might have been performed.

Modifications:
Added a shortCurcuit option to CorsConfig which when set will
cause a validation of the HTTP request's 'Origin' header and verify that
it is valid according to the configuration. If found invalid an 403
"Forbidden" response will be returned and not further processing will
take place.

This is indeed no help for non browser request, like using curl, which
can set the 'Origin' header.

Result:
Users can now configure if the 'Origin' header should be validated
upfront and have the request rejected before any further processing
takes place.
2014-05-06 12:16:57 +02:00
Norman Maurer
8a39b9fedc Fix buffer leak introduced by #2462
Motivation:
Because of not correctly release a buffer before null out the reference a memory leak shows up.

Modifications:
Correct call buffer.release() before null out reference.

Result:
No more leak
2014-05-06 10:03:24 +02:00
Jeff Pinner
1351349243 SPDY: ensure SpdyHeaderBlockRawDecoder always reads entire input 2014-05-05 07:33:05 +02:00
Jeff Pinner
326efe5b42 SPDY: release headerBlock in SpdyFrameCodec 2014-05-05 07:26:21 +02:00
Jeff Pinner
1d9c431604 SPDY: remove unused code in SpdyHeaderBlockRawEncoder 2014-05-05 07:01:56 +02:00
Sun Ning
d713f015da Made websocket maxFramePayloadSize configurable from WebSocketServerProtocolHandler.
Motivation:

Currently there's no way to configure maxFramePayloadSize from
WebSocketServerProtocolHandler, which is the most used entry point of
WebSocket server.

Modifications:

Added another constructor for maxFramePayloadSize.

Result:

We can configure max frame size for websocket packet in
WebSocketServerProtocolHandler. It will also keep backward compatibility
with default max size: 65536. (65536 is hard-coded max size in previous
version of Netty)
2014-05-05 06:24:28 +02:00
Norman Maurer
9752f37fa0 [#2454] Correctly return null when DefaultChannelPipeline.firstContext() is called on empty pipeline
Motivation:
DefaultChannelPipeline.firstContext() should return null when the ipeline is empty. This is not the case atm.

Modification:
Fix incorrect check in DefaultChannelPipeline.firstContext() and add unit tests.

Result:
Correctly return null when DefaultChannelPipeline.firstContext() is called on empty pipeline.
2014-05-04 21:13:17 +02:00
Jeff Pinner
9dbcf1af49 SPDY: allow subclassing of SpdyFrameCodec 2014-05-03 14:41:51 +02:00
Trustin Lee
296db210ab Generate the default JAR to make oss.sonatype.org happy
Motivation:

oss.sonatype.org refuses to promote an artifact if it doesn't have the
default JAR (the JAR without classifier.)

Modifications:

- Generate both the default JAR and the native JAR to make
  oss.sonatype.org happy
- Rename the profile 'release' to 'restricted-release' which reflects
  what it really does better
- Remove the redundant <quickbuild>true</quickbuild> in all/pom.xml
  We specify the profile 'full' that triggers that property already
  in maven-release-plugin configuration.

Result:

oss.sonatype.org is happy.  Simpler pom.xml
2014-05-03 17:32:45 +09:00
Trustin Lee
a566164b98 Enforce the release is performed only from RHEL 6.5 or equivalent
Motivation:

Netty must be released from RHEL 6.5 x86_64 or compatible so that:

1) we ship x86_64 version of epoll transport officially, and
2) we ensure the ABI compatibility with older GLIBC versions.
   The shared library built on a distribution with newer GLIBC will not
   run on older distributions.

Modifications:

- When 'release' profile is active, perform an additional check using
  maven-enforcer-plugin so that 'mvn release:*' fails when running on
  non-RHEL6.5.  This rule is active only when releasing, so a user
  should not be affected.
- Simplify maven-release-plugin configuration by removing redundant
  profiles such as 'linux'.  'linux' is automatically activated when
  releasing because we now enforce the release occurs on linux-x86_64.
- Remove the no-osgi profile, which is unused
- Remove the reference to 'sonatype-oss-release' profile in all/pom.xml,
  because we always specify 'release' profile when releasing
- Rename the profile 'linux-native' to 'linux' for brevity
- Upgrade oss-parent and maven-enforcer-plugin

Result:

No one can make a mistake to release Netty on an environment that can
produce incompatible or missing native library.
2014-05-02 21:20:34 +09:00
Trustin Lee
df93b35027 Upgrade os-maven-plugin to fix a build issue on windows-x86_32 2014-05-02 21:15:30 +09:00
Trustin Lee
2ff06676ca Fix a build failure due to missing classifier 2014-05-02 14:36:02 +09:00
Trustin Lee
6c1af9036f Simplify native library resolution using os-maven-plugin
Motivation:

So far, we used a very simple platform string such as linux64 and
linux32.  However, this is far from perfection because it does not
include anything about the CPU architecture.

Also, the current build tries to put multiple versions of .so files into
a single JAR.  This doesn't work very well when we have to ship for many
different platforms.  Think about shipping .so/.dynlib files for both
Linux and Mac OS X.

Modification:

- Use os-maven-plugin as an extension to determine the current OS and
  CPU architecture reliable at build time
- Use Maven classifier instead of trying to put all shared libraries
  into a single JAR
- NativeLibraryLoader does not guess the OS and bit mode anymore and it
  always looks for the same location regardless of platform, because the
  Maven classifier does the job instead.

Result:

Better scalable native library deployment and retrieval
2014-05-02 04:21:47 +09:00
Norman Maurer
93fab1d5a3 Remove ContinuationWebSocketFrame.aggregatedText()
Motivation:
Before we aggregated the full text in the WebSocket08FrameDecoder just to fill in the ContinuationWebSocketFrame.aggregatedText(). The problem was that there was no upper-limit and so it would be possible to see an OOME if the remote peer sends a TextWebSocketFrame + a never ending stream of ContinuationWebSocketFrames. Furthermore the aggregation does not really belong in the WebSocket08FrameDecoder, as we provide an extra ChannelHandler for this anyway (WebSocketFrameAggregator).

Modification:
Remove the ContinuationWebSocketFrame.aggregatedText() method and corresponding constructor. Also refactored WebSocket08FrameDecoder a bit to me more efficient which is now possible as we not need to aggregate here.

Result:
No more risk of OOME because of frames.
2014-04-30 14:49:17 +02:00
Norman Maurer
0aea9eaed5 Correctly write pending data after ssl handshake completes. Related to [#2437]
Motivation:
When writing data from a server before the ssl handshake completes may not be written at all to the remote peer
if nothing else is written after the handshake was done.

Modification:
Correctly try to write pending data after the handshake was complete

Result:
Correctly write out all pending data
2014-04-30 14:23:18 +02:00
Norman Maurer
3ba856a5f4 Correctly handle SocksCmdResponse. Related to #2428
Motivation:
Ports range check is not correct

Modification:
Allow port between 0 and 65535. 0 is wildcard / unknown port here

Result:
Correct validation
2014-04-30 10:44:17 +02:00
Norman Maurer
80f1dfaec0 [#2428] Proper fix of incorrect port range check
Motivation:

In the Internet Protocol, the valid port number range is from 1 to 65535
(inclusive on the both side.)  However, SocksCmdRequest and SocksCmdResponse
refuses to construct itself when the port number 65535 is specified. Beside
this it excepts 0 as port number which should not allowed.

    Modification:

* Not raise an exception when the specified port number is 65535.
* Raise an exception when the specified port number is 0

Result:

Fixes #2428
2014-04-30 08:17:40 +02:00
Trustin Lee
01ef614e77 Fix incorrect port range check
Motivation:

In the Internet Protocol, the valid port number range is from 1 to 65535
(inclusive on the both side.)  However, SocksCmdRequest refuses to
construct itself when the port number 65535 is specified.

Modification:

Do not raise an exception when the specified port number is 65535.

Result:

Fixes #2428
2014-04-29 17:58:08 +09:00