Commit Graph

5952 Commits

Author SHA1 Message Date
Frédéric Brégier
e1abfbb6c1 Correction on StringBuilder default init size
Motivation:
The default StringBuilder size is too small (data.length + 4) while it will be 2*data.length (byte to Hex) + 5 "-" char (since 5 peaces appended).

Modification:
Changing initial size to the correct one

Result:
Allocation of the correct final size from the beginning for this StringBuilder.
2014-05-30 07:33:38 +02:00
Trustin Lee
ea0eb4fdab Do not use a pseudo random for tree traversal
Motivation:

If we make allocateRun/SubpageSimple() always try the left node first and make allocateRun/Subpage() always tries the right node first,  it is more likely that allocateRun/Subpage() will find a node with ST_UNUSED sooner.

Modifications:

- Make allocateRunSimple() and allocateSubpageSimple() always try the left node first.
- Make allocateRun() and allocateSubpage() always try the right node first.
- Remove randome

Result:

We get the same performance without using random numbers.
2014-05-30 11:23:46 +09:00
Trustin Lee
1d0a79e11e Optimize PooledByteBufAllocator
Motivation:

We still have a room for improvement in PoolChunk.allocateRun() and
Subpage.allocate().

Modifications:

- Unroll the recursion in PoolChunk.allocateRun()
- Subpage.allocate() makes use of the 'nextAvail' value set by previous
  free().

Result:

- PoolChunk.allocateRun() optimization yields 10%+ improvements in
  allocation throughput for non-subpage allocations.
- Subpage.allocate() optimization makes the subpage allocations for
  tiny buffers as fast as non-tiny buffers even when the pageSize is
  huge (e.g. 1048576) because it doesn't need to perform a linear search
  in most cases.
2014-05-30 10:50:23 +09:00
Trustin Lee
b50f91f6d0 More realistic ByteBuf allocation benchmark
Motivation:

Allocating a single buffer and releasing it repetitively for a benchmark will not involve the realistic execution path of the allocators.

Modifications:

Keep the last 8192 allocations and release them randomly.

Result:

We are now getting the result close to what we got with caliper.
2014-05-29 19:50:43 +09:00
Trustin Lee
d54030266b Upgrade to netty-tcnative 1.1.30.Fork2 to support Windows 2014-05-28 10:55:52 +09:00
Trustin Lee
d90207b3ac Work around the system configuration issue that causes NioSocketChannelTest to fail
Motivation:

On some ill-configured systems, InetAddress.getLocalHost() fails.  NioSocketChannelTest calls java.net.Socket.connect() and it internally invoked InetAddress.getLocalHost(), which causes the test failures in NioSocketChannelTes on such an ill-configured system.

Modifications:

Use NetUtil.LOCALHOST explicitly.

Result:

NioSocketChannelTest should not fail anymore.
2014-05-28 09:38:42 +09:00
Trustin Lee
e044fc3e56 Upgrade os-maven-plugin to fix an issue with IntelliJ IDEA on Windows 2014-05-27 04:41:58 +09:00
Trustin Lee
7667a4a32e Use a forked exec-maven-plugin instead of maven-antrun-plugin
Motivation:

maven-antrun-plugin does not redirect stdin, and thus it's impossible to
run interactive examples such as securechat-client and telnet-client.
org.codehaus.mojo:exec-maven-plugin redirects stdin, but it buffers
stdout and stderr, and thus an application output is not flushed timely.

Modifications:

Deploy a forked version of exec-maven-plugin which flushes output
buffers in a timely manner.

Result:

Interactive examples work.  Launches faster than maven-antrun-plugin.
2014-05-23 20:03:14 +09:00
Trustin Lee
3f8194557b Clean up the examples
Motivation:

The examples have not been updated since long time ago, showing various
issues fixed in this commit.

Modifications:

- Overall simplification to reduce LoC
  - Use system properties to get options instead of parsing args.
  - Minimize option validation
  - Just use System.out/err instead of Logger
  - Do not pass config as parameters - just access it directly
  - Move the main logic to main(String[]) instead of creating a new
    instance meaninglessly
    - Update netty-build-21 to make checkstyle not complain
  - Remove 'throws Exception' clause if possible
- Line wrap at 120 (previously at 80)
- Add an option to enable SSL for most examples
- Use ChannelFuture.sync() instead of await()
- Use System.out for the actual result. Use System.err otherwise.
- Delete examples that are not very useful:
  - applet
  - websocket/html5
  - websocketx/sslserver
  - localecho/multithreaded
- Add run-example.sh which simplifies launching an example from command
  line
- Rewrite FileServer example

Result:

Shorter and simpler examples.  A user can focus more on what it actually
does than miscellaneous stuff.  A user can launch an example very
easily.
2014-05-23 16:58:47 +09:00
Trustin Lee
47f961b6b9 Fix a bug in DefaultPromise.notifyLateListener() where the listener is not notified
Motivation:

When (listeners == null && lateListeners == null) and (stackDepth >= MAX_LISTENER_STACK_DEPTH), the listener is not notified at all. The discard client does not work.

Modification:

Make sure to submit the notification task.

Result:

The discard client works again and all listeners are notified.
2014-05-23 09:47:22 +09:00
Trustin Lee
53e35480f0 Fix a regression introduced by 00853d9453
.. by fixing incorrect bitwise operation for pipeline traversal
2014-05-23 01:15:41 +09:00
Trustin Lee
e23a688c63 Fix packagine issues in all-in-one JAR
Motivation:

- dependencyVersionsDir property is not resolved during the build
  process. The build doesn't fail because of this, but it creates an
  ugly directory.
- All-in-one JAR contains libnetty-tcnative.so, which is not part of the
  all-in-one JAR.

Modifications:

- Fix an incorrect property name
  (dependencyVersionDir -> dependencyVersionsDir)
- Exclude libnetty-tcnative.so
- Remove unnecessary includes in source expanding configuration

Result:

- Cleaner pom.xml
- We do not ship libnetty-tcnative.so in all-in-one JAR anymore, which
  is correct, because strictly speaking the native library belongs to
  org.apache.tomcat.jni package.
2014-05-22 19:11:01 +09:00
Trustin Lee
3e8c80a26f Use maven-antrun-plugin instead of exec-maven-plugin
Motivation:

exec-maven-plugin does not flush stdout and stderr, making the console
output from the examples invisible to users

Modification:

Use maven-antrun-plugin instead

Result:

A user sees the output from the examples immediately.
2014-05-22 18:53:09 +09:00
Trustin Lee
b58737b898 Fix an incorrect path problem in run-example.sh 2014-05-22 17:58:08 +09:00
Trustin Lee
a99371a4a0 Display example names in 2 columns
- ported from ba28679775
2014-05-22 17:54:31 +09:00
Trustin Lee
00853d9453 Fix an event ordering issue
Motivation:

In the early days of 5.0, we merged ChannelInboundHandler and
ChannelOutboundHandler into ChannelHandler, and introduced the
annotation called 'Skip'.  The annotation 'Skip' was introduced to
determine which handler methods are no-op (i.e. simply forwarding the
event to the next handler) so that DefaultChannelHandlerContext doesn't
even need to submit an event-invoking task to an EventExecutor,
significantly reducing the context switches.

However, this introduced a regression for the handlers which implemented
write() but not flush(). Because flush() was skippable for such
handlers, flush() event went through to the next handler before write() does.

To address this problem, we came up with a naive workaround that sets
MASK_FLUSH when MASK_WRITE is set.

Although the previous workaround works fine for many cases, we still
seem to have an event ordering problem.  We keep seeing the intermittant
failure of LocalTransportThreadModelTest.testStagedExecution(), because
other handler methods are still skipped.

Modifications:

We do not skip the execution of handler methods annotated with 'Skip'
unless all inbound methods (or all outbound methods) are marked with
'Skip'.

Result:

This change Brings back the event ordering behavior of 4.x, making
LocalTransportThreadModelTest.testStagedExecution() pass.
2014-05-22 11:14:36 +09:00
Trustin Lee
d803a75e2f Fix NPE 2014-05-22 10:28:38 +09:00
Trustin Lee
af3b358892 Checkstyle 2014-05-22 10:14:42 +09:00
Trustin Lee
cb66866730 Future compatibility with TLS ALPN
Motivation:

According to TLS ALPN draft-05, a client sends the list of the supported
protocols and a server responds with the selected protocol, which is
different from NPN.  Therefore, ApplicationProtocolSelector won't work
with ALPN

Modifications:

- Use Iterable<String> to list the supported protocols on the client
  side, rather than using ApplicationProtocolSelector
- Remove ApplicationProtocolSelector

Result:

Future compatibility with TLS ALPN
2014-05-22 09:59:58 +09:00
Nathan Mittler
2fb7af0163 Merge pull request #2496 from nmittler/http2
Adding a general handler for upgrading protocols
2014-05-21 09:58:21 -07:00
nmittler
054e4c5233 Merge branch 'master' of https://github.com/netty/netty into http2
Conflicts:
	example/src/main/java/io/netty/example/http2/client/Http2Client.java
	example/src/main/java/io/netty/example/http2/client/Http2ClientInitializer.java
	example/src/main/java/io/netty/example/http2/server/Http2Server.java
	example/src/main/java/io/netty/example/http2/server/Http2ServerInitializer.java
2014-05-21 07:44:18 -07:00
Trustin Lee
46ed3d4c97 Fix NPE in OpenSslEngine 2014-05-21 20:02:15 +09:00
Trustin Lee
cf3e4e7043 Escape a colon in protocol names 2014-05-21 17:45:28 +09:00
Trustin Lee
ddb59fbc01 Update the instruction for running SPDY examples 2014-05-21 17:29:50 +09:00
Trustin Lee
e167ec51eb Add unified NextProtoNego extension support to SslContext
Motivation:

- OpenSslEngine and JDK SSLEngine (+ Jetty NPN) have different APIs to
  support NextProtoNego extension.
  - It is impossible to configure NPN with SslContext when the provider
    type is JDK.

Modification:

- Implement NextProtoNego extension by overriding the behavior of
  SSLSession.getProtocol() for both OpenSSLEngine and JDK SSLEngine.
  - SSLEngine.getProtocol() returns a string delimited by a colon (':')
    where the first component is the transport protosol (e.g. TLSv1.2)
    and the second component is the name of the application protocol
- Remove the direct reference of Jetty NPN classes from the examples
- Add SslContext.newApplicationProtocolSelector

Result:

- A user can now use both JDK SSLEngine and OpenSslEngine for NPN-based
  protocols such as HTTP2 and SPDY
2014-05-21 17:21:18 +09:00
Trustin Lee
3051c1ee0f Bash 3 compatibility
Motivation:

Mac OS X ships Bash 3, and it does not have an associative array
(declare -A).

Modifications:

Do not use an associative array.

Result:

Can run examples on Mac OS X using run-example.sh
2014-05-21 13:35:28 +09:00
Trustin Lee
579973e35a Fix resource leak in DefaultHttp2FrameIOTest 2014-05-21 12:54:36 +09:00
Trustin Lee
50ae950203 Improve run-example.sh
- More usage example
- Newlines for prettier output
2014-05-20 23:39:43 +09:00
Trustin Lee
72ccf83861 Clean up the execution mechanism of examples
Motivation:

- There's no way to pass an argument to an example.
- Assigning a Maven profile for each example is an overkill.
  It makes the pom.xml crowded.

Modifications:

- Remove example profiles from example/pom.xml
- Keep the list of examples in run-example.sh
- run-example.sh passes all options to exec-maven-plugin.
  For example, we can now do this:

    ./run-example.sh -Dssl -Dport=443 http-server

Result:

- It's much easier to add a new example and provide an easy way to
  launch it.
- We can still pass an arbitrary argument to the example being launched.
  (I'll update all examples to make them get their options from system
  properties rather than from args[].
2014-05-20 23:24:34 +09:00
Trustin Lee
66d969b453 Fix a build problem with JDK 8
Motivation:

Build fails with JDK 8 because npn-boot does not work with JDK 8

Modifications:

Do not specify bootclasspath when on JDK 8

Result:

Build is green again.
2014-05-20 22:37:55 +09:00
Trustin Lee
59f3d550fe Fix a problem where all classes are compiled again
Motivation:

Due to a known problem[1] of maven-compiler-plugin, our build always
compiles everything from scratch, which is waste of time.

Modifications:

Exclude package-info.java from the source list.

Result:

Much shorter build time.

[1]: https://jira.codehaus.org/browse/MCOMPILER-205
2014-05-20 20:04:41 +09:00
Trustin Lee
b27fd634f5 Clean up example execution
Motivation:

- example/pom.xml has quite a bit of duplication.
- We expect that we depend on npn-boot in more than one module in the
  near future. (e.g. handler, codec-http, and codec-http2)

Modification:

- Deduplicate the profiles in example/pom.xml
- Move the build configuration related with npn-boot to the parent pom.
- Add run-example.sh that helps a user launch an example easily

Result:

- Cleaner build files
- Easier to add a new example
- Easier to launch an example
- Easier to run the tests that relies on npn-boot in the future
2014-05-20 19:46:19 +09:00
Trustin Lee
087e95e899 Checkstyle 2014-05-20 17:38:52 +09:00
Trustin Lee
5c6c8da0c9 Limit the number of bytes to copy per Unsafe.copyMemory()
Motivation:

During a large memory copy, safepoint polling is diabled, hindering
accurate profiling.

Modifications:

Only copy up to 1 MiB per Unsafe.copyMemory()

Result:

Potentially more reliable performance
2014-05-20 17:16:15 +09:00
nmittler
086dc9140a Adding a general handler for upgrading protocols
Adding full implementation for cleartext upgrade from HTTP to HTTP/2.
2014-05-19 07:30:18 -07:00
Trustin Lee
108dc23cab Work around the JVM crash that occurs when cipher suite uses GCM
Motivation:

For an unknown reason, JVM of JDK8 crashes intermittently when
SslHandler feeds a direct buffer to SSLEngine.unwrap() *and* the current
cipher suite has GCM (Galois/Counter Mode) enabled.

Modifications:

Convert the inbound network buffer to a heap buffer when the current
cipher suite is using GCM.

Result:

JVM does not crash anymore.
2014-05-19 11:44:50 +09:00
Trustin Lee
46c79422ab Update os-maven-plugin again to address an IDEA integration issue 2014-05-19 01:32:31 +09:00
Trustin Lee
c99939628c Add JVM crash logs to .gitignore 2014-05-18 21:34:51 +09:00
Trustin Lee
998afe09bf Upgrade os-maven-plugin to fix the build issue 2014-05-18 17:30:29 +09:00
Trustin Lee
3f4c2bbaf0 Reduce memory usage of SslHandler when OpenSslEngine is in use
Motivation:

JDK's SSLEngine.wrap() requires the output buffer to be always as large as MAX_ENCRYPTED_PACKET_LENGTH even if the input buffer contains small number of bytes.  Our OpenSslEngine implementation does not have such wasteful behaviot.

Modifications:

If the current SSLEngine is OpenSslEngine, allocate as much as only needed.

Result:

Less peak memory usage.
2014-05-18 05:09:22 +09:00
Trustin Lee
45457645d9 Add netty-tcnative dependency to netty-example
Motivation:

It's useful to have netty-tcnative dependency in netty-example because
we can play with OpenSslEngine from our IDE.

Modifications:

Add netty-tcnative to example/pom.xml
2014-05-18 04:27:43 +09:00
Trustin Lee
97a4277be1 Fix buffer leaks during PEM to KeyStore conversion 2014-05-18 04:26:16 +09:00
Trustin Lee
44699afd79 Fix JDK 8 compatibility issue with OpenJdkSelfSignedCertGenerator
- X509CertInfo.setSubject/setIssuer() requires X500Name instead of
  CertificateSubjectName/CertificateIssuerName.
2014-05-18 03:45:58 +09:00
Trustin Lee
1095e4486f Optimize SslHandler in an OpenSslEngine-friendly way
Motivation:

Previous fix for the OpenSslEngine compatibility issue (#2216 and
18b0e95659) was to feed SSL records one by
one to OpenSslEngine.unwrap().  It is not optimal because it will result
in more JNI calls.

Modifications:

- Do not feed SSL records one by one.
- Feed as many records as possible up to MAX_ENCRYPTED_PACKET_LENGTH
- Deduplicate MAX_ENCRYPTED_PACKET_LENGTH definitions

Result:

- No allocation of intemediary arrays
- Reduced number of calls to SSLEngine and thus its underlying JNI calls
- A tad bit increase in throughput, probably reverting the tiny drop
  caused by 18b0e95659
2014-05-18 03:34:12 +09:00
Trustin Lee
a72230061d Add an OpenSslEngine and the universal API for enabling SSL
Motivation:

Some users already use an SSLEngine implementation in finagle-native. It
wraps OpenSSL to get higher SSL performance.  However, to take advantage
of it, finagle-native must be compiled manually, and it means we cannot
pull it in as a dependency and thus we cannot test our SslHandler
against the OpenSSL-based SSLEngine.  For an instance, we had #2216.

Because the construction procedures of JDK SSLEngine and OpenSslEngine
are very different from each other, we also need to provide a universal
way to enable SSL in a Netty application.

Modifications:

- Pull netty-tcnative in as an optional dependency.
  http://netty.io/wiki/forked-tomcat-native.html
- Backport NativeLibraryLoader from 4.0
- Move OpenSSL-based SSLEngine implementation into our code base.
  - Copied from finagle-native; originally written by @jpinner et al.
  - Overall cleanup by @trustin.
- Run all SslHandler tests with both default SSLEngine and OpenSslEngine
- Add a unified API for creating an SSL context
  - SslContext allows you to create a new SSLEngine or a new SslHandler
    with your PKCS#8 key and X.509 certificate chain.
  - Add JdkSslContext and its subclasses
  - Add OpenSslServerContext
- Add ApplicationProtocolSelector to ensure the future support for NPN
  (NextProtoNego) and ALPN (Application Layer Protocol Negotiation) on
  the client-side.
- Add SimpleTrustManagerFactory to help a user write a
  TrustManagerFactory easily, which should be useful for those who need
  to write an alternative verification mechanism. For example, we can
  use it to implement an unsafe TrustManagerFactory that accepts
  self-signed certificates for testing purposes.
- Add InsecureTrustManagerFactory and FingerprintTrustManager for quick
  and dirty testing
- Add SelfSignedCertificate class which generates a self-signed X.509
  certificate very easily.
- Update all our examples to use SslContext.newClient/ServerContext()
- SslHandler now logs the chosen cipher suite when handshake is
  finished.

Result:

- Cleaner unified API for configuring an SSL client and an SSL server
  regardless of its internal implementation.
- When native libraries are available, OpenSSL-based SSLEngine
  implementation is selected automatically to take advantage of its
  performance benefit.
- Examples take advantage of this modification and thus are cleaner.
2014-05-18 02:33:26 +09:00
Trustin Lee
942db3aa23 Preparation for porting OpenSSL support in 3.10
- Add licenses and dependencies
2014-05-17 20:01:09 +09:00
Norman Maurer
c3bd7a8ff1 Better implementation of AttributeMap and also add hasAttr(...). See [#2439]
Motivation:
The old DefaultAttributeMap impl did more synchronization then needed and also did not expose a efficient way to check if an attribute exists with a specific key.

Modifications:
* Rewrite DefaultAttributeMap to not use IdentityHashMap and synchronization on the map directly. The new impl uses a combination of AtomicReferenceArray and synchronization per chain (linked-list). Also access the first Attribute per bucket can be done without any synchronization at all and just uses atomic operations. This should fit for most use-cases pretty weel.
* Add hasAttr(...) implementation

Result:
It's now possible to check for the existence of a attribute without create one. Synchronization is per linked-list and the first entry can even be added via atomic operation.
2014-05-15 06:47:58 +02:00
Norman Maurer
7bf7c7b16a Correctly release buffer when testing DelegatingHttp2ConnectionHandler 2014-05-14 06:19:12 +02:00
Jake Luciani
856c89dd70 Fix capacity check bug affecting offheap buffers 2014-05-13 07:25:26 +02:00
Norman Maurer
4a92ed1e0c Fix buffer leaks in HTTP2 2014-05-13 06:50:38 +02:00