Commit Graph

63 Commits

Author SHA1 Message Date
田欧
e8efcd82a8 migrate java8: use requireNonNull (#8840)
Motivation:

We can just use Objects.requireNonNull(...) as a replacement for ObjectUtil.checkNotNull(....)

Modifications:

- Use Objects.requireNonNull(...)

Result:

Less code to maintain.
2019-02-04 10:32:25 +01:00
田欧
6222101924 migrate java8: use lambda and method reference (#8781)
Motivation:

We can use lambdas now as we use Java8.

Modification:

use lambda function for all package, #8751 only migrate transport package.

Result:

Code cleanup.
2019-01-29 14:06:05 +01:00
田欧
9d62deeb6f Java 8 migration: Use diamond operator (#8749)
Motivation:

We can use the diamond operator these days.

Modification:

Use diamond operator whenever possible.

Result:

More modern code and less boiler-plate.
2019-01-22 16:07:26 +01:00
Norman Maurer
4a82d107d2
Require Java8 as minimum (#8739)
Motivation:

While we are not yet quite sure if we want to require Java11 as minimum we are at least sure we want to use java8 as minimum.

Modifications:

Change minimum version to java8 and update some tests which failed compilation after this change.

Result:

Use Java8 as minimum and be able to use Java8 features.
2019-01-22 08:48:18 +01:00
Norman Maurer
d9a6cf341c
Remove support for marking reader and writerIndex in ByteBuf to reduce overhead and complexity. (#8636)
Motivation:

ByteBuf supports “marker indexes”. The intended use case for these is if a speculative operation (e.g. decode) is in process the user can “mark” and interface and refer to it later if the operation isn’t successful (e.g. not enough data). However this is rarely used in practice,
requires extra memory to maintain, and introduces complexity in the state management for derived/pooled buffer initialization, resizing, and other operations which may modify reader/writer indexes.

Modifications:

Remove support for marking and adjust testcases / code.

Result:

Fixes https://github.com/netty/netty/issues/8535.
2018-12-11 14:00:49 +01:00
Nick Hill
583d838f7c Optimize AbstractByteBuf.getCharSequence() in US_ASCII case (#8392)
* Optimize AbstractByteBuf.getCharSequence() in US_ASCII case

Motivation:

Inspired by https://github.com/netty/netty/pull/8388, I noticed this
simple optimization to avoid char[] allocation (also suggested in a TODO
here).

Modifications:

Return an AsciiString from AbstractByteBuf.getCharSequence() if
requested charset is US_ASCII or ISO_8859_1 (latter thanks to
@Scottmitch's suggestion). Also tweak unit tests not to require Strings
and include a new benchmark to demonstrate the speedup.

Result:

Speed-up of AbstractByteBuf.getCharSequence() in ascii and iso 8859/1
cases
2018-10-26 15:32:38 -07:00
Björn Kautler
1b6e47ab2b Fix incorrectly encoded empty SOCKS5 address (#8292)
Motivation:

If you encode a SOCKS5 message like new DefaultSocks5CommandResponse(FAILURE, DOMAIN, "", 0)
you correctly get a result of 05010003000000.
But if the bndAddr is null, for example like new DefaultSocks5CommandResponse(FAILURE, DOMAIN)
the encoded result is 0501000301000000 which means the domain name has a length of one and consists of a 0-byte.

Modification:

With this commit it is also correctly encoded as a string of 0 length.

Result:

Correctly encode empty SOCKS5 address
2018-09-18 14:20:28 -07:00
Jason Tedor
3fe1f71511 Do not treat errors as decoder exception (redux)
Motivation: Today when Netty encounters a general error while decoding
it treats this as a decoder exception. However, for fatal causes this
should not be treated as such, instead the fatal error should be carried
up the stack without the callee having to unwind causes. This was
probably done for byte to byte message decoder but is now done for all
decoders.

Modifications: Instead of translating any error to a decoder exception,
we let those unwind out the stack (note that finally blocks still
execute) except in places where an event needs to fire where we fire
with the error instead of wrapping in a decoder exception.

Result: Fatal errors will not be treated as innocent decoder exceptions.
2017-10-07 18:17:00 +02:00
Norman Maurer
f5bea11ee4 DefaultSocks5CommandRequest incorrectly rejects SOCKS5 commands with dstPort=0
Motivation:

According to SOCKS 5 spec, dstPort = 0 is a valid value in case of UDP ASSOCIATE.

Modifications:

- Allow 0 as port.
- Add unit tests.

Result:

Fixes [#7156].
2017-08-29 15:03:09 +02:00
Nikolay Fedorovskikh
8d0e0922a5 SocksCmdRequest and SocksCmdResponse are trying to convert host from IDN for the non-DOMAIN address types
Motivation:

In the SocksCmdRequest and SocksCmdResponse constructors a host param converts from IDN to ascii compatible form regardless address type.

Modifications:

Use `IDN#toASCII` only for `DOMAIN` address type.

Result:

More correct host handling in socks commands.
2017-06-28 20:34:45 +02:00
Nikolay Fedorovskikh
ba3616da3e Apply appropriate methods for writing CharSequence into ByteBuf
Motivation:

1. `ByteBuf` contains methods to writing `CharSequence` which optimized for UTF-8 and ASCII encodings. We can also apply optimization for ISO-8859-1.
2. In many places appropriate methods are not used.

Modifications:

1. Apply optimization for ISO-8859-1 encoding in the `ByteBuf#setCharSequence` realizations.
2. Apply appropriate methods for writing `CharSequences` into buffers.

Result:

Reduce overhead from string-to-bytes conversion.
2017-06-27 07:58:39 +02:00
Nikolay Fedorovskikh
c3bd1245c5 SocksCommonUtils cleanup
Motivation:

1. Internal class `SocksCommonUtils` contains a method `intToIp` that also exists in the `NetUtil`.
2. A `SocksCommonUtils#ipv6toCompressedForm` is never used.

Modifications:

1. Replace `intToIp` method usage with `NetUtil#intToIpAddress`.
2. Remove unused methods from `SocksCommonUtils`.

Result:

Less code for supports.
2017-06-23 19:04:03 +02:00
Nikolay Fedorovskikh
0692bf1b6a fix the typos 2017-04-20 04:56:09 +02:00
Nikolay Fedorovskikh
2993760e92 Fix misordered 'assertEquals' arguments in tests
Motivation:

Wrong argument order in some 'assertEquals' applying.

Modifications:

Flip compared arguments.

Result:

Correct `assertEquals` usage.
2017-03-08 22:48:37 -08:00
Nikolay Fedorovskikh
f49bf4b201 Convert fields to the local variable when possible
Motivation:

Some classes have fields which can be local.

Modifications:

Convert fields to the local variable when possible.

Result:

Clean up. More chances for young generation or scalar replacement.
2017-03-08 17:09:17 -08:00
Tim Brooks
3344cd21ac Wrap operations requiring SocketPermission with doPrivileged blocks
Motivation:

Currently Netty does not wrap socket connect, bind, or accept
operations in doPrivileged blocks. Nor does it wrap cases where a dns
lookup might happen.

This prevents an application utilizing the SecurityManager from
isolating SocketPermissions to Netty.

Modifications:

I have introduced a class (SocketUtils) that wraps operations
requiring SocketPermissions in doPrivileged blocks.

Result:

A user of Netty can grant SocketPermissions explicitly to the Netty
jar, without granting it to the rest of their application.
2017-01-19 21:12:52 +01:00
Norman Maurer
ae1234c303 Replace internal class usage to make it compile on java9. This was missed in f332a00. 2016-12-03 21:01:36 +01:00
Scott Mitchell
506ac2ca71 NetUtil.bytesToIpAddress bug
Motivation:
NetUtil.bytesToIpAddress does not correctly translate IPv4 address to String. Also IPv6 addresses may not follow minimization conventions when converting to a String (see rfc 5952).

Modifications:
- NetUtil.bytesToIpAddress should correctly handle negative byte values for IPv4
- NetUtil.bytesToIpAddress should leverage existing to string conversion code in NetUtil

Result:
Fixes https://github.com/netty/netty/issues/5821
2016-09-22 17:06:21 -07:00
Trustin Lee
3a9f472161 Make retained derived buffers recyclable
Related: #4333 #4421 #5128

Motivation:

slice(), duplicate() and readSlice() currently create a non-recyclable
derived buffer instance. Under heavy load, an application that creates a
lot of derived buffers can put the garbage collector under pressure.

Modifications:

- Add the following methods which creates a non-recyclable derived buffer
  - retainedSlice()
  - retainedDuplicate()
  - readRetainedSlice()
- Add the new recyclable derived buffer implementations, which has its
  own reference count value
- Add ByteBufHolder.retainedDuplicate()
- Add ByteBufHolder.replace(ByteBuf) so that..
  - a user can replace the content of the holder in a consistent way
  - copy/duplicate/retainedDuplicate() can delegate the holder
    construction to replace(ByteBuf)
- Use retainedDuplicate() and retainedSlice() wherever possible
- Miscellaneous:
  - Rename DuplicateByteBufTest to DuplicatedByteBufTest (missing 'D')
  - Make ReplayingDecoderByteBuf.reject() return an exception instead of
    throwing it so that its callers don't need to add dummy return
    statement

Result:

Derived buffers are now recycled when created via retainedSlice() and
retainedDuplicate() and derived from a pooled buffer
2016-05-17 11:16:13 +02:00
Norman Maurer
d081851156 Remove ByteBuf.readBytes(int) calls when possible
Motivation:

We use ByteBuf.readBytes(int) in various places where we could either remove it completely or use readSlice(int).retain().

Modifications:

- Remove ByteBuf.readBytes(int) when possible or replace by readSlice(int).retain().

Result:

Faster code.
2016-04-09 18:40:57 +02:00
Xiaoyan Lin
e2d4e22243 Add CharsetUtil.encoder/decoder() methods
Motivation:

See #3321

Modifications:

1. Add CharsetUtil.encoder/decoder() methods
2. Deprecate CharsetUtil.getEncoder/getDecoder() methods

Result:

Users can use new CharsetUtil.encoder/decoder() to specify error actions
2016-03-07 10:48:31 +00:00
Trustin Lee
1d061bbb27 Hide password in exception messages of SocksAuthRequest
Related: #3504

Motivation:

There are two places in the SocksAuthRequest constructor where an
IllegalArgumentException is thrown with a password as part of the
exception message.

This constitutes mishandling of confidential information, which can
compromise user privacy and is flagged as critical by security scanners.

Modifications:

Mask the password in the exception messages

Result:

No unexpected password leak
2015-03-17 17:25:09 +09:00
Trustin Lee
7aeafb9c09 Fixed buffer leaks in DefaultSocks5CommandResponseTest 2015-02-20 16:54:26 +09:00
Trustin Lee
976db9269d Revamp io.netty.handler.codec.socksx
While implementing netty-handler-proxy, I realized various issues in our
current socksx package. Here's the list of the modifications and their
background:

- Split message types into interfaces and default implementations
  - so that a user can implement an alternative message implementations
- Use classes instead of enums when a user might want to define a new
  constant
  - so that a user can extend SOCKS5 protocol, such as:
    - defining a new error code
    - defining a new address type
- Rename the message classes
  - to avoid abbreviated class names. e.g:
    - Cmd -> Command
    - Init -> Initial
  - so that the class names align better with the protocol
    specifications. e.g:
    - AuthRequest -> PasswordAuthRequest
    - AuthScheme -> AuthMethod
- Rename the property names of the messages
  - so that the property names align better when the field names in the
    protocol specifications
- Improve the decoder implementations
  - Give a user more control over when a decoder has to be removed
  - Use DecoderResult and DecoderResultProvider to handle decode failure
    gracefully. i.e. no more Unknown* message classes
- Add SocksPortUnifinicationServerHandler since it's useful to the users
  who write a SOCKS server
  - Cleaned up and moved from the socksproxy example
2015-02-10 09:14:13 +09:00
Trustin Lee
2309a75d15 Add proxy support for client socket connections
Related issue: #1133

Motivation:

There is no support for client socket connections via a proxy server in
Netty.

Modifications:

- Add a new module 'handler-proxy'
- Add ProxyHandler and its subclasses to support SOCKS 4a/5 and HTTP(S)
  proxy connections
- Add a full parameterized test for most scenarios
- Clean up pom.xml

Result:

A user can make an outgoing connection via proxy servers with only
trivial effort.
2014-10-14 12:29:08 +09:00
Trustin Lee
0b935b85ce Fix an incorrect use of ByteBuf.array() in Socks5CmdRequestDecoder
Motivation:

Socks5CmdRequestDecoder uses ByteBuf.array() naively assuming that the
array's base offset is always 0, which is not the case.

Modification:

- Allocate a new byte array and copy the content there instead

Result:

Another bug fixed
2014-10-14 12:29:08 +09:00
Norman Maurer
56fd33df23 codec-socks test cleanup
Motivation:

The unit tests in codec-socks contained redundant casts and empty test classes.

Modifications:

- Remove redundant casts
- Delete empty test classes

Result:

Cleanup
2014-08-20 06:53:37 +02:00
Trustin Lee
fc1429c00c Overall clean-up on socksx package
- SocksV[45] -> Socks[45]
- Make encodeAsByteBuf package private with some hassle
- Split SocksMessageEncoder into Socks4MessageEncoder and
  Socks5MessageEncoder, and remove the original
- Remove lazy singleton instantiation; we don't need it.
- Remove the deprecated methods
- Fix Javadoc errors
2014-08-14 16:41:04 -07:00
bk1te
1a05004523 Add socksx package which supports SOCKS 4/4a/5
Motivation:

SOCKS 4 and 5 are very different protocols although they share the same
name.  It is not possible to incorporate the two protocol versions into
a single package.

Modifications:

- Add a new package called 'socksx' to supercede 'socks' package.
- Add SOCKS 4/4a support to the 'socksx' package

Result:

codec-socks now supports all SOCKS versions
2014-08-14 16:40:46 -07:00
Trustin Lee
d0912f2709 Fix most inspector warnings
Motivation:

It's good to minimize potentially broken windows.

Modifications:

Fix most inspector warnings from our profile
Update IntObjectHashMap

Result:

Cleaner code
2014-07-02 19:55:07 +09:00
Trustin Lee
448b0105b4 Deprecate SocksMessage.encodeAsByteBuf()
It was an internal use only method which became public by a mistake
during the review process.
2014-06-24 16:40:44 +09:00
Trustin Lee
45fde9abba Rename fromByte() to valueOf()
Motivation:

Persuit the consistency in method naming

Modifications:

Rename fromByte(byte) to valueOf(byte)

Result:

Consistency
2014-06-24 16:34:52 +09:00
Trustin Lee
223efc5f99 Clean up the examples
Motivation:

The examples have not been updated since long time ago, showing various
issues fixed in this commit.

Modifications:

- Overall simplification to reduce LoC
  - Use system properties to get options instead of parsing args.
  - Minimize option validation
  - Just use System.out/err instead of Logger
  - Do not pass config as parameters - just access it directly
  - Move the main logic to main(String[]) instead of creating a new
    instance meaninglessly
    - Update netty-build-21 to make checkstyle not complain
  - Remove 'throws Exception' clause if possible
- Line wrap at 120 (previously at 80)
- Add an option to enable SSL for most examples
- Use ChannelFuture.sync() instead of await()
- Use System.out for the actual result. Use System.err otherwise.
- Delete examples that are not very useful:
  - applet
  - websocket/html5
  - websocketx/sslserver
  - localecho/multithreaded
- Add run-example.sh which simplifies launching an example from command
  line
- Rewrite FileServer example

Result:

Shorter and simpler examples.  A user can focus more on what it actually
does than miscellaneous stuff.  A user can launch an example very
easily.
2014-05-23 17:13:09 +09:00
Norman Maurer
3ba856a5f4 Correctly handle SocksCmdResponse. Related to #2428
Motivation:
Ports range check is not correct

Modification:
Allow port between 0 and 65535. 0 is wildcard / unknown port here

Result:
Correct validation
2014-04-30 10:44:17 +02:00
Norman Maurer
80f1dfaec0 [#2428] Proper fix of incorrect port range check
Motivation:

In the Internet Protocol, the valid port number range is from 1 to 65535
(inclusive on the both side.)  However, SocksCmdRequest and SocksCmdResponse
refuses to construct itself when the port number 65535 is specified. Beside
this it excepts 0 as port number which should not allowed.

    Modification:

* Not raise an exception when the specified port number is 65535.
* Raise an exception when the specified port number is 0

Result:

Fixes #2428
2014-04-30 08:17:40 +02:00
Trustin Lee
01ef614e77 Fix incorrect port range check
Motivation:

In the Internet Protocol, the valid port number range is from 1 to 65535
(inclusive on the both side.)  However, SocksCmdRequest refuses to
construct itself when the port number 65535 is specified.

Modification:

Do not raise an exception when the specified port number is 65535.

Result:

Fixes #2428
2014-04-29 17:58:08 +09:00
Trustin Lee
df346a023b Change the return type of EmbeddedChannel.read*() from Object to an ad-hoc type parameter
.. so that there's no need to explicitly down-cast.

Fixes #2067
2014-02-13 17:19:26 -08:00
Trustin Lee
c4c71e6d28 Fix the potential copyright issue in SocksCommonUtils
- Add StringUtil.toHexString() methods which are based on LoggingHandler's lookup table implementation, and use it wherever possible
2014-02-06 15:00:06 -08:00
Vladimir Schafer
147e08a30e #2177 Adding support for bound host and port for the SOCKS5 command response. Changes are fully backward compatible. 2014-02-04 19:28:35 +01:00
Trustin Lee
cbd8817905 Remove MessageList from public API and change ChannelInbound/OutboundHandler accordingly
I must admit MesageList was pain in the ass.  Instead of forcing a
handler always loop over the list of messages, this commit splits
messageReceived(ctx, list) into two event handlers:

- messageReceived(ctx, msg)
- mmessageReceivedLast(ctx)

When Netty reads one or more messages, messageReceived(ctx, msg) event
is triggered for each message.  Once the current read operation is
finished, messageReceivedLast() is triggered to tell the handler that
the last messageReceived() was the last message in the current batch.

Similarly, for outbound, write(ctx, list) has been split into two:

- write(ctx, msg)
- flush(ctx, promise)

Instead of writing a list of message with a promise, a user is now
supposed to call write(msg) multiple times and then call flush() to
actually flush the buffered messages.

Please note that write() doesn't have a promise with it.  You must call
flush() to get notified on completion. (or you can use writeAndFlush())

Other changes:

- Because MessageList is completely hidden, codec framework uses
  List<Object> instead of MessageList as an output parameter.
2013-07-09 23:51:48 +09:00
Trustin Lee
14158070bf Revamp the core API to reduce memory footprint and consumption
The API changes made so far turned out to increase the memory footprint
and consumption while our intention was actually decreasing them.

Memory consumption issue:

When there are many connections which does not exchange data frequently,
the old Netty 4 API spent a lot more memory than 3 because it always
allocates per-handler buffer for each connection unless otherwise
explicitly stated by a user.  In a usual real world load, a client
doesn't always send requests without pausing, so the idea of having a
buffer whose life cycle if bound to the life cycle of a connection
didn't work as expected.

Memory footprint issue:

The old Netty 4 API decreased overall memory footprint by a great deal
in many cases.  It was mainly because the old Netty 4 API did not
allocate a new buffer and event object for each read.  Instead, it
created a new buffer for each handler in a pipeline.  This works pretty
well as long as the number of handlers in a pipeline is only a few.
However, for a highly modular application with many handlers which
handles connections which lasts for relatively short period, it actually
makes the memory footprint issue much worse.

Changes:

All in all, this is about retaining all the good changes we made in 4 so
far such as better thread model and going back to the way how we dealt
with message events in 3.

To fix the memory consumption/footprint issue mentioned above, we made a
hard decision to break the backward compatibility again with the
following changes:

- Remove MessageBuf
- Merge Buf into ByteBuf
- Merge ChannelInboundByte/MessageHandler and ChannelStateHandler into ChannelInboundHandler
  - Similar changes were made to the adapter classes
- Merge ChannelOutboundByte/MessageHandler and ChannelOperationHandler into ChannelOutboundHandler
  - Similar changes were made to the adapter classes
- Introduce MessageList which is similar to `MessageEvent` in Netty 3
- Replace inboundBufferUpdated(ctx) with messageReceived(ctx, MessageList)
- Replace flush(ctx, promise) with write(ctx, MessageList, promise)
- Remove ByteToByteEncoder/Decoder/Codec
  - Replaced by MessageToByteEncoder<ByteBuf>, ByteToMessageDecoder<ByteBuf>, and ByteMessageCodec<ByteBuf>
- Merge EmbeddedByteChannel and EmbeddedMessageChannel into EmbeddedChannel
- Add SimpleChannelInboundHandler which is sometimes more useful than
  ChannelInboundHandlerAdapter
- Bring back Channel.isWritable() from Netty 3
- Add ChannelInboundHandler.channelWritabilityChanges() event
- Add RecvByteBufAllocator configuration property
  - Similar to ReceiveBufferSizePredictor in Netty 3
  - Some existing configuration properties such as
    DatagramChannelConfig.receivePacketSize is gone now.
- Remove suspend/resumeIntermediaryDeallocation() in ByteBuf

This change would have been impossible without @normanmaurer's help. He
fixed, ported, and improved many parts of the changes.
2013-06-10 16:10:39 +09:00
Andrei Pozolotin
a3e760a003 fix #1234 - duplicate package-info.java errors in eclipse requires release of netty-build v 19 and netty-parent update. 2013-04-05 05:38:05 +09:00
Norman Maurer
af4b71a00e Remove special handling of Object[] in codec framework (a.k.a unfolding)
- Fixes #1229
- Primarily written by @normanmaurer and revised by @trustin

This commit removes the notion of unfolding from the codec framework
completely.  Unfolding was introduced in Netty 3.x to work around the
shortcoming of the codec framework where encode() and decode() did not
allow generating multiple messages.

Such a shortcoming can be fixed by changing the signature of encode()
and decode() instead of introducing an obscure workaround like
unfolding.  Therefore, we changed the signature of them in 4.0.

The change is simple, but backward-incompatible.  encode() and decode()
do not return anything.  Instead, the codec framework will pass a
MessageBuf<Object> so encode() and decode() can add the generated
messages into the MessageBuf.
2013-04-03 21:44:54 +09:00
Prajwal Tuladhar
05850da863 enable checkstyle for test source directory and fix checkstyle errors 2013-03-30 13:18:57 +01:00
Trustin Lee
f67441354a Move logging classes from internal to internal.logging
.. because internal is crowded
2013-02-26 14:54:25 -08:00
Trustin Lee
1011227b88 Remove apiviz tags - we are focusing on user guide instead and putting diagrams there 2013-02-14 12:09:16 -08:00
Trustin Lee
5218f38ed0 Use InternalLogger in our code / Use jul in examples 2013-02-11 20:37:31 +09:00
Trustin Lee
a2e5cd94be Prettify APIviz / Tighten visibility / Move subclasses to top level / Remove unused UnknownSocksMessage 2013-02-11 19:42:23 +09:00
Trustin Lee
82c46180c9 Tighten access modifier of encode/decode() 2013-02-08 17:37:16 +09:00
Trustin Lee
76eb40a4d2 Make ChannelOutboundMessageHandlerAdapter similar to ChannelInboundMessageHandlerAdapter 2013-02-08 17:07:01 +09:00