Commit Graph

116 Commits

Author SHA1 Message Date
Trustin Lee
08646afc1e Do not fail a DNS query promise prematurely
Motivation:

DnsNameResolverContext completes its DNS query promise automatically
when no queries are in progress, which means there's no need to fail the
promise explicitly.

Modifications:

- Do not fail a DNS query promise explicitly but add an informational
  trace

Result:

- Fixes #6600
- Unexpected exception on one question type does not fail the promise
  too soon. If the other question succeeds, the query will succeed,
  making the resolver more robust.
2017-04-06 17:58:54 -07:00
Scott Mitchell
e074df2ae6 DNS Resolve ambiguity in which DNS servers are used during resolution
Motivation:
Recently DnsServerAddressStreamProvider was introduced to allow control for each query as to which DNS server should be used for resolution to respect the local host's default DNS server configuration. However resolver-dns also accepts a stream of DNS servers to use by default, but this stream is not host name aware. This creates an ambiguity as to which method is used to determine the DNS server to user during resolution, and in which order. We can remove this ambiguity and provide a more general API by just supporting DnsServerAddressStreamProvider.

Modifications:
- Remove the fixed DnsServerAddresses and instead only accept a DnsServerAddressStreamProvider.
- Add utility methods to help use DnsServerAddressStreamProvider for a single entry, a list of entries, and get the default for the current machine.

Result:
Fixes https://github.com/netty/netty/issues/6573.
2017-03-31 15:29:49 -07:00
Norman Maurer
2b8c8e0805 [maven-release-plugin] prepare for next development iteration 2017-03-10 07:46:17 +01:00
Norman Maurer
1db58ea980 [maven-release-plugin] prepare release netty-4.1.9.Final 2017-03-10 07:45:28 +01:00
Norman Maurer
fbf0e5f4dd Prefer JDK ThreadLocalRandom implementation over ours.
Motivation:

We have our own ThreadLocalRandom implementation to support older JDKs . That said we should prefer the JDK provided when running on JDK >= 7

Modification:

Using ThreadLocalRandom implementation of the JDK when possible.

Result:

Make use of JDK implementations when possible.
2017-02-16 15:44:00 -08:00
Scott Mitchell
9ce74d46c1 Correct unit test flaw introduced in 54c9ecf682
Motivation:
54c9ecf682 introduced a unit tests which attempted to exclude addresses which resolved to loop back addresses from an assert statement. This was done with a static check for localhost but depending on machine configuration it is possible for other interfaces to be resolved.

Modifications:
- Use InetAddress#isLoopbackAddress() instead of string match on localhost

Result:
DnsNameResolverTest#testNameServerCache is more reliable.
2017-02-13 18:36:06 -08:00
Scott Mitchell
54c9ecf682 DnsNameResolver should respect /etc/resolv.conf and /etc/resolver
Motivation:
The JDK uses gethostbyname for blocking hostname resoltuion. gethostbyname can be configured on Unix systems according to [1][2]. This may impact the name server that is used to resolve particular domains or just override the default fall-back resolver. DnsNameResolver currently ignores these configuration files which means the default resolution behavior is different than the JDK. This may lead to unexpected resolution failures which succeed when using the JDK's resolver.

Modifications:
- Add an interface which can override what DnsServerAddressStream to use for a given hostname
- Provide a Unix specific implementation of this interface and implement [1][2]. Some elements may be ignored sortlist, timeout, etc...

Result:
DnsNameResolver behaves more like the JDK resolver by default.

[1] https://linux.die.net/man/5/resolver
[2] https://developer.apple.com/legacy/library/documentation/Darwin/Reference/ManPages/man5/resolver.5.html
2017-02-13 11:54:09 -08:00
Norman Maurer
667cbe9923 Fix compilation error introduced by 81f9de423c 2017-02-11 09:17:13 +01:00
Stephane Landelle
81f9de423c HostsFileParser should allow both IPv4 and IPv6 for a given host
Motivation:

HostsFileParser only retains the first address for each given hostname.
This is wrong, and it’s allowed to have both an IPv4 and an IPv6.

Modifications:

* Have `HostsFileParser` now return a `HostsFileEntries` that contains IPv4 entries and IPv6 entries
* Introduce `ResolvedAddressTypes` to describe resolved address types preferences
* Add a new `ResolvedAddressTypes` parameter to `HostsFileEntriesResolver::address` to account for address types preferences
* Change `DnsNameResolver` constructor to take a `ResolvedAddressTypes`, allowing for a null value that would use default
* Change `DnsNameResolverBuilder::resolvedAddressTypes` to take a `ResolvedAddressTypes`
* Make `DnsNameResolver::resolvedAddressTypes` return a `ResolvedAddressTypes`
* Add a static `DnsNameResolverBuilder::computeResolvedAddressTypes` to ease converting from `InternetProtocolFamily`

Result:

We now support hosts files that contains IPv4 and IPv6 pairs for a same
hostname.
2017-02-10 20:09:32 -08:00
Scott Mitchell
007048dddd DnsNameResolver empty/null hostname missed by a416b79
Motivation:
a416b79 introduced a check for null or empty host name to be compatible with the JDK resolution. However the doResolve(String, Promise) method, and if the doResolve(String, DnsRecord[], Promise, DnsCache) method was overridden the empty/null hostname would not be correctly resolved.

Modifications:
- Move the empty/null host name check into the lowest level doResolve method in DnsNameResolver
- Remove the duplicate logic in InetNameResolver.java which can be bypassed anyways

Result:
By default (unless behavior is overridden) DnsNameResolver resolves null/empty host names to local host just like the JDK.
2017-02-09 09:22:27 -08:00
Norman Maurer
3462a86a3a Ensure we release the previous retained AddressedEnvelope when we fail to notify the promise.
Motivation:

We need to ensure we release the AddressedEnvelope if we fail to notify the future (as it may be notified before because of an timeout). Otherwise we may leak.

Modifications:

Call release() if we fail to notify the future.

Result:

No more memory leak on notify failure.
2017-02-09 10:11:07 +01:00
Norman Maurer
661ff2538e Implement correct handling of recursive DNS
Motivation:

DnsNameResolver does not handle recursive DNS and so fails if you query a DNS server (for example a ROOT dns server) which provides the correct redirect for a domain.

Modification:

Add support for redirects (a.k.a. handling of AUTHORITY section').

Result:

Its now possible to use a DNS server that redirects.
2017-02-06 20:33:52 +01:00
Norman Maurer
0d5b665fba Automatically decode DNS domain name to unicode
Motivation:

DnsNameResolver will return the domain / host name as ascii code using punycode (https://tools.ietf.org/html/rfc3492). This is different to what the JDK does which always convert it to unicode. We should do the same by default but allow to also not do it.

Modifications:

- Add new builder method on DnsNameResolverBuilder which allow to disable / enable converting. Default is to convert just like the JDK does.
- Add unit tests for it.

Result:

DnsNameResolver and JDK impl behave the same way.
2017-01-31 09:28:57 +01:00
Norman Maurer
735d6dd636 [maven-release-plugin] prepare for next development iteration 2017-01-30 15:14:02 +01:00
Norman Maurer
76e22e63f3 [maven-release-plugin] prepare release netty-4.1.8.Final 2017-01-30 15:12:36 +01:00
Norman Maurer
a416b79d86 DnsNameResolver.resolve*(...) never notifies the Future when empty hostname is used.
Motivation:

When an empty hostname is used in DnsNameResolver.resolve*(...) it will never notify the future / promise. The root cause is that we not correctly guard against errors of IDN.toASCII(...) which will throw an IllegalArgumentException when it can not parse its input. That said we should also handle an empty hostname the same way as the JDK does and just use "localhost" when this happens.

Modifications:

- If the try to resolve an empty hostname we use localhost
- Correctly guard against errors raised by IDN.toASCII(...) so we will always noify the future / promise
- Add unit test.

Result:

DnsNameResolver.resolve*(...) will always notify the future.
2017-01-24 21:21:49 +01:00
Tim Brooks
3344cd21ac Wrap operations requiring SocketPermission with doPrivileged blocks
Motivation:

Currently Netty does not wrap socket connect, bind, or accept
operations in doPrivileged blocks. Nor does it wrap cases where a dns
lookup might happen.

This prevents an application utilizing the SecurityManager from
isolating SocketPermissions to Netty.

Modifications:

I have introduced a class (SocketUtils) that wraps operations
requiring SocketPermissions in doPrivileged blocks.

Result:

A user of Netty can grant SocketPermissions explicitly to the Netty
jar, without granting it to the rest of their application.
2017-01-19 21:12:52 +01:00
Norman Maurer
ead87b7df8 Respect resolvedAddressTypes when follow CNAME records.
Motivation:

When we follow CNAME records we should respect resolvedAddressTypes and only query A / AAAA depending on which address types are expected.

Modifications:

Check if we should query A / AAAA when follow CNAMEs depending on resolvedAddressTypes.

Result:

Correct behaviour when follow CNAMEs.
2017-01-19 19:36:56 +01:00
Norman Maurer
d7ff71a3d1 Check if DnsCache is null in DnsNameResolver constructor.
Motivation:

We miss checking if DnsCache is null in DnsNameResolver constructor which will later then lead to a NPE. Better fail fast here.

Modifications:

Check for null and if so throw a NPE.

Result:

Fail fast.
2017-01-19 07:54:36 +01:00
Norman Maurer
7f01da8d0f [maven-release-plugin] prepare for next development iteration 2017-01-12 11:36:51 +01:00
Norman Maurer
7a21eb1178 [maven-release-plugin] prepare release netty-4.1.7.Final 2017-01-12 11:35:58 +01:00
Norman Maurer
89e93968ac Remove usage of own Atomic*FieldUpdater in favor of JDKs
Motivation:

In later Java8 versions our Atomic*FieldUpdater are slower then the JDK implementations so we should not use ours anymore. Even worse the JDK implementations provide for example an optimized version of addAndGet(...) using intrinsics which makes it a lot faster for this use-case.

Modifications:

- Remove methods that return our own Atomic*FieldUpdaters.
- Use the JDK implementations everywhere.

Result:

Faster code.
2016-12-15 08:09:06 +00:00
Norman Maurer
705e3f629a Not use InternalThreadLocalMap where access may be done from outside the EventLoop.
Motivation:

We should not use the InternalThreadLocalMap where access may be done from outside the EventLoop as this may create a lot of memory usage while not be reused anyway.

Modifications:

Not use InternalThreadLocalMap in places where the code-path will likely be executed from outside the EventLoop.

Result:

Less memory bloat.
2016-11-10 14:37:16 +01:00
Dmitry Spikhalskiy
eb7f8e4dc5 Expose RoundRobinInetAddressResolver
Motivation:
Make small refactoring for recently merged PR #5867 to make the code more flexible and expose aggressive round robin as a NameResolver too with proper code reuse.

Modifications:
Round robin is a method of hostname resolving - so Round robin related code fully moved to RoundRobinInetAddressResolver implements NameResolver<InetAddress>, RoundRobinInetSocketAddressResolver is deleted as a separate class, instance with the same functionality could be created by calling #asAddressResolver.

Result:
New forced Round Robin code exposed not only as an AddressResolver but as a NameResolver too, more proper code and semantic reusing of InetNameResolver and InetSocketAddressResolver classes.
2016-11-02 06:52:19 +01:00
Norman Maurer
5f533b7358 [maven-release-plugin] prepare for next development iteration 2016-10-14 13:20:41 +02:00
Norman Maurer
35fb0babe2 [maven-release-plugin] prepare release netty-4.1.6.Final 2016-10-14 12:47:19 +02:00
James Yuzawa
efd118ddec Support aggressive round-robin dns
Motivation:

Suppose the domain `foo.example.com` resolves to the following ip
addresses `10.0.0.1`, `10.0.0.2`, `10.0.0.3`. Round robin DNS works by
having each client probabilistically getting a different ordering of
the set of target IP’s, so connections from different clients (across
the world) would be split up across each of the addresses. Example: In
a `ChannelPool` to manage connections to `foo.example.com`, it may be
desirable for high QPS applications to spread the requests across all
available network addresses. Currently, Netty’s resolver would return
only the first address (`10.0.0.1`) to use. Let say we are making
dozens of connections. The name would be resolved to a single IP and
all of the connections would be made to `10.0.0.1`. The other two
addresses would not see any connections. (they may see it later if new
connections are made and `10.0.0.2` is the first in the list at that
time of a subsequent resolution). In these changes, I add support to
select a random one of the resolved addresses to use on each resolve
call, all while leveraging the existing caching and inflight request
detection. This way in my example, the connections would be make to
random selections of the resolved IP addresses.

Modifications:

I added another method `newAddressResolver` to
`DnsAddressResolverGroup` which can be overriden much like
`newNameResolver`. The current functionality which creates
`InetSocketAddressResolver` is still used. I added
`RoundRobinDnsAddressResolverGroup` which extends
DnsAddressResolverGroup and overrides the `newAddressResolver` method
to return a subclass of the `InetSocketAddressResolver`. This subclass
is called `RoundRobinInetSocketAddressResolver` and it contains logic
that takes a `resolve` request, does a `resolveAll` under the hood, and
returns a single element at random from the result of the `resolveAll`.

Result:

The existing functionality of `DnsAddressResolverGroup` is left
unchanged. All new functionality is in the
`RoundRobinInetSocketAddressResolver` which users will now have the
option to use.
2016-10-10 11:08:44 +02:00
Norman Maurer
dfa3bbbf00 Add support for Client Subnet in DNS Queries (RFC7871)
Motivation:

RFC7871 defines an extension which allows to request responses for a given subset.

Modifications:

- Add DnsOptPseudoRrRecord which can act as base class for extensions based on EDNS(0) as defined in RFC6891
- Add DnsOptEcsRecord to support the Client Subnet in DNS Queries extension
- Add tests

Result:

Client Subnet in DNS Queries extension is now supported.
2016-09-06 07:16:57 +02:00
Norman Maurer
54b1a100f4 [maven-release-plugin] prepare for next development iteration 2016-08-26 10:06:32 +02:00
Norman Maurer
1208b90f57 [maven-release-plugin] prepare release netty-4.1.5.Final 2016-08-26 04:59:35 +02:00
Trustin Lee
9fef4ba1bf Disable IPv6 address lookups when -Djava.net.preferIPv4Stack=true
Motivation:

According to the Oracle documentation:

> java.net.preferIPv4Stack (default: false)
>
> If IPv6 is available on the operating system, the underlying native
> socket will be an IPv6 socket. This allows Java applications to connect
> to, and accept connections from, both IPv4 and IPv6 hosts.
>
> If an application has a preference to only use IPv4 sockets, then this
> property can be set to true. The implication is that the application
> will not be able to communicate with IPv6 hosts.

which means, if DnsNameResolver returns an IPv6 address, a user (or
Netty) will not be able to connect to it.

Modifications:

- Move the code that retrieves java.net.prefer* properties from
  DnsNameResolver to NetUtil
- Add NetUtil.isIpV6AddressesPreferred()
- Revise the API documentation of NetUtil.isIpV*Preferred()
- Set the default resolveAddressTypes to IPv4 only when
  NetUtil.isIpv4StackPreferred() returns true

Result:

- Fixes #5657
2016-08-10 11:10:34 +02:00
Trustin Lee
b2f1ef57c8 Fix RejectedExecutionException when using DnsAddressResolverGroup
Motivation:

AddressResolverGroup adds a listener to the termination future of an
EventExecutor when a new AddressResolver is created. The listener calls
AddressResolver.close() when the EventExecutor is terminated to give the
AddressResolver a chance to release its resources.

When using DnsAddressResolverGroup, the AddressResolver.close() will
eventually trigger DnsNameResolver.close(), which closes its underlying
DatagramChannel.

DatagramChannel.close() (or any Channel.close()) will travel through
pipeline and trigger EventExecutor.execute() because
DnsNameResolver.close() has been invoked from a non-I/O thread.
(NB: A terminationFuture is always notified from the GlobalEventExecutor
thread.)

However, because we are doing this in the listener of the termination
future of the terminated EventLoop we are trying to execute a task upon,
the attempt to close the channel fails due to RejectedExecutionException.

Modifications:

- Do not call Channel.close() in DnsNameResolver.close() if the Channel
  has been closed by EventLoop already

Result:

No more RejectedExecutionException when shutting down an event loop.
2016-08-01 10:21:10 +02:00
alexlehm
ba80fbbe05 UnknownHostException mentions hostname with search domain added
Motivation:

When a hostname cannot be resolved, the message in the UnknownHostException mentions the hostname with the last attempted search domain appended, which is kind of confusing. I would prefer to see the original hostname supplied to the method in the exception.

Modifications:

Store the pristine hostname in the resolver context and use it to create the exception message instead of the hostname with search domain.
Add unit test to check that the exception does not mention the search domain.

Result:

The exception mentions the unmodified hostname in the message.
2016-08-01 07:20:19 +02:00
Norman Maurer
cb7cf4491c [maven-release-plugin] prepare for next development iteration 2016-07-27 13:29:56 +02:00
Norman Maurer
9466b32d05 [maven-release-plugin] prepare release netty-4.1.4.Final 2016-07-27 13:16:59 +02:00
Julien Viet
8d4cfd9002 Allow ndots=0 in DnsNameResolver and search domains - fixes #5570
Motivation:

The ndots = 0 is a valid value for ndots, it means that when using a non dotted name, the resolution should first try using a search and if it fails then use subdomains. Currently it is not allowed. Docker compose uses this when wiring up containers as names have usually no dots inside.

Modification:

Modify DnsNameResolver to accept ndots = 0 and handle the case in the resolution procedure. In this case a direct search is done and then a fallback on the search path is performed.

Result:

The ndots = 0 case is implemented.
2016-07-24 20:41:14 +02:00
alexlehm
73c0fb0e23 Construct LOCALHOST4 and LOCALHOST6 object with hostname "localhost"
Motivation:

When resolving localhost on Windows where the hosts file does not contain a localhost entry by default, the resulting InetAddress object returned by the resolver does not have the hostname set so that getHostName returns the ip address 127.0.0.1. This behaviour is inconsistent with Windows where the hosts file does contain a localhost entry and with Linux in any case. It breaks at least some unit tests.

Modifications:

Create the LOCALHOST4 and LOCALHOST6 objects with hostname localhost in addition to the address.
Add unit test domain localhost to DnsNameResolverTest to check the resolution of localhost with ipv4 at least.

Result:

The resolver returns a InetAddress object for localhost with the hostname localhost in all cases.
2016-07-20 05:55:45 +02:00
Norman Maurer
047f6aed28 [maven-release-plugin] prepare for next development iteration 2016-07-15 09:09:13 +02:00
Norman Maurer
b2adea87a0 [maven-release-plugin] prepare release netty-4.1.3.Final 2016-07-15 09:08:53 +02:00
Julien Viet
79c8ec4d33 DnsNameResolver search domains support
Motivation:

The current DnsNameResolver does not support search domains resolution. Search domains resolution is supported out of the box by the java.net resolver, making the DnsNameResolver not able to be a drop in replacement for io.netty.resolver.DefaultNameResolver.

Modifications:

The DnsNameResolverContext resolution has been modified to resolve a list of search path first when it is configured so. The resolve method now uses the following algorithm:

if (hostname is absolute (start with dot) || no search domains) {
 searchAsIs
} else {
  if (numDots(name) >= ndots) {
    searchAsIs
  }
  if (searchAsIs wasn't performed or failed) {
    searchWithSearchDomainsSequenciallyUntilOneSucceeds
  }
}

The DnsNameResolverBuilder provides configuration for the search domains and the ndots value. The default search domains value is configured with the OS search domains using the same native configuration the java.net resolver uses.

Result:

The DnsNameResolver performs search domains resolution when they are present.
2016-07-08 22:04:01 +02:00
Norman Maurer
4676a2271c [maven-release-plugin] prepare for next development iteration 2016-07-01 10:33:32 +02:00
Norman Maurer
ad270c02b9 [maven-release-plugin] prepare release netty-4.1.2.Final 2016-07-01 09:07:40 +02:00
Julien Viet
804e058e27 DnsNameResolver should not bind locally. Fixes #5457 Motivation: Dns resolution failures happen when using the DnsNameResolver and the JVM is not authorized to bind datagram channels. The current DnsNameResolver binds locally a DatagramChannel which is not necessary (and not always permitted).
Modifications:
The DnsNameResolver Bootstrap does not bind anymore, instead it registers and use the channel directly. The localAddress has also been removed from the DnsAddressResolverGroup, DnsNameResolver and DnsNameResolverBuilder as it is not necessary anymore and the API is marked as @UnstableApi.

Result:
Dns resolution does not require anymore to bind locally.
2016-06-28 15:39:42 +02:00
joymufeng
2562ef7cbe Fix a bug of DnsNameResolver while working with NoopDnsCache.
Motivation:

If DnsNameResolver works with NoopDnsCache, IndexOutOfBoundsException will
be thrown.

Modifications:

Test if the result of DnsNameResolver.get(hostname) is empty before
accessing it's elements.
2016-06-27 13:43:06 +02:00
Norman Maurer
a725b97092 [#5386] DnsNameResolver does not resolve localhost on Windows
Motivation:

On Windows localhost is not in hosts file and the DNS server does not resolve this address either, i.e it is handled by the Windows API. So using a Bootstrap (among others) with the resolver based on DnsNameResolver will not resolve localhost.

Modifications:

Workaround behavior of Windows

Result:

Correctly resolve localhost on Windows when using DnsNameResolver
2016-06-22 09:07:39 +02:00
Norman Maurer
c7a0a0f325 [#5391] DnsNameResolver does not resolve property A+CNAME answer
Motivation:

The current DnsNameResolver fails to resolve an A+CNAME answer. For example:

dig moose.rmq.cloudamqp.com

...
;; ANSWER SECTION:
moose.rmq.cloudamqp.com. 1800   IN  CNAME   ec2-54-152-221-139.compute-1.amazonaws.com.
ec2-54-152-221-139.compute-1.amazonaws.com. 583612 IN A 54.152.221.139
...

The resolver constructs a map of cnames but forgets the trailing "." in the values which lead to not resolve the A record.

Modifications:

Reuse the code of DefaltDnsRecordDecoder which correctly handles the trailing dot.

Result:

Correctly resolve.
2016-06-20 07:13:00 +02:00
Guido Medina
c3abb9146e Use shaded dependency on JCTools instead of copy and paste
Motivation:
JCTools supports both non-unsafe, unsafe versions of queues and JDK6 which allows us to shade the library in netty-common allowing it to stay "zero dependency".

Modifications:
- Remove copy paste JCTools code and shade the library (dependencies that are shaded should be removed from the <dependencies> section of the generated POM).
- Remove usage of OneTimeTask and remove it all together.

Result:
Less code to maintain and easier to update JCTools and less GC pressure as the queue implementation nt creates so much garbage
2016-06-10 13:19:45 +02:00
Norman Maurer
4dec7f11b7 [maven-release-plugin] prepare for next development iteration 2016-06-07 18:52:34 +02:00
Norman Maurer
cf670fab75 [maven-release-plugin] prepare release netty-4.1.1.Final 2016-06-07 18:52:22 +02:00
Norman Maurer
ee5969edfd Use higher maxQueriesPerResolve and make exception message more clear.
Motivation:

We use a default of 3 for maxQueriesPerResolve when using the DnsNameResolverBuilder, which is too low if you want to resolve a hostname that uses a lot of CNAME records.

Modifications:

- Use higher default (16)
- Make exception message more clear why it failed.

Result:

Be able to resolve more domains by default and be able to better trouble shoot why a resolver failed.
2016-06-06 09:22:11 +02:00