Page 1 of 1

Very Odd DNS Issue

Posted: 2017/08/10 01:28:50
by sirkillalot
I have a centos bind server setup that seems to work perfectly. My question I believe is an issue with centos clients resolving against the bind server but i'm not positive.

My issue is that ping and other applications (web browsers) by FQDN does NOT work, oddly enough pinging by single label hostnames with the correct search domain in /etc/resolv.conf DOES work. Also using dig or nslookup on the FQDN or single hostname also resolves. The lookup issues occur for every host in my environment, works by everything but FQDN.

Examples:

Code: Select all

ping ns01.wcbc.local
ping: ns01.wcbc.local: Name or service not known

#adding a period at the end of the FQDN
ping ns01.wcbc.local.
ping: ns01.wcbc.local.: Name or service not known

#single label name works
ping ns01
PING ns01.wcbc.local (10.22.2.10) 56(84) bytes of data.
64 bytes from ns01.wcbc.local (10.22.2.10): icmp_seq=1 ttl=62 time=1.04 ms

#dig FQDN
dig ns01.wcbc.local

; <<>> DiG 9.9.4-RedHat-9.9.4-50.el7_3.1 <<>> ns01.wcbc.local
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 59202
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;ns01.wcbc.local.		IN	A

;; ANSWER SECTION:
ns01.wcbc.local.	604800	IN	A	10.22.2.10

;; AUTHORITY SECTION:
wcbc.local.		604800	IN	NS	ns01.wcbc.local.

;; Query time: 2 msec
;; SERVER: 10.22.2.10#53(10.22.2.10)
;; WHEN: Wed Aug 09 21:23:50 EDT 2017
;; MSG SIZE  rcvd: 74

#nslookup on fqdn
nslookup ns01.wcbc.local
Server:		10.22.2.10
Address:	10.22.2.10#53

Name:	ns01.wcbc.local
Address: 10.22.2.10
/etc/resolv.conf:

Code: Select all

# Generated by NetworkManager
search wcbc.local
nameserver 10.22.2.10

Re: Very Odd DNS Issue

Posted: 2017/08/11 14:07:53
by moonpup
What does your zone file look like? If you did something like the below, it would cause the FQDN to get appended to the record name making it ns01.wcbc.local.wcbc.local. Hence, it would append wcbc.local to the short name of ns01 and the lookup would work. Whereas using the FQDN would append the domain again and it would fail. If you want to use the whole FQDN as the record name, put a . at the end of it like this ns01.wcbc.local.

ns01.wcbc.local IN A 10.22.2.10

Re: Very Odd DNS Issue

Posted: 2017/08/11 14:12:19
by sirkillalot
Update:

With the help of some folks more knowledgeable on Linux than I we were able to track this down.

The root cause of this was the Avahi-mdns daemon hijacking .local DNS queries which is the TLD for me, I discovered this when doing:

Code: Select all

sudo strace ping host.domain.com

Code: Select all

connect(4, {sa_family=AF_LOCAL, sun_path="/var/run/avahi-daemon/socket"}, 110) = 0
fcntl(4, F_GETFL)                       = 0x2 (flags O_RDWR)
fstat(4, {st_mode=S_IFSOCK|0777, st_size=0, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fc2b37ba000
write(4, "RESOLVE-HOSTNAME-IPV4 vcs01.wcbc"..., 39) = 39
read(4, 0x7fc2b37ba000, 4096)           = ? ERESTARTSYS (To be restarted if SA_RESTART is set)
--- SIGWINCH {si_signo=SIGWINCH, si_code=SI_KERNEL, si_value={int=125265496, ptr=0x7fdf07776658}} ---
read(4, "-15 Timeout reached\n", 4096)  = 20
close(4)                                = 0
munmap(0x7fc2b37ba000, 4096)            = 0
open("/usr/share/locale/locale.alias", O_RDONLY|O_CLOEXEC) = 4
fstat(4, {st_mode=S_IFREG|0644, st_size=2502, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fc2b37ba000
read(4, "# Locale name alias data base.\n#"..., 4096) = 2502
read(4, "", 4096)                       = 0
close(4)                                = 0
munmap(0x7fc2b37ba000, 4096)            = 0
i
stopping the services as a test:

Code: Select all

systemctl disable avahi-daemon avahi-daemon.socket
systemctl stop avahi-daemon avahi-daemon.socket

Not knowing how this got onto my machine I ran

Code: Select all

rpm -q --whatrequires avahi
rpm -q --whatrequires avahi
nss-mdns-0.10-12.el7.x86_64
Inspecting nss-mdns (note the last line about .local domains)

Code: Select all

Installed Packages
Name        : nss-mdns
Arch        : x86_64
Version     : 0.10
Release     : 12.el7
Size        : 95 k
Repo        : installed
From repo   : epel
Summary     : glibc plugin for .local name resolution
URL         : http://0pointer.de/lennart/projects/nss-mdns/
License     : LGPLv2+
Description : nss-mdns is a plugin for the GNU Name Service Switch (NSS) functionality of
            : the GNU C Library (glibc) providing host name resolution via Multicast DNS
            : (aka Zeroconf, aka Apple Rendezvous, aka Apple Bonjour), effectively allowing
            : name resolution by common Unix/Linux programs in the ad-hoc mDNS domain .local.

and then a final check to see what would be removed if I erase the package (be sure to NOT put the -y)

Code: Select all

yum erase avahi 
=======================================================================================================================
 Package                   Arch                    Version                                Repository              Size
=======================================================================================================================
Removing:
 avahi                     x86_64                  0.6.31-17.el7                          @base                  1.0 M
Removing for dependencies:
 chromium                  x86_64                  59.0.3071.104-1.el7                    @epel                  148 M
 nss-mdns                  x86_64                  0.10-12.el7                            @epel                   95 k

Transaction Summary
=======================================================================================================================
I had this installed from having chromium on my machine.

Re: Very Odd DNS Issue

Posted: 2017/08/11 14:18:42
by sirkillalot
moonpup wrote:What does your zone file look like? If you did something like the below, it would cause the FQDN to get appended to the record name making it ns01.wcbc.local.wcbc.local. Hence, it would append wcbc.local to the short name of ns01 and the lookup would work. Whereas using the FQDN would append the domain again and it would fail. If you want to use the whole FQDN as the record name, put a . at the end of it like this ns01.wcbc.local.

ns01.wcbc.local IN A 10.22.2.10
In the OP I tried pinging with a period at the end of the FQDN and it did not change the behavior. I did find the solution though

Re: Very Odd DNS Issue

Posted: 2017/08/11 14:22:10
by moonpup
Nice find!! Thanks for the update.

Interesting dependency with chromium though... :\