Here's a CentOS 6 "feature" which leads to a security problem.
Try this:
> ping example
I get:
PING example.com (192.0.43.10) 56(84) bytes of data.
64 bytes from 43-10.any.icann.org (192.0.43.10): icmp_seq=1...
Note that "ping" was given only "example", but some lower level appended ".com". The same thing occurs for wget, and even for programs that call "getaddrinfo" directly.
I've tested this from Python. The program actually displayed the ".com". It appears that the library for getaddrinfo is appending ".com" and retrying if a lookup fails.
Unfortunately, whatever appends ".com" doesn't check to see if it's there already. So
> ping noexample.com
PING phx1-ss-2-lb.cnet.com (64.30.224.112) 56(84) bytes of data.
64 bytes from phx1-ss-2-lb.cnet.com (64.30.224.112): icmp_seq=1 ttl=246 time=11.8 ms
...
How did we get to "cnet.com"? There is no "noexample.com". So something added ".com" without
checking, creating "noexample.com.com". That resolves because "com.com" is a domain owned
by CNET, and CNET has set up a wildcard A record to capture all subdomains in "com.com".
Which, inevitably, are directed to a fake search page with ads.
This isn't a browser-level thing. It affects "ping", "wget", and any application which uses
"getaddrinfo". But not "nslookup" or "host", which talk directly to the DNS servers.
The machine with this problem isn't running a "named"l; it just refers everything
to Codero's DNS servers using resolv.conf:
> cat /etc/resolv.conf
nameserver 69.64.66.11
nameserver 69.64.66.10
Those name servers are NOT appending the ".com" suffix on no-finds. It's local.
Where is this in the code, and who put it there?
CentOS 6, x86-64, GCC libraries /usr/lib/gcc/x86_64-redhat-linux 4.4.4 or 4.4.6
getaddrinfo NXDOMAIN hijack
Re: getaddrinfo NXDOMAIN hijack
Here's someone else who reported the same problem on serverfault last December.
http://serverfault.com/questions/341383/possible-nxdomain-hijacking
http://serverfault.com/questions/341383/possible-nxdomain-hijacking
getaddrinfo NXDOMAIN hijack
Is that the entire content of your /etc/resolv.conf file? Here's what I get on a fully patched CentOS 6.2 machine
[code]
$ ping example
ping: unknown host example
$ ping noexample.com
ping: unknown host noexample.com
[/code]
[code]
$ ping example
ping: unknown host example
$ ping noexample.com
ping: unknown host noexample.com
[/code]
Re: getaddrinfo NXDOMAIN hijack
Yes, those two lines are that's all that's in resolv.conf. That's what Webmin put there.
/etc/hosts is equally bland; it just has "localhost" stuff.
There's no local DNS daemon ("named") running on this machine. This is a dedicated server at Codero,
not virtualized.
I, too, thought that it had to be a problem with an upstream DNS server being set up to send NXDOMAIN
results to a junk site. But it's not. "nslookup" and "host" do NOT resolve nonexistent domains.
If I try "ping noexample.net", that's a no find.
I've had this server for about three months; it replaced a previous 32-bit x86 server, also at Codero, which
ran an earlier Red Hat version. That server didn't have this problem, but used the same DNS servers.
I've been looking at the code for "getaddrinfo.c", and don't see anything that could do this. But there are
so many versions of "getaddrinfo.c" out there that I may be looking at the wrong one.
/etc/hosts is equally bland; it just has "localhost" stuff.
There's no local DNS daemon ("named") running on this machine. This is a dedicated server at Codero,
not virtualized.
I, too, thought that it had to be a problem with an upstream DNS server being set up to send NXDOMAIN
results to a junk site. But it's not. "nslookup" and "host" do NOT resolve nonexistent domains.
If I try "ping noexample.net", that's a no find.
I've had this server for about three months; it replaced a previous 32-bit x86 server, also at Codero, which
ran an earlier Red Hat version. That server didn't have this problem, but used the same DNS servers.
I've been looking at the code for "getaddrinfo.c", and don't see anything that could do this. But there are
so many versions of "getaddrinfo.c" out there that I may be looking at the wrong one.
Re: getaddrinfo NXDOMAIN hijack
Try running
[code]
strace -f -o /tmp/pingnoex.txt ping noexample.com
[/code]
then look at the /tmp/pingnoex.txt file and hunt through it to see what's asking for noexample.com.com.
[code]
strace -f -o /tmp/pingnoex.txt ping noexample.com
[/code]
then look at the /tmp/pingnoex.txt file and hunt through it to see what's asking for noexample.com.com.
Re: getaddrinfo NXDOMAIN hijack
strace was a very good idea.
The result is below.
Points of interest:
Libraries loaded:
/lib64/libidn.so.11
/lib64/libc.so.6
- Opens /etc/resolv.conf, as expected.
- Opens /etc/nsswitch.conf . The key line in that file is just
hosts: files dns
so name search goes to "files" (hosts.txt), then DNS.
- Opens /etc/hosts, as expected. There's nothing in there but the local host itself.
- Opens /etc/host.conf, which contains one line with "multi on".
Then it gets interesting. Here's the communications with the DNS server, with some comments added.
[quote]
10398 socket(PF_INET, SOCK_DGRAM|SOCK_NONBLOCK, IPPROTO_IP) = 4
10398 connect(4, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("69.64.66.11")}, 16) = 0
10398 poll([{fd=4, events=POLLOUT}], 1, 0) = 1 ([{fd=4, revents=POLLOUT}])
***First DNS query: asking 69.64.66.11 (Codero's DNS server) for "noexample.com")***
10398 sendto(4, "\246O\1\0\0\1\0\0\0\0\0\0\tnoexample\3com\0\0\1\0\1", 31, MSG_NOSIGNAL, NULL, 0) = 31
10398 poll([{fd=4, events=POLLIN}], 1, 5000) = 1 ([{fd=4, revents=POLLIN}])
10398 ioctl(4, FIONREAD, [31]) = 0
***Reply comes back, but strace only prints the beginning, so we don't get to see enough of what the reply said.***
10398 recvfrom(4, "\246O\205\203\0\1\0\0\0\0\0\0\tnoexample\3com\0\0\1\0\1", 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("69.64.66.11")}, [16]) = 31
10398 close(4) = 0
10398 socket(PF_INET, SOCK_DGRAM|SOCK_NONBLOCK, IPPROTO_IP) = 4
10398 connect(4, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("69.64.66.11")}, 16) = 0
10398 poll([{fd=4, events=POLLOUT}], 1, 0) = 1 ([{fd=4, revents=POLLOUT}])
***Second DNS query: asking 69.64.66.11 (Codero's DNS server) for "noexample.com.com")***
***So it really is the local client library appending the extra ".com"***
10398 sendto(4, "\372a\1\0\0\1\0\0\0\0\0\0\tnoexample\3com\3com\0\0"..., 35, MSG_NOSIGNAL, NULL, 0) = 35
10398 poll([{fd=4, events=POLLIN}], 1, 5000) = 1 ([{fd=4, revents=POLLIN}])
10398 ioctl(4, FIONREAD, [83]) = 0
***Reply comes back from second DNS query***
10398 recvfrom(4, "\372a\201\200\0\1\0\2\0\0\0\0\tnoexample\3com\3com\0\0"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("69.64.66.11")}, [16]) = 83
10398 close(4) = 0
***Done with name resolution - setting up pinging***
10398 socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 4
***Preparing to ping 64.30.224.112, which is "phx1-ss-2-lb.cnet.com"***
10398 connect(4, {sa_family=AF_INET, sin_port=htons(1025), sin_addr=inet_addr("64.30.224.112")}, 16) = 0
10398 getsockname(4, {sa_family=AF_INET, sin_port=htons(46615), sin_addr=inet_addr("69.64.68.204")}, [16]) = 0
10398 close(4) = 0
10398 setsockopt(3, SOL_RAW, ICMP_FILTER, ~(ICMP_ECHOREPLY|ICMP_DEST_UNREACH|ICMP_SOURCE_QUENCH|ICMP_REDIRECT|ICMP_TIME_EXCEEDED|ICMP_PARAMETERPROB), 4) = 0
10398 setsockopt(3, SOL_IP, IP_RECVERR, [1], 4) = 0
10398 setsockopt(3, SOL_SOCKET, SO_SNDBUF, [324], 4) = 0
10398 setsockopt(3, SOL_SOCKET, SO_RCVBUF, [65536], 4) = 0
10398 getsockopt(3, SOL_SOCKET, SO_RCVBUF, [-1318403725483442176], [4]) = 0
10398 fstat(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 0), ...}) = 0
10398 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fdded933000
***ping reports to the user what it is pinging***
10398 write(1, "PING phx1-ss-2-lb.cnet.com (64.3"..., 65) = 65
[/quote]
So that extra ".com" [b]is being added to the domain name locally.[/b] The DNS server didn't do it. It's a problem in "getaddrinfo".
Exactly what source for "getaddrinfo.c" is CentOS 6 running?
Here's the whole trace, for completeness.
[quote]
10398 execve("/bin/ping", ["ping", "noexample.com"], [/* 21 vars */]) = 0
10398 brk(0) = 0x7fddedf34000
10398 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fdded934000
10398 access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
10398 open("/etc/ld.so.cache", O_RDONLY) = 3
10398 fstat(3, {st_mode=S_IFREG|0644, st_size=32064, ...}) = 0
10398 mmap(NULL, 32064, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7fdded92c000
10398 close(3) = 0
10398 open("/lib64/libidn.so.11", O_RDONLY) = 3
10398 read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0/\340\2302\0\0\0"..., 832) = 832
10398 fstat(3, {st_mode=S_IFREG|0755, st_size=209120, ...}) = 0
10398 mmap(NULL, 2301768, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fdded4e4000
10398 mprotect(0x7fdded516000, 2093056, PROT_NONE) = 0
10398 mmap(0x7fdded715000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x31000) = 0x7fdded715000
10398 close(3) = 0
10398 open("/lib64/libc.so.6", O_RDONLY) = 3
10398 read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\360\355\241\2222\0\0\0"..., 832) = 832
10398 fstat(3, {st_mode=S_IFREG|0755, st_size=1979000, ...}) = 0
10398 mmap(NULL, 3803304, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fdded143000
10398 mprotect(0x7fdded2da000, 2097152, PROT_NONE) = 0
10398 mmap(0x7fdded4da000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x197000) = 0x7fdded4da000
10398 mmap(0x7fdded4df000, 18600, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fdded4df000
10398 close(3) = 0
10398 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fdded92b000
10398 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fdded92a000
10398 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fdded929000
10398 arch_prctl(ARCH_SET_FS, 0x7fdded92a700) = 0
10398 mprotect(0x7fdded4da000, 16384, PROT_READ) = 0
10398 mprotect(0x7fdded935000, 4096, PROT_READ) = 0
10398 munmap(0x7fdded92c000, 32064) = 0
10398 brk(0) = 0x7fddedf34000
10398 brk(0x7fddedf55000) = 0x7fddedf55000
10398 open("/usr/lib/locale/locale-archive", O_RDONLY) = 3
10398 fstat(3, {st_mode=S_IFREG|0644, st_size=99158704, ...}) = 0
10398 mmap(NULL, 99158704, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7fdde72b2000
10398 close(3) = 0
10398 socket(PF_INET, SOCK_RAW, IPPROTO_ICMP) = 3
10398 getuid() = 0
10398 setuid(0) = 0
10398 getpid() = 10398
10398 open("/etc/resolv.conf", O_RDONLY) = 4
10398 fstat(4, {st_mode=S_IFREG|0644, st_size=46, ...}) = 0
10398 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fdded933000
10398 read(4, "nameserver 69.64.66.11\nnameserve"..., 4096) = 46
10398 read(4, "", 4096) = 0
10398 close(4) = 0
10398 munmap(0x7fdded933000, 4096) = 0
10398 uname({sys="Linux", node="sitetruth.com", ...}) = 0
10398 socket(PF_FILE, SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK, 0) = 4
10398 connect(4, {sa_family=AF_FILE, path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)
10398 close(4) = 0
10398 socket(PF_FILE, SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK, 0) = 4
10398 connect(4, {sa_family=AF_FILE, path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)
10398 close(4) = 0
10398 open("/etc/nsswitch.conf", O_RDONLY) = 4
10398 fstat(4, {st_mode=S_IFREG|0644, st_size=1683, ...}) = 0
10398 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fdded933000
10398 read(4, "#\n# /etc/nsswitch.conf\n#\n# An ex"..., 4096) = 1683
10398 read(4, "", 4096) = 0
10398 close(4) = 0
10398 munmap(0x7fdded933000, 4096) = 0
10398 open("/etc/ld.so.cache", O_RDONLY) = 4
10398 fstat(4, {st_mode=S_IFREG|0644, st_size=32064, ...}) = 0
10398 mmap(NULL, 32064, PROT_READ, MAP_PRIVATE, 4, 0) = 0x7fdded92c000
10398 close(4) = 0
10398 open("/lib64/libnss_files.so.2", O_RDONLY) = 4
10398 read(4, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\360!\0\0\0\0\0\0"..., 832) = 832
10398 fstat(4, {st_mode=S_IFREG|0755, st_size=65928, ...}) = 0
10398 mmap(NULL, 2151824, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 4, 0) = 0x7fdde70a4000
10398 mprotect(0x7fdde70b0000, 2097152, PROT_NONE) = 0
10398 mmap(0x7fdde72b0000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 4, 0xc000) = 0x7fdde72b0000
10398 close(4) = 0
10398 mprotect(0x7fdde72b0000, 4096, PROT_READ) = 0
10398 munmap(0x7fdded92c000, 32064) = 0
10398 open("/etc/host.conf", O_RDONLY) = 4
10398 fstat(4, {st_mode=S_IFREG|0644, st_size=9, ...}) = 0
10398 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fdded933000
10398 read(4, "multi on\n", 4096) = 9
10398 read(4, "", 4096) = 0
10398 close(4) = 0
10398 munmap(0x7fdded933000, 4096) = 0
10398 open("/etc/hosts", O_RDONLY|O_CLOEXEC) = 4
10398 fcntl(4, F_GETFD) = 0x1 (flags FD_CLOEXEC)
10398 fstat(4, {st_mode=S_IFREG|0644, st_size=280, ...}) = 0
10398 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fdded933000
10398 read(4, "# =============================="..., 4096) = 280
10398 read(4, "", 4096) = 0
10398 close(4) = 0
10398 munmap(0x7fdded933000, 4096) = 0
10398 open("/etc/ld.so.cache", O_RDONLY) = 4
10398 fstat(4, {st_mode=S_IFREG|0644, st_size=32064, ...}) = 0
10398 mmap(NULL, 32064, PROT_READ, MAP_PRIVATE, 4, 0) = 0x7fdded92c000
10398 close(4) = 0
10398 open("/lib64/libnss_dns.so.2", O_RDONLY) = 4
10398 read(4, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\20\0\0\0\0\0\0"..., 832) = 832
10398 fstat(4, {st_mode=S_IFREG|0755, st_size=27424, ...}) = 0
10398 mmap(NULL, 2117880, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 4, 0) = 0x7fdde6e9e000
10398 mprotect(0x7fdde6ea3000, 2093056, PROT_NONE) = 0
10398 mmap(0x7fdde70a2000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 4, 0x4000) = 0x7fdde70a2000
10398 close(4) = 0
10398 open("/lib64/libresolv.so.2", O_RDONLY) = 4
10398 read(4, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\00009\240\2242\0\0\0"..., 832) = 832
10398 fstat(4, {st_mode=S_IFREG|0755, st_size=113952, ...}) = 0
10398 mmap(NULL, 2202248, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 4, 0) = 0x7fdde6c84000
10398 mprotect(0x7fdde6c9a000, 2097152, PROT_NONE) = 0
10398 mmap(0x7fdde6e9a000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 4, 0x16000) = 0x7fdde6e9a000
10398 mmap(0x7fdde6e9c000, 6792, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fdde6e9c000
10398 close(4) = 0
10398 mprotect(0x7fdde6e9a000, 4096, PROT_READ) = 0
10398 mprotect(0x7fdde70a2000, 4096, PROT_READ) = 0
10398 munmap(0x7fdded92c000, 32064) = 0
10398 socket(PF_INET, SOCK_DGRAM|SOCK_NONBLOCK, IPPROTO_IP) = 4
10398 connect(4, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("69.64.66.11")}, 16) = 0
10398 poll([{fd=4, events=POLLOUT}], 1, 0) = 1 ([{fd=4, revents=POLLOUT}])
10398 sendto(4, "\246O\1\0\0\1\0\0\0\0\0\0\tnoexample\3com\0\0\1\0\1", 31, MSG_NOSIGNAL, NULL, 0) = 31
10398 poll([{fd=4, events=POLLIN}], 1, 5000) = 1 ([{fd=4, revents=POLLIN}])
10398 ioctl(4, FIONREAD, [31]) = 0
10398 recvfrom(4, "\246O\205\203\0\1\0\0\0\0\0\0\tnoexample\3com\0\0\1\0\1", 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("69.64.66.11")}, [16]) = 31
10398 close(4) = 0
10398 socket(PF_INET, SOCK_DGRAM|SOCK_NONBLOCK, IPPROTO_IP) = 4
10398 connect(4, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("69.64.66.11")}, 16) = 0
10398 poll([{fd=4, events=POLLOUT}], 1, 0) = 1 ([{fd=4, revents=POLLOUT}])
10398 sendto(4, "\372a\1\0\0\1\0\0\0\0\0\0\tnoexample\3com\3com\0\0"..., 35, MSG_NOSIGNAL, NULL, 0) = 35
10398 poll([{fd=4, events=POLLIN}], 1, 5000) = 1 ([{fd=4, revents=POLLIN}])
10398 ioctl(4, FIONREAD, [83]) = 0
10398 recvfrom(4, "\372a\201\200\0\1\0\2\0\0\0\0\tnoexample\3com\3com\0\0"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("69.64.66.11")}, [16]) = 83
10398 close(4) = 0
10398 socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 4
10398 connect(4, {sa_family=AF_INET, sin_port=htons(1025), sin_addr=inet_addr("64.30.224.112")}, 16) = 0
10398 getsockname(4, {sa_family=AF_INET, sin_port=htons(46615), sin_addr=inet_addr("69.64.68.204")}, [16]) = 0
10398 close(4) = 0
10398 setsockopt(3, SOL_RAW, ICMP_FILTER, ~(ICMP_ECHOREPLY|ICMP_DEST_UNREACH|ICMP_SOURCE_QUENCH|ICMP_REDIRECT|ICMP_TIME_EXCEEDED|ICMP_PARAMETERPROB), 4) = 0
10398 setsockopt(3, SOL_IP, IP_RECVERR, [1], 4) = 0
10398 setsockopt(3, SOL_SOCKET, SO_SNDBUF, [324], 4) = 0
10398 setsockopt(3, SOL_SOCKET, SO_RCVBUF, [65536], 4) = 0
10398 getsockopt(3, SOL_SOCKET, SO_RCVBUF, [-1318403725483442176], [4]) = 0
10398 fstat(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 0), ...}) = 0
10398 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fdded933000
10398 write(1, "PING phx1-ss-2-lb.cnet.com (64.3"..., 65) = 65
10398 setsockopt(3, SOL_SOCKET, SO_TIMESTAMP, [1], 4) = 0
10398 setsockopt(3, SOL_SOCKET, SO_SNDTIMEO, "\1\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0", 16) = 0
10398 setsockopt(3, SOL_SOCKET, SO_RCVTIMEO, "\1\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0", 16) = 0
10398 rt_sigaction(SIGINT, {0x7fdded93ca40, [], SA_RESTORER|SA_INTERRUPT, 0x7fdded175900}, NULL, 8) = 0
10398 rt_sigaction(SIGALRM, {0x7fdded93ca40, [], SA_RESTORER|SA_INTERRUPT, 0x7fdded175900}, NULL, 8) = 0
10398 rt_sigaction(SIGQUIT, {0x7fdded93ca50, [], SA_RESTORER|SA_INTERRUPT, 0x7fdded175900}, NULL, 8) = 0
10398 ioctl(1, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
10398 ioctl(1, TIOCGWINSZ, {ws_row=24, ws_col=80, ws_xpixel=0, ws_ypixel=0}) = 0
10398 sendmsg(3, {msg_name(16)={sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("64.30.224.112")}, msg_iov(1)=[{"\10\0b\315\236(\0\1\254xxO\0\0\0\0\21n\2\0\0\0\0\0\20\21\22\23\24\25\26\27"..., 64}], msg_controllen=0, msg_flags=0}, 0) = 64
10398 recvmsg(3, {msg_name(16)={sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("64.30.224.112")}, msg_iov(1)=[{"E\0\0T\0\0@\0\366\1\332\r@\36\340pE@D\314\0\0j\315\236(\0\1\254xxO"..., 192}], msg_controllen=32, {cmsg_len=32, cmsg_level=SOL_SOCKET, cmsg_type=0x1d /* SCM_??? */, ...}, msg_flags=0}, 0) = 84
10398 open("/etc/hosts", O_RDONLY|O_CLOEXEC) = 4
10398 fstat(4, {st_mode=S_IFREG|0644, st_size=280, ...}) = 0
10398 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fdded932000
10398 read(4, "# =============================="..., 4096) = 280
10398 read(4, "", 4096) = 0
10398 close(4) = 0
10398 munmap(0x7fdded932000, 4096) = 0
10398 socket(PF_INET, SOCK_DGRAM|SOCK_NONBLOCK, IPPROTO_IP) = 4
10398 connect(4, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("69.64.66.11")}, 16) = 0
10398 poll([{fd=4, events=POLLOUT}], 1, 0) = 1 ([{fd=4, revents=POLLOUT}])
10398 sendto(4, "\1w\1\0\0\1\0\0\0\0\0\0\003112\003224\00230\00264\7in-ad"..., 44, MSG_NOSIGNAL, NULL, 0) = 44
10398 poll([{fd=4, events=POLLIN}], 1, 5000) = 1 ([{fd=4, revents=POLLIN}])
10398 ioctl(4, FIONREAD, [79]) = 0
10398 recvfrom(4, "\1w\201\200\0\1\0\1\0\0\0\0\003112\003224\00230\00264\7in-ad"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("69.64.66.11")}, [16]) = 79
10398 close(4) = 0
10398 write(1, "64 bytes from phx1-ss-2-lb.cnet."..., 85) = 85
10398 poll([{fd=3, events=POLLIN|POLLERR}], 1, 965) = 0 (Timeout)
10398 sendmsg(3, {msg_name(16)={sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("64.30.224.112")}, msg_iov(1)=[{"\10\0~\305\236(\0\2\255xxO\0\0\0\0\364t\2\0\0\0\0\0\20\21\22\23\24\25\26\27"..., 64}], msg_controllen=0, msg_flags=0}, MSG_CONFIRM) = 64
10398 recvmsg(3, {msg_name(16)={sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("64.30.224.112")}, msg_iov(1)=[{"E\0\0T\0\0@\0\366\1\332\r@\36\340pE@D\314\0\0\206\305\236(\0\2\255xxO"..., 192}], msg_controllen=32, {cmsg_len=32, cmsg_level=SOL_SOCKET, cmsg_type=0x1d /* SCM_??? */, ...}, msg_flags=0}, 0) = 84
10398 write(1, "64 bytes from phx1-ss-2-lb.cnet."..., 85) = 85
10398 poll([{fd=3, events=POLLIN|POLLERR}], 1, 989) = 0 (Timeout)
10398 sendmsg(3, {msg_name(16)={sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("64.30.224.112")}, msg_iov(1)=[{"\10\0\23\277\236(\0\3\256xxO\0\0\0\0^z\2\0\0\0\0\0\20\21\22\23\24\25\26\27"..., 64}], msg_controllen=0, msg_flags=0}, MSG_CONFIRM) = 64
10398 recvmsg(3, {msg_name(16)={sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("64.30.224.112")}, msg_iov(1)=[{"E\0\0T\0\0@\0\366\1\332\r@\36\340pE@D\314\0\0\33\277\236(\0\3\256xxO"..., 192}], msg_controllen=32, {cmsg_len=32, cmsg_level=SOL_SOCKET, cmsg_type=0x1d /* SCM_??? */, ...}, msg_flags=0}, 0) = 84
10398 write(1, "64 bytes from phx1-ss-2-lb.cnet."..., 85) = 85
10398 poll([{fd=3, events=POLLIN|POLLERR}], 1, 988) = 0 (Timeout)
10398 sendmsg(3, {msg_name(16)={sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("64.30.224.112")}, msg_iov(1)=[{"\10\0l\271\236(\0\4\257xxO\0\0\0\0\4\177\2\0\0\0\0\0\20\21\22\23\24\25\26\27"..., 64}], msg_controllen=0, msg_flags=0}, MSG_CONFIRM) = 64
10398 recvmsg(3, {msg_name(16)={sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("64.30.224.112")}, msg_iov(1)=[{"E\0\0T\0\0@\0\366\1\332\r@\36\340pE@D\314\0\0t\271\236(\0\4\257xxO"..., 192}], msg_controllen=32, {cmsg_len=32, cmsg_level=SOL_SOCKET, cmsg_type=0x1d /* SCM_??? */, ...}, msg_flags=0}, 0) = 84
10398 write(1, "64 bytes from phx1-ss-2-lb.cnet."..., 85) = 85
10398 poll([{fd=3, events=POLLIN|POLLERR}], 1, 988) = 0 (Timeout)
10398 sendmsg(3, {msg_name(16)={sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("64.30.224.112")}, msg_iov(1)=[{"\10\0e\264\236(\0\5\260xxO\0\0\0\0\n\203\2\0\0\0\0\0\20\21\22\23\24\25\26\27"..., 64}], msg_controllen=0, msg_flags=0}, MSG_CONFIRM) = 64
10398 recvmsg(3, {msg_name(16)={sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("64.30.224.112")}, msg_iov(1)=[{"E\0\0T\0\0@\0\366\1\332\r@\36\340pE@D\314\0\0m\264\236(\0\5\260xxO"..., 192}], msg_controllen=32, {cmsg_len=32, cmsg_level=SOL_SOCKET, cmsg_type=0x1d /* SCM_??? */, ...}, msg_flags=0}, 0) = 84
10398 write(1, "64 bytes from phx1-ss-2-lb.cnet."..., 85) = 85
10398 poll([{fd=3, events=POLLIN|POLLERR}], 1, 988) = ? ERESTART_RESTARTBLOCK (To be restarted)
10398 --- SIGINT (Interrupt) @ 0 (0) ---
10398 rt_sigreturn(0x2) = -1 EINTR (Interrupted system call)
10398 write(1, "\n", 1) = 1
10398 write(1, "--- phx1-ss-2-lb.cnet.com ping s"..., 46) = 46
10398 write(1, "5 packets transmitted, 5 receive"..., 63) = 63
10398 write(1, "rtt min/avg/max/mdev = 11.885/11"..., 53) = 53
10398 exit_group(0)
[/quote][Moderator edit: Changed [i]code[/i] to [i]quote[/i] tags to wrap [b]long[/b] lines and prevent running-off-the-right-side syndrome.]
The result is below.
Points of interest:
Libraries loaded:
/lib64/libidn.so.11
/lib64/libc.so.6
- Opens /etc/resolv.conf, as expected.
- Opens /etc/nsswitch.conf . The key line in that file is just
hosts: files dns
so name search goes to "files" (hosts.txt), then DNS.
- Opens /etc/hosts, as expected. There's nothing in there but the local host itself.
- Opens /etc/host.conf, which contains one line with "multi on".
Then it gets interesting. Here's the communications with the DNS server, with some comments added.
[quote]
10398 socket(PF_INET, SOCK_DGRAM|SOCK_NONBLOCK, IPPROTO_IP) = 4
10398 connect(4, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("69.64.66.11")}, 16) = 0
10398 poll([{fd=4, events=POLLOUT}], 1, 0) = 1 ([{fd=4, revents=POLLOUT}])
***First DNS query: asking 69.64.66.11 (Codero's DNS server) for "noexample.com")***
10398 sendto(4, "\246O\1\0\0\1\0\0\0\0\0\0\tnoexample\3com\0\0\1\0\1", 31, MSG_NOSIGNAL, NULL, 0) = 31
10398 poll([{fd=4, events=POLLIN}], 1, 5000) = 1 ([{fd=4, revents=POLLIN}])
10398 ioctl(4, FIONREAD, [31]) = 0
***Reply comes back, but strace only prints the beginning, so we don't get to see enough of what the reply said.***
10398 recvfrom(4, "\246O\205\203\0\1\0\0\0\0\0\0\tnoexample\3com\0\0\1\0\1", 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("69.64.66.11")}, [16]) = 31
10398 close(4) = 0
10398 socket(PF_INET, SOCK_DGRAM|SOCK_NONBLOCK, IPPROTO_IP) = 4
10398 connect(4, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("69.64.66.11")}, 16) = 0
10398 poll([{fd=4, events=POLLOUT}], 1, 0) = 1 ([{fd=4, revents=POLLOUT}])
***Second DNS query: asking 69.64.66.11 (Codero's DNS server) for "noexample.com.com")***
***So it really is the local client library appending the extra ".com"***
10398 sendto(4, "\372a\1\0\0\1\0\0\0\0\0\0\tnoexample\3com\3com\0\0"..., 35, MSG_NOSIGNAL, NULL, 0) = 35
10398 poll([{fd=4, events=POLLIN}], 1, 5000) = 1 ([{fd=4, revents=POLLIN}])
10398 ioctl(4, FIONREAD, [83]) = 0
***Reply comes back from second DNS query***
10398 recvfrom(4, "\372a\201\200\0\1\0\2\0\0\0\0\tnoexample\3com\3com\0\0"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("69.64.66.11")}, [16]) = 83
10398 close(4) = 0
***Done with name resolution - setting up pinging***
10398 socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 4
***Preparing to ping 64.30.224.112, which is "phx1-ss-2-lb.cnet.com"***
10398 connect(4, {sa_family=AF_INET, sin_port=htons(1025), sin_addr=inet_addr("64.30.224.112")}, 16) = 0
10398 getsockname(4, {sa_family=AF_INET, sin_port=htons(46615), sin_addr=inet_addr("69.64.68.204")}, [16]) = 0
10398 close(4) = 0
10398 setsockopt(3, SOL_RAW, ICMP_FILTER, ~(ICMP_ECHOREPLY|ICMP_DEST_UNREACH|ICMP_SOURCE_QUENCH|ICMP_REDIRECT|ICMP_TIME_EXCEEDED|ICMP_PARAMETERPROB), 4) = 0
10398 setsockopt(3, SOL_IP, IP_RECVERR, [1], 4) = 0
10398 setsockopt(3, SOL_SOCKET, SO_SNDBUF, [324], 4) = 0
10398 setsockopt(3, SOL_SOCKET, SO_RCVBUF, [65536], 4) = 0
10398 getsockopt(3, SOL_SOCKET, SO_RCVBUF, [-1318403725483442176], [4]) = 0
10398 fstat(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 0), ...}) = 0
10398 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fdded933000
***ping reports to the user what it is pinging***
10398 write(1, "PING phx1-ss-2-lb.cnet.com (64.3"..., 65) = 65
[/quote]
So that extra ".com" [b]is being added to the domain name locally.[/b] The DNS server didn't do it. It's a problem in "getaddrinfo".
Exactly what source for "getaddrinfo.c" is CentOS 6 running?
Here's the whole trace, for completeness.
[quote]
10398 execve("/bin/ping", ["ping", "noexample.com"], [/* 21 vars */]) = 0
10398 brk(0) = 0x7fddedf34000
10398 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fdded934000
10398 access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
10398 open("/etc/ld.so.cache", O_RDONLY) = 3
10398 fstat(3, {st_mode=S_IFREG|0644, st_size=32064, ...}) = 0
10398 mmap(NULL, 32064, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7fdded92c000
10398 close(3) = 0
10398 open("/lib64/libidn.so.11", O_RDONLY) = 3
10398 read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0/\340\2302\0\0\0"..., 832) = 832
10398 fstat(3, {st_mode=S_IFREG|0755, st_size=209120, ...}) = 0
10398 mmap(NULL, 2301768, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fdded4e4000
10398 mprotect(0x7fdded516000, 2093056, PROT_NONE) = 0
10398 mmap(0x7fdded715000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x31000) = 0x7fdded715000
10398 close(3) = 0
10398 open("/lib64/libc.so.6", O_RDONLY) = 3
10398 read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\360\355\241\2222\0\0\0"..., 832) = 832
10398 fstat(3, {st_mode=S_IFREG|0755, st_size=1979000, ...}) = 0
10398 mmap(NULL, 3803304, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fdded143000
10398 mprotect(0x7fdded2da000, 2097152, PROT_NONE) = 0
10398 mmap(0x7fdded4da000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x197000) = 0x7fdded4da000
10398 mmap(0x7fdded4df000, 18600, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fdded4df000
10398 close(3) = 0
10398 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fdded92b000
10398 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fdded92a000
10398 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fdded929000
10398 arch_prctl(ARCH_SET_FS, 0x7fdded92a700) = 0
10398 mprotect(0x7fdded4da000, 16384, PROT_READ) = 0
10398 mprotect(0x7fdded935000, 4096, PROT_READ) = 0
10398 munmap(0x7fdded92c000, 32064) = 0
10398 brk(0) = 0x7fddedf34000
10398 brk(0x7fddedf55000) = 0x7fddedf55000
10398 open("/usr/lib/locale/locale-archive", O_RDONLY) = 3
10398 fstat(3, {st_mode=S_IFREG|0644, st_size=99158704, ...}) = 0
10398 mmap(NULL, 99158704, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7fdde72b2000
10398 close(3) = 0
10398 socket(PF_INET, SOCK_RAW, IPPROTO_ICMP) = 3
10398 getuid() = 0
10398 setuid(0) = 0
10398 getpid() = 10398
10398 open("/etc/resolv.conf", O_RDONLY) = 4
10398 fstat(4, {st_mode=S_IFREG|0644, st_size=46, ...}) = 0
10398 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fdded933000
10398 read(4, "nameserver 69.64.66.11\nnameserve"..., 4096) = 46
10398 read(4, "", 4096) = 0
10398 close(4) = 0
10398 munmap(0x7fdded933000, 4096) = 0
10398 uname({sys="Linux", node="sitetruth.com", ...}) = 0
10398 socket(PF_FILE, SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK, 0) = 4
10398 connect(4, {sa_family=AF_FILE, path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)
10398 close(4) = 0
10398 socket(PF_FILE, SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK, 0) = 4
10398 connect(4, {sa_family=AF_FILE, path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)
10398 close(4) = 0
10398 open("/etc/nsswitch.conf", O_RDONLY) = 4
10398 fstat(4, {st_mode=S_IFREG|0644, st_size=1683, ...}) = 0
10398 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fdded933000
10398 read(4, "#\n# /etc/nsswitch.conf\n#\n# An ex"..., 4096) = 1683
10398 read(4, "", 4096) = 0
10398 close(4) = 0
10398 munmap(0x7fdded933000, 4096) = 0
10398 open("/etc/ld.so.cache", O_RDONLY) = 4
10398 fstat(4, {st_mode=S_IFREG|0644, st_size=32064, ...}) = 0
10398 mmap(NULL, 32064, PROT_READ, MAP_PRIVATE, 4, 0) = 0x7fdded92c000
10398 close(4) = 0
10398 open("/lib64/libnss_files.so.2", O_RDONLY) = 4
10398 read(4, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\360!\0\0\0\0\0\0"..., 832) = 832
10398 fstat(4, {st_mode=S_IFREG|0755, st_size=65928, ...}) = 0
10398 mmap(NULL, 2151824, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 4, 0) = 0x7fdde70a4000
10398 mprotect(0x7fdde70b0000, 2097152, PROT_NONE) = 0
10398 mmap(0x7fdde72b0000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 4, 0xc000) = 0x7fdde72b0000
10398 close(4) = 0
10398 mprotect(0x7fdde72b0000, 4096, PROT_READ) = 0
10398 munmap(0x7fdded92c000, 32064) = 0
10398 open("/etc/host.conf", O_RDONLY) = 4
10398 fstat(4, {st_mode=S_IFREG|0644, st_size=9, ...}) = 0
10398 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fdded933000
10398 read(4, "multi on\n", 4096) = 9
10398 read(4, "", 4096) = 0
10398 close(4) = 0
10398 munmap(0x7fdded933000, 4096) = 0
10398 open("/etc/hosts", O_RDONLY|O_CLOEXEC) = 4
10398 fcntl(4, F_GETFD) = 0x1 (flags FD_CLOEXEC)
10398 fstat(4, {st_mode=S_IFREG|0644, st_size=280, ...}) = 0
10398 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fdded933000
10398 read(4, "# =============================="..., 4096) = 280
10398 read(4, "", 4096) = 0
10398 close(4) = 0
10398 munmap(0x7fdded933000, 4096) = 0
10398 open("/etc/ld.so.cache", O_RDONLY) = 4
10398 fstat(4, {st_mode=S_IFREG|0644, st_size=32064, ...}) = 0
10398 mmap(NULL, 32064, PROT_READ, MAP_PRIVATE, 4, 0) = 0x7fdded92c000
10398 close(4) = 0
10398 open("/lib64/libnss_dns.so.2", O_RDONLY) = 4
10398 read(4, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\20\0\0\0\0\0\0"..., 832) = 832
10398 fstat(4, {st_mode=S_IFREG|0755, st_size=27424, ...}) = 0
10398 mmap(NULL, 2117880, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 4, 0) = 0x7fdde6e9e000
10398 mprotect(0x7fdde6ea3000, 2093056, PROT_NONE) = 0
10398 mmap(0x7fdde70a2000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 4, 0x4000) = 0x7fdde70a2000
10398 close(4) = 0
10398 open("/lib64/libresolv.so.2", O_RDONLY) = 4
10398 read(4, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\00009\240\2242\0\0\0"..., 832) = 832
10398 fstat(4, {st_mode=S_IFREG|0755, st_size=113952, ...}) = 0
10398 mmap(NULL, 2202248, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 4, 0) = 0x7fdde6c84000
10398 mprotect(0x7fdde6c9a000, 2097152, PROT_NONE) = 0
10398 mmap(0x7fdde6e9a000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 4, 0x16000) = 0x7fdde6e9a000
10398 mmap(0x7fdde6e9c000, 6792, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fdde6e9c000
10398 close(4) = 0
10398 mprotect(0x7fdde6e9a000, 4096, PROT_READ) = 0
10398 mprotect(0x7fdde70a2000, 4096, PROT_READ) = 0
10398 munmap(0x7fdded92c000, 32064) = 0
10398 socket(PF_INET, SOCK_DGRAM|SOCK_NONBLOCK, IPPROTO_IP) = 4
10398 connect(4, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("69.64.66.11")}, 16) = 0
10398 poll([{fd=4, events=POLLOUT}], 1, 0) = 1 ([{fd=4, revents=POLLOUT}])
10398 sendto(4, "\246O\1\0\0\1\0\0\0\0\0\0\tnoexample\3com\0\0\1\0\1", 31, MSG_NOSIGNAL, NULL, 0) = 31
10398 poll([{fd=4, events=POLLIN}], 1, 5000) = 1 ([{fd=4, revents=POLLIN}])
10398 ioctl(4, FIONREAD, [31]) = 0
10398 recvfrom(4, "\246O\205\203\0\1\0\0\0\0\0\0\tnoexample\3com\0\0\1\0\1", 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("69.64.66.11")}, [16]) = 31
10398 close(4) = 0
10398 socket(PF_INET, SOCK_DGRAM|SOCK_NONBLOCK, IPPROTO_IP) = 4
10398 connect(4, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("69.64.66.11")}, 16) = 0
10398 poll([{fd=4, events=POLLOUT}], 1, 0) = 1 ([{fd=4, revents=POLLOUT}])
10398 sendto(4, "\372a\1\0\0\1\0\0\0\0\0\0\tnoexample\3com\3com\0\0"..., 35, MSG_NOSIGNAL, NULL, 0) = 35
10398 poll([{fd=4, events=POLLIN}], 1, 5000) = 1 ([{fd=4, revents=POLLIN}])
10398 ioctl(4, FIONREAD, [83]) = 0
10398 recvfrom(4, "\372a\201\200\0\1\0\2\0\0\0\0\tnoexample\3com\3com\0\0"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("69.64.66.11")}, [16]) = 83
10398 close(4) = 0
10398 socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 4
10398 connect(4, {sa_family=AF_INET, sin_port=htons(1025), sin_addr=inet_addr("64.30.224.112")}, 16) = 0
10398 getsockname(4, {sa_family=AF_INET, sin_port=htons(46615), sin_addr=inet_addr("69.64.68.204")}, [16]) = 0
10398 close(4) = 0
10398 setsockopt(3, SOL_RAW, ICMP_FILTER, ~(ICMP_ECHOREPLY|ICMP_DEST_UNREACH|ICMP_SOURCE_QUENCH|ICMP_REDIRECT|ICMP_TIME_EXCEEDED|ICMP_PARAMETERPROB), 4) = 0
10398 setsockopt(3, SOL_IP, IP_RECVERR, [1], 4) = 0
10398 setsockopt(3, SOL_SOCKET, SO_SNDBUF, [324], 4) = 0
10398 setsockopt(3, SOL_SOCKET, SO_RCVBUF, [65536], 4) = 0
10398 getsockopt(3, SOL_SOCKET, SO_RCVBUF, [-1318403725483442176], [4]) = 0
10398 fstat(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 0), ...}) = 0
10398 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fdded933000
10398 write(1, "PING phx1-ss-2-lb.cnet.com (64.3"..., 65) = 65
10398 setsockopt(3, SOL_SOCKET, SO_TIMESTAMP, [1], 4) = 0
10398 setsockopt(3, SOL_SOCKET, SO_SNDTIMEO, "\1\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0", 16) = 0
10398 setsockopt(3, SOL_SOCKET, SO_RCVTIMEO, "\1\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0", 16) = 0
10398 rt_sigaction(SIGINT, {0x7fdded93ca40, [], SA_RESTORER|SA_INTERRUPT, 0x7fdded175900}, NULL, 8) = 0
10398 rt_sigaction(SIGALRM, {0x7fdded93ca40, [], SA_RESTORER|SA_INTERRUPT, 0x7fdded175900}, NULL, 8) = 0
10398 rt_sigaction(SIGQUIT, {0x7fdded93ca50, [], SA_RESTORER|SA_INTERRUPT, 0x7fdded175900}, NULL, 8) = 0
10398 ioctl(1, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
10398 ioctl(1, TIOCGWINSZ, {ws_row=24, ws_col=80, ws_xpixel=0, ws_ypixel=0}) = 0
10398 sendmsg(3, {msg_name(16)={sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("64.30.224.112")}, msg_iov(1)=[{"\10\0b\315\236(\0\1\254xxO\0\0\0\0\21n\2\0\0\0\0\0\20\21\22\23\24\25\26\27"..., 64}], msg_controllen=0, msg_flags=0}, 0) = 64
10398 recvmsg(3, {msg_name(16)={sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("64.30.224.112")}, msg_iov(1)=[{"E\0\0T\0\0@\0\366\1\332\r@\36\340pE@D\314\0\0j\315\236(\0\1\254xxO"..., 192}], msg_controllen=32, {cmsg_len=32, cmsg_level=SOL_SOCKET, cmsg_type=0x1d /* SCM_??? */, ...}, msg_flags=0}, 0) = 84
10398 open("/etc/hosts", O_RDONLY|O_CLOEXEC) = 4
10398 fstat(4, {st_mode=S_IFREG|0644, st_size=280, ...}) = 0
10398 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fdded932000
10398 read(4, "# =============================="..., 4096) = 280
10398 read(4, "", 4096) = 0
10398 close(4) = 0
10398 munmap(0x7fdded932000, 4096) = 0
10398 socket(PF_INET, SOCK_DGRAM|SOCK_NONBLOCK, IPPROTO_IP) = 4
10398 connect(4, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("69.64.66.11")}, 16) = 0
10398 poll([{fd=4, events=POLLOUT}], 1, 0) = 1 ([{fd=4, revents=POLLOUT}])
10398 sendto(4, "\1w\1\0\0\1\0\0\0\0\0\0\003112\003224\00230\00264\7in-ad"..., 44, MSG_NOSIGNAL, NULL, 0) = 44
10398 poll([{fd=4, events=POLLIN}], 1, 5000) = 1 ([{fd=4, revents=POLLIN}])
10398 ioctl(4, FIONREAD, [79]) = 0
10398 recvfrom(4, "\1w\201\200\0\1\0\1\0\0\0\0\003112\003224\00230\00264\7in-ad"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("69.64.66.11")}, [16]) = 79
10398 close(4) = 0
10398 write(1, "64 bytes from phx1-ss-2-lb.cnet."..., 85) = 85
10398 poll([{fd=3, events=POLLIN|POLLERR}], 1, 965) = 0 (Timeout)
10398 sendmsg(3, {msg_name(16)={sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("64.30.224.112")}, msg_iov(1)=[{"\10\0~\305\236(\0\2\255xxO\0\0\0\0\364t\2\0\0\0\0\0\20\21\22\23\24\25\26\27"..., 64}], msg_controllen=0, msg_flags=0}, MSG_CONFIRM) = 64
10398 recvmsg(3, {msg_name(16)={sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("64.30.224.112")}, msg_iov(1)=[{"E\0\0T\0\0@\0\366\1\332\r@\36\340pE@D\314\0\0\206\305\236(\0\2\255xxO"..., 192}], msg_controllen=32, {cmsg_len=32, cmsg_level=SOL_SOCKET, cmsg_type=0x1d /* SCM_??? */, ...}, msg_flags=0}, 0) = 84
10398 write(1, "64 bytes from phx1-ss-2-lb.cnet."..., 85) = 85
10398 poll([{fd=3, events=POLLIN|POLLERR}], 1, 989) = 0 (Timeout)
10398 sendmsg(3, {msg_name(16)={sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("64.30.224.112")}, msg_iov(1)=[{"\10\0\23\277\236(\0\3\256xxO\0\0\0\0^z\2\0\0\0\0\0\20\21\22\23\24\25\26\27"..., 64}], msg_controllen=0, msg_flags=0}, MSG_CONFIRM) = 64
10398 recvmsg(3, {msg_name(16)={sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("64.30.224.112")}, msg_iov(1)=[{"E\0\0T\0\0@\0\366\1\332\r@\36\340pE@D\314\0\0\33\277\236(\0\3\256xxO"..., 192}], msg_controllen=32, {cmsg_len=32, cmsg_level=SOL_SOCKET, cmsg_type=0x1d /* SCM_??? */, ...}, msg_flags=0}, 0) = 84
10398 write(1, "64 bytes from phx1-ss-2-lb.cnet."..., 85) = 85
10398 poll([{fd=3, events=POLLIN|POLLERR}], 1, 988) = 0 (Timeout)
10398 sendmsg(3, {msg_name(16)={sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("64.30.224.112")}, msg_iov(1)=[{"\10\0l\271\236(\0\4\257xxO\0\0\0\0\4\177\2\0\0\0\0\0\20\21\22\23\24\25\26\27"..., 64}], msg_controllen=0, msg_flags=0}, MSG_CONFIRM) = 64
10398 recvmsg(3, {msg_name(16)={sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("64.30.224.112")}, msg_iov(1)=[{"E\0\0T\0\0@\0\366\1\332\r@\36\340pE@D\314\0\0t\271\236(\0\4\257xxO"..., 192}], msg_controllen=32, {cmsg_len=32, cmsg_level=SOL_SOCKET, cmsg_type=0x1d /* SCM_??? */, ...}, msg_flags=0}, 0) = 84
10398 write(1, "64 bytes from phx1-ss-2-lb.cnet."..., 85) = 85
10398 poll([{fd=3, events=POLLIN|POLLERR}], 1, 988) = 0 (Timeout)
10398 sendmsg(3, {msg_name(16)={sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("64.30.224.112")}, msg_iov(1)=[{"\10\0e\264\236(\0\5\260xxO\0\0\0\0\n\203\2\0\0\0\0\0\20\21\22\23\24\25\26\27"..., 64}], msg_controllen=0, msg_flags=0}, MSG_CONFIRM) = 64
10398 recvmsg(3, {msg_name(16)={sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("64.30.224.112")}, msg_iov(1)=[{"E\0\0T\0\0@\0\366\1\332\r@\36\340pE@D\314\0\0m\264\236(\0\5\260xxO"..., 192}], msg_controllen=32, {cmsg_len=32, cmsg_level=SOL_SOCKET, cmsg_type=0x1d /* SCM_??? */, ...}, msg_flags=0}, 0) = 84
10398 write(1, "64 bytes from phx1-ss-2-lb.cnet."..., 85) = 85
10398 poll([{fd=3, events=POLLIN|POLLERR}], 1, 988) = ? ERESTART_RESTARTBLOCK (To be restarted)
10398 --- SIGINT (Interrupt) @ 0 (0) ---
10398 rt_sigreturn(0x2) = -1 EINTR (Interrupted system call)
10398 write(1, "\n", 1) = 1
10398 write(1, "--- phx1-ss-2-lb.cnet.com ping s"..., 46) = 46
10398 write(1, "5 packets transmitted, 5 receive"..., 63) = 63
10398 write(1, "rtt min/avg/max/mdev = 11.885/11"..., 53) = 53
10398 exit_group(0)
[/quote][Moderator edit: Changed [i]code[/i] to [i]quote[/i] tags to wrap [b]long[/b] lines and prevent running-off-the-right-side syndrome.]
Re: getaddrinfo NXDOMAIN hijack
Other libraries loaded, just for completeness.
/lib64/libnss_dns.so.2
/lib64/libresolv.so.2
/usr/lib/locale/locale-archive
/etc/ld.so.cache
/lib64/libnss_files.so.2
Somewhere in all those libraries is the code that's doing this.
/lib64/libnss_dns.so.2
/lib64/libresolv.so.2
/usr/lib/locale/locale-archive
/etc/ld.so.cache
/lib64/libnss_files.so.2
Somewhere in all those libraries is the code that's doing this.
Re: getaddrinfo NXDOMAIN hijack
I'm looking at
__libc_res_nsearch
in res_query.c
in glibc sources at
http://sourceware.org/git/?p=glibc.git;a=blame;f=resolv/res_query.c;h=abccd4a92105d4bc105b2b04307c7d03440bea38;hb=master
There's code there which constructs a "name" and a "domain", and concatenates them before sending the result
to the DNS server. Take a look at the loop beginning at line 407. That has code which, on a no find, appends
domains from a list of domains and tries again. Note the comment,
* If name isn't found in this domain,
* keep trying higher domains in the search list
* (if that's enabled).
What that code does is determined by the "statp" variable, of type res_state.
I'm working on where that comes from.
(There are three decades of special case hacks in this code. It's a mess.)
__libc_res_nsearch
in res_query.c
in glibc sources at
http://sourceware.org/git/?p=glibc.git;a=blame;f=resolv/res_query.c;h=abccd4a92105d4bc105b2b04307c7d03440bea38;hb=master
There's code there which constructs a "name" and a "domain", and concatenates them before sending the result
to the DNS server. Take a look at the loop beginning at line 407. That has code which, on a no find, appends
domains from a list of domains and tries again. Note the comment,
* If name isn't found in this domain,
* keep trying higher domains in the search list
* (if that's enabled).
What that code does is determined by the "statp" variable, of type res_state.
I'm working on where that comes from.
(There are three decades of special case hacks in this code. It's a mess.)
Re: getaddrinfo NXDOMAIN hijack
I think I see what's going on. See the man page for "resolv.conf:
http://linux.die.net/man/5/resolv.conf
Note what the default is:
[b]domain Local domain name.[/b]
Most queries for names within this domain can use short names relative to the local domain.
If no domain entry is present, the domain is determined from the local hostname returned by gethostname(2);
the domain part is taken to be [b]everything after the first '.'[/b]. Finally, if the hostname does not contain a domain part,
the root domain is assumed.
In this case, the default name of the server is "sitetruth.com". So the "domain part" is ".com", and any failed
lookups are retried with ".com" appended.
Why doesn't this happen all the time? Because most servers have names assigned by some hosting service, like "gator123.hostgator.com".
In such cases, the default domain is "hostgator.com", and that's what gets appended on failed domain searches. If your server
has a two-component name as its main name, though, there's a problem. The default in "resolv" is badly chosen.
Setting "ndots" to 0, or providing an empty "search" line ought to disable this behavior, but so far, it's not doing so.
==
Submitted glibc bug report:
http://sourceware.org/bugzilla/show_bug.cgi?id=13935
http://linux.die.net/man/5/resolv.conf
Note what the default is:
[b]domain Local domain name.[/b]
Most queries for names within this domain can use short names relative to the local domain.
If no domain entry is present, the domain is determined from the local hostname returned by gethostname(2);
the domain part is taken to be [b]everything after the first '.'[/b]. Finally, if the hostname does not contain a domain part,
the root domain is assumed.
In this case, the default name of the server is "sitetruth.com". So the "domain part" is ".com", and any failed
lookups are retried with ".com" appended.
Why doesn't this happen all the time? Because most servers have names assigned by some hosting service, like "gator123.hostgator.com".
In such cases, the default domain is "hostgator.com", and that's what gets appended on failed domain searches. If your server
has a two-component name as its main name, though, there's a problem. The default in "resolv" is badly chosen.
Setting "ndots" to 0, or providing an empty "search" line ought to disable this behavior, but so far, it's not doing so.
==
Submitted glibc bug report:
http://sourceware.org/bugzilla/show_bug.cgi?id=13935