CentOS 8 not getting DCHP IP at boot

Issues related to configuring your network
Post Reply
prasanna@cloudraxak
Posts: 5
Joined: 2020/11/10 19:37:35

CentOS 8 not getting DCHP IP at boot

Post by prasanna@cloudraxak » 2020/11/10 19:57:01

I just brought up a CentOS 8 VM under KVM on a CentOS 7 host. The network on the host is a MACVTAP bridge. CentOS 7 guests come up fine and get DHCP addresses assigned so I know the overall system works.

My CentOS 8 VM comes up but the ens3 device does not get a DHCP address. journalctl shows DHCP times out. However manually running dhclient ens3 after booting works just fine 100% of the time, and without any long delay.

Any ideas? I cannot find anything on this forum other than one other issue with virtual box that sounds similar. But no resolution.

aks
Posts: 3073
Joined: 2014/09/20 11:22:14

Re: CentOS 8 not getting DCHP IP at boot

Post by aks » 2020/11/14 19:10:06

Look at what's happening on the wire.
Ask is usually broadcast, answer if often unicast.
What;s happening in your scenario?

prasanna@cloudraxak
Posts: 5
Joined: 2020/11/10 19:37:35

Re: CentOS 8 not getting DCHP IP at boot

Post by prasanna@cloudraxak » 2020/11/18 19:00:18

I have not done a wireshark.

Looking at the log entries in journalctl, I see that dhcp times out during the boot sequence:
Nov 18 13:44:10 CentOS8 NetworkManager[728]: <info> [1605725050.2378] dhcp-init: Using DHCP client 'internal'
Nov 18 13:44:10 CentOS8 NetworkManager[728]: <info> [1605725050.3457] dhcp4 (ens3): activation: beginning transaction (timeout in 45 seconds)
Nov 18 13:44:55 CentOS8 NetworkManager[728]: <warn> [1605725095.6430] dhcp4 (ens3): request timed out
Nov 18 13:44:55 CentOS8 NetworkManager[728]: <info> [1605725095.6434] dhcp4 (ens3): state changed unknown -> timeout
Nov 18 13:44:55 CentOS8 NetworkManager[728]: <info> [1605725095.6457] dhcp4 (ens3): canceled DHCP transaction
Nov 18 13:44:55 CentOS8 NetworkManager[728]: <info> [1605725095.6457] dhcp4 (ens3): state changed timeout -> done
Nov 18 13:44:55 CentOS8 NetworkManager[728]: <info> [1605725095.6718] dhcp4 (ens3): activation: beginning transaction (timeout in 45 seconds)
Nov 18 13:45:40 CentOS8 NetworkManager[728]: <warn> [1605725140.6323] dhcp4 (ens3): request timed out
Nov 18 13:45:40 CentOS8 NetworkManager[728]: <info> [1605725140.6333] dhcp4 (ens3): state changed unknown -> timeout
Nov 18 13:45:40 CentOS8 NetworkManager[728]: <info> [1605725140.6402] dhcp4 (ens3): canceled DHCP transaction
Nov 18 13:45:40 CentOS8 NetworkManager[728]: <info> [1605725140.6403] dhcp4 (ens3): state changed timeout -> done
Nov 18 13:45:40 CentOS8 NetworkManager[728]: <info> [1605725140.7001] dhcp4 (ens3): activation: beginning transaction (timeout in 45 seconds)
Nov 18 13:46:25 CentOS8 NetworkManager[728]: <warn> [1605725185.6323] dhcp4 (ens3): request timed out
Nov 18 13:46:25 CentOS8 NetworkManager[728]: <info> [1605725185.6324] dhcp4 (ens3): state changed unknown -> timeout
Nov 18 13:46:25 CentOS8 NetworkManager[728]: <info> [1605725185.6394] dhcp4 (ens3): canceled DHCP transaction
Nov 18 13:46:25 CentOS8 NetworkManager[728]: <info> [1605725185.6395] dhcp4 (ens3): state changed timeout -> done
Nov 18 13:46:25 CentOS8 NetworkManager[728]: <info> [1605725185.6962] dhcp4 (ens3): activation: beginning transaction (timeout in 45 seconds)
Nov 18 13:47:10 CentOS8 NetworkManager[728]: <warn> [1605725230.6323] dhcp4 (ens3): request timed out
Nov 18 13:47:10 CentOS8 NetworkManager[728]: <info> [1605725230.6325] dhcp4 (ens3): state changed unknown -> timeout
Nov 18 13:47:10 CentOS8 NetworkManager[728]: <info> [1605725230.6392] dhcp4 (ens3): canceled DHCP transaction
Nov 18 13:47:10 CentOS8 NetworkManager[728]: <info> [1605725230.6392] dhcp4 (ens3): state changed timeout -> done
Nov 18 13:52:10 CentOS8 NetworkManager[728]: <info> [1605725530.6710] dhcp4 (ens3): activation: beginning transaction (timeout in 45 seconds)
Nov 18 13:52:55 CentOS8 NetworkManager[728]: <warn> [1605725575.6323] dhcp4 (ens3): request timed out
Nov 18 13:52:55 CentOS8 NetworkManager[728]: <info> [1605725575.6326] dhcp4 (ens3): state changed unknown -> timeout
Nov 18 13:52:55 CentOS8 NetworkManager[728]: <info> [1605725575.6401] dhcp4 (ens3): canceled DHCP transaction
Nov 18 13:52:55 CentOS8 NetworkManager[728]: <info> [1605725575.6401] dhcp4 (ens3): state changed timeout -> done
Nov 18 13:52:55 CentOS8 NetworkManager[728]: <info> [1605725575.7034] dhcp4 (ens3): activation: beginning transaction (timeout in 45 seconds)
Nov 18 13:52:59 CentOS8 dhclient[4869]: /etc/dhcp/dhclient.conf line 12: Expecting numeric lease time
Nov 18 13:53:40 CentOS8 NetworkManager[728]: <warn> [1605725620.6322] dhcp4 (ens3): request timed out
Nov 18 13:53:40 CentOS8 NetworkManager[728]: <info> [1605725620.6326] dhcp4 (ens3): state changed unknown -> timeout
Nov 18 13:53:40 CentOS8 NetworkManager[728]: <info> [1605725620.6398] dhcp4 (ens3): canceled DHCP transaction
Nov 18 13:53:40 CentOS8 NetworkManager[728]: <info> [1605725620.6401] dhcp4 (ens3): state changed timeout -> done
Nov 18 13:53:40 CentOS8 NetworkManager[728]: <info> [1605725620.7006] dhcp4 (ens3): activation: beginning transaction (timeout in 45 seconds)
However, once the system completes booting, a manual `dhclient ens3` works and gets a ip address fine.

The difference seems to be that the boot sequence uses dhcp4. I don't know if that is the same as the /etc/dhcp/dhclient that gets called with a manual dhclient execution.

User avatar
jlehtone
Posts: 4523
Joined: 2007/12/11 08:17:33
Location: Finland

Re: CentOS 8 not getting DCHP IP at boot

Post by jlehtone » 2020/11/19 12:42:14

Rather than 'dhclient ens3' you should try

Code: Select all

sudo nmcli con up ens3
(assuming 'ens3' is the name of the connection)

That should try to activate the connection "the NetworkManager way".

The usual diagnostic commands that I run are:

Code: Select all

nmcli con show
nmcli dev state
nmcli
ip ro
Software bridge on CentOS 6 host and (libvirt) KVM VM did fail DHCP at PXEboot (install) due to the bridge having (STP) forward delay set. I presume that MACVTAP does not have that issue.

You should nevertheless be able to wireshark (or tcpdump) at the host (or does MACVTAP hide the port from host?)

prasanna@cloudraxak
Posts: 5
Joined: 2020/11/10 19:37:35

Re: CentOS 8 not getting DCHP IP at boot

Post by prasanna@cloudraxak » 2020/11/19 16:48:36

Well. I tried switching from MACVTAP to a regular bridged connection, following the directions in the libvirt wiki: https://wiki.libvirt.org/page/Networking

Same issue as before. There is no ip address assigned.

Code: Select all

nmcli con up ens3
hangs and times out after 90 seconds

Code: Select all

nmcli con show
shows ens3 as a ethernet type device.
nmcli dev shows the state as

Code: Select all

 connecting (getting IP config information)
Doing a nmcli con down ens3 before the up command does not help.

But after the dhclient ens3 command everything comes up and running.
The /etc/sysconfig/network-scripts/ifcfg-ens3 script looks like this:

Code: Select all

[root@CentOS8 prasanna]# cat /etc/sysconfig/network-scripts/ifcfg-ens3 
TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=dhcp
NAME=ens3
DEVICE=ens3
ONBOOT=yes
DNS1=8.8.8.8
DNS2=8.8.4.4
PERSISTENT_DHCLIENT="yes"
IPV4_DHCP_TIMEOUT=180
After doing a dhclient ens3 and getting an ip address, I can connect to it via ssh and all. And the nmcli dev at that point shows:

Code: Select all

[root@CentOS8 prasanna]# nmcli dev
DEVICE  TYPE      STATE      CONNECTION 
ens3    ethernet  connected  ens3       
lo      loopback  unmanaged  --  

For what it is worth: During the boot, or if I do a systemctl restart NetworkManager, journalctl shows errors of the form:

Code: Select all

journalctl -b | grep -Ei dhcp
Nov 19 13:03:46 CentOS8 NetworkManager[1337]: <info>  [1605809026.9901] dhcp-init: Using DHCP client 'internal'
Nov 19 13:03:47 CentOS8 NetworkManager[1337]: <info>  [1605809027.0173] dhcp4 (ens3): activation: beginning transaction (timeout in 180 seconds)
Nov 19 13:06:47 CentOS8 NetworkManager[1337]: <warn>  [1605809207.5822] dhcp4 (ens3): request timed out
Nov 19 13:06:47 CentOS8 NetworkManager[1337]: <info>  [1605809207.5824] dhcp4 (ens3): state changed unknown -> timeout
Nov 19 13:06:47 CentOS8 NetworkManager[1337]: <info>  [1605809207.5826] device (ens3): DHCPv4: trying to acquire a new lease within 360 seconds
Nov 19 13:08:25 CentOS8 NetworkManager[1337]: <info>  [1605809305.8666] dhcp4 (ens3): canceled DHCP transaction
Nov 19 13:08:25 CentOS8 NetworkManager[1337]: <info>  [1605809305.8666] dhcp4 (ens3): state changed timeout -> done
Nov 19 13:08:49 CentOS8 NetworkManager[1427]: <info>  [1605809329.3717] dhcp-init: Using DHCP client 'internal'
Nov 19 13:08:49 CentOS8 NetworkManager[1427]: <info>  [1605809329.4076] dhcp4 (ens3): activation: beginning transaction (timeout in 180 seconds)
Nov 19 13:09:10 CentOS8 NetworkManager[1427]: <info>  [1605809350.2877] dhcp4 (ens3): canceled DHCP transaction
Nov 19 13:09:10 CentOS8 NetworkManager[1427]: <info>  [1605809350.2877] dhcp4 (ens3): state changed unknown -> done
If I manually do a dhclient ens3, journalctl shows:

Code: Select all

journalctl -b | grep -Ei dhcp
Nov 19 13:09:16 CentOS8 dhclient[1450]: DHCPREQUEST on ens3 to 255.255.255.255 port 67 (xid=0x3948481a)
Nov 19 13:09:18 CentOS8 dhclient[1450]: DHCPACK from 192.168.1.1 (xid=0x3948481a)

prasanna@cloudraxak
Posts: 5
Joined: 2020/11/10 19:37:35

Re: CentOS 8 not getting DCHP IP at boot

Post by prasanna@cloudraxak » 2020/11/19 22:53:45

Update:
On a second CentOS 8 VM configured like the first (talking to bridge br0, not macvtap), I replaced the NetworkManager with network-scripts (basically following the instructions in: https://www.thegeekdiary.com/how-to-dis ... os-rhel-8/

With systemd networking, everything works smoothly. VM gets a DHCP address as it should at boot.

So the issue seems to be pointing to NetworkManager and not the virtualization/networking setup.

--prasanna

User avatar
jlehtone
Posts: 4523
Joined: 2007/12/11 08:17:33
Location: Finland

Re: CentOS 8 not getting DCHP IP at boot

Post by jlehtone » 2020/11/20 11:14:09

prasanna@cloudraxak wrote:
2020/11/19 22:53:45
So the issue seems to be pointing to NetworkManager and not the virtualization/networking setup.
Perhaps.

Note that you can set NetworkManager to use dhclient:
man 5 NetworkManager.conf wrote:dhcp
This key sets up what DHCP client NetworkManager will use. Allowed values are dhclient, dhcpcd, and internal. The dhclient and dhcpcd options require the indicated clients to be installed. The internal option uses a built-in DHCP client which is not currently as featureful as the external clients.

If this key is missing, it defaults to internal. It the chosen plugin is not available, clients are looked for in this order: dhclient, dhcpcd, internal.

prasanna@cloudraxak
Posts: 5
Joined: 2020/11/10 19:37:35

Re: CentOS 8 not getting DCHP IP at boot

Post by prasanna@cloudraxak » 2020/11/20 16:50:33

Bingo!

Setting the dhcp=dhclient solved the issue and the CentOS 8 with NetworkManager now boots and gets the IP address just fine.

Thanks jlehtone! :D

--prasanna

User avatar
jlehtone
Posts: 4523
Joined: 2007/12/11 08:17:33
Location: Finland

Re: CentOS 8 not getting DCHP IP at boot

Post by jlehtone » 2020/11/20 17:22:55

I would not say "solve" as it does not explain why the internal DHCP client fails in your environment.

However, it is a valid workaround.

Post Reply