9-stream no dhcp lease to a new vm with bridged networking

Issues related to configuring your network
user9452
Posts: 14
Joined: 2020/11/14 11:36:08

9-stream no dhcp lease to a new vm with bridged networking

Post by user9452 » 2022/08/09 04:09:15

I have a openbsd as a vm firewall which takes care of the routing and acts as an dhcp server. I migrated a couple of vm's from fedora34 hypervisor to 9-stream and on those the networking works as expected. As a test I destroyed the old leases and the all the old vm's got a new ones as expected. So as I understand, the issue shouldn't be on the firewall.

However when I create a new vm with virsh, the networking wont work at all. It wont get a dhcp lease.

I have tried disabling the usual suspects firewalld and selinux as test to see if those are the problem but no they're not.

First I created the bridges with nm but now changed the method to systemd-networkd but on both same issue arises.

Also when I have to shutdown the new vms which runs ubuntu 20.04 and 22.04, I get no response. I have to destroy them.

Non working vm interface setup:
<interface type='bridge'>
<mac address='52:54:00:8b:63:ea'/>
<source bridge='br1'/>
<model type='virtio'/>
<address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
</interface>
and a working one:
<interface type='bridge'>
<mac address='52:54:00:3a:c4:13'/>
<source bridge='br1'/>
<model type='virtio'/>
<address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
</interface>
bridges to firewall vm:
<interface type='bridge'>
<mac address='52:54:00:97:d4:8e'/>
<source bridge='br0'/>
<model type='virtio'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
</interface>
interface type='bridge'>
<mac address='52:54:00:15:8d:7f'/>
<source bridge='br1'/>
<model type='virtio'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
</interface>
<interface type='bridge'>
<mac address='52:54:00:b0:0a:6d'/>
<source bridge='br2'/>
<model type='virtio'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
</interface>
<interface type='bridge'>
<mac address='52:54:00:23:db:22'/>
<source bridge='br3'/>
<model type='virtio'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x0a' function='0x0'/>
</interface>
output of neworkctl:
1 lo loopback carrier unmanaged
2 enp1s0 ether routable configured
3 ens1f0 ether enslaved configured
4 ens1f1 ether enslaved configured
5 ens1f2 ether enslaved configured
6 ens1f3 ether no-carrier configured
7 br0 bridge routable configured
8 br1 bridge routable configured
9 br2 bridge routable configured
10 br3 bridge degraded-carrier configured
11 vnet0 ether enslaved unmanaged
12 vnet1 ether enslaved unmanaged
13 vnet2 ether enslaved unmanaged
14 vnet3 ether enslaved unmanaged
15 vnet4 ether enslaved unmanaged
16 vnet5 ether enslaved unmanaged
22 vnet10 ether enslaved unmanaged
br3 is currently not on use

Also when I chech ips with ip a on hypervisor I get ip on every bridge when I want one only on br1...I don't understand what I have misconfigured since br2 and br3 didnt get ip's on previous fedora34 hypervisor with similar setup
ip a:
r1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 7a:6d:56:af:bc:ec brd ff:ff:ff:ff:ff:ff
inet 192.168.10.101/24 metric 1024 brd 192.168.10.255 scope global dynamic br1
valid_lft 39782sec preferred_lft 39782sec
inet6 fe80::786d:56ff:feaf:bcec/64 scope link
valid_lft forever preferred_lft forever
9: br2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 06:6a:75:91:a1:4b brd ff:ff:ff:ff:ff:ff
inet 192.168.20.102/24 metric 1024 brd 192.168.20.255 scope global dynamic br2
valid_lft 34448sec preferred_lft 34448sec
inet6 fe80::46a:75ff:fe91:a14b/64 scope link
valid_lft forever preferred_lft forever
10: br3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 9a:12:7e:54:b5:c9 brd ff:ff:ff:ff:ff:ff
inet 192.168.30.100/24 metric 1024 brd 192.168.30.255 scope global dynamic br3
valid_lft 34351sec preferred_lft 34351sec
inet6 fe80::9812:7eff:fe54:b5c9/64 scope link
valid_lft forever preferred_lft forever

User avatar
TrevorH
Site Admin
Posts: 33202
Joined: 2009/09/24 10:40:56
Location: Brighton, UK

Re: 9-stream no dhcp lease to a new vm with bridged networking

Post by TrevorH » 2022/08/09 11:27:53

I believe NetworkManager in Stream 8 and 9 is currently broken for DHCP use.
The future appears to be RHEL or Debian. I think I'm going Debian.
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke

User avatar
jlehtone
Posts: 4523
Joined: 2007/12/11 08:17:33
Location: Finland

Re: 9-stream no dhcp lease to a new vm with bridged networking

Post by jlehtone » 2022/08/09 12:29:51

First, I don't know systemd-networkd so can't tell how it should activate interfaces without addresses.
With nmcli the options would be ipv4.method disabled ipv6.method ignore

In any case one should be able to listen with tcpdump (or other tool) at various points.
At DHCP server whether requests from 52:54:00:8b:63:ea do arrive.
At ens1fX of host that is enslaved to the bridge br1 and at br1.

If no DHCP traffic from guest does appear at br1, then issue is between them or in the guest.
If no DHCP traffic from guest does reach/leave the ens1fX, then issue is in host config.
If request arrives to BSD, but there is no reply, then it is the DHCP server that should be checked.

user9452
Posts: 14
Joined: 2020/11/14 11:36:08

Re: 9-stream no dhcp lease to a new vm with bridged networking

Post by user9452 » 2022/08/11 04:17:59

jlehtone wrote:
2022/08/09 12:29:51
First, I don't know systemd-networkd so can't tell how it should activate interfaces without addresses.
With nmcli the options would be ipv4.method disabled ipv6.method ignore

In any case one should be able to listen with tcpdump (or other tool) at various points.
At DHCP server whether requests from 52:54:00:8b:63:ea do arrive.
At ens1fX of host that is enslaved to the bridge br1 and at br1.

If no DHCP traffic from guest does appear at br1, then issue is between them or in the guest.
If no DHCP traffic from guest does reach/leave the ens1fX, then issue is in host config.
If request arrives to BSD, but there is no reply, then it is the DHCP server that should be checked.
I get dhcp requests from old imported vms but nothing from new ones. The setup is identical. Even when I just copy all the virsh settings, the behaviour between vms is different.

Also I can't watch from vnc if the vm even boots properly since this isn't supported in 9-stream for whatever reason. I had to remove all vnc from settings it when I migrated from fedora34. I guess I need to figure out to get an image with enabled vga console since that is the only way to connected to a vm without networking.

User avatar
jlehtone
Posts: 4523
Joined: 2007/12/11 08:17:33
Location: Finland

Re: 9-stream no dhcp lease to a new vm with bridged networking

Post by jlehtone » 2022/08/11 06:42:55

The bridge that the VM is connected to does show DHCP traffic only for "working" guests?

Do the bridges have STP off?
user9452 wrote:
2022/08/11 04:17:59
Also I can't watch from vnc if the vm even boots properly since this isn't supported in 9-stream for whatever reason. I had to remove all vnc from settings it when I migrated from fedora34.
That is interesting. Red Hat did remove Spice and QXL from libvirt of RHEL9 and chose to keep VNC.
I had to shift to VNC on VM's that I migrated to EL9 (Alma 9) host.
The 9-stream being what it is, it should not be able to lack VNC.

user9452
Posts: 14
Joined: 2020/11/14 11:36:08

Re: 9-stream no dhcp lease to a new vm with bridged networking

Post by user9452 » 2022/08/11 09:55:13

jlehtone wrote:
2022/08/11 06:42:55
The bridge that the VM is connected to does show DHCP traffic only for "working" guests?

Do the bridges have STP off?
yes.

I need to take a look of that STP tomorrow when I have the time since I don't even know what that is. This is a homelab setup for learning.
jlehtone wrote:
2022/08/11 06:42:55
user9452 wrote:
2022/08/11 04:17:59
Also I can't watch from vnc if the vm even boots properly since this isn't supported in 9-stream for whatever reason. I had to remove all vnc from settings it when I migrated from fedora34.
That is interesting. Red Hat did remove Spice and QXL from libvirt of RHEL9 and chose to keep VNC.
I had to shift to VNC on VM's that I migrated to EL9 (Alma 9) host.
The 9-stream being what it is, it should not be able to lack VNC.
I remembered wrong. It was only Spice and QXL that got removed and NOT VNC. My mistake. I used Spice before.

User avatar
jlehtone
Posts: 4523
Joined: 2007/12/11 08:17:33
Location: Finland

Re: 9-stream no dhcp lease to a new vm with bridged networking

Post by jlehtone » 2022/08/11 10:26:35

STP is spanning tree protocol. Network switches do use it to detect and prevent loops. A bridge is a switch.

Lets say you have PC connected to switch A that is connected to switch B: PC--A--B
The PC sends a broadcast (bcast). A will send it to all its other ports. B gets broadcast from A and sends it to all other ports. Everyone did receive the bcast as they should.

Add switch C that creates a loop:

Code: Select all

PC -- A -- B
       \  /
        C
Now both B and C receive bcast from A.
B forwards it to C, which forwards it to A, which ...
At the same time C forwards packets to B, which forwards them to A, which ...
A broadcast storm. Not good.
With STP the switches detect the loop and start to limit where they forward packets.
If we know for certain that a bridge is not part of a loop, then we can disable STP.


I do use virt-manager to manage VM's (even though I'm not fond of GUI). (Red Hat has deprecated it, but at least Alma 9 still has that package.)
In virt-manager I did change the 'Type' of 'Display' of VM from "Spice server" into "VNC server" and the 'Model' of "Video" from "QXL" to "Virtio". There were some additional "Spice" devices in VM that I did remove. The virt-manager can show console of the VM.

user9452
Posts: 14
Joined: 2020/11/14 11:36:08

Re: 9-stream no dhcp lease to a new vm with bridged networking

Post by user9452 » 2022/08/11 15:25:39

ok now all of a sudden it works and I have no idea why. I did run a bigger update but I don't think that is the reason. But I didn't change anything.
TrevorH wrote:
2022/08/09 11:27:53
I believe NetworkManager in Stream 8 and 9 is currently broken for DHCP use.
Do you have any more info on this? How is this possible on a distro which should be a upstream for tons and tons of production server

User avatar
TrevorH
Site Admin
Posts: 33202
Joined: 2009/09/24 10:40:56
Location: Brighton, UK

Re: 9-stream no dhcp lease to a new vm with bridged networking

Post by TrevorH » 2022/08/11 15:49:45

Do you have any more info on this? How is this possible on a distro which should be a upstream for tons and tons of production server
This is Stream. It's a beta testing ground for RH engineers to push changes to. A certain proportion of those are always going to be broken. This is the reason why many of us would not use Stream if you paid us.
The future appears to be RHEL or Debian. I think I'm going Debian.
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke

user9452
Posts: 14
Joined: 2020/11/14 11:36:08

Re: 9-stream no dhcp lease to a new vm with bridged networking

Post by user9452 » 2022/08/12 05:27:29

TrevorH wrote:
2022/08/11 15:49:45
Do you have any more info on this? How is this possible on a distro which should be a upstream for tons and tons of production server
This is Stream. It's a beta testing ground for RH engineers to push changes to. A certain proportion of those are always going to be broken. This is the reason why many of us would not use Stream if you paid us.
My experience with fedora34 as a hypervisor was very positive and I didn't have any significant issues. So my thought stream-9 would be more matrue platform since as I understarnd it's based on fedora34. The migration has been more difficulat than I expected. Anyway I'm doing this to learn and I did so again despite I'm a bit baffled what the issue actually was.

And thank you jlehtone for your STP and VNC setup explanation!

Post Reply