Cant get Network Bridge to work propperly

[AimLikeAProtato]

Hello,
I have a problem setting up our Centos 8.2.2004 as Hypervisor. I want to have a network Bridge with an IP Adress to access the server. The Guest systems would connect to this bridge to get their own ip adress.
Our Network is:

192.168.0.0/24
Gateway: 192.168.0.1
DNS: 192.168.0.1,192.168.0.4
Hypervisor: 192.168.0.202 (temporary)

i used nmcli to add the bridge br1 to the interface enp1s0f3

My Problem is that i am able to ping the gateway from the hypervisor. I can not ping the hypervisor from the gateway
I can not ping out of out network like 8.8.8.8

Does anyone know were there might be a mistake or what i am missing???

here is some usefull information like connections and config files:
[root@localhost ~]# nmcli connection show
NAME UUID TYPE DEVICE
enp1s0f2 96e96965-4caf-4505-992e-f7f3d7bc879a ethernet enp1s0f2
br1 50661144-28c6-47a8-a8cb-63e5874094e5 bridge br1
enp1s0f3 1c257f17-96ab-4f20-92ca-808f1bc62b67 ethernet enp1s0f3
enp1s0f0 7e9d5322-5f5f-4e33-a2b4-3cab4a93a487 ethernet --
enp1s0f1 d5e0ce14-a2ae-43e3-8b46-5dab78c1a27b ethernet --
enp7s0f3u2u3c2 4c4d06fc-c0b3-4993-94c6-40321d0b1900 ethernet --

[root@localhost ~]# cat /etc/sysconfig/network-scripts/ifcfg-br1
STP=no
BRIDGING_OPTS=priority=32768
TYPE=Bridge
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=none
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=br1
UUID=50661144-28c6-47a8-a8cb-63e5874094e5
DEVICE=br1
ONBOOT=yes
IPADDR=192.168.0.202
PREFIX=24
GATEWAY=192.168.0.1
DNS1=192.168.0.4
DNS2=192.168.0.1
DNS3=1.1.1.1

[root@localhost ~]# cat /etc/sysconfig/network-scripts/ifcfg-enp1s0f3
TYPE=Ethernet
NAME=enp1s0f3
UUID=1c257f17-96ab-4f20-92ca-808f1bc62b67
DEVICE=enp1s0f3
ONBOOT=yes
BRIDGE=br1

[jlehtone]

Code: Select all

enp1s0f2 96e96965-4caf-4505-992e-f7f3d7bc879a ethernet enp1s0f2
br1 50661144-28c6-47a8-a8cb-63e5874094e5 bridge br1
enp1s0f3 1c257f17-96ab-4f20-92ca-808f1bc62b67 ethernet enp1s0f3

I see that br1 is not your only connection. The config of enp1s0f2 can probably explain some of your issues.

Elementary check, what you get with:

Code: Select all

ip ro

[AimLikeAProtato]

in the meantime i tried to redo the whole configuration. Nothing changed except i deleted the old bridge and the new name is br2. still the same issues.
I have also seen the behaviour that sometimes if i ping the system i get 1 reply and then nothing.

ip ro command is now:

default via 192.168.0.1 dev br2 proto static metric 425
192.168.0.0/24 dev br2 proto kernel scope link src 192.168.0.202 metric 425
192.168.122.0/24 dev vibro proto kernel scope link src 192.168.122.1 linkdown

[leifh]

Hi, have an (almost) indentical setup. Using vlans on my bridges, but config should be similar.
I am using nmcli commands to setup my network configuration (tested on centos 8.1 and 8.2)

the example underneath have not been tested, but should work.

ipv4 and ipv6 disabled on my interface when i start. no other default routes active on any other interfaces.

# cat /etc/sysconfig/network-scripts/ifcfg-ens1f0
TYPE=Ethernet
DEVICE=ens1f0
ONBOOT=no
IPV6INIT=no
PROXY_METHOD=none
BROWSER_ONLY=no
NAME="System ens1f0"

First i create my bridge:
nmcli con add type bridge ifname br1 con-name br1 ipv4.method manual ipv4.addresses "192.168.0.202" ipv4.dns "192.168.0.4,192.168.0.1" ipv4.gateway "192.168.0.1" connection.autoconnect yes ipv6.method disabled

]# cat /etc/sysconfig/network-scripts/ifcfg-br1
STP=yes
BRIDGING_OPTS=priority=32768
TYPE=Bridge
PROXY_METHOD=none
BROWSER_ONLY=no
IPV6_DISABLED=yes
IPV6INIT=no
NAME=br1
DEVICE=br1
ONBOOT="yes"
BOOTPROTO="static"
IPADDR="192.168.0.202"
NETMASK="255.255.255.0"
GATEWAY="192.168.0.1"
IPV6INIT="no"
DNS1="192.168.0.4"
DNS2="192.168.0.1"
DNS3=""


Then ! tell ens1f0 br1 is master (added autostart connection, not sure it is needed)
#nmcli connection modify ens1f0 master br1 slave-type bridge connection.autoconnect yes

#nmcli con up br1

[s91066]

I have an identical issue with the original poster.
Setup: Centos 8.1

NM with only one interface and only 2 connections:

nmcli c s server_virt-slave

Code: Select all

connection.id:                          centos_virt-slave
connection.uuid:                        8a6be03b-debc-472d-a44d-eac7145b6ae0
connection.stable-id:                   --
connection.type:                        802-3-ethernet
connection.interface-name:              eth0
connection.autoconnect:                 yes
connection.autoconnect-priority:        0
connection.autoconnect-retries:         -1 (default)
connection.multi-connect:               0 (default)
connection.auth-retries:                -1
connection.timestamp:                   1600532164
connection.read-only:                   no
connection.permissions:                 --
connection.zone:                        --
connection.master:                      centos_virt
connection.slave-type:                  bridge
connection.autoconnect-slaves:          -1 (default)
connection.secondaries:                 --
connection.gateway-ping-timeout:        0
connection.metered:                     unknown
connection.lldp:                        default
connection.mdns:                        -1 (default)
connection.llmnr:                       -1 (default)
connection.wait-device-timeout:         -1
802-3-ethernet.port:                    --
802-3-ethernet.speed:                   0
802-3-ethernet.duplex:                  --
802-3-ethernet.auto-negotiate:          no
802-3-ethernet.mac-address:             --
802-3-ethernet.cloned-mac-address:      --
802-3-ethernet.generate-mac-address-mask:--
802-3-ethernet.mac-address-blacklist:   --
802-3-ethernet.mtu:                     auto
802-3-ethernet.s390-subchannels:        --
802-3-ethernet.s390-nettype:            --
802-3-ethernet.s390-options:            --
802-3-ethernet.wake-on-lan:             default
802-3-ethernet.wake-on-lan-password:    --
bridge-port.priority:                   32
bridge-port.path-cost:                  100
bridge-port.hairpin-mode:               yes
bridge-port.vlans:                      --
GENERAL.NAME:                           centos_virt-slave
GENERAL.UUID:                           8a6be03b-debc-472d-a44d-eac7145b6ae0
GENERAL.DEVICES:                        eth0
GENERAL.IP-IFACE:                       eth0
GENERAL.STATE:                          activated
GENERAL.DEFAULT:                        no
GENERAL.DEFAULT6:                       no
GENERAL.SPEC-OBJECT:                    --
GENERAL.VPN:                            no
GENERAL.DBUS-PATH:                      /org/freedesktop/NetworkManager/ActiveConnection/4
GENERAL.CON-PATH:                       /org/freedesktop/NetworkManager/Settings/2
GENERAL.ZONE:                           --
GENERAL.MASTER-PATH:                    /org/freedesktop/NetworkManager/Devices/5
IP4.GATEWAY:                            --
IP6.GATEWAY:                            --

nmcl c s server_virt

Code: Select all

connection.id:                          centos_virt
connection.uuid:                        c1e299da-45b5-40e5-a6e1-00d1da17102c
connection.stable-id:                   --
connection.type:                        bridge
connection.interface-name:              centos_virt
connection.autoconnect:                 yes
connection.autoconnect-priority:        0
connection.autoconnect-retries:         -1 (default)
connection.multi-connect:               0 (default)
connection.auth-retries:                -1
connection.timestamp:                   1600533430
connection.read-only:                   no
connection.permissions:                 --
connection.zone:                        --
connection.master:                      --
connection.slave-type:                  --
connection.autoconnect-slaves:          -1 (default)
connection.secondaries:                 --
connection.gateway-ping-timeout:        0
connection.metered:                     unknown
connection.lldp:                        default
connection.mdns:                        -1 (default)
connection.llmnr:                       -1 (default)
connection.wait-device-timeout:         -1
ipv4.method:                            manual
ipv4.dns:                               192.168.0.1
ipv4.dns-search:                        example.com
ipv4.dns-options:                       --
ipv4.dns-priority:                      0
ipv4.addresses:                         192.168.0.1/24
ipv4.gateway:                           192.168.0.254
ipv4.routes:                            --
ipv4.route-metric:                      -1
ipv4.route-table:                       0 (unspec)
ipv4.routing-rules:                     --
ipv4.ignore-auto-routes:                no
ipv4.ignore-auto-dns:                   no
ipv4.dhcp-client-id:                    --
ipv4.dhcp-timeout:                      0 (default)
ipv4.dhcp-send-hostname:                yes
ipv4.dhcp-hostname:                     --
ipv4.dhcp-fqdn:                         --
ipv4.never-default:                     no
ipv4.may-fail:                          yes
ipv4.dad-timeout:                       -1 (default)
ipv6.method:                            auto
ipv6.dns:                               --
ipv6.dns-search:                        --
ipv6.dns-options:                       --
ipv6.dns-priority:                      0
ipv6.addresses:                         --
ipv6.gateway:                           --
ipv6.routes:                            --
ipv6.route-metric:                      -1
ipv6.route-table:                       0 (unspec)
ipv6.routing-rules:                     --
ipv6.ignore-auto-routes:                no
ipv6.ignore-auto-dns:                   no
ipv6.never-default:                     no
ipv6.may-fail:                          yes
ipv6.routes:                            --
ipv6.route-metric:                      -1
ipv6.route-table:                       0 (unspec)
ipv6.routing-rules:                     --
ipv6.ignore-auto-routes:                no
ipv6.ignore-auto-dns:                   no
ipv6.never-default:                     no
ipv6.may-fail:                          yes
ipv6.ip6-privacy:                       -1 (unknown)
ipv6.addr-gen-mode:                     stable-privacy
ipv6.dhcp-duid:                         --
ipv6.dhcp-send-hostname:                yes
ipv6.dhcp-hostname:                     --
ipv6.token:                             --
bridge.mac-address:                     A8:A1:59:00:35:88
bridge.stp:                             no
bridge.priority:                        32768
bridge.forward-delay:                   15
bridge.hello-time:                      2
bridge.max-age:                         20
bridge.ageing-time:                     300
bridge.group-forward-mask:              0
bridge.multicast-snooping:              yes
bridge.vlan-filtering:                  no
bridge.vlan-default-pvid:               1
bridge.vlans:                           --
proxy.method:                           none
proxy.browser-only:                     no
proxy.pac-url:                          --
proxy.pac-script:                       --

The relevant /etc/sysconfing files are:

Code: Select all

STP=no
TYPE=Bridge
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=none
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=centos_virt
UUID=c1e299da-45b5-40e5-a6e1-00d1da17102c
DEVICE=centos_virt
ONBOOT=yes
IPADDR=192.168.0.1
PREFIX=24
GATEWAY=192.168.0.254
DNS1=192.168.0.1
BRIDGE_MACADDR=A8:A1:59:00:35:88
DOMAIN=example.com


TYPE=Ethernet
NAME=centos_virt-slave
UUID=fd8826ad-7f7b-45f4-922c-4bf455bea684
DEVICE=enp8s0
ONBOOT=yes
BRIDGE=centos_virt
BRIDGING_OPTS=hairpin_mode=1

I can ping the gateway as soon as the connection is up, I get ONE reply and after that nothing.
And not just that, but my whole lan is not working, like STP is enabled (but as you can see it's not!)
I am completely stuck here because I cannot use the server as a KVM host as intented.

I don't have a managed switch, just a plain dump TP-Link and the VDSL router.

I fail to see what is wrong, so I would appreciate any help.

[jlehtone]

One thing to try:
Use connection.uuid of the bridge as the connection.master, not the connection's name like you have now.

[s91066]

I tried it. Unfortunately not a lot changed.

Before going to the next step.
I used this topology:

CentOS ---> Switch
openSUSE --> Switch
Fedora 31 --> Switch
Windows 10 --> Switch

On the Fedora libvirt is running. On CenOS/openSUSE it's not (this is where I want to have the libvirt with bridges)
Now, the test was:

bridge is up on either CenOS or openSUSE (in the following lines noted as bridged)

ping from bridge to any other PC: 1st packet passes. All other are lost
Ping TO bridge: WORKS
SSH TO bridge: Works for openSUSE, not for CentOS

Direct connection of the Fedora to either CentOS or openSUSE didn't change anything.

Then I powered off the Fedora and did a power cycle on the switch. No change.

So, something is wrong with the routing of the bridge but what?

Code: Select all

centos:~ # ip route
default via 192.168.0.254 dev centos_virt proto static metric 425 
192.168.0.0/24 dev centos_virt proto kernel scope link src 192.168.0.1 metric 425 
[code]

[jlehtone]

I do often debug with tcpdump. In this case I would listen eth0 to see what the NIC gets and centos_virt to see what does reach the bridge.

How is the firewall?

[s91066]

Firewalls are down.
tcpdump, obviously show nothing.

BUT, now it works for the openSUSE.
What I did is to configure the slave bridge interface via KDE's NM applet. I used exactly the same settings, however, now it works.

The slave configuration from the NM has the following settings:

Code: Select all

..
connection.interface-name:              --
..
802-3-ethernet.auto-negotiate:          yes
802-3-ethernet.mac-address:             D0:50:99:17:3F:E6
..

while the old connection has the following settings:

Code: Select all

...
connection.interface-name:              eth0
...
802-3-ethernet.auto-negotiate:          no
802-3-ethernet.mac-address:             --
...

[jlehtone]

Oh yes.

When NM auto-generates connections, it sets the mac-address.
When we run 'nmcli con add', we usually specify interface-name.

Does it matter?

I had one VM with more than one interface. Config set/added with nmcli.
So, there were 'eth0', 'eth1', ...

On every reboot those names did enumerate differently. Wrong connections were bound to interfaces at least every second boot.
Once I did change the config to bind with mac-address and not with flaky "ethN" names, we got solid setup.


"Names are not important". It seems very important to believe that.

Of course, when firewall does specify interfaces ... but firewalld reads zone from NM's connection.
Nftables.service and ethN names ... luckily, I don't have that combo.