Network bridge

[fatcharly@gmx.de]

Hi,

I'm running two CentOS 8 4.18.0-305.25.1.el8_4.x86_64 #1 SMP Wed Nov 3 10:29:07 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux Nodes on an Microsoft Hyper-V 2012.
My Idea was to build with one node a bridge with firewall function, so I can reach the other node from the local network but protect him via the bridge.
I put two virtual nic's in the bridge node, and configured them as slave for the bridge. So now I can see with tcpdump on both nics the network traffic.
The second node has only one nic and is with a virtual switch with the second nic of the bridgenode connected. If I start a tcpdump on the second node, then I can't see any networt traffic at all. When I configure both nodes with normal nic properties in between in a same network, they can communicate in a propper way.
I have no Idea what is wrong, is the virtual switch not really fully switching ? I 'm lost, here is my config of the network config od the bridge node:

[root@BR01 ~]# nmcli connection show
NAME UUID TYPE DEVICE
br0 0f67976c-2f38-4691-9eb9-b80152f1428b bridge br0
eth0 29ab8a41-2322-4c5f-bbdd-2131c807bf2e ethernet eth0
eth1 e818e40b-ea19-47e6-86b5-2c429e3b6d94 ethernet eth1

[root@BR01 ~]# nmcli connection show br0
connection.id: br0
connection.uuid: 0f67976c-2f38-4691-9eb9-b80152f1428b
connection.stable-id: --
connection.type: bridge
connection.interface-name: br0
connection.autoconnect: yes
connection.autoconnect-priority: 0
connection.autoconnect-retries: -1 (default)
connection.multi-connect: 0 (default)
connection.auth-retries: -1
connection.timestamp: 1637768406
connection.read-only: no
connection.permissions: --
connection.zone: --
connection.master: --
connection.slave-type: --
connection.autoconnect-slaves: -1 (default)
connection.secondaries: --
connection.gateway-ping-timeout: 0
connection.metered: unknown
connection.lldp: default
connection.mdns: -1 (default)
connection.llmnr: -1 (default)
lines 1-23...skipping...
connection.id: br0
connection.uuid: 0f67976c-2f38-4691-9eb9-b80152f1428b
connection.stable-id: --
connection.type: bridge
connection.interface-name: br0
connection.autoconnect: yes
connection.autoconnect-priority: 0
connection.autoconnect-retries: -1 (default)
connection.multi-connect: 0 (default)
connection.auth-retries: -1
connection.timestamp: 1637768406
connection.read-only: no
connection.permissions: --
connection.zone: --
connection.master: --
connection.slave-type: --
connection.autoconnect-slaves: -1 (default)
connection.secondaries: --
connection.gateway-ping-timeout: 0
connection.metered: unknown
connection.lldp: default
connection.mdns: -1 (default)
connection.llmnr: -1 (default)
connection.wait-device-timeout: -1
ipv4.method: manual
ipv4.dns: 192.168.3.16
ipv4.dns-search: --
ipv4.dns-options: --
ipv4.dns-priority: 0
ipv4.addresses: 192.168.3.2/21
ipv4.gateway: 192.168.1.9
ipv4.routes: --
ipv4.route-metric: -1
ipv4.route-table: 0 (unspec)
ipv4.routing-rules: --
ipv4.ignore-auto-routes: no
ipv4.ignore-auto-dns: no
ipv4.dhcp-client-id: --
ipv4.dhcp-iaid: --
ipv4.dhcp-timeout: 0 (default)
ipv4.dhcp-send-hostname: yes
ipv4.dhcp-hostname: --
ipv4.dhcp-fqdn: --
ipv4.dhcp-hostname-flags: 0x0 (none)
ipv4.never-default: no
ipv4.may-fail: yes
ipv4.required-timeout: -1 (default)
ipv4.dad-timeout: -1 (default)
ipv4.dhcp-vendor-class-identifier: --
ipv4.dhcp-reject-servers: --
ipv6.method: auto
ipv6.dns: --
ipv6.dns-search: --
ipv6.dns-options: --
ipv6.dns-priority: 0
ipv6.addresses: --
ipv6.gateway: --
ipv6.routes: --
ipv6.route-metric: -1
ipv6.route-table: 0 (unspec)
ipv6.routing-rules: --
ipv6.ignore-auto-routes: no
ipv6.ignore-auto-dns: no
ipv6.never-default: no
ipv6.may-fail: yes
ipv6.required-timeout: -1 (default)
ipv6.ip6-privacy: -1 (unknown)
ipv6.addr-gen-mode: stable-privacy
ipv6.ra-timeout: 0 (default)
ipv6.dhcp-duid: --
ipv6.dhcp-iaid: --
ipv6.dhcp-timeout: 0 (default)
ipv6.dhcp-send-hostname: yes
ipv6.dhcp-hostname: --
ipv6.dhcp-hostname-flags: 0x0 (none)
ipv6.token: --
bridge.mac-address: --
bridge.stp: yes
bridge.priority: 32768
bridge.forward-delay: 15
bridge.hello-time: 2
bridge.max-age: 20
bridge.ageing-time: 300
bridge.group-forward-mask: 0
bridge.multicast-snooping: yes
bridge.vlan-filtering: no
bridge.vlan-default-pvid: 1
bridge.vlans: --
proxy.method: none
proxy.browser-only: no
proxy.pac-url: --
proxy.pac-script: --
GENERAL.NAME: br0
GENERAL.UUID: 0f67976c-2f38-4691-9eb9-b80152f1428b
GENERAL.DEVICES: br0
GENERAL.IP-IFACE: br0
GENERAL.STATE: activated
GENERAL.DEFAULT: yes
GENERAL.DEFAULT6: no
GENERAL.SPEC-OBJECT: --
GENERAL.VPN: no
GENERAL.DBUS-PATH: /org/freedesktop/NetworkManager/ActiveC>
GENERAL.CON-PATH: /org/freedesktop/NetworkManager/Setting>
GENERAL.ZONE: --
GENERAL.MASTER-PATH: --
IP4.ADDRESS[1]: 192.168.3.2/21
IP4.GATEWAY: 192.168.1.9
IP4.ROUTE[1]: dst = 192.168.0.0/21, nh = 0.0.0.0, mt>
IP4.ROUTE[2]: dst = 0.0.0.0/0, nh = 192.168.1.9, mt >
IP4.DNS[1]: 192.168.3.16
IP6.ADDRESS[1]: fe80::4d7:d6a3792/64
IP6.GATEWAY: --
IP6.ROUTE[1]: dst = fe80::/64, nh = ::, mt = 425
IP6.ROUTE[2]: dst = ff00::/8, nh = ::, mt = 256, tab>
lines 51-114/114 (END)



[root@BR01 ~]# nmcli connection show eth0
connection.id: eth0
connection.uuid: 29ab8a41-2322-4c5f-bbdd-2131c807bf2e
connection.stable-id: --
connection.type: 802-3-ethernet
connection.interface-name: eth0
connection.autoconnect: yes
connection.autoconnect-priority: 0
connection.autoconnect-retries: -1 (default)
connection.multi-connect: 0 (default)
connection.auth-retries: -1
connection.timestamp: 1637768406
connection.read-only: no
connection.permissions: --
connection.zone: --
connection.master: br0
connection.slave-type: bridge
connection.autoconnect-slaves: -1 (default)
connection.secondaries: --
connection.gateway-ping-timeout: 0
connection.metered: unknown
connection.lldp: default
connection.mdns: -1 (default)
connection.llmnr: -1 (default)
connection.wait-device-timeout: -1
802-3-ethernet.port: --
802-3-ethernet.speed: 0
802-3-ethernet.duplex: --
802-3-ethernet.auto-negotiate: no
802-3-ethernet.mac-address: --
802-3-ethernet.cloned-mac-address: --
802-3-ethernet.generate-mac-address-mask:--
802-3-ethernet.mac-address-blacklist: --
802-3-ethernet.mtu: auto
802-3-ethernet.s390-subchannels: --
802-3-ethernet.s390-nettype: --
802-3-ethernet.s390-options: --
802-3-ethernet.wake-on-lan: default
802-3-ethernet.wake-on-lan-password: --
bridge-port.priority: 32
bridge-port.path-cost: 100
bridge-port.hairpin-mode: no
bridge-port.vlans: --
GENERAL.NAME: eth0
GENERAL.UUID: 29ab8a41-2322-4c5f-bbdd-2131c807bf2e
GENERAL.DEVICES: eth0
GENERAL.IP-IFACE: eth0
GENERAL.STATE: activated
GENERAL.DEFAULT: no
GENERAL.DEFAULT6: no
GENERAL.SPEC-OBJECT: --
GENERAL.VPN: no
GENERAL.DBUS-PATH: /org/freedesktop/NetworkManager/Active>
GENERAL.CON-PATH: /org/freedesktop/NetworkManager/Settin>
GENERAL.ZONE: --
GENERAL.MASTER-PATH: /org/freedesktop/NetworkManager/Device>
IP4.GATEWAY: --
IP6.GATEWAY: --
lines 1-57/57 (END)...skipping...
connection.id: eth0
connection.uuid: 29ab8a41-2322-4c5f-bbdd-2131c807bf2e
connection.stable-id: --
connection.type: 802-3-ethernet
connection.interface-name: eth0
connection.autoconnect: yes
connection.autoconnect-priority: 0
connection.autoconnect-retries: -1 (default)
connection.multi-connect: 0 (default)
connection.auth-retries: -1
connection.timestamp: 1637768406
connection.read-only: no
connection.permissions: --
connection.zone: --
connection.master: br0
connection.slave-type: bridge
connection.autoconnect-slaves: -1 (default)
connection.secondaries: --
connection.gateway-ping-timeout: 0
connection.metered: unknown
connection.lldp: default
connection.mdns: -1 (default)
connection.llmnr: -1 (default)
connection.wait-device-timeout: -1
802-3-ethernet.port: --
802-3-ethernet.speed: 0
802-3-ethernet.duplex: --
802-3-ethernet.auto-negotiate: no
802-3-ethernet.mac-address: --
802-3-ethernet.cloned-mac-address: --
802-3-ethernet.generate-mac-address-mask:--
802-3-ethernet.mac-address-blacklist: --
802-3-ethernet.mtu: auto
802-3-ethernet.s390-subchannels: --
802-3-ethernet.s390-nettype: --
802-3-ethernet.s390-options: --
802-3-ethernet.wake-on-lan: default
802-3-ethernet.wake-on-lan-password: --
bridge-port.priority: 32
bridge-port.path-cost: 100
bridge-port.hairpin-mode: no
bridge-port.vlans: --
GENERAL.NAME: eth0
GENERAL.UUID: 29ab8a41-2322-4c5f-bbdd-2131c807bf2e
GENERAL.DEVICES: eth0
GENERAL.IP-IFACE: eth0
GENERAL.STATE: activated
GENERAL.DEFAULT: no
GENERAL.DEFAULT6: no
GENERAL.SPEC-OBJECT: --
GENERAL.VPN: no
GENERAL.DBUS-PATH: /org/freedesktop/NetworkManager/Active>
GENERAL.CON-PATH: /org/freedesktop/NetworkManager/Settin>
GENERAL.ZONE: --
GENERAL.MASTER-PATH: /org/freedesktop/NetworkManager/Device>
IP4.GATEWAY: --
IP6.GATEWAY: --



[root@BR01 ~]# nmcli connection show eth1
connection.id: eth1
connection.uuid: e818e40b-ea19-47e6-86b5-2c429e3b6d94
connection.stable-id: --
connection.type: 802-3-ethernet
connection.interface-name: eth1
connection.autoconnect: yes
connection.autoconnect-priority: 0
connection.autoconnect-retries: -1 (default)
connection.multi-connect: 0 (default)
connection.auth-retries: -1
connection.timestamp: 1637768406
connection.read-only: no
connection.permissions: --
connection.zone: --
connection.master: br0
connection.slave-type: bridge
connection.autoconnect-slaves: -1 (default)
connection.secondaries: --
connection.gateway-ping-timeout: 0
connection.metered: unknown
connection.lldp: default
connection.mdns: -1 (default)
connection.llmnr: -1 (default)
connection.wait-device-timeout: -1
802-3-ethernet.port: --
802-3-ethernet.speed: 0
802-3-ethernet.duplex: --
802-3-ethernet.auto-negotiate: no
802-3-ethernet.mac-address: --
802-3-ethernet.cloned-mac-address: --
802-3-ethernet.generate-mac-address-mask:--
802-3-ethernet.mac-address-blacklist: --
802-3-ethernet.mtu: auto
802-3-ethernet.s390-subchannels: --
802-3-ethernet.s390-nettype: --
802-3-ethernet.s390-options: --
802-3-ethernet.wake-on-lan: default
802-3-ethernet.wake-on-lan-password: --
bridge-port.priority: 32
bridge-port.path-cost: 100
bridge-port.hairpin-mode: no
bridge-port.vlans: --
GENERAL.NAME: eth1
GENERAL.UUID: e818e40b-ea19-47e6-86b5-2c429e3b6d94
GENERAL.DEVICES: eth1
GENERAL.IP-IFACE: eth1
GENERAL.STATE: activated
GENERAL.DEFAULT: no
GENERAL.DEFAULT6: no
GENERAL.SPEC-OBJECT: --
GENERAL.VPN: no
GENERAL.DBUS-PATH: /org/freedesktop/NetworkManager/Active>
GENERAL.CON-PATH: /org/freedesktop/NetworkManager/Settin>
GENERAL.ZONE: --
GENERAL.MASTER-PATH: /org/freedesktop/NetworkManager/Device>
IP4.GATEWAY: --
IP6.GATEWAY: --



Any suggestions are welcome

Kind regards

fatcharly

[jlehtone]

In other words, you have "physically":
Real cable == NIC -- some Hyper-V -- vnic0 :: br0 :: vnic1 -- Hyper-V vswitch -- vnic2

The vnic0 :: br0 :: vnic1 is VM A. It (br0) is in subnet 192.168.0.0/21. The br0 is like a "managed switch".
The Hyper-V vswitch is a "managed switch"?
The vnic2 is in VM B. VM B should also have address in subnet 192.168.0.0/21.
Machines outside, behind "real cable" are also in subnet 192.168.0.0/21.

When somebody outside generates traffic, you can see packets in vnic0, br0, and vnic1, but not in vnic2. Can you monitor the vswitch?
If you generate packets in VM B, do you see them anywhere but vnic2?

You can set bridge.stp no, if you are sure that that does create a loop. (If both vnic0 and vnic1 are connected to same, undivided vswitch, then you do have a loop. A packet created at br0 would go out from vnic0 to vswitch, which would propagate it to vnic1, which gives to br0 that propagates to vnic0, ...)

[fatcharly@gmx.de]

>>In other words, you have "physically":
>>Real cable == NIC -- some Hyper-V -- vnic0 :: br0 :: vnic1 -- Hyper-V vswitch -- vnic2 second Hyper-V VM -All on the same Hyper-V Server
Right !

>>The vnic0 :: br0 :: vnic1 is VM A. It (br0) is in subnet 192.168.0.0/21. The br0 is like a "managed switch".
Right !
>>The Hyper-V vswitch is a "managed switch"?
it's the normal virtual switch you get when you define one on Hyper-V 2012

>>The vnic2 is in VM B. VM B should also have address in subnet 192.168.0.0/21.
>>Machines outside, behind "real cable" are also in subnet 192.168.0.0/21.
Right !

>>When somebody outside generates traffic, you can see packets in vnic0, br0, and vnic1, but not in vnic2. Can you monitor the vswitch?
Right, I don't know if there is a way to monitor a vswitch in HYper-V 2012

>>If you generate packets in VM B, do you see them anywhere but vnic2?
After I switched off all firewalls I can see a ping request (ping IP of BR0 on VM A)from VM B on br0 of VM A, but VM A is not responding. Also when I ping from VM B a IP on the real cable, I see the traffic on BR0 but I get no response.

Do I have to activate the ip forward in the kernel ?

>>You can set bridge.stp no, if you are sure that that does create a loop. (If both vnic0 and vnic1 are connected to same, undivided vswitch, then you do have a loop. A packet created at br0 would go out from vnic0 to vswitch, which would propagate it to vnic1, which gives to br0 that propagates to vnic0, ...)
vnic0 and vnic1 are connected to different virtual swiches.

[jlehtone]

Do I have to activate the ip forward in the kernel ?

IP forwarding means routing to different subnet. Bridge is a L2 switch; it does not route.
Bridget traffic is not filtered by firewall either, so firewall should not block bridging.
(It is possible to enable filtering, but that is for peculiar setups.)

Default firewall rules do not block ping.

Does VM A respond to all other traffic, but with B?

You should see some state with:

Code: Select all

bridge link
bridge fdb
ip neigh

[fatcharly@gmx.de]

>>You should see some state with:

>>bridge link
>>bridge fdb
>>ip neigh

[root@BR01 ~]# bridge link
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master br0 state forwarding priority 32 cost 100
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master br0 state forwarding priority 32 cost 100
looks good to me

grepping for the mac of the VM B
[root@BR01 ~]# bridge fdb|grep 00:15:5d:03:ba:34
00:15:5d:03:ba:34 dev eth1 master br0
looks good to me, mac was seen on the right interface

[root@BR01 ~]# ip neigh
192.168.3.15 dev br0 lladdr 00:15:5d:03:ba:34 STALE
192.168.3.225 dev br0 lladdr 6c:3b:e5:11:96:48 REACHABLE
192.168.3.10 dev br0 lladdr 18:a9:05:3c:81:74 STALE
192.168.1.9 dev br0 lladdr 28:94:0f:86:03:18 STALE
192.168.3.16 dev br0 lladdr 00:15:5d:03:18:0b STALE
192.168.3.59 dev br0 lladdr 00:15:5d:03:18:0b STALE
VM B is 192.168.3.15 so it looks ok to me.

But there is still no communication between VM B and VM A
possible

Any suggestions are welcome

[jlehtone]

stp off forward-delay 0
for br0 could affect how bridge behaves.

If MAC has been seen, then ARP goes through.

Can you run tcpdump on both VMs? (That is my usual debug tool; listen in many points while generating traffic. nmap is another.)

How much packets does B see, and where do they originate from?


PS. Two of your "other IPs" have same MAC?

[fatcharly@gmx.de]

stp off forward-delay 0
for br0 could affect how bridge behaves.

If MAC has been seen, then ARP goes through.

Can you run tcpdump on both VMs? (That is my usual debug tool; listen in many points while generating traffic. nmap is another.)

How much packets does B see, and where do they originate from?


PS. Two of your "other IPs" have same MAC?

how do I set stp off forward-delay 0 ?

Yes, If I run tcpdump on eth1 on VM A, I see normal traffic, when I run it on eth0 of VM B, I see nothing.

They have the same because it's a system with two IP's

[jlehtone]

Code: Select all

sudo nmcli con mod br0 bridge.stp off bridge.forward-delay 0

Yes, If I run tcpdump on eth1 on VM A, I see normal traffic, when I run it on eth0 of VM B, I see nothing.

Code: Select all

[VM A]# tcpdump -vv -nn -i br0
[VM B]$ ping -c 1 forums.centos.org

The A would not detect ARP, DNS query, nor ICMP that B has to do for that ping?

[fatcharly@gmx.de]

Code: Select all

sudo nmcli con mod br0 bridge.stp off bridge.forward-delay 0

Now I can see the request vom the VM B faster in the tcpdump of the br0 on VM A

Yes, If I run tcpdump on eth1 on VM A, I see normal traffic, when I run it on eth0 of VM B, I see nothing.

Code: Select all

[VM A]# tcpdump -vv -nn -i br0
[VM B]$ ping -c 1 forums.centos.org

The A would not detect ARP, DNS query, nor ICMP that B has to do for that ping?

I'm sorry but I dont understand this question, why is it not detecting the protocolls ? I can see them in the tcpdump of br0 of VM A

[jlehtone]

The question was, do you see traffic from/to B when you listen on br0 of A?

If neither B sees anything on "wire" nor A sees anything from B on "wire", then I would suspect the wire, which is the vswitch.