Accessing nspawn via network-bridge

Issues related to configuring your network
Post Reply
Centriole
Posts: 15
Joined: 2016/01/29 04:43:37

Accessing nspawn via network-bridge

Post by Centriole » 2016/09/15 13:17:52

Hi helpful community,

I have a CentOS-7 system I have set up some CentOS-7 systemd-nspawn containers on. By default, booting them makes them share the same range of network ports with the host so when the host is already serving on port 22 or 80, the container must either be configured to serve on another port or be remapped using (for example) the --port=tcp:2222:22 switch when booting. This isn't so bad when running only one nspawn container hosting only one service that uses only one port.

For more functional containers, I could either dedicate a USB-to-ethernet adapter to each container using the --network-interface=hosteth99 boot switch, or use a bridge with --network-bridge which apparently behaves like a network hub / switch in that it does not show up as an entry in mtr and allows real machines on my network to access a distinct IP address with its full range of standard ports per container. (Have I understood that correctly?) So as the nspawn container boots, the bridge notices it attach to one of the bridge's ports, request an IP address from my DHCP server via the bridge, and thereafter real computers should be able to ping, ssh to and http from the container as if it were a standalone system because the bridge forwards external network traffic to the internal virtual network.

I used nmtui to create a bridge. I set the bridge to get IPv4 address from DHCP and set the physical NIC as a slave. I installed bridge-utils and ran brctl while the container was running and it shows the container connected to the bridge. Now:
  • booting the host assigns a DHCP address to the bridge
  • mtr to that IP address (the IP address of the bridge) resolves the hostname of the container host
  • sshing to the host's hostname works, and requests (eg curl) from the host work (so incoming and outgoing traffic to the host is fine)
  • booting the container(s) with --network-bridge (with or without --network-veth) results in a host0 device within the container with no IPv4 address (even though I used nmtui within the container to set it to DHCP); the host has a corresponding vb-container device that gets set up and attached to the bridge's port as the container boots and torn down as it halts
  • containers cannot make outgoing requests and external machines cannot ping the containers
I must be missing something. Has someone who has already set up something similar got a few pointers they could kindly donate? I'd be much obliged!

Centriole
Posts: 15
Joined: 2016/01/29 04:43:37

Re: Accessing nspawn via network-bridge

Post by Centriole » 2016/09/16 11:11:01

Here are (some of) the relevant gory details.

On the host:

Code: Select all

user@host:~# cat /etc/sysconfig/network-scripts/ifcfg-Bridge0
DEVICE=br0
STP=yes
TYPE=Bridge
BOOTPROTO=dhcp
DEFROUTE=yes
IPV4_FAILURE_FATAL=yes
IPV6INIT=no
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
NAME=Bridge0
UUID=1630cb02-e16b-4263-84c7-ae374c950b10
ONBOOT=yes
IPV6_PEERDNS=yes
IPV6_PEERROUTES=yes
BRIDGING_OPTS=priority=32768
PEERDNS=yes
PEERROUTES=yes

user@host:~# cat /etc/sysconfig/network-scripts/ifcfg-br0enp4s0 
TYPE=Ethernet
NAME=br0enp4s0
UUID=6e955ba5-6552-4edb-a580-bdf69355ae10
DEVICE=enp4s0
ONBOOT=yes
BRIDGE=1630cb02-e16b-4263-84c7-ae374c950b10

user@host:~# cat /var/lib/machines/container/etc/sysconfig/network-scripts/ifcfg-virbrhost0
HWADDR=CE:6A:41:08:FA:80
TYPE=Ethernet
BOOTPROTO=dhcp
DEFROUTE=yes
PEERDNS=yes
PEERROUTES=yes
IPV4_FAILURE_FATAL=yes
IPV6INIT=no
NAME=virbrhost0
UUID=514a767e-cf78-495b-93fc-8a568f378689
ONBOOT=yes

user@host:~# systemctl status firewalld
● firewalld.service - firewalld - dynamic firewall daemon
   Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: enabled)
   Active: inactive (dead)

user@host:~# ifconfig
br0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.1.113  netmask 255.255.255.0  broadcast 192.168.1.255
        inet6 fe80::82ee:73ff:fe32:8bbb  prefixlen 64  scopeid 0x20<link>
        ether 80:ee:73:32:8b:bb  txqueuelen 0  (Ethernet)
        RX packets 75  bytes 7142 (6.9 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 54  bytes 4653 (4.5 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

enp4s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether 80:ee:73:32:8b:bb  txqueuelen 1000  (Ethernet)
        RX packets 91  bytes 9152 (8.9 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 118  bytes 8571 (8.3 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 0  (Local Loopback)
        RX packets 4  bytes 340 (340.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 4  bytes 340 (340.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

user@host:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: enp4s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br0 state UP qlen 1000
    link/ether 80:ee:73:32:8b:bb brd ff:ff:ff:ff:ff:ff
3: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
    link/ether 80:ee:73:32:8b:bb brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.113/24 brd 192.168.1.255 scope global dynamic br0
       valid_lft 604692sec preferred_lft 604692sec
    inet6 fe80::82ee:73ff:fe32:8bbb/64 scope link
       valid_lft forever preferred_lft forever

user@host:~# brctl show
bridge name     bridge id               STP enabled     interfaces
br0             8000.80ee73328bbb       yes             enp4s0

user@host:~# machinectl list-images
NAME        TYPE      RO  USAGE CREATED MODIFIED
container   directory no  n/a   n/a     n/a     

user@host:~# systemd-nspawn --network-veth --network-bridge=br0 --bind /var/cache/yum -D /var/lib/machines/container -b

user@host:~# machinectl
MACHINE   CLASS     SERVICE
container container nspawn 

1 machines listed.

user@host:~# brctl show
bridge name     bridge id               STP enabled     interfaces
br0             8000.80ee73328bbb       yes             enp4s0
                                                        vb-container

user@host:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: enp4s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br0 state UP qlen 1000
    link/ether 80:ee:73:32:8b:bb brd ff:ff:ff:ff:ff:ff
3: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
    link/ether 80:ee:73:32:8b:bb brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.113/24 brd 192.168.1.255 scope global dynamic br0
       valid_lft 603962sec preferred_lft 603962sec
    inet6 fe80::82ee:73ff:fe32:8bbb/64 scope link
       valid_lft forever preferred_lft forever
4: vb-container@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br0 state UP qlen 1000
    link/ether a6:83:06:e4:53:52 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet6 fe80::a483:6ff:fee4:5352/64 scope link
       valid_lft forever preferred_lft forever
On the guest container:

Code: Select all

guest@container:~$ sudo systemctl status firewalld
● firewalld.service - firewalld - dynamic firewall daemon
   Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: enabled)
   Active: inactive (dead)

guest@container:~$ ifconfig
host0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether ce:6a:41:08:fa:80  txqueuelen 1000  (Ethernet)
        RX packets 192  bytes 12021 (11.7 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 0  (Local Loopback)
        RX packets 12  bytes 852 (852.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 12  bytes 852 (852.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

guest@container:~$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: host0@if4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether ce:6a:41:08:fa:80 brd ff:ff:ff:ff:ff:ff link-netnsid 0

guest@container:~$ ping 192.168.1.1
connect: Network is unreachable

matt_garman
Posts: 40
Joined: 2006/10/18 14:14:21

Re: Accessing nspawn via network-bridge

Post by matt_garman » 2017/06/13 20:49:14

Bump... did you ever find a solution?

Post Reply

Return to “CentOS 7 - Networking Support”