I am attempting to better learn CentOS 7 in my dev playground at home. This playground is really just my overly complex KVM setup.
The problem is that a lot of the information I am finding seems to be for EL6, and that doesn't always relate to EL7. I tried for an entire weekend trying to get networking configured properly with the command line version of Network Manager, until I found that the official Red Hat docs for KVM and RHEL 7 say not to use Network Manager with KVM due to briding issues. Thus, I have ripped out all of Network Manager and I am trying to get it to work with just networking. I have now slogged my way through various blogs/articles/forums/mailing-lists and I have something that "works" but I really don't know if I set this up correctly.
The real problem is that the networking doesn't always come up properly on boot. If I manually log into the box and 'service network restart' (yeah yeah, supposed to use systctl), I get this error:
Code: Select all
# service network restart
Restarting network (via systemctl): Job for network.service failed. See 'systemctl status network.service' and 'journalctl -xn' for details.
[FAILED]
# journalctl -xn
-- Logs begin at Tue 2014-11-11 12:18:42 EST, end at Tue 2014-11-11 13:39:59 EST. --
Nov 11 13:39:59 KVMHost network[7687]: RTNETLINK answers: File exists
Nov 11 13:39:59 KVMHost network[7687]: RTNETLINK answers: File exists
Nov 11 13:39:59 KVMHost network[7687]: RTNETLINK answers: File exists
Nov 11 13:39:59 KVMHost network[7687]: RTNETLINK answers: File exists
Nov 11 13:39:59 KVMHost network[7687]: RTNETLINK answers: File exists
Nov 11 13:39:59 KVMHost network[7687]: RTNETLINK answers: File exists
Nov 11 13:39:59 KVMHost network[7687]: RTNETLINK answers: File exists
Nov 11 13:39:59 KVMHost systemd[1]: network.service: control process exited, code=exited status=1
Nov 11 13:39:59 KVMHost systemd[1]: Failed to start LSB: Bring up/down networking.
-- Subject: Unit network.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit network.service has failed.
--
-- The result is failed.
Nov 11 13:39:59 KVMHost systemd[1]: Unit network.service entered failed state.
I am really hoping someone might spot my problem.
Questions:
1) What would cause the networking to have issues on boot but not on a service restart?
2) If I just want a VM to have a bridge to the network, does the card actually need it's own IP address? (see below for brOrso. The network card has an address and the VM has a different address. This is essentially two IP's for one VM. I would really rather this not be the case).
3) Is there anything blantently dumb that I have done? Anything that I should change?
Setup:
The box has 4 network cards (with matching 4 bridges) and a private-to-KVM network virbr0. I loath how EL7 names network devices now, so I gave the interface/bridges names I understand.
enp11s0-> brSystemNetwork: This is the main port to my 192.168.100.x network.
enp4s0f0->brenp4s0f0: I want to use this similiar to brOrso but I haven't started on this project yet until I feel better about the networking.
enp5s5->brOrso: This is a VM that I want to have a deidcated network port to my 192.168.100.x network. I don't want any other vm/or traffic that isn't to this VM. Because I eventually would like to learn/play-with fail-over to another host, I made it a bridge device so it isn't tied to a specific piece of hardware.
enp4s0f1->brClusterSwitch: This is a "private" 10.10.10.x network that I use for all kinds of weird testing when I want to attempt to learn something new. One of the VM's is set up as a pass-through between the 192.168.100.x network on brSystemNetwork and the 10.10.10.x network on brClusterSwitch so that those hosts can get out to the internet (this VM is the gateway 10.10.10.1).
Thus after getting my networking "functioning" here is what my 'ip a' looks like:
Code: Select all
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp11s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master brSystemNetwork state UP qlen 1000
link/ether 00:19:b9:2c:23:42 brd ff:ff:ff:ff:ff:ff
inet6 fe80::219:b9ff:fe2c:2342/64 scope link
valid_lft forever preferred_lft forever
3: enp4s0f0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast master brenp4s0f0 state DOWN qlen 1000
link/ether 00:17:08:7e:5c:1a brd ff:ff:ff:ff:ff:ff
4: enp5s5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master brOrso state UP qlen 1000
link/ether 00:0e:0c:58:ee:6e brd ff:ff:ff:ff:ff:ff
inet6 fe80::20e:cff:fe58:ee6e/64 scope link
valid_lft forever preferred_lft forever
5: enp4s0f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master brClusterSwitch state UP qlen 1000
link/ether 00:17:08:7e:5c:1b brd ff:ff:ff:ff:ff:ff
inet6 fe80::217:8ff:fe7e:5c1b/64 scope link
valid_lft forever preferred_lft forever
6: brClusterSwitch: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
link/ether 00:17:08:7e:5c:1b brd ff:ff:ff:ff:ff:ff
inet 10.10.10.2/24 brd 10.10.10.255 scope global brClusterSwitch
valid_lft forever preferred_lft forever
inet6 fe80::217:8ff:fe7e:5c1b/64 scope link
valid_lft forever preferred_lft forever
7: brOrso: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
link/ether 00:0e:0c:58:ee:6e brd ff:ff:ff:ff:ff:ff
inet 192.168.100.126/24 brd 192.168.100.255 scope global dynamic brOrso
valid_lft 3380sec preferred_lft 3380sec
inet6 fe80::20e:cff:fe58:ee6e/64 scope link
valid_lft forever preferred_lft forever
8: brenp4s0f0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
link/ether 00:17:08:7e:5c:1a brd ff:ff:ff:ff:ff:ff
9: brSystemNetwork: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
link/ether 00:19:b9:2c:23:42 brd ff:ff:ff:ff:ff:ff
inet 192.168.100.101/24 brd 192.168.100.255 scope global brSystemNetwork
valid_lft forever preferred_lft forever
inet6 fe80::219:b9ff:fe2c:2342/64 scope link
valid_lft forever preferred_lft forever
10: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN
link/ether 52:54:00:77:ee:4a brd ff:ff:ff:ff:ff:ff
inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
valid_lft forever preferred_lft forever
11: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN qlen 500
link/ether 52:54:00:77:ee:4a brd ff:ff:ff:ff:ff:ff
12: vnet0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master brOrso state UNKNOWN qlen 500
link/ether fe:54:00:e3:ea:56 brd ff:ff:ff:ff:ff:ff
inet6 fe80::fc54:ff:fee3:ea56/64 scope link
valid_lft forever preferred_lft forever
Code: Select all
# cat ifcfg-Physicalenp4s0f0
NAME=Physicalenp4s0f0
HWADDR=00:17:08:7e:5c:1a
ONBOOT=yes
NM_CONTROLLED=no
BRIDGE=brenp4s0f0
Code: Select all
# cat ifcfg-SystemNetwork
NAME=SystemNetwork
HWADDR=00:19:B9:2C:23:42
ONBOOT=yes
NM_CONTROLLED=no
BRIDGE="brSystemNetwork"
Code: Select all
# cat ifcfg-brSystemNetwork
DEVICE="brSystemNetwork"
ONBOOT="yes"
TYPE="Bridge"
BOOTPROTO="static"
IPADDR=192.168.100.101
NETMASK=255.255.255.0
GATEWAY=192.168.100.1
DNS1=192.168.100.1
DEFROUTE="yes"
Code: Select all
# cat ifcfg-PhysicalClusterSwitch
NAME=PhysicalClusterSwitch
HWADDR=00:17:08:7e:5c:1b
ONBOOT=yes
NM_CONTROLLED=no
BRIDGE=brClusterSwitch
Code: Select all
# cat ifcfg-brClusterSwitch
DEVICE=brClusterSwitch
ONBOOT=yes
TYPE=Bridge
STP=on
DELAY=0
NM_CONTROLLED="no"
BOOTPROTO=none
IPADDR=10.10.10.2
NETMASK=255.255.255.0
GATEWAY=10.10.10.1
Code: Select all
# cat ifcfg-PhysicalOrso
NAME=PhysicalOrso
HWADDR=00:0E:0C:58:EE:6E
ONBOOT=yes
NM_CONTROLLED=no
BRIDGE=brOrso
Code: Select all
# cat ifcfg-brOrso
DEVICE=brOrso
ONBOOT=yes
TYPE=Bridge
BOOTPROTO=dhcp
STP=on
DELAY=0
Code: Select all
# ifup brOrso
set forward delay failed: Numerical result out of range
I would really appreciate any comments/suggestions.
Thanks!
[EDIT] Quick update. Apparently when I rebooted and manually restarted the networks, I had all kinds of routing issues. As soon as I remembered I forgot to do ifdown/ifup on enp11s0, the routing started working again...Very strange...