Two gateways with different set metrics - how?
Posted: 2019/08/20 13:36:13
Hello folks,
I'm begging for help here. Been spending hours researching how to solve what seemed like a simple problem: on a CentOS 7.6 host with two network interfaces on two different networks, how to have two default gateways with a set metric to favour one of these interfaces. Can't do it. Would anybody please tell me what I'm missing before I have no hair left?
First, I've tried using the METRIC=xxx setting in /etc/sysconfig/network-scripts-/ifcfg-<interface> files based on some advice found on the web. Obviously if that ever was supported, it's no longer. Next.
Then I've tried using policy-based routing following the tons of documentation that can be found online. Unfortunately very few mention adding metric values.
So, the full details (sorry it's long but I don't want to skip anything): this machine has two active interfaces enp10s4f0 (IP 10.149.247.23, prefix /24) and enp2s0f0 (IP 10.149.160.21, prefix /24).
It has two gateways: 10.149.247.254 and 10.149.160.254. I want 10.149.160.254 to be the "best" gateway, therefore with the lower metric.
I also want to avoid asymetric routing.
I can reach my goal of having different metrics on gateways by manually entering routes with "ip route add default ... dev ... metric" but I want it to be persistent.
So I've created two routing tables in /etc/iproute2/rt_tables:
247 enp10s4f0table
160 enp2s0f0table
And I've made the following rule and route files in /etc/sysconfig/network-scripts:
route-enp10s4f0:
10.149.247.0/24 dev enp10s4f0 src 10.149.247.23 table enp10s4f0table
default via 10.149.247.254 dev enp10s4f0 metric 110 table enp10s4f0table
rule-enp10s4f0:
from 10.149.247.23/32 table enp10s4f0table
to 10.149.247.23 table enp10s4f0table
route-enp2s0f0:
10.149.160.0/24 dev enp2s0f0 src 10.149.160.21 table enp2s0f0table
default via 10.149.160.254 dev enp2s0f0 table enp2s0f0table
rule-enp2s0f0:
from 10.149.160.21/32 table enp2s0f0table
to 10.149.160.21 table enp2s0f0table
Then I've used the proper incantations:
yum install NetworkManager-config-routing-rules
systemctl enable NetworkManager-dispatcher.service
systemctl start NetworkManager-dispatcher.service
...and reboot.
Obviously the configuration is read and processed properly but the metric is *not* applied to the 10.149.247.254 gateway:
# ip route show table enp10s4f0table
default via 10.149.247.254 dev enp10s4f0 metric 110
10.149.247.0/24 dev enp10s4f0 scope link src 10.149.247.23
# ip route show table enp2s0f0table
default via 10.149.160.254 dev enp2s0f0
# ip rule show
0: from all lookup local
32762: from all to 10.149.160.21 lookup enp2s0f0table
32763: from 10.149.160.21 lookup enp2s0f0table
32764: from all to 10.149.247.23 lookup enp10s4f0table
32765: from 10.149.247.23 lookup enp10s4f0table
32766: from all lookup main
32767: from all lookup default
But:
# ip route show
default via 10.149.247.254 dev enp10s4f0 proto static metric 100
default via 10.149.160.254 dev enp2s0f0 proto static metric 102
10.149.160.0/24 dev enp2s0f0 proto kernel scope link src 10.149.160.21 metric 102
10.149.247.0/24 dev enp10s4f0 proto kernel scope link src 10.149.247.23 metric 100
192.168.2.0/24 dev enp10s4f1 proto kernel scope link src 192.168.2.14 metric 101
[OK, I do have a third interface enp10s4f1 but this one has DEFROUTE=no and no GATEWAY= in its ifcfg file so I haven't mentioned it so far]
See? no metric 110 on the "default via 10.149.247.254 dev enp10s4f0..." line. And the result is the exact opposite to what I'm trying to achieve: enp10s4f0's default gateway has a lower metric than enp2s0f0's
I can confirm this by tracerouting to a host outside of these two networks, packets do get out through 10.149.247.254
I'm stuck at this point. Can't figure it out. Please kindly help if you can.
I'm begging for help here. Been spending hours researching how to solve what seemed like a simple problem: on a CentOS 7.6 host with two network interfaces on two different networks, how to have two default gateways with a set metric to favour one of these interfaces. Can't do it. Would anybody please tell me what I'm missing before I have no hair left?
First, I've tried using the METRIC=xxx setting in /etc/sysconfig/network-scripts-/ifcfg-<interface> files based on some advice found on the web. Obviously if that ever was supported, it's no longer. Next.
Then I've tried using policy-based routing following the tons of documentation that can be found online. Unfortunately very few mention adding metric values.
So, the full details (sorry it's long but I don't want to skip anything): this machine has two active interfaces enp10s4f0 (IP 10.149.247.23, prefix /24) and enp2s0f0 (IP 10.149.160.21, prefix /24).
It has two gateways: 10.149.247.254 and 10.149.160.254. I want 10.149.160.254 to be the "best" gateway, therefore with the lower metric.
I also want to avoid asymetric routing.
I can reach my goal of having different metrics on gateways by manually entering routes with "ip route add default ... dev ... metric" but I want it to be persistent.
So I've created two routing tables in /etc/iproute2/rt_tables:
247 enp10s4f0table
160 enp2s0f0table
And I've made the following rule and route files in /etc/sysconfig/network-scripts:
route-enp10s4f0:
10.149.247.0/24 dev enp10s4f0 src 10.149.247.23 table enp10s4f0table
default via 10.149.247.254 dev enp10s4f0 metric 110 table enp10s4f0table
rule-enp10s4f0:
from 10.149.247.23/32 table enp10s4f0table
to 10.149.247.23 table enp10s4f0table
route-enp2s0f0:
10.149.160.0/24 dev enp2s0f0 src 10.149.160.21 table enp2s0f0table
default via 10.149.160.254 dev enp2s0f0 table enp2s0f0table
rule-enp2s0f0:
from 10.149.160.21/32 table enp2s0f0table
to 10.149.160.21 table enp2s0f0table
Then I've used the proper incantations:
yum install NetworkManager-config-routing-rules
systemctl enable NetworkManager-dispatcher.service
systemctl start NetworkManager-dispatcher.service
...and reboot.
Obviously the configuration is read and processed properly but the metric is *not* applied to the 10.149.247.254 gateway:
# ip route show table enp10s4f0table
default via 10.149.247.254 dev enp10s4f0 metric 110
10.149.247.0/24 dev enp10s4f0 scope link src 10.149.247.23
# ip route show table enp2s0f0table
default via 10.149.160.254 dev enp2s0f0
# ip rule show
0: from all lookup local
32762: from all to 10.149.160.21 lookup enp2s0f0table
32763: from 10.149.160.21 lookup enp2s0f0table
32764: from all to 10.149.247.23 lookup enp10s4f0table
32765: from 10.149.247.23 lookup enp10s4f0table
32766: from all lookup main
32767: from all lookup default
But:
# ip route show
default via 10.149.247.254 dev enp10s4f0 proto static metric 100
default via 10.149.160.254 dev enp2s0f0 proto static metric 102
10.149.160.0/24 dev enp2s0f0 proto kernel scope link src 10.149.160.21 metric 102
10.149.247.0/24 dev enp10s4f0 proto kernel scope link src 10.149.247.23 metric 100
192.168.2.0/24 dev enp10s4f1 proto kernel scope link src 192.168.2.14 metric 101
[OK, I do have a third interface enp10s4f1 but this one has DEFROUTE=no and no GATEWAY= in its ifcfg file so I haven't mentioned it so far]
See? no metric 110 on the "default via 10.149.247.254 dev enp10s4f0..." line. And the result is the exact opposite to what I'm trying to achieve: enp10s4f0's default gateway has a lower metric than enp2s0f0's
I can confirm this by tracerouting to a host outside of these two networks, packets do get out through 10.149.247.254
I'm stuck at this point. Can't figure it out. Please kindly help if you can.