network interfaces order is changed after upgrading CentOS 6.3 to CentOS 7.5

Issues related to configuring your network
User avatar
moradiv
Posts: 21
Joined: 2019/04/18 09:34:53

Re: network interfaces order is changed after upgrading CentOS 6.3 to CentOS 7.5

Post by moradiv » 2019/04/22 09:07:02

Thank you for your reply.
aks wrote:
2019/04/22 07:57:20
I guess you'd have to remove all of them then add them one at a time. Or set one of them to be (say) eth9 and then shuffle about.
Sounds correct. Do you have a way do do that automatically? Any idea how to perform that?
aks wrote:
2019/04/22 07:57:20
I don't really understand why you need a specific order
The issue happens after upgrading (export -> reinstall -> import) our system. The product is configured to use specific interface for management or data, so the upgraded system needs to use the same interfaces it used before. The interfaces naming has to remain without change.

User avatar
moradiv
Posts: 21
Joined: 2019/04/18 09:34:53

Re: network interfaces order is changed after upgrading CentOS 6.3 to CentOS 7.5

Post by moradiv » 2019/04/23 13:37:58

One more thing I tried (Spoiler: it didn't help) is creating new initramfs-$(uname -r).img file, which I injected into the following udev files:
- /etc/udev/rules.d/69-vmxnet3-net.rules with the following content

Code: Select all

SUBSYSTEM=="net", ACTION=="add", KERNELS=="0000:0b:00.0" DRIVERS=="vmxnet3", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"
SUBSYSTEM=="net", ACTION=="add", KERNELS=="0000:13:00.0" DRIVERS=="vmxnet3", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1"
SUBSYSTEM=="net", ACTION=="add", KERNELS=="0000:1b:00.0" DRIVERS=="vmxnet3", ATTR{type}=="1", KERNEL=="eth*", NAME="eth2"
SUBSYSTEM=="net", ACTION=="add", KERNELS=="0000:04:00.0" DRIVERS=="vmxnet3", ATTR{type}=="1", KERNEL=="eth*", NAME="eth3"
SUBSYSTEM=="net", ACTION=="add", KERNELS=="0000:0c:00.0" DRIVERS=="vmxnet3", ATTR{type}=="1", KERNEL=="eth*", NAME="eth4"
SUBSYSTEM=="net", ACTION=="add", KERNELS=="0000:14:00.0" DRIVERS=="vmxnet3", ATTR{type}=="1", KERNEL=="eth*", NAME="eth5"
SUBSYSTEM=="net", ACTION=="add", KERNELS=="0000:1c:00.0" DRIVERS=="vmxnet3", ATTR{type}=="1", KERNEL=="eth*", NAME="eth6"
SUBSYSTEM=="net", ACTION=="add", KERNELS=="0000:05:00.0" DRIVERS=="vmxnet3", ATTR{type}=="1", KERNEL=="eth*", NAME="eth7"
- /usr/lib/udev/rules.d/80-net-name-slot.rules with the following content

Code: Select all

[root@localhost ~]# cat /usr/lib/udev/rules.d/80-net-name-slot.rules
# do not edit this file, it will be overwritten on update

ACTION!="add", GOTO="net_name_slot_end"
SUBSYSTEM!="net", GOTO="net_name_slot_end"
NAME!="", GOTO="net_name_slot_end"

IMPORT{cmdline}="net.ifnames"
ENV{net.ifnames}=="0", GOTO="net_name_slot_end"

NAME=="", ENV{ID_NET_NAME_ONBOARD}!="", NAME="$env{ID_NET_NAME_ONBOARD}"
NAME=="", ENV{ID_NET_NAME_SLOT}!="", NAME="$env{ID_NET_NAME_SLOT}"
NAME=="", ENV{ID_NET_NAME_PATH}!="", NAME="$env{ID_NET_NAME_PATH}"

LABEL="net_name_slot_end"
created a new initramfs using the command:
dracut -f /tmp/initramfs-$(uname-r).img $(uname-r) -I "/etc/udev/rules.d/69-vmxnet3-net.rules /usr/lib/udev/rules.d/80-net-name-slot.rules"

And replaced it with the one located in /boot.
I found a solution on a forum advising to comment the following line: NAME=="", ENV{ID_NET_NAME_ONBOARD}!="", NAME="$env{ID_NET_NAME_ONBOARD}". tried injecting the file with the commented line
Bottom line, it changed nothing and the problem still exists.

Any other idea there? I'm helpless. :(

aks
Posts: 3073
Joined: 2014/09/20 11:22:14

Re: network interfaces order is changed after upgrading CentOS 6.3 to CentOS 7.5

Post by aks » 2019/04/23 19:40:49

Sounds correct. Do you have a way do do that automatically? Any idea how to perform that?
Um, remove all ethX devices at first thing (ACTION="remove" I seem to recall)?
product is configured to use specific interface
Don't do that, it's called technical debt and will pile up longer than you (or I) could imagine. Seriously. I know that devs/product owner/<insert another layer of crap that thinks this is a good idea> don't appreciate that, but then they are the cause of all bugs! LOL. So what do "they" know. An interface should have no context. It's a "modern" way and it's 100% guaranteed to get more important. Ja for sure (just to highlight this is a bad thing).

User avatar
moradiv
Posts: 21
Joined: 2019/04/18 09:34:53

Re: network interfaces order is changed after upgrading CentOS 6.3 to CentOS 7.5

Post by moradiv » 2019/04/24 13:28:01

It didn't solve the problem:
I tried to change the rules file to be as the attached image, first remove all vmxnet3, and then add it.
Also created new initramfs with this new rules file.

But no change, same error, same situation.

Am I doing it right?
Attachments
Capture.PNG
Capture.PNG (64.89 KiB) Viewed 2772 times

User avatar
moradiv
Posts: 21
Joined: 2019/04/18 09:34:53

Re: network interfaces order is changed after upgrading CentOS 6.3 to CentOS 7.5

Post by moradiv » 2019/04/25 06:59:59

Some new information: this issue happens also on a new installation:
When installing a our CentOS 7.5 based image on a vmware machine with 4 network interfaces, when the installation is done through net install (PXE server) - the issue occurs.
The machine starts up with the wrong network interfaces order, and the same behavior described in this thread.
As shown in the attached image: the first network interface on the VM is eth1 (instead of eth0), the second is eth2 (instead of eth1) and so on.
After adding the udev file: 69-vmxnet3-net.rules, and restarting. I'm getting the udev errors described while trying to fix the interfaces order:
"Error changing interface name 'ethX' to 'ethY': File exists"

Why does it happen?
Attachments
Capture.PNG
Capture.PNG (19.43 KiB) Viewed 2752 times

aks
Posts: 3073
Joined: 2014/09/20 11:22:14

Re: network interfaces order is changed after upgrading CentOS 6.3 to CentOS 7.5

Post by aks » 2019/04/25 19:37:01

Am I doing it right?
I seem to think that you have to remove all "the things", not the generalization - in the sense of find out what has happened, remove all of it and then add.

Here's a story (that I hope may inspire):

Many, many years ago I was responsible for many machines that would have (at a minimum) 4 interfaces for the various communication providers we used. I can't use the local MAC address (it's kind of "made up" - it's a VM). All interfaces had a DHCP provided address (which can change - but there was no "collision" across networks - one was 10.0.10.0/24 and the other was 192.168.0.0/24 and so on). The only things I could use was:

1. Can I get to a "large" (like google.com) internet provider from this particular interface? If so that's "default interface" - i.e.: Internet bound.
2. Can I get to a (a known, or a set of known) mobile endpoints - then that's the interface for that provider (so Voda phone is Voda phone).
3. Repeat the above for other network providers.
4. Re-number interfaces (eth0, eth1 etc.) depending on what can get to from where. Use ifrename like a b*tch.
5. Adjust local routing tables as is required. Make default Internet bound. Make Voda Voda bound as so on.
6. Test that I can get to the specific end points beased on the interface(s) and fail on any of the tests (which never happened in 10 years - it's not that hard).
7. Sit back and admire mnself. LOL.

The "harder" things:

1) Doing this in a rc.local script - CentOS/RHEL 7 really doesn't "like" you do this, it is depreciated. A bucket load of experimentation, especially with SE enabled.
2) What to do on fail. What to do when the logic is wrong. I was fortunate in the sense that I knew what (interface) sould be where (to get to).
3) Doing this all in a kick start to make sure the "default route"/"default interface" can get to the YUM repositories required to install/updates the OS. Although I think I ignored most things for the install - I only need internet bound (for PXE install).

File exists means it's already there. Seriously the story has not changed in your post.

Regards

tails_uk
Posts: 2
Joined: 2019/11/10 20:00:04

Re: network interfaces order is changed after upgrading CentOS 6.3 to CentOS 7.5

Post by tails_uk » 2019/11/12 12:56:44

Did you ever find a solution to this? This is the same problem I am battling with now

User avatar
moradiv
Posts: 21
Joined: 2019/04/18 09:34:53

Re: network interfaces order is changed after upgrading CentOS 6.3 to CentOS 7.5

Post by moradiv » 2019/11/12 13:55:44

for vmware i ran in the post installation script:
# back to CentOS7 default consistent network interfaces naming
sed -i 's/\(GRUB_CMDLINE_LINUX=.*\)net.ifnames=0 biosdevname=0\s\?/\1/' /etc/default/grub
grub2-mkconfig -o /boot/grub2/grub.cfg

i=0
for nic_bus in $(ls -la /sys/class/net/e* | awk '{t=$11"::"$9; print t}' | sort); do
nic_name="$(echo ${nic_bus} | awk -F'::' '{print $1}' | awk -F/ '{print $NF}')"
nic_mac=$(cat /sys/class/net/${nic_name}/address)
echo "HWADDR=${nic_mac}" >> /etc/sysconfig/network-scripts/ifcfg-eth${i}
((i++))
done

and for ovf i used udev file the I manually created to set up the order correctly:
cat <<"EOF" > /etc/udev/rules.d/69-vmxnet3-net.rules
SUBSYSTEM=="net", ACTION=="add", KERNELS=="0000:03:00.0" DRIVERS=="vmxnet3", ATTR{type}=="1", KERNEL=="e*", NAME="eth0"
SUBSYSTEM=="net", ACTION=="add", KERNELS=="0000:0b:00.0" DRIVERS=="vmxnet3", ATTR{type}=="1", KERNEL=="e*", NAME="eth1"
SUBSYSTEM=="net", ACTION=="add", KERNELS=="0000:13:00.0" DRIVERS=="vmxnet3", ATTR{type}=="1", KERNEL=="e*", NAME="eth2"
SUBSYSTEM=="net", ACTION=="add", KERNELS=="0000:1b:00.0" DRIVERS=="vmxnet3", ATTR{type}=="1", KERNEL=="e*", NAME="eth3"
SUBSYSTEM=="net", ACTION=="add", KERNELS=="0000:04:00.0" DRIVERS=="vmxnet3", ATTR{type}=="1", KERNEL=="e*", NAME="eth4"
SUBSYSTEM=="net", ACTION=="add", KERNELS=="0000:0c:00.0" DRIVERS=="vmxnet3", ATTR{type}=="1", KERNEL=="e*", NAME="eth5"
SUBSYSTEM=="net", ACTION=="add", KERNELS=="0000:14:00.0" DRIVERS=="vmxnet3", ATTR{type}=="1", KERNEL=="e*", NAME="eth6"
SUBSYSTEM=="net", ACTION=="add", KERNELS=="0000:1c:00.0" DRIVERS=="vmxnet3", ATTR{type}=="1", KERNEL=="e*", NAME="eth7"
SUBSYSTEM=="net", ACTION=="add", KERNELS=="0000:05:00.0" DRIVERS=="vmxnet3", ATTR{type}=="1", KERNEL=="e*", NAME="eth8"
EOF

for other appliances I used net.ifnames=0 and it worked correctly

bupkes
Posts: 4
Joined: 2017/06/12 18:08:27

Re: network interfaces order is changed after upgrading CentOS 6.3 to CentOS 7.5

Post by bupkes » 2020/10/21 03:23:17

I know this is an old post but this may be helpful for those landing here from a search.

The naming format of "eth#" should be thought of as "reserved for kernel". There are plenty of notes about using something else but none that provide a hint of the trouble it will cause if you don't and have multiple interfaces.

Solution:
You can use almost anything other than "eth#" as the interface name and not have trouble (lan, lan0, myeth0...).
Reason:
The interface names are assigned by the kernel as they are detected at boot starting with eth0 and sent to udev to process. Multiple threads are used by udev so it's possible for eth0 and eth1 to be handled by separate threads nearly simultaneously. If the thread handling eth1 determines from an ifcfg file that the MAC address is assigned to eth0, it will attempt to rename it but will get an error because eth0 is already taken. Mixing and matching udev rules and ifcfg files can get you into trouble if you don't understand the details here (I think it applies to both CentOS 7 & 8):

If you absolutely must use eth# format, you can first boot using other names and then rename prior to launching whatever requires the old naming. However, you will then have config files or udev rules that do not match your actual used names. The renaming must account for all situations that might cause the device or driver to reload.

Additional notes:
  • Notice that all the articles stating "How to rename the interface back to eth0" are only referring to a single interface.
  • You should not assume devices are discovered in any particular order or even consistently over boots.
  • Linux will consistently name interfaces regardless of detected order just like you expect from Windows if you follow the rules.
  • If in doubt about what's happening, enable udev logging. It's cryptic but readable.
  • In fact, you can use eth# with numbers higher than the number of interfaces the kernel will detect but why would you? (ie use eth4 through eth7 if kernel will only use eth0 through eth3)
  • This post is the result of pouring over docs and code trying to figure out workable solutions for an old app that mandated the use of eth# for multiple interfaces. The app should have been better designed, udev should have been better documented and I should get those hours back. I hope what I posted is correct.

Post Reply