/boot/efi on raid(mdadm) partition

Issues related to applications and software problems and general support
Post Reply
User avatar
javajox
Posts: 25
Joined: 2019/10/11 10:08:05
Location: Internet
Contact:

/boot/efi on raid(mdadm) partition

Post by javajox » 2019/10/28 08:59:10

I have the following setup (# lsblk -f) :

NAME FSTYPE LABEL UUID MOUNTPOINT
sda
├─sda1 linux_raid_member test123:swap a8e7756b-729b-f84c-9171-abaee360b443
│ └─md127 swap SWAP 5d9f3b69-6f48-4d06-8e31-4387ce57765d [SWAP]
├─sda2 linux_raid_member test123:boot 8b594736-5e80-5354-b3ae-4c84adf27302
│ └─md126 xfs BOOT 42356354-d2bb-41a4-841d-84acec4f4f9b /boot
├─sda3 linux_raid_member test123:boot_efi 873f27e7-3420-7c15-2c1f-b4dfed8438f4
│ └─md124 vfat BOOT_EFI 7EAB-DA36 /boot/efi
└─sda4 linux_raid_member test123:root cf71f772-31a1-1ef5-e04e-ce575843ac56
└─md125 xfs ROOT 450ded12-6923-489f-a71f-69d31024fd89 /
sdb
├─sdb1 linux_raid_member test123:swap a8e7756b-729b-f84c-9171-abaee360b443
│ └─md127 swap SWAP 5d9f3b69-6f48-4d06-8e31-4387ce57765d [SWAP]
├─sdb2 linux_raid_member test123:boot 8b594736-5e80-5354-b3ae-4c84adf27302
│ └─md126 xfs BOOT 42356354-d2bb-41a4-841d-84acec4f4f9b /boot
├─sdb3 linux_raid_member test123:boot_efi 873f27e7-3420-7c15-2c1f-b4dfed8438f4
│ └─md124 vfat BOOT_EFI 7EAB-DA36 /boot/efi
└─sdb4 linux_raid_member test123:root cf71f772-31a1-1ef5-e04e-ce575843ac56
└─md125 xfs ROOT 450ded12-6923-489f-a71f-69d31024fd89 /

This scheme has been created with the standard (Anaconda) CentOS 8 installer. The system boots correctly and everything is working fine. If you see the /boot/efi partition is located on a md124 software raid, the file system is FAT. From what I understand during boot process in UEFI mode, core.img can't work with a software/mdadm raid device, but it sees the individual partition in this case sdb3 and sda3 (with fs of type FAT), so it transfers the control to one of this partition and the system boots normally as expected. If a disk fails (let say sda) it will boot from sdb.

Is this reasoning correct ?

User avatar
KernelOops
Posts: 428
Joined: 2013/12/18 15:04:03
Location: xfs file system

Re: /boot/efi on raid(mdadm) partition

Post by KernelOops » 2019/11/15 22:07:52

Interesting, I would expect the system to fail to boot when /boot/efi is on software raid, apparently it boots!

In my case, here is my file system structure:

Code: Select all

sda1	/boot/efi
sda2	linux_raid_member		(/boot partition)
sda3	linux_raid_member		(/ root partition)
sdb is identical to sda.

the idea is, to copy the structure of sda to sdb (and sd*) so that all drives are bootable via /boot/efi.

all I had to do, was sfdisk to dump the sda structure to a text file, create new UUIDs and create the new structure on the other drives (each time re-creating new UUIDs of course).

PS:
As you will notice, I don't use swap partitions, I haven't used them for over a decade.
--
R.I.P. CentOS :cry:
--

User avatar
javajox
Posts: 25
Joined: 2019/10/11 10:08:05
Location: Internet
Contact:

Re: /boot/efi on raid(mdadm) partition

Post by javajox » 2019/11/19 18:07:22

KernelOops,

It boots because, grub will transfer control to either sdb3 or sda3 (depending on bios settings), of course at that time mdadm is not active, but grub sees the files on the disk and starts to execute code from those files.

Actually I had to abandon my scheme, because executable code located in /boot/efi can create or change files on that partition, at that time mdadm is not active, and as a result it will corrupt the raid. So I switched to legacy bios mode.

You have a nice solution, but what will happen when executable code from /boot/efi will create/modify files from sda1 or sdb1 ?

User avatar
KernelOops
Posts: 428
Joined: 2013/12/18 15:04:03
Location: xfs file system

Re: /boot/efi on raid(mdadm) partition

Post by KernelOops » 2019/11/20 12:17:11

Interesting question. My thinking is as follows, the system always boots and mounts /dev/sda1, since I don't care about changes that may happen during booting, I can ignore them. The only changes I care about, is when "dnf" (or "yum") update packages in /boot/efi, so when that happens I 'dd' the /dev/sda1 to the other drives (/dev/sdb1, etc).

So far, the above scheme has worked fine over the years and when I had to replace the /sda with another drive, booting worked fine and I moved on like nothing had happened.
--
R.I.P. CentOS :cry:
--

jabuzzard
Posts: 30
Joined: 2008/09/18 12:35:09

Re: /boot/efi on raid(mdadm) partition

Post by jabuzzard » 2019/11/21 13:15:05

My solution for booting when putting root on a software RAID is to cheat and put /boot on a USB flash device. This way you can have root on software RAID 5 and 6 too.

Sure it creates a single point of failure, however /boot is very infrequently written or read so you are unlikely to wear out the flash in the lifetime of the machine. Secondly I rsync /boot onto the the software RAID from crontab; if some goes horribly wrong in the worst case scenario I can install new USB flash device, boot from a rescue CD/USB key copy the contents onto the new USB boot device and reinstall grub. Thirdly the point in time when it is most likely to fail is during a dnf update when installing a new kernel at which point I will know it is broken and can move to replacing the flash drive when I reboot. Also if it fails on a read I will get an email notification from a failed crontab entry.

Using quality USB flash drives from known reputable sources, and making sure they are not fakes with f3 before using and I have never had a flash drive fail.

The only issue with this scheme is not all motherboard have a type A USB socket on them, and the USB DOM are stupidly expensive. However if you are any use with a soldering iron just buy a physically small USB flash device without an enclosed USB connector. I originally used Sony Micro Vault Tiny drives like this

https://www.ebay.co.uk/itm/143436132561

but due to cheaping out and getting 256MB ones I have had to replace them due to the limited capacity. I used some Cruzer Pop's like this

https://www.ebay.co.uk/itm/264292920627

then just hand crimp with small pliers some solid copper wire onto the female terminal contact and secure with the smallest amount of solder. Insert into the header trim to length and solder to the USB drive. The parts are not very expensive, see below though I can't immediately find the polarising key's but you don't technically need them and a you can just ram a match stick in if want.

https://uk.rs-online.com/web/p/pcb-conn ... s/6812878/
https://uk.rs-online.com/web/p/pcb-conn ... s/6812856/

Of course if your motherboard has a type A header then a Cruzer Fit style drive is the way to go. I have had very limited (as in none) success in removing a USB plug from a dismantled drive, hence the use of drives with open connectors.

I would highly recommend this scheme to anyone doing software RAID. It makes life way simpler than the traditional scheme of root on software RAID which is limited to RAID 1 anyway.

BShT
Posts: 584
Joined: 2019/10/09 12:31:40

Re: /boot/efi on raid(mdadm) partition

Post by BShT » 2019/11/21 13:25:03

:o

Post Reply