No Boot Media in EFI - Custom ISO Image

[CarbonDPG]

Hey folks,

Hoping someone can help me with this. I'm also certain this is a stupid mistake that I haven't caught, but you never know.

Background: I'm customizing my own CentOS 9 Stream ISO. We'll be phasing out support for the CentOS 7 Linux servers within the next year (along with their software suite). The goal is to automate as much of the deployment as I can. Given the various environments where this will be deployed (40+ unique environments), I figured the easiest way was KickStart in the ISO. Ansible wasn't going to be easy to do as these VMs will be deployed in locations that I can't control DHCP, PXE, TFTP, etc. Not to mention, spread across the world with no centralized access, and in most cases, restricted internet access. Ansible-pull might be an option down the line. But the goal is an install that only prompts for IP information and a hostname. Everything else is automated. Kickstart is filling that function for me.

Anyways, onto talking about the customizations. Note: A lot of these were necessary because CentOS does not release a "minimal" ISO (I'm aware of 'boot.iso'), which is a bit asinine... but whatever.

• Ripped apart and removed significant portions of AppStream/
• Ripped apart and removed *i686* packages from BaseOS/
• Created new repo data (AppStream/repodata/ and BaseOS/repodata/)
• Added kickstart.cfg that:
  • Sets up default accounts
  • Sets up high-level default firewall policies
  • Sets up SE linux
  • Sets up Security Policies
• Added addition '3rd Party' repos (self-hosted and synced).
• Configures %packages
  • @^custom-environment
  • @ standard
  • @ base
  • @ hardware-monitoring
  • @ network-tools
  • @ performance
  • <3rd Party apps found in the custom repos>
• Configures %post sections
  • Re-writes default SSH/Management policies.
  • Rewrites Default Logging Policies.
  • Rewrites config for 3rd Party Application(s).
  • Attaches MGMT profile.
  • Installs additional 3rd party apps without public repos.
  • Configures granular firewall policy.
  • <and others>
• Modifications to:
  • isolinux/isolinux.cfg
  • isolinux/grub.conf
  • efi/boot/grub.conf
  • Screenshots of the modified files: https://imgur.com/a/ueON3mS

All of these changes work when I boot in via BIOS/MBR. When I switch my hypervisors over to EFI (with or without Secure Boot), it no longer reads the media. While I could force people to use BIOS/MBR, I'd prefer to minimize any additional configurations, and also retain the additional features present in UEFI (notably, larger drive sizes and secure boot!)

I spent time reading both RedHat's documentation on customizing ISOs under solution #60959 (https://access.redhat.com/solutions/60959) and VMWare's documentation (https://docs.vmware.com/en/VMware-vSphe ... B1F9C.html). RedHat's is somewhat more applicable here, obviously.

RedHat's documentation ends with 3 commands to theoretically generate your own hybrid EFI/BIOS-bootable image:

Code: Select all

# mkisofs -o /tmp/<Appliance Name>.iso -b isolinux/isolinux.bin -J -R -l -c isolinux/boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table \
-eltorito-alt-boot -e images/efiboot.img -no-emul-boot -graft-points -joliet-long -V "Appliance" .
# isohybrid --uefi /tmp/rhel7test.iso 
# implantisomd5 /tmp/rhel7test.iso

Here's <..>/EFI/BOOT/grub.cfg

Code: Select all

menuentry 'Install Appliance via UEFI' --class fedora --class gnu-linux --class gnu --class os {
    linuxefi /images/pxeboot/vmlinuz inst.repo=cdrom inst.ks=cdrom:/kickstart.cfg inst.stage2=hd:LABEL=Appliance quiet
    initrdefi /images/pxeboot/initrd.img
}

Which leads to this > https://imgur.com/a/ye0u1lB

Running 'File' and 'isoinfo' against both the default CentOS 9 Stream DVD, and my own custom DVD, I can't find any differences that would impact the EFI side of it. (Notably: mkisofs executes fully and does output a *.iso file to my tmp directory.) The even more interesting part is what happens if I rip the image to a USB stick and attempt baremetal installs. The Intel NUC (test PC) will read the media just fine in UEFI/EFI. Ripped to a USB stick using rufus.

I'm assuming this is a stupid oversight somewhere in a command. Obviously, I'm doing something wrong; does anyone know what it is?

Cheers,
CDP

[TrevorH]

You really don't want to use CentOS Stream for production work. It is too unstable and lacks timely security updates (though 9 is better than 8). I'd look at Alma/Rocky or OEL instead as those are still 1:1 rebuilds of RHEL, unlike Stream which gets updates before they go to RHEL (except when they are embargoed security updates - i.e. the important ones! when they don't come out until after RHEL gets them).

The iso images need to be hybrid.

[CarbonDPG]

I think Rocky is supported by the software we use - though I'll have to double check that. (Even being 1:1 RHEL builds, the checks implemented by one vendor are strict, and will abort on unsupported operating systems. I have tried to force the install, several times...)

If CentOS 9 is that unstable, I will transition over to a full RHEL9 build or Rocky (assuming it's supported). Regardless of which RHEL-based system we use, Rocky, RHEL, or CentOS Stream, I suspect I will encounter similar issues. I'll poke around with Rocky tomorrow though. I'll update some when I've had a chance to test out a Rocky install.

Thanks Trevor!

[jlehtone]

If CentOS 9 is that unstable, I will transition over to a full RHEL9 build or Rocky (assuming it's supported). Regardless of which RHEL-based system we use, Rocky, RHEL, or CentOS Stream, I suspect I will encounter similar issues.

First, CentOS Stream is not "RHEL-based", unlike the Alma and Rocky. Check both Alma and Rocky with that application.
(If neither is "regognized", then reach out to those projects; they are probably happy to contact the application vendor as extra incentive.)


Years ago I had a custom system. Gigabyte board, IIRC. CentOS did install fine and I got GPT on the drive. Then MS Windows (7?), from USB made by MS. "Can't install.". It did turn out that the USB could boot only in legacy mode. Windows installer luckily aborted as it could not use the existing GPT.

Why only legacy? In that case the case of filenames was not what the board's EFI could accept. Considering that filesystem was case-insensitive FAT, that EFI implementation was obnoxious. I did look at filenames in CentOS USB, copied MS USB to new stick and renamed the "EFI" subdirectory contents. It is unlikely that your various systems would have that issue.

Ripped to a USB stick using rufus.

I'd rather use dd -- rufus with default options is notorious.

[TrevorH]

Good catch, I missed the Rufus bit. It works but only if you select the correct options and the developer was here and basically said it was a problem with the CentOS isos and refused to fix it. Use dd.

[CarbonDPG]

Good catch, I missed the Rufus bit. It works but only if you select the correct options and the developer was here and basically said it was a problem with the CentOS isos and refused to fix it. Use dd.

I'm less concerned with Rufus. The rufus part of that was just my curiosity on whether this would work on a baremetal machine as opposed to a type 1 hypervisor. I mentioned it in brief because the USB stick that was written by Rufus was booting into the UEFI section and BIOS sections if I forced one or the other. I found it interesting that Rufus USB was working, but the ISO I provided Rufus wasn't working when loaded into the hypervisors. I have no plans on supporting baremetal installs and intend on forcing people into a Type 1 hypervisor (Hyper-V, ESX, Nutanix, etc). To be frank, anyone who's using this will be large enough to have virtualization anyways. The abstracted physical hardware into virtual hardware is easier to deal with. Same reasoning that we don't support other appliances on baremetal, unless that baremetal was specially ordered from a specific vendor (and is still supported by said vendor...)

Rocky 9 is not (yet) supported by our software manufacturer, though Rocky 8 is explicitly mentioned, along with CentOS 7 Linux, CentOS 8 Stream, and RHEL 7 and 8. I reached out to the vendor who's confirmed support for RHEL9 but needed to check with another team for Rocky 9. Given their track history, I'd say support is likely though, so I'll move forward with testing Rocky.

On that note, in case anyone else encounters EFI boot issues when generating custom ISOs, this worked for me:

Code: Select all

mkisofs \
 -o /tmp/<name>.iso \
 -b isolinux/isolinux.bin \
 -c isolinux/boot.cat \
 -boot-load-size 4 \
 -boot-info-table \
 -no-emul-boot \
 -eltorito-alt-boot \
 -e images/efiboot.img \
 -no-emul-boot \
 -R \
 -J \
 -V <LABEL> \
 -T \
 .

TLDR of that command - you need to know the LABEL specified in ./EFI/BOOT/grub.cfg and ./isolinux/isolinux.cfg. By default, it's "Rocky-9-1-x86_64-dvd" for the Rocky 9.1 DVD ISO file. If you're using a different version, check the above files and look for LABEL=<...>. I also disabled the quiet mode for testing.

Thanks folks!