PXE boot / netboot

PXE boot / netboot

What is it?

PXE booting is the process of booting a .pxe (or .efi) file over the network. Usually these PXEs can be chainloaded, meaning you can boot from one into the next. This allows for example for multiple deplyment steps, or (in our case) to load the netboot.xyz PXE first, select an installation or live image, and have netboot.xzy download and boot into the netboot PXE of the chosen distro.

netboot.xyz

netboot.xyz provides a menu where the user can see carious information and select out of a wide range of netboot installations and live systems to boot into. netboot.xyz is based on iPXE.

iPXE

There have been many solutions for PXE booting in Linux land. Syslinux and others have a lot of legacy documentation und forum threads out there. The current way to solve PXE booting in Linux currently (2024) is to use the open-source iPXE project. iPXE provides PXE files for legacy and UEFI systems and allows for a trusted start of the PXE chain.

PXE booting flow

The PXE boot works like this: The DHCP server of the network advertises to its clients that it provides PXE boot files. The PXE boot files have to be served via tftp and can be located on another server than the DHCP server.

In OpenWrt we can configure dnsmasq to advertise the PXE information and serve the PXE files as well! The configuration has to be done through the command line, LuCI has no options for this.

SSH into OpenWrt and create a new directory to download the PXE files into. This will be the ftpd root later as well. Then download the PXE files from https://netboot.xyz/downloads/:

mkdir /mnt/ftpd
cd /mnt/ftpd/
wget https://boot.netboot.xyz/ipxe/netboot.xyz.kpxe
wget https://boot.netboot.xyz/ipxe/netboot.xyz.efi
wget https://boot.netboot.xyz/ipxe/netboot.xyz-snponly.efi
wget https://boot.netboot.xyz/ipxe/netboot.xyz-arm64.efi
wget https://boot.netboot.xyz/ipxe/netboot.xyz-arm64-snponly.efi
wget https://boot.netboot.xyz/ipxe/netboot.xyz-rpi4-snp.efi
👾
These are the files for x86_64 based systems. You can download additional PXE files if you want to boot other architectures as well (ARM64, Raspberry Pi 4, …)

Add this to /etc/config/dhcp (replace the tftp_root, serveraddress and servername with yours):

/etc/config/dhcp
config dnsmasq
    option enable_tftp '1'
    option tftp_root '/mnt/tftp'

config match
        option networkid 'bios'
        option match '60,PXEClient:Arch:00000' # x86 bios
config match
        option networkid 'efi64'
        option match '60,PXEClient:Arch:00007' # x64 UEFI
config match
        option networkid 'efi64'
        option match '60,PXEClient:Arch:00006' # x86 UEFI
config match
        option networkid 'efiarm64'
        option match '60,PXEClient:Arch:00011' # ARM 64-bit UEFI

config match
        option networkid 'efirpi3'
        option match '60,PXEClient:Arch:00000:UNDI:002001"' # rpi 3b+

config match
        option networkid 'efirpi'
        option match '60,PXEClient:Arch:00041' # arm rpiboot

config userclass
        option networkid 'ipxe'
        option userclass 'iPXE'

config boot
        option filename         'tag:bios,tag:!ipxe,netboot.xyz.kpxe'
        option serveraddress    '192.168.1.1'
        option servername       'OpenWrt'
config boot
        option filename         'tag:bios,tag:ipxe,netboot.xyz.kpxe'
        option serveraddress    '192.168.1.1'
        option servername       'OpenWrt'
config boot
        option filename         'tag:efi64,tag:!ipxe,netboot.xyz-snponly.efi'
        option serveraddress    '192.168.1.1'
        option servername       'OpenWrt'
config boot
        option filename         'tag:efi64,tag:ipxe,netboot.xyz.efi'
        option serveraddress    '192.168.1.1'
        option servername       'OpenWrt'

config boot
        option filename         'tag:efiarm64,tag:ipxe,netboot.xyz-arm64.efi'
        option serveraddress    '192.168.1.1'
        option servername       'OpenWrt'
config boot
        option filename         'tag:efiarm64,tag:!ipxe,netboot.xyz-arm64-snponly.efi'
        option serveraddress    '192.168.1.1'
        option servername       'OpenWrt'
config boot
        option filename         'tag:efirpi3,raspios/'
        option serveraddress    '192.168.1.1'
        option servername       'OpenWrt'
config boot
        option filename         'tag:efirpi,netboot.xyz-rpi4-snp.efi'
        option serveraddress    '192.168.1.1'
        option servername       'OpenWrt'
👾
These are the files for x86_64 based systems. You can download additional PXE files if you want to boot other architectures as well (ARM64, Raspberry Pi 4, …) You can find a full list of all client architectures at iana.org

Commit changes and restart dnsmasq:

uci commit
service dnsmasq restart

You should now be able to boot any BIOS or UEFI based system via PXE over your network.

credits:

Raspberry Pi 3B+

All Raspberry Pis up to and including this model can netboot through a bootcode.bin file on the SD card or the via the burnt in bootcode (later models). Later models (Pi 4 and up) have a separate EEEPROM that handles netboot and is configured in a different way. Even though netboot without an SD card/bootcode.bin is possible on later models, it is recommended to use the most current bootcode.bin on a SD card, as it contains fixes.

Special stuff

Serve symlinks with TFTP

TFTPD keeps the tftpdboot directory chrooted so it cannot follow symlinks outside of it’s directory. That means two things:

  1. The symlink needs to be set to a relative path, not an absolute path. In the context of the tftp root.
  2. The tftp server in OpenWrt can’t serve from outside its root, meaning paths like ../some.file won’t work.

How to serve NFS mount with TFTP

Use bind mounts to mount a directory into another (empty) directory:

mount --bind /some/where /else/where
# Download and extract image
wget https://downloads.raspberrypi.com/raspios_lite_arm64/images/raspios_lite_arm64-2023-12-11/2023-12-11-raspios-bookworm-arm64-lite.img.xz
xz -T 0 -vv -d ./2023-12-11-raspios-bookworm-arm64-lite.img.xz

# Create mountpoints for partitions and create devices for each partition
mkdir -p ./bootfs ./rootfs
LOOPDEV=$(sudo losetup -f --show -P -r 2023-12-11-raspios-bookworm-arm64-lite.img)
sudo mount -r /dev/loop0p1 ./raspios/img1/
sudo mount -r /dev/loop0p2 ./raspios/img2/

# Copy the partition content to disk
mkdir -p ./raspios/img1 ./raspios/img2
cp ./bootfs/* ./raspios/img1/
cp ./rootfs/* ./raspios/img2/

# unmount partitions and devices
sudo umount ./bootfs ./rootfs
sudo losetup -d $LOOPDEV

credits:

iPXE menu

tftproot/menu.ipxe
#!ipxe

# Display login form
:login
login || goto login_failed
Noted, let's get to booting..!

#console --picture https://online.ilove.rocks/

set pxe-address http://${username}:${password}@pxe.lan
set arch ${buildarch}

# Figure out if client is 64-bit capable
# cpuid --ext 29 && set arch x64 || set arch x86
# cpuid --ext 29 && set archb 64 || set archb 32
# cpuid --ext 29 && set archl amd64 || set archl i386

# Some menu defaults
set menu-timeout 15000
set submenu-timeout ${menu-timeout}
isset ${menu-default} || set menu-default exit

###################### MAIN MENU ####################################

:start
menu Please choose an operating system to boot (You are drving: ${buildarch})
item --gap --             ------------------------- Operating systems ------------------------------
item --key n netboot.xyz netboot.xyz - your favorite operating systems in one place
item --key a archinstall Unattended archinstall (DELETES ALL YOUR DATA WITHOUT ASKING!)
#item --gap --            ------------------------- Tools and utilities ----------------------------
item --gap --             ------------------------- Advanced options -------------------------------
item shell                    Drop to iPXE shell
item reboot                   Reboot computer
item --key x exit             Exit iPXE and continue BIOS boot
choose --timeout ${menu-timeout} --default ${menu-default} selected || goto cancel
set menu-timeout 0
goto ${selected}

:archinstall
menu ALL YOUR DATA WILL BE DELETED! ARE YOU OKAY WITH THAT?
item no                 NO
item no                 HELL NO!
item no                 Never
item no                 Take me back
item archinstall-yes                yes
item no                 WAH! No!
item no                 Please no..
item --gap --             ------------ GET ME OUT OF HERE! --------------
item --key x back        Back to main menu
choose --timeout ${menu-timeout} --default ${menu-default} selected || goto cancel
set menu-timeout 0
goto ${selected}

:archinstall-yes
kernel ${pxe-address}/archlinux/arch/boot/x86_64/vmlinuz-linux ip=dhcp archisobasedir=archlinux archiso_http_srv=${nas_address}/archlinux/ cms_verify=n
initrd ${pxe-address}/archlinux/arch/boot/intel-ucode.img
initrd ${pxe-address}/archlinux/arch/boot/amd-ucode.img
initrd ${pxe-address}/archlinux/arch/boot/x86_64/initramfs-linux.img
boot || goto failed

:netboot.xyz
iseq ${arch} i386 && chain ${pxe-address}/netboot.xyz.kpxe ||
iseq ${arch} x86_64 && chain ${pxe-address}/netboot.xyz.efi ||
# iseq ${arch} arm32 && chain ${pxe-address}/netboot.xyz.kpxe ||
iseq ${arch} arm64 && chain ${pxe-address}/netboot.xyz-arm64.efi



:cancel
echo You cancelled the menu, dropping you to a shell
shell

:back
goto start

:failed
echo Booting failed, going back to start
goto start

:login_failed
echo ...not that easy. Try again.
goto login

:exit
exit

Compile iPXE with background picture support

0001-Enable-console-command-and-graphical-framebuffer.patch
Subject: [PATCH] Enable graphical framebuffer

---
 src/config/console.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/config/console.h b/src/config/console.h
index 9f770d09..bfc00d76 100644
--- a/src/config/console.h
+++ b/src/config/console.h
@@ -35,7 +35,7 @@ FILE_LICENCE ( GPL2_OR_LATER_OR_UBDL );
  */

 //#define      CONSOLE_SERIAL          /* Serial port console */
-//#define      CONSOLE_FRAMEBUFFER     /* Graphical framebuffer console */
+#define        CONSOLE_FRAMEBUFFER     /* Graphical framebuffer console */
 //#define      CONSOLE_SYSLOG          /* Syslog console */
 //#define      CONSOLE_SYSLOGS         /* Encrypted syslog console */
 //#define      CONSOLE_VMWARE          /* VMware logfile console */
--
2.43.0

credits: