Host Your Own Services With FreeBSD: Advanced Installation - UEFI Gmirror GPT UFS

Hosting services is much more fun when we can be sure that malfunction of a single disk won't shut down services and cause data loss. FreeBSD Handbook has a chapter about Creating a Mirror with Two New Disks, but it suggests some outdated practices such as legacy BIOS boot and MBR partitioning scheme as opposed to UEFI boot and GPT partitoning scheme described in this article. Furthermore, contrary to examples in FreeBSD Handbook which use device nodes such as ada directly, this article describes using geom labels for gmirror creation. Finally, we set our fstab using custom gpt labels as oposed to automatically created partitions under /dev/mirror/ described in FreeBSD Handbook.

Assuming we have three disks, da0 (FreeBSD USB install media), and ada0 and ada1 (NVMes):

sysctl kern.disks
kern.disks: da0 nda1 nda0

We label disks:

glabel label OS0 /dev/nda0
glabel label OS1 /dev/nda1

create gmirror from geom labels:

gmirror load
gmirror label OS /dev/label/OS0 /dev/label/OS1

create gpt scheme and gpt partitions with their own gpt labels on top of gmirror:

gpart create -s gpt /dev/mirror/OS
gpart add -t efi -a1m -s200m -l OS-efi /dev/mirror/OS
gpart add -t freebsd-ufs -a1m -s1g -l OS-root /dev/mirror/OS
gpart add -t freebsd-ufs -a1m -s4g -l OS-var /dev/mirror/OS
gpart add -t freebsd-ufs -a1m -s2g -l OS-usr /dev/mirror/OS
gpart add -t freebsd-ufs -a1m -s8g -l OS-usr-local /dev/mirror/OS
gpart add -t freebsd-ufs -a1m -s10g -l OS-home /dev/mirror/OS
gpart add -t freebsd-swap -a1m -s8g -l OS-swap /dev/mirror/OS

before we continue let's check we partitioned correctly:

gpart show -l /dev/mirror/OS
=>      40  83886000  mirror/OS  GPT  (40G)
        40      2008             - free -  (1.0M)
      2048    409600          1  OS-efi  (200M)
    411648   2097152          2  OS-root  (1.0G)
   2508800   8388608          3  OS-var  (4.0G)
  10897408   4194304          4  OS-usr  (2.0G)
  15091712  16777216          5  OS-usr-local  (8.0G)
  31868928  20971520          6  OS-home  (10G)
  52840448  16777216          7  OS-swap  (8.0G)
  69617664  14268376             - free -  (6.8G)

Let's handle EFI partition first:

newfs_msdos -F32 -c1 /dev/gpt/OS-efi 
mount_msdosfs /dev/gpt/OS-efi /mnt/
mkdir -p /mnt/EFI/BOOT
cp /boot/loader.efi /mnt/EFI/BOOT/BOOTX64.efi
umount /mnt/

Format all the UFS partitions:

newfs -U /dev/gpt/OS-root
newfs -U /dev/gpt/OS-var
newfs -U /dev/gpt/OS-usr
newfs -U /dev/gpt/OS-usr-local
newfs -U /dev/gpt/OS-home

mount our to-be root partition:

mount /dev/gpt/OS-root /mnt/

create dirs for other mountpoints:

mkdir /mnt/var /mnt/usr /mnt/home

mount them:

mount /dev/gpt/OS-var /mnt/var/
mount /dev/gpt/OS-usr /mnt/usr/
mount /dev/gpt/OS-home /mnt/home/

this one needed /usr to be mounted beforehand, thus comes last:

mkdir /mnt/usr/local
mount /dev/gpt/OS-usr-local /mnt/usr/local/

We'll use vi to create /tmp/bsdinstall_etc/fstab:

# jailhost.mimar.rs:/etc/fstab
/dev/gpt/OS-swap      none       swap   sw 0 0
/dev/gpt/OS-root      /          ufs    rw 1 1
/dev/gpt/OS-home      /home      ufs    rw 2 2
/dev/gpt/OS-usr       /usr       ufs    rw 2 2
/dev/gpt/OS-usr-local /usr/local ufs    rw 2 2
/dev/gpt/OS-var       /var       ufs    rw 2 2
proc                  /proc      procfs rw 0 0
tmpfs                 /tmp       tmpfs  rw 0 0

After this we type exit which returns us to the installer where we continue as in standard install. When we get to "Final Configuration" menu and confirm "Exit", choose "Yes" when asked to "... make any final manual modifications" and use vi to create /boot/loader.conf:

# jailhost.mimar.rs:/boot/loader.conf
geom_mirror_load="YES"

Type exit, choose reboot, remove USB install media.

After reboot we can verify state of mirror:

gmirror status
     Name    Status  Components
mirror/OS  COMPLETE  label/OS0 (ACTIVE)
                     label/OS1 (ACTIVE)

Also our mountpints made of gpt labels:

df -h
Filesystem               Size    Used   Avail Capacity  Mounted on
/dev/gpt/OS-root         992M    238M    674M    26%    /
devfs                    1.0K      0B    1.0K     0%    /dev
/dev/gpt/OS-home         9.7G     40K    8.9G     0%    /home
/dev/gpt/OS-usr          1.9G    778M    1.0G    43%    /usr
/dev/gpt/OS-usr-local    7.7G    8.0K    7.1G     0%    /usr/local
/dev/gpt/OS-var          3.9G    5.7M    3.6G     0%    /var
procfs                   8.0K      0B    8.0K     0%    /proc
tmpfs                     16G    4.0K     16G     0%    /tmp

Here's transcript of terminal session: