Installing keyboot + an OS on bare metal

A step-by-step guide to take a blank bare-metal server to a keyboot-managed, ZFS-on-LUKS host running one of Debian, Gentoo, or Alpine. You pick one OS here; adding more later (multi-distro on one pool) is §6.

This is the vendor-neutral guide. For Hetzner-specific quirks (vKVM=QEMU, the rescue’s on-demand ZFS compile, BIOS-on-md) see hetzner-deploy.md. For the design, architecture.md; when it breaks, disaster-recovery.md.

What you’ll end up with

GRUB ─▶ keyboot (unlock env) ─▶ kexec ─▶ your OS (ZFS-on-LUKS root)

Every data disk LUKS-encrypted; one passphrase unlocks all of them.
The OS lives in a ZFS boot environment; keyboot picks + kexecs it.
Headless-reachable: the unlock env and the OS both come up networked over SSH.

Before you start — what you need

A build host (x86_64, lots of cores) to build the keyboot boot image (kernel + initramfs). This is NOT the target. See “Build the boot image” below.
A provisioning environment on the target: a keyboot rescue USB (usb-rescue.md), netboot, or the vendor’s rescue system.
SSH access to that provisioning env, and your SSH public key.
The target disks’ device paths (/dev/sda …) or serials.
A passphrase for the LUKS keyfile (you’ll type it at each boot, or enroll an automation slot later — operator-guide.md §3).
UEFI or BIOS? Both work; it only changes the GRUB-install details (§4).

⚠️ Installing destroys the target disks. Back up anything first.

Build the boot image (on the build host)

keyboot boots its own kernel + initramfs. Build them with all NIC drivers (so the unlock env comes up networked on real hardware) and your SSH key baked in:

bash kernel/build.sh                                   # -> kernel/out/vmlinuz
INCLUDE_MODULES=all AUTHORIZED_KEYS=you.pub \
    bash ci/build-image.sh                             # -> the initramfs

Copy kernel/out/vmlinuz and the built initramfs to the target’s provisioning env (you’ll pass them to the installer as --keyboot-kernel / --keyboot-initramfs).

1. Boot the target into a provisioning environment

Boot the keyboot rescue USB / netboot, or the vendor rescue, and get an SSH shell as root. You need: the target disks visible (lsblk), network up, and enough tooling to run the installer (next step provides it).

2. Bootstrap the installer

The installer (keyboot-install) is a musl binary. How you get it depends on the provisioning env’s libc:

musl live env (Alpine, keyboot rescue): one-liner bootstrap —
```
curl -fsSL https://packages.osterman.co/keyboot/install-os | sh
```
(fetches + minisign-verifies keyboot-install + the install-os toolkit + the be-tools). See installer-bootstrap.md.
glibc rescue (Debian/Hetzner): the musl binary won’t link there; build it natively + stage the toolkit with the helper —
```
KEYBOOT_SRC=/path/to/keyboot bash tools/hetzner-prep.sh
```
(also recompiles ZFS for the rescue kernel; see hetzner-deploy.md).

Each bootstrap installs keyboot-install and prints the exact install-os command to copy — including the KEYBOOT_INSTALL_OS_DIR (and, for the curl|sh path, KB_BE_TOOLS_DIR) environment variables the installer needs. Run install-os the way the bootstrap tells you; don’t drop those variables. If KB_BE_TOOLS_DIR is missing when you run install-os, the install still succeeds but the new BE silently ships without the be-tools (autosnap / be-upgrade / be-rollback / snap / update-check / keys) — the GL#43 footgun.

Tip — skip the variables entirely (curl|sh): let the bootstrap run the installer for you in the same process (where the variables are already live):
curl -fsSL https://packages.osterman.co/keyboot/install-os \
  | sh -s -- --run <distro> --disk /dev/sda --hostname h --confirm   # + the §3 flags

3. Install ONE operating system

Dry-run first (omit --confirm) to see the plan; add --confirm to execute. Pick one of the following. Common flags: --disk (repeat per pool member), --hostname, --authorized-keys you.pub (so the booted OS is SSH-reachable), --passphrase-from (prompt | file:PATH | env:VAR), and the boot image you built (--keyboot-kernel / --keyboot-initramfs). Disk count picks the default pool layout (1→single, 2→mirror, 3+→raidz1); override with --template or the --topology grammar (ADR 0010).

Debian (systemd, bookworm)

keyboot-install install-os debian \
    --disk /dev/sda --disk /dev/sdb \
    --hostname my-debian --authorized-keys you.pub \
    --passphrase-from prompt \
    --keyboot-kernel /root/keyboot-vmlinuz \
    --keyboot-initramfs /root/keyboot-initramfs.img \
    --confirm

Notes: debootstraps the base, builds ZFS via zfs-dkms in the chroot, wires the re-unlock as a boot=keyboot initramfs-tools script. Pool is created compatibility=openzfs-2.1-linux so Debian’s 2.1.x ZFS can import it.

Gentoo (openrc)

keyboot-install install-os gentoo \
    --disk /dev/sda --disk /dev/sdb \
    --hostname my-gentoo --authorized-keys you.pub \
    --passphrase-from prompt \
    --keyboot-kernel /root/keyboot-vmlinuz \
    --keyboot-initramfs /root/keyboot-initramfs.img \
    --confirm

Notes: extracts a stage3, emerges gentoo-kernel-bin + zfs-kmod + a dracut 90keyboot module from the binhost. --firmware curated keeps a small server-NIC firmware set; full ships everything; none ships nothing.

Alpine (openrc)

keyboot-install install-os alpine \
    --disk /dev/sda --disk /dev/sdb \
    --hostname my-alpine --authorized-keys you.pub \
    --passphrase-from prompt \
    --keyboot-kernel /root/keyboot-vmlinuz \
    --keyboot-initramfs /root/keyboot-initramfs.img \
    --confirm

Notes: the most CI-exercised path. Curated mkinitfs BE image bundling the keyboot binary + keyfile + a be-unlock init; comes up DHCP + sshd + your key.

Declarative alternative: put any of the above in a YAML profile and run keyboot-install install-os <distro> --profile profile.yaml --confirm (SPEC §13.7; see tools/keyboot-install-os/profile.example.yaml).

What the installer does, in order: partition (GPT + mdraid1 ESP + crypt) → create the keyfile + enroll each disk → unlock as sn-<serial> → zpool create → create the BE → bootstrap the distro → configure (network/ssh/hostname) → build the BE kernel + initramfs → install GRUB + the keyboot image → snapshot → teardown. --substrate-only stops after the pool+BE (no distro).

4. First boot

Reboot the target off the installer env onto its own disks.

UEFI: firmware reads BOOTX64.EFI from the mdraid1 ESP → GRUB → keyboot. Nothing extra needed.
BIOS: GRUB’s core.img is installed per-disk on each ESP member (the installer does this; the array is stopped first so the member mount doesn’t EBUSY). Set the BIOS to boot the disk you installed to.

The box comes up in the keyboot unlock env, networked. SSH in (the key you baked into the boot image) and you’ll get the passphrase prompt — or, with --passphrase-from prompt, keyboot waits on both the console and SSH:

ssh root@<target>          # to the keyboot env (dropbear)
# enter the LUKS passphrase -> keyfile opens -> disks open as sn-<serial>
#   -> pool imports -> kexec -> your OS boots

After kexec the OS comes up (also networked, your key). Verify:

ssh root@<target>          # now the booted OS
zpool status               # ONLINE
cat /etc/os-release        # the distro you installed

5. Verify it’s healthy

zpool status -x            # 'all pools are healthy'
zfs list -o name,used,mountpoint -r rpool
systemctl is-active ssh    # (Debian)  / rc-status (openrc)

Add a recovery passphrase now so a forgotten daily passphrase isn’t fatal:

keyboot-install keyfile add-recovery <keyfile>     # slot 1

(Back up the keyfile container off-box — disaster-recovery.md §1.)

6. (Optional) Add more operating systems later

keyboot’s own env is the only provisioning environment you need after this. To add a second/third distro to the same pool (each its own boot environment):

From the running OS or the keyboot provision shell, with the pool imported:
```
keyboot-install install-os <distro> --add-be --pool rpool \
    --authorized-keys you.pub --confirm
```
No repartition, no new keyfile/pool, no GRUB rewrite — keyboot auto-discovers the new BE. Reboot and pick it from the keyboot menu.
Provision mode (boot keyboot itself into an installer shell): arm it with keyboot-install keyboot provision --confirm, reboot — keyboot unlocks + imports read-write and drops you at an install-os --add-be prompt, then reverts to normal boot. See operator-guide.md §4.

If something goes wrong

keyboot brings networking + SSH up before unlock, so most failures leave a reachable recovery shell. See disaster-recovery.md for the symptom→fix table (won’t boot, unlock fails, wrong entry, feature drift, failed disk) and hardware-test-runbook.md for the console-attended recovery model.

Keyboard shortcuts

keyboot