Guide: Using banlieue-imagebuilder¶

This guide installs banlieue-imagebuilder (ADR-0010) — the provider-agnostic controller that turns a VMImage sourced from an OCI image (e.g. a nightly Kairos build) into a build artifact via kairos-operator: a raw cloud image for libvirt sources, or a bootable ISO with a baked-in default cloud-config for vSphere sources (ADR-0020). Everything uses the released image ghcr.io/firestoned/banlieue:v0.4.0.

What this pipeline delivers today

banlieue-imagebuilder drives a VMImage's build all the way to status.buildArtifact.phase: Ready, and both in-tree providers complete the per-zone import from there:

vSphere (ADR-0020, ADR-0021): one image-import Job per failure domain uploads the ISO to the zone's datastore, creates an EFI VM (pvscsi disk, vmxnet3 NIC), attaches the ISO as a CD-ROM, powers it on, waits for the cloud-config's unattended Kairos install to generalize the disk and power the VM off itself, removes the CD-ROM, and marks it as a template — clones need no ISO at boot. The Content Library path (Provider.spec.useContentLibrary: true, default off) is the one remaining stub — it reports ContentLibraryNotImplemented.
libvirt (ADR-0011): one import Job per declared storage pool streams the raw cloud image into a volume over mTLS.

Prerequisites¶

The core controller installed (CRDs + controller running in banlieue-system).
kairos-operator installed, smoke test passed.

The repo checked out at the release tag (for the imagebuilder manifests):

git clone --branch v0.4.0 --depth 1 https://github.com/firestoned/banlieue
cd banlieue

1. Install banlieue-imagebuilder¶

banlieue-imagebuilder is the same banlieue image, run with the imagebuilder subcommand. Its RBAC is deliberately narrow: it never reads backend credentials, and only touches vmimages, vmimages/status, kairos-operator's osartifacts, and the cloudConfigs[] Secrets/ConfigMaps you point it at — no vCenter or libvirt access of any kind passes through it. Since ADR-0037 it does read Secret content (to merge layered cloud-configs), but only inside the build namespace, and only through the namespaced Role described below — never through a ClusterRole.

kubectl apply -f deploy/imagebuilder/namespace.yaml
kubectl apply -R -f deploy/imagebuilder/rbac/
kubectl apply -f deploy/imagebuilder/configmap.yaml
kubectl apply -f deploy/imagebuilder/deployment.yaml
kubectl apply -f deploy/imagebuilder/service.yaml

kubectl -n banlieue-system rollout status deploy/banlieue-imagebuilder --timeout=120s

BANLIEUE_BUILD_NAMESPACE (default banlieue-imagebuild, set in deploy/imagebuilder/configmap.yaml) is where OSArtifact CRs — and the artifacts PVCs kairos-operator creates for them — live. A provider's per-zone import Jobs run in this same namespace to reach the shared PVC (ADR-0010 / ADR-0016); leave it at the default unless you have a specific reason to change it, and pass the same value to every provider via its own --build-namespace flag so the Jobs land where the PVC is.

Overriding the build namespace also moves an RBAC grant

deploy/imagebuilder/rbac/role.yaml is a namespaced Role + RoleBinding pinned to banlieue-imagebuild — it is what lets the reconciler read your cloudConfigs[] Secrets and write the merged one (ADR-0041). The grant is namespaced on purpose: a ClusterRole rule for Secrets has no namespace scope at all and would reach every Secret in the cluster, including every Provider's hypervisor credentials.

If you change BANLIEUE_BUILD_NAMESPACE, re-create that Role and RoleBinding in the new namespace, keeping the subject pointed at ServiceAccount/banlieue-imagebuilder in banlieue-system. Otherwise the first VMImage carrying cloudConfigs fails with a 403 on the Secret read. banlieue bootstrap imagebuilder emits this pair for the default namespace only.

2. Create a `VMImage` with a `Url` source¶

Unlike a Template source (which names something that must already exist in vCenter), a Url source names an OCI image banlieue-imagebuilder builds for you. For a vSphere source the build produces a bootable ISO (auroraboot build-iso); spec.cloudConfigs[] bakes a (layered) cloud-config into it, and spec.template controls how the per-zone import turns the ISO into a vCenter template (ADR-0020):

Cloud-config contract (ADR-0021)

The merged cloudConfigs[] result must set install.poweroff: true / install.reboot: false, at least one admin-group user, and an after-install-chroot stage that wipes per-machine identity before the disk is ever booted:

install:
  auto: true
  device: /dev/sda
  reboot: false
  poweroff: true
users:
  - name: kairos-admin
    groups: ["admin"]
    passwd: <hashed-or-plaintext-per-your-policy>
stages:
  after-install-chroot:
    - name: "banlieue: strip per-machine identity before templating"
      commands:
        - truncate -s 0 /etc/machine-id
        - rm -f /etc/ssh/ssh_host_*

This is what lets the import Job know the install finished (the VM powers itself off) and produces a template every clone's first boot generalizes for itself — no post-install reboot, no per-clone install. Without it, the import Job's wait times out after spec.template.installTimeoutSeconds (default 1800s) because the VM never powers off.

The users entry is not optional in Kairos v2.29.4+ (the v3.3.x line): with none, the install stage halts immediately with No users found in any stage that are part of the 'admin' group and — same symptom as a missing poweroff/reboot pair — the VM never powers off, so the import Job's wait times out. Set install.nousers: true instead if a userless system is genuinely intended.

Not building a Kairos image, or managing install yourself? Set spec.template.installMode: manual to skip this contract entirely — the import Job reverts to ADR-0020's original behavior: create the VM, attach the ISO, MarkAsTemplate immediately, no power-on. banlieue never reads or edits your cloudConfigs[] Secrets/ConfigMaps either way (installMode defaults to immediate); the contract above is documentation for what a Kairos-managed install needs, never something banlieue auto-injects.

Building a tpmEnabled: true VMClass's image? Use spec.template.installMode: deferred instead of manual — mechanically identical (no power-on at build time), but it signals intent: the install runs once per clone, at that clone's own first boot, with that clone's own vTPM already attached (ADR-0039). Kairos's install.encrypted_partitions only ever seals against a TPM present during install — it cannot encrypt an already-installed disk — so a deferred-mode image's cloud-config needs the opposite contract from the one above: install.reboot: true / install.poweroff: false (the VM must keep running as the production workload after install, not power itself off for templating) and no after-install-chroot identity-wipe stage (each clone installs fresh and gets its own machine-id/SSH host keys naturally). See examples/15-vmimage-kairos-deferred-install-tpm.yaml and ADR-0040. Note that VSphereMachine.status.initialization.provisioned still flips true the instant the clone powers on — for a deferred-mode VM that now means "the 8-12 minute install just started," not "the VM is ready" (a known, documented gap, not solved by ADR-0040).

Overlaying extra files onto the ISO (ADR-0022)

spec.isoOverlay lets you overlay additional files — e.g. a hand-verified /boot/grub2/grub.cfg — onto the built ISO, via kairos-operator's own overlayISOVolume mechanism (auroraboot build-iso --overlay-iso). As with cloudConfigs[], only the Secret's name and the key/path list you declare are ever read — banlieue-imagebuilder never reads the Secret's content:

isoOverlay:
  secretRef:
    name: kairos-iso-overlay
  files:
    - key: grub.cfg
      path: boot/grub2/grub.cfg

Under the hood this adds a small busybox init container (the "materializer") to the OSArtifact, to work around an auroraboot overlay bug (ADR-0022 Decision #3/#4, kairos-io/kairos#4324). On an air-gapped cluster that image must come from your own mirror — set on banlieue-imagebuilder itself (cluster-wide, not per-VMImage):

banlieue imagebuilder \
  --build-importer-image your-registry.example.com/mirror/busybox:1.36 \
  --build-importer-image-pull-secret your-mirror-pull-secret

or via env var (the repeatable pull-secret flag is CLI-only, same as --build-node-selector): BANLIEUE_BUILD_IMPORTER_IMAGE=your-registry .example.com/mirror/busybox:1.36. The pull secret must be a kubernetes.io/dockerconfigjson Secret in the imagebuild namespace — it becomes the OSArtifact's pod-wide imagePullSecrets, covering the main build image too if it comes from the same mirror.

Trusted Boot (UKI) artifacts (ADR-0051)

If your importFrom image was built with Kairos Trusted Boot (TRUSTED_BOOT=true), it's a Unified Kernel Image — no discrete /boot/vmlinuz//boot/initrd, so the default auroraboot build-iso path fails with No initrd file found. Set spec.trustedBoot to request spec.artifacts.uki instead (auroraboot build-uki), which understands the UKI shape:

trustedBoot:
  secretRef:
    name: kairos-trusted-boot-keys

The referenced Secret must hold six files auroraboot build-uki requires — PK.auth, KEK.auth, db.auth, db.key, db.pem, tpm2-pcr-private.pem — generated out-of-band, once, via:

auroraboot genkey --expiration-in-days 365 -o /keys "your-org"
kubectl create secret generic kairos-trusted-boot-keys \
  --from-file=/keys/PK.auth --from-file=/keys/KEK.auth \
  --from-file=/keys/db.auth --from-file=/keys/db.key \
  --from-file=/keys/db.pem --from-file=/keys/tpm2-pcr-private.pem \
  -n banlieue-imagebuild

cloudConfigs[] keeps working transparently alongside trustedBoot — you don't need to change how you declare cloud-config. Under the hood, banlieue-imagebuilder stops routing it through cloudConfigRef (a kairos-operator bug means auroraboot build-uki rejects the --cloud-config flag it would otherwise generate) and instead bakes the merged cloud-config into the ISO root via the same isoOverlay mechanism (ADR-0051 Decision #4) — the exact file kairos-agent's installer already looks for.

As with isoOverlay/cloudConfigs, only the Secret's name is ever read by banlieue-imagebuilder — never its content. This is a throwaway, self-signed key set (not an enterprise PKI/HSM): Kairos auto-enrolls it as the VM's UEFI PK/KEK/db on first boot when the firmware starts in UEFI Setup Mode, or you enroll it once manually — you are both the CA and the enroller. Pair this with spec.template.firmware: efi-secure and a tpmEnabled: true VMClass (ADR-0039/0040) for TPM-sealed kcrypt disk encryption end to end — see examples/16-vmimage-kairos-trusted-boot-uki.yaml. Live-verified: vSphere UEFI Secure Boot key enrollment works via the uefi.secureBoot.{pk,kek,db}Default.file0 VMX extraConfig mechanism (Broadcom KB 377306) — upload the three DER-encoded certs (auroraboot genkey's output already has them) to the VM's own datastore folder and set the three extraConfig keys plus uefi.secureBoot.dbDefault.append=FALSE before first boot. Only takes effect on a VM whose .nvram has no existing Secure Boot config, so this must happen before the VM's first power-on.

Experimental — first boot is extremely slow on vSphere

Live-testing Trusted Boot/UKI images against vSphere (govc-driven clone → vTPM attach → Secure Boot key pre-seed → cloud-config → power on, bypassing banlieue entirely to isolate the failure) found the initial UEFI → OS handoff to be extremely slow — multi-minute stalls at each Secure Boot stage transition (shim → systemd-boot → UKI). ESXi's own vmware.log shows complete silence at the hypervisor level during these stalls (no vTPM command traffic, no disk I/O), which rules out slow vTPM emulation as the cause and points to something inside guest space (kernel/systemd/dracut UKI-stub behavior specific to vSphere's firmware) that has not yet been root-caused. Observed on a Debian-based Trusted Boot image; a follow-up attempt with a Hadron-based image also failed to boot cleanly and was not further diagnosed. Treat VMImage.spec.trustedBoot on the vSphere provider as experimental until this is root-caused — see ADR-0051's follow-ups.

vmimage-kairos.yaml

apiVersion: banlieue.io/v1alpha1
kind: VMImage
metadata:
  name: kairos-ubuntu-2404
spec:
  osFamily: linux
  osDistribution: ubuntu
  osVersion: "24.04"
  architecture: amd64
  guestAgent: cloud-init
  sources:
    - providerClass: vsphere
      kind: Url
      ref: unused-for-url-sources # required by the schema; ignored for kind: Url
      # Digest-pinned: the banlieue-vmimage-import-source admission policy
      # (security review 2026-07-31) rejects mutable tags when installed.
      importFrom: quay.io/kairos/ubuntu:24.04-standard-amd64-generic-v3.7.2-k0s-v1.34.3-k0s.0@sha256:e4860078c024269e81ce561ce91cf9639a4e75c23ea4cd32d3405005087192a7
  # Optional, ordered, layered cloud-config sources baked into the built ISO
  # (ADR-0020/0037). Names Secrets/ConfigMaps in the imagebuild namespace;
  # merged and passed to the OSArtifact as cloudConfigRef
  # (auroraboot build-iso --cloud-config).
  cloudConfigs:
    - secretRef:
        name: kairos-base-cloud-config
        key: cloud-config.yaml
  # Optional extra files overlaid onto the built ISO (ADR-0022), e.g. a
  # hand-verified grub.cfg. Only the Secret's name + declared keys are read.
  isoOverlay:
    secretRef:
      name: kairos-iso-overlay
    files:
      - key: grub.cfg
        path: boot/grub2/grub.cfg
  # How the backend template is built from this Url source (ADR-0020).
  template:
    rootFolder: templates/kairos  # root vCenter folder, created if missing — the
                               # template lands at <rootFolder>/<failure-domain-name>,
                               # never at <rootFolder> itself (every zone gets its own
                               # subfolder, since two zones commonly share a
                               # datacenter and vSphere folders are scoped
                               # per-datacenter, not per-cluster)
    network:                   # template NIC(s); omit for zone default (ADR-0031)
      - network: vmnet-prod
        adapter: vmxnet3        # vmxnet3 | vmxnet2 | e1000 | e1000e
    disk:
      size: 100                # GiB; default 100
      type: thin               # thin | thick | eagerZeroed
      controller: pvscsi       # pvscsi | lsiLogic | lsiLogicSas | busLogic
    forceUpload: false         # delete + re-upload the ISO even if present
    forceCreate: false         # destroy + recreate the template even if present
    installTimeoutSeconds: 1800   # bound on the unattended-install wait (ADR-0021)
    installMode: immediate        # immediate (default) | deferred | manual — see ADR-0040

(Also available as examples/09-vmimage-kairos-url-source.yaml.)

kubectl apply -f vmimage-kairos.yaml

All of cloudConfigs and template are optional: omit them for a vanilla ISO and a thin 100 GiB pvscsi template, one per zone, each in its own subfolder named after the zone directly under the datacenter's VM-folder root. The per-zone import is idempotent — it skips an already-uploaded ISO and an existing template; the force knobs replace a bad one without manual vCenter cleanup.

3. Watch the artifact build¶

kubectl get vmimage kairos-ubuntu-2404 -o yaml | yq '.status.buildArtifact'

phase progresses Pending -> Building -> Ready (kairos-operator's own Exporting phase is folded into Building; Error maps to Failed). You can watch the underlying OSArtifact directly too:

kubectl -n banlieue-imagebuild get osartifact kairos-ubuntu-2404-build -w

Once buildArtifact.phase is Ready, kind, pvcRef, and file are populated — that's the handoff a provider's per-zone import reads:

buildArtifact:
  kind: iso                 # iso for vsphere sources, cloudImage for libvirt
  phase: Ready
  osArtifactRef: kairos-ubuntu-2404-build
  pvcRef:
    name: kairos-ubuntu-2404-build-artifacts
  file: kairos-ubuntu-2404-build.iso

4. Check per-provider / per-zone status¶

If a vsphere Provider is registered (see the vSphere Provider guide), banlieue-provider-vsphere picks up the Url source once the ISO is ready and starts one import Job per failure domain:

kubectl get vmimage kairos-ubuntu-2404 -o yaml | yq '.status.perProvider'

perProvider:
  - providerName: prod-vsphere
    providerNamespace: banlieue-system
    ready: true
    reason: Reconciled
    zones:
      - name: prod-vsphere-dc1-az1
        ready: true
        resolvedRef: "[ds-cluster-a] templates/kairos/kairos-ubuntu-2404"
      - name: prod-vsphere-dc1-az2
        ready: true
        resolvedRef: "[ds-cluster-b] templates/kairos/kairos-ubuntu-2404"

While a Job runs, its zone reports reason: Importing; a failed Job reports ImportFailed (see the troubleshooting table below). The import Jobs themselves live in the build namespace:

kubectl -n banlieue-imagebuild get jobs -l banlieue.io/vmimage=kairos-ubuntu-2404

Cloud Hypervisor hosts: pushed to a registry¶

A Cloud Hypervisor provider runs on its KVM host, outside the cluster, and cannot mount the artifacts PVC. For a Url source with providerClass: cloud-hypervisor, the imagebuilder pushes the finished artifact to an OCI registry instead (ADR-0064), and each host pulls it by digest:

# banlieue-imagebuilder-config (or the matching flags)
BANLIEUE_REGISTRY_REPOSITORY: "registry.internal:5000/banlieue/disks"   # --registry-repository
BANLIEUE_REGISTRY_CREDENTIALS_SECRET: "banlieue-registry-push"          # --registry-credentials-secret

The Secret is kubernetes.io/basic-auth (username, password) in the build namespace; leave it unset for an anonymous registry. --registry-plain-http is for a registry on a private network or a test.

Two settings matter on a cluster behind a private CA or an image mirror, and both fail only after the build has finished and the push starts:

Symptom on the push Job's pod	Setting
`invalid peer certificate: UnknownIssuer` in its log	`BANLIEUE_REGISTRY_CA_BUNDLE_CONFIG_MAP` or `BANLIEUE_REGISTRY_CA_BUNDLE_SECRET` (and `BANLIEUE_REGISTRY_CA_BUNDLE_KEY`, default `ca.crt`): the CA that signed the registry's certificate (ADR-0110)
`ImagePullBackOff` on its pod	`BANLIEUE_PUSH_IMAGE` (`--push-image`): the Job's image defaults to the public banlieue image; name the mirrored one, digest-pinned like the imagebuilder's own

The CA bundle is a PEM file, one or several certificates, in a ConfigMap or a Secret in the build namespace. The Job mounts only the named key, read-only, and trusts its certificates in addition to the public roots; a bundle with no certificate fails the push rather than falling back to the public roots, and there is no option to skip verification. Restart the imagebuilder after changing either setting, and delete a failed push Job (kubectl -n banlieue-imagebuild delete job <vmimage>-build-push) so it is recreated with the new settings: a Job's pod template cannot be edited. The host pulls with its own trust store, so a host that already trusts the CA needs nothing here.

The push Job runs the banlieue image itself (--push-image) with no ServiceAccount token, and each build is tagged with its OSArtifact uid:

kubectl -n banlieue-imagebuild get jobs -l app.kubernetes.io/component=registry-push
kubectl get vmimage kairos-ubuntu-2404-ch -o jsonpath='{.status.buildArtifact.ociArtifact}'
# {"phase":"Ready","reference":"registry.internal:5000/banlieue/disks@sha256:…"}

With no repository configured, a cloud-hypervisor Url source reports ociArtifact.phase: Failed saying so; installs without one need no registry. The host side is in the Cloud Hypervisor host guide.

First-boot sealing layer (ADR-0088)¶

When a VMImage declares spec.firstBootSealing (honoured by default; a controller run with --enable-first-boot-sealing=false refuses its VMs), the builder appends its own cloud-config layer after every cloudConfigs entry, so no operator layer can override it. The merged Secret is created even when cloudConfigs is empty.

Artifact	What the layer adds
`iso` (vSphere)	An `after-install-chroot` step that writes `/oem/91_banlieue_first_boot_sealing.yaml` (`kcrypt.encrypt_on_boot: true` and `install.encrypted_partitions`) into the installed disk's `COS_OEM`. The install configuration itself never asks for encryption: the template installs once, with no vTPM.
`cloudImage` (libvirt)	The same two keys at the top level, since no installer runs and the image's cloud-config is what lands in `COS_OEM`.
both	A `boot` step, guarded to the installed system, that writes the listed partitions found as LUKS containers to `guestinfo.banlieue.sealed` and `/run/banlieue/sealed`. The provider reads it for `GuestReady`.

status.buildArtifact.firstBootSealingLayer: true records that the layer is in the build, and status.firstBootSealing becomes Built.

The build is refused (phase: Failed) rather than produced when:

an operator layer of an immediate image sets install.encrypted_partitions (FirstBootSealingConflict): drop it from the template cloud-config;
a listed partition is not a plain label, or is COS_OEM, COS_STATE, COS_RECOVERY or COS_GRUB (FirstBootSealingInvalid).

The builder cannot read the source image's Kairos version, so it cannot refuse an image too old to seal. Such a clone boots unsealed and is caught by GuestReady=False (FirstBootSealingUnverified or FirstBootSealingTimeout), never reported healthy. Changing firstBootSealing on an existing VMImage does not rebuild an artifact that already exists; recreate the VMImage.

Rebuilding after a worker is replaced (ADR-0115)¶

On local-path storage the artifacts PVC is a directory on one worker's disk. When that worker is rebuilt it returns under the same name with an empty disk, while the PV, the PVC and the OSArtifact still read Bound and Ready, and every import Job fails with cannot read /artifacts/<image>-build.iso.

The imagebuilder now notices. It reads the PV's node (nodeAffinity on kubernetes.io/hostname) and compares that node's creationTimestamp with the PV's. A node created after the PV, or no longer present, means the volume's data is gone, and the OSArtifact is deleted and rebuilt (about eight minutes per image). The VMImage status message reads replaced an OSArtifact whose volume was lost with its node.

It needs the imagebuilder ClusterRole from deploy/imagebuilder/rbac/clusterrole.yaml (get on persistentvolumes, list on nodes). Without it the check logs a warning and does nothing; nothing is deleted.
It does not apply to storage without node affinity, and it cannot see a file deleted from a node that was not replaced.
A node whose Node object is deleted and re-registered with its disk intact is also rebuilt; the cost is one rebuild.

Pre-laying a raw image (ADR-0113)¶

You build the pre-lay image

ADR-0113 is accepted (2026-11-02: a Hadron Url image pre-laid in a cluster, its clone sealed and GuestReady in 30 s), but banlieue does not publish the pre-lay image yet: build it from Dockerfile.prelay (docker build -f Dockerfile.prelay -t <registry>/banlieue-prelay:<tag> .) and push it where the build namespace can pull it.

A raw Kairos image makes every clone run a recovery boot, a reset that lays out COS_STATE and COS_PERSISTENT, and a reboot before its real first boot. spec.prelay does that once, at build time: a sealed clone then reaches GuestReady in about 40 s instead of about 106 s on Cloud Hypervisor, and a two-member pool warms in 65 s instead of 125 s on libvirt.

spec:
  sources:
    - providerClass: cloud-hypervisor
      kind: Url
      ref: unused-for-url-sources
      importFrom: ghcr.io/example/kairos-debian:13@sha256:<digest>
  firstBootSealing: {}
  prelay:
    diskSizeGiB: 20   # the clones' OS disk size; give their class exactly this

What happens, in order:

kairos-operator builds the raw cloudImage as usual.
The imagebuilder starts a pre-lay Job beside the artifacts PVC, running --prelay-image. buildArtifact stays Building, reason Prelaying, so no import Job or registry push starts on the raw file.
The Job checks the built file against the source's checksum (when set), boots it once under QEMU with no network and no TPM, writes <osartifact>-prelaid.raw beside it and reports that file's digest.
buildArtifact becomes Ready with file the pre-laid image, checksum its digest and prelaidDiskSizeGiB the size; imports and the registry push carry on from there and verify against that digest.

The Job runs the pre-lay image banlieue publishes, ghcr.io/firestoned/banlieue-prelay (linux/amd64, Cosign-signed, built from Dockerfile.prelay; ADR-0113's Amendment: a published pre-lay image). Every published image passes its own Grype gate first: no high or critical finding that Debian has already fixed. Findings with no Debian fix ship, and are listed in the repository's Code Scanning (category grype-release-Prelay for a release); the image's SBOM is a release asset, so you can scan it against your own policy. --prelay-image (BANLIEUE_PRELAY_IMAGE) defaults to it at the imagebuilder's own version, and banlieue bootstrap imagebuilder writes it explicitly. To use it:

make sure the build nodes have /dev/kvm, and select them with --build-node-selector: the Job runs privileged in the build namespace, like the kairos-operator builds already do;
pin it by digest: banlieue bootstrap imagebuilder --prelay-image-digest sha256:<digest> (the digest is not --image-digest's: it is a different image). Look it up with crane digest ghcr.io/firestoned/banlieue-prelay:<version> and check its signature with cosign verify as for the banlieue image;
on a cluster that mirrors images, mirror it too: under --registry, bootstrap points the Job at <registry>/banlieue-prelay:<version>.

--prelay-image "" turns pre-laying off.

`buildArtifact`	Means
`Building`, `Prelaying`	the pre-lay Job is running
`Failed`, `PrelayFailed`	the Job failed or reported something other than a digest; its pod log, and the console it kept beside the output, say why
`Failed`, `PrelayNeedsCloudImage`	the source builds an `iso` (vSphere); only a raw image can be pre-laid
`Failed`, `PrelayImageUnset`	`--prelay-image` is set to an empty value (pre-laying off)

To retry a failed pre-lay, delete the Job (kubectl -n banlieue-imagebuild delete job <osartifact>-prelay; it allows no retries of its own) and touch the VMImage (kubectl annotate vmimage <image> banlieue.io/retry="$(date +%s)" --overwrite): a Failed artifact is otherwise only looked at again on the long requeue.

A pre-laid image is laid out for one disk size, and COS_PERSISTENT does not grow on a larger disk. Stages bound to the reset (after-reset and friends) no longer run on clones. See the Cloud Hypervisor guide's "Pre-laid images" section for the details and the measurements.

Probing only the datasource your backend uses¶

Every Kairos boot that has not fetched user-data yet runs a datasource probe (/system/oem/00_datasource.yaml) with 13 providers: the NoCloud seed (cdrom), config drives, and a dozen cloud metadata services. The probe checks the seed first and stops if it finds one. Otherwise it runs every other provider in parallel, each with a 2 s timeout, and waits for all of them, so a boot that finds no seed spends about 2 s on services that cannot exist on a hypervisor banlieue manages. Measured on a sealed, pre-laid Debian clone with no seed on Cloud Hypervisor (roadmap 18, 2026-11-09): narrowing the list cut immucore's initramfs-hook step from 3.3 s to about 1 s and the member's time to GuestReady from 25 s to 23 to 24 s.

Who pays it depends on how banlieue delivers user-data:

Backend	How banlieue delivers user-data	Pays the 2 s	Keep
libvirt	a NoCloud seed CD-ROM, always attached (ADR-0054)	no	`cdrom`
Cloud Hypervisor, Proxmox	a NoCloud seed CD-ROM, attached only with user-data or a static NIC (ADR-0054, ADR-0114)	only machines with neither	`cdrom`
vSphere	`guestinfo.userdata` (ADR-0024), read by the `vmware` provider, which runs in the parallel group	every boot: the probe waits for the other providers to time out even after `vmware` answers	`vmware` (not yet verified against a real vCenter)

The file is in the image's read-only system partition, so narrow it where the image is built, after kairos-init has run. Both Debian images and Hadron (whose BusyBox provides sed) take the same line:

RUN --mount=type=bind,from=quay.io/kairos/kairos-init:v4.4.0,src=/kairos-init,dst=/kairos-init \
    /kairos-init --version v0.1.0
# Probe only the NoCloud seed. Use ["vmware"] for an image that boots on vSphere.
RUN f=/system/oem/00_datasource.yaml; test -f "$f" && \
    sed -i 's/providers: \[.*\]/providers: ["cdrom"]/' "$f" && \
    grep -q 'providers: \["cdrom"\]' "$f"

The grep fails the build if a future kairos-init changes the file's shape, rather than shipping an image with the probe silently untouched.

Narrowing changes nothing for a machine whose seed is found: the probe stops at the seed either way.

Narrow it; do not remove it. Creating /oem/95_userdata would skip the probe entirely, but it would also skip the seed, and per-VM user-data (including ADR-0114's static addressing on Kairos guests) would stop reaching the guest. An image built for one backend only probes that backend's datasource; build one per backend if you serve both NoCloud and vSphere from the same base.

Integrity and lifecycle (security review 2026-07-31)¶

Two guarantees hold over everything above:

The build is bound to the VMImage. The OSArtifact is owned by its VMImage (cluster-scoped owner of a namespaced dependent — deleting the image garbage-collects the build). banlieue-imagebuilder only mirrors a Ready from an OSArtifact that carries the current VMImage's UID and requests the current importFrom; anything else — a stale object from before a spec change, or a foreign pre-created one — is deleted and rebuilt. kairos' status has no observedGeneration or digest to bind a Ready to the spec, so object identity is the anchor.
The artifact can be verified end to end. Set checksum: <alg>:<hex> (sha256 or sha512) on the Url source. It is copied to status.buildArtifact.checksum, and provider import Jobs hash the built artifact before any byte reaches a backend — both the libvirt and the vSphere import Jobs fail closed on mismatch or an unsupported algorithm, so a substituted or corrupted artifact never lands in a storage pool or datastore.

banlieue-imagebuild pod-create is node-root-equivalent (SEC-009)

The build namespace enforces PSA privileged because kairos' build pods need loop devices. That means anyone granted pod-create there can mount the host filesystem — treat every RoleBinding in banlieue-imagebuild as a node-root grant. Nothing but kairos' build pods, the providers' import Jobs, the registry push Jobs, and the artifacts PVC should ever run there. The import Jobs mitigate this by running under a dedicated read-only ServiceAccount, never the provider controller's own identity (ADR-0016 §4).

Troubleshooting¶

VMImage.status.buildArtifact not appearing at all:

Confirm banlieue-imagebuilder is running: kubectl -n banlieue-system logs deploy/banlieue-imagebuilder
Confirm the VMImage actually has a Url-kind source — banlieue-imagebuilder ignores Template/BackingFile-only images.

buildArtifact.phase stuck at Pending or Building:

Check the OSArtifact directly: kubectl -n banlieue-imagebuild describe osartifact <vmimage-name>-build
Check kairos-operator's own logs — a bad importFrom reference (typo, private registry needing imageCredentialsSecretRef, which banlieue-imagebuilder does not set) shows up there, not in banlieue-imagebuilder's.

buildArtifact.phase: Failed:

buildArtifact.message mirrors kairos-operator's own OSArtifact.status.message — usually a pull failure (bad importFrom, missing registry auth) or a build failure inside kairos-operator's builder pod.

VMImage.status.perProvider[].reason (vSphere provider):

Reason	Meaning
`BuildPending`	`buildArtifact` isn't `Ready` yet (missing, `Pending`, or `Building`)
`BuildFailed`	`buildArtifact.phase == Failed` — see its `message`
`WrongArtifactKind`	Artifact is `Ready` but not `kind: iso` — the build pipeline is misconfigured (imagebuilder always requests `iso` for vSphere sources)
`NoFailureDomains`	ISO is `Ready`, but the `Provider` has no `status.failureDomains[]` published yet
`Importing`	A per-zone import Job is running (uploading the ISO / creating the template)
`ImportFailed`	The per-zone import Job failed — check the Job's logs in the build namespace
`ContentLibraryNotImplemented`	`Provider.spec.useContentLibrary: true` — the Content Library import path is a planned follow-up; leave it `false`
`UnsupportedSourceKind`	The vsphere source is `BackingFile` — not a vsphere concept, never supported here

ImportFailed with "did not power itself off within <N>s of the unattended Kairos install starting" (ADR-0021): the merged cloudConfigs[] result is missing the install.poweroff: true / install.reboot: false + after-install-chroot wipe stage described above — the VM installed fine but never shuts itself down, so the Job times out and leaves the VM running (not destroyed) for console debugging via govc vm.console -h5 <vmimage-name>. Fix the cloud-config and re-run (spec.template.forceCreate: true to replace the half-built VM).

kubectl -n banlieue-system logs deploy/banlieue-imagebuilder
kubectl -n banlieue-system logs deploy/banlieue-provider-vsphere
kubectl -n banlieue-imagebuild logs job/<import-job-name>
kubectl describe vmimage kairos-ubuntu-2404   # Events

Full schema reference¶

Every field of every CRD: API Reference.