Threat Model¶
Status: Living document. Last full pass 2026-11-09, against the architecture defined by ADR-0001 … ADR-0124 (0116–0122 reserved by roadmap 20, 0123 Rejected; 0057–0059 unallocated, 0066 and 0068–0073 reserved; 0074 and 0075 Accepted 2026-09-28; 0076–0080 reserved by roadmap 19; 0081–0082 recorded but not implemented in banlieue, and kept canonically in mediatore since ADR-0097; ADR-0115 (2026-11-04: the imagebuilder's rebuild gate reads the artifacts PV and its node, two read-only cluster-scoped grants; a volume lost with a replaced node is rebuilt: §2, TB-3, §7, §8); ADR-0101, amended 2026-11-04 (a vSphere pass decides from a live read of the machine, not the watch cache, and a recorded
vmRefwhose VM is gone is recovered by the ownership marker: TB-4 vSphere ownership, two rows, and the §8 marker risk); 0088 Accepted 2026-10-26, superseding 0048; 0108 Accepted 2026-11-05 (proposed 2026-10-15); 0049 amended 2026-11-05 (SPIRE TPM node attestation supersedes Decision 4,status.nonceretired); 0114 amended 2026-11-06 (a static machine reports its declared address, labelledaddressSource: Static, when nothing observes one; TB-1 row; no new component, identity, grant or boundary, §5 diagram unchanged); 0109 Accepted 2026-11-07 (the isolated sandbox segment: TB-18); 0108 and 0109 renumbered 2026-10-26 from duplicate 0100 and 0103; 0112 Accepted 2026-10-28, renumbered 2026-10-30 from 0110, which main had meanwhile taken twice; 0113 Proposed, its tool, the provider's pacing and the imagebuilder's pre-lay Job implemented; 0087 amended 2026-11-01 and 2026-11-03; 0113 accepted 2026-11-02 and amended twice 2026-11-04; 0114 Accepted 2026-11-04; 0054 amended 2026-11-04: every NoCloud seed carriesuser-data, an empty cloud-config when none is set, so cloud-init stops ignoring seeds without it; no row changes, the seed's contents and who controls them are as before). The most recent pass, 2026-11-09, is ADR-0124 Decision 2's amendment: when placement fails after the cordon removed a candidate, the scheduler re-runs the same pure function with the cordon ignored, and reportsProviderUnschedulablenaming the cordoned Providers if that would have placed the VM. It changes a reason and a message, not who can place what: no actor, identity, grant, boundary or accepted risk changes. The second pass runs only on failure. The pass before it, 2026-11-08, is ADR-0124: a Provider labelledbanlieue.io/unschedulable: "true"(a cordoned host) is dropped by the scheduler right after the selector and takes no new VM; VMs already on it are not moved and the provider keeps reconciling them. Implementing it found that ADR-0102's stale filter moved placed VMs after all (the scheduler re-runs for them, and drift under the defaultmigrationPolicyrecreates a VM); a VM's own Provider now stays a candidate through both filters, and ADR-0102 is amended. Three TB-17 rows. Setting the label takespatchon Providers, which already lets its holder change every label a selector matches, so no grant, identity, boundary or accepted risk changes. The pass before it, also 2026-11-08, is ADR-0043 Decision 8's third amendment: the libvirt provider polls a machine still expecting its guest's report every 2 s for its first 90 s (then 30 s, as before). The reads are the same read-only guest-agent calls over the same mutual-TLS libvirt session (TB-4), only sooner; the extra load on libvirtd is bounded per member (at most 45 probes) by a window counted from creation, which a guest cannot extend. No actor, identity, grant, boundary or accepted risk changes; nothing changes beyond the stamp. The pass before it, 2026-11-07, is ADR-0109's acceptance: a new boundary, TB-18 (sandbox segment → egress gateway → routable network), in the §5 diagram and table; a §6 section mapping its threats to banlieue's topology controls (no default route for members, egress first in the gateway class, ADR-0083) and to the host and image controls that are not banlieue's; §7.29 (a segment routes nowhere); two §8 risks (a gateway's image-setsandboxaddress is unseen by ADR-0083; the gateways' hardening is the image author's). The pass before it, 2026-11-06, is ADR-0114's amendment, the declared address: a TB-1 row for a declared address read as an observed one (labelledStatic, replaced by any observation, ADR-0083 keeps it from naming another VM's address). The pass before it, 2026-11-05, is ADR-0108's acceptance and ADR-0049's amendment: the handshake the consumer built is SPIRE's TPM node attestation (a credential-activation challenge sealed to the vTPM's EK), not quote-over-nonce, so the controller stops mintingstatus.nonce. The TB row on who receives the subject's token now names that control and banlieue's published anchors (EK certificate,ekTrustBundle); A-6a, the EK-mismatch row and the broker risk drop "quote"; §8 gains the ADR-0108 residual (a key that is not TPM-bound baked into a sandbox image), besideAsserted. Less is published, no grant, identity or boundary changes, and the §5 diagram stands. The pass before it is ADR-0115's (noted above); the one before that, the second of 2026-11-04, is ADR-0114: libvirt and Cloud Hypervisor now write a cloud-initnetwork-configinto the NoCloud seed for a static NIC, through the renderer Proxmox already used, and libvirt derives MACs for such a machine. A tenant's address, gateway, nameserver and domain strings now reach two more seeds as YAML: TB-1 gains a row (the renderer quotes every value; the author already controls the guest through user-data) and A-2 names the file. No component, identity, grant or boundary changes and the §5 diagram stands. The pass before it, the first of 2026-11-04, covers three changes. ADR-0113's second amendment of that day, a Grype gate for the pre-lay image:make grype-prelay(thegrype-prelayCI job) scans every pushedbanlieue-prelayimage, reports every finding to Code Scanning, and fails on a high or critical finding Debian has already fixed, with curated statements from.vex/prelay/; the release's asset upload waits for it. CI only: no component, actor, identity or boundary changes and the §5 diagram stands. The TB-6 Pre-lay image and Scanning rows name the gate, and the §8 risk it replaces narrows to the findings Debian has not fixed. ADR-0113's first amendment, a published pre-lay image: CI publishesghcr.io/firestoned/banlieue-prelay(Cosign-signed; provenance and SBOM on main and releases), the imagebuilder's--prelay-imagedefaults to it andbanlieue bootstrap imagebuilderpins it with--prelay-image-digest. The privileged Job's code now comes from the project's supply chain instead of each operator's: A-5 and the §2 row name it, TB-6 gains a row, the TB-10 row's trust sentence changes, and §8 gained a risk (the image had no Grype gate) that the second amendment narrows. And ADR-0087's amendment of 2026-11-03: vSphere'sbanlieue.io/force-reimportis handled perProvider, each workload writing its own marker annotation on theVMImageand the last clearing the request. Those writes use theVMImagepatch the workloads already held; whoever can forge a marker can already delete the request, so no row changes. The pass before it, 2026-11-02, is ADR-0113's acceptance: the pre-lay Job ran in a cluster. Its one new behaviour, the script making missing loop partition device nodes inside its privileged container (no udev there), adds no capability the Job did not already hold as a privileged pod (§4 row, TB-10 rows and the §8 risk stand), so no row changes; the failure path (PrelayFailed, nofile) was observed live, as the TB-10 row says. The pass before it, 2026-11-01, is ADR-0087's amendment: a provider workload started for oneProvider(as the operator deploys them) reconciles only thatProvider'sVMImagerows, under its own field manager, on libvirt, Proxmox and vSphere, and vSphere's deletion becomes per-Provider. Found live as a v0.4.9 libvirt regression (workloads reading each other's Secrets, refused, and overwriting each other's rows), rolled back; vSphere (rows, and a delete that hung) and Proxmox (rows erased by a shared manager) were latent. It adds one TB-4 row and one §8 risk (a deleted vSphereProvider's templates are left in vCenter); no new identity, grant or boundary: it makes the workloads stay inside the grants they already had. The pass before it, 2026-11-01, is ADR-0113's amendment: the imagebuilder pre-lays aUrlimage when itsVMImagesetsspec.prelay, with a new privileged Job in the build namespace that boots the build once under QEMU/KVM and reports only a digest. It adds one §4 identity row, three TB-10 rows (a wrong digest, a QEMU escape, a rewritten build) and one §8 risk (a privileged Job running a guest on a build node); no new namespace, RBAC grant or boundary, since the namespace already runs privileged builds. The pass before it, 2026-10-31, is ADR-0113: pre-laid images, where the Kairos recovery reset every raw clone runs on first boot is done once at build time (scripts/prelay-kairos-raw.sh), and the Cloud Hypervisor provider requeues a VMM start step after 1 s. A pre-laid image is a disk artifact made by booting the image once, so the pass adds five TB-5 rows: an image sealed to the builder, build-boot identity and logs, operator stages run at build time, a reset that never ran, and a host rewriting an artifact. Each maps to a check in the script or to ADR-0064's existing rule that the provider never writes the image cache. No actor, identity, RBAC grant, namespace or boundary is added (the build boot runs on an operator's machine, with no network and no TPM), so §4, §5, §7 and §8 stand; the pacing change alters timing only. The pass before it, 2026-10-30, reconciles ADR-0112 with ADR-0110 and ADR-0111, all three passed in parallel. They do not interact: ADR-0110 changes what the imagebuilder's push Job trusts (TB-10), ADR-0111 how many adapters a vSphere clone gets (its own TB-4 table), and ADR-0112 what a Cloud Hypervisor guest reports over vsock (the TB-4 and TB-5 sealing rows, the TB-8 vTPM row). A sealing report is per guest, not per adapter, so a second vSphere NIC changes nothing it reads; nothing changes beyond the stamp. ADR-0112's acceptance, 2026-10-28, followed a live pool run on a Cloud Hypervisor host. The run found that a clone which never reports was markedFirstBootSealingTimeoutup to one long requeue (300 s) late; the provider now re-checks every 30 s while a report is due. That tightens the halted-clone row; no actor, identity, grant, boundary or accepted risk changes. ADR-0112 itself, 2026-10-27: first-boot sealing on Cloud Hypervisor. AnImmediateclone there is the guest's own reflinked or copiedos.rawwith a fresh swtpm, so sealing never writes the cached image, and the guest reports its sealed partitions as asealed=line on the ADR-0065 vsock report, parsed by the shared provider-SDK parser and gated through the same deadline as libvirt and vSphere. The scheduler no longer excludes Cloud Hypervisor. That pass widened A-8 and the two sealing-report rows to the vsock channel, rewrote the CH vTPM row and the §7 sealing requirement, and extended the halted-clone row; no actor, identity, RBAC grant or boundary changed, so §4 and §5 stand. The pass on 2026-10-30 before these is ADR-0111: a cloned vSphere VM now gets one NIC per declared interface, where it used to get one. Nothing new is trusted and no identity, RBAC grant, namespace or boundary is added; what changes is that a second interface now exists, so a VM can sit on two segments. It adds one §2 clause, one A-4 clause, one TB-1 clause, a TB-4 table (five rows), §7.28 and two §8 risks, and a line in the §5 diagram. The control that matters is old and was already per interface: a tenant picks aVMClass, it cannot add a NIC (networkOverrideshas nonetworkClass). The pass before it, 2026-10-29, was ADR-0110: the registry push Job can trust an operator-named CA bundle, for a registry served by a private CA. That is a new trust anchor in the one Job that holds a registry credential. It added one §2 sentence, five TB-10 rows, a clause in §7.13 and one §8 risk, and no actor, identity, RBAC grant or boundary. Before that, 2026-10-26, was ADR-0088's acceptance: Kairos v4.4.0 shipped first-boot sealing and the three kcrypt fixes its Accept when named, a rawcloudImagebuild was shown to carry the sealing layer ontoCOS_OEM(Open question 1), and--enable-first-boot-sealingnow defaults on, with=falserestoring ADR-0048. It rewrites the §2 controller row, A-12, two TB-5 rows, one TB-8 row and §7's sealing requirement, retires the §8 "pre-release Kairos" risk in favour of a narrower one (an image older than v4.4.0 is caught only by the readback) and adds one §8 risk (blocks written toCOS_PERSISTENTbefore sealing are not guaranteed wiped). No actor, identity, RBAC grant or boundary is added, so §4 and §5 are unchanged; Cloud Hypervisor stayed excluded until ADR-0112. The pass before that, 2026-10-24, was ADR-0107: the scheduler places a VM only where it fits the capacity a provider publishes on its failure domain (memory strictly, with a per-VM overhead; vCPUs up toProvider.spec.scheduling.cpuOvercommitRatio, default 4), and the Cloud Hypervisor provider publishes the smaller of the host's resources and its guests' slice budget, read from systemd. A provider's status now steers placement, so this pass adds two TB-8 rows (false capacity from a stolen host token, bounded to that host; fixed, read-only slice reads), one TB-1 row (the ratio), §7.27 (one host, one Provider) and three §8 risks (placement is not a reservation; capacity is counted per Provider; memory outside banlieue's VMs is not deducted). No identity, RBAC grant, boundary or diagram change. The 2026-10-23 pass before it is ADR-0106: management-cluster nodes on Cloud Hypervisor run inbanlieue-nodes.slice, beside ADR-0105'sbanlieue-guests.slice, sobanlieue.sliceholds every VM on a host and an operator can bound guests and nodes together withsystemctl set-property; andbanlieue host cloud-hypervisor overhead, a read-only verb, reports each VM's guest CPU against host-side overhead. It updates two §2 rows and the §5 diagram, adds three TB-9 rows, §7.26 and one §8 risk (a compromised VMM can skew the measurement); the polkit tests now refuseset-propertyandstopon all three slices. No identity, RBAC grant, boundary or cluster object changes. It also fixes two citations of a removed reference unit. The 2026-10-23 pass before it covers three amendments found by a live vSphere run (the third, ADR-0034's, is described after the first two): ADR-0044 on vSphere (once the guest is installed the provider ejects an ISO-backed CD-ROM, asgovc device.cdrom.ejectdoes, and only then canGuestReadybeTrue; recorded asVSphereMachineStatus.installMediaDetached) and ADR-0020's (aforceUploadre-import refuses, naming the VMs, before destroying anything when a running VM holds the ISO, and a failed import Job is retried after 10 minutes). It rewrites the TB-5 prose and three TB-5 rows (they were libvirt-only) and adds one TB-5 row. The provider's vCenter role now usesVirtualMachine.Interact.SetCDMedia/.DeviceConnectionon the provisioning path too (vsphere-least-privilege.md). ADR-0034's amendment makes the vSphere provider drive a provisioned VM tospec.desiredPowerState, as the other providers always did, so a VM stopped in vCenter is started again; it adds one TB-4 row and one §8 risk. No actor, identity, RBAC grant or boundary is added, so §5 is unchanged. Before it, the 2026-10-22 pass was ADR-0105: Cloud Hypervisor guests and vTPMs run in one root-owned, operator-budgetedbanlieue-guests.slice, so a host that also runs other workloads can bound them. It adds a root-owned unit file, four installer flags rendered into it, one TB-9 row, one TB-11 row and one §8 risk (a slice is a ceiling, not a reservation); no actor, identity, RBAC grant or boundary is added, so §5 is unchanged. Before it, the 2026-10-22 pass was ADR-0088: first-boot sealing forImmediateimages, behindbanlieue-controller --enable-first-boot-sealing. The imagebuilder appends a layer writing the sealing policy intoCOS_OEMand a guest sealing report; the controller's pairing rule gains the declaration and keeps such VMs off Cloud Hypervisor; both providers gateGuestReadyon the report, with a deadline. It adds one VAP (vmimage-first-boot-sealing.yaml), amends A-6 and A-8, the §2 controller and imagebuilder rows, three TB-4 rows and three TB-5 rows, §7.9, and five §8 risks (theManualrow is narrowed). No component, identity, RBAC grant or boundary is added (the report travels the ADR-0043 channels), so §5 stands. The 2026-10-21 pass is ADR-0104 (Accepted, implemented): a vSphereTemplate-kindVMImage(an existing vCenter template banlieue adopts and never builds) is looked up per failure domain, in<rootFolder>/<failure-domain>, and publishes per-zone status like aUrlimage. That would have made deleting such an image destroy the administrator's own templates, sofinalizenow destroys only templates aUrlsource imported, decided on the source kind. This pass adds a TB-4 block (four rows: adopted-template deletion, a planted same-named template, forged zone status, per-zone lookup load), extends A-3, the vSphere provider in §2 and the hypervisor-operator actor, and adds one §8 row (an adopted template is trusted by name, not provenance). It also records that the model never coveredTemplatesources or template destruction onVMImagedeletion (ADR-0028). No component, boundary, identity, RBAC grant or vCenter privilege changes (the lookups are the same read calls), so the §5 diagram stands. The earlier 2026-10-21 pass is ADR-0103:banlieue bootstrap --image-digestpins every banlieue image it writes (the controller, operator, imagebuilder and provider Deployments as well as the seededProviderClasses, which were the only ones pinned before) and refuses a malformed digest, and each release'sinstall.yamland clusterctl components are rendered after the image push with its multi-arch digest; a release with no digest fails. It closes the §8 risk that published manifests named the image by tag (that row also wrongly said bootstrap had no digest flag), adds two TB-6 rows, widens A-5 and adds §7.25. No component, boundary, identity or RBAC grant changes, so §5 stands. The 2026-10-15 pass is ADR-0102: a provider's liveness is judged outside it, from the leader-election Lease it renews. banlieue-operator reads each Provider's Lease (oneget) intostatus.workload.heartbeat, and the scheduler places nothing new on aStaleprovider. The Lease thereby becomes an input to scheduling, so this pass adds TB-17 (§5 table and diagram, §6) and asset A-22, updates the operator and controller in §2, and adds two §8 risks (clock skew, the operator as the evaluator). It also found and fixed a wider grant: every shared ClusterRole but the operator's let its identity write any Lease in the cluster, so Lease rights are now a namespaced Role narrowed to each role's own Lease (banlieue bootstrap, the GitOps manifests, and the operator's per-Provider Role, which already was). The earlier 2026-10-14 pass is ADR-0101: every vSphere clone carries itsVSphereMachine's UID in theextraConfigkeybanlieue.machine.uid, outsideguestinfo.*and set byCloneVM_Taskitself, and a create or a deletion with nostatus.vmRefsearches the clone's target folder and destroys only a VM carrying that UID. It widens A-16 to vSphere, adds a vSphere ownership block to TB-4 (four rows), extends the hypervisor-operator and sandbox-workload actors, widens the §8 marker risk to vSphere and adds one §8 row (the guest cannot reach the key, not yet verified live). No component, boundary, identity, RBAC grant or vCenter privilege changes, so the §5 diagram stands. The earlier 2026-10-14 pass is ADR-0100: every release publishesbanlieue-apito crates.io at the release's version, from abuild.yamljob of its own that alone reads the new repository secretCARGO_REGISTRY_TOKEN. It adds one asset (A-21, that token), one actor (a Rust consumer of the published types), the crates.io leg of the §5 TB-6 line, three TB-6 rows, §7.24 for consumers and two §8 risks (a crate upload is unsigned and immutable; the token is long-lived until Trusted Publishing). No runtime component, CRD, identity or RBAC grant changes: the crate carries types, not a client, and a consumer reaches the API server under its own RBAC and the same admission policies askubectl. The 2026-10-13 pass is ADR-0099: the management cluster's bootstrap can run a node on Cloud Hypervisor (a plain VM under three systemd units, no banlieue component) and can add workers and controllers to a running cluster with a short-lived token (join; a controller copies a running one's/etc/k0s), so a cluster's nodes can be swapped from libvirt to CH one at a time. Amended the same day (Decision 5), CH is now the default VMM: the host sudo it needs is the default rather than an opt-in, every command first refuses a node whose VMM disagrees with what exists on the host (so a libvirt-built cluster is neither doubled nor half-destroyed), and with no bridge configured CH nodes join the libvirt network's own bridge, read from libvirt (two rows on the CH management-node table, §4, §7.23, §8). It adds one component (§2), one asset (A-20, the join tokens), one actor (the bootstrap operator with sudo on the host), one boundary (TB-16, bootstrap operator → running cluster), a §5 diagram block, rows on TB-9 for a management node's VMM, §7.23 and three §8 risks. Found and fixed during the pass: the units' runtime directory defaulted to0755, and the VMM ran without Landlock (now on, with read access to its own tap's sysfs entry only). The 2026-10-12 pass is ADR-0097 (banlieue's API ends atVirtualMachineClaim;AgentSandboxmoves to mediatore as its CRD), and its conclusion is no boundary change.AgentSandboxandagent.banlieue.iowere never built, so no component, CRD, identity or RBAC grant leaves the deployed posture, and the broker stays what §4 already models: an external claim consumer bounded by §7.6. The pass updates the broker actor row, one §8 risk (mediatore's declarative request path) and §9. The 2026-10-13 pass covers the ADR-0047 Decision 9 amendment: the claim subject policy takes a username prefix per issuer (issuerPrefixes), falling back tousernamePrefix. No component, asset, actor or boundary changes. It narrows one §8 risk: for every issuer with anissuerPrefixesline, a claim'ssubject.issueris now verified (the prefix on the caller's username names the issuer that authenticated them) rather than only allowlisted. It updates two TB-7 rows and §7.6. It also covers ADR-0098: the scheduler spreads placement across failure domains (least loaded for the VM's pool, then overall, then a rendezvous hash) instead of taking the alphabetically first. No component, asset, actor or boundary changes; it closes one availability gap (a pool's whole warm set on one host) with a TB-1 row. Load is counted only from VMs in the same namespace, so placement can be steered only by someone who can already create VMs there. A second 2026-10-12 pass covers the ADR-0044 amendment: a libvirt machine with install media now boots its disk first and the installer only while the disk is empty, fixing a deadlock found live (the guest rebooted into its installer after every install). No component, asset, actor, identity or boundary changes. It strengthens one TB-5 row (a guest re-entering its still-attached installer: the disk now wins whenever it is bootable, before the eject as well as after) and changes nothing else. The 2026-10-08 pass is ADR-0091 to ADR-0096 (roadmap 12): Prometheus metrics and opt-in OpenTelemetry export, path-aware health, opt-in NetworkPolicies, credential rotation without Secret watches, and clusterctl packaging with a scoped CAPI aggregate role. It adds no component and no namespace. It adds two actors (CAPI's manager, the telemetry consumers), one asset (A-19, telemetry), two boundaries (TB-14 telemetry, TB-15 CAPI controllers → infrastructure objects), rows on TB-2 and TB-4, §7.21 and §7.22, and four §8 risks. It closes two things: the CAPI manager no longer inherits the controller's whole role (it had create and delete on everybanlieue.iokind), and the "fixed200health endpoint" risk, which ADR-0093 retires. A fourth 2026-10-06 pass is ADR-0090 (VirtualMachinefinalize cascade: isolate each backend kind): no new component, actor, asset or trust boundary — found live, a staleClusterRolemissing aproxmoxmachinesgrant madefinalize_vm's?-chained backend lookups abort before the vSphere branch below them ran, so a permission gap on one backend blocked deletion for everyVirtualMachine, any backend. Adds one TB-1 row for the fixed availability threat (classified D: the blast radius of a lookup failure, now scoped to the affected backend's own VMs byplan_finalize). The 2026-10-06 pass before it is ADR-0089 (VSphereMachinecustom attributesTemplate/CreatedAt/CreatedBy): adds A-18, a TB-1 row for a forgedbanlieue.io/created-byannotation, and a §7.7 note on the newMutatingAdmissionPolicy's higher (1.36+) floor andfailurePolicy: Ignoreposture — no new component, actor or trust boundary. A third 2026-10-03 pass is driven by evidence rather than an ADR: an OIDC issuer (Dex) and workload identity (SPIRE) were deployed on a reference management cluster, and every link's cryptography was measured by offering one TLS group at a time. It changes no component or boundary. It adds TB-7 rows for an issuer that runs on the management cluster (its signing keys become cluster resources; its default token lifetime; any upstream account can log in; the API server's own read access to its authentication config), corrects TB-12 (sslmode=requireencrypts but does not authenticate the server; PostgreSQL 17 cannot negotiate post-quantum key exchange), adds two TB-13 rows (the default allow-all tailnet policy, and a roaming client whose VIP route is not yet approved), adds §7.20 (post-quantum key exchange on every link that carries cluster secrets) and records three §8 risks: banlieue's own TLS clients use rustls with theringprovider (ADR-0009), which has no ML-KEM, so every connection banlieue makes is classical; classical key exchange on links banlieue does not terminate; and no post-quantum signatures anywhere. The 2026-10-03 ADR pass covers ADR-0086: a management cluster with more than one controller puts its API behind a keepalived VRRP virtual IP that k0s's reverse proxy balances over every controller, and runs without konnectivity. No banlieue component, asset or actor changes: keepalived and the proxy run inside k0s, and the VRRP password is not a secret by design. It adds one trust boundary (TB-13, the LAN to the VIP), with a §5 diagram line, a STRIDE table, a TB-8 row (host providers now dial the VIP), §7.19 and two §8 entries (VIP takeover from the LAN, and API servers reaching kubelets directly). It removes a single-controller availability dependency that was never recorded.The later 2026-10-03 pass covers ADR-0087: a field manager names one writer, not a provider class. No component, asset, actor or boundary changes, because the writers and what they write are unchanged. What changes is that this document previously asserted something false: the TB-8 row for a stolen host token said the token "writes only its own
perProviderrow", which was true of the RBAC and of the intent but not of the effect. Every host applied under one class-wide field manager, and because a manager owns exactly the field set it last applied, each host's single-row apply made the API server delete every other host's row; the resulting watch storm was a write loop any host could cause with its own legitimate credential, observed at ~170 writes/sec sustained on one object. That row is corrected and a new TB-8 row records the threat and its control. Theforceon that apply is removed, so a cross-manager claim is now a conflict rather than a silent takeover. §§1–5 and 7–10 are unchanged: this was never a boundary that was missing, it was a control that did not do what the table claimed. The second 2026-10-02 pass covers ADR-0085: the management cluster's bootstrap can keep cluster state in an external SQL database through kine (PostgreSQL on the hypervisor host), put its nodes on a host bridge, and boot Kairos on Debian. It adds one asset (A-17, the datastore, which holds every object including every Secret) and one trust boundary (TB-12, controllers to datastore), with a §5 diagram line, a STRIDE table, §7.18 and two §8 entries (the datastore as a single point of failure, and nodes reachable from the LAN). Etcd, the default, was already inside the nodes and moves nothing. No component, actor or other boundary changes. The first 2026-10-02 pass covers ADR-0084: the installer is nowbanlieue host cloud-hypervisor <verb>(aliasch; it wasbanlieue host <verb>); it no longer installs OS packages on any distribution (the host's package manager supplies systemd and swtpm;preflightchecks for them by command); and the VMM,ch-remoteand firmware each come from a URL (github.com by default, or an operator's mirror) at the pinned release or any release at or above the client's version gate, still verified against a sha256 before anything is installed. No new component, actor or trust boundary: TB-11 gains a second source (a mirror) and, for a non-pinned release without a digest flag, a read of GitHub's published digest fromapi.github.com; the §5 diagram's TB-11 line says so. §2's installer row and A-14 name the new digest sources. TB-11 gains five rows (a version or tag escaping/opt/banlieue, a plain-HTTP mirror, a hostile release document, the[vmm]rewrite, and a spoofed GitHub digest) and its two pin rows are revised. §7.11 gains one bullet, §8 two entries (a non-pinned release trusts GitHub's or the operator's digest; a newer VMM runs against types checked only against the pinned API), §9 one (the host's OS packages).2026-10-01 pass. Covers ADR-0083 as revised in review: a
VirtualMachinewhose static address another banlieue VM already claims on the samenetworkClassis blocked before scheduling (Ready=False reason=DuplicateAddress, no infra CR), on all four infra kinds including Proxmox (ADR-0075); incumbency comes from the new, controller-writtenstatus.heldAddresses, never from guest-reported addresses; a blocked VM that is already provisioned keeps its infra CR but still has power applied. No new component, asset, actor or trust boundary, so the §5 diagram is unchanged. The controller readsVirtualMachines across every namespace it watches, from its existing reflector store. §2's controller row names the check. TB-1 gains three rows: the traffic hijack/denial a declared duplicate enabled (controlled), cross-namespace disclosure through the condition message (controlled), and a forgedstatus.heldAddresses(bounded by RBAC). §7 gains 17 (onenetworkClassper real segment). §8 gains two residues: first-come address squatting and a one-bit "address in use" oracle across namespaces. Guest-reported addresses do not feed the check, so it adds no guest-influenced residue. §9 gains machines banlieue did not create.2026-09-28 pass — Proxmox VE (ADR-0074, ADR-0075). A third hypervisor backend, reached over HTTPS with a privilege-separated API token. One new component (
banlieue-provider-proxmox), one new actor (a stolen Proxmox token), one new asset (A-16, which of a Proxmox cluster's VMs banlieue considers its own), a Proxmox block under TB-4, rows added to TB-1, TB-2 and TB-5, hardening item §7.16 and five §8 entries. No new trust boundary: the provider is an in-cluster pod like the vSphere and libvirt ones, so it sits on TB-4. Every section was re-walked; TB-3, TB-6 to TB-11 are unchanged. What is verified: the client and the provider are unit-tested against an in-memory Proxmox that refuses what the real API refuses, and the read-only live tier (make proxmox-live-test) passed against a real node on 2026-09-28: token auth, TLS verified against the node's own CA, inventory decoding, and the reason phrase on a 401. On 2026-09-30 the lifecycle tier (make proxmox-lifecycle-test) passed too: clone, configure, resize (a task), multipart seed upload, start, stop, delete, task polling with real UPIDs, and a same-named foreign VM surviving the machine's delete.A 2026-09-28 pass covered ADR-0043 Decision 8 as amended 2026-09-28 (an unreachable guest agent backs off only after a 10-minute agent-bootstrap grace window, so the machine reconciler's cadence can no longer lose the race against a pool's
provisioningTimeoutSeconds): no change — no new component, asset, actor, boundary or control; A-8's liveness-not-integrity posture and bothGuestReadyrows in TB-4/§8 are unaffected by when the signal is polled. Recorded because the stamp is the deliverable, not because anything moved.A 2026-09-27 pass covered ADR-0049 Decision 10 (
Provider.spec.attestation.ekTrustBundle— the per-backend EK trust anchor is admin-supplied on the Provider, never discovered, and banlieue neither resolves nor verifies against it; ADR-0049 is now Accepted, with the in-guest agent and broker deliberately outside banlieue). The pass re-walked every section: no new component, actor, namespace or trust boundary (the broker and agent were already modelled as external). §3 gains A-15 (the trust bundle — zero confidentiality, high integrity: a rogue CA added to it makes every forged EK certificate verify); TB-1 gains its tampering row, shape-validated at admission bybanlieue-provider-attestation-ektrustbundlewhile content stays a §7.14 custody obligation; §4 names the sandbox workload (a prompt-injectable AI agent, hostile by design) as an actor, making explicit what TB-4/TB-5 already assumed; TB-1 also gains the stale-image-member row (bounded by pick order, TTL and rollout); §7 gains 14 (trust-bundle custody) and 15 (vSphere VM Encryption on sandbox storage classes, roadmap 17 phase F); §8's two trust-anchor residues are updated now that the bundle exists.The 2026-09-27 pass for ADR-0067 (
banlieue host, the installer in the binary) adds a component, an asset (A-14), an actor row, a trust boundary (TB-11, with the §5 diagram), eight STRIDE rows and one accepted risk. A pass the same day covers the host-resident Cloud Hypervisor provider (ADR-0060 to ADR-0065), the first banlieue component that runs outside the cluster, on the hypervisor host itself: a new component, a new actor (a compromised VMM), two new assets and two new trust boundaries (TB-8 host ↔ cluster, TB-9 guest VMM ↔ host and other guests), a new hardening requirement (§7.11) and five new accepted risks. It also corrects §1: the workspace is no longerunsafe-free — the new provider has two auditedioctlblocks, compiler-confined to one module. What is implemented and what is not is stated per row: ADR-0061, 0062, 0063 and ADR-0064 Decision 5 are live-verified; ADR-0060'sExternalmode and token self-renewal, the rest of ADR-0064 and all of ADR-0065 are not built, and the rows that depend on them say so.Amended 2026-09-26 for ADR-0060 Decisions 3–5 (External mode and token self-renewal), now implemented: A-11, TB-8 and §7.11 updated; the §8 "namespace-wide Role" entry narrowed to machines (Providers are now
resourceNames-scoped); the "token not renewed" entry replaced by two new ones — a stolen token can renew itself (revocation: delete the ServiceAccount), and the operator's namespaced token grant.Amended 2026-09-26 for ADR-0064 Decisions 1–3 (registry delivery), now implemented: two components (the registry push Job, the host's image import unit), A-3 extended and a new asset A-13 (registry credentials), a new actor (the registry), a new boundary TB-10 (build namespace → OCI registry → host) with its STRIDE table, rows added to TB-3 and TB-8, hardening item §7.13, and four §8 entries. Every section was re-walked; TB-1, TB-2, TB-4, TB-6, TB-7 and TB-9 are unchanged. A second pass the same night covers Decision 4 (cache eviction, and the
banlieue.io/host-image-cachefinalizer, held by the controller so host tokens gain noVMImagewrite): two TB-8 rows, TB-10 and §8 updated. A third pass (2026-09-27) covers ADR-0065's vTPM (Decisions 1, 2, 6): two components (swtpm and its one-shot manufacture), A-6 and A-6a extended, the TB-8 vTPM row rewritten and four TB-9 rows added. The EK certificate is read host-side, from a provider-only directory, so this backend avoids libvirt's guest-reported residue. The same pass coversDeferredinstall and the vsock report (Decisions 3–5): the report is a new guest → host input path, with three TB-9 rows. A fourth pass (2026-09-27) covers ADR-0063's amendment: guests, their vTPMs and image imports are instances of root-owned template units, not transient units. The transient design let the provider's user start a unit as root, because polkit sees only a unit's name; that is closed (verified on a host) and recorded as two TB-9 rows. Components, A-12, the §5 diagram and the §8 "compromised provider" entry are updated. The decompressed size of a pulled image is now capped by a digest-covered annotation, which removes that §8 entry. A fifth pass (2026-09-27) covers ADR-0065 Decision 3's amendment, found by the first liveDeferredrun: the VMM, running as the guest, could not read the provider's image cache, so each machine now gets its own read-only copy of its installer, deleted after the eject. A-12 lists it and TB-9 gains one row; no component, actor or boundary changes, and §8 is unchanged.Deferredinstall and the vsock report are now verified live (make ch-deferred-e2e).The 2026-09-24 pass covered ADR-0045 (the vTPM endorsement key certificate is published on the infra CR, mirrored onto a bound claim, and — for a
tpmEnabledmachine — gatesGuestReady). A-6 is split so the EK certificate is its own asset; TB-4 gains three rows for the guest-reported read path libvirt forces; §8 records the two residues it leaves — swtpm certificates that never expire, and a per-host local CA that is only as trustworthy as the host. The 09-24 pass re-walked every section after the vSphere half was brought level with libvirt: no new component, actor, asset or trust boundary, one control strengthened (theGuestReadygate now covers both backends, so atpmEnabledvSphere member with no certificate is no longer bindable) and one durability defect closed (patch_status_failedno longer retracts the anchor under SSA). Nothing moved into §8 — both residues the 09-23 pass had flagged on vSphere were fixed rather than accepted. banlieue acquires no new privilege in the guest: the certificate is read with ADR-0043's read-onlyguest-file-openpath, neverguest-exec. A second 09-24 pass covers ADR-0056 (VirtualMachinePool.spec.addressinggains a list of address-pool entries in place of one range): no new component, actor, asset or trust boundary. The entries are parsed entirely insidebanlieue-controller, still tenant-namespace-scopedVirtualMachinePoolspec data, and a malformed or wide entry (e.g. a/0CIDR) cannot be used to force unbounded work —pool_plan::plan()only ever walks as many addresses asmax_surgelets it create in one pass, so the CIDR's declared size never drives iteration by itself. Against the architecture defined by ADR-0001 … ADR-0056, with ADR-0049 as amended 2026-09-27 (banlieue's side implemented; the attestation exchange itself lives in the external broker and agent). ADR-0057 … ADR-0059 are reserved-unwritten (roadmap 15); ADR-0060 … ADR-0065 are Proposed (roadmap 09) and outside this stamp. 2026-10-11 TB-7 update (not a full pass; the stamp above is unchanged). No ADR:scripts/dev-oidc-k0s.shcan now add Google and Auth0 as direct issuers beside Dex in one structured authentication file. No component, asset or boundary changes; TB-7 now has more than one issuer behind it. It adds four TB-7 rows (prefixes shared between issuers, unverified emails at a self-signup issuer, tokens minted for another application at the same provider, a refused authentication file that only fails at the next restart), a §7.6 obligation, and a note on the §8 issuer-verification risk.Method: asset/actor enumeration, trust-boundary decomposition, STRIDE per boundary, control mapping to the manifests in
deploy/and the crates incrates/.This document describes what banlieue defends, from whom, and how. It is the companion to
SECURITY.md, which describes how to report a vulnerability. Specific unremediated findings are handled through private vulnerability reporting, not this page.
1. What banlieue is, in security terms¶
banlieue is a Kubernetes control plane that turns a namespaced VirtualMachine
custom resource into a real virtual machine on a hypervisor (vSphere, libvirt,
Proxmox). It holds two things an attacker wants:
- Hypervisor credentials. A
Providernames a Secret containing infrastructure-admin credentials for a vCenter or libvirtd — a username and password for vCenter, an mTLS client certificate and private key for libvirtd (there is no password in the libvirt path by design), and a privilege-separated API token for Proxmox VE (there is no password or ticket path there either, ADR-0074). Those credentials are, by construction, more powerful than the Kubernetes cluster banlieue runs in — they can create, delete, and read the disks of every VM on the backend, including VMs banlieue never created. - Guest bootstrap material. cloud-config / user-data routinely carries SSH authorized keys, cluster join tokens, and registration secrets. banlieue moves that material from a Kubernetes Secret into a guest, across several intermediate representations.
Everything below follows from those two facts. The memory-safety surface is
small and well-controlled: no subprocess execution anywhere in the workspace,
a fuzzed libvirt wire decoder, and unsafe in exactly two places — the
tap and bridge ioctls of the Cloud Hypervisor provider, which have no safe
wrapper in any crate. Both are in
crates/banlieue-provider-cloud-hypervisor/src/sys.rs, take their request
number from a closed enum so no request can be paired with the wrong
argument, and are confined there by #![deny(unsafe_code)] on the crate.
The meaningful risk in banlieue is authorization and data-flow, not memory
corruption.
The Cloud Hypervisor provider (ADR-0060) changes the first fact's shape
rather than adding to it. It holds no hypervisor credential — the
hypervisor is local — but it runs on the hypervisor host, holding a
cluster credential there, and it manages guests with host capabilities
(CAP_NET_ADMIN, CAP_CHOWN, CAP_FOWNER). Its two boundaries, TB-8 and
TB-9, are the host's relationship to the cluster and each guest's
relationship to the host.
2. Components¶
| Component | Identity | Namespace | Scope |
|---|---|---|---|
banlieue-controller |
banlieue-controller |
banlieue-system |
Watches VirtualMachine, schedules onto a Provider (only where the VM fits the failure domain's published capacity, ADR-0107, which lists VMClasses once per scheduling pass), creates provider infra CRs. Refuses a VM whose static address another VM already claims (ADR-0083), which reads VirtualMachines across every namespace it watches. Also runs the VirtualMachinePool loop (ADR-0046) and the VirtualMachineClaim loop (ADR-0047): the latter binds a pool member to a subject and destroys it on release. Unless run with --enable-first-boot-sealing=false (on by default since ADR-0088 was accepted, 2026-10-26) it accepts a tpmEnabled class with an Immediate image that declares spec.firstBootSealing, copies the declaration onto the infra CR, refuses it without a vTPM or with the flag off, and keeps such VMs off Cloud Hypervisor (ADR-0088): reconciler/virtualmachine.rs (image_class_mismatch), reconciler/infra.rs, reconciler/scheduler.rs. |
banlieue-operator |
banlieue-operator |
banlieue-system |
Provider lifecycle (ADR-0012): creates provider Deployments, ServiceAccounts, Roles, RoleBindings. Reads each Provider's leader-election Lease (one get) and records its heartbeat in status.workload (ADR-0102) |
banlieue-provider-vsphere / -libvirt |
per-Provider SA |
banlieue-system |
Talks to the hypervisor; reconciles infra CRs (VSphereMachine, LibvirtMachine — ADR-0050) and realises them as real VMs/domains. For a Template-kind VMImage it looks the adopted template up per zone, read-only, and never destroys it (ADR-0104) On vSphere a clone gets one NIC per declared VMClass interface (ADR-0111): the template's own NIC is edited onto the first interface's port group and each further interface is added in the same CloneVM_Task (client/vim.rs, build_clone_nic_changes) |
banlieue-provider-proxmox |
per-Provider SA |
banlieue-system |
Talks to Proxmox VE through the first-party banlieue-proxmox client (ADR-0074); reconciles ProxmoxMachines (ADR-0075) into full clones of a template VM, verifies the template VMIDs VMImages name, and publishes one failure domain per node. Spawns no Jobs — deploy/provider-proxmox/rbac/clusterrole.yaml |
banlieue-provider-cloud-hypervisor |
ServiceAccount banlieue-provider-cloud-hypervisor (a bound token in a kubeconfig on the host); host user banlieue |
Not in the cluster — a systemd service on the KVM host; its Provider lives in banlieue-system |
Watches its own Provider, CloudHypervisorMachines and VMImages over the API; creates taps, disks, seeds and one transient systemd unit per guest (ADR-0060, ADR-0063). Holds CAP_NET_ADMIN, CAP_CHOWN, CAP_FOWNER and nothing else — deploy/provider-cloud-hypervisor/host/banlieue-provider-cloud-hypervisor.service |
| Cloud Hypervisor VMM, one per guest | Its own host uid and private gid (banlieue-g<uid>, registered in /etc/userdb), plus kvm |
The host, as banlieue-ch@<guest uid>.service, an instance of a root-owned template |
Runs one guest. No capabilities; seccomp and Landlock on; systemd sandbox: deploy/provider-cloud-hypervisor/host/banlieue-ch@.service |
banlieue-imagebuilder |
banlieue-imagebuilder |
banlieue-system |
Drives kairos OSArtifact builds; merges cloud-config (ADR-0037) ; for a VMImage with firstBootSealing appends a banlieue-generated layer after every operator layer: the sealing policy into the installed disk's COS_OEM and a boot stage that reports the sealed partitions (ADR-0088, src/first_boot_sealing.rs); reads the artifacts PV and the node it is pinned to (read-only, cluster-scoped) to rebuild an artifact whose node-local volume was lost with a replaced node (ADR-0115) |
swtpm, one per tpmEnabled Cloud Hypervisor guest |
The guest's uid and private group | The host, as banlieue-swtpm@<guest uid>.service |
Emulates the guest's TPM from state in /var/lib/banlieue/tpm/<guest uid>/ (ADR-0065) — plan.rs::swtpm_unit |
| vTPM manufacture, once per guest | Host user banlieue |
The host, as banlieue-swtpm-setup@<guest uid>.service |
Runs swtpm_setup: creates the TPM and signs its EK and platform certificates with the host's swtpm_localca key — plan.rs::swtpm_setup_unit |
| per-zone import Job | banlieue-import |
banlieue-imagebuild |
Uploads a built ISO to a datastore, creates a template (ADR-0020) |
| registry push Job | None — automountServiceAccountToken: false |
banlieue-imagebuild |
Pushes a Ready build to the operator's OCI registry for host-resident providers (ADR-0064); reads the artifacts PVC read-only, the push Secret and, if the operator names one, one key of a CA bundle (ADR-0110, a trust anchor, mounted read-only), nothing else — crates/banlieue-imagebuilder/src/reconciler/push.rs |
| pre-lay Job (ADR-0113) | None: automountServiceAccountToken: false |
banlieue-imagebuild |
Boots a Ready raw build once under QEMU/KVM (no network, no TPM) to pre-lay it, for a VMImage with spec.prelay; privileged (loop devices, mounts, /dev/kvm, as the namespace's kairos-operator builds), mounts the artifacts PVC read-write to write <osartifact>-prelaid.raw beside the untouched build, runs --prelay-image (by default the published ghcr.io/firestoned/banlieue-prelay at the imagebuilder's version; banlieue bootstrap imagebuilder --prelay-image-digest pins it), and reports only a digest: crates/banlieue-imagebuilder/src/reconciler/prelay.rs (build_prelay_job), scripts/prelay-kairos-raw.sh |
| image import unit (Cloud Hypervisor) | Host user banlieue |
The host, as banlieue-ch-import@<vmimage uid>.service |
Pulls one image by digest into the storage classes' image caches and exits; sandboxed, writable only in images/ — crates/banlieue-provider-cloud-hypervisor/src/vmimage.rs (import_unit), import.rs |
| kairos build pod | kairos-operator's SA | banlieue-imagebuild |
Privileged — loop devices, mount, chroot |
| Management-cluster node on Cloud Hypervisor (ADR-0099) | Host user CH_VM_USER (default k0s-vm, no login, kvm), shared by every such node on the host; its tap is created by a root oneshot |
The host, as k0s-vm-<name>.service, with k0s-vm-<name>-net.service and k0s-vm-<name>-swtpm.service; not a banlieue object |
Runs one management-cluster node, written by scripts/bootstrap-k0s-cluster.sh through sudo. No capabilities, NoNewPrivileges, ProtectSystem=strict, DevicePolicy=closed (/dev/kvm, /dev/net/tun), Landlock with the tap's sysfs entry read-only, runtime directory 0700; writes only CH_VM_DIR/<name>/ (0750) and its runtime directory: render_ch_vm_unit. VMM and vTPM in banlieue-nodes.slice (ADR-0106) |
banlieue host cloud-hypervisor (installer) |
root, once, at an operator's request; no cluster identity | A Cloud Hypervisor host, as a command, never a service | Installs the VMM and firmware (the pinned release, or one the operator names at or above the client's gate, from github.com or a mirror; ADR-0084), the banlieue user, guest uid records, directories, the host config, the EK CA, the polkit rule and the units (ADR-0067). Installs no OS packages. Its overhead verb (ADR-0106) is read-only and runs as any user: it reads /proc/*/task/*/stat and cgroup counters and writes nothing. A separate crate no provider depends on: crates/banlieue-host/ |
3. Assets¶
| ID | Asset | Where it lives | Impact if lost |
|---|---|---|---|
| A-1 | Hypervisor credentials — vCenter username/password, a libvirt mTLS client key, or a Proxmox API token (username = user@realm!tokenid, tokenValue) |
Secret named by Provider.spec.connection.credentialsRef |
Critical — full virtualization-layer compromise, independent of Kubernetes |
| A-2 | Guest bootstrap material (cloud-config, SSH keys, join tokens) | Secrets/ConfigMaps → VSphereMachine.spec.userData / LibvirtMachine.spec.userData / CloudHypervisorMachine.spec.userData / ProxmoxMachine.spec.userData → guestinfo.userdata or a NoCloud ISO (on Cloud Hypervisor, seed.iso in the machine directory on the host; on Proxmox, banlieue-<machine uid>.iso on an iso-content storage, attached for the machine's life; beside it a generated network-config for a static NIC, ADR-0114, which holds addresses, not secrets) → built image |
High: guest compromise, lateral movement into provisioned fleet |
| A-3 | VM image artifacts (ISO / raw disk) | OSArtifact PVC, then a vSphere datastore under banlieue-images/; on Cloud Hypervisor, the operator's OCI registry (a single-layer artifact, addressed by digest) and then each storage class's image cache on the host (<storage class>/images/sha256-<hex>.raw, 0750 banlieue), or a file an admin placed there (ADR-0064); and, for a vSphere Template source, the administrator's own template in <rootFolder>/<failure-domain>, which banlieue only reads and clones, never writes or destroys (ADR-0104) |
Critical — a tampered image compromises every VM built from it |
| A-4 | Integrity of the control plane's own decisions | Provider, ProviderClass, VMImage, VMClass CRs |
High — a forged Provider redirects credentials; a forged VMImage redirects the fleet's boot media; a forged or edited VMClass also chooses which segments a VM joins, since a VM gets one NIC per interface its class declares (ADR-0111) |
| A-5 | Released binaries, container images (banlieue, and banlieue-prelay, which the privileged pre-lay Job runs, ADR-0113) and install manifests (install.yaml, clusterctl components; both pin the image by digest, ADR-0103) |
GHCR, GitHub Releases | Critical: downstream supply-chain compromise |
| A-9 | The subject's own credential (a JWT) — the thing a sandbox is handed so it can act as its subject | Never in banlieue. Broker → in-guest agent over mTLS, after attestation (ADR-0049). Not on a disk, not in a CR, not in a hypervisor channel | Critical — it is the subject's identity. Kept out of banlieue entirely, which is why no banlieue compromise discloses it |
| A-8 | Guest readiness marker: a value the guest writes and the provider reads back: /run/banlieue/phase on libvirt, guestinfo.banlieue.phase on vSphere, and on Cloud Hypervisor the phase=installed and (ADR-0112) sealed= lines of its vsock report |
guest tmpfs → qemu-guest-agent → LibvirtMachine.status.guestInstalled, or guest vmware-rpctool → config.extraConfig → VSphereMachineStatus.guestInstalled (ADR-0043; both transports verified against a real backend) |
Low on its own, but it gates pool membership: a guest that can assert it early gets handed out early. Not an integrity signal: see §6/TB-4 and §8 Since ADR-0088 the same channels carry the sealing report of a first-boot sealing guest (/run/banlieue/sealed, guestinfo.banlieue.sealed) into status.sealedPartitions, with the same trust: it can only withhold GuestReady. |
| A-7 | Claim bindings — which subject was given which VM, and when | VirtualMachineClaim.spec.subject + status, mirrored onto the member as banlieue.io/claim-subject-* annotations (ADR-0047) |
Medium — discloses who was using which sandbox to every reader of the namespace; a forged binding makes the record say someone requested a VM they never asked for |
| A-10 | The consumer's cached Kubernetes credential — the ID token kubectl oidc-login writes to disk after a browser flow |
~/.kube/cache/oidc-login on the consumer's own machine, outside every boundary below |
High — it authenticates as that consumer, so it can create claims attributed to them. banlieue has no control here; see §8 |
| A-11 | The Cloud Hypervisor provider's cluster credential — a bound ServiceAccount token, renewed by the provider itself at half-life | /etc/banlieue/credentials/token on the host, beside a kubeconfig that only points at it; directory 0700 banlieue (ADR-0060 Decision 5) |
High — it is the provider's cluster identity: its own Provider and status, every CloudHypervisorMachine in the namespace, and minting further tokens for itself. No Secret access — the operator-built Role (crates/banlieue-operator/src/workload.rs) and deploy/provider-cloud-hypervisor/rbac/clusterrole.yaml — so it discloses no other credential |
| A-12 | Guest disks, seeds and API sockets on a Cloud Hypervisor host | <storage class>/<machine uid>/ (os.raw, seed.iso, serial.log, and install.iso while a Deferred installer is attached) and /run/banlieue/ch/<guest uid>/api.sock, each directory 2770 guest-uid:banlieue |
High — a guest's whole disk, its rendered user-data (A-2) in the seed, and control of its VMM. Encrypted at rest only for a tpmEnabled machine installed Deferred (ADR-0065), which seals to its own vTPM; otherwise plaintext, and tpmEnabled with an Immediate image is refused: ADR-0048's rule, which ADR-0088 keeps on this backend by never scheduling a first-boot sealing VM here |
| A-13 | Registry credentials (ADR-0064) — push: a kubernetes.io/basic-auth Secret in banlieue-imagebuild; pull: username/password files in the host's [registry] credentials_dir (0750 root:banlieue) |
Build namespace; each Cloud Hypervisor host | Push: High — combined with a VMImage status write, it chooses what hosts boot (TB-10). Pull: Medium — reads every pushed image, including cloud-config baked into it (A-2) |
| A-14 | What makes a Cloud Hypervisor host safe to run guests on: the VMM and firmware, the template units, the polkit rule, the host config, the userdb records and directory modes | /opt/banlieue/, /etc/systemd/system/, /etc/polkit-1/rules.d/, /etc/banlieue/, /etc/userdb/, placed by banlieue host cloud-hypervisor install (ADR-0067, ADR-0084) |
Critical: a tampered unit or polkit rule is root on the host, a tampered VMM runs every guest; integrity comes from the banlieue binary (A-5) and a sha256 per file: the pins compiled into it for the pinned release, otherwise the operator's digest flag or GitHub's published digest (ADR-0084) |
| A-16 | Which Proxmox and vSphere VMs banlieue considers its own: on Proxmox the banlieue-machine-uid=<uid> line in a VM's description (ADR-0075 Decision 4); on vSphere the extraConfig key banlieue.machine.uid (ADR-0101) |
Each VM's description on the Proxmox cluster, or its config.extraConfig in vCenter; written by the clone itself on both |
High for integrity: it is what stops a machine adopting, reconfiguring and finally destroying a VM it did not create (a same-named VM in another namespace, or an administrator's own). Not a secret |
| A-6 | vTPM identity and sealed disk-encryption keys | vSphere VM, per-clone (ADR-0039/0040); on libvirt, swtpm state keyed by domain UUID (ADR-0050); on Cloud Hypervisor, swtpm state in <storage class>/<machine uid>/tpm/, owned by the guest's uid, deleted with the machine (ADR-0065) |
High — a shared or surviving TPM identity breaks per-VM disk-encryption isolation With an Immediate image that declares firstBootSealing (ADR-0088) the template carries the sealing policy and no key material or TPM state; each clone's key is born on its first boot against its own vTPM, so nothing sealed is copied. |
| A-6a | vTPM endorsement key certificate: the public anchor a guest's attestation is checked against | vCenter-issued and read host-side on vSphere; swtpm_localca-issued into the vTPM's NVRAM on libvirt, exported by the guest to /run/banlieue/ek.pem and mirrored to VirtualMachineClaim.status (ADR-0045); on Cloud Hypervisor, written by swtpm_setup at manufacture to <state_root>/ek/<machine uid>/ (0700 banlieue) and read host-side (ADR-0065) |
Low confidentiality: it is a public key, deliberately readable by every reader of the claim. Its value is integrity of binding: it must name the VM banlieue actually created, or the consumer verifies an attestation from the wrong machine |
| A-17 | The management cluster's datastore: every Kubernetes object, Secrets included (A-1 hypervisor credentials, A-13 registry credentials, ServiceAccount signing material references), in plaintext unless API-server encryption at rest is configured | etcd on the controllers by default; with ADR-0085's kine, an external SQL database (PostgreSQL on the hypervisor host), reached over the network | Critical: read access is every Secret in the cluster, write access is cluster-admin. With kine it also lives outside every node, so it is backed up, and must be protected, as a database |
| A-18 | VSphereMachine custom attributes — Template, CreatedAt, CreatedBy (ADR-0089), the fleet-auditing fields sre-automations' cf-node tool also writes |
vCenter CustomFieldsManager, per cloned VM; CreatedBy's source is VirtualMachine's banlieue.io/created-by annotation, stamped at CREATE by the optional banlieue-virtualmachine-created-by MutatingAdmissionPolicy |
Zero confidentiality — all three are free text, visible to anyone with vCenter read access. Low integrity: CreatedBy can be forged (TB-1); nothing downstream trusts any of the three for an authorization decision |
| A-19 | Telemetry: Prometheus series on each role's metrics port, and, only when OTEL_EXPORTER_OTLP_ENDPOINT is set, trace spans exported to a collector (ADR-0091, ADR-0092) |
The scraper's storage; the operator's collector and wherever it forwards | Low confidentiality: metric labels carry no object names, namespaces or error text (runner_tests.rs asserts it); spans carry object names, namespaces and backend ids (node, vmid, moref, domain), never credentials, user-data or Secret contents. Its value is integrity for operations: forged or suppressed telemetry hides an outage, it does not change state |
| A-20 | k0s join tokens, and a joining controller's copy of /etc/k0s (ADR-0099). A worker token joins any machine as a worker; a controller token hands over the cluster CA and service-account signing keys, and /etc/k0s carries the datastore password and the VRRP secret |
Minted on a controller by join with --expiry=15m, piped over SSH straight to the joining node (never written on the operator's machine), stored 0600 root at /etc/k0s/{worker,controller}-token-file on that node; /etc/k0s crosses the same kind of pipe |
Worker token: high while valid (a worker runs any pod scheduled to it and reads the Secrets those pods mount). Controller token or /etc/k0s: critical, the whole cluster and its datastore. A token is worthless after expiry; the file stays on the node, like k0sctl's and the vSphere backend's |
| A-21 | CARGO_REGISTRY_TOKEN, the crates.io API token that publishes banlieue-api (ADR-0100) |
A GitHub Actions repository secret, read only by build.yaml's publish-crate job on the release event; never available to a pull request from a fork |
High: whoever holds it can publish any new version of banlieue-api, and every consumer that resolves a semver range takes it on their next build. It cannot replace a published version (uploads are immutable) and holds no cluster or hypervisor access |
| A-22 | Provider liveness: each provider's leader-election Lease, and the status.workload.heartbeat banlieue-operator derives from it (ADR-0102) |
A coordination.k8s.io Lease named after the provider's workload, in its workload namespace; Provider.status.workload |
Medium for integrity and availability: a forged Alive hides a dead provider and keeps VMs scheduling onto it; a forged Stale stops new VMs reaching a healthy one. Not a secret |
| A-15 | EK trust-anchor bundle — the set of CA certificates trusted to issue this backend's vTPM EK certificates | Provider.spec.attestation.ekTrustBundle — inline PEM, or a ConfigMap/Secret it names in the Provider's namespace. Admin-supplied, never discovered; resolved by the broker, never by any banlieue identity (ADR-0049 Decision 10) |
Zero confidentiality — public CA material. High integrity: a rogue CA added here makes every EK certificate that CA forges verify, defeating the whole attestation chain; and on libvirt removing a host's issuer is the revocation mechanism, so an entry an attacker can re-add is a revocation undone |
4. Actors¶
| Actor | Assumed capability | Trusted? |
|---|---|---|
| Cluster admin | Full Kubernetes API | Yes — out of scope by policy (SECURITY.md) |
| Platform admin | Creates ProviderClass, Provider, VMImage |
Semi-trusted — must be treated as infrastructure-admin-equivalent |
| Tenant / VM author | Creates VirtualMachine in a namespace |
Untrusted for confidentiality of A-1/A-2 — see §7 |
| Claim consumer / sandbox broker | Creates VirtualMachineClaims, holding create on them in a namespace |
Bounded by admission: spec.subject.id must equal the authenticated username unless the requester is a declared broker (§7.6). A broker is trusted for attribution by definition. The reference broker is mediatore, which creates each claim from an AgentSandbox it owns (ADR-0097); to banlieue it is one declared broker, whatever its front door |
| Compromised controller pod | RCE inside one banlieue pod | Untrusted |
| Sandbox workload — the AI agent (or anything else) running inside a claimed VM, including one gone hostile via prompt injection | Arbitrary code as root inside its own guest: can write the readiness marker, present or withhold an EK certificate, answer on any port, read the still-attached NoCloud seed | Untrusted, by design — the VM boundary is the isolation model (ADR-0047), and every TB-4/TB-5 control involving the guest assumes it is hostile. What it cannot do from inside: escape its vTPM identity (the EK private key is the one thing it cannot substitute, TB-4), reach another member's claim, rewrite its vSphere ownership marker (vmware-rpctool reaches only guestinfo.*; ADR-0101, not yet verified live, §8), or make banlieue carry a credential to it (A-9) |
| Compromised hypervisor endpoint | Attacker-controlled host reachable at spec.connection.endpoint |
Untrusted |
| Compromised VMM (Cloud Hypervisor) | A guest that has escaped into its VMM process: code execution as that guest's host uid, with kvm and write access to its own two directories |
Untrusted. The unit's sandbox and the per-guest identity are what contain it (TB-9); the provider must never act on anything it can influence without checking |
| Stolen Proxmox API token | Holds A-1 for Proxmox, without the cluster | Untrusted; bounded by the token's ACLs, not the user's: it is privilege-separated and holds only the BanlieueProvider role on /vms, one node, one SDN zone and the ISO storage — it can create and destroy VMs there, and cannot open a console, run commands in a guest, migrate, snapshot, or change permissions (scripts/bootstrap-proxmox-host.sh, ROLE_PRIVS). Its one storage-deleting privilege, Datastore.Allocate, is held only on banlieue-seed, a storage holding nothing but seed ISOs (SEED_ROLE_PRIVS): it cannot delete backups, templates or ISOs elsewhere, nor edit another storage's definition |
| Stolen host credential (Cloud Hypervisor) | Holds A-11 — the provider's ServiceAccount token — without the host | Untrusted; bounded by the provider's namespaced RBAC (TB-8) and the token's lifetime |
| OCI registry and whoever operates it (ADR-0064) | Stores every pushed build; can read, withhold or delete one. With a private CA (ADR-0110), the CA that signs its certificate is trusted by the push Job, so whoever controls that CA can also read the push credential and the artifact in transit | Untrusted for integrity — hosts pull by digest and verify it, so the registry cannot substitute content. Trusted for confidentiality and availability: it sees every image in full (§7.13) |
Bootstrap operator running scripts/bootstrap-k0s-cluster.sh with Cloud Hypervisor nodes (the default VMM) or join (ADR-0099) |
Passwordless sudo on the hypervisor host (CH nodes; only DEFAULT_VMM=libvirt with no CH node avoids it), and an SSH key every management node trusts |
Trusted, like the hypervisor operator. join asks it to hold no new secret: the token goes controller → node over its SSH sessions |
Host operator running banlieue host cloud-hypervisor install |
Root on the host, for one command | Trusted, like the hypervisor operator; the installer's controls protect the host from the provider's user during that run, not from the operator |
| External contributor | Opens a PR from a fork | Untrusted |
Rust consumer of banlieue-api (ADR-0100): a broker such as mediatore, an operator's tooling |
Builds against the published types and calls the API server with its own credential | Neither more nor less trusted than any API client: the crate carries types only (no client, credential or TLS stack), so it grants nothing that RBAC and admission (TB-1, TB-7) do not |
Cluster API's manager (CAPI core, installed by the platform operator; clusterctl init --infrastructure banlieue, ADR-0096) |
Creates, updates and deletes banlieue infrastructure objects it owns through Machines and MachineSets, through the aggregated banlieue-capi-infrastructure ClusterRole |
Semi-trusted, like any controller the platform operator installs. Bounded to infrastructure.banlieue.io machines, templates and clusters: nothing in banlieue.io, no Secrets, IPAM, Events or Leases (TB-15) |
Telemetry consumers: the metrics scraper (from a namespace labelled banlieue.io/monitoring=true) and, if configured, the OTLP collector |
Read A-19; the collector receives spans pushed to it | Trusted for confidentiality of what they receive, untrusted for integrity of anything banlieue does: nothing banlieue decides reads telemetry back (TB-14) |
| OIDC identity provider (and any bridge in front of it, e.g. Dex for GitHub) | Mints the ID tokens the API server accepts, and therefore decides what request.userInfo.username is |
Semi-trusted, and entirely outside banlieue's control. Every guarantee the claim-subject policy makes is downstream of this actor: banlieue checks subject.id against a username it did not derive. Compromise or misconfiguration here makes every claim attribution meaningless — §8 |
| Hypervisor operator | vCenter/libvirt/Proxmox privileges outside Kubernetes; root on a Cloud Hypervisor host | Semi-trusted: can read datastores, storage pools and Proxmox storages banlieue writes to (including a Proxmox machine's seed ISO), can edit or copy the ownership marker on a Proxmox VM, or on a vSphere VM with VirtualMachine.Config.AdvancedConfig (§8), and on libvirt can read swtpm state on the host filesystem. On a Cloud Hypervisor host, root can read every guest's disk, seed and memory, and the provider's cluster token (A-11). Can also plant or replace the template a Template-source image resolves to in a zone's folder, which banlieue trusts by name (ADR-0104, §8) |
5. Trust boundaries¶
┌───────────────────────── TB-7 ───────────────────────────┐
│ OIDC identity provider — external, unmanaged by banlieue │
│ consumer ──▶ browser flow ──▶ signed ID token │
│ token cached on the consumer's laptop (A-10) │
│ API server verifies via JWKS ──▶ userInfo.username │
└────────────────────────────┬─────────────────────────────┘
│ the username every claim's
▼ spec.subject.id is checked against
┌──────────────────────── TB-1 ─────────────────────────┐
tenant namespace │ banlieue-system (restricted PSA) │
┌──────────────┐ │ ┌────────────┐ ┌──────────┐ ┌─────────────────┐ │
│VirtualMachine├──┼─▶│ controller ├──▶│ operator ├──▶│ provider pod │ │
├──────────────┤ │ └─────┬──────┘ └──────────┘ └────────┬────────┘ │
│ Pool / │ │ │ pool fills; a claim BINDS one │ │
│ Claim ├──┼───────▶│ member to a subject and │ │
└──────────────┘ │ │ DESTROYS it on release │ │
│ │ TB-2 (Secret read) │ │
│ ▼ │ TB-4 │
│ ┌─────────┐ │ │
│ │ Secrets │ │ │
│ └─────────┘ │ │
└──────────────────────────────────┬────────┼───────────┘
TB-3 │ │
┌──────────────────────────────────▼──┐ │
│ banlieue-imagebuild (PRIVILEGED PSA) │ │
│ cloud-config Secrets │ Job │ kairos │ │
└───────────────────┬──────────────────┘ │
TB-5 │ ▼
┌───────────────────▼────────────────────────────────────┐
│ Hypervisor — vCenter (HTTPS + creds) │
│ libvirtd (mTLS, native RPC; ADR-0011/0050) │
│ Proxmox VE (HTTPS + API token; ADR-0074) │
│ datastores / storage pools, VMs & domains, vTPM/swtpm │
│ a vSphere VM: one NIC per declared interface (ADR-0111)│
│ guest → qemu-guest-agent → EK cert, phase (read-only) │
└────────────────────────────────────────────────────────┘
TB-6: GitHub Actions / GHCR ──▶ released images & binaries
GitHub Actions ──▶ crates.io ──▶ banlieue-api in consumers' builds
(A-21 token, release job only; ADR-0100)
TB-12: management cluster controllers (kine) ──▶ SQL datastore on the host
(A-17; TLS, password from an untracked env file; ADR-0085)
TB-13: LAN / tailnet clients ──▶ control plane VIP (keepalived, VRRP)
──▶ k0s reverse proxy ──▶ any controller's API server
(TLS passes through; konnectivity off, API servers reach kubelets directly; ADR-0086)
TB-14: metrics scraper (banlieue.io/monitoring ns) ──▶ :8080 /metrics on every role
kubelet ──▶ :8081 /livez /readyz; role ──▶ OTLP collector (opt-in, ADR-0092)
TB-15: CAPI manager ──▶ API server ──▶ infrastructure.banlieue.io objects only
(aggregated banlieue-capi-infrastructure ClusterRole, ADR-0096)
TB-16: bootstrap operator (SSH key) ──▶ controller: k0s token create (15 min)
──pipe──▶ joining node: /etc/k0s/<role>-token-file (0600) ──▶ k0s worker
controller: tar /etc/k0s (no tokens) ──pipe──▶ joining controller ──▶ k0s controller
(never written on the operator's machine; never writes a controller; ADR-0099)
TB-17: provider pod / host ──▶ its own Lease only (namespaced, resourceNames)
operator ──get──▶ that Lease ──▶ Provider.status.workload.heartbeat
controller scheduler ──▶ skips a Stale Provider (ADR-0102)
TB-18: sandbox members (one NIC, static, no default route)
──▶ sandbox segment (layer 2, no router, no DHCP, no resolver)
──mTLS CONNECT──▶ egress gateway VM: sandbox NIC │ egress NIC (ip_forward=0)
──▶ routable network, from the gateway's own address (ADR-0109)
KVM host, management nodes on Cloud Hypervisor (ADR-0099; no banlieue component):
bootstrap operator ──sudo──▶ /etc/systemd/system/k0s-vm-<n>{,-net,-swtpm}.service
systemd ──▶ k0s-vm-<n>-net root oneshot: tap kv-<n> owned by k0s-vm ──▶ bridge
──▶ k0s-vm-<n>-swtpm as k0s-vm, state in CH_VM_DIR/<n>/tpm (0700)
──▶ k0s-vm-<n> VMM as k0s-vm (kvm, no caps), sandbox + Landlock,
writes CH_VM_DIR/<n>/ (0750) and /run/k0s-vm-<n>/ (0700)
(k0s-vm is shared by every management node on the host; banlieue guests'
uids and 2770 dirs are separate from it ─ TB-9)
TB-10: push Job (banlieue-imagebuild, no SA token; public roots + one
operator-named CA bundle key, ADR-0110) ──▶ OCI registry
──▶ banlieue-ch-import@<V>.service on a KVM host (by digest, host-pinned repository)
┌──────────── KVM host running banlieue-provider-cloud-hypervisor ────────────┐
│ │
│ provider (user banlieue, CAP_NET_ADMIN/CHOWN/FOWNER) ──TB-8──▶ API server │
│ │ dials out with a bound SA token (A-11); nothing dials in │
│ │ D-Bus + polkit: start/stop/reset-failed root-owned template instances │
│ ▼ for uids in the guest range only │
│ systemd ──▶ banlieue-ch@<A>.service banlieue-ch@<B>.service │
│ VMM as guest A (uid:gid A) VMM as guest B (uid:gid B) │
│ seccomp · Landlock · sandbox ─ TB-9 ─ seccomp · Landlock │
│ <storage>/<A>/ 2770 A:banlieue <storage>/<B>/ 2770 B:banlieue │
│ tap A ─────────── host bridge ─────────── tap B │
│ guests in banlieue-guests.slice ⊂ banlieue.slice (ADR-0105/6) │
│ │
│ banlieue host cloud-hypervisor install (root, once) ──TB-11──▶ /opt, /etc, │
│ ▲ in banlieue-owned dirs by O_NOFOLLOW units, polkit, userdb, EK CA │
│ └── release assets: github.com or an operator mirror, HTTPS, each file │
│ sha256-verified (compiled pin, operator flag, or api.github.com) │
└─────────────────────────────────────────────────────────────────────────────┘
| ID | Boundary | Crossing |
|---|---|---|
| TB-1 | Tenant namespace → banlieue-system |
A VirtualMachine causes privileged work in another namespace |
| TB-2 | Control-plane pod → Kubernetes Secrets | Credential and user-data reads |
| TB-3 | banlieue-system (restricted) → banlieue-imagebuild (privileged) |
Image builds and per-zone imports. The imagebuilder and provider pods run in banlieue-system; what sits across the boundary is the build inputs (cloud-config Secrets), the import Job and kairos' privileged builder — so both the imagebuilder's cloud-config Role (ADR-0041) and the operator-minted import Role are cross-namespace bindings |
| TB-4 | Cluster → hypervisor | Authenticated API calls carrying A-1 (vCenter, libvirtd, Proxmox VE) |
| TB-5 | Cluster → shared datastore | ISO/disk artifacts written to storage other people can read |
| TB-6 | Contributor / CI → published artifact | Build and release |
| TB-7 | External identity provider → API server | The assertion of who the caller is, on which the whole claim attribution model rests |
| TB-8 | Cloud Hypervisor host ↔ cluster | A process on the hypervisor holds a cluster credential and acts on cluster state; cluster state tells a host what to run |
| TB-9 | Guest VMM → host and other guests | Guest code that escapes into its VMM runs on the host, next to the provider and every other guest |
| TB-10 | Build namespace → OCI registry → Cloud Hypervisor host | A build leaves the cluster for a registry the operator runs, and enters a host that cannot mount cluster storage (ADR-0064) |
| TB-12 | Management cluster controllers → external datastore | With kine (ADR-0085), cluster state leaves the controllers for a database on another host; whoever reaches that database with the credential holds the cluster |
| TB-13 | LAN and tailnet → management cluster control plane VIP | With API_VIP (ADR-0086), every client outside the cluster, kube-proxy and joining workers reach the API through an address that VRRP elects among the controllers, an election anything on the LAN can take part in |
| TB-14 | banlieue pods ↔ telemetry consumers | Inbound scrapes of metrics and probes of health; outbound OTLP spans when configured (ADR-0091 to ADR-0093) |
| TB-15 | CAPI controllers → banlieue infrastructure objects | CAPI's manager creates, owns and deletes infrastructure machines through its aggregated role (ADR-0096) |
| TB-16 | Bootstrap operator → running management cluster (join) |
A join credential (A-20) is minted on a controller and carried to a new node, which then joins a cluster whose controllers the script does not reconfigure; a joining controller also receives a running controller's /etc/k0s (ADR-0099) |
| TB-17 | Provider identity → its Lease → operator → scheduler | Since ADR-0102 a Lease a provider writes is read back as an input to placement: whoever can write it can hide a dead provider or starve a live one |
| TB-18 | Sandbox segment → egress gateway → routable network | Untrusted code on the segment reaches the outside only through a gateway VM, the one component with a NIC on both sides (ADR-0109) |
| TB-11 | Installer (root, once) → Cloud Hypervisor host | A root process writes the host's trust base (A-14), in part inside directories the provider's user owns, from artifacts downloaded from upstream or an operator's mirror (ADR-0067, ADR-0084) |
6. Threats by boundary¶
TB-1 — Tenant namespace → control plane¶
| Threat | STRIDE | Control |
|---|---|---|
A Provider points at an attacker host and ships real credentials to it |
S, I | banlieue-provider-connection VAP: absolute URL, https:// for vsphere/proxmox, no plain http://, no @ userinfo, no # fragment |
| TLS verification silently disabled | T, I | Same VAP: insecureSkipTLSVerify: true requires the explicit, auditable annotation banlieue.io/allow-insecure-tls: "true" |
A Provider names a Secret its author cannot read (confused deputy) |
E, I | banlieue-provider-credentialsref-authorization VAP uses the CEL authorizer — the creating principal must itself be able to get that Secret |
A ProviderClass mints RBAC or places pods in kube-system |
E | banlieue-providerclass-guardrails VAP: additionalRules may not name secrets, *, escalate, bind, impersonate; system namespaces rejected for workloadNamespace |
| Ref-swapping an existing VM onto another class/image | T | banlieue-virtualmachine-immutable-refs, banlieue-provider-immutable-class VAPs |
Resource-exhaustion via absurd specs (numCpus, disk counts) |
D | schemars range/length/maxItems constraints on VMClass, VSphereMachine and ProxmoxMachine (cores, memoryMiB, dataDisks ≤ 30, nics ≤ 16); cross-field rules (a seed needs isoStorage, unique NIC and disk names, VLAN 1–4094) in ProxmoxMachineSpec::validate, called by the controller when it builds the machine — crates/banlieue-api/src/infrastructure/proxmox_machine.rs |
A VirtualMachine names user-data its author cannot read (confused deputy) |
E, I | banlieue-virtualmachine-userdata-authorization VAP (ADR-0042) uses the CEL authorizer: the creating principal must itself be able to get the Secret / ConfigMap named by spec.userData, in the VirtualMachine's own namespace — deploy/admission/virtualmachine-userdata-authorization.yaml |
A tenant's static-address strings (address, gateway, nameservers, domain) add keys to a guest's NoCloud network-config (ADR-0114) |
T | The one renderer, banlieue_provider_sdk::cloudinit::network_config (crates/banlieue-provider-sdk/src/cloudinit/network.rs), single-quotes every value and flattens line breaks, so a value stays one scalar (quotes_in_values_cannot_break_the_document). The author gains nothing new: the same principal already controls the guest's user-data, which the seed runs as root |
| A consumer takes a declared address in status as one the guest answers on (ADR-0114, amendment 2026-11-06) | S, T | A running static machine with no observed address reports its static NICs' declared addresses with addressSource: Static, never as an observation: banlieue_provider_sdk::status::declared_addresses (crates/banlieue-provider-sdk/src/status.rs), applied in libvirt's converge (crates/banlieue-provider-libvirt/src/reconciler/libvirtmachine.rs) and Cloud Hypervisor's build_status (crates/banlieue-provider-cloud-hypervisor/src/machine.rs). Any observed address replaces it, a stopped machine reports none, and the guides say Static means configured, not proven: GuestReady or an observed source is the proof. A tenant cannot use it to show another machine's address: a static address already held by another VM is refused before any infra CR exists (address_conflict.rs, ADR-0083, the row below) |
Rendered user-data is readable from VSphereMachine.spec / LibvirtMachine.spec |
I | No code control — this is the accepted reflection of ADR-0025. See §7.1 and §8 |
| Two claims bind the same warm member, so one VM is handed to two subjects | I, E | The bind is a JSON merge patch carrying the member's resourceVersion, so a member written since the snapshot is rejected 409 and the loser re-picks — crates/banlieue-controller/src/reconciler/claim.rs. A member already carrying banlieue.io/claim reads as MemberPhase::Claimed (reconciler/pool.rs::member_view) and pick_member filters to Ready only (reconciler/claim_plan.rs) |
| One claim binds two warm members, so one subject drains the pool's warm capacity and holds a sandbox nobody tracks | D, E | Before picking, the reconciler looks in the fresh member list for a member already labelled for this claim and finishes that bind instead — reconciler/claim.rs::pick, reconciler/claim_plan.rs::member_bound_to. Found live on 2026-09-28 (a reconcile from a stale cached claim); regression test tests/live_claim.rs::a_stale_copy_of_a_bound_claim_does_not_bind_a_second_member against a real API server |
| A released member is recycled to a second subject | I | Release is always deletion: claim_plan.rs::next_step has no transition back to an unclaimed state, and pool_plan's invariants 1–2 keep the pool from reclaiming a labelled member. The VM is the isolation boundary, so reuse is the one outcome the design must exclude (ADR-0047) |
A stale-image member is handed to a fresh claim after the pool's VMImage moved on — an old userland, old agent, old patch level |
T | Bounded, not prevented: pick_member prefers the freshest image revision and falls back to a stale one only when nothing fresh is Ready (crates/banlieue-controller/src/reconciler/claim_plan.rs); spec.maxIdleSeconds reaps members that sat unclaimed too long (reconciler/pool_plan.rs, invariant 5); the mandatory claim TTL bounds how long a stale member lives once bound; and a pool rollout replaces unclaimed stale members while holding available ≥ warmReplicas (ADR-0046). A claim that must never get a stale member is a claim the pool should refuse instead — not modelled today |
| A claim attributes a sandbox to a subject that never requested one | S, R | banlieue-virtualmachineclaim-subject-authorization VAP (ADR-0047 Decision 10): spec.subject.id must equal the authenticated username, subject.issuer must be in an operator allowlist, and spec is immutable so the check cannot be undone by a later patch — deploy/admission/virtualmachineclaim-subject-authorization.yaml. Declared brokers are exempt from the id check by design (§7.6) |
A credential is written into spec.subject, which is world-readable in the namespace and copied onto the member |
I | Partly controlled: subject.id must now equal the authenticated username, so it cannot be an arbitrary string, and issuer is allowlisted. Neither stops a determined author from putting a secret in a field shaped like a username — banlieue never reads it as a credential and never forwards it to a guest, but nothing rejects one. See §7.6, §7.10 |
delete virtualmachineclaims destroys running VMs |
D | Equivalent to delete virtualmachines by design — releasing a claim is destroying the sandbox. RBAC is the only control; §7.10 |
The controller's patch on virtualmachineclaims is used to rewrite a claim's spec (its pool, subject or TTL) rather than only its finalizer |
T, E | Bounded: the reconciler's only writes to the claim are metadata.finalizers (banlieue-provider-sdk/src/finalizer.rs) and status; the claim's spec is immutable at admission (deploy/admission/virtualmachineclaim-subject-authorization.yaml), so a patch from any principal cannot change it. The grant exists because a finalizer is a merge patch on the object itself (deploy/controller/rbac/clusterrole.yaml; bootstrap_tests.rs::the_controller_may_patch_every_kind_it_finalizes) |
User-influenced strings (domainName, pool, disk/volume names) injected into libvirt domain XML |
T, E | Every value is escaped on the way in by esc() — all five XML entities, uniformly in text and attributes, so there is no context-dependent rule to get wrong — crates/banlieue-provider-libvirt/src/xml/escape.rs, applied throughout xml/domain.rs; both have dedicated _tests.rs |
A rogue CA is slipped into attestation.ekTrustBundle (A-15) — on the Provider spec, or by editing the ConfigMap/Secret it references — so an attacker-forged EK certificate verifies |
S, T | Bounded, not prevented. The banlieue-provider-attestation-ektrustbundle VAP validates shape (exactly one of inline/configMapRef/secretRef — deploy/admission/provider-attestation-ektrustbundle.yaml); content cannot be validated by banlieue, which has no idea which CAs are legitimate (ADR-0049 Decision 10 makes that an explicit admin assertion, the same posture as capabilities.features). Provider writes are platform-admin-only (§7.5), and the referenced object lives in the Provider's own namespace, out of tenant reach. Custody of that object is §7.14 |
Provider.spec.scheduling.cpuOvercommitRatio is set to disable the CPU fit or make it meaningless (ADR-0107) |
T, D | The CRD schema bounds it to 1 to 64 (deploy/crds/banlieue.io_providers.yaml, from MIN_/MAX_CPU_OVERCOMMIT_RATIO in crates/banlieue-api/src/banlieue/provider.rs, tested the_crd_bounds_the_cpu_overcommit_ratio), and Provider writes are platform-admin-only (§7.5). Memory has no ratio: it is never overcommitted |
A VirtualMachine declares another VM's address in spec.networkOverrides, by mistake or to intercept traffic meant for it and knock it off the network (ARP contention on the shared segment) |
S, T, I, D | ADR-0083: banlieue-controller blocks the later claimant on the same networkClass before scheduling: Ready=False reason=DuplicateAddress, and no infra CR for a VM never provisioned, so the address never reaches a provider. A VM that holds the address (status.heldAddresses, written by the controller only when it applies the infra CR) keeps it whatever its age, including while its guest reboots and on vSphere, which reports no guest addresses; a running VM edited onto a held address is blocked and its infra CR is not re-applied. crates/banlieue-controller/src/reconciler/address_conflict.rs (find_duplicate_address), called from reconciler/virtualmachine.rs before schedule; unit tests in address_conflict_tests.rs, reconcile wiring in tests/live_duplicate_address.rs against a real API server. Banlieue VMs and declared addresses only: a machine banlieue did not create is not seen (§9), and root in a guest can still configure any address on its NIC, which is the existing no-anti-spoofing residue in §8. Every declared interface is checked (ADR-0111): declared_claims walks all of networkOverrides, so a second NIC's address is held to the same (networkClass, address) rule as the first |
| The duplicate-address condition message discloses another namespace's VM: its name, namespace, or that it exists | I | The message names a holder as namespace/name only when it shares the blocked VM's namespace; otherwise it says "a VirtualMachine in another namespace" (address_conflict.rs (Holder::OtherNamespace), tested by collision_across_namespaces_is_detected_but_holder_is_not_named and tests/live_duplicate_address.rs::holder_in_another_namespace_blocks_without_being_named). The residual one-bit oracle ("is this address in use on this networkClass?") is §8 |
A principal forges status.heldAddresses to reserve addresses it was never given, or to release one a running guest still uses |
T, D | Bounded by RBAC, not prevented. Only the controller writes VM status; writing it takes update/patch on virtualmachines/status, which no tenant role needs and which is already infrastructure-admin-equivalent (it can rewrite status.infrastructureRef and every mirrored field). The controller rewrites the field from its own spec on every successful apply, so a forgery lasts until the next one. deploy/controller/rbac/clusterrole.yaml is the only shipped grant |
A VirtualMachine's banlieue.io/created-by annotation is forged, so the vCenter CreatedBy custom attribute later misattributes who built a VM (ADR-0089) |
S | Not prevented on UPDATE, and not prevented at all on a cluster without the policy — both accepted by design (ADR-0089 Decision 2). On CREATE where banlieue-virtualmachine-created-by applies, the MutatingAdmissionPolicy's ApplyConfiguration merge always sets the annotation to request.userInfo.username, overwriting any value the creator supplied in the same request — a self-set value on CREATE does not survive. But the policy matches operations: ["CREATE"] only, so a later UPDATE by any principal holding update on the VirtualMachine can edit or remove it uncorrected; and a cluster that never applies the policy (it needs Kubernetes 1.36+, §7) gets no stamping at all, leaving the annotation fully attacker/author-controlled. Unlike VirtualMachineClaim.spec.subject (the claim-attribution row above), created-by carries no authorization weight — nothing in banlieue reads it to decide who may do what — so a forged value only misleads an operator reading vCenter's fleet-audit attributes, the same exposure already accepted for the vCenter annotation field other tooling writes |
A permission or connectivity fault on one infra backend (vSphere/libvirt/cloud-hypervisor/Proxmox) blocks the VirtualMachine deletion cascade for every backend, holding every pending VirtualMachine delete — including ones with no relationship to the broken backend — Terminating indefinitely (ADR-0090) |
D | Fixed, not merely bounded. Found live: a ClusterRole missing proxmoxmachines returned 403 from apis.proxmox.get_opt, and the old sequential ?-chained lookup in finalize_vm aborted before the vSphere branch below it ran, even though that vSphere lookup had already succeeded — the finalizer stayed on and the backend VM in vCenter was never destroyed. finalize_vm now resolves each backend kind independently (infra_state) into InfraState::{Absent,Present,Terminating,Unknown}, and the pure plan_finalize decides per kind: a lookup or delete failure on one kind (Unknown) never blocks another kind's Present delete, and never lets the finalizer come off either — crates/banlieue-controller/src/reconciler/virtualmachine.rs, unit-tested in virtualmachine_tests.rs |
| A pool's whole warm set sits on one host, because placement took the first eligible failure domain, so losing that host loses every warm member at once (found live on Cloud Hypervisor, 2026-10-09) | D | ADR-0098: schedule() orders eligible domains by load of the VM's pool, then total load, then a rendezvous hash, so members spread and refills go where the pool is short (crates/banlieue-controller/src/reconciler/scheduler.rs, placement_load; tests in scheduler_tests.rs). Load comes only from status.scheduled of VMs in the same namespace: a principal who can create VMs there can push later placements away from a domain, which is within what that principal can already do. Concurrent members that cannot see each other can still collide; the next placement corrects it |
TB-2 — Pods → Secrets¶
The design principle is: no banlieue identity holds cluster-wide Secret access, and each identity reads only the objects it can name.
| Control | Where |
|---|---|
Provider ClusterRoles grant zero Secret/ConfigMap access |
deploy/provider-{vsphere,libvirt}/rbac/clusterrole.yaml |
Per-Provider Role is resourceNames-scoped to exactly that Provider's credentials Secret (+ CA bundle if named) |
crates/banlieue-operator/src/workload.rs (build_import_role, named_rule) |
An empty resourceNames list is treated as a bug, not a default — the CA-ConfigMap rule is emitted only when the Provider names one |
same |
banlieue-controller's ClusterRole carries no Secret rule at all; its Secret/ConfigMap get is a namespaced Role, which banlieue bootstrap operator emits from the same file (it did not before 2026-10-09, so no VirtualMachine with user-data scheduled on a bootstrap install). The Role is un-scoped by name, so it is safe only with ADR-0042's VAP applied (deploy/admission/, a separate step) |
deploy/controller/rbac/clusterrole.yaml, deploy/controller/rbac/role.yaml, crates/banlieue-operator/src/bootstrap.rs (build_controller_userdata_role), tested in bootstrap_tests.rs |
banlieue-imagebuilder's cloud-config Secret access is a namespaced Role in the build namespace, not a ClusterRole rule (ADR-0041) |
deploy/imagebuilder/rbac/role.yaml |
banlieue-imagebuilder's ClusterRole reads persistentvolumes (get) and nodes (list) and writes neither; a unit test pins the exact verbs (ADR-0115) |
deploy/imagebuilder/rbac/clusterrole.yaml, crates/banlieue-operator/src/bootstrap_tests.rs |
No ClusterRole grants Secret access except banlieue-operator's deliberate get — held only so RBAC's escalation-prevention permits it to delegate that verb into a per-Provider resourceNames-scoped Role, and never exercised by the operator itself (SEC-007, accepted-with-monitoring) |
deploy/operator/rbac/clusterrole.yaml |
That invariant is mechanically enforced, not merely checkable: a unit test fails the build if any other ClusterRole grants secrets/configmaps |
crates/banlieue-operator/src/bootstrap_tests.rs (only_the_operator_cluster_role_may_grant_secret_access) |
The CLI install path emits the same namespaced RBAC as the manifests, so banlieue bootstrap and GitOps cannot drift apart |
crates/banlieue-operator/src/bootstrap.rs (build_cloud_config_role) |
Credentials has a hand-written redacting Debug |
crates/banlieue-provider-vsphere/src/client/mod.rs |
TlsIdentity likewise — the libvirt credential is the client private key, so client_key_pem renders as <redacted> and the public CA/cert halves render as byte counts. A regression test asserts the key never reaches {:?} |
crates/banlieue-libvirt/src/transport.rs, transport_tests.rs (tls_identity_debug_redacts_the_private_key) |
The Proxmox provider ClusterRole grants no Secret or ConfigMap access, no create and no delete on proxmoxmachines; the API token and CA bundle are read by name through the operator's per-Provider resourceNames Role — deploy/provider-proxmox/rbac/clusterrole.yaml, crates/banlieue-operator/src/bootstrap.rs |
|
ApiToken and ClientConfig have hand-written redacting Debug; the token is validated (user@realm!tokenid, no whitespace in the secret) before any request, so a mis-pasted value is reported as such and never echoed. Regression tests assert the secret never reaches {:?} |
crates/banlieue-proxmox/src/token.rs, client.rs (token_tests.rs, client_tests.rs) |
Rotating a credential needs no Secret list/watch: each provider reads its one Secret by name on every reconcile and builds a fresh client from it, so a rotation lands within the ~30 s requeue (ADR-0095). Granting a watch would have meant read access to every Secret in the namespace. Regression tests pin each production client factory as stateless |
crates/banlieue-provider-{vsphere,proxmox,libvirt}/src/client* (production_factory_holds_no_client_between_reconciles) |
The libvirt provider ClusterRole grants no create and no delete on libvirtmachines — the controller owns their lifecycle; a compromised provider cannot mint machines the scheduler never placed |
deploy/provider-libvirt/rbac/clusterrole.yaml |
TB-3 — Restricted → privileged namespace¶
banlieue-imagebuild enforces Pod Security Admission privileged because
kairos' builder needs loop devices, mount, and chroot. This is deliberate
and isolated (ADR-0010, ADR-0016), but it has a consequence operators must
internalise:
Any principal granted pod-create in
banlieue-imagebuildis effectively node root, and can assume any ServiceAccount in that namespace. Treat every RoleBinding there as a node-root-equivalent grant and review them accordingly.
The banlieue-import identity is deliberately not the provider controller's
own identity (which can create Jobs), and starts with zero permissions;
banlieue-operator grants it narrowly-scoped, per-Provider read access. See
§7 for the hardening this still requires.
Since ADR-0064 the imagebuilder itself creates a Job here: the registry
push Job. Its build-namespace Role therefore holds jobs
get/create/patch/delete and pods list (deploy/imagebuilder/rbac/role.yaml,
crates/banlieue-operator/src/bootstrap.rs::build_cloud_config_role). By the
statement above, that makes the imagebuilder's identity node-root-equivalent
in this namespace. It already was in effect — it creates OSArtifacts,
which kairos turns into privileged pods — but the grant is now direct. §8.
The push Job it creates runs with no ServiceAccount token, non-root, with a
read-only root filesystem and all capabilities dropped.
ADR-0115: a lost volume is rebuilt, not trusted. On node-local storage an
artifacts PVC can stay Bound and its OSArtifact Ready after the worker's
disk was replaced, so the build namespace would keep reporting an image whose
file is gone (an availability failure, and a status that lies). The imagebuilder
now reads the bound PV and the node it is pinned to, and deletes and rebuilds an
OSArtifact whose node was replaced after the PV was made
(crates/banlieue-imagebuilder/src/reconciler/vmimage.rs: pinned_hostname,
volume_lost, artifact_gate). Its identity gains get on persistentvolumes
and list on nodes (deploy/imagebuilder/rbac/clusterrole.yaml, pinned by
the_imagebuilder_cluster_role_reads_volumes_and_nodes_and_writes_neither):
read-only, cluster-scoped, no Secret content, no new write. The delete it
performs is the one SEC-005 already grants. A failed read never deletes
(a missing grant degrades to the old behaviour). STRIDE: D/T (a node that
re-registers makes a needless rebuild, bounded, §8); I (node and PV names
and timestamps are visible to the identity, the same facts kubectl get nodes
shows any cluster reader).
TB-4 — Cluster → hypervisor¶
| Threat | Control |
|---|---|
| MITM / attacker-presented certificate | banlieue owns the reqwest client and honours connection.caBundle (ADR-0008); CA source validated by banlieue-provider-cabundle-source VAP |
| Hostile or unresponsive endpoint stalls every reconcile | 10 s connect / 120 s request timeouts on the vSphere client; timeouts on libvirt connect, recv, and Session::send |
| Malformed libvirt RPC frames | Wire decoder is continuously fuzzed (crates/banlieue-libvirt/fuzz, .github/workflows/fuzz.yaml, ClusterFuzzLite); ADR-0050's domain decode_* halves are pure and unit-tested against captured wire bytes, so they are in that fuzz surface too |
| A compromised provider pod uses its network reach to scan or attack hosts other than its backend | Opt-in deploy/network-policies/ (ADR-0094): default deny, then egress only to DNS, the API server and the provider's own backend port (443 vCenter, 8006 Proxmox, 16514 libvirtd), selected by app.kubernetes.io/component so operator-spawned providers in any namespace match. Destinations stay "any address" until the site narrows them (§7.21, §8); crates/banlieue-operator/tests/network_policies.rs pins the port set against the labels the operator stamps |
| An operator quarantines or holds a VM by powering it off in vCenter, and banlieue starts it again | The provider drives every provisioned vSphere VM to spec.desiredPowerState on each pass (ADR-0034 amendment 2026-10-23: crates/banlieue-provider-vsphere/src/reconciler/vspheremachine.rs, converge_power_state), as libvirt, Proxmox and Cloud Hypervisor already do. The supported hold is desiredPowerState: PoweredOff on the VirtualMachine (the provider then holds it off), or deleting it; both are Kubernetes writes, gated by RBAC on VirtualMachine. A vCenter-side power-off is not a hold. Recorded in §8 |
| Backend call tracing (ADR-0092) leaks hypervisor credentials or guest bootstrap material into a span | Every backend span is skip_all and records only identifiers (node, vmid, moref, pool, volume, domain): crates/banlieue-proxmox/src/client.rs, crates/banlieue-libvirt/src/procs.rs, crates/banlieue-cloud-hypervisor/src/client.rs, crates/banlieue-provider-vsphere/src/client/vim.rs. Export is off unless OTEL_EXPORTER_OTLP_ENDPOINT is set |
| A deleted VM leaves its sealed-key material behind (swtpm state, UEFI NVRAM varstore) | domain_undefine takes no flags parameter and unconditionally sends MANAGED_SAVE\|NVRAM\|TPM (ADR-0050 Decision 5) — the flag cannot be forgotten at a call site — crates/banlieue-libvirt/src/procs.rs, proven against a real libvirtd in tests/live_libvirtd.rs. Live since ADR-0050: LibvirtMachine's finalizer calls it on every teardown — crates/banlieue-provider-libvirt/src/machine_client.rs (undefine), invoked from reconciler/libvirtmachine.rs::finalize_backend, which then verifies the domain is actually gone before deleting its volumes |
| Something other than the intended guest answers the broker's mTLS connection and receives the subject's token | S |
| A token minted for a different service is presented to the agent and accepted | S |
| A claim names an attacker-controlled issuer, so the agent fetches that attacker's JWKS and every forged token verifies | S, T |
A guest asserts GuestReady while still installing, or a compromised guest asserts it to be handed out sooner |
S, T |
A guest returns a huge or malformed payload to guest-file-read, hoping to fault the provider's reconcile loop |
D |
| A guest reports another member's EK certificate, so a verifier later checks an attestation against the wrong machine | S, T |
| A guest reports partitions sealed that are not, to be handed out (or kept) as encrypted | S, T |
| A guest returns a huge or malformed sealing report, hoping to fault the reconcile or smuggle text into status | D, T |
A VMImage author smuggles shell into the sealing report stage through a partition name, which runs as root on every boot of every clone |
E, T |
| A guest publishes bytes that are not a certificate at all, into a status field consumers feed to a certificate library | T |
| banlieue's read of the certificate becomes a way to run code inside a sandbox | E |
| A half-failed teardown silently leaves domains defined | libvirt 11.3 fails undefine on a UEFI domain without NVRAM rather than warning; the error is returned, never swallowed, and the live lifecycle test asserts the domain is actually gone — crates/banlieue-libvirt/tests/live_libvirtd.rs |
| Credentials leak into logs | No secret is ever logged; redacting Debug; provider condition messages are the only verbatim text mirrored to user-facing status |
A provider workload acts on another Provider's state: reads its credentials Secret, overwrites or erases its VMImage.status.perProvider row, or (vSphere) blocks or performs the deletion of its zone templates |
ADR-0087, Amendment 2026-11-01. A workload started with --provider-name reconciles only that Provider's rows (providers_in_scope, crates/banlieue-provider-sdk/src/reconciler.rs) and writes them under its own field manager (status_field_manager, ssa.rs), on libvirt (reconciler/vmimage.rs), Proxmox (reconciler/image.rs) and vSphere (reconciler/vmimage.rs); its operator-rendered Role grants get on its own credentials Secret only, so reaching another's is refused by the API server, not just avoided. vSphere's delete destroys only the workload's own templates, clears its own row's zones, and removes the shared finalizer once no existing Provider lists templates (rows_to_clean, with_zones_cleared, finalizer_can_go, tested in vmimage_finalize_tests.rs). Found live 2026-11-01 (libvirt, v0.4.9, rolled back); vSphere and Proxmox were latent |
vSphere VM ownership (ADR-0024, ADR-0026, ADR-0101)¶
status.vmRef is a cache written at the end of a successful pass; the VM's
own marker is what decides ownership when the cache is empty or wrong. A pass
reads the machine live from the API server and decides from that, never from
the controller's watch cache (ADR-0101, amended 2026-11-04).
| Threat | STRIDE | Control |
|---|---|---|
| A machine adopts or destroys a VM banlieue did not create (an administrator's VM of the same name, another namespace's machine in the same folder) | T, E, D | Ownership is the machine's UID in extraConfig banlieue.machine.uid, written by CloneVM_Task in the same CloneSpec as the guestinfo keys, so no banlieue VM exists unmarked (A-16). The name lookup runs only when status.vmRef is unset and only over the direct children of the clone's own target folder; a VM without a matching UID is never destroyed, and a create that meets one reports VMNameConflict instead of cloning. A recorded vmRef is still destroyed by moref, as ADR-0026 decided. crates/banlieue-provider-vsphere/src/reconciler/vspheremachine.rs (reclaim_or_refuse, finalize_unrecorded), client/vim.rs (find_vms_named); tested against a fake that refuses DuplicateName per folder, as vCenter does |
A pass that loses its moref (a fault after the clone, a provider crash before the status write, a machine deleted meanwhile) strands a VM that no object accounts for, still running with the machine's user-data (A-2), or wedges the machine on DuplicateName |
D, I | A fault in any step after the clone destroys it (finish_clone, roll_back_clone); a crash leaves a marked VM that the next pass destroys and clones afresh; deletion with no vmRef destroys it (finalize_unrecorded). If the Provider is already gone and nothing is recorded, the finalizer is released with a warning rather than blocking deletion forever. Same files |
| A pass destroys the VM it has just built, or records a VM that no longer exists, because it decided from a watch-cache snapshot older than its own status write (seen 2026-11-04: every OpenBao VM rolled onto a new image, 13 ms between passes) | T, D | A pass reads the VSphereMachine live (live_base, ApiMachineSource: an uncached GET) after the cheap serves check and uses that object for the deletion branch, the route (route) and the base of every status write, so Decision 3's destroy only runs when the API server itself holds no vmRef, and no write regresses a newer one. A live object that is gone, or has another UID, ends the pass. crates/banlieue-provider-vsphere/src/reconciler/vspheremachine.rs; regression tests a_second_pass_from_a_stale_cache_does_not_destroy_the_vm_the_first_pass_built and a_pass_builds_on_the_live_status_not_the_stale_cached_one in vspheremachine_live_tests.rs. Residual: the live read and the write that follows are not atomic, but the provider runs one pass per object and owns status, so nothing else writes it between them. The other providers' create paths have not been checked for the same pattern (ADR-0101 Consequences) |
A recorded vmRef names a VM that is gone, and recovering it by marker adopts a VM that is not the machine's |
T, E | Adoption needs the exact banlieue.machine.uid match (Decisions 1 and 2), in the machine's own folder, under its own name, and exactly one such VM (find_own_clone): none reports BackendMissing, several report it naming the morefs and adopt none, and a failed lookup is an error that adopts nothing. The adopted moref is written into status with every other field kept (status_adopting). A user with VirtualMachine.Config.AdvancedConfig can plant a UID on a same-named VM, which is the §8 marker risk, now also meaning that VM can be adopted. Same files, same tests |
| A compromised guest forges, copies or erases the marker, to pass its VM off as another machine's or to orphan itself | S, T | The key is outside guestinfo.*, the only extraConfig namespace a guest reaches through vmware-rpctool; the guest's own writes (guestinfo.banlieue.phase, A-8) never touch it, and a machine's UID is assigned by the API server, not by whoever writes the VirtualMachine. Not yet verified against a real vCenter (§8) |
| Two namespaces' machines of the same name, placed in one vCenter folder, collide | D | The second reports VMNameConflict; it never adopts or destroys the first one's VM. Making VM names unique across namespaces is undecided (ADR-0101 Consequences), so the denial is bounded, not prevented |
vSphere VM network adapters (ADR-0111)¶
A cloned VM gets one NIC per interface its VMClass declares. The guest's
addressing for each reaches it as guestinfo.network.<i>.*.
| Threat | STRIDE | Control |
|---|---|---|
| A VM author puts a VM on a segment they should not reach, by declaring another interface | E | A tenant cannot add a NIC. The interface list is the VMClass's (cluster-scoped, platform-admin, §7.5), and VirtualMachine.spec.networkOverrides carries only a static address keyed by an interface the class already declares: NetworkInterfaceOverride has name and static, no networkClass (crates/banlieue-api/src/banlieue/virtualmachine.rs). The port groups that exist at all are the ones the Provider advertises in networkClasses (§7.17) |
| A second interface's address duplicates another VM's, to intercept its traffic or knock it off the segment | T, D | The ADR-0083 check covers every interface, not the first: declared_claims iterates all of networkOverrides and keys on (networkClass, address) (crates/banlieue-controller/src/reconciler/address_conflict.rs). It only means something for a later NIC now that one exists |
| A clone is left half built: the first NIC moved, a later one missing or on the wrong network | T, D | Every declared interface's port group is resolved from the target cluster before CloneVM_Task, and one that is not reachable stops the clone and names the interface (ensure_vm, reconciler/vspheremachine.rs; an_unreachable_later_interface_clones_nothing_and_names_the_port_group). All NIC changes are in the one CloneVM_Task (build_clone_nic_changes), so the clone exists with all of them or not at all |
A guest rewrites its own guestinfo.network.<i>.* to confuse its own configuration |
T | Nothing reads them back: banlieue writes them once, in the clone's extraConfig, and the guest reads them at boot. The keys hold addresses, prefixes and names, never credentials (A-2 is guestinfo.userdata, unchanged), and the ownership marker is outside guestinfo.* (above) |
| A VM on two segments forwards between them, defeating an isolated segment | E | Not prevented by banlieue, which cannot see inside a guest. Recorded as an accepted risk in §8 and asked of the operator in §7.28 |
vSphere adopted templates (ADR-0020, ADR-0028, ADR-0104)¶
A Template-kind source adopts a template that already exists in vCenter.
banlieue looks it up in each failure domain's own folder and clones from it;
it never builds, writes or destroys it. A Url source is the opposite: the
per-zone import Job creates the template, so VMImage deletion may destroy it.
| Threat | STRIDE | Control |
|---|---|---|
Deleting a VMImage destroys an administrator's own template: per-zone status on a Template row would otherwise feed finalize, which destroys every template listed in zones[] |
T, D | Ownership is decided on the source kind, not on status: image_owns_templates is true only for a vsphere Url source, finalize skips the vCenter connection and every destroy for anything else, and destroy_owned_zone_templates re-checks it before destroy_zone_templates, so a Template row cannot become destroyable by what zones[] contains. The other mutating paths (--force-create's destroy_if_present, --force-upload's datastore delete, import_iso_template) run only inside the import Job, which reconcile_for_provider never creates for a Template source; spec.template.retainOnDelete and the forceCreate/forceUpload knobs are ignored for it. crates/banlieue-provider-vsphere/src/reconciler/vmimage.rs; regression tests vmimage_finalize_tests.rs (template_kind_image_never_destroys_*) against a fake that holds the template in the zone's folder |
| A same-named template planted elsewhere in the datacenter is cloned instead of the intended one (the old lookup took the first match anywhere in the datacenter) | S, T | The lookup is confined to <rootFolder>/<failure-domain> of that zone's own datacenter (template_zone_status; client/vim.rs::find_template roots its view at that folder), so planting one needs write access to that specific folder, and the fake refuses a cross-datacenter folder match as vCenter does (client/fake.rs). Whoever can write there is the hypervisor operator (§4); the template is still trusted by name (§8) |
A forged status.perProvider[].zones[] steers VMs into a zone, or marks a missing template ready |
T | Bounded by RBAC, not prevented. The provider's field manager writes vmimages/status, a grant no tenant role carries and already infrastructure-admin-equivalent (the status.heldAddresses row in TB-1). The scheduler's per-zone gate (reconciler/scheduler.rs::image_ready_in_zone) can only narrow placement: candidates still come from Provider.status.failureDomains[]. A clone into a zone whose template is absent fails ProvisionFailed (exact-name lookup in that folder), never clones a different template |
Per-zone lookups multiply the vCenter work of every VMImage reconcile |
D | One folder-scoped view per failure domain, each narrower than the old datacenter-wide walk; the client's 10 s connect / 120 s request timeouts apply; an error in one zone reports LookupFailed on that zone only and leaves the others' status intact — template_zone_status. No rate limit beyond the requeue interval |
Proxmox VE (ADR-0074, ADR-0075)¶
The same boundary, with a credential shaped differently: an API token rather than a password or a client certificate, and a protocol whose errors live in the HTTP reason phrase and whose mutations are asynchronous tasks.
| Threat | STRIDE | Control |
|---|---|---|
| MITM / attacker-presented certificate, so the token is sent to the wrong host | S, I | TLS is verified by default against connection.caBundle (the node's own CA, written by the bootstrap script) or the system roots; a bundle that does not parse is a configuration error, never a silent fallback. The endpoint must be https:// (banlieue-provider-connection VAP, which names proxmox); insecureSkipTLSVerify needs the audited annotation — crates/banlieue-proxmox/src/client.rs (Client::new), deploy/admission/provider-connection.yaml |
A redirect carries the Authorization header to another host |
I | Redirects are never followed (redirect::Policy::none()); a 3xx is an error — client.rs, test redirects_are_not_followed_so_the_token_stays_put |
| A hostile or wedged endpoint stalls every reconcile | D | 10 s connect and 30 s whole-request timeouts on every call; every task wait is bounded, and a task still running at the deadline is TaskTimeout, never a silent return — client.rs (DEFAULT_*_TIMEOUT), api.rs (wait_task_with) |
| A hostile endpoint returns an unbounded response body | D | The body is read through a hard cap and refused beyond it — client.rs (MAX_RESPONSE_BYTES), test an_oversized_response_is_refused |
| A stolen or over-privileged token | E | The token is privilege-separated and the role holds only what ADR-0074 Decision 4 lists, on /vms, one node, one SDN zone and the ISO storage. Absent by design: console, guest exec, migrate, snapshot, backup, Sys.Modify, Permissions.Modify. Revocation is pveum user token remove and takes effect immediately. Deleting a seed needs Datastore.Allocate — on a shared storage that also deletes backups and templates and edits the storage definition — so it is a separate BanlieueSeed role granted on the dedicated banlieue-seed storage alone (ADR-0074 Decision 4, amended after the first live run found the missing privilege). The privilege list is a contract: a client change that needs a new privilege must update the role, the script, the guide and this table together — scripts/bootstrap-proxmox-host.sh |
| A machine adopts, reconfigures or destroys a VM banlieue did not create (a same-named VM in another namespace, an administrator's own VM) | T, E, D | Ownership is the machine's UID, carried in the VM description by the clone itself (A-16), and every adopt and delete checks it; a VM that shares only the name is never touched — crates/banlieue-provider-proxmox/src/reconciler/proxmoxmachine.rs (allocate_vmid, finalize_backend) |
| Two controllers allocate the same VMID | T | Proxmox refuses the second clone onto an existing id; the provider records status.vmid before cloning, abandons an id whose VM is not its own, and retries — proxmoxmachine.rs (allocate_vmid) |
| A hostile node, storage or volume name injects a path component or a header | T | Every path segment is percent-encoded, .. and / included (wire.rs::encode_segment); an upload filename containing a quote, CR/LF, /, \ or a leading dot is refused before any request (client.rs::check_filename) |
A Template source names a VMID that is a running guest, and banlieue clones it |
T, I | The provider verifies template=1 before publishing the image and again before cloning, and fails naming the VMID — reconciler/image.rs, proxmoxmachine.rs |
| A half-failed delete leaves the VM or its seed ISO behind | I | The finalizer is released only after the VM is destroyed (with its unreferenced disks) and the seed volume is deleted; "already gone" is success at each step and every other failure keeps the finalizer — proxmoxmachine.rs (finalize_backend), tested for orphaned volumes against the fake |
| A Proxmox error message, which the hypervisor authors, is mirrored onto user-facing status | I | Same posture as §8's verbatim condition messages; the message is the reason phrase plus parameter names, and never the request body or the token — wire.rs::api_message |
| Proxmox's wire behaviour differs from what the client assumes | — | The first live lifecycle run (2026-09-30) found two drifts the fake had hidden, both fixed with the fake corrected in the same change: resize is a task on PVE 9 (it was treated as synchronous, so a start could race it), and seed deletion needs Datastore.Allocate (above). Partly verified live (2026-09-28): the read-only tier passed against a real node — verified TLS against the node CA, token auth, inventory decoding, and Proxmox's reason phrase surfacing on a 401. The lifecycle tier then passed live (2026-09-30): clone, resize, upload, start/stop/delete and UPID polling. Still unverified live: the Kubernetes-side reconcile loop (the provider binary in a cluster), guest-agent addresses, EFI/vTPM, and a failed task's exit status. The fake refuses what the real API is documented to refuse. Recorded in §8 |
TB-5 — Cluster → shared datastore¶
Built ISOs are uploaded to banlieue-images/<vmimage>.iso on a vSphere
datastore — or to a libvirt storage pool — and, under deferred install
(ADR-0040), stay CD-ROM-attached to each clone until its installed guest
announces itself, when the provider ejects them (ADR-0044: libvirt since
2026-09-23, vSphere since 2026-10-23). Datastore-browse in
vCenter, or filesystem access on a libvirt host, is a much broader privilege
than banlieue admin. Anything embedded in that ISO — including cloud-config
supplied through VMImage.spec.cloudConfigs or isoOverlay — should be
treated as readable by every hypervisor operator, not just by banlieue's own
principals. Put per-VM secrets in VirtualMachine.spec.userData
(guest-delivered per clone) rather than baking them into a shared image.
A second exposure at this boundary is the guest's own disk, which is only outside an operator's reach if it was actually encrypted:
| Threat | STRIDE | Control |
|---|---|---|
A VM presents every outward sign of a sealed disk — vTPM attached, tpmEnabled: true on its class, Ready — while its disk is plaintext on the datastore or storage pool, readable by any hypervisor operator and by the next tenant of the same host |
I | ADR-0048, since 2026-09-23: banlieue-controller rejects tpmEnabled: true paired with an installMode: Immediate image (or an image with no template block, which is a pre-built and therefore pre-laid disk) — Ready=False, reason=ImageClassMismatch, and no infrastructure CR is created, so the machine never reaches a provider that would build it. crates/banlieue-controller/src/reconciler/virtualmachine.rs (image_class_mismatch), checked after the class and image resolve and before scheduling. This is the only place the combination is visible: tpmEnabled is on the VMClass and installMode on the VMImage, so no ValidatingAdmissionPolicy can see both. ADR-0088 (Accepted 2026-10-26, superseding ADR-0048; on unless --enable-first-boot-sealing=false) allows the pairing only when the image declares firstBootSealing, refuses the declaration without tpmEnabled or with the flag off (FirstBootSealingDisabled), and otherwise leaves ADR-0048's refusal in place: same function, unit-tested in virtualmachine_tests.rs. |
A clone of a first-boot sealing image boots plaintext (a source image older than Kairos v4.4.0, or an Asserted image whose policy is missing from COS_OEM) and is used as if sealed. A Url build's policy does land there for both artifacts: the iso path was verified on vSphere (2026-10-22), and for a raw cloudImage AuroraBoot copies the merged cloud-config verbatim to /oem/90_custom.yaml (ADR-0088 Open question 1, answered 2026-10-26) |
I | GuestReady for a sealing machine requires every declared partition in the guest's report, else False/FirstBootSealingUnverified, and past timeoutSeconds FirstBootSealingTimeout: crates/banlieue-provider-sdk/src/sealing.rs (sealing_gate), applied in crates/banlieue-provider-vsphere/src/reconciler/vspheremachine.rs and crates/banlieue-provider-libvirt/src/reconciler/libvirtmachine.rs, unit-tested on both. A VirtualMachinePool binds only on GuestReady. Ready is not gated (§8) |
| An ISO template's install tries to encrypt with no vTPM because an operator layer asked for it, or the policy reaches the install configuration | D | The builder never puts encryption in an ISO's install configuration (the policy is a file written from after-install-chroot), and refuses an Immediate build whose operator layers set install.encrypted_partitions (FirstBootSealingConflict) — first_boot_sealing.rs (sealing_layer) |
| A clone that cannot seal (no vTPM, KMS unreachable, an upstream defect) | D | Fails closed in Kairos: the boot halts in the initramfs and never reaches plaintext userspace. banlieue refuses firstBootSealing without tpmEnabled, and reports the halted clone as FirstBootSealingTimeout, on libvirt even though no guest agent answers (libvirtmachine.rs, build_status) and on Cloud Hypervisor even though nothing was reported (crates/banlieue-provider-cloud-hypervisor/src/machine.rs, build_status, ADR-0112), re-checked every 30 s while the report is due so the deadline is not overshot (reconciler.rs, sealing_report_due; verified live 2026-10-28) |
| A pre-laid image (ADR-0113) is sealed to the build machine's TPM, so every clone shares one key and no clone's seal is its own | I, E | scripts/prelay-kairos-raw.sh boots the image with no TPM, with every cloud-config moved off COS_OEM except AuroraBoot's 01_reset.yaml and a throwaway user, so no kcrypt section is present at build time; it refuses the result if any partition is crypto_LUKS (twice: after the build boot, and before writing the image). Each clone seals its own copy of the plaintext partition with its own vTPM: make ch-sealing-isolation-e2e passed against a pre-laid image (2026-10-31) |
A pre-laid image carries the build boot's identity or logs (a machine-id, a journal, wtmp) into every clone |
I | Found while designing it: the build boot logs into COS_PERSISTENT and names a journal directory after its machine-id. The script gives COS_PERSISTENT a fresh, empty filesystem after the build boot and refuses the image unless it is empty (scripts/prelay-kairos-raw.sh). The build user's password is random, never stored, lives only in the recovery boot's in-memory /etc, and its config is deleted from COS_OEM |
| An operator's cloud-config stages run once at build time and their output is shared by every clone (keys, hostnames, anything a stage generates) | I, T | The build boot runs with only 01_reset.yaml and banlieue's throwaway user; the image's own configs are restored afterwards, so they run in each clone, never at build. Reset-bound stages (after-reset*) then run nowhere, recorded as ADR-0113 Open question 2 |
A tool reports a pre-laid image good when the reset never ran (empty COS_STATE), and clones boot nothing |
D | Found live: 01_reset.yaml's layout stage creates the partitions before the reset, and QEMU exits 0 on SIGTERM. The script requires cOS/active.img on COS_STATE and deletes the output (keeping the console) on any failure (scripts/prelay-kairos-raw.sh) |
| A host, not the builder, pre-lays (or otherwise rewrites) an image, so what runs is not the artifact that was built and pinned by digest | T | ADR-0113 Decision 3 places pre-laying in the builder. The Cloud Hypervisor provider only reflinks or copies a cached image into a guest's own os.raw and never writes the cache (ADR-0064 Decision 4, crates/banlieue-provider-cloud-hypervisor/src/hostfs.rs). Pre-laying today is an operator running the script on an image before placing it, the same trust as any operator-built BackingFile image |
A sandbox workload mounts the still-attached install ISO and reads the build-time cloud-config overlay baked into it (VMImage.spec.cloudConfigs, isoOverlay) |
I | ADR-0044, since 2026-09-23: the medium is ejected when the ADR-0043 guestInstalled marker flips, via virDomainUpdateDeviceFlags with AFFECT_LIVE\|AFFECT_CONFIG — crates/banlieue-libvirt/src/procs.rs (DEVICE_MODIFY_EJECT). GuestReady is published only after the eject, so a VirtualMachinePool — whose sole readiness input is that condition — cannot bind a member whose installer is still attached. converge() also suppresses the ISO from the domain XML it redefines each pass, or a redefine would restore it while status claimed otherwise. vSphere (ADR-0044 amendment, 2026-10-23): one ReconfigVM_Task gives every ISO-backed CD-ROM a client-device backing with connected, startConnected and allowGuestControl false — crates/banlieue-provider-vsphere/src/client/vim.rs (build_eject_cdroms_reconfigure_spec), called by vspheremachine.rs (settle_install_media) before the status that may publish GuestReady; GuestReady=False, InstallMediaAttached, until it succeeds; the eject is verified live (2026-10-23, vCenter 8.0.3, two installed Deferred VMs). A guest-side eject (Kairos eject-cd) is not the control: it cannot clear startConnected or allowGuestControl, so the ISO would return at the next power-on |
| A guest reboots into its still-attached installer and re-runs the install, re-sealing a fresh disk over the previous tenant's workload | T, D | Same control. Both flags are passed deliberately: a LIVE-only eject leaves the medium in the persistent definition, where it returns at the next boot. The rendered <os> block also stops offering <boot dev='cdrom'/> once detached. Since the ADR-0044 amendment (2026-10-12) the disk is listed before the installer (crates/banlieue-provider-libvirt/src/xml/domain.rs), so a bootable disk wins even before the eject; the installer is reached only while the disk is empty or unbootable. On vSphere a Deferred clone still boots the CD-ROM first (ADR-0044 amendment 2026-10-12), so the window is from install to the first pass that sees guestInstalled; after the eject the drive has no medium and is not connected at power-on |
| A guest holds the cdrom tray locked so the eject fails, and is handed out anyway | D | VIR_DOMAIN_DEVICE_MODIFY_FORCE is not passed. A failed eject leaves GuestReady unpublished, so the member never becomes available and provisioningTimeoutSeconds (ADR-0046) reaps it as poisoned. Failing toward an unavailable member rather than an exposed one is the intended direction. On vSphere the opposite choice (ADR-0044 amendment, Decision 4 revised 2026-10-23): a locked tray makes vCenter ask msg.cdromdisconnect.locked, and the provider answers Yes to that question only — client/vim.rs (cdrom_lock_question_answer, wait_for_eject_task). Found live: installed Kairos guests hold the tray locked, and the VMs could not be logged into, so not overriding meant the ISO stayed attached forever. The eject happens only after guestInstalled, when the installer is no longer needed. Any other question is left for a human; a failed eject is retried at the long requeue |
A forced re-import (forceUpload) cannot replace an ISO a running VM still has connected, and the zone is left with no template because the destroy ran first |
D | Found live 2026-10-22. ADR-0020 amendment (2026-10-23): with forceUpload the Job resolves the target datastore and lists the VMs on it holding that ISO open before the forceCreate destroy, and refuses naming them — crates/banlieue-provider-vsphere/src/import.rs (ensure_iso_replaceable, iso_delete_failed_message); client/vim.rs (vms_holding_iso, holds_iso). A failed Job is deleted and recreated once its Failed condition is 10 minutes old — reconciler/vmimage.rs (import_retry_due). The root cause, an installed VM keeping the ISO, is the ADR-0044 control above |
A Proxmox machine's rendered user-data (A-2) sits in a seed ISO on an iso storage that anyone with storage-audit rights on the Proxmox side can read |
I | Uploaded as banlieue-<machine uid>.iso, so it traces to one machine; deleted with the VM (finalize_backend); stays attached for the machine's life like the libvirt and Cloud Hypervisor seeds (§8). Put per-VM secrets in userData, not in the shared template |
An installMode: Manual image asserts a deferred install it does not perform, and seals nothing |
I | Not controlled. Manual is ADR-0040's escape hatch for a non-Kairos build and banlieue cannot inspect what such an image does. Recorded in §8 |
TB-6 — Supply chain¶
This is the strongest area of the project and is largely already ADR-0006.
| Control | Detail |
|---|---|
| Provenance | SLSA build provenance on every release. Its subjects are every installable asset: the binary tarballs, deploy-manifests.tar.gz, and the single-file YAMLs (install.yaml, crds.yaml, admission-policies.yaml, network-policies.yaml, the clusterctl files), all rendered by make release-manifests in .github/workflows/build.yaml and listed in checksums.sha256 |
| SBOM + VEX | Generated per release; VEX statements are reachability- and presence-derived, and the generators fail closed on empty or oversized inputs (256 MiB cap) |
| Signing | cosign signatures on published images |
| Base images | Digest-pinned (Chainguard + distroless; Debian 13-slim for the pre-lay image), tracked by Dependabot; the banlieue images run non-root |
| Actions | SHA-pinned, with one documented exception (slsa-github-generator must be tag-referenced or it rejects its own ref) |
| Privileged triggers | No pull_request_target, no issue_comment. docs.yaml's workflow_run is hard-gated to same-repository runs, with a defence-in-depth re-check before any checkout — fork SHAs are never checked out privileged, and the default-branch cache cannot be poisoned |
| Untrusted input | Event fields reach run: steps only through env: indirection, never inline interpolation |
| Auto-merge | Gated on pull_request.user.login == 'dependabot[bot]' (the PR author, not github.actor); major-version updates are held for human review |
| Scanning | CodeQL, OpenSSF Scorecard, SAST, grype/OSV, cargo audit, cargo deny, gitleaks; Grype gates on both the banlieue image (grype, VEX from .vex/ and the auto-VEX tools) and the pre-lay image (grype-prelay, make grype-prelay, VEX from .vex/prelay/) |
| Crate publication: token custody (S, E) | CARGO_REGISTRY_TOKEN (A-21) is read by one job, publish-crate in .github/workflows/build.yaml, gated on github.event_name == 'release', holding only contents: read; fork pull requests receive no secrets. It goes through the SHA-pinned firestoned/github-actions/rust/publish-crate (ADR-0100) |
| Crate publication: what is published (T) | The crate is built from the release's own checkout with the tag's version stamped by make set-version, after the test job passed; make crate-package builds the packaged copy alone against crates.io dependencies on every pull request (crate-package job) and again before the upload. Only banlieue-api is published; no path dependency or file outside the crate can enter it |
| Crate publication: repudiation and re-runs (R) | One version per release tag, equal to the binary's and the CRDs'; make crate-published skips a version already on crates.io, so a re-run cannot fail half-way or publish twice. crates.io's own audit log records each upload against the token |
| Pull-request images (T, S) | Images of a pull request are pushed to GHCR only when its head branch is in this repository (.github/workflows/build.yaml, docker job: github.event.pull_request.head.repo.full_name == github.repository), which needs repository write access, the same trust as a push to main; fork PRs stay build-only. They are Cosign-signed like every pushed image, and tagged only pr-<N> and sha-<short> (with the variant suffix), never latest or a version, so nothing that resolves a release reaches them; release manifests pin a digest (ADR-0103). A PR image is still unreviewed code under the project's namespace: pull it only to test that PR |
| Install manifests: image pinning (T) | ADR-0103. A moved or overwritten GHCR tag cannot change what a fresh install from a release runs: install.yaml and the clusterctl components name every banlieue image <repository>:<tag>@<digest> (install_image and the seeded ProviderClass.spec.image.digest in crates/banlieue-operator/src/bootstrap.rs), and the kubelet pulls the digest. The release-manifests job in .github/workflows/build.yaml runs after docker and passes docker.outputs.digest-chainguard; make release-digest-check (RELEASE_REQUIRE_DIGEST=true) fails the release rather than publish an unpinned manifest. Tests: a_digest_pins_every_installed_workload, a_seeded_class_can_be_pinned_to_a_digest |
| Pre-lay image (T, S) | ADR-0113, amendment 2026-11-04. ghcr.io/firestoned/banlieue-prelay is what the privileged pre-lay Job runs. It is built from Dockerfile.prelay by the docker-prelay job in .github/workflows/build.yaml under the same push rules and tags as the banlieue image (fork PRs build only; PR tags never latest or a version), Cosign-signed by digest, with build provenance and an SBOM on main and releases, and the release's asset upload waits for it. banlieue bootstrap imagebuilder --prelay-image-digest pins it (prelay_install_image in crates/banlieue-operator/src/bootstrap.rs, parsed by parse_image_digest; tests a_prelay_digest_pins_the_prelay_image, the_banlieue_digest_does_not_pin_the_prelay_image); without it the imagebuilder pulls the tag. Grype gate (amendment 2026-11-04, a Grype gate for the pre-lay image): the grype-prelay job runs make grype-prelay on every pushed image by digest, uploads every finding as SARIF (grype-<event>-Prelay), and fails on a finding at or above high with a Debian fix (--only-fixed --fail-on high); upload-release-assets needs it, so a release whose pre-lay image misses a fix publishes no assets. Its VEX is .vex/prelay/, never merged into the banlieue image's. Findings Debian has not fixed ship: §8 |
| Install manifests: digest input (T, D) | --image-digest is parsed by parse_image_digest, which accepts only sha256: and 64 lowercase hex digits (banlieue_oci::reference::require_digest), before anything is applied, so a typo cannot pass the install and then fail every pull (the_digest_flag_accepts_only_a_sha256_digest) |
TB-7 — External identity provider → API server¶
Every other boundary in this document is one banlieue can place a control on.
This one is not: the API server derives request.userInfo.username from a
token minted elsewhere, and the claim-subject policy compares
spec.subject.id against that derived name. banlieue's strongest
attribution guarantee is therefore no stronger than the issuer behind it,
which is worth stating plainly rather than leaving implicit in §8.
| Threat | STRIDE | Control |
|---|---|---|
| The issuer is compromised, or mints a token for an attacker under a victim's name | S, R | None in banlieue. The API server vouches for the username; nothing downstream can second-guess it. Recorded in §8 — the mitigation is issuer-side (MFA, short lifetimes, key custody) and is the operator's, not banlieue's |
usernamePrefix in the policy's ConfigMap disagrees with the API server's --oidc-username-prefix |
T | Fails in the safe direction, but silently. Too short a prefix makes the comparison unsatisfiable and every claim is refused; too long forces authors to store the prefixed name in subject.id, which is the exact shape ADR-0047 Decision 9 was amended to eliminate and which leaves the in-guest agent a value it cannot compare to any JWT. Neither is a bypass. It is a configuration coupling between two independently-managed objects, so §7.6 now names it. The same holds for each issuerPrefixes line (ADR-0047 Decision 9, amended 2026-10-13): a wrong prefix refuses that issuer's claims, never admits another's |
| A stolen cached ID token is used to create claims in the victim's name | S, R | None in banlieue, and not specific to claims — a stolen bearer token authenticates as its owner everywhere in Kubernetes. It is called out because the consequence here is an audit record that says the victim asked for a sandbox. §8 |
An issuer the site does not use is named in spec.subject.issuer |
S, R | The issuers allowlist in banlieue-virtualmachineclaim-subject-authorization. This is a check on the claim, not on the caller — nothing reveals which issuer actually minted the caller's token (§8) |
The agent is pointed at an attacker's JWKS via spec.subject.issuer |
S, T | Same allowlist, doing double duty — see TB-4. Load-bearing for verification, not merely for audit tidiness (ADR-0049 Decision 7) |
The issuer runs on the management cluster (e.g. Dex with storage: kubernetes), so its signing keys are cluster resources; anyone who can read them mints a token for any username |
S, E | Not a banlieue control. Operator requirement (§7.20): read on the issuer's storage API group only for cluster-admin, and the issuer in its own namespace at Pod Security restricted. Residual: a cluster admin can mint any identity, which is consistent with cluster-admin being out of scope (§9); §8 records when to move the issuer off the cluster |
| A stolen cached ID token stays valid for the issuer's default lifetime (24 h for Dex) | S | Operator requirement (§7.20): a short ID-token lifetime with refresh tokens. Observed on a reference deployment: 24 h until configured |
| Any account at the upstream identity provider (e.g. any GitHub user, through Dex) can complete a login | S, E | Authentication is not authorization: nothing is bound to system:authenticated beyond the Kubernetes defaults, and scripts/dev-oidc-k0s.sh grant (grant_me) binds one named identity to one namespaced Role. Verified: a freshly logged-in identity is forbidden from listing namespaces |
| More than one issuer is trusted (e.g. GitHub through Dex, Google and Auth0 direct) and two share a username prefix, or one prefix nests inside another, so an account at one issuer authenticates as a same-named identity from another | S, E | The prefix is the only part of request.userInfo.username that names the issuer. scripts/dev-oidc-k0s.sh (validate_issuers) refuses empty, equal, nested and system: prefixes before writing the file; defaults are github:, google:, auth0:. Operator requirement for hand-written files: §7.6. On the claim path the policy's issuerPrefixes then checks each claim against its own issuer's prefix, so a user from one issuer cannot record a subject under another (deploy/admission/virtualmachineclaim-subject-authorization.yaml, tested by crates/banlieue-controller/tests/live_claim_subject_policy.rs) |
A direct issuer that allows self-signup (an Auth0 database connection) issues a token for an email address the holder never proved, and email is the username claim |
S | claimValidationRules requiring claims.?email_verified.orValue(false) == true on every direct issuer, written by authn_google / authn_auth0 in scripts/dev-oidc-k0s.sh. Stricter than the implicit rule claim: email applies, which accepts a token without the claim |
| A validly signed ID token minted for another application at the same provider (Google and Auth0 sign every tenant client's tokens with the same keys) is presented to the API server | S | Each jwt entry's audiences is this cluster's own client ID, so aud must name it (authn_google / authn_auth0) |
| The API server refuses an edited authentication file and keeps serving the previous one, so the change looks applied, and the bad file stops kube-apiserver at its next restart | D | scripts/dev-oidc-k0s.sh (await_reload) waits until the hash label of apiserver_authentication_config_controller_last_config_info equals the file's sha256 on each controller, and otherwise restores the previous file it kept as .prev |
| The API server cannot read its authentication configuration and crash-loops on every controller | D | scripts/dev-oidc-k0s.sh (attach_controller) owns the file by the API server's user, because k0s runs kube-apiserver unprivileged with gid 0 and no supplementary groups (group ownership does not help), restarts one controller at a time, and restores the previous config if the flag is not in the running process within RESTART_TIMEOUT_SECS |
TB-8 — Cloud Hypervisor host ↔ cluster¶
The provider runs on the hypervisor and dials the API server; nothing dials the host (ADR-0060). Two directions matter: what a stolen host credential can do in the cluster, and what cluster state can make the host do.
| Threat | STRIDE | Control |
|---|---|---|
| A stolen host token reads Secrets or other credentials | I, E | The provider's identity has no Secret or ConfigMap access at all. It is the operator-built per-Provider Role of an External class (ADR-0060 Decision 3), which never emits a Secret rule without a credentialsRef and a cloud-hypervisor Provider may not have one — crates/banlieue-operator/src/workload.rs (build_role, external_rules), deploy/admission/provider-connection.yaml, and the provider's own CredentialsNotAllowed refusal (provider.rs) — plus a ClusterRole limited to reading VMImages and patching their status, deploy/provider-cloud-hypervisor/rbac/clusterrole.yaml |
| A stolen host token mints or deletes machines | T, D | No create or delete on cloudhypervisormachines; the controller owns their lifecycle — same file |
A stolen host token patches another host's Provider or its status |
S, T | The External Role scopes providers and providers/status by resourceNames to the one Provider; the provider's watch filters on metadata.name, which resourceNames honours for list/watch — workload.rs::external_rules, tested in workload_tests.rs |
| A stolen host token publishes false capacity on its own Provider: inflated, so the scheduler over-places VMs there, or zero, so it places none (ADR-0107) | T, D | Bounded to that host. The same resourceNames-scoped Role means a token can only misreport its own Provider, so placement elsewhere is unaffected. Over-placement on that host is still bounded by each VMM's MemoryMax and the guests' slice MemoryMax (ADR-0105): it ends in guests on that host being OOM-killed, never the host itself. The same token could already withdraw the host by publishing no failure domain |
| The provider's slice reads are turned into reads of something else | I | systemd.rs::Systemd::guest_slice_budget loads only the two fixed names GUEST_SLICE and VMS_SLICE and reads two properties; LoadUnit and unit properties need no privilege and change nothing (verified unprivileged on systemd 257), and no cluster value reaches the unit name |
| A stolen host token patches another host's machines in the same namespace | T, D | Not controlled — machine names are unknowable when the Role is written, so cloudhypervisormachines is namespace-wide. §8 |
| The host loses the cluster when the one controller its kubeconfig names goes down | D | With ADR-0086 the host's kubeconfig names the control plane VIP, which moves between controllers; the host still verifies the API server's certificate against the cluster CA (TB-13) |
| A stolen token is renewed by the thief and never expires | S | Revocable, not self-limiting. The token can create tokens for its own ServiceAccount (that is how the host renews), so a thief who holds it can too. The control is revocation: bound tokens are tied to the ServiceAccount's UID, so deleting the ServiceAccount invalidates every token ever issued for it at once, and the operator recreates it for a fresh banlieue bootstrap cloud-hypervisor-host (§7.11). Token creation appears in the API server's audit log. §8 |
| A host's token outlives the host | S | Bound, 24 h by default, renewed only while the provider runs: a host down for longer than one lifetime comes back with a dead token and must be re-issued one — crates/banlieue-provider-cloud-hypervisor/src/token.rs |
| Cluster state chooses a host path or bridge | T, E | Machines name classes, never paths: the host resolves a class through its own /etc/banlieue/cloud-hypervisor.toml, which is 0640 root:banlieue and read-only to the provider (ProtectSystem=strict) — crates/banlieue-provider-cloud-hypervisor/src/host_config.rs, plan.rs. An unknown class is refused (PlanError::UnknownStorageClass, UnknownNetworkClass) |
| Cluster state injects a path through an image name, MAC or machine UID | T, E | plan.rs accepts an image only as a plain file name (no /, no leading .), a MAC as six hex octets, and the machine UID as a canonical UUID before any of them reaches a path, unit name or tap name; tap names are derived, never taken from the spec |
| A machine for another host is realised here | T | reconciler.rs::is_ours: only machines whose providerRef names this host's Provider, in its namespace, are reconciled |
Cluster state chooses where a host downloads what it boots — a VMImage status naming an attacker's registry |
T, E | The host pulls only a digest in the one repository its own config names ([registry] repository); anything else is ForeignReference and never fetched — vmimage.rs::check_reference, re-checked by the import itself (import.rs). The cache file name is derived from the digest, never taken from the cluster. See TB-10 |
| Host paths leak into the cluster through status | I | The Provider failure domain and the VMImage row carry class names only; unit tests assert no / in either — provider_tests.rs, vmimage_tests.rs |
A stolen host token holds a VMImage in Terminating by never reporting Released, or reports it falsely |
D, R | The host token writes only its own perProvider row (it has vmimages/status patch, no vmimages write), and since ADR-0087 that is true of the effect as well as the intent: it applies under a field manager scoped to its own Provider, so the write cannot reach another host's row. Withholding blocks deletion of images that host held (removing that Provider releases it); a false Released only leaves a file on that host. The finalizer itself is the controller's (crates/banlieue-controller/src/reconciler/vmimage.rs, HOST_CACHE_FINALIZER), so no host token needed VMImage metadata write, which would also reach the spec |
A stolen host token erases every other host's VMImage readiness rows, or holds the object in a write loop |
T, D | ADR-0087. status.perProvider is merge-keyed on providerName/providerNamespace (ADR-0015), and each host applies under banlieue.io/provider-cloud-hypervisor/<ns>/<provider> built by provider_field_manager in crates/banlieue-provider-sdk/src/ssa.rs, via host_config.rs::field_manager. A field manager owns exactly the field set it last applied, so under the former class-wide manager one host's single-row apply made the API server delete every other host's row; each host's watch then fired and re-applied, which is a write loop reachable with nothing but a host's own legitimate credential. The single-row apply no longer passes force, so claiming a row owned by another manager is a conflict rather than a silent takeover. Pinned by crates/banlieue-provider-libvirt/tests/e2e_vmimage_ssa.rs in both directions |
| A cluster author asks for a vTPM or install media the host cannot provide | I | vtpm is advertised only when the host config has [tpm] and its swtpm binaries, setup configuration and CA certificate exist (provider.rs::gather_facts); plan.rs refuses tpmEnabled on a host without [tpm]. A tpmEnabled VM with an Immediate image is refused before scheduling (ADR-0048), and one whose image declares firstBootSealing seals the guest's own reflinked os.raw with its fresh swtpm and is GuestReady only once its vsock report names every listed partition sealed, or FirstBootSealingTimeout past the deadline (ADR-0112; crates/banlieue-provider-cloud-hypervisor/src/machine.rs, build_status), so none is reported healthy while unsealed |
TB-9 — Guest VMM → host and other guests¶
Assume a guest has escaped into its VMM process. What contains it:
| Threat | STRIDE | Control |
|---|---|---|
| The VMM reads or writes another guest's disk, seed or socket | I, T | Each guest runs as its own uid and its own private group (User=<uid>, Group=<uid>, registered in /etc/userdb by the bootstrap); machine and run directories are 2770 guest-uid:banlieue under 0711 roots, and no guest is ever in the banlieue group; a uid is given to one machine at a time, under one lock, even when a pool creates members together — plan.rs::vmm_unit, plan.rs::UidLedger, hostfs.rs::prepare_dirs, scripts/bootstrap-cloud-hypervisor-host.sh. Live-verified: another user cannot even list a guest's directory |
| The VMM writes anywhere else on the host | T, E | ProtectSystem=strict with ReadWritePaths= its own two directories, ProtectHome, PrivateTmp, NoNewPrivileges; Cloud Hypervisor's own seccomp filters and Landlock, with Landlock widened only by one read-only rule per tap on /sys/devices/virtual/net/<tap>: systemd.rs, banlieue-cloud-hypervisor/src/types.rs; reference unit deploy/provider-cloud-hypervisor/host/banlieue-ch@.service |
| The VMM opens other devices | E | DevicePolicy=closed with only /dev/kvm and /dev/net/tun allowed |
| The VMM exhausts host memory or PIDs | D | MemoryMax= guest memory + 512 MiB, TasksMax=1024 in the unit's cgroup |
| All guests together exhaust a shared host's CPU, memory or PIDs (a pool refill storm, or one escaped VMM spawning load) and starve the host's other workloads | D | Every guest and vTPM runs in banlieue-guests.slice (Slice= in banlieue-ch@.service and banlieue-swtpm@.service), whose AllowedCPUs, CPUWeight, MemoryMax and TasksMax the operator declares at install. The slice is a root-owned file and polkit never allows set-property on it, so neither the provider nor a guest can lift it — deploy/provider-cloud-hypervisor/host/banlieue-guests.slice, banlieue-host/src/settings.rs::GuestSlice, tested the_guest_slice_is_installed_with_the_declared_budget, guest_and_vtpm_units_run_inside_the_slice. Unset limits leave that dimension unbounded, and install says so. §8 |
| All VMs on a host together (guests and management-cluster nodes) starve its other workloads | D | Nodes from scripts/bootstrap-k0s-cluster.sh run in banlieue-nodes.slice (Slice= from render_ch_vm_unit/render_ch_swtpm_unit, or the 50-slice.conf drop-in ensure_ch_slice adds to an older node), so banlieue.slice holds every VM and one systemctl set-property banlieue.slice bounds guests and nodes together (§7.26). Nothing ships a limit on the parent or the nodes' slice: by default they are bounded only by the host, and the guests by ADR-0105's declared budget (ADR-0106) |
The provider lifts a budget on banlieue.slice or banlieue-nodes.slice, or moves a VM out of its slice |
T, E | The polkit rule matches only guest and import instances, so it grants nothing on any slice: scripts/test-cloud-hypervisor-polkit.js refuses set-property on banlieue.slice, banlieue-guests.slice and banlieue-nodes.slice, and stop on banlieue-guests.slice. Slice= is fixed in root-owned unit files and systemd accepts only resource-control properties at runtime on a unit with a file |
VMM-controlled /proc data (its thread names, its counters) reaches banlieue host cloud-hypervisor overhead, possibly run as root |
T, D | crates/banlieue-host/src/overhead.rs only parses: stat is split after the last ) and read as numbers, thread names are only compared against two fixed patterns, nothing from /proc is printed (VM names come from cgroup directory names, which systemd and root create), and the verb writes nothing. It needs no root; run it unprivileged. Skewed numbers are §8 |
The VMM plants a symlink in its own directory so the provider's privileged chown/chmod/truncate lands on another guest's disk or the shared image |
T, E | Every privileged change the provider makes inside a guest's directory goes through a file handle, never a path: disks and seeds are created O_CREAT\|O_EXCL, then grown, owned and re-moded through that handle (fchown, fchmod, set_len); directories are opened O_NOFOLLOW\|O_DIRECTORY before being re-owned; the seed is read O_NOFOLLOW — hostfs.rs (own, ensure_os_disk, write_seed, prepare_dirs), sys.rs::clone_file. Tests plant symlinks and assert their targets are untouched (hostfs_tests.rs, sys_tests.rs) |
The VMM replaces its API socket so the provider talks to something else, or so the provider's CAP_FOWNER re-modes an arbitrary file |
S, E | hostfs::grant_api_socket opens the path O_PATH\|O_NOFOLLOW, requires a socket owned by the guest uid in the banlieue group with no bits for others, and changes only that opened inode via /proc/self/fd; the VMM client re-checks owner and mode before every connect — banlieue-cloud-hypervisor/src/socket.rs |
| The VMM returns a malformed or huge API response to the provider | D, T | Typed decoding into bounded structs; errors are reported on the machine's status, never panics — banlieue-cloud-hypervisor/src/wire.rs. Not fuzzed, unlike the libvirt decoder. §8 |
| The guest (or its VMM) replaces its EK certificate with one it made, carrying the expected CN | S, T | The certificate is written by swtpm_setup, run by the provider's user, to <state_root>/ek/<uid>/ (0700 banlieue), outside every directory a guest uid owns, and read from there — plan.rs (TpmPlan::ek_dir), hostfs.rs::ek_certificates. Nothing a guest sends is published as its EK |
The guest deletes its TPM state to get a TPM manufactured again, this time with symlinks planted where swtpm_setup (run as the provider's user, which can read the CA key) will write |
T, E | Whether a TPM was manufactured is the provider-only EK directory, not the guest-owned state: a machine with EK files is never manufactured again; lost state is an error (hostfs.rs::tpm_manufactured, machine.rs::ensure_tpm, tested in machine_tests.rs) |
A symlink in the TPM state directory turns the provider's CAP_CHOWN onto another file when the state is handed to the guest |
T, E | Each entry is opened O_NOFOLLOW relative to the directory handle and re-owned through that handle; anything but a regular file is refused — hostfs.rs::adopt_tpm_state, tested with a planted symlink in hostfs_tests.rs |
The TPM state is handed to the guest while swtpm_setup is still writing it, so setup's final state file lands owned by the provider's user and the guest's swtpm cannot read it: the vTPM enters failure mode and the VM never boots |
D | Nothing is adopted and swtpm is not started while the manufacture unit is still running; the EK files alone do not mean manufacture finished (machine.rs::ensure_tpm), tested by machine_tests.rs::swtpm_waits_for_the_manufacture_unit_to_finish with a fake that now writes the EK files before the unit exits, as swtpm_setup does. Found live 2026-10-09 (roadmap 17 phase G) |
| A guest's swtpm reads another guest's TPM state, or the CA key | I, E | swtpm runs as the guest's own uid and private group, writing only its state directory and run directory (plan.rs::swtpm_unit); the CA key is 0600 banlieue in a 0700 directory, and manufacture — the only step that reads it — runs as the provider's user (plan.rs::swtpm_setup_unit) |
| A guest floods or stalls the vsock report listener, or sends crafted bytes | D, T | Each connection is read with a 10 s timeout and a 16 KiB cap; only an exact phase=installed line counts, everything else is ignored, and nothing is ever sent back or executed — report.rs (parse_report, Listeners), tested in report_tests.rs. The listener is a Unix socket in the guest's own run directory, never a network port |
| A guest claims another guest's installation, or claims "installed" while its installer is still running | S | A report can only arrive on the socket in the reporting guest's own run directory, so it speaks for itself only. A guest that lies about itself makes only itself GuestReady early — the same bound as ADR-0043's marker (§8); with tpmEnabled, GuestReady also waits for the host-minted EK |
A symlink planted at the report socket path turns the provider's CAP_CHOWN onto another file |
T, E | The path is unlinked (a link, never its target) before binding; the bound socket is opened O_PATH\|O_NOFOLLOW, checked to be a socket the provider owns, and re-owned through /proc/self/fd — report.rs::hand_to_guest, tested with a planted symlink |
| The installer survives into the installed system and re-runs on a reboot | T, D | When the installed system reports, vm.remove-device install in the same pass, installMediaDetached sticky, every later start planned without it — machine.rs, plan.rs::without_install_media, reconciler.rs; GuestReady stays False, InstallMediaAttached until then |
| A guest's VMM reads the shared image cache, or another guest's installer, or follows a symlink the guest planted where its installer copy goes | I, T | The cache stays 0750 banlieue, closed to guest uids; each Deferred machine gets its own copy in its own directory, 0440 owned by the guest, created O_EXCL under a temporary name and re-owned by handle; anything but a regular file at the path is removed, never followed, and the copy is deleted on the pass after the eject — hostfs.rs::ensure_install_media, tested with a planted symlink in hostfs_tests.rs; verified live by tests/e2e_deferred.rs |
| The VMM starts, stops or changes other systemd units | E | It holds no D-Bus authority; only the provider's user may manage units, through the polkit rule below — deploy/provider-cloud-hypervisor/host/60-banlieue-cloud-hypervisor.rules |
| The provider's user starts a unit as root, or as a user outside the guest range (a compromised provider process, or anyone holding its uid) | E | Units are instances of root-owned templates that fix User=, ExecStart= and the sandbox (deploy/provider-cloud-hypervisor/host/*@.service); the provider chooses only the instance. polkit allows start/stop/reset-failed only on those templates' instances, with the uid instance inside the guest range (so not banlieue-ch@0), and set-property only on VMM instances; systemd itself refuses a transient unit under a name with a unit file and refuses User/ExecStart changes through set-property. Tested by scripts/test-cloud-hypervisor-polkit.js (22 cases) and verified on a host as the banlieue user (ADR-0063, amended 2026-09-27). Previously the provider created transient units, which polkit could only check by name, so this was possible |
| The provider passes crafted arguments through a template's environment file | T, E | Only the two templates that run as the provider's own user read one, so nothing is gained; values are refused unless single safe words (no whitespace, quotes, $, \\), and reach ExecStart as ${NAME} — systemd.rs::EnvFile::render |
| A guest spoofs its address, or another guest's, on the bridge | S | Not controlled at L2: the tap is a plain bridge port. The provider reports only complete neighbour entries for the guest's own MAC on its own bridge (neigh.rs), but a guest chooses what IP it answers for. §8 |
| A failed VMM loops silently, hiding a fault | R | Failed units are kept (CollectMode=inactive), their systemd result is reported as Ready=False, VmmExited, then cleared and retried with backoff — machine.rs, systemd.rs::failure |
| Deleting a machine leaves its disk, seed, tap or unit behind | I | machine.rs::teardown stops the unit, deletes the taps and removes both directories, then verifies each is gone before the finalizer is released (hostfs::remove_machine, host.rs::remove_taps) — live-verified repeatedly |
Management-cluster nodes on Cloud Hypervisor (ADR-0099). These are not
banlieue guests: scripts/bootstrap-k0s-cluster.sh writes their units, and
nothing in banlieue manages them. Assume one node's guest escapes into its
VMM:
| Threat | STRIDE | Control |
|---|---|---|
| The VMM reads or writes a banlieue guest's disk, seed, socket or TPM state | I, T | It runs as CH_VM_USER (k0s-vm), which is neither a guest uid nor in the banlieue group, so the 2770 guest-uid:banlieue directories under 0711 roots are closed to it, as to any other user: render_ch_vm_unit, and TB-9 above for the guest side |
| The VMM reads or writes another management node's disk or vTPM state | I, T | Not separated: every management node on the host runs as the same CH_VM_USER. All of them are nodes of one cluster, and a management node already holds what a worker or controller holds; §8 |
| The VMM writes anywhere else on the host | T, E | ProtectSystem=strict, ProtectHome=read-only, PrivateTmp, NoNewPrivileges, ReadWritePaths= its node directory and /run/k0s-vm-<name> only; Cloud Hypervisor's seccomp filters and Landlock, widened only by a read-only rule on its own tap's sysfs entry (found live: without it the VMM cannot read tun_flags and refuses to boot): render_ch_vm_unit, tested in scripts/bootstrap-k0s-cluster-test.sh |
| The VMM opens other devices | E | DevicePolicy=closed with only /dev/kvm and /dev/net/tun |
| A local user drives the VMM through its API or serial socket, or the vTPM through its control socket | E, I | RuntimeDirectoryMode=0700 and UMask=0077 on both units: the sockets are reachable by CH_VM_USER and root only (found during this pass: the runtime directory had defaulted to 0755): render_ch_vm_unit, render_ch_swtpm_unit |
| The VMM reconfigures the bridge or other interfaces | E | It holds no capabilities. The tap is created by a root oneshot, k0s-vm-<name>-net.service, owned by CH_VM_USER, so opening it needs no CAP_NET_ADMIN: render_ch_net_unit |
| A management node spoofs another LAN host's address on the bridge | S | Not controlled at L2, as for banlieue guests above and libvirt nodes on LIBVIRT_BRIDGE (§8) |
| A VMM that cannot start loops fast enough to take its vTPM unit into systemd's start limit, leaving it down after the fault clears (found live) | D | RestartSec=5s on the VMM unit: render_ch_vm_unit |
With CH the default, a command run against a cluster built on libvirt creates a second VM with a running node's name and MAC, or destroy reports the cluster removed while every domain still runs |
T, D | Every command, destroy included, first compares each node's resolved VMM with the host (a libvirt domain, or a CH node's unit, of that name) and refuses on a mismatch, naming DEFAULT_VMM=libvirt: check_vmm_matches_host, tested in scripts/bootstrap-k0s-cluster-test.sh |
| A CH node lands on a network nobody chose | T | CH_BRIDGE, else LIBVIRT_BRIDGE, else the bridge libvirt itself reports for LIBVIRT_NETWORK; refused when none resolves, and the chosen bridge is logged: resolve_ch_bridge. A bridge without a host address is refused as before: check_deps_ch |
TB-10 — Build namespace → OCI registry → Cloud Hypervisor host¶
A Cloud Hypervisor host cannot mount the artifacts PVC, so a build it needs is pushed to an OCI registry and pulled by digest (ADR-0064). Integrity rests on content addressing; confidentiality and availability rest on the registry.
| Threat | STRIDE | Control |
|---|---|---|
| The registry, or anyone with push access, substitutes an image's content | T | Digest pinning end to end. The push Job reports the manifest digest; the imagebuilder records the reference only if it is a digest in its configured repository (push.rs::pushed_reference); the host pulls that digest, verifies the manifest and the layer against their digests while streaming, and renames into place only on a match — crates/banlieue-oci/src/client.rs (manifest, pull_file). A moved tag changes nothing |
A VMImage status writer points hosts at another registry or repository |
T, E | Host-pinned repository — TB-8 |
A VMImage status writer points hosts at another digest in the same repository |
T | RBAC only. vmimages/status patch (the imagebuilder, every Cloud Hypervisor provider's token, cluster admins) together with push access to the repository chooses what hosts boot. Digest pinning gives integrity, not provenance; signature verification is ADR-0064's follow-up. §8 |
| A compromised push Job redirects hosts | T, S | It has no API credential at all (automountServiceAccountToken: false); its only output is a termination message the imagebuilder validates as above — push.rs::build_push_job |
| The push Job escalates or tampers with the build | E, T | Non-root, readOnlyRootFilesystem, allowPrivilegeEscalation: false, all capabilities dropped, PVC mounted read-only, an emptyDir for the compressed copy — push.rs::build_push_job, tested in push_tests.rs |
| A compromised or wrong pre-lay Job publishes content it should not, or a digest for something else (ADR-0113) | T, S | It holds no API credential; its only output is a termination message the imagebuilder accepts only as sha256: and 64 lowercase hex digits (prelay.rs::parse_prelay_result), and every consumer verifies the file against that digest. It verifies the built raw file against the source's checksum before touching it (PRELAY_VERIFY_CHECKSUM, verify_checksum in scripts/prelay-kairos-raw.sh), so SEC-004's chain runs source checksum → builder digest. The Job's code is --prelay-image: by default the published, signed banlieue-prelay image (TB-6), or the operator's own; the same trust as the build images the namespace already runs |
| The image booted at build time escapes QEMU onto the build node, into a privileged container | E | QEMU runs with its seccomp sandbox (-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny), no network device and no TPM (scripts/prelay-kairos-raw.sh); the guest is the image being built, which every clone will run anyway. Residual risk in §8 |
| The pre-lay Job rewrites the built file, so what was verified is not what was transformed | T | It writes a new file (prelaid_file_name) and refuses an existing output; the built file is read only. Only Done moves the artifact to the new file; a failed or refused pre-lay publishes Failed with no file, so nothing imports a half-made image (prelay.rs::apply_prelay, apply_refusal, tested in prelay_tests.rs) |
| The registry reads images, including cloud-config baked into them (A-2) | I | Not controlled by banlieue. Treat the registry as holding A-2 (§7.13); per-VM secrets belong in VirtualMachine.spec.userData, delivered per clone, not in a shared image — as TB-5 says for datastores |
| The registry withholds or deletes an image | D | The import unit fails, the row reports ImportFailed with systemd's reason, and the next reconcile retries. Images already cached, and running guests, are unaffected |
| An oversized or hostile layer exhausts the host | D | The layer is capped at its manifest-declared size while streaming, and so is the decompressed stream: the push records the uncompressed length as a layer annotation (io.banlieue.disk.size), covered by the manifest digest the host pins, and the pull refuses to write past it or to finish short of it — crates/banlieue-oci/src/sparse.rs (with_limit), client.rs::pull_file. Superseded pulls are evicted beyond [registry] keep_unreferenced, and a deleted image's file is removed (vmimage.rs::eviction_candidates, release), so steady-state growth is bounded by the images in use. §8 |
| Eviction deletes an image a guest depends on | D, T | Only digest-named pulls are candidates, never admin-placed files; a file is kept while this host's row of a live VMImage resolves to it or one of its machines names it (referenced_files). A running guest's OS disk is its own file (reflink or sparse copy), so removing the cache file cannot touch it |
| The import process writes outside the image cache | T, E | Its own unit, an instance of the root-owned banlieue-ch-import@.service, as the provider's user: ProtectSystem=strict, ReadWritePaths= only the storage classes' images/, ProtectHome, PrivateTmp, NoNewPrivileges, DevicePolicy=closed with no devices — the template file, checked by systemd_tests.rs. Writes go through a temporary name and a rename; a copy into a second class is created O_EXCL (sys.rs::clone_file) |
| A bundle naming a rogue or over-broad CA makes the push Job trust an impostor registry (a man in the middle reads the push credential and the artifact, A-13 and A-2/A-3) | S, I | The trust anchor is the operator's choice and nothing else's: it is named only by the imagebuilder's own configuration (--registry-ca-bundle-*, set by whoever edits its ConfigMap and Deployment), never by a VMImage, Provider or tenant object. The Job mounts one key, read-only, and adds the certificates to the public roots (tls.rs::http_client). A man in the middle cannot change what a host boots: the host pulls with its own trust store, a digest in its own pinned repository (TB-8). It can capture the push credential, so scope that credential to the one repository (§7.13). §8 |
A --registry-ca-bundle-key that is not a plain file name escapes the mount or names another file |
T, E | The key is validated when the flags are resolved, before any Job exists: letters, digits, ., _, -, not . or .., at most 253 bytes, and only that key is projected as a volume item — push.rs::CaBundle::new, tested in push_tests.rs and app_tests.rs |
| An empty, unreadable or unparsable bundle silently leaves the Job on the public roots while the operator believes a CA is pinned | S | Fails closed: a bundle with no certificate, or that cannot be read, fails the push with its path, and the VMImage row reports the failure — crates/banlieue-oci/src/tls.rs::{http_client, root_certificates}, tls_tests.rs |
| Verification is switched off to get past a certificate error | S, I | There is no such option, in the client or the Job: the only alternative to a CA is --registry-plain-http, which is explicit, named for a private network, and unchanged by ADR-0110 — oci_push.rs, app.rs |
| The CA bundle is tampered with in the build namespace | T | A ConfigMap or Secret in banlieue-imagebuild: the same RBAC that protects the push Secret (A-13) and the build namespace's other Secrets (§6 TB-3). Its worst case is the first row; a host is unaffected |
| Registry credentials leak | I | Push: a Secret in banlieue-imagebuild, mounted only into the push Job. Pull: files readable by root:banlieue only, never a cluster Secret, so ADR-0060's "no Secret reads" holds for the host (host_config.rs, bootstrap script) |
TB-11 — Installer (root, once) → Cloud Hypervisor host¶
banlieue host cloud-hypervisor install runs as root, at an operator's request, and places
everything A-14 names. It is the one piece of banlieue that is root by
design, so its controls are about two things: what it installs is what
banlieue pinned, and a provider's user that was compromised earlier cannot
use a later run as a lever.
| Threat | STRIDE | Control |
|---|---|---|
| A tampered or substituted VMM or firmware download, from upstream or a mirror | T | HTTPS only (rustls), and a sha256 per file, every file fetched and verified before any is installed; on a mismatch nothing is written and the previous symlinks stay: stages.rs::vmm, tested (a_pin_mismatch_installs_nothing, make ch-host-install-test step 5). For the pinned release the digest is compiled in whatever URL is used, and a digest flag for a pinned file is refused, so a mirror chooses where bytes come from, never what they must be: release.rs::resolve, tested (a_mirror_url_keeps_the_compiled_pins, a_digest_flag_for_a_pinned_file_is_refused). Any other release: §8 |
| The installed VMM is older than the release the client is written against | T | The default is the pinned release (pins_tests.rs asserts it equals spec/PIN and the client's gate). A chosen --vmm-version below the gate's major.minor is refused before any fetch, by the comparison the provider makes at runtime: release.rs::check_vmm_version, tested (a_vmm_older_than_the_client_gate_is_refused, make ch-host-install-test step 6). The provider's own gate still refuses an older VMM at runtime (ADR-0061 Decision 5). A newer one: §8 |
A --vmm-version or --firmware-tag names a path outside /opt/banlieue (../..), so root writes or links there |
T, E | Both become one directory name: letters, digits, ., _, -, not starting with ., refused otherwise before anything is fetched: release.rs::check_dir_name, tested (unsafe_or_unparseable_versions_and_tags_are_refused) |
A mirror URL downgrades the transport (http://, another scheme) |
T, I | Refused when the flag is read (release.rs::check_url, tested a_url_that_is_not_https_is_refused), and the client is https_only regardless (fetch.rs::https_client). The sha256 check above holds even over a hostile transport |
A spoofed digest for a non-pinned release, from api.github.com |
S, T | Read over HTTPS verified against the system trust store, only when no digest flag was given, and only by the installer, never the provider: fetch.rs::GitHubDigests. An unreachable or unparseable answer stops the install before any fetch and names the flag to pass (an_unreachable_github_names_the_digest_flag, githubs_release_json_yields_each_assets_digest). What GitHub itself publishes is trusted: §8 |
| A hostile server makes the installer buffer a release document without bound | D | fetch.rs::MAX_RELEASE_DOCUMENT_BYTES, enforced while streaming (get_capped) |
Following a VMM change rewrites more of the host config than [vmm], or a file an admin broke |
T | Only [vmm] is replaced, through the provider's own type, and the result is parsed back before it is written (render.rs::with_vmm, tested with_vmm_replaces_only_the_vmm_section); a file that does not parse is left untouched (the_host_config_is_kept_unless_forced); the EK CA is not rotated (the_host_configs_vmm_section_follows_the_release). The file lives in root-owned /etc/banlieue/, so this write gives the provider's user no lever |
| The provider's user (compromised earlier) plants a symlink in a directory it owns — state root, EK CA directory, storage class — so the root installer chowns, chmods or writes another file | T, E | Every ownership and mode change acts on a handle opened O_NOFOLLOW; every write is a temporary file created O_EXCL beside its target and renamed over it; mkdir refuses anything but a directory — real.rs, tested with a planted symlink (a_symlink_planted_where_a_directory_goes_is_refused). Residual: §8 |
The installer weakens a directory another package owns (polkit's rules.d is root:polkitd) |
T | Directories the OS or a package owns are created when missing and otherwise never re-moded or re-owned — stages.rs::shared_dir, tested; found by a dry run on a host |
| A read-only verb changes a host | T | preflight and status take ops::Probe, which has no mutating methods; selftest removes its scratch directory; asserted in unit tests and in the container test (step 3) |
| A reconcile path reaches root installer code, or its subprocesses | E | Crate boundary: no provider crate depends on banlieue-host (boundary_tests.rs); the subcommand is behind the binary's host feature |
| A setting (flag, cloud-init environment) breaks out of the TOML or a unit file it is rendered into | T, E | Class names are [a-z0-9-], values refuse whitespace and quotes; the host config is serialized from the provider's HostConfig and parsed back before it is written; a template placeholder left unfilled is an error — settings.rs, render.rs, tested |
A guest-slice flag (--guest-cpus, --guest-memory-max …) injects a directive into the root-owned slice unit |
T, E | Each value is parsed against a strict grammar (cpuset list, size with K/M/G/T or 1–100%, weight 1–10000, positive count) before it is rendered, and the unit is written by temp-and-rename — settings.rs::guest_slice, tested budget_values_that_could_corrupt_a_unit_are_refused |
| A hostile server makes the installer buffer without bound | D | Downloads stop at fetch.rs::MAX_ARTIFACT_BYTES before the digest check |
TB-12: Management cluster controllers → external datastore¶
Only with K0S_STORAGE_TYPE=kine (ADR-0085). The datastore is A-17. The
database itself is configured by the operator, not by banlieue, so most
controls here are requirements (§7.18), not code.
| Threat | STRIDE | Control |
|---|---|---|
| The data source's password leaks from the bootstrap host | I | It is read from the operator's untracked env file (BANLIEUE_ENV_FILE), never a flag; the generated k0sctl.yaml that carries it is created 0600 before it is written: scripts/bootstrap-k0s-cluster.sh (generate_k0sctl_config). k0sctl then writes it into each controller's root-owned /etc/k0s/k0s.yaml |
| A crafted data source breaks out of the YAML string into k0s's config | T | A KINE_DATASOURCE containing " or \ is refused: generate_k0sctl_config |
| Traffic between controllers and database is read or altered on the LAN | I, T | Operator requirement: sslmode=verify-full with a CA the operator controls, and a TLS-only hostssl rule on the server (§7.18). sslmode=require encrypts but does not authenticate the server: a LAN host presenting any certificate can intercept the session, the kine password included. Not enforced by the script |
| Recorded datastore traffic is decrypted later by a quantum-capable adversary ("harvest now, decrypt later") | I | Operator requirement (§7.20): PostgreSQL 18 or later with ssl_groups offering X25519MLKEM768, TLS 1.3 minimum. PostgreSQL 17 cannot: its ssl_ecdh_curve takes exactly one classical curve (default prime256v1), which a reference deployment confirmed by refusing both X25519 and X25519MLKEM768. Residual: §8 |
| Anyone on the LAN connects to the database | S, E | Operator requirement: the database listens only where the controllers reach it, and allows only the k0s role from the controllers' addresses with SCRAM (§7.18). Residual: §8 |
| The database is lost or corrupted | D | Operator requirement: backups of the database are the cluster's backups (§7.18). Residual: §8 |
TB-13: LAN and tailnet → management cluster control plane VIP¶
Only with API_VIP (ADR-0086), which the bootstrap script requires for a
cluster with more than one controller. keepalived and the reverse proxy are
k0s components, configured by scripts/bootstrap-k0s-cluster.sh
(render_cplb, require_api_vip).
| Threat | STRIDE | Control |
|---|---|---|
| A LAN host sends VRRP adverts and takes the VIP, then impersonates the API server | S | The proxy passes TLS through without terminating it, and the VIP is only a SAN on the controllers' own certificates: a client that verifies against the cluster CA (every generated kubeconfig, kube-proxy, kubelets, the host providers' kubeconfig) rejects any other server. The VRRP password does not authenticate and is not relied on. Residual (denial of service): §8 |
| The same takeover blackholes the API | D | Not controlled beyond VRRP re-election once the impostor stops. §8 |
| A second cluster on the LAN claims the same virtual router ID and fights over the VIP | D | A per-cluster VRRP password, generated once into $WORKDIR/api-vip-authpass (0600) and identical on every controller, so keepalived ignores the other cluster's adverts; API_VIP_ROUTER_ID to separate them outright (§7.19). api_vip_auth_pass in the script |
| The VIP collides with an address something else holds | D | Operator requirement: reserve it outside DHCP and every MetalLB pool (§7.19). The script has no default VIP and refuses one without a prefix length: validate_api_vip |
A crafted API_VIP or password breaks out of the YAML string into k0s's config, or out of the cloud-init tailscale up line |
T | validate_api_vip accepts only [0-9A-Fa-f.:]+/<digits> for the VIP, digits for the router ID, and 1-8 letters or digits for the password |
| One controller's loss takes the API away from clients outside the cluster | D | The VIP moves to a surviving controller, and the proxy balances over the kubernetes endpoints, which each API server publishes for itself once control plane load balancing is on. A multi-controller cluster without a VIP is refused unless NO_API_VIP=true: require_api_vip |
A tailnet member reaches the API through the advertised /32 route |
S, E | Only the VIP's /32 is advertised, and only after an admin approves it. Reaching the address grants nothing: the API server authenticates every request. Tailnet ACLs decide who may route to it (§7.19) |
| The tailnet's policy is the default allow-all, so every member device and every tagged device (for example a Kubernetes operator's ingress proxies) reaches the VIP | S, E | Reaching the address grants nothing (the API server authenticates every request), but the reachable population is the whole tailnet. Operator requirement (§7.19): grants by group and tag instead of src * dst * |
| The VIP's route is advertised but not yet approved, so a client off the LAN sends API traffic to whatever local network it is on | I, S | The kubeconfig pins the cluster CA: the TLS handshake fails before any client credential is sent (client certificates and bearer tokens both follow server verification). Approve the route before switching kubeconfigs to the VIP (§7.19) |
| With konnectivity off, the API servers dial kubelets (10250) and pods directly | I | The kubelet authenticates and authorizes the API server's client certificate; the nodes were already reachable on the LAN (§8, ADR-0085). Konnectivity stays available (K0S_DISABLE_KONNECTIVITY=false) where the controllers cannot reach the nodes |
TB-14: banlieue pods ↔ telemetry consumers¶
Every role serves /metrics on port 8080 and /livez and /readyz on 8081,
from a hand-rolled listener in crates/banlieue-provider-sdk/src/httpd.rs
(ADR-0091, ADR-0093). Trace export leaves the pod only when an OTLP endpoint
is configured (ADR-0092).
| Threat | STRIDE | Control |
|---|---|---|
Anyone on the pod network reads /metrics |
I | Metric labels carry no object name, namespace or error text, only controller kinds, error variants, Provider names and VM phases; runner_tests.rs asserts no label carries an object's name. Reach is limited by the opt-in ingress rule: metrics only from namespaces labelled banlieue.io/monitoring=true (deploy/network-policies/, §7.21) |
| Label cardinality explodes the scraper (one series per VM or per error message) | D | Bounded by design (ADR-0091): banlieue_virtualmachines{phase} replaces a per-object series; runner_tests.rs checks 50 objects share one series |
| A malformed, slow or oversized request ties up the health or metrics listener | D | The request line is capped at 1 KiB with a 5 s read timeout; anything else is a 400 and the connection is closed (httpd.rs, MAX_REQUEST_LINE_BYTES; httpd_tests.rs) |
| A standby falsely reports Ready, or a healthy standby is reported broken and stalls rollouts | T, D | Readiness is "serving or able to take over" (crates/banlieue-provider-sdk/src/health.rs, evaluate): leader with controllers started, or a standby that read the Lease within one lease duration; leadership is in the body and the banlieue_leader gauge (ADR-0093) |
| The process keeps running with no health server after a failed bind | D | A bind failure is fatal (start_health_server returns BootstrapError::Bind; every role exits non-zero) |
| Spans reach a collector the operator did not intend, or carry sensitive content | I | Off unless OTEL_EXPORTER_OTLP_ENDPOINT/OTEL_EXPORTER_OTLP_TRACES_ENDPOINT is set (crates/banlieue-provider-sdk/src/bootstrap.rs, otlp_enabled); span attributes exclude credentials, user-data and Secret contents (TB-4). The shipped policies open no egress to a collector; the operator adds that rule knowingly (§7.21) |
| Telemetry is forged or suppressed to hide an outage | T, R | Not controlled, and harmless to state: nothing banlieue decides reads telemetry back. Status on the CRs remains the record (§8) |
TB-15: CAPI controllers → banlieue infrastructure objects¶
| Threat | STRIDE | Control |
|---|---|---|
CAPI's manager, or anything that compromises it, creates or deletes VirtualMachines, pools, claims or IPAM claims through banlieue's RBAC |
E | Closed by ADR-0096. The aggregation label is off banlieue-controller; CAPI aggregates only banlieue-capi-infrastructure (deploy/capi/clusterrole-aggregate.yaml), which grants verbs on infrastructure.banlieue.io machines, templates and clusters and nothing else. bootstrap_tests.rs (capi_aggregate_role_grants_nothing_outside_the_infrastructure_group, only_the_capi_aggregate_role_is_aggregated_to_the_capi_manager) and e2e_capi_contract.rs (against real CAPI v1.14.3) enforce it |
| A new infrastructure kind is added without the CAPI role, or the role drifts wider than the contract | E, D | capi_aggregate_role_covers_exactly_the_capi_contract_kinds derives the expected set from the CRDs carrying the contract label, so either drift fails the build |
| CAPI's manager deletes a banlieue machine it owns, destroying a real VM | D | By design: a Machine owns its infrastructure object, and deleting the Machine is the request. A machine banlieue's controller created is owned by its VirtualMachine, not a CAPI Machine |
The clusterctl components file installs something other than what banlieue bootstrap installs |
T | One definition: the file is banlieue bootstrap operator --dry-run, escaped for envsubst, and make clusterctl-components checks that unescaping it gives back the dry-run byte for byte. It ships with the release's provenance (TB-6) |
A published install.yaml installs something other than what banlieue bootstrap installs |
T | Same single definition: make install-yaml writes banlieue bootstrap operator --image-digest <release digest> --dry-run for the release tag, unescaped, with only a comment header added. It is a SLSA subject and in checksums.sha256 (TB-6) |
TB-16: Bootstrap operator → running management cluster (join)¶
Only when the operator runs join (ADR-0099). The tokens, and the copied
/etc/k0s, are A-20.
| Threat | STRIDE | Control |
|---|---|---|
| Growing a cluster rewrites its controllers with stale settings (the k0sctl config predates a VIP, a disabled component or an API server flag), restarting them misconfigured | T, D | join writes nothing to an existing controller and never reads or applies k0sctl.yaml; on a controller it only reads: k0s version, k0s token create, systemctl cat k0scontroller, tar -c /etc/k0s, k0s kubectl get node/controlnode: join_nodes, wait_node_ready, wait_controlnode |
| A joining controller runs with settings the other controllers do not have (another datastore, VIP or API server configuration), splitting the control plane | T, D | It is a copy of a running controller, not of the script's settings: that controller's /etc/k0s and its ExecStart flags, with only the worker flags following the node's role; a --single source and unparseable flags are refused: join_controller_node, controller_install_flags, tested in scripts/bootstrap-k0s-cluster-test.sh |
The datastore password or VRRP secret in /etc/k0s leaks on the way to a joining controller |
I | Read by sudo tar on the controller and extracted by sudo tar on the node, through one pipe between two SSH sessions; nothing is written on the operator's machine; ownership and modes are kept (--numeric-owner, -p), so k0s.yaml stays 0600 root: join_controller_node |
| A join token leaks and joins an attacker's machine, as a worker or, with a controller token, holding the cluster CA key | I, E | Minted per run with --expiry=15m (JOIN_TOKEN_EXPIRY); carried controller → node in one pipe between two SSH sessions, never written on the operator's machine; stored umask 077 under /etc/k0s on the node: join_worker_node, join_controller_node. A node that does join is visible in kubectl get nodes (and a controller in kubectl get controlnodes) and can be removed, but a leaked CA key cannot be revoked short of rotating the cluster CA |
| A node joins at a different k0s version from the cluster | D | The controller's k0s version must equal K0S_VERSION, otherwise join stops before minting a token: join_nodes |
| The k0s binary on the new node is tampered with | T | Downloaded for the pinned K0S_VERSION from K0S_BINARY_BASEURL and checked against the release's sha256sums.txt: stage_k0s_binary. Only when that file can be fetched: if it cannot, the binary is used unverified (§8) |
| Something on the LAN answers as the controller or the new node and receives the token, or hands the node another cluster's token | S, I | Not controlled by the script: ssh_run connects with StrictHostKeyChecking=no, as every step of this script always has. The SSH key authenticates the operator to the node, not the node to the operator (§8) |
| A join runs twice, or after a partial failure | D | A node whose role's service (k0sworker or k0scontroller) is already active is skipped; k0s install --force replaces a half-installed service; a node running k0s in another role is refused rather than reset: join_worker_node, join_controller_node |
| A reset worker joins as a controller over its old Node object, which keeps the worker's labels, so the control plane runs on a node nothing selects as control plane | T | join refuses a controller while a Node of that name exists, and says to delete it: join_controller_node, tested in scripts/bootstrap-k0s-cluster-test.sh |
A swap retires a node, and a later vms recreates it as an empty VM that then waits for a join |
D | Retired nodes are listed in RETIRED_NODES, which vms does not recreate and the node table skips: node_retired, tested in scripts/bootstrap-k0s-cluster-test.sh |
TB-17: Provider identity → its Lease → operator → scheduler¶
ADR-0102 turns each provider's leader-election Lease into an input to placement. The boundary is who may write a Lease, and what the operator and the scheduler do with what they read.
| Threat | STRIDE | Control |
|---|---|---|
A compromised provider rewrites another provider's Lease: renews a dead one so it reads Alive and keeps receiving VMs, lets a live one age into Stale, or takes over its leadership so the real one exits |
S, T, D | No shared ClusterRole but the operator's grants any Lease verb: deploy/{controller,imagebuilder,provider-vsphere,provider-proxmox,provider-libvirt}/rbac/clusterrole.yaml, guarded by bootstrap_tests.rs::no_shared_cluster_role_but_the_operators_grants_leases. Each role writes its Lease through a namespaced Role whose get/update/patch name only its own Lease: the operator's per-Provider Role (crates/banlieue-operator/src/workload.rs, named_rule), banlieue bootstrap's (bootstrap.rs::build_leader_election_role, with --leader-election-namespace set to the install namespace) and the GitOps deploy/*/rbac/role-leader-election.yaml, checked equal by gitops_lease_roles_match_what_bootstrap_builds. Verified with kubectl auth can-i on a real API server: each identity may update its own Lease and no other |
The same grant reaches beyond banlieue: node heartbeats in kube-node-lease, control-plane leader Leases in kube-system |
S, T, E | Removed with the row above: Lease rights are namespaced Roles, so no banlieue identity but the operator can write a Lease outside its own namespace, and none but the operator one it does not name. The operator's cluster-wide Lease grant is §8's "holds the union" |
A provider creates a Lease under another provider's name before that provider ever starts, so an Absent provider reads Alive |
S | Bounded: create cannot be narrowed by resourceNames, so it stays a namespaced rule. A Lease it created but cannot update goes Stale after staleAfter, and Absent schedules exactly like Alive, so the forgery gains no placement it would not have had |
A compromised provider writes status.workload.heartbeat on another Provider directly (every provider ClusterRole holds providers/status patch) |
T, D | Bounded, not prevented: the operator owns that field and re-applies it with force on every reconcile; the forged write is itself a Provider change that wakes the operator, and the requeue is never longer than REQUEUE_DEFAULT_SECS (reconciler/provider.rs::publish_status, heartbeat.rs::requeue_after). The same identity could already rewrite that Provider's conditions and failure domains; ADR-0102 adds nothing it lacked |
A provider that is down keeps Ready=True and keeps receiving VMs (the 2026-10-13 outage) |
D | ADR-0102: crates/banlieue-operator/src/heartbeat.rs (evaluate) records Stale once the Lease is staleAfter (default 60 s) old; crates/banlieue-controller/src/reconciler/scheduler.rs (is_stale) skips it, ProviderNotAlive when none is left; ProviderHeartbeatStale Event and banlieue_provider_heartbeat gauge. Live-tested against a real API server |
Absent is used to slip past the filter |
D | By design Absent (no Lease) is not filtered, so --no-leader-elect runs and Providers from before ADR-0102 keep working. It is a different state from Stale and visible as such (column, gauge); a provider that was ever running leaves a Lease behind and reads Stale, not Absent, when it stops |
| Clock skew between a host and the operator misjudges liveness | D | Bounded: a renewal stamped in the future reads Alive for at most staleAfter, and staleAfter is four lease durations. Skew larger than that is §8 |
Someone labels a Provider banlieue.io/unschedulable to stop placement on it |
D | ADR-0124: the label is a placement filter only (crates/banlieue-controller/src/reconciler/scheduler.rs, is_unschedulable); setting it takes patch on Providers, which already lets its holder change every label a selector matches, so it is no new power. A VM whose every candidate is cordoned is Scheduled=False, reason ProviderUnschedulable, visible and retried. |
| A cordon is bypassed, so a new VM still lands on a host being drained | T | No per-VM override: every VM passes the filter after its selector, before liveness (ADR-0124 decision 4; scheduler_tests.rs: an_unschedulable_provider_takes_no_new_vm, every_candidate_unschedulable_is_provider_unschedulable, unschedulable_is_reported_before_liveness). Only the exact value "true" cordons (only_true_makes_a_provider_unschedulable). |
Cordoning a host, or its provider going stale, recreates the VMs already on it elsewhere (the reconciler re-runs the scheduler every pass, and drift under the default migrationPolicy recreates a VM, losing a local disk) |
D, T | A VM's own Provider (its status.scheduled) stays a candidate through both filters, so neither changes its decision; only new placements avoid the Provider (scheduler.rs, is_own; scheduler_tests.rs: a_placed_vm_keeps_its_provider_when_it_is_cordoned_or_stale, the_exemption_is_only_for_the_vms_own_provider). The provider keeps reconciling its VMs: that is spec.paused, which the label does not touch (ADR-0124 decision 3; ADR-0102 amended, whose stale filter had this gap). |
TB-18: Sandbox segment → egress gateway → routable network¶
ADR-0109 puts sandboxes on a segment with no router and lets them out only through gateway VMs. banlieue's controls are the topology it provisions; the segment's isolation is host configuration and the gateways' behaviour is image content, both recorded where they belong.
| Threat | STRIDE | Control |
|---|---|---|
| A sandbox reaches the routable network directly, bypassing the gateways | I, T | banlieue gives a member no default route: the pool's addressing carries no gateway, so the stamped networkOverrides has none, and the seed's network-config renders a route only when one is set (crates/banlieue-provider-sdk/src/cloudinit/network.rs, test a_static_nic_without_a_gateway_has_no_route); a Kairos member writes only Address= (examples/25-isolated-sandbox-segment.yaml, its shape enforced by crates/banlieue-api/tests/examples.rs, isolated_segment_example_matches_the_types: one NIC on the segment, no pool gateway). A guest that adds a route itself finds nothing that routes: the segment has no router, and on libvirt the isolated network rejects anything forwarded off its bridge (host configuration, §7.29; verified live 2026-11-05: rejected packets counted on the bridge) |
| A gateway forwards between its two NICs, so the segment gains a route | I, E | The image author's control (ip_forward=0, a listener on the sandbox NIC only): §7.28 and the §8 row on two-NIC guests. banlieue puts egress first in the gateway class, so the first static interface (the one placeholders and guestinfo describe, and the only one with a gateway) is the routable one (isolated_segment_example_matches_the_types asserts it) |
| A sandbox attacks a gateway, the one component on both sides | E | Bounded by the gateway's own controls (mutual TLS per claim, per-claim allowlist, no forwarding), which are mediatore's (its ADR-0028), and by the gateway being an ordinary VM: its vTPM, sealing and RBAC are banlieue's as for any VM. §8 records the image's hardening as the image author's |
A sandbox's address collides with a gateway's sandbox address and steals its traffic |
S, D | Partly: two static addresses held by banlieue-managed VMs collide only if ADR-0083 lets them, and it does not (address_conflict.rs). A gateway's sandbox address is set by its image on libvirt and Cloud Hypervisor, which banlieue never sees: §8 |
| A member reaches the gateway's control relay or proxy as another claim | S | Not banlieue's boundary: the gateway authorizes on the caller's claim SVID (mediatore ADR-0027, ADR-0028). banlieue mints no identity for it (ADR-0049, amended) |
| An inbound path into a sandbox appears through a gateway | E | ADR-0082: banlieue injects no SSH key or listener through guest data; a gateway accepts nothing on egress and is not a jump host (image content, Decision 8) |
7. Deployment hardening requirements¶
These are properties of the current design that operators must enforce themselves. They are not bugs; they are the trust model, and deploying against a different assumption is unsafe.
- Every infra machine CR is a credential-bearing resource — restrict
geton all of them.banlieue-controllerresolvesspec.userDatareferences and inlines the rendered content into the infra CR in plaintext (ADR-0025/ADR-0038) —VSphereMachine.spec.userDataand, since ADR-0050,LibvirtMachine.spec.userData— and, since ADR-0062,CloudHypervisorMachine.spec.userData, and, since ADR-0075,ProxmoxMachine.spec.userData— all built by the samebuild_*_machinepath incrates/banlieue-controller/src/reconciler/infra.rs. Anyone who can read one of these can read the user-data that produced it, SSH keys and join tokens included. This reflection is ADR-0025's accepted trade-off (§8), not a bug — but it makesget vspheremachines,get libvirtmachinesorget proxmoxmachinesequivalent to reading every user-data Secret any VM in that namespace has referenced. A new provider inherits this property the moment it gains auserDatafield; it is a contract-level consequence, not a per-provider one. VirtualMachinecreate is no longer a Secret-read grant — provided the admission policies are installed. The controller's Role grantsgeton Secrets and ConfigMaps across the wholebanlieue-systemnamespace with noresourceNames, because the names a validVirtualMachinemay cite are not knowable when the manifest is written. What stops that from becoming an effective namespace-wideget secretsfor anyone holdingcreate virtualmachinesis admission, not RBAC:banlieue-virtualmachine-userdata-authorization(ADR-0042) requires the requesting principal to be authorized for the same read. A cluster that appliesdeploy/controller/withoutdeploy/admission/gets the un-checked version of this grant — see requirement 7. Automation that createsVirtualMachines must therefore holdgeton the user-data it names.- Do not co-locate unrelated Secrets in
banlieue-system. The controller's grant is namespace-wide, notresourceNames-scoped; admission bounds who can trigger a read, not what the controller identity could read if compromised. - Treat
banlieue-imagebuildRoleBindings as node-root grants (§TB-3), and do not grant pod-create there to anyone who should not hold every Provider's hypervisor credentials. ProviderandProviderClasscreation is platform-admin-only. The VAPs bound the damage; they do not make these safe to delegate.- Install the claim subject policy, and audit its ConfigMap.
deploy/admission/virtualmachineclaim-subject-authorization.yamlis what makesspec.subjectan attribution the API server vouched for rather than free text (ADR-0047 Decision 10). It bindssubject.idto the authenticated username, confinessubject.issuerto an allowlist, and makesspecimmutable so the check cannot be undone by a later patch.
Three operator obligations come with it:
- The issuers list ships with a placeholder. A cluster that does not
edit it rejects every real claim — visibly, which is the intended
failure direction.
- The brokers list is the trust concentration. Anyone named there
may attribute a sandbox to any identity, so that ConfigMap is as
sensitive as the audit trail it underwrites. It is empty by default;
alert on changes to it.
- usernamePrefix must match the API server's
--oidc-username-prefix. These are two independently-managed objects
that have to agree, and nothing checks that they do. spec.subject.id
stores the raw subject as the issuer spells it — a JWT carries no
github: prefix, so the prefixed form would leave the in-guest agent a
value it cannot compare (ADR-0047 Decision 9, amended; ADR-0049) — and
the policy re-applies the prefix at comparison time instead. A mismatch
fails safe but silently: see §6/TB-7. Re-check it whenever the cluster's
authentication configuration changes, not only at install.
- One username prefix per issuer. When the API server trusts more
than one issuer, each jwt entry needs its own non-empty prefix, none a
prefix of another (§6/TB-7). Give the claim subject policy the same
pairs in issuerPrefixes (<issuer> <prefix> per line) and list each
issuer in issuers. An issuer without a line falls back to
usernamePrefix, so its users' claims are refused unless that is its
prefix: it fails closed.
The binding is parameterNotFoundAction: Deny, so a missing ConfigMap
blocks claims rather than degrading to "any subject is fine".
Re-applying the policy file resets the ConfigMap, because the file
ships it — the same property vmimage-import-source.yaml has. A
kubectl apply -f deploy/admission/ therefore reverts issuers,
brokers and usernamePrefix to their shipped defaults, and the
placeholder issuer rejects every real claim. Keep site values in your
own overlay or re-apply them after.
- Install the admission policies. Every control in §6/TB-1 is a
ValidatingAdmissionPolicyindeploy/admission/. A cluster that skips them reverts to the pre-hardening threat surface.banlieue bootstrap(ADR-0013) does not emit these policies — it installs workloads, RBAC and namespaces only. Applyingdeploy/admission/is a separate, mandatory step in every install path, including GitOps. Two of the policies (*-credentialsref-authorization,*-userdata-authorization) need an API server new enough for the CELauthorizervariable and are shipped as separate files for exactly that reason; both arefailurePolicy: Fail.virtualmachine-created-by.yamlis the one exception to both of those patterns (ADR-0089): it is aMutatingAdmissionPolicy, needs Kubernetes 1.36+ (higher than everyValidatingAdmissionPolicyfloor above), and isfailurePolicy: Ignoreon purpose — it stamps an audit-convenience annotation, not a security check, so its own failure must never block aVirtualMachinecreate. Skipping it has no security consequence (see the TB-1 row above); it only meansVSphereMachine'sCreatedByvCenter custom attribute is never populated. - Pin
VMImage.spec.sources[].importFromto digests. Thebanlieue-vmimage-import-sourceVAP enforces a registry allowlist supplied as a parameter ConfigMap and fails closed if that ConfigMap is absent — configure it. - If you enable
VMClass.spec.tpmEnabled, use deferred install. The requirement is the same on every backend, for different reasons: - vSphere — the default clone policy duplicates a template's vTPM and
its secrets onto every clone, and banlieue does not set
vpxd.clone.tpmProvisionPolicy. Only per-clone install (ADR-0040) yields a unique, per-VM sealed key. - libvirt — swtpm state is keyed by domain UUID, so a new domain always gets a fresh TPM and the vSphere duplication problem does not arise. Deferred install is still required, because Kairos only ever seals partitions to a TPM present during install (ADR-0040); an already-installed image cannot be encrypted later on any backend.
The pairing is now enforced (ADR-0048, 2026-09-23), closing what
ADR-0040 Decision 5 left open. banlieue-controller rejects a
tpmEnabled: true VMClass paired with an installMode: Immediate
image — or with an image carrying no template block, which is a
pre-built and therefore pre-laid disk — before scheduling, and creates
no infrastructure CR. What used to attach a real TPM and silently
encrypt nothing is now Ready=False, reason=ImageClassMismatch. The
residual operator responsibility is installMode: Manual, the escape
hatch for non-Kairos builds, which banlieue cannot inspect (§8).
First-boot sealing is the one exception (ADR-0088, Accepted
2026-10-26), and it is on by default. An Immediate image may declare
firstBootSealing, so each clone seals COS_PERSISTENT to its own vTPM
on its first boot; --enable-first-boot-sealing=false restores ADR-0048
for every such image. Declare it only on images built from Kairos v4.4.0
or later, which carries first-boot sealing and its fixes
(kairos-io/kairos#4924; #5150, #5160 and #5194, closing #5149, #5158 and
#5193). Keep the operator cloud-config of such images free of
install.encrypted_partitions, and consume such VMs on GuestReady, not
Ready (§8). On Cloud Hypervisor (ADR-0112) the image must also send its
partitions as a sealed= line on the vsock report; banlieue's sealing
layer does, and an Asserted image that does not times out rather than
being reported healthy.
On libvirt, a tpmEnabled image must also export its EK certificate
(ADR-0045). The guest writes the PEM to /run/banlieue/ek.pem — two
lines of cloud-config beside the ADR-0043 phase marker, needing
tpm2-tools in the image — because swtpm keeps no host-side copy for
banlieue to read. An image that cannot do this never reports
GuestReady for a tpmEnabled class and so is never bound: the failure
is a member visibly stuck with reason=TpmEndorsementPending, not one
handed out without an attestation anchor. Machines with
tpmEnabled: false are unaffected.
10. A claim isolates at the VM boundary, not the Kubernetes one — grant
create and delete on virtualmachineclaims narrowly. A
VirtualMachineClaim guarantees that one VM is used by one subject and
then destroyed (ADR-0047). It does not partition the Kubernetes
namespace: two subjects' sandboxes are ordinary VirtualMachines side by
side, so anyone with namespace read sees both bindings, and — per
requirement 1 — anyone who can get the infra CRs can read the user-data
behind either. This is the same single-tenant posture as ADR-0025, applied
to a feature whose name invites the opposite assumption.
Three consequences to enforce by RBAC until Decision 10's policy exists:
- `create virtualmachineclaims` lets the holder record **any**
`spec.subject`, including someone else's. The binding record (A-7) is
only as trustworthy as that grant.
- `delete virtualmachineclaims` destroys running VMs.
- `spec.subject` is world-readable in the namespace and is copied onto the
member as an annotation. Put an identifier there, never a token: the
subject's credential belongs on the consumer's attested channel,
anchored on the member's EK certificate (ADR-0049).
- Cloud Hypervisor hosts are part of the trust base; prepare them with
banlieue host cloud-hypervisor installand keep them to it. The provider on the host is only as contained as the host configuration around it (docs/src/guides/cloud-hypervisor-host-systemd.md):- Install with
banlieue host cloud-hypervisor installfrom a banlieue binary you verified (A-5: its signature and SBOM), since it is what places A-14 (ADR-0067; the script is now an SSH wrapper around it), which registers per-guest uids and private groups, sets0711/2770directories, and installs the polkit rule and the sandboxed provider unit. A host set up by hand without the userdb records cannot start guests; one set up without the directory modes loses guest-to-guest isolation. /etc/banlieue/kubeconfigis0600 banlieue; the host config is0640 root:banlieue. Nothing else on the host should read either.- The provider renews its own token (ADR-0060 Decision 5). To
revoke a host — decommissioned, stolen disk, suspected compromise —
delete its ServiceAccount; every token for it dies at once and the
operator recreates the account. Alert on
serviceaccounts/tokencreation for these accounts from anywhere but their host. - Do not put two hosts' Providers in one namespace unless they are
equally trusted — each host's Role reaches every
CloudHypervisorMachinein its namespace (§8). External Providers must live in the operator's install namespace, the only one where it holds the token grant. - Keep
nsswitch.conf'spasswd/grouplines includingsystemd;preflightrefuses a host without it. - Prefer the pinned VMM release. For another one (ADR-0084), get
each file's sha256 from the upstream release page on a machine that
can reach GitHub, and pass it with
--vmm-sha256/--ch-remote-sha256/--firmware-sha256, rather than trusting a mirror or the host's own view ofapi.github.com. Try a new major version on one host first.
- Install with
- Recommended audit rule: alert on any
ClusterRoleBindingcreated by thebanlieue-operatoridentity whoseroleRefis notbanlieue-provider-*(accepted-risk monitoring for the operator's RBAC-minting capability). - Treat the OCI registry used for Cloud Hypervisor images as holding
guest bootstrap material (ADR-0064, TB-10). Images can carry
cloud-config baked in by
VMImage.spec.cloudConfigs. Use a private repository with access control and TLS (if its certificate is signed by a private CA, name that CA to the imagebuilder with--registry-ca-bundle-*, ADR-0110, and name no more than the one CA); give the imagebuilder a push credential scoped to that one repository, and hosts a pull-only credential. Push access to that repository, together withVMImagestatus write, decides what hosts boot: keep both lists short. Configure retention yourself — banlieue never deletes from the registry. On hosts,keep_unreferencedbounds superseded pulls; a host that is down holds deleted images inTerminatinguntil it returns or itsProvideris removed. - Guard the EK trust bundle like the attestation root it is.
Provider.spec.attestation.ekTrustBundle(A-15) is the list of CAs whose EK certificates a verifier will believe, and banlieue validates its shape, never its content (ADR-0049 Decision 10). Whoever can edit the Provider, or the ConfigMap/Secret it references, can add a CA that vouches for a fake vTPM — and can re-add an issuer an administrator removed, which on libvirt undoes a revocation (§8). Keep the referenced object in the Provider's namespace under the same write discipline as the Provider itself (§7.5), and alert on changes to it — the same monitoring posture as the claim-subject policy'sbrokerslist (§7.6). Populate it per backend: vCenter's issuing CA on vSphere; each host'sswtpm_localcaissuer certificate on libvirt, one entry per host. - Put sandbox VMs on an encrypted storage class where the backend offers
one (roadmap 17 phase F). On vSphere, VM Encryption on the storage
policy the sandbox
VMClass'sstorageClassmaps to makes deleting the VM a cryptographic erase at the datastore layer as well — a second, independent layer under the guest's own TPM-sealed partitions, and the one that still holds for the disk regions Kairos never encrypts. banlieue does not configure this; it is a property of the datastore / storage policy the platform admin maps the class to. - Prepare a Proxmox node with
scripts/bootstrap-proxmox-host.sh, and keep the token privilege-separated. The script creates theBanlieueProviderrole with exactly the privileges ADR-0074 Decision 4 lists, a user and a token whose own ACLs are the only permissions it holds (privilege separation), and grants them on/vms, the node, the SDN zone and the image storages, plus aBanlieueSeedrole on the dedicatedbanlieue-seedstorage alone. Do not point theseed-isoclass at a shared storage, and do not widen the ACLs to/or addVM.Console, guest-exec orPermissions.Modify"to make it work": with them, a leaked token stops being "can create and destroy VMs on this node" and becomes a way into guests. The token's secret cannot be read back, so the Secret inbanlieue-systemis the only copy — treatget secreton it asA-1(§7.2, §7.3). Supply the node's own CA ascaBundlerather than settinginsecureSkipTLSVerify, and re-issue the certificate when the names clients connect by change. Leave thebanlieue-machine-uid=line in a banlieue VM's description alone (§8). -
Give each real network segment its own
networkClass. The duplicate-address check (ADR-0083) keys on(networkClass, address), becausenetworkClassis the finest network identity the controller has. TwonetworkClasses mapped to one segment let two VMs collide undetected. OnenetworkClassmapped to isolated segments on differentProviders makes a legitimate reuse of a subnet read as a collision. The mapping lives inProvider.spec.capabilities.networkClassesand is a platform-admin decision (§7.5). -
Protect a kine datastore as the cluster itself (A-17, ADR-0085).
- Listen only on the address the controllers use, and allow only the
k0s role from the controllers' addresses:
hostsslwithscram-sha-256, nevertrustormd5. Narrow the rule to the node addresses once they are reserved. - Require an authenticated TLS session:
sslmode=verify-fullwith a CA you control.sslmode=requireencrypts but accepts any server certificate. Setssl_min_protocol_version = TLSv1.3. - Back the database up, encrypt the backups, and test a restore: they are the only copy of the cluster's state, Secrets included.
- Consider API-server encryption at rest, so a copy of the database or a backup does not disclose Secrets in plaintext.
- Reserve each node's address in DHCP (
VM_MACSfixes the MACs): a node that changes address breaks its k0s install and the API server's certificate names.
- Listen only on the address the controllers use, and allow only the
k0s role from the controllers' addresses:
- Treat the control plane VIP as an address you own (TB-13, ADR-0086).
- Reserve
API_VIPoutside every DHCP range and MetalLB pool. k0s does no address management, and a collision is silent. - Give each cluster on a LAN its own
API_VIP_ROUTER_ID, and keep the generatedapi-vip-authpassfile with the cluster's other secrets. - Keep kubeconfigs verifying the cluster CA. Never set
insecure-skip-tls-verify: certificate verification is what stops a host that takes the VIP from impersonating the API. - Approve only the VIP's
/32route on the tailnet, and restrict who may reach it with tailnet ACLs. - When converting a cluster built before ADR-0086, move
/var/lib/k0s/manifests/konnectivityoff every controller and delete the objects labelledk0s.k0sproject.io/stack=konnectivity(agent DaemonSet, its ServiceAccount, thesystem:konnectivity-serverClusterRoleBinding): k0s prunes none of them, and the binding keeps grantingsystem:auth-delegatorto an identity nothing uses. - Replace a default allow-all tailnet policy with grants by group and tag before relying on the tailnet as an access boundary.
- Reserve
-
Negotiate post-quantum key exchange on every link that carries cluster secrets, and an issuer on the cluster is a cluster secret. Recorded traffic and stored backups can be decrypted later; signatures cannot be forged retroactively, so key exchange comes first.
- Verify each endpoint by offering only the hybrid group:
openssl s_client -connect <host>:<port> -groups X25519MLKEM768(OpenSSL 3.5 or later; add-starttls postgresfor the datastore). A completed handshake proves the server negotiates it. - Datastore (TB-12): PostgreSQL 18 or later,
ssl_groupsincludingX25519MLKEM768. PostgreSQL 17 cannot. - Backups of the datastore (A-17): encrypt to a hybrid ML-KEM recipient
(age 1.3 or later
mlkem768x25519, or GnuPG 2.5 or later composite keys), not a classical OpenPGP key. - An issuer on the management cluster (TB-7): restrict read on its
storage API group to
cluster-admin, give it its ownrestrictednamespace, and shorten its ID-token lifetime (15 minutes with refresh tokens, not Dex's 24 hour default). - Observed on a reference deployment (2026-10-03): kube-apiserver,
kubelet, the SPIRE server, an issuer behind a Go TLS proxy, and
OpenSSH 10 all negotiate ML-KEM hybrids by default; PostgreSQL 17,
libvirtd's GnuTLS build, WireGuard and banlieue's own clients (§8) do
not. On 2026-10-04 its datastore moved to PostgreSQL 18 with
ssl_groups = 'X25519MLKEM768'(no classical fallback), TLS 1.3 only andverify-fullagainst a private CA; kine (Go) and SPIRE reconnected under that offer, which shows Go clients negotiate the hybrid. - Upgrading the datastore with Debian's
pg_upgradecluster: it does not copyconf.d/, so alisten_addresseskept there is lost and every controller loses the API until it is restored. Copyconf.d/first.
- Verify each endpoint by offering only the hybrid group:
-
Apply the NetworkPolicies, and narrow them (TB-4, TB-14, ADR-0094).
deploy/network-policies/is opt-in, so nothing is restricted until you apply it to every namespace running banlieue pods, aProviderClassworkloadNamespaceincluded. Then replace each "any address" API-server and backend rule with your cluster's API CIDR and your hypervisors' CIDRs; until you do, egress is restricted by port only, and on vSphere port 443 stays open everywhere. Label only your monitoring namespacebanlieue.io/monitoring=true. If you enable trace export, add an egress rule to your collector and nothing wider. Cloud Hypervisor providers run on the host; use its firewall. -
Point trace export only at a collector you operate (A-19, ADR-0092). Spans name objects, namespaces and backend ids. Leave
OTEL_EXPORTER_OTLP_ENDPOINTunset unless you want them, and usehttps://to a collector outside the pod's node. -
Grow a running management cluster with
join, and keep management nodes on Cloud Hypervisor apart from banlieue's guests (TB-9, TB-16, ADR-0099).- Add workers and controllers to a cluster you have changed since
bootstrap with
join, neverconfig+apply, which push the generated configuration back onto every controller. When swapping nodes, add the new controller before retiring the old one. - Treat a controller
joinas a cluster-admin action: its token carries the cluster CA key. Admit the new controller's address to the datastore yourself (on PostgreSQL, onehostsslline for that address), and remove the retired controller's line when it goes. - Run
joinfrom the hypervisor host or another machine on the nodes' network, not across an untrusted segment: the script does not check SSH host keys (§8). - Keep
CH_VM_USERa dedicated account: notbanlieue, not a banlieue guest uid, in no group but its own andkvm. - Cloud Hypervisor is the default VMM, and it needs passwordless sudo on
the host. Set
DEFAULT_VMM=libvirtin the env file of a cluster built on libvirt; the script refuses to run against it otherwise. - Leave
CH_VM_DIRon a filesystem banlieue's storage classes do not share a directory with, and keep its parents searchable but not writable byCH_VM_USER. - Reserve each CH node's MAC in DHCP, as for libvirt nodes: the script finds a node by its MAC, and a k0s node must not change address.
- Add workers and controllers to a cluster you have changed since
bootstrap with
- Build against the
banlieue-apirelease that matches the installed CRDs (A-21, ADR-0100). The crate's version is the release's version; a consumer pinned to it serializes exactly the schema the API server enforces. CommitCargo.lock, so a new upload reaches a build only when someone updates it, and enable one k8s-openapi version feature in the final binary. The crate grants a consumer nothing: give its identity only the RBAC its calls need. -
Install banlieue by digest (A-5, ADR-0103). Apply a release's
install.yaml(already pinned), or pass--image-digesttobanlieue bootstrap operator, taking the digest from the release you verified (crane digest ghcr.io/firestoned/banlieue:<tag>, checked against its cosign signature). A GitOps repository rendered with--dry-runkeeps the digest, so an upgrade is a reviewed change of digest and every Deployment rolls. A mirror (--registry) is pinned to the same digest, which the mirror must serve unchanged. -
Budget the VMs on a shared host (ADR-0105, ADR-0106). Declare the guests' budget to
install(--guest-cpus,--guest-cpu-weight,--guest-memory-max,--guest-tasks-max); change it by re-runninginstall, never withset-propertyonbanlieue-guests.slice, whose drop-in would silently override the declared value. Where management nodes share the host, bound every VM together on the parent:systemctl set-property banlieue.slice CPUWeight=50 MemoryMax=<host RAM less what the host needs>. Size eachMemoryMaxfrom the VMs under it: a cap below their sum makes the kernel kill a VMM. -
One backend host is one
Provider(ADR-0060, ADR-0107). Capacity is counted perProvider: twoProviders pointing at the same Cloud Hypervisor host would each count only their own VMs and together over-place it. On a host shared with other workloads or management nodes, declare a guests' budget (§7.26), so the published capacity is what guests may really have. Setspec.scheduling.cpuOvercommitRatioto1where vCPUs must map to real CPUs. -
A VM with two adapters is a bridge until the guest says otherwise (ADR-0111). A
VMClassthat declares an isolated segment beside a routable one gives its VMs a path between them. Give such a class only to the VMs that are meant to sit on both (a proxy or gateway, ADR-0109), keep IP forwarding off in the guest's own configuration (net.ipv4.ip_forward = 0, and the IPv6 equivalent) and test that, and give an isolated segment its ownnetworkClass(§7.17) so no other class reaches it. banlieue creates the NICs; it does not and cannot check what the guest does with them. Rebuild an image after the in-guestconfigure-networkchanges, or its NICs beyond the first come up unconfigured. - A sandbox segment routes nowhere (ADR-0109). On libvirt define it as
an isolated network, with no
<forward>,<dhcp>or<dns>, and give the host an address on it only if something on the host must be reached from the segment. On vSphere use a port group on a VLAN with no SVI, trunked on every host a member can land on, and list a zone only after a two-host test passes. Keep the pool's address range clear of the gateways'sandboxaddresses. Check it with the guide's table (docs/src/guides/isolated-sandbox-segment.md).
8. Accepted risks¶
| Risk | Why accepted | Revisit when |
|---|---|---|
banlieue-guests.slice is a ceiling, not a reservation (ADR-0105): AllowedCPUs confines guests to a CPU set but does not keep other workloads out of it, and nothing stops the host's own processes from using the memory the budget leaves free. Guests and the host also share one kernel |
Reserving capacity for foreign workloads is the host owner's decision (their own cpuset or slice); banlieue declares only what it may take, and never infers it | Provider.status reports the slice budget instead of physical totals (ADR-0105 follow-up), or a multi-tenant host is supported |
A VM stopped in vCenter is started again by the vSphere provider (ADR-0034 amendment 2026-10-23): an operator's out-of-band power-off is reverted within one long requeue, so it cannot be used to quarantine a VM that banlieue still declares PoweredOn |
The declared state is the source of truth, as for every other provider; observe-only left a poweroff: true install stuck off forever. The hold is a desiredPowerState: PoweredOff write |
A hold mechanism exists that is not a power state (for example a banlieue.io/hold annotation the provider honours), or an incident process needs to freeze a VM without a Kubernetes write |
banlieue-operator holds the union of every permission it can delegate |
Kubernetes escalation-prevention requires it; granting escalate/bind instead would be strictly worse. The ceiling is auditable in one file. This includes Lease get/create/update/patch cluster-wide: it delegates them, narrowed, into every provider's Role, and since ADR-0102 it reads each provider's Lease, so a compromised operator can forge any heartbeat (TB-17) |
A provider needs a materially more dangerous permission |
Liveness assumes clocks agree to within staleAfter (ADR-0102): a provider writes renewTime by its own clock and the operator judges it by its own |
Leader election already assumes the same; staleAfter defaults to four lease durations, and a future timestamp is capped at one staleAfter of grace |
A host's clock is found to drift past staleAfter, or staleAfter is lowered close to the lease duration |
The operator is the only evaluator of liveness: while it is down, every heartbeat stays at its last value, and a provider that dies then keeps Alive |
Liveness is a status field of the operator's (ADR-0012); a second evaluator would be a second writer. The operator's own banlieue_leader gauge is what to alert on, and docs/src/guides/observability.md says so |
Liveness needs to survive an operator outage (e.g. the scheduler reading the Lease itself) |
banlieue-imagebuild runs privileged |
kairos' builder genuinely requires loop devices and chroot; isolation is by namespace | kairos supports rootless builds |
Rendered user-data is visible in VSphereMachine.spec and LibvirtMachine.spec |
Single-tenant, single-namespace posture (ADR-0025). ADR-0042 closed the escalation (a principal reaching user-data it could not read); the reflection to anyone who can already get the infra CR is unchanged and deliberate, and ADR-0050 extends it to a second kind rather than introducing a new risk |
A second tenant or namespace becomes real — ADR-0025's superseded per-VM Role design is the shape that scales |
The controller's user-data Role is namespace-wide, not resourceNames-scoped |
The names a validly admitted VirtualMachine may cite are unknowable when the manifest is written; authorization moves to admission, where the requesting identity still exists (ADR-0042). A compromise of the controller identity itself is still bounded only by the namespace |
The install stops shipping deploy/admission/, or per-VM RBAC becomes tractable |
| A libvirt guest's TPM is emulated by swtpm on the host, so a host-root adversary can read the sealed-key material that a physical TPM would protect | This is the libvirt trust model, not a banlieue choice; the hypervisor operator is already semi-trusted (§4) and hypervisor compromise is out of scope (§9). EK trust anchors differ per backend, and since 2026-09-27 that is explicit rather than implicit: Provider.spec.attestation.ekTrustBundle (ADR-0049 Decision 10, A-15) names which issuers count, per backend, as an admin assertion |
The attestation exchange itself ships (the broker and in-guest agent, outside banlieue), or a libvirt host is no longer operator-trusted |
swtpm EK certificates never expire — the observed notAfter is 9999-12-31 — so validity-period checks are not a revocation mechanism on libvirt |
Nothing banlieue controls: swtpm_localca issues them that way. Expiry would be a weak control regardless, since a sandbox's whole life is measured in minutes. Revocation on libvirt is removing the issuing host's CA from ekTrustBundle — a per-host decision an administrator now makes on a field that exists (ADR-0049 Decision 10, landed 2026-09-27) rather than one a certificate makes for them. §7.14 records the corollary: whoever can re-add an entry can undo a revocation |
A verifier needs a per-certificate revocation story rather than per-host bundle membership |
| On libvirt the EK certificate is reported by the guest, not read from the hypervisor, because swtpm persists no host-side copy | Forced by swtpm's design, not chosen (ADR-0045): the certificate is loaded into the vTPM's NVRAM and the issuing temp directory is deleted, and no libvirt RPC exposes it. It is sound because an EK certificate is a public key — substituting another member's does not yield its private half, so ADR-0049's activation fails — and the subject-CN binding catches the substitution earlier still. The residue is that a guest can withhold its certificate, which denies only its own readiness | libvirt or swtpm grows a host-side read, or a member withholding its certificate becomes something worth distinguishing from one that is merely slow |
GuestReady can be asserted by any code running as root inside the guest, so it proves which disk booted only for a guest that has not been compromised, true on both transports (qemu-guest-agent file read on libvirt, config.extraConfig on vSphere) |
It is a liveness signal by construction (ADR-0043 Decision 9) and is consumed only to decide when a fresh, unclaimed VM joins a warm pool, before any subject has touched it. Treating it as integrity would be the error; the document and the ADR both say so explicitly | The consumer's attestation (SPIRE TPM node attestation against the member's EK, ADR-0049 as amended 2026-11-05) is the integrity signal wherever it runs; this one stays what it is |
| A broker both holds subject credentials and decides which attested guest receives them, so its compromise (or that of the consumer's SPIRE server, which runs the attestation) is the design's worst case | Somebody has to hold the credential to deliver it, and somebody has to decide which guest attested; concentrating both in one audited deployment is preferable to spreading either. banlieue is deliberately not that component (ADR-0049 Decision 2), so a controller compromise discloses no subject credential | The broker is split into deliver/verify roles, or hardware-backed key custody becomes available to it |
| A declared broker may attribute a sandbox to any identity, so the audit trail is only as honest as the broker is | Handing sandboxes out on behalf of other people is a broker's entire purpose; a broker that could only name itself could not broker. The concentration is explicit, empty by default, and confined to one auditable ConfigMap (§7.6) rather than diffused across everyone holding create. mediatore also accepts declarative AgentSandbox requests, so the honesty of the trail rests on mediatore's own subject-pinning admission policy as well as on mediatore; that control is mediatore's and is not verified here |
A broker is compromised; claims need per-request proof of the subject's consent rather than the broker's assertion; or a broker that accepts declarative requests ships without pinning their subject to the caller |
A guest's vTPM can be failed by host CPU contention on stock Cloud Hypervisor v53.0: the VMM gives swtpm 100 ms per response, then marks the TPM fatally failed for the guest's lifetime. A busy host (another guest, a burst of installs) is enough; on a Deferred tpmEnabled member it aborts the install, which the pool reaps at provisioningTimeoutSeconds (D) |
Not banlieue code: it is in the VMM's TPM backend (tpm/src/emulator.rs). The cost is bounded by the pool's reaper, and nothing is disclosed or forged: the TPM refuses, it does not answer wrongly. The fix is known and tested (docs/src/reference/cloud-hypervisor-pool-timings.md); reported upstream as cloud-hypervisor#9053 |
Every Cloud Hypervisor host runs a VMM with the response-deadline fix (upstream release, or a patched VMM installed with --vmm-url) |
| The identity provider is trusted absolutely, and is outside banlieue | The API server is the only thing that can attest a caller, and it attests whatever the configured issuer asserted. banlieue cannot verify an upstream IdP without becoming an IdP. The claim-subject policy is still worth having: it binds an attribution to whatever identity the cluster does authenticate, which is strictly better than free text | banlieue ever needs an attribution stronger than the cluster's own authentication — at which point the answer is per-request proof from the subject (a signed consent, or the attested channel of ADR-0049), not a better check on the caller |
A consumer's cached ID token sits on their laptop (~/.kube/cache/oidc-login) and authenticates as them if stolen |
Not specific to claims — every Kubernetes bearer token behaves this way, and client-side credential custody is out of scope (§9). Recorded because the claim-specific consequence is distinctive: the audit trail records the victim requesting a sandbox, which is exactly the fiction §6/TB-1 exists to prevent. Short token lifetimes are the issuer-side mitigation | Claims carry per-request proof of the subject's intent rather than only the caller's identity |
subject.issuer is allowlisted but never verified: the API server does not reveal which issuer minted the caller's token |
Nothing in Kubernetes attests it directly, but with one distinct prefix per issuer (§7.6) the username prefix identifies the issuer, and since ADR-0047's 2026-10-13 amendment the policy checks it for every issuer with an issuerPrefixes line. For an issuer that falls back to usernamePrefix the allowlist remains the strongest check: it stops a claim naming an issuer the site does not use, which is what would make the recorded attribution meaningless. The claim deliberately carries no token to verify (ADR-0047 Decision 9) |
The in-guest agent's JWT validation lands (roadmap phase C), at which point the guest verifies issuer, audience and oid against the claim |
An installMode: Manual image can claim a deferred install it does not perform, so a tpmEnabled VM built from it is unencrypted and says nothing |
Manual exists precisely for builds banlieue does not drive (ADR-0040), so inspecting it is not possible without becoming its build system. ADR-0048 closes the case banlieue can see (Immediate, and an absent template) and fails closed there; Manual is a deliberate operator assertion, narrower than the blanket gap it replaced |
A backend reports sealed-partition state back to banlieue, making the assertion verifiable rather than trusted Since ADR-0088 an image that declares firstBootSealing is checked by the guest's sealing report whatever its install mode; a Manual image without the declaration remains uncontrolled. |
Ready does not reflect first-boot sealing: a clone whose first boot did not seal is Ready=True with GuestReady=False (ADR-0088 Decision 5 keeps Ready unchanged) |
GuestReady is the readiness input of the consumer that hands VMs out (VirtualMachinePool); gating Ready would regress every image that never reports, as ADR-0043 found |
A consumer other than a pool relies on Ready alone for VMs it treats as sealed |
An Asserted first-boot sealing image (a Template or BackingFile source) can carry a forged report stage that says sealed while the disk is plaintext |
banlieue did not build the disk and cannot inspect it before boot; the declaration is the operator's claim, the same class as Manual (ADR-0088 Consequences). Url builds are not affected: banlieue wrote both policy and report |
Image provenance can be verified (signed images), or a backend can read the partition table host-side |
| A sandbox image that bakes in a key that is not TPM-bound gives every pool member the same identity (ADR-0108) | banlieue ships no sandbox guest software and cannot tell a public certificate from a private key inside a consumer layer, so ADR-0108 Decisions 2 and 3 (no per-VM identity in the image; identity made on first boot onto a sealed partition) are a contract on the image author, stated in the pools guide (Building a sandbox image), the same class as Asserted and Manual. A cloned TPM-bound key fails loudly, since it cannot attest against the clone's own EK; only a key that is not TPM-bound can be cloned silently, and only the consumer's identity layer can notice it, by binding each agent's identity to the EK banlieue publishes |
The consumer's attestation binds every workload identity to the member's EK, or banlieue gains a way to verify an image's contents |
A first-boot sealing image built on a Kairos older than v4.4.0 boots plaintext (no encrypt-pending) or cannot unlock after a reboot (kairos-io/kairos#5193). This replaces the pre-acceptance risk "first-boot sealing runs on pre-release Kairos", retired 2026-10-26 when v4.4.0 shipped #4924 and the fixes |
The builder never learns the source image's Kairos version, so it cannot refuse an old one (ADR-0088 Amendment 2026-10-22, B). Such a clone never reports its partitions sealed, so it is GuestReady=False (FirstBootSealingUnverified or FirstBootSealingTimeout) and a pool never hands it out (§6/TB-5); the docs and example 23 require v4.4.0 |
The builder can read the source's Kairos version (for example /etc/kairos-release in the unpacked rootfs) and refuse an older one |
Blocks written to COS_PERSISTENT before it is sealed are not guaranteed wiped: on the iso path the template install lays the partition out once, and on the raw cloudImage path each clone's recovery reset does; first-boot sealing then turns it into a LUKS container, and nothing in banlieue or ADR-0088 asserts the old blocks are overwritten |
What is written before sealing is image content (the install or reset), not a per-VM secret: per-VM user-data goes to COS_OEM (the row below), and nothing runs from COS_PERSISTENT before encrypt-pending |
A per-VM secret is written to COS_PERSISTENT before first-boot sealing, or Kairos documents that sealing wipes the data area |
firstBootSealingLayer can overclaim on an image changed after its first build: adding firstBootSealing updates the merged cloud-config, but an existing OSArtifact is not rebuilt, so status says the layer is in the build while the artifact predates it |
The guest readback still withholds GuestReady from such a clone, so the overclaim is in provenance, not in what gets handed out; the guides say to recreate the VMImage |
The OSArtifact rebuild is keyed on the merged cloud-config's content |
An artifact is rebuilt needlessly when a worker's Node object is deleted and re-registered with its disk intact (ADR-0115): the node's new creationTimestamp reads as a replacement |
The cost is one rebuild (about eight minutes) per image on that node, and the opposite error, an ISO gone while Ready is reported, is what the gate exists to stop. A volume with no node affinity, or a read that fails, is never judged lost |
Revisit when storage that survives node loss is the default, or when a probe of the file itself is added |
First-boot sealing does not protect COS_OEM, where per-VM user-data (A-2) is persisted in plaintext for the life of the VM |
Kairos reads its sealing policy from COS_OEM before anything is unlocked, so OEM stays plaintext by design (ADR-0088 Consequences); the example and guides say so |
Secrets stop travelling in user-data (short-lived join tokens, fetching secrets at boot from an authenticated source) |
The NoCloud cidata seed stays attached after install, so a guest can read its own rendered user-data (A-2) from inside the sandbox |
ADR-0044 Decision 4, deliberate and scoped: cloud-init re-reads its datasource on every boot, so removing the seed risks regressing per-boot modules in a way that needs its own live verification on a reboot — not a first boot. Ejecting the installer removes the build-time overlay shared across every VM, which is the broader exposure; what remains is each guest's own material, which that guest's workload could in principle obtain anyway | The seed eject is verified live across a reboot, or user-data delivery stops needing a persistent datasource |
A stolen host token (A-11) can patch any CloudHypervisorMachine in its namespace — another host's included — clearing finalizers or rewriting status |
Machine names are not known when the Role is written, so resourceNames cannot scope them. Providers and their status are scoped since External mode (2026-09-26). The token still cannot read Secrets or create or delete machines. Operators bound it by namespace (§7.11) |
Per-host machine scoping becomes possible (a label-selector authorization, or one namespace per host) |
| A stolen host token can renew itself, so it does not expire on its own | Renewal by TokenRequest is what lets a host run unattended (ADR-0060 Decision 5), and the API cannot tell the host's request from a thief's. Revocation is immediate and total — delete the ServiceAccount (§7.11) — and every renewal is an audited API call | Token requests can be bound to a host attestation (a TPM-bound key), or the API server gains per-token revocation |
banlieue-operator holds serviceaccounts/token create in its install namespace, so a compromised operator can mint tokens for other ServiceAccounts there, the controller's included |
RBAC requires a grantor to hold what it grants, and each External provider's Role carries renewal on its own ServiceAccount. Held namespaced, never cluster-wide (deploy/operator/rbac/role.yaml; a test fails if the ClusterRole ever gains it). The operator already holds broader grants (SEC-007) |
The operator stops needing to delegate token creation — e.g. the provider renews through a narrower API |
| Guest addresses reported on status come from the host's neighbour table, which the guest influences; the tap is a plain bridge port with no MAC/IP filtering | Same posture as a libvirt NAT network. Addresses are a convenience for consumers, not an identity — the same reasoning as GuestReady (ADR-0043 Decision 9). Static IPAM (ADR-0024) gives an address the guest did not choose |
Bridge-level anti-spoofing (nftables bridge family) is added, or an address becomes load-bearing for authorization |
A compromised provider process reaches every guest on its host: it can read their disks and reconfigure taps (CAP_NET_ADMIN) |
Inherent in a host-resident hypervisor manager — something on the host has to own guests' storage and networking, and the alternative (root) is strictly worse. It holds three capabilities and no others, can manage only instances of its root-owned templates for uids in the guest range (so it cannot start anything as root), and writes only the run root, the storage classes and its own state directories | A split into a minimal privileged helper and an unprivileged reconciler becomes worth its complexity |
Hosts verify an image's integrity (digest) but not its provenance: whoever can write VMImage status and push to the configured repository chooses what hosts boot |
Digest pinning already defeats a registry that substitutes content, and the host pins the repository. Signing needs a key-management story of its own, which ADR-0064 defers to its own ADR (cosign/sigstore) | Signature verification on pull lands, or the repository is shared with parties outside the platform team |
The imagebuilder's identity can create Jobs in banlieue-imagebuild, a privileged namespace, so its compromise is node-root-equivalent there |
It already drove privileged builds through OSArtifacts; the push Job must run beside the PVC it reads. The Job it creates holds no API credential and no privilege |
The push can run outside the privileged namespace (e.g. a restricted namespace with a read-only clone of the artifacts volume) |
| The push Job's CA bundle (ADR-0110) is a trust anchor, not a pin: any certificate the named CA issues for the registry's host name is accepted, and a CA that issues for other names widens what the Job would trust | The operator names the CA, banlieue adds it to the public roots rather than replacing them, the bundle is reachable only through the imagebuilder's own configuration, and the host never uses it (it pulls by digest from its own repository with its own trust store). The exposure is the push credential and the artifact in transit | A per-registry certificate or public-key pin is added, or the same bundle is offered to anything that reaches a host or a tenant |
| Images in the registry can carry cloud-config (A-2), readable by the registry's operator | The registry is the operator's, like the datastore in TB-5; banlieue cannot encrypt what a host must boot without a key-distribution scheme it does not have | Per-VM secrets move entirely out of shared images, or images are encrypted for their hosts |
The Cloud Hypervisor API decoder (banlieue-cloud-hypervisor) is not fuzzed |
The peer is a local VMM the provider started, not a network endpoint; decoding is into typed, bounded structs and failure is reported, not fatal. The libvirt decoder is fuzzed because its peer is remote | A fuzz target is added alongside the libvirt one, or the client ever talks to a VMM it did not start |
| A non-pinned VMM release is verified against GitHub's published digest or the operator's flag, not a pin reviewed in a banlieue change, so a release replaced on GitHub itself (a compromised upstream account) would verify | Only when an operator names a version other than the pinned one, per host and explicitly (ADR-0084 Decision 4). It still catches a corrupted or substituted download from any mirror, and Cloud Hypervisor publishes no signatures to check instead. The pinned release keeps its compiled digests | Cloud Hypervisor signs its releases (then verify the signature), or a fleet needs non-pinned releases routinely enough to pin them in banlieue instead |
| A newer VMM than the pinned one runs against client types checked only against the pinned release's API document | The client sends only fields that exist in the pinned schema (spec_tests.rs), and the gate refuses older releases, which is where removals would bite. A newer release could still change a field's meaning; the operator chose it, and the guide says to try a new major on one host first |
A newer release is pinned in banlieue (vendored spec, digests, tests), or the client learns to read the running VMM's API document |
| With kine, the management cluster's datastore is one database on one host: if it is down the API server cannot read or write, and if it is lost so is the cluster's state | A choice the operator makes per cluster (K0S_STORAGE_TYPE=kine, ADR-0085), for state outside the VMs and ordinary database backups; etcd remains the default. Running guests are unaffected while the API is down; controllers resume when it returns |
The management cluster needs to survive its database host, then a replicated database or etcd |
Every TLS connection banlieue itself makes uses classical key exchange. All binaries use rustls with the ring crypto provider (ADR-0009), which implements X25519, P-256 and P-384 but no ML-KEM, so the controller's and operator's API traffic, the providers' sessions with vCenter, Proxmox and libvirtd (carrying A-1), and the Cloud Hypervisor provider's cluster token (A-11) can be recorded now and decrypted by a future quantum-capable adversary |
ADR-0009 chose ring to avoid aws-lc-rs and OpenSSL in the build; post-quantum key exchange was not a requirement then. Changing the provider is an architectural decision that needs its own ADR |
An ADR supersedes ADR-0009's provider choice with rustls's aws-lc-rs provider (X25519MLKEM768 by default), or ring gains ML-KEM |
| Classical key exchange on links banlieue does not terminate: the kine datastore on PostgreSQL 17, libvirtd's mutual TLS (GnuTLS builds without ML-KEM negotiate X25519 only), and the WireGuard tailnet (Curve25519, no pre-shared key) | Each is fixed upstream or by the operator, not in banlieue: PostgreSQL 18's ssl_groups, a distribution GnuTLS with ML-KEM, Tailscale post-quantum key exchange. Inside the tailnet, API traffic is itself post-quantum TLS where the endpoints support it (§7.20) |
The operator moves to PostgreSQL 18; a GnuTLS with ML-KEM reaches the hypervisor distributions; Tailscale ships post-quantum key exchange |
| No post-quantum signatures anywhere: the cluster CA, kubelet and libvirt certificates, SPIRE's X.509 CA, OIDC ID tokens (RS256) and public CA certificates are all RSA or ECDSA | Forgery needs a quantum computer at the time of the attack, not a recording, and ML-DSA is not yet supported by Kubernetes, Go's crypto/x509 issuance paths, SPIRE or JOSE implementations |
ML-DSA (FIPS 204) is supported by Kubernetes certificates, SPIRE, the OIDC issuer and the API server's JWT verification |
| An OIDC issuer running on the management cluster lets a cluster admin mint any identity, so claim attribution (TB-7) is no stronger than the cluster-admin boundary | Cluster-admin is already out of scope (§9), and a separate identity cluster is more infrastructure than a single-operator site needs | Management-cluster admins and claim subjects become different trust populations; then run the issuer outside the management cluster |
| Any LAN host can take the control plane VIP by sending VRRP adverts at a higher priority, and blackhole the API until it stops (ADR-0086) | VRRP's simple password is not authentication, and keepalived offers nothing stronger that k0s exposes. The takeover cannot impersonate the API (TLS is end to end, §6/TB-13); it is the same class of LAN denial of service as ARP spoofing a node's own address, which the bridged nodes already accept | The LAN carries hosts that should not be able to disturb the cluster, then a dedicated VLAN, or unicast VRRP with a host firewall admitting only the controllers |
| With konnectivity disabled, the API servers reach every kubelet and pod directly (ADR-0086) | Both backends build flat networks where that reach exists anyway; the kubelet authenticates the API server; konnectivity's alternative pinned every agent to one controller and broke webhooks on the others | A node joins from a network the controllers cannot reach, then K0S_DISABLE_KONNECTIVITY=false with node-local load balancing |
With LIBVIRT_BRIDGE, the management cluster's nodes are on the LAN, so every LAN host reaches their API server, kubelet and node ports |
Needed for host-resident providers and DNS clients to reach the cluster without tailscale (ADR-0085). The API server and kubelet authenticate every request; nothing banlieue adds listens unauthenticated | The LAN carries hosts that should not reach the cluster at all, then a dedicated VLAN or host firewall rules |
The installer's O_NOFOLLOW protects only a path's last component: a directory the provider's user owns could be swapped for a symlink between the installer's check and its use, one level up |
The window is one run of a root command started by an operator; the directories are created (and a symlink there refused) earlier in the same run; openat2(RESOLVE_NO_SYMLINKS) would forbid legitimate symlinks on the path, such as a storage class under a linked mount |
A host where the provider's user is untrusted while an install runs; then resolve beneath each banlieue-owned root with openat2(RESOLVE_BENEATH) |
The Proxmox provider's ClusterRole can update/patch any ProxmoxMachine in any namespace, so a compromised provider pod can clear finalizers or rewrite status on machines placed on other Providers |
Machine names are not known when the role is written, so resourceNames cannot scope them, exactly as for libvirt. It still cannot read Secrets it is not named for, and cannot create or delete machines (deploy/provider-proxmox/rbac/clusterrole.yaml); its Proxmox reach is bounded by its own Provider's token ACLs |
Per-provider machine scoping becomes possible (a label-selector authorization, or one namespace per Provider) |
The ownership marker is not a secret and is not signed: anyone with VM.Config.Options on the Proxmox side can delete the banlieue-machine-uid= line (orphaning a VM: banlieue then clones a second) or copy it onto another VM (which banlieue would then treat as its own, and configure, start and finally destroy). On vSphere, anyone with VirtualMachine.Config.AdvancedConfig can do the same to banlieue.machine.uid (ADR-0101): deleting it orphans an unrecorded clone (the next pass reports VMNameConflict), copying it onto a same-named VM in the target folder gets that VM destroyed on the machine's next pass without a vmRef, or adopted when the machine's recorded VM is gone (ADR-0101 amendment 2026-11-04, and only when it is the single marked VM) |
The party who can do that is the hypervisor operator, already semi-trusted and already able to destroy any VM directly (§4); the marker exists to stop accidental collisions between namespaces and with unrelated VMs, which the name could not, not to defend against the hypervisor's own administrators. The token itself holds VM.Config.Options on /vms, so a compromised provider can edit descriptions too (bounded by §4's stolen-token row); the vSphere provider's account writes extraConfig on every clone, so the same holds there. A tenant cannot plant a marker ahead of time: a machine's UID is assigned by the API server, not chosen by whoever writes the VirtualMachine |
A machine's UID becomes something a tag ACL can protect, or a per-VM signed marker is worth its key management |
That a vSphere guest cannot reach banlieue.machine.uid is documented, not tested (ADR-0101): the control rests on the guest RPC (vmware-rpctool info-get / info-set) being confined to guestinfo.* |
That confinement is how VMware Tools' guestinfo channel is defined, and the impact is low even if it failed: a guest could orphan its own VM on an interrupted pass, or mark it with another machine's UID, which matters only for a VM of that machine's name in that machine's folder | The first ADR-0101 run against a real vCenter: from inside the guest, vmware-rpctool "info-get banlieue.machine.uid" must fail and info-set must not change the key |
A vSphere Template source trusts whatever template of that name sits in the zone's folder: banlieue checks the name and the template flag, not what boots from it, so anyone with write access to <rootFolder>/<failure-domain> chooses what every clone there boots (ADR-0104) |
The same posture as the Proxmox Template row below and ADR-0064's deferred signing; per-zone folders narrow it from the whole datacenter (first match) to one folder, and the party who can write there is the hypervisor operator (§4), who can already destroy any VM directly |
Template provenance (a signed image reference recorded on the template) is available, or images are imported by banlieue |
A Template source trusts whatever the administrator put in that VMID: banlieue verifies only that it is a template, not what boots from it |
Proxmox has no image provenance to check, and banlieue builds none of these templates (the bootstrap script imports a cloud image over HTTPS from its published URL). The same posture as vSphere Template sources and as ADR-0064's deferred signing |
Template provenance (a signed image reference recorded on the template) is available, or images are imported by banlieue |
| The Proxmox provider has not run as a deployed controller: the lifecycle passed live through the reconciler functions (2026-09-30), but not through a cluster (finalizers, SSA status, controller dispatch), and not with EFI, vTPM, guest-agent addresses or a failing task | Each of those is unit-tested against the fake and the API calls under them are now observed live | A kind-cluster e2e runs a VirtualMachine to Ready on a Proxmox node |
| The Proxmox seed ISO stays attached for the machine's life (as for libvirt and Cloud Hypervisor), so a guest can read its own rendered user-data | Same reasoning as the NoCloud entry above: cloud-init re-reads its datasource on every boot, and ejecting it needs its own live verification across a reboot. What remains is each guest's own material | Detaching the seed is verified live across a reboot on Proxmox, or delivery stops needing a persistent datasource |
Address squatting: anyone who can create a VirtualMachine on a networkClass can claim an address on it first, and ADR-0083 then blocks everyone after them |
First-come-first-served is how the network itself behaves. ADR-0083 makes that visible rather than creating it. The alternative (pre-assigned address ownership) is IPAM, and CAPI IPAM pools (ADR-0033/0053) are where per-tenant address ownership belongs | A networkClass is shared between tenants who do not trust each other, or CAPI IPAM lands and static overrides can be restricted to IPAM-issued addresses |
Cross-namespace "address in use" oracle: a tenant can learn whether an address is claimed on a networkClass by any VM in any namespace, by declaring it and reading the condition |
One bit per probe, never a name (the holder is unnamed across namespaces, §6/TB-1). The same fact is observable on the wire by anyone on the segment, which is where the address actually lives. Consistent with the single-tenant posture (ADR-0025) | A second tenant becomes real on a shared networkClass; then scope the check per tenant or move address ownership into IPAM |
Health and metrics endpoints bind 0.0.0.0 without authentication, and /readyz says whether this replica leads |
Probes come from the kubelet, not a pod, so they cannot authenticate; leadership is also readable from the Lease. Metrics carry no object names (TB-14). ADR-0093 retired the old "fixed 200" entry: the answers are now real |
A metric gains a label that names an object, or a non-probe client needs these ports |
| The NetworkPolicies are opt-in, and their destinations are "any address" until narrowed | banlieue cannot know a site's API server or hypervisor CIDRs, and a wrong egress rule breaks the product silently; there is no templating layer to fill them in (ADR-0094). Port-level restriction already stops most lateral movement | The operator gains RBAC to write per-workload policies with addresses from the Provider (ADR-0094 follow-up) |
| A rotated credential is used for up to ~30 s before banlieue picks it up, and a revoked one keeps failing reconciles until the Secret is updated | Watching Secrets would grant every provider read access to every Secret in its namespace (ADR-0095). Immediate cut-off belongs on the backend, which stops a credential everywhere at once | A provider caches a client across reconciles (the cache must then key on resourceVersion), or Kubernetes can scope a watch to one named Secret |
| Telemetry is not integrity-protected: a party on the path or with scraper access can forge or suppress it | Nothing banlieue decides reads telemetry back; the CRs' status is the record (TB-14) | An automated decision (autoscaling, remediation) starts consuming banlieue metrics |
Management nodes on Cloud Hypervisor share one host user (CH_VM_USER), so a guest that escapes into one node's VMM can read and write the other management nodes' disks and vTPM state on that host (ADR-0099) |
They are nodes of one cluster, and one compromised node already reaches what the cluster's workloads reach; a uid per node, as banlieue gives its guests, would add bookkeeping without separating different trust. banlieue's guests remain separate from it (TB-9) | Nodes of more than one cluster, or of different trust, are put on one host as Cloud Hypervisor nodes (the default VMM) |
The bootstrap script does not verify SSH host keys (StrictHostKeyChecking=no in ssh_run); with join that session also carries a join token, and for a controller the cluster CA key's token and the datastore password (A-20) |
Every node is new, or rebuilt, when the script first reaches it, so there is no key to pin yet; a token expires in 15 minutes, though a controller's is enough to take the CA key within them. Running it from the host or the nodes' own network keeps it off untrusted paths (§7.23) | The script records each node's host key at creation (the seed can carry one) and checks it afterwards |
stage_k0s_binary uses the k0s binary unverified when the release's sha256sums.txt cannot be fetched, which join and the vSphere backend both rely on |
An air-gapped mirror may publish no checksum file, and failing closed would break those estates; the binary is fetched over HTTPS from K0S_BINARY_BASEURL |
A pinned digest setting (as IMAGE_SHA256 does for the installer ISO) is added, or the mirror case is handled explicitly |
Provider condition messages are mirrored verbatim onto user-facing VirtualMachine status |
Useful diagnostics; providers are in-tree | A third-party provider ships |
A compromised VMM can skew banlieue host cloud-hypervisor overhead: naming a thread vcpu<N> (or its process swtpm) moves its time between columns, so a VMM's own overhead can be reported as guest time |
The verb is a diagnostic an operator reads, not an input to scheduling, billing or any control; the guest's total is unchanged, and the slice's own cpu.stat (also reported) is kernel-accounted and cannot be renamed. Telling vCPU threads apart without trusting their names needs the VMM's API (vm.info), which the installer crate does not speak |
Overhead numbers feed an automated decision (placement, billing, alerts) |
| Capacity is checked at placement, not reserved (ADR-0107): VMs scheduled at the same moment can each fit against the same free memory, and a VM can be placed against capacity a provider has not re-published since another VM landed | The controller reconciles one VM at a time per leader and counts every VM whose status.scheduled names the failure domain, so the window is between two concurrent passes, not across them; an over-placement still ends at the guests' slice MemoryMax, killing a guest rather than the host. A reservation would need a write-ahead record per placement |
Concurrent scheduling (more than one controller replica placing at once) is introduced, or an over-placement is observed |
Capacity is counted per Provider (ADR-0107): two Providers on one host each see only their own VMs |
ADR-0060 makes one host one Provider, and §7.27 says so; nothing in the CH provider's install or status flow creates a second Provider for a host | Several Providers per host become a supported topology |
Memory used outside banlieue's VMs is not deducted from published capacity unless a slice budget leaves it out: the host's own services, and management nodes in banlieue-nodes.slice |
The guests' budget (ADR-0105) is the way to state what guests may have on a shared host (§7.26); measuring and subtracting everything else would make capacity move under load | A host's non-guest load is measured reliably enough to publish |
A banlieue-api upload carries no signature or provenance, unlike the binary and images (cosign, SLSA, ADR-0006), so a consumer cannot verify which build produced it |
crates.io offers none to sign with; the upload comes from the tagged checkout in the release run, and Cargo.lock checksums pin what a consumer got the first time |
crates.io supports artifact attestations or Sigstore signing |
CARGO_REGISTRY_TOKEN is a long-lived secret (A-21): if it leaks, it publishes until revoked |
The first upload needs a token; scope it to publish-new and publish-update and, once the name exists, to banlieue-api only, with an expiry (ADR-0100 Decision 6) |
The crate exists on crates.io and the publish action accepts Trusted Publishing (GitHub OIDC), so the secret can be deleted |
| A guest with two NICs can forward between its segments, and banlieue does not check that it does not (ADR-0111) | banlieue provisions the adapters and cannot inspect a guest; the isolation of a segment is a property of the guest that bridges it. §7.28 asks for forwarding off, and the first consumer (the egress gateways, ADR-0109) writes it in its own cloud-config and tests it | A policy can state which VMClasses may join an isolated networkClass, or the guest's forwarding state is readable from outside (a status field or a scan) |
| A gateway's sandbox address is image content on libvirt and Cloud Hypervisor (ADR-0109 Decision 6), so a pool range that overlaps it is not detected (TB-18) | Three gateways are three per-image constants; ADR-0083 sees only addresses banlieue manages. The guide and §7.29 say to keep the range clear | A gateway's sandbox address is set through banlieue (as ADR-0111 does on vSphere), so ADR-0083 covers it |
| The egress gateways' hardening is the image author's control (ADR-0109): forwarding off, a listener on the sandbox NIC only, the per-claim allowlist, no inbound on egress. banlieue provisions the VM and cannot inspect what runs in it | The software is consumer image content by design (ADR-0108, ADR-0109 Decision 5); a check banlieue made would see a disk, not a policy. mediatore's ADR-0028 and its image tests hold these controls | banlieue gains a way to verify an image's runtime configuration, or the gateway becomes banlieue code (ADR-0097 rules that out) |
| A VMClass edit does not change a VM that already exists (ADR-0111, as ADR-0024): the NICs a VM was cloned with are the NICs it keeps, so removing an interface from a class does not detach running VMs from that segment | Reconciling a changed NIC list on a running VM is a reconfigure with its own failure modes (a NIC that is in use, a guest that has configured it) and is out of scope of the clone; the operator who retires a segment must rebuild its VMs | A VM's NIC list is reconciled after creation, or VMClass becomes immutable once a VM uses it |
| The pre-lay Job is privileged and runs a guest under QEMU/KVM on a build node (ADR-0113): a QEMU escape lands in a privileged container on that node | The build namespace already runs privileged kairos-operator builds (row above), so the namespace boundary is unchanged; QEMU runs seccomp-sandboxed, with no network and no TPM, and the guest is the image being built. --build-node-selector keeps it on dedicated build nodes | QEMU can run unprivileged in the Job (non-root, a KVM device plugin instead of privileged) |
| The pre-lay image ships the findings Debian has not fixed (ADR-0113, amendments 2026-11-04): it is a Debian 13 userland (QEMU, OVMF, e2fsprogs, gdisk, util-linux) run in a privileged Job, and its Grype gate (make grype-prelay) fails only on a high or critical finding with a fix. Built 2026-11-04, it carried 82 high or critical findings, every one wont-fix or not-fixed in Debian. A Debian fix is also missed while the cached apt-get install layer is reused, until the base is re-pinned or the image rebuilt without the cache | No rebuild can remove an unfixed finding, and a gate on them would be switched off. Every finding is in Code Scanning; the Job holds no API token, QEMU runs seccomp-sandboxed with no network and no TPM, and the guest it boots is the image being built. The image's SBOM is published with each release | A slimmer base (QEMU and OVMF without a full Debian userland), the gate is widened to unfixed findings with curated .vex/prelay/ statements, or Code Scanning shows an unfixed critical in a package the Job reaches from its input |
| A VMImage deleted after one of its vSphere Providers leaves that Provider's zone templates in vCenter (ADR-0087, Amendment 2026-11-01) | Nothing can reach that vCenter with a deleted Provider's credentials; before the amendment the delete hung forever instead. The workload logs which Provider's templates it left | An operator-side cleanup for orphaned templates exists, or Provider deletion learns to wait for its VMImage templates |
9. Out of scope¶
- Anything requiring cluster-admin as a starting position (
SECURITY.mdpolicy). - Compromise of the hypervisor itself, or of vCenter/libvirt/Proxmox authorization — including root on a Cloud Hypervisor host, which owns every guest on it.
- Guest-OS hardening after boot; banlieue's responsibility ends at delivering the bootstrap material.
- Address conflicts with machines banlieue did not create. ADR-0083
checks
VirtualMachines only; the provider-side lookup that would catch another writer on the same segment is deferred until it can be verified against a real backend. deploy/kind/— development-only, not held to production standard.- The MkDocs documentation toolchain (
docs/), which ships nothing at runtime. - The Cloud Hypervisor host's OS packages (systemd, polkit, swtpm,
ca-certificates): installed and kept current by the host's own package manager or image build, never by banlieue (ADR-0084 Decision 1).preflightchecks that the commands exist, not where they came from. - The agent layer above
VirtualMachineClaim:AgentSandbox, token brokering, the in-guest agent and their attestation. They are mediatore's (ADR-0097), covered by mediatore's own threat model; banlieue's share ends at the claim and the EK certificate it mirrors. - Removing banlieue from a Cloud Hypervisor host (
uninstall, host drain): not built (ADR-0067 Decision 8);banlieue host cloud-hypervisor statusshows what a manual removal must cover.
10. Maintenance¶
This threat model is a first-class artifact under Architecture Driven Development (ADR → CALM → TDD → implement → docs → threat model). It is the last step of the cycle: after any ADR is implemented, a full pass over this document is mandatory, and an ADR does not count as implemented until that pass is done.
A full pass walks every section above — components, assets, actors, trust
boundaries (including the §5 diagram), the STRIDE tables, hardening
requirements, and the accepted-risk register — rather than appending a row to
whichever table obviously changed. Each new or changed threat must name a
control that actually exists in deploy/ or crates/, or be recorded in §8
with an explicit Revisit when. The pass then bumps the header stamp above:
the date and the ADR range. "No change" is a valid conclusion, but the
stamp still advances — an unchanged stamp means no pass happened.
Sections most likely to move: a new provider or binary (§2, §4), a new CRD or contract (§3, §6), a new namespace or PSA level (§5), a new identity or RBAC grant (§6, §7), a new external dependency in the boot path (TB-6), or any change to the single-tenant assumption in §7.
Auditing the CALM model counts as a trigger. TB-7 exists because
architecture.json gained an network-oidc-issuer node and the wires around
it, and the question "is that actor in the threat model?" answered no — for
an actor every claim attribution already depended on. The two documents
describe the same system from different angles, so a component that is new in
one is a prompt to check the other.