Changelog
User-visible changes only. Internal refactors and infrastructure work omitted.
2026-11-17 — IPv6 egress#
- Bunkers on
registry,allowlistedandunrestrictednow have IPv6 as well as IPv4. Outbound IPv6 leaves from the worker's public address; when a name has both, IPv4 is used. The resolver forregistryandallowlistedanswers AAAA queries for permitted names. IPv6 traffic is filtered, audited and counted inegress_byteslike IPv4. Bunkers started earlier have IPv4 only until recreated. The bunker record has a newipv6field (nullfor bunkers created before this release), and the worker record shows whether a worker offers IPv6. See Network profiles.
2026-10-24 — Exec log retention and erasure; stream keep-alives; per-user rate limit#
- The exec log is now kept 30 days, and a tenant admin can erase a user's exec log and large outputs at once:
POST /admin/erasure/user/{user_id}. See API reference. - Streamed execs send a
: keepalivecomment every 15 seconds while the command prints nothing, so proxies and browsers don't drop a quiet stream. env_varsare encrypted at rest.- The shell WebSocket refuses browser pages from origins that aren't allowed (
403before the socket opens). Clients that send noOriginheader, such as servers, are unaffected. - Requests made with a user token (JWT) are rate-limited per user instead of per client IP, so a service that relays many users' tokens no longer shares one limit across all of them.
- Quota profiles assigned to a group now apply when the caller's token doesn't list the user's groups (for example, exchanged tokens and API keys).
- Corrected the docs: idle
sessionbunkers are paused, not snapshotted, and resume only onPOST /resume; the response envelope includesstatusandmeta.
2026-10-22 — Quota usage fix; clearer placement error#
GET /quota-profiles/usageworks again. It returned404because the request was read as a profile namedusage.- A
label_setthat no worker carries now returns422 NO_WORKER_WITH_LABELSinstead of503 WORKER_UNAVAILABLE.label_setselects workers; it isn't a tag. - The API reference now shows the real list response shape:
data.itemswithpagination.has_moreandpagination.next_cursor.
2026-10-21 — Exec users and root; allowlist ports; more egress audit#
- Bunkers have a default
user(POST /bunkers), and exec and the shell take one per call (user,?user=): a name, uid or uid:gid. The bunker record returns it asuser. scaibunker:rootis enforced. Without it, bunkers run asbunker, root is refused (403 BUNKER_PERMISSION_DENIED), and file calls on a bunker that runs as root are refused. Until now every command ran as root.- New errors:
422 INVALID_USER,422 USER_NOT_FOUND,409 USER_UNSUPPORTED(a bunker started before this release; recreate it). - Without
scaibunker:root, exec and the shell also refuse any user that is uid 0 in the image, such as atooralias (403 ROOT_REFUSED). Bunkers started before this check answer409 NO_ROOT_UNSUPPORTED; recreate them. - File calls now run as the user, like exec: a path that user can't access gets
403 FILE_PERMISSION_DENIED, and file calls take?user=. Withoutscaibunker:root, file calls on a bunker that runs as root now run asbunkerinstead of being refused. Before this, reads and deletes ran as root whatever the bunker's user. - Breaking for images that give users
sudo: withoutscaibunker:root, exec and the shell can no longer gain root throughsudo,suor other setuid programs; setuid bits are ignored for those calls. Grantscaibunker:rootto the identity that needssudo. Rootless Podman (newuidmap/newgidmap) andpingkeep working. Bunkers started before this change keep the old behaviour until recreated. - Allowlist entries can name a TCP port:
github.com:22. Without a port, 80 and 443. - Egress audit now covers
allowlistedandregistrybunkers too, withaction: allow|denyper record. POST /bunkersnow checks the whole request body: invalid allowlist entries, out-of-range values and unknown enum values return422 INVALID_BUNKER_REQUESTinstead of failing on the worker.- Bunkers now support containers: rootful and rootless Podman run inside a bunker when the image has them.
2026-10-20 — Ready for GPU bunkers; placement and snapshot fixes#
- The API is ready for GPU bunkers:
gpu_count(and optionallygpu_type, e.g.rtx-4090) onPOST /bunkers, gated by the new permissionscaibunker:create:gpu. Whole GPUs are passed through and ScaiBunker supplies the NVIDIA driver. No GPU capacity is available yet; until it is, a GPU create returns503 WORKER_UNAVAILABLE. See API reference. - Images marked
gpu_required(such aspython-3.12-ml) now refuse bunkers without a GPU:IMAGE_REQUIRES_GPU. - GPU bunkers can be paused and resumed but not snapshotted or restored into:
SNAPSHOT_NOT_SUPPORTED. Persistent GPU bunkers get no checkpoints. - Bunkers without GPUs avoid GPU workers while another worker fits.
- A create that reaches a worker that turned out to be full now tries the other workers, and returns
503 WORKER_UNAVAILABLEif none fits, instead of200with afailedbunker. - GPU access is opt-in per tenant:
scaibunker:create:gpumust be granted explicitly (admin roles don't include it), and the default quota profile allows no GPUs. GPU tenants get thegpu-standardprofile: up to 8 cores, 32 GiB memory, 100 GiB disk and 1 GPU per bunker, and 2 GPUs per tenant. - Large-file uploads work:
POST /files/uploadnow returns anupload_urlto PUT the file to, andPOST /files/commitonly accepts that bunker's own upload keys (UPLOAD_NOT_FOUNDotherwise). - Output over 1 MiB is saved: exec responses carry
exec_idandfull_output_available, andGET /bunkers/{id}/exec/{exec_id}/outputreturns a download link. Kept 30 days. - Egress audit is on for
unrestrictedbunkers:GET /bunkers/{id}/audit-batcheslists per-flow records, kept 30 days. - Streamed execs are now recorded in the bunker's exec log like other execs.
- Failed bunkers say why: new fields
failure_codeandfailure_messageon the bunker record (e.g.GPU_SETUP_FAILED,WORKER_LOST). - Worker records list their hypervisors in
runtimes. - The
minimalimage's shell no longer loses input typed before its prompt appears (scripted shells over the WebSocket). max_lifetime_snow has a ceiling: 24 h forephemeral, 72 h forsession(MAX_LIFETIME_TOO_LONG). Negative timeouts are refused (INVALID_BUNKER_REQUEST); before, a negativemax_lifetime_smeant no limit.env_varsare deleted when a bunker ends.
2026-10-19 — Interactive shell, private registries, archive uploads; bunkers survive worker outages; stricter restores#
-
The interactive shell works:
WebSocket /bunkers/{id}/shell(needsscaibunker:shell), one per bunker, with resize, anexitframe carrying the exit code, and a 15-minute idle close. In a browser, pass the token as thebearer.<token>sub-protocol. The admin console has a terminal on the bunker page. Sessions are audited; their content isn't recorded. See API reference. -
Custom images can come from private registries: store a login once with
POST /registry-credentials(encrypted, never returned) and register images from that registry as usual. See Register a custom image. -
Custom images can be uploaded: register with
build_source: {"kind": "archive", "size_bytes": …}, PUT adocker saveor OCI archive to the returned URL, then callPOST /images/{id}/upload/complete. New build statusuploading, new errorIMAGE_UPLOAD_INVALID. -
Image endpoints now check ownership: you can read your tenant's, your partner's and platform images, and change only your tenant's. Partner images need a partner admin, platform images a super admin.
-
When a worker is gone for good, an admin retires it (
POST /workers/{id}/retire) and its session and persistent bunkers come back on another worker, under the same id, from their latest snapshot. A worker silent for 2 hours is retired automatically, as long as another worker is still running. Ephemeral bunkers and bunkers without a snapshot fail. See Architecture. -
Persistent bunkers are snapshotted automatically (
trigger: "checkpoint"): every 6 hours while running, and before an idle pause if they changed. The newest two are kept. -
A
lostbunker past itsmax_lifetime_sis now terminated even if its worker never returns. -
A running bunker whose VM stops answering (about 70 seconds without a response from inside the VM) now becomes
losttoo: it isn't billed, and comes back by itself when the VM answers again. Terminate it, or restore one of its snapshots into a new bunker, if it doesn't. -
Deactivating an image now also removes it from the workers' caches.
-
Bunkers and snapshots show the image version they run on (
image_version). A restore always uses the snapshot's version, so snapshots keep restoring after an image is updated. -
nodejs-22now ships npm 12 instead of the npm 10 bundled with Node.js 22, clearing the high-severity findings in npm's own dependencies. Node.js itself is unchanged (22.23.3). Snapshots taken on the previous version restore on that version. -
If a bunker's worker stops sending heartbeats for 5 minutes, the bunker becomes
lostinstead offailed. It isn't billed while lost and comes back by itself when the worker returns, in the state the worker reports. If it passed itsmax_lifetime_smeanwhile, it's terminated; if the worker no longer has it, it becomesfailed. See Architecture. -
A restore (
snapshot_idon create) keeps the source bunker's CPU, memory and disk. Asking for different values returnsSNAPSHOT_RESOURCES_MISMATCH, and an incompatible network profile returnsSNAPSHOT_PROFILE_MISMATCH. -
Restores are placed on a worker whose CPU can run the snapshot.
-
Images are scanned once, on ingress, and again only when their content changes; the daily re-scan is gone. Platform images carry the scan from their build, tied to their checksum.
-
Custom images are now built by ScaiBunker from a public OCI reference: pulled, flattened, boot-tested without network and scanned before any worker sees them. A fixable critical vulnerability or a missing shell refuses the image. New image fields
build_statusandbuild_error; new errorsIMAGE_NOT_READY,IMAGE_NAME_CONFLICT,IMAGE_SOURCE_UNSUPPORTED. Tarball sources are no longer accepted. See Register a custom image.
2026-10-17 — Platform images built from distribution sources#
- The six built-in images are now built by ScaiLabs from Ubuntu and Alpine sources and official runtime downloads, instead of public container images.
- Every build is boot-tested in a Firecracker microVM, scanned, and published with an SBOM and an
image.jsonmanifest. - Images are rebuilt monthly, plus urgent security rebuilds. Versions are immutable (
1.<YYYYMMDD>.<n>). - See Platform images.
v0.1 — Initial release#
First public iteration.
- Firecracker microVMs with three lifecycle modes: ephemeral, session, persistent.
- Structured execution API: command run (inline and streaming), file read/write/delete/list, staged upload/commit.
- Interactive PTY shell over WebSocket (relay implementation in progress).
- Five network profiles: isolated, registry, allowlisted, unrestricted, plus v0.2 transit appliances with tenant-scoped Linux bridges and per-interface
spoof_guard. - Per-bunker egress shaper (
bandwidth_mbit) on every profile. - Image system: OCI and tar sources, ext4 size cap, lazy-pull flag, automatic Trivy security scans with daily refresh.
- Availability groups: tenant- and platform-scoped, image / worker membership drives fan-out warming.
- P2P image distribution: workers fetch ext4 bytes from peers in the same group before falling back to the controller storage proxy.
- Snapshots: manual, idle, terminate, checkpoint, failure triggers; tar.gz archives in S3; per-file download without full restore.
- Egress audit batches: NDJSON in S3 for
unrestricted-profile bunkers, listed and fetched through the API. - Quota profiles: per-user
individual, per-groupsharedandper_userassignment modes; most-restrictive-wins composition; live/usageendpoint. - Capability tokens for the storage proxy: short-lived JWTs scoped to a specific verb and path; admin-mintable via
POST /storage/capabilities. SCAIGRID_MODE=bunker_proxydeployment: stripped-down storage-proxy-only FastAPI app, no DB or Redis, horizontally scalable behind a load balancer.- ScaiCore integration: cores declare bunker capabilities in their manifest; activation auto-provisions a sandbox bound to the entity.