---
title: Your first provisioning recipe
path: tutorials/your-first-provisioning-recipe
status: published
---

# Your first provisioning recipe

This walks you through writing, validating, and shipping a provisioning recipe end-to-end. By the end you'll have a working `provision.json` for a service called `scaifoo` and you'll know enough to do the same for any new ScaiLabs service.

For the conceptual model behind every step in this tutorial, see [Concepts → Provisioning workflows](../concepts/provisioning-workflows).

## What you'll build

A 6-step provisioning recipe for a hypothetical `scaifoo` service:

```mermaid
flowchart TD
    A[validate] --> B[gen_creds]
    B --> C[store_vault]
    B --> D[call_provisioning_api]
    C --> D
    D --> E[health_check]
    E --> F[notify]
```

When a tenant subscribes to scaifoo, ScaiControl will:

1. Verify `scaikey` is already provisioned (a prerequisite for scaifoo)
2. Mint a `(client_id, client_secret)` pair scoped to the tenant
3. Store the credentials in ScaiVault
4. Call scaifoo's provisioning API to create the tenant on its side (async — scaifoo callbacks when done)
5. Verify scaifoo's health endpoint comes back 200
6. Email the tenant admin that provisioning is complete

## Prerequisites

- A local checkout of the service repo where this recipe will live (`scaifoo/`).
- The `scaicontrol` CLI on your PATH (`pip install -e scaicontrol/backend/` in a venv).
- Optional: the sandbox stack running locally for end-to-end testing — see [Sandbox](../sandbox).

You **don't** need a database connection for `validate` — that's by design. CI in your service repo can validate the JSON without any ScaiControl infrastructure.

## Step 1 — scaffold the file

Create `scaifoo/provisioning/provision.json` with just the wrapper:

```json
{
  "max_retries": 3,
  "timeout_seconds": 3600,
  "steps": []
}
```

The two top-level fields are workflow-wide defaults: `max_retries` is the global retry budget for the whole instance (not per-step — steps get their own); `timeout_seconds` is the workflow's hard ceiling.

Validate:

```bash
scaicontrol admin workflow-defs validate scaifoo/provisioning/provision.json
```

You'll get a single error: `EMPTY_DEFINITION — definition must include a non-empty steps[] array`. Expected.

## Step 2 — first step: validate dependencies

Scaifoo depends on scaikey being provisioned first. Add the preflight check:

```json
{
  "max_retries": 3,
  "timeout_seconds": 3600,
  "steps": [
    {
      "name": "validate",
      "type": "sync",
      "handler_class": "ValidateDependencies",
      "depends_on": [],
      "config": {
        "dependencies": ["scaikey"]
      }
    }
  ]
}
```

A few things to notice:

- `name` must be unique within the workflow. Use snake_case; the validator doesn't enforce that, but it makes input-mapping references readable.
- `handler_class` is a short name resolved against the [handler registry](../concepts/provisioning-workflows#handler-registry). `ValidateDependencies` is one of the seven built-ins.
- `config` is handler-specific. The `dependencies` key tells `ValidateDependencies` which service slugs to check. The shape is documented per handler in the [handlers reference](https://www.scailabs.ai/docs/scaicontrol/reference/api/admin-provisioning) (`GET /admin/provisioning/handlers`) or by clicking the handler in the visual designer's Handlers tab.

Re-validate:

```bash
scaicontrol admin workflow-defs validate scaifoo/provisioning/provision.json
```

Now you get `OK — definition is valid` with a single warning: `NO_COMPENSATE_HANDLER — step 'validate' has no compensate_handler`. That's fine — `validate` has no side effects, nothing to compensate.

## Step 3 — mint credentials

Add the credentials step. It depends on `validate` and produces outputs that downstream steps will need:

```json
{
  "name": "gen_creds",
  "type": "sync",
  "handler_class": "GenerateServiceCredentials",
  "depends_on": ["validate"],
  "config": { "credential_type": "api_key", "key_length": 32 },
  "compensate_handler": "GenerateServiceCredentials"
}
```

`compensate_handler` points back at the same class — the handler implements both `execute()` (mint) and `compensate()` (revoke). On rollback after a partial provisioning failure, the credential gets cleanly revoked instead of orphaned.

What does this step produce? Click the handler in the visual designer (or fetch `GET /admin/provisioning/handlers` and find the entry for `GenerateServiceCredentials`):

```
produces_outputs:
  credential_type: str
  client_id:       str
  client_secret:   str
```

We'll wire those into the next two steps.

## Step 4 — store in vault

The credentials go into ScaiVault at a tenant-derived path. This step **depends on `gen_creds`** and **consumes its outputs**:

```json
{
  "name": "store_vault",
  "type": "sync",
  "handler_class": "StoreCredentialsInVault",
  "depends_on": ["gen_creds"],
  "config": {
    "vault_path_template": "tenants/{tenant_id}/services/scaifoo"
  },
  "input_mapping": {
    "client_id":     "gen_creds.client_id",
    "client_secret": "gen_creds.client_secret"
  }
}
```

The `input_mapping` is the explicit wire between steps. Format:

- `gen_creds.client_id` — pull from the upstream step's `result` dict, key `client_id`.
- `context.tenant_id` — pull from the workflow's frozen [context](../concepts/provisioning-workflows#workflow-context).

The vault path template uses `{tenant_id}` etc. — those are interpolated from context by the handler itself (not by the engine). Check each handler's docs for which template tokens it accepts.

## Step 5 — call scaifoo's provisioning API

This is the async part. ScaiControl POSTs to scaifoo's `provisioning_api_url` with the tenant + credentials, and waits for scaifoo to callback `/api/v1/provisioning/callback` with the result.

```json
{
  "name": "call_provisioning_api",
  "type": "async_callback",
  "handler_class": "CallProvisioningApi",
  "depends_on": ["store_vault"],
  "config": {
    "method": "POST",
    "path": "/v1/tenants",
    "timeout": 30
  },
  "input_mapping": {
    "client_id":     "gen_creds.client_id",
    "client_secret": "gen_creds.client_secret"
  },
  "compensate_handler": "CallDeprovisioningApi",
  "max_retries": 5,
  "retry_delay_seconds": 60,
  "timeout_seconds": 600
}
```

What changed:

- `type` is `async_callback` (not `sync`). The handler returns immediately with `status: waiting_callback`. The step sits there until scaifoo POSTs back with the matching `callback_token`.
- `depends_on` is `["store_vault"]`, but `input_mapping` references `gen_creds.client_id` — that's allowed because `store_vault` itself depends on `gen_creds`, so `gen_creds` is in this step's transitive dep set. The validator does NOT enforce direct-dep wiring for inputs; if you'd rather be strict, add `gen_creds` to `depends_on` explicitly.
- `compensate_handler: "CallDeprovisioningApi"` — on rollback, ScaiControl asks scaifoo to tear down whatever it created.
- Bigger retry budget + longer timeout — async network calls deserve more headroom.

## Step 6 — health check + notify

Two final sync steps:

```json
{
  "name": "health_check",
  "type": "sync",
  "handler_class": "HealthCheck",
  "depends_on": ["call_provisioning_api"],
  "config": { "timeout": 10, "expected_status": 200, "retries": 3, "retry_delay": 2.0 }
},
{
  "name": "notify",
  "type": "sync",
  "handler_class": "SendProvisioningNotification",
  "depends_on": ["health_check"],
  "config": { "template": "provisioning_complete", "channel": "email" }
}
```

`health_check` GETs the service's `health_check_url` (from the service registry, available in context). It has its own internal retry-with-backoff via the `retries` config key.

`notify` is a fire-and-forget — failure here doesn't unwind the provisioning.

## Step 7 — validate the full recipe

```bash
scaicontrol admin workflow-defs validate scaifoo/provisioning/provision.json
```

Expected output:

```
OK — definition is valid.
  [warning] NO_COMPENSATE_HANDLER  at /steps/0/compensate_handler  step 'validate' has no compensate_handler — rollback will skip it
  [warning] NO_COMPENSATE_HANDLER  at /steps/3/compensate_handler  step 'health_check' has no compensate_handler — rollback will skip it
  [warning] NO_COMPENSATE_HANDLER  at /steps/5/compensate_handler  step 'notify' has no compensate_handler — rollback will skip it
```

Warnings are fine — they're for steps that don't need compensation. Errors would block import.

For CI integration, use `--format json` to get machine-readable output:

```bash
scaicontrol admin workflow-defs validate provision.json --format json | jq '.ok'
```

Exits non-zero on any error, so it's pipeline-friendly.

## Step 8 — test in the sandbox

Before shipping to prod, run the recipe end-to-end against a local stack. See [Sandbox](../sandbox) for setup. Quick version:

```bash
docker compose -f tools/sandbox/docker-compose.sandbox.yml up -d
python tools/sandbox/seed.py

# Import the recipe into the sandbox DB:
DATABASE_URL=mysql+pymysql://root:sandbox@localhost:3307/scaicontrol_sandbox \
  scaicontrol admin workflow-defs import scaifoo/provisioning/provision.json \
    --service scaifoo --type provision

# Trigger a provisioning run by creating a sandbox subscription on scaifoo
# (use the admin API or the seeded tenant_admin user via the portal):
curl -X POST http://localhost:8001/api/v1/admin/subscriptions \
  -H "Authorization: Bearer $SANDBOX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "tenant_id":   "tnt_sandbox_platform_a",
    "partner_id":  "prt_sandbox_platform",
    "service_slug":"scaifoo",
    "plan_id":     "<plan-uuid>"
  }'
```

Watch the resulting workflow at `http://localhost:8001/admin/provisioning` → Workflows tab. Each step should turn green; the `call_provisioning_api` step will sit on `waiting_callback` until scaifoo's mock POSTs the callback.

## Step 9 — deploy to prod

CI in your service repo runs `validate` on every PR. On merge, your CD pipeline imports against prod:

```bash
scaicontrol admin workflow-defs import scaifoo/provisioning/provision.json \
    --service scaifoo --type provision \
    --description "Initial provisioning recipe for scaifoo"
```

Output: `Imported scaifoo/provision v1 (id=<uuid>)`.

From this point on, every new subscription to scaifoo triggers your recipe. Re-running the import with identical content is a no-op (`No-op: identical to existing v1`); changing the content auto-bumps to v2.

## Editing later

The big rule from [Concepts](../concepts/provisioning-workflows#definitions-are-versioned): **once any workflow has attached to a definition, the `steps_definition` is immutable.** Edits go through clone-as-new-version, either:

- CLI: edit the JSON file, then `scaicontrol admin workflow-defs import` again — auto-bumps.
- Admin UI: open `/admin/provisioning` → Definitions → `Clone & Edit` on the current row → make changes → save. Pops out as v+1.

Old in-flight workflows keep running on the version they started on. New ones pick up the new version.

## Authoring in the visual designer instead

Everything above can be done in the designer instead of by hand-editing JSON:

1. Click `+ New Definition` on the Definitions tab.
2. Fill in `service_slug` + `workflow_type`.
3. Click handlers in the left palette to drop step nodes onto the canvas.
4. Drag from a node's bottom port to another node's top port to wire `depends_on`.
5. Click a step → property panel on the right. Edit `name`, `handler_class`, `type`, retries, etc. The `config` form is auto-rendered from the handler's declared `config_schema`. The `input_mapping` form lets you pick source step + output via dropdowns — picking a step that isn't yet a dep wires the edge for you.
6. `Auto-layout` re-flows nodes into clean layers.
7. `Validate` round-trips to the same validator as the CLI.
8. `Create` saves.

The visual designer and the JSON file produce the same `steps_definition` — they're two interfaces over one underlying model. The "Visual" / "JSON" tab toggle inside the editor switches between them mid-edit.

## Common pitfalls

| Symptom | Cause | Fix |
|---|---|---|
| `HANDLER_NOT_FOUND` on a name you expected to exist | Handler isn't in the built-ins and the entry-point plugin isn't installed on this ScaiControl instance | Either fix the handler name, or have the service team ship a package with the right `[project.entry-points."scaicontrol.provisioning.handlers"]` | 
| `CYCLE` error | A → B → A somewhere in `depends_on` | Walk the listed steps to find the loop; visual designer highlights the affected nodes |
| Step stuck on `waiting_callback` | Service never POSTed `/provisioning/callback`, or used the wrong `callback_token` | Check the service's logs. The token lives on the step row + the workflow row — service should grab it from the body of the inbound provisioning request |
| Step retries to exhaustion + workflow ends `failed` | Genuine handler failure | Admin recovery: `/admin/provisioning` → Workflows → expand row → `Retry Step`. Once you've fixed the underlying problem, retry resets `status=pending` and the scheduler picks it up |
| `IDEMPOTENCY_KEY_MISMATCH` from a write | A retry sent the same key with different body | The retry layer in your call site is wrong — the body must be byte-identical across retries |
| Subscription is created but no workflow appears | No active definition for `(service, type)` | `scaicontrol admin workflow-defs list --service scaifoo --include-inactive` — confirm a v1 is active |

## See also

- [Concepts → Provisioning workflows](../concepts/provisioning-workflows) — the model behind the moving parts.
- [Reference → CLI](../reference/cli) — every `workflow-defs` subcommand.
- [Reference → admin-provisioning API](../reference/api/admin-provisioning) — endpoints, request/response schemas.
- [Reference → state machines](../reference/state-machines) — workflow + step lifecycles, transitions.
- [Sandbox](../sandbox) — local docker-compose stack for safe end-to-end runs.