Platform
ScaiWave ScaiGrid ScaiCore ScaiBot ScaiDrive ScaiKey Models Tools & Services
Solutions
Organisations Developers Internet Service Providers Managed Service Providers AI-in-a-Box
Resources
Support Documentation Blog Downloads
Company
About Research Careers Investment Opportunities Contact
Log in

Migrate from .env Files

You have a fleet of services reading credentials from .env files or environment variables, set by your deployment pipeline. This tutorial walks through moving them into ScaiVault with no downtime and a path back if anything breaks.

The pattern works for any "credentials in env" setup: 12-factor apps, Docker Compose, Kubernetes ConfigMap-as-env, GitHub Actions secrets, Heroku config vars. The principle is the same — read from a single canonical place at startup.

What you need#

  • ScaiVault token with secrets:read and secrets:write.
  • One service to start with. Pick a low-stakes one for the first pass.
  • Optionally: the Python SDK or CLI for scripting bulk imports.

1. Inventory what you have#

Before importing anything, list what you're trying to migrate. For a typical Python service:

bash
1
2
3
4
5
6
7
$ cat .env.production
DATABASE_URL=postgresql://...
STRIPE_SECRET_KEY=sk_live_...
SALESFORCE_CLIENT_ID=3MVG9...
SALESFORCE_CLIENT_SECRET=...
SLACK_WEBHOOK_URL=https://hooks.slack.com/...
SENTRY_DSN=https://...

Categorize each one:

Variable Sensitive? Should migrate? Notes
DATABASE_URL Yes (contains password) Yes Split into DB_HOST, DB_USER, DB_PASS, DB_NAME later
STRIPE_SECRET_KEY Yes Yes High-value, rotate quarterly
SALESFORCE_* Yes Yes Group as one JSON secret
SLACK_WEBHOOK_URL Yes-ish (URL contains a token) Yes
SENTRY_DSN No (intentionally public) No Keep in env

Anything not sensitive — port numbers, feature flags, public URLs, Sentry DSNs — stays in env. ScaiVault is for things that should not appear in kubectl describe pod.

2. Decide your path scheme#

A path scheme is a contract: a service knows where its secrets live, the policy model knows what to permit, the dashboard knows what to show. Bad path schemes haunt you for years.

A scheme that scales:

text
1
<scope>/<env>/<service>/<credential>

Examples:

text
1
2
3
4
5
environments/production/billing/database
environments/production/billing/stripe
environments/staging/billing/database
shared/integrations/salesforce/oauth        # used by multiple services
infra/postgres/reporting/root               # privileged backing creds

Recommendations:

  • environments/<env>/<service>/... for per-service per-env credentials.
  • shared/... for things multiple services use (one Salesforce credential, many consumers).
  • infra/... for backing-service root credentials (DB roots, AWS deploy keys) that should rarely be read by application code.

Don't put environment in the credential name (stripe-prod) — that gets out of sync with the actual deployment environment. Put it in the path.

3. Import one service#

For the billing service:

bash
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
# Stripe secret
scaivault secrets write environments/production/billing/stripe \
  secret_key="$STRIPE_SECRET_KEY" \
  --type api_key \
  --tag billing,stripe,production

# Salesforce — group fields under one secret
scaivault secrets write environments/production/billing/salesforce \
  --type json \
  --json-file - <<EOF
{
  "client_id": "$SALESFORCE_CLIENT_ID",
  "client_secret": "$SALESFORCE_CLIENT_SECRET"
}
EOF

# Database — split connection components
scaivault secrets write environments/production/billing/database \
  --type json \
  --json-file - <<EOF
{
  "host": "db.internal",
  "port": 5432,
  "database": "billing",
  "username": "billing_app",
  "password": "..."
}
EOF

Verify:

bash
1
scaivault secrets list --prefix environments/production/billing/

4. Grant the service access#

Create a service account in ScaiKey for the billing service (or use one that exists), then a narrow policy:

bash
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
curl -X POST https://scaivault.scailabs.ai/v1/policies \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "billing-prod-reader",
    "rules": [{
      "path_pattern": "environments/production/billing/**",
      "permissions": ["read"]
    }]
  }'
# -> {"id": "pol_billing_prod"}

# Bind
curl -X POST https://scaivault.scailabs.ai/v1/policies/pol_billing_prod/bindings \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"identity_type": "service_account", "identity_id": "sa:billing-prod"}'

The billing service's token can now read its own credentials and nothing else.

5. Adapt the service code#

Goal: minimal diff, ability to fall back to env vars if ScaiVault is unreachable at startup. This rollback path is the whole reason to do staged migration — without it, ScaiVault becomes a hard dependency before you've earned trust in it.

python
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
import os
from scaivault_sdk import ScaiVaultClient, ScaiVaultError

def load_config():
    use_vault = os.environ.get("USE_SCAIVAULT", "false").lower() == "true"

    if use_vault:
        try:
            client = ScaiVaultClient(
                base_url=os.environ["SCAIVAULT_URL"],
                token=os.environ["SCAIVAULT_TOKEN"],
                timeout=5.0,
            )
            stripe = client.secrets.read("environments/production/billing/stripe")
            sf     = client.secrets.read("environments/production/billing/salesforce")
            db     = client.secrets.read("environments/production/billing/database")
            return {
                "stripe_secret_key": stripe.data["secret_key"],
                "salesforce_client_id": sf.data["client_id"],
                "salesforce_client_secret": sf.data["client_secret"],
                "database_url": f"postgresql://{db.data['username']}:{db.data['password']}@{db.data['host']}:{db.data['port']}/{db.data['database']}",
            }
        except ScaiVaultError as e:
            if not bool(os.environ.get("SCAIVAULT_FALLBACK_ALLOWED", "")):
                raise
            print(f"WARN: ScaiVault failed ({e.code}); falling back to env", flush=True)
            # fall through to env path

    return {
        "stripe_secret_key":         os.environ["STRIPE_SECRET_KEY"],
        "salesforce_client_id":      os.environ["SALESFORCE_CLIENT_ID"],
        "salesforce_client_secret":  os.environ["SALESFORCE_CLIENT_SECRET"],
        "database_url":              os.environ["DATABASE_URL"],
    }

6. Roll out gradually#

Stage 1: deploy the code to staging with USE_SCAIVAULT=true and SCAIVAULT_FALLBACK_ALLOWED=true. Verify it picks up values from ScaiVault.

Stage 2: roll the same code to production with USE_SCAIVAULT=false. Wait a day. Nothing should change — you've just deployed code that can read from ScaiVault but isn't yet.

Stage 3: flip to USE_SCAIVAULT=true in production, keep fallback on. If anything breaks (token problem, network issue), the env values bail you out and you see warnings in logs.

Stage 4: after a week with no fallback warnings, set SCAIVAULT_FALLBACK_ALLOWED=false. The service now hard-fails if ScaiVault is unreachable. This is what you want long-term — silent fallback hides problems.

Stage 5: remove the env vars from the deployment pipeline. Now they only exist in ScaiVault.

7. Audit-check the migration#

bash
1
2
3
4
5
6
# Verify the service is actually reading from ScaiVault
scaivault audit query \
  --identity-id sa:billing-prod \
  --start "1 hour ago" \
  --json \
  | jq -r '.logs[] | "\(.timestamp) \(.action) \(.secret_path)"'

You should see steady reads from the paths you imported. If sa:billing-prod shows zero activity but the service is running, it's still using env vars — something didn't roll out.

8. Wire rotation policies#

Now that the secret is canonical, put a rotation policy on the ones that warrant it:

bash
1
2
3
4
5
6
7
# Stripe — quarterly rotation; you'll trigger it via Stripe admin then PUT back
scaivault rotation create --name "api-key-90d" --interval 90d --grace-period 48h
scaivault rotation assign rot_api-key-90d environments/production/billing/stripe

# Database — weekly, auto-generated
scaivault rotation create --name "db-passwords-weekly" --interval 7d --grace-period 24h --auto-generate
scaivault rotation assign rot_db-passwords-weekly environments/production/billing/database

For database with auto_generate: true, you also need a webhook to actually ALTER ROLE in Postgres — see the Postgres tutorial.

9. Bulk-import a fleet (optional)#

For dozens of services with similar .env files, scripting helps:

python
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
import os
from pathlib import Path
from dotenv import dotenv_values
from scaivault_sdk import SyncScaiVaultClient

client = SyncScaiVaultClient(base_url=..., token=...)

SERVICES = {
    "billing":   "deploy/billing/.env.production",
    "reporting": "deploy/reporting/.env.production",
    "scheduler": "deploy/scheduler/.env.production",
}

# Map env var name -> (secret path, field name, type)
MAPPING = {
    "STRIPE_SECRET_KEY":          ("stripe",     "secret_key",    "api_key"),
    "SALESFORCE_CLIENT_ID":       ("salesforce", "client_id",     "json"),
    "SALESFORCE_CLIENT_SECRET":   ("salesforce", "client_secret", "json"),
    "DATABASE_URL":               ("database",   "url",           "kv"),
}

for service, envfile in SERVICES.items():
    env = dotenv_values(envfile)
    grouped: dict[str, dict] = {}
    types: dict[str, str] = {}
    for k, v in env.items():
        if k not in MAPPING:
            continue
        sub, field, secret_type = MAPPING[k]
        grouped.setdefault(sub, {})[field] = v
        types[sub] = secret_type
    for sub, data in grouped.items():
        path = f"environments/production/{service}/{sub}"
        client.secrets.write(
            path=path,
            data=data,
            secret_type=types[sub],
            metadata={"tags": [service, sub, "production"], "owner": f"team:{service}"},
        )
        print(f"imported {path}")

Run it once. The output is deterministic and rerunnable — re-running just creates new versions, which is fine.

What you have now#

  • Credentials in one canonical place per environment, identifiable by path.
  • Audit log shows which service reads which credential when.
  • A rollback path (env vars) in place for as long as you want it.
  • A scheme to extend: when you add another service or another environment, the pattern carries over.

Failure modes worth knowing#

ScaiVault is unreachable at service startup. With fallback off, the service hard-fails to start. Good. Investigate, fix, restart. If you can't fix immediately, set fallback to true and rotate the values out of env later.

Secret missing. 404 secret_not_found from client.secrets.read(). Either the path is wrong (typo) or the import never ran. Check audit for the import and scaivault secrets list --prefix ....

Access denied even though scopes look right. Probably a group membership issue — the service account is bound via a group it doesn't belong to. scaivault auth whoami from inside the service shows current group membership; scaivault policies test <path> read --identity <sa> shows which rule allows or blocks.

What's next#

Updated 2026-05-17 13:26:51 View source (.md) rev 1