--- title: Migrate from .env Files path: tutorials/migrate-from-env-files status: published --- # Migrate from .env Files You have a fleet of services reading credentials from `.env` files or environment variables, set by your deployment pipeline. This tutorial walks through moving them into ScaiVault with no downtime and a path back if anything breaks. The pattern works for any "credentials in env" setup: 12-factor apps, Docker Compose, Kubernetes ConfigMap-as-env, GitHub Actions secrets, Heroku config vars. The principle is the same — read from a single canonical place at startup. ## What you need - ScaiVault token with `secrets:read` and `secrets:write`. - One service to start with. Pick a low-stakes one for the first pass. - Optionally: the [Python SDK](../sdks/python) or [CLI](../sdks/cli) for scripting bulk imports. ## 1. Inventory what you have Before importing anything, list what you're trying to migrate. For a typical Python service: ```bash $ cat .env.production DATABASE_URL=postgresql://... STRIPE_SECRET_KEY=sk_live_... SALESFORCE_CLIENT_ID=3MVG9... SALESFORCE_CLIENT_SECRET=... SLACK_WEBHOOK_URL=https://hooks.slack.com/... SENTRY_DSN=https://... ``` Categorize each one: | Variable | Sensitive? | Should migrate? | Notes | |----------|-----------|-----------------|-------| | `DATABASE_URL` | Yes (contains password) | Yes | Split into `DB_HOST`, `DB_USER`, `DB_PASS`, `DB_NAME` later | | `STRIPE_SECRET_KEY` | Yes | Yes | High-value, rotate quarterly | | `SALESFORCE_*` | Yes | Yes | Group as one JSON secret | | `SLACK_WEBHOOK_URL` | Yes-ish (URL contains a token) | Yes | | | `SENTRY_DSN` | No (intentionally public) | No | Keep in env | Anything not sensitive — port numbers, feature flags, public URLs, Sentry DSNs — stays in env. ScaiVault is for things that should not appear in `kubectl describe pod`. ## 2. Decide your path scheme A path scheme is a *contract*: a service knows where its secrets live, the policy model knows what to permit, the dashboard knows what to show. Bad path schemes haunt you for years. A scheme that scales: ``` /// ``` Examples: ``` environments/production/billing/database environments/production/billing/stripe environments/staging/billing/database shared/integrations/salesforce/oauth # used by multiple services infra/postgres/reporting/root # privileged backing creds ``` Recommendations: - `environments///...` for per-service per-env credentials. - `shared/...` for things multiple services use (one Salesforce credential, many consumers). - `infra/...` for backing-service root credentials (DB roots, AWS deploy keys) that should rarely be read by application code. Don't put environment in the credential name (`stripe-prod`) — that gets out of sync with the actual deployment environment. Put it in the path. ## 3. Import one service For the billing service: ```bash # Stripe secret scaivault secrets write environments/production/billing/stripe \ secret_key="$STRIPE_SECRET_KEY" \ --type api_key \ --tag billing,stripe,production # Salesforce — group fields under one secret scaivault secrets write environments/production/billing/salesforce \ --type json \ --json-file - < {"id": "pol_billing_prod"} # Bind curl -X POST https://scaivault.scailabs.ai/v1/policies/pol_billing_prod/bindings \ -H "Authorization: Bearer $TOKEN" \ -H "Content-Type: application/json" \ -d '{"identity_type": "service_account", "identity_id": "sa:billing-prod"}' ``` The billing service's token can now read its own credentials and nothing else. ## 5. Adapt the service code Goal: minimal diff, ability to fall back to env vars if ScaiVault is unreachable at startup. This rollback path is the whole reason to do staged migration — without it, ScaiVault becomes a hard dependency before you've earned trust in it. ```python import os from scaivault_sdk import ScaiVaultClient, ScaiVaultError def load_config(): use_vault = os.environ.get("USE_SCAIVAULT", "false").lower() == "true" if use_vault: try: client = ScaiVaultClient( base_url=os.environ["SCAIVAULT_URL"], token=os.environ["SCAIVAULT_TOKEN"], timeout=5.0, ) stripe = client.secrets.read("environments/production/billing/stripe") sf = client.secrets.read("environments/production/billing/salesforce") db = client.secrets.read("environments/production/billing/database") return { "stripe_secret_key": stripe.data["secret_key"], "salesforce_client_id": sf.data["client_id"], "salesforce_client_secret": sf.data["client_secret"], "database_url": f"postgresql://{db.data['username']}:{db.data['password']}@{db.data['host']}:{db.data['port']}/{db.data['database']}", } except ScaiVaultError as e: if not bool(os.environ.get("SCAIVAULT_FALLBACK_ALLOWED", "")): raise print(f"WARN: ScaiVault failed ({e.code}); falling back to env", flush=True) # fall through to env path return { "stripe_secret_key": os.environ["STRIPE_SECRET_KEY"], "salesforce_client_id": os.environ["SALESFORCE_CLIENT_ID"], "salesforce_client_secret": os.environ["SALESFORCE_CLIENT_SECRET"], "database_url": os.environ["DATABASE_URL"], } ``` ## 6. Roll out gradually Stage 1: deploy the code to staging with `USE_SCAIVAULT=true` and `SCAIVAULT_FALLBACK_ALLOWED=true`. Verify it picks up values from ScaiVault. Stage 2: roll the same code to production with `USE_SCAIVAULT=false`. Wait a day. Nothing should change — you've just deployed code that *can* read from ScaiVault but isn't yet. Stage 3: flip to `USE_SCAIVAULT=true` in production, keep fallback on. If anything breaks (token problem, network issue), the env values bail you out and you see warnings in logs. Stage 4: after a week with no fallback warnings, set `SCAIVAULT_FALLBACK_ALLOWED=false`. The service now hard-fails if ScaiVault is unreachable. This is what you want long-term — silent fallback hides problems. Stage 5: remove the env vars from the deployment pipeline. Now they only exist in ScaiVault. ## 7. Audit-check the migration ```bash # Verify the service is actually reading from ScaiVault scaivault audit query \ --identity-id sa:billing-prod \ --start "1 hour ago" \ --json \ | jq -r '.logs[] | "\(.timestamp) \(.action) \(.secret_path)"' ``` You should see steady reads from the paths you imported. If `sa:billing-prod` shows zero activity but the service is running, it's still using env vars — something didn't roll out. ## 8. Wire rotation policies Now that the secret is canonical, put a rotation policy on the ones that warrant it: ```bash # Stripe — quarterly rotation; you'll trigger it via Stripe admin then PUT back scaivault rotation create --name "api-key-90d" --interval 90d --grace-period 48h scaivault rotation assign rot_api-key-90d environments/production/billing/stripe # Database — weekly, auto-generated scaivault rotation create --name "db-passwords-weekly" --interval 7d --grace-period 24h --auto-generate scaivault rotation assign rot_db-passwords-weekly environments/production/billing/database ``` For database with `auto_generate: true`, you also need a webhook to actually ALTER ROLE in Postgres — see the [Postgres tutorial](./dynamic-postgres-credentials). ## 9. Bulk-import a fleet (optional) For dozens of services with similar .env files, scripting helps: ```python import os from pathlib import Path from dotenv import dotenv_values from scaivault_sdk import SyncScaiVaultClient client = SyncScaiVaultClient(base_url=..., token=...) SERVICES = { "billing": "deploy/billing/.env.production", "reporting": "deploy/reporting/.env.production", "scheduler": "deploy/scheduler/.env.production", } # Map env var name -> (secret path, field name, type) MAPPING = { "STRIPE_SECRET_KEY": ("stripe", "secret_key", "api_key"), "SALESFORCE_CLIENT_ID": ("salesforce", "client_id", "json"), "SALESFORCE_CLIENT_SECRET": ("salesforce", "client_secret", "json"), "DATABASE_URL": ("database", "url", "kv"), } for service, envfile in SERVICES.items(): env = dotenv_values(envfile) grouped: dict[str, dict] = {} types: dict[str, str] = {} for k, v in env.items(): if k not in MAPPING: continue sub, field, secret_type = MAPPING[k] grouped.setdefault(sub, {})[field] = v types[sub] = secret_type for sub, data in grouped.items(): path = f"environments/production/{service}/{sub}" client.secrets.write( path=path, data=data, secret_type=types[sub], metadata={"tags": [service, sub, "production"], "owner": f"team:{service}"}, ) print(f"imported {path}") ``` Run it once. The output is deterministic and rerunnable — re-running just creates new versions, which is fine. ## What you have now - Credentials in one canonical place per environment, identifiable by path. - Audit log shows which service reads which credential when. - A rollback path (env vars) in place for as long as you want it. - A scheme to extend: when you add another service or another environment, the pattern carries over. ## Failure modes worth knowing **ScaiVault is unreachable at service startup.** With fallback off, the service hard-fails to start. Good. Investigate, fix, restart. If you can't fix immediately, set fallback to `true` and rotate the values out of env later. **Secret missing.** `404 secret_not_found` from `client.secrets.read()`. Either the path is wrong (typo) or the import never ran. Check audit for the import and `scaivault secrets list --prefix ...`. **Access denied even though scopes look right.** Probably a group membership issue — the service account is bound via a group it doesn't belong to. `scaivault auth whoami` from inside the service shows current group membership; `scaivault policies test read --identity ` shows which rule allows or blocks. ## What's next - [Rotation tutorial](./rotate-oauth-credentials) — put quarterly rotation on imported credentials. - [Migrate from HashiCorp Vault](../migrations/from-hashicorp-vault) — different starting point, same destination. - [Cookbook](../api-guides/cookbook) — common patterns for the steady state.