Federation Deep Dive

Federation lets ScaiVault front an existing secret store: HashiCorp Vault, AWS Secrets Manager, Azure Key Vault, or GCP Secret Manager. You get the ScaiVault API, audit trail, and policy model while the secret data stays where it is. This is the preferred path for migrating an existing deployment — you don't need a hard cutover.

Two modes#

Proxy. ScaiVault forwards each read to the backend in real time. Writes to proxied paths are rejected — the backend is authoritative. No secret data is stored in ScaiVault's own database.

Sync. ScaiVault periodically pulls the backend into its own storage. Reads hit local cache. Writes can be configured to write-through or be disabled.

Proxy is the conservative default during migration. Sync is a better fit once you've decided ScaiVault is the primary API. Migrating an individual path from proxy to sync is a config change, not a data operation.

Proxy mode read flow#

Sync mode read flow#

Configure a backend#

The standard pattern: store the backend's credentials in ScaiVault first (yes, bootstrapping is a little uncomfortable — a narrow static credential for a single purpose), then reference that path in the federation config.

HashiCorp Vault#

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

# Store AppRole role_id and secret_id
curl -X PUT https://scaivault.scailabs.ai/v1/secrets/infra/hashicorp/approle \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "data": {"role_id": "...", "secret_id": "..."},
    "secret_type": "json"
  }'

# Configure the backend
curl -X POST https://scaivault.scailabs.ai/v1/federation/backends \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "hashicorp-production",
    "type": "hashicorp-vault",
    "config": {
      "endpoint": "https://vault.internal:8200",
      "auth_method": "approle",
      "auth_config_path": "infra/hashicorp/approle",
      "tls_verify": true,
      "namespace": "acme"
    },
    "path_mapping": {
      "external/hashicorp/**": "secret/data/**"
    },
    "mode": "proxy"
  }'

Now a read on external/hashicorp/app/db/password in ScaiVault proxies to secret/data/app/db/password in Vault, transforms the response, records an audit entry, returns it.

Supported auth methods: approle, token, kubernetes, aws, gcp.

AWS Secrets Manager#

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

curl -X POST https://scaivault.scailabs.ai/v1/federation/backends \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "aws-prod-sm",
    "type": "aws-secrets-manager",
    "config": {
      "region": "us-east-1",
      "credentials_path": "infra/aws/sm/reader",
      "role_arn": "arn:aws:iam::123456789012:role/scaivault-sm-reader"
    },
    "path_mapping": {
      "external/aws/**": "prod/**"
    },
    "mode": "sync",
    "sync_interval": "15m"
  }'

Provide either credentials_path (static IAM user keys) or role_arn (assume role). IAM role with only secretsmanager:GetSecretValue and secretsmanager:ListSecrets on the relevant ARN pattern is sufficient.

AWS Secrets Manager charges per secret per month and per API call. sync with a local cache is usually cheaper than proxy.

Azure Key Vault#

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

{
  "name": "azure-prod-kv",
  "type": "azure-key-vault",
  "config": {
    "vault_url": "https://acme-prod.vault.azure.net",
    "auth_method": "managed_identity"
  },
  "path_mapping": {
    "external/azure/**": "**"
  },
  "mode": "proxy"
}

Auth methods: managed_identity (for pods running in Azure), client_secret (static client ID + secret), certificate.

GCP Secret Manager#

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

{
  "name": "gcp-prod-sm",
  "type": "google-secret-manager",
  "config": {
    "project_id": "acme-prod",
    "credentials_path": "infra/gcp/sm/reader-sa"
  },
  "path_mapping": {
    "external/gcp/**": "**"
  },
  "mode": "sync",
  "sync_interval": "30m"
}

credentials_path points at a stored GCP service account JSON key. For workload identity deployments, use "auth_method": "workload_identity" and omit credentials.

Path mapping#

path_mapping is an ordered array of templates. The first match wins.

1
2
3
4

"path_mapping": {
  "external/hashicorp/apps/**": "secret/data/apps/**",
  "external/hashicorp/shared/*": "secret/data/shared/*"
}

Globs capture and interpolate: the ** match in the ScaiVault path is substituted into ** in the backend path. This works for both reads and (for sync-mode writes) writes.

Migration strategy#

Typical path from "everything in Vault" to "everything in ScaiVault":

1. Stand up ScaiVault alongside the existing Vault. Both operational.
2. Federate. Proxy mode. Applications keep reading from Vault directly; dashboards and new integrations read through ScaiVault's /external/hashicorp/*. Audit trail is unified.
3. Switch one subtree to read through ScaiVault. Applications now talk to ScaiVault but data is still in Vault.
4. Flip to sync mode for that subtree. Data is now in ScaiVault's store, but the upstream Vault still has a copy.
5. Migrate writes. Update the app to write to ScaiVault. The sync is now bidirectional (if configured) or the upstream becomes stale (if not).
6. Decommission the upstream subtree when you're confident nothing reads it.

Don't do all six steps for everything at once. Pick the highest-value subtree, complete the sequence, repeat.

Conflicts#

In sync mode with write-through disabled, conflicts can arise: someone writes to Vault directly while ScaiVault has a cached value. On next sync, ScaiVault detects the divergence:

1
2

curl -H "Authorization: Bearer $TOKEN" \
     https://scaivault.scailabs.ai/v1/federation/backends/fed_abc/conflicts

Response lists paths where local and remote disagree. Resolution policy is configured per backend:

• backend_wins — remote wins, local overwritten.
• local_wins — local wins, remote overwritten (if write-through is enabled).
• manual — neither wins; the path is flagged and excluded from sync until resolved.

Observability#

federation.sync.completed and .failed events fire on every sync cycle. federation.backend.unreachable fires on persistent connection failures. Wire these to your alerting.

The /v1/federation/backends/{id}/status endpoint is suitable for polling dashboards — include last sync time, secrets synced, connection latency.

Limitations#

• Path format compatibility. Vault and AWS-SM allow characters ScaiVault doesn't. During sync, incompatible paths are logged and skipped. The sync report tells you which.
• Metadata translation. Not every field maps cleanly. Custom metadata is preserved; provider-specific fields (AWS SM's KmsKeyId, Vault's cas) are not.
• Version history. Proxied reads return only the current version. Sync preserves version history only if the backend exposes it (Vault KV v2: yes. AWS SM: versions exist but labeling is different; ScaiVault maps AWSCURRENT to current).

What's next#

• Federation Reference — endpoints.
• Multi-tenancy — federation operates per tenant.