---
title: Federation Deep Dive
path: advanced/federation
status: published
---

# Federation Deep Dive

Federation lets ScaiVault front an existing secret store: HashiCorp Vault, AWS Secrets Manager, Azure Key Vault, or GCP Secret Manager. You get the ScaiVault API, audit trail, and policy model while the secret data stays where it is. This is the preferred path for migrating an existing deployment — you don't need a hard cutover.

## Two modes

**Proxy.** ScaiVault forwards each read to the backend in real time. Writes to proxied paths are rejected — the backend is authoritative. No secret data is stored in ScaiVault's own database.

**Sync.** ScaiVault periodically pulls the backend into its own storage. Reads hit local cache. Writes can be configured to write-through or be disabled.

| Aspect | Proxy | Sync |
|--------|-------|------|
| Freshness | Real-time | Bounded by `sync_interval` |
| Latency | Backend latency + ScaiVault overhead | Local only |
| Availability | Tied to backend | Independent of backend |
| Data at rest | Nowhere in ScaiVault | Encrypted in ScaiVault storage |
| Write support | Read-only | Optional write-through |
| Cost per read | One backend call | None after first sync |

Proxy is the conservative default during migration. Sync is a better fit once you've decided ScaiVault is the primary API. Migrating an individual path from proxy to sync is a config change, not a data operation.

### Proxy mode read flow

```mermaid
sequenceDiagram
    participant App
    participant SV as ScaiVault
    participant Vault as Backend (Vault / AWS SM / ...)

    App->>SV: GET /v1/secrets/external/hashicorp/path
    SV->>Vault: GET /v1/kv/data/path
    Vault-->>SV: value
    SV-->>App: value (real-time)
    Note over SV: audit row written;<br/>nothing persisted locally
```

### Sync mode read flow

```mermaid
sequenceDiagram
    participant Job as Sync job
    participant SV as ScaiVault
    participant Vault as Backend
    participant App

    Note over Job: every sync_interval
    Job->>Vault: list + read paths
    Vault-->>Job: values
    Job->>SV: write to local storage<br/>(encrypted)

    Note over App: independent timeline
    App->>SV: GET /v1/secrets/external/hashicorp/path
    SV-->>App: value from local cache
    Note over SV: staleness ≤ sync_interval
```

## Configure a backend

The standard pattern: store the backend's credentials in ScaiVault first (yes, bootstrapping is a little uncomfortable — a narrow static credential for a single purpose), then reference that path in the federation config.

### HashiCorp Vault

```bash
# Store AppRole role_id and secret_id
curl -X PUT https://scaivault.scailabs.ai/v1/secrets/infra/hashicorp/approle \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "data": {"role_id": "...", "secret_id": "..."},
    "secret_type": "json"
  }'

# Configure the backend
curl -X POST https://scaivault.scailabs.ai/v1/federation/backends \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "hashicorp-production",
    "type": "hashicorp-vault",
    "config": {
      "endpoint": "https://vault.internal:8200",
      "auth_method": "approle",
      "auth_config_path": "infra/hashicorp/approle",
      "tls_verify": true,
      "namespace": "acme"
    },
    "path_mapping": {
      "external/hashicorp/**": "secret/data/**"
    },
    "mode": "proxy"
  }'
```

Now a read on `external/hashicorp/app/db/password` in ScaiVault proxies to `secret/data/app/db/password` in Vault, transforms the response, records an audit entry, returns it.

Supported auth methods: `approle`, `token`, `kubernetes`, `aws`, `gcp`.

### AWS Secrets Manager

```bash
curl -X POST https://scaivault.scailabs.ai/v1/federation/backends \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "aws-prod-sm",
    "type": "aws-secrets-manager",
    "config": {
      "region": "us-east-1",
      "credentials_path": "infra/aws/sm/reader",
      "role_arn": "arn:aws:iam::123456789012:role/scaivault-sm-reader"
    },
    "path_mapping": {
      "external/aws/**": "prod/**"
    },
    "mode": "sync",
    "sync_interval": "15m"
  }'
```

Provide either `credentials_path` (static IAM user keys) or `role_arn` (assume role). IAM role with only `secretsmanager:GetSecretValue` and `secretsmanager:ListSecrets` on the relevant ARN pattern is sufficient.

AWS Secrets Manager charges per secret per month and per API call. `sync` with a local cache is usually cheaper than `proxy`.

### Azure Key Vault

```json
{
  "name": "azure-prod-kv",
  "type": "azure-key-vault",
  "config": {
    "vault_url": "https://acme-prod.vault.azure.net",
    "auth_method": "managed_identity"
  },
  "path_mapping": {
    "external/azure/**": "**"
  },
  "mode": "proxy"
}
```

Auth methods: `managed_identity` (for pods running in Azure), `client_secret` (static client ID + secret), `certificate`.

### GCP Secret Manager

```json
{
  "name": "gcp-prod-sm",
  "type": "google-secret-manager",
  "config": {
    "project_id": "acme-prod",
    "credentials_path": "infra/gcp/sm/reader-sa"
  },
  "path_mapping": {
    "external/gcp/**": "**"
  },
  "mode": "sync",
  "sync_interval": "30m"
}
```

`credentials_path` points at a stored GCP service account JSON key. For workload identity deployments, use `"auth_method": "workload_identity"` and omit credentials.

## Path mapping

`path_mapping` is an ordered array of templates. The first match wins.

```json
"path_mapping": {
  "external/hashicorp/apps/**": "secret/data/apps/**",
  "external/hashicorp/shared/*": "secret/data/shared/*"
}
```

Globs capture and interpolate: the `**` match in the ScaiVault path is substituted into `**` in the backend path. This works for both reads and (for sync-mode writes) writes.

## Migration strategy

Typical path from "everything in Vault" to "everything in ScaiVault":

1. **Stand up ScaiVault** alongside the existing Vault. Both operational.
2. **Federate.** Proxy mode. Applications keep reading from Vault directly; dashboards and new integrations read through ScaiVault's `/external/hashicorp/*`. Audit trail is unified.
3. **Switch one subtree** to read through ScaiVault. Applications now talk to ScaiVault but data is still in Vault.
4. **Flip to sync mode** for that subtree. Data is now in ScaiVault's store, but the upstream Vault still has a copy.
5. **Migrate writes.** Update the app to write to ScaiVault. The sync is now bidirectional (if configured) or the upstream becomes stale (if not).
6. **Decommission** the upstream subtree when you're confident nothing reads it.

Don't do all six steps for everything at once. Pick the highest-value subtree, complete the sequence, repeat.

## Conflicts

In `sync` mode with write-through disabled, conflicts can arise: someone writes to Vault directly while ScaiVault has a cached value. On next sync, ScaiVault detects the divergence:

```bash
curl -H "Authorization: Bearer $TOKEN" \
     https://scaivault.scailabs.ai/v1/federation/backends/fed_abc/conflicts
```

Response lists paths where local and remote disagree. Resolution policy is configured per backend:

- `backend_wins` — remote wins, local overwritten.
- `local_wins` — local wins, remote overwritten (if write-through is enabled).
- `manual` — neither wins; the path is flagged and excluded from sync until resolved.

## Observability

`federation.sync.completed` and `.failed` events fire on every sync cycle. `federation.backend.unreachable` fires on persistent connection failures. Wire these to your alerting.

The `/v1/federation/backends/{id}/status` endpoint is suitable for polling dashboards — include last sync time, secrets synced, connection latency.

## Limitations

- **Path format compatibility.** Vault and AWS-SM allow characters ScaiVault doesn't. During sync, incompatible paths are logged and skipped. The sync report tells you which.
- **Metadata translation.** Not every field maps cleanly. Custom metadata is preserved; provider-specific fields (AWS SM's `KmsKeyId`, Vault's `cas`) are not.
- **Version history.** Proxied reads return only the current version. Sync preserves version history only if the backend exposes it (Vault KV v2: yes. AWS SM: versions exist but labeling is different; ScaiVault maps `AWSCURRENT` to current).

## What's next

- [Federation Reference](../reference/federation) — endpoints.
- [Multi-tenancy](../core-concepts/multi-tenancy) — federation operates per tenant.