Architecture

ScaiVault is a stateless API service in front of a PostgreSQL database, Redis cache, and an object store for large artifacts. It depends on ScaiKey for identity, optionally on a KMS/HSM for root key protection, and optionally on external secret stores for federation.

The big picture#

Components#

API service#

A stateless HTTP/JSON server. Horizontal scale by running more replicas behind a load balancer. All state lives in PostgreSQL, Redis, or the object store — API instances can be killed at any time without data loss.

Every request flows through the same pipeline: authenticate the bearer token against ScaiKey, resolve the caller's identity and tenant, evaluate policies, execute the operation, write an audit entry, respond.

PostgreSQL#

Canonical store for metadata and small secret payloads:

• Secret records and version history
• Policies, bindings, and rotation schedules
• PKI certificate metadata, CAs, trust anchors, CSRs
• Dynamic engines, roles, and leases
• Audit log
• Identity cache (mirrored from ScaiKey)

Encrypted columns use envelope encryption: each secret is encrypted with a per-row data key, which is itself encrypted with the KMS root key. Rotating the root key re-wraps the data keys without rewriting secret data.

Redis#

• Rate limiter state (token bucket per identity and endpoint category)
• Short-lived tokens and session handles
• Cached policy evaluations (with TTL)
• Lease countdowns for dynamic secrets

Redis is a cache — losing it costs you a warmup, not data.

Object storage#

Large payloads (certificate chains with many intermediates, CRLs, large JSON configs) are offloaded to S3-compatible storage. The DB row holds the reference and the decryption envelope; the object store holds the ciphertext.

ScaiKey#

ScaiKey is the identity provider for the whole ScaiLabs stack. ScaiVault does not implement its own login, user management, or JWT issuance. It accepts JWTs minted by ScaiKey, validates them against ScaiKey's public keys, and resolves the caller's tenant, groups, and roles.

ScaiVault keeps a cache of ScaiKey identity data so policy evaluation is a local lookup, not a cross-service call. The cache is kept fresh by webhooks — ScaiKey pushes identity events (user.updated, group.member.added, etc.) to ScaiVault's /v1/identity/webhooks/scaikey endpoint.

KMS / HSM#

ScaiVault stores a single root key in a KMS (AWS KMS, GCP KMS, Azure Key Vault, HashiCorp Vault, or an on-prem HSM). Every secret is encrypted with a per-row data key, and data keys are wrapped by the root key. The plaintext root never leaves the KMS.

Rotation of the root key re-wraps data keys in place — existing ciphertexts don't need to be rewritten.

Request lifecycle#

A read request GET /v1/secrets/environments/production/salesforce/api-key:

1. TLS termination at the load balancer. mTLS optional for service identities.
2. Token validation. The Authorization: Bearer ... header is parsed. The token's signature is checked against the cached ScaiKey JWKS. Expired or invalid tokens return 401.
3. Identity resolution. The token's sub, tenant_id, and groups claims are used to look up the caller in the local identity cache.
4. Path and tenant scoping. The requested path is qualified with the caller's tenant. Cross-tenant access requires the /t/{tenant_id}/ prefix and partner-admin role.
5. Policy evaluation. Every policy bound to the caller (directly, via group membership, or via service-account impersonation) is checked. The first matching rule wins. If no rule matches, the request is denied with access_denied.
6. Rate limit check. Redis token bucket for the secrets:read category. A budget miss returns 429 with Retry-After.
7. Data retrieval. The current version row is fetched from PostgreSQL. The encrypted payload is unwrapped: data key → KMS decrypt → payload plaintext.
8. Audit log write. A row is appended with request ID, identity, path, outcome, and duration. Writes are asynchronous but guaranteed-at-least-once.
9. Response. JSON envelope with the plaintext secret and metadata. X-RateLimit-* and X-Request-ID headers included.

Deployment shapes#

Single tenant, self-hosted. One scaivault container plus Postgres + Redis + a KMS of your choice. Suitable for small organizations.

Multi-tenant, managed. Several regional clusters, each running the API horizontally. Postgres in HA (primary + replicas). Redis in cluster mode. Root keys in a regional KMS with cross-region replication. This is how scaivault.scailabs.ai runs.

Federated. ScaiVault fronts an existing HashiCorp Vault (or AWS SM, Azure KV, GCP SM). Reads go through ScaiVault's API; ScaiVault either proxies to the upstream or maintains a local synced cache. Writes can be read-only against the upstream if you want ScaiVault purely as a uniform API layer.

See Deployment for concrete setup.

Trust boundaries#

• Client ↔ API: TLS. Bearer JWT from ScaiKey. Optionally mTLS with a service-account cert.
• API ↔ Postgres: TLS. Dedicated Postgres user per environment. No read replicas exposed externally.
• API ↔ KMS: cloud provider SDK (AWS IAM / GCP IAM / Azure MI) or PKCS#11 for on-prem HSMs. Root key material never crosses this boundary in plaintext.
• API ↔ ScaiKey: outbound-only HTTPS. Inbound webhooks verified by HMAC signature (X-ScaiKey-Signature).
• API ↔ federated backends: credentials stored in ScaiVault itself (bootstrap-chicken-and-egg handled by a boot-time root token).

What's next#

• Quickstart — hands on.
• Multi-tenancy — the tenant / partner model.
• Deployment — putting it in your infrastructure.