Topics, types, and HITL specs

A ScaiQueue queue is a typed channel: every message carries a declared message_type, an optional schema-validated body, structured labels, and a priority. Some messages also carry a HITL spec that tells the admin UI how a human reviewer should see them. This page is about that vocabulary.

Message fields#

Every message has the same core shape, regardless of which queue it lands in.

Field	What it is
`id`	Server-assigned UUID.
`queue_id`, `scope_id`, `tenant_id`	Where it lives.
`source_type`, `source_id`	Who published it (`user`, `agent`, etc., plus that actor's id).
`correlation_id`	A user-supplied id that ties many messages together across queues. Carried through routing.
`causation_id`	The id of the message that caused this one, when routing or fan-out creates new messages.
`parent_id`	Direct parent — set by routing actions that emit child messages.
`message_type`	Free-form short string. Convention is dotted, e.g. `order.new`, `refund.review`.
`priority_tier`	`critical`, `high`, `normal`, `low`, or `background`.
`priority_score`	0–1000 within the tier; higher fires earlier in `priority` ordering.
`labels`	A `{key: value}` map. Used by routing conditions and consumer-side filtering.
`content_type`	`application/json` by default; opaque bytes otherwise.
`body`	The actual payload, encoded per `content_type`.
`ttl_seconds`, `deadline`	Lifetime bounds; `expiry_enforcer` cleans these up.
`idempotency_key`	Optional. Repeated publishes with the same key return the original message id.
`hitl_spec`	Optional. See below.

correlation_id is the load-bearing field for cross-queue tracing. Set it once at the start of a pipeline and let it ride; GET /scopes/{scope_id}/messages?correlation_id=... returns the whole chain, and GET /scopes/{scope_id}/audit/trace/{correlation_id} returns every audit event for it in order.

Message schemas#

A scope can register named JSON Schemas via /scopes/{scope_id}/schemas. Each is identified by name + version. You can then validate any payload against a schema before publishing:

bash
curl -X POST "$SCAIGRID_HOST/v1/modules/scaiqueue/scopes/$SCOPE_ID/schemas/order.new/1.0/validate" \
  -H "Authorization: Bearer $SCAIGRID_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"payload": {"order_id": "ord_1", "total": "127.50"}}'

The response is {"valid": bool, "errors": [{"path": "...", "message": "..."}]}. Schemas are draft-2020-12 JSON Schema. Deprecating a schema sets a flag — older messages remain valid; new publishes typically use newer versions.

HITL specs#

A HITL spec is an optional object on a message that declares what a human reviewer should see and decide. The admin UI renders it as a structured form; the spec format is implementation-defined per tenant, but reusable patterns can be stored in the scope's HITL-pattern registry and expanded by parameters at publish time.

bash
curl -X POST "$SCAIGRID_HOST/v1/modules/scaiqueue/scopes/$SCOPE_ID/hitl-patterns" \
  -H "Authorization: Bearer $SCAIGRID_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "approve-or-reject",
    "version": "1.0",
    "spec": {
      "sections": [
        {"type": "context", "title": "{{title}}", "content": "{{summary}}"},
        {"type": "decision", "field": "decision", "options": ["approve","reject"]}
      ]
    }
  }'

Then expand it with parameters when you want to preview the rendered spec:

bash
curl -X POST "$SCAIGRID_HOST/v1/modules/scaiqueue/scopes/$SCOPE_ID/hitl-patterns/approve-or-reject/expand" \
  -H "Authorization: Bearer $SCAIGRID_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"version": "1.0", "parameters": {"title": "Refund request", "summary": "Customer asks for $500"}}'

Expansion is a {{key}} template substitution — the result is what a publisher would attach to a message as its hitl_spec. The registry is for governance: legal and product can curate the spec catalogue; engineers reference patterns by name.

Labels and routing#

Labels are the join column between messages and routing rules. A rule can say "if labels.region == 'eu' and message_type starts with 'order.', route to queue X." Labels travel through routing actions unchanged unless a transform rewrites them. Use labels for facts that affect routing (region, customer tier, severity) and the body for the payload itself.

Priority and aging#

Priority is two-dimensional: a tier (one of five categorical buckets) and a score (0–1000 inside the tier). In priority ordering, higher tier-plus-score wins. The priority_aging_worker agent runs every five minutes and increments priority_score on messages that have been pending for too long — so low-priority work doesn't starve behind a steady stream of high-priority arrivals.

Streams (chunked messages)#

When a producer is generating output progressively — for example, a long LLM response — it can open a stream instead of one big message. Each chunk is a message with stream_id, stream_sequence, and stream_final. The stream service collects chunks and exposes the assembled result via GET /scopes/{scope_id}/streams/{stream_id}/assembled. Default assembly_mode is concatenate; streams time out after timeout_seconds (default 300).

Tenant boundaries#

Every message, scope, queue, schema, pattern, ACL, and audit row is scoped to a tenant. Cross-tenant message flow exists only via explicit cross-scope trust grants (/scopes/{scope_id}/acl/trusted-scopes), which name an allowed direction (unidirectional or bidirectional), an identity mode (passthrough, proxy), and optional allowed-types / allowed-queues lists.