---
summary: "WebSocket protocol for the ScaiVatars surface \u2014 every message type,\
  \ parameter, and response shape."
title: Avatar API reference
path: reference/avatar-api
status: published
---

# Avatar API reference

The avatar surface is a single WebSocket endpoint. All I/O is JSON; no binary frames; all audio is base64-encoded inside JSON payloads.

```
wss://scaigrid.scailabs.ai/v1/public/scaibot/avatar/ws-interactive?token=<embed token>
```

## Connection

- **Transport**: WSS (TLS 1.2+ required; plain `ws://` is rejected).
- **Auth**: per-bot embed token in the `?token=` query. Issued from the Embed & Deploy admin page.
- **Origin gate**: the browser's `Origin` header must match an entry in the bot's `avatar_allowed_origins` list. CLI clients without an `Origin` header pass through.
- **Close codes**:
   - `4401 INVALID_ACCESS_TOKEN` — token missing, revoked, or invalid.
   - `4403 INVALID_ORIGIN` — origin not on the bot's allow-list.
   - `4401 AVATAR_NOT_FOUND` — token resolved, but the bot doesn't have `avatar_enabled=true`.

## Message envelope

Every WebSocket frame is JSON. Outer shape:

**Client → Server (requests)**:

```json
{
  "type": "request",
  "request_id": "<UUIDv4>",
  "payload": {
    "action": "...",
    "params": { ... },
    "response_config": { ... }   // optional, see below
  }
}
```

**Server → Client (responses, system, events, errors)**:

```json
{
  "type": "response|system|error|event",
  "request_id": "<UUIDv4>",     // matches the request; omitted on protocol-level errors
  "timestamp": "2026-06-23T12:34:56.789Z",
  "payload": { ... }
}
```

Multiple requests can be in flight simultaneously — match server frames to your request via `request_id`. Server frames may stream out of request order.

## Actions

### start_session

Create a new session. Required before any chat action.

**Request params:**

| Field | Required | Notes |
|---|---|---|
| `avatar_id` | yes | The bot's slug (visible in the admin UI's bot picker as the part after the dash). |
| `user_id` | no | Optional end-user identifier for tying the session to a known user. |
| `access_token` | no | Required when `user_id` is set. Validates `user_id`. |
| `metadata` | no | Arbitrary JSON the server stores against the session. |

**Response data:**

```json
{
  "session_id": "<UUIDv4>",
  "avatar_id": "<bot slug>",
  "avatar_name": "<bot display name>",
  "user_id": "<user id or null>",
  "created_at": "2026-06-23T12:34:56.789Z"
}
```

### resume_session

Re-attach to an existing session — used when the WebSocket reconnects.

**Request params:**

| Field | Required | Notes |
|---|---|---|
| `session_id` | yes | The `session_id` returned by an earlier `start_session`. |
| `user_id` | no | Must match the session's original `user_id` when set. |
| `access_token` | no | Required when `user_id` is set. |

### text_to_speech

Make the avatar say a specific piece of text. No LLM involvement — the text passes straight to TTS + lipsync.

**Request params:**

| Field | Required | Notes |
|---|---|---|
| `session_id` | yes | From `start_session`. |
| `text` | yes | What to say. |
| `voice_id` | no | Override the bot's default voice for this turn. |

**Response chunks** carry `data.text`, `data.voice` (base64 PCM block), and `data.blendshapes` per `response_config` (below). `chunk_info.is_final=true` marks the last chunk.

### send_text

Pass user text through the bot's LLM with the bot's persona + the appended avatar-mode rules, synthesize the reply as audio + lipsync, and emit it as a response chunk.

**Request params:**

| Field | Required | Notes |
|---|---|---|
| `session_id` | yes | From `start_session`. |
| `text` | yes | The user's message. |

The server persists both the user's message and the assistant's reply against the session so subsequent calls to `get_history` include them. The reply's text + voice + blendshapes ship in a single response chunk with `chunk_info.is_final=true`. The same `response_config` block as `text_to_speech` controls what gets included.

### send_voice

Send a base64-encoded microphone clip. The server transcribes it via ScaiInfer's STT, then delegates to `send_text` for the LLM + TTS + lipsync round-trip.

**Request params:**

| Field | Required | Notes |
|---|---|---|
| `session_id` | yes |  |
| `audio_format` | yes | `pcm` or `wav`. |
| `audio_data` | yes | Base64-encoded PCM or WAV bytes. |
| `sample_rate` | when `pcm` | Hz. Required for raw PCM; WAV reads it from the RIFF header. |
| `channels` | no | Defaults to 1. |
| `bit_depth` | no | Defaults to 16. |

When the client sets `response_config.include_transcription=true`, the transcript appears on the response chunk as `data.transcription`.

## avatar_update event — how clients learn the 3D config

Right after `start_session` (and `resume_session`) succeeds, the server emits an `avatar_update` event carrying the bot's current visual configuration. The client uses this to pick the 3D model URL, the lipsync engine variant, and the idle animation — without an extra HTTP round-trip.

```json
{
  "type": "event",
  "timestamp": "2026-06-23T12:34:56.789Z",
  "payload": {
    "event_type": "avatar_update",
    "data": {
      "avatar_id": "support-bot",
      "avatar_name": "Poolnoodle",
      "model_url": "https://cdn.customer.example.com/poolnoodle.glb",
      "blendshape_profile": "audio.lipsync.scailipsync",
      "idle_animation": "idle_loop_01"
    }
  }
}
```

| Field | Type | Notes |
|---|---|---|
| `avatar_id` | string | The bot's slug, matching what the client passed in `start_session`. |
| `avatar_name` | string | The bot's display name. |
| `model_url` | string or null | GLB / GLTF / VRM URL the client loads. Null when the operator hasn't set one — the client falls back to whatever default it bundles. |
| `blendshape_profile` | string | Lipsync engine variant. Currently always `"audio.lipsync.scailipsync"`; future expressive variants ship under their own engine kinds. |
| `idle_animation` | string or null | Animation name the client plays between speaking turns. Null = client picks its own default. |

The event is emitted on every `start_session` and `resume_session` success — even when none of the bot's avatar fields are set. Clients use **field presence** (model_url etc.) to decide what to load, not whether to handle the event.

Compatibility: `avatar_update` is in the standalone protocol's event_type enum, and `EventPayload.data` is `dict[str, Any]`, so this carries zero compatibility risk. Clients that don't handle `avatar_update` drop the event silently per protocol.

### get_history

Page through the session's message history.

**Request params:**

| Field | Required | Notes |
|---|---|---|
| `session_id` | yes |  |
| `limit` | no | Default 50, max 1000. |
| `offset` | no | Default 0. |
| `order` | no | `asc` (default) or `desc`. |

### stop_command

Cancel an in-flight request mid-stream.

**Request params:**

| Field | Required | Notes |
|---|---|---|
| `target_request_id` | yes | The `request_id` of the in-flight action to interrupt. |

The cancelled request emits a single `event` frame with `event_type: "stopped"` before its asyncio task settles.

## response_config

Optional block on chat actions that controls what the server emits. Server-side defaults match the standalone ScaiVatars service.

| Field | Default | Notes |
|---|---|---|
| `include_text` | `true` | Echo the assistant text in `data.text`. |
| `include_voice` | `false` | Synthesize audio; populate `data.voice`. |
| `include_blendshapes` | `false` | Generate lipsync; populate `data.blendshapes`. |
| `include_transcription` | `false` | For `send_voice`: include the STT transcript. |
| `voice_format` | `"pcm"` | Only PCM today. |
| `voice_sample_rate` | `24000` | Hz. |
| `voice_bit_depth` | `16` | 8/16/32. 8 is unsigned per the WAV convention. |
| `voice_channels` | `1` | 1 or 2. |
| `blendshapes_frame_rate` | `60` | 1–60 fps. |

## Blendshape format

Each blendshape frame is a list:

```
["HH:MM:SS.mmm", c0, c1, c2, ..., c67]
```

- Index 0 is a timecode string (`HH:MM:SS.mmm`, millisecond precision).
- Indices 1–68 are ARKit-compatible blendshape coefficients (68 floats in `[-1.0, 1.0]`).
- At the default 60 fps, frames are 16.67 ms apart.

Drive your 3D model's morph target weights from indices 1–68, synchronized to the audio's playback time.

## Avatar mode rules

When the LLM lands (v1.1), every request will carry an appended system-prompt block reminding the model it's a 3D avatar. The block tells the LLM:

- The user SEES its face and HEARS its words; expressions and tone are rendered live.
- Never use Markdown — asterisks get read aloud by the TTS.
- Never reference visual artifacts ("as shown below", "see the diagram") that don't translate to speech.
- It cannot see the user; don't claim to read their expression.
- It can convey emotion through tone; write words you'd want spoken with the right feeling.
- The user can interrupt at any time; get to the answer in the first sentence.

The full block is visible read-only on the Avatar Settings admin page so operators can see exactly what gets appended to their persona.

## Error codes

| Code | Meaning |
|---|---|
| `INVALID_REQUEST` | Malformed frame, unknown action, or invalid params. |
| `SESSION_NOT_FOUND` | `session_id` doesn't resolve, was closed, or belongs to another bot. |
| `AVATAR_NOT_FOUND` | `avatar_id` doesn't match the embed-token-bound bot's slug. |
| `INVALID_ACCESS_TOKEN` | Embed token missing, revoked, or invalid. |
| `USER_ID_MISMATCH` | Resume attempted with a different `user_id` than the session's original. |
| `INVALID_ORIGIN` | Origin not in the bot's allow-list. |
| `AUDIO_FORMAT_UNSUPPORTED` | `send_voice` audio in a format we can't decode. |
| `TTS_GENERATION_FAILED` | Synthesis failed; check the voice_id and bot voice config. |
| `NOT_IMPLEMENTED` | Action not yet wired. No actions return this in v1.1 — reserved for future protocol additions. |
| `INTERNAL_SERVER_ERROR` | Catch-all for unexpected server failures. |

## Admin endpoints

For programmatic management (the UI uses these too):

```
GET  /v1/modules/scaibot/bots/{bot_id}/scaivatars
PUT  /v1/modules/scaibot/bots/{bot_id}/scaivatars
GET  /v1/modules/scaibot/scaivatars/sessions
GET  /v1/modules/scaibot/scaivatars/sessions/{id}
DELETE /v1/modules/scaibot/scaivatars/sessions/{id}
```

The `PUT` accepts any subset of: `avatar_enabled`, `avatar_model_url`, `avatar_blendshape_profile`, `avatar_idle_animation`, `avatar_default_response_config`, `avatar_allowed_origins`, `avatar_system_prompt`.

Embed tokens for the avatar surface come from the **existing** ScaiBot embed-token endpoints — one token type, three surfaces (chat widget, voice widget, avatar WebSocket):

```
GET    /v1/modules/scaibot/bots/{bot_id}/embed-tokens
POST   /v1/modules/scaibot/bots/{bot_id}/embed-token
DELETE /v1/modules/scaibot/bots/{bot_id}/embed-token
```