Resilience & Failure Handling

← Back to System Internals Reference

How the system behaves under network failures, disconnects, and device problems. There are three reconnection tiers, only loosely coordinated:

The headline risk: tiers 2 and 3 are not wired together for custom non-SDK clients. The SDK wires them via _recoverMedia → _coldReconnect. When the WebRTC layer exhausts retries and emits 'failed', a custom client that does not use the SDK's KalturaAvatarSession must handle this itself.

Device permissions (mic/camera)

connecting → gettingUserMedia calls getUserMedia(audio:true, video:false) on the built-in client's media layer. This is audio only by default; the avatar doesn't need your camera. On denial, the runtime client's device-media handler routes to error with reason: DevicesPermissionDenied, skipDisconnect:true, suppressNotification:true (and shouldPurge=false). This is a clean, retryable abort with no scary toast. The SDK (KalturaAvatarSession) surfaces distinct NotAllowed/NotFound/NotReadable codes; the platform's built-in client's classification is coarse (no NotAllowedError vs NotFoundError vs NotReadableError distinction). No pre-flight navigator.permissions.query, no mid-call device-loss handling.

WebRTC media peer (the built-in client's media layer)

• Config: maxReconnectAttempts = 5, reconnectDelayMs = 2000 (fixed). ICE timers (rtc-core constants): connect-start 20s, no-SDP-answer 30s, disconnect-grace 60s.
• ASR reconnection (handleAsrReconnection): closes the peer, fully re-joins (new offer/answer via socket relay), preserves mute state. After 5 tries → emits onConnectionState({type:'asr', status:'failed'}).
• STV reconnection (handleStvReconnection): re-runs WHEP joinSTV. If WHEP returns 404 NO_ACTIVE_SESSION, gives up immediately (server session gone; only the app can recreate it). After 5 tries → 'failed'.

Control socket & session machine (the platform's built-in client)

• Socket.IO built-in recovery: on disconnect with socket.active, auto-reconnects (may restore the same socket via connectionStateRecovery).
• But the runtime client's error handler converts every disconnect/connect_error/error/removePeer/throwTo* into a machine Disconnect → teardown. The session machine has no auto-reconnect; recovery is user-initiated.
• Connect-phase hang protection is strong: every sub-state has a timeout (5–30s). Teardown is reliable (routes through disconnected even on failure; resets the player-ready singleton). Disconnect reasons (the runtime client's notification layer) drive user-readable, severity-tagged messages.

Failure-mode matrix

Tool-call spiral: what happened and how it's mitigated

A tool-eager brain can retry the same client command dozens or hundreds of times in one turn instead of narrating. KalturaAvatarSession defends against this with a two-tier circuit breaker.

Soft tier: signal only. Once a turn accumulates toolSpiralLimit (default 10) raw type:"tool" segments, counted before dedup since a spiral is the same call repeating, the SDK emits toolSpiralDetected once. The soft tier only signals. It never calls interrupt(): interrupting a spiral already running server-side has no effect on it, and can truncate the turn's own narration.

A legitimate turn can double its raw tool-segment count when speak()'s barge-in branch (still-playing TTS audio from a prior turn) spawns a parallel tap-to-talk stream for the same question. For example, a 3-tool turn duplicates into 6 raw segments this way. The default limit of 10 is high enough to absorb that duplication without tripping the breaker on an ordinary turn, not a real spiral.

Hard tier: the actual fix. A session-scoped hard counter (hardToolSpiralLimit, default toolSpiralLimit * 3) counts raw tool segments since the last perceivable output. It is immune to turn-boundary resets, so an idle wake-up nudge mid-spiral cannot hide it. Once it's crossed, the SDK emits toolSpiralRecovering (carrying lastTurnText, the abandoned turn) and forces _coldReconnect(). This is the same full media rebuild already used for a dead media channel, replaying threadId so brain memory continues. This turns the eventual uncontrolled JoinRoomTimeout into a deliberate, bounded, self-healing reconnect.

Because the control socket is still live at this point (unlike a genuine transport drop), _coldReconnect() opens a brand-new socket rather than re-join-ing the still-connected one: re-sending join on an already-live socket is a silent no-op server-side. _coldReconnect() detects this case (this.state !== 'reconnecting' at entry means the socket never actually dropped) and opens a genuinely new socket via the same factory connect() uses, before re-join-ing on it. The one path that safely reuses the existing socket is the genuine-transport-disconnect case, reached only after a real drop already set state to 'reconnecting'. There, the prior session is already gone, so re-join-ing it is not a no-op.

The hard guard re-arms on a successful cold reconnect, not just on perceivable output. A spiral by definition never produces spoken or GenUI content, so that's the only reset path that can actually fire while one is running. Without this re-arm, a second spiral later in the same session would find the guard permanently latched from the first recovery, and would hang indefinitely instead of recovering.

A cold reconnect restores connectivity and brain memory (threadId) but otherwise abandons the turn that triggered it. With recoverFromSpiral (default true), the SDK auto-resends that turn's tracked text once, from speak() or ASR's userTranscription. It prefixes the resend with SPIRAL_RECOVERY_PREFIX (the same nudge used on the headless Conversations#send({recoverFromSpiral:true}) path), and emits spiralRecovered {text}. recoverFromSpiral:false suppresses the resend and leaves it to the app via lastTurnText. All three thresholds (brainStallMs, toolSpiralLimit, hardToolSpiralLimit) are configurable at construction; 0 disables any of them. The platform's built-in client has no such breaker. Author-side mitigation (a tool-call budget in the system prompt) and the headless-path equivalent are covered in Client-Side Commands's "Tool spirals starve the voice". This section documents only the SDK's own recovery mechanism.

KalturaChatSession (the HTTP text transport) ports the soft tier only: cfg.toolSpiralLimit (default 10, same counting rule) emits the same toolSpiralDetected {count, limit} once per turn. There's no hard tier here. A chat turn is one stateless HTTPS request with no socket to cold-reconnect, so a stuck turn is bounded by the caller's own sendText({signal}) abort, not by a session-level recovery mechanism.

Session-completion signal (session_completed): telling the backend a conversation is truly over

Without this, the backend only learns a thread is done when its idle timeout fires (about 10 minutes by default). End-of-conversation lifecycle rules (summaries, insights, CRM pushes) then fire minutes late. A closed tab looks identical to a user who just walked away.

KalturaAvatarSession, KalturaChatSession, and KalturaAgentSession all POST {genieUrl}/thread/session_completed ({"id":"<threadId>"}, the same conversation KS as every other client call) the moment a conversation genuinely ends. This includes tab-close, backgrounding, and bfcache freeze. It never fires on an internal transition like a mode switch, and never ends a thread another tab is still using. Full config surface: README.md § Ending a conversation cleanly. Wire shape: Wire Protocol · Events Catalog § Session-completion signal.

The signal is idempotent: a repeat POST for the same thread is a server-side no-op. It is never awaited on the unload path; the SDK uses fetch(url, {keepalive:true}), not navigator.sendBeacon, because sendBeacon can't carry the Authorization header. Cross-device duplicate tabs are out of scope by design (BroadcastChannel is same-origin/same-device only). The backend's own self-healing on the next real message covers that case.

What's already solid (don't regress)

• Connect-phase hang protection (per-substate timeouts).
• Clean teardown (no stale connection state; player-ready reset).
• Permission-denied UX (silent, retryable).
• TURN relay for connectivity behind hostile NATs (STV forces relay; ASR relays in practice regardless of policy; see Endpoints & Credentials).
• Mute-state preservation across ASR reconnects.
• Capacity queue (graceful waiting vs hard failure).
• Distinct, user-readable disconnect reasons.
• WHEP 404 short-circuit.
• SDK (KalturaAvatarSession) implements: ICE restart (_recoverMedia), socket-transport recovery, a repeating brain-stall watchdog plus brainStalled event, a two-tier tool-call-spiral circuit breaker (toolSpiralDetected soft, toolSpiralRecovering plus cold reconnect hard), granular device error codes (NotAllowed/NotFound/NotReadable), and online/offline/visibilitychange handling. These limitations apply only to custom clients that bypass the SDK.

Related docs