# Authoring & Consuming Widgets

[← Back to GenUI Reference](https://kaltura.github.io/intelligent-agents-sdk/reference/genui-reference/index.md)


## Authoring: which capability turns each widget on

Set capabilities **at intellect creation** (the agent's configuration). Do not change them after creation, see [Client-Side Commands](https://kaltura.github.io/intelligent-agents-sdk/guides/client-commands/index.md#gotcha-2-set-capabilities-at-creation-not-after). Source of truth: `src/management/capabilities.js` (`CAPABILITY_INFO`, `CAPABILITY_DEFAULTS`, `OFF_BY_DEFAULT`).

`kind` is `tool`, `segment`, `mode`, or `prompt`. This only names the mechanism that gates the capability on or off. It says nothing about whether the capability's *content* is persona-steerable.

`avatar_filler` (`kind: 'prompt'`) is the exception to watch for. Its filler phrasing is NOT reliably steerable via `base_directive`, even though it streams as a "spoken" segment alongside `avatar`/`text` (see [Wire Protocol · Events Catalog](https://kaltura.github.io/intelligent-agents-sdk/reference/wire-protocol/events-catalog/index.md)). Disable the capability if the default phrasing doesn't fit your persona.

| Capability | Default | Kind | Gates runtime | Notes |
|---|---|---|---|---|
| `kaltura_genie_experiences` | **ON** | mode | (master) | Master switch for structured GenUI. Leaving it on can make the agent prefer built-in widgets over a custom tool. Turn it OFF for command-only agents. See [External API Integrations § Don't skip `kaltura_genie_experiences: 'off'`](https://kaltura.github.io/intelligent-agents-sdk/guides/external-api-integrations/index.md#dont-skip-kaltura_genie_experiences-off) |
| `generate_followup_questions` | **ON** | segment | `followups` | None |
| `include_sources` | **ON** | segment | `sources` | Pairs with `use_knowledge_base` (RAG) |
| `video_gallery` | **OFF** | segment | `video-gallery` (and `content-gallery`) | None |
| `external_video` | **OFF** | segment | `external-video` | None |
| `show_link` | **OFF** | segment | `show-link` | None |
| `avatar_show_content` | **OFF** | prompt | (avatar visual push) | None |
| `use_knowledge_base` | **ON** | tool | (feeds `sources`) | `async_search_knowledge_base` RAG |

The structured data-collection form (`user-properties-form`) is configured via the intellect's `user_properties_forms` field, not a boolean capability. On the wire this field is a LIST of `{call_stage, properties:[{key,type}]}` forms. `setUserPropertiesForms` accepts a single form object and wraps it into a one-element list.

Every capability and its default is listed in [Configure an Intellect](https://kaltura.github.io/intelligent-agents-sdk/reference/api/build/intellect/index.md#configure-an-intellect).

## Consuming widgets in your app

**The 2-line happy path:** pass a DOM Element as `mount` and the SDK renders for you, live:

```js
import { ExperienceRenderer } from '@kaltura/intelligent-agents/experience/genui';

// session: an active KalturaAvatarSession from your app
new ExperienceRenderer({
  session,
  mount: document.getElementById('widgets'),
  onAction: (action, payload) => console.log(action, payload),
}).start();
```

`mountWidget` (`genui/renderers/mount.js`, exported from `./experience/genui`) is the SDK's last-mile descriptor→DOM renderer:

- Zero-dep and isomorphic. Returns `null` with no DOM.
- **Never `innerHTML`.**
- Accessible by construction.
- Ships **zero styling**. It emits the `kgenui` / `kgenui__*` class contract for you to theme.

Call it directly with `mountWidget(descriptor, targetEl, { replace?, onAction? })`, or let `ExperienceRenderer` call it when `mount` is an Element (or `target`).

### `ExperienceRenderer` options

`new ExperienceRenderer({ session?, mount, target?, onAction?, renderers?, replace?, onUnhandled?, urlPolicy?, partnerId?, uiConfId?, clearOnTurnStart?, maxRendered? })` (`genui/renderer.js`). `partnerId` alone enables `video-gallery` to build a player-embed iframe URL from each `entryId`. `uiConfId` (string or number) is optional and pins a specific player uiConf in that URL.

`mount` is a `(descriptor)=>void` function (full control) **or** a DOM Element (auto-rendered via `mountWidget`). `maxRendered` (default `100`) caps the `rendered` history: the oldest descriptor is dropped when the cap is exceeded, so a long-lived session can't grow memory unboundedly.

### Live vs. headless dispatch

In the live socket runtime, `.start()` subscribes to `session.on('brainSegment')` and flushes on `turnEnd`/`avatarStopTalking`, resetting on `interrupted`. `clearOnTurnStart` (default `true`) also resets the assembler and `clear()`s `rendered`/`last` on the session's `turnStart` event (re-emitted from the raw `agent_start_speech` socket event). So a widget from turn N never lingers into turn N+1. Set `false` for intentional cross-turn persistence.

Headless, call `.render(runtimeName, widget)` (or `.render(segment)`) per segment from a `conversations.stream()` feed. This is the reliable path.

### Registration, fallback, and provenance

- `.register(runtimeName, fn)` adds/overrides a renderer (name normalized); `.has`, `.runtimes`, `.last`, `.rendered`, `.clear()`. `replace:true` keeps only the latest widget per turn.
- An unknown runtime renders `{kind:'unknown', data:{runtime, model}}` and fires `onUnhandled` (never throws). A throwing custom renderer degrades to `{kind:'error', ...}`.
- `_meta` receipt stamps `{partnerId, source:'experience/genui', scope:'conversation (entitlement ON)', known, firstClass}`. `known` means this instance has a renderer for it (a registered 10th runtime is `known:true`). `firstClass` means it's one of the built-in set.
- `graded-question` ([GenUI · Per-Runtime Widget Detail § 10](https://kaltura.github.io/intelligent-agents-sdk/reference/genui/widgets/index.md#10-graded-question-rendergradedquestion-a-host-registered-10th-runtime)) is a concrete, shipped example of a registered 10th runtime. It's exported (`renderGradedQuestion`) but deliberately excluded from `DEFAULT_RENDERERS`/`RUNTIMES`, so it stays `known:false` until a host explicitly registers it.

### `onAction`, `WIDGET_KINDS`, and the hand-rolled escape hatch

`onAction(action, payload)` surfaces interactions `mountWidget` can't fulfil itself:

- `'followup'` `{question}` → `session.speak`
- `'submit'` `{values}` → `session.submitStructuredDataForm`
- `'play'` `{entryId, url, embedUrl}`
- `'open'` `{url}`
- `'answer'` `{questionId, variant, correct, value, explanation, optionId?}`: a `graded-question` was answered (see [GenUI · Per-Runtime Widget Detail § 10](https://kaltura.github.io/intelligent-agents-sdk/reference/genui/widgets/index.md#10-graded-question-rendergradedquestion-a-host-registered-10th-runtime))

`WIDGET_KINDS` (exported) is the frozen list of every `kind` (the nine first-class GenUI runtimes, plus `unknown` and `error`). Use it for an exhaustive host switch or a parity test.

You can still read `seg.metadata.runtimeName` and build DOM yourself (use `renderSafeLink`/`safeText`/`safeUrl` so nothing un-sanitized hits the DOM). Prefer `mountWidget`. The hand-rolled path is for a fully custom design system only.

### Markdown rendering (opt-in)

`mountWidget(descriptor, el, {markdown:true})` parses a `summary` widget's text (`genui/renderers/markdown.js`, `renderMarkdown`) as markdown instead of flat text. It supports:

- Headings (`#`-`######`)
- **bold**/__bold__ and *italic*/_italic_
- Inline `` `code` ``
- Links: `[text](https://example.com)`
- Unordered and ordered lists
- Fenced code blocks (with a `language-<token>` class)
- GFM tables

A markdown table renders through the `tableEl` builder (`genui/renderers/dom-helpers.js`), one shared, safe `<table>` construction path with no duplicate table logic.

This is markdown-**in**-plain-text rendering, not a new wire segment type. The underlying descriptor is still `{kind:'summary', data:{summary, ...}}`. Only the render path changes.

It is `never innerHTML`. Every text run is built via `textContent`/`createTextNode`, so a raw `<script>` tag in the LLM output is inert text, not markup. Every extracted URL goes through `safeUrl` too. A `javascript:` or other unsafe-scheme link degrades to plain safe text, never a dead or unsafe `href`.

Without `opts.markdown`, the summary renders as flat escaped text.

### Theming + a11y contract (`kgenui` classes)

`mountWidget` emits semantic, accessible DOM and a stable `kgenui`/`kgenui__*` class contract. Theme it in your CSS. The SDK ships none.

| Scope | Classes |
|---|---|
| Root | `kgenui kgenui--{kind}` (`role="group"`, `aria-label`); `kgenui--unknown`/`kgenui--error` for the two fallback kinds |
| Shared across widgets | `kgenui__title`, `kgenui__text`, `kgenui__list`, `kgenui__card`, `kgenui__link`, `kgenui__img`, `kgenui__gallery`, `kgenui__meta`, `kgenui__muted`, `kgenui__sr-only` |
| followups | `kgenui__chip`, `kgenui__chips` |
| flashcards | `kgenui__flip`, `kgenui__back` |
| sources | `kgenui__source`, `kgenui__snippet`, `kgenui__score` |
| summary | `kgenui__bullet` |
| video-gallery | `kgenui__player`, `kgenui__play`, `kgenui__play-glyph` |
| external-video | `kgenui__embed` (iframe wrap) |
| user-properties-form | `kgenui__form`, `kgenui__field`, `kgenui__label`, `kgenui__help` |
| user-properties-form and graded-question (free-text input/submit) | `kgenui__input`, `kgenui__submit` |
| graded-question | `kgenui__prompt`, `kgenui__feedback`, `kgenui__explanation`, `kgenui__options-group`, `kgenui__options`, `kgenui__option-label`, `kgenui__radio`, `kgenui__answer-echo` |
| Opt-in markdown (`{markdown:true}`) | `kgenui__markdown`, `kgenui__md-h`, `kgenui__md-p`, `kgenui__md-list`, `kgenui__md-code`, `kgenui__md-pre`, `kgenui__md-link`, and the shared `kgenui__table` (also used by other structured-table output) |

Built-in accessibility:

- Flashcards are `<button aria-pressed>` flip toggles.
- Followups are `<button>` chips in a labeled list.
- Links carry a visually-hidden "(opens in a new tab)" cue and `rel=noopener noreferrer`, and are **dropped** when `safe:false`.
- Images always have `alt`.
- Form fields are real `<label for>` and `<input>` elements, with `type`/`inputmode` from the field type and `aria-required`/`aria-describedby` from `required`/`description`.

The SDK ships zero CSS for these class names. Style the `.kgenui__*` block in your own stylesheet.

## `screen_share_analysis` / `sendScreenShot(data)`

An avatar agent can look at the user's screen. Turn it on with the agent's `screenShare` setting ([agent fields](https://kaltura.github.io/intelligent-agents-sdk/reference/api/build/avatar-and-agent/index.md)). Setting the `screen_share_analysis` capability on the intellect is not enough.

On a live session, call `session.startScreenShare()`. It opens the browser's screen picker and sends a still image every 2 seconds. `session.stopScreenShare()` ends it, and so does the user's own "stop sharing" control. See [Screen share](https://kaltura.github.io/intelligent-agents-sdk/reference/api/build/avatar-and-agent/index.md#screen-share) for options, events and error codes.

`session.sendScreenShot(data)` sends one still you captured yourself (`data` is an `ArrayBuffer` or `Uint8Array`). Use it only when you need your own capture path. Both calls throw `capability_disabled` unless the agent's `screenShare` setting is on.

## Related docs

| Doc | Covers |
|---|---|
| [GenUI · Model and Runtimes](https://kaltura.github.io/intelligent-agents-sdk/reference/genui/model-and-runtimes/index.md) | Runtime catalog, data flow, delivery paths, `force_experience` |
| [GenUI · Per-Runtime Widget Detail](https://kaltura.github.io/intelligent-agents-sdk/reference/genui/widgets/index.md) | Per-runtime model keys, constraints, and descriptor shapes |
| [GenUI · Widget Analytics](https://kaltura.github.io/intelligent-agents-sdk/reference/genui/analytics/index.md) | Reporting widget interactions to KAVA without double-counting |
| [GenUI Reference](https://kaltura.github.io/intelligent-agents-sdk/reference/genui-reference/index.md) | Back to the index |