# OneSay Developer Platform: full documentation The API and SDK reference pages are generated from the OneSay specifications; where a guide and a reference page differ, follow the reference. Never share Production credentials, device secrets, connection tickets, customer audio or transcripts. MCP does not carry realtime audio. ## OneSay Developer Platform URL: / # OneSay Developer Platform The Developer Platform brings together the Docs, the Console, the Playgrounds, examples, the changelog and the community page for building with the OneSay Platform APIs and SDKs. The Playgrounds run on fixed sample data and need no credential. Never paste a Production key, device credential, Realtime ticket, customer audio or transcript into a guide, playground or support report. To give a Task explicit context, create an immutable ContextSnapshot with `POST /v1/context-snapshots` and pass its `ctx_` ID as a typed TaskInput. Creating a snapshot needs the `context_snapshot.create` Grant; reading it back with `GET /v1/context-snapshots/{context_snapshot_id}` needs the separate `context_snapshot.read` Grant. ## Documentation URL: /docs # Documentation The guides explain how clients use the OneSay Control API, Task voice, Realtime RTC, and the Room, Source, SignalStream, Task and Artifact resources. When a guide and the generated reference differ, follow the reference. Neither the JavaScript SDK nor the ESP32 SDK is published to a package registry yet. ## Quickstarts - [JavaScript Quickstart](JAVASCRIPT_QUICKSTART.md): Task voice (audio you supply in, a Task result out, over the OneSay Realtime WebSocket) and Realtime RTC (publications, subscriptions, data, RPC and explicit Agents) with the installed package. - [ESP32 Quickstart](ESP32_QUICKSTART.md): Task voice from an ESP-IDF application and the Realtime RTC component. It covers building the application; flashing and provisioning a board are not covered. - [Multi-party voice Room](MULTI_PARTY_VOICE_ROOM.md): three or four people in one voice Room from the browser, step by step with the JavaScript SDK, and the current preview limits. ## SDK reference - [API Reference](../reference/generated/public-api.md): every Control API operation with its request and response fields, examples and errors. - [JavaScript SDK Reference](../reference/generated/javascript-sdk-draft.md): environments, Task and RTC entry points, session states and declared capabilities. - [ESP32 SDK Reference](../reference/generated/esp32-sdk.md): the component, Task and RTC entry points, and declared capabilities with their limitations. - [SDK Capability and Compatibility](../reference/generated/sdk-compatibility.md): the support matrix of both SDKs, unsupported and not exposed entries, the RTC lifecycle, Task recovery, limits and privacy. - [Canonical Reference](../reference/generated/catalog.md): the public API operations, events, Grants and errors. ## AI clients and diagnostics - [MCP Guide](MCP_GUIDE.md): the OneSay MCP server for AI clients, its sign-in, tools, resources and prompts. - [Device Diagnostics](DEVICE_DIAGNOSTICS.md): a Device's state and the developer-safe logs of its Project Environment. ## Security and identity - [Credential Safety](CREDENTIAL_SAFETY.md): browser, server and device boundaries. - Authentication: people sign in with Authorization Code and PKCE, servers use an environment-bound API key or Service Principal token, and devices use their own per-device credential. - API keys: the prefix shows where a key belongs (`osp_stg_` on Staging, `osp_test_` and `osp_live_` for Test and Live in Production); it is a hint, not a permission. A key is shown once, and its revocation is checked on every request. - Device provisioning: a five-minute, single-use provisioning code, then a per-device secret that the device exchanges for short-lived access tokens. Realtime uses a separate 60-second, one-time connection ticket. - Authorization: the server decides the Project and Environment of every resource. Grants are explicit, a deny always wins, and credential scopes can narrow Grants but never widen them. ## JavaScript Quickstart URL: /docs/quickstart/javascript # JavaScript Quickstart Choose [Task Voice](#javascript-task-voice) for explicit audio input and a Task result, [Realtime RTC](#javascript-realtime-rtc) for live publications and subscriptions, or [OneSay Agent](#javascript-onesay-agent) for a spoken conversation with an Agent in a Room. All use the OneSay JavaScript SDK; their lifecycles are separate. The SDK is in preview and not yet published. ## JavaScript + Task Voice Start with the [public Task sample](#public-task-sample-using-the-installed-package) and its consumer layout below. It exports `runPublicTaskSample(input, ports)` from `examples/javascript-quickstart/public-task.mjs`. For application admission, `runAuthorizedTaskSample(input, ports)` in `examples/javascript-quickstart/authorized-task.mjs` composes the public APIs and explicit Consent decisions; the [authorization walkthrough](#application-authorization-through-existing-public-apis) lists its inputs. Until the SDK is published, install `@onesay/sdk` from the package archive (`.tgz`) provided for the preview: place it next to your `package.json`, depend on it with a `file:` reference as `examples/javascript-quickstart/package.json` does, and install with `pnpm install --ignore-scripts`. Supply current Project/Environment and Source bindings, short-lived credentials through the SDK's token providers, and already captured 16 kHz mono PCM s16le. The Task client owns WSS/OSAF framing, sequence, ACK, backpressure, replay/resume and cancellation; it does not capture audio or request microphone permission. A Room created for a Task does not start an RTC session or an Agent. A Task voice session runs in this order: 1. Choose the Test Project Environment. 2. In a browser, provide a short-lived access token through `tokenProvider`; never embed an `osp_*` credential. 3. Obtain a single-use Realtime ticket from the Control API for the intended Room and Project Environment. 4. Connect with `onesay.realtime.v1`, open an Audio SignalStream, and send 16 kHz mono PCM s16le frames that your application captured. An AudioWorklet is one optional way to capture them in a browser. 5. The server sends `task.started`, then `audio.ack` and `artifact.delta` while audio flows. Keep each frame until a contiguous `audio.ack` covers it. 6. Send `signal.end`. The server answers with `signal.ended`, `artifact.finalized` and `task.completed`; wait for that terminal Task state. A ticket can be used once. A second connection with the same ticket is refused with close code 1008 `ticket_invalid`, even while the first connection is still open; request a new ticket instead. ### Try Task Voice in the Console To see a Task Voice dictation before you write code, sign in to the [Console](../developer/CONSOLE.md) on Staging and open its Task Voice section. Choose a microphone Source and a language, then speak: the page shows the text as it arrives and the final text with its usage when you stop. It runs the same chain as the sample above, a OneSay dictation Task over Task WSS/OSAF with the SDK's Task client, under your signed-in identity in the selected Project and Environment; the page's own part is capturing the microphone as 16 kHz PCM frames. Every step that creates, sends or ends something (the Room, the microphone signal, cancelling a dictation, closing the Room) is first prepared as a Consent request, and it runs only after you allow it in the page; Deny leaves it undone. The server checks each Action again, so the page grants nothing by itself. A dictation uses the trial's audio allowance, and the panel stops it by itself before the Task's audio limit. You can cancel a dictation instead of stopping it, and leave and close the Room when you are done. The section is available on Staging only, for accounts enrolled in the trial. It shows what the chain does; it is not a measure of recognition quality. ## JavaScript + Realtime RTC Use the browser example in `examples/javascript-rtc/`. Its `sample.mjs` exports `createRTCExample`; the page uses the installed `@onesay/sdk` package and the public `client.rtc.connect` facade. Copy the directory, place the SDK package archive next to its `package.json` under the file name its `file:` reference expects, then install and start the local server, which prints the page's local URL: ```text pnpm install --ignore-scripts pnpm start ``` Ctrl+C stops the server. The example creates no API proxy. The full operation examples are in `examples/javascript-rtc/README.md`. The page's SDK environment is `staging`. A connection needs an environment you are authorized to use, an application origin allowed by Control CORS, existing Room/AppSession/Source resources and a short-lived Human/AppSession token. The page keeps the token in memory; never supply a Platform API key or Provider key. Connect with an explicit initial receive set, then request publication and playback through the named controls. Empty subscriptions deny all reception. RTC connect does not create a Task or Agent; Agent creation and stop are separate authorized operations. Closing the session locally does not confirm that the server revoked the connection. ## JavaScript + OneSay Agent A OneSay Agent is a voice participant that joins your Room, listens to the publications you select and answers out loud. You can interrupt it: speak while it answers, and it stops and listens. To try it without code, sign in to the [Console](../developer/CONSOLE.md) and use its Voice call trial section (Staging only; your account must be enabled for the trial). In code, connect to the Room as in the RTC section, publish your microphone, then start the Agent with the explicit input set and a duration: ```ts const idempotencyKey = crypto.randomUUID(); const agent = await client.rtc.agents.create({ roomId, appSessionId, mode: "realtime", publicationIds: [microphonePublicationId], maxDurationSeconds: 120, idempotencyKey, }); ``` Every call that creates, changes or ends something needs the Human's confirmation (Consent): until the Human confirms, the server answers `consent_required`. After the confirmation, retry with the same `idempotencyKey`. Keep the idempotency key: if the response is lost, the same call with the same key returns the same Agent instead of starting another. Read the Agent with `client.rtc.agents.get(roomId, agent.agentExecutionId)` at most once every 2 seconds until `playback.facts.outputPublicationId` names its output, then subscribe to that publication to hear it. | `observedState` | Meaning | |---|---| | `pending` | Accepted; the Agent is not running yet | | `running` | The Agent has started | | `failed` | The Agent ended by itself before you stopped it | | `unknown` | OneSay cannot confirm what the Agent is doing (it stopped reporting, ended after your stop, or reached its maximum duration) | | `stopped` | A confirmed stop | Staging as of 2026-10-26 reports `running` while the Agent works (also after your stop request) and, once it ends, `unknown` (or `failed` if it ended by itself before any stop). `unknown` can still change to `failed` if a late report shows the Agent ended by itself first. OneSay does not report `stopped` yet; a null `effectiveStoppedAt` means the stop is not confirmed. `desiredState` is what you asked for; `observedState` is what OneSay observed. While you listen, the RTC session's `audio.agentStates` reports the Agent's output as `initializing`, `idle`, `listening`, `thinking`, `speaking` or `unknown`: a low-frequency hint for your interface, never proof of what the Agent did. Stop it with `client.rtc.agents.stop(roomId, agentExecutionId, { idempotencyKey })`; an accepted stop is a request, not proof that the Agent has left. Staging as of 2026-10-26, on the standard trial: a conversation lasts at most 120 seconds and has a cost budget; an interrupted answer uses it like any other. How many conversations an account can have is set by its free allowance, not by a count; automatic enrollment of every new account is not live yet. Longer conversations, with a very long answer cut off early (at the earliest after about 30 seconds of speech), are implemented and arrive with a later Staging update. ## Public Task sample using the installed package The entry is [`runPublicTaskSample(input, ports)`](../../examples/javascript-quickstart/public-task.mjs). It imports the public `@onesay/sdk` entry. It does not provision an identity, Project Environment, Room, Source, Consent or Grant, and it does not issue its own connection ticket. The caller must already hold authorized resource IDs and a fresh single-use ticket for the intended Room and Project Environment. Room authority is carried by that server-issued ticket; this function does not take a separate `roomId` argument or decode the ticket to authorize itself. ### Prepare the consumer layout Use a separate project with `@onesay/sdk` installed as described in [Task Voice](#javascript-task-voice). Do not replace it with a workspace alias or remove its declared dependencies. `public-task.mjs` imports two support modules from `./support/`: `pcm-session.mjs` frames the sample PCM and observes ACK, backpressure and replay, and `task-usage-readback.mjs` reads the Task and its Usage back through the public API. Place them in a `support/` folder next to `public-task.mjs`: ```text consumer/ package.json node_modules/@onesay/sdk/ public-task.mjs support/ pcm-session.mjs task-usage-readback.mjs ``` ### Call the exported function The following is application code stored next to `public-task.mjs`. Its parameters are values supplied by your authorized application; no token is embedded or printed. `observeAuthorizedTask` below is this guide's wrapper, not an additional OneSay SDK export. ```js import { buildRealtimeConnectPlan } from "@onesay/sdk"; import { runPublicTaskSample } from "./public-task.mjs"; export async function observeAuthorizedTask({ projectId, environmentId, sourceId, issuedSingleUseTicket, signal, ports, }) { return runPublicTaskSample( { projectId, environmentId, sourceId, plan: buildRealtimeConnectPlan("staging", issuedSingleUseTicket), signal, timeoutMs: 5000, }, ports, ); } ``` `buildRealtimeConnectPlan(environment, ticket)` is the public SDK function. It plans the WSS connection and performs no ticket exchange. The sample reads Usage back from the Staging Control environment; it is not a Production endpoint configuration. Only run it against an environment you are authorized to use. All four ports are required application callbacks, not SDK methods: | Port | Callback shape and responsibility | | --- | --- | | `webSocketFactory` | `(url, protocols, options) => socket`. Return the SDK-compatible WebSocket transport. Honor the supplied connection and size bounds, disable redirects, and do not relax certificate/hostname validation. | | `confirmedControlsFactory` | `(client, { signal }) => controls`, or a Promise of controls. Connect the application's explicit Grant/Consent selection through the public contracts; never approve by default. | | `readbackTokenProvider` | `({ signal }) => shortLivedToken`, or a Promise of that token. Supply a current principal authorized to read the Task and Usage in the requested scope. No long-lived platform or Provider secret belongs here. | | `readbackTransport` | `(url, init) => Promise`. Provide a Fetch-compatible transport, honor `init.signal`, and preserve the SDK's headers, no-redirect policy and scope. | The controls returned by `confirmedControlsFactory` must implement this shape: ```text send(type, data, messageId, { signal }) flush({ signal }) observe(event, { signal }) ``` `send` and `flush` may return Promises; `observe` is a synchronous event observer. Consent selection is an explicit application decision backed by server authority. Returning these methods does not grant permission: the server still validates the ticket, Source, Action, Grant and Consent. ### PCM, completion and failure behavior This sample generates exactly two 640-sample PCM16LE mono frames at 16 kHz: 80 ms of fixed synthetic input. It deliberately probes bounded backpressure and replays the first unacknowledged frame through the SDK. Duplicate replay is not new audio input or a second Task. There is no microphone capture, AudioWorklet, OS dependency or media permission request in the sample. For an application that records live audio, capture and permissions are caller responsibilities. An AudioWorklet is one optional acquisition technique; the SDK has no audio-capture export. This Task PCM boundary does not change RTC `publishMicrophone` behavior. The sample waits for contiguous ACK, the matching terminal Task and Artifact, and a bounded public Task/Usage readback, and resolves only after the SDK observes the transport closed. The result holds counts only: `pcm` reports the frames sent, acknowledged and replayed, client backpressure rejections, dropped frames and ACK latency percentiles; `readback` reports the Tasks observed, the Usage events scanned and the pages read. It contains no audio, transcript, ticket or token, and it is not a billing statement. HTTP 401/403 is terminal for this invocation and is not automatically retried. Missing, duplicate or foreign-scope Usage, wrong Task/Artifact correlation, missing ACK, caller cancellation, deadline expiry and failed cleanup reject the invocation. Obtain fresh authorization through the application's explicit flow before considering another invocation; do not replay a consumed ticket. `timeoutMs` bounds the operation and separately bounds waiting for close. The sample propagates abort to asynchronous factory/control/token/HTTP ports. A callback must honor its supplied signal and release its own resources. Bounded waiting cannot forcibly stop an uncooperative callback or undo its external side effects; synchronous callbacks must also avoid blocking the event loop. Never log the plan URL, ticket, token, raw PCM or transcript to diagnose a failure. ### Application authorization through existing public APIs `runAuthorizedTaskSample(input, ports)` in `examples/javascript-quickstart/authorized-task.mjs` is the application example that obtains its own connection material and explicitly confirms protected operations. It composes `runPublicTaskSample`; it does not replace the SDK's Task lifecycle, ACK, replay, backpressure or cancellation logic. This is an example export, not a new OneSay SDK API. Place `authorized-task.mjs` next to `public-task.mjs`. It imports one more support module, `support/confirmed-controls-core.mjs`: a bounded queue that holds each protected command until its confirmation completes. It never approves anything by itself. ```js import { runAuthorizedTaskSample } from "./authorized-task.mjs"; export async function runApplicationTask(resources, signal, ports) { return runAuthorizedTaskSample({ organizationId: resources.organizationId, projectId: resources.projectId, environmentId: resources.environmentId, deviceId: resources.deviceId, roomId: resources.roomId, sourceId: resources.sourceId, signal, timeoutMs: 60000, delegationTtlSeconds: 120, }, ports); } ``` All resources must already exist in the intended environment. Provisioning, login, Device registration, credentials and capability policy remain owned by the application's existing authorized flows. This example does not create Grants, change permissions or grant itself access. | Application port | Responsibility | | --- | --- | | `humanTokenProvider({ signal })` | Supply a current short-lived Human token authorized for the public reads, Delegation management and interactive Consent decisions. | | `deviceTokenProvider({ signal })` | Supply a separate current short-lived Device token for the intended Device. Do not substitute the Human token or pass the Device's long-lived bootstrap secret. | | `confirmConsent(snapshot, { signal })` | Display the validated, immutable server presentation and return only `"allow"` or `"deny"` after the user's explicit decision. Honor cancellation. Never default to Allow. | | `fetch(url, init)` | Preserve the SDK's request headers, scope, abort signal and redirect restrictions. | | `webSocketFactory(url, protocols, options)` | Supply a compatible transport with certificate/hostname checks and bounded connection options. | The application must implement those callbacks. The confirmation callback provides UX only: a compatible third-party client receives exactly the same server-side checks. Interactive Consent also requires a trusted, signed OIDC client identity recognized by the server; being an Owner, returning `"allow"` or inventing an Origin header cannot replace it. This Node example does not add an Origin header. The application's browser login and CSRF requirements still belong to its established browser flow. The example reads `/me` and the Project, Environment, Device, Room and Source through public Control operations. The Human token performs those reads; the Device token requests the connection. After joining, the example reads the observed Participant to check that it belongs to the configured Device before sending the protected action. These observations do not replace the server's current authorization checks. For each protected command the example explicitly confirms the exact Delegation management action and the exact Device action. Both confirmations use the public Consent helpers and server snapshots, digests, wording versions and execution bindings. A per-invocation command identity stays identical between confirmation and the actual wire send. A rejected, expired, changed or mismatched confirmation stops the invocation. The connection response's `websocket_url` contains the short-lived ticket. There is no separate response `ticket` or AppSession field. The example validates the URL through `buildRealtimeConnectPlan` and does not rewrite it. The sample uses the Staging environment. The result adds `authorization_observation` to the Task sample's result: which public checks ran, how many Delegations were created (`retained_delegations`) and the requested TTL (`requested_ttl_seconds`). It is not an authorization certificate. The client waits for transport cleanup, but created server Delegations remain until their bounded expiry or a separately confirmed revocation. Client close does not revoke a Delegation. The default and maximum requested Delegation TTL is 120 seconds; no cleanup step automatically approves a revocation. Normal identity, scope, HTTP 401/403, rejected Consent, expiry, deadline and transport failures reject the invocation without automatic retry. Public errors use fixed stage messages and preserve only supported SDK error metadata; they omit original causes, tokens, ticket URLs, resource IDs, presentation content and provider details. Use the bounded operation's application context to decide the next explicit action. Never dump transport errors or credentials. The overall authorization and Task operation shares the caller's cancellation and deadline. The sample separately bounds its close wait. Ports must honor abort; bounded waiting cannot undo an already accepted server operation or forcibly stop an uncooperative callback. If failure follows creation of a Delegation, its bounded TTL still applies; failure does not revoke it. ## ESP32 Quickstart URL: /docs/quickstart/esp32 # ESP32 Quickstart Choose [Task Voice](#esp32-task-voice) for a Task-only application or [Realtime RTC](#esp32-realtime-rtc) for live audio in a Room. OneSay OS is not required. The SDK is in preview and not yet published; use it as an ESP-IDF component in your project. All server permissions remain authoritative even if the client binary or local capability flags change. ## ESP32 + Task Voice Start from `sdks/esp32/examples/voice_button`, a standalone Task-only ESP-IDF application. Copy it into a new project directory and add the OneSay ESP32 component (the contents of `sdks/esp32`) as `components/onesay`; the example expects the component there and has no build of its own without it. Keep the component's dependency pins in `idf_component.yml`; the ESP-IDF component manager downloads missing dependencies on the first build. In that project, use ESP-IDF 6.0.2 to build, then flash your board and watch its serial output: ```text idf.py set-target esp32s3 idf.py build idf.py -p flash monitor ``` Building needs no board. Flashing writes the application only; it does not provision Device credentials. Follow `sdks/esp32/examples/voice_button/README.md` and the example-local `main/voice_button.h`. Supply `onesay_voice_button_configure` with secure Device credential storage, existing authorized Project/Environment/Application and Source/Room bindings, network access, trusted UTC/certificates and a bounded 16 kHz mono PCM s16le producer. Its default hook returns `ONESAY_ERR_AUTH` before connecting, so the unmodified example does not connect. Capture, GPIO, I2S, codec and board tuning belong to your board code. The caller serializes `onesay_voice_button_init`, `onesay_voice_button_connect` and bounded `onesay_voice_button_step` calls. The SDK owns WSS/OSAF, sequence/ACK, backpressure, replay/resume and cancellation. Keep retained PCM until ACK release; a successful cancel send is not a terminal Task result. `onesay_voice_button_close` must release the transport before its storage and dependencies can be freed. This path does not open an RTC session. ## ESP32 + Realtime RTC The component in `sdks/esp32` exposes `onesay_esp32_rtc.h` when `CONFIG_ONESAY_RTC_ENABLED` is selected. RTC is built on the LiveKit ESP32 SDK; `sdks/esp32/idf_component.yml` pins the tested version. Use ESP-IDF 6.0.2 and keep the component's dependency graph as it is. The component's `sdkconfig.rtc.defaults` is the RTC configuration fragment, loaded before application defaults; it is not a board configuration. Enabling RTC in the Task example does not turn it into an RTC application; start from the RTC project below. ### Runnable project: `rtc_smoke` The board-agnostic project [`sdks/esp32/examples/rtc_smoke`](../../sdks/esp32/examples/rtc_smoke/README.md) is the runnable RTC starting point. It is an independent ESP-IDF application that uses only the OneSay component and the `voice_button` Task flow. It has no GPIO, I2S, codec, BSP, AEC, Wi-Fi or NVS code, and it does not depend on OneSay OS. One run streams a Task, then connects RTC with synthetic capture, sends one packet, releases everything and prints one content-free summary line on the serial console. Create the project the same way as the Task example: copy `rtc_smoke` into a new directory, add the component as `components/onesay`, and copy `main/voice_button.c` and `main/voice_button.h` from the `voice_button` example into its `main/`. Then build and flash it with the `idf.py` commands above. Provide a strong `onesay_rtc_smoke_configure`. Its weak default returns `ONESAY_ERR_AUTH` before any network activity. Before it returns `ONESAY_OK`, your firmware brings up the network, sets trusted UTC and fills the configuration from your secure store. The project's README lists every field. Board settings (flash size, PSRAM, console) go in a third defaults file. ### Device credential to RTC bearer The RTC owner takes a caller bearer and never exchanges a Device secret. To go from the Device credential to RTC with public calls only, set `rtc_bearer_from_device_credential = true` in the smoke configuration. The chain is: 1. **Owner authority, before the device runs.** The owner creates the Delegation and decides the Consent requests. The Device creates its AppSession under that Delegation with `createAppSession`. The device receives the Room, AppSession, Source and delegation IDs. IDs grant nothing; the server checks the actual Grant, Delegation and Consent on every request. 2. **Device secret to short-lived token.** The `rtc_bearer` step loads the Device secret once through your existing secure-store callback (`task.credentials.load`). It sends the secret only as the `Authorization` of one `exchangeDeviceAccessToken` request (`POST /v1/device-access-tokens`, body `{}`) through the public `onesay_esp32_control_request`. The RTC Control configuration must name the same environment, Device, Project and Environment as the Task one, or the step fails before the secret is loaded. The secret is erased whatever happens. Only the `access_token` of the `{"data":{"access_token","token_type":"Bearer","expires_at"}}` response envelope is kept. The store's `erase`, a factory reset, is never called. 3. **Token to RTC.** `onesay_esp32_rtc_prepare`, `onesay_esp32_rtc_connect_audio` and the packet policy receive only that short-lived Device access token, on the same Control owner. The smoke erases its copy after release. Your idempotency-key callback returns a fresh key for `exchangeDeviceAccessToken` in each run, because a replayed key returns the original expiry. It returns the original Consent-bound key for each RTC operation. To use a bearer from another established flow instead, supply the `load_rtc_bearer` callback. Exactly one of the two sources is accepted. Never put a long-lived Device secret, Platform key, Provider key or LiveKit secret in firmware sources, logs or documentation. ### Prepare the application inputs Initialize the `onesay_esp32_control_context_t` using `onesay_esp32_control_init` and your `onesay_esp32_control_config_t`. Supply the expected Device/Project/Environment/Application bindings, certificate or verified certificate bundle, bounded request timeout and `next_idempotency_key` callback. The shared Control initializer requires a valid `device_id` even when the separate RTC bearer is a Human; that configured ID does not change the bearer's identity or authorize RTC. Set `environment` to the SDK enum value for the environment you are authorized to use. Network setup, trusted UTC and certificate storage remain application responsibilities. Do not disable TLS hostname or date validation. The audio sequence below needs these caller-owned inputs: | Input | Public contract | | --- | --- | | `control` | Initialized Control owner; retain its certificate and key-callback dependencies while using it. | | `bearer` / `bearer_length` | Current short-lived Human or delegated Device/AppSession bearer, for example the Device access token from the [Device credential chain](#device-credential-to-rtc-bearer). RTC prepare does not exchange a Device secret or create an AppSession. Do not embed Platform, Provider or LiveKit long-term secrets. | | `input` | `onesay_action_rtc_connection_create_input_t` containing the existing `room_id` and `app_session_id`, with terminated strings. Resources must already exist and be authorized. | | `connection_delegation` | The selected connection operation's `delegation_id`, or `NULL` when omitted. It is request metadata, not permission. | | `audio` | `onesay_rtc_audio_config_t`: authorized `source_id`, that publication operation's separate `delegation_id` (empty omits it), capture/render callbacks and `now_unix_ms` with its context. | | `owner` | A caller-retained pointer initialized to `NULL`; never pass a slot already owning a session to prepare. | The key callback must return the original key for each logical operation: `createRTCConnection` and `createRTCPublication` are separate operations. When Consent/Delegation applies, each key must match its own confirmed action. The connection's selector is not inherited by publication. Server checks remain authoritative; a valid local selector or callback cannot grant access. ### Prepare, connect, observe and release This is application code using the public header, not another SDK export or a complete board application. It requires the inputs above. ```c #include "onesay_esp32_rtc.h" onesay_err_t example_begin_rtc_audio( onesay_esp32_control_context_t *control, const uint8_t *bearer, size_t bearer_length, const onesay_action_rtc_connection_create_input_t *input, const char *connection_delegation, const onesay_rtc_audio_config_t *audio, onesay_esp32_rtc_t **owner) { if (owner == NULL || *owner != NULL) return ONESAY_ERR_INVALID_ARG; onesay_err_t result = onesay_esp32_rtc_prepare( control, bearer, bearer_length, input, connection_delegation, owner); if (result != ONESAY_OK) return result; /* Keep *owner even on failure: the caller still owns cleanup. */ return onesay_esp32_rtc_connect_audio( *owner, control, bearer, bearer_length, audio); } ``` Serialize these public calls on the application thread, outside all media, network and clock callbacks. Keep the pointer and callback contexts alive after the function returns. Failed prepare leaves the initially empty pointer `NULL`, but an ambiguous HTTP outcome still needs server reconciliation. After successful prepare, even failed connect leaves an owned pointer requiring close. `ONESAY_OK` from connect means that the connection attempt was accepted; it does not mean connected, published, heard or effectively revoked. | Next call, where `owner` is the retained session pointer | Meaning and required handling | | --- | --- | | `onesay_esp32_rtc_read_binding(owner, &connection)` with `onesay_rtc_binding_t connection = {0}` | Copies original OneSay IDs, generation, deadline and observed state. Keep PREPARED/CONNECTING/CONNECTED/RECONNECTING/FAILED/CLOSED distinct; none replaces server authority. | | `onesay_esp32_rtc_read_audio_binding(owner, &publication)` with `onesay_rtc_audio_binding_t publication = {0}` | Copies content-free publication IDs when available, including after a connection failure. Check the return value: unavailable publication returns `ONESAY_ERR_INVALID_ARG` and cleared output, not a fabricated ID. | | `onesay_esp32_rtc_close(&owner)` when `owner != NULL` | Cancels board I/O, stops the RTC session and releases media owners. Only `ONESAY_OK` clears the pointer. On failure retain the pointer and all callback contexts for an explicit later cleanup attempt; do not free or reinitialize them. | Read and retain any IDs needed for server reconciliation before successful close. The `CLOSED` observation alone does not prove cleanup: a failed close can leave an owned pointer. Local close does not execute `endRTCPublication` or `revokeRTCConnection`; server cleanup uses those public operations with their own current authorization and idempotency/Consent bindings. Capture must supply exactly the requested 16 kHz mono s16le samples, at most 640 per read. Render must accept the actual decoded s16le format: 8/12/16/24/48 kHz, mono or stereo, at most 120 ms per callback (5,760 frames at 48 kHz). All callbacks borrow their buffers until return and have a 40 ms budget. The idempotent `cancel` callback may run concurrently and must wake I/O without freeing its context. Callbacks must not invoke SDK lifecycle functions. GPIO, codec, resampling and board tuning are not supplied by this application code; the SDK's media pipeline handles encoding and format conversion. On denied authorization, inspect `onesay_esp32_control_last_error(control)` for the error code, request ID and retryability; do not dump tokens, connection material or transport bodies. Expired first-use material cannot be rescued by a longer business deadline or a changed client clock. A failed or ambiguous publication/connect attempt must not be retried by calling connect again or minting a replacement key. Preserve the original IDs/key, release local owners and reconcile through the application's explicit authorization flow. For the full interface, see the [SDK audio-owner documentation](../../sdks/esp32/README.md#rtc-audio). The `rtc_smoke` project uses synthetic capture and a counting render sink. It does not supply microphone, speaker, AEC or audio-quality tuning; those belong to your board. The Task-only `voice_button` example does not use RTC, and Task or JavaScript support does not imply ESP32 RTC support for a feature. ## Credentials on your board Provision each device with its own Device credential and keep it in protected storage, for example encrypted NVS. Never compile a Project API key or Device credential into firmware, and never print it to the serial log. Supply certificates and trusted UTC before the first connection. ## Credential Safety URL: /docs/security/credentials # Credential Safety - Browser: short-lived token provider only; no Platform API key. - Server: environment-scoped API key or token provider, never client-shipped. - ESP32: future per-device credential in encrypted NVS when supported; never a Project API key, Kconfig plaintext or serial-log value. - Realtime: single-use ticket, passed only to the OneSay Realtime endpoint. - Test and live credentials are never interchangeable. The Docs, the Developer Portal and the Playgrounds never ask for a credential; only the Console signs you in. ## MCP Guide URL: /docs/guides/mcp # MCP Guide The OneSay MCP server lets an AI client read your OneSay Projects, Devices, Rooms, Tasks, logs and Usage, and ask for a few bounded changes, through the Model Context Protocol. It is a thin layer over the public Control API: every tool calls a public API operation or Action, and the Control API decides what your account may do. The server is available on Staging; see [Availability](#availability). MCP never carries audio. Voice goes through the SDKs: Task voice over the OneSay Realtime WebSocket, or Realtime RTC. ## Connect - **Endpoint:** `https:///mcp`, one per environment. On Staging it is `https://mcp.staging.onesay.com/mcp`. - **Transport:** stateless Streamable HTTP. The server keeps no session between requests. - **Protocol:** MCP `2026-07-28` only. Every request sends `Mcp-Protocol-Version: 2026-07-28`; another version is refused with `mcp_protocol_version_unsupported`, naming the supported one. - **Request headers:** `Mcp-Method` must equal the body's method, and for `tools/call`, `prompts/get` and `resources/read` `Mcp-Name` must equal the tool name, prompt name or resource URI. A mismatch is answered with HTTP 400 and JSON-RPC error `-32020` before any token is read. ## Sign in The server is an OAuth 2.1 protected resource. Its Protected Resource Metadata is served at `/.well-known/oauth-protected-resource` (and at `/.well-known/oauth-protected-resource/mcp`), and it names the environment's authorization server. - **No token** is needed for public discovery: the R0 tools, the quickstart resources, the prompts and the tool, resource and prompt lists. - **Account data needs a token.** An R1 or R2 tool, or a resource other than a quickstart, needs a Bearer access token whose audience is this MCP resource. Without one the server answers HTTP 401 with a `WWW-Authenticate` challenge that points to the metadata. The body is a JSON-RPC error carrying your request id, with the `oauth_resource_invalid` error envelope as its data. - **Your token never reaches the Control API.** For each Control API call the server exchanges it (OAuth 2.0 Token Exchange, RFC 8693) for a short-lived token whose subject is you and whose actor is the MCP service. No exchange is made after your token expires, and no token is logged or returned. ## Tools Each tool has a risk class. Read-only tools are marked read-only to the client; `rooms.close` and `tasks.cancel` are marked destructive. | Risk | Tools | Needs | |---|---|---| | R0 | `onesay.capabilities.list`, `onesay.capabilities.get`, `onesay.quickstarts.get` | Nothing: public metadata | | R1 | `onesay.projects.list`, `onesay.projects.get`, `onesay.applications.list`, `onesay.devices.list`, `onesay.devices.get`, `onesay.devices.diagnose`, `onesay.rooms.get`, `onesay.tasks.get`, `onesay.logs.search`, `onesay.logs.timeline`, `onesay.usage.summary` | A token and the Grants the matching API operation requires | | R2 | `onesay.rooms.create`, `onesay.rooms.close`, `onesay.tasks.cancel` | A token, the Grant and the human's consent | An R2 tool runs its Action only after the human has consented in OneSay. Without that consent the call answers `mcp_confirmation_required`, and nothing changes. On Staging the R2 tools are switched off for now: they answer `mcp_tool_not_exposed` and change nothing. ## Resources | URI template | Reads | |---|---| | `onesay://projects/{project_id}` | The Project, like `onesay.projects.get` | | `onesay://devices/{device_id}` | The Device's state, like `onesay.devices.get` | | `onesay://tasks/{task_id}` | The Task's state, like `onesay.tasks.get` | | `onesay://docs/quickstarts/{name}` | A quickstart page (`javascript` or `esp32`); public | The server lists templates only. Every read result has `ttlMs` 0, so a client treats it as immediately stale. ## Prompts - `build_voice_app` follows the [JavaScript Quickstart](JAVASCRIPT_QUICKSTART.md). - `diagnose_device` follows the [Device Diagnostics](DEVICE_DIAGNOSTICS.md) guide. ## Errors A tool error carries the OneSay error envelope, the same as the Control API's: | Code | Meaning | |---|---| | `oauth_resource_invalid` | The token is missing, invalid, for another resource, or was refused by the token exchange. | | `capability_authority_unavailable` | OneSay could not check your access right now. Retry. | | `mcp_confirmation_required` | An R2 call has no human consent yet. | | `consent_required` | The Control API needs a Consent the account has not given. | | `mcp_tool_input_invalid` | An argument is missing or does not match the tool's schema. | | `mcp_tool_not_exposed` | The tool is not enabled on this server. | | `mcp_tool_output_invalid` | The Control API answered with something the tool cannot return. | Errors from the Control API itself, such as a missing Grant or an unknown resource, are passed through with their own codes. The [Canonical Reference](../reference/generated/catalog.md) lists every code. ## Availability - The server runs on Staging at `https://mcp.staging.onesay.com/mcp`. Production is not available yet. - On Staging the R2 tools are switched off and answer `mcp_tool_not_exposed`. - Tools, prompts and resources are fixed per release, so the server sends no list-changed notifications. ## Device Diagnostics URL: /docs/guides/device-diagnostics # Device Diagnostics When a Device misbehaves, start from two server-side facts: its canonical state, and the developer-safe logs of the Project Environment it belongs to. Both come from the Control API, and the Device itself is never queried. The same steps work from your own server and from an AI client through the [MCP Guide](MCP_GUIDE.md). ## What you need - The Device id (`dev_…`), its Project id and its Environment id. - An access token for your account with the `device.read` and `logs.read` Grants in that Project Environment. ## Read the Device's canonical state `GET /v1/devices/{device_id}` returns the Device as OneSay knows it. The Control API, not the Device, is the source of this state: a Device that cannot reach OneSay still has its last recorded state here. ## Search the developer-safe logs `GET /v1/logs?project_id=…&environment_id=…` lists the Project Environment's developer-safe logs, a page at a time: pass `next_cursor` back as `cursor` while `has_more` is true. An entry has an event, a severity, a time, a latency, the ids of the resources it concerns (look for your `device_id` there) and safe fields such as an `error_code`. It never carries audio, a transcript or a credential. Each entry names its `request_id` (`req_…`). `GET /v1/logs/{request_id}` returns that request's timeline, which shows where a failing call stopped. ## With an AI client - `onesay.devices.diagnose` takes `device_id`, `project_id` and `environment_id` (and optionally `cursor` and `limit`). It returns one object with the Device's canonical state under `getDevice` and one page of logs under `listDeveloperLogs`. - `onesay.logs.timeline` takes a `request_id` from those logs. - The `diagnose_device` prompt walks a client through these steps, using this page. Both tools are R1: they need a token and change nothing. ## What these steps do not cover Board hardware, firmware and radio quality are diagnosed by the device's OS and board vendor, not by the Platform. A Device's own local logs never reach OneSay unless the device runtime sends them. ## Multi-party voice Room URL: /docs/guides/multi-party-voice # Multi-party voice Room Three or four people can talk in one voice Room, each from their own browser. A Room holds up to four people. LiveKit carries the audio, and OneSay decides who may join, speak and listen: the server checks every step, whatever the client sends. The [Console](../developer/CONSOLE.md) has a ready-made Voice room panel that walks you through these steps. This page explains them, so that you can build the same flow in your own application. This is a preview with a fixed group. Read the current limits at the end of this page before you invite people. ## What you need - A Project Environment on Staging, and an Application in it. - For each person: their own OneSay sign-in through your application, which gives a short-lived Human token (never put an API key in a browser), and a microphone Source. - For each person, these Grants in that Project Environment: `room.read`, `participant.read`, `room.join`, `source.use`, `signal.audio.publish`, `rtc.subscription.configure`, `signal.audio.subscribe`, `signal.audio.end`, `app.session.create` and `app.session.close`. The person who creates the Room also needs `room.create` and `room.close`. ## Steps Each person walks these steps in order. Join one at a time: wait until the previous person appears in the participant list (`listRoomParticipants`; in the Console, under Who is here) before you join; people who join or start listening at the same moment can fail to connect or stay unheard. 1. **Create or join the Room.** One person creates it with `POST /rooms` (`createRoom`), with `max_participants` set to the group size (3 or 4), and shares its `room_` id. The others read it with `GET /rooms/{room_id}` (`getRoom`) to check that it is open and has not started. 2. **Start an app session** with `POST /app-sessions` (`createAppSession`) for your Application. 3. **Connect** with the SDK's `client.rtc.connect` (`createRTCConnection`). Nobody is heard yet. 4. **Publish your microphone** with `session.audio.publishMicrophone` (`createRTCPublication`) and your microphone Source. 5. **Let the next person join.** The next person starts step 3 only when you appear in the participant list (step 6). 6. **Read the roster.** `GET /rooms/{room_id}/participants` (`listRoomParticipants`) lists who is in the Room, and `GET /rooms/{room_id}/signals` (`listRoomSignals`) lists their audio. A person is speaking when one of their signals has the profile `audio.rtc.v1` and the status `active`. Name only `active` signals: a `pending` one is not published yet. 7. **Listen, once.** When everyone is speaking, each person sets one subscription with `session.audio.setSubscriptions` (`updateRTCSubscriptions`): the `pub_` ids from the `publication_id` field of the others' `active` signals, sorted, with `expected_policy_version` 0. Start listening one person at a time, a few seconds apart, and do not change the set afterwards. 8. **Turn on sound** with `session.audio.enablePlayback`, called from the person's own click, as browsers require. 9. **Leave.** End the microphone publication (`endRTCPublication`), then close the session, which revokes the connection (`revokeRTCConnection`), and close the app session (`closeAppSession`). The creator may also close the Room (`closeRoom`). ## JavaScript This sketch uses only the public JavaScript SDK. `humanToken`, the ids and the idempotency keys come from your application; keep each key, so that a retry of the same step sends the same key. ```js import { OneSayClient } from "@onesay/sdk"; // A short-lived Human token from your own sign-in; never an API key. const client = new OneSayClient({ environment: "staging", tokenProvider: async () => humanToken }); // Step 3: connect with your app session. const session = await client.rtc.connect({ roomId, appSessionId, idempotencyKey: connectKey }); // Step 4: publish your microphone Source. const microphone = await session.audio.publishMicrophone({ sourceId, idempotencyKey: publishKey }); // Step 7: when everyone speaks, listen once to the others' active publications. await session.audio.setSubscriptions({ publicationIds: otherPublicationIds, expectedPolicyVersion: 0, idempotencyKey: listenKey }); // Step 8: sound starts from the person's own click. soundButton.onclick = () => session.audio.enablePlayback(); // Step 9: leave. await microphone.end({ idempotencyKey: endKey }); await session.close({ idempotencyKey: closeKey }); ``` The Room and the app session are created through the Control API (`createRoom`, `createAppSession`), as in step 1 and step 2. ## Consent On Staging, every step that creates, changes or ends something asks the person for confirmation first: creating the Room, starting the app session, connecting, publishing, listening and each part of leaving. Your application asks OneSay for a confirmation of that exact step (`createConsentRequest`), shows it, records the person's own Allow or Deny (`decideConsentRequest`) and then runs the step with the same input. Nothing is allowed by default, and reads such as the roster need no confirmation. The Console's Voice room panel shows this pattern. ## Current limits (preview) - **A fixed group.** The creator picks three or four people. Everyone joins and speaks before anyone listens, and each person sets their subscription once. - **Someone leaving no longer ends the others' connections, but the group cannot change.** Nobody can be added or removed in this preview: do not change your subscription set after someone leaves. To talk again as a different group, leave and start a new Room. - **One at a time.** Join one at a time: wait until the previous person appears in the participant list (`listRoomParticipants`; in the Console, under Who is here) before you join; people who join or start listening at the same moment can fail to connect or stay unheard. If a step fails because the service is busy, try again with the same key. If you still cannot hear someone, leave and start a new Room. - **At most four people** in one Room. - **Voice only.** Video, screen sharing, recording, moderation, guest links and dial-in are not part of this preview. ## Canonical Reference URL: /docs/reference # Reference Catalog The capabilities, actions, Control API operations, Grants, events, error codes and MCP tools of the OneSay Platform, generated from its registries. Risk levels: R0 is public metadata; R1 is an authorized read; R2 is a bounded change that may need confirmation; R3 is a destructive or live-sensitive change that always needs confirmation. ## Capabilities Capabilities a client can discover, with the clients that use them. | ID | Clients | |---|---| | `speech.transcription.streaming.v1` | javascript, esp32 | | `capability.platform.discovery` | javascript, mcp | ## Actions Each action with its risk level, the Grants it requires and the MCP tool that offers it. | ID | Risk | Grants | MCP tool | |---|---|---|---| | `room.create` | R2 | `room.create` | `onesay.rooms.create` | | `room.close` | R2 | `room.close` | `onesay.rooms.close` | | `task.cancel` | R2 | `task.cancel` | `onesay.tasks.cancel` | | `application.requirements.read` | R1 | `application.requirements.read` | `onesay.application.requirements.read` | | `application.requirements.replace` | R2 | `application.manage` | `onesay.application.requirements.replace` | | `application.compatibility.read` | R1 | `application.compatibility.read` | `onesay.application.compatibility.read` | | `app.session.create` | R2 | `app.session.create` | `onesay.app.session.create` | | `app.session.read` | R1 | `app.session.read` | `onesay.app.session.read` | | `app.session.close` | R2 | `app.session.close` | `onesay.app.session.close` | | `app.session.revoke` | R3 | `app.session.revoke` | `onesay.app.session.revoke` | | `app.session.capabilities.read` | R1 | `app.session.capabilities.read` | `onesay.app.session.capabilities.read` | | `signal.open` | R2 | `source.use`, `signal.audio.publish` | — | | `delegation.create` | R3 | `delegation.create` | — | | `delegation.read` | R1 | `delegation.read` | — | | `delegation.revoke` | R3 | `delegation.revoke` | — | | `data.sync.scope.enable` | R3 | `data.sync.control` | — | | `data.sync.scope.disable` | R3 | `data.sync.control` | — | | `rtc.connection.create` | R2 | `room.join` | — | | `rtc.connection.read` | R1 | `room.join` | — | | `rtc.connection.revoke` | R2 | `room.join` | — | | `rtc.media_key.read` | R1 | `room.join` | — | | `rtc.publication.create` | R2 | `room.join`, `source.use` | — | | `rtc.publication.end` | R2 | `room.join` | — | | `rtc.subscription.update` | R2 | `room.join`, `rtc.subscription.configure` | — | | `rtc.packet.policy.update` | R2 | `room.join`, `rtc.packet.configure` | — | | `agent.execution.create` | R2 | `room.join`, `source.use`, `signal.audio.subscribe`, `agent.execution.create` | — | | `agent.execution.read` | R1 | `agent.execution.read` | — | | `agent.execution.stop` | R2 | `agent.execution.stop` | — | ## Public API operations Every operation of the Control API with the Grants it requires. [API Reference](public-api.md) shows the request, response, examples and errors of each. | Operation | Method | Path | Grants | |---|---|---|---| | `getCurrentUser` | GET | `/me` | none (authentication policy) | | `createOrganization` | POST | `/organizations` | none (authentication policy) | | `listOrganizations` | GET | `/organizations` | `organization.read` | | `getOrganization` | GET | `/organizations/{organization_id}` | `organization.read` | | `updateOrganization` | PATCH | `/organizations/{organization_id}` | `organization.manage` | | `listOrganizationMemberships` | GET | `/organizations/{organization_id}/memberships` | `organization.read` | | `createOrganizationMembership` | POST | `/organizations/{organization_id}/memberships` | `organization.manage` | | `createProject` | POST | `/projects` | `project.manage` | | `listProjects` | GET | `/projects` | `project.read` | | `getProject` | GET | `/projects/{project_id}` | `project.read` | | `updateProject` | PATCH | `/projects/{project_id}` | `project.manage` | | `archiveProject` | POST | `/projects/{project_id}/archive` | `project.manage` | | `listProjectEnvironments` | GET | `/projects/{project_id}/environments` | `project.read` | | `createApplication` | POST | `/applications` | `application.manage` | | `listApplications` | GET | `/applications` | `application.read` | | `getApplication` | GET | `/applications/{application_id}` | `application.read` | | `updateApplication` | PATCH | `/applications/{application_id}` | `application.manage` | | `revokeApplication` | POST | `/applications/{application_id}/revoke` | `application.manage` | | `createCredential` | POST | `/credentials` | `credential.create` | | `listCredentials` | GET | `/credentials` | `credential.read` | | `getCredential` | GET | `/credentials/{credential_id}` | `credential.read` | | `rotateCredential` | POST | `/credentials/{credential_id}/rotate` | `credential.create` | | `revokeCredential` | POST | `/credentials/{credential_id}/revoke` | `credential.revoke` | | `createDevice` | POST | `/devices` | `device.create` | | `listDevices` | GET | `/devices` | `device.read` | | `getDevice` | GET | `/devices/{device_id}` | `device.read` | | `updateDevice` | PATCH | `/devices/{device_id}` | `device.update` | | `publishDeviceCapabilities` | PUT | `/devices/{device_id}/capabilities` | `device.capabilities.publish` | | `createDeviceProvisioningSession` | POST | `/devices/{device_id}/provisioning-sessions` | `device.update` | | `exchangeDeviceAccessToken` | POST | `/device-access-tokens` | none (authentication policy) | | `exchangeDeviceProvisioningCode` | POST | `/device-provisioning/exchange` | none (authentication policy) | | `rotateDeviceCredential` | POST | `/devices/{device_id}/credentials/rotate` | `device.update` | | `revokeDevice` | POST | `/devices/{device_id}/revoke` | `device.revoke` | | `createSource` | POST | `/sources` | `source.create` | | `listSources` | GET | `/sources` | `source.read` | | `getSource` | GET | `/sources/{source_id}` | `source.read` | | `createContextSnapshot` | POST | `/context-snapshots` | `context_snapshot.create` | | `getContextSnapshot` | GET | `/context-snapshots/{context_snapshot_id}` | `context_snapshot.read` | | `createRoom` | POST | `/rooms` | `room.create` | | `listRooms` | GET | `/rooms` | `room.read` | | `getRoom` | GET | `/rooms/{room_id}` | `room.read` | | `closeRoom` | POST | `/rooms/{room_id}/close` | `room.close` | | `listRoomParticipants` | GET | `/rooms/{room_id}/participants` | `participant.read` | | `listRoomSignals` | GET | `/rooms/{room_id}/signals` | `room.read` | | `createRealtimeConnection` | POST | `/realtime/connections` | `room.join` | | `resumeRealtimeConnection` | POST | `/realtime/connections/resume` | `room.join` | | `getRealtimeCapabilities` | GET | `/realtime/capabilities` | `capability.read` | | `createTask` | POST | `/tasks` | `task.dictation.create` | | `listTasks` | GET | `/tasks` | `task.read` | | `getTask` | GET | `/tasks/{task_id}` | `task.read` | | `cancelTask` | POST | `/tasks/{task_id}/cancel` | `task.cancel` | | `getUsageSummary` | GET | `/usage/summary` | `usage.read` | | `listUsageEvents` | GET | `/usage/events` | `usage.read` | | `listDeveloperLogs` | GET | `/logs` | `logs.read` | | `getDeveloperLogTimeline` | GET | `/logs/{request_id}` | `logs.read` | | `listCapabilities` | GET | `/capabilities` | `capability.read` | | `getCapability` | GET | `/capabilities/{capability_id}` | `capability.read` | | `listActions` | GET | `/actions` | `capability.read` | | `getAction` | GET | `/actions/{action_id}` | `capability.read` | | `getApplicationRequirements` | GET | `/applications/{application_id}/requirements` | `application.requirements.read` | | `replaceApplicationRequirements` | PUT | `/applications/{application_id}/requirements` | `application.manage` | | `evaluateApplicationCompatibility` | GET | `/applications/{application_id}/compatibility` | `application.compatibility.read` | | `createAppSession` | POST | `/app-sessions` | `app.session.create` | | `getAppSession` | GET | `/app-sessions/{app_session_id}` | `app.session.read` | | `closeAppSession` | POST | `/app-sessions/{app_session_id}/close` | `app.session.close` | | `revokeAppSession` | POST | `/app-sessions/{app_session_id}/revoke` | `app.session.revoke` | | `getAppSessionCapabilities` | GET | `/app-sessions/{app_session_id}/capabilities` | `app.session.capabilities.read` | | `createConsentRequest` | POST | `/consent-requests` | none (authentication policy) | | `getConsentRequest` | GET | `/consent-requests/{consent_request_id}` | none (authentication policy) | | `decideConsentRequest` | POST | `/consent-requests/{consent_request_id}/decision` | none (authentication policy) | | `createDelegation` | POST | `/delegations` | `delegation.create` | | `listDelegations` | GET | `/delegations` | `delegation.read` | | `getDelegation` | GET | `/delegations/{delegation_id}` | `delegation.read` | | `revokeDelegation` | POST | `/delegations/{delegation_id}/revoke` | `delegation.revoke` | | `enableDataSyncScope` | POST | `/data-sync/scopes/enable` | `data.sync.control` | | `disableDataSyncScopeRetainingData` | POST | `/data-sync/scopes/disable-retain` | `data.sync.control` | | `readDataSyncChanges` | GET | `/data-sync/changes` | `data.sync.read` | | `createDataSyncSnapshot` | POST | `/data-sync/snapshots` | `data.sync.read` | | `readDataSyncSnapshot` | GET | `/data-sync/snapshots/{snapshot_id}` | `data.sync.read` | | `completeDataSyncSnapshot` | POST | `/data-sync/snapshots/{snapshot_id}/complete` | `data.sync.read` | | `createRTCConnection` | POST | `/rtc/connections` | `room.join` | | `getRTCConnection` | GET | `/rtc/connections/{connection_id}` | `room.join` | | `revokeRTCConnection` | POST | `/rtc/connections/{connection_id}/revoke` | `room.join` | | `getRTCMediaKeys` | GET | `/rtc/connections/{connection_id}/media-keys` | `room.join` | | `createRTCPublication` | POST | `/rooms/{room_id}/rtc/publications` | `room.join`, `source.use` | | `endRTCPublication` | POST | `/rooms/{room_id}/rtc/publications/{publication_id}/end` | `room.join` | | `updateRTCSubscriptions` | PUT | `/rooms/{room_id}/rtc/subscriptions` | `room.join`, `rtc.subscription.configure` | | `createAgentExecution` | POST | `/rooms/{room_id}/agent-executions` | `room.join`, `source.use`, `signal.audio.subscribe`, `agent.execution.create` | | `getAgentExecution` | GET | `/rooms/{room_id}/agent-executions/{agent_execution_id}` | `agent.execution.read` | | `stopAgentExecution` | POST | `/rooms/{room_id}/agent-executions/{agent_execution_id}/stop` | `agent.execution.stop` | | `updateRTCPacketPolicy` | PUT | `/rooms/{room_id}/rtc/packet-policy` | `room.join`, `rtc.packet.configure` | ## Grants The permissions an operation or tool can require. - `organization.read` - `organization.manage` - `capability.read` - `project.read` - `project.manage` - `application.read` - `application.manage` - `credential.create` - `credential.read` - `credential.revoke` - `device.create` - `device.read` - `device.update` - `device.capabilities.publish` - `device.revoke` - `room.read` - `room.create` - `room.join` - `room.close` - `participant.read` - `source.create` - `source.read` - `source.use` - `signal.audio.publish` - `signal.audio.end` - `task.dictation.create` - `task.rtc.transcription.create` - `task.read` - `task.cancel` - `context_snapshot.create` - `context_snapshot.read` - `artifact.read` - `artifact.use` - `artifact.subscribe` - `logs.read` - `usage.read` - `delegation.create` - `delegation.read` - `delegation.revoke` - `application.requirements.read` - `application.compatibility.read` - `app.session.create` - `app.session.read` - `app.session.close` - `app.session.revoke` - `app.session.capabilities.read` - `data.sync.read` - `data.sync.write` - `data.sync.control` - `data.sync.delete` - `rtc.subscription.configure` - `rtc.packet.configure` - `rtc.packet.send` - `rtc.packet.receive` - `signal.audio.subscribe` - `agent.execution.create` - `agent.execution.read` - `agent.execution.stop` - `rtc.stream.configure` - `rtc.stream.send` - `rtc.stream.receive` - `rtc.rpc.configure` - `rtc.rpc.call` - `rtc.rpc.handle` - `signal.data.publish` - `signal.data.end` - `signal.data.subscribe` ## Device grant ceiling The most a Device credential can be granted. - `device.capabilities.publish` - `capability.read` - `room.read` - `room.join` - `participant.read` - `source.read` - `source.use` - `signal.audio.publish` - `signal.audio.end` - `task.dictation.create` - `task.read` - `task.cancel` - `context_snapshot.read` - `artifact.read` - `artifact.subscribe` - `application.requirements.read` - `application.compatibility.read` - `app.session.create` - `app.session.read` - `app.session.close` - `app.session.capabilities.read` ## Realtime events The events the Realtime WebSocket delivers. - `connection.welcome` - `connection.resumed` - `connection.resume_rejected` - `connection.closing` - `room.joined` - `room.left` - `participant.joined` - `participant.left` - `signal.opened` - `audio.ack` - `signal.ended` - `signal.failed` - `task.created` - `task.started` - `artifact.delta` - `artifact.finalized` - `task.completed` - `task.failed` - `task.cancelled` - `pong` - `error` ## Error codes The values of `error.code` in an error response. - `authentication_required` - `token_invalid` - `token_expired` - `token_audience_invalid` - `credential_revoked` - `credential_expired` - `ticket_invalid` - `ticket_expired` - `ticket_consumed` - `grant_missing` - `resource_owner_mismatch` - `project_environment_mismatch` - `principal_suspended` - `device_revoked` - `resource_not_found` - `resource_state_invalid` - `room_closed` - `participant_limit_reached` - `source_not_found` - `signal_not_found` - `artifact_not_found` - `artifact_unavailable` - `task_input_invalid` - `task_profile_input_unsupported` - `task_not_found` - `idempotency_conflict` - `protocol_version_unsupported` - `message_invalid` - `message_id_conflict` - `frame_invalid` - `frame_too_large` - `audio_format_unsupported` - `sequence_gap` - `protocol_sequence_conflict` - `resume_state_expired` - `heartbeat_timeout` - `task_cancelled` - `task_timeout` - `no_speech` - `provider_unavailable` - `provider_timeout` - `provider_authentication_failed` - `provider_rate_limited` - `provider_invalid_response` - `quota_exceeded` - `rate_limited` - `budget_exceeded` - `reservation_failed` - `usage_pipeline_unavailable` - `mcp_protocol_version_unsupported` - `mcp_client_capability_missing` - `mcp_extension_required` - `mcp_tool_not_found` - `mcp_tool_not_exposed` - `mcp_tool_input_invalid` - `mcp_tool_output_invalid` - `mcp_resource_not_found` - `mcp_resource_forbidden` - `mcp_confirmation_required` - `mcp_consent_expired` - `mcp_request_state_invalid` - `mcp_operation_handle_expired` - `mcp_audio_transport_unsupported` - `oauth_resource_invalid` - `oauth_client_not_allowed` - `delegation_required` - `delegation_expired` - `delegation_revoked` - `delegation_scope_exceeded` - `consent_required` - `consent_denied` - `consent_request_mismatch` - `skill_version_incompatible` - `skill_package_revoked` - `client_buffer_overflow` - `client_audio_capture_failed` - `client_network_unavailable` - `client_no_memory` - `resource_busy` - `application_requirements_missing` - `application_incompatible` - `capability_revision_conflict` - `capability_registry_mismatch` - `capability_evidence_invalid` - `capability_authority_unavailable` - `authority_revision_exhausted` - `app_session_expired` - `app_session_revoked` - `app_session_closed` ## MCP tools The tools of the OneSay MCP server. - `onesay.capabilities.list` — risk R0; Grants: `capability.read` - `onesay.capabilities.get` — risk R0; Grants: `capability.read` - `onesay.projects.list` — risk R1; Grants: `project.read` - `onesay.projects.get` — risk R1; Grants: `project.read` - `onesay.applications.list` — risk R1; Grants: `application.read` - `onesay.devices.list` — risk R1; Grants: `device.read` - `onesay.devices.get` — risk R1; Grants: `device.read` - `onesay.devices.diagnose` — risk R1; Grants: `device.read`, `logs.read` - `onesay.rooms.create` — risk R2; Grants: `room.create` - `onesay.rooms.get` — risk R1; Grants: `room.read` - `onesay.rooms.close` — risk R2; Grants: `room.close` - `onesay.tasks.get` — risk R1; Grants: `task.read` - `onesay.tasks.cancel` — risk R2; Grants: `task.cancel` - `onesay.logs.search` — risk R1; Grants: `logs.read` - `onesay.logs.timeline` — risk R1; Grants: `logs.read` - `onesay.usage.summary` — risk R1; Grants: `usage.read` - `onesay.quickstarts.get` — risk R0; Grants: none ## API Reference URL: /docs/reference/api # API Reference The public operations of the OneSay Control API, generated from its OpenAPI document. [Reference Catalog](catalog.md) lists the capabilities, actions, Grants, events, error codes and MCP tools. The operation explorer on this page shows the address, request and response fields, examples and errors of each operation. ## Operations Every operation with its method, path and the Grants it requires. | Operation | Method | Path | Grants | |---|---|---|---| | `getCurrentUser` | GET | `/me` | none (authentication policy) | | `createOrganization` | POST | `/organizations` | none (authentication policy) | | `listOrganizations` | GET | `/organizations` | `organization.read` | | `getOrganization` | GET | `/organizations/{organization_id}` | `organization.read` | | `updateOrganization` | PATCH | `/organizations/{organization_id}` | `organization.manage` | | `listOrganizationMemberships` | GET | `/organizations/{organization_id}/memberships` | `organization.read` | | `createOrganizationMembership` | POST | `/organizations/{organization_id}/memberships` | `organization.manage` | | `createProject` | POST | `/projects` | `project.manage` | | `listProjects` | GET | `/projects` | `project.read` | | `getProject` | GET | `/projects/{project_id}` | `project.read` | | `updateProject` | PATCH | `/projects/{project_id}` | `project.manage` | | `archiveProject` | POST | `/projects/{project_id}/archive` | `project.manage` | | `listProjectEnvironments` | GET | `/projects/{project_id}/environments` | `project.read` | | `createApplication` | POST | `/applications` | `application.manage` | | `listApplications` | GET | `/applications` | `application.read` | | `getApplication` | GET | `/applications/{application_id}` | `application.read` | | `updateApplication` | PATCH | `/applications/{application_id}` | `application.manage` | | `revokeApplication` | POST | `/applications/{application_id}/revoke` | `application.manage` | | `createCredential` | POST | `/credentials` | `credential.create` | | `listCredentials` | GET | `/credentials` | `credential.read` | | `getCredential` | GET | `/credentials/{credential_id}` | `credential.read` | | `rotateCredential` | POST | `/credentials/{credential_id}/rotate` | `credential.create` | | `revokeCredential` | POST | `/credentials/{credential_id}/revoke` | `credential.revoke` | | `createDevice` | POST | `/devices` | `device.create` | | `listDevices` | GET | `/devices` | `device.read` | | `getDevice` | GET | `/devices/{device_id}` | `device.read` | | `updateDevice` | PATCH | `/devices/{device_id}` | `device.update` | | `publishDeviceCapabilities` | PUT | `/devices/{device_id}/capabilities` | `device.capabilities.publish` | | `createDeviceProvisioningSession` | POST | `/devices/{device_id}/provisioning-sessions` | `device.update` | | `exchangeDeviceAccessToken` | POST | `/device-access-tokens` | none (authentication policy) | | `exchangeDeviceProvisioningCode` | POST | `/device-provisioning/exchange` | none (authentication policy) | | `rotateDeviceCredential` | POST | `/devices/{device_id}/credentials/rotate` | `device.update` | | `revokeDevice` | POST | `/devices/{device_id}/revoke` | `device.revoke` | | `createSource` | POST | `/sources` | `source.create` | | `listSources` | GET | `/sources` | `source.read` | | `getSource` | GET | `/sources/{source_id}` | `source.read` | | `createContextSnapshot` | POST | `/context-snapshots` | `context_snapshot.create` | | `getContextSnapshot` | GET | `/context-snapshots/{context_snapshot_id}` | `context_snapshot.read` | | `createRoom` | POST | `/rooms` | `room.create` | | `listRooms` | GET | `/rooms` | `room.read` | | `getRoom` | GET | `/rooms/{room_id}` | `room.read` | | `closeRoom` | POST | `/rooms/{room_id}/close` | `room.close` | | `listRoomParticipants` | GET | `/rooms/{room_id}/participants` | `participant.read` | | `listRoomSignals` | GET | `/rooms/{room_id}/signals` | `room.read` | | `createRealtimeConnection` | POST | `/realtime/connections` | `room.join` | | `resumeRealtimeConnection` | POST | `/realtime/connections/resume` | `room.join` | | `getRealtimeCapabilities` | GET | `/realtime/capabilities` | `capability.read` | | `createTask` | POST | `/tasks` | `task.dictation.create` | | `listTasks` | GET | `/tasks` | `task.read` | | `getTask` | GET | `/tasks/{task_id}` | `task.read` | | `cancelTask` | POST | `/tasks/{task_id}/cancel` | `task.cancel` | | `getUsageSummary` | GET | `/usage/summary` | `usage.read` | | `listUsageEvents` | GET | `/usage/events` | `usage.read` | | `listDeveloperLogs` | GET | `/logs` | `logs.read` | | `getDeveloperLogTimeline` | GET | `/logs/{request_id}` | `logs.read` | | `listCapabilities` | GET | `/capabilities` | `capability.read` | | `getCapability` | GET | `/capabilities/{capability_id}` | `capability.read` | | `listActions` | GET | `/actions` | `capability.read` | | `getAction` | GET | `/actions/{action_id}` | `capability.read` | | `getApplicationRequirements` | GET | `/applications/{application_id}/requirements` | `application.requirements.read` | | `replaceApplicationRequirements` | PUT | `/applications/{application_id}/requirements` | `application.manage` | | `evaluateApplicationCompatibility` | GET | `/applications/{application_id}/compatibility` | `application.compatibility.read` | | `createAppSession` | POST | `/app-sessions` | `app.session.create` | | `getAppSession` | GET | `/app-sessions/{app_session_id}` | `app.session.read` | | `closeAppSession` | POST | `/app-sessions/{app_session_id}/close` | `app.session.close` | | `revokeAppSession` | POST | `/app-sessions/{app_session_id}/revoke` | `app.session.revoke` | | `getAppSessionCapabilities` | GET | `/app-sessions/{app_session_id}/capabilities` | `app.session.capabilities.read` | | `createConsentRequest` | POST | `/consent-requests` | none (authentication policy) | | `getConsentRequest` | GET | `/consent-requests/{consent_request_id}` | none (authentication policy) | | `decideConsentRequest` | POST | `/consent-requests/{consent_request_id}/decision` | none (authentication policy) | | `createDelegation` | POST | `/delegations` | `delegation.create` | | `listDelegations` | GET | `/delegations` | `delegation.read` | | `getDelegation` | GET | `/delegations/{delegation_id}` | `delegation.read` | | `revokeDelegation` | POST | `/delegations/{delegation_id}/revoke` | `delegation.revoke` | | `enableDataSyncScope` | POST | `/data-sync/scopes/enable` | `data.sync.control` | | `disableDataSyncScopeRetainingData` | POST | `/data-sync/scopes/disable-retain` | `data.sync.control` | | `readDataSyncChanges` | GET | `/data-sync/changes` | `data.sync.read` | | `createDataSyncSnapshot` | POST | `/data-sync/snapshots` | `data.sync.read` | | `readDataSyncSnapshot` | GET | `/data-sync/snapshots/{snapshot_id}` | `data.sync.read` | | `completeDataSyncSnapshot` | POST | `/data-sync/snapshots/{snapshot_id}/complete` | `data.sync.read` | | `createRTCConnection` | POST | `/rtc/connections` | `room.join` | | `getRTCConnection` | GET | `/rtc/connections/{connection_id}` | `room.join` | | `revokeRTCConnection` | POST | `/rtc/connections/{connection_id}/revoke` | `room.join` | | `getRTCMediaKeys` | GET | `/rtc/connections/{connection_id}/media-keys` | `room.join` | | `createRTCPublication` | POST | `/rooms/{room_id}/rtc/publications` | `room.join`, `source.use` | | `endRTCPublication` | POST | `/rooms/{room_id}/rtc/publications/{publication_id}/end` | `room.join` | | `updateRTCSubscriptions` | PUT | `/rooms/{room_id}/rtc/subscriptions` | `room.join`, `rtc.subscription.configure` | | `createAgentExecution` | POST | `/rooms/{room_id}/agent-executions` | `room.join`, `source.use`, `signal.audio.subscribe`, `agent.execution.create` | | `getAgentExecution` | GET | `/rooms/{room_id}/agent-executions/{agent_execution_id}` | `agent.execution.read` | | `stopAgentExecution` | POST | `/rooms/{room_id}/agent-executions/{agent_execution_id}/stop` | `agent.execution.stop` | | `updateRTCPacketPolicy` | PUT | `/rooms/{room_id}/rtc/packet-policy` | `room.join`, `rtc.packet.configure` | ## JavaScript SDK Reference URL: /docs/reference/javascript-sdk # JavaScript SDK Package: `@onesay/sdk`; license: `Apache-2.0`. ## Environments Each environment has its own Control API and Realtime WebSocket endpoint. | Environment | Control API | Realtime WebSocket | |---|---|---| | Production | `https://api.onesay.com/v1` | `wss://realtime.onesay.com/v1` | | Staging | `https://api.staging.onesay.com/v1` | `wss://realtime.staging.onesay.com/v1` | ## Room events Each SDK event name and the OneSay event it reports. - `Connected` -> `room.connected` - `Disconnected` -> `room.disconnected` - `Reconnecting` -> `room.reconnecting` - `Reconnected` -> `room.reconnected` - `ParticipantJoined` -> `participant.joined` - `ParticipantLeft` -> `participant.left` - `SignalOpened` -> `signal.opened` - `SignalEnded` -> `signal.ended` - `Error` -> `error` ## Task events Each SDK event name and the OneSay event it reports. - `Created` -> `task.created` - `Started` -> `task.started` - `TranscriptPartial` -> `artifact.delta` - `TranscriptFinal` -> `artifact.finalized` - `Completed` -> `task.completed` - `Failed` -> `task.failed` - `Cancelled` -> `task.cancelled` In the browser the SDK authenticates only through the token provider you supply, never with a secret key. Task voice sends the audio you pass in, 16000 Hz mono `pcm_s16le`; the SDK does not open a microphone for it and never drops audio silently. Helpers convert sample buffers. ## Task voice - Entry points: `client.tasks.create`, `client.tasks.get`, `client.tasks.list`, `client.tasks.cancel`, `RealtimeClient` - Transport: onesay.realtime.v1 WebSocket with canonical PCM frames, contiguous ACK, unacked-only replay and bounded resume - Results: Task and Artifact readback ## Realtime RTC - Entry points: `client.rtc.connect`, `client.rtc.get`, `client.rtc.revoke`, `client.rtc.endPublication`, `client.rtc.agents.create`, `client.rtc.agents.get`, `client.rtc.agents.stop` - Transport: OneSay RTC session over the locked native SDK, loaded lazily on the first RTC operation - Results: publications, subscriptions, packets, streams, RPC, data tracks and explicit Agent executions ## RTC session states - `connected` - `reconnecting` - `closed` The session state is what this client observes. The server decides every authorization, and a revocation is complete only when the server reports it. ## Capabilities | Capability | Support | Detail | |---|---|---| | Task voice (WSS, ACK, replay, resume) | supported | Supported (browser, Node) | | RTC audio | supported | Publish, subscribe, mute, playback (browser) | | Packets | supported | Send and receive | | Streams | supported | Send and receive | | RPC | supported | Call and handle | | DataTrack | supported | Supported (`livekit-client` 2.22.3) | | Agents | supported | Explicit Agent executions and read-only Agent state | | State, device, error, release | supported | Session state, quality, microphone selection, OneSay errors, close | | Selected-mode media encryption | conditional | Conditional: where the browser can run the packaged worker | Not exposed: native room, track and participant objects; native token or connection material; native SDK types; an escape hatch to the native SDK. An unsupported capability fails with an explicit error. See the [SDK capability and compatibility page](sdk-compatibility.md). ## ESP32 SDK Reference URL: /docs/reference/esp32-sdk # ESP32 SDK - Component: `onesay-esp32` for ESP-IDF - Audio: 16000 Hz mono `pcm_s16le`, 40 ms frames - Replay buffer: 3000 ms (96000 bytes) by default; an overflow is reported as an error, never dropped silently - Credentials: read through a secure-store interface that your firmware provides - Network: TLS WebSocket through ESP-IDF - Audio input: PCM that your firmware supplies; the SDK does not capture audio The SDK is a fixed-capacity C client. It binds the connection, frames audio, handles ACKs, replays unacknowledged audio, orders events, dispatches callbacks and reconnects. Your firmware provides the audio, the credential store and the hardware drivers. ## Task voice - Entry points: `onesay_client_init`, `onesay_client_connect`, `onesay_client_start_dictation`, `onesay_client_write_audio`, `onesay_client_replay_unacked`, `onesay_client_reconnect`, `onesay_client_end_signal`, `onesay_client_dispatch`, `onesay_client_disconnect` - Transport: onesay.realtime.v1 WebSocket with canonical PCM frames, contiguous ACK, unacked-only replay and bounded resume - Results: Task events through the bounded event queue ## Realtime RTC - Entry points: `onesay_esp32_rtc_prepare`, `onesay_esp32_rtc_connect_audio`, `onesay_esp32_rtc_read_media_status`, `onesay_esp32_rtc_read_binding`, `onesay_esp32_rtc_close` - Transport: OneSay RTC binding over the pinned official ESP32 runtime behind one internal adapter - Results: RTC audio, packet send, stream receive, RPC callee and explicit Agent Control ## Capabilities | Capability | Support | Detail | |---|---|---| | Task voice (WSS, ACK, replay, resume) | supported | Supported | | RTC audio | supported | RTC audio owner | | Packets | partial | Send only | | Streams | partial | Receive only | | RPC | partial | Callee only (answers 1–4 `app.*` methods, replies to the original caller only); no caller | | DataTrack | unsupported | Not supported on ESP32 yet | | Agents | supported | Explicit Agent Control (create, get, stop) | | State, device, error, release | supported | Binding and media status, OneSay error codes, close | | Selected-mode media encryption | unsupported | Unsupported: any selected value is refused before any request | Not exposed: native room or track handles; native token or connection material; the upstream ESP32 runtime API. An unsupported capability fails with an explicit error. See the [SDK capability and compatibility page](sdk-compatibility.md). ## SDK Capability and Compatibility URL: /docs/reference/sdk-compatibility # SDK Capability and Compatibility The JavaScript and ESP32 SDKs side by side. Each SDK's support is stated on its own: one never implies the other. The server decides every operation, so a capability in a client is not a permission. ## Support matrix | Capability | JavaScript | ESP32 | |---|---|---| | Task voice (WSS, ACK, replay, resume) | Supported (browser, Node) | Supported | | RTC audio | Publish, subscribe, mute, playback (browser) | RTC audio owner | | Packets | Send and receive | Send only | | Streams | Send and receive | Receive only | | RPC | Call and handle | Callee only (answers 1–4 `app.*` methods, replies to the original caller only); no caller | | DataTrack | Supported (`livekit-client` 2.22.3) | Not supported on ESP32 yet | | Agents | Explicit Agent executions and read-only Agent state | Explicit Agent Control (create, get, stop) | | State, device, error, release | Session state, quality, microphone selection, OneSay errors, close | Binding and media status, OneSay error codes, close | | Selected-mode media encryption | Conditional: where the browser can run the packaged worker | Unsupported: any selected value is refused before any request | Support: supported; partial means one direction or one role only; conditional depends on the client environment; unsupported is refused with an explicit error and never replaced by another transport. Details per SDK: [JavaScript SDK](javascript-sdk-draft.md), [ESP32 SDK](esp32-sdk.md). ## Unsupported and not exposed - JavaScript, not fully supported: Selected-mode media encryption (conditional). - JavaScript, not exposed: native room, track and participant objects; native token or connection material; native SDK types; an escape hatch to the native SDK. - ESP32, not fully supported: Packets (partial); Streams (partial); RPC (partial); DataTrack (unsupported); Selected-mode media encryption (unsupported). - ESP32, not exposed: native room or track handles; native token or connection material; the upstream ESP32 runtime API. ## RTC lifecycle - Connection details come from the Control API and are valid for at most 60 s. The SDK never stores them and gives you a OneSay session, never the underlying room or token. - A revocation is requested as `revoking` and becomes `revoked` only when it is effective. A closed connection never reopens. - JavaScript session states: `connected`, `reconnecting`, `closed`. Reconnecting clears the receive handlers; the SDK restores a subscription only with its original version and key. - Media encryption modes: `none`, `e2ee` (default `none`). A client that cannot use the selected mode is refused with an explicit error, never downgraded. ## Task recovery - JavaScript: at most 5 reconnect attempts with bounded exponential backoff and jitter; no retry after an authentication or permission failure; a rejected resume ends as `fail`. - ESP32: at most 5 attempts with jitter and a single-use ticket per attempt; no retry after an authentication failure, a revoked credential or a rejected resume. - Both: a contiguous ACK releases audio, only unacknowledged audio is replayed, and an overflow is an explicit `client_buffer_overflow`, never a silent drop. ## Limits and privacy - Audio: 16000 Hz mono `pcm_s16le`. A JavaScript message carries at most 4096 samples; the ESP32 replays at most 3000 ms (96000 bytes) by default. - For Task voice your application supplies the audio; neither SDK opens a microphone for it. - ESP32 diagnostics never carry `wifi_ssid`, `raw_mac_address`, `raw_ip`, `raw_audio`, `transcript`, `credential`, `realtime_ticket`. - RTC connection details are sensitive and served with `no-store`; the SDK never stores them. ## Console URL: /console # Console The Console is where you manage your OneSay resources in a browser. It signs you in with Authorization Code and PKCE and keeps your session in a host-only, Secure, HttpOnly cookie; tokens never go into persistent browser storage. Signing out, an expired session or a failed sign-in clears the data on the page. The Docs and the Developer Portal hold no sign-in session. The Console calls the public Control API, the same operations your own application can use. The server decides what you can see and change from your membership and Grants, never from the IDs you select, and a denied request shows the error instead of sample data. Results keep their request ID, pagination and the `as_of` time of Usage. ## Projects and resources On first sign-in, name a project: it is created in your personal organization with its Test and Live Environments, and Live starts disabled. Then choose a Project and an enabled Project Environment to open its views: Organizations, Projects, Environments, Applications, Credentials, Devices, Rooms, Tasks, Usage summary, Usage events and Developer Logs. Credentials need Owner access. From these views you can: - create an Organization, a Project or an Application, update a resource's details, archive a Project, and revoke an Application, a Credential or a Device; - create an API key for a Service Principal (the secret is shown once) and rotate it, create a Device with a short-lived provisioning code, issue a new provisioning code, and rotate a Device credential; - read a Room, an RTC connection, an Agent execution or a Task, revoke an RTC connection, request an Agent stop and cancel a Task; - create and revoke bounded delegations, which let a Device or Service run exact operations in one Environment for up to 15 minutes; - load or prepare a confirmation (Consent request) for an Action, and allow or deny it. Each change is shown for review before it runs. Where an operation needs the Human's Consent, you allow or deny it first and then run it separately; the server checks your permissions again when it runs. ## OneSay Agent and Voice call trial Both sections run on Staging, with your signed-in identity in the selected Project and Environment. The "OneSay Agent" section (`#agent-demo`) is a guided voice conversation with the OneSay Agent: select Start, then allow each step in one place. Each Allow runs its step and prepares the next request; nothing is decided for the Human. The section shows the Agent's live listening, thinking or speaking state from the SDK, a hint that speaking interrupts the Agent, the time left and a note that the conversation has a time limit and a cost budget. The step list stays available under Details. The "Voice call trial" section (`#voice-trial`) runs the same conversation as explicit steps: Room, app session, connection, microphone, Agent (realtime, 120 s), listening, sound and the teardown. Each step that creates, changes or ends something is prepared as a Consent request when you click it and runs only after you allow it in the page; Deny leaves it undone. When "Create a new microphone Source" is chosen, Prepare of the microphone step also creates the `sensor.microphone` Source first; Source creation has no Consent Action of its own and needs Organization membership. ## Task Voice The "Task Voice" section (`#task-voice`) also runs on Staging with your signed-in identity in the selected Project and Environment. Speak in the browser and watch a OneSay dictation Task turn it into text, live: create a Room, connect, open the microphone signal, start dictation, then Stop for the final text and usage, or Cancel the dictation; leave and close the Room to finish. Each step that creates, sends or ends something is a Consent request you allow or deny in the page, as above. A dictation uses the trial's audio allowance, and the panel stops it by itself before the Task's audio limit. "Try Task Voice in the Console" in the JavaScript Quickstart explains the same flow. ## API Playground URL: /playgrounds/api # API Playground The API Playground runs three read operations of the Control API through the OneSay JavaScript SDK: `listProjects`, `listCapabilities` and `listActions`. Each run shows the operation's method and path, the Grants it requires and a sample response. The Playground answers from fixed sample data and sends nothing over the network, so it needs no credential and cannot change any resource. In your own application, the SDK sends every Control API request with the access token in a Bearer `Authorization` header and with `cache: no-store`, `credentials: omit` (no cookies), `redirect: error` and `referrerPolicy: no-referrer`. A browser gets its token from a short-lived token provider and never holds a Platform API key. ## Realtime Playground URL: /playgrounds/realtime # Realtime Playground The Realtime Playground walks through one Task voice exchange on the OneSay Realtime WebSocket (protocol `onesay.realtime.v1`), step by step: `signal.open → signal.opened → OSAF frame → audio.ack → artifact.delta → signal.end → signal.ended → artifact.finalized → task.completed`. Each step shows its direction and what it carries. Control messages are JSON. Each binary audio frame starts with a 32-byte OSAF header, followed by 40 ms of 16 kHz mono `pcm_s16le` audio by default. `audio.ack` reports the highest contiguous sequence the server has received, and the client releases buffered audio only up to that sequence. The Playground shows the messages only: it opens no WebSocket, asks for no microphone and creates no audio. MCP never carries audio. ## RTC This page covers the Task protocol and opens no RTC session. To try Realtime RTC, use the JavaScript RTC browser example in [Examples](EXAMPLES.md): with the installed OneSay SDK and a short-lived credential you already have, it connects, publishes, subscribes, sends data, calls RPC and starts Agents. RTC needs real server authorization, so the Developer Portal does not simulate it. ## Device Playground URL: /playgrounds/device # Device Playground The Device Playground calls the read-only `getDevice` operation through the OneSay JavaScript SDK and shows a sample microphone Device. The sample is fixed data, sends nothing over the network and changes nothing; the [Canonical Reference](../reference/generated/catalog.md) lists every field of a Device. A Device record never contains a credential. The device secret and the provisioning code are returned only once, by the operations that issue them. Provisioning a Device and rotating or revoking its credentials happen in the Console or through the Control API, not in this Playground. ## Examples URL: /examples # Examples The examples use the public OneSay JavaScript SDK (`@onesay/sdk`). The SDK is not published to a package registry yet. The examples do not set up a Project or Room for you: they use the resources, credentials and Consent decisions that your application already has. ## Choose the service | Service | Example | What it shows | | --- | --- | --- | | Task Voice | [JavaScript Quickstart](../guides/JAVASCRIPT_QUICKSTART.md) | Sending PCM audio that you supply, ACK and replay, Task completion, and reading back the Artifact and Usage. Your application supplies the identity, the existing resources and the Human's Consent decisions. | | RTC audio, data and RPC | The JavaScript RTC browser example, below | Connect, publish, subscribe, receive, end and close, each as an explicit button. | | RTC Agent | The same RTC example | Choose the input publications, then create, read and stop an Agent execution. A separate listener needs its own authorization and a user gesture to play the Agent's voice. | Task Voice and RTC are separate services. Connecting to RTC or publishing a microphone starts neither a Task nor an Agent. A pipeline Agent's transcription result is not the Task dictation service. The examples use public OneSay objects only; the SDK exposes no native Room, Track or token API. For a device, see the [ESP32 Quickstart](../guides/ESP32_QUICKSTART.md). ## Run the RTC example The RTC example is a plain browser page (`index.html`, `app.mjs`, `sample.mjs`, `style.css`) with a small static server, `serve.mjs`. Put these files in a directory where `@onesay/sdk` is installed and run `node serve.mjs`. The server prints `http://127.0.0.1:/`; open that address in a browser, and press Ctrl+C to stop the server. It serves only the page, its modules and its stylesheet: it never sees your credential, forwards no API request and grants no authority. The page uses the SDK's `staging` environment. You need an authorized Room, AppSession, Source and publication IDs, and a short-lived Human or AppSession token from your application's own sign-in. Paste the token into the password field and select **Use token in memory**: the page keeps the token in memory only and clears the field at once. The Control API must allow the page's origin through CORS. Never put a Platform API key, Service key or Provider key into a browser page. ## Handle the lifecycle - Keep each resource ID, idempotency key and policy version you used. If a response is lost, send the same request with the same key, or read the original resource. - An accepted request records intent: an accepted subscription, packet policy, Agent stop or revoke does not prove that it has taken effect. - A data receiver must select the publication and start its reader. In the example, `nextDataFrame` reads one frame and then closes that reader. - Cancelling a local wait does not roll back an operation the server has already committed. - Close the RTC session when you are done, and stop every Agent execution you created: closing the session does not stop an Agent. - When an outcome is uncertain, read the original resource and use your application's Consent flow. Do not change keys, chase newer policy versions or create replacement work automatically. To hear an Agent, read its execution with `client.rtc.agents.get(roomId, agentExecutionId)`. When `playback` is not null, it names the Agent's output publication; null means unknown, not silence. Share the public IDs with the listener through your application. The listener connects with its own identity and AppSession, subscribes to that publication and calls `session.audio.enablePlayback` from a user gesture. ## Report a problem Keep the operation, the returned error code and the resource IDs together in an access-controlled report. Leave out bearer tokens, connection material, private exception text and customer media. ## Changelog URL: /changelog # Changelog ## 2026-10-26 — Voice room: clearer messages - In the Console Voice room, someone leaving no longer ends the others' connections. The group is still fixed: to talk again as a different group, leave and start a new Room. - When you connect to a full Room, or the Environment's realtime limits are reached, the Voice room says so and what to do. - A Room made for other than three or four people is refused when you try to join it, with its own message. ## 2026-10-26 — Reference pages in Chinese and Japanese - The [reference catalog](../reference/generated/catalog.md), the [JavaScript SDK](../reference/generated/javascript-sdk-draft.md) and [ESP32 SDK](../reference/generated/esp32-sdk.md) references and the [SDK capability and compatibility](../reference/generated/sdk-compatibility.md) page are available in 中文 and 日本語. Descriptions that come from the API definitions are still in English. - The reference pages now describe only what you can use. An RTC feature the ESP32 SDK lacks reads "Not supported on ESP32 yet". - This changelog lists developer-visible changes only. ## 2026-10-25 — Multi-party voice Room guide - New guide: [Multi-party voice Room](../guides/MULTI_PARTY_VOICE_ROOM.md) walks three or four people through one voice Room from the browser with the public API and the JavaScript SDK: create or join the Room, connect, publish, read the roster, listen and leave, with Consent on every changing step. It states the preview limits. ## 2026-10-25 — OneSay Agent in the JavaScript Quickstart; MCP Guide and Device Diagnostics - The [JavaScript Quickstart](../guides/JAVASCRIPT_QUICKSTART.md) shows how to start a OneSay Agent in your Room with `client.rtc.agents.create`, read it with `client.rtc.agents.get` and end it with `client.rtc.agents.stop`. A call that creates, changes or ends something needs your confirmation in OneSay: until you confirm, the server answers `consent_required`; then retry with the same idempotency key. - New guides: the [MCP Guide](../guides/MCP_GUIDE.md) for AI clients and [Device Diagnostics](../guides/DEVICE_DIAGNOSTICS.md). - The MCP server has 17 tools, 4 resources and 2 prompts. R0 tools need no account, R1 tools read what your account can read, and R2 tools change something and need your confirmation in OneSay. The MCP server runs on Staging; Production is not available yet. ## 2026-10-24 — Voice call trial in the Console - The [Console](CONSOLE.md) has a Voice call trial section: talk with the OneSay Agent from your browser, one step at a time, from creating the Room to closing it. Every step that creates, changes or ends something asks for your approval first. Staging only. - JavaScript SDK: `RealtimeClient` keeps its own heartbeat replies to itself and delivers every other `pong` to your application, also when an `error` answers a heartbeat ping. ## 2026-10-23 — The Developer Portal in three languages - The Developer Portal, the Docs and the Console are available in English, 中文 and 日本語. The language switcher keeps you on the same page. ## 2026-10-22 — Endpoints on onesay.com - Control API `https://api.onesay.com/v1` (Staging `https://api.staging.onesay.com/v1`). - Realtime WebSocket for Task voice `wss://realtime.onesay.com/v1` (Staging `wss://realtime.staging.onesay.com/v1`). - MCP `https://mcp.onesay.com/mcp` (Staging `https://mcp.staging.onesay.com/mcp`). - The SDKs' environment tables use these addresses. ## 2026-10-20 — RTC connections and the SDK capability matrix - `POST /v1/rtc/connections` on a Room that is still being prepared answers `202` with `retry_after_ms`. Ask again with a new `Idempotency-Key` after that time; the answer is `201` once the Room is ready. The JavaScript SDK's `rtc.connect` waits and retries for you. - The [SDK capability and compatibility](../reference/generated/sdk-compatibility.md) page shows what each SDK supports. - ESP32: `onesay_esp32_rtc_prepare` refuses any selected `media_encryption` with `ONESAY_ERR_INVALID_ARG` before any request; leave the field unset. ## 2026-10-17 — One Room for Task voice and RTC - A Room created with `POST /v1/rooms` serves Task voice and RTC at the same time. `POST /v1/rooms/{room_id}/close` ends both: connections close and SDK sessions report `closed`. A closed Room cannot be joined or reopened. - A realtime connection can request the `context_snapshot.read` Grant, so a Task created on it can carry a ContextSnapshot. A dictation Task whose ContextSnapshot cannot be used is refused at creation with `task_profile_input_unsupported`. - ESP32 RPC: the SDK answers RPC calls and replies to the original caller only; it cannot make RPC calls yet. ## 2026-10-15 — End-to-end encrypted RTC audio in JavaScript - `rtc.connect({ mediaEncryption: "e2ee" })` encrypts audio frames end to end with per-participant keys that the server issues and rotates every 30 seconds. A browser that cannot run it fails before joining and never joins unencrypted. `session.capabilities.e2ee` is true only in this mode. OneSay issues the keys, so OneSay can decrypt the audio. - New error code `resource_busy`: the JavaScript SDK raises it from `publishMicrophone` when another capture holds the microphone. On ESP32, your capture and playback callbacks can return `ONESAY_ERR_RESOURCE_BUSY`, and `onesay_esp32_rtc_read_media_status` reports it. ## 2026-10-14 — Console resource views - The Console shows resource details, request diagnostics with a request timeline, Usage by dimension, and Agent details with a confirmed stop. - It creates resources and updates their metadata, creates and rotates API keys (a rotation revokes the previous key at once), creates Devices and their provisioning codes, and rotates Device credentials, each after you check the change. A returned secret stays masked until you reveal it and is cleared when you change scope or leave the page. ## Community URL: /community # Community The OneSay developer community is not open yet: there are no discussion forums or community channels.