ESP32 Quickstart

# ESP32 Quickstart

Choose [Task Voice](#esp32-task-voice) for a Task-only application or
[Realtime RTC](#esp32-realtime-rtc) for the current RTC component integration.
OneSay OS is not required. The SDK is a local component candidate; an SDK build
is not registry publication or real-device qualification. All server permissions
remain authoritative even if the client binary or local capability flags change.

## ESP32 + Task Voice

Reuse `sdks/esp32/examples/voice_button`, the standalone Task-only ESP-IDF
application. Export it with the existing OneSay component into a new directory:

```text
node tools/sdk/export-esp32-oem-example.mjs <new-output-project>
```

In that exported project, use the pinned ESP-IDF 6.0.2 toolchain:

```text
idf.py set-target esp32s3
idf.py build
```

The exporter records source hashes and supplies `components/onesay`; the source
directory is an export template, not an independently configured build. Preserve
its exact dependency pins and checksum checks. Missing components may require
dependency downloads. These commands compile an application; they do not flash
a board or provision credentials.

Follow `sdks/esp32/examples/voice_button/README.md` and the example-local
`main/voice_button.h`. Supply `onesay_voice_button_configure` with secure Device
credential storage, existing authorized Project/Environment/Application and
Source/Room bindings, network access, trusted UTC/certificates and a bounded
16 kHz mono PCM s16le producer. Its default hook returns `ONESAY_ERR_AUTH` before
connecting. Capture, GPIO, I2S, codec and board tuning belong to the OEM.

The caller serializes `onesay_voice_button_init`, `onesay_voice_button_connect`
and bounded `onesay_voice_button_step` calls. The existing SDK owns WSS/OSAF,
sequence/ACK, backpressure, replay/resume and cancellation. Keep retained PCM
until ACK release; a successful cancel send is not a terminal Task result.
`onesay_voice_button_close` must release the transport before its storage and
dependencies can be freed. This path does not require a LiveKit RTC session.

Existing host, target-build and native Task fixtures retain their distinct
source-bound results. This page does not execute the example. Real-device smoke,
real Provider, Staging and Production remain **NOT-RUN** here; the compile-only
instructions are not a claim that the default application can connect.

## ESP32 + Realtime RTC

The existing component in `sdks/esp32` exposes `onesay_esp32_rtc.h` when
`CONFIG_ONESAY_RTC_ENABLED` is selected. It privately uses the pinned official
LiveKit ESP32 component. Use ESP-IDF 6.0.2 and the exact dependency graph in
`sdks/esp32/idf_component.yml`; its current LiveKit commit is
`fbf09edf27a504d82119f730a35bf00fa287f7c1`.
The component's `sdkconfig.rtc.defaults` is the existing RTC configuration
fragment, loaded before application defaults; it is not a board configuration.
Preserve the component's configuration and dependency checks rather than
substituting a registry version or enabling RTC in the Task template and
assuming that it becomes an RTC application.

### Runnable OEM project: `rtc_smoke`

The board-agnostic OEM project
[`sdks/esp32/examples/rtc_smoke`](../../sdks/esp32/examples/rtc_smoke/README.md)
is the runnable RTC starting point. It is an independent ESP-IDF application
that consumes only the exported OneSay component and the reused `voice_button`
Task flow. It has no GPIO, I2S, codec, BSP, AEC, Wi-Fi, NVS or `livekit_*`
call, and it does not depend on OneSay OS. One run streams a Task, then
connects RTC with synthetic capture, sends one packet, releases everything
and prints one content-free summary line. The two commands below do different
things. The first exports the project into a new directory, which you then
build with your own ESP-IDF 6.0.2 installation (`idf.py set-target esp32s3`,
then `idf.py build`). The second is the repository's verification build: it
exports, resolves and compiles the same project from this repository inside
the pinned ESP-IDF 6.0.2 environment and checks the dependency lock and linked
symbols. It is not a build of your exported directory.

```text
node tools/sdk/export-esp32-oem-example.mjs --example rtc_smoke <new-output-project>
node tools/sdk/build-esp32-idf-component.mjs --rtc-smoke
```

Provide a strong `onesay_rtc_smoke_configure`. Its weak default returns
`ONESAY_ERR_AUTH` before any network activity. Before it returns `ONESAY_OK`,
your firmware brings up the network, sets trusted UTC and fills the
configuration from your secure store. The project's README lists every field.
Board settings (flash size, PSRAM, console) go in a third defaults file.

### Device credential to RTC bearer

The RTC owner takes a caller bearer and never exchanges a Device secret. To go
from the Device credential to RTC with public calls only, set
`rtc_bearer_from_device_credential = true` in the smoke configuration. The
chain is:

1. **Owner authority, before the device runs.** The owner creates the
   Delegation and decides the Consent requests. The Device creates its
   AppSession under that Delegation with `createAppSession`. The device
   receives the Room, AppSession, Source and delegation IDs. IDs grant
   nothing; the server checks the actual Grant, Delegation and Consent on
   every request.
2. **Device secret to short-lived token.** The `rtc_bearer` step loads the
   Device secret once through your existing secure-store callback
   (`task.credentials.load`). It sends the secret only as the `Authorization`
   of one `exchangeDeviceAccessToken` request (`POST /v1/device-access-tokens`,
   body `{}`) through the public `onesay_esp32_control_request`. The RTC Control
   configuration must name the same environment, Device, Project and
   Environment as the Task one, or the step fails before the secret is loaded.
   The secret is erased whatever happens. Only the `access_token` of the canonical
   `{"data":{"access_token","token_type":"Bearer","expires_at"}}` envelope is
   kept. The store's `erase`, a factory reset, is never called.
3. **Token to RTC.** `onesay_esp32_rtc_prepare`,
   `onesay_esp32_rtc_connect_audio` and the packet policy receive only that
   short-lived Device access token, on the same Control owner. The smoke
   erases its copy after release.

Your idempotency-key callback returns a fresh key for
`exchangeDeviceAccessToken` in each run, because a replayed key returns the
original expiry. It returns the original Consent-bound key for each RTC
operation. To use a bearer from another established flow instead, supply the
`load_rtc_bearer` callback. Exactly one of the two sources is accepted. Never
put a long-lived Device secret, Platform key, Provider key or LiveKit secret
in firmware sources, logs or documentation.

The host recorder checks this chain without a network. The ESP32-S3 target
build checks that the project links only public OneSay calls. Running the
project on a board is the required real-device smoke:
NOT-RUN — requires an authorized reference target. Its operator procedure and
evidence format are in [GX09 SDK smoke](../development/GX09_SDK_SMOKE.md).

### Prepare the application inputs

Initialize the existing `onesay_esp32_control_context_t` using
`onesay_esp32_control_init` and your `onesay_esp32_control_config_t`. Supply the
expected Device/Project/Environment/Application bindings, certificate or verified
certificate bundle, bounded request timeout and `next_idempotency_key`
callback. The shared Control initializer requires a valid `device_id` even when
the separate RTC bearer is a Human; that configured ID does not change the
bearer's identity or authorize RTC. Set `environment` to the existing SDK enum
for the environment you are authorized to use; the host check below does not
provision that environment or add a loopback override. Network setup, trusted
UTC and certificate storage remain application responsibilities. Do not disable
TLS hostname or date validation.

The audio sequence below needs these caller-owned inputs:

| Input | Existing public contract |
| --- | --- |
| `control` | Initialized Control owner; retain its certificate and key-callback dependencies while using it. |
| `bearer` / `bearer_length` | Current short-lived Human or delegated Device/AppSession bearer, for example the Device access token from the [Device credential chain](#device-credential-to-rtc-bearer). RTC prepare does not exchange a Device secret or create an AppSession. Do not embed Platform, Provider or LiveKit long-term secrets. |
| `input` | `onesay_action_rtc_connection_create_input_t` containing the existing `room_id` and `app_session_id`, with terminated strings. Resources must already exist and be authorized. |
| `connection_delegation` | The selected connection operation's `delegation_id`, or `NULL` when omitted. It is request metadata, not permission. |
| `audio` | `onesay_rtc_audio_config_t`: authorized `source_id`, that publication operation's separate `delegation_id` (empty omits it), capture/render callbacks and `now_unix_ms` with its context. |
| `owner` | A caller-retained pointer initialized to `NULL`; never pass a slot already owning a session to prepare. |

The key callback must return the original key for each logical operation:
`createRTCConnection` and `createRTCPublication` are separate operations.
When Consent/Delegation applies, each key must match its own confirmed action.
The connection's selector is not inherited by publication. Server checks remain
authoritative; a valid local selector or callback cannot grant access.

### Prepare, connect, observe and release

This is application glue using the current public header, not another SDK
export or a complete board application. It requires the inputs above. The C
fragment was syntax/type-checked against the current public headers only;
target linking, board compilation and execution were not performed for it.

```c
#include "onesay_esp32_rtc.h"

onesay_err_t example_begin_rtc_audio(
    onesay_esp32_control_context_t *control,
    const uint8_t *bearer, size_t bearer_length,
    const onesay_action_rtc_connection_create_input_t *input,
    const char *connection_delegation,
    const onesay_rtc_audio_config_t *audio,
    onesay_esp32_rtc_t **owner) {
    if (owner == NULL || *owner != NULL) return ONESAY_ERR_INVALID_ARG;
    onesay_err_t result = onesay_esp32_rtc_prepare(
        control, bearer, bearer_length, input, connection_delegation, owner);
    if (result != ONESAY_OK) return result;
    /* Keep *owner even on failure: the caller still owns cleanup. */
    return onesay_esp32_rtc_connect_audio(
        *owner, control, bearer, bearer_length, audio);
}
```

Serialize these public calls on the application thread, outside all media,
native and clock callbacks. Keep the pointer and callback contexts alive after
the function returns. Failed prepare leaves the initially empty pointer `NULL`,
but an ambiguous HTTP outcome still needs server reconciliation. After successful
prepare, even failed connect leaves an owned pointer requiring close.
`ONESAY_OK` from connect means that native connect was
accepted; it does not mean connected, published, heard or effectively revoked.

| Next call, where `owner` is the retained session pointer | Meaning and required handling |
| --- | --- |
| `onesay_esp32_rtc_read_binding(owner, &connection)` with `onesay_rtc_binding_t connection = {0}` | Copies original OneSay IDs, generation, deadline and observed state. Keep PREPARED/CONNECTING/CONNECTED/RECONNECTING/FAILED/CLOSED distinct; none replaces server authority. |
| `onesay_esp32_rtc_read_audio_binding(owner, &publication)` with `onesay_rtc_audio_binding_t publication = {0}` | Copies content-free publication IDs when available, including after a native failure. Check the return value: unavailable publication returns `ONESAY_ERR_INVALID_ARG` and cleared output, not a fabricated ID. |
| `onesay_esp32_rtc_close(&owner)` when `owner != NULL` | Cancels OEM I/O, joins native ownership and releases media owners. Only `ONESAY_OK` clears the pointer. On failure retain the pointer and all callback contexts for an explicit later cleanup attempt; do not free or reinitialize them. |

Read and retain any IDs needed for canonical reconciliation before successful
close. The `CLOSED` observation alone does not prove cleanup: a failed close
can leave an owned pointer. Local close does not execute `endRTCPublication`
or `revokeRTCConnection`; server cleanup uses those existing public operations
with their own current authorization and idempotency/Consent bindings.

Capture must supply exactly the requested 16 kHz mono s16le samples, at most
640 per read. Render must accept the actual decoded s16le format: 8/12/16/24/48
kHz, mono or stereo, at most 120 ms per callback (5,760 frames at 48 kHz).
All callbacks borrow their buffers
until return and have the existing 40 ms budget. The idempotent `cancel`
callback may run concurrently and must wake I/O without freeing its context.
Callbacks must not invoke SDK lifecycle functions. GPIO, codec, resampling and
board tuning are not supplied by this application glue; the pinned media
pipeline retains its existing encoding and format-conversion responsibilities.

On denied authorization, inspect `onesay_esp32_control_last_error(control)` for
the canonical code, request ID and retryability; do not dump tokens, connection
material or transport bodies. Expired first-use material cannot be rescued by
a longer business deadline or a changed client clock. A failed or ambiguous
publication/connect attempt must not be retried by calling connect again or
minting a replacement key. Preserve the original IDs/key, release local owners
and reconcile through the application's explicit authorization flow.

### Existing local checks and qualification boundaries

From the repository root inside the existing Linux development environment:

```text
corepack pnpm esp32:rtc:audio-owner
```

This existing bounded runner requires GCC, pthread and ASan/UBSan support.
It writes source-bound results under `.build/rtc-audio-owner-proof/` and uses
Control/native/PCM recorders. Its existing cases cover call order, separate
delegation selectors, material expiry, failed connect and retained cleanup.
It does not open a real RTC connection, exercise a microphone, or qualify an
ESP32 board. Do not invoke it as a Windows native executable.

For the current interface details and the original independent host/target
evidence, see the [SDK audio-owner documentation](../../sdks/esp32/README.md#onesay-rtc-audio-owner)
and [source-bound audio acceptance](../development/M1_ESP32_SDK_ACCEPTANCE.md#esp32-public-audio-owner-and-exact-component-graph--2026-10-09).
Those results keep their original source bindings; they were not rerun for
this guide. The component's host checks and target compilation are distinct.

Scope of this RTC path: the runnable project is the board-agnostic
`rtc_smoke` OEM project above, with synthetic capture and a counting render
sink. It is not a board application, and it does not supply microphone,
speaker, AEC or audio-quality tuning; those belong to your board. The
Task-only Voice Button export is not an RTC demonstration. Host recorders and
the ESP32-S3 target build are local evidence. They do not establish native
network/media on a device, Device media quality or the real-device smoke, which
remain **NOT-RUN** here, as do real Provider, Staging and Production.
Unsupported functionality must remain explicit; Task and JavaScript support
must not be inferred to cover ESP32 RTC.

## Historical physical harness context

Status: `PRIVATE PHYSICAL HARNESS CANDIDATE / OPERATOR-GATED`

The earlier physical harness guide planned supported ESP-IDF and board versions,
per-device provisioning, NVS encryption, I2S mapping, Test-environment
endpoints, build/flash, bounded memory, reconnect and serial-log safety.

That private C11 host proof validates canonical endpoint constants, OSAF
framing, bounded replay, ACK/reconnect/resume rules and a safe fixed-capacity
diagnostic snapshot. Required CI compiles the component for ESP32-S3 from
pinned ESP-IDF source. Real provisioning exchange, NVS/I2S/network execution,
physical heap/TLS measurement, flashed-device serial inspection and physical-
device steps remain `NOT EXECUTED`.
Never compile a Project API key or device credential into firmware.

## Real-device smoke

No hardware command is offered here. The earlier inert
`physical_acceptance` program and its `esp32:physical:run` wrapper were retired
by the repository boundary correction, and the repository boundary check keeps
them removed. The required minimal real-device SDK smoke is the external
hardware action `EXT-HARDWARE-SMOKE`: it needs written board-specific
connect/flash/test authorization before the smoke firmware is flashed to that
board. Do not call `idf.py flash` on an unauthorized board.