Speech and recognition API: /tts and /ws/stt
For developers, in code: POST /tts for one sentence of speech, WS /ws/stt for streaming recognition with language identification, and GET /stt/languages — frames, limits and error codes.
Both halves of voice run on the server. The widget synthesizes an answer one sentence at a time and streams the microphone to a recognition socket; a custom front end does the same.
POST /tts
POST /tts
X-Widget-Key: wk_…
Content-Type: application/json
{ "code": "default", "text": "Yes — on Saturdays until two.", "lang": "en" }| Field | Meaning |
|---|---|
code |
the widget — it selects the voice map and the gate |
text |
one sentence; clipped at 1,200 characters |
lang |
the language of the text; picks the voice: the owner’s override for that language, else the language’s default voice, else a provider default that speaks it |
Returns audio/mpeg with Cache-Control: private, max-age=86400. Identical sentences are
replay-cached server-side by provider, voice and text, so a repeated greeting costs nothing.
| Status | When |
|---|---|
400 |
conversation mode is off on the widget — spoken answers exist only inside a voice conversation |
404 |
unknown or switched-off widget code |
422 |
no voice can speak that language, or nothing to speak after cleaning |
429 |
over 120 requests a minute for the account or 60 for this visitor; Speech limit reached for today past this visitor’s 30,000 characters a day, past what the account’s answers earned today, or before a plan past the account’s 20,000 — a replay from the cache is never refused |
502 |
the speech provider failed |
Speed is the owner’s per-widget setting (slow · normal · fast · faster), applied
server-side per provider; the client sends no rate.
WS /ws/stt
One socket per utterance stream. The first frame authenticates and describes the visitor; then binary audio; then an optional stop.
Client → server
| Frame | Content |
|---|---|
| auth (text, first) | { "widget_key", "code", "lang", "lang_manual", "browser_langs" } — lang the page language (biases identification), lang_manual: true when the visitor pinned a language in the chip (skips detection), browser_langs the browser’s navigator.languages |
| audio (binary) | PCM, 16-bit little-endian, mono, at the sample_rate the server announced |
{ "type": "stop" } |
end the utterance; the server flushes and finalizes |
Server → client
| Frame | Content |
|---|---|
ready |
{ sample_rate } — send audio at this rate (16,000 Hz) |
partial |
{ text, language } — an interim transcript |
final |
{ text, language } — a finished utterance; may be empty when the server blanked low-confidence noise, and the client should reopen the mic |
error |
{ code, message } |
code |
When |
|---|---|
bad_request |
the first frame is not a JSON object (closes 1008) |
unauthorized · forbidden |
the key or the origin (closes 1008) |
not_found |
unknown or switched-off widget code |
voice_disabled |
neither dictation nor conversation mode is on for the widget |
rate_limited |
over 60 streams a minute for the account, over 20 a minute or 300 a day for this visitor, over what the account’s answers earned today, or before a plan over the account’s 100 a day |
stt_unavailable |
the provider could not be opened |
stream_too_long |
over 120 s on one stream — the stream is closed, and what was heard is finalized |
stt_error |
a provider error mid-stream |
Language identification runs per utterance over a small candidate list the server builds — the page language, the browser’s languages, the site’s languages, capped at five — so a visitor may speak a language the site does not declare. A pinned language skips it.
GET /stt/languages
No authentication. { "languages": ["uk", "en", …], "autonyms": { "uk": "Українська", … } } —
every language the recognizer supports that the assistant can also speak, with native names, cached a day. The widget’s language
chip is built from it.
What a custom client must do
Capture with echo cancellation on, send 16 kHz mono PCM, treat an empty final as «reopen», and
on a phone gate the microphone while a clip plays — the loudspeaker trips the voice detector and
the assistant’s own voice would come back as a question. The widget’s src/audio.js and
src/voice.js are the reference implementation.