Speech recognition
Deepgram, Sarvam, Gnani, AssemblyAI, Gladia, Gemini and OpenAI speech recognition — each model, the caller languages it recognises, how it decides the caller has finished, and every setting it takes, with its range and default.
An engine and its model are chosen in Voice Settings → Speech recognition engine and Recognition model, in each language's editor in Languages, and in the flow builder's Voice & languages. Each model shows only the caller languages it recognises and the settings it uses: a model that can't recognise the agent's language is shown but can't be chosen, and a setting the model doesn't use gives way to a line saying so. A setting left empty uses the model's default.
The same rule applies to every save, through the API too: a save that pairs a model with a language it doesn't recognise is refused with a 422 that names the language and what the model recognises, such as Deepgram Nova 2 doesn't recognise Tamil. Nova 3 does, and so does Sarvam. An agent already saved that way keeps saving its other settings; changing its model or language must land on a pair the model recognises.
AssemblyAI, Gladia, Gemini and OpenAI are offered where your platform can run them (see Voices and recognition).
Turn-taking
Pause Before Replying (how long the caller must be silent before the agent replies) is used by Deepgram's Nova models. Every other model decides for itself when the caller has finished, so the slider gives way to the model's own setting, or to a line saying the engine decides. Recognition Pace applies to every model.
Words to expect
Keyword boosting, the words the agent should never mishear:
| Model | Takes |
|---|---|
| Deepgram Nova 2 | Words, each with an optional weight after a colon, such as Wishlink:5 |
| Deepgram Nova 3 | Plain words, for English and Multi-language |
| Deepgram Flux | Plain words |
| Sarvam Saaras v4 | Plain words, at most 50 of up to 64 characters |
| AssemblyAI, Gladia | Plain words (AssemblyAI at most 100) |
| Sarvam's other models, Gnani, Gemini, OpenAI | None: the box steps aside, and the words already saved stay for another model |
For a model that takes plain words, a weight such as :5 is removed when you
save.
Hinglish mode
For Hindi callers who mix in English. Turn on Hinglish mode and the model listens for both, so English words stay in English in the transcript instead of being written out in Devanagari: "मेरा order अभी तक deliver नहीं हुआ", not "मेरा ऑर्डर अभी तक डिलीवर नहीं हुआ". The agent's model reads the words the caller actually used.
The switch appears in Voice Settings, and in a flow agent's Voice tab, when the language is Hindi and the model reads it:
| Model | What it does |
|---|---|
| Sarvam Saaras v3, Saaras v4 | Transcribes in code-mix mode |
| Deepgram Flux (Multilingual), listening for Hindi alone | Listens for Hindi and English |
AssemblyAI universal-3-6-pro | Listens for Hindi and English |
It is off until you turn it on. Through the API it is hinglish_mode under configuration.runtime.speech_recognition.
Other models and languages ignore it.
Deepgram
| Model | Recognises | Pause Before Replying |
|---|---|---|
| Nova 3 (default) | Hindi, English, Multi-language (Hindi with English), Bengali, Gujarati, Kannada, Marathi, Punjabi, Tamil, Telugu and Urdu | Yes |
| Flux (English) | English | No: its own turn-taking |
| Flux (Multilingual) | Hindi, English and Multi-language, listening for any of ten languages | No: its own turn-taking |
| Nova 2 | Hindi and English | Yes |
No Deepgram model recognises Odia: for an Odia language, choose Sarvam or Gnani. Changing the agent's language keeps its model where that model recognises the new language; otherwise it moves to one that does (Flux (English) to Flux (Multilingual), else Nova 3).
Flux
Flux listens for the end of a sentence rather than for a fixed silence, so the agent answers sooner without talking over a caller who pauses mid-thought. Choosing a Flux model shows its Turn-taking: three named choices, and the exact settings under More settings. An agent moved to Flux starts on Balanced.
| Choice | Certainty to end a turn | Start replying early | Longest pause |
|---|---|---|---|
| Quick | 0.7 | At 0.5 | 700 ms |
| Balanced | 0.7 | Off | 1 s |
| Patient | 0.8 | Off | 2 s |
| Setting (More settings) | What it does | Range | Default |
|---|---|---|---|
| Certainty to end a turn | How sure Flux must be that the caller has finished before the agent replies. Higher waits until they are clearly done; lower answers sooner and may cut in. | 0.5 to 0.9 | 0.7 |
| Longest pause | After this much silence the turn ends, however unsure Flux is. | 500 ms to 3 s | 1 s |
| Start replying early | The agent starts preparing its reply while the caller is finishing and drops it if they carry on: quicker answers, more AI model use. Its certainty is no higher than Certainty to end a turn; 0.4 to 0.5 gives the quickest replies. | Off, or 0.3 to 0.9 | Off (0.5 when switched on) |
| Languages it listens for (Flux Multilingual) | The languages your callers mix, from Hindi, English, Spanish, French, German, Portuguese, Italian, Dutch, Russian and Japanese. | Any of the ten | The agent's own language; with Multi-language, Flux works it out |
Settings saved another way show as Custom, with their values.
Sarvam
| Model | Recognises | Turn-taking | Words it should expect |
|---|---|---|---|
| Saaras v4 | 22 Indian languages, English (Indian and global accents) and Indic Multi (auto-detect) | Its own settings, below | Yes |
| Saaras v3 realtime | The same languages | Its own settings, below | Yes |
| Saaras v3 (default) | The same languages | Sarvam decides | Yes |
| Saarika v2.5 (legacy) | Hindi, English, Bengali, Gujarati, Kannada, Malayalam, Marathi, Odia, Punjabi, Tamil and Telugu | Sarvam decides | No |
Words it should expect is a sentence about the call, such as the brand and product names the agent says a lot. Sarvam reads it as a sentence, not a list.
The realtime models (Saaras v4 and Saaras v3 realtime) take their own turn-taking:
| Setting | What it does | Range | Default |
|---|---|---|---|
| Partial transcripts | Fast suits a conversation; Balanced wobbles less if you show live captions; Off sends one result per turn. | Fast, Balanced, Off | Fast |
| End of turn | Silence before the agent answers. | 100 to 5,000 ms | 500 ms |
| Mic sensitivity | Raise it on a noisy line. | 0 to 1 | 0.3 |
| Shortest turn | Sound shorter than this is not a turn. | 0 to 2,000 ms | 250 ms |
Gnani
Vachana v3 recognises Hindi, English, Bengali, Gujarati, Kannada, Malayalam, Marathi, Punjabi, Tamil and Telugu. Gnani decides when the caller has finished and takes no words to expect.
AssemblyAI
Streaming recognition that decides when the caller has finished, with words and a description of the call to expect.
| Model | Recognises |
|---|---|
| Universal 3.6 Pro (default) | English, Hindi, Marathi, Urdu, Arabic, Chinese, Dutch, French, German, Italian, Japanese, Korean, Portuguese and Spanish |
| Universal 3.5 Pro | The same, without Marathi |
| Universal Streaming English | English only |
| Universal Streaming Multilingual | English, Spanish, French, German, Italian and Portuguese, detecting which |
| Setting | What it does | Range | Default |
|---|---|---|---|
| Mode | Lowest latency answers soonest; Most accurate waits a little longer to get words right. | Lowest latency, Balanced, Most accurate | AssemblyAI's own |
| Shortest pause before the turn ends | The silence after which AssemblyAI may end the caller's turn, once it is confident they have finished. | 50 to 10,000 ms | AssemblyAI's own |
| Longest pause before the turn ends | The silence after which the turn always ends, even when AssemblyAI isn't sure the caller has finished. It can't be shorter than the shortest pause. | 50 to 10,000 ms | AssemblyAI's own |
| Interruption delay | How long the caller must speak over the agent before it counts as an interruption. | 0 to 1,000 ms | AssemblyAI's own |
| What the call is about | A sentence or two about the call, so AssemblyAI expects its words. | Text, at most 1,750 characters | None |
| Speech threshold (More settings) | How sure AssemblyAI must be that a sound is speech. Raise it on noisy lines. | 0 to 1 | AssemblyAI's own |
| Voice focus (More settings) | Near listens for a voice close to the phone, as when it is held to the ear; Far for one further away, as on a speakerphone. | Near, Far | None |
| Voice focus strength (More settings) | How strictly the voice focus leaves out other voices. Choose a voice focus first. | 0 to 1 | AssemblyAI's own |
Mode, Interruption delay, What the call is about and the voice focus settings apply to the Universal 3 models; the streaming models don't take them.
Gladia
Recognition in about a hundred languages, that can follow a caller mixing in English. Among this dashboard's languages, Gladia recognises all but Odia.
| Model | Recognises |
|---|---|
| Solaria 1 | About 100 languages |
| Setting | What it does | Range | Default |
|---|---|---|---|
| Pause before the turn ends | How long the caller must be silent before their turn ends. | 10 to 10,000 ms | 500 ms |
| Longest turn | A turn ends after this many seconds even if the caller hasn't paused. | 5 to 60 seconds | 5 seconds |
| Speech threshold | How sure Gladia must be that a sound is speech. Raise it on noisy lines. | 0 to 1 | 0.6 |
| Follow a switch to English | Recognises English mixed into the call's language, as in Hinglish. | On or off | Off |
Gemini
Live recognition for English and nine Indian languages: Hindi, Bengali, Gujarati, Kannada, Malayalam, Marathi, Odia, Punjabi and Telugu. Gemini doesn't recognise Tamil: for a Tamil language, choose another engine.
| Model | Notes |
|---|---|
| Gemini 3.5 Transcribe Live | Live recognition |
| Setting | What it does | Range | Default |
|---|---|---|---|
| Pause before the turn ends | How long the caller must be silent before their turn ends. 500 to 800 ms suits most calls. | 100 to 2,000 ms | 500 ms |
| Transcript | Verbatim keeps every word, fillers and self-corrections included; Smart leaves them out. | Verbatim, Smart | Verbatim |
| Speech start padding (More settings) | Audio kept from just before the caller starts speaking, so a first word isn't clipped. | 0 to 1,000 ms | Gemini's own |
| Wait for a finished turn (More settings) | How long to wait for Gemini to finish a turn before taking what it has heard. | 1 to 30 seconds | 8 seconds |
OpenAI
Realtime recognition in Whisper's languages, with noise reduction. Among this dashboard's languages, OpenAI recognises all but Odia.
| Model | Notes |
|---|---|
| GPT Realtime Whisper (default) | The default |
| GPT Live Transcribe | Live recognition |
| GPT Transcribe | Recognition |
| Setting | What it does | Range | Default |
|---|---|---|---|
| Pause before the turn ends | How long the caller must be silent before their turn ends. | 100 to 2,000 ms | 400 ms |
| Noise reduction | Filters background noise out before recognising, for a phone held close. | On or off | Off |
| Speech loudness threshold (More settings) | How loud audio must be to count as the caller speaking. Raise it if background noise interrupts the agent. | 0 to 32,767 | 400 |
| Speech threshold (More settings) | How sure the detector must be that a sound is speech. | 0 to 1 | 0.5 |
| Speech start padding (More settings) | Audio kept from just before the caller starts speaking, so a first word isn't clipped. | 0 to 1,000 ms | 300 ms |
Switching engines
Moving to another engine keeps the words to expect and starts the new engine at its own defaults: one engine's settings are never carried to another.