Voice

The agent is a conversation. Text is the fallback.

The whole integration

<GraineProvider
  publishableKey="pk_live_…"
  navigationRef={navigationRef}
  autoConnect={false}
  loadFont={Font.loadAsync}   // expo-font: the SDK loads the design's typeface
  svg={{ Svg, Path }}         // react-native-svg: icons in the design's style
>
  {/* active: the customer opened the agent. autoStart then places the call. */}
  <GraineVoiceLauncher webView={WebView} active={open} autoStart onError={(m) => console.warn("[graine]", m)}>
    {(api) => (open ? <GraineVoiceBar api={api} onClose={() => setOpen(false)} /> : null)}
  </GraineVoiceLauncher>
  {!open && <GraineLauncherButton onPress={() => setOpen(true)} style={{ position: "absolute", right: 16, bottom: 24 }} />}
</GraineProvider>

react-native-webview autolinks. No MainApplication.kt edit, no AppDelegate.swift edit, no ProGuard rule, no babel plugin ordering.

GraineVoiceBar is the bar the dashboard's Garage previews — the design's surface, corners and shadow, the avatar, the name and status in the design's typeface with a meter that moves at the design's pace, the microphone, the hang-up, the caption and the chat — so what the brand approves there is what the customer sees. See Your design.

Why a WebView

React Native has no getUserMedia, so every route to a microphone is a native module — and a native module means an app-store release for every fix. The audio path is where nearly every fix lands: decoding, jitter buffering, echo suppression, the mute latch. Through a WebView those ship over the air.

It is one pixel, invisible, unreachable — laid out at 1×1 rather than hidden, because a view with no box can have its media suspended by the platform, which would kill the call the moment your bar collapses.

Driving it from your own UI

Your own UI covers every platform — React Native, Android, iOS, Flutter and a website — with each state to draw. In React Native, only if the SDK's bar will not do: from here on you own matching the brand's design yourself — useGraineTheme() gives you its tokens.

<GraineVoiceLauncher webView={WebView}>
  {({ connected, connecting, muted, start, end, setMuted }) => (
    <MyBar
      status={connecting ? "connecting" : connected ? "live" : "idle"}
      onCall={() => (connected ? end() : start())}
      onMute={() => setMuted(!muted)}
      muted={muted}
    />
  )}
</GraineVoiceLauncher>

While the call is coming up, GraineVoiceBar draws your loader — the mark and words from Appearance → Loading — in place of the status line, and again while the agent reconnects and while it works out its answer ("Thinking…"). A bar of your own gets the same from api.phase and <GraineLoader moment="connecting" />.

A turn arrives many times as its words come in — each fuller, with live: true — then once settled with live: false. The launcher folds those into one entry, so captions holds turns rather than keystrokes. The transcript is cleared when a call starts and when you set active={false}; copy it in onEnded if you need it afterwards.

While it connects

A connect on a good network takes under a second warm and about two cold. On mobile data the commonest slow connect is a line the phone's network dropped while the app sat idle — after sleep, or a switch between 5G and 4G — and the engine handles it without you: the line is checked as the customer taps (beside the microphone, so a healthy one costs nothing), a dead one is replaced at once, a call nobody answers is placed once more on a fresh connection — up to twice more when the network lost it on the way, and never while the device has no network — and only then does the call move to the WebSocket. A call answered with no audio path is reconnected inside the call first (at once when nothing at all is getting through) and moves only if that fails: after 8 seconds on a good network, longer on a slow one, never later than 30 seconds.

The launcher says where it is, and GraineVoiceBar draws it:

To see how connects go on your customers' networks, count them from the same signals:

useEffect(() => { if (api.slowConnect) analytics.track("agent_connect_slow", { stage: api.connectStage }); }, [api.slowConnect]);
useEffect(() => { if (api.connectStage === "retrying") analytics.track("agent_connect_retried"); }, [api.connectStage]);
useEffect(() => { if (api.callFailed) analytics.track("agent_connect_failed"); }, [api.callFailed]);

Hanging up

Two paths, both idempotent, so a host using one, the other, or both ends exactly one call: api.end(), and active={false} when your UI closes. Deactivating also clears connected / connecting and the transcript, so a reopened launcher never shows a call that has ended.

When the page cannot load

The launcher watches the download, not a clock. A page still arriving is left alone however slow the link; a page that has gone silent is reloaded on a bounded, jittered ladder, and the bar is told where it is:

{(api) => api.connectivity.state === "failed"
  ? <Button title="Try again" onPress={api.retry} />
  : <MyBar … />}

connectivity.state is loading · ready · retrying · failed. A page carrying a live call is never reloaded from under it, and closing always ends the call even if the page never answered. The knobs, if a field app on 2G needs them: timing={{ stallMs, bootMs, hardCapMs, retryDelaysMs }} — silence before a load is abandoned, a downloaded page's grace to say ready, the ceiling on one attempt (120 s), and the ladder.

Microphone permission

The microphone is the one permission voice needs. Your app declares it; when a call starts without it, the prompt comes from the SDK, the system or your own code, depending on the platform:

<!-- Android: AndroidManifest.xml (React Native and Flutter) -->
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.MODIFY_AUDIO_SETTINGS" />

<!-- iOS: Info.plist (every platform) -->
<key>NSMicrophoneUsageDescription</key>
<string>Used to talk to the in-app assistant.</string>

"When a call starts" includes autoStart. A launcher that is active from the moment it mounts, with autoStart, starts the call as soon as the engine is ready — and on Android the prompt appears then, before anyone has tapped. To choose the moment yourself, ask during onboarding with the permission library your app already uses: the SDK then finds the microphone granted and asks nothing. Or mount the launcher with active={false} and make it active when the customer opens it. Every permission each feature needs, with the words to ask with, is on Permissions.

A refused microphone is reported through onMicDenied (and micDenied on the launcher's api) and never falls back to another transport. The SDK does not ask again by itself: the next call asks again only where the system still allows it — iOS shows its dialog once, and after that the customer turns the microphone on in Settings. Draw the refusal, and offer the way to Settings.

Interruption

Handled for you. When the customer talks over the agent it stops, and whatever it had queued is discarded rather than played out behind them.

The SDK also keeps the agent from hearing itself: on a phone the speaker is two inches from the microphone, so a large part of what it hears while talking is itself — passed on unfiltered, those words are transcribed as the customer's and the agent interrupts itself after a word or two. The guard measures the echo on the device rather than assuming a level, and holds about 240 ms back so a real interruption is not clipped.

How the audio travels

One integration, two transports. Omit transport and the agent's dashboard setting decides (Embed & Widgets → Call audio), so this can change for every app running your build without a release.

Screen context, actions and components work identically on both.

On a weak network

The SDK owns the network, and your app draws what it reports. Nothing here needs code in your app.

Quality, in one word. While a call is up the engine reads the peer connection's own statistics — round-trip time, jitter, and the packets lost over the last few seconds — every two seconds, and the SDK reduces them to excellent · good · poor · lost with the thresholds VoIP has used for twenty years (150 ms / 30 ms / 2 % is the edge of good; 400 ms / 80 ms / 8 % is poor). Audio that stops arriving altogether reads poor at once, and the word recovers as soon as the call does. It changes only when the word changes.

const { quality, online } = useGraineConnection();
// or, on the launcher's api: api.quality, api.network, api.reconnecting

GraineVoiceBar already says "Weak connection" and "No connection". If you draw your own bar, say them too: a customer whose agent goes quiet on a train reads "Listening" and concludes the agent is broken.

A dead media path is a failure, not a call. WebRTC can complete its signalling and never find a path for the audio — the call answers, and no sound flows either way. The engine watches the connection. When its audio path fails or drops — a lift, a tunnel, Wi-Fi to mobile data — the engine reconnects it inside the same call: the conversation carries on and nothing is said twice, reconnecting is true meanwhile (the voice bar shows your reconnecting words), and a moment with no network at all is waited out. Only a path that cannot come back — about 15 seconds with a network, longer on a slow one — hangs that leg up and carries the conversation on the WebSocket, reported through onTransport with reason: "media".

Reconnect, not restart. If the device goes offline mid-call, the SDK holds the call as reconnecting (the transcript stays, phase says so) for up to 30 seconds. Network back inside that window: the call is placed again on the same session and the agent picks up. Not back: graine:ended, as an honest end.

It is all on the record. Which transport carried the call, a fallback and why, every quality change and every network loss are written to the call as quiet rows under What happened in the app in Execution Logs — so "the agent went quiet" reads as "the network went offline at 0:42". The agent is never told about them and never speaks about them.

Mobile networks

Behind a mobile carrier's network, or on Wi-Fi that drops UDP, a direct audio path is often impossible — and the failure is not a refused call: the call answers and no audio flows. WebRTC calls therefore include a relay (TURN) that carries the audio when a direct path cannot be found. Graine provides it; there is nothing to configure. On a network that blocks UDP (some offices and mobile networks) the engine's next try uses only the relays it reaches over TCP and TLS, port 443 included. If a network blocks even those, the call moves to the standard path (reason: "media", below).

When rtc cannot be used

It falls back to ws rather than failing, and tells you:

onTransport={({ transport, requested, reason, message }) => {
  if (transport !== requested) console.warn(`[graine] ${reason}: ${message}`);
}}

Diagnose not_configured without rebuilding anything — this sends no Origin, exactly as a native app does, so a 403 here is the 403 your app is getting:

curl -i -X POST https://www.graine.ai/api/embed/rtc-session \
  -H 'Content-Type: application/json' -d '{"publishableKey":"pk_live_YOUR_KEY"}'

Bringing your own audio

Only if you already ship a native audio module and have a reason to prefer it. It is more work and it cannot be updated over the air.

const adapter = liveAudioStreamAdapter(LiveAudioStream, myPcmPlayer);
const { listening, start, stop } = useGraineVoice(adapter);
// and the provider needs `voice`, since its own socket then carries the audio:
// <GraineProvider … voice>

The boundary is four methods: startCapture(onChunk), stopCapture(), play(pcm16, sampleRate), clear() — plus optional bufferedSeconds(), which lets the agent know when a reply has actually been heard, not just received.

Do not resample in JavaScript — that is how a mid-range Android starts dropping frames and the agent hears every third word. Read MIN_PLAYOUT_BUFFER_SECONDS before writing the player: a cushion enforced as a hard gate strands the end of every reply.

Next steps