Graine AI

Showing, not just saying

Point at an element on screen, and draw components instead of sentences — UI highlighting and generative UI.

An agent that can only talk is doing half the job. Two things let it use the screen the customer is already looking at.

Pointing at something

"Tap Continue" is useless if the customer cannot find Continue.

import { useGraineHighlight } from "@graineai/inapp-react-native";
 
function ContinueButton() {
  const { highlighted } = useGraineHighlight("continue_button");
  return (
    <Pressable style={[styles.btn, highlighted && styles.glow]}>
      <Text>Continue</Text>
    </Pressable>
  );
}

That is the whole integration. Registering the name tells the agent the element exists and can be pointed at; the boolean tells you when it is being pointed at.

The SDK does not draw the highlight. It does not own your design language, and a ring that looks right in one app looks broken in the next. You get a boolean and style it the way the rest of your app would — a border, a pulse, a scroll-into-view.

Highlights expire after six seconds. One still lit after the conversation moved on is worse than none: the customer keeps looking at the wrong thing.

Scoped to the component. Unmounting deregisters it, so the agent is never offered a target on a screen the customer has left.

The one dashboard step

Declare highlight_element on the agent, under Integrations & Tools, alongside your other actions:

{
  "name": "highlight_element",
  "description": "Draw the customer's attention to something on their screen. Use it when you tell them to tap or look at a specific control.",
  "parameters": {
    "type": "object",
    "properties": { "name": { "type": "string", "description": "The element to point at." } },
    "required": ["name"]
  }
}

You do not implement it. The SDK answers it: it checks the name against what is actually registered on the current screen and refuses with the real list when the agent invents one — so a wrong guess produces a useful sentence rather than a dead action.

Turning one on

Four things have to be true before a component can appear, and one of them fails silently if your agent is older.

YOU, IN THE DASHBOARDAT CALL TIME1 · Build itlayout and content,and a tool name2 · Point at your APIoptional — leave itblank to stay static3 · Savepushes it intoagent config4 · On screenwhich agentsmay show itThe agent calls itby its tool name,when it decides toYour API is calledfrom our servers,merged over your contentControl channelkind + data, verbatimIt drawsembed and app,from one definition

An agent with no web-call variant is SKIPPED

open that agent, save it once, and try again

Saving is what pushes a component into an agent’s config — a live agent reads a snapshot of its tools, not your database. Components are deliberately never written to a phone agent, so an embed with no web-call variant is skipped and the dashboard tells you which.

Load the content from your API

Leave it blank and the component is static. Fill it in and it is called the moment the agent reaches for the component, and the JSON object it returns is merged over what you authored — so the layout stays yours and the items, prices, images or times are live.

{ "items": [
  { "id": "creta-sx", "title": "Creta SX(O)", "price": "₹19.2L",
    "imageUrl": "https://cdn.yours.com/creta.jpg",
    "facts": ["6 airbags", "Panoramic sunroof"] }
] }
Called fromOur servers, not the customer's device — so a key in a header is not shipped to the app
Must behttps, reachable from the internet. Private addresses and the cloud metadata endpoint are refused
Timeout8 seconds, and only on a channel that can draw — a phone call never pays it
If it failsThe component still renders what you authored. A slightly emptier card beats an agent that went quiet

Ask your API for what the customer said

A component was called the same fixed way whatever the customer said, so it could show a catalogue but never the right part of one. Write a placeholder into the URL, a query value or the body, and it becomes a parameter the model fills from the conversation:

https://api.yoursite.com/cars?max_price={budget}&type={body_type}

"I want a new SUV under twenty lakh" then reaches your API as max_price=2000000&type=SUV. The editor lists every parameter it found under the URL, so you can see what the model will be asked for.

Where they workThe URL, any query value, and the body at any depth
Where they do notHeaders, on purpose — a placeholder there would let the model choose an authorization value
Required?Never. A customer who has not named a budget still sees the default list, rather than the model stalling or inventing one
Left blankA query parameter with nothing to fill is dropped, not sent empty — ?max_price= asks something different from not asking
SafetyA value cannot re-point the request: it is encoded on substitution, and the finished URL is checked again before the call goes out

Is it actually on the agent?

There are two copies of a component and only one is callable: the library in the dashboard, and the snapshot inside the agent's own config that the runtime reads. They used to look identical from outside. Embed & Widgets → On screen now reads the snapshot back and says one of three things:

  • The agent can call — it is on the agent; mention it in the prompt.
  • Nothing — press Save selection to push. Until then the agent cannot show a component however it is asked, and will answer in words or reach for something else.
  • No web-call variant — components are never written to a phone agent. Open the agent, save it once to create one, and come back.

A component is a tool the model calls, so it needs a tool name and a reason to call it. Refer to it in the prompt the way you would any other tool — by what it does, not with any special syntax. An agent that was never told it exists will answer in words and reach for whatever else it has.

Drawing components

Sometimes a card, a set of choices or a three-field form serves the customer better than another sentence. The agent decides that itself and sends one.

import { useGraineWidget } from "@graineai/inapp-react-native";
 
const { widget, submit, dismiss } = useGraineWidget();
 
if (widget?.kind === "choices") {
  return (
    <Choices
      options={widget.data.options}
      onPick={(o) => submit({ chosen: o.id }, `I'll go with ${o.label}.`)}
    />
  );
}

A product list, with pictures

Reading three cars and their prices out loud loses the customer by the second one. product_list is the same three as rows they can see and tap, and it is the kind most worth drawing in your app.

if (widget?.kind === "product_list") {
  const { title, prompt, items, selectLabel } = widget.data as any;
  return items.map((it) => (
    <Row key={it.id}
      image={it.imageUrl} title={it.title} subtitle={it.subtitle}
      facts={it.facts}        /* up to three short lines */
      price={it.price}        /* a string: "₹19.2L", "From ₹499/mo" */
      disabled={it.unavailable}
      buttonLabel={selectLabel ?? "Choose"}
      onPress={() => submit({ id: it.id, title: it.title, price: it.price },
                            `I'll take a look at ${it.title}.`)} />
  ));
}

The contents are authored in Embed & Widgets, not in your app, and may be filled at call time from your own API — so a catalogue that changes weekly needs no release. The same component reaches a website embed and a native app: one definition, both surfaces.

A new product list starts already filled, with sample rows and sample pictures, so you can see the shape before you replace it.

Pictures

imageUrl takes three forms, and all three work on both surfaces:

FormExampleNotes
Absolutehttps://cdn.yours.com/creta.jpgWhat you will use in production
Root-relative/samples/car-blue.pngResolved against the agent's own origin before it reaches a device
Data URIdata:image/png;base64,…Fine for something small; it travels with every send

The middle one is the one that used to break. A path with no host is an ordinary image on a web page and nothing at all on a phone — React Native's Image renders an empty box and says nothing, which looks exactly like an author who forgot to set a picture. The SDK resolves it against the agent's origin on the way in, so what you author in the dashboard is what the app draws.

Keep them small and roughly 4:3. A row draws the picture at about 64pt, so a 2000px photograph costs the customer their data and buys nothing.

Embed & Widgetsbuilt once, forthe whole workspaceOn screenwhich ones THISagent may showControl channelnot the audio legWebsite embeddrawn for youNative appsame fields, your componentstoolkind + dataverbatimverbatim

a tap returns as a sentence for the conversation and data for the webhook

The kind travels untranslated, so a component you add tomorrow reaches a browser and a phone without either being taught about it. Without the returning sentence the customer taps a card, sees a tick, and the agent carries on as though nothing happened.

One library, chosen per agent

Components belong to the workspace, not to an agent: a lead form is the same lead form whichever agent shows it, and a copy per agent is how five slightly different versions of "Share your details" end up in production.

Which of them an agent may SHOW is per agent, under Agent → Embed & Widgets → On screen. Every component you give an agent becomes a tool it can call, so a car catalogue on a loan agent is not just clutter — it is a tool a model will eventually reach for, and tokens on every turn until it does.

Pick nothing and the agent gets every enabled component, including ones added later. That is the default and it is right for a workspace with one agent. Tick even one and the agent sees only what is ticked. Untick them all and the agent draws nothing and answers in words, which is a real choice and not the same as having picked nothing at all.

The selection is applied in both places it has to be: what the SDK is told, and what the runtime registers as tools. Saving stores it; Sync to agent config is what hands it to the runtime.

The kinds the dashboard ships

lead_form, plan_comparison, info_card, calendar_slots, quick_actions, product_list, blank. The SDK relays kind verbatim, so an agent may also send one of your own — handle what you draw and fall back for the rest.

submit takes two things, and both matter

datathe structured answer your backend wants
summarya sentence for the conversation

Send only the first and the customer taps a card, sees a tick, and the agent carries on as though nothing happened. The sentence is what moves the conversation.

Handle the kinds you know, fall back for the rest

widget.fallbackText is what the agent would have said instead. Render it for any kind you do not draw — a component you cannot show should still move the conversation forward, and a renderer that guesses produces something worse than words.

Dismissing

dismiss() is local only. A customer closing a card has not answered the question, so the agent is not told; it will ask again in words, which is right.

Both work on either transport. On rtc they travel the control channel beside screen context and actions — for the same reason: they must arrive whole and in order.

Next steps