An LLM feature is not a frontend feature with an API call in it. It is a network boundary, a secret, an unbounded cost and an untrusted output — and Vue makes it very easy to put all four in the wrong place.
The demo is deceptively easy. Install the provider SDK, call it from a component, bind the response to a ref, and a chat panel is working in twenty minutes. That version is also unshippable, and usually for reasons that only become visible in production.
I have reviewed enough of these to know the failure modes are consistent: a key in the bundle, provider calls scattered across components, a streaming implementation that re-renders the component tree on every token, and model output written straight into application state without validation. Each one is cheap to avoid up front and expensive to unpick later.
What follows is the structure I use for Vue 3 and Nuxt apps. Four boundaries, each one enforcing a single rule, with the code for the parts people most often get wrong — stream parsing and reactivity under streaming.
Boundary 1 — the key never reaches the browser
This is not a best practice, it is the whole game. Any value that reaches the client is public: bundling is not obfuscation, and a key in a Vue app is one devtools Network tab away from being someone else's inference budget. Vite makes the mistake particularly easy, because anything prefixed VITE_ is inlined into the built JavaScript by design.
The fix is a server route that holds the key and forwards the request. In Nuxt that is a file under server/api; in a plain Vue SPA it is a small Hono, Fastify or Express service, or a serverless function. The browser talks only to you.
Which means the route is now a paid endpoint you own, and it needs the things paid endpoints need: authentication, a per-user rate limit, and a cap on request size. An unauthenticated proxy to a provider account is not an AI feature, it is a public API on your credit card — and it will be found. Put the limit on the server, because a limit enforced in the component is a suggestion.
- Anything VITE_-prefixed is public: Vite inlines it at build time. If a secret has ever been in a VITE_ variable, treat it as leaked and rotate it — it is in every build artefact and probably in your CDN cache.
- Rate-limit per user, not per IP alone: One authenticated user looping a request is the cheapest way to burn a month of budget in an afternoon. Cap per minute and per day, and return 429 rather than silently queueing.
- Keep prompts on the server: A system prompt in client code is readable and, worse, replaceable. Anything that constrains the model belongs behind the boundary.
- Cap max_tokens on every call: It is the only hard stop between a bad prompt and an unbounded bill. Set it per endpoint, not globally.
import { z } from 'zod'
// Validate the request before it costs anything.
const ChatRequest = z.object({
messages: z
.array(
z.object({
role: z.enum(['user', 'assistant']),
content: z.string().min(1).max(4000),
}),
)
.min(1)
.max(20),
})
export default defineEventHandler(async (event) => {
const user = await requireUser(event) // your auth, not the model's
await enforceRateLimit(user.id, { perMinute: 10, perDay: 200 })
const { messages } = ChatRequest.parse(await readBody(event))
const upstream = await fetch('https://api.provider.com/v1/chat', {
method: 'POST',
headers: {
// Server-only env var. No VITE_ prefix, never sent to the client.
authorization: 'Bearer ' + process.env.LLM_API_KEY,
'content-type': 'application/json',
},
body: JSON.stringify({
model: 'small-fast-model',
stream: true,
max_tokens: 800, // a hard ceiling per call
// The system prompt lives here, not in the browser, where it would
// be both readable and editable by anyone using your app.
messages: [{ role: 'system', content: SYSTEM_PROMPT }, ...messages],
}),
signal: AbortSignal.timeout(30_000),
})
if (!upstream.ok || !upstream.body) {
throw createError({ statusCode: 502, statusMessage: 'Upstream failed' })
}
setResponseHeaders(event, {
'content-type': 'text/event-stream',
'cache-control': 'no-cache, no-transform',
connection: 'keep-alive',
})
return sendStream(event, upstream.body)
})Boundary 2 — no component ever knows which provider you use
The second failure is architectural rather than dangerous: provider calls spread through components until swapping models means touching fifteen files. Components should know about messages, loading and errors. They should not know about roles, token limits, SSE frames or which vendor is answering.
One composable owns that. It exposes state and two functions — send and stop — and nothing else. Whether it is talking to one provider, two behind a router, or a local model in development is invisible to everything above it.
This pays off faster than people expect. Provider pricing, latency and availability all change; the teams that can move between them in an afternoon are the ones that drew this line on day one. It also makes testing possible: a fake implementation of the same interface replays a recorded stream, so your component tests never make a network call.
// composables/useChat.ts — the public surface, provider-agnostic
export interface ChatMessage {
id: string
role: 'user' | 'assistant'
content: string
status: 'streaming' | 'complete' | 'error'
}
export interface UseChat {
messages: Readonly<ShallowRef<ChatMessage[]>>
isStreaming: Readonly<Ref<boolean>>
error: Readonly<Ref<Error | null>>
send(text: string): Promise<void>
stop(): void
}
// A component uses it like any other composable, and would not change
// if you switched provider, added a router, or moved to a local model:
//
// const { messages, isStreaming, send, stop } = useChat()Boundary 3 — parse the stream properly, then batch it
Two bugs live here, and almost every hand-rolled integration has both. The first is assuming a network chunk equals a message. It does not: server-sent events arrive split across chunk boundaries, so a JSON payload routinely turns up as two halves in two reads. You need a buffer, and you split on the blank line that terminates an event — not on every newline, and never on each chunk.
The second is reactivity. A fast model emits tokens at a rate that, wired naively to a ref, triggers a render pass per token. If the component renders markdown, each of those passes re-parses the entire response so far, and the cost grows with the length of the answer. The symptom is a chat panel that is smooth for two sentences and janky by the fourth paragraph, which people then misdiagnose as the model being slow.
Fix it by buffering into a plain, non-reactive string and flushing once per animation frame. You keep the streaming effect — the human eye cannot tell 60 updates a second from 300 — and the render count drops by an order of magnitude. Use shallowRef for the message array so Vue does not deep-track every message object, and defer markdown parsing until the stream completes, rendering plain text with preserved whitespace while it runs.
- Buffer across reads, split on the blank line: Splitting per chunk or per newline works in development and fails under real network conditions, where frames arrive fragmented. The bug looks like occasional missing words.
- Do not use axios for streaming: It buffers the whole response in the browser. Use fetch with res.body, which gives you a ReadableStream you can read incrementally.
- Render markdown once, not per token: Parsing the full response on every token is quadratic in answer length. Stream plain text with preserved whitespace, parse when the stream completes, or throttle parsing to a few times a second.
- onScopeDispose, always: A user navigating away mid-answer must abort the request and cancel the pending frame, or you keep a dead component's closure alive and keep paying for tokens nobody will read.
Measure this rather than trusting it. Open the Vue devtools performance panel, send a long prompt, and count component updates — the difference between per-token and per-frame is usually a factor of five or more.
import { ref, shallowRef, onScopeDispose, triggerRef } from 'vue'
export function useChat(): UseChat {
// shallowRef: Vue tracks the array reference, not every message object.
const messages = shallowRef<ChatMessage[]>([])
const isStreaming = ref(false)
const error = ref<Error | null>(null)
let controller: AbortController | null = null
let frame: number | null = null
let pending = '' // plain string: deliberately not reactive
function flush() {
frame = null
if (!pending) return
const last = messages.value[messages.value.length - 1]
if (!last || last.role !== 'assistant') return
last.content += pending
pending = ''
triggerRef(messages) // one render, however many tokens arrived
}
function schedule() {
if (frame === null) frame = requestAnimationFrame(flush)
}
async function send(text: string) {
stop()
error.value = null
isStreaming.value = true
controller = new AbortController()
messages.value = [
...messages.value,
{ id: crypto.randomUUID(), role: 'user', content: text, status: 'complete' },
{ id: crypto.randomUUID(), role: 'assistant', content: '', status: 'streaming' },
]
try {
const res = await fetch('/api/chat', {
method: 'POST',
headers: { 'content-type': 'application/json' },
body: JSON.stringify({ messages: toWire(messages.value) }),
signal: controller.signal,
})
if (!res.ok || !res.body) throw new Error('Request failed: ' + res.status)
const reader = res.body.pipeThrough(new TextDecoderStream()).getReader()
let buffer = ''
while (true) {
const { value, done } = await reader.read()
if (done) break
// Events are separated by a blank line, and a single read may
// contain half an event, several events, or both.
buffer += value
const events = buffer.split('\n\n')
buffer = events.pop() ?? '' // keep the incomplete tail
for (const event of events) {
const line = event.split('\n').find((l) => l.startsWith('data:'))
if (!line) continue
const payload = line.slice(5).trim()
if (payload === '[DONE]') continue
try {
pending += extractDelta(JSON.parse(payload))
schedule()
} catch {
// A malformed frame is not a reason to kill the stream.
}
}
}
flush()
markLastComplete(messages)
} catch (e) {
if ((e as Error).name === 'AbortError') return // user pressed stop
error.value = e as Error
markLastError(messages)
} finally {
isStreaming.value = false
controller = null
}
}
function stop() {
controller?.abort()
controller = null
if (frame !== null) cancelAnimationFrame(frame)
frame = null
pending = ''
}
// Leaving the page mid-stream must cancel the request and the frame.
onScopeDispose(stop)
return { messages, isStreaming, error, send, stop }
}Boundary 4 — model output is input, so validate it
The moment a feature does more than display prose — extracting fields, choosing a category, filling a form, driving a UI state — the model's output becomes untrusted input to your application. Treating it as anything else is how a hallucinated enum value ends up in a database column with a foreign key constraint.
Ask for structured output through the provider's JSON or schema mode, and then validate it anyway on the server, against the same schema your application uses. Providers get this right most of the time, which is precisely the problem: a failure mode that appears in one call in five hundred will not show up in testing and will show up on a customer's screen.
Decide the fallback before you ship. One retry with the validation error fed back usually fixes a malformed response; after that, degrade to a state your UI can actually render. A feature that renders a helpful empty state when the model misbehaves is finished. A feature that throws is not.
const Extraction = z.object({
urgency: z.enum(['low', 'medium', 'high']),
category: z.string().max(60),
summary: z.string().max(400),
// Never let the model invent a value your schema will reject downstream.
assignee: z.string().uuid().nullable(),
})
export type Extraction = z.infer<typeof Extraction>
export async function extract(input: string): Promise<Extraction | null> {
for (let attempt = 0; attempt < 2; attempt++) {
const raw = await callModel(input, { jsonSchema: toJsonSchema(Extraction) })
const parsed = Extraction.safeParse(raw)
if (parsed.success) return parsed.data
// One retry, with the validation error as feedback. Then stop:
// a third attempt rarely succeeds and always costs.
input = input + '\n\nPrevious reply was invalid: ' + parsed.error.message
}
logInvalidExtraction(input)
return null // the UI renders an empty state, not an error
}| Concern | Component | Composable | Server route |
|---|---|---|---|
| API key and prompts | Never | Never | Here |
| Auth, rate limits, quotas | No | No | Here |
| Provider choice and model selection | No | Hidden behind it | Here |
| Stream parsing and buffering | No | Here | Pass-through |
| Cancellation and cleanup | Triggers it | Owns it | Honours the disconnect |
| Schema validation of output | No | Type only | Here, authoritative |
| Rendering, scroll, empty and error states | Here | No | No |
The operational details that decide whether it survives
Four smaller things separate an AI feature that runs quietly from one that generates support tickets. None is difficult; all are easy to leave until after launch, which is exactly when they become expensive.
Log token counts and latency per request from the server route, keyed by user and feature. Without it, “the bill went up” is unanswerable; with it, you know which feature and which cohort, usually in one query. Build a fake provider for development that replays a recorded stream with realistic pauses — it makes the UI work testable, keeps CI free, and lets you reproduce the slow-stream case deliberately.
Handle the degenerate cases explicitly in the UI: an empty response, a refusal, a timeout, and a stream that stops mid-sentence because the upstream connection dropped. And keep transcripts out of a global store unless the product genuinely needs them there — an ever-growing array of conversations in Pinia is a memory leak with a product justification attached.
- Log tokens and latency per call: Prompt tokens, completion tokens, model, duration, user, feature. Six fields make cost attributable; without them you are guessing at the invoice.
- Ship a fake provider: A fixture stream replayed with delays gives you deterministic tests, offline development and a free way to reproduce slow or truncated responses.
- Design the failure states: Timeout, refusal, empty answer, truncated stream. All four will happen; decide now what the panel shows rather than letting an exception decide.
- Do not accumulate transcripts in a global store: Keep the active conversation in the composable's scope and persist the rest server-side. A store that only ever grows is the classic long-session leak.
Frequently asked questions
Can I call an LLM API directly from a Vue component?
Only with a key you are happy to publish, which in practice means never. Anything in the client bundle is readable, and Vite inlines VITE_-prefixed variables at build time by design. Put a server route in front — a Nuxt server/api handler, or a small Hono, Fastify or Express service for a plain SPA — and give it authentication and a per-user rate limit, because it is now a paid endpoint you own.
How do I stream LLM responses in Vue 3?
Use fetch and read res.body as a stream; axios buffers the entire response and cannot stream in the browser. Decode with TextDecoderStream, accumulate into a buffer, and split on the blank line that terminates each server-sent event — chunks do not align with event boundaries. Then batch updates into the reactive state rather than writing every token directly.
Why does my Vue chat UI get slow as the answer gets longer?
Almost always per-token rendering, and usually per-token markdown parsing on top of it. Each token triggers a render, and each render re-parses the whole response so far, so cost grows with answer length. Buffer tokens in a plain string, flush once per animation frame, use shallowRef with triggerRef for the message list, and parse markdown only when the stream finishes.
Should LLM state live in Pinia or in a composable?
The active stream belongs in a composable, because it owns a cancellable request whose lifetime should match the component scope. Put in Pinia only what genuinely needs to be shared across routes — a conversation id, a summary, a feature flag. An ever-growing array of full transcripts in a global store is a memory leak that survives every navigation.
How do I stop users from running up my inference bill?
Four controls, all server-side: authentication on the route, a per-user rate limit by minute and by day, a hard max_tokens on every call, and a request-size cap so nobody pastes a novel into the prompt. Add per-user daily spend tracking if the feature is generous, and return 429 rather than queueing — a queue just delays the cost.
How do I test a Vue component that streams from an LLM?
Do not call the model. Implement the same composable interface with a fake that replays a recorded stream from a fixture, with small delays so the streaming path is genuinely exercised. Your component tests then cover the cases that actually break — cancellation mid-stream, a truncated response, an empty answer — deterministically and for free.