The UX of streaming AI responses
Latency you cannot remove can still be designed around. Interface patterns that make a slow model feel responsive and an uncertain one feel honest.
Model latency is largely outside your control. Perceived latency is almost entirely a design problem, and the gap between the two is where a good AI interface is won.
Stream, but stream something meaningful
Token-by-token output is table stakes. What matters more is what appears first. Leading with the retrieved sources, or a one-line restatement of what the system understood you to ask, gives the reader something to evaluate while the answer assembles.
What should appear, in order
- 1
What it understood
A one-line restatement gives the reader something to evaluate immediately.
- 2
The sources
Retrieved citations arrive before the prose and set expectations.
- 3
The plan, on long tasks
Three visible steps make a wait feel like progress.
- 4
The answer, streaming
By now the reader has context for what they are reading.
Show the work when the work takes time
For multi-step tasks, a visible plan — searching, reading, drafting — converts dead time into progress. It also sets expectations: a user who can see three steps remaining waits far more patiently than one watching a spinner.
Make uncertainty visible and useful
Confidence is not a percentage badge. It is inline citations you can click, a clear statement when the sources disagree, and a plain refusal when there is nothing to go on. Users forgive a system that says it does not know; they do not forgive one that sounds certain and is wrong.
Designing the wait
Do
- Show sources or a restatement first, so there is something to read
- Show a visible plan on multi-step tasks
- Cite inline, and say plainly when sources disagree
- Make stop actually stop, and keep what arrived
Don't
- A bare spinner for a ten-second generation
- Confidence percentages nobody can interpret
- Forcing a fresh thread to correct one word
- Hiding that a cheaper fallback model answered
Design the interruption
Long generations need a stop button that actually stops, and a way to redirect without starting over. Being trapped watching output you already know is wrong is the fastest way to lose trust in an assistant.
- Stop that halts generation and keeps what has arrived
- Edit-and-resend on the previous message rather than a fresh thread
- Regenerate with a hint — shorter, more formal, cite more
- A visible token or time budget on genuinely long tasks