Product Design · Self-initiated
Kouto
An agentic voice agent for your Mac. You hold a key and talk, and it answers out loud, runs your automations, sits in on meetings and makes you a podcast. No cloud, no API keys, no account.
01 · The brief
The problem was presence, not features
Most voice assistants are chat boxes with a microphone bolted on. The interface is still a window full of text and the voice is a feature inside it. I wanted to find out what is left when you take the window away.
That leaves a hard constraint. A voice agent has no screen to point at: no list to scroll, no button to press, no cursor to follow. Everything the user knows about what is happening has to arrive through sound and one small object on screen, and in its smallest form there is nowhere to put text at all.
So I wrote the constraint down first and designed against it: the main screen has exactly one control.
02 · The interface
Seven states, and none of them is a spinner
The sphere is the entire interface. It is a cloud of particles, and the state of the conversation changes both its colour and the way it moves. Every state below is the real sphere, captured from the app’s own renderer rather than recreated.
Idle
at rest
Nothing is happening. It follows the level only when sound is playing.
Listening
ripple
The microphone is open. This is the only state that matters when the sphere is alone in a corner, because there is no text anywhere.
Transcribing
pulse
A short wait on words that already exist.
Thinking
pulse
The model is writing. It squeezes in a slow rhythm rather than spinning, so waiting reads as work rather than as a hang.
Speaking
flow
The reply is playing, so the live level is the truth again.
Working
ring
A tool is running, so every particle is pulled onto a turning circle. A mechanism, not a mood: it has gone away and cannot hear you.
Something failed
a shake, not a shape
A flash, not a phase. It takes the danger colour for about 1.4 seconds and returns to whatever was happening.
Why the motion matters
Each state has its own motion vocabulary rather than its own spinner. Listening ripples with your voice, thinking pulses, speaking scatters and flows, and working pulls every particle onto a turning circle because a tool call is a mechanism rather than a mood. The whole palette is a set of CSS custom properties, one block per state, and the renderer reads them without knowing what a theme is. That one decision is what made every state work in light and dark mode without a second code path.
03 · Latency is a design problem
Fifty-six milliseconds, and why it is a design decision
A naive voice assistant waits for the whole reply, synthesises all of it, then plays it. You sit in silence while it composes an essay. That silence is the product experience, and it is terrible. Nothing about the shape of the reply changed here. What changed is when you hear the first word.
56 ms
to the first word of audio, measured from the transcript being in hand
~0.5 s
to transcribe a short utterance, which happens before that clock starts
04 · What it does
The rest, and why it is a list rather than a pitch
These are ordinary features on purpose. The interesting part of this product is not that it can do them, it is that a voice agent doing them needs different answers than an app with a screen.
A memory that writes itself
It keeps its own notes and reviews the conversation to decide what is worth keeping. Notes about the machine are capped at 2,200 characters and your profile at 1,375, and the files are readable and editable by hand.
Automations
Say "brief me at 8:30 on weekdays" and it sets one up. A schedule is either a line it says word for word, with no model involved, or a job it goes and does first.
Meetings
It notices a call starting and records both sides, your microphone and the system audio, then transcribes it with each speaker marked.
Reading aloud
Hand it a book and it reads chapter by chapter, resuming where you left off. Or select text anywhere on the Mac and have it read that.
Podcasts
Give it a topic and some links and two voices write and perform an episode from what the pages actually say.
A canvas
Ask for a diagram or a chart and it draws one on a visual page instead of reading a table out loud. The particle cloud opens to make room for it.
05 · The system
One machine, and nothing leaves it
Speech recognition, the model and the voice all run locally. That is not a privacy claim added at the end: an assistant that holds your calendar, your meetings and your notes is a different proposition when none of it leaves the machine, and it is the reason this product can exist at all.
The case study
Everything above is the product. What follows is how it was decided, including the decisions that were wrong first.
01 · One control
The main screen has exactly one button
The microphone is the only thing that starts or stops listening. Nothing auto-starts, not on launch and not on a stray keystroke. You press it, it stays armed across turns so a conversation flows, and you press it again to stop.
The default is hands-free: the microphone stays open and you interrupt by talking. The agent stops mid-sentence, drops what it was about to say, and listens. A voice assistant that cannot be interrupted is walkie-talkie software, and waiting is the thing this product exists to remove.
02 · The hard part of interruption
The agent can hear itself
The obvious fix for barge-in is a fixed multiplier on the trigger while the agent speaks. That is wrong in a way you only find by using it in a real room: one number either lets the agent interrupt itself on laptop speakers or makes you shout on headphones.
So the page measures how much of the agent’s own output is arriving back through the microphone and learns the room. The level also has to stay up longer during a reply than outside one, 320 milliseconds against 140, because a door slam should not cancel a sentence.
03 · Memory
Two limits, deliberately different sizes
An assistant that forgets you every morning is a search box with a nice voice. This one keeps notes about the machine and a profile of you, and the two are bounded at different sizes on purpose: facts about an environment accumulate, so they get 2,200 characters, and the profile is held to 1,375 so it stays a profile instead of a diary.
Because the notes are full, the assistant has to choose what to drop. Asking where the memory is and who controls it is the first question any memory feature gets, and the answer here is two files you can open and edit.
04 · Failure
The error state is a flash, not a mode
When something goes wrong the sphere takes the danger colour for about 1.4 seconds and returns to whatever it was doing, because the failure is over and the app has moved on. An error state that persists is a second thing to dismiss.
Two failures were treated as one for a while, and that cost thirteen hours. An empty reply and a refused request looked identical to the user, so a broken model server produced a polite apology for losing a thread that was never lost. A failure the user cannot read is a failure that was not handled.
06 · The machine
Six phases and one flash
Seven names exist in the code, but only six are phases of a turn. The seventh is the failure, which fires from any state, lasts about a second and a half, and is not something the conversation can be in.
07 · Decision log
Four things I got wrong first
A design decision is only worth reading if the alternative is visible. Each of these was shipped, used, and then changed.
01
Amber for a running tool
Rejected because it read as a warmer version of thinking. Two states that mean different things cannot look like one state twice. It is a cool blue now.
02
Viewport height for the sphere
Rejected because the sphere resized continuously while the window was being dragged, so it visibly inflated and drifted for the length of every toggle. It is a fixed pixel value.
03
White ink for the tool icons
Rejected, twice. It was green, it was asked to become white, and white vanished entirely in light mode. The token went back to the colour it always was, because a value that reads on a near-black card is invisible on a near-white one.
04
A blue glow nobody chose
Nothing selected blue. It was an inherited colour, which is the worst reason for anything to look the way it does, and it is the kind of thing that survives ten reviews because it looks deliberate.
08 · The system underneath
A design system with a motion language in it
Fifty-nine custom properties, sixteen type sizes, and five named motion verbs that the states are assembled from rather than hand-animated one at a time. The palette is declared once per state and the renderer discovers it at runtime, which is what let the sphere be themed without a single branching `if`.
09 · Progressive disclosure
Thirty sections, and none of them on the main screen
The settings sheet is larger than the entire main interface, which sounds like the opposite of everything above. The difference is that the default state of the app is one sphere and one button, and every knob sits behind a button for the person who wants it.
The defaults are the product. The model is one field that is both a text box and a picker, the voice list follows whichever engine you choose, and changing the model changes nothing else in the interface.
10 · What is not built
Three things it still cannot do
There is no wake word, so it needs a key press before it listens. It does not transcribe while you are still speaking, so the transcript arrives after you finish rather than with you. And it does not keep a conversation across a restart.
Those three are the difference between an assistant you visit and one that is simply present. That is the part I have not solved.