AI Tooling · Design Systems
Robby 5 v2
The rebuild: describe a screen and get a working prototype that follows the design system — from a tool that refuses to build until it has read how the product works.
- Role
- Sole designer & engineer
- Timeline
- 3 months, ongoing
- Setting
- Independent, after Levelpath
- Result
- The AI can’t build until it has read how the product works
01 / 09 · What it is
Describe a screen. Get a working prototype that follows the rules.
Robby 5 v1 could build prototypes fast, but nothing made it follow the design system. v2 is the rebuild. A designer describes a screen and gets back a working prototype made from the real components — and the tool won’t build anything until it has read how the product works.
Deciding what an AI is allowed to know, and what it gets refused for, turned out to be interface design. The user just happens to be a machine.
02 / 09 · The problem, measured
A design system is a rule. To an AI, it’s a suggestion.
AI tools invent components you don’t have, reach for colours that aren’t yours, and describe your product in generic terms. I wanted to measure how often, not guess.
I set up a test version of Zendesk and planted three made-up words in its product description: a record type called Playbooks, a status called Waiting on vendor, and a screen called Frontline. No AI could know them without reading the description. Then I asked for eleven screens, each in a fresh session.
Not one of the eleven read the product description. They checked how to build — never what they were building.
03 / 09 · What came back
Every one of them passed every check
One built a screen from a different product entirely, complete with ServiceNow’s ticket numbers. One copied deliberately wrong vocabulary straight from the prompt. One built an invoice-escalation flow wired to NetSuite — the wrong product altogether.
All eleven still compiled, passed every automated check, and followed every layout rule.
There was nothing to catch. The mistakes are invisible unless you already know the product — so they survive review by everyone except the person they’d mislead.
04 / 09 · Why it happens, and the fix
Make the design system something the AI looks up, not something it’s told
Designers learn a design system by being in the room — in reviews, in what gets sent back. An AI has no room. It knows what it’s handed, plus a guess based on everyone else’s software. So I turned what the team knows into files the AI can look up and be checked against.
Product vocabulary · status
Waiting on vendor
- Use it for
- A request paused on the supplier’s side
- Never call it
- PendingOn holdBlocked
- Checked
- Every string in the prototype
01
Written as data, not prose
Components, rules, and product vocabulary live in structured files that are reviewed like code — and that someone who doesn’t code can still edit.
02
Looked up, not remembered
Fifteen tools let the AI ask what actually exists — real components, real colours, real rules — instead of guessing.
03
Only real building blocks
The AI builds from 21 approved building blocks. It can’t invent a component that isn’t on the list.
The rules stopped being a document and became an input.
05 / 09 · When asking wasn’t enough
So asking became refusing
The instructions had always asked the AI to read the product description first. The test showed what asking was worth. Now Robby 5 blocks the AI from writing a prototype until it has actually read the description in that session — and when it can’t tell, it says no.
There’s an override for people editing by hand, set where the AI can’t reach it. An override the AI could flip itself would just be an opt-out.
It’s the first time the rules held.
Asked to read the product first
0 of 11
Blocked until it had
2 of 2
06 / 09 · How it works
Two checkpoints, with ordinary generation between them
One checkpoint before the AI writes anything, one after. The generation in between is the least interesting part — the checkpoints are what make the output trustworthy.
Input
A sentence of intent
“A triage queue for open requests.” No component names, no token values, no product description.
Gate — before
— otherwise, refuses the write
Context gate
Has this session actually loaded the product context? A marker says yes or nothing does. Ambiguity resolves to no.
Bounded by
Generation
The assistant queries what exists rather than assuming it — real components with their props, tokens read from source, composition rules, the product's own vocabulary. Output can be constrained further to a spec the block registry resolves.
Gate — after
— otherwise, fails the run
Composition gate
Ten composition rules and the vocabulary check run against the result. Only high-confidence checks are allowed to block; the rest stay advisory on purpose.
Output
A prototype in the repo
Typed TSX in a folder, live at its own route. It reviews, diffs and branches like any other code.
07 / 09 · Where enforcement stops
Deciding what not to enforce
Every prototype is checked against ten design rules — empty states, loading states, colour never being the only signal — and against the product’s own vocabulary.
But only the checks that are almost never wrong are allowed to block. Some rules I care about, like confirming before deleting, stay as warnings, because a scan of the code can’t always tell whether they’re met.
A check that’s wrong one time in ten doesn’t cost you ten percent. It costs you the whole system, because people learn to ignore it.
08 / 09 · The evidence
Proof, not testimonials
Each test round sets up a fresh fake product, plants words no AI could guess, runs sessions blind, and searches the results for those words. The most revealing test asked for a word the product description forbids: the right answer is to use the correct word and say why.
- 8 of 8
- Greenhouse test runs where both checkpoints caught what they should
- 15
- tools the AI uses to look things up
The rules are mechanical now. They don’t depend on anyone remembering to be careful — including me.
09 / 09 · What I’d do differently
The gaps were where I reasoned instead of tested
01
I built the tests last
They’re my strongest evidence and my most fragile process — manual, and easy to spoil. One round was nearly ruined by an old server quietly serving outdated information. The scoring should have been built alongside the block.
02
The block proves less than it looks
It proves the session read the product description — not that every helper the AI spins up did. That’s a limit of how it works, and I’d rather say so.
03
I missed a door
A way of creating prototypes inside the app skips the block entirely. Leaving it open is defensible, since a person is driving it — but I found it by accident, not by design.