AI has made building so fast that a new problem has appeared. It is now easy to ship something every few days that nobody has actually tried.
The code works. The tests pass. The demo looks good. And then real people use it and get stuck on the second screen.
That is why, in my process, real users try every preview before anything ships. Not once at the end. Every preview, every round.
Why testing has to keep up with building
In the old rhythm, design took weeks and building took months, so a usability study near the end still had time to change things. Now a working preview arrives every few days. If testing only happens at the end, you have built weeks of features on top of assumptions nobody checked.
The fix is to make testing part of the loop. Each preview gets tried by real people on real tasks, and what they struggle with shapes the next preview.
Who tests
The people who will use the product. Not the founder, not the team, and not me.
Founders know too much. Teams know where the buttons are. I have 30+ years of using software and can find my way through almost anything. None of us are representative of a new user arriving cold.
So I recruit people who match the actual users: the staff member who will use the dashboard every morning, the parent who will open the app once a week, the reviewer who has to approve a release before a deadline.
When I work with an existing team, I run the first sessions with their designers watching, then coach them to run the next rounds themselves.
How a round works
Each round is short and focused.
- Pick the tasks. Three to five real tasks drawn from the spec's core flows, like "add a new applicant" or "find out whether this release is safe to ship."
- Moderated sessions. A handful of people try the tasks while thinking aloud. I watch where they hesitate, what they expect to happen, and what they say under their breath.
- Unmoderated tests. Quick remote tests with more people give wider coverage on one or two specific questions.
- Measure. Task success, time on task and the moments people get stuck.
- Rank. Every finding goes into a list ranked by severity.
The ranked list is the output that matters. It turns "people seemed a bit confused" into "four of five people could not find the save button, and that blocks the core task."
Fix the worst problems first
Findings from testing and feedback on each preview become the next round of changes, worst problems first. Not the easiest fixes. Not the most interesting ones. The ones that hurt users most.
This sounds obvious, but it is easy to get wrong. When a round turns up fifteen issues, the temptation is to fix the ten quick ones and feel productive. Meanwhile the one problem that stops people completing the main task waits another round.
We review every preview together against the spec, and the spec is updated as decisions change, so it stays the source of truth.
What to watch for
A few things I have learned to pay attention to in sessions:
- What people do, not what they say. "This is easy" while clicking the wrong thing three times means it is not easy.
- The first five seconds. If someone cannot tell what a screen is for almost immediately, the screen has a problem.
- Workarounds. When testers invent their own way around something, the intended way is not working.
- Every screen size. A flow that works on a laptop can fall apart on a phone. Having written a book on responsive design, I test at more than one size by habit.
The code works and the tests pass. That only tells you the software does what it was told. Testing with people tells you whether it was told the right thing.
Shipping is not the end
Nothing reaches production until it passes automated tests, a security review and sign-off on the preview. Then it deploys, and we watch the metric we agreed on at the start.
But launch is the start of the loop, not the end of the project. What real users do after launch feeds straight back into testing and iteration. In my process, the last step, Ship, loops back to Test.
Production usage shows things no test session can: which features people actually use, where they drop off, and what they do that nobody predicted. That data plans the next round.
Tools help; people decide
Tools make this faster. Previews are deployed automatically, sessions are recorded, and AI can help summarize what happened across many sessions. On my own products, BlastRocket makes sure every visual change is seen by a person before release, across six screen sizes from phone to ultra-wide.
But the judgment stays human. A summary can tell you that people paused on a screen. Watching them tells you why.
If you are shipping previews every few days but nobody outside the team is trying them, email me. Setting up a simple testing rhythm is one of the quickest ways to make AI-speed building actually pay off.