Many products start with a pile of documents. PDFs, scanned forms, spreadsheet exports, a folder of files someone emails around every month. The data the product needs is in there somewhere.
AI is now good at pulling it out. That part looks impressive in a demo. Upload a file, get a tidy table.
But people do not act on a tidy table just because it is tidy. They act on it when they can see where each value came from, how sure the system is, and when they get to check and correct it before anything is saved.
After 30 years of technical UX work, my view is simple. The extraction is the easy part. Trust is the design problem.
Keep the source next to the value
Every extracted value should point back to where it came from. Not "somewhere on page 4," but the exact passage, cell or region, one click from the value.
When a reviewer sees a start date next to the sentence it came from, checking takes a second. When they have to open the PDF and hunt for it, they either stop checking or stop trusting.
Anthropic's citations feature helps here: it returns the exact passages that support each answer, with page numbers for PDFs. Its docs also note limits worth designing around: a scanned PDF with no extractable text cannot be cited, and your code still has to check that each field came back with a value and a citation.
The same principle applies to any AI summary on a review screen. Let it open to the evidence behind it. Summaries without evidence get distrusted. Evidence without summaries gets skipped.
Show confidence, and route the uncertain items
Not every value deserves the same attention. A clean, typed total is different from a handwritten number on a crooked scan.
Microsoft's documentation for Azure Document Intelligence describes field confidence as an estimated probability that a prediction is correct. Where accuracy is critical, it says confidence can decide whether to accept a prediction automatically or flag it for human review. The same page notes that not every field returns a confidence score, so plan for gaps.
In practice I would design three lanes:
- Accept. High confidence and low consequence. Visible, but not in the way.
- Review. Low confidence, or a field that matters too much to skip. Sent to a person, with the source open beside it.
- Missing. Nothing found, or no score returned. Treated as a gap to fill, never as a blank that passes.
Where the line sits should follow the cost of being wrong, not a number that feels safe. And a confidence score is an estimate, not a promise. Spot-check the accepted lane too.
Let people correct before anything is saved
This is the rule I care about most. AI proposes. A person checks and corrects. Then the system saves.
Mentor Intelligence works this way: a founder's pitch is transcribed and summarized into a list of needs, and program staff edit that list before anything is saved. The model never writes to the database on its own.
That costs a few clicks. In return, every saved record traces to a person who looked at it. It is the same thinking behind where an AI agent should ask for human approval: the person should see exactly what is about to be saved, not a "Continue?" button.
Make correction cheap. Edit in place, next to the source. If fixing a value takes longer than typing it from scratch, people will stop fixing and start ignoring.
Keep the original file
Store the document as it arrived, unchanged, alongside what was extracted from it.
You will need it. A reviewer will recheck a value months later. A better model will arrive and you will want to run extraction again. A customer will ask why a number is what it is.
When you do re-extract, keep the earlier result. The new one supersedes it without rewriting it, so the history of what was reviewed stays intact.
Record who approved what
For every saved value, keep:
- what the AI proposed;
- what the person changed;
- who approved it, and when;
- which file, and which version of it, the value came from.
That record is what lets someone say "I checked this" and mean it. It also shows you where extraction fails, because the corrections tell you exactly which fields the model gets wrong. It is the same reason humans keep launch authority in my release process: there is a clear record of who said yes.
Start with the documents customers already have
Founders often want to start with integrations that pull data from the customer's system automatically. That can be a good end state. It is a slow first step.
Customers already have the files. Accepting them as they are gets real data into the product sooner, and shows you which fields matter before you build any connection.
A sensible order is an importer for the files they already send, then a live connection once you know what is worth connecting. When you do connect, consider syncing into your own database, so pages load fast and keep working when the other system is slow.
Design the decision, not just the extraction
Extraction is never the point. Someone will act on the data: approve it, price it, match it, file it.
So design backwards from that decision. Which fields does it actually need? Which ones change the outcome if they are wrong? Those get the review lane, the source beside them and the strictest rules. Fields that do not affect the decision may not need extracting at all.
If the data feeds a match, the fields that drive the match need the most care. If it feeds a payment, the amounts and dates do. Getting those right, with a person checking them, is what makes everything downstream worth using.
People do not trust extraction. They trust a value they can trace, check and correct.
A checklist for document-driven products
- Show the source next to every extracted value.
- Route low-confidence and high-consequence fields to review, and treat missing values as gaps.
- Let people correct before anything is saved.
- Keep the original file and earlier extractions.
- Record what was proposed, what changed and who approved it.
- Start with the files customers already have.
- Design for the decision the data feeds.
If your product starts with a pile of documents and you are not sure people will trust what comes out, email me. I am happy to look at the files with you.