Introducing How to Do User Research on AI Products
The methods that work on deterministic software break on probabilistic systems. Here is what to do instead.
At some point in the last two years, most product research teams started getting asked to evaluate AI features.
Some of them had done this before. Most hadn’t. And the ones who hadn’t quickly discovered that the standard toolkit, the usability tests, the satisfaction surveys, the A/B tests, the interview guides, worked fine until they didn’t. The methods that had always produced reliable signal on deterministic software started producing something murkier on probabilistic systems. Users said the AI was helpful. Trust scores looked fine. And then something would go wrong in a way the research hadn’t anticipated and couldn’t explain.
The problem is not that traditional research methods are bad. They are well-designed for the systems they were designed to study. Software that behaves the same way every time is a different research subject than software that produces different outputs for different users under different conditions, degrades silently when inputs drift from the training distribution, and can be confidently wrong in ways users cannot detect.
AI products require a research practice built for that specific set of properties. Most teams don’t have one yet. They are adapting methods that were built for a different problem and discovering the gaps in real time, usually when something has already gone wrong.
This series is about closing those gaps before they become failures.
The first two series on Ground Truth built the case for why rigorous measurement matters in AI product development: series one on evaluation frameworks, failure modes, and governance; series two on the evidence infrastructure problems that get encoded into AI systems upstream. This series gets specific about the research practice itself. Not the organizational conditions or the strategic framing, the actual methods. What works, what breaks, and why.
Four parts covering the research challenges specific to AI products that traditional methods weren’t designed to handle.
The Series: Four Research Challenges
Part 1: Why Standard UX Research Methods Break on AI Products
The foundational piece. Names the specific properties of AI systems that create research challenges traditional methods weren’t designed for: probabilistic outputs, calibration failure, silent degradation, and the gap between user satisfaction and decision quality. Establishes the frame for the three parts that follow.
Part 2: How to Study Trust Calibration Without a Survey
Trust is the central research challenge in AI product work and a single Likert item does not measure it. This piece covers what trust calibration actually means as a research construct, why self-reported trust diverges from behavioral trust in AI contexts specifically, and what methods actually surface whether users are relying on AI outputs appropriately.
Part 3: Designing Evaluative Research for Probabilistic Systems
Evaluative research on deterministic software asks: does this work as intended? On probabilistic systems the question is more complex: does this work well enough, for which users, under which conditions, and how would we know if it stopped? This part covers how to design evaluative research that answers the right version of the question.
Part 4: Neither and Both
The quant versus qual debate is the wrong debate. Here is the right one. Mixed-methods research on AI products is not about balancing two types of evidence. It is about sequencing them correctly so each type of evidence answers the question it is actually suited to answer. This part covers what that looks like in practice.
This series is for UX researchers, product researchers, research leads, and anyone responsible for generating insight on AI products who has noticed that their existing methods are not quite equipped for what they are actually dealing with.
It is also for product managers and directors who commission research on AI features and want to understand what good research practice looks like so they can evaluate what they are getting.
The methods exist. They are not exotic or inaccessible. What has been missing is a coherent account of which methods apply to which AI-specific research challenges and why. That is what this series provides.
Start with Part 1: Why Standard UX Research Methods Break on AI Products.
Part 2: How to Study Trust Calibration Without a Survey
Part 3: Designing Evaluative Research for Probabilistic Systems
Part 4: Neither and Both
Previously on Ground Truth: The Capability Question: What AI Enablement Actually Does to People — Series 3 Intro
Ground Truth is written by Shannon Coleman, PhD. Product strategy leader and researcher with a background in psychology and quantitative methods. Her work spans govtech, healthtech, fintech, and consumer product contexts, with particular depth in enterprise AI platforms. Portfolio: shannonlcoleman.com