I built a proprietary research panel from zero once, for an agency serving a roster of regulated pharma and banking clients. No panel existed when I started, so every study meant going through a third-party vendor and accepting whatever quality they offered. I proposed building our own and my manager agreed we needed one. Where we split was how to fill it. She wanted to buy email lists and seed the panel that way, fast.
I refused. Buying the list probably would have worked in the short term. But a panel built on contacts who never opted in can’t survive the moment a pharma client’s legal team asks how these participants were sourced. The list would have bought us speed and nothing a regulated client could trust.
Building it to almost 1,000 quality participants took roughly a year and meant creating the recruitment mechanism from nothing. That looked like an opt-in flow, a referral system with real attribution, and a compliance stack built with legal from day one. It was slower and harder. It was also the only version of the panel that could hold up to an audit.
I think about that decision a lot now, because anyone spending the first dollar in a new research function is facing the same choice. AI tooling has taken the place of the purchased list. The tradeoff between buying volume and buying something trustworthy is the same.
Tooling produces output immediately. Whether that output is any good is a separate question, and the tool can’t answer it. A synthesis tool will summarize interviews whether the sample was solid or garbage, and a panel service will keep delivering responses long after the panel stopped representing anyone real. Without someone in the room who can tell the difference, the function is producing something that looks like research and may or may not be.
A new function is where this mistake is easiest to make and most expensive to undo. If the first dollar buys a tool and the second buys another tool, nobody is positioned to evaluate any of it until much later, after bad habits and bad outputs have already piled up.
So the first hire should be someone who can tell good research from confident-sounding research, and that hire should happen before a single tool gets purchased. Tooling without judgment produces volume, and volume nobody can evaluate is a liability, no matter how good the dashboard looks.
Once that person is hired, the actual build-versus-buy math changes. The question stops being people or tools and becomes what this person’s time should go toward, and what should get automated so their time isn’t wasted. A skilled researcher spending hours on transcription and first-pass coding is a bad use of a scarce, expensive skill. The same researcher spending that freed-up time validating whether an AI-generated synthesis actually holds up is a genuinely good use of it. The tooling didn’t replace the need for judgment; it moved judgment to a different part of the job.
A few signals worth checking before the first significant tooling purchase:
Is there anyone on the team who could catch a tool getting it wrong? If the honest answer is no, that’s the gap to close first, not the tool to buy first.
Is the tool being evaluated for what it produces, or for how fast it produces it? Speed is real value, but it’s not the same value as accuracy, and a purchase decision that only weighs speed is skipping half the question.
Would the team notice if the tool quietly started producing worse output six months from now? If nobody’s specifically watching for that, the tool is running unsupervised, whatever anyone assumes about it.
None of this is an argument against AI tooling in a research function. It’s an argument for sequencing. Buy the judgment first and everything after that is a real efficiency gain instead of an unmonitored risk.
This is part 2 of Who’s Actually Running This?
Previously: Centralize or Embed
Next: Proving the Function’s Worth
Ground Truth is written by Shannon Coleman, PhD. Strategy and research leader with a background in psychology and quantitative methods. Her work spans consumer product contexts, with particular depth in enterprise AI platforms. Portfolio: shannonlcoleman.com


