Ankur Goyal from Braintrust
Braintrust founder Ankur Goyal explains how a list of roughly 50 AI frontrunners created focus, why complaints are stronger product signals than praise, and how two weeks preparing user interviews exposed evals as the urgent problem.
Ankur Goyal is the founder of Braintrust, an LLM evaluation platform used by companies including Figma, Vercel and Stripe. At the time of the conversation, the company was about a year old and had raised a $36 million Series A.
Ankur breaks down the customer concentration, structured research and problem-first thinking behind that start, then shows how those same users pulled Braintrust from evals into logs, datasets and a more complete environment for building AI software.
The Q&A below is experimental — extracted from the episode transcript with AI. If you spot a mistake, let us know at jack@scalingdevtools.com.
How did a list of roughly 50 companies help Braintrust focus?
At the start, very few companies had shipped interesting generative-AI products, so Elad Gil and I listed the organizations already doing it or about to. Most eventually became customers or investors. That narrow group removed much of the usual prioritization debate: if a frontrunner such as Zapier or Notion needed something, other companies were likely to need it shortly. Concentration made customer requests feel repeatable rather than bespoke.
Why target the innovators before the broader market?
AI was a new way of building software for most engineers, while the earliest teams had already accumulated useful practices. Working with them let us codify those lessons into the product. New customers could then begin much closer to the frontier instead of relearning every mistake. Vercel’s eval-first workflow is a good example: once evaluation entered the pull-request loop, systematic progress became much faster.
What did Vercel’s early feedback do for the product?
Guillermo and Max were skeptical and gave very direct criticism. A production problem eventually pushed the team to install Braintrust, and adoption spread once the eval loop worked. When copying results into GitHub became painful, a strongly worded message led us to build a GitHub Action. Specific complaints were valuable because they proved the product was being used in a real workflow and showed exactly where to improve it.
Why are complaints a better signal than enthusiasm?
Someone can say a product looks great without ever trying it. A complaint usually means the person invested enough effort to hit a real limitation. That is both evidence of value and a precise opportunity to make the product better. I feel energized by it: criticism is the heartbeat showing that a customer is deriving enough value to care what happens next.
How did Elad Gil help uncover the Braintrust opportunity?
I had built similar internal tooling at Impira and Figma, which made the idea feel boring to me. Elad supplied the market perspective: once generative AI expanded the audience from specialists to software engineers, tooling for building AI became a much larger opportunity. Open-ended interviews then produced an unusually strong signal—leaders such as Brian at Zapier followed up asking for the product immediately.
How should a founder prepare for user interviews?
Do not wing them. We spent one or two weeks debating questions, their sequence and the assumptions behind them in a shared document. The questions were non-leading and structured enough to compare answers across companies. That preparation revealed counterintuitive facts quickly: customers did not want the operational burden implied by an open-source product, and many were building AI software in TypeScript rather than defaulting to Python.
Which interview questions exposed evals as the bottleneck?
We asked what languages, clouds and databases people used; how they chose internal versus external projects; which modalities they handled; how they logged performance; what they wanted tomorrow; what made them most anxious; and what blocked wider rollout. The emotional, open question about anxiety was especially revealing. Across interviews, the answers repeatedly pointed to evaluation rather than a problem we had named for them.
What should happen after the research call?
We used the last ten minutes of a half-hour conversation to explain the direction. Polite approval during a pitch means very little, so the point was to seed the idea rather than count compliments. The signal came afterward, when several people independently returned and said they needed it now. I had not written code yet, but prior experience meant a first prototype took only three or four days once demand was clear.
How are customers expanding Braintrust beyond evals?
Evals created demand for data, so we built logging that helps diagnose a user issue and turn it into datasets for later evaluation or fine-tuning. With logs already present, customers then wanted monitoring. They also preferred iterating on prompts and test cases in a collaborative interface rather than inside a format string in an IDE. The product is moving toward a developer platform where code supplies scaffolding and much of the AI iteration happens in Braintrust.
What problem test should an AI DevTools founder apply?
Have a very high bar for the business problem, not just an appealing implementation. Fine-tuning, for example, is one possible method for improving model quality and cost; customers care about the outcome, not that method by itself. A useful heuristic is whether you can sustain an entire customer conversation about the problem without discussing your proposed solution. If you cannot, you may be attached to a technique rather than building a company.
Links & references
This episode is brought to you by WorkOS.