Developer User Research extends your runway, with Ana Hevesi
Chatting with users in your Discord and shipping their feature requests on the spot feels like user research. Ana Hevesi's argument: it's the opposite — it's so unstructured you can't actually isolate which problem connects to which solution, which is exactly what burns runway.
Ana Hevesi is a developer experience researcher and ecosystem consultant, with a background in developer relations, open source, and community management at Stack Overflow, Nodejitsu, and MongoDB.
We get into the single 'anchor question' she starts every research engagement with, why she deliberately interviews as wide a range of users as possible early on, and why treating research as a deliberate, isolated practice — rather than constant ad hoc Discord conversations — is what actually lets a startup extend its runway.
The Q&A below is experimental — extracted from the episode transcript with AI. If you spot a mistake, let us know at jack@scalingdevtools.com.
What actually drew you to focus specifically on user research for developer tools?
I spent a long time in open source community management and developer relations, across several companies and ecosystems, and kept noticing that a lot of what we did to reach users wasn't always as clearly justified as I felt it could be — we weren't always sure why a given action was the right one, or why it was having the impact we wanted. At one company, on the community team of a fairly mature public organization, I suggested we sit down and actually do structured developer user research given the product's maturity. What came out of it were very specific opportunities that hadn't surfaced through existing marketing or advocacy work — exciting use cases we hadn't known about, or clear appetite for peer learning in a specific area of the product. When I shared the findings, people asked why we hadn't been doing this sooner, and that's when I realized I might be onto something worth doing independently.
How do you typically structure a developer user research engagement with an early-stage startup?
I always start with what I think of as an anchor question — big, open-ended, and genuinely crucial to the business. Early on with a client, that's often something like 'in what ways does this product solve meaningful problems for our users,' though it can be more specific depending on where the company is — why do people get involved in our support community, why aren't we attracting more open-source contributors, or is this pivot actually the right evolution for the product. With early-stage clients especially, I recommend going as wide as possible with who we interview — different educational backgrounds, different amounts of time in the field, different roles, different demographics. Going wide surfaces the most potential opportunities to drill into later.
What does the actual interview and analysis process look like?
We reach out and ask users for their time, treating that time as a genuine gift. I write a script that uncovers two things together: where someone is in their journey as a technologist, so we understand their broader career goals, and the granular detail of how they actually use the product. We either record conversations or have someone taking notes live. From there I go into an analysis phase — I'm a big fan of the research tool Dovetail for this — looking first for clearly observable facts, like years of experience or job title, and then digging deeper into more subjective signals, like frustration or excitement in how someone describes their experience. Those observations start forming patterns and themes, which I layer on top of each other and pull out as quotes the whole team can refer back to and recognize together.
Once you've synthesized those findings, how does a team actually act on them?
The details vary enormously depending on the people, product, and situation — that variation is actually the point and the leverage. But broadly, after this process you'll have a much clearer, demographic-level picture of which kinds of users are having the best experience with the product right now — maybe heavy TypeScript usage correlates with strong adoption, or a lot of your active users are tech leads straddling management and individual contribution. You'll also understand where the product gives people something like superpowers versus where it's become onerous — gaps in the product itself, in error messaging clarity, or in documentation. That can point toward very specific next actions: deepening documentation past the initial getting-started stage for particular use cases, prioritizing integrations with adjacent tools, or even organizing structured peer-learning sessions if research reveals users want to grow in a specific broader skill area the product only touches the surface of.
How exactly does user research extend a startup's runway?
It gives you insights you wouldn't otherwise have access to, which lets you run far more deliberate experiments. If you're a founder thinking about time spent, headcount, and which direction to invest effort in, research lets you draw a much more specific, evidence-based picture of what you're doing, over what timeframe, and why — which means a much clearer navigational path with fewer wasted cycles and fewer surprises.
Some founders say they're already talking to users constantly, in Discord, shipping feature requests on the spot. Isn't that the same thing?
It looks and sounds similar from the outside, but it's actually distinct. What I do with clients is put a deliberate box around the process — essentially building a small laboratory — which lets you isolate variables and see clearly how one specific problem connects to one specific potential solution, and how much of your effort that's actually worth. Constant, unstructured interaction and on-the-spot implementation based on whoever happens to ask in Discord makes it extremely hard to isolate anything — you can't cleanly tell which change actually mattered. To be clear, I'm not saying don't talk to your users or don't build those relationships. I'm saying that stepping back and running this deliberately, then deciding your next move based on tangible evidence rather than whoever spoke up most recently, lets you chart a course that's actually paced to your runway, with far fewer surprises along the way.
Why has figuring out which action drives which impact gotten harder over time?
The rate at which people are becoming software developers has been accelerating continuously. Decades ago, being a programmer meant something fairly singular. A market research firm, SlashData, reported around 25 million developers worldwide in 2019, expected to grow to 45 million by 2030. Between bootcamps, traditional computer science paths, and the sheer variety of roles and sub-disciplines within tech now, it's genuinely harder to know which specific action connects to which specific outcome in your product experience and adoption efforts than it used to be — which is exactly why building that structured 'lab' and calibrating your instruments more precisely matters more now, not less.
What's the core message you'd want a founder to take away about this?
You're already doing developer user research whether or not you've planned it that way — the real question is how expensive and unstructured that version currently is, how much sooner you could have that information in a clearer form, and what you could do with the extra runway that clarity buys you. Structured developer user research is really a way of calibrating your navigational instruments far more finely, so everyone in the org understands where they're headed, and you as a founder can make a genuinely informed call about how much fuel you're spending to get to the next milestone.