Baseten CEO and co-founder Tuhin Srivastava on inference and feedback loops
Baseten cofounder Tuhin Srivastava explains how years of pre-GenAI infrastructure work met a suddenly accelerating market, why customer feedback speed defines company speed, and how “more inference” became a filter for every product decision.
Tuhin Srivastava is the CEO and cofounder of Baseten, an inference platform that had just announced $150 million in Series D funding at the time of this conversation.
Baseten spent its early years building machine-learning primitives before market demand arrived. Its later acceleration required narrowing the product, staying close enough to customers to avoid becoming their bottleneck, and preserving quality while both the company and its users moved dramatically faster.
The Q&A below is experimental — extracted from the episode transcript with AI. If you spot a mistake, let us know at jack@scalingdevtools.com.
What is the Baseten origin story?
The founders started in 2019 with a broad belief that machine learning would matter and that a picks-and-shovels company should exist beside it. From 2019 into 2022 or 2023, a team of roughly 10–20 built useful primitives without matching market demand. When the GenAI market appeared, Baseten adjusted its direction rather than replacing its underlying thesis, then grew beyond 120 people largely within 18 months.
Was the market change immediately obvious?
Something clearly changed, but it did not arrive as a single ChatGPT moment that removed all doubt. Demand accumulated through 2023 while the market remained fickle and stressful. Only in retrospect could the team see that it had found a great market with the right focus. In the moment, the shift felt incremental, and success created something valuable that the company could now lose.
How did the product narrow?
Baseten initially combined an application builder, laptop-to-production model deployment and serverless code for stitching systems together. The company removed the first and third pieces and concentrated on deployment and inference. Some ideas later returned in a different form, such as execution sandboxes for LLMs, but the surface area became narrower while the chosen problem became much deeper.
What is hard about meeting explosive demand?
Every choice becomes a tradeoff between customers, product, recruiting and quality, only at much greater speed. Baseten serves some of the fastest-growing AI companies, whose own expectations and infrastructure change rapidly. The company must keep pace without becoming their bottleneck, while operating beneath them in a market whose technical landscape is still moving.
How does a small company actually move fast?
Compress every feedback loop between the team and the customer: the time to hear a problem, ship something, observe the result, communicate and improve it. Baseten embeds engineers closely enough to resemble pair programming with customers and begins from what the user is trying to unlock rather than from a technology. Direct engineer-to-engineer contact preserves the reason behind a request instead of reducing it to a feature ticket.
Why is “more inference” Baseten’s product filter?
The company believes most businesses will become AI-first or AI-enabled, model capability will increasingly converge and both open and custom models will proliferate. Inference is where those models create user value and generate data that can improve future models. Training, evaluations, capacity partnerships and other adjacent work qualify when they directly or indirectly help customers run more models and create more inference.
What matters most in inference developer experience?
Speed prevents waiting, debuggability explains the often difficult packaging and deployment process, and observability shows what happens once a model is in production. The guiding rule is to make easy tasks easy while keeping difficult tasks possible. Abstraction should remove unnecessary work without hiding the information and controls an advanced user needs when something fails.
Why add training to an inference platform?
Customers repeatedly asked for the Baseten inference experience when training models. Sending them to another provider created a leak: the other platform might retain the later inference workload, where most spending occurs. A training product keeps the workflow together and applies Baseten’s developer-experience layer above compute, while still serving the North Star that models trained there should ultimately generate more inference there.
Is access to GPUs the competitive advantage?
Flexible capacity is part of the advantage: Baseten worked across ten clouds and 40 regions, could aggregate demand and could add compute to the cluster where needed. But raw compute is not the product, and cheaper sources exist for buyers who only want machines. Customers pay for the abstraction, developer experience and inference system with capacity included rather than for the lowest possible GPU price.
What advice does Tuhin give earlier DevTools founders?
Stay close to customers, but do not rush immature abstractions into broad use. Foundational technical and developer-experience decisions become very difficult to replace once users depend on them. Keep a beta small, combine customer contact with the intuition that comes from being a developer, and wait until the experience is genuinely delightful before earning scale that freezes the wrong choices in place.