Quinn Favret from Tavus: AI video API that saved our episode
Tavus's Quinn Favret on why customers asked for interrupt-sensitivity toggles and got a dedicated turn-taking model instead, why being a research company and a product company pull in different directions, and the digital-human 'Turing test' the team runs internally.
Quinn Favret works on product at Tavus, which builds the models and APIs behind "digital humans" — AI video interfaces that hold face-to-face conversations, from AI tutors to sales reps to companions.
We get into why Tavus thinks empathy has to be engineered rather than assumed to emerge from raw intelligence, how the company balances being a research lab with being a product company, and the turn-taking problem that taught them to solve the underlying need instead of shipping the parameter customers asked for.
The Q&A below is experimental — extracted from the episode transcript with AI. If you spot a mistake, let us know at jack@scalingdevtools.com.
You've said empathy and emotion aren't an emergent property of intelligence. What does that mean for building a real digital human?
As intelligence gets better and better, our belief is that it doesn't become easier to actually communicate with — you've talked to ChatGPT and told it to do something and it does something completely different. In a real conversation, a lot of what's being communicated is in micro-expressions, peripheral vision and focus, patterns of speech — the full picture of what someone means. When you combine that right-brain, expressive layer with the left-brain intelligence layer, that's when you get what we consider a true digital human. And the goal is to make it so simple to use that people don't have to realize the complexity running underneath — they just get the result.
Looking ten years out, who do you think ends up being Tavus's biggest customers?
Tavus is a series of models for building digital humans, and I see a world where anyone can go in and say "I want to create an AI blank," connect it to their knowledge base, and deploy it through a natural language interface, the same way people use ChatGPT today. Right now, the spaces we're most excited about are healthcare — pre-op and post-op care, patient intake, companionship, elderly care — and recruiting, on both sides of the table. The one that personally excites me most, and was a lot of my own inspiration for joining, is education: a K-through-12 use case where a student who isn't quite clicking with the material in class can spend an hour one-on-one later in the day with a digital AI teacher that actually understands the context of what happened, reads them a story at their level, and answers their questions — the same way great teachers already do it, just made accessible to everyone.
What is Tavus, in plain terms — is it fair to call it the Twilio of conversational AI?
To a degree — an infinitely more complex, more varied version of Twilio, but you could use that as a reference point. We build the core models — rendering, perception, turn-taking, and others — and bring them together into what we call the conversational video interface, effectively the operating system for human-AI conversation, which developers and product teams build on top of through dev-first APIs. We're not quite an infrastructure layer, more an operating system that other teams build their actual products and experiences on top of. We build everything in-house except the LLM and text-to-speech — for text-to-speech we work with partners like Cartesia and ElevenLabs, and on the LLM side you can plug in your own knowledge base and model, whether that's Llama, ChatGPT, or something else, either using our fine-tuned pipeline end to end or swapping in your own pieces as building blocks.
How does building something like an AI tutor actually work as a developer, if it's not a traditional if-this-then-that application?
It's not an if-then tree. We have a layer we call personas, which inform the LLM who it is, what the conversation's goals are, and what it can and can't talk about — memory, guardrails, that kind of thing. Prompting is really important here; a lot of people still underestimate it, and a one-word change can fundamentally change how smooth a conversation feels. You might prompt it as an interviewer with ten specific questions and instructions to ask for elaboration on thin answers, in a firm, one-by-one style, or leave it much more open-ended — some people even use a second LLM as an orchestrator on top. The persona is what informs the LLM's identity, goal, and context for that specific conversation.
Customers asked you for interrupt-sensitivity toggles and turn-taking parameters. Why didn't you just build that?
That's a good example of research-versus-product tension. Early on we got a lot of feedback that the bot was responding too fast or too slow — cutting people off, or leaving an awkward pause. What people actually asked for was interrupt-sensitivity toggles and more parameters to control it directly. But that wasn't the right answer — the right answer was solving the deeper problem of how AI understands when a person is actually done speaking versus just pausing to think. So we built a dedicated turn-taking model, trained on a lot of data, that understands whether what it's hearing is an affirmation or an actual interruption, and adapts to a person's natural flow of speech far better than any rigid parameter could. If turn-taking were perfect and we never had to expose a single parameter for it, that would be the win — nobody actually wants to think about ASR or VAD settings, they're only asking for them out of necessity because the underlying thing isn't solved yet.
You're both a research company and a product company. How do you reconcile those two modes?
They're different in some ways and similar in others. At the end of the day we're a research company — the models and tools to build what we're building just don't exist anywhere else, so research is where we spend most of our time. But research is also product: we have to deeply understand what users need and make the complexity underneath invisible to them. Where it gets different from a typical product org is that we also have to tell users what's possible — think several years out, not just about what people want today but what the interface should ultimately feel like. What we landed on is that talking to a computer should feel as natural as a face-to-face conversation, and everything gears around that. We call it the Tavus Turing test internally — you should be able to have a conversation with a digital human and not realize it's AI. Being research-oriented means planning months and years out rather than putting a band-aid on today's problem.
What advice do you have for DevTools founders, especially ones straddling research and product?
Beyond the traditional advice — know your users deeply, make APIs easy to use, docs are a product — a few things I've learned at Tavus that weren't obvious to me earlier. First, make it a magical experience: someone should feel surprise and delight the first time they use your product, the way people log into Linear and almost want to use it more. We try to do the same thing at Tavus — even small details, like how a conversation wraps up instead of just abruptly stopping, or intro music that plays before a session starts, all add up to shaping someone's first impression. Second, support is product — really understanding the questions users ask and getting them fast answers, so it's clear you're building with them, not just supporting them from a distance, more like an extension of their own team. Third, have fun and have a personality. In a world where every website looks like the same Webflow template, people want to understand your vision, your story, the emotion behind it — I remember reading Linear's manifesto for the first time and immediately getting it. That vision, plus a genuine magic moment, covers for a lot of other things you might get wrong.
How do you find the kind of people who do genuinely great work in a space like this?
In some ways, those people find you. If you're mission-driven and public about your philosophy and principles, it attracts people who are genuinely passionate about what you're doing rather than just looking for a job — some of our best hires found us and reached out themselves. We also don't look for a traditional background — someone who's been at Google for two years and done a startup for two years. We look for people from diverse backgrounds, fresh out of college or very senior, where a specific piece of their work is genuinely interesting to us — we've read someone's paper, reached out to the author, and it turned into a hire because it was clear they cared deeply about a similar problem. My cofounder has a line I use a lot: in a different world, everyone on our team would be a craftsman, an artist, a poet, a woodworker — people who take real pride and see their work as craft, not just a job. You see that difference in the day-to-day, and in how the product comes together.
Links & references
This episode is brought to you by WorkOS.