Siddhartha Borah from Brackett: teaching agents work that was never written down (Heavybit DevGuild)
Brackett AI teaches agents the undocumented 10% of a business process by watching people work—then turns what they learn into a repeatable workflow that can hand unfamiliar cases back to a human.
Siddhartha Borah is the co-founder and CTO of Brackett AI, which is building agents for complex business work, particularly in supply-chain and operations environments where essential knowledge often lives in people's heads.
Recorded at Heavybit DevGuild, this conversation covers why apparently small gaps in context stop end-to-end automation, how a screen-sharing session can teach an agent a process, and how confidence scores, human handoffs, memory and knowledge graphs help the system handle a changing real world.
The Q&A below is experimental — extracted from the episode transcript with AI. If you spot a mistake, let us know at jack@scalingdevtools.com.
What is Brackett AI building?
We are building agents that can learn complex business work. Many agent systems fail in the last mile because they lack the context behind the process. Work spans systems and people, and in areas such as last-mile supply chain, important details remain in an end user's head. We want an agent to operate like a teammate that can learn work which was never written down.
How do you teach an agent knowledge that only exists in someone’s head?
Think of it as a Zoom call with the agent. The person speaks to it, shares their screen and performs the work as usual. The agent watches, listens and asks questions along the way. That lets someone pass on nuances—such as a vendor whose invoices are handled on a different schedule—without first turning their entire job into formal documentation.
How do you make a nondeterministic model reliable enough for enterprise work?
The exploratory nature of an LLM is both a feature and a challenge. An enterprise needs an agent it can run thousands of times with consistent behavior. Brackett converts the learned knowledge into a verifiable and repeatable workflow, so the agent qualifies how the work should be done and does not need the same correction every time.
How much undocumented context does it take to break an automation?
Even if 90% of a workflow is captured, the missing 10% can prevent the whole job from being completed. Context is not just data; it includes procedures and knowledge distributed across people. A coding agent asked to change a feature across three repositories will fail if it can see only one. Real-world work is harder because the equivalents of those repositories might be a spreadsheet, an ERP system and a fact known only by a person.
Why is watching someone work the best way to capture that context?
Operations and supply-chain workers may not be technical, but they are experts at performing the job. Asking them to formalize everything first creates another difficult project. Learning through demonstration is how apprentices have acquired complex skills throughout history: the mentor does the work and the learner observes. The product makes converting that demonstration into a workflow as effortless as possible.
What happens when the agent encounters an invoice or format it has never seen?
The system assigns confidence to what it understands. If the environment changes, it can fall back to another agent; if that agent is still not confident, it sends the case to the human. The person explains the exception—for example, that a particular vendor has more relaxed rules—and the system writes that information into memory so it can decide correctly when the same situation appears again.
How does Brackett’s memory system work?
An agent works alongside the user and takes notes about potentially important information. At intervals, another process organizes those observations, similar to the idea of dreaming. A knowledge graph with an ontology tailored to the business then gives the memories structure, so they are not merely an ever-growing pile of transcript fragments.
How do background agents help without overwhelming the primary agent’s context?
The primary agent stays focused on helping the user. Behind it, an orchestra of agents searches for relevant information and offers hints about what may matter. The primary agent can then explore the useful leads without filling its working context with every possible piece of organizational knowledge.