Hamzah Chaudhary from Lightdash: bringing developer tools to Business Intelligence
Lightdash started as an internal tool Hamzah Chaudhary and his co-founder built so they wouldn't have to pay for an expensive BI tool at their own data consultancy. A customer spotted it, said it looked "a bit janky" but asked to use it anyway — and the consultancy quietly stopped being the business.
Hamzah Chaudhary is the co-founder and CEO of Lightdash, an open-source, code-driven business intelligence tool. Before that he was a software engineer running data pipelines and machine learning models at Cytora, where he found the data team's tooling decades behind what software engineers had come to expect.
We get into how a side project inside a data consultancy became the whole company, why holding onto your own product vision matters even when smart people keep offering compelling-sounding alternatives, and the specific challenge of building one tool that has to work for both the data engineer writing SQL and the business user who just wants a number.
The Q&A below is experimental — extracted from the episode transcript with AI. If you spot a mistake, let us know at jack@scalingdevtools.com.
What gap did you see in tooling for data teams specifically?
At Cytora I joined as the second engineering hire and ended up responsible for our entire data pipeline, from ingestion through modeling to serving machine learning models to customers in insurance and banking. What I found was that our data teams were woefully underprepared, tooling-wise, compared to the software engineering side of the business — it was the Wild West: CSVs being thrown around, things saved in Google Drive, no concept of Git version control, no preview or dev environments versus production. My first job leading that team was figuring out what tools could give data people the same kind of leverage software engineers had enjoyed for years. It turned out those tools basically didn't exist yet.
How did Lightdash actually start?
My co-founder Oliver and I originally set out to bootstrap a data consultancy in January 2020 — great timing — building and managing data stacks for other companies, including one linked to the UK's Department of Education, aggregating viewership data on online curriculums during the pandemic. Lightdash started as an internal tool, just for the two of us to manage that consultancy business, because we didn't want to pay for one of the big BI tools ourselves. Someone on the customer side saw it and said it looked a bit janky but asked to use it anyway. A few people kept pulling it out of us even though we insisted the consultancy was the real business, and eventually we realized this might actually be the thing — the consultancy stopped, and Lightdash carried on.
What happened when you actually launched it publicly?
We wrote the first lines of what became Lightdash toward the end of 2020, then posted it to Hacker News with an embarrassingly long, overly technical title trying to precisely describe what it was. Someone in the comments said, 'this is basically open-source Looker' — which became our tagline for the next year. In the earliest days, the open-source version didn't even have a login page; there was very little people could actually do with it. But we were surprised by how far people pushed that limited version anyway, which gave us real conviction people needed this. We probably didn't handle the resulting inbound especially well with just three of us — we spun up a community and told people to come talk to us, and that was mostly it.
How did you avoid getting pulled in a hundred directions by early user feedback?
Staying focused on the original vision matters a lot, because you'll always get more requests than you can act on — it's easy to lose sight of what you set out to build if you chase every piece of feedback. We got a bit lucky in that early requests tended to cluster: seven or eight people asking for almost exactly the same thing in a given week, rather than wildly different asks, because we were deliberately focused on one specific type of user from the start. I also think founders often value other people's opinions more than their own, especially from people who sound smart and can produce a compelling-sounding take in five minutes — when in reality you've spent a hundred hours thinking about that exact problem. Trusting your own judgment, including with a large partner like dbt, matters just as much: know what's genuinely important to you and hold that, even while looking for where your incentives and theirs actually align.
Who's your actual target user, and where do you find them?
A technical data analyst — SQL-savvy, Python-dangerous, comfortable in an IDE and the CLI, but not someone spinning up their own Kubernetes clusters. dbt has built the biggest open-source community for this persona, which they now call 'analytics engineering,' and we still work closely with them — the majority of our inbound today still comes from open-source communities like theirs. Beyond initial discovery, growth compounds in a way I didn't fully expect: once Lightdash gets rolled out inside a company, it tends to spread wide, because when a team builds its core KPI dashboards in it, everyone from data to marketing to sales to spend ops ends up exposed to it. We're also now old enough that people who used Lightdash at one company are moving to a new one and bringing it with them.
You mentioned contributors building really impressive things on top of Lightdash. What does that look like?
There are roughly three buckets. Some people contribute genuinely massive features directly to core Lightdash — engineering work I honestly wouldn't have expected them to be capable of pulling off. Others build a separate product or plugin with Lightdash powering it underneath, almost like an OEM arrangement. And some just build and share really compelling open datasets using it — one company that monitors ocean health, tracking plastic and microplastic pollution, built an open interactive dashboard in Lightdash that started as something they shared at a conference and now works with us directly. Another company with the largest dataset of Pokémon card auction data — some of those cards go for seven figures — built interactive dashboards in Lightdash to share that data publicly too.
What does bringing software engineering practices to BI actually look like in the product?
Most BI tools are UI-driven — drag-and-drop editors, at best a cloud IDE — which makes it very hard to build a real developer workflow. Take something as simple-sounding as 'revenue': in practice it usually means joining two datasets, deduplicating, and applying business-specific adjustments, because data is quirky and it's never just 'sum the Stripe total.' In Lightdash, those kinds of business-critical metric definitions get codified, version-controlled, and stored in GitHub — immutable enough that changing one requires deliberate intent, but easy to pull from once defined. That gives you two workflows: a developer workflow, where you write metrics as code with local IDE and CLI support and hot-reloading against a live dashboard so you see a chart update as you tweak a calculation, with the usual staging, PR, and validation steps before production — and a much simpler UI-driven workflow for the business user who just wants to look at revenue cut by region and get on with their day.
How do you think about giving people the right level of control without overwhelming them?
Think of it as levels of abstraction: at one end, someone who knows SQL can write any raw query they want and do anything; at the other, someone just wants you to tell them a number, like how much revenue came in last week. Different levels of abstraction should allow different actions — the trick is giving people guardrails without giving them stop signs, so it doesn't feel overwhelming but also doesn't let people break things unknowingly. For the developer, that means making the system incredibly configurable; for the business user, it means making it genuinely easy. Balancing both of those in a single product is the actual hard problem BI tools have to solve.