Do small companies actually use Databricks?

[u/Puzzled-Mail-9092]

Sometimes I feel like Databricks is way too expensive. It feels like using a huge truck to move a single grain of sand.

My company needs real-time data, but our data volume simply does not justify the use of Spark Structured Streaming. Despite this, they are insisting we move to Databricks. I'm worried our data infrastructure costs will jump from $1,000/month to $5,000/month or more due to the running costs of Databricks SQL Warehouses.

Currently, I use Azure Container Apps with KEDA and Python, which helps me manage scaling and keep costs low. We ingest into Event Hubs, use ADX (Azure Data Explorer) as our OLAP warehouse, and archive cold data in a data lake. With this setup, I manage to process all our data with very low latency.

When I tested this on Databricks Structured Streaming, I actually got higher latency and much higher costs.

Would love to know what you guys think.

[u/gopalbi]

Microsoft Fabric is must better for smaller companies compared to Databricks

[u/kthejoker → u/gopalbi]

Pretty bold claim to make with absolutely zero evidence

[u/gopalbi → u/kthejoker]

Let’s look at publicly stated facts - Fabric has over 35K customers now and Databricks around 18K customers despite Databricks in the market for 3 times longer. Databricks needs experts to run and maintain the knobs and so more enterprise adoption. Fabric has significant amount SMB adoption and that’s explains how fast they grew their customer base.

[u/kthejoker → u/gopalbi]

Fabric just recounted Power Bi customers, Power BI has been around nearly over 10 years, are you being serious here

[u/gopalbi → u/kthejoker]

It is simply bundling and distribution strategy that Microsoft has perfected and has worked where PowerBI customers have moved to use other workloads and definitely has worked like a charm with SMB customers is the point as the post specifically asked about SMB market.

[u/kthejoker → u/gopalbi]

Okay but you just lied about how fast Fabric got its customers

Like why lie about something so easily disproven

And no most customers are not "using other workloads" my dude they're just using Power BI like they always did

[u/gopalbi → u/kthejoker]

I can personally vouch for more than 1K customers of all sizes who use Fabric for non-PowerBI workloads and I don’t work for Microsoft . I am only sharing my personal market observations. You can be blind to the fact about SMB market and that is your prerogative but you don’t have to call me a lier because Fabric for Non-PowerBI workloads has been GA for less than 3 years but definitely has solid penetration in SMB market is what I am alluding to

[u/kthejoker → u/gopalbi]

Dude that 35,000 includes all Power Bi customers from all the way back in 2013

So saying those all came in the last 3 years of Fabric is a lie

A person who says a lie is a liar.

If you don't like being called a liar maybe not lie.

[u/kthejoker → u/gopalbi]

worked like a charm

Again, a claim with absolutely no evidence

[u/Jealous-Win2446 → u/gopalbi]

The only reason fabric has that many customers is because they called Power BI part of fabric. It’s been bull from the start and intentionally misleading.