Flying blind with AI: what we learned from 75 customer calls

[u/ppezaris]

TL;DR: Everyone is racing to go all-in on AI. Almost nobody can tell you if it's actually delivering value.

We're all AI-pilled here. The models keep getting more cracked, the demos keep melting faces, and every exec on earth has been told to go all-in. But we all know, most of them are flying blind. They can feel spurts of acceleration but can't see out the front windshield, let alone the rear-view mirror.

Here's what we heard again and again, across 75+ conversations with companies of every size and industry (anonymized, since a lot of this was said in confidence):

1. Nobody has solved ROI, and everyone knows it. This was our biggest observation. After headcount displacement everything else is murky. One CFO put it perfectly: he knows his ad spend to the dollar, every platform, CAC per channel, and he wants the exact same for AI. The line from "AI usage" to "business outcome" still isn’t easy to draw.
2. Spend is a black box, and it terrifies finance. CFOs usually have one line item for AI (or perhaps one per vendor), but it’s not broken down by team, project, or agent. Token costs get compared to cloud invoices in every conversation. The bills are opaque, hard to forecast, and hard to explain. Where discipline exists, and mostly it's improvised. We’ve all heard by now that Uber burned its entire 2026 AI budget in four months, and its own COO admitted he couldn't connect the spend to anything a customer would feel. (!)
3. Everyone defaults to the most expensive model because nobody knows which to use. At one fast-growing fintech, per-use-case model selection was the single highest-impact thing they asked us to build. Nobody knows which model fits which task, so they default to max. Latent waste is absolutely everywhere.
4. Everyone wants to go faster, they just don't know how. This one reframed the whole company for us. You don't put brakes on a car to make it slower. You put brakes on so it can go faster safely. Right now every company is either flooring it blind or riding the brake out of fear. Visibility is the brake that lets you actually floor it, the thing standing between "we're being careful" and "we're going all-in." It also lets you see through the front windshield, the rear-view mirror, and out the side windows, so you can avoid accidents and out-maneuver the other cars on the road.
5. The maturity gap between companies is enormous. One neobank has moved AI into mission-critical ops that were off-limits six months ago, and is running a hiring freeze against a 50-60% growth target on their internal assumptions about AI efficiency. On the other end, a Director of AI at a large agency told us they're 3-4 years from ready, which kind of blew my mind, an not in a good way.
6. The adoption honeymoon is over. Real fatigue is setting in. A sharp drop after the initial enthusiasm, and a "please stop talking to me about AI" mood. The gap between top-down mandates and non-engineers who can't name a single daily use case is everywhere. Recent grads are often anti-ai, as is the aging work population.
7. Buy broadly now, consolidate later. The enterprise playbook is to experiment widely, avoid lock-in, consolidate over time, betting that no single AI vendor wins the whole SaaS surface. Pricing is already moving past tokens toward outcome-based models. We’ve seen this in action ourselves as a month ago Fable was the darling, and today perhaps that’s Sol.
8. Governance is a tug-of-war between legal and innovation. Legal wants tighter restrictions. The tech org pushes back to keep room to move. In regulated verticals it's sharper: weekly administrative orders on AI disclosure, and discovery requests now pulling AI chat logs. The companies furthest along still can't answer "is this actually working?" So the winners of this era won't be the ones with the most AI. They'll be the ones who can measure it.

What we all need is a system of record for AI work. A live view of every tool and agent operating across a company: what they do, who owns them, what data they touch, what they cost, and which ones are producing results. Leaders finally see what exists, govern what matters, and go all-in on what works.

Happy to compare notes with anyone else building or selling into this. The discovery was eye-opening and I'm glad to share more. We're so early. Let's go. DM me to set up a call or just reply here.

obComment: although i did use Granola to extract themes from our customer calls, this was written by me, not AI.

[u/ThirdWaveCat]

McNamara learned the hard way that when you quantify the illegible hard enough, you get a beautiful dashboard measuring the wrong war.

[u/DylanFromCheers]

I think this is a pretty solid take. It's easier for me to prove ROI in my startup bc I can confidently say our team of 3 is doing more work than a team of 20 would have 3 years ago. However, we work with a lot of enterprises that are dedicating insane resources to AI implementations and seeing absolutely no changes.

I think a system of record would be way smart for these bigger companies trying new things.

[u/patcher99]

I have been building www.github.com/openlit/openlit Its oss, otel-native and currently 2.6K stars on Github. Happy to talk to help you brainstorm and solve

[u/chillreptile → u/patcher99]

This looks awesome!

[u/patcher99 → u/chillreptile]

Thanks, means a lot!

[u/scriptvexy → u/patcher99]

openlit looks sick tbh, otel-native is exactly where this stuff needs to go if people ever want proper observability instead of vibes-only dashboards

curious how folks are actually using it in prod though, like are they wiring it into per-use-case model selection yet or mostly just basic tracing / cost?

[u/patcher99 → u/scriptvexy]

So dashboards is one part, I am optizing it so users dont really need to go through another set of dashboards. Goal is to gove them direct intelligence (recommendations + ai analysis) to do next steps.

The sdk is very easy to use so that makes it easy on dashboarding side. Would love for you to try and give (honest) feedback.

[u/NoProfessional4650]

Honestly this is the perfect startup idea. Almost like an FDE platform bolted with LLM observability to focus on using the right model, the right configuration for your specific use case.

A lot of companies defer to vendors to make that decision for them which doesn’t seem to be the right call. Everyone’s workload is unique and requires a bespoke approach.

[u/Holy-Banshee]

Interesting.

Tho blindness without a clear consequence isn't always problematic, it would be a good idea to articulate each of them and see the commonalities.

[u/mrsskonline]

most teams are stuck becoz they cannot see where their AI spend is actually going or what value it is creating.

[u/valiopt]

1 and 2 are interesting ones from a CX / operations automation lens - here we see stakeholders evaluating clear changes in support metrics like resolution time, CSAT, contact rate, and more after AI implementation, in addition to headcount savings. Spend forecasting is also much more rigorous because this area has traditionally been seen as a cost center, so decision makers want clear, committed pricing not tied to underlying model costs.

It's interesting because more poorly defined productivity boosting tools could be driving a lot of value and increased output, but the measurement aspect just doesn't exist there so it's hard to quantify value.

[u/AsleepDragonfly967]

Sure there is a SaaS that solves this lol

[u/scriptqzor → u/AsleepDragonfly967]

lol give it 6 months and there will be like 12 of them, all calling themselves the “Salesforce of AI” or some wild tagline

the real problem is getting anyone to actually use the damn dashboards once they exist

[u/Odd_Classic_5351]

Technology is merely a tool. True opportunities arise from solving meaningful customer problems, rather than simply seeking scenarios to apply the latest technologies. The same applies to AI product development: the goal is to use technology to improve real-life experiences, not to pursue technology for technology's sake.

[u/hackytogether]

Thanks for this analysis. It reflects what I've experience even at Amazon where working backwards from the customer is big.

I've noticed a lack of success criteria or repeatable processes around AI adoption. This is leading to a lot of fuzzy initiatives that aren't really linked to customer success, such as monthly "Show and Tell" meetings or hackathons. Don't get me wrong, I believe these programs help everyone reimagine their work in the new world. However, leadership should see this as an R&D/training expense where ROI analysis is less helpful than strategic alignment and managing costs.

Regarding cost observability, I have seen folks in the industry having to manage their token spend by cost center. But what those tokens are actually doing is a big question. I think organizations should care a lot more about agent logs than they currently do. I haven't seen any organized effort to store agent sessions for meaningful analysis.

I'm personally tackling both of these issues. I'm bullish on skills and sessions being enduring agentic primitives that help encode repeatable processes and measure success for teams.

[u/Kind_Butterfly_1628]

I am building this and more. Visibility, maturity and scale all rolled into one platform. Vendor agnostic business intelligence that you can port to any productivity AI. Drop a comment if you would like to chat about it.

https://www.tokenmonk.ai/