Your AI is only as smart as the data fabric it walks on

By the end of 2025, half of all companies had AI running in at least three business functions. Copilots, agents, predictive systems—they’re everywhere now, from finance to HR to supply chains. The hype cycle is over, and deployment is real.

But here’s the thing nobody likes to admit: the hardest part isn’t the model. It’s not the GPU shortage or the cost of inference. It’s the data. Specifically, it’s the lack of context baked into that data.

Irfan Khan, president and chief product officer of SAP Data & Analytics, puts it bluntly: “AI is incredibly good at producing results. It moves fast, but without context it can’t exercise good judgment, and good judgment is what creates a return on investment for the business. Speed without judgment doesn’t help. It can actually hurt us.”

He’s right. If your AI doesn’t understand the business logic behind the numbers, you’re just generating wrong answers faster.

The context premium

Traditional data strategies are built for aggregation. Over the last twenty years, companies have dumped everything into warehouses, lakes, and dashboards. That works fine for reports. But when you extract data from operational systems and strip away the semantics—how it relates to policies, processes, and real-world decisions—you lose the meaning.

Consider two companies trying to manage supply-chain disruptions with AI. Both systems ingest inventory levels, lead times, and supplier scores. One also has context: which customers are strategic accounts, what tradeoffs are acceptable during shortages, the status of extended supply chains. Both systems analyze data quickly. Only one makes the right call.

“Both systems move very quickly, but only one moves in the right direction,” Khan says. “This is the context premium.”

In the old days, humans filled that gap. A procurement manager knew which supplier had a long history of reliability. A sales director knew which client was worth bending the rules for. But when AI acts autonomously, there’s no human to say “wait, that doesn’t make sense.” The system just executes.

Consolidation is the wrong answer

The obvious fix sounds like “put everything in one place.” But that’s not the solution. Moving data into a single repository doesn’t preserve context—it usually destroys more of it.

What companies actually need is a data fabric: an abstraction layer that sits across infrastructure, clouds, and operational systems. For agentic AI, the fabric becomes the primary interface. Agents don’t query raw databases. They interact with business knowledge through knowledge graphs that encode relationships, policies, and priorities.

This is a fundamentally different approach from the ETL pipelines of the past. Instead of flattening data into rows and columns, you’re preserving the semantics. The fabric doesn’t just integrate data—it orchestrates it. It lets AI systems understand that a late payment from a strategic partner matters differently than a late payment from a one-off customer.

The maturity gap

Most organizations know they’re not ready. According to recent surveys, only one in five companies considers its data approach highly mature. Only 9% feel fully prepared to integrate and interoperate their data systems. That’s a brutal reality check for anyone betting big on AI.

I’ve seen this play out firsthand. Companies spend millions on LLMs and agents, then wonder why the results are mediocre. The answer is almost always the same: garbage in, garbage out, but with a fancy API wrapper.

Khan’s point about context resonates because it shifts the conversation from “how do we build better models” to “how do we build better data foundations.” That’s a harder sell—it’s less glamorous than fine-tuning a new open-source model—but it’s where the actual ROI lives.

What this means for practitioners

If you’re building AI systems today, stop optimizing for model accuracy and start optimizing for data context. Here’s what that looks like in practice:

• Map your business processes before you map your data. Understand which decisions matter and what information drives them.
• Invest in knowledge graphs, not just vector databases. Vectors capture similarity; graphs capture relationships and rules.
• Don’t let your data engineers work in isolation. They need to talk to domain experts who understand the business logic.
• Build data fabrics that span operational systems, not just analytical ones. Your AI needs real-time context, not yesterday’s snapshot.

The companies that get this right will pull ahead. The ones that don’t will wonder why their AI is so fast at making bad decisions.