# Why AI Agents Break in Production: The Missing Harness in Your Data Stack

*Listen to the full conversation on DataScienceWithSam, available on* [*Apple Podcasts*](https://podcasts.apple.com/us/podcast/ep-45-why-ai-agents-break-in-production-the-missing/id1587954336?i=1000776852944) *and* [*Spotify*](https://open.spotify.com/episode/0nKtoDZCNpcgD2NOAyFVzO?si=Vzos9X0YTkSeS0bUXoF9Mw&nd=1&dlsi=3d298d39cd99420e)*.*

On this episode of DataScienceWithSam, Pradnesh Patil — Co-Founder and CEO of Altimate AI, and a former product leader at Palo Alto Networks, Cisco, and VMware — argues that this isn't a model failure. It's a missing harness.

**The core distinction: prompts vs. harnesses**

A system prompt tells a model what to do. A harness tells it what is actually true. Patil breaks the agentic data engineering harness into five components: context, governance, MCP tools, shared skills, and agent infrastructure. Miss any one, and the failures aren't loud — they're silent. The query runs, returns a result, and looks correct. It just isn't.

That gap explains two of the episode's sharper numbers: 27–33% of AI-generated queries reference phantom tables that don't exist in the schema, and 78% contain silent wrong joins — joins that execute cleanly but return the wrong data. None of this is the LLM being bad at SQL. It's the absence of infrastructure that grounds the model in the real state of the data.

**Harness beats model size**

Patil points to Altimate Code's performance on ADE-Bench as proof: it topped the leaderboard running on Sonnet while competitors relied on Opus. The takeaway isn't "bigger model wins" — it's that the surrounding system matters more than raw model capability.

**Where LLMs should — and shouldn't — make decisions**

A recurring theme is the deterministic/LLM boundary. Validation, cost checks, and query correctness are deterministic problems with deterministic answers, and Patil argues they should never be handed to an LLM to reason about probabilistically. Save the model for judgment calls; let hard rules handle hard rules.

**Fixing context compaction for long-running tasks**

Standard LLM context compaction — built for general chat — tends to destroy long-running data engineering tasks by discarding the schema and lineage context an agent needs mid-task. Altimate built a different compaction approach specifically to preserve that context over time.

**The $5,000 query bill**

Governance isn't abstract here — Patil cites a real $5,000 Cortex AI query bill as the kind of outcome permission-based governance is meant to prevent. Without controls on what an agent can query and at what cost, agents can go rogue on cloud spend just as easily as they can go rogue on correctness.

**Where data engineering is headed**

Patil's broader claim: the days of hand-writing SQL are ending, and the data engineer's role shifts toward building and governing the systems that let agents do that work reliably. His advice to the industry: build open source, build cross-platform, and avoid siloed AI features that don't move the field forward as a whole.

*Listen to the full conversation on DataScienceWithSam, available on* [*Apple Podcasts*](https://podcasts.apple.com/us/podcast/ep-45-why-ai-agents-break-in-production-the-missing/id1587954336?i=1000776852944) *and* [*Spotify*](https://open.spotify.com/episode/0nKtoDZCNpcgD2NOAyFVzO?si=Vzos9X0YTkSeS0bUXoF9Mw&nd=1&dlsi=3d298d39cd99420e)*.*