When Your Analytics Agent Can Trace Why a Number Looks Wrong
The Data Report: Weekly market signals on modern data platform shifts | Week ending July 13, 2026
The 30-second version
A useful next job for an analytics agent is tracing why a number looks wrong through lineage and a read-only warehouse; the platform decision it forces is which systems become governed, read-only agent surfaces.
An agent-written pipeline that runs is weak evidence it’s right; MotherDuck’s tutorial puts correctness in an explicit data contract, realistic tests, and a publish gate.
In the Radar: regression-testing a chat-analytics agent before a model swap, QuickSight’s semantic-layer move, and the AI-versus-data-engineers debate.
The Agent Reaches Below the Dashboard
You know the ticket. A number on a dashboard looks wrong, someone raises it, and answering “why” means a person opens dbt to read the lineage, checks the compiled SQL, queries the warehouse to see what the source data actually did, and reconciles all of it against the semantic model. That person is usually an engineer. It is almost never the work they had planned for the day, and the question that produced it, “why is this number wrong,” is the one an analytics chatbot has always been allowed to shrug at.
Integral Ad Science (IAS) built an architecture aimed at exactly that question. In dbt Labs’ account of the design, the IAS team connected their analytics agent to dbt, Databricks SQL, and Looker through dedicated Model Context Protocol (MCP) servers, the emerging standard that lets an agent discover and call a tool’s capabilities without a bespoke integration for each one. One server exposes dbt’s model metadata, lineage, and compiled SQL. One exposes Databricks SQL for governed, read-only checks under Unity Catalog. One exposes Looker, the dashboard on-ramp where the question starts. The post’s worked example is a hypothetical supermarket chain whose revenue report inflates the Food category because a popular drink got miscategorized: the exact trace a human runs today across LookML, dbt models, and validation queries. IAS offers one production reference architecture for tracing a suspect metric through lineage and read-only warehouse checks; the published example shows the mechanism, not an independently verified benchmark.1 The team’s own report is that investigations that used to take an analyst an afternoon now take minutes.
What’s new here isn’t “agents need a governed semantic layer.” That has been the advice all quarter, and it is about the agent answering correctly over a layer someone curated for it. This is the same plumbing pointed the other direction: the agent isn’t answering over the layer, it’s reaching through it to debug the pipeline underneath. Grounding gets you a correct answer. This gets you the reason a previous answer was wrong.
That reframes the decision you face as a platform owner, and it is wider than read versus write. Whose identity the agent acts under: IAS routes users through Google OAuth and an authorized group, and the MCP servers keep raw API credentials out of the agent’s hands. What each server exposes: the MCP server is the control point, and IAS turned on seven of the dbt server’s roughly 37 tools, not all of them. What gets logged: access is scoped and audited at the server boundary rather than inside the agent. And how failures and sensitive context are contained: every sub-agent adds a routing failure mode, and the agent’s business mode strips model names and compiled SQL before answering a non-technical user. IAS’s three-server design is one reference architecture worth testing.
Bottom line: The wrong-number question is exactly where analytics agents were supposed to stop and hand off to a human. IAS’s published design is one production case for not stopping there.
The Green Pipeline That Was Quietly Wrong
You’ve started letting an agent scaffold a pipeline or two. It runs, the rows land, nothing throws an error, and you ship it. Here’s the trap: “it runs” is the condition the agent is most likely to satisfy and the weakest evidence that the pipeline is correct. An execution check can’t catch a units or data-assumption error, and a code review can miss one when the units and the data contract live only in someone’s head.
MotherDuck’s tutorial on building robust pipelines with AI makes the trap concrete. The author points Claude Code at public weather data from NOAA, the US government’s climate record, and the first green pipeline hardcodes its partitions, reads tenths-of-a-degree temperatures as whole degrees, and keeps rows the dataset’s own quality flags mark as bad. Each failure executes cleanly and quietly corrupts the output. The reusable pattern is the workflow that follows: inspect the real data first, define the data contract explicitly, run realistic tests against fixture rows, then write, audit, and only then publish. Units, partitions, quality flags, deduplication, and idempotency all live inside that gate, not in a reviewer’s memory.2
None of it is exotic. It’s the checklist a careful engineer already runs, written down and pointed at the agent’s output, which is what makes it a gate the agent can’t satisfy by accident.
Bottom line: MotherDuck’s tutorial shows why a green run is weak evidence: correctness comes from an explicit data contract, realistic tests, and a publish gate.
The Radar
📊 If you ship a chat-analytics agent. Once your agent answers business questions, the missing piece is a pre-deploy gate, and Omni’s AI Evals is one shape of it: replay a fixed question set against the semantic model, with a judge scoring whether the agent picked the right topic, fields, and filters. The question it forces is whether your natural-language surface has any regression test before you swap the model under it.
🧩 If you’re deciding where your semantic layer lives. Amazon QuickSight’s multi-dataset Topics, now in public preview, let one governed Topic span up to 12 related datasets with auto-generated joins, potentially pulling more semantic work into the BI layer, the dashboard tool itself, rather than the warehouse. It’s an AWS-specific preview, not a settled market direction, and you can’t mix SPICE, its in-memory cache, with live Direct Query inside one Topic.
🧠 If you’re rethinking the data-engineer role. The recurring “is AI replacing data engineers” question gets a practitioner answer in Kirill Bobrov’s conversation with Joe Reis: coding throughput changes; accountability does not. Useful framing to pressure-test how your team adopts AI without quietly outsourcing the judgment.
Where would you draw the read-only boundary for an analytics agent investigating a wrong number? Reply and tell me where the line sits.
Published by RepublicOfData.io. Curated by Olivier Dupuis.
Single source, vendor-authored: a dbt Labs blog post recounting one customer’s architecture (Integral Ad Science). What transfers is the shape: dedicated read-only MCP servers per system, with access scoped and audited at the server boundary. Treat it as a reference design to test on your own stack, not a corroborated standard.
Single source, vendor-authored: MotherDuck’s own worked example on its own stack. What transfers is the operating pattern: inspect the data, define the contract, test realistically, gate the publish. Treat it as a worked example, not a corroborated standard.