Your Next Data Model Might Not Have a Human Author
The Data Product Report: Weekly State of the Market in Data Product Building | Week ending June 1, 2026
This Week
Most data engineers still build the model layer by hand. It is the part of the job that feels furthest from automation, because it is where the judgment lives: which grain, which joins, what counts as a customer. So the question worth sitting with this week is what happens to that judgment when an agent drafts the model for you, and whether you would ship what it hands back. The reason to treat that as a now-question and not a someday-question is dltHub Transformation, which hit public preview this week with a real rebuild behind it instead of a demo: an agent scaffolded a company’s model layer and closed a data freshness problem that had outlived months of backlog grooming. What makes it a signal worth acting on is that all of it is open to inspection, the toolkit, the numbers, the engines that can read the result, and the way to check the agent’s work.
An Agent Rebuilt the Model Layer, and the Dashboard Caught Up
The thing that decides whether agent-drafted modeling earns a place in your stack is mundane: does it save real work, or just move the work around? At Navit, the answer was a number that had been stuck for months. On-time delivery for a row of dashboards sat around 80 percent against a target near 99, and the fix kept losing to the rest of the backlog. An agent-scaffolded rebuild with dltHub Transformation, which hit public preview this week, closed the gap in about three weeks, and time-to-metric dropped from days to hours.
The toolkit is an agent-guided workflow with four steps. You point it at your raw sources and it proposes a taxonomy of what is in them. It turns that into an ontology, then into a common data model (a single, tidy description of your core tables), then into the actual Python transformation code that the dlt engine runs. The agent doing the scaffolding is your choice of Claude, Codex, or Cursor. The engine executes the result, so the agent is writing the plan, not babysitting the pipeline.
What makes this worth your attention is not that an agent wrote some code. It is that the claim is testable against your own work. dltHub used the same toolkit to migrate its own stack from HubSpot to Attio in two weeks. The reported time-to-metric improvement is the kind of number you can hold up against your current migration backlog and check.
The honest caveat: this is public preview, on the paid tier, and one company’s rebuild is one data point, not a benchmark. But the shape of the claim is the news. Agent-authored modeling has moved from “watch this demo” to “here is a before-and-after you can try to reproduce.”
The bottom line: The teams that pointed an agent at a stuck migration this week got a common data model and runnable transforms out the other side, and at least one of them watched a months-old freshness problem close. What changed is not that an agent wrote code. It is that the result came with numbers a skeptic can try to break.
The Catalog Became the Place Everything Reads From
Say the agent did hand you a clean model. The next question is mundane and a little political: which of the engines your team actually runs can read it, and at what cost in copies and per-tool permissions? For most teams the honest answer is still “some of them, once we duplicate the data and wire up access for each one.” This week Databricks made Unity Catalog’s Iceberg support generally available, and the news underneath the announcement is that the catalog, not the storage format, is becoming where that question gets settled.
In plain terms: a catalog is the index that tells every tool where your tables live and who is allowed to touch them. Unity Catalog now exposes a standard catalog API that Spark, Trino, Flink, Snowflake, DuckDB, and even pandas can call to read and write the same Iceberg tables, without each of them needing broad permissions on the underlying storage. It can also federate catalogs you do not own, governing and sharing tables that live in AWS Glue or Snowflake Horizon without moving the data into Databricks first.
For anyone running a lakehouse, this is a concrete decision, not a vibe. The question on the table is whether your current Glue or Hive metastore setup still earns its place once one catalog can broker reads and writes across every engine you run. There is a roadmap promise too, to converge the Iceberg and Delta metadata so the format war quietly ends, but that part is not shipped, so treat it as a direction and not a feature.
The bottom line: The teams evaluating catalog strategy this week have a generally-available way to let any engine read one set of tables through one governed API. The ones still wiring per-engine permissions onto raw storage are maintaining plumbing the catalog layer now offers to absorb.
Before You Trust the Agent, You Have to Measure It
Here is the question that should nag at you the moment you let an agent near the warehouse: how would you even know it got the join right? The first two stories assume the output is correct. Nothing in them tells you whether the model the agent chose, the join it assumed, or the filter it applied actually holds. Hex wrote up the lab it built for exactly that question, and the part worth stealing is the pattern, not the product.
The lab, which Hex calls Shoebox, runs pairwise experiments: a candidate version against a baseline, changing one thing at a time so you can see what a new model, prompt, or piece of context actually did. A local development stack iterates fast while a shared remote workspace holds a baseline that updates daily, so every comparison is apples to apples instead of drifting underneath you. The test data is synthetic but modeled on a realistic business, not a generic public benchmark that looks nothing like your schema.
You do not need Hex to copy this. The reusable idea is three moves: isolate one variable at a time, keep a stable reference you trust, and measure against realistic data before you ship. That is a pattern any team shipping a text-to-SQL or natural-language query feature can stand up with the tools they already have.
The bottom line: The teams that built a measurement loop before turning an agent loose on their warehouse this week can answer “did this change help?” with a number. Without that loop, “the agent seems better now” is the whole quality story.
The Radar
If you’re shipping AI query interfaces:
Snowflake published Arctic Text2SQL R2, a compact model trained specifically on Snowflake’s SQL dialect that beats much larger frontier models on enterprise SQL. If you ship a text-to-SQL feature, the cost math just changed: a smaller model tuned to your warehouse’s dialect may beat the biggest general model you are paying for. And AWS published a primary walkthrough of evaluating text-to-SQL agents with LangSmith, built on a tasks, trials, and transcripts structure with offline pytest checks and online production monitoring. It pairs well with the Hex pattern above as a concrete starting point.
If you’re building pipelines:
dlt published its first throughput benchmark: roughly 65 GB/hr from Postgres to BigQuery on a small 2-core, 4 GB worker at about a dollar an hour of compute, scaling linearly. If your current extract-load job moves similar volumes, you now have a baseline to measure against. And Dagster showed how to wrap Snowflake Dynamic Tables as declarative pipeline nodes without running any Dagster-side compute, which is a clean fit if you want orchestration to declare intent and let the warehouse do the work.
If you’re rethinking your stack:
A walkthrough on building durable workflows directly on Postgres argues you can use row locking and transactional checkpoints instead of adopting Temporal or stretching Airflow. The discussion was lively and not one-sided. Teams with lighter orchestration needs may find the trade-off genuinely different from the pitch they have been hearing.
If you care about data residency:
A sharp analysis of EU sovereignty makes the case that running in an EU region is not the same as being sovereign: storage, identity, and DNS calls can still route through us-east-1 even when your workloads sit in Frankfurt. If you carry GDPR or Schrems II obligations, that control-plane exposure is an architecture question, not a checkbox.
If you’re running Databricks:
Two Lakebase updates worth a look: Always-On pricing that saves about 25 percent on steady-state workloads (with a deeper promotional discount running into January 2027), and a Change Data Feed that captures row changes natively so the operational database can feed your Bronze layer without bespoke connectors.
Have you let an agent scaffold a data model you actually shipped? Reply and tell us what you kept and what you threw away.
The Data Product Report is published every Tuesday by RepublicOfData.io.