Author: Jonathan Dufault

In March I spent a weekend running test queries through Power BI Copilot on a Fabric F2 capacity, reviewing the diagnostic JSON to understand how to improve my reports. One of those diagnostics is the reason for this post. When I asked a question about product growth trends, the answer had taken 107 seconds to arrive. The DAX query behind it executed in 173 milliseconds. The other 106.8 seconds was LLM overhead, and the why didn’t exist outside of the diagnostics.

Copilot tackles a data question in two tiers. A semantic parser gets the first shot, matching your words against the model’s tables, columns, and relationships, and it plays by the model’s rules, so a column marked Hidden (IsHidden = 1) might as well not exist. My question needed Product[Category], which I had hidden. Refused. The question dropped through to the second tier, an LLM that writes DAX from scratch.

The DAX generator has auto-retry logic. If the generated DAX fails to execute, it tries again with a different approach, up to a limit. Mine hit the limit:

"activityDetails": {
  "fallbackReason": "queryNotSupported"
},
"daxExecution": {
  "autoRetryCount": 2,
  "notRetryableReason": "MaxAutoRetry",
  "executeDaxDuration": 172.5
},
"generateDaxDuration": 106814.5

Three generation attempts, each with its own token cost. The first produced invalid GROUPBY DAX. The second tried a join with no common join columns. The third ran. Interestingly, the DAX generator does not respect column visibility the way the parser does (Microsoft’s documentation mentions this). The generated DAX reached the hidden Product[Category] through TREATAS without complaint. The tier that honors your configuration refused the question. The tier that ignores your configuration billed three attempts and then answered using the column you hid.

Four Layers

The reflex is to blame the question. Growth trends means comparing periods, comparing periods means real DAX, expensive question, expensive answer. The diagnostic says otherwise. A question of comparable complexity on the same model, four tables, three filters, three aggregations, resolved through the parser in about 2 seconds. The price didn’t come from what I asked. It came from a checkbox in the model view, and between that checkbox and the invoice there are four layers a user can’t see through.

They can’t see the visibility flags. IsHidden lives in the model properties, set by whoever built the semantic model, possibly years ago, possibly for cosmetic reasons. Hiding a column is how you tidy a field list. Nobody hides a column thinking they’re routing future questions to the expensive tier.

They can’t see which tier is handling the query. There is a tell, the loading message switches from “Checking the underlying data” to “Generating a DAX query,” but it only reads as a tell if you’ve been in the diagnostics. To everyone else it’s a progress message with vocabulary.

They can’t see the retries. Three generations happened behind one spinner. The UI shows a wait, not an attempt count, and each attempt carried its own tokens at Copilot’s rates, 100 CU seconds per 1,000 input tokens and 400 CU seconds per 1,000 output tokens.

And they can’t see the cost land. Copilot operations are classified as background jobs, so they get spread across 24 hours. The 400 CU seconds from one query gets divided into roughly 16.67 CU seconds per hour across the next day. By the time consumption is visible in the Capacity Metrics app, the question, the retries, and the hidden column are a day in the past. The Capacity Metrics app is retrospective, not preventive.

Four layers, one property. Everything that determines the price sits upstream of anything the person paying can observe. The cost of a Copilot query is coupled to model hygiene, and model hygiene is invisible from the chat pane.

Why the Estimate Missed

Before that weekend I went into the Azure calculator, pulled up the resource usage, all told would be no more than $5.00. The bill came to $18.78, and at the time I filed the gap under salespeople. That was unfair. The calculator estimates provisioned capacity, and it got that part right. The F2 itself cost $1.80 for the hours I ran it. The rest was consumption, and consumption is retry behavior multiplied by token counts. Retry behavior is determined by whether the semantic model gives the parser a clean path. No estimator can price whether somebody hid a column three weeks before you typed your question.

What the System Does

The purpose of a system is what it does. Nobody at Microsoft sat down and designed a meter that charges people for a checkbox they can’t see. The parser honors visibility because that’s the whole point of visibility. Retries exist because a second attempt beats an error message, and the smoothing is there to protect shared capacity from bursts. Every one of those choices is defensible on its own. Chain them together and you get a machine that spends 107 seconds of paid reasoning on a hidden column, then scatters the evidence across the next 24 hours. Configuration is priced in. Nobody sees the price until it’s been paid.

The full pipeline teardown, with the diagnostic exports all of this comes from, is in Looking Under the Hood for Power BI Copilot.

This is a companion piece to How Power BI Copilot Works Under the Hood. In the main post I traced the full pipeline from question to answer, but one of the more practical findings was about the two paths Copilot can take to resolve a data question. One is fast and reliable. The other is slow, expensive, and prone to hallucination. Which path you land on depends on how your model is set up and how you phrase the question.

Two tiers, one question

When you ask Copilot a data question through answerDataQuestion, it first tries a semantic parser. This is a structured query engine that resolves your question against the model’s relationships, column names, and hierarchy levels. In the UI, you’ll see “Checking the underlying data…” while it works. The query I described in the main post (warehouse clothing sales in the US by month, joining four tables with three filters and three aggregations) resolved in about 2 seconds through this path.

If the parser can’t handle the question, Copilot falls back to generating DAX with an LLM. The UI shifts to “Generating a DAX query…” and the wait gets longer. One of my test queries took 107 seconds to generate DAX, then 173 milliseconds to execute it. The bottleneck isn’t running the query; it’s writing it. And because it’s freeform code generation, the results are less predictable.

So the practical question becomes: what keeps you in the first tier?

What the parser handles

The parser is good at questions that map to a single pass over the data with grouping, filtering, and basic aggregation. Think of it as roughly equivalent to what you could express in a single SELECT with GROUP BY. From the 24 queries I tested:

Simple aggregations work. “Total sales amount,” “average unit price,” “count of orders.” Filters on specific values work. “Sales in the United States” resolves because the parser matches “United States” to an actual value in the country column. Top N queries have native support; “top 10 customers by sales amount” stays in the fast path. Sorting works in both directions.

Multi-table joins work as long as the tables are connected through your model’s defined relationships. “Sales by customer country” resolves because sale→customer→country follows the relationship chain. The parser also has built-in relative date functions, so “last year” and “this year” work regardless of what dates are actually in your dataset. These are more robust than hardcoded years, which sometimes appeared in Copilot’s own suggested follow-up queries.

What surprised me was how much the parser could combine. That warehouse clothing query joined four tables (sale, reseller, product, sales territory, date), applied three filters (Warehouse, Clothing, United States), computed three aggregations (sum of quantity, average unit price, average discount), sorted by month, and resolved in 2 seconds. All in the fast path.

What falls through to DAX

The boundary is roughly: if the answer requires comparing results across different slices or building on intermediate calculations, the parser can’t express it. Year-over-year growth needs two separate period totals and a comparison between them. Percentage changes need the same. “Which products grew the most AND which declined?” asks for both sides of a ranking simultaneously, which requires multiple passes.

In my testing, every query that involved conditional logic across different analytical dimensions (“products where sales increased but returns also increased”) or multi-step analysis (“average sales for customers who bought more than 10 items”) fell through to DAX generation. The parser is single-pass by design; anything that needs a subquery or self-join is out of scope.

Setting your model up for the fast path

The single highest-leverage thing is descriptive names. The parser resolves natural language against your table, column, and measure names. sales_amount gives it something to work with; sa_amt_v2 does not.

Synonyms on hierarchy levels matter more than I expected. In the main post I documented a case where the parser refused a query about “category” because Product[Category] was hidden, even though the hierarchy level Product → Product → Category was visible. The parser resolves through hierarchy paths, not just columns. Adding a single Terms: ["category"] synonym to the hierarchy level turned a DomainModelLimitation refusal into a working query. One word.

Column visibility is important and asymmetric. The parser respects IsHidden; if a column is hidden, the parser won’t use it and may refuse the query entirely. The DAX fallback tier doesn’t always respect this boundary the same way. I found cases where generated DAX used hidden columns through TREATAS. This is another reason to prefer staying in the parser tier: it actually honors the access controls you set.

Field descriptions have a side effect worth knowing about. Setting descriptions on columns in “Prep data for AI” can trigger auto-generated synonyms. In one case, a field description caused a synonym to appear that made a previously-failing query succeed. The description itself wasn’t what fixed it; the synonym it generated was.

If you know users will ask questions that would normally fall to DAX (year-over-year growth, percentage changes), write measures for them. The parser can use pre-built measures in a single pass, turning a two-tier question into a one-tier question.

Phrasing

Refer to columns and measures by something close to their actual names. “Total sales amount” resolves better than “how much money did we make” if your measure is called Sales Amount. Be specific about filter values: “sales in the United States for clothing” gives the parser concrete strings to match against column values, where “US clothing revenue” requires it to infer that “US” means “United States” and “revenue” means “sales.”

Split compound questions. “What are the top customers and how have they changed over time?” is two questions. The first part stays in the parser; the second (change over time) forces a DAX fallback. Ask them separately and both answers come back faster.

One thing I didn’t expect: conversation context can poison the parser. If a query gets refused in one turn, the refusal persists in the conversation’s context events and can contaminate subsequent queries. I tested the same question in a session where a prior refusal existed and in a fresh session. It failed in the contaminated session; it succeeded in the fresh one. If Copilot starts refusing things it should be able to handle, start a new chat.

How to tell which tier you’re in

Watch the status text in the Copilot pane. “Checking the underlying data…” means the parser is working. “Generating a DAX query…” means it fell through. In the Service (browser) environment you’ll also see a “View DAX query” button when the DAX tier was used. Desktop doesn’t surface this, but the fallback still happens behind the scenes.

If you’re consistently seeing “Generating a DAX query” on questions you think should be simple, that’s a signal to check your model setup: column names, visibility, and synonyms. The parser’s capabilities are wider than you’d expect. Usually when it falls through, the model is the bottleneck, not the question.

The Bill

Power BI is part of the Fabric ecosystem. Fabric is an end-to-end data platform, including ingestion, warehousing, outflows, and dashboarding, including Copilot for Power BI. The costing for Fabric involves you purchasing capacity units (CU), from F2 for 2, to F2048. Capacity units are measured in seconds. If you have 2 capacity units per second run, then you have 60 seconds available to you per 30 seconds of runtime. When I ran my workspace, I paid for 5 hours x 60 minutes / hour x 60 seconds / minute x 2 capacity units = 360,000 seconds, for a total of $1.80. But that’s not the full story.

Fabric bills Copilot at 100 CU seconds per 1,000 input tokens and 400 CU seconds per 1,000 output tokens. A request with 2,000 input tokens and 500 output tokens consumes 400 CU seconds, about 6.67 CU minutes. An average Power BI Copilot query in my instance ran about 1,500 CU seconds, though summaries have cost more. At the pay-as-you-go rate of $0.18 per CU hour, that’s roughly $0.075 per question. But the CUs come out of the same capacity pool that runs your Spark notebooks, warehouse queries, dataflows, and semantic model refreshes.

Regardless, you’re paying for fixed capacity, if you consume more than that then the service should throttle.

When you open a dashboard in Power BI or run a SQL query, the cost for that is spread out over the next 5 minutes to 24 hours. Copilot operations are classified as background jobs, so they get spread across 24 hours. This helps because it means the capacity you have running idle at 2am takes some of the cost of a burst of user queries at 2pm. This is a good thing.

That also means the 400 CU seconds from one query gets divided into roughly 16.67 CU seconds per hour across the next day. It’s also what makes the cost hard to notice until it accumulates. I ran my Fabric environment for 5 hours, but during that time, fabric was assuming that I would be running it for the next 24 hours.

If you have twenty queries a day from one user, 22 working days a month: about 660,000 CU seconds, 183 CU hours, roughly $33 in shared capacity consumption. Their Power BI Pro license costs $10. A team of ten moderate users on an F64 ($8,400/month) would put about 39% of the entire capacity toward Copilot alone. The remaining 61% handles everything else. The Capacity Metrics app is retrospective, not preventive.

The F2 made all of this visible because there was nothing else running. On a production F64, the same $17 of Copilot consumption would be a rounding error in a monthly bill that’s already four figures. Same cost. Same CUs consumed. Different signal-to-noise ratio.

I spent last weekend testing out Copilot for Power BI. I’ve been using it extensively as a developer, trying to make something that’s useful for people. I’ve been growing more skeptical of all my technical skills.

I went into the Azure calculator, pulled up the resource usage, all told would be no more than $5.00. Salespeople. I made the Power BI report, ran the Copilot queries. Documented the telephone game, deleted the resources.

Now I’m looking at what the telephone game cost. Not much, but for 5 hours for a single user running queries for only a small fraction of that time, a bill for $18.78. How did that happen?

I encountered the same when running my queries in Roo. This isn’t a post about money.

The Telephone Game

I showed in the last post that the Copilot pipeline is a chain: outer LLM rephrases the question, Q&A parser tries to match against the semantic model schema, if that fails a DAX generator fires as fallback, results go back to the outer LLM for formatting. Each step consumes CUs. Each retry consumes more.

The 107-second DAX generation with three retries was because a column got hidden in the semantic model. Metadata, not complexity of the ask. The Q&A parser kicked it to the DAX generator, which tried bad DAX three times before giving up. The diagnostic JSON shows all of it. Each attempt had its own token cost.

This is one tradeoff of a multi-agent system. Flashlight on the ground, looking for a problem that falls within its torch-light. Each node in the chain simplifies the problem to something it can solve. If you remove enough context or focus enough detail, anything becomes solvable. This means that four can chain together, with irrelevant details getting amplified. Telephone game.

The Q&A parser is good at what it does. The DAX generator is good at what it does. The handoff between them is where meaning drops out. That’s not a capability problem that gets fixed by making each node smarter. It’s a structural property of chaining agents.

Build vs. Run

The $18.78 bill on a restarted F2 capacity warrants another look, but not as a measure of cost, because if it means not hiring a third data analyst, then $3.70 per hour washes out. The problem is $3.70/hr/user. It runs counter to the trend of the last 40 years. Build vs Run.

Build. When you build a dashboard, it is a capital expense. Build it, done. Whether 5 people or 5,000 people look at it, the marginal cost per user is the same, nearly zero. If you use or rent a GPU, same.

Run. A Copilot query is an operating expense. Every question costs CUs. The more people use it, the more capacity it consumes. Until you invest in infrastructure or reserve capacity, success and cost scale together. Copilot answering “what were sales last month” for the 400th day in a row is going to incur 400 times the cost. The next person who asks pays again. To its credit, Fabric caches identical queries for 24 hours, though in my experience that also means you might get a “sorry Dave” question to multiple users if the first one had somehow triggered a refusal.

In between. I’ve seen people using Claude code to write the MCP tools where the tools exist and are deterministic after. The artifact is created and the marginal cost returns to zero.

The $0.91 in that Roo screenshot produced a working query and a colleague who now understands the data model. Copilot’s $0.075 produced an answer that evaporated when the session ended.

The Leverage Inversion

I went looking for data on how many people one person’s work typically serves.

The low end is Baumol’s stagnant sector. William Baumol and William Bowen, 1966: the string quartet can’t play faster. Four musicians, same duration, every performance. The performer’s labor IS the output. You can’t increase productivity because there’s nothing to optimize away. But wages rise anyway because the progressive sector (manufacturing, tech) pulls them up. So the cost of live performance, healthcare, education rises without the work itself changing, called Baumol’s cost disease.

Software was supposed to escape this. Build once, serve at scale. Zero marginal cost of replication. Naval Ravikant calls code “permissionless leverage”: it works for you at scale without needing anyone’s permission. WhatsApp served 42 billion messages daily with a team that fit in a conference room. That’s what breaking out of the string quartet looks like.

The consumption model puts something interesting on the table. The “performance” (the query, the generation, the agent session) has a real cost every time it runs. A dashboard builder at 1:500 creates an artifact once and it serves viewers indefinitely. Copilot answering questions about that same dashboard is a separate interaction for each person, each question. The leverage ratio looks different.

“Pay for what you use” was cloud computing’s promise, and it was a good one. The meter was on your infrastructure. A VM you spun up, controlled, sized, shut down. This meter is on something else. Every person who asks a question, anywhere in the org, adds to the same shared capacity pool. The consumption is emergent. It’s the aggregate of individual curiosity across the entire company, billed to a shared resource nobody is watching at the question level.

Google handled this differently. Gemini is bundled into all Workspace plans at a flat rate, $7-26/user/month depending on tier. Prices went up 16-22% when Gemini was added. No per-query metering. Everyone pays whether they use it or not. Microsoft gives you the per-query meter but lets you opt out by not enabling Copilot. Different risk transfer models for the same underlying cost. When employees have access to both Copilot and ChatGPT, 76% choose ChatGPT. Both companies’ real moat is forced bundling with the productivity suite.

The Design Question

The consumption cost is proportional to how much reasoning you’re asking the LLM to do per interaction. That much seems clear from the data.

Copilot’s Q&A parser tier is a good example. Cheap pattern matching handles the easy stuff. If the user asks “what were total sales last month” and the semantic model has a measure for that, it matches directly. The LLM only fires on fallback. That tiered design keeps the per-interaction cost low for the 60-70% of questions that are straightforward. The same principle shows up in the MCP: five deterministic tools, the LLM picks which one and interprets results. Most of the work is in the tool, not the reasoning.

Contrast that with overly broad agent instructions. “You are a helpful data analyst, figure it out” with a dozen tools. The LLM is in the hot path for everything. Every instruction that could have been a tool, every decision that could have been a rule, every piece of context that could have been in the schema metadata, that’s reasoning you’re paying for per query.

People at my company are building Copilot Studio agents that encode their instructions into a prompt. That’s middle management in a different medium. Compare that to a coworker who wrote a script that bundles an entire repo into a file you can paste as context to an agent, with response parsing built in. One fills the existing pattern. The other gives people something they didn’t have before.

The “just make an agent” thing. A virtual employee that can be cloned infinitely sounds like an advantage until you notice that everyone else can clone one too. If the advantage can be replicated at marginal cost, it’s table stakes within six months. The question underneath that: what work stays valuable when everyone has agents?

Where I keep landing: general-purpose insight extraction layered on specific artifacts. Build a table, the AI uses it to answer a range of questions. Build a Gantt chart, the AI applies it to a suite of problems. The artifact is still the leverage. The AI is an interpretation layer on top. My colleague isn’t valuable because he has an agent. He’s valuable because he understands the business well enough to know which questions to ask and whether the answers make sense.

This isn’t only about LLMs either. World models, spatial reasoning, domain-specific architectures for things like protein folding. None of these are LLMs and all of them are moving fast. Yann LeCun left Meta and raised over a billion dollars betting that LLMs “will become useless within five years.” Whether he’s right about that is a different post. The design principle is the same regardless: minimize the expensive reasoning surface area, build specific tools underneath. If you build around a model type, you’re locked in. If you build around the principle, the reasoning layer is swappable.

The Turn

“Is It Okay” ended uneasy. The meter doesn’t help with that.

But the Power BI tool is important. It lets people find insights faster than I could. I’m not the bottleneck anymore. My colleague learned the domain through the MCP in ways he couldn’t have without it. He’s finding discrepancies nobody had surfaced before. That’s real. The MCP isn’t designed to teach him the business, but that’s what it does.

I entered the workforce at a time where the challenge was how to survive in a world where the model was front-load on education and hope it translates to experience before you retire. Drucker’s knowledge worker problem. Continuous learning as the strategy. Now I’m in a world where finding information is cost zero, but finding context is the whole game. The strategy was continuous learning. What’s the model for continuous experience?

Jack Dorsey cut Block from 10,000 to 6,000 employees and shipped 40% more production code per engineer. Tobi Lutke told Shopify employees to prove AI can’t do a job before asking for headcount. Salesforce stopped hiring software engineers. The investment thesis is headcount reduction. The “augmentation” framing and the “replacement” framing aren’t in tension. They’re the same companies saying both things in different rooms.

McKinsey says 6% of enterprises qualify as “AI high performers.” PwC says 56% of CEOs report AI has produced neither increased revenue nor decreased costs. S&P Global says 42% of companies abandoned most AI initiatives in 2025. IBM found that GenAI ROI averaged 31% in pilots but collapsed to 7% when scaled, below the typical 10% cost-of-capital hurdle. These numbers don’t mean AI doesn’t work. They mean the deployment model matters, and most organizations haven’t figured it out yet.

Both things are true at the same time. The consumption model changes the economics. And the tools let people do work that wasn’t possible before. There’s a version of this where the cost analysis is the whole story and the conclusion is “it’s bad.” There’s another version where the tools story is the whole story and the conclusion is “it’s great.” Neither version is honest.

The Landing

The purpose of a system is what it does.

The system isn’t designed to hide costs, but the 24-hour smoothing makes them hard to see. The system isn’t designed to invert leverage, but per-query consumption does change the math. Microsoft didn’t design Fabric to be opaque about Copilot costs. The Capacity Metrics app exists, the CU rates are published, the smoothing mechanism is documented. But the purpose of a system is what it does, and what the system does is make it very easy to not notice.

And the MCP isn’t designed to teach my colleague the business, but that’s what it does. The tool isn’t designed to surface discrepancies nobody found before, but that’s what it does.

Same principle. Both directions.

Look at the systems you’re building. Not what they’re designed to do. What they actually do.