Skip to content

Publishing a dashboard

The end of the promotion path. A candidate the promoter released (see 12-promotion) becomes a dashboard in every tool DAS_DASHBOARD_TARGETS names — created under the identity of the person who asked where the tool can carry it, checked against the query it came from, and recorded in OpenMetadata with lineage to the tables it reads.

Power BI is the first target and for a long time it was the only one, which is why most of this page is about TMSL and PBIR. What is not Power BI’s is the order — build, create, evaluate, compare, record — and the Plan that order operates on. See 15-adding-a-dashboard-target for the seam, and §20 of 00-plan for why it exists.

Terminal window
python -m promoter.run --from compose # what keeps being asked
python -m publisher.run --user erin@… # publish the top candidate

Who publishes is configuration: a person passes --user, a scheduled job reads DAS_PUBLISH_USER. With neither, the job stops before minting any token and says which to set — no identity is built into the code.

DAS_DASHBOARD_TARGETS names which are live; a name nothing is built for is refused at startup rather than silently doing nothing. Each candidate is offered to every configured target, and each target’s accepts() returns the reason it cannot take one — so a question that recurs against PostgreSQL lands in Superset while Power BI explains, in a sentence, that Direct Lake binds to a Fabric item.

Target Artefact Verified through authz_tier Witnessed
Power BI Direct Lake semantic model (TMSL) + PBIR report its own DAX, evaluated user — on-behalf-of to Fabric e2e.run phase16
Apache Superset virtual dataset + chart + dashboard POST /api/v1/chart/data service — no OBO exists e2e.run phase19
Tableau .twb workbook VizQL Data Service user — connected app, direct trust e2e.run phase20 generator only

Three properties hold across all of them, and they are the reason the seam is worth having rather than three integrations:

The tool never sees more than the template. A Direct Lake binding, a virtual dataset and a custom-SQL relation are three spellings of the same decision: the target receives plan.comparison_sql, not a grant on the tables behind it. It matters most for Superset, which reads with one shared credential — nothing else bounds what that credential could reach.

The tier is named honestly. Power BI and Tableau carry the asking person into the tool. Superset cannot, so it says service, and the asker is recorded as an OpenMetadata owners reference instead. No target is labelled user because we would like it to be.

A target that renders a QUERY reads the measure back rather than re-applying its function. The guarded SQL has already aggregated, so the dataset holds one row per group with the answer in it. Re-aggregating is harmless for SUM and AVG and wrong for COUNT, which returns 1 for every group — a plausible number on a dashboard that nobody questions. It is also the one class of error a second implementation cannot catch, because both would re-aggregate identically and agree. Superset uses MAX, Tableau Sum; Power BI is exempt only because it binds a model over the source tables instead.

Tableau has no container, so the phase splits where the evidence does. The workbook, the VDS query and the connected-app token are pure functions of the Plan, recorded in publisher/contract/cases.json and witnessed on every push. publish() and evaluate() refuse by name, and parity carries the live hop red. make tableau-check asks a real site whether it accepts a token this repo signed — the first hop, and the only part a person can check by hand.

The definitions are deterministic functions of the template and the catalog. A dashboard that says something different on Tuesday is not a dashboard anyone can rely on, and the title, the measures, the visual and the slicers are all decided by rules short enough to argue with:

Decision Rule
Title the catalog’s name for each measure and dimension (see 12)
Measure the template’s aggregate over the same column
Visual no dimensions → card · one → bar chart · several → table
Slicers one per recorded slot, never with a default

The slicer rule follows from the promoter keeping no literals: there is no default to restore, and inventing one would put a filter on the page nobody chose and everybody would read as the organisation’s own.

Direct Lake, and the copy that isn’t made

Section titled “Direct Lake, and the copy that isn’t made”

The model binds to the warehouse and the engine reads it in place. The emulator also accepts an import model carrying its rows inside the definition as a data.json part, and that is the wrong shape for a reason the sibling project wrote down before we met it: a model with an embedded copy of its own source is two things that can disagree, and Power BI Desktop opens such a model to empty tables with nothing in the definition saying why.

Direct Lake partitions need compatibilityLevel 1604 and are refused below it. Since fabric-emulator 0.21.0 they resolve over a Warehouse too, so one definition answers DAX on both targets — no emulator-only path.

The publish sequence is create → evaluate → compare → record, and the compare step can refuse:

  1. the DAX measure is evaluated through executeQueries, the same REST surface Power BI Desktop and SemPy use;
  2. the template’s SQL runs through the executor — guard, access rules and all — because a verification that queried the database directly would be checking a path nobody uses;
  3. the two answers are compared as sets, rounded to four places: ordering is not a disagreement about a number, and a check that failed on the fifteenth decimal is one nobody keeps.

Only a dashboard whose two answers agree is recorded in the catalog. A report that quietly disagrees with the query it was promoted from is worse than no report, because people stop checking a dashboard after the first week.

The SQL side drops the slot predicates, because the slicers open unset. Running the template as-is would compare an unfiltered dashboard against a filtered query and call the difference a defect.

What can be expressed, and what is refused

Section titled “What can be expressed, and what is refused”

The measure has to be one the DAX evaluator can express: SUM, AVERAGE, COUNT, COUNTROWS, DISTINCTCOUNT, MIN, MAX, DIVIDE. Notably not CALCULATE — which is why the filter belongs to the report rather than the measure. Anything else is refused rather than approximated, along with a column no table owns and a column two tables own. Guessing which table a column belongs to produces a model that answers, which is the failure nobody notices.

Publishing is privileged, and Fabric says so

Section titled “Publishing is privileged, and Fabric says so”

Creating an item needs Contributor on the workspace; asking a question needs only Viewer. The first attempt to publish was refused for exactly that reason — as the asking user, which is the property the whole service advertises. So the seeded personas now include one who may publish:

Persona Role Workspace Can publish
alice Data.Analyst Viewer no
carol Data.Finance Viewer no
erin Data.Admin Contributor yes
bob no

That is how it works in an organisation: an analyst proposes a dashboard, someone who owns the workspace publishes it. e2e/run.py witnesses both sides — a viewer is refused, a contributor succeeds.

Publishing reaches two more resources as the user, and each needs its delegated scope exposed on its resource app:

  • https://api.fabric.microsoft.com/user_impersonation — create the items
  • https://analysis.windows.net/powerbi/api/user_impersonation — evaluate the DAX

Without them the exchange fails with AADSTS70011, which names a scope rather than a resource and so reads like a typo in the string instead of a missing registration. seed/apps.py exposes every scope this deployment exchanges for, in one list, because the failure is identical whichever one is missing — the Databricks adapter hit it in phase 13 and the publisher hit it again here.

executeQueries takes a Power BI token, not the control-plane one. A surface that accepted either would teach the wrong thing about Fabric’s auth model, and the emulator enforces the distinction.

Rendering. The emulator persists a report definition byte-for-byte and does not interpret it — deliberately, since an emulator that parsed a report definition would be claiming knowledge of a format whose renderer it does not have. That the report draws correctly is prod-only, and docs/parity.md says so.

Why this is a job, and the executor is a service

Section titled “Why this is a job, and the executor is a service”

They look similar — both speak to a governed system on a user’s behalf, both have an adapter seam, both refuse rather than guess. They are not the same kind of thing, and the difference decides how much machinery each deserves.

Executor Publisher
Shape a service on the hot path of every question a job, run by a person or a schedule
Load concurrent, latency-bound, measured by k6 one candidate, a few REST calls, seconds
Identity the on-behalf-of hop, every user, every call one exchange per publication
Adapter SourceBackend — read from an engine DashboardTarget — write to a tool, then read back to verify
Contract services/contract/openapi.json + the conformance suite publisher/contract/plan.schema.json + recorded artefacts

The Go executor exists for two reasons: to answer a performance question (§4, row 4 of the plan), and to prove the MCP contract is specifiable rather than being “whatever the Python does”. That is why access.go, tagindex.go and vaultref.go all had to be ported — they sit inside the service.

The publisher has no load to compare, so the first reason does not apply, and one Python implementation is all it needs. publisher-go/ exists for the second reason alone. §18 claims the definitions are deterministic functions of the template and the catalog; a sentence is not a check. A second generator, held to the same recorded bytes, is what turns that claim into one. Where the two disagree, the contract was underspecified — which is the finding, and the reason to have built it.

Only the pure part is ported. The REST plumbing — Fabric, on-behalf-of, Superset’s login, the catalog writes — stays Python: a second implementation of that would have nothing to disagree about.

Terminal window
uv run python publisher/contract/gen_cases.py # re-record
make publisher-contract # regenerate, diff, hold Go to the bytes

publisher/contract/cases.json records two things.

Cases — for each candidate, the executor’s column lists, the Plan built from them, and every artefact each target emits, canonically serialised, so two generators either match to the byte or name the line they disagree on.

Bindings — what table_of decides before there are any artefacts: which table a column belongs to, or the exact message explaining why it cannot be decided. Refusals are recorded beside successes, because a corpus of only successes passes against a function that never refuses anything, which is precisely what table_of exists to prevent. A divergence here surfaces as one generator refusing a candidate the other publishes — which no comparison of artefact bytes can catch, since there are no bytes.

CI regenerates and diffs both, so the file cannot drift from the Python it describes; the same discipline as services/contract/guard_corpus.json.

Every one of these was found by the machinery rather than by reading:

Found by What
the first contract run Go’s Canonical kept struct field order while sorting map keys — one value canonicalising two ways depending on how the caller declared it
FuzzTableOfNeverGuesses a self-join was refused as “ambiguous across ['dbo.a', 'dbo.a']” — ambiguous with itself. Unreachable today only because the promoter’s canonicaliser deduplicates first, which the function never said it relied on
the first bindings run Go’s TableOf did not strip a t0. alias where the Python did. The artefacts matched anyway, because DAX() happens to pass already-bare columns — a divergence waiting for the first caller that used it differently
a stripped-environment test the generator imported publisher/fabric.py, which built a Credential at import, so the pure half needed a live identity. CI would have failed on a missing secret rather than on a byte difference

The properties are cheap and worth more than the cases. Canonical must be a fixed point over its own output; TableOf must bind a column only when exactly one distinct table owns it. Both are about forty lines and both found something the same day they were written.