Design: one toggle between fabric-emulator and real Fabric

Status: T0 + T1 + T2 shipped (python/fabric-target/, CI fabric-target; conformance emulator leg on every push, real leg via the secret-gated real-fabric workflow). Goal: a user’s Python functionality — SDK calls, fabric-cicd pipelines, notebooks on the notebookutils shim, dbt projects, plain requests — runs against either the local emulator family or the real Fabric service, switched by one setting, with zero code edits.

Why this is nearly free

Everything this project did to run real clients unmodified against the emulator is exactly the machinery a toggle needs, pointed the other way:

• Every client is already parameterized by an API root (FABRIC_API_ROOT_URL / DEFAULT_API_ROOT_URL), a token authority + credential (azure-identity), a storage endpoint (OneLake), and a vault URL — because that’s how the e2es aim them at localhost.
• The emulator deliberately speaks v2.0/JWKS/challenge auth exactly like production, so azure-identity, MSAL, and the SDKs cannot tell the difference — only the endpoints and the credential values change.

So the toggle is not an emulation feature; it is one resolver that turns a target name into a coherent set of endpoints + credentials, plus guardrails for the places the two worlds genuinely differ.

Where your artifacts persist — definition parts, the {name}.{type}/ + .platform source format, and the hard line that OneLake data is never in Git — is docs/46. This document resolves where to talk; that one says what you are storing, and both have to hold for a run to be portable.

The contract

One switch: FABRIC_TARGET=emulator | real (default emulator).

Resolved value	emulator (zero-config defaults)	real (from standard env)
API root	https://localhost:9443/v1	https://api.fabric.microsoft.com/v1
Token authority	entra-emulator (https://localhost:8443/{tenant})	https://login.microsoftonline.com/{AZURE_TENANT_ID}
Credential	seeded daemon SP (cccccccc-…0002 / daemon-app-secret)	AZURE_CLIENT_ID/AZURE_CLIENT_SECRET, or DefaultAzureCredential (az CLI, managed identity, browser)
OneLake	https://localhost:9443 + Host/--resolve (or az:// via Sail)	https://onelake.dfs.fabric.microsoft.com
Key Vault	azure-keyvault-emulator (https://localhost:8444)	the user’s real vault URI
TLS verify	off (self-signed family certs)	on
Workspace	by id or name against the emulator	by name (FABRIC_WORKSPACE), resolved to the real GUID at startup

Ids are the one thing that can never match across targets — so the contract is name-based: user code holds workspace/item display names; the resolver translates to GUIDs per target.

Variable names, and why there are two sets

Emulator-mode knobs are named FABRIC_*/*_EMULATOR_URL; real mode reads the standard AZURE_* names. A consumer driving both targets from one compose file writes the Azure names, because real mode leaves it no choice — so emulator mode accepts them as aliases rather than making the same URL be declared twice:

Resolved value	Preferred	Also accepted
Fabric API	FABRIC_EMULATOR_URL	FABRIC_URL
Entra	ENTRA_EMULATOR_URL	ENTRA_URL
Key Vault	VAULT_EMULATOR_URL (emulator), FABRIC_VAULT_URL (real)	AZURE_KEY_VAULT_URL
Tenant	FABRIC_TENANT	AZURE_TENANT_ID
Client id / secret	FABRIC_CLIENT_ID / FABRIC_CLIENT_SECRET	AZURE_CLIENT_ID / AZURE_CLIENT_SECRET

The FABRIC_* name wins when both are set, so the aliases change nothing for anyone already using them. They do widen one hole, which is closed explicitly: because emulator mode now reads AZURE_CLIENT_SECRET, a shell left over from a local run could carry the seeded daemon into real mode. Real mode therefore refuses the seeds by value, not by variable name — see “Seeded values never leak into real mode” below.

The family — trust direction constrains the toggle

Tokens flow one way: entra (or real AAD) issues; fabric and keyvault only validate. That means the toggle is not three independent switches — only chains rooted in a single issuer are coherent:

Combination	Works?	Why
All-emulator	✅	the default family — one trust chain rooted in entra-emulator
All-real	✅	real AAD issues; real Fabric + real Key Vault validate
Real AAD + emulated fabric/keyvault (hybrid)	✅	both emulators already accept any configured issuer — FABRIC_ENTRA_ISSUER / KV_ENTRA_ISSUER can point at login.microsoftonline.com/{tenant} today
Emulated entra + real fabric/keyvault	❌	real Microsoft services will never trust the emulator’s JWKS

So FABRIC_TARGET flips the whole chain; the one supported refinement is a hybrid profile (FABRIC_TARGET=emulator + AUTH_TARGET=real) for teams with a real AAD app registration but no Fabric capacity.

Per member, “real” resolves as:

• entra-emulator → real Entra ID. azure-identity is the toggle: the resolver builds ClientSecretCredential(..., authority=<entra-emulator>) with the seeded SP in emulator mode, and exactly DefaultAzureCredential() in real mode. Its chain order does the rest with no branching of ours: explicit AZURE_TENANT_ID/AZURE_CLIENT_ID/ AZURE_CLIENT_SECRET win when set (CI, service contexts); otherwise az login wins — tokens are minted from the developer’s own CLI session (delegated identity; workspace RBAC and Conditional Access apply to them), refresh handled by azure-identity re-invoking az. All three family scopes (Fabric, Storage, Vault) mint through the CLI. Non-Python tools follow the same split via the env emitter: fabric-cicd already uses DefaultAzureCredential internally, dbt-fabric takes authentication: CLI, azcopy takes AZCOPY_AUTO_LOGIN_TYPE=AZCLI.
• azure-keyvault-emulator → the user’s actual vault. Key Vault’s challenge-based auth does the discovery: the SDK hits the vault, reads the 401 WWW-Authenticate challenge naming the authority, and follows it — the AKV emulator implements that same challenge advertising entra-emulator’s authority, so identical SecretClient(vault_url, credential) code walks either chain. The resolver supplies only the vault URL per target (https://localhost:8444 and its default vault vs https://{name}.vault.azure.net). Vault names are the cross-target contract, exactly like workspace names. A connection references the vault by account name, not URL, so that shape is identical on both sides while the URL each target composes differs.
• Seeded values never leak into real mode. Tenant 11111111-…, the daemon SP, and daemon-app-secret are emulator-mode defaults only; in real mode the resolver requires a real credential source — env SP vars or a live az login (the DefaultAzureCredential chain probe) — and refuses to fall back to seeds. No source found → fail at startup with “run az login or set AZURE_* credentials”, never a silent seed. Since emulator mode accepts the AZURE_* aliases, “not a seed” is checked by value: if AZURE_TENANT_ID, AZURE_CLIENT_ID or AZURE_CLIENT_SECRET equals a seeded constant, real mode refuses to construct at all. A leftover shell fails loudly at startup instead of authenticating against nothing.

Deliverable A — fabric_target (Python helper, python/fabric-target/)

Published. The release workflow builds it beside the fixture wheels and attaches it to the GitHub Release, stamped at the release version:

uv pip install "fabric-target[real,sessions] @ \
  https://github.com/calvinchengx/fabric-emulator/releases/download/vX.Y.Z/fabric_target-X.Y.Z-py3-none-any.whl"

The extras matter. The core is stdlib-only and imports azure-identity (real credentials) and requests (session()) lazily, so a bare install succeeds and then fails at the first authenticated call.

Why it is published at all: while it was not, a consumer could only restate the contract, and contoso-data-platform did — losing the DefaultAzureCredential branch in the copy, so its real target demanded a client secret. That made az login unusable, a managed identity unusable, and a Fabric notebook — which has no client secret to give — unable to run the platform at all. The emulator never noticed, because it does not care which identity shows up. A contract that must be copied is a contract that gets its untested branch wrong.

Small sibling of the notebookutils shim, same env-driven style:

from fabric_target import target

t = target()                     # reads FABRIC_TARGET, resolves the profile
t.credential                     # azure.identity credential for this target
t.session()                      # requests.Session: base URL, bearer auth,
                                 # verify flag, retry-on-429 — same object
                                 # whichever target is active
ws = t.workspace("analytics")    # name → id, either target
t.session().post(f"/workspaces/{ws.id}/items", json={...})

t.onelake                        # adlfs/azure-storage-blob-ready endpoint + credential
t.vault_url                      # keyvault base for this target
t.emulator_only("clock freeze")  # raises TargetError under FABRIC_TARGET=real

Implementation notes: authority override is plain azure-identity (ClientSecretCredential(..., authority=...) — the e2es already prove entra works as an authority); verify=False only in emulator mode; the profile is resolved once and printable (python -m fabric_target show).

Deliverable B — env emitter (non-Python tools, one command)

The same resolver, exported for tools that only read env — fabric-cicd, dbt profiles, azcopy, the notebookutils shim, fab CLI:

eval "$(python -m fabric_target env real)"      # or: emulator

Emits the full coherent set: FABRIC_API_ROOT_URL, DEFAULT_API_ROOT_URL, AZURE_TENANT_ID/AZURE_CLIENT_ID/…, NOTEBOOKUTILS_* (mapped onto real endpoints in real mode), REQUESTS_CA_BUNDLE/SSL_CERT_FILE handling, and — emulator mode only — the DNS-pin guidance for hostname-strict tools (05-tls-and-hosts.md).

Guardrails — where the worlds differ on purpose

The toggle must make these differences loud, not paper over them:

1. Emulator-only surfaces hard-fail in real mode. /_emulator/* (clock, faults, portal data), forged tokens, seeded principals: the helper’s emulator_only() raises TargetError("clock control does not exist on real Fabric") rather than letting a test silently no-op.
2. Time is real. No frozen clock: LROs poll for real minutes; the helper’s session() bakes in Retry-After-honoring polling either way, so code written against the emulator’s instant LROs still behaves.
3. Real mode costs money and touches real state. Destructive verbs (workspace/item DELETE) require FABRIC_TARGET_ALLOW_DESTRUCTIVE=1 in real mode; the resolver refuses to start in real mode without an explicit FABRIC_WORKSPACE scope, so nothing ever enumerates a whole tenant.
4. Throttling exists. 429/Retry-After handling is on by default in the session (the emulator can rehearse it via fault injection).
5. RBAC is real. The SP needs actual workspace roles; the resolver’s startup probe (GET /workspaces + the scoped workspace) fails fast with a “grant your SP access” message instead of 403s mid-run.

Running an example against a real tenant, step by step

What a person actually types, and what happens at each boundary. The example is not edited: everything below is environment.

az login                                   # as YOURSELF for a trial (see below)

curl -s -H "Authorization: Bearer $(az account get-access-token \
  --resource https://api.fabric.microsoft.com -o tsv --query accessToken)" \
  https://api.fabric.microsoft.com/v1/capacities

cd examples/medallion-pyspark && FABRIC_TARGET=real FABRIC_WORKSPACE=contoso-analytics FABRIC_CAPACITY_ID=<capacity-guid> uv run --frozen python provision.py

Then, with a Key Vault, the next three steps in order: secret.py, extract_load.py, bronze.py — adding FABRIC_VAULT_URL=https://<vault>.vault.azure.net.

Step	Real Fabric	Note
provision.py	workspace + lakehouse + warehouse + workspace identity	needs FABRIC_CAPACITY_ID; the resolver refuses without it rather than creating a capacity-less workspace whose every item then fails
secret.py	secret in your vault + an AKV-reference connection	the workspace identity needs get on the vault
extract_load.py	notebookutils.credentials.getSecret, then ~170 MB into Files/landing	the brokered path, unchanged from local
bronze.py	deploys the Notebook and DataPipeline definitions, runs the pipeline	the notebook activity executes on the workspace’s starter pool; its body is already portable (spark, abfs://<ws>@onelake.dfs.fabric.microsoft.com/...)
engine.py	skips	Fabric ran the notebook itself
silver.py	deploys silver.Notebook, submits RunNotebook, verifies the Delta tables with delta-rs	the transform runs on the starter pool; this file never touches Spark
reflect.py	queries the lakehouse SQL analytics endpoint	address discovered from sqlEndpointProperties
gold.py, dq_gate.py	dbt-fabric builds the star in the warehouse	address discovered from properties.connectionString
semantic_model.py	publishes and queries the model over executeQueries	—
lineage.py	skips	the flow graph is the emulator’s own record; Purview is Fabric’s answer and a different integration

Verified against a real trial on 2026-08-11, and every one of these was a code-reading until then:

==> provisioned workspace=f2c82a4e-… lakehouse=450d5027-… warehouse=c6c4ba99-…
lakehouse  server=<opaque>-<opaque>.datawarehouse.fabric.microsoft.com
           database='lake' encrypt=True endpoint_id=803c8e33-…
warehouse  same server, database='dw' encrypt=True endpoint_id=None

Four things that only a real run could establish:

1. FABRIC_CAPACITY_ID works: the workspace came up with the trial capacity assigned, from an unmodified example.
2. A Warehouse create is ASYNCHRONOUS on real Fabric — 202 with an operation and no body, where the emulator answers 201 with the item. provision.py read None["id"] and failed three calls after the assumption that caused it. Every create now resolves an operation when one is offered (post_and_wait).
3. AZURE_TENANT_ID did not reach the az CLI credential. DefaultAzureCredential takes tenant hints for the browser, VS Code, shared-cache and workload-identity links and has none for the CLI, so a developer whose az default tenant differs from the configured one got tokens for the wrong tenant — and Fabric answered UserNotLicensed, which reads as a licensing problem rather than a tenant one. Fixed in fabric_target.
4. The lakehouse’s sqlEndpointProperties.id is a DIFFERENT GUID from the lakehouse (803c8e33… vs 450d5027…), and a Warehouse has none. That confirms the emulator is right to omit the field rather than report the lakehouse id: code using it as an endpoint id would work locally and address the wrong thing on a tenant.

Three more facts that decide whether this works, all Microsoft’s rather than ours:

• A trial capacity can only be assigned by the account that started the trial. A service principal cannot, so use az login as that user. CI uses an F-SKU capacity for the same step.
• A workspace with no capacity is created successfully and then rejects every Fabric item in it. That is why the resolver refuses instead.
• Workspace identity can be created in any workspace except My workspace, so a trial is fine; trusted access to firewalled storage additionally needs an F SKU.

Verification — the same tests, both targets

A pytest marker ties it together:

@pytest.mark.target          # runs under either FABRIC_TARGET
def test_publish_roundtrip(t): ...

• CI (every push): the marked suite runs with FABRIC_TARGET=emulator — free, deterministic, offline.
• Nightly / manual (workflow_dispatch): the same suite with FABRIC_TARGET=real, gated on repo secrets (AZURE_TENANT_ID, SP creds, a dedicated throwaway workspace). Every divergence found feeds the parity map — the toggle doubles as a fidelity oracle: the emulator’s behavior is continuously diffed against the real service.

Phasing

Phase	Lands	Proves
T0 ✅	python/fabric-target/ (resolver, TokenCredential-shaped emulator credential, guarded session + LRO poll, env emitter) + unit tests + e2e/fabric-target (CI 3-OS) + quickstart section	toggle commands are real; emulator profile CI-verified end to end
T1 ✅	conformance/ suite (7 target-agnostic tests: per-scope minting, name resolution, item+LRO lifecycle, throttle shape, both guards) — emulator leg in the fabric-target CI job; real leg in .github/workflows/real-fabric.yml (workflow_dispatch + weekly, self-skips until AZURE_*/FABRIC_TEST_WORKSPACE secrets exist)	same suite, one env var apart; divergences feed the parity map
T2 ✅	notebookutils reads FABRIC_TARGET: real mode resolves the real control plane, the real OneLake endpoint (a different host, not Host-routing), real Entra, TLS verification on, and mints via DefaultAzureCredential — the seeded dev identity is None in real mode by construction	notebook code runs unchanged locally and as a genuine Fabric notebook

Non-goals

Proxying or recording real Fabric traffic through the emulator, translating ids between targets persistently (names are the contract), emulating tenant onboarding/capacity purchase, and hiding real-mode latency or cost.