Fabric Emulator

latest release →

Build a Microsoft Fabric data product without a tenant.

A clean-room Go emulator of Microsoft Fabric: the control plane, a real OneLake ADLS Gen2 and Blob data plane, a T-SQL warehouse over TDS, native Livy sessions on a real Spark engine, Data Factory pipelines, Apache Airflow jobs on a real scheduler, KQL eventhouses on Microsoft's own engine, and Eventstream on a real Kafka broker. It validates every incoming bearer token against entra-emulator's JWKS, the way real Fabric validates against Entra.

The same pipeline runs unmodified here and against real Fabric, one environment variable apart. Switching target is configuration, not a code path.

Every capability below carries the grade the parity ledger gives it, and every figure on this page is read from that ledger at build time rather than typed into the page.

distinct witnesses, each one a thing that runs
parity claims, every one naming a witness
/ledger rows graded Real, not merely shaped
end-to-end suites, one per real client
/Spark probes passing on the default engine
rows compared against a real Fabric tenant

The problem this solves

Delivering a data product on Fabric means proving it on Fabric, and Fabric is a tenant with a capacity attached. That makes the feedback loop slow, billed, shared, and awkward to reset.

A tenant is the only place it runs

Provision, wait, run, read the result, then try to get the workspace back to where it started. Resetting the state is usually the expensive half, and two people on one capacity cannot both do it.

The whole surface on your machine, in seconds, with state you can delete

An API shape is not the work

A mock that answers 200 to a Copy activity tells you nothing about whether the bytes moved, whether the Delta commit landed, or whether the principal was allowed to write. Those are the failures that reach production.

Real signed JWTs, real Delta bytes on disk, real RBAC, a real pipeline interpreter

Green against a mock proves only the mock

An emulator that agrees with its own tests is a mirror. The evidence worth having comes from clients nobody here wrote, run unmodified against the emulator.

Microsoft's own tools drive it in CI, and the ledger names which one covers which claim

What it does

Each capability carries its grade from the parity ledger and the evidence behind it. Evidence tiers are ranked: an unmodified external client in CI outranks Microsoft's own client in process, which outranks our own tests on both ends.

Control plane witnessed

Workspaces, items and the documented typed collections, folders, role assignments enforced from the validated bearer principal, deterministic long-running operations, connections and the credential model, git integration and deployment pipelines. A Fabric Core MCP server sits on the same handlers at POST /v1/mcp/core.

fabric-cicd, the Fabric CLI and the microsoft/fabric Terraform provider, all unmodified in CI

OneLake witnessed

One store behind two surfaces: the ADLS Gen2 DFS API and the Blob API, with managed folders, put-if-absent Delta commits, shortcuts, and read-through shortcuts to real Amazon S3 through a Connection.

delta-rs, the Azure Blob and Data Lake SDKs, and azcopy drive it. docs/08

T-SQL warehouse over TDS witnessed

Entra FedAuth is terminated at the emulator and the session is byte-spliced to a SQL Server sidecar, so a driver speaks real TDS end to end. Row-level and column-level security, dynamic data masking and time travel are enforced rather than described.

three independent drivers: go-mssqldb, ODBC Driver 18, mssql-python. dbt-fabric 1.11 runs through it

Spark, on a real engine

Native Livy sessions, interactive and high concurrency, with notebook cell execution, the notebookutils shim, and runMultiple DAGs carrying Fabric's failure contract. Sail, a Rust Spark Connect engine, is the default and starts in seconds; the JVM overlay is one compose flag away when a test needs the RDD surface or Java UDFs.

the engine matrix is generated by running both engines, never written by hand

Data Factory pipelines witnessed

A pure-Go interpreter for Fabric's own activity model, with the expression language and the control-flow set. Copy moves real bytes, Lookup reads real rows, Script runs real T-SQL, Web makes the real HTTP call, and per-activity retry and timeout behave as documented.

external-connector leaves such as ServiceNow are stubbed successes, and the ledger grades them that way

Airflow, KQL and Eventstream witnessed

ApacheAirflowJob items run on genuine Apache Airflow at the versions Microsoft documents, with no orchestration emulation at all. Eventhouse and KQL Database execute on Microsoft's own kustainer behind --profile rti, and Eventstream provisions a real Kafka topic behind --profile eventstream.

the last two are opt-in profiles, so they are real but not part of a default docker compose up

One toggle to real Fabric shipped, unwitnessed in CI

FABRIC_TARGET=emulator|real resolves the API root, the token authority, the credential and the OneLake endpoint as one coherent set, so a pipeline, a dbt project or a fabric-cicd run moves between the two without a code edit.

the conformance suite has been run against a real tenant from a developer machine; the CI leg is secret-gated and has not. docs/21

Runtime conformance

Seven contracts a Fabric notebook runtime should hold, checked on each backend and confirmed out of band, so the engine that wrote is never the one that confirms. of applicable cells are proven so far: the write-landing contract, on all three backends. The rest need a running notebook session and are recorded red rather than skipped quietly.

the matrix is generated; a gap points at the prose that closes it

Deliberately not implemented honest 501

Dataflow Gen2 manages its item and definition but refuses to execute, because the Power Query M engine is not something to fake. Purview's built-in classifiers are proprietary detection patterns published in no contract, and that row can never move. Graph, Real-Time Hub, Copilot and the Workload Dev Kit are absent.

each refusal is a named error, not a plausible empty result. parity ledger

Compared against real Fabric

The differential workflow is written and the toggle it needs is shipped, but the workflow is opt-in by repository secrets and skips loudly without them. No parity row cites it. Until one does, every green row on this page rests on Microsoft's published contracts and on unmodified third-party clients, which is the strongest evidence available here and still not Fabric.

.github/workflows/real-fabric.yml, and no witness in the manifest names it

Honest status

The figures above are not typed into this page, and the build fails if they are.

Where the evidence comes from

Every capability the ledger grades green must name a witness in docs/witnesses.json, and check_witnesses.py --strict fails CI if a named witness stops existing. A renamed test that leaves a green row behind is a lie the reader cannot see, so the manifest is what makes the claim checkable at all.

ci:a CI job driving a real, unmodified external client. The strongest tier. sdk:Microsoft's own client does the talking, in process rather than over a network. go:a Go test: real HTTP, real signed JWTs, real RBAC, but our own client on both ends. py:a Python test, same standing as the Go tier. boundary:the claim is scoped by a documented limitation, with the reason recorded.

A witness whose name exists is not a witness that ran. Witnesses that can skip, such as the warehouse tests that need a real SQL Server, must be declared with their reason, and a declaration that has gone stale fails the same gate. What none of this proves is that a witness asserts its claim, or that the code behind the claim executes at all. Coverage answers the second, and a green row with no coverage behind it is recorded in AUDIT.md rather than smoothed over.

The engine and conformance matrices are generated by running the probes, never edited, so a row cannot claim a behaviour the runtime lacks. The parity map argues where each boundary sits and why, and it is the only place a capability is graded.

Get it

One static Go binary with no CGO and no runtime dependencies, or one container. Tested on Linux, macOS and Windows.

docker pull ghcr.io/calvinchengx/fabric-emulator:latest

Homebrew, winget, go install and the release archives are all in installation. State lives in /data, so mount it to keep it and delete it to start over.

Run the whole thing

One command brings up the family wired together, with real engines attached by default.

git clone https://github.com/calvinchengx/fabric-emulator
cd fabric-emulator
make doctor    # checks the runtime, the tools and the ports before anything starts
make up        # entra-emulator, key vault, fabric-emulator, plus Spark and SQL Server

Then run a full medallion against it. The example asserts its own results, so a run that completes is a passing test rather than a demo script.

cd examples/medallion-advanced-pyspark
uv sync --frozen && uv run python pipeline.py

Three source systems land, get conformed to bronze, resolve to one customer identity in silver, and reach a Warehouse star that Power BI queries: PySpark on a real engine for bronze and silver, dbt-fabric over real TDS for gold. Follow it live at https://localhost:9443/#flow. Give the container runtime 8 GB for the default services, or 13 GB with governance and Airflow attached: the running modes page has the per-service measurements.

Start from the quickstart for the five-minute version, or the end-to-end tutorial for the walked-through one.