Quickstart

Five minutes from nothing to a workspace, an item, and a file in OneLake — no tenant, no capacity. Every value below is a seeded dev default from entra-emulator; nothing needs registering.

0. Prerequisites — one line per platform

You need a container runtime with Compose v2, plus GNU Make for the convenience targets. Python 3 is optional (only make spark and make seed use it). Pick your platform:

# Linux — engine + compose plugin, then join the docker group
curl -fsSL https://get.docker.com | sh && sudo usermod -aG docker "$USER" && newgrp docker

# macOS — Make and Python come with the Xcode CLT; then any runtime, e.g. Colima
xcode-select --install && brew install colima docker docker-compose && colima start --memory 8

# Windows — Git supplies sh.exe + grep/awk/curl; ezwinports supplies make
winget install Git.Git; winget install ezwinports.make

Then clone, and confirm the machine is actually wired up before starting anything:

git clone https://github.com/calvinchengx/fabric-emulator
cd fabric-emulator
make doctor

It checks the shell tools, a runnable Python, the docker CLI and the daemon behind the active context, and the ports the stack publishes — and names what is missing rather than letting it surface later as a broken recipe or an unreachable socket. Allow the runtime 8 GB of memory — enough for the six services below while they are working; they idle at about 530 MB. make up adds OpenMetadata and Airflow and wants 13 GB. Per-service measurements, and why idle and working differ so much, are in 27-running-modes.md.

Full per-platform detail — including Rancher Desktop’s context selection on Windows and the Apple-silicon sidecar constraints — is 26-platform-setup.md.

1. Start the family — one command

docker compose up          # or: make up (which also adds OpenMetadata)

That single command brings up the whole emulator family with Spark-compatible compute attached (the override file auto-loads):

Service	Port	Role
entra-emulator	https://localhost:8443	the STS — issues every token
azure-keyvault-emulator	https://localhost:8444	Key Vault data plane (secrets, AKV references, notebookutils.credentials.getSecret)
fabric-emulator	https://localhost:9443	Fabric control plane + OneLake + portal
sail	sc://localhost:50051	LakeSail’s Sail — the Spark engine (Rust Spark Connect, no JVM)
spark-agent	—	statement executor behind native Livy / RunNotebook, runs on Sail
sqlserver	:1433 via fabric	the T-SQL/TDS warehouse surface

keyvault and fabric both validate bearers against entra’s JWKS — the same trust relationships as production Azure. Everything serves self-signed TLS, hence -k below.

To confirm the stack is actually usable (not just that containers exist):

./scripts/status.sh      # or: make status  (works on Windows too)

It rolls up container health, the HTTP surfaces, and emulator state in one view, and exits non-zero when something is wrong. It reports what docker compose ps cannot: a service with no healthcheck (sail, spark-agent report running, never healthy, so serving is unverified), and a container that is up and healthy while attached to no network — the state Docker leaves behind when a port bind fails during creation. Compose then reuses that container because it looks healthy, and every peer fails DNS on its name.

The engine is LakeSail’s Sail — there is no JVM anywhere in the stack. Sessions start in milliseconds, Delta is native Rust, and the same stack is available with explicit file flags (naming -f files skips the auto-loaded override, so use the compute overlay to keep the full stack):

docker compose -f docker-compose.yml -f docker-compose.compute.yml up

Any PySpark client connects directly with no JVM installed:

# pip install "pyspark-client==4.2.0"   (the thin Connect client — no JVM)
from pyspark.sql import SparkSession
spark = SparkSession.builder.remote("sc://localhost:50051").getOrCreate()
spark.sql("SELECT 1").show()

Sail is the default local engine. The slower Spark 3.5 JVM compatibility oracle is test-only and does not change this stack.

Contract-only, no engines: docker compose -f docker-compose.yml up (naming the file skips the override) — lightest start; Spark/SQL surfaces answer with honest 501s.

Without Docker: run entra-emulator locally (see its quickstart), then

go run ./cmd/fabric-emulator \
  -entra-issuer "https://localhost:8443/6f89cf12-978b-4d23-ac18-9ef0c127cf87/v2.0" \
  -entra-tls-insecure

2. Mint a Fabric-audience token

entra-emulator seeds a confidential daemon app. Client credentials against the Fabric resource:

TOKEN=$(curl -sk https://localhost:8443/6f89cf12-978b-4d23-ac18-9ef0c127cf87/oauth2/v2.0/token \
  -d grant_type=client_credentials \
  -d client_id=00d88624-f0d7-46f6-a641-6232c2608928 \
  -d client_secret=daemon-app-secret \
  -d scope=https://api.fabric.microsoft.com/.default | jq -r .access_token)

(The legacy https://analysis.windows.net/powerbi/api/.default scope works too — most fabric-docs samples use it, and both audiences are accepted.)

3. Create a workspace — and meet the LRO

Nearly every Fabric mutation is async: 202 Accepted + poll. The emulator implements this faithfully:

curl -sk -D- https://localhost:9443/v1/workspaces \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"displayName": "quickstart"}'

A 201 returns the workspace directly (create is one of the sync paths); note the id — and that capacityId is already set: the emulator seeds a default capacity and auto-assigns it, so tools like fabric-cicd that refuse capacity-less workspaces work out of the box.

Async calls (item create with a definition, git sync, …) return 202 with x-ms-operation-id and Location headers; poll until the status leaves Running:

curl -sk https://localhost:9443/v1/operations/<operation-id> \
  -H "Authorization: Bearer $TOKEN"

By default operations complete on the next poll. Pin them Running with -lro-delay or the clock control to test polling loops.

4. Create an item

curl -sk https://localhost:9443/v1/workspaces/<workspace-id>/items \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"displayName": "lake", "type": "Lakehouse"}'

Typed collections (/lakehouses, /notebooks, …) serve the same items — see the control-plane API.

5. Write a file into OneLake

The data plane wants a Storage-audience token and is Host-routed at onelake.dfs.fabric.microsoft.com (the self-signed cert covers that name — see TLS & hosts):

STOKEN=$(curl -sk https://localhost:8443/6f89cf12-978b-4d23-ac18-9ef0c127cf87/oauth2/v2.0/token \
  -d grant_type=client_credentials \
  -d client_id=00d88624-f0d7-46f6-a641-6232c2608928 \
  -d client_secret=daemon-app-secret \
  -d scope=https://storage.azure.com/.default | jq -r .access_token)

OL="https://onelake.dfs.fabric.microsoft.com:9443"
R="--resolve onelake.dfs.fabric.microsoft.com:9443:127.0.0.1"

# create, append, flush — the ADLS Gen2 protocol
curl -sk $R -X PUT "$OL/<workspace-id>/<item-id>/Files/hello.txt?resource=file" -H "Authorization: Bearer $STOKEN"
printf 'hello onelake' | curl -sk $R -X PATCH "$OL/<workspace-id>/<item-id>/Files/hello.txt?action=append&position=0" -H "Authorization: Bearer $STOKEN" --data-binary @-
curl -sk $R -X PATCH "$OL/<workspace-id>/<item-id>/Files/hello.txt?action=flush&position=13" -H "Authorization: Bearer $STOKEN"
curl -sk $R "$OL/<workspace-id>/<item-id>/Files/hello.txt" -H "Authorization: Bearer $STOKEN"

Fabric-audience tokens are rejected on the data plane and vice versa, matching real OneLake. Managed-folder rules apply — try to DELETE /Files itself and watch it refuse (OneLake).

Governance (optional): OpenMetadata

Data catalog/governance over the same state, one flag — nothing runs (or pulls) without it:

docker compose --profile governance up        # + OpenMetadata at http://localhost:8585
docker compose run --rm govern-ingest         # catalog workspaces → lakehouses → Delta tables

Schemas are read from the real Delta logs in OneLake, so governance sees exactly what your pipelines wrote. Details: 22-openmetadata.md.

Run it on a fresh stack and there is nothing to catalog yet (the seed is one capacity row, 06), so govern-ingest first creates a small demo: a workspace, a lakehouse, and a real Delta table written via delta-rs. It fires only when the emulator is completely empty, so it never touches state you seeded yourself, and skipping straight to steps 3 to 5 above gets your own data cataloged instead. GOVERN_SEED_DEMO=0 turns it off and restores the plain “nothing to catalog” exit; GOVERN_DEMO_WORKSPACE, GOVERN_DEMO_LAKEHOUSE and GOVERN_DEMO_TABLE rename what it creates.

Same code against real Fabric — the toggle

Python code written against the emulator runs against the real service by flipping one env var — the fabric-target package resolves endpoints + credentials; your code holds names and never branches:

pip install ./python/fabric-target        # once

export FABRIC_TARGET=emulator             # local (the default — zero config)
python my_pipeline.py

az login                                  # real: your own identity…
export FABRIC_TARGET=real
export FABRIC_WORKSPACE=my-workspace-name # …scoped to one workspace, always
python my_pipeline.py                     # same code

from fabric_target import target
t = target()
ws = t.workspace("analytics")             # names, not GUIDs — they differ per target
s = t.session()                           # authed, TLS-aware, 429-honoring
s.post(f"/workspaces/{ws.id}/items", json={"displayName": "nb", "type": "Notebook"})

Real mode refuses to start without a credential source (az login or AZURE_* vars) and never falls back to the seeded dev values. Env-only tools get the same switch via eval "$(python -m fabric_target env real)".

Where next

• Change engine, go lite, or point at real Fabric: running modes.
• Point the real fabric-cicd tool at the emulator: testing with fabric-cicd.
• Freeze time and inject faults: testing.
• Every endpoint: control-plane API and OneLake.
• Eventstream (Kafka, Lakehouse dest, Reflex dest): 51-eventstream-kafka.md.
• Optional full DAX against msmdsrv: 52-msmdsrv-hosts.md.
• What is real vs emulated: parity map.