How the emulator’s surface maps to real Fabric (as documented at
learn.microsoft.com/fabric /
MicrosoftDocs/fabric-docs), and
— the point of this table — whether real work happens or just the API shape.
The emulator’s design bet is that the durable, testable surface is
contracts + storage + identity + orchestration, and those are done for real
(real signed JWTs, real Delta bytes on disk, real RBAC, a real pipeline
interpreter, real cross-engine SQL, real Livy high-concurrency session packing).
The heavyweight or proprietary compute engines are either bring-your-own
(Spark behind the Livy proxy — which is how Fabric itself layers a Livy endpoint
over Spark) or honestly stubbed.
“Real via our own wire-protocol implementation.” A row is 🟢 Real not
only when an external engine/client does the work, but also when the emulator
itself implements Fabric’s wire protocol and the logic behind it — so a real,
unmodified client gets byte- and behaviour-identical responses. Fabric’s
control plane, OneLake’s ADLS/Blob surfaces, the Data Pipeline expression
language + control flow, and the Livy high-concurrency session-packing layer
are all in this category: no engine is being proxied, yet the observable
contract matches real Fabric because we built the protocol, not a mock of it.
Where a row’s execution still needs a heavyweight engine (a REPL’s Spark
statements, a notebook’s cells), that part is split out as 🟠 BYO-engine or 🔴.
| Meaning |
|---|
| 🟢 Real | Genuine work: real signed JWTs, real bytes on disk, a real engine/client computes, real logic enforced — no pretending. |
| 🟡 Emulated | Faithful API contract + persisted state, but no engine — status is clock-derived / management-only. |
| 🟠 Bring-your-own-engine | Real when a real external engine is attached (Spark via the Livy proxy; notebook cells on the Spark sidecar); contract-only (honest 501) otherwise. |
| 🔴 Not implemented | Honest 501 or absent. |
| Fabric feature | Emulator | Type |
|---|
| Workspaces CRUD | Full. Display names are unique tenant-wide — duplicates 409 WorkspaceNameAlreadyExists (uniqueness per the REST reference; fabric-docs covers workspace naming portal-side only) | 🟢 Real |
| Items CRUD + 12 typed collections | Full. Display names are unique per (workspace, type) — duplicates 409 ItemDisplayNameAlreadyInUse; names stay reusable across types, which is why OneLake addresses items as name.Type | 🟢 Real |
| Role assignments / workspace RBAC | Enforced from the validated bearer principal | 🟢 Real |
| Folders | Full | 🟢 Real |
| Capacities (list, assign / unassign) | Full state, no billing/SKU enforcement | 🟢 Real state |
| Long-running operations (202 → poll) | Clock-derived | 🟡 Emulated |
Item job execution (jobs/instances) | Generic items: status clock-derived. DataPipeline jobs really run the interpreter (see Data Factory) and set terminal status from the run | 🟡 Emulated / 🟢 Real (pipelines) |
| Fabric feature | Emulator | Type |
|---|
| Entra OAuth2 tokens / JWKS / client-credentials | entra-emulator mints real signed JWTs | 🟢 Real |
| Workspace managed identity handshake | Provisioned via entra admin API; the identity’s own token passes RBAC | 🟢 Real |
| Key Vault references in connections | Resolved against azure-keyvault-emulator | 🟢 Real |
| Tenant settings / audit / admin-portal APIs | — | 🔴 Not implemented |
Purview / sensitivity labels (governance/) | — | 🔴 Not implemented |
| Lineage (catalog graph) | Via the optional OpenMetadata profile: OneLake shortcut edges and executed pipeline Copy source→sink edges are persisted exactly and witnessed in OM’s graph API. Notebook/Script code is not guessed. Catalog SSO can also be pointed at entra-emulator (22-openmetadata.md) | 🟢 Real (shortcuts + Copy) |
| Fabric feature | Emulator | Type |
|---|
| ADLS Gen2 DFS surface (create → append → flush, ranged read, list) | Full, incl. the x-ms-range dialect | 🟢 Real (real bytes) |
| Blob surface | Full | 🟢 Real |
| Delta commits (put-if-absent atomicity) | Real; -race-tested concurrent-commit race | 🟢 Real |
| Shortcuts (OneLake → OneLake) | Symlinks with target-side RBAC (trusted-workspace-access) | 🟢 Real |
| Shortcuts to external targets (S3 / ADLS Gen2 / Dataverse) | ADLS Gen2 and Amazon S3 HTTP(S) read-through with Connection-backed Anonymous/Basic/Key/SAS credentials; Dataverse remains an explicit 501 | 🟢 ADLS/S3 reads / 🔴 Dataverse |
| Fabric feature | Emulator | Type |
|---|
| Lakehouse item + Tables/Files storage | Full (via OneLake) | 🟢 Real |
| Notebook authoring / definition round-trip | Full | 🟢 Real |
notebookutils / mssparkutils (fs, credentials, getSecret, lakehouse, runtime) | Functional stdlib shim (python/notebookutils) | 🟢 Real |
| Spark session / statement / batch via the Livy API | Native termination (--spark-agent-url): the emulator implements the Livy contract and drives a persistent statement-executor agent. The default agent is a PySpark Connect client of Sail, not Apache Spark. An external Livy backend remains configurable with --spark-livy-url | 🟢 protocol / 🟠 Sail execution |
| Notebook cell execution | The emulator parses and records the notebook run, resolves attached lakehouse/Environment metadata, and Sail executes cells by default. The same fixture runs on Spark 3.5 JVM and proves unqualified saveAsTable/spark.table bind to OneLake Tables/ | 🟢 orchestration+binding / 🟠 Sail subset |
| Livy High-Concurrency (5-REPL) sessions | Fabric’s packing layer is implemented directly: sessionTag packing, 5-REPL cap + spill, independent lifecycle and slot reuse. Statements use Sail by default, so engine compatibility is limited to the Spark Connect subset | 🟢 protocol / 🟠 Sail execution |
| Environments | Run binding resolves Python requirements, Spark properties, and JAR declarations. Python packages are provisioned per run; config is applied to the real session; JAR-bearing runs explicitly require JVM Spark | 🟢 portable subset / 🟠 engine-specific JARs |
| Spark Job Definitions | V1 definition/main/arguments/libraries parsing, attached lakehouse+Environment resolution, Pending→Completed/Failed callback lifecycle, and real Sail/JVM execution witness | 🟢 orchestration / 🟠 selected engine |
The engine behind the agent is Sail (Rust
Spark-Connect, no JVM). Every row is probed in CI (e2e/sail), not
inferred — the fidelity deltas a Fabric notebook author actually hits.
A Spark 3.5 JVM image (docker/spark-runtime, Fabric Runtime 1.3’s engine
baseline) exists as a CI compatibility oracle (e2e/spark-jvm), and the
statement agent still has its classic-session path. It is also exposed as a
user-facing overlay — docker compose -f docker-compose.yml -f docker-compose.override.yml -f docker-compose.spark-jvm.yml up swaps the
statement agent onto it — so the JVM-only rows below are graded 🟠
(BYO-engine): real with that overlay attached, unavailable on the default
engine. Verified live, not inferred: sc.parallelize([1,2,3,4]).map(x*2) .sum() returns 20 through the Livy agent, and spark._jvm.io.delta.tables .DeltaTable resolves.
| Notebook pattern | Emulator (Sail) | Type |
|---|
abfss://…@onelake.dfs.fabric.microsoft.com/… production paths | Work unmodified (endpoint override routes the Hadoop URL form) | 🟢 Real |
| Delta write/read/append; SQL over temp views | Full | 🟢 Real |
Time travel option("versionAsOf", n) | Works (SQL VERSION AS OF is a Sail gap) | 🟢 Real / 🔴 SQL form |
MERGE INTO | Works against a registered table target (CREATE TABLE … USING delta LOCATION); path-based delta.`az://…` merge targets don’t resolve | 🟢 Real (registered) / 🔴 path target |
createDataFrame(local_rows) | Works (runners preset localRelationSizeLimit) | 🟢 Real |
sc / RDD API / spark._jvm | Fidelity inversion: works on real Fabric, impossible on Spark Connect — the agent binds sc to a guide-rail stub that raises a clear pointer instead of NameError. Restored by the JVM overlay (classic session): sc.parallelize(…).map(…).sum() verified | 🔴 default / 🟠 JVM overlay |
DML row-count envelopes (INSERT/MERGE counts) | Statement executes; DataFusion’s uint64 count is absorbed as an empty result by the SQL agent | 🟡 Emulated envelope |
Structured streaming, OPTIMIZE/VACUUM, Java/Scala UDFs | Execution fails on Sail v0.6.6; available on the JVM overlay (Delta JVM classes verified present) | 🔴 default / 🟠 JVM overlay |
CDF options, spark.jars | Accepted but inert on Sail: CDF returns a normal snapshot and JARs have no classloader. The JVM overlay has a real one (verified) | 🔴 default / 🟠 JVM overlay |
| Concurrent Delta overwrite writers | Two independent Connect sessions race at one barrier; one commits and the other receives a transaction failure from the conditional Delta-log create | 🟢 probed conflict rejection |
| Fabric feature | Emulator | Type |
|---|
| SQL-analytics-endpoint semantics over lakehouse Delta | DuckDB runs real SQL (aggregation / join / filter), e2e | 🟢 Real (engine in e2e) |
| Warehouse item management | Full | 🟢 Real |
| T-SQL over TDS + Entra FedAuth | Pure-Go TDS front (internal/tds) terminates the FedAuth handshake (real Entra token, database.windows.net audience), then byte-splices the client’s post-login session to a real per-item SQL Server connection so the engine emits every token itself. Unmodified go-mssqldb and Microsoft ODBC Driver 18 (pyodbc) clients connect and run T-SQL — including RPCs, prepared statements, and transactions. Verified against a real SQL Server; Microsoft’s real dbt-fabric adapter passes debug/seed/run/test end-to-end (e2e/dbt-fabric/) | 🟢 Real (front) / 🟠 SQL Server sidecar |
| Lakehouse SQL analytics endpoint — Delta → engine | The emulator reads the lakehouse’s Tables/<t> Delta in pure Go and reflects (CREATE+INSERT) it into the sidecar on connect, so SELECT hits real OneLake data (matches DuckDB), read-only (writes rejected). Not PolyBase — SQL Server reading Delta in place is a proven dead-end on the Linux container (a throwaway spike; see 16-warehouse-tds.md) | 🟢 Real (reflection) |
| Warehouse — read-write T-SQL | Client CREATE/INSERT/SELECT relay straight to the sidecar; the warehouse owns its data (no reflection) | 🟢 Real (relay) |
Fabric SQL Database (database/) — OLTP + OneLake mirror | Same read-write TDS/FedAuth path (its own SQL Server database), plus mirroring: POST …/sqlDatabases/{id}/refreshMirror snapshots every table to OneLake as Delta (real Parquet + _delta_log), so Spark / DuckDB / delta-rs query the operational data. Verified with a go-mssqldb-writes → mirror → Delta-reads-back e2e (gated). Continuous/CDC mirroring and write-back-to-Delta are the deferred edge | 🟢 Real (snapshot mirror) |
| Per-item isolation (each item = its own SQL Server database) | Lakehouse/Warehouse routed by type; per-item databases so they never collide | 🟢 Real |
| RBAC → SQL permissions | Workspace role enforced on connect: no role → rejected; Viewer → read-only; Contributor+ → read-write (warehouse) | 🟢 Real |
information_schema / sys.* introspection | Relays natively — reflected/warehouse tables are real SQL Server tables | 🟢 Real (relay) |
| Per-column type fidelity (real SQL types over the wire) | The splice forwards SQL Server’s own COLMETADATA, so every column carries its true native type over the wire (the re-encode fallback, used only by fake test backends, synthesizes INTN/FLTN/BITN and falls back to NVARCHAR text) | 🟢 Real (native) |
| Connection by item name (vs GUID) | Workspace read from the server name (<workspace>.datawarehouse.fabric.microsoft.com), item resolved by display name; a GUID still resolves by id (back-compat). Verified with a real go-mssqldb client | 🟢 Real |
| Fabric feature | Emulator | Type |
|---|
Data Pipeline control flow (If / ForEach / Until / Switch / Filter / Fail, expression language, dependsOn) | Pure-Go interpreter that really executes | 🟢 Real (orchestration) |
| Per-activity policy — retry + backoff + timeout | Applied to every activity type: policy.retry re-runs a failed activity (each retry from scratch; only the final outcome is recorded, carrying retryAttempt); policy.retryIntervalInSeconds is folded into the run’s durationInSeconds as virtual backoff; policy.timeout fails an attempt whose own virtual duration exceeds the limit. No real sleeping — backoff and timeouts are exercised in milliseconds on the controllable clock | 🟢 Real |
ForEach sequential / parallel (isSequential, batchCount) | Iterations run in array order (deterministic); the mode sets the reported wall-clock — sequential iterations add, a parallel batch costs its slowest — matching how real Fabric overlaps them | 🟢 Real |
List pagination (continuationToken) | Opt-in via ?maxPageSize on list endpoints (workspaces, items, capacities, folders, connections, role assignments, shortcuts): returns a page + a continuationToken/continuationUri when more remain; omitted → the full set | 🟢 Real |
| Invoke pipeline (ExecutePipeline) | Resolves the referenced DataPipeline (GUID or name, optional other workspace) and runs it through a fresh interpreter — real recursive interpretation, one level deeper on the same engines. waitOnCompletion (default) gates the parent on the child’s terminal status; parameters flow into the child; a cycle or excessive nesting fails loudly | 🟢 Real |
| Pipeline → notebook activity (TridentNotebook) | Resolves the notebook reference and creates a real RunNotebook job instance the pipeline gates on — the pipeline→jobs linkage is real; the notebook’s cells execute only on the Spark sidecar (otherwise the job is clock-derived, like any RunNotebook job) | 🟢 Real chain / 🟠 exec |
queryactivityruns detail | Full | 🟢 Real |
| Activity-level lineage | Successful Copy execution persists its resolved workspace/item/path source→sink edge, returns it in activity output, and exposes workspace lineage for OpenMetadata ingestion | 🟢 Real (Copy) |
| Copy activity — OneLake → OneLake | Really moves the bytes through the storage layer: a file, or a directory subtree preserving structure; source/sink locations {workspaceId?, itemId, path} are expression-resolved (GUID or name); returns real filesWritten / dataWritten. External stores / format transformation are out of scope and fail loudly | 🟢 Real (in-family) / 🔴 external |
| Lookup activity — OneLake CSV/JSON/Parquet/Delta | Reads real rows from a CSV, JSON, or standalone Parquet file, or a lakehouse Delta table (Tables/<name>, auto-detected — no format hint needed) in OneLake; honors firstRowOnly; the result flows into @activity(…).output for downstream steps. Parquet/Delta reuse the warehouse’s own Parquet reader — a real Delta column keeps its native type (int/float/bool), not a stringified cell | 🟢 Real (CSV/JSON/Parquet/Delta) |
| GetMetadata activity — OneLake path | Stats a real OneLake path: exists / itemType / size / lastModified / childItems; a missing path honestly returns exists:false | 🟢 Real |
| Script / SqlServerStoredProcedure activities | Run real T-SQL against a Warehouse/Fabric-SQL-Database item’s own SQL Server database — the same per-item backend the TDS endpoint and the SQLDatabase mirror share. Script runs each scripts[] entry (Query → real rows back, NonQuery → rows affected); SqlServerStoredProcedure calls a real stored procedure with named parameters. The target is named directly as {workspaceId?, itemId} (the emulator’s own scoped mapping — real Fabric’s linkedService/connection reference isn’t modeled), the same shape Copy/Lookup/GetMetadata already use. Honest error without a warehouse SQL backend attached | 🟢 Real (scoped) |
| Web / external-connector leaves | Stubbed success — reached in dependsOn order and inputs resolved, but nothing executes: Web calls to arbitrary URLs would break the offline/deterministic guarantee | 🟡 Emulated |
| Apache Airflow Job | Typed item + beta file APIs; uploaded Python DAGs sync to an opt-in real Airflow 2.10.5/Python 3.12 sidecar, whose scheduler/executor and REST state determine the Fabric job result (e2e/airflow) | 🟢 Real (sidecar) |
| Dataflow Gen2 (Power Query M engine) | Typed item/definition management round-trips. Refresh, Publish, and in-pipeline execution fail with DataflowEngineNotImplemented; no open Power Query M engine exists to attach | 🟡 mgmt / 🔴 exec |
| Connectors / on-prem gateways | — | 🔴 Not implemented |
| Fabric feature | Emulator | Type |
|---|
| Git integration (connect / status / commit / update / disconnect) | Full, real state | 🟢 Real |
fabric-cicd tool publishing | The real client round-trips definitions (e2e) | 🟢 Real |
| Deployment pipelines — model, assignment, item pairing, Deploy Stage Content (D0–D2) | Real promotion: definitions really copy, pairs decide (not names), metadata only — a deployed lakehouse arrives empty — and target-only items survive. 202 LRO + /result detail. 23-deployment-pipelines.md | 🟢 Real |
| Deployment pipelines — role-assignment CRUD (D3) | Add / Delete / List; Admin is the only role a pipeline defines; mutations require Admin, reads require membership | 🟢 Real |
| Fabric area | Emulator | Type |
|---|
Real-Time Intelligence — Eventhouse / KQL Database (real-time-intelligence/) | Full item management (including the default child database an eventhouse creates, and creationPayload.parentEventhouseItemId), plus the Kusto REST protocol on the eventhouse’s published properties.queryServiceUri — /v1/rest/mgmt, /v1/rest/query, /v2/rest/query — terminated by the emulator (Kusto-audience bearer, workspace RBAC, one isolated engine database per Fabric KQL Database) and executed by Microsoft’s own KQL engine container (kustainer) when the rti profile attaches it. No engine attached → honest 501. 25-rti-kusto.md | 🟡 mgmt / 🟠 exec (BYO Kusto engine) |
Real-Time Intelligence — Eventstream (real-time-intelligence/event-streams/) | Item management only. The attached Kusto engine is a query/ingest engine with no streaming ingestion — a streaming pipeline is a different service, deferred with cause | 🟡 mgmt / 🔴 exec |
Mirroring — Mirrored Database (mirroring/) | POST …/mirroredDatabases/{id}/refreshMirror mirrors an external SQL Server source (reached via a Connection with Basic credentials) to OneLake as real Delta — reusing the exact same mirror writer the Fabric SQL Database uses (warehouse.Mirror; same code, external source). Proven by a gated e2e: a table seeded directly on an external database (bypassing the emulator’s own per-item routing entirely) mirrors and reads back correctly. Snapshot-on-trigger, not continuous/CDC replication; other source engines (Snowflake, CosmosDB, on-prem via gateway) are out of scope | 🟢 Real (snapshot mirror, SQL Server sources) |
Power BI — Semantic Model query (executeQueries) | Real bounded DAX engine — EVALUATE, SUMMARIZECOLUMNS, measures, SUM/DIVIDE, relationship filter propagation — over imported data.json or compatibility-level-1604 Direct Lake entity partitions backed by current OneLake Delta. Proven by the golden DAX/GX suites and the Spark-written Direct Lake witness. | 🟢 Real (DAX subset + Direct Lake) |
| Power BI — Reports / rendering; full DAX; SemPy over XMLA | No report rendering; DAX beyond the fixture subset; and the native ADOMD.NET/XMLA transport SemPy uses (no CI oracle) — all deferred with cause | 🟡 mgmt / 🔴 render |
Data Science — ML models / experiments / MLflow (data-science/) | Authenticated, workspace-scoped proxy to a real MLflow 3 tracking/model-registry server. Experiment/model creation synchronizes typed Fabric items; experiment/run references are isolated by workspace; successful artifact uploads are mirrored under the experiment item’s OneLake Files/mlflow-artifacts. | 🟢 Real (sidecar) |
Fabric SQL Database (database/), Graph (graph/), Real-Time Hub, Copilot / IQ (iq/), Embed, Workload Dev Kit | — | 🔴 Not implemented |
| Capability | Purpose |
|---|
Controllable clock (/_emulator/clock) | Advance virtual time to drive LRO / job status transitions deterministically. |
Fault injection (/_emulator/faults, /_emulator/permissions) | Force failures / throttling / RBAC denials to test client resilience. |
| Svelte management portal | Dashboard, workspaces, operations, clock, and fault controls. |
Parity isn’t claimed from our own tests alone — each 🟢 surface is pinned against
the real, unmodified client a Fabric user runs, executed against the emulator
in CI (e2e/<client>/). If Microsoft’s own tool round-trips unchanged, the
contract holds better than any assertion we could write ourselves.
| Real client (pinned) | Surface exercised | Status |
|---|
fabric-cicd (Microsoft) | Control plane / CI-CD publish | 🟢 e2e/fabric-cicd |
Fabric CLI fab (Microsoft) | Control plane — SPN auth (MSAL) + workspace/item CRUD (Notebook, SemanticModel, Report, DataPipeline, Lakehouse), ls/get/api | 🟢 e2e/fabric-cli |
| Fabric Data Engineering VS Code extension 1.18.1 contract (Microsoft) | Shared-backend/MWC authoring routes through api.powerbi.com; interactive kernel websocket is not claimed | 🟢 e2e/vscode-extension |
| Apache Airflow 2.10.5 | ApacheAirflowJob DAG discovery, scheduling, execution, and status | 🟢 e2e/airflow |
| MLflow 3 + dbt-duckdb | Workspace-scoped experiment/run/artifact/model lifecycle, followed by dbt’s real Delta plugin over the same Spark-written OneLake table | 🟢 e2e/data-science-loop |
deltalake (delta-rs) | OneLake Delta write/read | 🟢 e2e/delta-rs |
azure-storage-file-datalake + Blob SDK | OneLake ADLS Gen2 DFS + Blob | 🟢 e2e/adls-sdk |
azcopy (Microsoft) | OneLake Blob multi-block transfer | 🟢 e2e/azcopy |
| DuckDB | Lakehouse SQL over Delta/Parquet | 🟢 e2e/duckdb |
| PySpark behind the Livy API | Spark sessions / statements | 🟢 e2e/spark, e2e/livy, e2e/notebook-run |
notebookutils | Notebook utility shim | 🟢 e2e/notebookutils |
go-mssqldb | Warehouse/Lakehouse TDS + FedAuth | 🟢 internal/server, internal/tds |
dbt-fabricspark (Microsoft) | Fabric Spark via Livy HC sessions | 🟠 e2e/dbt-fabricspark — debug→seed→run→test on Sail |
dbt-fabric (Microsoft) | Warehouse TDS via ODBC Driver 18 | 🟢 e2e/dbt-fabric — debug→seed→run→test through the TDS splice |
azure-kusto-data (Microsoft) + raw Kusto REST, over kustainer (Microsoft’s own KQL engine) | Eventhouse / KQL Database: /v1/rest/mgmt, /v1/rest/query, /v2/rest/query on the published queryServiceUri — create table, ingest, query values back, per-database isolation | 🟠 e2e/rti — witness of record is CI (amd64). The engine needs AVX2, which Rosetta does not provide, so the default Docker setup on Apple silicon cannot run it; a QEMU x86-64 VM with --cpu-type max can, and does (25-rti-kusto.md) |
The TDS surface now has two independent driver witnesses: go-mssqldb and
the Microsoft ODBC Driver 18 (via dbt-fabric). That second driver mattered —
it exposed a real gap: go-mssqldb tolerated a synthesized FedAuth login, but
ODBC Driver 18 took a compatibility path (prepared-statement RPCs +
sp_reset_connection under mandatory connection pooling) that desynced against a
re-encoding relay. The fix was to byte-splice the post-login session straight
to the real SQL Server (so it emits every token itself), which is exactly the
kind of driver-family gap a single-driver test never surfaces. dbt-fabricspark
likewise drives the high-concurrency Livy layer over its real Livy-session
protocol (method: livy, service-principal auth via entra-emulator).
The emulator targets Microsoft Fabric — the convergence/successor product —
not the earlier Azure analytics services Fabric replaced. That boundary is why
some adjacent dbt adapters and Azure surfaces are intentionally not built:
they belong to predecessor (often retired) products, and their Fabric-native
successors are what we emulate instead.
| Adjacent product / client | Why out of scope | Fabric-era equivalent (in scope) |
|---|
Azure Synapse dedicated SQL pool (dbt-synapse) | Different product: its own control plane (Synapse workspaces) and an MPP T-SQL dialect (DISTRIBUTION = HASH, clustered-columnstore / resource-class DDL) that our vanilla SQL Server sidecar rejects. dbt-synapse layers on dbt-fabric, so the shared SQL path is already covered by the dbt-fabric witness | Fabric Warehouse — 🟢 TDS relay |
Azure Data Lake Analytics — U-SQL / SCOPE (dbt-scope) | Retired service (EOL Feb 2024), proprietary batch language, no Fabric embodiment. The only overlap (Delta on a lake) is Spark/OneLake, already witnessed | Fabric Spark — 🟠 Livy |
| ADLS Gen1 | Retired (Feb 2024), superseded by Gen2 | — |
| ADLS Gen2 (standalone storage account) | Not missing — OneLake is the Gen2 endpoint: hierarchical namespace, the dfs filesystem API, onelake.dfs.fabric.microsoft.com. Fabric has no separate storage account to emulate | OneLake — 🟢 e2e/adls-sdk |
Rule of thumb: if a capability exists only in a product Fabric replaced, it’s out
of scope; its Fabric-native successor is what we build. “We already have the
TDS/SQL Server foundation” makes Synapse cheaper, not done — the remaining
delta is a whole MPP dialect plus a second control plane, for a superseded
target. So the two dbt adapters we build (dbt-fabricspark, dbt-fabric) are
exactly the two that hit live Fabric surfaces; the other two (dbt-synapse,
dbt-scope) target predecessor products outside the emulator’s remit.
Real Fabric’s own Livy endpoint is Microsoft’s implementation of the Livy REST
contract over their Spark platform — they honor the protocol, not the retired
Apache Livy server. And where Fabric adds its own layer on top of that
protocol — high-concurrency REPL packing, which a vanilla Livy server has no
concept of — the emulator implements that layer directly rather than proxying,
because there is nothing to proxy it to. That is the same stance throughout: the
protocol and control plane are the durable, real things (built, not mocked,
so real clients can’t tell the difference), and the compute engine is attached
(Spark; T-SQL on SQL Server; KQL on Microsoft’s own Kusto engine —
25-rti-kusto.md) or deferred when proprietary or without an
implementation to attach at all (Dataflow Gen2’s M engine, Power BI rendering,
Eventstream’s streaming ingestion). Every deferral fails loudly rather than
pretending to succeed. See 13-roadmap.md for the milestone
history and the deferred-with-cause rationale.