Databricks & Fabric Study Guide

Start here

What this role asks for

The role is a lead architect for an Azure lakehouse platform. Azure Databricks is the main engine. Microsoft Fabric is the second platform, mostly for new builds and migrations. Strong SQL runs through everything.

Probe area 1

Architecture judgement

Why you pick one design over another. Medallion layers, Lakehouse or Warehouse, batch or streaming.

Probe area 2

Hands-on depth

PySpark, Delta Lake, Auto Loader, declarative pipelines, Databricks SQL. Expect "how does it work inside" questions.

Probe area 3

Azure integration

ADLS Gen2, Data Factory, Synapse, Power BI, on-premises sources, Key Vault, networking.

Probe area 4

Fabric and migration

Fabric items, OneLake, Eventhouse, and how you move Synapse or Databricks workloads into Fabric.

Probe area 5

Engineering discipline

Git, CI/CD, infrastructure as code, code quality, security and governance.

Probe area 6

Leadership

Mentoring engineers, explaining technical choices to business people, owning delivery end to end.

Job description line	Module	Priority
Lead architecture and design of analytics solutions on Databricks / Fabric	Lakehouse & medallion	Essential
PySpark pipelines, ETL, Hadoop / Spark knowledge	Spark & PySpark	Essential
Delta tables, Delta Lake, ADLS Gen2, cluster management	Delta Lake	Essential
Auto Loader and Delta Live Tables; Spark Streaming	Auto Loader & pipelines	Essential
Integrate with ADLS, Synapse, ADF, on-premises, Power BI	Azure integration	Essential
Strong SQL, Databricks SQL query layer, NoSQL familiarity	SQL & query layer	Essential
Fabric Pipeline, Lakehouse, Warehouse, Eventhouse	Microsoft Fabric	Essential
Migrate Synapse, Databricks, Data Mart workloads to Fabric	Migration to Fabric	Essential
CI/CD, Git, ARM templates, cloud security, governance	CI/CD, IaC & security	Essential ARM: nice to have
Mentoring, stakeholder communication, documentation	Leadership & stakeholders	Essential

How to use this guide

Open one module. Read the short concept section.
Run the animation. Use the step buttons and read the caption below the diagram.
Read the "Interview angle" box. Say the answer out loud in your own words.
Do the quick check. Then click "Mark as studied".
Finish with the mock interview flashcards.

Names changed recently. Use the current names in the interview, and mention the old ones. I checked these against vendor documentation on 10 Nov 2026. They can change again, so verify before the interview.

Delta Live Tables (DLT) is now Lakeflow Spark Declarative Pipelines. Some Databricks pages shorten this to "Lakeflow pipelines". Old DLT code still runs, with no migration needed.
Databricks Asset Bundles is now Declarative Automation Bundles (renamed 16 March 2026). The databricks bundle commands stay the same.
Databricks Workflows appears in current docs as Lakeflow Jobs.
Fabric "Real-Time Analytics" is now part of Real-Time Intelligence. Eventhouse is its storage item.

Foundations · 1

Lakehouse and medallion architecture

A lakehouse keeps data as open files in cheap cloud storage. It adds warehouse features on top: transactions, schemas, SQL and governance. The medallion pattern organizes that data in three quality layers.

JD link: "Lead the architecture, design, and implementation of advanced analytics solutions" and "Experience with ETL/ELT processes, data warehousing, and building data lakes."

Records flow through Bronze, Silver and Gold

Landed in Bronze 0Quarantined 0Served from Gold 0

Press Play. Each dot is a record. Its color shows the layer it is in. Red dots fail a data quality rule at Silver.

Bronze

Keep the source as it is

Append-only. Do not change values.
Add metadata: load time, source file, batch ID.
Lets you replay history when Silver logic changes.

Silver

Make it correct

Cast types, remove duplicates, standardize codes.
Apply data quality rules. Send bad rows to quarantine.
Conform keys across sources (one customer ID).

Gold

Make it useful

Star schemas, aggregates, feature tables.
Shaped for one consumer group or domain.
Serve to Power BI, Databricks SQL, APIs.

Design decisions an architect owns

Managed or external tables. Managed tables let Unity Catalog control files and lifecycle. External tables keep files at a path you control. Choose external when other engines must own the path.
Catalog layout. Common options: one catalog per environment (dev, prod), or one catalog per business domain with schemas per layer. Pick one rule and document it.
Batch or streaming per layer. Bronze is often streaming (Auto Loader). Gold is often batch or materialized views.
Medallion is a flexible pattern. Some teams add a landing zone before Bronze or split Gold into "serving" and "sandbox". Explain the reason for each layer.

Interview angle

"Why not load data straight into Gold?"

Bronze gives an audit trail. You can prove what the source sent.
When business logic changes, you rebuild Silver and Gold from Bronze. You do not ask the source to resend.
Layers isolate failures. A bad source file stops at Silver and does not break the dashboard.
Trade-off: more storage and more jobs. Say how you control that cost (retention, VACUUM, fewer Gold tables).

Business logic for "active customer" changes. What lets you rebuild Gold for the last two years without asking the source to resend data?

Back: Role map

Foundations · 2

Spark and PySpark internals

Spark splits data into partitions. The driver plans the work. Executors run one task per partition. Most performance questions come down to one thing: how much data moves between executors.

JD link: "Proficiency in big data technologies (e.g., Hadoop, Spark) and data processing frameworks (e.g., PySpark)."

Narrow and wide transformations

Each square is a partition. The letter is the key (for example, region). Try filter(), then Reset, then groupBy().

Core vocabulary

Transformation (select, filter, join) is lazy. It only adds to the plan.
Action (count, write, collect) starts a job.
Job → stages → tasks. A shuffle creates a new stage. One task runs per partition.
Narrow: each output partition needs one input partition. No network.
Wide: groupBy, join, distinct, repartition. Data moves across the network (shuffle).

Performance levers

Broadcast join: copy a small table to every executor. No shuffle of the big table.
Adaptive Query Execution (AQE): changes the plan at runtime. It merges small partitions and splits skewed ones.
Skew: one key has most rows. One task runs much longer than others.
Small files: too many tiny files slow reads. Compact with OPTIMIZE.
Photon: Databricks vectorized engine for SQL and DataFrame work.

# PySpark: read Silver, aggregate to Gold, broadcast the small dimension
from pyspark.sql import functions as F

orders  = spark.read.table("main.silver.orders")
regions = spark.read.table("main.silver.dim_region")          # small table

daily = (orders
  .filter(F.col("status") == "COMPLETE")                     # narrow
  .join(F.broadcast(regions), "region_id")                       # no shuffle of orders
  .groupBy("order_date", "region_name")                         # wide: shuffle
  .agg(F.sum("amount").alias("revenue")))

daily.write.mode("overwrite").saveAsTable("main.gold.daily_revenue")  # action: job starts here

Verify these numbers before you quote them. I believe the default broadcast threshold (spark.sql.autoBroadcastJoinThreshold) is 10 MB in open-source Spark, and AQE is on by default since Spark 3.2. Databricks runtimes can use different defaults. "Aim for partitions of about 128 MB" is a common rule of thumb, not a fixed rule.

Interview angle

"A Spark job is slow. What do you check?"

Open the Spark UI. Find the slowest stage.
Compare task times in that stage. One long task means skew. Fix with AQE skew join, salting, or a better key.
Check shuffle read and write size, and disk spill. Large spill means not enough memory per task or too few partitions.
Check the number and size of input files. Many small files means compaction is needed.
Check the plan for joins that could broadcast, and for filters that do not push down.
Only then change cluster size. Bigger clusters hide bad design and cost more.

Hadoop in one paragraph

The JD lists Hadoop. Know the history: HDFS stores blocks across nodes, YARN manages cluster resources, and MapReduce runs batch jobs by writing to disk between steps. Spark keeps intermediate data in memory where it can, so it is usually much faster for multi-step work. Hive added SQL tables over HDFS. In Azure, ADLS Gen2 replaces HDFS and Databricks replaces YARN-managed clusters. If they ask about migrating Hadoop, talk about moving data to ADLS Gen2, converting Hive tables to Delta, and rewriting MapReduce jobs as Spark.

Which operation creates a stage boundary?

Back: Lakehouse

Foundations · 3

Delta Lake

A Delta table is Parquet data files plus a transaction log (_delta_log). The log lists which files make up each version. Readers use the log to decide which files to read. That one idea explains ACID, time travel, MERGE and VACUUM.

JD link: "Solid understanding and hands-on experience with Delta tables, Delta Lake, and Azure Data Lake Storage Gen2."

Transaction log simulator

_delta_log/

Data files in ADLS Gen2

part of selected versionstill on disk, not in versionphysically deleted by VACUUM

Read as of version0

Each button writes a commit. Move the slider to time travel. Then run VACUUM and try to read an old version.

How Delta gives ACID on object storage

Atomic: a write is visible only when its log entry exists. A failed job leaves no log entry, so readers never see half a write.
Consistent: schema enforcement rejects writes that do not match the table schema.
Isolated: optimistic concurrency. Two writers try to commit the same version number. One wins. The other checks for conflicts and retries or fails.
Durable: files and log live in cloud storage.
The log is periodically summarized into a Parquet checkpoint so readers do not replay every JSON file.

Commands to know well

MERGE INTO for upserts and CDC.
VERSION AS OF / TIMESTAMP AS OF for time travel and audit.
RESTORE to roll a table back.
OPTIMIZE to compact small files.
VACUUM to delete unreferenced files after the retention period.
DESCRIBE HISTORY to see who changed what.
Change Data Feed to read row-level changes downstream.

-- Upsert today's changes into Silver
MERGE INTO main.silver.customers t
USING updates s
  ON t.customer_id = s.customer_id
WHEN MATCHED AND s.op = 'D' THEN DELETE
WHEN MATCHED THEN UPDATE SET *
WHEN NOT MATCHED THEN INSERT *;

-- Audit: what did the table look like before last night's load?
SELECT * FROM main.silver.customers VERSION AS OF 41;

-- Layout: liquid clustering replaces partitioning and Z-order for new tables
ALTER TABLE main.silver.orders CLUSTER BY (order_date, customer_id);
OPTIMIZE main.silver.orders;

Table layout: partitioning, Z-order, liquid clustering

Current Databricks documentation recommends liquid clustering for all new tables, and predictive optimization for Unity Catalog managed tables. It gives size guidance: below 1 TB, do not partition. From 1 TB to 100 TB, use liquid clustering instead of partitioning. At 100 TB or more, partitioning might help, but try liquid clustering first and measure. Liquid clustering lets you change the clustering keys later without a full rewrite. I believe you should not use liquid clustering and Z-order together on one table, but I did not confirm this in the docs, so verify it.

Old answer you may still hear: "Partition by date and Z-order by customer ID." Say you know it, and explain why the newer default is better: no fixed folder boundaries, fewer small files, keys can change.

Cluster management basics

All-purpose compute for interactive notebooks. Job compute for scheduled jobs; it starts per run and is cheaper.
Serverless compute for notebooks, jobs and SQL warehouses removes cluster sizing work. Check region and feature availability.
Cluster policies limit node types, sizes and auto-termination. Use them to control cost.
Autoscaling and auto-termination on every all-purpose cluster.
Access modes: Unity Catalog needs a supported access mode. I believe the current names are "standard" (formerly shared) and "dedicated" (formerly single user). Verify in current docs.
Pin jobs to an LTS Databricks Runtime and plan upgrades.

Check before you quote: I believe the default VACUUM retention is 7 days (168 hours). Running VACUUM with a shorter period can break time travel and running readers. Newer Delta versions also support deletion vectors, which can mark rows as deleted without rewriting the whole file. The simulator above shows the classic copy-on-write behavior.

Interview angle

"Two jobs write to the same Delta table at the same time. What happens?"

Both read the latest version, for example version 10.
Both try to write version 11. Storage allows only one to create that log entry.
The loser checks whether the winner changed files it read. If not, it commits as version 12. If yes, it fails with a concurrent modification error.
Design fix: separate writers by partition or clustering range, or serialize writes through one job.

You ran VACUUM with a short retention. Now VERSION AS OF 3 fails. Why?

Back: Spark

Build · 4

Auto Loader and declarative pipelines

Auto Loader finds new files in cloud storage and loads each file once. Lakeflow Spark Declarative Pipelines (formerly Delta Live Tables) let you declare the tables you want. Databricks runs them in order, retries them and checks data quality.

JD link: "Experience in efficiently using Auto Loader and Delta Live tables" and "Spark Streaming for processing real-time data streams."

How Auto Loader finds new files

Pick a mode, then press Play. Every fifth file is a re-upload of a file already loaded. Watch the checkpoint skip it.

Auto Loader essentials

Source format cloudFiles on Structured Streaming.
Checkpoint stores which files were loaded. This gives exactly-once file processing.
Schema inference and evolution with cloudFiles.schemaLocation. Unknown fields go to a rescued data column instead of failing the load.
Directory listing needs no extra setup. It slows down on very large folders.
File notification scales better. Current docs recommend file events on a Unity Catalog external location (cloudFiles.useManagedFileEvents, Databricks Runtime 14.3 LTS or above). The legacy mode creates an Event Grid subscription and queue per stream.
trigger(availableNow=True) runs it like a batch job: load everything new, then stop.

Declarative pipelines essentials

Streaming table: a Delta table that processes each input row once. Good for Bronze and Silver appends.
Materialized view: results are stored and refreshed. Good for Gold aggregates.
Expectations are data quality rules. On failure you can keep the row and log it, drop the row, or fail the update.
CDC: apply inserts, updates and deletes from a change feed, including SCD Type 1 and Type 2.
The pipeline builds the dependency graph from your code. You do not write orchestration.

# Auto Loader into Bronze (classic Structured Streaming)
(spark.readStream.format("cloudFiles")
   .option("cloudFiles.format", "json")
   .option("cloudFiles.schemaLocation", "/Volumes/main/ops/schemas/orders")
   .load("abfss://landing@mylake.dfs.core.windows.net/orders/")
   .writeStream
   .option("checkpointLocation", "/Volumes/main/ops/checkpoints/orders")
   .trigger(availableNow=True)
   .toTable("main.bronze.orders"))

# Same idea as a declarative pipeline with a quality rule
from pyspark import pipelines as dp          # older code: import dlt

@dp.table(comment="Orders with valid amounts")
@dp.expect_or_drop("positive_amount", "amount > 0")
def silver_orders():
    return spark.readStream.table("bronze_orders")

Naming check. Databricks docs say DLT is now Lakeflow Spark Declarative Pipelines, and import dlt can be replaced with from pyspark import pipelines as dp. Old code still runs. The SQL command APPLY CHANGES INTO may appear under a newer name (I think "AUTO CDC") in current docs. I am not certain, so verify. In the interview, say "DLT, now called Lakeflow Declarative Pipelines". The JD uses the old name.

Interview angle

"Auto Loader, COPY INTO or Data Factory Copy activity: which one?"

Auto Loader: continuous or frequent file arrival, millions of files, schema drift. Built-in checkpoint.
COPY INTO: simple SQL, idempotent batch loads of a modest number of files.
ADF Copy: source is not files in the lake. For example, on-premises SQL Server through a self-hosted integration runtime, or a SaaS API. ADF lands the files, then Auto Loader picks them up.

Spark Structured Streaming in brief

Micro-batch model. Each trigger processes new data since the last checkpoint.
Watermark tells Spark how late events can arrive. It lets Spark drop old state.
Sources: Azure Event Hubs (Kafka-compatible endpoint), Kafka, Auto Loader, Delta tables.
Output modes: append, update, complete.
foreachBatch lets you run a MERGE per micro-batch. This is a common upsert pattern.

A pipeline restarts after a failure. Why does Auto Loader not load yesterday's files again?

Back: Delta Lake

Build · 5

Azure integration

Databricks rarely works alone. Data Factory brings data in, ADLS Gen2 stores it, Databricks transforms it, Unity Catalog governs it, and Power BI, Synapse or Fabric serve it. Know what each service does best.

JD link: "Integrate Azure Databricks with other Azure services (e.g., Azure Data Lake, Azure Synapse, Azure Data Factory) and on-premise systems."

Reference architecture: pick a scenario

Choose a scenario. The active path animates and the caption explains each hop.

Azure Data Factory

Pipeline holds activities (Copy, Databricks Notebook or Job, Lookup, ForEach, If).
Linked service = connection. Dataset = shape of data at that connection.
Integration runtime: Azure IR for cloud, self-hosted IR for on-premises and private networks, Azure-SSIS IR for SSIS packages.
Triggers: schedule, tumbling window (with backfill and dependency), storage event.
Metadata-driven pattern: a control table lists sources. One pipeline with ForEach loads all of them.

ADLS Gen2

Blob storage with hierarchical namespace: real folders, atomic renames, POSIX-style ACLs.
Path format abfss://container@account.dfs.core.windows.net/path.
Databricks access through Unity Catalog storage credential (managed identity via the Access Connector) and external location. Avoid account keys in notebooks.
Combine Azure RBAC and ACLs. Use private endpoints for both dfs and blob.

Azure Synapse Analytics

Dedicated SQL pool: MPP warehouse. Tables use hash, round-robin or replicated distribution. Sized in DWUs.
Serverless SQL pool: pay per query over lake files.
Spark pools and pipelines (ADF engine inside Synapse).
Status: Microsoft has not announced retirement of Synapse as a whole. It steers new work to Fabric and is retiring some parts. For example, Synapse Link for Cosmos DB NoSQL cannot be enabled on accounts created on or after 31 March 2026, and existing accounts are supported until 31 March 2029 (per an Azure update notice).

Unity Catalog

Three-level names: catalog.schema.table.
One metastore per region, attached to workspaces.
GRANTs on catalogs, schemas, tables, volumes, functions, models.
Row filters and column masks for fine-grained access.
Automatic lineage and audit logs.
Mirroring into Fabric is driven from Unity Catalog.

Interview angle

"Who orchestrates: Data Factory or Databricks Jobs?"

If most steps run in Databricks, use Lakeflow Jobs. It is simpler, has task dependencies, retries, and repair runs.
If the flow spans many systems (on-premises copy, SaaS APIs, stored procedures, other services), use Data Factory as the outer orchestrator and call Databricks jobs from it.
Avoid two schedulers for one flow. Pick one owner and document it.
On-premises sources need a self-hosted integration runtime, or a private network path that Databricks can reach.

Power BI on Databricks

Connect with the Azure Databricks connector to a Databricks SQL warehouse, ideally serverless for fast start.
Import mode for small to medium Gold tables. DirectQuery for large or near-real-time data. Composite models mix them.
Build a star schema in Gold: facts plus conformed dimensions. Power BI performs best on that shape.
Use Entra ID single sign-on so Unity Catalog permissions apply per user, where the setup supports it.
With Fabric, mirror the Unity Catalog tables and use Direct Lake semantic models (see the Fabric module).
For "develop, optimize, deploy reports": incremental refresh, aggregations, row-level security, deployment pipelines.

Data Factory must copy from SQL Server in a private data centre. Which component makes that possible?

Back: Auto Loader

Build · 6

SQL and the query layer

The JD says "Strong SQL" twice. Expect live SQL questions: deduplication, window functions, slowly changing dimensions, and how you make Databricks SQL fast for Power BI users.

JD link: "Proficiency in building and optimizing query layers using Databricks SQL" and "Strong SQL skills and familiarity with NoSQL databases."

SCD Type 2: keep history when an attribute changes

sk	customer_id	city	valid_from	valid_to	is_current

Press the button. The MERGE closes the current row and inserts a new current row. Old facts still join to the old city.

-- SCD2 in one MERGE: the "staged" trick makes changed rows appear twice
MERGE INTO gold.dim_customer t
USING (
  SELECT s.customer_id AS merge_key, s.* FROM silver.customer_changes s
  UNION ALL
  SELECT NULL AS merge_key, s.*                        -- forces an INSERT for changed rows
  FROM silver.customer_changes s
  JOIN gold.dim_customer d
    ON d.customer_id = s.customer_id AND d.is_current AND d.city <> s.city
) src
ON t.customer_id = src.merge_key AND t.is_current
WHEN MATCHED AND t.city <> src.city THEN UPDATE SET
  is_current = false, valid_to = src.change_date
WHEN NOT MATCHED THEN INSERT (customer_id, city, valid_from, valid_to, is_current)
  VALUES (src.customer_id, src.city, src.change_date, NULL, true);

-- Deduplicate: keep the latest record per key
SELECT * FROM bronze.orders
QUALIFY ROW_NUMBER() OVER (PARTITION BY order_id ORDER BY ingest_ts DESC) = 1;

-- Running total and month-over-month change
SELECT month, revenue,
  SUM(revenue) OVER (ORDER BY month) AS running_total,
  revenue - LAG(revenue) OVER (ORDER BY month) AS mom_change
FROM gold.monthly_revenue;

Databricks SQL warehouses

Types: serverless, pro, classic. Serverless starts fastest and scales without cluster management.
Size (T-shirt) sets compute per cluster. Scaling (min/max clusters) handles many users at once.
Query history and query profile show where time goes.
Result cache and disk cache speed up repeated queries.

Query optimization checklist

Filter on clustering keys so file skipping works.
Select only needed columns. Parquet is columnar.
Keep table statistics current (ANALYZE TABLE, or predictive optimization).
Pre-aggregate in Gold or use materialized views for heavy dashboards.
Watch for exploding joins and SELECT DISTINCT that hides a bad join.

NoSQL familiarity

Know the four families: key-value, document (Azure Cosmos DB for NoSQL, MongoDB), wide-column (Cassandra), and graph. Cosmos DB scales by partition key; a bad key creates hot partitions. For analytics, do not run heavy queries on the operational store. Copy changes out: Cosmos DB change feed into Databricks, or Cosmos DB mirroring into Fabric OneLake. Microsoft now points new projects to Fabric mirroring instead of Synapse Link.

Interview angle

"A Power BI report on Databricks takes 40 seconds. Walk me through the fix."

Find the slow visual with Power BI Performance Analyzer. Copy the generated SQL.
Run it in Databricks SQL. Read the query profile: scan size, files read, joins, spill.
Fix the data model first: star schema, correct relationships, fewer high-cardinality columns.
Fix the table: clustering on filter columns, OPTIMIZE, fresh statistics.
Add aggregation tables or switch suitable tables to Import mode.
Then size the SQL warehouse for the number of users who query at the same time.

The business needs sales reported by the customer's city at the time of each sale. Which SCD type?

Back: Azure

Fabric · 7

Microsoft Fabric

Fabric is a SaaS analytics platform. All its items store tables in one tenant-wide lake, OneLake, in Delta Parquet format. You buy capacity (F SKUs) and every workload shares it. The key idea for an architect: one copy of data, many engines.

JD link: "Design and implement solutions on Microsoft Fabric, including Fabric Pipeline, Lakehouse, Warehouse and Real-Time Analytics (EventHouse)."

OneLake: how data gets in and out

Choose a pattern to see the path it takes into or out of OneLake.

Data Engineering

Lakehouse

Spark notebooks and jobs write Delta tables. A Files area holds raw files.
Auto-generated SQL analytics endpoint. It is read-only: views and functions, no INSERT or UPDATE.
Best for engineering, medallion layers, data science.

Data Warehouse

Warehouse

Full T-SQL: DDL, DML, multi-table transactions.
Data still stored as Delta in OneLake.
Best for SQL teams, stored-procedure ELT, Synapse dedicated pool migrations.

Real-Time Intelligence

Eventhouse

Container of one or more KQL databases.
Fast ingestion and query of time-series, logs, telemetry with KQL.
Successor of Azure Data Explorer patterns. Eventstream feeds it; Activator acts on it.

Data Factory

Pipelines

Orchestration like ADF: Copy, Notebook, Stored procedure, ForEach.
Dataflow Gen2 for low-code Power Query transformation.
Copy job for simple bulk or incremental copy.

OneLake

Shortcuts and mirroring

Shortcut: a pointer to data in ADLS, S3 or another item. No copy.
Mirroring: near real-time replica of a database in OneLake.
Azure Databricks Unity Catalog mirroring is generally available. Microsoft describes it as a no-copy integration, and the data is read-only in OneLake.

Power BI

Direct Lake

Semantic model reads Delta files in OneLake directly.
Import-like speed without scheduled data copies.
Falls back to DirectQuery in some cases. Learn the guardrails per capacity size.

Decision helper: Lakehouse or Warehouse?

Main team skill

Need INSERT / UPDATE / MERGE in T-SQL?

Semi-structured or unstructured files?

Migrating a Synapse dedicated SQL pool?

Suggested

Lakehouse

Fast-moving product. Fabric ships features monthly. Examples I have seen in 2026 sources but did not fully verify: materialized lake views reached general availability around March 2026, and SQL database in Fabric is generally available. Check the Fabric "What's new" page in the week of your interview.

Interview angle

"The company has Databricks. Why would they add Fabric?"

Business users already live in Power BI and Microsoft 365. Fabric puts reporting, semantic models and self-service in the same place.
Mirroring Unity Catalog into OneLake gives Direct Lake reports without copying data or building export jobs.
Real-Time Intelligence covers telemetry and alerting scenarios with less custom code.
Keep Databricks for heavy engineering and ML if it already works. Avoid running the same pipeline on both platforms.
Risks to name: two governance models (Unity Catalog and Fabric workspace/OneLake security), capacity cost planning, feature gaps in preview items.

A SQL developer tries to run UPDATE on a table through the Lakehouse SQL analytics endpoint. What happens?

Back: SQL

Fabric · 8

Migrating workloads into Fabric

A migration is mostly a project management problem with technical risks. Treat it in phases. Each source type has its own path and its own traps.

JD link: "Knowledge on Migrate existing workloads (e.g., Synapse, Databricks, Data Mart) into Fabric."

Migration phases

Source	Usual target in Fabric	Tooling	Watch out for
Synapse dedicated SQL pool	Fabric Warehouse	Fabric Migration Assistant for Data Warehouse. Input is a DACPAC file or a direct connection. It translates T-SQL metadata, offers Copilot-assisted fixes, copies data with a copy job, and helps reroute connections.	Distribution and index options do not carry over. Some data types and T-SQL features differ. Workload management and security model change.
Synapse Spark pools	Fabric Data Engineering (Lakehouse, notebooks, Spark job definitions)	Migration assistant for Spark artifacts (announced; check status). Manual: export notebooks, recreate environments and libraries.	Runtime versions, library management, `mssparkutils` versus Fabric `notebookutils`, linked service references.
ADF / Synapse pipelines	Fabric Data Factory pipelines	Pipelines migration assistant (public preview when announced). Manual rebuild for unsupported activities.	Linked services become Fabric connections. Self-hosted IR becomes an on-premises data gateway. Triggers and parameters differ.
Azure Databricks	Often: keep Databricks, add Fabric for serving	Mirror Unity Catalog into OneLake, or create OneLake shortcuts to the Delta files. Full move: port PySpark notebooks to Fabric notebooks.	Declarative pipelines, Unity Catalog features, Photon and `dbutils` have no one-to-one Fabric match. Do a cost and capability comparison first.
Power BI datamarts	Fabric Warehouse	Microsoft guidance moves datamarts to Warehouse.	I am not certain of the current retirement dates. Verify before you mention dates.

Interview angle

"How do you prove the migrated warehouse is correct?"

Define reconciliation rules before the move: row counts, sums of key measures, distinct key counts, per table and per day.
Automate the checks. Run them on both platforms after every load during a parallel-run period.
Compare a fixed set of business reports, signed off by business owners.
Compare performance on the top queries from query history.
Cut over only when checks pass for an agreed number of cycles. Keep a rollback plan and a date to decommission the old system.

Your PM advantage

Migration questions are a good place to show delivery experience. Talk about inventory and dependency mapping, wave planning (low-risk workloads first), a RACI with business owners, parallel runs, cutover checklists and hypercare. Then show the technical depth: T-SQL translation issues, security mapping, capacity sizing.

Which Fabric item does the Migration Assistant for Data Warehouse create from a Synapse dedicated SQL pool?

Back: Fabric

Operate · 9

CI/CD, infrastructure as code and security

Notebooks in a workspace are not a product. Code lives in Git, tests run on every change, and the same definition deploys to dev, test and prod. Infrastructure is code too.

JD link: "Familiarity with modern CI/CD practices, especially in the context of Databricks", "ARM templates for IaC" and "Knowledge of cloud security best practices."

A Databricks release pipeline

Press Run pipeline. The run stops at the approval gate before production.

# databricks.yml (Declarative Automation Bundles, formerly Databricks Asset Bundles)
bundle:
  name: sales_lakehouse
resources:
  jobs:
    daily_sales:
      name: daily_sales
      tasks:
        - task_key: silver
          notebook_task: { notebook_path: ./src/silver_orders.py }
        - task_key: gold
          depends_on: [{ task_key: silver }]
          notebook_task: { notebook_path: ./src/gold_revenue.py }
targets:
  dev:  { mode: development, default: true, workspace: { host: https://adb-dev.azuredatabricks.net } }
  prod: { mode: production,                workspace: { host: https://adb-prod.azuredatabricks.net } }

# CI steps
databricks bundle validate -t dev
databricks bundle deploy   -t prod
databricks bundle run daily_sales -t prod

Databricks CI/CD

Git folders link workspace folders to a branch for development.
Declarative Automation Bundles (renamed from Databricks Asset Bundles in March 2026) define jobs, pipelines, permissions and targets in YAML.
New bundles from Databricks CLI 1.3.0 use a direct deployment engine by default instead of Terraform (per June 2026 release notes).
Run from Azure DevOps or GitHub Actions with a service principal. Do not use a personal token.
Unit test pure Python functions with pytest. Integration test on a test workspace with sample data.

Fabric CI/CD

Workspace Git integration with Azure DevOps or GitHub.
Deployment pipelines promote items dev → test → prod with deployment rules.
Fabric REST APIs for automation. A Python library for Fabric deployments exists (fabric-cicd); verify its current status.
Not every item type supports Git yet. Check the support list.

Infrastructure as code

ARM templates (JSON) or Bicep (simpler syntax, compiles to ARM) for Azure resources: workspace, storage, Key Vault, networking, Access Connector.
Terraform with the azurerm and databricks providers is very common for both Azure and workspace objects.
Keep environment values in parameter files. Never hard-code secrets.
Run what-if or plan in the pull request. Apply after approval.

Security checklist

Identity: Entra ID groups, service principals, managed identities. No shared accounts.
Secrets: Azure Key Vault, Key Vault-backed secret scopes.
Network: VNet injection, secure cluster connectivity (no public IP), private endpoints for workspace and storage.
Data: Unity Catalog least-privilege grants, row filters and column masks, encryption with customer-managed keys where needed.
Audit: system tables, diagnostic logs to Log Analytics, lineage.
Fabric: workspace roles, item permissions, OneLake security, Purview sensitivity labels.

Interview angle

"How do you enforce code quality across a team of data engineers?"

Branch policy: no direct commits to main. Pull requests need one reviewer and passing checks.
Linting and formatting in CI (for example ruff and sqlfluff). Tests for transformation logic.
Shared templates: a standard bundle layout, a logging module, a data quality pattern.
Definition of done includes documentation, lineage check and a runbook entry.
Review a sample of merged pull requests each sprint. Use findings for coaching.

Which identity should the production deployment pipeline use to deploy to Databricks?

Back: Migration

Lead · 10

Leadership and stakeholder communication

The JD asks you to explain complex topics to non-technical people, mentor engineers and own delivery. Prepare short analogies and real stories.

JD link: "Excellent communication skills", "Leadership skills with the ability to manage and mentor a team of data engineers", and "Ensure proper documentation of architecture, processes, and data flows."

Analogies that work for business people

Medallion layers: a kitchen. Bronze is groceries as delivered, Silver is washed and cut ingredients, Gold is the plated dish.
Delta transaction log: a bank statement. The balance is not stored; it is the sum of all recorded transactions. You can see the balance on any past date.
Shuffle: sorting mail. Every post office sends letters to the right city before delivery. That transport is the slow part.
OneLake shortcut: a desktop shortcut. It opens the file where it lives. No copy.
Data quality expectations: a factory quality gate. Bad parts go to a reject bin with a reason. The customer gets only good parts.

Mentoring practices to describe

Pair on the first pipeline. Then review, then let them lead.
Code review as teaching: explain the "why" in comments.
Short internal sessions: one topic per week (for example, reading the Spark UI).
Certification paths with study time in the plan.
Architecture decision records (ADRs) so juniors see how decisions are made.

STAR stories to prepare

Prepare one story for each line below. Use Situation, Task, Action, Result. Put a number in the result (time saved, cost reduced, defects avoided).

You designed or changed a data architecture and explained the trade-off to management.
A pipeline failed in production. You found the root cause and stopped it from happening again.
You disagreed with a stakeholder on scope or approach and reached an agreement.
You raised the skill level of a team member or the whole team.
You delivered a migration or platform change with a fixed deadline.
You introduced a standard (code review, CI/CD, documentation) that the team adopted.

Gap check

If your hands-on years are below the JD number

The JD asks for 8+ years of building pipelines. Interviewers will test hands-on depth with follow-up questions.
Do not overclaim. State what you built yourself, what you led, and what you reviewed.
Show current hands-on practice: a personal Databricks project on GitHub, a Fabric trial workspace build, or a recent certification.
Relevant certifications to consider: Databricks Certified Data Engineer Associate or Professional, and Microsoft Fabric Data Engineer (DP-700). Verify current exam codes and names on the vendor sites.

Questions to ask them

How is the work split between Databricks and Fabric today, and where is it going?
Is the Fabric migration already planned, or will this role define it?
How many engineers are on the team, and at what levels?
What does the current CI/CD and environment setup look like?
Who are the main business consumers, and how do they use the data now?

Back: CI/CD

Practice · 11

Mock interview

Read the question. Answer out loud for about 90 seconds. Then flip the card and compare with the key points.

Topic

Click "Show key points" to flip.

Back: Leadership

Study

200 practice questions

Ten topics, twenty questions each. Open a question to see a short answer. Try to answer out loud first.

Reference

Tools and websites

Websites to read and practice on, and tools to install on your own computer. Start with the free ones. You do not need a company account to try most of the ideas in this guide.

Name	Type	Use it for	Module

Check links before you rely on them. I did not open every link. Vendors move pages and rename products. If a link fails, search for the page name. Free tiers and trial limits change, so read the current terms on each site.

A free practice setup in one evening

Install Python, then run pip install pyspark delta-spark. Build a small Delta table on your laptop.
Install VS Code, the Databricks extension and the Databricks CLI. Connect to a free Databricks workspace.
Start a Fabric trial, if your account allows it. Create a Lakehouse and load a CSV.
Install Power BI Desktop. Connect it to the SQL endpoint and build one report.
Put everything in a GitHub repository with a short README. Interviewers can open it.

Reference

Glossary in plain words

Short meanings for terms in the job description and in this guide. Use the search box.

Reference

Sources and verification notes

I checked product names and current recommendations against these pages on 10 Nov 2026. Everything else in this guide comes from my general knowledge. Statements marked "verify" are ones I am not fully certain about.

What happened to Delta Live Tables (DLT)? Databricks docs. Rename to Lakeflow Spark Declarative Pipelines, new Python module.
Data engineering with Databricks Databricks docs. Lakeflow Connect, Declarative Pipelines, Jobs; streaming tables and materialized views.
Configure Auto Loader streams in file notification mode Databricks docs. File events recommendation, requirements.
Auto Loader with file events overview Microsoft Learn.
When to partition tables on Azure Databricks Microsoft Learn. Liquid clustering recommendation.
Declarative Automation Bundles release notes Databricks docs. Rename (16 Mar 2026), direct deployment engine.
CI/CD with bundles and GitHub Actions Microsoft Learn.
Fabric decision guide: Warehouse or Lakehouse Microsoft Learn.
Manage and monitor a KQL database Microsoft Learn. Eventhouse and KQL databases.
Get started with Real-Time Intelligence in Fabric Microsoft Learn training.
Migration planning: Synapse dedicated SQL pools to Fabric Warehouse Microsoft Learn.
Migrate with the Fabric Migration Assistant Microsoft Learn.
From Azure Synapse and ADF to Microsoft Fabric Fabric Updates Blog. Pipelines and Spark migration assistants.
Mirroring Azure Databricks Unity Catalog to OneLake (GA) Fabric Updates Blog.
Retirement: Azure Synapse Link for Cosmos DB NoSQL Azure Updates mirror.
Synapse Analytics roadmap Directions on Microsoft (independent analyst, Sep 2026). No full retirement announced; Microsoft prefers migration to Fabric.

Not verified from a source in this guide: Spark default thresholds, VACUUM default retention, Databricks access mode names, the "AUTO CDC" name, Power BI datamart retirement dates, fabric-cicd status, and certification exam codes. Check these on official documentation before you use them as facts.