Databricks Data Engineer Interview: A Broadcast Join Case Study

Why a 50MB table can cause a driver OOM crash, and how concurrency is the real culprit.

Senior Databricks Interview: ‘I forced a Broadcast Join on a tiny 50MB table. The driver crashed with OOM. Why?’

(Hint: It’s not the table size).

A junior engineer blames the configuration. A senior engineer blames the concurrency.

Here is the offer-winning breakdown:

❌ The Junior Answer (Rejected) “The table must have been bigger than you thought, or spark…