▸ Series
▸ Standalone Posts
// no posts yet
▸ Pills
-
Inside the Delta Lake Transaction Log
How Delta Lake uses a sequence of JSON commit files and Parquet checkpoints to turn a directory of Parquet files into an ACID-compliant table — and what that means for reads, writes, and time travel.
delta-lake internals storage -
Spark's Physical Execution Model: From DAG to Task
The DAG is a logical plan. This post covers what actually happens between calling .count() and getting a number back — how stages are cut, what the task scheduler does, and where the shuffle lands on disk.
spark internals execution