BigQuery: working of "LEAF NODE"

I’ve spent my first week diving into BigQuery's internals. Everyone talks about "serverless," but the real magic happens at the Leaf Node level.

If you’re used to Spark Executors or traditional MPP workers, the way BigQuery handles "Leaf Nodes" (also known as Slots) is a total paradigm shift. Here is the deep-dive on how they actually handle your data.

1. The Shard Discovery & Pickup

When you submit a query like "Find the best employee per department" .
BigQuery doesn't see a single table. It sees a collection of Capacitor files (shards) stored in Colossus.

• The Mechanism: The Root Node identifies all relevant shards. Leaf Nodes don't "own" data; they are assigned a set of physical file metadata pointers.

• "If": If your query uses a specific WHERE clause, the Leaf Node uses columnar pruning to scan only the necessary bytes. It doesn't even pull the rest of the row into its memory.

2. Handling "Heavy" Shards (Dynamic Re-sharding)

What happens if one shard is massive or contains highly skewed data? In a traditional system, you’d wait for that one "Straggler" node to finish. BigQuery solves this with Dynamic Work Rebalancing:

// Pseudocode of the Control Plane Logic:
IF LeafNode_A is processing Shard_101 for > 500ms
AND Idle_Slots > 0
THEN 
    INSTRUCT LeafNode_A to "STOP at Row 5,000,000"
    ASSIGN Shard_101 (Rows 5,000,001+) to LeafNode_B
    MERGE results in the Shuffle Tier

• "But": This only works because storage is decoupled. Any node can pick up any piece of data at any time via the 1Pbps Jupiter Network.

3. The Stateless "Shuffle Tier" Handshake

Unlike Spark, where executors often write shuffle data to their own local disks (leading to No space left on device errors), BigQuery Leaf Nodes are stateless.

• The Execution: The Leaf Node performs local filtering and partial sorting, then pushes intermediate results to a shuffle layer. While the exact implementation is abstracted, the key idea is that shuffle state lives outside the leaf node.

• The Benefit: If a Leaf Node dies mid-query, BigQuery doesn't care. It simply re-assigns that shard's offset to a new node. No data is lost because the node was never the "source of truth."

The "Iffs and Buts" of Slot Performance

My Takeaway for Week 1

The Leaf Node is essentially a disposable worker. Because it doesn't "own" the data, BigQuery can treat it like a commodity spinning it up, killing it, or splitting its work in milliseconds. It’s a beautifully orchestrated dance of thousands of tiny workers.