← All Notes January 3, 2025

The Software Stack of Cards

The Analogy

Software is a stack of cards. The smallest cards are at the bottom—closest to the hardware. The largest cards are at the top—the highest abstractions. Each card rests on the one below it.

A small movement at the bottom shifts everything above.

This is what makes low-level code so critical, and why “just change the database” is never simple. The foundation isn’t big and sturdy—it’s small and precise. That’s what makes it powerful. And terrifying.

What Are the Cards?

Each card represents a layer of abstraction over two fundamental resources:

Compute — CPU cycles, instructions, work
Memory — Allocations, storage, state

Every layer translates your intent into something the layer below can understand. This continues until you hit actual electrons moving through silicon.

Going up = more convenience, more hidden work Going down = more control, more visibility

The Innocent One-Liner

Consider this simple operation:

np.array([1,2,3]) + np.array([1,2,3])

Output: [2, 4, 6]

That innocent + is standing on top of many cards:

Hardware — Transistors flipping, electrons moving, registers and cache
CPU instructions — ADD, MOV, memory access patterns
OS Kernel — Memory allocation, scheduling, syscalls
System libraries — glibc, memory allocators
Language runtime — Python interpreter, garbage collection, GIL
C extensions — numpy’s core doing loops, SIMD optimizations
Python C API — Bridging to the numpy extension
numpy array object — The abstraction you interact with
Your code — The friendly + that “just works”

Each card is a little bigger than the one below. More abstraction, more convenience, more stuff happening—all balanced on that tiny, precise foundation of bits and cycles.

Stacking Stacks

Now imagine that numpy operation running in a typical cloud environment:

np.array([1,2,3]) + np.array([1,2,3])
        ↓
   Python interpreter
        ↓
   numpy C extensions
        ↓
   glibc / memory allocation
        ↓
   Container's isolated filesystem & namespaces
        ↓
   Docker runtime (containerd, runc)
        ↓
   MicroVM kernel (Firecracker, gVisor, etc.)
        ↓
   Hypervisor (KVM, Xen, etc.)
        ↓
   Host OS kernel
        ↓
   Actual hardware

All that coordination, all those context switches, all that abstraction overhead… to compute [2, 4, 6].

Every layer has a cost:

Context switches
Memory copies
Scheduling overhead
Coordination code

The industry has traded efficiency for convenience, isolation, and developer velocity. Sometimes that tradeoff makes sense. Sometimes we’re running a simple addition through seventeen layers of indirection because “that’s how we deploy things now.”

Counting Your Cards

What if we chose a different stack?

The same operation in Rust + WASM on Cloudflare Workers:

Rust code → compiled to WASM
        ↓
   V8 isolate (lightweight, no cold start)
        ↓
   Edge node hardware

That’s it. A handful of cards.

WASM is already close to metal
V8 isolates are tiny and fast
No container orchestration
No nested virtualization

The work is the same. The path to get there is dramatically shorter.

The Cost of a Card

Every layer has measurable costs:

Cost Dimension	What It Means
Lines written	Your code—what you see
Lines executed	Library code, runtime code, OS code—what actually runs
CPU instructions	The actual work the processor does
Memory allocated	Heap allocations, stack frames, buffers
Syscalls	Crossings between user space and kernel
Context switches	Scheduler interruptions, VM boundaries
Time	Wall clock duration
Energy / Cost	Compute → electricity → dollars

Abstraction Budget

Think of abstraction as a budget. Every layer you add spends some of that budget.

The question isn’t “should we use abstractions?”—of course we should. The question is:

For this workload, is this layer worth its cost?

Some examples:

Workload	Good Fit	Poor Fit
Stable API, constant traffic	Containers, Kubernetes	—
Tiny, bursty data pipeline jobs	WASM, lightweight runtimes	Containers (overhead dominates)
Tight numerical loops	Rust, C, Polars	Pure Python
Rapid prototyping	Python, high-level frameworks	Low-level manual memory management

Finding Wins Within Constraints

You don’t always control the whole stack. Team dynamics, company uniformity, existing infrastructure—these matter for reasons beyond pure architecture.

But you can find workarounds.

Example: Polars in a Python shop

df.lazy()
  .filter(...)
  .group_by(...)
  .agg(...)
  .collect()

It looks like Python. But underneath? Rust. Vectorized. No Python GIL strangling your tight loops.

You’ve collapsed part of the stack without fighting the team’s workflow.

The Benchmark Framework (Future Work)

What if we could measure, for any given operation:

Input: 1 line of high-level code

Output:

Lines of library code executed
Interpreter instructions
Syscalls
CPU instructions (via perf, etc.)
Memory allocated
Wall time
Estimated cost ($)

Then compare implementations:

Implementation	Your Code	Instructions	Syscalls	Time	Est. Cost
numpy + Python + Docker + VM	1 line	50M	200	15ms	$$$
Polars (lazy) + Python + Docker	1 line	5M	80	2ms	$$
Rust + WASM + edge	1 line	50K	2	0.1ms	$

Same work. Different stack. Wildly different cost.

This gives concrete data for:

Teaching — “Here’s what your innocent one-liner actually costs”
Decision-making — “Is this abstraction layer worth it for this workload?”
Optimization — “Where in the stack is the bloat?”
Cost awareness — More compute → more $ → should be a conscious choice

The Key Question

For any piece of software, ask:

How many cards are we going through, and what does each card cost?

Most developers live entirely in the top few cards and have no idea what’s beneath them. Which is fine—until something wobbles at the bottom.

Seeing the whole stack doesn’t mean you have to optimize everything. It means you can make informed tradeoffs instead of accidental ones.

Summary

Software is a stack of cards: small at the bottom, large at the top
Each card is an abstraction over compute and memory
Every layer has a cost in instructions, allocations, time, and money
The same work can take wildly different paths through the stack
Abstraction is a budget—spend it wisely
You don’t need to control the whole stack to make it better
The goal: informed tradeoffs, not accidental complexity