↓ Skip to main content

Posts

2026

Fitting a 12B Model into Two Small CPU Nodes: Gemma 4 on Graviton with vLLM

In the last post I argued that you should not split a model that fits on one machine: the 1B model I benchmarked was faster, steadier and cheaper in a single pod. The flip side of that argument is the case where splitting is not a choice. When the weights do not fit, pipeline parallelism is how you run the model at all.

Don't Split a Model That Fits: One Pod vs Two for Gemma on CPU

In the last post I split a 1B-parameter Gemma model across two CPU nodes on purpose, to learn how LeaderWorkerSet and vLLM’s multi-node mode behave. I said then that splitting a model that fits on one machine makes it slower, and I listed “compare the split against a single replica” among the things I had not done.

Serving Gemma Across Two Nodes with vLLM and LeaderWorkerSet, on CPU

I wanted to try two things at once: Kubernetes’ LeaderWorkerSet (LWS), the API for running a group of pods as one unit, and vLLM’s multi-node serving. The cheapest way to exercise both is not a GPU cluster. It is two small CPU nodes, one small model, and a deliberate decision to split that model across the nodes even though it fits on one.

DOM Caching to Avoid Terminal Re-initialization in Electron

··719 words·4 mins
Worklayer has workspaces, and each workspace has multiple panels including terminals. The naive implementation of switching workspaces would be: destroy the current workspace’s DOM, render the new workspace’s DOM. This works for web panels and file editors which can reload their state from a URL or file path. But terminals are different.

Capturing OAuth Flows Inside an Electron App

··646 words·4 mins
CLI tools like gh auth login, gcloud auth login, and aws sso login all share a pattern: they print a URL or open the system browser for OAuth authentication, wait for the callback, and then continue in the terminal. When running these commands inside Worklayer’s terminal panels, the browser opens externally and the user has to leave the app to complete the flow.

Modern Filesystems: Journaling vs Copy-on-Write, the Kernel Write Path, and Data Correctness

··3861 words·19 mins
When engineers discuss storage and file systems, the conversation often starts and stops with the classics: ext4 on Linux, NTFS on Windows, and perhaps XFS for large database servers. For decades, the dominant answer to filesystem reliability was straightforward: journaling.

The Isolation Boundary of Multikernel Linux Containers

··2348 words·12 mins
The phrase “one kernel per container” sounds like a stronger isolation claim than “one process namespace per container.” In a Multikernel Linux runtime, each child really does have its own kernel, process table, boot ID, CPUs, memory description, and root filesystem. That still does not make it equivalent to a virtual machine.

Running OCI Processes Inside a Multikernel Linux Child

··1541 words·8 mins
In Turning Container Images into Multikernel Linux Roots, I described how an unpacked image becomes a private root for a child kernel. That gets the files into the right place, but files are not yet a running container. Something inside the child still has to interpret the OCI process configuration, establish its safety boundaries, start the executable, and translate later exec, signal, terminal, and wait requests.

Recovering Multikernel Runtime State After a Crash

··1357 words·7 mins
My earlier walkthrough of a Multikernel container from the primary Linux system followed a successful task from containerd through a shim, mkruntimed, Kerf, and finally a child kernel. The normal path is only half of a runtime, though. The harder question is what to do when one of those steps succeeds but the caller disappears before learning the result.

Anatomy of a Multikernel Container from the Primary Linux Host

··3145 words·15 mins
My earlier posts looked at individual parts of the Multikernel container path: building image-derived roots, providing mediated networking, and implementing a containerd Runtime v2 shim. This post puts those pieces together from one particular viewpoint: what happens on the primary Linux system when containerd creates a container whose process will run under another kernel?

Turning Container Images into Multikernel Linux Roots

··2663 words·13 mins
My first Multikernel Linux experiment on GCE started child kernels with roots that I built ahead of time. That was useful for proving that the kernels could boot, but it was not how a container runtime should consume an image. A runtime should accept the root filesystem that containerd has already prepared and should not become another registry client, layer downloader, or snapshotter.

Persistent ext4 Roots for Multikernel Linux on Google Compute Engine

··1340 words·7 mins
My first Multikernel Linux experiment on Google Compute Engine (GCE) showed that one VM could run a primary kernel and multiple child kernels at the same time. The children could use different kernel binaries, but their root filesystems were either minimal initramfs images or DAXFS. That left an important question unanswered: could a child kernel have a persistent, read-write ext4 root filesystem?

Can Multikernel Linux Run on Google Compute Engine?

··1336 words·7 mins
The idea behind Multikernel Linux is unusual: one physical machine can run several independent Linux kernels at the same time, without placing a conventional hypervisor between those kernels and the hardware. A primary kernel owns the machine’s resources, moves selected CPUs, memory, and devices into a pool, and starts child kernels from that pool.