↓ Skip to main content
  1. Article Categories/

Ai

Fitting a 12B Model into Two Small CPU Nodes: Gemma 4 on Graviton with vLLM

In the last post I argued that you should not split a model that fits on one machine: the 1B model I benchmarked was faster, steadier and cheaper in a single pod. The flip side of that argument is the case where splitting is not a choice. When the weights do not fit, pipeline parallelism is how you run the model at all.

Don't Split a Model That Fits: One Pod vs Two for Gemma on CPU

In the last post I split a 1B-parameter Gemma model across two CPU nodes on purpose, to learn how LeaderWorkerSet and vLLM’s multi-node mode behave. I said then that splitting a model that fits on one machine makes it slower, and I listed “compare the split against a single replica” among the things I had not done.

Serving Gemma Across Two Nodes with vLLM and LeaderWorkerSet, on CPU

I wanted to try two things at once: Kubernetes’ LeaderWorkerSet (LWS), the API for running a group of pods as one unit, and vLLM’s multi-node serving. The cheapest way to exercise both is not a GPU cluster. It is two small CPU nodes, one small model, and a deliberate decision to split that model across the nodes even though it fits on one.

Building a Custom MCP Server with Chrome DevTools Protocol

··718 words·4 mins
When using Claude Code inside Worklayer’s terminal panel, I wanted it to be able to interact with web pages displayed in adjacent web panels. The standard approach would be to use the Playwright MCP server, but that spawns a separate Chromium instance outside the app. The page Playwright controls and the page the user sees are two different browser sessions with no shared state.

Measuring Coding Tool Effectiveness

··520 words·3 mins
Most online content regarding AI coding tools focuses heavily on input and output token counts. While these metrics are useful for understanding the raw volume of data processed, they often fail to address the actual effectiveness of those tokens in solving real-world engineering problems. Measuring the true impact of these tools on development workflows remains a challenge because volume does not equate to value.