About
Learning inference infrastructure the slow way
I'm Min Liu. For the past several years I've built B2B SaaS billing systems — subscription lifecycles, proration, invoice generation, the kind of software where a rounding error shows up on someone's bank statement. It taught me to chase a number until it reconciles.
I'm now pointing that habit at GPUs. This site is the working log: what I measure, what I got wrong, and what the arithmetic actually says. Not tutorials — I'm not far enough along to teach anyone. Closer to lab notes that happen to be public.
What gets written here
The plan, roughly in order:
- Reading the hardware. Roofline analysis, arithmetic intensity, and using a profiler to tell memory stalls from launch overhead.
- Kernels. Triton, then the fused patterns that matter for attention and normalisation.
- Engines. Building a minimal inference engine end to end — KV cache, batching, scheduling — then reading vLLM to find out how wrong I was.
- Scale. Sharding, and what actually happens when a node fails mid-run.
House rules
Every performance claim comes with the hardware, the shapes, and how it was measured. If a number is from a spec sheet rather than my own run, it says so. Posts get corrected in place, with the change noted at the bottom — I'd rather be usefully wrong in public than quietly vague.