All Posts · 0xWay1 Blog

All Posts

Sep 13, 2026

LLM Inference AI Infra

Once a large language model is actually deployed, the main bottlenecks are often no longer about whether the model can compute the answer. They are about how data moves, how GPU memory is managed, and how requests are scheduled. KV Cache, Prefill and Decode, PagedAttention, parallelism, and inference engines may look like separate topics, but they all serve the same goal: making limited GPU resources support longer contexts, higher concurrency, and lower latency.

26 min read
AI InfraNotes
A girl playing guitar beside a glowing carousel at night
Sep 12, 2026

Large Model Training and Inference

The hardest part of getting into AI Infra is not the concepts themselves, but the sheer number of tools: CUDA, NCCL, DeepSpeed, Megatron-LM, vLLM. Once they are placed back into the actual execution path of a large model, the picture becomes much clearer. Most of them are solving a small set of recurring problems: limited GPU memory, slow communication, underutilized GPUs, and inefficient resource use during inference. AI Infra is essentially about these system bottlenecks.

23 min read
AI InfraNotes