#Notes

Sep 13, 2026

LLM Inference AI Infra

Once a large language model is actually deployed, the main bottlenecks are often no longer about whether the model can compute the answer. They are about how data moves, how GPU memory is managed, and how requests are scheduled. KV Cache, Prefill and Decode, PagedAttention, parallelism, and inference engines may look like separate topics, but they all serve the same goal: making limited GPU resources support longer contexts, higher concurrency, and lower latency.

26 min read

AI InfraNotes

A girl playing guitar beside a glowing carousel at night

Sep 12, 2026

Large Model Training and Inference

The hardest part of getting into AI Infra is not the concepts themselves, but the sheer number of tools: CUDA, NCCL, DeepSpeed, Megatron-LM, vLLM. Once they are placed back into the actual execution path of a large model, the picture becomes much clearer. Most of them are solving a small set of recurring problems: limited GPU memory, slow communication, underutilized GPUs, and inefficient resource use during inference. AI Infra is essentially about these system bottlenecks.

23 min read

AI InfraNotes

Sep 8, 2026

From Scale to Capability Scaling Laws and Emergence

As large language models grow, so do their parameter counts, training data, and compute budgets. Performance generally improves with them. But saying that “bigger models are stronger” only describes the outcome. It does not explain whether the improvement follows a stable pattern, or how a fixed compute budget should be divided between model size and data. Scaling Law sdescribe the relationship between scale and loss. Emergent Abilities focus on a different layer of behavior: average loss may decrease smoothly, while performance on a specific task can rise sharply within a relatively narrow range of model scales. Together, they address two closely related questions: how large models improve as they scale, and why some capabilities appear to emerge suddenly.

14 min read

AI InfraNotes

A girl holding flowers beneath a bright blue sky