All Posts · 0xWay1 Blog

All Posts

Sep 8, 2026

From Scale to Capability Scaling Laws and Emergence

As large language models grow, so do their parameter counts, training data, and compute budgets. Performance generally improves with them. But saying that “bigger models are stronger” only describes the outcome. It does not explain whether the improvement follows a stable pattern, or how a fixed compute budget should be divided between model size and data. Scaling Law sdescribe the relationship between scale and loss. Emergent Abilities focus on a different layer of behavior: average loss may decrease smoothly, while performance on a specific task can rise sharply within a relatively narrow range of model scales. Together, they address two closely related questions: how large models improve as they scale, and why some capabilities appear to emerge suddenly.

14 min read
AI InfraNotes
A girl holding flowers beneath a bright blue sky
Sep 6, 2026

Before Learning AI Infra, I Went Back to Review C++

I thought learning AI Infra meant starting with CUDA. But once I began reading real code, the first thing that kept slowing me down was C++: What does a pointer actually point to? When is an object already gone? What does std::move really move? Why does contiguous memory matter so much for tensors? And why are runtimes full of threads, locks, and CMake? So I went back and reviewed C++, focusing only on the parts that keep showing up in AI Infra code.

36 min read
C++