My blogs
Explore by topic
Statistics & Inference (Theory)
Modeling
Systems & Infrastructure
Interpretability & Understanding
AI for Science (Domain)
Agents & Scientific Workflows
-
From Memory to Photonics: Solving the Next Bottleneck in AI Scaling
A future-me note on how FlashAttention is IO-aware, why AI scaling turns memory movement into communication movement, and why photonics matters for the next interconnect bottleneck.
-
Training a Language Model from Scratch (Part 2: FlashAttention and Device Memory)
A future-me note on why naive attention becomes a device-memory problem, how FlashAttention uses tiling, online softmax, log-sum-exp, and recomputation, and how Triton exposes the tile-level implementation model.
-
Training a Language Model from Scratch (Part 1: Building Blocks)
A future-me refresher on the main pieces behind a small Transformer language model: byte-level BPE, embeddings, RoPE, attention, normalization, loss, optimization, and decoding.
-
Data 145: Evidence and Uncertainty - Topic Map
A compact topic map for my Data 145 Phase 1 and Phase 2 notes.
-
Data 145 Phase 1: From MLE to Neyman-Pearson to Reward Models
My Data 145 Phase 1 notes: a broad roadmap of statistical inference, with connections to modern reward-based AI.