Daily build log
One short doc per day: what was added, why, and what was learned. Newest entries sort to the bottom; the full plan they follow is in The 100-day plan.
Table of contents
- Day 01: scaffolding and the plan
- Day 02: weights, config, and a verification harness
- Day 03: the loader, and the name-mapping rabbit hole
- Day 04: RMSNorm and RoPE, position as a rotation
- Day 05: SwiGLU, the gated MLP every modern model uses
- Day 06: GQA attention, the one sublayer that mixes tokens
- Day 07: one full transformer block, end to end vs HuggingFace
- Day 08: the full forward pass, matching HuggingFace to 2.4e-5
- Day 09: greedy decode, the whole continuation matches HuggingFace
- Day 10: sampling, pinned bit-for-bit to the HuggingFace warpers
- Daily template