44% on ARC-AGI-1 in 67 cents

The author trained a small transformer from scratch in 1.5 hours on an RTX 5090, achieving 44% accuracy on the ARC-AGI-1 benchmark for 67 cents and 7% on ARC-2, surpassing many larger LLMs. Improvements include a modern architecture (SwiGLU, RMSNorm), increased layer depth, reduced augmentations, and a supervised loss that focuses only on output tokens, yielding higher sample efficiency and lower compute cost.