Mini-AGI – dynamic continual learning model trained from scratch on 8GB VRAM

Mini-AGI is a byte-level continual learning language model that trains from scratch on a single 8 GB VRAM GPU, storing weights on disk and paging them into memory as needed. It expands capacity during training, prunes unused parameters, and learns from a continuous data stream without catastrophic forgetting. The architecture uses two dense prelude blocks followed by up to 24 recurrent block applications, each selecting experts from a shared pool, with adaptive depth controlled by a halting head.