Breaking through AI’s memory wall with token warehousing

WEKA CTO Shimon Ben-David discusses the "memory wall" limiting AI scalability, citing 40GB of GPU memory needed for 100,000-token sequences, and introduces "token warehousing" as a solution, enabling 4.2x more tokens per GPU and potential millions in daily savings.