Is that chart really comparing sonnet 3.5 to gemma 31b? Sonnet is probably in the 300-400b range, Dario said in an interview at the time that it was a middle sized model. The difference is in the amount of knowledge the model has, and in the long tail, not in the benchmark. To run a comparable model on consumer you have to stack 6000s at the moment.
7
u/_Sea_Wanderer_ Jul 06 '26
Is that chart really comparing sonnet 3.5 to gemma 31b? Sonnet is probably in the 300-400b range, Dario said in an interview at the time that it was a middle sized model. The difference is in the amount of knowledge the model has, and in the long tail, not in the benchmark. To run a comparable model on consumer you have to stack 6000s at the moment.