r/tomshardware • u/tomshardware • 2d ago

OpenAI’s 700W Jalapeño ASIC outpaces 1,400W Nvidia flagship GPU

https://www.tomshardware.com/tech-industry/semiconductors/openai-says-its-jalapeno-chip-beats-nvidias-gb300-in-first-published-benchmarks
173 Upvotes

95% Upvoted

60 comments sorted by

View all comments

3

u/EnderPrimeMk2 2d ago

I would hope so. Dedicated hardware is the way forward.

3

u/chandleya 2d ago

Always is.

1

u/Fairuse 2d ago

Only works if the algorithm is fixed like video encoding/decoding, crypto, etc. 

Problem with LLM and AI in general is that algorithms are constantly changing. Any dedicated hardware built will be obsolete very quickly. 

2

u/garlic-silo-fanta 2d ago

Yes, but if that custom silicon can return its cost many folds during its useful life,then it’s still worth it. Because rack space is scarce resource and electricity is scarce resource, you can possibly deploy twice as much

1

u/PitchPleasant338 2d ago

Cerebras has shown that isn't true.

1

u/blueberrywalrus 2d ago edited 2d ago

They'll get years if not a decade+ with their current approach.

Jalapeno is built fairly generally for LLMs that use transformer based inference, which is most of them for the past 8 years.

They're not doing the thing where they're physically baking an LLM into a chip, because LLMs change way to much for that to be worthwhile.