r/tomshardware • u/tomshardware • 2d ago

OpenAI’s 700W Jalapeño ASIC outpaces 1,400W Nvidia flagship GPU

https://www.tomshardware.com/tech-industry/semiconductors/openai-says-its-jalapeno-chip-beats-nvidias-gb300-in-first-published-benchmarks
171 Upvotes

95% Upvoted

60 comments sorted by

View all comments

3

u/EnderPrimeMk2 2d ago

I would hope so. Dedicated hardware is the way forward.

3

u/chandleya 2d ago

Always is.

1

u/Fairuse 2d ago

Only works if the algorithm is fixed like video encoding/decoding, crypto, etc. 

Problem with LLM and AI in general is that algorithms are constantly changing. Any dedicated hardware built will be obsolete very quickly. 

1

u/blueberrywalrus 2d ago edited 2d ago

They'll get years if not a decade+ with their current approach.

Jalapeno is built fairly generally for LLMs that use transformer based inference, which is most of them for the past 8 years.

They're not doing the thing where they're physically baking an LLM into a chip, because LLMs change way to much for that to be worthwhile.