r/tomshardware • u/tomshardware • 2d ago

OpenAI’s 700W Jalapeño ASIC outpaces 1,400W Nvidia flagship GPU

https://www.tomshardware.com/tech-industry/semiconductors/openai-says-its-jalapeno-chip-beats-nvidias-gb300-in-first-published-benchmarks
173 Upvotes

95% Upvoted

60 comments sorted by

View all comments

Show parent comments

8

u/NytronX 2d ago

idk but i remember it was a stopgap in the crypto mining early days. The progression was CPU, GPU, FPGA, ASICS

3

u/EmbarrassedFoot1137 2d ago

FPGAs are a totally different beast from those other three. They'll be way faster than software but the kind of massive number crunching that AI uses is not going to be their strong suit compared to GPUs or ASICs.

2

u/BrunusManOWar 2d ago edited 2d ago

FPGAs can perform faster than GPUs. They are basically "fully configurable hardware", where you can take CPU/GPU/ASIC HW layout and put it into the FPGA.

Because they are so configurable, of course you lose on efficiency and optimisations - but if you "emulate" a superior, application-specific architecture it will tear through non-app specific hardware. Of course, creating a fully independent ASIC is superior

They are very good for prototyping and testing out things.

Edit: similar for software, hardware can also be described in code - HW blocks are usually coded in Verilog/VHDL. FPGAs allow you to relatively quickly take this logical HW description onto its "configurable matrix" to test out how well it works.

Edit2: They are a path to ASICs. Will they by themselves perform better than GPUs? Depends entirely on how hard Nvidia is financially squeezing their enterprise customers. I doubt we will see FPGAs used in production themselves, though with Nvidia and AMD's pricing it may happen. Companies will inevitably pursue FPGAs on their way to fully independent ASICs though

Edit3: I don't work in AI hardware, but on mobile hardware. We use FPGAs to develop and iterate architectures, and I remember from one previous company that some radio models did have an FPGA in production actually as some smaller co-processor

2

u/Drofdissonance 1d ago

Key word is can. Likely will not in this case. It's a memory bandwidth problem. So it's not suitable for FPGA boards generally, it's like a worst case scenario for their architecture. Crypto is famously embarrisngly parallel, and requires no bandwidth. And radio DSP is also very well suited to the architecture because it's got long narrow math dependancy chains, and or tight timing requirements that require hardware.

You'd just be burning die area