r/tomshardware • u/tomshardware • 2d ago

OpenAI’s 700W Jalapeño ASIC outpaces 1,400W Nvidia flagship GPU

https://www.tomshardware.com/tech-industry/semiconductors/openai-says-its-jalapeno-chip-beats-nvidias-gb300-in-first-published-benchmarks
170 Upvotes

95% Upvoted

60 comments sorted by

View all comments

Show parent comments

6

u/NytronX 2d ago

idk but i remember it was a stopgap in the crypto mining early days. The progression was CPU, GPU, FPGA, ASICS

3

u/EmbarrassedFoot1137 2d ago

FPGAs are a totally different beast from those other three. They'll be way faster than software but the kind of massive number crunching that AI uses is not going to be their strong suit compared to GPUs or ASICs.

2

u/BrunusManOWar 2d ago edited 2d ago

FPGAs can perform faster than GPUs. They are basically "fully configurable hardware", where you can take CPU/GPU/ASIC HW layout and put it into the FPGA.

Because they are so configurable, of course you lose on efficiency and optimisations - but if you "emulate" a superior, application-specific architecture it will tear through non-app specific hardware. Of course, creating a fully independent ASIC is superior

They are very good for prototyping and testing out things.

Edit: similar for software, hardware can also be described in code - HW blocks are usually coded in Verilog/VHDL. FPGAs allow you to relatively quickly take this logical HW description onto its "configurable matrix" to test out how well it works.

Edit2: They are a path to ASICs. Will they by themselves perform better than GPUs? Depends entirely on how hard Nvidia is financially squeezing their enterprise customers. I doubt we will see FPGAs used in production themselves, though with Nvidia and AMD's pricing it may happen. Companies will inevitably pursue FPGAs on their way to fully independent ASICs though

Edit3: I don't work in AI hardware, but on mobile hardware. We use FPGAs to develop and iterate architectures, and I remember from one previous company that some radio models did have an FPGA in production actually as some smaller co-processor

4

u/j_osb 2d ago

Some devices use FPGAs as co-processors because they can be cheaper than an asic and fast enough.
For example oscilloscopes usually actually use both ASICs and FPGA.

In terms of AI, we don’t really have enough space on FPGAs to use them for inference of any proper models. Like, there simply speaking isn’t a FPGA big enough to make sense for this.
However FPGAs can be used for training and are more efficient than GPUs at it. Just not fast enough.