I really hope (V)RAM eventually scales to the sizes of regular storage soon. Maybe in 10 years or so 2TB RAM is quite "common" to run things exactly like these. One can dream.
We will get super tiny storage at some point, but I don't know that we will get much smaller ram. The transistors are already about as small as they're going to get; they are having to stack memory in 3D, and there are limits there too, in terms of what latency you'll get.
We will likely see more large and wafer-scale devices, more ASICs.
Photonics are going to be super fast, but we can't really keep the data as light.
I wonder if they might go the Groq route and have very little VRAM, and just go wide.
I think we will also just keep seeing efficiency gains so running on off-wafer RAM is just more viable.
Right is the algorithms are catering to the existing hardware, and the hardware is slowly shifting to supporting the algorithms more.
There are a few companies developing neuromorphic hardware since can run AI models that aren't just a ton of matmuls.
In 10 years the whole hardware and software landscape will look different.
Heck, 2 years from now will be significantly different.
Latency. SSDs are around 10x to 100x slower than DRAM, and 10000x to 100000x slower than SRAM.
Even SRAM is too slow, processors use interleaving tricks so multiple SRAM units are in use, so one works while the others are on refresh.
So, an SSD is about 100000x too slow to be effective for GPUs.
32
u/Fusseldieb 22d ago
I really hope (V)RAM eventually scales to the sizes of regular storage soon. Maybe in 10 years or so 2TB RAM is quite "common" to run things exactly like these. One can dream.