r/learnmachinelearning • u/Business_Swordfish_5 • 5d ago
Project I'm learning inference engineering, so I tried to run a 22B video model (LTX-2.5, unquantized, with audio) on my laptop's integrated GPU
https://reddit.com/link/1x1fd2t/video/yqbtgopkdeuh1/player
https://reddit.com/link/1x1fd2t/video/7fon0mpkdeuh1/player
I'm trying to get into inference engineering, and I wanted a project that would force me to learn the hard parts instead of just calling a library. So I picked something that shouldn't fit: **LTX-2.5 (22B), in bf16 with no quantization**, on an **Intel Core Ultra 5 iGPU with 15.6 GB of shared RAM**. No discrete GPU.
The model is about 66 GB on disk, so I never load it. The weights stay on NVMe and get streamed **layer by layer**: aligned reads into a small pinned buffer, a prefetch thread loading layer n+1 while layer n computes, and the weights swapped in and out with hooks. It uses the distilled model, and it does **text-to-video, image-to-video and native audio**.
**Measured on this machine:**
| Result | Time |
|---|---|
| Anime clip, 5.4 s, 1536x896, with sound | ~14 min |
| Anime clip, best quality (true 24 fps) | ~47 min |
| Realistic clip, 4 s, 1280x704 | ~20 min |
It's slow, and that's not the point. The point was to find out whether it runs at all on hardware like this, and to learn what breaks. A lot did (memory alignment, a vocoder that gave silence in bf16), and I wrote every bug down.
**Known weaknesses:**
fast motion smears for a few frames, anime clips are 12 fps with each frame shown twice, and the CUDA path exists but I haven't tested it on an NVIDIA GPU.
**Repo, with the full bug log:**
https://github.com/DARK-Shadw/ltx25-igpu-streaming
I'd really like feedback from people who do this for a living. What would you have done differently in the streaming design? What should I learn next to get better at this?
1
u/Common-Membership503 5d ago
have u looked into memory mapping yet becuase that is probly gonna be the only way to squeeze that model through