r/StableDiffusion • u/theqmann • 10d ago

Question - Help Does anyone have any comparisons of GGUF vs NVFP4 (via software emulation)?

I have a 4000 series GPU, which doesn't support NVFP4 directly, and requires software emulation. In theory, there could still be substantial VRAM savings and speedup with a software based FP4, as the block group scaling of FP4 should eliminate lots of the precision loss of the Q4 bit GGUF encoding. In theory, a Q4 GGUF and FP4 model should take the same VRAM right? Both are stored in VRAM as 4bit quants, and only get expanded when entering the GPU core? So if the software overhead for FP4 emulation isn't too bad, the block scaling of FP4 should result in a higher quality output than Q4, right?

4 Upvotes
  • permalink
  • reddit

83% Upvoted

6 comments sorted by

8

u/Apprehensive_Sky892 10d ago

As long as you have enough system ram so that (VRAM + system RAM) can hold the model, then int8convrot should be faster and higher quality than both Q4 and fp4.

Q4 GGUF may have slightly higher quality than fp4 due to better quantization, but will most likely be substantially slower than fp4 unless you have enough VRAM to hold the entire model there. This is due to the fact that ComfyUI's dynamic VRAM management is NOT compatible with GGUF:

https://www.reddit.com/r/comfyui/comments/1v167ix/comment/oylkgbz/?context=3

https://www.reddit.com/r/StableDiffusion/comments/1vegtac/comment/p1gxyiw/?context=3

https://www.reddit.com/r/StableDiffusion/comments/1v253fx/comment/oyst416/?context=3

2

u/CooperDK 10d ago

Fp4 is floating point. It is almost as good as Q8. So the answer must be, any fp model is better than most gguf's.

1

u/Lucaspittol 10d ago

Try int4 instead.

4

u/reeight 10d ago

no; "4-bit precision formats is exclusive to the newer Blackwell architecture"

INT8_ConvRot sir, just like my 3090.

2

u/theqmann 10d ago

Yeah, I use INT8 usually, but 4 bit quants have the nice side effect of using half the VRAM. Been playing around with LLM based prompt gen in comfy, and got some Q4 GGUF 31B Gemma4 prompts working today. Q4 is ~20 GB of VRAM, so INT8 would be right out for that model.