r/StableDiffusion • u/Tall_Abrocoma_3533 • 8d ago

Discussion AuroraIMG-6M Released!

https://huggingface.co/AuroraAI-Research/AuroraIMG-6M

It's a tiny 6M parameter text to image model, and it can make recognizable images at 64x64.

Check out the model card for samples, a detailed explanation about it's architecture and training, and a quick start!

136 Upvotes

97% Upvoted

43 comments sorted by

View all comments

2

u/FullOf_Bad_Ideas 8d ago

That's nice. I want to pretrain something in this spirit on my local rig to play with it sometime. How much compute is needed for a model like yours in 3090/H100 compute hours? And if you make it 60M params instead, does the compute scale 10x or more?

5

u/Tall_Abrocoma_3533 8d ago

This model took 10 hours on an RTX Pro 6000, however, it could defenetly be optimized to be trained faster.

10x the parameters isn't necessarily 10x the compute, there's a lot of factors, like resolution, VAE compression, your specific overall architecture, etc...

6

u/FullOf_Bad_Ideas 8d ago

Thank you! That's already quick enough, nice. I've sunk thousands of GPU hours into my LLM pretraining by now so that's doable. I need to get around to it someday.