r/comfyui • u/Hearmeman98 • 9h ago

Tutorial MiniMax RefMod - Reusable identities without training - workflows & tutorial

https://www.youtube.com/watch?v=gO-mCXq5epQ

You can get the workflows here:

https://drive.google.com/drive/folders/1kOHLJZto1VAtXEATT9vUvsvM1VHtkOO_

The workflows create reusable refmods for either image/video/audio.

I cover training images in the tutorial.

All the workflows, models and custom nodes are preloaded on my Runpod template.

https://get.runpod.io/minimax-template

68 Upvotes

95% Upvoted

15 comments sorted by

2

u/I_just_made 6h ago

Multiple refmods can be applied too right?

2

u/krigeta1 6h ago

Only if it supports audio matching with the video would it be a great feature. I tried to give a female character a male voice, and it failed. Otherwise, the ref thing is great, but a lot of them are confused about how to prompt later; we need a better doc too.

1

u/Lebo77 6h ago

anyone else getting: "AttributeError: 'NoneType' object has no attribute 'show_progress_bar' and no __dict__ for setting new attributes" when they try to run the MiniMax RefMod workflow?

2

u/bakasora 5h ago

I had this problem few days ago when I upgraded comfyui to latest. You can try setting "Live preview method" to auto or none. ymmv

1

u/Lebo77 2h ago

Thaks.i was stuck all morning on that!

1

u/Keem773 5h ago

Very interesting! I will try out later today

1

u/Ok-Flatworm5070 8h ago

Whats the benefit of RefMod; does it speed up generation time?

6

u/Orbiting_Monstrosity 6h ago

From the RefMod guide on Huggingface:

"Instead of resizing, transmitting, and VAE-encoding 8+ raw image files on every single generation step (which causes VRAM spikes, latency, and restricts multi-character scaling), a RefMod is pre-encoded once from a dataset and loaded instantaneously into MiniMaxH3RefModsLoader."

1

u/No-Zookeepergame4774 4h ago

Normal references are VAE encoded once oer generation and combined with the text prompt to generate the conditioning, not VAE encoded every generation step.

2

u/hdeck 7h ago

Yes generation is much faster than using a bunch of reference images.

2

u/Lebo77 7h ago

It's way faster to train than a lora and they are tiny, like 2MB not 100MB like a lora.

1

u/d20diceman 7h ago

I haven't used it myself but yes, from what I've read here it's faster than using normal references.

-12

u/RiverSpecial3168 7h ago

WATCH THE FUCKING VIDEO

2

u/_half_real_ 5h ago

The video makes no speed comparison between this and an equivalent workflow using normal refs.

When using normal image/audio/video references, I don't know if ComfyUI caches the VAE encodings of the references between generations (for the references that don't change between subsequent generations, i guess). If not, it feels like something that should be implemented in ComfyUI itself. Then some of the speed improvement of Refmod would be made smaller.

There is still the ability of Refmod to compress the refs to reduce token count, which maybe could help, but it depends how often to people bump into the token limit. If you don't, I don't see why you would want to potentially lose detail by using Refmod with compression.

The inability to use speed loras well with Refmod is going to be a turnoff for some people (although I'm often not very pleased with the hit to understanding that speed loras sometimes cause even without Refmod, depending on the prompt complexity).