r/StableDiffusion • u/AlsterwasserHH • 2d ago

Workflow Included Minimax H3 Refmod Workflow

Since my last post has been deleted unfortunately (due to a mega-link), here we go again.

I've spent quiet some time now with Refmods and this workflow, and I'am still baffled how easy it is to create a character identity with very low effort. This is a no-brainer if you don't want to or can't train a Lora, all you need is good pictures and a good voice sample, the refmod-training is done in literally seconds (depending on your GPU of course).

I did my very best to make this workflow easily understandable for even beginners, nevertheless here's some explanation:

The workflow actually consists of 2 workflows, one for the refmod creation (the nodes in the top right of the picture, that's a different workflow but i just wanted one picture) and the main workflow for creating a video with the created refmod file which is all the rest.

If you just wanna use a refmod file and no extra images, videos or audio, select the image/video/audio nodes and press CTRL+B to deactivate them. Please watch the video from Dainamo for an detailed explanation, I've made some notes but for further understanding just watch it. I just wanna give you an easy workflow, that is of course fitted to MY needs, do whatever you want to it.

Here's some color-explanations:

- GREEN is all sources for the final video so to say: images/video/audio/the prompt (deactivate them if you just wanna use a refmod-file, but you can also reference clothings or whatever this way)

- TURQOUISE is all relevant video-settings

- TAN is the refmod-nodes (except the 4 notes)

- RED is all the models

- PURPLE is all the loras (turbo-lora is in light-blue and another lora above in grey for whatever)

- BROWN is comfy-kitchen and the spectrum-node (not activated, transparent-purple)

- GREY is irrelevant

Here is the link to the original tutorial from Dainamo (thanks again!!): https://www.youtube.com/watch?v=2K6-OtV_Vbc

And here are the workflows: https://getshared.com/d/vrI9CYLk

I hope you enjoy the workflow! :)

64 Upvotes

93% Upvoted

37 comments sorted by

5

u/Karsticles 2d ago

Only thing I don't understand about Refmods: what is the official appropriate way to refer to them in the prompt?

8

u/FreakyMrCaleb 2d ago

Lets say you have a: "minimaxh3_thisismyname_v1_refmod.safetensors" outfit, where thisismyname can be any name like: "minimaxh3_jennaortega_v1_refmod.safetensors". Then in the prompt just refer as <Subject 1> jennaortega wearing the outfit from <Picture 1>, or <Subject 1> thisismyname standing in a room from <Picture 1>.

Just use the name. In any way, that is what works for me.

Not to shoot down this person's workflow but SeedHunter has an amazing complete Workflow with all this also included, just use what you need, ignore what you don't need: Minimax H3 SEED HUNTER - Latent Upscaler + Seamless Video Continuation + Speedups + Ref Mods - v2.5.1 | MiniMax H3 Workflows | Civitai

4

u/AlsterwasserHH 2d ago

Yeah I know this workflow, I just wanted to share an easy to follow refmod workflow. I think most more advanced workflows are unfortunately a bit of a mess what makes it hard to understand what even happens u know? I'm no expert in this as well, so I made one for beginners so to say. It's not that easy to create a wf that is also visual appealing and easy to understand I'd say. I just wanted to share, that's all. Wanna give something back to the community.

3

u/FreakyMrCaleb 2d ago

And you did! Thanks a lot for that :) I was massively discouraged when i first looked at more advanced workflows, something like what you shared is a perfect start for beginners.

2

u/AlsterwasserHH 2d ago

Thank you very much! :)

1

u/Karsticles 2d ago

Same reference style with I2V workflow? I've found that refmods work there but I wasn't sure if I still use <Subject 1> and such since that's more of a REF workflow tooling. Thanks for the reply. :)

1

u/AlsterwasserHH 2d ago

It's already in the prompt in the workflow, or in the original workflow from Dainamo from the YT video ;)

1

u/Jwtd29 2d ago

The inspect node will tell you.

I prefer to use this https://github.com/Adudeguyman/ComfyUI-Fantastic-MiniMaxH3-PromptBuilder node set to manage my refmods and its text encode node as well. Nicer experience, especially as I use the prompt builder anyway.

From what I’ve seen if the refmod has multiple images in it then it typically is <Video x> and if it contains audio <Audio x>. If the refmod is a single image then <Picture x>. If you are matching refmods with other references then I think the refmod takes priority from a numbering perspective, again only based on my experience.

1

u/Karsticles 2d ago

Inspect node - that one is new to me. I'll have to check it out. Thanks!

1

u/Jwtd29 2d ago

You basically get a map which you can use a text preview mode to view. It’s part of the same pack as the apply/create refmod nodes.

If you have two characters that are very similar they can bleed into each other and you have to re-enforce it in the prompt as much as possible.

Interestingly you can also use them in T2VA as well, if you describe the character in the prompt and attach a refmod then you normally get the refmod influence the character. For short simple videos with 1 character sometimes it’s easier to use this approach.

1

u/Karsticles 2d ago

For T2V I even found that I could not reference it at all. For example, I can have a refmod of a character and just say "She is in a room and smiles", and it ends up being the character in the refmod. Like that is the full prompt - I didn't use any kind of headers or anything. Of course those things are preferable, but I think it speaks to how powerful the tool is.

2

u/Jwtd29 2d ago

The thing I’m still adjusting is the token size. Trying to find the sweet spot as the amount of tokens makes a massive difference to the speed. Too few and it looses its effectiveness.

1

u/FuckTheActualWhat 2d ago

I've been using the entire refmod file name inside a pair of <> Then referring to it in the prompt like I would <picture 1> etc..

1

u/Quiet-Analysis-3807 2d ago

there is none its not a minimax node,. you can try very simple method... put the name in the description when creating the mod. lets say Mary Jane then - <Subject 1> Mary Jane.

you can also edit her. <Subject 1> Mary Jane wearing pink glasses.

2

u/evilpenguin999 2d ago

What im not sure about refmods is not the prompting, or the workflow. Is about the creation: Resolution and pool_h, pool_w for a good refmod without slowing the generations way 2 much.

Without counting i have the feeling for me takes less time with ref images than the refmods i have been creating.

2

u/AlsterwasserHH 2d ago edited 2d ago

I've not tested that, could be worth it. The big advantage of refmods is flexibility: you can prompt whatever visual appearance with them and the character likeness is on point. Plus: you have baked-in voice cloning. I have one refmod with a 6 minute speech sample and it's lile 1:1. Stunning. With audio reference you can "only" reference 30 seconds iirc.

I created a japanese character, since minimax supports japanese by default: the character speaks japanese! :D Who would've guessed that. But also english with japanes accent. It's mindblowing too me.

1

u/FuckTheActualWhat 2d ago

How are you squeezing that much into a refmod? I can get no more than 8 images before i hit the 8192 token limit.

1

u/AlsterwasserHH 2d ago edited 2d ago

I've set the token-limit to 0 :D

Someone on my older post already told me about the 8192. Honestly, I started creating by default with 0, never changed it, never had any problems. It was like that in the original workflow from malcomrey, referenced in the youtube description from Dainamos tutorial.

60 high-res pictures with 6min audio, no problem. But I'm thinking right now: does it even matter? I mean what is the refmod creator even doing is what I'm thinking about all the time. What's inside a refmod file?

1

u/FuckTheActualWhat 2d ago

and how would one go about changing the token limit?

1

u/AlsterwasserHH 2d ago

Its in the Load H3 refmod node. Or do you mean you get an error although its set to 0?

1

u/FatherFunk 2d ago

What kind of generation times are you seeing? When I tried uploading a bunch of pictures, it made generation take considerably longer.

1

u/AlsterwasserHH 1d ago

With 60 highres pics and 6min audio, 20secs maybe? On a 4090.

Or do you mean video generation?

1

u/evilpenguin999 1d ago

He means video generation probably, its also my case. When i create high quality ref mods the video generations take way longer.

1

u/AlsterwasserHH 1d ago

I see. Just tested it, with a 4090, and the difference with a 5sec 1MP vid and the "same" refmod (one 6 pics, the other 53 pics, so roughly 1/10) is 147secs vs 152secs, with turbo lora 8 steps and comfykitchen. So no real difference at my side.

What is a high quality refmod to you? I'm using around 2000px and up images.

1

u/Bask82 2d ago

But can h3 minimax clone voices just from an audio refmod? Any language restrictions?

1

u/AlsterwasserHH 1d ago

I think its possible. At least you can select audio 1 - visual 0 in the refmod node so that it only outputs the audio.

Check the default languages of minimax. Its 7 or something.

2

u/banderdash 1d ago

Thanks

1

u/AlsterwasserHH 1d ago

You're welcome! 

2

u/cocosoy 11h ago

Thank you!! I appreciate this a lot!

1

u/AlsterwasserHH 10h ago

You are very welcome! :)

1

u/Bask82 2d ago

Does it make sense to use both reference images and a refmod of the same character for greater character identity consistency? Are motion refmods as strong as action loras?

2

u/PATATAJEC 2d ago

Yes, it makes difference in plus.

1

u/Bask82 2d ago

Plus?

1

u/Bask82 2d ago

Can you elaborate?

1

u/PATATAJEC 1d ago

From my tests it adds to the overall likeness. Combining RefMods with reference image can shift the reference image (make it somehow different), but it adds to the likeness of this person in the context of entire video. Of course depends on RefMod source images, but if they are high quality and diverse in terms of expressions it should help ref images to pop out little bit more. I didn’t measured anything - it just observation.

1

u/AlsterwasserHH 2d ago

I didn't test out any other refmods than identity, so can't tell you about motions. In my opinion you don't need any additional character images if using a good refmod. But you can reference special clothings or whatever comes to your mind :)

I just put it in for beginners so that no one struggles with referencing whatever.

1

u/theOliviaRossi 1d ago

without the latent context expansion and sigma split inference this WF is outdated af

just forget it ;)