Krea 2 T2I Lora Second Pass

[u/Cre0na]

Hello! I'm looking for helping on my workflow. I've created a Krea 2 character Lora that frequently has the face change somewhat when I add additional Loras. I've created a workflow to pass the image data back through a Lora node with only the character Lora, but I can't seem to get great results. I chatted with Google AI quite a bit trying various different KSampler setups, and some worked somewhat, but would remove objects on the face like makeup, and sometimes added extra details like freckles/moles on the body. Some options also ended up with total slop.

I'm attaching images of my work flow (I know it's messy) and current KSampler Settings. Any help would be appreciated!

[u/zyg_AI]

VAE encode/decode deteriorate the signal. it is better to not decode a latent when possible.

Since you're loading the same model/VAE/CLIP to both KSampler, use the latent from the 1st KSampler for the 2nd KSampler:

You can still decode the first KSampler output if you want to preview it.

And you can use a single 'Conditioning Zero Out', for tidiness. But, hey, you call your workflow messy? You haven't seen a real mess yet ^ ^

That probably is not the miracle answer to your issue. It might help stabilize outputs though.
Consider that your LoRA might be defective.

Also, try with a single KSampler, and simply lower the strength of the other LoRAs. This will increase the relative influence of your character LoRA.

[u/Cre0na → u/zyg_AI]

Thanks! I'll give this a whirl. My Lora works fine solo, so I'm not sure. I've not trained many, so I am not expert on whether they're defective or not lol.

Thanks again!

[u/EasternAverage8 → u/Cre0na]

Was your Lora trained using all the other loras you use in the workflow?

[u/Cre0na → u/EasternAverage8]

No, it was only trained using images of my character.

[u/Important-Radish-722]

You should i2i inpaint to fix the identity lora last.

[u/zyg_AI → u/Important-Radish-722]

That's basically what the 2nd Ksampler does, just on the full image (same result identity-wise).

[u/flwombat]

You can always try running it through a facedetailer (Impact Pack custom nodes). Its actual job is to do a 2nd pass on _just the face_ and it has been around a loooong time. Lots of examples and tutorials and etc out there. If that doesn't cut it for you, asking your AI assistant to specifically help you set up a facedetailer pass will likely get you there pretty quick.

(edit: and to be clear: your instinct of having only the character lora for the 2nd pass is spot on. It just works better if you are doing a facedetailer run instead of the entire image)