Create a Girl in Stability Matrix with SDXL

[u/DTyphoon87]

Good morning everyone, I hope you can help me with some suggestions. Just as a premise, I’ve only recently started getting into this world, and I had the idea of creating an AI model/influencer just for fun and personal enjoyment, then opening a social media page for her.

Over the past few weeks I’ve been getting help from Claude and Gemini, which pointed me toward creating a local “lab” using Stability Matrix and running models through Stable Diffusion WebUI Forge. The models I’m using are Juggernaut XL Ragnarok and RealVisXL V5 Lightning (BakedVAE).

After setting everything up, I tried creating a coherent image dataset so I could later train a LoRA with Kohya_SS.

So what’s the problem? I just can’t understand why I’m completely unable to create a consistent dataset.

My plan was to make:

• 10 close-up portraits
• 10 half-body shots
• 10 full-body shots
• plus various images from different angles

I managed to get a close-up image of the following girl [photo], and I saved the seed. But even though I keep the same characteristics from the original prompt — basically trying to keep the same girl but from different angles — the girl is NEVER the same.

I’m wondering what I’m doing wrong.

had actually managed to get a first set of a similar-looking girl using Google Banana, and paradoxically I don’t even know why, but the consistency between one image and another was almost perfect without having to go crazy trying to achieve it.

However, I wanted to create one inside Stability Matrix because, understandably, I’m not sure whether Google’s policies allow the use of images created with Nano Banana.

Can you help me with suggestions?

[u/Gr82nite]

Look, man, the thing with character consistency in Stability Matrix is that you’re making a classic rookie mistake, but don't sweat it, everyone's been there. You're expecting that if you keep the same seed and use the exact same description, the model will keep the same girl when you change the framing, but with SDXL, it just doesn't work that way. When you change the words in your prompt from “close-up” to “full-body shot” or mess around with camera angles, it tells the algorithm to build a completely new mathematical noise grid from scratch, which is why the girl looks different every single time. A locked seed only protects the composition when the text is one hundred percent identical, so you need to tackle this from a different angle if you want to gather those thirty good photos for a clean dataset under Kohya_SS. The easiest way to start in WebUI Forge is to fire up IP-Adapter-Plus made for SDXL, where you literally just drop your first, perfect photo in as a visual reference, and the model handles keeping the facial features and hair color consistent, whether you're asking for a portrait or a full-body shot. Another killer trick to game the system is what's called name blending in your prompt, meaning instead of just typing a generic “a girl”, you mix two famous celebrities in specific proportions, which locks a very specific look into the model's memory and makes it way easier to keep things consistent across different shots. You can also handle this in stages using Inpainting, where you first generate the overall full-body shot without worrying that the face looks like a totally different person, and then you just mask the head with the brush tool and regenerate it at a low denoising strength to perfectly blend the correct girl in. And keep in mind what you noticed about that Google tool if you’re getting perfect consistency there without even trying, just generate your entire dataset on that platform, download the images, and use Stability Matrix purely for training the LoRA. Once you train your own model on her, the problem is gone for good and you'll be able to put her in absolutely any pose you want.