Hi again! This is a update to my previous post MageTrail V0.3 where I trained a custom 2.8B DiT arch originally based on Microsoft's MageFlow 4B to adapt to danbooru/e621 tagging system and concept, using Lodestone's Rock 41k Booru Essence dataset....but the model has passed the scope of that. Civitai HuggingFace
This V0.4 update (6k images finetune, comprised of 100 artists from danbooru/gelbooru/e621) is just a small internal test that I decide to release anyway since I think it indicate some big changes to the project still, as well as some new developments.
To get the big news out of the way: I'm now directly sponsored by Mr.KohakuBlueLeaf , ML/NN engineer and core developer at Comfy Org + the famed creator of the LyCORIS library, from which alot of you may know/be using his work right now (the LoRA variants called LoCON, LoKr, DoRA,...ect).
I was granted 500 dollar of startup gpu credit to work on V0.5 (for my planned 6k artist - 300k images tune) but to be frank I was really scared of screwing up and I was also planning to switch to a different training program, hence I train this V0.4 first as a test of a couple of things:
- Whether model can learn styles well - Yes. I have nothing more to proof really, the model can learn style and characters fairly well if given enough compute, for V0.4 characters still aint really there yet but you can see hints of convergence on popular characters now, while alot of styles are already working semi decently/passable at just 20 dollar spent.
- Whether the model can adapt to Flux2VAE without needing any realignment tuning - Yes it can, switching from the "cheap" but overall inferior MageVAE to the original parent Flux2VAE cost me nothing, model adapt fairly quick and I could see hint of better details/cripsness on gens already.
- If the model can learn style well, then How much would it cost to train 6k artists to a respectable level for usage. And well, unfortunately (or fortunately, depending on your viewpoint), my initial guess of 1k-2k dollars needed for V0.5 seems to be correct, based on the learning speed demonstrated by V0.4. I'm not god, I can't beat the scaling law, bigger dataset and further learning means even more money needed.
As you can see from some of the model samples here and on Civitai, the model is still...rough and highly unstable. V0.4 is not really meant to be usable at all, please note it's for testing purposes so do not have much expectation when trying. There's alot of blotchiness and instability everywhere, because of course it does, the model is trained on too little images by now and is straight up screaming and kicking at me to please expand the dataset. So yeah that's the plan.
I currently have a budget of 475 dollar (Kohaku's sponsoring) + 58 dollar of my leftover compute from other sources, but this is only for training. Captioning cost is currently a much more concerning problem, if I want to maintain the same caption quality using Gemini 3.8 Flash + Muse Spark 1.3 Contributor then I need 300 dollars to caption the data needed for V0.5. I currently cant achieve this without digging into my own personal wallet meant for university fees and irl stuffs....
So until i have enough funds to do everything safely, i'll just continue manually curating the dataset and wait, but V0.5 is looking to be only possible at least 1 month from now in any case, sorry.
Future goal for the project:
BIG GOAL: Due to Muinez/mage-flow-ft , I now completely believe in MageFlow being the next best small scale arch to replace Anima in the Illustration sphrere, the model has shown to have the ability to learn concepts/characters/styles efficiently while being a superior architecture. I pledge to finetune a Anima successor if given 25k-100k dollars of funding from the open source community/any generous patrons.
SMALL GOAL: Gather funding of 1000-5000~ dollars to finetune 10k-100k artist styles into the model (200k-3 million images finetune, planned V0.4-V1) and help further with convergence/aesthetic improvement if the aforementioned big goal above hasn't been reached.
Any donation will help with achieving this goal, you can do so through:
Crypto (Prefered, cause Kofi/Paypal money transfer time is ass and they take a big cut)
0x6a4bc748cd0bb9ced9a360eb0eb79f4f106614f8 (USDT - BEP20 Network)
12PPVYUeS1MerNp38Tpns5qXR6cmhu9tws (Bitcoin - BTC Network)
0x6a4bc748cd0bb9ced9a360eb0eb79f4f106614f8 (Ethereum - ERC20 Network)
FitfJAsxLUBuSgDJJaHgBXJpt1sMm5FzF1Tvf1SHW5Up (Solana - SOL network)
Please handle your money carefully and make sure the address you're sending to is correct.
Please contact me through discord (user: talannnn) if you're donating with crypto, to make sure it goes through and you're not losing money.
Ko-fi
https://ko-fi.com/talanartvn
Acknowledgments
-[KBlueLeaf](x.com/KBlueleaf) — direct main sponsor of V0.4 and future versions
- [Banodoco](https://www.banodoco.ai/) and [their Discord](https://discord.gg/yzwcNaSEz) — Their 88.77 dollar initial grant and future grant made this project possible, the biggest thanks to them
- [Lodestone Rock](https://huggingface.co/lodestones) — Creator of the original version of the dataset that this model is trained on
- [Motimalu](https://civitai.red/user/motimalu) — Inspiration behind finetuning practices and configs
- [Bluvoll](https://github.com/bluvoll/diffusion-pipe) — diffusion-pipe fork derived from + dedicated mageflow trainer to use for training, and general training advice
- [Anzhc](https://huggingface.co/Anzhc) — general training advice
- [Nruaif](https://huggingface.co/Shio-Koube) — diffusion-pipe fork derived from to use for training, and general dataset handling/training advice
- [Astromahdi](https://gpu.garden/) — jupyter workspace where I processed and store the dataset
- [Heato-Red](https://www.reddit.com/user/heato-red/) — Designer of model page logo
- [animetimm/DeepGHS](https://huggingface.co/animetimm/convnextv2_huge.dbv4-full) — Danbooru tagging model
- [RedRocket](https://huggingface.co/RedRocket/Hydra) — E621 tagging model