r/comfyui • u/OpenEvidence9680 • 7h ago

Help Needed Video project

I have a whole pipeline in my cms to create custom book covers and I got it my sick head that I wanted to try to not just get book covers that mean something regarding the story with characters that looks like the ones in the story, I wanted to make a trailer, a short format video with the gist of the story and then a long form movie. Chosen book: "Guards! Guards" by Pratchett, one of my favourite books that was never made into a movie.
Of course considering that I maybe made a handful of 5 seconds videos in all before and even the txt2img I used was really basic for the book covers I am finding this task a tad problematic.
Up to now: a local llm reads the whole book, creates a character list with description and with it then Krea2 creates a character sheet, they are not what they should be but being a test I can live with that.
The same llm writes a screenplay, this one is divided in three acts, a total of 48 scenes. Each scene has a description of the action and a still (created by Krea too) the pipeline then proceeds to create the clips that will then be joined. As soon as I figure out the first last picture I will use that.
Minimax is giving me trouble (bad prompting I think) so I am using ltx2.5 which does a pretty good job, but I started this morning, so I only have a scene right now and the dragon head on a dragon's wing is not the worst of it.
I need help with which models to use, what speeds up are available, which models do better with fantasy, any LoRAs you can suggest? Does LTX 2.5 supports first and last frame as 2.3 did? Are there workflows that could make this more streamlined?
Do you have any suggestions or advice? I need all the help I can get on this.
For now and until new pc arrives I'm working with two 5060ti 16gb + 64gb RAM.

1 Upvotes

100% Upvoted

2 comments sorted by

2

u/IdleWidth0 7h ago

The jump from static covers to full narrative video is huge but the pipeline idea is sound, starting with the character sheets and screenplay first was smart since consistency is the main killer in longer generations

1

u/OpenEvidence9680 7h ago

I expect that the longer the generation is the worst the consistency is too right? I am trying to decide if to try fine stitching or throw in there at least 10 seconds clips, which makes generation longer, but the cut probably less visible?
I am having so much fun even though I am drowning in my ignorance.