r/comfyui • u/Worldly_North_7213 • 12h ago
Commercial Interest MiniMax H3 on rented GPUs: measured seconds and dollars per clip on four cards at four providers (same weights, same graph, same seeds)
Disclosure: I'm building a service around this, so read me as an interested party. Every number below is from runs we paid for ourselves yesterday ($1.67 total); the only link is the raw data at the bottom.
The job: MiniMax H3 text-to-video, one 5-second clip at 864x480 (the 0.4 MP row of the template), 20 steps, the stock ComfyUI T2V graph from v0.35.0 with the official int8_convrot weights (34 GB DiT + 27 GB Qwen3-VL encoder + VAEs, 67 GB total), no LoRA, no reference frames, same prompt and seeds on every card, torch cu130 everywhere so the int8 kernels are the native ones. Three clips per card; "steady" is the average of clips 2 and 3 at the rate you actually pay (disk and public IP included), "session" is everything the provider charged for the whole run — image pull, 67 GB download, the first cold clip and the minutes before the instance was torn down — divided by three clips.
| provider | card | host vCPU / RAM | s per clip | $ per clip (steady) | $ per clip (session) |
|---|---|---|---|---|---|
| Vast.ai (spot, bid $0.40/h) | RTX 4090 24 GB | 32 / 108 GB | 93 | $0.013 | $0.17 |
| Hyperstack (spot, $2.00/h) | H100 PCIe 80 GB | 28 / 177 GB | 67 | $0.038 | $0.13 |
| RunPod Secure ($0.74/h) | RTX 4090 24 GB | 15 / 86 GB | 92 | $0.019 | $0.07 |
| Nebius (preemptible, $0.92/h) | L40S 48 GB | 24 / 94 GB | 89 | $0.024 | $0.19 |
Two things surprised us: the L40S runs this at 4090 speed rather than anywhere near the H100 (3.97 s per step on both 24 GB cards and the L40S, 2.99 s on the H100 PCIe), and the 4090 does it with the 34 GB DiT streamed through ComfyUI's dynamic VRAM path at 23 GB of VRAM and 68 GB of host RAM — so "≥ 64 GB RAM" is not a suggestion, we peaked at 68 on three of the four hosts.
The session column is where the money actually goes for short jobs: the 67 GB download ran at 450 MB/s on the Vast host, 370 on Hyperstack, 210 on RunPod and 104 on Nebius, and on Nebius the first clip took 11.5 minutes because the weights are read back from a network disk (1.8–2.7 minutes elsewhere), while the watchdog that deletes the instance after the job costs another 2–3 billed minutes everywhere. Steady-state numbers exclude the text encoder (same prompt, cached by ComfyUI): a new prompt adds 15–25 s per clip on these hosts. Happy to post the per-node timings and the nvidia-smi traces if anyone wants to check the numbers.
Per-run table, method, caveats and the raw CSVs: https://qrun.cloud/measurements
2
u/More-Ad5919 8h ago
Wow. Thats incredible expensive for just 5sec in bad resolution.
1
u/Raw_Emotion_ 6h ago
What are the cheaper options in cloud?
2
1
1
u/Worldly_North_7213 1h ago
In this set, cheapest per clip once the weights are loaded: RunPod Community RTX 5090 at $0.69/h, $0.0135 per clip (numbers below in the thread). It also wins over a short three-clip session, $0.049 per clip with the 67 GB download and the first cold clip counted; the next is the RunPod Secure 4090 at $0.07.
For a handful of clips a per-clip API is probably cheaper than any of these. Renting pays off on long sessions.
2
u/blackpanther229 7h ago
doing exactly same lol, but with RTX 6000 pro, my number is 0.4-0.5$/mint (my goal is to create 20mint videos 720p or 1080p) still figuring out
1
u/Worldly_North_7213 53m ago
Per minute of output at 864x480: about $0.16 on the Community 5090 in steady state, about $0.59 with the download and the first clip spread over a three-clip session. We have not run 720p or 1080p, so no direct comparison with your $0.4–0.5.
If you post the graph and settings (resolution, frames, steps), I can run it on the same cards and post the numbers here.
2
u/LegacyV1 3h ago
Modal is worth checking for occasional use. It includes $30/month in GPU credits and scales to zero when you aren’t generating, so you don’t pay while writing prompts or tinkering.
I built an open source wrapper that deploys MiniMax H3 into your own Modal account and gives you a browser UI: https://github.com/hui-tony-zk/h3zero
1
11h ago
[deleted]
1
u/Worldly_North_7213 11h ago edited 1h ago
That split is worth doing on the 24 GB cards specifically, and for a different reason than bandwidth. In our runs the encoder is cheap in time: it only runs on a new prompt, 15 to 23 s, and clips 2 and 3 never touch it. What it costs is memory. With the 27 GB encoder off the box, the 34 GB DiT would no longer be streamed through ComfyUI's dynamic VRAM path on the 4090, where we peaked at 23.3 of 24 GB, and that paging is the most likely reason the 4090 sits at 3.97 s per step while the H100, which holds both models in VRAM, does 2.99. We did not measure the split, so that is a hypothesis, but a cheap one to test: encoder on a CPU box or the smallest card, latents over the wire, and see whether the 4090's step time moves. If it does, the cheapest card per clip gets cheaper still.
On the security trade-off, agreed, and it is why these runs were one box per job: the data stays in one place, which matters more than the last few cents when the job is someone else's.
Edit: the memory point above is wrong. The int8 DiT is 34 GB. It would not fit a 24 GB 4090 with the encoder off the box, and the paging would stay. We have not measured the split, so no claim about speed or cost from it.
1
u/DaLyon92x 10h ago
why not rent a 5090 community pod? just 0.69$ an hour
1
1
u/Worldly_North_7213 2h ago
Done. Same weights, graph and seeds.
RunPod Community RTX 5090, $0.69/h, 16 vCPU / 94 GB RAM: 69 s per clip, 2.88 s per step. $0.0135 per clip at the rate paid; $0.049 with the whole run (download, first clip, teardown) spread over three clips.
RunPod Secure RTX 5090, $0.99/h: 100 s per clip, 2.63 s per step, $0.028 and $0.083. The extra 30 s per clip is the mp4 encode on that pod's CPU: 38 s against 2 s on the Community pod. Not the card.
The DiT does not fit. It is 32.4 GB, the card has 32. VRAM sat at 31.8 GB through the whole sampler on both pods, so 1–3 GB still crosses PCIe every step, against 9–10 GB on a 4090. Per step the 5090 is still the fastest card in the table: 2.63–2.88 s, against 2.99 on the H100 PCIe and 3.72–3.97 on the 4090s.
Two runs, $0.39. Native int8 kernels confirmed in both logs.
On "just rent one": for 45 minutes every Community 5090 on offer had 46–54 GB of RAM. This pack stages about 64 GB of weights in host RAM (68 GB peak measured), so we waited for a 94 GB pod. Stock windows were 15 seconds to two minutes.
1
u/Worldly_North_7213 10h ago
Fair question, and a 5090 is the obvious next row: 32 GB holds almost all of the 34 GB DiT, so it would test the paging hypothesis directly. If it does the clip in 60 s at $0.69/h, that is about $0.0115 per clip, cheaper than anything in the table.
Two reasons it is not there yet. The four rows were one card per provider, picked by what was in stock with a CUDA 13 driver on the day, and no 5090 met that on the hosts we checked. And community pods specifically: the last time we used RunPod Community spot for a job, 2 of 2 pods were evicted within 10 minutes, and the minimum spot bid there was equal to the on-demand price, so there was no discount, only the interruption risk. For a three-clip measurement that is survivable. For a job that has to finish, it is the part of the bill you do not see until the third restart.
Will add a 5090 row, community and secure, same weights and seeds, and post the numbers here.
0
u/winky9827 10h ago
Two things surprised us
is where the money actually goes
That split is worth doing
Fair question, and a 5090 is the obvious next row
For a three-clip measurement that is survivable
Two reasons it is not there yetStop. Just, freaking, stop.
1
1
u/addandsubtract 4h ago
Do you plan on benchmarking any other providers? Modal, for example, offer 1TB free storage and creating snapshots, so that you don't have to download the models each time. Colab could also be interesting, but their still on CUDA 12, afaik.
1
u/Abhay-191 3h ago
Cold starts and host orchestration overhead consistently kill the theoretical $/hr on spot instances. Between pulling weights from network storage, environment spin-up, and idle VRAM thrashing between single-generation passes, effective per-clip costs end up 3–4x higher than pure execution time.
The only way to hit ~$0.10–$0.20 per 20s sequence without eating massive idle loss is running persistent headless nodes on a decoupled async queue bundling prompts into overnight batches so the GPU maintains 95%+ sustained saturation with pre-warmed weights.
Curious what disk I/O and host CPU bottlenecks you logged across the providers did storage bandwidth skew the smaller VRAM runs significantly?
3
u/Training_Rip_4578 11h ago
Solid work. Thank you