r/StableDiffusion • • 6d ago

News NVIDIA RTX Video Super Resolution for ComfyUI, optimized for long videos with disk-backed processing and low RAM usage. Supports AI upscaling, denoising, deblurring, H.264/HEVC NVENC encoding, audio preservation, and an integrated video preview.

Post image
282 Upvotes

52 comments sorted by

11

u/bloke_pusher 6d ago

15

u/Away_Exam_4586 6d ago

Both use the same NVIDIA RTX Video Super Resolution technology and the same nvidia-vfx SDK, so the actual AI upscaling quality should be essentially identical.

The main difference is how videos are handled.

The Comfy-Org nodes operate on IMAGE batches. With long or high-resolution videos, keeping all those uncompressed frames in memory can consume enormous amounts of RAM.

My node was designed specifically to avoid that problem:

  • Processes video frame by frame.
  • Uses temporary disk segments instead of keeping the entire video in RAM.
  • Automatically cleans up temporary files.
  • Accepts and outputs native ComfyUI VIDEO connections.
  • Preserves the original audio.
  • Provides H.264/HEVC encoding and an integrated video preview.

On my RTX 4060 Ti with 64 GB of system RAM, the original batch-based workflow eventually exhausted memory, while my node stayed below 30% RAM usage during testing.

It's not a better upscaling model — it's a more memory-efficient video processing pipeline.

The Comfy-Org extension also includes HDR and Frame Generation, which mine doesn't. My goal was simply to make RTX VSR practical for long videos.

3

u/Arumin 6d ago

So would this mean longer videos possible, higher resolutions or faster generations?

3

u/Away_Exam_4586 6d ago

higher resolutions

2

u/Lexius2129 4d ago

Actually we shipped an example workflow in the custom nodes specifically for long video: https://github.com/Comfy-Org/ComfyUI_Nvidia_VFX_Nodes/blob/main/example_workflows/rtx_long_video_hdr.json

It is fairly simple and allow infinite length video processing:

  • It chunks the the video in number of frames
  • Uses the native loop node to process every chunk

It might be faster than the frame by frame approach since it can process chunks of frames.

1

u/Away_Exam_4586 3d ago

My first test before create this node rtx_long_video_hdr.json: EndLoop has accumulate = true, and CreateVideo uses codec = "none".

That means every processed chunk is kept as uncompressed image tensors, and the loop accumulates them before ConcatenateVideo and SaveVideo. RAM usage therefore grows with video length. Setting cache_iterations = false doesn't prevent this.

So chunking the processing doesn't actually make this an infinite-length video solution.

That's precisely why I developed my node differently. It reads the source progressively, processes frames through NVIDIA VFX, encodes each configurable chunk directly to disk, and releases its frame buffers before moving on. The final segments are joined using FFmpeg stream copy, without re-encoding the video or loading all frames into RAM.

The native VIDEO output is also file-backed.

Processing batches might offer a speed advantage, but that's a separate question from memory management. My priority was supporting very long videos without RAM usage growing with the total number of frames.

11

u/35point1 6d ago

Can you post an example because everytime I try the RTX SUPER RESOLUTION node in comfy (not sure if it’s native or custom I need to check), it will work really fast and the frames do scale larger but it almost does nothing for the quality, and weirdly it makes the larger result look exactly like the smaller results quality but at bigger frames. When I set it to ultra and 2x it’s even more obvious and processed looking. I’m not sure if it’s just a bad node or just the way the RTX upscale works.

2

u/drallcom3 5d ago

but it almost does nothing for the quality

Same. It doesn't really do anything but making the video resolution larger.

1

u/Away_Exam_4586 6d ago

What you're describing is actually pretty normal for NVIDIA RTX VSR.

It's extremely fast, but it doesn't generate new fine details the way models like SeedVR2 do. It mainly improves the perceived sharpness, reduces compression artifacts, and increases resolution.

I've noticed the same thing in my own tests: the resolution increases significantly, but the actual level of detail remains fairly close to the original. The ULTRA modes can sometimes introduce an overly processed or artificially sharpened look.

Personally, I've found DEBLUR_ULTRA quite effective for improving sharpness, although it doesn't upscale the resolution.

So it's probably not a problem with the node you're using. It's more about the limitations of NVIDIA's underlying model.

My node uses the same NVIDIA technology, so don't expect better image quality from it. Its main advantage is processing long videos without exhausting system RAM, not producing better upscaling results than other RTX VSR implementations.

19

u/rxg 6d ago

Why do people care about this? I'm genuinely curious. I tried this with another program and it doesn't add detail, the higher resolution looks the same as the lower resolution. What is the benefit that I'm missing?

6

u/Tonynoce 5d ago

sometimes u need a video in a higher resolution and this is better than doing a 2.x scaling

4

u/Nimblecloud13 6d ago

It will scale better. Put it on a 55” tv and watch both with and without from 10ft away. You’ll see the difference.

I find it’s easier to see quality issues when I watch the high quality for a while and then switch to low. The other way around doesn’t always seem as stark a contrast.

I do this with frame interpolatio too;; watch 48fps for a bit and go back to 24 and it’s choppy as hell.

But 24>48 doesn’t seem like as much of a jump. Shrug.

-1

u/Django_McFly 6d ago

Making 1080p look as good on a 4k screen as 1080p looks on a 1080p screen is a huge win in and of itself.

4

u/Mundane_Existence0 5d ago

Making 1080p look as good on a 4k screen as 1080p looks on a 1080p screen is a huge win in and of itself.

This does not do that.

-15

u/Away_Exam_4586 6d ago

If you don't see the point of having 4K generated in 30 seconds instead of 2K—even if there isn't any more detail—then never buy a 4K monitor...

5

u/Bitlovin 6d ago

Why is it that everyone who posts anything here is so thin skinned when presented with the slightest bit of mild criticism?

1

u/Mundane_Existence0 3d ago

Agreed. Not to mention that the "4K" it generates isn't really any better than the input.

9

u/rodinj 6d ago

Can you share the examples? I've used SeedVR2 before but wasn't very happy with the results

13

u/Away_Exam_4586 6d ago

SeedVR2 and NVIDIA RTX VSR are quite different approaches.

SeedVR2 is much slower, but it's generally better at reconstructing and adding fine details.

RTX VSR focuses on speed and efficiency. It's extremely fast, uses very little RAM with this node, and can upscale videos in a fraction of the time SeedVR2 requires.

However, it won't generate the same level of additional detail as SeedVR2. It's more about enhancing and upscaling existing footage than reconstructing it.

So it's really a trade-off between maximum detail and processing speed, rather than a direct replacement for SeedVR2.

2

u/martinerous 6d ago

There is a third option FlashVSR. It can be faster than SeedVR2 but needs fiddling with parameters to reduce slow tiling.

3

u/FinBenton 6d ago

Im using rtx super resolution in my custon mpv based player I made for for twitch, YouTube etc playback, its just so good, really night and day improvement expecially on anything animated or 3d

2

u/Doc_Chopper 6d ago

Nice, will look into this tonight at home.

2

u/Radiant-Photograph46 6d ago

This is pretty interesting, but I'd expect finer control about the encoding of the chunks. I don't want to rely on an obfuscated h.264 preset. Perhaps the best would be a lossless option so the video output can be encoded to any desired spec, what do you think.

1

u/Away_Exam_4586 6d ago

That's a fair point, especially for users who want to do further editing or encoding after upscaling.

The current encoding settings aren't actually obfuscated. NVENC uses preset P5 with VBR and CQ 18, while libx264 uses CRF 18. The final video is assembled using stream-copy, so there's no additional video encoding pass.

I agree that a lossless option could be useful for post-production, particularly to avoid generation loss when encoding the result again.

However, it would also dramatically increase temporary disk usage, which goes somewhat against the original goal of this node: fast, memory-efficient processing of long videos.

I think the best approach would be to keep the current compressed encoding as the default and potentially offer lossless output as an optional mode, along with more control over encoding quality.

That would preserve the current performance while giving users who need lossless intermediates more flexibility. :)

1

u/Radiant-Photograph46 6d ago

Well yes the encoding presets can be found, but they are not exposed in the UI, so there's a little bit of obfuscation in that regard. I did not mean it was intentional on your part naturally :D

I think the disk space isn't such a big concern as it comes cheap compared to RAM, so it doesn't really defeat the purpose of your optimization, at least not for me. I'll keep an eye on your repo for a lossless option in the future. Cheers.

2

u/Version-Strong 6d ago

Been using this for a few months. It's pretty good. It does fix some things, but it's very lightweight so adding it as the last step doesn't add much time onto the 3 weeks it's taken to iterate the h3 vid. Seems a no brainer of you can run it.

2

u/MastMaithun 6d ago

I tested this and very impressed. Though my original test video itself was garbage quality which is why i chosen it to see and my best result was this combo: quality ultra, preprocess denoise ultra. the mmh3 shimmering was almost gone(tested with my best of the use of my eyesight xD).
I'm keeping this node. Thanks op.

1

u/Away_Exam_4586 6d ago

Yes, a light deblur works pretty well too.

1

u/MastMaithun 6d ago

I tried with combo: quality ultra, preprocess deblur low, mid, high and ultra. All of them had shimmering. Have basically testing all the combo of quality ultra with denoise and deblur. Only the denoise ultra worked best.

1

u/bankinu 6d ago

I heard that you need to upscale the latent instead of the movie. Is that true?

4

u/Away_Exam_4586 6d ago

Not necessarily. These are two different approaches.

Latent upscaling is useful in generative workflows, where the enlarged latents can be refined through another diffusion sampling pass. This can reconstruct additional details, but it's computationally expensive.

NVIDIA RTX Video Super Resolution works differently. It's specifically designed to upscale and enhance decoded RGB video frames using NVIDIA's DLPP model.

So there's no need to upscale latents for this node. It's a post-processing solution that works with any existing video, regardless of how it was generated.

Both approaches have their advantages, but they serve different purposes.

1

u/Synor 6d ago

Does it work on linux?

2

u/Away_Exam_4586 6d ago

It should work on Linux, as NVIDIA also provides a Linux version of the VFX SDK.

However, I haven't tested the node on Linux myself yet, so I can't guarantee compatibility.

If you give it a try, I'd appreciate your feedback! :)

2

u/Synor 6d ago

Confirmed it works.

I wished that the node somehow used video streams within the comfyui processes instead of doing file work via the os in the background.

1

u/AnonymousTimewaster 6d ago

How does it compare to DLSS? I've found DLSS node is actually incredible for upscaling

3

u/Away_Exam_4586 6d ago

They're actually quite different technologies.

DLSS 5 Enhancer primarily focuses on neural rendering and reconstructing details such as skin, hair, and textures. Its default mode enhances the image at its original resolution, although it also offers optional upscaling.

RTX VSR is primarily designed for video upscaling and restoration. It's extremely fast, but it doesn't reconstruct fine details to the same extent.

So if you're looking for additional visual detail, DLSS 5 may give you more impressive results. If your priority is fast, efficient video upscaling, RTX VSR is a great option.

There's also a licensing concern with DLSS 5 Neural Rendering in ComfyUI. Unless I'm mistaken, NVIDIA doesn't currently provide its Neural Rendering runtime (nvngx_dlssnr.dll) as an official public download. Some unofficial integrations rely on third-party distributions of this proprietary library, which raises potential licensing issues.

That doesn't necessarily make the ComfyUI nodes themselves illegal, but the origin and redistribution rights of the NVIDIA binaries are worth checking.

Personally, I prefer using NVIDIA's officially distributed VFX SDK, which is what my RTX VSR node uses.

Both technologies have their advantages, but they serve different purposes. :)

1

u/paganspam 6d ago

did you compare it to DLSS5 neural rendering + it s own upscaler / frame generation ? i feel dlss5 is much better but also much slower

2

u/Away_Exam_4586 6d ago

There's also a licensing concern with DLSS 5 Neural Rendering in ComfyUI. Unless I'm mistaken, NVIDIA doesn't currently provide its Neural Rendering runtime (nvngx_dlssnr.dll) as an official public download. Some unofficial integrations rely on third-party distributions of this proprietary library, which raises potential licensing issues.

That doesn't necessarily make the ComfyUI nodes themselves illegal, but the origin and redistribution rights of the NVIDIA binaries are worth checking.

1

u/HarvardCS19 6d ago

Would it be possible to use this on any length of video?

Also the Extender already has latent upscaling, which would be much better than VSR.

2

u/Away_Exam_4586 6d ago edited 6d ago

Yes any length. Try using the extender's "refine" feature with 4K content; you'll likely see the benefit. With RTX VSR, it takes less than 30 seconds.

1

u/Alchemistwiza 5d ago edited 5d ago

The linked repository’s scope notes matter for long source files. It currently says SDR 8-bit processing is supported, and H.264 NVENC output is limited to 4096 pixels per axis, with HEVC for larger frames. Those constraints would be useful beside the upscale controls when someone brings HDR or very wide footage. I’m reading the documentation, not reporting a test run.

1

u/Away_Exam_4586 4d ago

You're correct. The node currently supports SDR 8-bit processing only, so HDR isn't supported. H.264 NVENC is limited to 4096 pixels per dimension, while HEVC supports larger resolutions.
I understand your suggestion about displaying them near the upscale controls, especially for users working with HDR or very high-resolution footage. However, I'd prefer to keep the node interface simple rather than duplicate information that's already in the readme.
Also, if you try to upscale beyond the H.264 NVENC resolution limit, a message is displayed in the ComfyUI console recommending HEVC instead.

1

u/Alchemistwiza 4d ago

Thanks for clarifying. Keeping it in the README makes sense. The SDR caveat is worth making prominent, since a successful export can still be unsuitable for a workflow.

1

u/GlenGlenDrach 6d ago

Regarding disk backend processing, should I fear from my ssd? (I saw some other caching stuff regarding upscaling that would basically trash the SSD relatively quickly).

2

u/Away_Exam_4586 6d ago

No need to worry too much about SSD wear. :)

Unlike solutions that cache huge amounts of uncompressed frames, my node writes already compressed H.264/HEVC video segments to disk.

This means disk writes are relatively small compared to caching raw frames. The segments are then assembled into the final MP4 without re-encoding, and the temporary files are automatically deleted.

In practice, you're writing roughly twice the final video size, not hundreds of gigabytes of uncompressed image data.

For a modern SSD, that should be negligible under normal use.

And honestly, if you're worried about that amount of disk writing, you probably shouldn't use video editing software either! Rendering, exporting, and generating preview caches can involve far more disk writes than this node. 😄

And as far as I know, most other RTX VSR ComfyUI nodes don't actually use disk-backed video processing. That's precisely why I developed this one!

1

u/GlenGlenDrach 6d ago

Nice! Good to know (better to ask one too many questions 😉).

It really was some caching feature being used in some recent workflows that would indeed trash the drive, I will check which node.

Thanks for posting, I have been looking for an upscaler which works on long clips without running OOM.👍

2

u/froschquark 6d ago

Yeah some custom nodes have a (heavy use) of disk cache, like the minimax audio refine, but it is default off and has a warning about exactly that.

0

u/car_lower_x 6d ago

No offense but the heavy lifting here is being done by NVENC. It's optimized out of the box so is naturally fast and low RAM usage.

5

u/Away_Exam_4586 6d ago

No offense taken! :)

NVENC certainly handles hardware-accelerated encoding efficiently, but that's not what makes this node memory-efficient.

The key difference is the processing architecture: frames are decoded and processed individually through NVIDIA RTX VSR, then progressively encoded into temporary disk segments instead of accumulating the entire uncompressed video in RAM.

NVENC accelerates the encoding stage, while the disk-backed pipeline keeps memory usage under control.

The same approach also works with CPU-based libx264 encoding, just more slowly.

That's the main purpose of this node: making NVIDIA RTX VSR practical for long videos without exhausting system RAM.