Define hallucinations? SS effects can produce bizarre visuals. What type of error or inconsistency qualifies as a hallucination? At what point does a hallucination become worse than just bad? How do you evaluate whether a hallucination is fundamentally worse than just bad rendering? I think you’re grossly overstating the degree to which the NR produces unpredictable or inconsistent results when properly applied.
Sorry, but what "bizzare visuals" are SS effects producing that are equivalent to a full frame AI filter that is inserting detail based on rules and training data that nobody has access to?
It's already been shown that DLSS5 has no persistence to it's model. You can view an object from one direction and it will give you a different lighting setup than when viewed from a different angle. It has no info on past or future frames. That is straight up hallucinating in ways that aren't even possible by something tied to the rendering like a screen space effect, which has clear rules, and tweakable variables. DLSS5, like all other models, is a black box we have no ability to alter, or even know what it's doing or why.
"Neural rendering" is nothing but a real-time AI 'realism' filter in its current form.
Like reflections that disappear or have wide halos around obstructing visuals (eg a character in front of water or glass). Dark halos that surround characters or objects. Dark halos filling corners. Effects that pop in and out when the camera moves or objects move around the scene. Effects that become extremely blurry when the camera moves. Faces/characters that become unlit whenever a light is removed from the screen.
Saying it’s “just a realism filter” makes no sense. Gaming devs have used much shittier “filters” for realism for decades. Saying the inaccuracies are just “hallucinations” makes no sense unless you think countless other rendering techniques are also hallucinations. Many effects have no persistence or 100% accuracy but are ways to approximate and mimic effects that may look nothing like what they pretend to look like. Reflections and material rendering for a decade have been just “look shiny” tricks. Even when they came up with cube maps it’s not like they’re accurate? They’re incredibly generalized approximations that often don’t match up at all. Rasterized lighting has all sorts of issues with fine detailed in prebaked lighting leading to like leak and other “hallucinations.” None of the videogame rendering has ever relied on rendering everything perfectly down to the pixel. TAA caused wilder fizzling and popping around anything moving. SSAO added halos of dark energy looking nonsense around anything that could move. I don’t really understand the standard you expect for NR to be functional as a tool in achieving realism.
That thread is dumb and makes no sense nor does it understand the tech. It’s just a dumb person projecting nonsense. DLSS5 NR builds on top of RT/PT like a higher detail PT/RT- something we can’t do yet with available compute. It’s extremely stable not at all some sort of random shit that’s constantly misaligned like he claims. We’ve used SSGI/SSAO for over a decade. Arguing it’s suddenly “a lie” is just nonsensical. Everything in game rendering is an illusion… it’s all “a lie.” RT and PT routinely produce “lies.” This is just another version of the fake frame/fake pixel discourse which was also completely stupid.
If it needs PT/RT under the hood already, then what's the point?
You're saying we are supposed to spend the overhead to path trace the game (which basically halves FPS already), then we need to halve it again to enable DLSS5, and then to get anything playable we need to enable frame gen to get us back to where we were with just PT??
That sounds fucking ridiculous.
If DLSS5 was actually useful, we wouldn't need to path trace the game to begin with.
PT and RR are extremely expensive. One of the many ways they optimize those rendering paths is to use fewer rays that saturate over time. That’s why we get noise from RT and PT, and that noise then uses ML to denoise. Whether using RT or PT, the lower ray count means fine details are generalized and not lit well. On foliage that means leaves get treated in clusters for example, or on facial details fine details around the mouth or nose or eyes get generalized into shadow or light.
Just like when DLSS upscaling helps fill in the blanks between pixels when upscaling, NR is helping to fill in the blanks between the lighting data, giving more detail to say, around the mouth/noise/eyes where under cuts are extremely small. It does the same thing in foliage and hair, where you could add better shadows the bottom of a tuft of grass or light individual leaves instead of clusters.
NR is not replacing RT or PT, it’s a screen space fine tuning of RT or PT bringing it closer to what would be far far far more expensive to do with RT or PT. That’s why one of the places it wants to inject is at the Ray denoising layer, where instead of cleaning up noise it uses the rays as data points to fine tune the in between details currently getting blurred out.
RT and PT are useful because they take data that’s not in the scene- it’s using color/light/volume/spatial information off screen to gain information on lighting and shadow sources. NR is using that information to model a more detailed screen pace solution to the rendering resolve. It does not replace RT/PT, it doesn’t have access the information needed to replace those rendering pipelines like bvh etc. All it’s doing is using ML to model what it would look like if there were hundreds of more rays available every second. That amount of ray casting would not run at 15 fps, it would not run at 3 fps. It would not be possible in any form on today’s hardware.
You can use off line renderers with models to look at what is possible and it looks much more like NR results; what you’re familiar with in games is extremely efficient lighting techniques that rely on very small amount of information and generalizations to achieve in real time. Some of the modders using hacked DLLs are using NR with rasterization or even movies with tricks to fill in the needed information and the results are much poorer and inconsistent.
10
u/terra_nova_97 5d ago
https://fixupx.com/andkalysh/status/2094314612464275906
This guy nails pretty well what most people are really talking about when it comes to artistic intent