Search Unity

Question DLSS + Custom Pass with shader using HDSceneColor - is this scenario supported?

Discussion in 'High Definition Render Pipeline' started by mgeorgedeveloper, Feb 15, 2023.

  1. mgeorgedeveloper

    mgeorgedeveloper

    Joined:
    Jul 10, 2012
    Posts:
    324
    Hello all,

    We use a few different custom passes in our game.

    One is a custom pass to draw all the glass windows. It reads HDSceneColor to create a blurred effect on the glass surface (it is in a custom pass so that it can also blur the transparent water previously drawn - long story that I don't want to get into here).

    Another is our UI, which is in camera space, so that we can use a custom shader that again draws a blurred background using HDSceneColor.

    So far all is good and well...

    dlss-off-glass.png

    dlss-off-UI.png

    Now let's enable DLSS in the asset and on the camera. Just using some basic settings, like a forced 50% render size:

    dlss-enable.png

    dlss-camera-on.png


    Now this is where things start to go badly wrong.

    1. Any shader in a custom pass that samples from HDSceneColor, now reads bogus information for 75% of the screen. It's like the HDSceneColor buffer being read from is at the 50% scale, but the current shader is seemingly operating at full resolution.

    dlss-on-glass.png

    I can't upload more images - but basically... same issue in the UI.

    2. Furthermore - the UI "Tiles" shown before, now also jitter like crazy:



    So after a long and tedious wall of text, I guess my first question is... can DLSS play well alongside custom passes using HDSceneColor (shader graph node)?

    Secondly, can screen space UI be used at all without causing this crazy jitter?
     
    nehvaleem and akent99 like this.
  2. PutridEx

    PutridEx

    Joined:
    Feb 3, 2021
    Posts:
    1,136
    I believe I've seen this somewhere in the issue tracker, as a bug.
    But I'm not sure, it's been a while and it might've been a different issue so I can only hope so as it sounds like a critical issue.

    Hopefully someone with more info can help out.
     
    mgeorgedeveloper likes this.
  3. m0nsky

    m0nsky

    Joined:
    Dec 9, 2015
    Posts:
    257
    Which version of Unity are you using?
    If you're using _ScreenSize anywhere in your shaders, try swapping them for _PostProcessScreenSize
     
  4. mgeorgedeveloper

    mgeorgedeveloper

    Joined:
    Jul 10, 2012
    Posts:
    324
    Unity 2022.2

    It's nothing as fancy as hand-written shaders or access to parameters such a "_ScreenSize".

    Here's what we have... a pretty simple custom pass with Before Post Process injection point:

    dlss-issue-custom-pass-settings.png

    And then a Shader Graph that simply reads from HD Scene Color using screen co-ords. The screen co-ordinates are the issue, since they seem to be in full res, but the HD scene color buffer seems to be at 50% res?

    dlss-issue-shader.png
     
    Last edited: Feb 16, 2023
  5. mgeorgedeveloper

    mgeorgedeveloper

    Joined:
    Jul 10, 2012
    Posts:
    324
    And I have some kind of breakthrough, which is a hack/work-around that does not make a lot of logical sense.

    If I multiply the UV to match the DLSS percentage, then the HD Scene Color node reads correctly:

    dlss-hack-shader.png

    dlss-hack-working.png

    But the reason this is a little baffling, is because:
    - DLSS injection is set to AFTER POST PROCESS - this is when the final upscale should be happening in my understanding.
    - Custom pass injection point is set to BEFORE POST PROCESS, so all buffers should be in the 50% dimensions.

    So I guess "Screen Pos" is just always seen as full res... so how would one read from HD Scene Color in a manner that is agnostic to the DLSS percentage?

    NOTE - even reading HD Scene Color using default UV (no input), the issue is the same. So it seems the HD Scene Color node does not have a correct understanding of DLSS scaling.

    --------

    Anyway, this still leaves the insane Jittering shown in my original post in the video, which I will dig into separately to try to find a solution.
     
    Last edited: Feb 16, 2023
  6. larsbertram1

    larsbertram1

    Joined:
    Oct 7, 2008
    Posts:
    6,900
    if you look into the lit code for refraction you will find something like this:
    Code (CSharp):
    1. float2 samplingUV = samplingPositionNDC * _RTHandleScaleHistory.xy;
    when using a blurred sample you have to be even more carefully as here you will sample from lower mip levels of the color pyramid which will contain white spill at the upper and left edge:

    Code (CSharp):
    1. float ceilRefractionBlur = ceil( abs(MipLevel) );
    2. float2 maxCoord = _RTHandleScaleHistory.xy - pow(2.0, 2.0 + ceilRefractionBlur) * _ScreenSize.zw;
    3. samplingUV.xy = min(samplingUV.xy, maxCoord);
     
  7. reddotgames

    reddotgames

    Joined:
    Apr 5, 2011
    Posts:
    707
  8. mgeorgedeveloper

    mgeorgedeveloper

    Joined:
    Jul 10, 2012
    Posts:
    324
    I think in v2022.3.8, the use of HD Scene Color seems to work OK in a custom pass. I will have to check again to confirm this, because I only recently switched back to a custom pass for our glass, and I didn't notice an issue.

    But I also use FSR2, and this still has the issue for sure.

    So the only way I know to solve it, is to manually adjust the HD Scene Color input UV by the scaling factor that is currently active. This is on my to-do list, fixing the FSR2 incorrect sampling of scene color, so I will report back here as soon as I can.
     
  9. mgeorgedeveloper

    mgeorgedeveloper

    Joined:
    Jul 10, 2012
    Posts:
    324
    In the latest LTS 2022.3.18, no hacky adjustments are required to correctly read the HDSceneColor node for either DLSS or FSR3 (store package).