Search Unity

Feedback Recorder - Pathtracing with denoiser AND accumulation with recorder?

Discussion in 'Audio & Video' started by newguy123, Sep 26, 2022.

  1. newguy123

    newguy123

    Joined:
    Aug 22, 2018
    Posts:
    1,248
    Hi guys

    Its great that we can use accumulation with recorder, however, when I'm using denoiser, either Intel or nvidia denoiser, it works for a single frame in viewport, but how are we suppose to use it with recorder?!?

    For example I have pathtracing with 128 samples and intel denoiser (or nvidia)

    In recorder however, I set accumulation to 128 samples, however it goes to next frame without letting the denoiser do its thing...

    Unity 2023.1 alpha 11
    Recorder 4.0.0 release
     
    newguy123, Sep 26, 2022
    #1

  2. cguertin

    cguertin

    Unity Technologies

    Joined:
    Aug 20, 2019
    Posts:
    83
    Hey @newguy123! I just did a couple of tests and can't seem to reproduce this problem on my end.

    Just to double-check:
    1. Are the recorded images on disk also missing the denoising or only what appears in the GameView during the recording?
    2. Are you recording from the Recorder window or a Recorder clip (on a Timeline)?
    3. Which Recorder Type are you using (Movie or Image Sequence)?
     
    cguertin, Sep 26, 2022
    #2

  3. newguy123

    newguy123

    Joined:
    Aug 22, 2018
    Posts:
    1,248
    Hi @cguertin

    1) Images on disk is not denoised. In game view during recording I also dont see any denoising
    2) recorder direct from menu, not in a timeline
    3) image sequence

    Here's my settings with Intel denoiser (switching to NVidia makes no difference)
    upload_2022-9-27_9-27-55.png

    And here's my recorder settings:
    upload_2022-9-27_9-32-59.png

    This is just an initial quick test with jpg images. Eventually I will need to export EXR with various AOV
     
    newguy123, Sep 27, 2022
    #3

  4. cguertin

    cguertin

    Unity Technologies

    Joined:
    Aug 20, 2019
    Posts:
    83
    Thanks for the additional information! I tried the same setup and somehow the resulting images are always denoised.
    UnityRecording_denoise.png recorder_0000.jpg

    I'm a bit stumped on what else could be preventing the denoise from working correctly in your case. If you haven't tried already, would it be possible to redo your test in a fresh project? For example the HDRP template project available through the Unity Hub.
     
    cguertin, Sep 28, 2022
    #4

  5. newguy123

    newguy123

    Joined:
    Aug 22, 2018
    Posts:
    1,248
    is your camera animated @cguertin ?

    I have mine as a child of an animated object imported via fbx from max

    I'll try importing that object into the demo scene and redo the test and let you know the result...
     
    newguy123, Sep 28, 2022
    #5

  6. Mark_29

    Mark_29

    Joined:
    Aug 11, 2014
    Posts:
    79
    Edit - updated

    I just did a quick test and I seem to be getting similar issues. It was working for me for a percentage of the time, then stopped denoising. (120 frame animation and stopped denoising around 68 frames).

    I was using timeline, the recorder and cinemachine. No other plugins.

    PT settings 128
    Accumulation 128
    FPS on both timeline and 24
    Preferences TimeLine - playback locked to frame


    I can upload the video if you it helps
     
    Last edited: Oct 15, 2022
    Mark_29, Oct 15, 2022
    #6
    newguy123 likes this.

  7. cguertin

    cguertin

    Unity Technologies

    Joined:
    Aug 20, 2019
    Posts:
    83
    Hey everyone, just want to update that we haven't forgotten about this. I'm still going back to the issue from time to time but somehow haven't been able to reproduce the problem on my end yet.

    @newguy123, were you able to validate if this was also happening in a new project? I did multiple tests on my end with animated cameras and still no luck :(

    @Mark_29 I'm not sure if you noticed but when the denoising stopped mid-recording, did the non-denoised images appear to have 128 samples or only 1 sample (very very noisy)?
     
    cguertin, Nov 2, 2022
    #7

  8. newguy123

    newguy123

    Joined:
    Aug 22, 2018
    Posts:
    1,248
    Hi @cguertin sorry I have not had a chance to look at it again. I'm back in traditional DCC and rendering at the moment with other projects. However I'm hoping to revisit this very soon as I also need to get pathtracing and accumulation working with 360 rendering. We do a TON of 360 traditional rendering and would be awesome if I could move this to Unity
     
    newguy123, Nov 3, 2022
    #8

  9. Mark_29

    Mark_29

    Joined:
    Aug 11, 2014
    Posts:
    79
    Hi @cguertin - I am not 100% sure. I will need to try it again and see. In the timeline and the editor it was trying to "denoise" and it did, but the actual frame saved to the pro res was definitely 100% not denoised. I can try it again and upload a video.

    • Is there any particular release you want me to try out? (beta or alpha)
    • Is there anything else you want me to look at specifically?
     
    Mark_29, Nov 3, 2022
    #9

  10. cguertin

    cguertin

    Unity Technologies

    Joined:
    Aug 20, 2019
    Posts:
    83
    For tracking on our end, what Unity and Recorder versions are you using?

    One theory we have a the moment is that denoising might be stopping when the Unity Editor window is minimized, or another software window is in focus. It should still be able to render "in the background" there might be something weird around this.
    Another similar scenario could potentially happen if the GameView window becomes hidden mid-recording.

    It's not a solid lead yet but it's something I would try to keep a note of if the issue happens again.
     
    cguertin, Nov 8, 2022
    #10

  11. Mark_29

    Mark_29

    Joined:
    Aug 11, 2014
    Posts:
    79
    Recorder 4.0.0
     
    Mark_29, Nov 9, 2022
    #11
    cguertin likes this.

  12. cerrec

    cerrec

    Joined:
    Jan 19, 2017
    Posts:
    41
    SOLVED!

    I had this same problem - no denoising when using acumulation and rendering pathtracing. The culprit turned out to be DLSS. I had it enabled on the camera that was rendering. Once I turned off DLSS, the denoiser worked on every frame rendered with Recorder.

    I hope the Unity team looks into this, because DLSS is very useful in this application. It significantly speeds up rendering with virtually no loss in quality. And we need all the help we can get!
     
    cerrec, Dec 13, 2022
    #12
    BrettNexefy likes this.

  13. Mark_29

    Mark_29

    Joined:
    Aug 11, 2014
    Posts:
    79
    Ah..... Although you really don't need DLSS with path tracer, but it may be the cause for me too .... . I sometimes have DLSS on my camera for AA on RTX (not path tracer) when I was doing the renders. WIll go and double check to see if fixes the issue. Thanks for the info
     
    Mark_29, Dec 18, 2022
    #13