Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.

Question Denoiser for HDRP pathtracing

Discussion in 'High Definition Render Pipeline' started by molestium, Jan 20, 2021.

  1. chap-unity

    chap-unity

    Unity Technologies

    Joined:
    Nov 4, 2019
    Posts:
    533
    As you can see, there's still a few things that needs polishing but this should land "soon".
     
    chadfranklin47 likes this.
  2. alejobrainz

    alejobrainz

    Joined:
    Sep 10, 2005
    Posts:
    288
    Hi. Is this PR compatible with HDRP 12? We are currently using the 2021.2 staging branch for HDRP, so I was wondering if this is something we could check out on our end or if it's based on the HDRP 13 codebase.
     
  3. chap-unity

    chap-unity

    Unity Technologies

    Joined:
    Nov 4, 2019
    Posts:
    533
    Since it's a new feature, it will be compatible with latest HDRP (probably HDRP 14 when it comes out) and more but it won't be backported (only bugfixes are).
    There's no harm in trying to backport it but we can't guarantee anything. :|
     
    alejobrainz likes this.
  4. alejobrainz

    alejobrainz

    Joined:
    Sep 10, 2005
    Posts:
    288
    Looks super promising. Es there any ETA to when the denoising package may be available to install? I tried to backport it into HDRP 12 but was only missing that package.
     
  5. MCYCShadow

    MCYCShadow

    Joined:
    Jan 1, 2018
    Posts:
    14
    Woop Woop the denoiser branch has been merged into master.
     
  6. alejobrainz

    alejobrainz

    Joined:
    Sep 10, 2005
    Posts:
    288
    Nice!!! Now hoping to see it backported and
    com.unity.rendering.denoising
    available on 2021.
     
  7. chap-unity

    chap-unity

    Unity Technologies

    Joined:
    Nov 4, 2019
    Posts:
    533
    Sadly, it won't be backported to 2021.2 since it's a new feature and we only backport bugfixes.

    Here's a few gifs of the results with only 32 spp on the HDRP template using the different options available.
    Overall Nvidia filter is a bit faster but produce slightly more low frequencies noise. There's also a temporal options available with Nvidia filter but the resuts can vary wildely depending on your scenes / animations.. etc
     

    Attached Files:

  8. alejobrainz

    alejobrainz

    Joined:
    Sep 10, 2005
    Posts:
    288
    Thanks for the update. I pulled from master and opened our project in 2022.1.0b6, but it seems the com.unity.rendering.denoising is still not available. Will it be available on an upcoming 2022.1.0 release?
     
  9. chap-unity

    chap-unity

    Unity Technologies

    Joined:
    Nov 4, 2019
    Posts:
    533
    Yep, the package is still under review and will be included in a future 2022.2 alpha release.
     
    fra3point, TerraUnity, wm-VR and 2 others like this.
  10. al3d

    al3d

    Joined:
    Feb 7, 2013
    Posts:
    51
    Hi when the 2022.2 alpha will release ? thanks
     
  11. al3d

    al3d

    Joined:
    Feb 7, 2013
    Posts:
    51
    Please someone answer me.. also is someone have a denoiser for the pathtracing right now ? thanks
     
  12. chap-unity

    chap-unity

    Unity Technologies

    Joined:
    Nov 4, 2019
    Posts:
    533
    Sadly, we don't really know for now, 2022.2 alphas have not been published yet.
    Also, keep in mind that you will need the denoiser package to be able to use pathtracer denoiser and right now, it's still not available on the package manager.
    I will update here when a first public version will be available.
     
    Last edited: Mar 11, 2022
    Yanus3D, TerraUnity and wm-VR like this.
  13. al3d

    al3d

    Joined:
    Feb 7, 2013
    Posts:
    51
    Ok thanks for the answer !
     
  14. MCYCShadow

    MCYCShadow

    Joined:
    Jan 1, 2018
    Posts:
    14
    So the denoiser will be an additional package loaded aside form the HDRP and Recorder right?

    Looking forward to it ^^ using the time until then to built the scenes :)
     
    chap-unity likes this.
  15. Arlorean

    Arlorean

    Joined:
    Sep 7, 2014
    Posts:
    22
    For anyone interested in an example render, here is a scene generated in Unity 2022.1.0b13.2913 using Path Tracing:
    RebelstarPathTraced.png
    And here is that same image run through IODN using the Declan Russel's command line tool:
    RebelstarPathTraced-Denoised.png
    The tool took 0.764s to denoise this 1366x768 image. Not great but good enough for what we need for now.
     
    valarus and newguy123 like this.
  16. UnifierX

    UnifierX

    Joined:
    Mar 29, 2022
    Posts:
    1
    +1 on the path trace denoiser!

    I don’t even know how to place a point light yet;) but just jumping over from Unreal where there is a good implementation of it .

    don’t suppose there has been any update with regards to ETA on a main build denoiser has there?

    btw
    Hello world
     
    fra3point and TerraUnity like this.
  17. al3d

    al3d

    Joined:
    Feb 7, 2013
    Posts:
    51
    2022.2 alphas is now available is the denoiser too ?
     
  18. al3d

    al3d

    Joined:
    Feb 7, 2013
    Posts:
    51
    I have installed the 2022.2 alpha and when I add the Pathtracing I can read "Path Tracing Denoising is not active in this project. To activate it, install the Unity Denoising package" what is the name of the package I don't found it ?
     
  19. chap-unity

    chap-unity

    Unity Technologies

    Joined:
    Nov 4, 2019
    Posts:
    533
    For now, it's expected, the package has not been published yet.
    When the time comes, I'll update the thread and the button "fix" on the pathtracer override will work as it should :)
     
    newguy123, TerraUnity, wm-VR and 2 others like this.
  20. al3d

    al3d

    Joined:
    Feb 7, 2013
    Posts:
    51
  21. chap-unity

    chap-unity

    Unity Technologies

    Joined:
    Nov 4, 2019
    Posts:
    533
    That's a good and important question. For starters, Pathtracer in Unity has never been intended to be a real time feature, so it doesn't make sense to just "slap a denoiser" on it to make it real-time.

    The denoiser filters added are indeed not meant to be used in real-time. To be fair, they sort of can, with a sample per pixel (spp) at 1, at a low resolution but the result can be peculiar to say the least.
    Also, it won't be stable temporally with such a low spp.


    The main problem this feature is trying to solve is basically remove the leftover noise from complex scenarios when the pathtracer, even with a large amount of spp, is still noisy. Basically, after accumulation a denoiser pass is added to "smooth out" the result. Sub Surface Scattering and/or fog is a good example of this.

    Here's an example from the PR:
    No Denoising

    Denoised:
     
    fuzzy3d, wm-VR, newguy123 and 2 others like this.
  22. alejobrainz

    alejobrainz

    Joined:
    Sep 10, 2005
    Posts:
    288
    Exactly, the perfect use case is to make offline renderings from unity. We used it extensively on a NFT project called Champions Ascension . Using path tracing, we were able to render 10,000 unique animated NFTs in record time using Unity. Since there was no denoiser at that point, we made a wrapper for OIDN, using the DLL that was already on the editor with the added benefit that using the unity plugin system we could create stand-alone builds that would work like a render slave, and uploaded 30-40 of those on NVIDIA T4 equipped instances on AWS, which allowed us to render out the set much faster than with any CGI path tracer (eg, vray, cycles, arnold, etc).

    IMO, Unity's Path tracer is super capable and one of the fastest options out there for CGI quality offline rendering.
     
    Last edited: Apr 8, 2022
    chap-unity and PutridEx like this.
  23. MassiveTchnologies

    MassiveTchnologies

    Joined:
    Jul 5, 2016
    Posts:
    86
    Any updates on the Denoising package availibility in 2022.2a?
     
    newguy123 and alejobrainz like this.
  24. Yanus3D

    Yanus3D

    Joined:
    May 6, 2019
    Posts:
    434
    ...waiting :cool:
     
    newguy123 likes this.
  25. valentinwinkelmann

    valentinwinkelmann

    Joined:
    Nov 3, 2014
    Posts:
    166
    Oh man I'm waiting like crazy for that!
     
    newguy123 likes this.
  26. Yanus3D

    Yanus3D

    Joined:
    May 6, 2019
    Posts:
    434
    How about me?! Denoiser is like Oxygen for visualization.....
     
    newguy123 and valentinwinkelmann like this.
  27. Onigiri

    Onigiri

    Joined:
    Aug 10, 2014
    Posts:
    350
    @chap-unity how can I denoise my pathtraced screenshot? Denoiser API wants NativeArray<Vector4> as source but I have NativeArray<byte> after calling tex.GetRawTextureData(). I don't know how should i convert it from bytes to Vector4
     
  28. Yanus3D

    Yanus3D

    Joined:
    May 6, 2019
    Posts:
    434
  29. Yanus3D

    Yanus3D

    Joined:
    May 6, 2019
    Posts:
    434
    My beard grown and is at the floor right now :D ....at least working version of activated Denoiser with bugs in Alpha will be welcome ....
     
  30. PutridEx

    PutridEx

    Joined:
    Feb 3, 2021
    Posts:
    1,035
    upload_2022-5-14_9-51-9.png

    I think the denoiser package landed in 2022.2 alpha 13
     
  31. Yanus3D

    Yanus3D

    Joined:
    May 6, 2019
    Posts:
    434
    Last edited: May 14, 2022
  32. wm-VR

    wm-VR

    Joined:
    Aug 17, 2014
    Posts:
    123
    Hey, I am happy to see that the denoiser has finally landed.
    I conducted a few tests and found it very stable (no bugs nor crashes so far, works also very well with DLSS and the new 4.0.0 pre4 recorder when extracting videos and pictures)

    One question: can I manually adjust the amount of "blurriness" under Volume -> path tracing -> Denoiser - temporal?

    I also figured out that Nvidias OptiX is way faster than Intels Open Image Denoise. But since I am using an RTX 3070 I am fine with that.

    Here are a few first impressions rendered in 1080p

    100s, NO DENOISE.png 100s, denoised.png 500s, denoised.png 35s, NO DENOISER.png 35s, denoised.png

    (some more test pictures I extracted a few days ago with 500 samples without the denoising feature: https://forum.unity.com/threads/wip-small-works-art-thread.145172/page-151#post-8124845)
     
    Last edited: May 14, 2022
    PavlosM, m0nsky, Deleted User and 2 others like this.
  33. Yanus3D

    Yanus3D

    Joined:
    May 6, 2019
    Posts:
    434
    10 sec FullHD:
    Render_100_10sA.jpg Render_009_2022-05-14.jpg Render_012_2022-05-14.jpg Render_100_10s.jpg
     
  34. Yanus3D

    Yanus3D

    Joined:
    May 6, 2019
    Posts:
    434
    3 sec FullHD
    Render_10_3s.jpg
     
  35. Onigiri

    Onigiri

    Joined:
    Aug 10, 2014
    Posts:
    350
    I'm getting worse results with aovs, is this expected? Also do i need to enable Runtime AOV API checkbox in render pipeline asset for them to work? Doesn't seem to change anything
    Without AOVs(OID 32spp)
    oid-noaov.jpg
    With AOVs
    oid-aov.jpg
     
  36. jeremedia

    jeremedia

    Joined:
    Apr 21, 2015
    Posts:
    62
    Here's how the denoiser works in the editor. Very pleased with the speed and results!
     
  37. PavlosM

    PavlosM

    Unity Technologies

    Joined:
    Oct 8, 2019
    Posts:
    30
    Hey, thanks for the feedback. Do you get the same behaviour with both Optix and the OpenImageDenoise backends?


    You can ignore this setting, you only need it if you want to use HDRP's AOV API in a player build.
     
  38. PavlosM

    PavlosM

    Unity Technologies

    Joined:
    Oct 8, 2019
    Posts:
    30
    We don't provide such setting, because the third-party neural-network denoisers that we use don't have an equivalent option. What we can do (in the future) is to "blend" the denoised/blurry image with the original one and allow you to control the "blend factor" as a percentage from 0% to 100%, but we need to investigate how useful this would be.

    Also note that the temporal mode right now has some ghosting/artifacts, we try to fix this before the final release. If you see ghosting please disable the temporal mode for now.
     
    Last edited: May 16, 2022
    wm-VR, Yanus3D and newguy123 like this.
  39. Onigiri

    Onigiri

    Joined:
    Aug 10, 2014
    Posts:
    350
    Only with OpenImage, Optix is ok i think
     
  40. newguy123

    newguy123

    Joined:
    Aug 22, 2018
    Posts:
    1,156
    This will be very useful. VRay has a similar feature in its frame buffer
     
    PavlosM likes this.
  41. Yanus3D

    Yanus3D

    Joined:
    May 6, 2019
    Posts:
    434
    Yes, we also have this option in Kray Render. Useful blend range which we use is 80% (sweet spot)
     
    PavlosM and newguy123 like this.
  42. Yanus3D

    Yanus3D

    Joined:
    May 6, 2019
    Posts:
    434
    It would be nice to add some more visible Path Tracing bar progress (scalable with resolution, stronger color, etc). In Hires is almost impossible to see it.
     
    Cascho01, fra3point and newguy123 like this.
  43. chap-unity

    chap-unity

    Unity Technologies

    Joined:
    Nov 4, 2019
    Posts:
    533
    The scale part has been done recently (not public yet). Before, it was a few pixel high and not dependent on the resolution leaving you with an incredibly small progress bar on very high resolution. Now, it's at least 4px high and 0.5% on the resolution.

    For the color part, you can easily modify the function AddConvergenceCue() in
    com.unity.render-pipelines.high-definition/Runtime/RenderPipeline/Accumulation/Shaders/Accumulation.compute
     
    fra3point likes this.
  44. newguy123

    newguy123

    Joined:
    Aug 22, 2018
    Posts:
    1,156
    I suspect, most people that would turn to Unity for pathtracing, are not coders, but artists wanting faster offline renders. So a simpler solution to this would be appreciated, like a UI option to adjust it for example...
     
    fuzzy3d likes this.
  45. Yanus3D

    Yanus3D

    Joined:
    May 6, 2019
    Posts:
    434
    That`s correct.
    PT+denoiser is a new gate for new users like me.
     
  46. Cascho01

    Cascho01

    Joined:
    Mar 19, 2010
    Posts:
    1,347
    I would like to access the progress at runtime via API to implement a custom progressbar - will this be possible?
     
  47. chap-unity

    chap-unity

    Unity Technologies

    Joined:
    Nov 4, 2019
    Posts:
    533
    This thread might help. (don't forget to search for answers before asking :)
     
  48. Cascho01

    Cascho01

    Joined:
    Mar 19, 2010
    Posts:
    1,347
    Thanks, so it´s very easy:
    Just divide the amount of frames you are running the pathtracer through its samples - done.
     
    chap-unity likes this.
  49. Cascho01

    Cascho01

    Joined:
    Mar 19, 2010
    Posts:
    1,347
  50. rmuk

    rmuk

    Joined:
    Feb 18, 2015
    Posts:
    53
    Just installed the beta and denoiser and am LOVING the visual results so far. I'm noticing a REALLY odd behavior with the path tracer though. Not sure if this is new or not, but if i move my camera around and HOLD in the right click button while it accumulates/denoises it takes about 12 seconds with 128 samples on an RTX3080. If I move the camera and then LET GO of the right mouse button, the accumulation/progress stops and GPU usage goes to zero. If i just move the mouse around on top of the scene view with no buttons held down it does proceed a bit, but it's extremely slow. If i hold the right mouse button again without moving the camera, it speeds back up. Any clue what would cause that?

    EDIT: Seems like the same is true for the game-view. The main reason I ask is that I saw the video above and realized that what I was seeing wasn't normal.