Question Denoiser for HDRP pathtracing

I searched whole internet, and could not find a clear answer. Is there any way denoise image in real-time using patch tracing? I know that Unreal engine already has it (DLSS), but for the unity, I did not find anything. Please help, maybe there are some other projects which add denoiser in Unity.

 

Thanks! Strange, on my GeForce 3080 Unity patchtracing works is very fast. And for me it was a big disappointment that I cannot turn on denoiser . I would like to get something like Quake 2 RTX, but it seems not destiny. Primarily for realistic realtime lighting and shadows.

Hopefully in the future developers will add this function, because people with RTX cards getting more.

Yeah, DXR rendering is very cool, but it does not provide realistic lighting as with patch tracing.

 

I'm found this:

Source: https://blogs.unity3d.com/2019/04/16/introducing-unity-2019-1/#optix

This for only lightmaps?

 

[QUOTE="molestium, post: 6753148, member: 2441404"This for only lightmaps?[/QUOTE]

ye

 

Correct but since you added it, do the right thing and let us get some fast and nice renders for marketing purposes. It should not be that difficult to implement.

 

Why is this taking so long? Been on the roadmap for months.

 

Now with dlss 2.0, pathtracing works realy great. But if a realtime denoiser could be connected to this, it's realy going to be game changing. We work a lot with Chaos Vantage and they have a similar approach and this dramastically changed our workflow. So if a realtime game engine like Unity would have this, it would be amazing.

 

Im using unity by now as my main renderer for cinematics, the usage of having all in place beeing able to easily integrate effects, simulations and particles is just very comfortable.

But yes I would also love to have an update on the pathtracing which includes a denoiser to reduce render times.

 

Been using the path tracer lately, a denoiser would really be helpful.

 

Yes, denoiser - please!

 

Denoiser! please!

 

Well, of course its not but hey it is a nice thing for rendering.
And it can open the door for more people using it to make offline media instead.

Voila, you can see how noisy it is still. But overall its pretty nice.

 

+1 on denoiser for pathtracing please!

 

There has been a card for it in the roadmap for quite a while now. Any ETA on it please?

 

We are at point where we are almost going to develop our own denoiser from scratch

Ah thanks for the update. We will implement own temporal denoiser in that case.

 

Will you make this aviable on the asset store by any chance?

 

Awesome news, thanks all!

 

Amazing! Any plans for it to support animation (temporal denoising)?

 

Finally! Can't wait to see it in action!

 

We ended un using Intel Open Image Denoiser which we invoke using a command line process when the image is converged. Results are pretty great and integrated seamlessly using a noise image as input. We added albedo and normals AOVs for better result.

 

Is this a solution you would like to share?

 

It's pretty simple, really. You can download a command-line version of the denoiser at:

https://github.com/DeclanRussell/IntelOIDenoiser

You can simply pass in a beauty pass (the noisy image) and it will be denoised. If in addition, you are rendering out AOVs, you can add an Albedo and a Normal pass for better results. Something like this should work:

Code (CSharp):

1.  
2.     public static void SaveFrame(string outputDenoisedImagePath, string noisyImagePath, string albedoImagePath = null, string normalImagePath = null)
3.     {
4.         string denoiserPath = Path.Combine(Application.streamingAssetsPath, "oidn\\Denoiser.exe");
5.         string args = $"\"& \'{denoiserPath}\'\" -i \'{noisyImagePath}\' -o \'{outputDenoisedImagePath}\' -hdr 0 -affinity 1 -clean_aux 1";
6.         if (albedoImagePath != "")
7.         {
8.             args += $" -a \'{albedoImagePath}'";
9.         }
10.         if (normalImagePath != "")
11.         {
12.             args += $" -n \'{normalImagePath}'";
13.         }
14.         System.Diagnostics.ProcessStartInfo startInfo = new System.Diagnostics.ProcessStartInfo();
15.         startInfo.FileName = "powershell.exe";
16.         startInfo.Arguments = args;
17.         startInfo.WindowStyle = System.Diagnostics.ProcessWindowStyle.Hidden;
18.         startInfo.CreateNoWindow = true;
19.         System.Diagnostics.Process processTemp = new System.Diagnostics.Process();
20.         processTemp.StartInfo = startInfo;
21.         processTemp.EnableRaisingEvents = true;
22.         try
23.         {
24.             processTemp.Start();
25.         }
26.         catch (Exception e)
27.         {
28.             Debug.Log(e);
29.             throw;
30.         }
31.     }
32.  

 

Be sure to look at the command line denoiser parms to make sure you set things the way you need. In our case, we were using sRGB input, so HDR was off, we use affinity to maximize core count and clean aux since the AOVs seem to come out clean (although not antialiased).

Also note in our case we placed the denoiser in the streaming assets folder so it was available on standalone builds.

 

We are very happy with the results of this approach. Easier than waiting for the next HDRP version, and will literally produce the same results if given the same inputs and the same denoiser. That same author has a command-line flavor for the Optix Denoiser, but as far as I've tested on Blender, Open Image Denoiser tends to preserve details a bit better IMO, as I don't mind about the denoising speed too much, as renders in our Unity-based path tracing use case are being done offline

 

Hopefully, you make the denoising call public so that if devs are not using the multi-frame accumulation APIS, like in our case, we can still enjoy denoising features, just providing three Texture2D objects, or even better it's applied to the Camera prior to firing the IsComplete flag, so that it's done automatically.

 

@etur I see there is a pt_denoiser branch in the github repo. I tried to use it but it seems to require a package called com.unity.rendering.denoising, but no idea how to install it. Hope to get a glimpse of how it will work and provide feedback as we are actively working on offline rendering based on path tracing on an ongoing project.

 

Many thanks Alejandro!

 

Incidentally, you can also make a wrapper for the Open Image Denoise DLL and invoke it directly. I saw a couple of implementations of this out there on a google search. It might be the better option to avoid invoking a command line or powershell app.

 

Just being curious, will the denoiser be applied during the rendering-process or when it´s finished?

 

Could you guys give us updated timeline for the denoiser? Also if it will support temporal denoising (for consistent results across frames in an animation?)

 

Also, it would be nice if it works with the frame accumelation API (for offline rendering and cinematic motion blur)

 

Is there any third part denoiser, maybe from intel or amd? Or how could we make one?

 

https://docs.unity3d.com/Packages/c...high-definition@12.1/manual/Accumulation.html
This can help

 

You can integrate either Intel Open Image Denoise or Nvidia AI denoiser. I found Intel easier as there is a DLL you can make a wrapper around. For NVIDIA, you would need to first create a DLL and then a wrapper. The Intel DLL is actually already in the unity editor install path, so you only need to create a wrapper. I tried this one and it works great in Unity:

https://github.com/teamclouday/PathTracer/blob/main/Assets/Scripts/Denoise.cs

You can use the accumulation APIs or you can create your own simply updating to collect all samples and using the new API renderPipeline.IsFrameCompleted().

 

If it´s that easy it should be definitely in one of the next Unity-Versions.

 

Lol, ill try

 

Do you have a way to perform AA on the AOVs?

 

not really, I'm using the AOVs as they are outputted by unity, but images look pretty good IMO using Open Image denoise.

 

I'm seeing that the un-antialiased AOVs are introducing the jagged edges back into the final denoised output, so I'm just leaving off the AOVs for now.

 

Anyone knows why some meshes don't show up when using the pathtracer in 2021.2? Even the HDRP template has this issue, most meshes don't show up at all.

I did a test with 7 meshes, all the same material, and are cubes extended to create a room but only 2 show up. Any ideas?

 

Are they LOD?
If yes then this is the problem
Else, try restarting unity or try reactivating cameras

 

Nope, they're just gameobject cubes.
The problem happens when I press play and go in-game. In the scene it renders correctly.

 

Just going to make the case for a denoiser being given a higher priority if it hasn't been already. I'm using Unity / HDRP as a preferred method for producing fairly large amounts of video content for commercial clients. I use Unity for a whole bunch of different reasons, not the least of which is I just plain like it. It also fits well with my workflows and the renders you get from HDRP can be outstanding. I try to avoid tools such as TwinMotion because they have almost zero animation capabilities. Yeah, there's Unreal but project sizes end up being very large and Unity just feels a bit more flexible to me somehow.
Path tracing is becoming an increasingly big deal for photorealistic renders though. Some recent tutorial videos of TM's path-tracer also happened, almost incidentally, to show just how effective denoising is in improving image quality and reducing production time, which is at the end of the day the primary reason for using a game engine as opposed to offline renderers. I guess basically I want it all - quality, speed, ease of use, flexibility, marmalade even. Denoising seems like an important component in making this happen.
Just saying folks, but either way - keep up the good work, it's all appreciated.

 

Denoiser is actually in progress: [HDRP] Path tracer denoiser by pmavridis · Pull Request #6652 · Unity-Technologies/Graphics (github.com)