Bug Path Tracing - Denoiser, odd behaviour (due to Volumetric Fog)

So I've been toying around with the Path Tracing for a while, and I've figured out why the denoiser doesn't really give good results when you have point lights, spot lights, etc in your scene.
What I mean is that you get visible edges around the "cones" for all the lights. This includes cones for spotlights, halfdomes for arealights, and spheres for pointlights.
These are visible even through walls.
Here are a few examples:


It seems we're generating the AOVs using the Path Tracing itself, which includes the "cone" for the lights... in the Albedo and Normal used for the denoiser.
Here are a few examples of what is generated for the denoiser, albedo:

and normals


I can understand why one would want it generated by the Path Tracing as it would include depth of field, but it also gives you noisy albedo and normal results along with those anoying edges of the light cones.
Obviously it's not inteded behaviour to have those lights affect the albedo and normal passes, but this begs the question as to why it was left in from the beginning?

This has been here since the denoiser was implemented, has nobody noticed this before?

Of course those edges are just one part of the problem, as stated the same passes are also noisy.
For the best results you would think you'd feed the denoiser clean albedo and normal passes yet here we are feeding it terrible and noisy passes.

Possible solutions?
One way of mitigating these ugly artifacts would be to create custom passes for the denoiser but without lights in the acceleration structure. This would at least remove one part of the artifacting.

Would also want to feed the denoiser cleaner and non-noisy results, not sure how using Path Tracing as the method of generating these passes but using the realtime render for that would certainly do the trick. Of course this would lead to not supporting depth of field, so a more clever solution would have to be considered.

 

I manage to find out why those problems are there in the first place and how to fix it, unfortunately fixing it would require a rewrite of the denoising pipeline.

The Albedo and Normal passes are noisy because they use the same first segmentID (depth) of the Path Tracing, for some reason this first depth is noisy. But when we use any other value for our depth, 2 or higher, the are no longer noisy.
This is an easy fix, simply enforce using a deeper depth segment when generating Albedo and Normal passes for the denoiser right?
Well unfortunately they are linked at the moment. When you tick that checkbox to use AOVs with the denoiser, you are also generating those AOV passes at the same time. More accurately; we are simply writing the data we're already using into those AOV passes.
Using a deeper depth also gets rid of those ugly cones and outlines for the cones for lights.

In this example Albedo and Normal passes use the first depth segment, this is representing how those passes look when they are sent to the denoiser as is.
1spp is completely useless, while it gets better when you have higher sample count, it still stays fairly noisy.

In this example Albedo and Normal passes use any other segment apart from the first one, this represents how those passes should look when they are sent to the denoiser.
Here even 1spp is usable, it's aliased for sure but it's not unlike a rasterized albedo pass. 8spp or higher gives us more refined results with cleaner edges and less noise in textures and normals on surfaces.


In order to debug this I had to keep a local copy of the HDRP package and modify the relevant shaders in order to even display what is sent to the denoiser.
Fixing this would require rendering the AOV passes after the main render pass has finished.
There could even be seperate settings for them, since you won't need the same amount of samples as the main render you can get away with a faster render.

As it stand the denoiser is simply being fed bad data, no wonder it's generating worse results than it should.

TLDR:
Generating AOVs needs to be decoupled from the main render and be given it's own seperate settings in order for it to give the denoiser correct input.

 

Thanks for answering.
You are correct, disabling fog did indeed remove the noise. With that in mind, disabling volumetric for every light source did also remove the ugly visible cone in all renders.
Of course now I have no fog/volumetric fog in my scene, and no light will contribute to the volumetric fog. I have distance fog but not the nice volumetric goodness.
Now I have a much easier way for our own denoising method to grab albedo and normals from the path tracing itself.
This doens't solve the issue for the denoising in unity itself, since with this knowledge it means we can't have fog and clean denoising results.

here are a two screen grabs at 8spp, with fog and without. Very good result (when fog is off) considering the super low spp.

Fog:

No fog:


I still think they should be seperated considering the drawback of getting clean AOVs.
If they were seperate, one could render the noisy image at 200spp, and the AOVs at 8spp for denoising. And more considering they would be treated seperately.

 

Yes, I simply have volumetric off for the time being.

Though the AOVs are perhaps not the only "bug" related to volumetric fog and the path tracer.
Transparent surfaces occlude the volumetric fog, which I find a little strange if intended.
Say you have an interior scene, a living room with large windows and a balcony. Looking out through the windows doesn't show any volumetric fog at all. It shows the normal "depth fog" but not the nice volumetric stuff. But if you were to look through an open window or an open balcony door, now you'd be able to see the volumetric fog in all it's glory. Seems a little strange.
Glass or transparent surfaces do not stop volumetric fog from seeping through them however, meaning you can have both no visible volumetric fog on the outside and have loads of volumetric fog blaring through the window on the inside.