Discussion Unity Ray Tracing Enthusiasts Thread (sampling/denoising, optimizations, tricks, showcase)

The original thread was introduced to discuss HDRP's Experimental DXR project released back in 2019. Since the test project no longer exists, HDRP Ray Tracing is no longer in experimental, the discussion slowly drifted off into a general HDRP Ray Tracing discussion. A great place where we could learn and discuss, but it could also cause confusion for new people.

On a sidenote, the HDRP Ray Tracing subforum is still located under "Graphics Experimental Previews". Since Ray Tracing has now left experimental, maybe a moderator can move the entire subforum to HDRP? Thanks!

This thread is a place to discuss
- Sampling/denoising
- Optimizations
- Tricks
- Showcase
- Other implementations (how are they doing things, what is their performance like, what tricks do they use, what can we learn?)

Useful tricks
- Proxy Meshes (LOD0 for primary rays/gbuffer, LOD1 for RTAS), thanks to @Lex4art

Useful links
- HDRP Ray Tracing Samples
- HDRP DXR/Realtime Ray Tracing Lighting Troubleshooting Guide
- DXR in Virtual Reality Thread

Community Ray Tracing Projects
- TrueTrace (Github) (Asset Store)
- Buildmark (Steam)

Important Milestones
Please let me know if anything is missing/incorrect, I will also be looking into this later.

Unity 2020
- Ray Tracing skinned mesh renderer support was added in 2020.1
- DX12 performance was improved in 2020.2

Unity 2021
- DLSS support was added in 2021.2

Unity 2022
- Ray Tracing CPU performance was massively improved (8x-30x due to CullInstances API) in 2022.1
- DX12 left experimental in 2022.2
- Path Tracing Importance Sampling (Sky) was added in 2022.2
- Path Tracing Denoiser was added in 2022.2

Unity 2023
- Ray Tracing left experimental in 2023.1
- Ray Tracing VFX support was added in 2023.1
- Ray Tracing Terrain support was added in 2023.1

 

To kick things off, I am personally working on a HDRP real time path tracing implementation for my game called Hyper Project. I've spent about a year studying various topics, making proof of concepts and writing a lot of notes and have now started on the implementation. I am ray tracing the full specular and diffuse path with 2-3 bounces and hope to bring this to VR.

 

Here I start something new in my pipeline: Manual Tone Mapping - yes I stopped to use ACES.
First of all, I compare always quality to the Corona Render Engine as the best render engine (this is the CPU field so is a slow time render but not that like Maxwell or Blender, and is similar to V-ray speed)

Corona has many advantages like very fast Photon Mapping+QMC system render (same I have also in KrayTracing in Lightwave), the best ever shader system/Nodes (I cover practically all in Unity), and a super Tone Mapping system (this gives big boost quality in Corona).

The tone Mapping system in Unity is also very good.

here is Corona's render:

 

Here is Unity:


The next step will be External Tonemping.

 

Actually, I noticed that Custom Tone Mapping is Tanh system - so that is very good

 

Here is a trick on how to avoid GI "Raytracing Effect" (poor GI bounce light).
Create Area Light DXR with a distance from the ceiling max 5 mm (so almost with the same level) and a size a bit bigger than the ceiling. The direction of light points to the ceiling.



Before:


After:

 

Some new ideas with lighting and post-process (WiP)

 

Here is the finished scene:

 

#2

 

I'm wondering why there's no raytraced caustics in unity.

 

There are caustics in the HDRP path tracer, I tried it in a separate project a while ago. Do you have a screenshot of your setup?

 

I'm not about pathtracing. Just usual raytracing like rt caustics in unreal.

 

old dusted folder with silly scene:

 

Some renders from refreshed Sponza scene:

 

#2

 

Finished my 2-week dive in Cyberpunk 2077 with path tracing - got a lot (700+) of screenshots with inspiring lighting & detail level. If somebody don't have time/don't like gameplay - maybe those screenshots will be worthy - you can browse them on-line (to download all at once click on folder name there and pick appropriate option from drop down list):
https://drive.google.com/drive/folders/1unqCfilu25MAp1P5ChwnSKrgFJagsmGD

Path tracing implementation is far from perfect but in general it's working (and of course fog is not lit by GI and can ruin a lot of visuals when dust storm coming at day time). Anyway, amount of details in this game is impressive, monumental effort of the large game studio.

 

Yes, I installed it only for Path Tracing effect (The game is boring as hell itself...I do not even have plans to play it) On my hardware I got like 60 fps in 4K with maximum settings....pretty impressive performance!
I use this game only to walk in the night city and enjoy real-time GI/reflections and details.
Visible ghosting effect - somebody will complain - but this is real-time GI/reflection!
Next few years this will be standard in games because it is much easier to do it (no need fake solutions every 5 meters, optimization, baked maps, hundreds of reflection probes, etc)
I like in-game when moving objects have contact shadows, real area light shadows, raytracing reflections, and good contrast.

 

I improved the quality of Sponza by some fresh ideas:

 

#2

 

ReBLUR for ray traced reflections in HDRP

 

m0nsky
How can I add this to the actual 2022.2 Unity?

 

I think 2022.2 is too old, not sure if it will get backported, but it landed in 2023.2.0a14

 

Thanks!

 

Any chance this will also be implemented for the diffuse counterpart? It would solve a lot of headaches for me.

 

Hi!
Can somebody write me a basic script for recording images with specific procedures?
Of course, I will pay for it via the company.
Details via PM.
Thanks!

 

...or maybe somebody can post here some basic scripts with camera recording and delays recorded between frames?
This is really basic - unfortunately, I am not a coder (not enough length of life to learn everything... that's enough that I spent 30 years for CPU offline rendering...)

The script is very basic:
- The script must be attached to the null object
- should have a camera selector menu (because will be more cameras than one, simply selector for cams that you want to use in the script)
- should have start delay before recording the image and delay for the next frame also as well (in seconds)
- should record PNG
- whole procedure starts with pressing Play Mode and ends by the exit from Play Mode itself
- each camera will have simple animation of 0-20 frames maximum.
- each camera will be attached to the Null object (that's the best way I think when you want to change the position of this camera without changing animation)

 

chap-unity
Works but with a massive amount of render frames....I need the specific frame. After render, I have to fish those which I need which makes whole procedures useless (I can do it manually at the same time)

Here is a simple example:
- camera rotates in Y 360 degree
- rotate step 20 degree
- the number of frames: 18
- between frames and at the beginning "animation" must be a manual delay in seconds (to remove DXR ghost effect) for example 10 sec
- whole procedures: 180 sec

The script is easy to write down but I am not codder
It looks like I am looking for something dumb, but trust me is not

 

I think this should be possible with the recorder, which specific feature are you missing?

Edit
I just made this for you, I hope I understood you correctly. Attach it to your camera and press play. (using a "null object" makes things overly complex) Also I think it would be better to use x amount of frames for accumulation instead of seconds, but I think the recorder can also do this.

https://gist.github.com/m0nsky/1a41a56743b54c559010412e77137e2d

 

The Recorder missed:
- delay start record (in seconds)
- delay record between frames (in seconds)

First is important when you start animation in DXR. You lose 10-20 frames (it goes to the trash) and you can not render designed cameras in one shot (must be done manually)

Second, there is no solution to tell Unity that frame is finished completely. Unity does not recognize that frame is clean from artifacts because this varies from step frame, speed camera, and many others. It knows that the procedures of the engine for one frame are finished only.

It looks like it is a stupid idea to add an Offline mode delay record between frames, but is not.
When you use DXR in animation you face a couple of problems (offline mode is not solving them):
- artifacts in GI (HiRes render)
- flickering level of brightness randomly in a group of frames - this is a Unity bug
- ghosting effect

Adding 2-5 sec between frames will help to create PERFECT animation...I know that time rendering will extend 5-10 times or even more, but who cares! Still is in minutes for one camera shot. I remind you of CPU animation rendering: a few days of basic and very short animation without a render farm.

 

m0nsky
Great!

Here is the first bug (so this is something connected with a missing accumulation system?):
I gave 10 sec wait time for HiRes Images. In Game mode everything went well - the image looks good and clean in DXR before the next frame jump but the save Images are different.
frame 0 is ok because of start procedures...it looks like Unity saved images earlier or is something wrong?

 

m0nsky
the goal is that:...but before I need working procedures
Had to scale it down from 10.000 pix (max res limits on the forum)

 

That is not the correct way to do it, this won't work.

What you are looking for is capturing a cubemap and converting it to equirectangular projection. Real time DXR depends heavily on accumulation which does not play well with RenderToCubemap.

Your best bet is to use the path tracer and then use the Recorder package to capture a single frame 360 view, and save to EXR.

 

This is not for cube map render...that's impossible to render it in Unity in DXR mode with HiRes.
This is for further action: stitching images. Whole Panorama procedures are very complex and need a lot of experience and knowledge.
Path Tracing: out of use, forget it.
I need just a working script and I will show you what I can do with those images

 

Guys! It cannot be so challenging to do it!
You are so talented in C#.....

 

...I got the script from a talented H-Trace developer!
Works Great! I can now render Panorama in DXR without any problems!

 

1. and pathtracing also?
2. Does the script call a custom write, or it still uses Recorder but with fixed settings under the hood?
3. Does the script come with H-Trace, or how can the rest of us get a hold of it?

 

I do not use Path Tracing, only DXR. The script is custom-written with my suggestions without the recorder.
The script can be used with anything in Unity (Bake, DXR, Path Tracing):


It looks primitive but works exactly as I want:
- delay recording - "artifacts removing system"
- angle camera for two passes (up/down) + amount of frames for 360 degree
- Automatic render resolution - very important for heavy scenes - you can keep the free aspect in Game mode.
- automatically remove post-process modules (You do not render panorama with Viggent, DoF, and others) but for normal renders, you need them (no need to Disable it manually for Panorama)
- It has a queue camera list (full auto)
- Render in the background (yes, you can work on other programs at the same time)
- Back to the original scene state: For some heavy scenes you must start from wire view (crash VRam limits problem) and also you do not "burn" GPU after finishing Panorama (for example you are absent)

The score (original size is 12.000 pix but I had to rescale it...limits on forum):

 

How can the rest of us get the script?
is the h-trace guy selling it on asset store?

 

The script was written only for me. I am not sure that can be sold again or you have to pay with full price - you can ask him (Discord channel: Passeridae)

Also, except script, you need three extra software:
- Topaz for upscaling and improvement quality
- PTGui for stitching images (necessary and only one software which can stitching properly)
- Pano2VR (This I use personally)

About PTGui:
It is expensive but is worth every money.
Do not waste time with other software for stitching images (especially those for free) they are totally bad and they do not work properly at all in automatic stitching. You need AUTOMATIC stitching!!

 

Some real time path tracing updates. Previously I was using a dual reservoir setup (for both direct and indirect lighting) which I have now converted to a single reservoir setup (half the buffers, half the VRAM allocation/reads/writes).

All videos are recorded in HDRP (in real time) without a denoiser, on a RTX 3080. Don't get fooled by the scenery, these are not voxels, this is an actual 90MB .obj mesh.

This first video has an issue because I left the RAY_FLAG_ACCEPT_FIRST_HIT_AND_END_SEARCH flag enabled. This was causing some blobs/speckles due to some surfaces reaching occluded lights. It's not too apparent so I decided to leave the video up anyways.

This one has the flag disabled. The remaining noise comes from specular (there is no specular resampling yet) and indirect lighting (it's very simple, just a cos hemisphere + VNDF ray). I'm actually surprised how well it catches these small torches, because there is no NEE currently.

I did finish up the reflection probe based NEE code in an experimental project which now allows for both occlusion & emission on a single, real time, 64x64 probe. Taking a higher MIP of the reflection probe actually improved the result because it helps "smoothen" the misalignment that reflection probes have by nature. All green pixels managed to find the emissive sphere on their first TraceRay.



I have also finished:
- Camera relative sample cache, which should help with sample validation far from the world origin
- Multi-bounce ray validation, which provides snappy response for indirect lighting when conditions change
- Optimized ray lengths for sample validation, we know where the sample is supposed to be so there is no point in traveling any further
- Fast validation paths, which allows sample validation while skipping all of the surface contribution / lightloop evaluation (which is very heavy)
- Neighborhood search for failed reprojection

I also updated the existing VNDF code to Sampling Visible GGX Normals with Spherical Caps by Anis & Jonathan @ Intel.

To be honest, there is just way too much more to put in a list.



I've added a health property for the reservoirs (based on their current M, and the defined M cap) which will be used to allocate the ray budget. I've done some testing with turning off the actual initial candidate RIS and flying around a scene, purely relying on temporal reprojection of existing samples in the buffer, which works well. So the heavy work (ray tracing with actual surface contribution / lightloop) can be allocated to reservoirs killed by disocclusion & failed validation.

I've also added decoupled shading for spatial iterations. I don't shade the temporal and RIS reservoirs because they are low quality by design. This clears up a good amount of noise without tracing any extra rays, and does not introduce any bias.


Only shading the winner sample


Shading every spatial iteration (no extra rays compared to the previous image!)

I've also prepared the classic MIS scene from Veach's 1997 thesis, in which I will be finalizing the specular resampling.



I've prepared two methods, one which stores the NdotV in the .w (unused) component of the existing luminance cache to reweigh the sample in an estimation of the current specular lobe, and one which uses a separate buffer to store the original view vector to reconstruct the actual GGX lobe.

My graphics to-do (the must part, at least) is now so small that there is no longer a scroll bar, so hopefully I can round this up soon™ so I can get back to actually developing the game.

 

Looks impressive!
The pity is that it is your Engine for internal projects....