Search Unity

  1. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Graphics H-Trace: Global Illumination and Occlusion [ RELEASED ]

Discussion in 'Tools In Progress' started by Passeridae, Mar 11, 2022.

  1. blueivy

    blueivy

    Joined:
    Mar 4, 2013
    Posts:
    621
    IDK how i missed this post but the reflection probe + indirect shadows would be sufficient for 80% of scenes. With camera color buffer tracing taking it completely to the next level! Awesome work!

    • Question, do you see H Trace supporting the built in pipeline or would that be too much work?
    • Also, with the reflection probe fallback, does it still allow you to change the lighting (ie sun direction) without having to re bake the probes?
    • Last question, but when the final product is released can you provide a toggle of disabling the camera color buffer? Is there a big performance impact of with/without camera tracing?
     
    Last edited: Aug 9, 2022
  2. Passeridae

    Passeridae

    Joined:
    Jun 16, 2019
    Posts:
    395
    Hello :)
    No, I haven't faced any major difficulties so far. Of course there are some issues here and there: like hdrp-specific integration stuff, some bugs and so on. But overall, I'm wrapping the whole thing up and I finished the denoisers three days ago.

    I still can't promise anything about the exact release date, but maybe you can dm me and we can figure something out for your specific case.
     
    Felinde likes this.
  3. Felinde

    Felinde

    Joined:
    Sep 19, 2021
    Posts:
    18
    News of the urp version? If the work is going on.. is there a chance that it can be compatible in webgl? the ssgi on the asset store are not comparable to this, to have that result and performance at that point you use the pointlight...
     
  4. Passeridae

    Passeridae

    Joined:
    Jun 16, 2019
    Posts:
    395
    Hi! I think it's totally doable, but first we'll need to see how it goes with HDRP and URP versions. And we also have to spend our resources wisely, because world-space fallback (Voxels, SDFs, etc.) is also a major goal, and it will take some time to develop.

    You have to either rebake the probes or update them in real-time to make it work correctly. Nothing prevents you from changing the lighting and not rebaking the probes, of course, but then you'll get a mismatch between the primary tracing and the fallback.

    You can use the "time slicing" option to decrease the performance impact from real-time reflection probes. HDRP has this feature implemented in version 14 and later, and URP always had it. We have also prepared a script that can update a certain number of probes in a sequence (also using time slicing), sorting them by distance to the camera, but it's not fully tested yet. Plus, we can use a single real-time reflection probe attached to the camera. This is not an extremely robust fallback, but it may serve well for certain scenarios.

    But again, we are trying to find the right balance between investing our time into reflection probes or just switching to, for example, voxels, that can just replace probes :)

    You mean an option to use just the fallback alone? Yes, it could be done, but the performance gain might not be worth the visual downgrade.

    We have migrated all the features from the HDRP version the the URP version, except the reflection probe fallback support. Here's a bit old screenshot comparison from URP:

    upload_2022-8-11_16-18-17.png

    upload_2022-8-11_16-16-9.png

    We haven't tested this yet. I think we can take a look at it when we submit the 1.0 version to the Asset Store.
     
    saskenergy, Felinde, Marseol and 5 others like this.
  5. blueivy

    blueivy

    Joined:
    Mar 4, 2013
    Posts:
    621
    I have to compliment you every time you post new screenshots. Because the results are always beautiful!

    Also about world space fallback, are you planning on implementing that after the initial release? I see that taking a lot of effort.
     
  6. TyGameHead

    TyGameHead

    Joined:
    Apr 29, 2017
    Posts:
    21
    I really need H trace and the GTAO for urp feel like the fps would be better
     
  7. Passeridae

    Passeridae

    Joined:
    Jun 16, 2019
    Posts:
    395
    Hi everyone! :)
    As we are approaching our HDRP release, I wanted to share our current progress. This is H-Trace in the (99%) finished state:

    upload_2022-8-20_19-47-5.png

    upload_2022-8-20_19-47-11.png

    upload_2022-8-20_19-47-22.png

    upload_2022-8-20_19-47-29.png
    All the credit for this beautiful HDRP Sponza scene goes to @kristijonas_unity

    I'll try to post a video and write about our final feature list as soon as possible, stay tuned :)
     
  8. Blade_Master_2020

    Blade_Master_2020

    Joined:
    Aug 3, 2019
    Posts:
    38
    Passeridae, Felinde and MaximKom like this.
  9. blueivy

    blueivy

    Joined:
    Mar 4, 2013
    Posts:
    621
    Congratulations on this milestone! You've already done amazing work and can't wait to purchase this!
     
    Passeridae likes this.
  10. Passeridae

    Passeridae

    Joined:
    Jun 16, 2019
    Posts:
    395
    Here's a first video. It's recorded without TAA, because my goal was to show how the H-Trace denoiser works on its own. Though, I'm not sure it's really visible through YouTube's compression. But anyway, finally some motion!

     
  11. Blade_Master_2020

    Blade_Master_2020

    Joined:
    Aug 3, 2019
    Posts:
    38
    Very beautiful indirect soft shadows !! The GI looks amazing ! Great job !
     
  12. Felinde

    Felinde

    Joined:
    Sep 19, 2021
    Posts:
    18
    The denoiser is excellent! It also seems very reactive to 0:29. I'm curious to see how it behaves with moving emissions such as a flashlight or in very large environment. But above all, an estimate of how much time is missing for the release (considering the acceptance times of Unity)? In any case, the work done is amazing, we need it!
     
    hkalterkait and Passeridae like this.
  13. Passeridae

    Passeridae

    Joined:
    Jun 16, 2019
    Posts:
    395
    Thank you! :)

    I'll try to capture it on a video. But your question got me interested in how H-Trace in its current state behaves with emissive materials in general. I haven't tested this side for a while. So I made some quick comparisons with Unity's SSGI.

    Unity's SSGI (Hight Preset + Reflection Probes fallback)
    upload_2022-8-22_17-34-21.png

    H-Trace
    upload_2022-8-22_17-34-32.png

    Here's a side by side comparison:
    https://cdn.knightlab.com/libs/juxt...html?uid=ec5d539c-221e-11ed-b5bb-6595d9b17862

    Here's another one from a bit different perspective:
    https://cdn.knightlab.com/libs/juxt...html?uid=58ea8fe4-221e-11ed-b5bb-6595d9b17862

    The rough guesstimate is ~2 weeks to finish everything and the rest will depend on the speed of the submission process of the Asset Store.
     
  14. Passeridae

    Passeridae

    Joined:
    Jun 16, 2019
    Posts:
    395
    Yeah, GI for emissive materials has a lot of impact.

    upload_2022-8-23_2-13-58.png

    upload_2022-8-23_2-12-40.png
     
    Felinde, sjm-tech, MaximKom and 8 others like this.
  15. Passeridae

    Passeridae

    Joined:
    Jun 16, 2019
    Posts:
    395
    Final denoising optimizations. Small details are preserved way better now, even under strong blur with wide radius.

    Before:
    upload_2022-8-23_18-48-23.png

    After:
    upload_2022-8-23_18-48-39.png
     
  16. blueivy

    blueivy

    Joined:
    Mar 4, 2013
    Posts:
    621
    That's a HUGE difference/improvement! Nice work.
     
    Passeridae likes this.
  17. Felinde

    Felinde

    Joined:
    Sep 19, 2021
    Posts:
    18
    I'm appreciating Unity's SSGI, but in this comparison it almost makes you laugh...that blue ball is exciting. But I see that with H-trace active the triangles increase to 1.0m, what is it due to? Do the Gameobjects that project shadows from indirect light must necessarily be static or can they be dynamic?
     
  18. Passeridae

    Passeridae

    Joined:
    Jun 16, 2019
    Posts:
    395
    The increase in triangles is due to rendering backfaces of objects into a separate depth buffer. This buffer is needed to reconstruct correct object thickness. This is optional, and if you choose to disable it, then you can control thickness with a single value, like you do in Unity's SSGI. However, the performance cost of this feature is relatively small and can be compared to enabling a shadowmap on a single spot light. Objects are rendered with a unified and extremely light (almost empty) shader that only writes their depth values.

    Everything can be dynamic. Just remember that the emission ball will emit light as long as it's in the frame. If you want it to continue to illuminate everything even when it's outside the frame (or occluded by other objects), then you will need to capture it into the reflection probes (for fallback).
     
    Felinde, blueivy, MaximKom and 3 others like this.
  19. blueivy

    blueivy

    Joined:
    Mar 4, 2013
    Posts:
    621
    That gives me the idea of lighting entire scenes only with lots of static emissive lights that are captured in the reflection probes! Is the performance hit of emissives determined by how many are on screen?
     
  20. Passeridae

    Passeridae

    Joined:
    Jun 16, 2019
    Posts:
    395
    Specular Occlusion feature is added.

    Without Specular Occlusion (Example 1):
    upload_2022-8-25_1-9-9.png

    With Specular Occlusion (Example 1):
    upload_2022-8-25_1-9-40.png

    Without Specular Occlusion (Example 2):
    upload_2022-8-25_1-10-25.png

    With Specular Occlusion (Example 2):
    upload_2022-8-25_1-10-39.png

    Specular Occlusion is generated by screen-space ao + screen-space bent normals / bent cones and comes with no additional performance cost. It uses the latest algorithm Unity has implemented in HDRP and thus should be on par with the native SO that you can find in HDRP shaders.

    The additional benefit comes from its real-time nature: objects can be dynamic and can cast SO in real-time onto each other, creating correct occlusion in places of object intersections.

    Short version: The same story as with baked AO vs. screen-space AO

    Long version: When you are using typical SO in shaders, you usually provide bent normal + ao maps that are baked offline in 3rd party software. Baking process is typically done on a per object basis and can't account for other objects that will be placed in the actual scene. So, unless you bake objects in groups, you will end up with different objects not casting SO onto each other (only on themselves). And when some objects are dynamic, then there's virtually no way to create SO between them using baked maps. Real-time Specular Occlusion solves this for you and doesn't require any uv unwrapping and texture baking :)
     
    Last edited: Aug 25, 2022
    Hubster, Gooren, florianBrn and 7 others like this.
  21. Passeridae

    Passeridae

    Joined:
    Jun 16, 2019
    Posts:
    395
    More examples of Specular Occlusion in the Sponza Environment.

    Specular Occlusion disabled:
    upload_2022-8-25_1-47-3.png

    Specular Occlusion enabled:
    upload_2022-8-25_1-48-1.png

    Here's an extreme scenario where all reflection probes are disabled and sky-probe reflection is leaking everywhere. (Don't do this, it's just for the demonstration purposes ;))
    upload_2022-8-25_1-51-5.png

    upload_2022-8-25_1-53-39.png

    So, as you can see, Specular Occlusion will do its best to get rid off unwanted reflection leaking. It plays really nice with reflection probes (because they are never perfect) but is also applied to SSR. And of course you can control the intensity of this effect :)

    And a close-up. Here SO helps to "ground" reflective objects, creating this nice contact shadowing, that couldn't be baked otherwise. It also fights specular aliasing to a certain extent.


    P.S: deleted the last post to keep things clean and organized, recorded a gif and attached here instead.
     
    Last edited: Aug 25, 2022
    RyanJVR, Hubster, GuitarBro and 7 others like this.
  22. florianalexandru05

    florianalexandru05

    Joined:
    Mar 31, 2014
    Posts:
    1,771
    I'm loving this light so much, this looks better than everything I seen in Unity so far!!
     
    Blade_Master_2020 likes this.
  23. Passeridae

    Passeridae

    Joined:
    Jun 16, 2019
    Posts:
    395
    Hmm...haven't tested this yet. Generally, it's not recommended to use emissive materials as your only light source in the scene. Even offline renderers don't like this. So, I would be cautious with such an approach, but this is doable in theory. I haven't noticed anything wrong during my tests with this light ball (yet), but I don't know how the rest of Unity systems will handle it. From the top of my head, you will lose direct specular lighting. Maybe somethings else will break too, because the current rendering pipeline isn't meant to be used like this :)

    No, performance is not supposed to be affected, use as many as you want. However, I would recommend to keep your emissive objects relatively big (to keep noise under control) and avoid insane emissive exposure values.

    Thank you for the kind words! :)
     
  24. Passeridae

    Passeridae

    Joined:
    Jun 16, 2019
    Posts:
    395
    About the Fallback.
    As with any other GI in screen-space, there's always a question: "What will happen if you turn away from the directionally lit area?". If no fallback is used, then "black screen" is the answer. However, we do have a fallback and I hope this video can answer thе question:

    As you can see, I turn my camera away from the area lit by the directional light and even without a single pixel of direct lighting the indirect lighting is still there. And indirect soft shadows are also intact. That's how the fallback works.

    So, if you have reflection probes in your scene and you placed them decently enough, then the fallback will pick up the GI effect. The seamlessness of the transition will depend on the difference between the data in your probes and the data received during the primary color buffer tracing. As you can see in this particular Sponza example - the transition is almost seamless. I can't guarantee it for every scene, because I can't test all possible scenarios, but since Unity adopted this method as their official fallback, I guess, it's supposed to work in the majority of cases. I will also try to add AVP (Adaptive Probe Volume) support and Lightmap support as fallbacks as well.
     
    sacb0y, jiraphatK, Hubster and 7 others like this.
  25. Ookoohko

    Ookoohko

    Joined:
    Sep 29, 2009
    Posts:
    103
    Wow, this looks extremely good for SSGI! Congrats!

    How many reflection probes did you place in the scene? How much does the resolution of the reflection probes matter?

    I assume if you want semi-dynamic lighting, you could capture the probes on the fly if the lighting changes?

    br,
    Sami
     
    one_one likes this.
  26. Passeridae

    Passeridae

    Joined:
    Jun 16, 2019
    Posts:
    395
    Thank you! :)

    12 probes. They were placed by the author of the original scene and I haven't changed their placement or settings. Just disabled some probes that weren't really necessary in this case in my opinion. So it's good to see that the probes that weren't placed with GI fallback in mind work really well almost out of the box. Btw, GI performance isn't really affected by the probe count, so it's possible to add more of them.

    It doesn't, really. I tested this right now, and I don't see any visual or performance difference between 12 probes 128x128 and 12 probes 512x512 in this particular Sponza scene. Maybe there will be some visual difference in other scenarios, but the quality of probe placement (correct usage of shapes and proxy volumes) and their scene coverage will always matter more than their resolution.

    Sure. You can progressively update your probes using time-slicing to get fully dynamic fallback. We have a script to automate this process, but it's still experimental.
     
    Last edited: Aug 29, 2022
  27. TyGameHead

    TyGameHead

    Joined:
    Apr 29, 2017
    Posts:
    21
    is the video from urp because if it is great job
     
  28. Felinde

    Felinde

    Joined:
    Sep 19, 2021
    Posts:
    18
    Fallback looks magical and performing, I hope Unity release the reflection probe convex , also to maximize precision. Will objects with transparent material act with H-trace? Will it be possible to exclude objects via layers?
     
  29. Passeridae

    Passeridae

    Joined:
    Jun 16, 2019
    Posts:
    395
    The video is from HDRP version. But we'll try to do our best for URP as well ;)

    Thank you! The current plan is to add transparency support after the release. I've already done some initial tests, so transparency seems doable, but at the moment we're fully committed to finally releasing the asset with its current set of features :)

    Any object(s) can be excluded on the per-layer basis, yes.

    We really hope that we will be able to submit the asset in the next few days. At the current moment we are doing the final testing for different Unity & HDRP versions (and hell, it's very tiring).

    Btw, here's a new comparison made in Unity's HDRP sample scene:
    upload_2022-9-5_3-33-41.png
    upload_2022-9-5_3-34-18.png

    Unity's SSGI (with Reflection Probe fallback)
    upload_2022-9-5_3-34-33.png

    H-Trace (with Reflection Probe fallback + Specular Occlusion)
    upload_2022-9-5_3-34-47.png
     
  30. GuitarBro

    GuitarBro

    Joined:
    Oct 9, 2014
    Posts:
    179
    I'm assuming that for reflection probe fallback to work with a time of day system, the reflection probes would need to be updated independently to avoid strange lighting artifacts?
     
  31. Passeridae

    Passeridae

    Joined:
    Jun 16, 2019
    Posts:
    395
    Yep. Here's a quick test I made just now. I'm using a single real-time reflection probe (with time slicing) placed in the center of the room. The framerate with this setup is even higher than in the screenshots above.
     
    Last edited: Sep 5, 2022
    ontrigger, Hubster, Gooren and 12 others like this.
  32. Passeridae

    Passeridae

    Joined:
    Jun 16, 2019
    Posts:
    395
    Did one more test in the same scene. This time I tuned down the weight of the reflection probe to ~0.2 (the fallback impact is still 100%) so the stuttering in the reflections caused by time-slicing wouldn't be so obvious. And added Unity's SSR (with PBR accumulation) on top. SSR can pick up GI lighting, so they look quite nice together.


     
  33. Win3xploder

    Win3xploder

    Joined:
    Dec 5, 2014
    Posts:
    159
    Very impressive result!
     
    Passeridae likes this.
  34. Passeridae

    Passeridae

    Joined:
    Jun 16, 2019
    Posts:
    395
    Thanks to Marked who provided this beautiful HDRP scene for the testing purposes (and took part in the testing itself) I now have one more example to share with you:

    H-Trace (no fallback)
    upload_2022-9-12_22-58-18.png
    upload_2022-9-12_22-58-12.png

    H-Trace (with fallback)
    upload_2022-9-12_22-58-32.png
    upload_2022-9-12_22-58-27.png

    If it's hard to compare and detect differences between the last couple of screenshots, here's an interactive comparison and a couple of enlarged crops that better show what's going on with shadows, occlusion and small details:
    upload_2022-9-12_23-57-11.png

    I think, it's time to call the version 1.0 finished :). I will upload it to the Asset Store in a couple of days. In the meantime I will be writing a quick-start manual (the full documentation will come a bit later).

    Btw, if there is someone, who has a high-end interior archviz scene and wants to test H-Trace on it - feel free to DM me ;)
     
    Last edited: Sep 12, 2022
  35. Qleenie

    Qleenie

    Joined:
    Jan 27, 2019
    Posts:
    831
    This looks really good, much better in the shots compared to SSGI of HDRP!
    Have you tried it with VR? If it is integrated same as SSGI, it should work in principal, or?
     
  36. blueivy

    blueivy

    Joined:
    Mar 4, 2013
    Posts:
    621
    The occlusion really makes the world of a difference! Can't wait to throw my money at you.
     
    Passeridae likes this.
  37. jiraphatK

    jiraphatK

    Joined:
    Sep 29, 2018
    Posts:
    276
    Damn, great work! I'm buying this plug in as soon as it is available!
     
    Passeridae likes this.
  38. thwackstudio

    thwackstudio

    Joined:
    Mar 12, 2022
    Posts:
    7
    Awesome. I can't wait!.
     
    Passeridae likes this.
  39. Passeridae

    Passeridae

    Joined:
    Jun 16, 2019
    Posts:
    395
    It should work in theory, yes, but I haven't tested it in practice, so I can't guarantee. I'll take a look at it as soon as I have time ;)

    Sorry for keeping you waiting. I spent the last week designing graphics for the asset store page, writing the quick-start manual and doing more tests. And I also made it work on Metal and tested it on MacBook Air (M1). I really hope to submit the asset tomorrow or the day after :)
     
    Efif, Gooren, Felinde and 7 others like this.
  40. Passeridae

    Passeridae

    Joined:
    Jun 16, 2019
    Posts:
    395
    H-Trace is submitted to the Asset Store! :)

    Let's hope the submission process won't take too long. In the meantime, I will continue my active development. My immediate plans are:
    • update media on this thread
    • create a discord server
    • post a roadmap for my future development plans
    Stay tuned!
     
    Ziq-Zaq, Arycama, Marseol and 15 others like this.
  41. dicksucker2008

    dicksucker2008

    Joined:
    Sep 23, 2022
    Posts:
    1
    it's won't work with volumetric fog
     
  42. Blade_Master_2020

    Blade_Master_2020

    Joined:
    Aug 3, 2019
    Posts:
    38
    ?? How do you know it’s not compatible with Unity’s volumetric fog? It’s not available yet.
     
  43. jjojev

    jjojev

    Joined:
    Jan 6, 2021
    Posts:
    6
    Unity HDRP does not have an injection point before fog, note attention that all screenshots and videos are taken without the use of fog.
     
  44. jjojev

    jjojev

    Joined:
    Jan 6, 2021
    Posts:
    6
    I'm sure AO and GI will show through the fog, but i hope i'm wrong
     
  45. Marseol

    Marseol

    Joined:
    Jan 16, 2022
    Posts:
    10
    It's actually on page 2 "Here's H-Trace + Unity's SSR & Fog:"
     
  46. Blade_Master_2020

    Blade_Master_2020

    Joined:
    Aug 3, 2019
    Posts:
    38
    Yeah, You reminded me about this point, the asset "HBAO" has the same issue for several years, the AO just stack onto the fog, which makes it unuseful for HDRP project. I hope I'm wrong too.
     
  47. Passeridae

    Passeridae

    Joined:
    Jun 16, 2019
    Posts:
    395
    Okay, I see the confusion about the fog. Let me clarify this :)

    As I previously said, we have something that we internally call "integration modes". In practice it means that when you just installed the package you are in the "basic integration" mode which runs on the "Before Transparent" injection custom pass point. I haven't noticed anything wrong with the fog in this mode, but obviously I couldn't test every possible case, so maybe I'm missing something.

    But then we have a second mode, that we call "full integration". It works this way: there is a button called "Patch HDRP package" - you press this button and the automated patcher (that comes with the asset) changes certain files inside the HDRP package, injecting H-Trace code into them. This allows us to override HDRP buffers and execute our code at a different place inside the rendering pipeline. When the patching is finished, H-Trace GI will be recognized by Unity as its own GI, making it compatible with all rendering features: forward material support, visibility in SSR reflections and so on. So there will be not problems with fog as well.

    Here's a volumetric fog example with the basic integration mode:
    upload_2022-9-24_16-2-22.png

    Here's a volumetric fog example with the full integration mode:
    upload_2022-9-24_16-3-10.png

    I don't see any issues with fog in any mode here. But maybe this is not a representative case. Anyway, when you patch the HDRP package you won't face any issues with fog in any case.
     
    Last edited: Sep 24, 2022
    Ruchir, Gooren, florianBrn and 4 others like this.
  48. razzraziel

    razzraziel

    Joined:
    Sep 13, 2018
    Posts:
    380
  49. thwackstudio

    thwackstudio

    Joined:
    Mar 12, 2022
    Posts:
    7
    I cant wait. At least beautifull GI. Unity propably will review it in a month or so :/. Is there any hope for urp version?. Urp could be game changer.
     
  50. Blade_Master_2020

    Blade_Master_2020

    Joined:
    Aug 3, 2019
    Posts:
    38
    Yep, I can't wait too, our project almost abandoned and change to UE5 because procedurally generated world cannot bake light at all, without proper lighting, our game looks like a sh*t...From what I see,URP is not the game changer, H-trace is. There will be URP version,so be patient. The author is very active here,he will answer you soon.

    We should support him, the real game changer.
     
    FlightFight likes this.