Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Graphics HXGI Realtime Dynamic GI

Discussion in 'Tools In Progress' started by Lexie, May 24, 2017.

  1. Lexie

    Lexie

    Joined:
    Dec 7, 2012
    Posts:
    646
    I would say that older IGP are probably a little too slow for this, current gen IGP like the new iris pros should be fine though. That bunny scene runs at 40fps at 720p on my Intel HD4600 using a half res GI volume. there are more settings I can lower like removing the specular approximation or lowering the GI sample count, but at the end of the day that IGP is really not meant for something like this. Like that scene with my stuff turned off and 1 unity shadow casting point light runs at 40-45 fps... that bunny is pretty high polly.

    When I've finished everything and have all the settings exposed it will be a little easier to figure out what the cutoff is. till then its more of a guess.
     
  2. Lexie

    Lexie

    Joined:
    Dec 7, 2012
    Posts:
    646
    The AO in the corners of the wall is a side effect of how the system samples the GI. I have a couple of ideas on how to remove it with out introducing light bleeding but it will cost a little extra to render.
     
    Shinyclef, macdude2 and DMeville like this.
  3. Howard-Day

    Howard-Day

    Joined:
    Oct 25, 2013
    Posts:
    137
    ..As long as it's optional - I quite like that AO. :D
     
    MarkusGod, SteveB and buttmatrix like this.
  4. neoshaman

    neoshaman

    Joined:
    Feb 11, 2011
    Posts:
    6,493
    Wow that looks like really performant lol, didn't expect it to go that low, I understand why baking is less an issue now lol
     
    chiapet1021 likes this.
  5. buttmatrix

    buttmatrix

    Joined:
    Mar 23, 2015
    Posts:
    609
    moar eye candy plz
     
  6. zelmund

    zelmund

    Joined:
    Mar 2, 2012
    Posts:
    437
    enjoed by pictures =)
    keep it up, man =)
     
  7. Frpmta

    Frpmta

    Joined:
    Nov 30, 2013
    Posts:
    479
    Last edited: Aug 9, 2017
  8. Lexie

    Lexie

    Joined:
    Dec 7, 2012
    Posts:
    646
    My method of GI is pretty different for defuse lighting compared to segi. it doesn't do an tracing for defuse lighting at all so the lighting is stable and doesn't require any noise to accumulate the lighting over many frames for each pixel. In segi when you move the camera or a dynamic object moves you see all this ghosting. that's because those new pixels haven't had enough time to gather enough lighting information. My method has its own artifacts as well but only really show up when you move lights or light blockers around.

    I do use noise for the stochastic voxel traced specular though,This allows the reflections to support different roughness values and makes distant reflections go out of focus like they do in the real world. The noise is just a texture though, its very easy to switch out what type of noise the calculation uses. I use a blue noise texture as it works well with TAA

     
  9. Frpmta

    Frpmta

    Joined:
    Nov 30, 2013
    Posts:
    479
    I thought the blue noise was to give that 'CGI raytraced' look rather than making up for artifacts.

    Shows how clueless I am :D
     
    buttmatrix likes this.
  10. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    12,329
    Blue noise is useful because it is difficult to see repeating patterns in it.

    The below images are repeating 64x64 noise textures:
    (Images stolen from this page: http://momentsingraphics.de/?p=127)


    blue noise


    white noise​
     
    Last edited: Aug 9, 2017
  11. Lexie

    Lexie

    Joined:
    Dec 7, 2012
    Posts:
    646
    Also the clumping of the noise helps remove artifacts as near by pixels tender to sample a similar direction for that frame.
     
    Shinyclef likes this.
  12. magique

    magique

    Joined:
    May 2, 2014
    Posts:
    4,030
    I'd be interested in seeing how that exact same scene performs with standard Unity GI. That is the sort of comparison I think that gives us a true idea of just what we would be getting with this asset (at least at this stage).
     
    buttmatrix and SteveB like this.
  13. buttmatrix

    buttmatrix

    Joined:
    Mar 23, 2015
    Posts:
    609
    I assume you are talking about the 'femto-photography' sort of artifacts / banding, or no?
     
  14. Lexie

    Lexie

    Joined:
    Dec 7, 2012
    Posts:
    646
    I figured out a simple method for removing some of the dark AO that happens when sampling the GI near multiple surfaces. Left image is with this enabled. It can have the effect of removing some of the self AO but looks more believable. Its a float value so you can set it to anywhere in between these two images.




    Yeah the femto-photography artifacts, that's what happens when GI contribution lights and objects move. you can speed up the lighting at the cost of computation though, depends on your needs.

    Ill do some comparisons once its done. Really don't want to lock up my computer while it bakes out unity's GI
     
    DasBiot, hopeful, Reanimate_L and 7 others like this.
  15. Tzan

    Tzan

    Joined:
    Apr 5, 2009
    Posts:
    733
    Yeah I was worried about that going to black in the corners effect.
    Looks much better now.

    Do you expect that your with your GI there would be no need for an AO effect to be added?
     
  16. Lexie

    Lexie

    Joined:
    Dec 7, 2012
    Posts:
    646
    I think it might be useful to do an AO pass so dynamic objects that don't contribute to GI can look like they are effecting the scene and calculating some self AO. Also SSAO will capture small details that my GI will miss.
     
    Shinyclef likes this.
  17. Tzan

    Tzan

    Joined:
    Apr 5, 2009
    Posts:
    733
    I ask because that left image looks maxed out on the AO effect in the corners.
     
  18. tweedie

    tweedie

    Joined:
    Apr 24, 2013
    Posts:
    311
    Looks considerably better, but I think it's still a touch strong; evident nearest the green cube, where I'd expect almost no AO given how close it is to the wall. What exactly causes this effect?
    *chuckles*
     
  19. Lexie

    Lexie

    Joined:
    Dec 7, 2012
    Posts:
    646
    Yeah there shouldnt really be any AO in that corner. The hard thing is sampling the GI with out causing light leaking. I can change the way the lighting is calculated to remove all that AO but it introduced some light bleeding. Id rather have AO bleeding then light bleeding. When you put textures on everything and some color correction you really dont notice any of this.
     
  20. tweedie

    tweedie

    Joined:
    Apr 24, 2013
    Posts:
    311
    Ah, I see, thanks for the explanation. Would definitely agree with that preference :)
     
    Shinyclef likes this.
  21. Lexie

    Lexie

    Joined:
    Dec 7, 2012
    Posts:
    646
    Here is a textured scene so you can see that the AO really isn't that noticeable
     
    punk, DasBiot, Hitch42 and 7 others like this.
  22. tweedie

    tweedie

    Joined:
    Apr 24, 2013
    Posts:
    311
    Basically you're fine so long as you're not doing a white-walled Interior Arch Vis demo :) (Thankfully I don't think many are).

    Certainly looks good textured, appreciate the follow up post!
     
    Shinyclef likes this.
  23. macdude2

    macdude2

    Joined:
    Sep 22, 2010
    Posts:
    686
    Looks much better, very nice! Still looks a little off (as you wouldn't expect there to be any AO in direct light), but honestly it's good enough now and I'd definitely prefer this over light leakage anyhow.
     
  24. neoshaman

    neoshaman

    Joined:
    Feb 11, 2011
    Posts:
    6,493
  25. Zuntatos

    Zuntatos

    Joined:
    Nov 18, 2012
    Posts:
    612
    My guess is that it would be viable to program in clouds/fog for the lighting propagation, but the problem would be sampling the lighting that has no geometry at that position. It's either raytracing through it when rendering behind the fog/clouds, or lots of particles. It would work for lighting smoke etc I guess. (Disclaimer; not my asset)
     
    neoshaman likes this.
  26. Lexie

    Lexie

    Joined:
    Dec 7, 2012
    Posts:
    646
    Not for free, you still need to ray march all the data. Ill be adding support for my volumetric plug-in once the main features are done.
     
    Last edited: Aug 10, 2017
    punk, yohami, one_one and 6 others like this.
  27. yohami

    yohami

    Joined:
    Apr 19, 2009
    Posts:
    124
    I've got my finger on the buy button, firm and steady.
     
    TooManySugar and PhilippG like this.
  28. PhilippG

    PhilippG

    Joined:
    Jan 7, 2014
    Posts:
    257
    Buy button? WHERE? :eek:
     
  29. yohami

    yohami

    Joined:
    Apr 19, 2009
    Posts:
    124
    Exactly!
     
    PhilippG likes this.
  30. Yasunokun

    Yasunokun

    Joined:
    Feb 20, 2015
    Posts:
    8
    I am really late to this party, but I am really looking forward to this as well! I am willing to spend $100 on it. :D SEGI is nice, but mostly abandoned at the moment due to RL issues he seems to be facing. (Which is understandable!) Thank you for all the work you have been doing on this, it looks great!
     
  31. Shinyclef

    Shinyclef

    Joined:
    Nov 20, 2013
    Posts:
    502
    A good solution shouldn't be rushed. But yeah, I empathise :)
     
    yohami likes this.
  32. zenGarden

    zenGarden

    Joined:
    Mar 30, 2013
    Posts:
    4,538
    @Lexie
    What is performance on some scene like Unity vicking village ?
    Will it work only on static scene objects ?
     
  33. Lexie

    Lexie

    Joined:
    Dec 7, 2012
    Posts:
    646

    (realtime voxel traced specular reflections. Left: on Right: off)

    I've added realtime stochastic voxel traced specular reflection support, its necessary to make metal and shiny surfaces look real. The bunny and sphere have the properties of a mirror. It's physically based so rough surfaces will have more blurred/ elongated specular reflections. This can be a little expensive to calculate and is meant for modern GPU's. lower end GPU's will be better off using the specular approximation mode to keep the cost of rendering down. its not ideal but its better then having no specular.


    The reflections still pull screen space data where possible. there are artifacts so it isn't recommended to make completely mirror surfaces that are flat like this.


    Here is a shiny scene to see it in action. Contact reflections are failing a little bit so i need to work on that. also the effect can be a little noisy. TAA helps a bit but it would be better if i made a solution just for the specular reflections. I might mipmap the results for far away reflections as well.

    Because of the way the data is structured in this new version the reflections are more blob-like then hard voxels. this means the colors will blend together making it harder to recognize the voxel world in reflections.

    I have a lot of trips coming up so work will slow down a bit. i still need to optimize the reflections to make them faster and leverage the cascades to skip large empty areas while its tracing through the GI data.


    There is no concept of static object.

    GI contributing objects will emit/block/bounce light and will sample the GI per pixel.
    Non-GI contributing objects will not emit/block/bounce light but will sample the GI per pixel.

    Both types are free to move but moving GI contributing objects will cause some artifacts while the new lighting is calculated.

    I haven't tested on the viking village. Ideally you would use LOD meshs for the voxelization step rather then voxelizing the high poly buildings. but it should run that scene fine. But at the end of the day it depends on your settings.
     
    Last edited: Aug 19, 2017
    punk, tapawafo, SteveB and 14 others like this.
  34. nipoco

    nipoco

    Joined:
    Sep 1, 2011
    Posts:
    2,008
    This is coming along really nice.
     
  35. Shinyclef

    Shinyclef

    Joined:
    Nov 20, 2013
    Posts:
    502
    Looks fantastic! Even the off-screen approximated reflections are impressive.
    Do the reflections have similar light propagation limitations that we've seen so far in terms of the 'speed of light' being very noticeable for moving objects?

    Are you talking about a couple of weeks, or more like... a few months?

    Keep up the amazing work (when you're not on your trips)!
     
  36. Lexie

    Lexie

    Joined:
    Dec 7, 2012
    Posts:
    646
    no/yes? the reflections are ray traced every frame in realtime so they are instantaneous. but the result for the lighting of the reflected point needs to sample the GI so that part has the speed of light limitation. Ill make some videos once its done.
     
    punk, Shinyclef and Yasunokun like this.
  37. yohami

    yohami

    Joined:
    Apr 19, 2009
    Posts:
    124
    Fantastic.
     
    Shinyclef likes this.
  38. neoshaman

    neoshaman

    Joined:
    Feb 11, 2011
    Posts:
    6,493
    What's left now to do?
     
  39. Tzan

    Tzan

    Joined:
    Apr 5, 2009
    Posts:
    733
    The remaining 10% ( which takes 90% of the time )
     
    wetcircuit, Martin_H, hopeful and 2 others like this.
  40. benfattino

    benfattino

    Joined:
    Nov 26, 2010
    Posts:
    43
    Hi. I am very interested to realtime GI.
    Any chance to try this magic?
     
  41. Reanimate_L

    Reanimate_L

    Joined:
    Oct 10, 2009
    Posts:
    2,788
    Release it i hope :D
     
    neoshaman likes this.
  42. Lexie

    Lexie

    Joined:
    Dec 7, 2012
    Posts:
    646
    The reflections need some temporal solution to handle rough surfaces. it generally looks fine on perfectly smooth surface but if a surface is rough it needs more samples to be able to calculate a good result. most screen space reflection solutions just treat the surface as completely smooth but I need the reflections to act as accurate-ish specular. Also I need to optimize the reflections to be faster.

    Some pipeline needs to be made to enable forward shading support.

    Some general clean up and editor scripts to make it presentable.

    There are also a couple of bugs that I've been putting off fixing that need some work as well.
     
  43. MelvinEng

    MelvinEng

    Joined:
    Jan 11, 2016
    Posts:
    14
    Hi Lexie,

    Uhh..what about support for translucent surfaces that allow light passing through to be filtered/colored?
    Also, suppose we have light from an emissive surface passing through a colored glassy object, will we see caustic patterns on objects behind the glassy object?

    Cheers!
     
    Last edited: Aug 23, 2017
    ekergraphics likes this.
  44. Lexie

    Lexie

    Joined:
    Dec 7, 2012
    Posts:
    646
    Lighting from emissive surfaces and bounce lighting from unity's lights will change color as they pass through translucent surfaces. Direct lighting from unity's lights will not as unity does not support that.

    The GI is too low detail to calculate any caustics. The point of this GI is fast lighting for video games, not accurate light simulation.
     
    Last edited: Aug 23, 2017
  45. MariskaA

    MariskaA

    Joined:
    Feb 17, 2017
    Posts:
    11
    Hello!
    I've got a little request if possible, could you make 2 rendering, with both of those method to see the difference between stochastic voxel traced specular reflection and specular approximation please?

    Also, what do you think "modern GPU's is? (like, is a 2 years old GPU is modern to you?)

    Thank you so much!
     
    MarkusGod likes this.
  46. DanMeyer009

    DanMeyer009

    Joined:
    Sep 22, 2013
    Posts:
    23
    Well, It's official. Your product is now the last hope of a fast real time GI solution. SEGI dev just said he's at an impasse with his product and considers it to be the best state he can make it. Don't get me wrong, SEGI looks good for what it is but I'm more impressed with the performance you have managed to get. Here is to hoping one day you find it in your heart to let the rest of the world use your amazing piece of tech. Keep up the excellent work, Lexie.
     
    Mauri, tapawafo, S_Darkwell and 2 others like this.
  47. TooManySugar

    TooManySugar

    Joined:
    Aug 2, 2015
    Posts:
    864
    Don´t get too hyped, different approach may find a big rock at another point of the trail. GI has proven to kill all teh attemps to date ans yes this two attemps are the closest and most promising ever, but... Segi is still alive, I mean is not abandoned, so better keep two projects living that put all the hopes in a single boat. The dev from this asset even showed on Segi fórum to drop some ideas about performance (without luck for Segi dev). When open sources, help may come from some point and when this Project stalls (hopefully not) help may again come from elsewhere. Keep in mind that HXGI dev already stated that this Projects are not like so direct rivals as each one performs better on different scenarios.
     
    wetcircuit, Lexie and Shinyclef like this.
  48. Lexie

    Lexie

    Joined:
    Dec 7, 2012
    Posts:
    646
    The specular stuff really isn't finished yet but here is a comparison. Both Methods really need work though.

    (Left: Physically based specular traced per pixel. Right: Specular Approximation)

    Left is more accurate results but is more expensive to render and I still need to add some temporal filtering to remove the grain. Right is cheaper to calculate but doesn't really do a good job at representing the full range of materials or capturing the emissive surfaces but on the plus side it wont need temporal sampling.

    As for modern GPU I mean anything around a 660 GTX and above. Although the specular support will probably need something in the 9 series.

    I'm working on a side project right now so development will slow down for a couple of weeks. I cant really talk about the project but it does all the gameplay on the GPU using compute shaders (needs to simulate 260K+ entities). It has forced me to learn how to optimize compute shaders so I should be able to apply that knowledge to the light propagation shader and make that step run a lot faster. That means increasing the speed of light wont be as much of a performance hit. But ill have to wait and see.

    One update I haven't posted about is that I've improved the screen space reflections a bit. Before it would do the voxel tracing and then if the final location was on the screen it would pull data from it. The downside of this is non voxelized objects wouldn't create reflections unless they where very close to the final position, and even then they would become skewed.

    So instead I've made it do normal screen space reflections and then switch to marching the voxel grid when screen space reflections fail. This means I'm able to capture non GI contributing objects in the specular contribution!
    It also helps improve the contact reflections as well. Currently they don't pull from the emissive buffer yet and the formulate for calculating failed rays needs a lot of improving, but it gives some nice results. I should also be able to make it calculate reflections of reflections in screen space as well.


    (Adam model still contributes to specular reflections even though it doesn't get voxelized by the GI system)
     
    Last edited: Sep 3, 2017
    ftejada, sdutter, DasBiot and 12 others like this.
  49. Shinyclef

    Shinyclef

    Joined:
    Nov 20, 2013
    Posts:
    502
    In the screenshot with the apartment with the reflective floor, the reflections weren't alligned. Do these changes to screen space reflection have an impact on those artefacts you mentioned previously?

    Thanks for the update. Following your compute shader work on Twitter also, looks very nice.
     
  50. Lexie

    Lexie

    Joined:
    Dec 7, 2012
    Posts:
    646
    Oh that wasn't a floor that was a rotated floating block that wasn't aligned to an axis. so the reflections should have looked like that.

    The thing I'm talking about with things getting skewed from voxel tracing and then mapping back to screen space is visable in this old gif. See the reflection of the pot on the left, it looks like a LOD blocky thing with the texture mapped to the voxels when it pulls data from the screen. That's the skewing that doing screen space then voxel traced reflection solves.
     
    Shinyclef likes this.