Search Unity

  1. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice

Fully Dynamic Diffuse Global Illumination

Discussion in 'Works In Progress' started by sonicether, Mar 29, 2015.

Thread Status:
Not open for further replies.
  1. sonicether

    sonicether

    Joined:
    Jan 12, 2013
    Posts:
    265
    Hello, everyone. I'd like to start a thread about my unique solution for dynamic Global Illumination. I showed this technique off recently at GDC, and it was quite well-received, so I figured I'd share some information about it here.

    Unity 5's integration of Enlighten offers incredible image quality at an amazingly low run-time cost, but because it relies on precomputation, many developers are unable to leverage it in their games, especially if their scenes are being procedurally generated at run-time. If you're working on a procedurally generated game, or a game that otherwise is very dynamic, I believe that my algorithm may be a fantastic solution for you.

    I don't quite have a name for the technique yet, but it's basically a variation of Reflective Shadow Maps. I originally developed the algorithm back when I was working on lighting for Minecraft. Due to the dynamic nature of Minecraft, I needed a fully dynamic solution that didn't rely on precomputation.

    Here's the current simplistic state of the algorithm. The scene is rendered from the view of the light source at hand and information about each surface receiving illumination is stored. That information is then accessed in a post-process effect to generate the final bounced light and then blended using the albedo buffer (thank you Unity 5 for deferred rendering!).

    Here's a few screenshots of the effect.

    crytek sponza 1.jpg

    crytek sponza 2.jpg

    sponza 1.jpg

    2015-03-29_11_02.jpg


    As you can see, the technique yields very spatially-coherent results with quite good close-proximity detail. The cool thing is that it's calculated on-the-fly, so any material, lighting, or geometry changes are reflected immediately in the indirect lighting. Another strength of this algorithm is that it works very well for large outdoor scenes, because of its spacial coherency and retention of detail. Here are some comparisons showing the effect on and off in The Viking Village.

    village comparisons.jpg


    So, how's it perform? At the settings in these screenshots, this effect is running at around 6 ms on a GTX 980 at 1080p. It is very scalable, however, and can cost as little as 2.5 ms and as much as 10 ms depending on quality settings.

    Some of the drawbacks are:

    -Limited to a single bounce and a "simulated" second bounce (works well enough)
    -Currently only works for directional lights. I'm looking to find a solution for other light types
    -Only surfaces lit directly by sunlight cast indirect light
    -Does not (and cannot in its current state) handle indirect occlusion/shadows

    As quality settings increase, the result approaches each surface giving off a single perfectly accurate bounce of light without visibility/shadowing. Lowering quality consists of reducing radius of indirect light, reducing resolution of buffers for coarser approximation of surfaces, reducing sampling rate at the risk of low-frequency noise, and decreasing draw distance of the effect.


    Because I am also working on updating my SE Natural Bloom & Dirty Lens asset and working on finishing my SSAO asset, development may be slowed. I'm aiming to release this as an asset in the near future. Because of GDC, most of the preliminary work is done. I have an idea for a more lightweight approach, but that may have to wait until the initial version is released.

    Please, let me know what you think! If you have any questions, feel free to ask them. :)
     
    Last edited: Mar 29, 2015
    SeanHoward3D, Farelle, ksam2 and 28 others like this.
  2. jcarpay

    jcarpay

    Joined:
    Aug 15, 2008
    Posts:
    534
    Wow, very impressive!!
     
  3. nasos_333

    nasos_333

    Joined:
    Feb 13, 2013
    Posts:
    8,430
    Amazing :)

    Do you have to change materials to get the result ? Or works in any scene without any preparation ?
     
  4. sonicether

    sonicether

    Joined:
    Jan 12, 2013
    Posts:
    265
    It'll work with any shaders in any scene provided that you're using deferred rendering.
     
  5. protopop

    protopop

    Joined:
    May 19, 2009
    Posts:
    1,375
    That's so beautiful. Im hooked on outdoor scenes and still trying to understand what Global Illumination exactly is and how it works in these scenes. Looking at the examples brings me a bit closer to understanding it:)
     
  6. SAOTA

    SAOTA

    Joined:
    Feb 9, 2015
    Posts:
    220
    This is great.
     
  7. nasos_333

    nasos_333

    Joined:
    Feb 13, 2013
    Posts:
    8,430
    Great, that is a big perk. Is there a release date window ?
     
    RB_lashman likes this.
  8. RB_lashman

    RB_lashman

    Joined:
    Aug 15, 2013
    Posts:
    85
    well he did say "near future", so .... probably in something like 2-3 weeks? maybe? :)
     
  9. Deleted User

    Deleted User

    Guest

    Hmm, with it working from a DL and looking at the top pictures it reminds me of CE's LPV implementation. Which of course isn't a bad thing, as long as performance is good it looks like a great solution.
     
  10. thoorne

    thoorne

    Joined:
    Jul 22, 2012
    Posts:
    64
    Can't wait. Thread subscribed.
    Price?
    ETA?
     
  11. Luckymouse

    Luckymouse

    Joined:
    Jan 31, 2010
    Posts:
    481
    Didn't see any dx11 requirement, so it will be mac compatible, right?
     
  12. RB_lashman

    RB_lashman

    Joined:
    Aug 15, 2013
    Posts:
    85
    I don't think it needs DX11 to work .... just deferred rendering
     
  13. Pulov

    Pulov

    Joined:
    Feb 20, 2010
    Posts:
    824
    Looks aAaaaAwesome. I've just finished baking a super complex scene in unity 4 and its 32 bit limitation have almost made me kill myself with so much crashing. Tried with unity 5 trial and precomputation took forever so switched back to U4. Finally I baked with just 2 hours, very little time for the glory I got afterward and did not crash becaus had to do ridiculous tricks to keep ram comsumption low, but still this assets delivers same ressults as baking with no lenghtly baking so I would jump in with no doubts.
     
    RB_lashman likes this.
  14. Zomby138

    Zomby138

    Joined:
    Nov 3, 2009
    Posts:
    652
    Looks fantasmic :)
     
  15. Pulov

    Pulov

    Joined:
    Feb 20, 2010
    Posts:
    824
    I would like to know if this will support Unity 4 as it also has deferred lighting. As it computes fast, could it be used used to bake high quality lightmaps?

    Please paste more screenshots because this is looking freaking amazing.

    For that SSAO you mention please add depth cutoff.

    I've just finished a very complex scene of Tokyo Shibuya lit with Beast, find it in the WIP thread, if you license me a trial I could compare the results. I'll update screens later today as these are quick grabs from editor as I needed a rest.
     
    Last edited: Mar 31, 2015
  16. sonicether

    sonicether

    Joined:
    Jan 12, 2013
    Posts:
    265
    Yep, no dx11 requirement for now.

    Hmm.. Using this algorithm to do gpu-accelerated super fast baking of lightmaps is an interesting idea. I may think about this more.

    Unity 4 has deferred lighting. Unity 5 has deferred rendering. There's a big difference there, enough to where implementing something like this as a post effect in Unity 5 is trivial, and doing so in Unity 4 would be much trickier. It would at least involve custom shaders and would have an overhead cost.

    It's important to note a current limitation of this algorithm; only surfaces illuminated by a directional light will give off indirect light. That means that for now, spot and point lights are not supported, and neither are emissive surfaces. It's real-time GI, you gotta cut corners somewhere.
     
    Pulov and RB_lashman like this.
  17. Pulov

    Pulov

    Joined:
    Feb 20, 2010
    Posts:
    824
    Lol, I did not notice the deferred Rendering instead of Lighting. Sorry.

    You could consider implementing spots and emisive surfaces for the lightmap version, if the ligthmap feature is ever implemented. This would be like a blazing fast Beast for unity 5.
     
  18. Issam

    Issam

    Joined:
    Dec 18, 2012
    Posts:
    37
    Can I ask what buffer information from the deferred rendering you are using? I am curious if you need a lot of the info used or if when doing forward rendering we can do fast shader past to generate the buffer that you need? (or just write it to a another render target during the forward pass)
     
  19. sonicether

    sonicether

    Joined:
    Jan 12, 2013
    Posts:
    265
    Just albedo. Yeah, it would be possible to get albedo by rendering the scene once more with a replacement shader (though, a terrible option to render the scene again just to get that information, in my opinion), or by using custom shaders that write to an additional render target. I'm not sure if it was a bug or limitation, but I recall MRT not being possible with Deferred Lighting, only Forward (in Unity 4).
     
    Issam and RB_lashman like this.
  20. Issam

    Issam

    Joined:
    Dec 18, 2012
    Posts:
    37
    Cool, we are doing forward rendering for our game and using MRT would be doable (I need to test if it's still working in Unity 5 ;)). Our game is all about moving color around (http://www.troubleimpact.com/games/2013/2/25/color-thief.html) and realtime GI would be really useful, but the current Unity 5 precomputation step is just too slow to be useful for us. We only have one big directional light (the sun), so this sounds really good. Let me know if you need people to test it before release.
     
  21. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    26,727
    It would be cool if you had a ubershader version, ie combining ssao, gi and bloom. This would be available to download if people have purchased them all separately and quite probably lead to really good performance?

    Or better still a combined Sonic post that's generally all your post in one for a competitive price (for performance reasons, I'm most interested in combining post wherever possible).
     
    ZJP, Arkade and RB_lashman like this.
  22. chingwa

    chingwa

    Joined:
    Dec 4, 2009
    Posts:
    3,516
    Ooooooh pretty.... :)
     
    RB_lashman likes this.
  23. smetzzz

    smetzzz

    Joined:
    Mar 24, 2014
    Posts:
    145
    I am very interested in this.
     
    RB_lashman likes this.
  24. Pulov

    Pulov

    Joined:
    Feb 20, 2010
    Posts:
    824
    Now that you have the ssao almost finished hope you can focus into this.
     
    RB_lashman likes this.
  25. Arkade

    Arkade

    Joined:
    Oct 11, 2012
    Posts:
    619
    Just watched your video -- v. interested since (a) I'm integrating voxels atm and (b) the snow aesthetic in the video is very similar to what I'd like to achieve in my snow set game! Any chance for a write up of what's in there? (All your effects?) How did you prevent washing everything out while having Bloom in a predominantly snow white scene?

    Thanks!

    P.s. I'd echo @hippocoder comment re combined viability.

    low-poly environment:
     
    RB_lashman likes this.
  26. Arkade

    Arkade

    Joined:
    Oct 11, 2012
    Posts:
    619
    P.p.s. what sort of requirements do you think a full set would have? (PC / Mac / Lin) Viable for current gen consoles? (Xbone, PS4, ... WiiU?!)
     
    RB_lashman likes this.
  27. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    26,727
    It's all about how many setpasses and ms it takes up. Setpass is generally evil I've noticed.
     
    RB_lashman likes this.
  28. sonicether

    sonicether

    Joined:
    Jan 12, 2013
    Posts:
    265
    Sure. Basically, I've just got my GI, my SSAO, and my bloom going on in that scene with a bit of s-curve tonemapping. The s-curve really helped keeping things crisp and preventing a "washed out" look. It's nothing major, but I've considered working on a more comprehensive tonemapping plugin that could have an adjustable s-curve, contrast, and white and black clipping. It wouldn't be much, but having those few extra features has helped me and I think it could help others as well.

    All I can do is guess about the viability on other platforms as of now; I have no way of testing on them myself as I don't own them. I'd like to invest in those platforms for testing, but that'll have to wait. For now, I can search for people who have those platforms that can test things for me.
     
    Arkade, smetzzz and RB_lashman like this.
  29. smetzzz

    smetzzz

    Joined:
    Mar 24, 2014
    Posts:
    145
    I would be interested in this working on mobile. I would be fine if it worked non-dynamically in a procedural world. Where you could hit a button and render the bounces and SSAO one frame for camera only animation. Could you add that into the plugin? Keep up the great work.
     
    RB_lashman likes this.
  30. Mauri

    Mauri

    Joined:
    Dec 9, 2010
    Posts:
    1,749
    I'm sure you're busy, but is there any chance of getting an reply from you regarding my e-mail / PN? ^^

    Edit: No response, whatsoever. And I though Livenda's way of communication would be already the worst. Wow. Good job.
     
    Last edited: May 4, 2015
  31. I am da bawss

    I am da bawss

    Joined:
    Jun 2, 2011
    Posts:
    2,574
    Very interested. Definitely on my shopping list.
     
  32. IronDuke

    IronDuke

    Joined:
    May 13, 2014
    Posts:
    132
    Agreed. Baked or pre-computed lighting is useless in a procedural game, which gets worse when it's a space simulator, with nothing static. Ever.

    --IronDuke
     
    RB_lashman likes this.
  33. Pulov

    Pulov

    Joined:
    Feb 20, 2010
    Posts:
    824
    Hey. A bit of updating wouldn't hurt;)
     
  34. I am da bawss

    I am da bawss

    Joined:
    Jun 2, 2011
    Posts:
    2,574
    One thing, just to clarify, which one is the one with the effect on? They are all labelled "Unity". I think you should label your own solution with your own logo just for clarification. I am guessing the one that's "brighter" is your solution.
     
  35. smetzzz

    smetzzz

    Joined:
    Mar 24, 2014
    Posts:
    145
    Sonicether, I would much prefer your solution over your competitors. Its evident you have an artistic eye for design and color. In my experience this always translates to cleaner code and better user experience. I hope you are not discouraged by their push to make a similiar product. All the best.
     
    RB_lashman likes this.
  36. nasos_333

    nasos_333

    Joined:
    Feb 13, 2013
    Posts:
    8,430
    Are these assets released or presented in WIP forum ? Would be great to have a bigger picture on all available choices, though i must say the image effect approach and non invasive to shader method is my preference as well, by far.

    Is there a release window for this asset ?
     
  37. SteveB

    SteveB

    Joined:
    Jan 17, 2009
    Posts:
    1,448
    You know whats funny Nasos? You and are active in GI threads...the ONLY threads that never seem to catch any real momentum ever. The pattern is always the same. Enthusiastic OP, a bunch of questions and a bunch of replies then silence. I understand GI is a remarkable beast and terribly difficult, it's just still humorous. Either the authors are overzealous or we are. I'm perhaps in the minority but the GI in Unity 5 still seems inflexible and preprocess intensive and I find myself eager and equally disappointed often with threads like this when faced with the fact that even a solution that has been specialized falls short.

    All this said, I'd love for one of these to materialize, hehe so clearly I'm glass half full eh? :D

    Cheers
     
    Pulov likes this.
  38. nasos_333

    nasos_333

    Joined:
    Feb 13, 2013
    Posts:
    8,430
    Well, one can only hope :). I suppose integrating something that affects the whole game in such a big way is far from easy and making it fast enough to be usuable makes it even harder.

    Enlighten is probably a good solution for the final game, if somehow it could be prototyped and tested faster would be 90% there. I cut a 10% since you need static items to grab real time color from bounces, this would be helped more by a real time solution.
     
  39. SteveB

    SteveB

    Joined:
    Jan 17, 2009
    Posts:
    1,448
    Thats my issue with Enlighten right now...everything is static and precomputed. It's indeed quite wonderful when it works/doesn't hang, but the inability to change your environment makes it restrictive.

    I'm excited for Distance Field GI in Unreal 4, coming in 4.8, which seems to be a reasonable realtime solution. Otherwise we don't have anything of the sort in Unity, and not in the near future...
     
  40. smetzzz

    smetzzz

    Joined:
    Mar 24, 2014
    Posts:
    145
    Yeah, the pre-computation is a killer. As far as I know the technique above uses some type of HLGL deferred lighting computation. (shooting in the dark here) and seems like a great alternative that really should be added into the engine. I am specifically interested in procedural world lighting that Unitys current GI solution cant process due to pre-computation in the editor.
     
    kristisi and RB_lashman like this.
  41. SteveB

    SteveB

    Joined:
    Jan 17, 2009
    Posts:
    1,448
    So speaking of which then, why hasn't there been continued conversation by Sonicether? He's clearly on the boards and we're clearly discussing this in his thread. This is that bizarre thing that seems to only occur with GI threads, and iirc, it has been every GI thread that ever was created. Baffling...

    Cheers guys
     
    Pulov and Zeblote like this.
  42. zenGarden

    zenGarden

    Joined:
    Mar 30, 2013
    Posts:
    4,538
    I think real time GI just fails and don't provide good performance as it is heavy computation.
     
  43. SteveB

    SteveB

    Joined:
    Jan 17, 2009
    Posts:
    1,448
    Oh believe you me I appreciate that and do not doubt it in the least. Something however is compelling these authors to talk about this, and if it were completely unworkable they wouldn't bother. Call me glass half full, but I'm giving them the benefit of the doubt here...

    ...then again as many people have pointed out, if Epic can't do it, and it took a company dedicated to realtime GI to create what we currently have in Unity 5, how can we hope that 'some guy' is going to match that output. I get it. :)
     
  44. smetzzz

    smetzzz

    Joined:
    Mar 24, 2014
    Posts:
    145
    This does have some element of "too good to be true" when I first read it. But I know sonicether is a talented and smart guy and he isn't the first to claim non-precomputed GI.
     
  45. nasos_333

    nasos_333

    Joined:
    Feb 13, 2013
    Posts:
    8,430
    I have some doubts that real time GI would come close to what Enlighten does after the baking. If so this solution should be bought by Unity and used instead.

    The problem is that resolution of the solution cant be as high as the baked one, there will also probably be some form of artifacts, either in motion or something else, as having it completly smooth would seem like a miracle. I wish it happens, but i dont hold my breath either.

    After what i have seen, i am prepared that i will probably have to use a mix of all available systems of GI to get the effects i want for my various game areas, so more options will add up for my case and are always more than welcome.

    If all goes well, i will have 4 GI systems in my game :), if not just Enlighten and GI Proxy, which is fine as well, but more restrictive.

    Also i do believe that a single developer can do better than a company, it is all about focus. Maybe not in the polish or content areas, but in the tech area certainly is possible.
     
    Last edited: May 6, 2015
    JamesArndt likes this.
  46. Pulov

    Pulov

    Joined:
    Feb 20, 2010
    Posts:
    824
    If this goes the same path as livenda's vaporware its gonna kill me.
     
  47. chingwa

    chingwa

    Joined:
    Dec 4, 2009
    Posts:
    3,516
    Well, as a developer for the UAS I can say that sometimes this stuff just takes waaay longer than you anticipate. There are so many unforseen pitfalls that you run into while developing systems like this, it's not surprising that @sonicether is taking his time. I for one am still optimistic :)
     
  48. SteveB

    SteveB

    Joined:
    Jan 17, 2009
    Posts:
    1,448
    He can take his time but why do all these authors stop communicating? Aieth on his asset JOVE is very active and openly discusses his development.

    ...and yea Nasos, I agree with that sentiment, though it's also hard to deny having those added resources, but I do agree with you.

    Right now it's as Pulov said...we've all been through this with Livenda and it boggles my mind that the best course of action these authors think is the correct one is to pretty much ignore an interested and active user base simply because they're unable to deliver perhaps all of what they wanted. Conversely if they were open about it, we'd be far more understanding and maybe could help, just like what is happening over on JOVE's development.
     
    Pulov and Mauri like this.
  49. smetzzz

    smetzzz

    Joined:
    Mar 24, 2014
    Posts:
    145
    As a dev myself I understand why SonicEther is not communicating. It takes an enormous amount of time to write updates and answer questions. Time that is lost developing your product. I am just fine not hearing form SonicEther. I assume he is working hard on a good product and I would rather he focus.
     
  50. sonicether

    sonicether

    Joined:
    Jan 12, 2013
    Posts:
    265
    Hey, everyone. Just wanted to check in and hopefully dispel some of the doubts here.

    I completely understand the skepticism going on in this thread, and I could do better in keeping communication open to aid in that. As @gsmetzer mentioned, it's a little difficult to openly discuss the development process because it's so turbulent--I'm constantly experimenting with different approaches, comparing, reverting, etc. Not to mention, I've been busy getting my SSAO asset ready for release (just submitted it a few days ago) and working on an update for my bloom asset.

    There have been a lot of empty promises regarding real-time interactive GI in Unity. Working on a solution myself, I understand (as I'm sure many of you do to a degree being familiar with the complexity of the effect itself) how difficult it is to deliver on the promise of visually and computationally practical no-precomputation GI, so the fact that those promises haven't been met isn't surprising to me.

    Given the fact that I've already implemented my GI algorithm into Minecraft and it visually and computationally worked, and the fact that the algorithm has matured a ton since then, I'm fairly optimistic about this.



    Just to give some substance to my claims, I've put together a standalone demo (Windows) so you guys can interactively see where the algorithm is right now. Please keep in mind, it's still a work-in-progress and is far from complete. Please also consider the following notes about this demo.

    First of all, GI is being computed at half resolution, and because the whole system is still under construction, there is no bilateral upsampling or edge reconstruction happening, so you'll see pixelization along edges. Know that I have several concepts for how to best deal with this, and have already dealt with it in the past, I just haven't the time to fix it now (this is an extremely common situation in real-time graphics).

    Secondly, you'll see some temporal aliasing/flickering when moving the camera around. I already have a solution for this (it's another problem I've dealt with in the past), but again, haven't the time to implement it right now.

    You may see some artifacts related to the temporal sampling that is going on. I plan on tightening this up when I get back into serious development (which will be after SENBDL is updated).

    And, of course, I want to talk about performance. In the demo, GI is pretty much configured at the highest quality settings. Also, who knows what optimizations I'll find in the future during development (I've stumbled upon quite a few already since the first post in this thread). I'll also mention that along with GI, my SSAO and bloom are present, tonemapping is in there as well, and finally some post-AA, so computationally there's some overhead for those other effects.

    As for the lighting setup in the scene, there's just one directional light with no skylight/ambient light contribution. Just direct light and indirect light from my GI. I tossed in a spinning Buddha and wonky moving cloth thingy to demonstrate the dynamic capabilities of the effect.

    Here's the controls for the demo. Of course, WASD movement, Left Shift and Space to move down and up. You can change the azimuth angle of the sun with the left and right arrows, and the elevation of the sun with the up and down arrows.

    SEGI Alpha Preview 1 Demo

    Thank you guys for understanding and hanging in there. Hopefully the demo will make this project more believable and tangible. I'm curious to hear how the demo scene looks and performs for you, so definitely let me know! :)
     
Thread Status:
Not open for further replies.
unityunity