Search Unity

Fully Dynamic Diffuse Global Illumination

Discussion in 'Works In Progress' started by sonicether, Mar 29, 2015.

Thread Status:
Not open for further replies.
  1. sonicether

    sonicether

    Joined:
    Jan 12, 2013
    Posts:
    265
    You CAN use Unity's standard shader and any other shader framework that utilizes deferred rendering properly (Alloy, for example).
     
    Licarell, RB_lashman and zenGarden like this.
  2. TooManySugar

    TooManySugar

    Joined:
    Aug 2, 2015
    Posts:
    864
    If possible add the sponza scene to allow us play with that setup directly and see how each value affects on an already tuned scene.
     
    Martin_H likes this.
  3. gurayg

    gurayg

    Joined:
    Nov 28, 2013
    Posts:
    229
    @sonicether
    Looking beautiful as always!
    Are there any pre processing steps necessary for this system to work? Like baking some scene data.
    Are there any requirements for the modelled mesh?
     
    RB_lashman likes this.
  4. zenGarden

    zenGarden

    Joined:
    Mar 30, 2013
    Posts:
    4,538
    This is real time so there is no baking unlike Enlighten.
    But if a very fast baking option allowing faster real time GI with some precalculation it could be usefull also.

    You never model a mesh specifically for some lightening or shaders system.
     
    Last edited: Jan 5, 2016
  5. Mauri

    Mauri

    Joined:
    Dec 9, 2010
    Posts:
    1,688
    Seeing your latest screens on twitter, I wanna scream "Release it already" and throw money at your face :D
    Is there a vague timeframe on when you think you'll release this?
     
    Last edited: Jan 5, 2016
    Matthew-Schell and SteveB like this.
  6. zoid25

    zoid25

    Joined:
    Mar 30, 2015
    Posts:
    6
    Great, thanks for the heads-up. I think that price sounds very reasonable.
     
    Matthew-Schell likes this.
  7. SteveB

    SteveB

    Joined:
    Jan 17, 2009
    Posts:
    1,448
    All my money is going all over his face...

    ...man yea I keep forgetting to peep Twitter; this all looks so good! I'm so happy...

    :D
     
    RB_lashman and Mauri like this.
  8. Matthew-Schell

    Matthew-Schell

    Unity Technologies

    Joined:
    Oct 1, 2014
    Posts:
    238
    Just a plus one to say this looks exciting. My personal after hours project is 100% procedural so I can't use Enlighten at all. Hoping whatever you come up with fits in my current rendering budget.
     
  9. nasos_333

    nasos_333

    Joined:
    Feb 13, 2013
    Posts:
    8,033
    How does the solution scale depending on scene complexity ? For example tris count etc
     
    TooManySugar and RB_lashman like this.
  10. zenGarden

    zenGarden

    Joined:
    Mar 30, 2013
    Posts:
    4,538
    Enlighten real time is too slow and doesn't work with Unity PBR shaders.
    That's sad no Unity engineers working on the subject and proposing an official solution, they could buy the plugin or hire him.

    It is like grass, the more you want and put, the more it uses processing power.
     
  11. AcidArrow

    AcidArrow

    Joined:
    May 20, 2010
    Posts:
    6,300
    Wait what? Yes it does.
     
  12. zenGarden

    zenGarden

    Joined:
    Mar 30, 2013
    Posts:
    4,538
    Did you tried the CourtYard demo ? The shaders used is not Unity defaut one. This is a modified or another version of PBR.
     
  13. AcidArrow

    AcidArrow

    Joined:
    May 20, 2010
    Posts:
    6,300
    I tried my own scenes and enlighten works fine with the included PBR shaders, so I don't know what the courtyard demo has to do with anything.
     
    RB_lashman likes this.
  14. mkgm

    mkgm

    Joined:
    Apr 10, 2015
    Posts:
    134
    RB_lashman likes this.
  15. zenGarden

    zenGarden

    Joined:
    Mar 30, 2013
    Posts:
    4,538
    The included shaders are not Unity default shaders , they are not the same.
    I am not sure the demo could run with Unity default PBR shader or would run with the same frame rate.
    Anyway the demo has big errors when i tested it and was too slow, it is a no use for me.
     
  16. AcidArrow

    AcidArrow

    Joined:
    May 20, 2010
    Posts:
    6,300
    By included, I meant the Unity Standard Shader that comes with Unity.
     
    RB_lashman likes this.
  17. Matthew-Schell

    Matthew-Schell

    Unity Technologies

    Joined:
    Oct 1, 2014
    Posts:
    238
    Yeah not sure how Enlighten doesn't work with the built-in Standard shader, that's news to me. The fact that other demos may use custom shaders doesn't mean that those two don't work together.

    If you can't run a scene with Unity's precomputed GI (like courtyard) on your rig my guess is that this solution will be less, not more performant (just a guess) since this is pure real time. The precompute process with Unity's implementation of Enlighten is there to move some computation to edit time instead of run time. My non-work project is procedural and therefore I can't do any kind of precomputing to save performance. It has pretty simple scenes geometry wise though so I'm hoping this will fit in my performance budget. We shall see!
     
    DMeville and RB_lashman like this.
  18. zenGarden

    zenGarden

    Joined:
    Mar 30, 2013
    Posts:
    4,538
    Why having used custom shaders instead of the standard that the majority of people will use ?
    I don't have tested if the demo can run with Unity default shaders without graphic glicthes also , but i'll do a test.

    I can run it with a good Nvidia card (GTX960) , but the frame rate seemed not enought high , perhaps i am wrong so i'll try it on a real game level (terrain ,grass, characters buildings, effects, AI ...) and see how it can perform, this could be interesting.
    The courtyard demo was a first demo perhaps it will be improvments later (i had some big graphic glicthes that was a no use for me)
    I don't need real time GI right now, but i could be tempted, it will be interesting to see what is the best solution ? Unity Enlighten or Sonicether plugin ? More choice is always better.
     
  19. AcidArrow

    AcidArrow

    Joined:
    May 20, 2010
    Posts:
    6,300
    Enlighten needs you to bake things and for stuff to be static, while Sonic Ether plugin is all calculated at runtime (and hence more expensive to render, but on the upside, it presumably works even with dynamic geometry).

    They are not comparable, they are different things.

    And I am not sure why the courtyard demo keeps coming up.
     
  20. rebit

    rebit

    Joined:
    Nov 13, 2014
    Posts:
    133
    Yes, Sonic will release and i will buy !!! Go GO GO SonicEther ! very happy with this new GI
     
    RB_lashman likes this.
  21. sonicether

    sonicether

    Joined:
    Jan 12, 2013
    Posts:
    265
    Actual tracing of lighting is not dependent on complexity. Voxelization and secondary bounce calculation for infinite bounces is, however. The lower poly/simpler your GI geometry is, the better.

    I understand why a lot of voxel GI methods out there use pre-calculated voxel data. Voxelization is definitely the expensive part of this approach currently. That being said, don't be too discouraged if, when you get the beta, voxelization is too slow to be practical. Right now I'm pretty much doing the most straightforward and least conservative thing, which is voxelizing and updating secondary bounces for all geometry (static or not) for the whole volume every frame. There are a lot of options to explore regarding taking a more conservative approach here. I've already got a few ideas and It's one of the things that is on the development roadmap.

    I tend to avoid talking about time frames when it comes to software development, especially for something this complicated, because I'm constantly running into unforeseen obstacles. Even just working on the first demo scene, I'm finding things that need fixing (which, I suppose, is a good thing, and so far they've been relatively simple to fix). You guys know what's left before the first release, and I'm almost done with the most complicated demo. The other two are already modeled and pretty much ready to go. After that, the other preparations shouldn't take me more than a week, assuming that I don't run into any more unexpected setbacks.

    Believe me, I'm just as excited for the first release as you guys are. :)
     
    jcarpay and RB_lashman like this.
  22. nasos_333

    nasos_333

    Joined:
    Feb 13, 2013
    Posts:
    8,033
    Thanks for the details. Will it be possible to exclude certain items from the calculations to speed it up ?
     
  23. sonicether

    sonicether

    Joined:
    Jan 12, 2013
    Posts:
    265
    Yep, there's a culling mask that you can use to exclude layers.
     
    CWolf, mkgm, nasos_333 and 3 others like this.
  24. macdude2

    macdude2

    Joined:
    Sep 22, 2010
    Posts:
    685
    @sonicether — Got a got a couple questions!

    First, will it be possible to use your algorithm to generate a lightmap to use statically so that procedurally levels could be well lit on lower spec hardware?

    Second, if one made levels of the complexity and size of those that exist in Portal (that is to say simple and small) do you have any idea what computer spec might be needed to run your GI at a decent frame rate?

    Really hyped to see the demos!
     
    RB_lashman likes this.
  25. Golesy

    Golesy

    Joined:
    Apr 10, 2013
    Posts:
    33
    @sonicether still looks amazing!
    I've just got a small question. My project has a lot of emissive blocks people can build with that have attached spot lights to actually give off light. It's one of the biggest performance hits right now. With the voxelization method would that remove the need for point lights all together? Do you think there might be a possibility of it increasing performance in this situation? Of course I wont know for sure until I plug it in, just wondering what your thoughts are on it. Don't worry I'm buying this either way :)
     
    RB_lashman likes this.
  26. Kombiice

    Kombiice

    Joined:
    Jan 20, 2015
    Posts:
    64
    awesome !!
     
    RB_lashman likes this.
  27. Kombiice

    Kombiice

    Joined:
    Jan 20, 2015
    Posts:
    64
    @macdude2 would be awesome to see that in your game !!
     
    RB_lashman likes this.
  28. sonicether

    sonicether

    Joined:
    Jan 12, 2013
    Posts:
    265
    Being able to bake the lighting result would be awesome. I'm definitely going to look into it in the future. Considering that GI renders in milliseconds, it should be very quick to bake, which would be a great option to have

    As far as your scene complexity comparison, it's hard to say at this point. I tested the demo scene I'm working on currently on a GTX 760 yesterday. I was pretty happy with the results, seeing anywhere between 35 and 110 fps depending on the quality settings. That, however, is very likely going to change for the better with the optimizations I have planned. The most significant optimization to come will happen when Unity is updated to allow me to work with native mipmaps instead of using the separate texture workaround in use currently, since tracing is a lot cheaper with native mipmaps. Updating voxel data in a smarter way will hopefully speed things up as well. The addition of cascades may or may not speed things up, I'm not certain of that yet.

    Another possibility is decoupling voxelization and light injection. This would require more memory, since additional data would need to be stored like normal and albedo, but it would grant the benefit of only needing to inject updated light onto already voxelized geometry in the case of static geometry.


    Yeah, that's a possibility. With this system, you essentially get area lights for "free" since cone tracing is happening anyway, and making any geometry that is already there write an emissive value during voxelization is trivial.
     
  29. Misciagno

    Misciagno

    Joined:
    Jul 27, 2013
    Posts:
    88
    Does the GI ignore terrain detail objects? I imagine having a bunch of grass combined with your technique would kill the performance since it adds so many tris.
     
  30. Kombiice

    Kombiice

    Joined:
    Jan 20, 2015
    Posts:
    64
    Lokks like we'll have a real DYNAMIC "GI" soon :p
     
    Last edited: Jan 9, 2016
    RB_lashman likes this.
  31. zenGarden

    zenGarden

    Joined:
    Mar 30, 2013
    Posts:
    4,538
    +1 It would be a great option.
     
  32. TooManySugar

    TooManySugar

    Joined:
    Aug 2, 2015
    Posts:
    864
    I think no one asked this yet. Emisive movie textures would be possible? affecting the environment lighting ?
     
    RB_lashman likes this.
  33. sonicether

    sonicether

    Joined:
    Jan 12, 2013
    Posts:
    265
    That's a great point, I'll have to check that.

    Yep, totally possible.

    Also, I've got a bit of unfortunate news regarding temporal sampling. It's a little difficult to explain, but I'll try.

    A temporally-sampled GI result represents lighting values that are smoothed over time. Imagine an object moves and reveals a "fresh" area of the screen that hasn't been calculated in previous frames yet. That fresh area will be calculated and represent the scene lighting at that exact frame, whereas the other "stale" areas of the screen have had previous frames to blend with, so they have a different value than the fresh area. This is a situation I hadn't encountered or thought of until a few days ago. So, I'll leave the feature of temporal sampling in for those who want to use it, because it can still be useful in some rare situations, but especially with this recent news, it causes more problems than it solves. The good news is that it's entirely optional. The bad news is that temporal sampling is essentially an optimization that can't be taken advantage of most of the time because of the problems it causes (and that problem in particular). So, more samples per pixel will have to be taken in order to obtain the same result without temporal sampling. This extra cost will be helped when native mipmaps are possible (cheaper GI tracing).

    Temporal sampling was my main method for reducing noise, but there are other ways. You could disable stochastic sampling (randomization of cone directions), but without stochastic sampling, GI can look pretty blotchy if there are too few cones. You could also enable bilateral filtering to smooth out the noise, but that obviously tends to smudge high-frequency details. It all depends on you to decide what's best for your scene. I'm basically aiming to provide as many useful ways around problems as possible, each having their own strengths and weaknesses, so it'll be up to the user to decide.
     
  34. CWolf

    CWolf

    Joined:
    Oct 24, 2011
    Posts:
    106
    Thanks for the great updates @sonicether - even if things didn't go your way with the Temporal sampling.

    Out of curiosity, you keep mentioning the direct native mipmap access. Is this something you know Unity are adding, or more hoping they will add?
     
    RB_lashman likes this.
  35. HenriqueGomez

    HenriqueGomez

    Joined:
    Dec 13, 2012
    Posts:
    63
    RB_lashman and CWolf like this.
  36. CWolf

    CWolf

    Joined:
    Oct 24, 2011
    Posts:
    106
    Thanks =). Saw they were adding in Texture Array support but didn't link it with mipmaps.
     
    HenriqueGomez and RB_lashman like this.
  37. ksam2

    ksam2

    Joined:
    Apr 28, 2012
    Posts:
    1,001
    Can you please release the first demo?
     
  38. Nyt0x

    Nyt0x

    Joined:
    Feb 3, 2014
    Posts:
    18
    What? No.. Texture arrays is a texture format using an array of... texture! As input.

    Just think of a texture, it is a 2D image (put aside the filtering type and the way different shader stage handle it for the moment)

    If you take multiple texture and pack them as an array you can access them in one shot, just asking the shader to take the correct one by its index. it looks like a 3D texture right? 3 dimension, 2D for the texture, and 1D for the array.
    The trick is filtering.
    In a 3D texture, there's filtering in each dimension, for a texture array, 2D texture are filtered in their 2D dimension, but the dimension of the array is not filtered.

    So a 2D texture array is a 3D texture without filtering on the Z axis.

    What is filtering? Well to put it simple the pixel shader stage does not access a texture on a 1 pixel 1 index basis, texture are handle as floating values between 0 an 1, if you access the x = 0.5 and y = 0.5 you're in the middle of a texture but that texture might as well not be squared or an uneven number so there might not be a pixel index there, so you filter, meaning you have rules to estimate the value that should be there looking at the neighboring pixels.

    Mip map are part of that filtering, when estimating that texture pixel, you don't want to spend to much time accessing a texture for a pixel you are rendering that is far from the camera, in place of accessing the full scale texture to estimate the pixel value you're taking a less detailed version of the texture, the result might be off but you don't care as the pixel you're rendering do not contribute much to the details in the current frame.

    That's a rough explanation full of simplification, but yeah, texture array and mipmap are not the same. There"s a link between the two as texture array are still textures and can still have mip map.

    By the way a 3D texture also have mipmaps, 3D ones, where a texture array have an array of 2D mipmap ;)


    Also what sonicether means by native mipmap is "mipmap directly computed by the hardware"

    When you send your texture to the device (your CG) it does the mipmapping for you (with the setting you ask it to use)
    For GI I guess he uses 3D textures, and atm unity does not let you tel the device to build mipmap for them.
    Meaning that you have to compute them yourself (either on the CPU...ouch, or using a Compute buffer, less ouch but still ouch as you have to ask your device to change context to run the compute shader and then go back to the rendering pass, and also allocate buffer and ressources for)
     
    CWolf and RB_lashman like this.
  39. TooManySugar

    TooManySugar

    Joined:
    Aug 2, 2015
    Posts:
    864
    amm. some time ago we saw some unity staff around here interested on the bug that holds the developer to integrate some neat features. Any news on if this isue is being shorted from the unity side?.
     
    Licarell and RB_lashman like this.
  40. CWolf

    CWolf

    Joined:
    Oct 24, 2011
    Posts:
    106
    Thanks for the very informative post =).
     
    RB_lashman likes this.
  41. KyleOlsen

    KyleOlsen

    Joined:
    Apr 3, 2012
    Posts:
    218
    I'm so excited for this :eek:
     
    RB_lashman likes this.
  42. Hexaedre

    Hexaedre

    Joined:
    Jan 23, 2015
    Posts:
    110
    A wonderful new step for lighting systems is happening here.

    I am so glad from seeing this amazing progress.
     
    RB_lashman likes this.
  43. TooManySugar

    TooManySugar

    Joined:
    Aug 2, 2015
    Posts:
    864
    If this delivers it will be the reason to switch to U5 for my next project. Otherwise I feel pretty confortable in teh uber stable 4.6 platform.
     
    macdude2 and RB_lashman like this.
  44. sonicether

    sonicether

    Joined:
    Jan 12, 2013
    Posts:
    265
    Alright, I think the low poly demo scene is done! I'll provide a standalone to avoid the forced vsync in web player builds. I can only provide a Windows build for now.

    I've only tested this on four PCs so far, so I'm really crossing my fingers here regarding compatibility. If things are looking bleak, I might consider delaying the Asset Store release a bit to work on compatibility.

    A few things to consider: the Low and Medium presets rely on temporal sampling. So, you may see some artifacts from that (the most noticeable being when you pick up and move an object). Keep in mind that a velocity buffer will solve that problem, although, like I said, I'm still unsure whether temporal sampling will solve more problems than it causes in the long run. This will be worked out throughout the course of development on this project.

    You can figure out approximately how much time GI is taking to render with this simple formula:

    giTimeInMilliseconds = (1000 / frameRateWithGI) - (1000 / frameRateWithoutGI)

    Toggling voxel resolution with R can make a big impact on performance, as well as toggling infinite bounces with I.

    And, of course, everything is subject to change (and hopefully improve)!

    SEGI b0.8 Low Poly Demo.zip

    I'll get working on the other demo scenes right away and post them here when they're completed!
     
    zoid25, CWolf, chiapet1021 and 4 others like this.
  45. Shinyclef

    Shinyclef

    Joined:
    Nov 20, 2013
    Posts:
    364
    @sonicether your work looks fantastic! I'm also working on a completely procedural game.
    I'd like to ask about compatibility with my game.
    I have a voxel system in my game that uses a geometry shader to construct the geometry.

    It goes something like this:
    - add a vert to verts list.
    - add that vert's index three times to tris list.
    - on the GPU's geometry shader, convert that vert into 2 triangles whose size varies based on information stored in UV and other properties. Those tries get passed to the frag shader and drawn from there.

    Will I be able to get my game to work with your voxelisation algorithm? What are the requirements for it to work in terms of mesh data...?
    Also, will the process have any problem with concave geometry?
     
  46. Shinyclef

    Shinyclef

    Joined:
    Nov 20, 2013
    Posts:
    364
    Wow I'm trying the scene you just posted. It's truely amazing. I'm hoping I can get this to work with my geometry shader meshes...

    Performance feedback:
    CPU: Core i5 3.4GHz
    GPU: GTX970
    1440p

    Load scene, don't do anything: 51 fps.
    Without moving or rotating...
    Press 1: 74 fps
    Press 2: 48 fps
    Press 3: 37 fps
    Press 4: 30 fps
    Press 2, then R: 67 fps. <- I'm attracted to this for performance and reflections.

    Interestingly...
    Switching infinite bounces ON gives me MORE fps than when it's off. That's interesting...
     
    Last edited: Jan 14, 2016
    RB_lashman likes this.
  47. KyleOlsen

    KyleOlsen

    Joined:
    Apr 3, 2012
    Posts:
    218
    It looks perfect here, I'm getting a solid 60 fps with max settings and hundreds of spawned objects in the scene.

    Unity 5.3.1p1
    Windows 10
    980TI x2
     
    RB_lashman likes this.
  48. TooManySugar

    TooManySugar

    Joined:
    Aug 2, 2015
    Posts:
    864
    The demo:
    NVIDIA GeForce GTX 560 Ti
    i7-3820 CPU @ 3.60GHz

    View from the sky seeing all the world:
    2560*1080
    Withoug GI 100FPS
    GI all low = 40FPS
    +Infinite bounces=37FPS
    +High voxel res=28FPS
    --------------------------------------
    With all on switching GI quality
    2=25FPS
    3=18FPS
    4=14FPS

    Quality comes at a high performance cost with not so much visual enhancement.
    In general all low settings with Infinite bounces witch seems to have very low impact delibers good quality.
    -------------------------------------

    I think you should try with a more complex scene, something that could feel real if rendered Vray, something like an urban scene. I've something of very high end urban environment but we could test only on non public, if interested send me PM.
    Otherwiese simpler urban scenes with enlighten have been tested with impresive ressults you could try.
     
  49. amasinton

    amasinton

    Joined:
    Aug 12, 2006
    Posts:
    131
    I'm on a very old machine - a dual-booted MacPro2009 with an Nvidia GeForce GT 120 (not the best kit these days).

    When I turn on GI the whole game window flickers between black and then the world view very quickly.

    I can run the scene without GI, but that's obviously not the point.

    In the console I get a lot of the following: Invalid pass number (x) for Graphics.Blit. Material=Hidden/SEGI, npasses=1

    This is odd hardware and configuration, so don't worry about it, but I thought you might find the report useful.
     
    DMeville likes this.
  50. TooManySugar

    TooManySugar

    Joined:
    Aug 2, 2015
    Posts:
    864
    Further testing:
    Quality certainly is quite noticeable betwen level 1 and 2, specially when there are glowing objects.















     
    wetcircuit and RB_lashman like this.
Thread Status:
Not open for further replies.
unityunity