Search Unity

SEGI (Fully Dynamic Global Illumination)

Discussion in 'Assets and Asset Store' started by sonicether, Jun 10, 2016.

  1. castor76

    castor76

    Joined:
    Dec 5, 2011
    Posts:
    1,401
    Humm.. yeah it is probably because of the vertex shader deformation. But I am not 100% certain. However, it does make sense..
     
    RB_lashman likes this.
  2. edan_kuva

    edan_kuva

    Joined:
    May 11, 2017
    Posts:
    3
    I don't know how SEGI picks up the vertex position... If it can somehow use the transform feedback results or anyway to use a custom sub compute shader for the voxelization, that will be great.
     
    RB_lashman likes this.
  3. OnlyVR

    OnlyVR

    Joined:
    Oct 5, 2015
    Posts:
    50
    Works fine in 5.6.0f3.
    In VR works as well but 40FPS :-(
     
    FPires, RB_lashman and buttmatrix like this.
  4. Lex4art

    Lex4art

    Joined:
    Nov 17, 2012
    Posts:
    154

    Source code is available, maybe something useful here for SEGI )
     
    Malbers, ikazrima, arnoob and 2 others like this.
  5. zenGarden

    zenGarden

    Joined:
    Mar 30, 2013
    Posts:
    4,538
    Wooowww
    I hope someone will make a Unity plugin with Wicked ( pretty good it works also in Forward +).
     
    P_Jong and RB_lashman like this.
  6. TooManySugar

    TooManySugar

    Joined:
    Aug 2, 2015
    Posts:
    864
    Don't think that it looks any better than SEGI
     
    P_Jong, Crossway, neoshaman and 3 others like this.
  7. zenGarden

    zenGarden

    Joined:
    Mar 30, 2013
    Posts:
    4,538
    Perhaps but the video shows it's a finished and complete solution, even on outdoors or specific things like reflections.
    It just needs someone to be able make a plugin from it.
     
    Last edited: May 17, 2017
    arnoob and RB_lashman like this.
  8. Lexie

    Lexie

    Joined:
    Dec 7, 2012
    Posts:
    643
    @sonicether Your last post said you were trying to speed up voxelization so I had a look over your code base cause my method for realtime GI shares some similarities. I noticed that your voxelization shader is pretty rough. If you want to speed up the voxelization process you could optimize your current shader to not use soo much matrix math.

    On the scene I'm testing in, my voxelization shader costs 2.5ms to voxelize the full volume where as yours costs 8.5ms. (this is done with the same resolution and volume size)
    My voxelization shader also writes into more textures then yours does so you should be able to beat that 2.5ms.
     
    Last edited: May 17, 2017
  9. neoshaman

    neoshaman

    Joined:
    Feb 11, 2011
    Posts:
    4,190
    How does that translate into raw fps estimation?
     
    TooManySugar and RB_lashman like this.
  10. Lexie

    Lexie

    Joined:
    Dec 7, 2012
    Posts:
    643
    Hard to say, depends on a lot of factors. But if voxelization is something he is looking to improve then getting a 2-4x speed improvements for exactly the same result is probably be worth it.

    Edit: just off raw numbers my current scene would go from 41fps to 55fps using SEGI if the voxelization shader was optimized. (note: hard to say if it would be that big of an improvement or not)

    I might be reading this wrong cause the voxelization shader spikes every second frame to 16ms. I went with the lower number incase unity is reporting something strange. If you are splitting up the work over multiple frames then you are going about it the wrong way. you shouldn't have massive spikes like this, Sure it increase your average FPS but it makes your game unstable and jerky.


    Tested on a 970m (laptop GPU around the speed of a 660 GTX)

    Your shader compiles out to
    // Stats for Vertex shader:
    // d3d11: 7 math
    // Stats for Fragment shader:
    // d3d11: 121 math, 3 texture, 5 branch
    // Stats for Geometry shader:
    // d3d11: 79 math

    Mine compiles down to
    // Stats for Vertex shader:
    // d3d11: 10 math
    // Stats for Fragment shader:
    // d3d11: 50 math, 3 branch
    // Stats for Geometry shader:
    // d3d11: 23 math

    Looks like your doing something extra with texture reads, but still there is room for improvement there.
     
    Last edited: May 21, 2017
  11. gmatthews

    gmatthews

    Joined:
    Jan 13, 2016
    Posts:
    10
    Haven't quite gone live yet, but seeking to launch this Enterprise Architecture modelling tool before end of June.

    www.architectureinmotion.com.au



    It's an Enterprise Architecture / IT portfolio management tool for enterprises to better plan digital transformation. i.e. create a digital business model simulation and then overlay scope of projects as part of planning IT spend, which gets complex when there's hundreds of apps, and many dozens of projects occurring at once.

    Uses SEGI and some bloom effects, although is quite simple and understated compared to other game demos posted in this thread. SEGI works well here since every scene element is entirely dynamic.
     
    RB_lashman and Baldinoboy like this.
  12. sonicether

    sonicether

    Joined:
    Jan 12, 2013
    Posts:
    265
    Funny, I've actually been watching your progress on your GI solution on YouTube. I tried something similar (progressive/recursive light propagation) but wasn't thrilled about the "femto photography" light ripples artifact. Had I known it could look as good as your solution does, I might have kept going!

    Since the latest version of SEGI that's on the asset store, I have simplified the voxelization shader somewhat (only one matrix multiplication in the geometry shader now). Though, it definitely didn't make up the considerable difference between how fast your voxelization shader is versus mine. Perhaps I'm doing something superfluous in the interlocked atomic add function for writing to the texture, or maybe in encoding a pseudo-HDR HSV + alpha value into the one 32 bit unsigned integer texture. The fact that the interlocked add needs to repeatedly encode and decode that value might be causing more slowdown than it's worth. I could definitely reduce branching as well.

    I've also considered injecting irradiance from indirect light in a separate compute shader instead of during the fragment shader of the voxelization shader, perhaps there are gains to be had in that endeavor as well.
     
    CWolf, arnoob, buttmatrix and 7 others like this.
  13. Lexie

    Lexie

    Joined:
    Dec 7, 2012
    Posts:
    643
    Yeah its probably faster combining bounce from the last step in a compute shader, you could even look up the sunlight contribution there as well. you wont get as accurate of a color cause it would just be bouncing the color of the whole voxel though. But probably worth the trade off. Having a lightweight fragment shader is really important as the interlock is going to stall way more if its slow.

    You pretty much have to use the interlock when writing to the volume though. I encode the max color (1010102) to an int texture.

    If you use a do while loop rather then a for loop you should remove a few branchs and also remove some of the color flickering when you have a lot of fragments in the same voxel.

    My interlock code looks like this

    My pack and upack is just converting between an int and a 1010102 color.

    Edit: Also you don't need any matrix math in the geometry shader.
    It's orthographic rendering so you can just switch around some values of the position of the screen vertex.

     
    Last edited: May 22, 2017
    pcg, CWolf, nxrighthere and 12 others like this.
  14. buttmatrix

    buttmatrix

    Joined:
    Mar 23, 2015
    Posts:
    609
    ^this is why I come here. Awesome!
     
    pcg, arnoob, RB_lashman and 1 other person like this.
  15. Ogdy

    Ogdy

    Joined:
    Dec 26, 2014
    Posts:
    17
    Hey people,

    I have one question about SEGI and maybe voxel cone traced GI in general.
    In my game, which is 100% indoor, the lightning is done entirely with emissive materials on walls (kind of neons lights). It looks really good with SEGI, however I have a major issue when I place lights in rooms that are close to each other : I noticed that light isn't really stopped by my walls, and I can see the light from a room in the rooms nearby, which is not wanted.

    I joined a few pictures to better explain the issue :
    bleed.png

    bleed_2_planes.png

    voxels.png

    I can see that there is a "GI Blocker" shader in the documentation, but all my game being procedural, I went for SEGI to avoid the big baking hassle, and I'm afraid that tricking with GI Blockers would just killl all the benefits.

    Is there a solution for this ? I mean, without having more space between my rooms and without use GI Blockers everywhere ? Is this doable with cone tracing, and if yes, will SEGI support it one day ?

    Thanks in advance :)
     
    RB_lashman likes this.
  16. Mauri

    Mauri

    Joined:
    Dec 9, 2010
    Posts:
    1,494
    I guess that's the "Light Leaking", which is mentioned here. You can (kind of) tone it down a bit by playing with the Inner Occlusion Layers property and by using GI Blockers. Other than that, there is not much you can do right now.
     
  17. Ogdy

    Ogdy

    Joined:
    Dec 26, 2014
    Posts:
    17
    Thanks Mauri. Indeed, this is probably due too the lack of directional information. I see that the progress indicator says 20% for this feature. I hope that we will be able to see that working soon :)
     
    RB_lashman likes this.
  18. Giga_Steve

    Giga_Steve

    Joined:
    Nov 11, 2014
    Posts:
    13
    Unless I missed it in this thread, but is SEGI (or any other global illumination system on the asset store) compatible with custom lighting such as toon-like shaders? I know when it comes to lighting systems, that type of shader can be problematic when it comes to certain lighting conditions.
     
    RB_lashman likes this.
  19. neoshaman

    neoshaman

    Joined:
    Feb 11, 2011
    Posts:
    4,190
    Depend on what you want to achieve. But vanilla celshading don't have bounce light let alone smooth shading. Illustrative effect and complex NPR effect might benefit but that would call for solid art direction and if you ask the question you are not there yet. For celshading ambient volume is generally the closest equivalent.
     
    RB_lashman likes this.
  20. pointcache

    pointcache

    Joined:
    Sep 22, 2012
    Posts:
    525
    Hello i am unable to build the project i get these errors on unity

    u 5.5.2f1
     
    RB_lashman likes this.
  21. Lex4art

    Lex4art

    Joined:
    Nov 17, 2012
    Posts:
    154
    Solution.
     
    RB_lashman likes this.
  22. Elecman

    Elecman

    Joined:
    May 5, 2011
    Posts:
    1,330
    After days of futile work trying to get Enlighten to work in my A320 cockpit (see my blog), I am looking for an alternative. The problem is that the cockpit has thousands of tiny objects (switches, buttons, etc.), and even the larger panels have many small faces. This is where Enlighten struggled significantly, even with the use of light probes for the smaller objects.

    I was wondering if SEGI is a better fit for this use case.
     
    RB_lashman likes this.
  23. neoshaman

    neoshaman

    Joined:
    Feb 11, 2011
    Posts:
    4,190

    Small objects (hi frequency details) should work with generic solution like SSAO, SSR, lightprobe and environment. The trick is to understand the contribution of each and set them up accordingly, they will cover each others weakness.

    RT GI solution like enlighten and even SEGI are low frequency solution, great at room scale, but details will be paint over. They need to be complemented by the above.

    Here is a more thourough breakdown:
    SSAO, ideal to bring shadow contact of small details
    SSR, ideal to have contribution of self reflection and short contact reflection
    Lightprobe environement, work well to have the global reflection of the scene, is complemented with SSR to have what SSR is good for. Should be masked by AO to avoid reflecting in crevice (and let SSR fill in). Problem is that it's not dynamics.
    Enlighten is at the room of level scale, ie give a very smooth contribution

    It's like with shadow, you would have typical dynamic shadow, it is low resolution but can work on entire level. Screen space contact shadow allow to have the small length sharp shadowing of small details, but don't work at long distance (hence contact).

    There is no silver bullet at all. You need each tools and understand their role.
     
    RB_lashman and Elecman like this.
  24. Elecman

    Elecman

    Joined:
    May 5, 2011
    Posts:
    1,330
    Thanks. I have SE's screen space ray traced shadows too and it is amazing. I suppose SEGI won't work in such a small environment without the smaller objects looking out of place. A lot of small objects are fused together to reduce draw calls which makes them even harder to blend in.

    I didn't try SSR yet. I will give that a try too.
     
    Last edited: May 28, 2017
    RB_lashman likes this.
  25. neoshaman

    neoshaman

    Joined:
    Feb 11, 2011
    Posts:
    4,190
    It's not that segi won't work in such a small environment, it's that it's contribution is not the same, you can't use just one technique. You will likely need many techniques to get the result you want, and that's mean it also depend on your material, visual target, etc ....
     
    RB_lashman likes this.
  26. TooManySugar

    TooManySugar

    Joined:
    Aug 2, 2015
    Posts:
    864
    I would go for a lightmap backed light such an Vray or Like former Beast to get neat indirect lighting etc with a generic cloudy sky and then use screen space stuff like SE SSshadows for the hard shadow stuff.
     
    RB_lashman likes this.
  27. Elecman

    Elecman

    Joined:
    May 5, 2011
    Posts:
    1,330
    Thanks, I will try that too.
     
    RB_lashman likes this.
  28. jammer42777

    jammer42777

    Joined:
    Apr 21, 2014
    Posts:
    85
    I sent the creator this question, however I want to know of those who have used segi:

    what shaders and post processing effects will it break?

    I am currently using: Alloy (it does require custom project settings)
    Amplify bloom, Amplify Occlusion and Amplify Shader editor.

    I also Use Uber standard shader ultra and RTP.

    Will these things work under segi?

    Also I am wondering if Segi isn't meant for large out door areas, is it possible to turn segi off and Unity GI on during run time? Or upon a new area load?
     
    RB_lashman likes this.
  29. jammer42777

    jammer42777

    Joined:
    Apr 21, 2014
    Posts:
    85
    I did some digging in the forum.
    No shaders that don't work in deferred rendering or use a custom brdf. (nicks alloy, but it was annoying me anyways)
    I'm buying the beta on Thursday. I'll post my findings.
     
    RB_lashman likes this.
  30. Michael55

    Michael55

    Joined:
    Jun 15, 2017
    Posts:
    10
    I just bought SEGI. Meshes drawn with Graphics.DrawMesh are handled by SEGI but meshes drawn with Graphics.DrawMeshInstanced are not. Is this a technical limitation or something that can be worked around?
     
    RB_lashman likes this.
  31. LennartJohansen

    LennartJohansen

    Joined:
    Dec 1, 2014
    Posts:
    2,275
    That is a bit strange. you are using the same shader for the instanced and drawmesh when testing?
     
    RB_lashman likes this.
  32. Michael55

    Michael55

    Joined:
    Jun 15, 2017
    Posts:
    10
    Yes, both are using the same material with the standard shader. I created a new scene and used the following script to test:

    Code (CSharp):
    1. using UnityEngine;
    2.  
    3. public class SegiInstancingTest : MonoBehaviour
    4. {
    5.     public Mesh mesh;
    6.     public bool useInstancing;
    7.     public Material material;
    8.  
    9.     void Update () {
    10.         Matrix4x4[] matrices = new Matrix4x4[] {Matrix4x4.identity};
    11.         if (useInstancing)
    12.         {
    13.             Graphics.DrawMeshInstanced(mesh, 0, material, matrices);
    14.         }
    15.         else
    16.         {
    17.             Graphics.DrawMesh(mesh, Matrix4x4.identity, material, 0);
    18.         }
    19.     }
    20. }
    Enabling debugging from SEGI shows that the mesh drawn with DrawMeshInstanced does not get voxelized. This is problematic for my use case.
     
    RB_lashman likes this.
  33. LennartJohansen

    LennartJohansen

    Joined:
    Dec 1, 2014
    Posts:
    2,275
    aha. It does get lighting from SEGI? problem is that it does not add itself to the voxels?
     
    RB_lashman likes this.
  34. Michael55

    Michael55

    Joined:
    Jun 15, 2017
    Posts:
    10
    I double-checked so it does indeed get lighting from SEGI. Just doesn't get voxelized. If this is something that cannot be easily fixed, one workaround might be to use DrawMesh and proxy objects for voxelization.
     
    RB_lashman likes this.
  35. Michael55

    Michael55

    Joined:
    Jun 15, 2017
    Posts:
    10
    Segi kills my scene's performance on 4k (rendering to render texture, then downsampling to 1080p).

    Segi off: 13 ms gpu
    Segi on (bright preset): 34 ms gpu
    Segi on (low preset): 23 ms gpu

    ^The blit on bright preset is taking whopping 23ms of GPU time on Gtx 980Ti!

    Any ideas on how to get my framerate to acceptable levels? Quater resolution would likely help a bit since I am downsampling anyways.
     
    RB_lashman likes this.
  36. arnoob

    arnoob

    Joined:
    May 16, 2014
    Posts:
    151
    What is your framerate at 1080p? Is baking on? Infinite GI? What are the polys you are using for sampling?
     
    RB_lashman likes this.
  37. Michael55

    Michael55

    Joined:
    Jun 15, 2017
    Posts:
    10
    For 1080p:
    Segi off 230 FPS (instacing enabled, see my earlier posts)
    Segi off 64 FPS (instancing disabled, since instanced objects aren't voxelized)
    Segi on 43 FPS (bright and low presets)
    For 1080p the performance is mostly due to geometry and not the Blit step so I will be able to optimize my scene for Segi. The geometry is terrain and trees. For 4k the problem is the Blit step, though. I may be mistaken, but to my understanding geometry does not affect the Blit step which is taking most of the GPU time.

    Infinite bounces on/off does not cause a measurable difference in framerate. Same goes with Update GI, Voxel AA, Gaussian Mip Filter. No enlighten bake if that was the question.

    I also tried creating a 4k scene with nothing but a directional light and single cube with default material.
    Segi off 4k: 1550 FPS
    Segi on 1080p: 158 FPS (bright preset)
    Segi on 4k: 43 FPS (bright preset)
    1550 FPS drops to 43 FPS when enabling SEGI.
     
    arnoob and RB_lashman like this.
  38. neoshaman

    neoshaman

    Joined:
    Feb 11, 2011
    Posts:
    4,190
    Did you test the script posted in this thread where you only update voxel when you move?
     
    Crossway, TooManySugar and RB_lashman like this.
  39. christoph_r

    christoph_r

    Joined:
    May 20, 2013
    Posts:
    481
    That's not likely to help with screen space blit operations, which appear to be the bottleneck. That does sound odd, though.
     
    RB_lashman likes this.
  40. Michael55

    Michael55

    Joined:
    Jun 15, 2017
    Posts:
    10
    Thanks for the suggestion. Disabling Update GI only increases FPS by around ~10%, so it is not enough to achieve a playable framerate.

    Another issue I am still facing is that meshes drawn with "Graphics.DrawMeshInstanced" do not get voxelized unlike meshes drawn with "Graphics.DrawMesh". It won't help with the 4k issue, but instancing would help performance on 1080p since I have a large number of trees, unless there is a technical reason why instancing cannot work with voxelization.
     
    RB_lashman likes this.
  41. castor76

    castor76

    Joined:
    Dec 5, 2011
    Posts:
    1,401
    SEGI at 4k... I think you are pushing it too far. SEGI barely works at ok level on 1080p and you would need very high spec gpu on that too. Are you targeting some specific beefed up specs ?
     
    buttmatrix, RB_lashman and arnoob like this.
  42. Michael55

    Michael55

    Joined:
    Jun 15, 2017
    Posts:
    10
    I have a lot of vegetation and due to deferred rendering SSAA is just about the only way to get rid of aliasing without making everything look blurry. For most users without 4k monitors it's downsampled to 1080p in the end anyways. I imagine SEGI could run at a much lower resolution, there's no need for the global illumination to be pixel perfect at 4k. There is already the possibility to run at half resolution whereas quarter resolution would likely be enough at 4k.
     
    christoph_r and RB_lashman like this.
  43. Mauri

    Mauri

    Joined:
    Dec 9, 2010
    Posts:
    1,494
    Have you tried TAA by any chance?
     
  44. Lex4art

    Lex4art

    Joined:
    Nov 17, 2012
    Posts:
    154
    It was discussed some months ago [link], my guess is that path tracing/ray tracing algorithms drops performance linear to amount of pixels on the screen and they are used in some form inside SEGI; that tracing stuff gives most reliable results, though. Hope Sonicether will found a way to outsmart this math-intensive task. Like:
    Even on half resolution there is evil pixelisation artefacts on edges, but if there is possible to have quarter resolution and avoid those - this will do the job for me too, at least for small scenes ).
     
    Last edited: Jun 18, 2017
    RB_lashman likes this.
  45. arnoob

    arnoob

    Joined:
    May 16, 2014
    Posts:
    151
    Not op but sadly official TAA, while promising, is really glitchy beyond professional use as of now...


    On an other subject, does anybody has managed to use something else than the MainTex as source of color for sampled meshes?
     
    RB_lashman likes this.
  46. Michael55

    Michael55

    Joined:
    Jun 15, 2017
    Posts:
    10
    TAA from the Unity post processing stack is what I am currently using. The results are relatively good but there are some issues such as flickering shadows.

    @Lex4art Thanks for sharing. It's difficult to tell but I'm hoping vegetation might be a bit more forgiving with artefacts than say, a red cube on blue background.

    Anyways, getting 4k working is not an absolute must for me, I just wanted to hear if you guys had any thoughts on the subject. Getting DrawMeshInstanced working with SEGI voxelization, if such thing is possible, is a much higher priority right now.
     
    RB_lashman likes this.
  47. neoshaman

    neoshaman

    Joined:
    Feb 11, 2011
    Posts:
    4,190
    Inever tried it, but @brilliantgames had a crowd rendering tech where they basically a single vertex buffer custom managed to have all the objects in (ie a single object from unity). Maybe if you render proxy low poly objects using a similar implementation where group of vertex are sync with the instance position?

    I have never attempted that, so it just a last ditch solution.
     
    RB_lashman likes this.
  48. sonicether

    sonicether

    Joined:
    Jan 12, 2013
    Posts:
    265
    Hey everyone! I wanted to update you on how development is going.

    I took some time today to really compare the latest development version to v0.83 (the latest release available on the Asset Store), and I guess all the little improvements I've made have really added up and I hadn't noticed because the change has been gradual from my perspective. So, I'd like to share that with you now.

    My test scene of choice was the Viking Village. I've disabled all the small props to simulate an ideal poly count for GI-contributing geometry (since low poly proxies should be used for the buildings, and very small detail/clutter objects shouldn't be voxelized). So here, the high poly buildings are voxelized. The scene is rendered at 1920x1080 on a GTX 1080.

    This is how the scene looks without GI. Total render time is 2.9 ms


    This is the same scene view with SEGI v0.83. Total render time is 8.9 ms


    This is the same scene view with the latest development version of SEGI with cascades, with the same GI tracing settings (half resolution, stochastic sampling, bilateral blur, 3 cones and 12 trace steps with temporal blending). Total render time is 5.1 ms


    This means that in this particular scene, the total render time for the latest development version of SEGI is 2.2 ms, which means that it is approximately 2.8x faster than v0.83 (6 ms).


    Temporal blending and bilateral blur go a long way with the new blue noise approach to stochastic sampling. Here you can see what white noise looks like with no temporal blending or blurring (v0.83):


    And what using blue noise looks like with no temporal blending or blurring (latest development version):



    With the improved spatial and temporal coherence of blue noise, you can get away with using fewer cones per pixel, which helps offset the additional tracing cost of the new cascaded approach.

    I've made a lot of improvements to the speed of voxelization to the point where, even though GI tracing is slower, the total render cost of SEGI has decreased a lot, as you can see. I'm going to see what I can do to speed up GI tracing more.


    And finally here's a demonstration of the vastly improved render distance of GI by utilizing cascades.



    As you can see, the quality of GI near the camera is roughly the same, but with cascades the entire village can be rendered with GI.


    There is a new problem to be considered with cascades, however. Since only one cascade level is voxelized on a frame, certain frames are more expensive to render because some cascades are larger than others and contain more geometry to voxelize. I'm considering providing separate culling masks for each cascade level so that smaller detail objects can be omitted from the larger cascades (the largest cascades wouldn't even properly convey these small objects in their voxel data since they're so large). This means that there's a little more work to be done for the user, but it's the best solution I can think of for now.


    Something else that I've thought about is using the Albedo color (not texture, but constant color) alpha value in Standard shader materials as a "bounce light intensity from this material" property during voxelization. Since this value is pretty much unused (to my knowledge), it could be quite useful for situations like the open structure with the straw roof. The straw roof bounces light downward because the GI voxel data is non-directional (as has been discussed previously). However, it's almost pointless in this particular case to have this material bounce light because there's nothing above it to receive it. So, setting the Albedo color alpha value to zero prevents the straw roof from bouncing any light into the interior, but doesn't affect how the material looks.



    It's just an idea that could help in certain situations where you want to avoid light bleeding. What do you think of this idea?
     
    elias_t, overthere, ftejada and 24 others like this.
  49. Arganth

    Arganth

    Joined:
    Jul 31, 2015
    Posts:
    280
    looks very nice :)
    could you maybe upload a version of this to the asset store? :)
     
  50. Baldinoboy

    Baldinoboy

    Joined:
    Apr 14, 2012
    Posts:
    1,301
    Looks really amazing @sonicether! Thanks for the update!
     
    RB_lashman likes this.