Search Unity

SEGI (Fully Dynamic Global Illumination)

Discussion in 'Assets and Asset Store' started by sonicether, Jun 10, 2016.

  1. Mauri

    Mauri

    Joined:
    Dec 9, 2010
    Posts:
    1,559
    Take all the time you need :) Thank you for your hard work!
     
  2. nxrighthere

    nxrighthere

    Joined:
    Mar 2, 2014
    Posts:
    537
    @redemprez Please make sure that you have correctly applied this patch.
     
    RB_lashman likes this.
  3. redemprez

    redemprez

    Joined:
    Aug 14, 2017
    Posts:
    29
    Sorry for noob question but I'm not a coder - should I just save this RAW file as SEGI.cs in SEGI folder, place SEGI folder in Assets folder of my project and reload Unity?
     
    RB_lashman likes this.
  4. nxrighthere

    nxrighthere

    Joined:
    Mar 2, 2014
    Posts:
    537
    Yes, you can just replace SEGI.cs with a file from the repository if you want.
     
    Last edited: Sep 25, 2017
    RB_lashman likes this.
  5. redemprez

    redemprez

    Joined:
    Aug 14, 2017
    Posts:
    29
    EDIT: @nxrighthere found the source of flickering: It appears only when the scene view and the game view are opened at the same time. Temporary solution: closing one of views. @nxrighthere is working on a fix.

    I downloaded 0.9 release, replaced SEGI.cs with your file, infinite bounces (and without them too) still give me this flicker :/ Vanilla 0.9 works smoothly.

    That's vanilla 0.9 release:


    and the patched version:
     
    Last edited: Sep 23, 2017
    arnoob and RB_lashman like this.
  6. Nexusmaster

    Nexusmaster

    Joined:
    Jun 13, 2015
    Posts:
    53
    Hi guys,

    I have an idea to improve performance, we could try to replace the shadowmap generation by using unity's internal shadowmap using the code form this site: https://docs.unity3d.com/ScriptReference/Rendering.CommandBuffer.SetShadowSamplingMode.html

    I tried to implement the depthmap of the light but I can't find the correct matrix to get it working, any idea why it's not working? Would be cool if somebody else can try it out... nxrighthere or sonic?
    Code (CSharp):
    1.  
    2. //IN SEGICascaded.cs:
    3.    void Start()
    4.     {
    5.         InitCheck();
    6.         InitShadowmapCopy();
    7.     }
    8.  
    9.  
    10.     RenderTexture m_ShadowmapCopy;
    11.  
    12.     void InitShadowmapCopy()
    13.     {
    14.         RenderTargetIdentifier shadowmap = BuiltinRenderTextureType.CurrentActive;
    15.         m_ShadowmapCopy = new RenderTexture(1024, 1024, 0);
    16.         CommandBuffer cb = new CommandBuffer();
    17.  
    18.         // Change shadow sampling mode for sun's shadowmap.
    19.         cb.SetShadowSamplingMode(shadowmap, ShadowSamplingMode.RawDepth);
    20.  
    21.         // The shadowmap values can now be sampled normally - copy it to a different render texture.
    22.         cb.Blit(shadowmap, new RenderTargetIdentifier(m_ShadowmapCopy));
    23.  
    24.         // Execute after the shadowmap has been filled.
    25.         sun.AddCommandBuffer(LightEvent.AfterShadowMap, cb);
    26.  
    27.         // Sampling mode is restored automatically after this command buffer completes, so shadows will render normally.
    28.     }
    29. //....
    30. Shader.SetGlobalTexture("SEGISunDepth", m_ShadowmapCopy);// sunDepthTexture);
    31. //....
    32.  
    33.  
    // IN SEGIVoxelizeScene_C.shader:

    Code (CSharp):
    1.                 float4 getCascadeWeights_splitSpheres(float3 pos)
    2.                 {
    3.                     float3 fromCenter0 = pos - unity_ShadowSplitSpheres[0].xyz;
    4.                     float3 fromCenter1 = pos - unity_ShadowSplitSpheres[1].xyz;
    5.                     float3 fromCenter2 = pos - unity_ShadowSplitSpheres[2].xyz;
    6.                     float3 fromCenter3 = pos - unity_ShadowSplitSpheres[3].xyz;
    7.                     float4 distances2 = float4(dot(fromCenter0, fromCenter0), dot(fromCenter1, fromCenter1), dot(fromCenter2, fromCenter2), dot(fromCenter3, fromCenter3));
    8.                     float4 weights = float4(distances2 >= unity_ShadowSplitSqRadii);
    9.                     return weights;
    10.                 }
    11.  
    12.                 float4 getShadowCoord(float3 pos, float4 cascadeWeights)
    13.                 {
    14.                     return mul(unity_WorldToShadow[(int)dot(cascadeWeights, float4(1, 1, 1, 1))], float4(pos, 1));
    15.                 }
    16.  
    17. //...
    18.  
    19.                     float4 shadowPos = mul(SEGIVoxelProjectionInverse, float4(fcoord * 2.0 - 1.0, 1.0));
    20.                     shadowPos = mul(SEGIVoxelToGIProjection, shadowPos);
    21.                     //shadowPos = mul(unity_WorldToShadow[0], shadowPos);//Didn't work either
    22.                     shadowPos.xyz = shadowPos.xyz * 0.5 + 0.5;
    23.                  
    24.  
    25.                     float4 cascadeWeights = getCascadeWeights_splitSpheres(shadowPos.xyz);
    26.                     shadowPos.xyz = getShadowCoord(shadowPos.xyz, cascadeWeights).xyz;
    27.  
    28.  
    29.                     float sunDepth = tex2Dlod(SEGISunDepth, float4(shadowPos.xy, 0, 0)).x;
    30.                     #if defined(UNITY_REVERSED_Z)
    31.                     sunDepth = 1.0 - sunDepth;
    32.                     #endif
     
    RB_lashman likes this.
  7. redemprez

    redemprez

    Joined:
    Aug 14, 2017
    Posts:
    29
    Ok, @nxrighthere found the source of flickering: It appears only when the scene view and the game view are opened at the same time. temporary solution: closing one of views. @nxrighthere is working on a fix.
     
  8. AndyNeoman

    AndyNeoman

    Joined:
    Sep 28, 2014
    Posts:
    868
    Speaking of shadows. What is the best setup with Segi? No shadow on lights because they are produced by segi? No light probes needed on any objects? Is there any performance benefit for setup changes if you are using segi only with directional lights only (Sun and Moon).
     
    RB_lashman likes this.
  9. redemprez

    redemprez

    Joined:
    Aug 14, 2017
    Posts:
    29
    Ok, small test:


    I like SEGI very much. The only thing that it lacks right now is better quality of reflections + mirror shader support.
     
    Abuthar and RB_lashman like this.
  10. neoshaman

    neoshaman

    Joined:
    Feb 11, 2011
    Posts:
    4,348
     
    RB_lashman likes this.
  11. redemprez

    redemprez

    Joined:
    Aug 14, 2017
    Posts:
    29
    RB_lashman likes this.
  12. Mauri

    Mauri

    Joined:
    Dec 9, 2010
    Posts:
    1,559
    Usually, for clear mirrors, RenderTextures are the way to go. SRR and alike gives blurry reflections only. Even SEGI's reflections aren't clear, but that's intended - otherwise it would be too expensive on the performance side...

    SEGI uses RenderTextures, too - maybe both interfere with each other and that's why you get artifacts?
     
  13. tweedie

    tweedie

    Joined:
    Apr 24, 2013
    Posts:
    272
    I think what's really needed is some form of support for forward rendered objects. (I don't mean having Segi run completely in forward). For instance, alpha blended hair cards are currently a no-go which is a real shame if your game has characters. I'm a long time Segi user and it's fantastic, but for us, this is the only reason we can't really use it, because we have a lot of hair/fur, and cutout won't really do. If the community is going to take it anywhere, after stability updates, I think having these objects able to sample even a rough approximation of the GI would be invaluable. :)
     
    thelebaron, RB_lashman and arnoob like this.
  14. nxrighthere

    nxrighthere

    Joined:
    Mar 2, 2014
    Posts:
    537
    @redemprez A patch for this bug is almost completed. GI is still flickering a bit because the position of the scene view camera conflicts with the position of the main camera. At the moment, I have no idea how to fix this. Manually switching between the scene and the game tabs is the only solution that I have.

    @Nexusmaster I'll take a look at your code and see what I can do.

    @sonicether Cody, source code of SEGI is a goddamn puzzle... :D
     
    Last edited: Sep 24, 2017
  15. nxrighthere

    nxrighthere

    Joined:
    Mar 2, 2014
    Posts:
    537
  16. Shinyclef

    Shinyclef

    Joined:
    Nov 20, 2013
    Posts:
    364
    Wow SEGI is alive again! The dream of realtime GI lives on. This thread is full of energy.

    I'd love to try cascades in the next few days. What kind of performance improvement are people getting with cascades? Anyone got some before and after numbers they want to share?
     
    RB_lashman likes this.
  17. cerrec

    cerrec

    Joined:
    Jan 19, 2017
    Posts:
    26
    Non-programmer here ... Before this thread gets out of hand with excitement, could somebody explain how an artist like me can actually use the new features and fixes people are producing? When somebody posts a link to a page with code, that means absolutely nothing to me. I'm sure there are many more like myself, so please help us along.

    Thanks.
     
    RB_lashman likes this.
  18. buttmatrix

    buttmatrix

    Joined:
    Mar 23, 2015
    Posts:
    609
    SEGI is a .cs script that you attach to the camera, works in deferred, so apply AA from Unity post effects stack (available on Asset Store). The GitHub page contains an extensive ReadMe which should help you along. That's it!
     
    RB_lashman likes this.
  19. cerrec

    cerrec

    Joined:
    Jan 19, 2017
    Posts:
    26
    I know how it works. I've been using it for 6 months. I'm asking how do you use the additions and modifications people are now making.
     
    RB_lashman likes this.
  20. Pode

    Pode

    Joined:
    Nov 13, 2013
    Posts:
    112
    @cerrec : if you go to the github page (https://github.com/sonicether/SEGI) and click on 'Downlaod', you'll get the latest 'up to date' version of the code, with the latest modifications, as a zip file.
     
    RB_lashman likes this.
  21. nxrighthere

    nxrighthere

    Joined:
    Mar 2, 2014
    Posts:
    537
    @Shinyclef On my GTX 950, a difference between them with the almost identical settings is about ~2.5 ms of GPU time in favor of the cascaded version. But don't forget that the SEGI cascaded is still in the WIP, and currently doesn't support reflections.

    @cerrec You can just download the master branch since most of the recent changes already merged with it. If you want to apply changes to your local files from an open pull request, you need to learn how to use git.
    git clone https://github.com/sonicether/SEGI.git
    cd SEGI
    git ls-remote origin
    git pull origin pull/<pull-request-id>/head
     
    Last edited: Sep 25, 2017
  22. Yuki-Taiyo

    Yuki-Taiyo

    Joined:
    Jun 7, 2016
    Posts:
    72
    I just tried SEGI some days ago on a small interior scene with emissive material lights... and I must say it is not really what I expected. There's a lot of noise and bleeding going on ; half-resolution setting is ugly. After that, I may have to further experiment with it. I think it should be used on Low settings along some basic point lights for interior scenes.

    I think I would rather use the asset called NGSS, Next-Gen Soft Shadows, and place my lights inside and beyond the lamp meshes to simulate an area effect + some Ambient Occlusion solution to fake the GI maybe.
     
    Last edited: Sep 25, 2017
    RB_lashman likes this.
  23. Nexusmaster

    Nexusmaster

    Joined:
    Jun 13, 2015
    Posts:
    53
    guy_, arnoob, nxrighthere and 2 others like this.
  24. makeshiftwings

    makeshiftwings

    Joined:
    May 28, 2011
    Posts:
    3,298
    For people having issues with forward rendered objects: I've found most problems I've had with them can be solved by just putting them on a layer that is not included in your SEGI layer mask. I had to put most of my particle effects on separate layers because for some reason they would get skewed if they were on a SEGI layer. Putting them on a non-SEGI layer allows them to still receive SEGI light, they just won't produce SEGI emissiveness or bounce reflections.
     
    RB_lashman, arnoob and hopeful like this.
  25. thelebaron

    thelebaron

    Joined:
    Jun 2, 2013
    Posts:
    299
    The two arent mutually exclusive, and look quite beautiful together. But yeah there's a fair amount of tweaking necessary as well as some weird mesh problems with light leaks that can occur.
     
    RB_lashman and arnoob like this.
  26. arnoob

    arnoob

    Joined:
    May 16, 2014
    Posts:
    151
    Thanks for the input, I'll try that out!

    By the way is there still the problem with particles not facing camera? I know we can put it in different layer, but that still is a really rough workaround, knowing how the number of layers is limited in unity... Maybe one of the shader wizard could find what is creating this artefact in the code?

    Anyway, ones again, it's really great to see everyone working together for this asset. As an artist, I can't do much code wise, but I plan to do a quick guide through testing to show the good practices to have, mesh and setup wise, in order to hide the flaws of the shader. I still think that the modelers/level designer have to work to get the best of segi, not simply waiting that the code get flawless.
     
    RB_lashman likes this.
  27. makeshiftwings

    makeshiftwings

    Joined:
    May 28, 2011
    Posts:
    3,298
    Particles do still get skewed in the latest version, but I feel like it's better to have them on a non-SEGI layer anyway. Particle effects usually move quickly and are often temporary, so it's pointless to waste cycles trying to include them into Global Illumination. You can always just add a point light or even a GI-only emissive cube with a matching color if you really need the lighting around particles to be perfect. But given that particles themselves are usually there because you want a fast, low-impact way to render a ton of small low-quality objects, it would seem weird to really need perfect reflection from them anyway.
     
    arnoob and RB_lashman like this.
  28. nxrighthere

    nxrighthere

    Joined:
    Mar 2, 2014
    Posts:
    537
    @Nexusmaster I'm glad to hear that you found a solution. I still focused on the stability and bug fixes, but I have some ideas how to improve the overall SEGI performance.
     
    Last edited: Sep 26, 2017
  29. arnoob

    arnoob

    Joined:
    May 16, 2014
    Posts:
    151

    Actually this is more about the workaround technique. In my case, I am doing a 4 player split screen game. Therefore, I already need 8 layers for each camera setup alone. In order to have the particles behave correctly, I would need to add 5 new layers (1 for each player + one for the common world). Knowing that we are limited to 32 layers, you can easily see how it can become a problem...
     
    RB_lashman likes this.
  30. AndyNeoman

    AndyNeoman

    Joined:
    Sep 28, 2014
    Posts:
    868
    Maybe its because I've not had enough coffee yet so forgive me if that is the case but why do you need each particle on a separate layer?

    If it is to do with your game then it's not segi constricted or if it is just for segi masking purposes why don't you just have a particle layer, and mask from gi?

    Little things really do not need to be included in GI calculations.
     
    arnoob and RB_lashman like this.
  31. arnoob

    arnoob

    Joined:
    May 16, 2014
    Posts:
    151
    Don't worry I was probably not very clear :)

    I am doing a FPS. In order to make it look "good", the standard setup is to have two models for each player : 1 FPS and 1 TPS. During the game, we put each of the models on its own layer. We show the FPS hands to the player, and hide its body; while showing its body to other players and hiding the FPS part to them.
    So the problem is : if I need to have particles on the players, I need to show 2 sets of particles : one for the TPS and one for the FPS. With the current workaround, it therefore force to use 2 new layers per players!

    I know that I could be able to simply remove any particles from the players, but (sic) that would be a real bummer. That said, I know that a very few people will do that kind of setup, so I can cope with it for my own project.

    Anyway, I still think that there should be a way to fix that in the shader, and that maybe if we wait longer it will become harder and harder to remove, and maybe will this exception create some new artefacts on the future, that's why I thought I had to bring that up, now that there are a lot of people working on the shader.

    I hope I was clearer here, and by the way if any of you have some troubles setting up a FPS game and need some insight, just PM me! :)
     

    Attached Files:

    AndyNeoman and RB_lashman like this.
  32. BruceBarratt

    BruceBarratt

    Joined:
    Jun 20, 2015
    Posts:
    46
    Why 2 new layers per player? Surely each player just needs to know which character it is and change its layers accordingly at the start of the game. What's different about particles that every different player needs a new layer (or two)?.
     
    RB_lashman and arnoob like this.
  33. arnoob

    arnoob

    Joined:
    May 16, 2014
    Posts:
    151
    One for the FPS body (wich receives SEGI) and one for the FPS particles (wich musn't receive SEGI with this workaround). Same for the TPS, for each players. If the particle facing bug was fixed, we could put the body and the particles in the same layer.
     
    RB_lashman likes this.
  34. BruceBarratt

    BruceBarratt

    Joined:
    Jun 20, 2015
    Posts:
    46
    OK. So we are clear it's not like this is going to keep growing with the number of players. I thought that was what you were saying. So you would have 4 layers right? No matter how many players there are. FPS player, TPS player, FPS particles, TPS particles. What's the max number of layers? Isn't it 32? Should have plenty to spare?
     
    RB_lashman likes this.
  35. BruceBarratt

    BruceBarratt

    Joined:
    Jun 20, 2015
    Posts:
    46
    I mean the problem is the idea of assigning a new layer to each player when they should all share one TPS player layer and one TPS particle layer. And which character is on FPS layers should be set in code.
     
    RB_lashman likes this.
  36. BruceBarratt

    BruceBarratt

    Joined:
    Jun 20, 2015
    Posts:
    46
    Sorry I hadn't read up to the original part. About it being split-screen. SEGI doesn't work on multiple cameras does it? How can you do splitscreen?
     
    arnoob and RB_lashman like this.
  37. arnoob

    arnoob

    Joined:
    May 16, 2014
    Posts:
    151
    Np :)

    Last time I checked, it did work on multiple camera. The inconvenient being that it had to run for each of them (being pretty intensive). I may be wrong that's what I believed I saw o the forum...
     
    RB_lashman likes this.
  38. Nexusmaster

    Nexusmaster

    Joined:
    Jun 13, 2015
    Posts:
    53
    Hi, so the unity shadows are working as expected, but the problem is that unity only renders visible objects of the main camera into the shadowmap, this can lead to wrong GI lighting, so I will try another approach, but the performance gain was good, I got around 0.5 - 1.0 ms less (depending on the scene complexity).

    btw.: a dev thread for the new cascaded segi would be nice?
     
    RB_lashman, AlanCarter and arnoob like this.
  39. arnoob

    arnoob

    Joined:
    May 16, 2014
    Posts:
    151
    It might not be as precise as the old one, but it's still a good way to gain performances. Thank you a lot for digging into that! Shouldn't we keep that as an option for low end computers? Like a "medium" quality level?
     
    RB_lashman likes this.
  40. AlanCarter

    AlanCarter

    Joined:
    Sep 26, 2017
    Posts:
    3
    Ah interesting! Is the incorrect GI quite noticeable? It still may be an option worth exposing.

    Do you think it would be possible to do the inverse, and have the Unity directional light utilise SEGIs shadowmap?

    I get a considerable performance increase when disabling Unity shadows in my project, but of course it looks awful without them. I understand SEGI's shadow map would not be anywhere near as sharp and detailed as Unitys, but in my particular case (and hopefully others) it could be a worthy trade off for such an increase in performance, if it's even possible that is!
     
    RB_lashman likes this.
  41. arnoob

    arnoob

    Joined:
    May 16, 2014
    Posts:
    151
    Shadows are usually pretty cheap (compared to SEGI I mean). How many shadow casters does your stats show?
     
    RB_lashman likes this.
  42. AlanCarter

    AlanCarter

    Joined:
    Sep 26, 2017
    Posts:
    3
    About 600 shadow casters. I can optimize that for sure, but Unity shadows appear to affect my performance much more than SEGI shadows. The way I tested was to reduce SEGI Shadow Space Size to 0. I assumed that would disable the SEGI shadow map or at least lower the amount of work it was doing significantly. Perhaps I am wrong about that.
     
    RB_lashman likes this.
  43. Nexusmaster

    Nexusmaster

    Joined:
    Jun 13, 2015
    Posts:
    53
    It's noticeable, in the way that it gets darker (or lighter) from one frame to another, when the shadowcaster is no longer in the unity-shadowmap.
    Well the performance difference really depends on how you draw your meshes... SEGI draws all the meshes that need to be in the voxel area as shadow casters, in my approach there were no shadow casters drawn, because unity did it already. I think an alternative would be to draw lod meshes instead of the real once or use a volume shadow ray caster for everything.
     
    Last edited: Sep 27, 2017
    RB_lashman likes this.
  44. AlanCarter

    AlanCarter

    Joined:
    Sep 26, 2017
    Posts:
    3
    Ah ok, thanks for the answer!
     
    RB_lashman likes this.
  45. tweedie

    tweedie

    Joined:
    Apr 24, 2013
    Posts:
    272
    Unfortunately I'm pretty sure this isn't the case. Forward rendered objects don't receive GI. As far as I'm aware, the particle skewing you're talking about isn't because they're forward rendered, but because the voxelisation messes with the bilboarding, hence the need to put them on a separate layer, excluding them from being voxelised. Here's a test where the sphere is using the Standard Shader in Transparent mode, then again in Opaque; note how the Transparent version receives none of the orange bounce light.

    Transparent:



    Opaque:

     
    RB_lashman and arnoob like this.
  46. arnoob

    arnoob

    Joined:
    May 16, 2014
    Posts:
    151
    I'm in the same boat as you...

    But sadly, I don't think it will be possible from SEGI...
    As you can see, the transparent object doesn't receive the shadows themselves. I've been scratching my head a long time on this one.
    Transparent shaders can't get complex lighting informations due to the deferred pipeline.
    I'm pretty sure that in order to have it, we would first need to have transparent receiving shadows, the rest would work by itself.

    Maybe if we had a mean to read what is the segi contribution at a given point (for exemple if the SEGI calculation was stored in an accessible texture3D I guess) we could use it to feed the transparent shader, but that's about it...
     
    Last edited: Sep 26, 2017
    RB_lashman likes this.
  47. Monkeybro

    Monkeybro

    Joined:
    Sep 15, 2017
    Posts:
    5
    I just discovered this and it looks fantastic. I'm doing some VR testing (haven't been able to try SEGI on the vive yet) and I'm wondering what "undefined VR behaviour" means?
     
    RB_lashman likes this.
  48. Ninlilizi

    Ninlilizi

    Joined:
    Sep 19, 2016
    Posts:
    271
    My experience of having tried it with VR a number of months back was no support for single pass. And a higher than expected performance hit..... Like 30fps in VR vs 120fps without.
     
    RB_lashman likes this.
  49. Monkeybro

    Monkeybro

    Joined:
    Sep 15, 2017
    Posts:
    5
    Aha. Really hope that can be fixed in the future. Thanks.
     
    RB_lashman likes this.
  50. Nexusmaster

    Nexusmaster

    Joined:
    Jun 13, 2015
    Posts:
    53
    Hi, I added a VolumeRayCasting System (just for experimental reasons) and UnityShadowMap as an additional shadow caster, also converted the RG0 texture to 2D: https://github.com/CK85/SEGI
     

    Attached Files: