SEGI (Fully Dynamic Global Illumination)

@redemprez Please make sure that you have correctly applied this patch.

 

Sorry for noob question but I'm not a coder - should I just save this RAW file as SEGI.cs in SEGI folder, place SEGI folder in Assets folder of my project and reload Unity?

 

Yes, you can just replace SEGI.cs with a file from the repository if you want.

 

EDIT: @nxrighthere found the source of flickering: It appears only when the scene view and the game view are opened at the same time. Temporary solution: closing one of views. @nxrighthere is working on a fix.

I downloaded 0.9 release, replaced SEGI.cs with your file, infinite bounces (and without them too) still give me this flicker :/ Vanilla 0.9 works smoothly.

That's vanilla 0.9 release:

and the patched version:

 

Hi guys,

I have an idea to improve performance, we could try to replace the shadowmap generation by using unity's internal shadowmap using the code form this site: https://docs.unity3d.com/ScriptReference/Rendering.CommandBuffer.SetShadowSamplingMode.html

I tried to implement the depthmap of the light but I can't find the correct matrix to get it working, any idea why it's not working? Would be cool if somebody else can try it out... nxrighthere or sonic?

Code (CSharp):

1.  
2. //IN SEGICascaded.cs:
3.    void Start()
4.     {
5.         InitCheck();
6.         InitShadowmapCopy();
7.     }
8.  
9.  
10.     RenderTexture m_ShadowmapCopy;
11.  
12.     void InitShadowmapCopy()
13.     {
14.         RenderTargetIdentifier shadowmap = BuiltinRenderTextureType.CurrentActive;
15.         m_ShadowmapCopy = new RenderTexture(1024, 1024, 0);
16.         CommandBuffer cb = new CommandBuffer();
17.  
18.         // Change shadow sampling mode for sun's shadowmap.
19.         cb.SetShadowSamplingMode(shadowmap, ShadowSamplingMode.RawDepth);
20.  
21.         // The shadowmap values can now be sampled normally - copy it to a different render texture.
22.         cb.Blit(shadowmap, new RenderTargetIdentifier(m_ShadowmapCopy));
23.  
24.         // Execute after the shadowmap has been filled.
25.         sun.AddCommandBuffer(LightEvent.AfterShadowMap, cb);
26.  
27.         // Sampling mode is restored automatically after this command buffer completes, so shadows will render normally.
28.     }
29. //....
30. Shader.SetGlobalTexture("SEGISunDepth", m_ShadowmapCopy);// sunDepthTexture);
31. //....
32.  
33.  

// IN SEGIVoxelizeScene_C.shader:

Code (CSharp):

1.                 float4 getCascadeWeights_splitSpheres(float3 pos)
2.                 {
3.                     float3 fromCenter0 = pos - unity_ShadowSplitSpheres[0].xyz;
4.                     float3 fromCenter1 = pos - unity_ShadowSplitSpheres[1].xyz;
5.                     float3 fromCenter2 = pos - unity_ShadowSplitSpheres[2].xyz;
6.                     float3 fromCenter3 = pos - unity_ShadowSplitSpheres[3].xyz;
7.                     float4 distances2 = float4(dot(fromCenter0, fromCenter0), dot(fromCenter1, fromCenter1), dot(fromCenter2, fromCenter2), dot(fromCenter3, fromCenter3));
8.                     float4 weights = float4(distances2 >= unity_ShadowSplitSqRadii);
9.                     return weights;
10.                 }
11.  
12.                 float4 getShadowCoord(float3 pos, float4 cascadeWeights)
13.                 {
14.                     return mul(unity_WorldToShadow[(int)dot(cascadeWeights, float4(1, 1, 1, 1))], float4(pos, 1));
15.                 }
16.  
17. //...
18.  
19.                     float4 shadowPos = mul(SEGIVoxelProjectionInverse, float4(fcoord * 2.0 - 1.0, 1.0));
20.                     shadowPos = mul(SEGIVoxelToGIProjection, shadowPos);
21.                     //shadowPos = mul(unity_WorldToShadow[0], shadowPos);//Didn't work either
22.                     shadowPos.xyz = shadowPos.xyz * 0.5 + 0.5;
23.                  
24.  
25.                     float4 cascadeWeights = getCascadeWeights_splitSpheres(shadowPos.xyz);
26.                     shadowPos.xyz = getShadowCoord(shadowPos.xyz, cascadeWeights).xyz;
27.  
28.  
29.                     float sunDepth = tex2Dlod(SEGISunDepth, float4(shadowPos.xy, 0, 0)).x;
30.                     #if defined(UNITY_REVERSED_Z)
31.                     sunDepth = 1.0 - sunDepth;
32.                     #endif

 

Ok, @nxrighthere found the source of flickering: It appears only when the scene view and the game view are opened at the same time. temporary solution: closing one of views. @nxrighthere is working on a fix.

 

Speaking of shadows. What is the best setup with Segi? No shadow on lights because they are produced by segi? No light probes needed on any objects? Is there any performance benefit for setup changes if you are using segi only with directional lights only (Sun and Moon).

 

Ok, small test:

I like SEGI very much. The only thing that it lacks right now is better quality of reflections + mirror shader support.

 

Thanks for answer. Can you give me solutions for mirror-like reflection with SEGI? I tried to combine it with Mirror shader (from here: http://wiki.unity3d.com/index.php/MirrorReflection4) script but it is not working well - it gives artifacts.

 

Usually, for clear mirrors, RenderTextures are the way to go. SRR and alike gives blurry reflections only. Even SEGI's reflections aren't clear, but that's intended - otherwise it would be too expensive on the performance side...

SEGI uses RenderTextures, too - maybe both interfere with each other and that's why you get artifacts?

 

I think what's really needed is some form of support for forward rendered objects. (I don't mean having Segi run completely in forward). For instance, alpha blended hair cards are currently a no-go which is a real shame if your game has characters. I'm a long time Segi user and it's fantastic, but for us, this is the only reason we can't really use it, because we have a lot of hair/fur, and cutout won't really do. If the community is going to take it anywhere, after stability updates, I think having these objects able to sample even a rough approximation of the GI would be invaluable.

 

@redemprez A patch for this bug is almost completed. GI is still flickering a bit because the position of the scene view camera conflicts with the position of the main camera. At the moment, I have no idea how to fix this. Manually switching between the scene and the game tabs is the only solution that I have.

@Nexusmaster I'll take a look at your code and see what I can do.

@sonicether Cody, source code of SEGI is a goddamn puzzle...

 

SEGI cascaded shouldn't crash anymore for most people. Thanks to @BruceJRBarratt and @Mauri for testing an experimental version.

 

Wow SEGI is alive again! The dream of realtime GI lives on. This thread is full of energy.

I'd love to try cascades in the next few days. What kind of performance improvement are people getting with cascades? Anyone got some before and after numbers they want to share?

 

Non-programmer here ... Before this thread gets out of hand with excitement, could somebody explain how an artist like me can actually use the new features and fixes people are producing? When somebody posts a link to a page with code, that means absolutely nothing to me. I'm sure there are many more like myself, so please help us along.

Thanks.

 

SEGI is a .cs script that you attach to the camera, works in deferred, so apply AA from Unity post effects stack (available on Asset Store). The GitHub page contains an extensive ReadMe which should help you along. That's it!

 

I know how it works. I've been using it for 6 months. I'm asking how do you use the additions and modifications people are now making.

 

@cerrec : if you go to the github page (https://github.com/sonicether/SEGI) and click on 'Downlaod', you'll get the latest 'up to date' version of the code, with the latest modifications, as a zip file.

 

@Shinyclef On my GTX 950, a difference between them with the almost identical settings is about ~2.5 ms of GPU time in favor of the cascaded version. But don't forget that the SEGI cascaded is still in the WIP, and currently doesn't support reflections.

@cerrec You can just download the master branch since most of the recent changes already merged with it. If you want to apply changes to your local files from an open pull request, you need to learn how to use git.

Spoiler: Example

git clone https://github.com/sonicether/SEGI.git
cd SEGI
git ls-remote origin
git pull origin pull/<pull-request-id>/head

 

I just tried SEGI some days ago on a small interior scene with emissive material lights... and I must say it is not really what I expected. There's a lot of noise and bleeding going on ; half-resolution setting is ugly. After that, I may have to further experiment with it. I think it should be used on Low settings along some basic point lights for interior scenes.

I think I would rather use the asset called NGSS, Next-Gen Soft Shadows, and place my lights inside and beyond the lamp meshes to simulate an area effect + some Ambient Occlusion solution to fake the GI maybe.

 

@nxrighthere: I got the unity shadow input working, I just have to configurate it! This helped me a lot:
https://github.com/TheMasonX/UnityPCSS

 

For people having issues with forward rendered objects: I've found most problems I've had with them can be solved by just putting them on a layer that is not included in your SEGI layer mask. I had to put most of my particle effects on separate layers because for some reason they would get skewed if they were on a SEGI layer. Putting them on a non-SEGI layer allows them to still receive SEGI light, they just won't produce SEGI emissiveness or bounce reflections.

 

The two arent mutually exclusive, and look quite beautiful together. But yeah there's a fair amount of tweaking necessary as well as some weird mesh problems with light leaks that can occur.

 

Thanks for the input, I'll try that out!

By the way is there still the problem with particles not facing camera? I know we can put it in different layer, but that still is a really rough workaround, knowing how the number of layers is limited in unity... Maybe one of the shader wizard could find what is creating this artefact in the code?

Anyway, ones again, it's really great to see everyone working together for this asset. As an artist, I can't do much code wise, but I plan to do a quick guide through testing to show the good practices to have, mesh and setup wise, in order to hide the flaws of the shader. I still think that the modelers/level designer have to work to get the best of segi, not simply waiting that the code get flawless.

 

Particles do still get skewed in the latest version, but I feel like it's better to have them on a non-SEGI layer anyway. Particle effects usually move quickly and are often temporary, so it's pointless to waste cycles trying to include them into Global Illumination. You can always just add a point light or even a GI-only emissive cube with a matching color if you really need the lighting around particles to be perfect. But given that particles themselves are usually there because you want a fast, low-impact way to render a ton of small low-quality objects, it would seem weird to really need perfect reflection from them anyway.

 

@Nexusmaster I'm glad to hear that you found a solution. I still focused on the stability and bug fixes, but I have some ideas how to improve the overall SEGI performance.

 

Actually this is more about the workaround technique. In my case, I am doing a 4 player split screen game. Therefore, I already need 8 layers for each camera setup alone. In order to have the particles behave correctly, I would need to add 5 new layers (1 for each player + one for the common world). Knowing that we are limited to 32 layers, you can easily see how it can become a problem...

 

Maybe its because I've not had enough coffee yet so forgive me if that is the case but why do you need each particle on a separate layer?

If it is to do with your game then it's not segi constricted or if it is just for segi masking purposes why don't you just have a particle layer, and mask from gi?

Little things really do not need to be included in GI calculations.

 

Don't worry I was probably not very clear

I am doing a FPS. In order to make it look "good", the standard setup is to have two models for each player : 1 FPS and 1 TPS. During the game, we put each of the models on its own layer. We show the FPS hands to the player, and hide its body; while showing its body to other players and hiding the FPS part to them.
So the problem is : if I need to have particles on the players, I need to show 2 sets of particles : one for the TPS and one for the FPS. With the current workaround, it therefore force to use 2 new layers per players!

I know that I could be able to simply remove any particles from the players, but (sic) that would be a real bummer. That said, I know that a very few people will do that kind of setup, so I can cope with it for my own project.

Anyway, I still think that there should be a way to fix that in the shader, and that maybe if we wait longer it will become harder and harder to remove, and maybe will this exception create some new artefacts on the future, that's why I thought I had to bring that up, now that there are a lot of people working on the shader.

I hope I was clearer here, and by the way if any of you have some troubles setting up a FPS game and need some insight, just PM me!

 

Why 2 new layers per player? Surely each player just needs to know which character it is and change its layers accordingly at the start of the game. What's different about particles that every different player needs a new layer (or two)?.

 

One for the FPS body (wich receives SEGI) and one for the FPS particles (wich musn't receive SEGI with this workaround). Same for the TPS, for each players. If the particle facing bug was fixed, we could put the body and the particles in the same layer.

 

OK. So we are clear it's not like this is going to keep growing with the number of players. I thought that was what you were saying. So you would have 4 layers right? No matter how many players there are. FPS player, TPS player, FPS particles, TPS particles. What's the max number of layers? Isn't it 32? Should have plenty to spare?

 

I mean the problem is the idea of assigning a new layer to each player when they should all share one TPS player layer and one TPS particle layer. And which character is on FPS layers should be set in code.

 

Sorry I hadn't read up to the original part. About it being split-screen. SEGI doesn't work on multiple cameras does it? How can you do splitscreen?

 

Np

Last time I checked, it did work on multiple camera. The inconvenient being that it had to run for each of them (being pretty intensive). I may be wrong that's what I believed I saw o the forum...

 

Hi, so the unity shadows are working as expected, but the problem is that unity only renders visible objects of the main camera into the shadowmap, this can lead to wrong GI lighting, so I will try another approach, but the performance gain was good, I got around 0.5 - 1.0 ms less (depending on the scene complexity).

btw.: a dev thread for the new cascaded segi would be nice?

 

It might not be as precise as the old one, but it's still a good way to gain performances. Thank you a lot for digging into that! Shouldn't we keep that as an option for low end computers? Like a "medium" quality level?

 

Ah interesting! Is the incorrect GI quite noticeable? It still may be an option worth exposing.

Do you think it would be possible to do the inverse, and have the Unity directional light utilise SEGIs shadowmap?

I get a considerable performance increase when disabling Unity shadows in my project, but of course it looks awful without them. I understand SEGI's shadow map would not be anywhere near as sharp and detailed as Unitys, but in my particular case (and hopefully others) it could be a worthy trade off for such an increase in performance, if it's even possible that is!

 

Shadows are usually pretty cheap (compared to SEGI I mean). How many shadow casters does your stats show?

 

About 600 shadow casters. I can optimize that for sure, but Unity shadows appear to affect my performance much more than SEGI shadows. The way I tested was to reduce SEGI Shadow Space Size to 0. I assumed that would disable the SEGI shadow map or at least lower the amount of work it was doing significantly. Perhaps I am wrong about that.

 

It's noticeable, in the way that it gets darker (or lighter) from one frame to another, when the shadowcaster is no longer in the unity-shadowmap.
Well the performance difference really depends on how you draw your meshes... SEGI draws all the meshes that need to be in the voxel area as shadow casters, in my approach there were no shadow casters drawn, because unity did it already. I think an alternative would be to draw lod meshes instead of the real once or use a volume shadow ray caster for everything.

 

Ah ok, thanks for the answer!

 

Unfortunately I'm pretty sure this isn't the case. Forward rendered objects don't receive GI. As far as I'm aware, the particle skewing you're talking about isn't because they're forward rendered, but because the voxelisation messes with the bilboarding, hence the need to put them on a separate layer, excluding them from being voxelised. Here's a test where the sphere is using the Standard Shader in Transparent mode, then again in Opaque; note how the Transparent version receives none of the orange bounce light.

Transparent:



Opaque:

 

I'm in the same boat as you...

But sadly, I don't think it will be possible from SEGI...
As you can see, the transparent object doesn't receive the shadows themselves. I've been scratching my head a long time on this one.
Transparent shaders can't get complex lighting informations due to the deferred pipeline.
I'm pretty sure that in order to have it, we would first need to have transparent receiving shadows, the rest would work by itself.

Maybe if we had a mean to read what is the segi contribution at a given point (for exemple if the SEGI calculation was stored in an accessible texture3D I guess) we could use it to feed the transparent shader, but that's about it...

 

I just discovered this and it looks fantastic. I'm doing some VR testing (haven't been able to try SEGI on the vive yet) and I'm wondering what "undefined VR behaviour" means?

 

My experience of having tried it with VR a number of months back was no support for single pass. And a higher than expected performance hit..... Like 30fps in VR vs 120fps without.

 

Aha. Really hope that can be fixed in the future. Thanks.

 

Hi, I added a VolumeRayCasting System (just for experimental reasons) and UnityShadowMap as an additional shadow caster, also converted the RG0 texture to 2D: https://github.com/CK85/SEGI

 

@Nexusmaster It's broken as hell.