SEGI (Fully Dynamic Global Illumination)

Hey @Ninlilizi

this check ( in the Render function )

Code (CSharp):

1. if (previousGIResult.width != context.width || previousGIResult.height != context.height)
2. {
3.       Debug.Log("<SEGI> previousGIResult != Expected Dimensions. Resizing Render Textures");
4.       ResizeAllTextures();
5. }

returns 540(for previousGIResult.width), 512(for context.width), 512(for previousGIResult.height), 512(for context.height)

This triggers constant texture resizing which is really slow on my machine - may be a bug !?

 

Looking at that pic it also looks like the lighting is moved up 1 meter everywhere. See the 2 highlights above the arches on 1st floor at other end of the atrium, see the entire floor being brighter for the bottom 50-100 cm.

 

Don't add the script to the camera... Add it as a pp2 effect instead.

 

Also, yeah... Constant resizing would be something gone wrong. Weird

 

Yeah, the voxel coordinates for the cached rays are still a little off. the x/y is fine (I think)... Just that damned z has been massively challenging to figure out some workable math for. It gets more inaccurate the further away you get. I spent 10 hours repeatedly banging my head against it over the last day.

 

How much VRAM do you have?

 

also, what happens when you comment out that sanity check?

It's a legacy check from SEs base... I question if it's even necessary to retain with my current version of code. But left it in encase removing it broke something I'd not anticipated.

 

Tackling the voxelisation is on my radar... But more research needed. The sample caching came from a few weeks of reading and learning alone. I'd probably divide the voxel volume in 16 slices. And spread them over as many frames. But will have to cache the intermediary data a different way... As it stands the path trace cache volumes eat 3.2GB VRAM... Which definitely needs whittling down A LOT.

 

750Ti 2GB ram

 

If this question is related to my case: - i think it started to flicker - apparently there is a need to resize.

 

Ah, that'll be the problem... The cache volumes currently use more VRAM than you have... I'm working on bringing it down

 

Good job! That's quite the achievement!

Hadn't even have the time to look at it lol and I still need to do the final vanilla code commentary

 

Today so far... I've improved the lighting accuracy some more.
Fixed the VR depth buffer error bug.
Reduced the overall memory of the path tracer cache volume a little.
And most importantly, added a selector to adjust the path tracer cache volume resolution. Turning it right down will make it possible to run even on systems with as little as 2GB VRAM.

No idea if that fixes the exact bug encountered above.... But it's a step towards it till I can figure more closely whats going wrong.

edit:
Emissives appear to currently be broken. I'm aware of that.

 

Wow what an insane boost in performance.

How to use the cascades? how do they work?

 

Cascades are automatic. They 'just work' in so much as they're voxelized across 6 frames and then traced across as if they were a single large volume.

 

As far as I know, Voxel space size works like this: setting a high voxel space size gives you more range but less preccise detail, low voxel space size gives low range but high detail. I thought cascades were meant to be able to have a low voxel size at close range and have it increase with range, removing that trade-off limitation. So if they are enabled by default and for example I set a voxel space size of like 100, and I seeing a lower real voxel size near me? how does it work?

 

The voxel space size setting is the total size including cascades. cascade zero is at the centre. Then the others project out around it. When voxelizing. It moves the grid to relocate the center to the current transform of the camera. I don't think there's any LOD work happening during the process.

 

As requested (Just fixed emissives) ... The cyan near the ground is the area around a tiny emissive spot on the thing it's around.

Also, I just changed the reflection logic to only trace reflection for the first 50% of the voxel space size (where you can actually see them) to boost performance a little there.

 

Oh sweet. I'm gonna give it another go then.

 

I don't know if I'm doing something wrong, but I can't get the lighting to be at all stable.
Different kinds of lighting and artefacts keep popping in out.

Without changing the settings, and only moving the camera back and forth in the low poly scene, here are some of different states the scene can be in.

 

Also in the sponza scene, without moving the camera at all, here the voxel visualization is stable, but the GI visulization seems to be doing some weird stuff as it cycles through cascades (I guess).

 

Oh damn.. I thought I'd fixed that one.... Thanx for the report... Back to the drawing board.

 

On the bright side, it worked better for you than the last time.

In other news. For the VR buffs here... If anyone's using NVIDIA VRWorks in their projects. There's some highly experimental VRWorks support in there. It might explode and curdle all the milk in your refrigerator. Or it might work. But if it works. It reduces the GPU load of the path tracer by 15% in testing... Just add 'VRWORKS' to your platform defines and the option to enable it will magically appear in the inspector.

 

Are proper shadows cast from objects now?

 

Shadows are as of a few days ago encoded into the voxel data. But only for the main directional light.. I had that disabled for a while why figuring something else out... but they should be there now.

Here, shadows:

 

I have a 960 4gig of ram vga card but It's impossible to use SEGI in 4k! too expensive. Can someone make it a bit more optimize?

 

It's not very realistic to run any game at 4k on a 960, let alone one with real-time, dynamic GI.

 

But I can play Sleeping Dogs 4k on my 960 on 50-60 fps!!!
It's an open world game.

 

btw... What software did you use to create these gifs?
Been working on some new cache update logic to fix the stability issue.... And got it staggered updating and then continuing to converge over successes cycles if you stay in the same place for a bit.... Basically... mixes the samples with the previous cache content while updating and skips any dirty cache locations that contain bad data.
Would be useful to show how it works.

 

The reason you can't is not due to the performance of the card... It's that it doesn't have enough VRAM.... Running SEGI at that resolution along with breathing room for the needs of the games itself your gonna need about 8GB VRAM.

I can do 2k with 6GB VRAM on a GTX1060.... For a comfortable watermark.

4k is challenging... Not because it's hard to heavily optimize the thing... But that its a delicate triage between performance and VRAM hunger.

tl;dr.... I can make it faster... But at the expensive to exponential VRAM usage.

 

the inverted version looks better but still feels like tehre is something odd going on in that lighting.

 

I just took some screenshots and then made the gif at https://ezgif.com/maker.
Upload images, choose transition time, optimise if you want, done.

 

Awesome... Right.... Another long day of banging my head against random hard objects, and I think I've fixed the problem you had... Also made massive changes and improvements to the path trace cache logic.

Repos just been updated.

I've removed the Cones slider from the inspector.... As it now always runs at one cone per frame, which when run thru the cache gives a 64 cone result. It flips the buffer every 96 frames. Which is often enough the lighting should keep up with the environment. Though I guess I should add a function to cause an immediate flip if someone wants to programaticaly demand it happen NOW. Still not done with this part of the code. But hopefully it's a good step towards a usable end result ^_^

Screen shot of 1 sample per frame result

I know... And the closer I get to a decent result. The harder it becomes to define what exactly is still off.

 

I'll take another look in a few hours .

 

Take a canonically correct screenshot then your version with identical camera settings and do an image diff?

 

It got 1000x better since yesterday. But there is still heaps of light popping and instability. I made a video to show what I'm seeing in the low poly scene. The light pops are most evident when using the visualise GI debug setting, looking at the emissive crystal in the cave, and also near at the end of video as light pops back in.
https://drive.google.com/open?id=1yHRLuZmP5ooLboFSIqzfmOlk5M2j584p

I'm sort of guessing a lot of this instability is because of the cascades? Is there a way to turn cascades off to compare? Or are cascades pretty much baked into the solution at this point?

 

I also noticed a very large amount of graininess on objects close to the camera. Again, this is with Visualize GI.

 

Morning.

Afraid there is no easy way to turn off cascades... They touch the code all over the place and making them togglable will actually be a lot of work. That's how it was designed in the SE base. The popping is happening with the voxels themselves rather than my changes. I'll have a play with the original source later, see if it's present there too and not something I've introduced. Though I've not touched the voxelisation code myself.

The grainyness close to the cam. That is something to the math that calculates the cache coordinates. Something much more straightforward to adjust. So I guess I'll probs tackle that first. It looks like this at this point:

voxelCoord = float3(voxelOrigin.x + kernel.x - blueNoise.x * SEGITraceCacheScaleFactor, voxelOrigin.y + kernel.y - blueNoise.y * SEGITraceCacheScaleFactor, depth * voxelDepth - blueNoise.z * SEGITraceCacheScaleFactor);

Thanks for your help btw... Your doing a service for us all

 

Have a look in the vid when the camera is standing still next to the emissive gem. That flickering looks similar to what you fixed with the sunlight earlier. It's not voxelisation at fault there.

 

I think there's actually 2 things going wrong here... But regarding the near range noise... Should be fixed now... Give it a try

I'm getting pretty excited at this stage... Gone from never looked at a shader to massively improving on both performance and image quality of SEGI in less than a year ^_^

 

For those wandering what the staggered cach tracing code looks like at this point

Staggered updating:

Code (CSharp):

1. [numthreads(16, 16, 1)]
2. void CSMain4(uint3 id : SV_DispatchThreadID)
3. {
4.     uint slice = Resolution / 16;
5.     uint currentZ;
6.     half4 color;
7.  
8.     for (uint i = 0; i < slice; i++)
9.     {
10.         currentZ = zStagger * slice - slice + i;
11.         color = RG1[uint3(float3(id.xy, currentZ))] / 65536;
12.         if (color.r > 0 && color.g > 0 && color.b > 0)
13.         {
14.             color = Result[uint3(id.xy, currentZ)] * color;
15.             Result[uint3(id.xy, currentZ)] = color;
16.         }
17.     }

Path trace cache:

Code (CSharp):

1.         Pass //0
2.         {
3.             HLSLPROGRAM
4.             #pragma vertex VertSEGI
5.             #pragma fragment Frag
6.             #pragma fragmentoption ARB_precision_hint_fastest
7.             #pragma multi_compile_instancing
8.             #if defined (VRWORKS)
9.                 #pragma multi_compile VRWORKS_MRS VRWORKS_LMS VRWORKS_NONE
10.             #endif
11.  
12.             int FrameSwitch;
13.  
14.  
15.             sampler2D NoiseTexture;
16.  
17.             float4 Frag(VaryingsSEGI input) : SV_Target
18.             {
19.                 float2 coord = input.texcoord.xy;
20.                 float2 uv = input.texcoord;
21.  
22.                 //Get view space position and view vector
23.                 float4 viewSpacePosition = GetViewSpacePosition(coord, uv);
24.                 float3 viewVector = normalize(viewSpacePosition.xyz);
25.                 float4 worldViewVector = mul(CameraToWorld, float4(viewVector.xyz, 0.0));
26.  
27.                 //Get voxel space position
28.                 float4 voxelSpacePosition = mul(CameraToWorld, viewSpacePosition);
29.                 voxelSpacePosition = mul(SEGIWorldToVoxel0, voxelSpacePosition);
30.                 voxelSpacePosition = mul(SEGIVoxelProjection0, voxelSpacePosition);
31.                 voxelSpacePosition.xyz = voxelSpacePosition.xyz * 0.5 + 0.5;
32.  
33.                 //Prepare for cone trace
34.                 float2 dither = rand(coord + (float)FrameSwitch * 0.011734);
35.  
36.                 float3 worldNormal;
37.                 if (ForwardPath) worldNormal = GetWorldNormal(coord).rgb;
38.                 else worldNormal = normalize(SAMPLE_TEXTURE2D(_CameraGBufferTexture2, sampler_CameraGBufferTexture2, coord).rgb * 2.0 - 1.0);
39.  
40.                 float3 voxelOrigin = voxelSpacePosition.xyz + worldNormal.xyz * 0.003 * ConeTraceBias * 1.25 / SEGIVoxelScaleFactor;
41.  
42.                 float3 gi = float3(0.0, 0.0, 0.0);
43.                 float3 traceResult = float3(0, 0, 0);
44.  
45.                 const float phi = 1.618033988;
46.                 const float gAngle = phi * PI * 1.0;
47.  
48.                 //Get blue noise
49.                 float2 noiseCoord = (input.texcoord.xy * _MainTex_TexelSize.zw) / (64.0).xx;
50.                 float4 blueNoise = tex2Dlod(NoiseTexture, float4(noiseCoord, 0.0, 0.0)).x;
51.  
52.                 float depth = GetDepthTextureTraceCache(uv);
53.                 blueNoise *= (1 - GetDepthTexture(uv)) * voxelSpaceSize;
54.  
55.                 //Trace GI cones
56.                 uint3 voxelCoord = float3(0, 0, 0);
57.                 uint voxelDepth;
58.                 float3 kernel;
59.  
60.                 float fi = (float)blueNoise.x * StochasticSampling + tracedTexture1UpdateCount * StochasticSampling;
61.                 float fiN = fi / 92;
62.                 float longitude = gAngle * fi;
63.                 float latitude = asin(fiN * 2.0 - 1.0);
64.  
65.                 kernel.x = cos(latitude) * cos(longitude);
66.                 kernel.z = cos(latitude) * sin(longitude);
67.                 kernel.y = sin(latitude);
68.  
69.                 kernel = normalize(kernel + worldNormal.xyz * 1.0);
70.              
71.                 traceResult += ConeTrace(voxelOrigin.xyz, kernel.xyz, worldNormal.xyz, coord, -blueNoise.z * 0.125 * StochasticSampling, TraceSteps, ConeSize, 1.0, 1.0, depth, voxelDepth);
72.  
73.                 voxelCoord = float3(voxelOrigin.x + kernel.x - blueNoise.x * SEGITraceCacheScaleFactor, voxelOrigin.y + kernel.y - blueNoise.y * SEGITraceCacheScaleFactor, depth* voxelDepth - blueNoise.z * SEGITraceCacheScaleFactor);
74.                 tracedTexture1[uint3(voxelCoord)].rgba += float4(traceResult.rgb, 0);
75.  
76.                 uint scaledDepth = 256 * SEGITraceCacheScaleFactor;
77.                 if (voxelCoord.x < scaledDepth && voxelCoord.y < scaledDepth && voxelCoord.z < scaledDepth);
78.                 {
79.                     if (voxelCoord.x > 0 && voxelCoord.y > 0  && voxelCoord.z > 0);
80.                     {
81.                         if (traceResult.r > 0 || traceResult.g > 0 || traceResult.b > 0)
82.                         {
83.                             traceResult.rgb = lerp(tracedTexture0[uint3(voxelCoord)].rgb, traceResult.rgb, 0.25);
84.                         }
85.                     }
86.                 }
87.  
88.                 gi = traceResult.rgb * 1.18;
89.  
90.                 return float4(gi, 1.0);
91.             }
92.             ENDHLSL
93.         }

c#

Code (CSharp):

1.             else if (tracedTexture1UpdateCount > 80)
2.             {
3.                 context.command.SetComputeIntParam(clearComputeCache, "Resolution", (int)settings.traceCacheResolution.value);
4.                 context.command.SetComputeTextureParam(clearComputeCache, 1, "RG1", tracedTexture1);
5.                 context.command.SetComputeIntParam(clearComputeCache, "zStagger", tracedTexture1UpdateCount - 80);
6.                 context.command.DispatchCompute(clearComputeCache, 1, (int)settings.traceCacheResolution.value / 16, (int)settings.traceCacheResolution.value / 16, 1);
7.             }
8.             else if (tracedTexture1UpdateCount > 64)
9.             {
10.                 context.command.SetComputeTextureParam(transferIntsCompute, 3, "Result", tracedTexture0);
11.                 context.command.SetComputeTextureParam(transferIntsCompute, 3, "RG1", tracedTexture1);
12.                 context.command.SetComputeIntParam(transferIntsCompute, "zStagger", tracedTexture1UpdateCount - 64);
13.                 context.command.SetComputeIntParam(transferIntsCompute, "Resolution", (int)settings.traceCacheResolution.value);
14.                 context.command.DispatchCompute(transferIntsCompute, 3, (int)settings.traceCacheResolution.value / 16, (int)settings.traceCacheResolution.value / 16, 1);
15.             }
16.             tracedTexture1UpdateCount = (tracedTexture1UpdateCount + 1) % (96);

End result: (one cone per frame, for 64 cone result... The cyan is emissive result)

 

Just fixed Sub-sampling. (it now boosts performance rather than killing it.)

 

Now the path cache seems to be working correctly... I need to look into the weird voxelisation instability that sometimes occurs. It's also a problem in SEs source, so not something I've introduced. Anyone have any clever ideas for ways of stabilising it?

 

I think that's the reason he abandoned SEGI?

 

tbf, the more I work on it. The more I realise why he considered it a dead end... It's been pretty hard to figure ways to improve it.

 

I think the next step is to do what HxGI is doing, going to tree voxel structure to accelerate ray and have a better scene representation with sparse data

 

@Ninlilizi Again a huge huge improvement. The graininess is gone.
I made a script to help find issues. It automates the camera to highlight particular issues.
Drop this script onto the low poly scene's FirstPersonFlyingController game object, and disable the existing flying controller script. Also just enter a transition curve in the inspsctor.



Press play and select one of the two presets.
The voxel pop preset shows what I assume to be a bug in the voxelisation.
The emission pop preset shows a bug relating to the emissive material in the cave.

Are these both expected? Or are they fixable?

 

Awesome. I'll take a gander when I get back to work this afternoon.

Just found this blog via Reddit which has some interesting ideas that may be of use to us here.

http://blog.tuxedolabs.com/

http://blog.tuxedolabs.com/2017/11/22/a-better-depth-buffer-for-raymarching.html
http://blog.tuxedolabs.com/2018/10/17/from-screen-space-to-voxel-space.html
http://blog.tuxedolabs.com/2017/11/09/upscaling-half-resolution-screen-space.html
http://blog.tuxedolabs.com/2018/01/03/screen-space-path-tracing-diffuse.html

 

I see the problem now. Thank you for that... Yeah... The GI is being odd there.... The voxel thing is def weirdness from SE's code. I hope to tackle it.. But as I'm looking at some radical refactoring to the voxel generation, it's not going to be an immediate fix, but a little way down the road... The emmissive only emitting in one direction, I will take the time to investigate as the next thing on my list... Unfortunately I was unable to do so today as ended up in a teleconference for much longer than planned.