Graphics HXGI Realtime Dynamic GI

Try playing with the "Cones" and "Cone Trace Steps" and deactivate the Reflections. But like @buttmatrix said, your post belongs more to the SEGI thread ^^

 

i dont see any GI in this demo. i only see area light here.

 

Just getting started again, i was on a bunch of work trips followed by a holiday. Im working on a sparse voxel implementation now. Not too much to report.

That looks more like approximated skybox occlusion then GI. I don't see any light bouncing.

 

I actually thought this was the SEGI forum I'll unsub

 

Hey @Lexie, hope you're doing well. Will your solution support dynamic point / spot lights?

 

Yes, I've already added support for them. Although they are more expensive to calculate then an emissive surface. Page 6 has a bunch of images from dynamic point and spot lights

 

Is there a shadow solution available in Unity for emissive surfaces?

Dynamic/point light shadows are rather ugly in Unity as it is.

 

What your talking about is GI, The only realtime + runtime one available to the public is SEGI. I'm not sure when my version will be ready for public release

 

Before the voxel and GI data was stored using 3D textures. This meant that empty space would still take up large amounts of data and would limit the texture size of the voxel + GI texture to 128/256^3, any larger and it would take up too much VRAM.

I tried switching to cascades of this data to try and increase the view distance, but the quality of the distant GI became to poor and introduced too many artifacts. So the next step is using a sparse data structure to store all the GI and voxel data. Below is the debug data showing the data structure usage to store 1 voxel.

Only solid space and the nearby area take up space on the GPU. Also it only takes a few branches to get to the payload data. This data structure is completely dynamic and all calculated on the GPU using compute shaders.

Ill post some more interesting voxelized models soon. Its a slow process working with compute shaders as you have to write a lot of debug tools to visualize that the code is doing what it should.

 

Oh sweet. Glad you've got time on this again! Pumped to see your updates .

 

Hmm. Forgive this crude sketch, but aren't shadows and GI a separate thing, handled by separate assets?



And right now, we can only get good looking shadows from a single directional light source in Unity. Point and spot light sources have really ugly shadows, and emissive surfaces have no shadow support.

I get that your GI solution will give us some shadows, but if a character holds his hands, fingers spread out close to the floor, will we be able to make out his fingers on the shadow on the floor with only HXGI? From what I've seen so far, the answer is no and the shadow might even have a slight delay (note that I'm not asking for this either, for most situations this will probably be just fine).

Since the last post, I did a quick search, and one of the few threads asking about this that I found was my old thread, so either I'm missing a huge point here, or there's a surprising gap on the asset store market (especially after HXGI hopefully gets released which might make people use 100% emissive materials in a scene).

I'm going to try the area lights from the Adam demo when I get time, though. Maybe that holds the answer.

 

Yes. And no. Depends on how you look at it. For example, what is the "shadows" that form when it's bright outside and you have an open window, but no direct light comes in?

In any case, generally the direct shadows we use in games, use simplified and fast models to produce shadows of much higher fidelity of what is usually afforded by GI solutions. Arbitrary emissive surfaces cannot be handled similarly to the way point and spots or area lights are. (I mean, area lights are more advanced, but it's still a much more simplified case than "any kind of surface").

 

Right, I was sloppy in the text, but at least I wrote "projected shadows" in the image.

I just tried Adam's area lights and it does seem to actually be what I was looking for. Need further testing, but hopefully it will be a nice complement to HXGI (because I do hope it will work with HXGI).

https://github.com/Unity-Technologies/VolumetricLighting

Now to test it in VR, though...

 

Does not work with single pass rendering

 

Woo! Looking forward to release

 

The shadows in Adams area lights are just a spot light shadow with EVSM shadow maps, it's not area light shadows.

My GI is calcualated on a voxel representation of the world. The shadow quality from GI is very much linked to the size you set for the voxel grid.

GI means more then just bouncing direct light, It also means calculating the light from emissive materials. A non emissive material hit by direct sunlight becomes an emissive material.

 

I know GI means more, but for example in a third person game, you'd want a separate, more detailed shadow projected for your main character, right?

Or will HXGI produce detailed enough and fast enough shadows that you don't need anything else? If so, that's even more impressive than I thought.

EDIT: I just found this thread... I'm going to shut up and read some stuff now.

https://forum.unity.com/threads/dynamic-volumetric-lighting-replacement.379041/

 

I got some time to work on the sparse voxel structure again. Here is the debug rendering that shows what nodes are active. Each level of the tree gets subdivided into a 4x4x4 structure if a voxel is present in it, this goes all the way to the bottom node where the payload data is stored, This allows me to store any type of data in this sparse data structure.

I've hidden the lowest node level if its not a voxel so its easier to visualize the data. For example: the largest box is completely filled with 4x4x4 of the second largest boxes. But only the visible ones of those are filled with the third smallest boxes.



The largest box is a tile, you can set how many tiles you need, That is the one thing that cant really change at runtime. But you can set it to something massive like 128x64x128 with very little memory overhead and that still gives you a massive voxel range of 8192 x 4096 x 8192, The old system only supported 256x256x256.

 

Nice one!

Out of curiosity, did you choose a subdivision of 4x4x4 rather than 2x2x2 to reduce shader branching?
Is it always a set 'depth' from root 'tile' to 'payload'?

 

Less branching, its also a lot faster to construct as i can set aside a large portion of data each time it needs to split. Although depending on how I calculate the GI, it might be faster doing a 2x2x2. Its only a few lines of code to change it so ill have to see how it goes.

It's always a set depth to the payload, But the ID of the current branch can be used to store extra data along the way.

 

@Lexie So when you say you can set the voxel tile dimensions to at most 128 x 64 x 128, which ultimately gives the range of 8192 x 4096 x 8192, isn't that just the theoretical max? It seems like you'd have to allocate a ton of data to account for the possibility when all 274 billion voxels are filled. Or do you not actually need to preallocate memory for each sub voxel? Or is it based on hashing or something that isn't really pre allocated arrays?

 

You could set it a lot larger then that, It takes up 4 bytes per tile(So 4mb for a world size using the above) its kinda meant for the bounds of your world. Its a sparse data structure, So a tile with no voxel data in it wont take up any extra space.

You allocate how much resources you want to put into the storage of the voxel data. only 5-10% of the space will actually be filled with voxels.

 

Interesting, that's cool. So what happens if something like 20% is filled? Does that break things? Or can you just reallocate more memory?

 

I really love SEGI but performance isn't good for large map. Is there any demo for LXGI to test and compare performance with SEGI?

 

Nothing public yet unfortunately.

It is possible to resize the buffer if it's getting full. Although it wouldn't be wise. There will be little bars showing how much space you have spare so during testing you can make sure you allocate more then you would ever need.

 

Hello there! Is it possible to teleport lights/shadows through a plane? I have portals, that I can not make unnoticable due to the lights/shadows, that appear/disappear on the portal's edge

 

I might have to dilate nearby nodes to make sure that the light transport retains enough detail. So I went ahead and added support for octrees as my 4^3 nodes would use too much VRAM.

Its just a shader flag so its easier to switch between them. But from the test I'm seeing the octree saves about 40% memory over the 64Tree. Although it will make the cost of traversing the nodes a little more expensive. It does cut the light probe count by a lot, So the speed improvement of faster light propagation should offset that cost.

Overall I'm pretty happy with the data structure, It supports all the functions i needed while storing the data in a sparse structure and saving somewhere between 80-99% of VRAM per chunk. Couple more things to clean up and then I can move onto GI again.

No. you will have to write your own lighting system from scratch to get that working well. Even then it will most likely be a combination of faking it by rendering two lights for each side of the portal and then using some depth stencil to mask out areas.

 

Below is a demonstration of the new sparse voxelization system on a large scene. It's able to store all this information in less VRAM then a 256^3 texture. The image at the bottom represents the area a 256^3 volume in the old system would take up.

 

D

Dude that is fully amazing. Really glad to see your updates again!

What is the performance difference like between the two versions? Does the sparse octree take much of a performance penalty?

Also, when visualising the light transport in my mind, I struggle to understand how a payload voxel could pass light through a sparse container and then back to a payload voxel without the light getting all blurry. Is this a problem that the sparse structure introduces, or am I misconceiving now the light propagates?

Still keeping a keen eye on your progress.

 

LPV's Really don't accurately transport the light at all. But the way it wont be too bad is dilating the sparse voxel tree. Your not propagating lighting from a voxel to a massive open space, You dilate the structure so it only ever makes the step up/down one node level.

Im also messing around with calculating the GI lighting a different way. Something like cone tracing the sparse data structure per light probe. Will have to see what looks/performs the best.

 

This looks like progress! Good job Lexie. Excited to see this in action.

 

Did you time building the large scene structure?

Lennart

 

This is without a doubt, one my most anticipated "features" in modern game-dev. It would save soo much time + enable a whole new paradigm of procedural / destructable environments, without compromising on lighting etc. I keep checking your post weekly, and it's really exciting seeing the progress you are making with the new voxelization.

If the performance ends up being good, and VR compatible, I would buy it in an instant no matter the price.

Keep up the phenomenal work!

 

Looks great! I tried implementing a sparse structure last year for a project, but ran out of time. From what I gathered in your previous posts, sounds like you've got a predefined root grid with 3? internal 4x4x4 levels and 8x8x8 leaf blocks. Are you using a custom camera with multiple passes to store each level? What's the voxelization time looking like?

 

Close, there are two modes for storage right now, a 4x4x4 tree with 3 levels and a 2x2x2 tree with 6 levels (Both 64^3 voxels). It is then stored in a predefined root grid (Tile, dark blue boxes). The octree version looks like it uses a lot less space but will cost more to traverse, Ill do some testing to see what works best.

There are better sparse structures I've looked into that would be more efficient, But creating them dynamically at runtime all on the GPU is a lot harder/slower then this implementation.

The voxelization uses a camera in orthographic mode to render out a 64^3 * voxel scale volume to a 64x64 texture. The shader uses a geometry shaders to reorient each triangle towards the camera using its strongest normal. This way it can be done in 1 pass. A 3 pass shader can be used for Mac support. Each fragment is then written to the sparse data structure. Most of the cost comes from rendering the meshes, Adding the fragments to the data structure doesn't impact performance much at all.

The voxelization time really depends on how many polygons you intend to voxelize, Its recommended to use proxy/LOD meshes if possible. It can voxelize/re-voxelize as many or as little tiles per frame as you want. To voxelize this whole scene all at once (1013 tiles) took around 0.5 seconds, that's an average of 0.5ms per tile, But this world is pretty low poly so this cost would increase based on poly count. There are a few steps I need to remove that are used for debugging that might speed this up a bit more as well.

0.5 seconds for the whole scene. But you can spread this out over as many frames as you want.

Not sure yet, the data structure doesn't look like it added any measurable/noticeable overhead at all. The cost to sample the data in the lighting step will have an increased cost though.

 

That is fast. No problem at all to do this at scene load for procedural scenes.

 

This is just for the voxelization step, It will still need to do the GI Calculations on top of this, This will also be able to be spread over many frames though.

 

Just FYI, I would easily put $150 for an asset like that.

 

I wasn´t expecting large scene support anityme. Awesome.

 

Realtime Voxelization Test - Just voxelizing the dynamic objects helps illustrate how the data structure works.
I'd love to be able to voxelize static objects once, and then only update cells with dynamic objects... There just isnt a good way to do this in unity... I've asked for the ability to set a flag on the camera for static or dynamic. but i don't think they will add it.

Edit: no gif support?

 

I would gladly pay much more than that if it is performant, voxel realtime GI is the holy grail! Lexie maybe the asset store is not good for such a complex/specific/difficult to maintain plugin, but you could license it in a case by case basis at negotiated prices. If we wanted to implement this ourselves it would take us many months, $150 is just pennies for this.

 

sorry for noobish question, but..
with new unity job system your GI system will get benefit?

 

Yeah this is really a tough choice. It really depends on what lexie wants to market towards, "hobby indie" developers or "succesful / proven indie" developers. The "proven" developers have a lot more budget but are a lot rarer. Saving months off development time/cost is worth a lot.

P.S: If this asset would be editable/edited to also work nicely with cubic-meter-voxels (like minecraft), I'd be willing to pay significantly for it (since it saves me months of development time I can't spare).

 

Yeah I really don't know about pricing, I'm just focusing on making it work. Once that's done ill think about licensing costs etc. It would be nice if I could have two different licenses to be able to support both pricing models. I understand hobby devs cant afford a price point over 80$, But at the same time this asset would save a team many many months/years of R&D and is surely worth more then the cost of 30mins/1hour of contract work.

As for cubic-meter-voxels, you can set the center point of the voxel system as well as the scale. So as long as you set that correctly it should work.

I was thinking about having a way to override the voxelization code in case people have their own voxel system stored on the CPU. Then they could just push the data to the GPU rather then the GPU generating the voxel data from a mesh. Or even a mix of both could be useful.

We sort of use voxel data as well as meshes for our game so I know this functionality could be useful for speed purposes.

Unity's new job system is for C# CPU code. Its basically a sandbox for easier multi threading in unity. 99% of this GI system lives on the GPU. The code that is on the CPU wouldn't benefit from any multi threading as the few calls it does must be made on the main thread.

Yeah...

 

I've added the ability to raycast the data structure. Below is a comparison of octree casting vs voxel tracing. Each box represents 1 texture lookup. As you can see the octree implementation allows much faster tracing as its able to skip large spaces.

Cone tracing uses basically the same code as well so it should be really simple to add. I might use cone tracing for specular or as part of the lighting calculation. Time to do some tests.

 

A test of the octree raycasting per pixel. The image on the right represents the cost to traverse the octree.

If I was to use this stuff for lighting I would use cones instead of rays. That way it early outs long before the image gets that red. Over all its pretty fast, and if I use cone tracing it should remove most of the bright read areas.

In the old versions I had a raymarcher to view the voxel grid. It would tank the frame rate. The image on the right would mostly be bright red as soon as it goes past the first tree. So its a pretty large savings while having a massive view distance.


The little bright read dots on the close hill are some missing voxels, Might be a bug or i need to enable AA on the voxel renderer. Will look into it.

 

Man, I can't wait for this to release.

 

Very cool!

 

I know my question isn't directly related to what you are doing (which is awesome by the way!) but how do you store and send the octree information in the CPU and GPU? (I am wondering if I could use this technique for a custom cloud and smoke method I am trying to realize).

 

I send a mesh to the GPU, the GPU voxelizes it and then adds each voxel to an octree. All the work is done on the GPU in my implementation using a combination of compute and fragment shaders.

If you wanted to generate the Octree on the CPU and then send it to the GPU. The requirements would be that the data structure be stored in arrays as that's all the GPU can handle. Then send the data using ComputeBuffers.

If I wanted I could send the whole Octree from the GPU to the CPU, added stuff CPU side, then send it back.
Issue is that that would be slow. So best to just send the data I want to add and let the GPU do the work.

I use 3 Arrays for my tile+octree implementation.