Graphics HXGI Realtime Dynamic GI

I've been working on a Realtime GI solution for Unity. I'm not able to use Unity's pre-computed GI system, since my game is procedurally generated; additionally, I wasn't happy with the speed, quality, and limitations of other methods available, so I had to roll my own system.


(newest screen shot using the new Sparse Radiance volume. Everything below this are older pictures using LPV's to generate the data, the quality is now on par with unity's progressive light mapper)

Features

• Fully dynamic
• Dynamic Shadow casting Directional Light
• Emissive surfaces (supports single sided surfaces)
• Infinite bounces
• Works well with normal maps.
• Little to no light bleeding.
• Sparse data structure for larger view distance and less VRAM.
• Skybox Lighting

Road map

• Filter pass to remove any noise
• Half res rendering for faster performance.
• Dynamic Shadow/none shadow casting Point and Spot light
  
• Realtime offscreen reflections
• Forward/transparency support
• Integration with HXVolumetricLighting
• VR support

Performance is based on a lot of factors so its hard to nail down the cost of the effect. Scene complexity, view distance of GI and how fast you want light to respond to scene changes are the major factors. The amount of emissive surfaces has very little to no impact on performance. So a lot of expensive per pixel lights can be replaced with emissive meshes.

I made this post to catalog my progress and have a centralized place that people can follow along rather then through my twitter. I'm not sure yet if I will release this to the public or not since I don't know if I have the time to support another asset or if it's even financial worth my time, but I'd like to gauge interest in it.

 

I love the idea of emissive meshes actually - makes certain kinds of scenes so much easier to do. Strip lights, architectural lighting, etc...

 

This is undoubtably sick. I've been following it on twitter for the past few weeks so I've been watching the progress and have been hoping you'd decide to put it on the store!

I guess the main question I have is how does this compare with SEGI? I suppose its a fair bit faster as you say it would work with VR? If you would be willing to support this solution, I think the community would certainly appreciate it as it seems SE is having a hard time devoting time to SEGI.

 

How does it behave in vast mountain environment with small scale detail in the foreground?

 

Does it rely on real time voxelization? Can we bake/stream voxel? can we cache this for pgc level? Can we prebake objects that are snap to a grid and assemble their voxel module to recreate the whole voxel volume for light by inserting them?

 

SEGI Currently has 3 big advantages over my system.

1. Generally looks a lot more accurate on the first bounce.
2. Resolves changes in the scene a lot faster.
3. Supports skybox lighting.

But suffer from a lot of light bleeding due to the nature of cone traced GI.
Ghosting when camera or object moves due to GI being calculated in screen space with temporal re-projection sampling.
Only really handles 2 light bounces and fakes any after that which tends to look bad in low light situations.
It's pretty expensive to render especially at 1080p+, generally resulting in Half res rendering which give you an unacceptable aliasing effect. Limited to deferred. A lot of these issues might be fixed with optimizations, but the light leaking probably cant be.

If you need lighting that responds almost instantly, then SEGI is better suited.

My solution runs a lot faster, Should be able to support Forward rendering as the pixel shader is really lightweight, handles infinite light bounces. Lighting is calculated independently of the view so there isn't any ghosting from temporal re-projection, light leaking is limited to one voxel space. It will never leak through a wall but you might see a small amount of light on the ground of a really thin wall, light can also creep around corners a little bit. Currently i have no way of calculating Skybox occlusion, I have a few ideas though.

The biggest downside is it can take a 0 - 0.5 seconds to fully resolve the lighting when something contributing to GI moves, it looks like a ripple of light, You can see the effect in this video (old). There are settings to mitigate this at the cost of performance but isn't nearly as noticeable in a lit scene, The directional light in the above video is doing it, I bet you didn't notice. Adding cascade will also mostly mitigate this effect as well. Things like players and fast moving objects are best not to contribute to GI to mitigate this effect, They will still be lit correctly though. The effect does take up more VRAM.

You set the resolution and scale of the voxel gird. Only GI near the camera is calculated, That distance depends on the kinda of quality you want. Adding cascades will open it up to larger view distance. But for now its probably limited to 50-100 units depending on how good you want the effect to look.

It voxelizes the scene over X frames, it's currently a lot faster then SEGI voxelization. You can slow down the voxelization or turn it off completely to get a speed improvement. As long as no GI contributing object moves, everything will work fine. Dynamic none GI objects still get lit correctly with in the bounds. You can even stop the light propagation if you don't need to move the directional light and then the effect is almost free. So you could warm up the effects and then pause the voxelization and light propagation for really faster baked lighting that can light dynamic objects. But its only with in the bounds of the effect. You can do a similar thing in SEGI but it still needs to do the cone traced step. Honestly just turning down the voxelization and light propagation speed to something like 1/16 is pretty fast if you don't need moving GI stuff.

 

Is that picture lit entirely by the emissive surfaces? It looks as good or better than normal real-time lights in that screenshot.

 

Super interested in this, Lexie, but you knew that already. Hoping you'll find it worthwhile as a sellable asset to the community, which is why I wanted to post here, to add my name officially to the list of potential customers.

 

Yes. It does specular approximation from all the lighting, even bounce lighting contributes to specular. But it falls a little short on really smooth surfaces, you can kinda notice the blue shifting colors towards the end of the specular, Ill probably have to ray trace the voxel grid for fast reflections to make really smooth surfaces look good. The other option is enabling screen space reflections but those are SLOOOOOW and don't handle off screen reflections.

 

What I meant is that can we pre voxelize scene and use that as separate asset to load and save! so it only run teh lighting and not the voxel (for all question). Or would that be possible in the future? and if there is real advantage of doing it? Assuming no moving objects of course, but can these be inserted in real time without having to voxelize the whole geometry using per object prevoxelized data? Basically trying to avoid rt voxelization at run time and use any data as "tile" to insert in into the "voxel map"

 

Yeah, screen space reflections are generally problematic. Off-screen reflections generally prevent me from using them.

It's very impressive. It seems like it could solve a lot of the problems I've encountered lighting up big procedural worlds in Unity.

 

Screen space reflection is usually for contact and inter object dynamic reflection, it's generally used and gracefully failed using lightprobe, look at gdc vault case like mirror edge catalist, they explain the idea well. SSR is never ever to be used alone.

 

My issues w/ it are that I can never get reflection probes set up in any way where the transition is at all graceful.

 

It would be possible and you would see a speed improvement. Down side is it can take up a lot of space to have those all saved out. My last system did that, I had a pre-calculated volume for everything. It started taking up a lot of space though.

 

Looks awesome

 

Does this work with a volume of spherical harmonics and then iterate and spread them 1 (or more?) voxels every iteration? If so, I've been thinking about such a system and wondering 2 things:
1) How to iterate over them without causing artifacts. Going simply pixel 1,2,3,4,5 would make light spread faster to one side, would iterating like 1,3,5,2,4 (or more steps inbetween) fix it enough?
2) Are specular highlights done by having a 2nd volume do the same thing with tighter spreading?

I may be entirely wrong about the concept though, in which case this is off-topic.

 

do you have a timeframe for when to publish it? Early Access Style would be nice

 

While it looks weird and well, "wrong", I also think it looks really cool? Feels like I'm in some sort of liquid light and I'm throwing waves of light around?

I almost want to make a game around it.

EDIT: Also, since the original video has a bit of banding, which I'm assuming is because of Youtube's compression, can you post a couple of stills so I can see what sort of banding (if any) your solution generates?

 

I have to say I'm extremely impressed, I think the lack of light bleeding more than makes up for the "slow" light propagation and I will admit I didn't even notice this effect in the video you posted. One last question - does it work on a mac? I think macs can support compute shaders now, so I assume it wouldn't be issue?

 

From old reddit posts I've been creeping:
This one from the video:


1 month ago:


Last December 5 months ago:
(This one also has a demo looks pretty awesome: https://dl.dropboxusercontent.com/u/11572739/Linked/CascadeVoxelGI.rar )





It seems to have evolved a lot since those old photos (and demo), but the old ones are still impressive D:

 

Those first two are this new system. The last ones are an older system I made, It used SDF ray traced shadows for point lights. In the end that method was a bit slow and the shadow quality wasn't good enough. The new system uses some similar stuff for bounce lighting, but has come a lot further over the months.

 

I'll have to look into metal and see if it can handle writing to 3D textures and has the equivalent of a geometry shader. This should work on DX11/ps4/xbone/linux although ill have to do some platform optimizations for consoles.

 

No time frame on when this will be finished. Hopefully soon though as I really need to get back to working on gameplay for my game. I've been working on solving our lighting issues for well over a year now. If i release this to the public ill probably do something similar to SEGI, release a beta on the asset store and stress this isn't 100% complete.

 

Yes, it does work like that, I don't know if that method of speeding it up will work or not. it would create some interesting artifacts. A better one is when cascades are added. you sample some of the lighting from the above cascade as the light is traveling twice as fast. This can get the light to propagate enough to hide the effect with in a couple of frames.

Specular highlights are approximated from the SH of the diffuse lighting, it falls really short on smooth surfaces and doesn't look as good if you have a lot of lights right near the surface. But the results are pretty impressive for very little cost. The plan is to add some ray traced reflections of the voxel grid to help capture more accurate spec.

 

Banding was from the video compression, The effect adds some noise to the lighting to remove all the banding. There is some grain in the AO areas of that image though. The fragment samples the GI multiple times using a rotated poisson disk. So you get some world space grain. It's not noticeable at all on textured surfaces and the sample number can be increased to hide it even more. I might try some other sampling methods though.

 

This would be fantastic, Lexie. Feel free to do it at any stage. As I mentioned to you on twitter, at Facepunch Studios we could really use something like this for Rust. Our islands are also procedurally generated and I suppose we suffer from many of the same problems you're facing on Spire. We ruled out SEGI rather early during the performance evaluation.

Amplify's game team are also looking forward to evaluate HXGI after experiencing a few problems with Unity's built-in RTGI for Decay of Logos.

No need to have it perfect before release. We just really need to figure out if HXGI is a solution we can eventually ship with, even if it takes a few months to sort out all the details, or even if we have to customize it ourselves to fit our needs.

Take my money.

Cheers

 

I think it's trade off worth having as an option, now we can optimized on what we have most, memory or compute (old trade off) we could even switch back and forth depending on the situation, and a lot could be further optimize if we have "tile" (or basically just manipulating the voxel memory directly.)

It also remove the burden of voxelization to whatever solution and project specific optimization anyone can do.

 

Amazing work. We would definitely be interested in supporting/purchasing this asset. I see forward rendering is on the roadmap, so do you see any conflicts with single pass stereo rendering and your GI solution?

Also really happy to hear about the scaleability and potential "realtime baking" option. I think in many cases this is a great option as an interactivity/performance trade-off. Or even to use the system with really high poly scenes (if I understand correctly, poly-count only affects voxelization?)

 

Same for our company!

 

Hmm, wouldn't be too hard to port my old system for creating voxelized cached data of assets.

The pixel shader for looking up the GI is pretty cheap. That's why it can handle large resolution rendering. In forward rendering there would be some added cost for overdrawn fragments but it wouldn't be too much.

it will work fine with single pass rendering for VR. The only downside to forward rendering is that you would have to switch over to using different shaders. It would have to include custom version of all the ones that come with unity. This would mean if you have any custom shaders they would need to be edited to support GI. If anyone knows a good way to edit unity's include files for forward rendering with out having to ship custom shaders, Id love to know.

You're right, The voxelization step is the only thing that gets effected by poly count. Using proxy Geo can go a really long way to speeding up the voxelization step though. I know that can be a lot to ask of hobbyists but larger studios are generally use to the pipeline of proxy meshs for assets. For our game we actually generate really low polly geo for all the surfaces in our game for collision, We will be using this to voxelize the world instead of the actually geo you see. I might be able to add a system to generate proxy data for each asset offline though.

Right now we are internally debating if its worth releasing or not. Releasing on the asset store generally means a price tag of less then 100$ for it to even sell. We would have to sell a lot of copies for it to be worth the time needed to support it and adding features the community will inevitably ask for. The last time we released our stuff for sale at the request of the community, id say it was a net loss at the end of the day.

This asset thankfully doesn't use any of unity's backend so it shouldn't be prone to bugs with every unity update, but there will be times that will need to be spent updating it. Any data points or opinions are greatly appreciated.

 

I think the financial potential is vastly different from the Hx Volumetric Lighting asset.
Unlike Hx Volumetric Lighting, HXGI is a general lighting solution that most probably attracts more customers.

 

I understand your position completely. The Asset Store is a competitive place to do business and not always fair.

On one hand, even if you undersell, not providing timely support it might affect reviews. On the other hand, selling at a fair price will get you less sales. It's a complicated market.

My opinion, tbh, is that if you don't live off the Asset Store and number of sales isn't that important to you, just set the price you think it's fair and be done with it. Less costumers and less support. Also, keep in mind that selling outside the Asset Store is also a perfectly valid option, especially since you're a highly reputable developer.

Although, I do agree with jcarpay. I think the sales potential is higher with this one.

 

I'm really sorry to hear this. Hx Volumetric Lighting's really holding the graphics side of my project together. I'd have happily paid more for it than I did. It really sucks that it's turning out to be more trouble than it's worth to you guys. I just want to say I really appreciate all the work you've done and the features you've added.

 

I'd like to second this, HX is a fantastic asset

 

Throwing my hat in as well, LOVED the volumetric system you built, would also buy this GI system, even though I already own and enjoy SEGI, will also buy your game when it's done, pretty much just want to keep throwing money your guys way regardless of avenue, keep up the great work.

 

I'd be happy to throw money at the old system as well, stuff still looks awesome D:

 

Hell yes. We'd buy an reasonably more expensive pack of both of these with cross support (e.g. volumetric indirect lighting) in a heart beat.

 

(Surface set to be extremely reflective for testing)

By raymarching the voxel grid and GI lighting I am able to reconstruct a pretty convincing representation of the world, This handles offscreen reflections as well which is great. I plan to use this data to supplement the specular approximation to make it more accurate on smooth surfaces. One thing to note is that the reflection will be of the voxelized version of the world. It only looks this accurate because everything is aligned to the voxel grid.

In a normal setting you wouldn't have surfaces this reflective, so an organic shapes with normal maps should hide the voxel reflections pretty well.

The smooth shadows in the reflection are my custom VSM shadows. I can't use unity's shadow maps as they generally don't contain offscreen shadows for optimization reasons, the VSM aren't super accurate but help make the reflections look more correct, Ill try and solve some of the bleeding it can introduce.

The above image is with a pretty high sample count. I'm in the proccess of optimizing it now. I just wanted to see what the best case would look like. Hopefully I can get close to this quality while still keeping the performance up and remove the banding effect if the reflections

 

I've gotten pretty close to the quility of the above image with out crippling the FPS. It runs a lot faster now and I've managed to remove the banding effect that most SSRR suffer from. There are still some bugs that need to be fixed up, you can see the wall is floating a bit in the reflection.

Once I fix those issues all start on rendering the reflections out at half or quater resolution + some blurring to hide the fact that the reflections are made from voxels. Then ill calculate some mipmaps so rough surfaces pull from a less sharp image.

Over all I'm pretty happy with the results I have so far.

 

omg that is so beautiful ^^ just give it to us

 

That looks really nice.

Is the square under the sphere the voxelized reflection of the sphere?

 

Yeah, There are still some bugs that are causing the voxel tracing to start in the wrong voxel so it wont be as bad when that's fixed. Also the lowres rendering + blur should help make it look more like a sphere.

If i lift it up a bit you can see what the reflections look like of the bottom of the sphere.

 

That could be an interesting look… LOL

 

Here is a test to see how well the reflections perform in a standard scene. Bottom is just GI + specular approximation, Top is with the reflections on. Still some bugs to fix and I'm not completely happy with how it blends with the material properties but its a good start.

Definitely going to have to do some form of blurring for the voxel traced reflections to be acceptable in a scene with flat shiny surfaces.

 

That's pretty cool. It's interesting to see the underlying voxel structure. I'm looking forward to seeing it with the blur.

In the scene without big flat mirrors there's only a few places where I can tell the reflections are voxelized, although I'm sure it's easier in-game than in a screenshot. Still, it looks quite good.

Is it a requirement that all geometry that contributes to GI contributes to reflections, or can you disable it for things where the voxelization causes something to look weird?

Even if these reflections aren't perfect, if they're consistently okay they're an improvement over my experience with reflection probes. They take a lot of manual setup for results that only really look good in a few places, and I don't even know where I'd start setting them up for a procedural world. I'd much rather have reflections that are always pretty good than reflections that are nice in one place where I spent 30 minutes tuning them and garbage everywhere else.

 

Yeah its only really noticeable on flat surface, If i can blur it then it should make materials look better.

I could add the ability that you can flag what things reflect or not. But it would slow down the reflections. Right now that's the most expensive part so id probably pass on that.

 

Yeah, it wouldn't be worth any significant performance cost I don't think.

 

Great result, definitely buy it for my project

 

I've been following this thread for a ehile now and I am growing more and more impressed by your results! This is amazing! Compared to the GI solutions we used so far, or the fake reflection techniques like SSR, your result look way more convincing! Could you maybe give a couple of performance metrics for non-test-scenes?

PS: I completely agreemthat putting this on the asset store for <100€ would probably not work out in the end. Plus it being heavily underpriced!