Official Adaptive Probe Volumes (APVs) experimental release for HDRP in 2021.2

Eh, I think if APV lerping becomes a thing a lot of us use, we need a way to easily lerp sets of relfection probes as well, or have a system to easily low priority async rebake all reflection probes in the vicinity.

But we're probably not going to get that. Normalizing them reflections probes by the lightprobes will be deemed good enough and they'll call it a day, and it will be yet another half assed feature in Unity's toolset.

 

Kinda leaning toward a mix of relighting probes (possibly rgb cube, alpha contains depth which you can reconstruct normal from) and static (local) indoor ones.

Probe relighting is a staple of open world games out there because it's practically free, requires no camera nonsense or setup or cost. It would be nice if the option was built into the existing Unity probes. Naturally these probes do not contain shadow information, but they may well not require it for their use cases. Any extra thoughts on this from anyone would be cool.

Not sure how it would look API wise but you need the cubemap rgb and possibly alpha in the depth. Then you can calculate how a probe is supposed to look from any given position with any number of basic in-shader lights, like maybe a campfire or a primary dir light. It's for outdoor usage mainly or places where it's too expensive to update 6 camera directions for a cube. The memory cost would be a bit higher: the probe needs RGB+RGB+A so 7 floats to do with what you will.

Perhaps there's better data arrangements and people know about it.

In any case, for a large open world game, I think people need 3 types of probe:

• classic render camera for 6 faces / sliced (slow, but you can use proxy geo)
• static probes for indoors etc
• relit probes: store rgb+depth and use it to render a new probe relit for any time or lighting circumstances to loosely match. Custom shader can be used.

 

This is largely where we have been heading too. Open air being the most difficult to balance right.
Our early tests we had a few open air probes refreshing, code and timeline tests proved to work however no existence of time slicing in HDRP under 22, puts a heavy spike into the mix, interestingly enough more effective with timeline than hardcoding the update on change.

even with custom frame setups ignoring shadows for example there sadly was still large spikes causing a noticeable jitter where as you mentioned the 6 face updates we're just way too much.


In general I feel you're certainly on a strong path for your project, especially with your DOTS framework as well, that must have been very rewarding for you getting Into that early on.

It also seems like many open world projects are hitting the same pain point sadly.

 

To be honest ( from a Nvidia perspective) you could use most 1000/2000/3000 fine enough.

We tested on a 1070ti 2070s and 3070 and all was pretty adequate on our test ext/int scene. Though it wasn't extensive.
But I think your personal workflows wouldn't vary too much based on hardware, potentially just more volumes for lower end cards

Overall it's a much much MUCH better experience that manual probe placement and the additional processes APV does.

The APV system can be sliced up quite well with volumes so that your GPU isn't chugging too much at once.

I will however state that it's really still early days in the tools life so using at full production stage is dicey.

It does tend to freeze and hang the GI cache plenty, and of course full support for end hardware may not be present yet.

 

I'm switching to HDRP after the proto and HDRP has a heavy up front CPU cost. I hope staff will optimise it best they can. I really do need the fidelity it brings and I won't wait for APVs.

 

I feel like I'm generally more accustomed to HDRP general workflows, our team took a much longer time to adapt to it trying to change their solo work built-in mindsets to HDRP specifically.

I'd say focusing on your project outliner with the hardware targets and the RP assets set up first, you'll probably have a much better time working with it, profiling and benchmarking performance rounds then.

 

Well, I have to say it's come a long way since I last was here a few years ago. This blows Unreal away in the right hands because it has way less temporal dirt and the signal seems way cleaner. Some really clever heads have worked on this and it shows as soon as you engage. Wow! Loving this!

Sorry, off-topic. Now to figure out how APVs work in an open world lerpy context. Advice anyone? Basically plan is to emulate enlighten by lerping time of day. What is best practise/size/resolution/amount of APVs needed etc etc?

Advice please!

 

It currently doesn't work well with terrain, also don't lerp everyframe

 

Ouch - so there is a big hit to performance? I don't mind if something can be sliced to only update in parts over time, but what I CANNOT endure are spikes. Even if it's once every 5 frames. Because that's precisely sort of thing that becomes visible on a wide range of machines.

I do use terrain, sadly. I'll try meshes while I learn so I see the correct results.

@francescoc_unity - does it have a heavy cost (spike)? can we make it a constant cost but spread over frames?

 

Thanks that would be really great. Happy to test what you have planned!

 

Great, thank you.

Regarding APV volume, you said it is tied to the scene. Is there support for a simple offset for origin shifting? Perhaps for hybrid renderer / entities in future (if you needed context).

 

Found an issue with APV on 2021.3.5 and earlier:
- Planar reflection probes don't work anymore (probably wrong normal calculation) if "Normalize Reflection Probes" is enabled. Screen goes all black if Planar Reflection Probe is visible.

 

If the APV (which bakes very quick) aids in a sort of almost realtime GI, since the prodes are all over the place, I can understand how it makes lighting better.

At the same time though, can't the same system be used, in a way to aid reflections, without the need to place reflection probes? Altenatively, can't a similar system be created for automatic reflections everywhere in the scene?

 

If you are not able to reproduce, I would do, but would need to create a new project with APV setup, as current project is way too big to submit.

 

For large worlds that go further than 5-10K units, what do you recommend?

 

I'm also interested in this? Maybe in Unity 2023? It would be extremely helpful.

 

Reported as IN-8235 without repo project, but steps to reproduce.

 

Many thanks! Your are right, this was what I was looking for, a way to do the classic open world scenario. Still remaining is a question of how to deal with loss of precision far away from origin?

At 10k units, things start to move strangely, so we shift the world every so often, I have not tried shifting with APV yet, but I assume it will be streamed in at the wrong position. Will test next week.

 

How do I fix this blackness on the floor?
I'm using CPU light mapper
Probe volume setting at the default value
Unity 2021.3.4
HDRP 12.1.7

 

One comment / question on this; As reported by me earlier in this thread, in 2021.2.x and 2021.3.x "Virtual Offset" is not usable at all if a project uses non-kinematic physics, so I am running into same issues with blackness. What will be the first version where the physics issue has been fixed?

 

In the picture, I had both dilation distance and iteration count at 1.
Changing both to 3 seems to fix the problem. Thanks.
All these settings seem like black magic to me though. I don't understand what they do from the short tooltip shown in editor.
Questions
1. Is there an advantage of spliting the probe volume to multiple volumes instead of one big volume that cover everything?
2. How do you deal with large and complex scenes that need a very dense probe volume (aside from getting better hardware)? I can't bake the volume that has distance lower than 1 in large scene it because I ran out of ram or vram? 'Cannot allocate more brick chunks, probevolume brick pool is full.'
3. Is it possible to only rebake probe and not the entire lightmap?
4. Does the light probe sample multiplier in the lightmap setting do anything for APV?

 

HI!
Looks like some good works has been done.

However I struggle to evaluate the cost as a low spec game dev. I mean how does it work under the hood?

Given it's per pixel and seems sparse except at surface, how the hell the probe selection happen?

- first I thought of a system like sparse voxel octree, with texture brick as leaf holding the SH, but the traversal seems a bit too expensive for most low end target if that's a replacement for old style PV in unity.
- Then I thought it use hierarchical volume with various, we collect the volume on object overlap, then pass them on a light loop to sample the corresponding SH texture brick, which make the sampling O(1).
- Another idea was it might work by having the texture brick like the tetrahedral SH, but interpolating the brick instead of a single SH. Which probably wouldn't work as good, so I guess not.

Is there a paper or article about the underlying behavior? That would help understand a lot how to deal with it.

 

Hey! nevermind! I had to google video instead, site search only return unity forum in most pages! lol

 

In the video it showed that the scene also have somekind of global Sky Occlusion which help for the improve the indirect shadowing, we really need those kind of global Sky Occlusion even if it's static Sky Occlusion

 

Where is the ♥️ love button!

Really grateful to hear this.

 

What sort of APV features and worlkflow will HDRP 15 get?

 

Been trying it out in 2022.2.18a - surprised by the results! better then the last time I tried it out a while ago.
One question though: Where's the option to smooth noise when TAA is enabled?
I'm referencing this change:

• HDRP: Added: A new option to animate APV sample noise to smooth it out when TAA is enabled.

It's a part of the release notes for 2022.2.a18 so it should be there. But I can't find it anywhere (APV window, probe volume script, camera, etc)

one issue I'm facing is noise. It gets worse the higher the bounces are, and so you have to compensate with an even higher value for light probe samples the higher the samples are, the less noise there is, as you can imagine it gets crazy pretty fast

 

Some interior APV / lightmap comparisons:

- mixed lights
APV settings: 0.55m (very dense), 1024 indirect samples, 17 probe sample multiplier, 5 max bounces
lightmap settings: 45 texels, 1024 indirect samples, 5 max bounces
No directional/sky light

Right click on each picture and click open image in new tab, then jump between tabs quickly to notice all the differences:

APV:



Lightmap:



APV with SSGI active:


it seems to struggle more with emissions

 

What's your SSGI setting?

 

I can't find all of the options in 2022.2.0b1.
Did they move it somewhere else ?

 

Default medium with reflection probe + sky fallback

 

Seems to me this is a bit of an abuse of the APV, setting it so high as to replace local lighting on small details. But if that is possible, then colour me impressed.

Could staff elaborate the ideal use cases a bit clearer for the system? One can then be impressed and work with the original design intent (where it no doubt shines).

 

Hi there, is it possible to use apv to approximate lighting on a skinned mesh renderer? i'm not having any luck with that.

 

What does your skinned mesh inspector look like?

 

Hi there. Can anyone possibly test with Better Lit Shader? I’m trying to figure out why it doesn’t work with Better Lit Shader. Can someone from Unity perhaps test this to aid in the support of this feature? Thanks!

 

To the above it was because the mesh was using Better Lit Shader not HDRP Lit. I’ve been on to the developer but he is unsure what would cause the issue - it works with standard probes - perhaps you could point to the difference in what APV is doing and point him in the right direction for support for APV.

 

I'm wondering if APV would work as a simplification of the plenoptic equation. While researching lightfield rendering, it has been demonstrated that the 5d equation (x,y,z,phi,theta) can be reduce to a 4d equation (x,y,phi,theta). Typical lightprobe array are basically a 5d representation of lightfield, which mean it can be compressed from volume to at least boundaries of volume (assuming convex for simplicity). APV seems to aggregate data to boundaries of mesh rather than volume, so how is the empty space derived, through interpolation of hierarchical data, or through adapting the plenoptic equation?

 

Anyone?