Questions about reflection probe in Unity 5

Hi,

I wondered how reflection probes are compute in Unity 5 ? Is it compute shader ? Is it use importance sampling ?

I assume it is based on a blinn-phong model to match the direct specular term ( or I am wrong ), is it going to be possible to change it to use for example a GGX model or other ?

Thanks.

 

Does that mean that now when using Camera.RenderToCubemap in a custom script a convolution is always applied to mipmaps ?

 

I doubt it.

I would however like the option to apply the convolution to regular old render textures.

 

Ok well I what I wonder is would it be possible to change this convolution by an other one and keep Unity's 5 reflection probe.

I already have a way to convolve my probe and for that I'm using the GPU with flitered importance sampling (to use GGX or any) but for this I'm using my own scripts and I would like to take advantage of the new reflection probe system of unity.

 

That explains very poor performance.

 

Does it, though? I thought cubemaps were only baked once.

 

With the new realtime option, you can have a script to update them. You could even attach scripts on all of the to update all the time

A more sane solution though, would be to update only the relevant cubemaps when you are doing really major lighting changes.

 

Yup, you can use them as real-time from newest beta, but performance is so poor that.... its almost useless.

 

I disagree. I've been using a real time probe attached to my character and the performance is excellent. Admittedly it's only a 128 probe, which wouldn't look good on a wall, but it looks fantastic on the player character.

 

Performance when you're updating a couple of probes at a time is just fine. Especially with the timesliced bool on.

 

Ok I'll try that, thanks for the tip!

 

Hey Charkes, are you using Unity 5 now?

I posed a little reflection probe request in [Official] 5.0 - Feedback request for Reflection Probes threads, I think you will need it too

 

Hey,

Yeah I tried it a bit to see if I could bring Antonov Suit to it but get stuck with reflection, I dont want to go back to Blinn-Phong kind model.

Thanks I'll probably make a request too .

 

I tried that, but it doesn't seem to be updating when the character moves. Am I missing something?

 

Yes, you need to tell it to update in a script. The script I'm using at the moment looks like this:

Code (CSharp):

1. using UnityEngine;
2. using System.Collections;
3.  
4. public class ReflectionProbeUpdater : MonoBehaviour
5. {
6.     private ReflectionProbe probe = null;
7.  
8.     // Use this for initialization
9.     void Start ()
10.     {
11.         probe = GetComponent<ReflectionProbe>();
12.     }
13.    
14.     // Update is called once per frame
15.     void Update ()
16.     {
17.         if( probe == null )
18.             return;
19.    
20.         if( probe.IsFinishedRendering(0) )
21.             probe.RenderProbe( false );
22.     }
23. }

Later on I might make it so that it only updates it if it's position changes... but then I guess i want it to change if other things around the character as well. I dunno, performance seems fine anyway.

I've also set the Anchor override of my character's mesh to be the probe object, and the "reflection probes" variable to simple. Also I've removed the character's layer from the render mask of the probe.

For me it's working fantastically.

 

Oh, awesome, thanks for the prompt reply. I'll give that a shot.

 

Hey Zomby138, have you try to build your scene for the Reflection Probe update? Is that working for you?

I tried earlier with my own simple script ( almost same as your script) to update the Reflection Probe, it works very well only in the editor play mode. After i build the webplayer or standalone, the Reflection Probe is black.

I'm on my Macbook, not sure if it is the platform issue or not. (or maybe a bug?)
(Attached a screenshot of webplayer)

 

@Luckymouse Yeah looks like a bug. Doesn't work for me in the windows build either.

 

I'm seeing the convolution step taking 45-50 ms at 128 resolution with 0 reflection passes and size 10.

Setting time sliced to true, it's really chunky, about 1-2 fps update on the reflection.

 

According to Tomas1856 explained from above post how unity reflection probe does as following step: (Please correct me if I'm wrong)
1) Use Camera.RenderToCubemap for first mip level
2) Convert RenderTexture to texture asset. (assuming using Get.Pixels to save to a file )
3) Use TextureImporterCubemapConvolution.Specular to convolve the texture asset to Specular Cubemap
4) Apply this Specular Cubemap to reflection probe.

This workflow is ok for baked/precomputed cubemap in the Editor, definitely not a good idea for realtime update per-frame based.

I assume the new ReflectionProbe.RenderProbe API will do the better job for reflection probe update, maybe implemented a GPU base convolution? (Directly convolve the RenderTexture without to save it to file first?) Or it is just doing the same steps above. If it is so this API is just not that meaningful.

 

This is some real time test done with Unity 4 :

https://dl.dropboxusercontent.com/u/1812933/AntonovSuit/ProbeSky/ProbeSky.html

I render them with RenderToCubemap() in ARGBHalf format with mipmap (both diffuse and specular)
I send them to my filtered importance sample shader to do real time convolution.

It's slow as hell but gives good idea of what real time can do.

But I don't think full real time can be done for an entire scene with multiple cubemap.

I used uSky for sky : https://www.assetstore.unity3d.com/en/#!/content/24830

EDIT : Makes me think that some engine (Fox Engine and Far Cry 3) pre bake their cubemaps for each time of the day to use them later at correct time.

Convolution is I think done with GPU during the bake, at least for Fox Engine. But still nothing is in real time, it's just too much.

 

Amazing, especially in the right golden material looks awesome. Now that is what i'm talking about! Great job!

 

I made a quick update with two little sphere also.

Yeah it looks good but it is slow (64 samples for both diffuse and specular less reduce qualtiy a lot for diffuse).

I didn't tested with compute shader yet though.

 

@Charkes
I found out that Unity 5 is using GGX ( Googled ), It is a derivative of Disney GGX BRDF. So it will fit your Antonov Suit shader, right?
If yes, It is will be very easy for you to integrate Antonov Suit for Unity 5.

 

Specular shape looks like Blinn-Phong to me and I remember in : Physically Based Shading in Unity by Aras Pranckevičius reading that it was inspired by Disney, but still use Blinn-Phong.

 

hi charkes,
i just wanted to correct you but had a look at the latest (well: beta 16...) version of the standard brdf first: it seems as if you were right. unity has switched back to blinnphong... now that anybody moves to cook torrance.

 

I honestly think Unity should switch to GGX specular - it has a very nice falloff, much better than either Blinn-Phong or Cook-Torrance.

 

I sort of agree, but isn't GGX more expensive?

 

Of course a bit but not that much really.