Video light emission

I'm trying to make a video material emit light into the room, using the colors being shown in the video. I've tried using Video Player component and map the video output to the TV materials emission, and then updating realtime GI with the material. But even if the update is per-frame, there is a delay of 150-200ms before GI have taken the TV materials new emission video frame and used it to affect the environment.

I also tried mapping video to a spotlight cookie, but it's limited to one color instead of using the videos colors.

This is ultimately what I'd like to achieve (seen in the Netflix and Youtube apps from Oculus Store as well), where there's no delay between video frame and environment lighting:

I'm in Unity 2019.4 LTS.

 

Are these apps with dynamic light emission from video not using Unity maybe?

 

You can definitely do this with Unity but it isn't built into the engine. I have done the same thing essentially a while back using a dynamic planar area light where it gets its color from a texture that comes from a video playing. If you look for LTC area lights in Google it should show the basis of what I'm talking about. I think I remember Unity having an early implementation of it somewhere on Github for the built-in renderer as they used it for the Adam demo. Although it won't have everything you need to get the video playback functionality you want. Hope this helps!

 

You can call UpdateGIMaterials on each item you want the screen to light up. Make sure your items are set to static. Something like this:

Code (CSharp):

1. public class UpdateGI : MonoBehaviour
2. {
3.  
4.     public List<Renderer> rends;
5.  
6.     // Update is called once per frame
7.     void Update()
8.     {
9.             foreach (Renderer rend in rends)
10.                 rend.UpdateGIMaterials();
11.     }
12. }

 

This could be helpful :
http://answers.unity.com/answers/1856523/view.html