Resolved Dynamic (on and off) baked/mixed lights?

I'm trying to create a system that will allow me to turn baked/mixed lights on and off while affecting the lightmap. My solution is to have one default lightmap where lights that aren't dynamic get baked into, and for each dynamic light baking a separate lightmap influenced only by that light. When a light is on, just take its lightmap and add it to the default lightmap. Whether it looks accurate or not is completely irrelevant to me. This has been much harder to implement than I originally thought, and searching around has led me to a few solutions, each with their own difficulties and problems. Maybe I'm not using the right keywords?

Solution 1: Using a lightmap switcher.

This yielded very great results, I was able to switch the lightmap, light probes, and reflection probes (I'm only looking for lightmap changes). The only problem is that this is something that is great for large lightmap changes (i.e day/night), but not for what I'm wanting to achieve. In order to achieve the effect I'm looking for, I would need to bake each lighting condition, and this would increase exponentially with each dynamic light added (8 dynamic lights would result in about 65 different lighting conditions, default condition included), and would then need to find out how to tell the lightmap changer which condition it should currently be at.

Solution 2: Combining lightmaps on the GPU via custom shader.

Haven't attempted to implement this solution, but thinking about its implementation has me doubtful about its effort vs results turnout and how optimal it'll be. Since light probes likely wont be viable for something like this I was thinking of simply sampling the objects position and transferring it to UV coordinates to sample the lightmap from and setting its ambient color to that color. Since this solution composites the combined lightmap on the GPU, I would have to composite the lightmap per object and sample the color. This seems like a more preferable solution that would get close to the effect I'm looking for, but I would like to get some input before I pour a couple weeks into developing a system like this. I'd likely have to build a custom lightmap shader solution instead of using Unitys.

Solution 3: Combining lightmaps on the CPU via script (the only one I have screenshots and code for).

This one was the most appealing to me, simply combine the lightmaps via script and apply them to the scenes lightmap settings. I wasn't able to apply the results to the scene lightmap, but I was able to apply it to the materials texture as a test. It combined them, but it also resulted in a much darker lightmap. I'm pretty sure it has to do with using .GetPixels() on lightmaps. This would be a near perfect solution for me besides those two issues.

Lightmap applied as a texture, no combination function.


Ran through combination function, with no texture to add.


Ran through combination function, with one texture added.


Here are the lightmaps



Here is the script

Code (CSharp):

1. using System.Collections;
2. using System.Collections.Generic;
3. using UnityEngine;
4.  
5. public class AdditiveLighting : MonoBehaviour
6. {
7.     [SerializeField]
8.     private Texture2D[] defaultLtMaps;
9.  
10.     private Color[] ltMapPixels;
11.  
12.     public Texture2D[] additiveLights;
13.     private Texture2D[] finalDest;
14.  
15.     private int arrayLength;
16.  
17.     public Material testMat;
18.  
19.     // Start is called before the first frame update
20.     void Start()
21.     {
22.  
23.         //defaultLtMaps = LightmapData.lightmapColor;
24.  
25.         testMat.SetTexture("_MainTex", defaultLtMaps[0]);
26.  
27.         ltMapPixels = defaultLtMaps[0].GetPixels(0);
28.         arrayLength = ltMapPixels.Length;
29.  
30.     }
31.  
32.     void Update()
33.     {
34.  
35.         if (Input.GetKeyDown("r"))
36.         {
37.  
38.             AddLighting();
39.  
40.         }
41.  
42.     }
43.  
44.     private void AddLighting()
45.     {
46.         Color[] pixels = new Color[arrayLength];
47.         Color[] mixedColors = new Color[arrayLength];
48.  
49.         foreach (Texture2D dM in defaultLtMaps) {
50.  
51.             ltMapPixels = dM.GetPixels(0);
52.  
53.             if(additiveLights.Length > 0) {
54.                 foreach (Texture2D tx in additiveLights)
55.                 {
56.  
57.                     pixels = tx.GetPixels(0);
58.  
59.                     for (int i = 0; i < arrayLength; i++)
60.                     {
61.                         Color c1 = ltMapPixels[i];
62.                         Color c2 = pixels[i];
63.                         Color result;
64.  
65.                         result.a = 1;
66.                         result.r = (c2.r + c1.r);
67.                         result.g = (c2.g + c1.g);
68.                         result.b = (c2.b + c1.b);
69.  
70.                    
71.  
72.                         mixedColors[i] = result;
73.  
74.                     }
75.                 }
76.             } else
77.             {
78.                 mixedColors = ltMapPixels;
79.             }
80.         }
81.        
82.         Texture2D mixedImage = new Texture2D(256, 256);
83.  
84.         mixedImage.SetPixels(mixedColors);
85.         mixedImage.Apply(true);
86.  
87.         LightmapData data = new LightmapData();
88.  
89.         data.lightmapColor = mixedImage;
90.  
91.         testMat.SetTexture("_MainTex", mixedImage);
92.  
93.         //LightmapSettings.lightmaps = lightDatas;
94.  
95.     }
96.  
97. }
98.  

If anyone knows places I can learn more about systems like this, please point me towards them.

I've also been thinking whether or not additive combination is also viable for light probes and potentially reflection probes (just isolate each lights contribution to a duplicate light/reflection probe). I have my doubts on if it'll work or not, but I will read up on light and reflection probes to learn how they function and how to alter them at runtime. Right now I'm more focused on lightmaps.

 

Small update on the shader GPU solution.

So far I have something that works decently, sort of helps prove the concept.


Unfortuntately, I cant find a way to properly sample the array in the same way the default lightmap is sampled. I assumed I would be able to use some component of unity_LightmapST that lets the shader know which lightmap index to use, but I may misunderstand completely how that variable works.

I also can't seem to pass a Texture2DArray into the shader, had to create a texture2Darray asset in order to see if the lightmap sampling was working the way I expected (it isn't).

This is the function that adds the lightmaps together into one texture2Darray. It seems to work, but I have no real way to test it as I cant seem to pass the results into the test shader.

Code (CSharp):

1. using System.Collections;
2. using System.Collections.Generic;
3. using UnityEngine;
4. using UnityEngine.Experimental.Rendering;
5.  
6. public class AdditiveLighting : MonoBehaviour
7. {
8.  
9.  
10.     public Transform[] DynamicLights;
11.  
12.     private Texture2DArray compLightmap;
13.     private int dnmcLightsLength;
14.  
15.     [SerializeField]
16.     private Material testShader;
17.  
18.     private int lightmapArrayLength;
19.  
20.     void Start()
21.     {
22.         lightmapArrayLength = 2;
23.  
24.      
25.  
26.         if (dnmcLightsLength > 0) {
27.             compLightmap = new Texture2DArray(128, 128, lightmapArrayLength * dnmcLightsLength, TextureFormat.RGBAHalf, true, true); // the width and height values are placeholders, couldn't find a way to get the lightmap width and height
28.  
29.  
30.             GatherLights();
31.          
32.         }
33.     }
34.  
35.     public void GatherLights()
36.     {
37.         int arrayPos = 0;
38.         DynamicLight lightInfo;
39.  
40.         dnmcLightsLength = DynamicLights.Length;
41.  
42.         TextureFormat fmt = compLightmap.format;
43.  
44.         foreach (Transform light in DynamicLights)
45.         {
46.             if (light.gameObject.activeSelf) {
47.                 Debug.Log("I activate");
48.                 lightInfo = light.gameObject.GetComponent<DynamicLight>();
49.                 int dept = lightInfo.gatherLightmaps.Length;
50.              
51.              
52.                 for (int i = 0; i < dept; i++)
53.                 {
54.                     int mip = lightInfo.gatherLightmaps[i].mipmapCount;
55.  
56.                  
57.  
58.                     for (int o = 0; o < mip; o++)
59.                     {
60.                      
61.                         Graphics.CopyTexture(lightInfo.gatherLightmaps[i], 0, o, compLightmap, i + arrayPos, o);
62.                      
63.                     }
64.                  
65.                  
66.  
67.                 }
68.             }
69.             arrayPos += lightmapArrayLength;
70.         }
71.         compLightmap.Apply();
72.         UpdateLightmap();
73.     }
74.  
75.     public void UpdateLightmap()
76.     {
77.         testShader.SetTexture("_ltMapAdd", compLightmap);
78.         //testShader.SetFloat("_lengthOfArray", lightmapArrayLength);
79.         //testShader.SetFloat("_loopAmount", dnmcLightsLength);
80.  
81.         Shader.SetGlobalTexture("_ltMapAdd", compLightmap);
82.         Shader.SetGlobalFloat("_lengthOfArray", lightmapArrayLength);
83.         Shader.SetGlobalFloat("_loopAmount", dnmcLightsLength);
84.         Debug.Log(lightmapArrayLength);
85.         Debug.Log(dnmcLightsLength);
86.     }
87.  
88. }
89.  

This is the script attached to dynamic lights

Code (CSharp):

1. using System.Collections;
2. using System.Collections.Generic;
3. using UnityEngine;
4.  
5. public class DynamicLight : MonoBehaviour
6. {
7.     [SerializeField]
8.     public Texture2D[] gatherLightmaps;
9.  
10. }
11.  

This is the test shader.

Code (HLSL):

1. Shader "Unlit/TestShader"
2. {
3.     Properties
4.     {
5.         _MainTex ("Texture", 2D) = "white" {}
6.         _ltMapAdd("Lightmap", 2DArray) = "" {}
7.     }
8.     SubShader
9.     {
10.         Tags { "RenderType"="Opaque" }
11.         LOD 100
12.  
13.         Pass
14.         {
15.             CGPROGRAM
16.             #pragma vertex vert
17.             #pragma fragment frag
18.             // make fog work
19.             #pragma multi_compile _ LIGHTMAP_ON
20.             #pragma target 3.5
21.  
22.             #include "UnityCG.cginc"
23.             #include "UnityShaderVariables.cginc"
24.             #include "UnityStandardBRDF.cginc"
25.  
26.             struct appdata
27.             {
28.                 float4 vertex : POSITION;
29.                 float2 uv : TEXCOORD0;
30.                 float4 uv1 : TEXCOORD1;
31.             };
32.  
33.             struct v2f
34.             {
35.                 float2 uv : TEXCOORD0;
36.                 float4 vertex : SV_POSITION;
37.                 float4 uv1 : TEXCOORD1;
38.             };
39.  
40.             UNITY_DECLARE_TEX2DARRAY(_ltMapAdd);
41.             sampler2D _MainTex;
42.             float4 _MainTex_ST;
43.             uniform float _loopAmount;
44.             uniform float _lengthOfArray;
45.  
46.             v2f vert (appdata v)
47.             {
48.                 v2f o;
49.                 o.vertex = UnityObjectToClipPos(v.vertex);
50.                 o.uv = TRANSFORM_TEX(v.uv, _MainTex);
51.                 o.uv1.xy = v.uv1 * unity_LightmapST.xy + unity_LightmapST.zw;
52.                 o.uv1.zw = unity_LightmapST.zw;
53.                 return o;
54.             }
55.  
56.             half3 GatherLightmaps(float3 defaultMap, float4 uv) {
57.  
58.                 if (_loopAmount < 1) {
59.                     return defaultMap;
60.                 }
61.  
62.                 float3 results = defaultMap;
63.  
64.                 for (int t = 0; t < 1; t += _lengthOfArray) {
65.  
66.                     results += DecodeLightmap( UNITY_SAMPLE_TEX2DARRAY( _ltMapAdd, float3(uv.xy, uv.z + t)));
67.  
68.                 }
69.  
70.                 return results;
71.  
72.             }
73.  
74.             fixed4 frag (v2f i) : SV_Target
75.             {
76.  
77.              
78.                 half3 lightmap = GatherLightmaps( DecodeLightmap( UNITY_SAMPLE_TEX2D( unity_Lightmap, i.uv1) ), i.uv1);
79.                 fixed4 col = tex2D(_MainTex, i.uv) * half4(lightmap, 1);
80.  
81.                 return col;
82.             }
83.             ENDCG
84.         }
85.     }
86. }
87.  

I'm hoping theres a way to utilize unity_LightmapST to get the lightmap index, because thats where I assume its coming from with my limited knowledge about how it functions. I dont wanna have to pass in the lightmap indexes per each renderer, but if I have to I have to.

 

Found some errors with the AdditiveLighting script and test shader, fixed them and its properly passing in the texture2Darray. Unfortunately still haven't gotten the indexes figured out.

Code (CSharp):

1.    void Start()
2.     {
3.         lightmapArrayLength = 2;
4.  
5.          dnmcLightsLength = 0; // this variable will tell the shader how many times
6.                                // it needs to iterate through the lightmap loop,
7.                                // so dont use it to tell how long the actual dynamic light array is.
8.  
9.         if (DynamicLights.Length > 0) // if there are no dynamic lights in the scene
10.        {                              // there is no need for this at all
11.             compLightmap = new Texture2DArray(128, 128, lightmapArrayLength * DynamicLights.Length, TextureFormat.RGBAHalf, true, true);
12.  
13.  
14.             GatherLights();
15.        
16.         }
17.     }

Its somewhat functional, despite the lightmap index issues. If the light is active it'll add the lights lightmaps to the texture2Darray, if its off it wont bother.

 

After some meddling around, I've finally got it fully functioning.

I've added a second light but this system could, as far as I'm aware, handle an infinite amount, although that would not be optimal.


Turning off the yellow light will turn off its baked information as well.

Same with the red light.

The cyan light isn't supposed to be dynamic, its part of the default bake.