Feature Request Support supplying lightmaps as "Texture" instances, rather than "Texture2D"

An in-development project I'm a part of necessitates dynamic manipulation of lightmap textures. Currently this is being handled by using CustomRenderTextures and setting the unity_Lightmap shader property to them on individual renderers. This however is unwieldy to use, and complicates operations such as raycasting into the lightmap.

I managed to throw together a (very dangerous) prototype that can forcefully store any texture type in the lightmaps array, and with the exception of erroring or crashing when the engine tries to clean up the texture instance it actually works perfectly fine (the textures even preview correctly in the lightmap inspector!).

Code (CSharp):

1. using System;
2. using System.Collections.Specialized;
3. using System.Runtime.InteropServices;
4. using System.Runtime.CompilerServices;
5. using UnityEngine;
6. using Object = UnityEngine.Object;
7.  
8. class ConvertLightmapsToRenderTextures
9. {
10.     [StructLayout(LayoutKind.Sequential)]
11.     class UnityObject
12.     {
13.         public unsafe NativeUnityObject* CachedPtr;
14.  
15.         // Partial version of the internal C++ Unity object layout.
16.         // Due to being partial, this is only safe to use by reference!
17.         public unsafe struct NativeUnityObject
18.         {
19.             static readonly BitVector32.Section s_MemLabelIdentifier = BitVector32.CreateSection(1 << 11);
20.  
21.             static readonly BitVector32.Section s_TemporaryFlags = BitVector32.CreateSection(1 << 0, s_MemLabelIdentifier);
22.  
23.             static readonly BitVector32.Section s_HideFlags = BitVector32.CreateSection(1 << 6, s_TemporaryFlags);
24.  
25.             static readonly BitVector32.Section s_IsPersistent = BitVector32.CreateSection(1 << 0, s_HideFlags);
26.  
27.             static readonly BitVector32.Section s_CachedTypeIndex = BitVector32.CreateSection(1 << 10, s_IsPersistent);
28.  
29.             public IntPtr* MethodTable;
30.  
31.             public int InstanceID;
32.  
33.             public BitVector32 BitFields;
34.  
35.             public RuntimeTypeIndex CachedTypeIndex
36.             {
37.                 get => (RuntimeTypeIndex)BitFields[s_CachedTypeIndex];
38.                 set => BitFields[s_CachedTypeIndex] = (int)value;
39.             }
40.         }
41.     }
42.  
43.     enum RuntimeTypeIndex : uint
44.     {
45.         WebCamTexture       = 239,
46.         RenderTexture       = 244,
47.         CustomRenderTexture = 245,
48.         SparseTexture       = 246,
49.         Texture2D           = 247,
50.         Cubemap             = 248,
51.         Texture2DArray      = 249,
52.         Texture3D           = 250,
53.     }
54.  
55.     static unsafe void ChangeTypeIndex(Object obj, RuntimeTypeIndex index)
56.     {
57.         Unsafe.As<UnityObject>(obj).CachedPtr->CachedTypeIndex = index;
58.     }
59.  
60.     [RuntimeInitializeOnLoadMethod(RuntimeInitializeLoadType.AfterSceneLoad)]
61.     static void Execute()
62.     {
63.         var lightmaps = LightmapSettings.lightmaps;
64.        
65.         for (int i = 0; i < lightmaps.Length; i++)
66.         {
67.             // Create a RenderTexture clone of the lightmap, as an example.
68.             var lightmap      = lightmaps[i].lightmapColor;
69.             var renderTexture = new RenderTexture(lightmap.width, lightmap.height, 0);
70.             Graphics.Blit(lightmap, renderTexture);
71.  
72.             // Trick engine into thinking that the RenderTexture is a Texture2D.
73.             ChangeTypeIndex(renderTexture, RuntimeTypeIndex.Texture2D);
74.             lightmaps[i].lightmapColor = Unsafe.As<Texture2D>(renderTexture);
75.         }
76.  
77.         // Ensure original type indices are restored before the game quits, this is presumably
78.         // because the engine calls the Texture2D cleanup function on each lightmap despite
79.         // them not actually being Texture2D objects. It's also possible that this is the result
80.         // of C++ virtual call optimization if we assume the engine stores them as Texture2D fields.
81.         Application.quitting += () =>
82.         {
83.             for (int i = 0; i < lightmaps.Length; i++)
84.             {
85.                 var lightmap = lightmaps[i].lightmapColor;
86.  
87.                 // If this lightmap is not actually a Texture2D, then
88.                 // its type index must be restored to prevent a crash.
89.                 switch (lightmap.GetType())
90.                 {
91.                 case Type type when type == typeof(WebCamTexture):
92.                     ChangeTypeIndex(lightmap, RuntimeTypeIndex.WebCamTexture);
93.                     break;
94.                 case Type type when type == typeof(RenderTexture):
95.                     ChangeTypeIndex(lightmap, RuntimeTypeIndex.RenderTexture);
96.                     break;
97.                 case Type type when type == typeof(CustomRenderTexture):
98.                     ChangeTypeIndex(lightmap, RuntimeTypeIndex.CustomRenderTexture);
99.                     break;
100.                 case Type type when type == typeof(SparseTexture):
101.                     ChangeTypeIndex(lightmap, RuntimeTypeIndex.SparseTexture);
102.                     break;
103.                 case Type type when type == typeof(Cubemap):
104.                     ChangeTypeIndex(lightmap, RuntimeTypeIndex.Cubemap);
105.                     break;
106.                 case Type type when type == typeof(Texture2DArray):
107.                     ChangeTypeIndex(lightmap, RuntimeTypeIndex.Texture2DArray);
108.                     break;
109.                 case Type type when type == typeof(Texture3D):
110.                     ChangeTypeIndex(lightmap, RuntimeTypeIndex.Texture3D);
111.                     break;
112.                 }
113.             }
114.         };
115.  
116.         LightmapSettings.lightmaps = lightmaps;
117.     }
118. }

(I'm almost hoping that the sheer hackiness of this workaround will convince somebody at Unity to take a look at implementing this properly in the engine )

The problem is that I absolutely do not want to rely on a hack like this, as it's reliant on implementation details within the engine (and would break if, for example, the internal layout of the Object class changed).

Would it be possible for this functionality to be added to the engine within the Unity 2021 (or even 2020) release cycle? Perhaps via the addition of APIs along the lines of:

Code (CSharp):

1. public sealed partial class LightmapData
2. {
3.     public Texture polymorphicColor { get; set; }
4.     public Texture polymorphicDir { get; set; }
5.     public Texture polymorphicShadowMask { get; set; }
6. }

The Texture2D property variants could either throw or return null if another texture kind is used.

 

Correct, more specifically a CustomRenderTexture so I can make changes to the lightmap at runtime without paying texture copy costs.

 

As I need to bypass the lightmaps array by using MaterialPropertyBlocks to override unity_Lightmap currently, the lightmaps array needs to be filled with dummy data which the Renderer components reference. Any third party code that deals with lightmaps also needs to be tweaked to account for this system (granted, if polymorphic texture properties were added they'd need to be tweaked regardless, just on a much smaller scale).

That's completely understandable and I have no complaints there. Asking for this to arrive in 2020.2 is a huge stretch and I was expecting 2021.x to be a much more realistic request from the get-go.

Yep, that's a very good point. I'd be perfectly fine with the properties just being typed as RenderTexture instead of Texture, since that'd be fine for my use case as I'm only using CustomRenderTexture instances.

 

Correct, that's exactly it.

I agree, this is a tough problem to think of a solution for. In the OP I suggested that the Texture2D properties could throw or return null if the other properties are set, but that wouldn't alleviate confusion in the API (rather, it could even make it worse if a third party library changed the properties without your knowledge).

This would definitely be ideal in the long term, but a change of that scale does seem excessive to enable a use case as niche as this. That said, it'd have some pretty nice benefits for other projects I'm sure.

Given that Unity 2020 has already begun introducing significant changes to the lightmapping APIs, I wonder if a better alternative in the short term could be to deprecate the LightmapData class in Unity 2021 and replace it with something else. The problem is figuring out what that "something else" should even look like.

It's no problem at all! I never had the impression you were intending to obstruct anything. Rather I very much appreciate the responses, the design in the OP leaves much to be desired so it's only natural to want to explore alternative routes.

 

That's completely understandable. The project in question that would benefit from the feature likely wouldn't be able to jump to Unity 2021 before release regardless, so right now this is merely an attempt to explore potential optimizations for lower end computers in a patch (as the texture copies aren't much of an issue for higher end devices.)

I really appreciate you taking the time to have this discussion and look forward to any potential news on this in the future!