Lighting Data Asset was a mistake

I have a faint hope that some of the developers will read this post.
I'm developing a GPU lightmapper: https://forum.unity.com/threads/bakery-gpu-lightmapper-v1-2-released.536008/

To my regret, every time I have to deal with patching Unity lighting data, I have to fight it. Sometimes it's tougher than writing the actual lightmapping code itself. I even had to spend a few days reversing LightingDataAssets in a hex editor to get the job done.

This is not the way it should be, and it's not the way it was before. Before LightingDataAssets were introduced things were nice and easy - there was a lightmap array, lightmapIndex, lightmapScaleOffset, they could be altered by a script and saved with the scene.
LightingDataAsset is a black box with no API. It can be only modified indirectly (e.g. light probe colors seem to be saved) or not at all.
Light Probes can't be created without calling Bake(). Lightmap-renderer mapping can't be saved. Reflection probes get in the way by being saved in the same asset, even though they have literally nothing to do with baked lighting. Asking Unity to bake a cubemap can instead force lightmap rendering for the whole scene with the purpose of just initializing a new LightingDataAsset.

It was mentioned by devs somewhere that the reason for LightingDataAsset was to make project merging easier (when working with a team), but to me this solution created much more problems than it solved.

What would make sense:
- Add complete read/write API to LightingDataAsset
- Possibly separate Reflection probes and even Light probes into different files

 

- Give us low level documentation about Unity systems and what it's actually doing so people do not need to reverse engineer.

 

- [Bonus point] Expose atlasing API used by built-in lightmappers, so it can be used separately from baking

 

I asked for this a couple of years ago. Since in most of my scenes I want to use just one 1k or 2k lightmap, I wanted to be able to run just the atlasing to see what I get, then increase or decrease the resolution until I get exactly 1 lightmap at maximum possible res.

A dev saw it, told me it makes sense and they'd look into it.

Frankly, I think something has gone horribly wrong with the lightmapping team. I'm guessing they're having trouble with the GPU lightmapper (that switch from powervr to radeon rays must have been fun), since we haven't really gotten any more features for the CPU one, nor a lot of bug fixes.

LightingDataAsset is "fine" as a concept. It should not be a black box and it should not auto-do stuff on its own. We need more control on this.

 

Hey Frank,

The reason
LightingDataAsset was introduced so that the scene (source data) doesn't need to be touched when we bake/precompute lighting (generated data). This separation is crucial for a decent end-user workflow.
Initially the asset wasn't really meant to be modified by the user as it contains a lot of realtime GI specific mappings that you really don't want to deal with. The other reason was that realtime GI and then lighting modes were in a lot of flux until last year, so exposing APIs would make it much harder to shift things around.

Reflection probes do belong in the lighting data asset as they are as much "baked GI" as lightmaps and light probes are.

That said, I can see how its black-box nature can be limiting, especially in your case.

The workaround
We could take a step back and look at what you want to achieve: apply lightmaps and light probes in the scene. One approach is the one that you took, so trying to modify the lighting data asset. The other is to store the data in your own asset (or possibly as references in the scene itself, as otherwise referencing renderers would be tricky) and patching it at scene load and after entering play mode. Something that other users are already doing to keep lightmap data together with prefabs.

The solution
In the short term we can add an API that will allow you to modify:

• the lightmap references
• per renderer information for baked GI
• baked light probes coefficients
• baked ambient probe
• baked reflection probes

In the mid term we can add APIs also to the lighting data asset that will help with lighting modes:

• light:
  • shadowmask index - a light points to a shadowmask layer
  • light index for occlusion probes - an occlusion probe points to a light
• occlusion probes (attenuation for up to 4 lights affecting the probe)

In the long term we can add APIs that will help with creating everything from scratch:

• LightingDataAsset
• LightProbes

But that will likely come after we've done some changes when it comes to storing inputs to lightmapping (to separate them from scenes) and changes wrt to multi-scene and light probes.


I'm sad to hear that it's been frustrating for you. We never want you to feel that way. So as soon as something like that comes up, please poke early. We might miss it sometimes, we might have other priorities some other times, but we'll hear you out and try to help as much as we can. <3

 

Thanks for response Kuba!

I know - that's the first approach I took, and that's how it works in most cases. However:

- light.bakingOutput property did not exist on pre-2017.3 versions. That means there is no public API to set shadowmasks. On these versions shadowmask can be still accessed via SerializedObject, but it won't work in builds. To support shadowmasking on older versions was the only reason I had to reverse.

- Applying lightmaps via scripts can be slower than doing it in the native code, especially when there are lots of objects, and especially when scenes are loaded at runtime, leading to performance hiccups.

- Also for some reason when loading multiple scenes, Unity may rearrange lightmap IDs on objects, so I have to set them more than once. Setting IDs in Awake() seems to be the only option to make it work with static batching, but when it gets to Start(), IDs can somehow change, and I'm setting them again.

What if instead of one fat asset, it would be a collection of smaller, more focused sub-assets? So LightingDataAsset would be just a list of references to a LightProbes asset, ReflectionProbes asset, BakedGI asset, Realtime GI asset... etc. In this case:
- Scene still won't be modified after rebaking.
- You can have a stable API for some sub-assets that don't change much (lightmap-renderer mappings, light probes).
- You can separate more experimental features into isolated assets.
- In my opinion, there are benefits in separately baking parts of the scene or just probes only, and in this case there would be less merge conflicts, if only one sub-asset changes.

That would be definitely amazing

 

Awesome reply. And while you're at it - don't forget to enable all this content for mipmap streaming if not already

 

And just to confirm - is there any API at all to read/write occlusion probe values at the moment?

 

This is by far the most painful part of Unity... overall, the engine is very open ended. But lighting? Complete black box. It's beyond frustrating since there is essentially 0 api calls to look at it. It's mind blowing.

 

I'm trying to figure this out too, I got lightmapped prefabs working after about a day of struggling but they're kind of useless without light probed prefabs too.

@guycalledfrank does Bakery make the process of lightmapped prefabs anyless painful? Or support including bakery light probes in prefabs and loading them at runtime (I'm piecing together a dungeon room-by-room at runtime)? If it does, i'd by it right now to make this headache go away.

 

There is no proper way to do it yet, but you can hack by putting the invisible scene object I'm storing lightmaps on into the prefab. Then it kinda just works.

Unfortunately Bakery currently just uses standard Unity probe system, only overriding their color values. I'm thinking of making a separate system for probes.

 

I see. In your experience with them is there any way to get pre-baked light probes to load at runtime and add to the "scene" light probe group (it seems like no matter what you do unity looks at all light probe groups in the scene as one, but doesn't honor or remember the baked probe data of a light probe group that was in a prefab)?

 

Unity doesn't seem to be able to do that out of the box. I heard of some people doing crazy hacks/reverse engineering for that:

 

I've been using Bakery to bake lighting Data into prefabs, and the Prefabs can be instantiated at Runtime with the baked lightmaps and realtime reflection probes intact. Which is great. The only limitation I can see is that I am not supposed to use Nested Prefabs, but I think I can cope with this.

However, when it comes to lightprobes, I seem to have to update the 'Scene Lighting Data Asset' reference to get at least one of my instantiated Prefabs to work with its baked lightprobes. I cannot move the Prefab without causing an issue where the dynamic object still thinks its at the location where I originally baked the Prefab, and references the lightprobe group from there.

I'm left with the situation that I'll need to use a limited number of dynamic lights to illuminate non-static objects at runtime, which is an expense that I was trying to avoid, and not ideal.

 

Re: light probes, unity can now additively load light probes and merge them with a tetrahedralize call as of 2019.3 so that's a thing now, not sure if helpful?

 

I agree if just for the need for prefab-baked lightmapping. It feels like unity is always forcing a choice between making something well lit and making the world truly dynamically streamed, even if only streaming static content in dynamic transforms. Choosing not to implement it is one thing, but not providing the tools for us to implement it ourselves outside of creating your own lightmapper, which is what frank did, is worse. @Kuba Is bundling realtime & baked lightmaps in with prefabs something that is still on the roadmap? I've scoured for resources on how to do this, and they've all boiled down to hacks that only work with the baked lightmaps and I can't get the baked only lightmaps to look right in comparison to realtime..

 

@Kuba any updates or estimates on LightingDataAsset API? It would make my life waaaaay easier if there was a way to create these assets from code and set occlusion probe values manually.

 

Hi - I am running into the same issue. Instantaited prefabs have all of the baked lighting, but the lightprobes aren't working. Did you get this working or does someone have a solution? When you say that you updated the 'SceneLighting Asset' - how did you go about this? Thanks.

 

Lighting data asset is still a mistake.
I'm running into more and more weird behaviours. In latest 2019 versions under some circumstances light probe data seems to be not saved, unless I reload the scene and write it again (I guess it's related to the recent merge-able probes change).
It would be awesome to be able to completely opt out from using the asset and save everything to the scene file, the oldschool way. Make lighting data asset a [preview] package? Or at least make it transparent. Or at least publish the format spec.

 

It is really frustrating to see that someone from unity said that they'd add an API almost two years ago and we've had no progress since then. My game is heavily dependent on bakery, so seeing this get fixed is also important to me and other users of that asset - which is currently the only good way to lightmap in unity

 

How about this recent progress in the documentation
https://docs.unity3d.com/ru/2019.4/Manual/LightmapSnapshot.html

Before 2019.4:


2019.4 and after:


I guess that's the Unity's answer

 

I wonder, would it be possible to pop open the lighting data assets ourselves? Now that we know Unity isn't planning on implementing an API, I'd say it's time for us to give it a try. However, I assume the code related to this asset is on the C++ side, which may make it nearly impossible to deconstruct? :/

I could *swear* someone, somewhere, had made some sort of program to read these files, but I could be wrong. I can't find it anywhere on google.

 

That would be definitely cool. Except for the fact its file format can change between versions. But as I observed, the changes are not that huge.
I have some notes on LightingDataAsset binary structure, I can share them somewhere, but they are far from complete file description.
Github? Possibly other users can extend the spec by submitting pull requests?

 

Github would be great!

 

Created: https://github.com/guycalledfrank/lighting-data-asset-reverse
Anyone willing to dig a bit deeper: you are welcome!

 

@Kuba any updates on new API´s? Its pretty much impossible to use Unity for anything semi-procedural if you want lightmapping and lightprobes. Specifically, not being able to move lightprobes or attach them to a prefab is extremely frustrating. How has there been nothing but silence for 2 years?

 

Seconding access to the lightingdata asset. I'm generating composite lightmaps and want to put the result back into the automatic lightmap system.

 

Not being able to have light probes in our procedurally generated game is really hurting our performance. I've been searching for ~84 hours now in forums, google, source code. It looks like all the important code I require is not open source? I'm not sure where to find it, and I've tried hex mapping the LightingDataAsset to see if I can salvage it. The only thing I haven't tried is reading through all those research papers and reinventing the wheel ><

I would be incredibly grateful if you guys could help out at all. Any sort of direction to point us in. Getting Light Probes to move at run time is the final, last step in having baked lighting for procedurally generated maps.

Thank you for your time!

 

I actually needed to access the internals of LightingDataAsset a little while ago, so I reverse engineered the whole thing. I figured I'd release it after coming across this thread, so enjoy: https://github.com/NewBloodInteractive/com.newblood.lightdata

Here's some example usage:

Code (CSharp):

1. // Retrieve the lighting data and convert the SceneAsset to a Scene.
2. var data  = LightingData.CreateFromAsset(Lightmapping.lightingDataAsset);
3. var scene = SceneManager.GetSceneByPath(AssetDatabase.GetAssetPath(data.scene));
4.  
5. // Object references are stored as SceneObjectIdentifiers, so they need conversion.
6. var ids    = data.lights;
7. var lights = new Light[ids.Length];
8. SceneObjectIdentifier.SceneObjectIdentifiersToObjectsSlow(scene, ids, lights);
9.  
10. foreach (var light in lights)
11. {
12.     Debug.LogFormat(light, "{0}: {1}", light.type, light.name);
13. }

 

Amazing! You have my respect!

 

I think we really need LightProbes.positions to not be readonly.

 

You can modify LightProbes.positions using a SerializedObject in the editor, but that's not available at runtime.

The LightProbe API is annoyingly limited at runtime, especially so when you know that you can construct an AssetBundle in memory to get around it (a similar issue applies to the Font class, and I've used the same technique to get around it before).

The problem with that approach though is that you need to have knowledge of the file format and you need to know how to construct one correctly for your particular Unity version.

 

This is definitely something I'm not skilled enough to do. I am going to start a new forum on this subject though because I feel it is quite important.

 

you are too kind
you could've not shared it but you shared it
thank you

 

i wouldn't also mind if you were to show the wizardly methods by which you can construct an AssetBundle in memory to get around LightProbes.positions

 

+1

 

Very disappointed that it seems Unity hasn't made any progress on this. Baked lighting is kind of big deal, especially on the Oculus Quest. Or has it and this thread was just never updated to reflect it?

 

Completely agree with OP.

 

After all this years it baffle me how there is no progress about this. Even the new system they are working on, APV, is still locked spatially. I don't know what they are thinking, especially as it go in opposition with this under consideration feature I seriously hope that they will work soon. If you can save lightmaps with prefab, without APV or light probes being movable it look useless.

Please Unity if you can't do that yourself, please at least let us manage lighmaps, light probes and reflection probes ourself. There is no need to hide that, we are not "users" but developers. Right now we have to do workaround and fight against this black box system to make things work like we want.

 

Hey @TheZombieKiller , this package is still very valid, great work! Thanks!
Only one request - when I imported it in 2020.3.x (I do not know about newer versions) I get the following compile errors:


Basically saying something needs to be passed on to the ctor here

Do you have an idea what that could be?

Thanks a bunch!

 

That's a C# 9 feature, which Unity 2020 doesn't support. You can fix it by changing

new()

to

new LightingDataAssetRoot()

.

 

If that's the case, you probably ought to set a minimum editor version on the package, or at least restrict yourself to C# 8 (or 7.3 if any editors on that are still relevant / supported).

 

The functionality should work as far back as Unity 2019 I believe (maybe even earlier), we're just using it in Unity 2021 internally so target-typed new not being supported in 2020 slipped past me. I'll likely push a commit to fix that up later.

 

Bump. Please help us! Anything!

 

Is there a way to reassign the lighting data asset to a new scene with this package? I am wanting to duplicate scenes and not have to rebake. (Simply duplicating a scene the lightingdataasset still works in the new scene but it points to the old scene so if I export a package it wants to bring the old scene as well)

 

Yes, you should be able to duplicate the LightingDataAsset and then modify its internal scene reference:

Code (csharp):

1. var data = ScriptableObject.CreateInstance<ScriptableLightingData>();
2. data.Read(lightingDataAsset);
3. data.scene = newSceneAsset;
4. data.Write(lightingDataAsset);

However, if you don't want to pull in a big dependency just to change the scene reference, this should work too:

Code (csharp):

1. var id   = GlobalObjectId.GetGlobalObjectIdSlow(newSceneAsset);
2. var json = $@"{{
3.    ""LightingDataAsset"": {{
4.        ""m_Scene"": {{
5.            ""fileID"": {id.targetObjectId},
6.            ""guid"": ""{id.assetGUID}"",
7.            ""type"": {id.identifierType}
8.        }}
9.    }}
10. }}";
11.  
12. EditorJsonUtility.FromJsonOverwrite(json, lightingDataAsset);
13. EditorUtility.SetDirty(lightingDataAsset);

 

Oh great thank you!

 

As of Unity 2023.2.0a8, there is a new set of APIs for modifying light probe data (it's not even editor-only!):

Code (CSharp):

1. using UnityEngine;
2. using UnityEngine.SceneManagement;
3.  
4. namespace UnityEngine
5. {
6.     public sealed partial class LightProbes : Object
7.     {
8.         public static LightProbes GetSharedLightProbesForScene(Scene scene);
9.         public static LightProbes GetInstantiatedLightProbesForScene(Scene scene);
10.         public Vector3[] GetPositionsSelf();
11.         public bool SetPositionsSelf(Vector3[] positions, bool checkForDuplicatePositions);
12.         public int countSelf { get; }
13.         public int cellCountSelf { get; }
14.     }
15. }

If no

LightProbes

object is assigned to the scene, then Unity will create one for you with this API, that's awesome! And then you learn the unfortunate news that makes this still useless for many runtime use cases: you still can't change the number of positions/probes stored in the

LightProbes

object.

SetPositionsSelf

will fail if the array length does not match, so while

GetInstantiatedLightProbesForScene

will give you a new

LightProbes

object if there isn't one already, you can't do anything with it at runtime because its internal probe array is empty

 

Can you relay this feedback at:

https://forum.unity.com/threads/prefab-level-generation-light-probe-support.999266/#post-8914573