Question AssetDatabase.CreateAsset() performance

I have created an editor script which converts terrains into meshes.

Since the game world is huge, the script is used to generate a lot (tens of thousands) of meshes.

It only takes a few minutes to generate all the meshes. Saving them, however, can take several hours!

The code used to save them is simmilar to this:

Code (CSharp):

1. // generate meshes here
2. Mesh[] meshes = /* ... */;
3.  
4. try
5. {
6.     AssetDatabase.StartAssetEditing();
7.     AssetDatabase.DisallowAutoRefresh();
8.  
9.     for (int i = 0; i < meshes.Length; ++i)
10.     {
11.         string path = $"Assets/DataFiles/TerrainMesh/TerrainMesh_{i}.asset";
12.         meshes[i].UploadMeshData(true);
13.         AssetDatabase.CreateAsset(meshes[i], path);
14.     }
15. }
16. finally
17. {
18.     AssetDatabase.StopAssetEditing();
19.     AssetDatabase.AllowAutoRefresh();
20.     AssetDatabase.SaveAssets();
21.     AssetDatabase.Refresh();
22. }

Further profiling has revealed that more than 99% of the time is spent on AssetDatabase.ImportAtPathImmediate. EnumerateFiles alone can take more than 100ms!

Furthermore, it seems that saving assets becomes even slower the larger the project gets.

Is there a way to make this faster?

I'm using unity version 2021.3.5f1. The entire project is saved on Samsung980 PRO NVMe SSD.

 

I see this all the time ... hey, let's call every imaginable AssetDatabase method just to be safe! And it gets copied and copied here and there and it never stops ... seriously, you only need those calls that I haven't uncommented (please, call me an idiot if I'm wrong since I don't know your exact use case but I'm 99% sure you won't notice a difference without those lines since the whole idea of "StopEditing" is to stop auto-refresh and for the ADB to perform the refresh right on "StopEditing" while "CreateAsset" naturally implies "SaveAsset"):

Code (CSharp):

1. try
2. {
3.     AssetDatabase.StartAssetEditing();
4. //    AssetDatabase.DisallowAutoRefresh();
5.     for (int i = 0; i < meshes.Length; ++i)
6.     {
7.         string path = $"Assets/DataFiles/TerrainMesh/TerrainMesh_{i}.asset";
8.         meshes[i].UploadMeshData(true);
9.         AssetDatabase.CreateAsset(meshes[i], path);
10.     }
11. }
12. finally
13. {
14.     AssetDatabase.StopAssetEditing();
15. //    AssetDatabase.AllowAutoRefresh();
16. //    AssetDatabase.SaveAssets();
17. //    AssetDatabase.Refresh();
18. }

This probably won't affect your performance issues however. Just something I have a grudge with.

FWIW thankfully you read the docs and put this in a try/finally block.

I wonder, do you really have to call UploadMeshData ? Seems odd that the mesh has to be on the GPU for it to be saved as an asset. Can't remember that I had to do that when generating a mesh but then again, I can't see the mesh gen code and terrain might be different.

What do you do in the case that an asset already exists at "path"? Do you delete the assets before creating them again? I think it *might* be faster if you keep the assets and just update them, using Load/SaveAsset in that case (could make things slower when saving the assets for the first time as the LoadAssets will fail every time).

Tens of thousands of meshes is actually a lot! How big is each mesh, in terms of vertices and size on disk?

Note that you can assign more CPU time to the importer, maybe playing with that helps a bit:

 

Btw, do you change the ENTIRE world every time you do this? You may be able to save a ton of time in this process if you process only the parts of the world that DID change.

Unless you do something like "randomly scatter trees in the world" I bet you rarely change the entire world. Try thinking about how you can track changes and save only modified terrain chunks.

 

Thanks for the clarification @Unity_Javier! Any workaround would be appreciated.

@CodeSmile The script does edit a large portion of the world. However, it does so efficiently (as you have said). It will not open or edit scenes needlesly.

All meshes are optimized before saving, so they only have a few hundred vertices each (at most). Since they are rather simple, the file size is low.

Passing true to UploadMeshData() method will mark the asset as unreadable, which stops unity from keeping a copy of the mesh in memory.

As far as I know, AssetDatabase.StartAssetEditing() only stops asset importing. Shouldn't I also disable auto refresh via DisallowAutoRefresh()?

 

No, that wouldn't make sense. You stopped importing, why would it continue to auto refresh assets? Refresh is essentially the same thing as importing.

And as the example shows, but the description doesn't state, the main purpose of Refresh() is to pick up any changes made by external programs as well as changing the file system where you bypass the AssetDatabase, such as CreateDirectory or file methods like WriteAllBytes().

In essence, everyone and their dog keep adding AssetDatabase.Refresh() after modifying assets, even though it's very rarely needed. It could still be useful because of the garbage collection mentioned in the docs. But I would only add it whenever I find that some process doesn't pick up all changes, such as the new directory/file not showing up in the Project explorer.

 

@CodeSmile , Hello!
I trying to use StartAssetEditing and StopAssetEditing for make better performance of CreateAsset method (creating about 2000+ assets files). But after CreateAsset I need use the link to asset. For example:

Code (CSharp):

1. AssetDatabase.StartAssetEditing();
2. Mesh mesh = CreateMesh();
3.  
4. AssetDatabase.CreateAsset(mesh, _meshPath);
5. _meshFilter.mesh = mesh; //thats line is not working well
6.  
7. AssetDatabase.StopAssetEditing();

After execute that code meshFilter has empty link of mesh. without using methods StartAssetEditing and StopAssetEditing I lose performance, but the link of mesh is correct - file in assets folder. How I can save performance and has a correct link to asset?

 

The easiest way would be to add each created mesh to a list, and then after StopAssetEditing foreach over this list and make the meshFilter.mesh assignment.

But it might also be a flawed thinking here, since you create an in-memory mesh (I assume) and then create an asset out of it, but you assign the in-memory mesh and not the asset you just created. Those might actually be two different instances. To confirm this theory you could try this instead:

Code (CSharp):

1. _meshFilter.mesh = AssetDatabase.LoadAssetAtPath(_meshPath, typeof(Mesh));

But then I'm not sure if LoadAsset will work within Start/StopAssetEditing since the AssetDatabase won't be updated and may not find that new asset. So you may still have to use the list and deferred assignment, except you'd have to store the asset path in the list.

 

@CodeSmile , yes, I generate mesh in-memory, after that I save them through CreateAsset. Without Start/StopAssetEditing the link to in-memory mesh is replaces to link on asset. But within the link is stays in-memory. LoadAssetAtPath not working within Start/StopAssetEditing. But after StopAssetEditing that will work well, you right. But loading asset after Stop is not best way to me, but maybe it be fastest. I check that, thanks!

Is there another way to solve this problem?

 

@Unity_Javier , thanks! I try you code later and will let you know.
I wrote the _meshFilter just for example, is not real code. In my case I generate mesh and ScriptableObject with link on mesh and save them in assets. And after the process the link in ScriptableObject is empty.