dynamic, server-side asset bundle creation odyssey

Hi folks,

I'm creating dynamic asset bundles server side with some editor scripts. Unfortunately the process for doing this is rather tedious and error-prone (as the last Update to Unity 5.3 proved again).

I'd be super glad if anyone's got tips on how to make it better or if some Unity guys would consider looking at this and improve the situation long-term.

I'm using the AssetPostprocessor API, which is handy, but due to it's asynchron interface far from ideal in our case. Examples:
There's no way to get notified when a prefab (which I created before via script) is finished importing/saving, which is necessary for building it into an asset bundle Therefore I use OnPostprocessAllAssets for it, which doesn't get called once for all objects, but multiple times with varying subsets of imported objects. I have to manually keep track of my stuff, which is very very prone to errors.

Things that would make it easier:
- Something like OnPostprocessModel for other types (Prefabs, Materials, ...)
- A dependable Callback when AssetDatabase.Refresh or AssetDatabase.SaveAssets are finished. Additionally feedback on what they actually did (like how many things were dirty).

Here's how it works in a bit more detail:

1. Fire up Unity from command line (headless) with all information passed through parameters
2. Analyze source folders. They're outside of the project, usually contain FBX files, textures and JSONs with further meta-data. Main result is the number of Models (FBX) to be imported.
   
3. Copy folder(s) into the Assets folder
4. call AssetDatabase.Refresh();
5. Hook into OnPreprocessModel to change importer settings
6. Hook into OnPostprocessModel an remember the item.
7. Hook into OnPostprocessAllAssets and loop over imported assets:
   
   1. If we're in the process of saving (Like in step 7.5), skip
      
   2. If it's a material, change it according to some rules (Which will trigger another number of calls to OnPostprocessAllAssets again).
   3. If it's an item we remembered from OnPostProcessModel, create a prefab of of it (I add some scripts and stuff) and keep track of all prefabs created so far (Which will trigger another number of calls to OnPostprocessAllAssets again).
      
   4. If it's a previously generated prefab, add an AssetBundleBuild entry to a list.
   5. Check if the list is complete ( by comparing the number of entries to the number found at step 2.). If so: Save everything by calling AssetDatabase.SaveAssets(); AssetDatabase.Refresh(); (Which will trigger another number of calls to OnPostprocessAllAssets again)
      
   6. If the list is complete AND we've saved before, finally build all asset bundles via BuildPipeline.BuildAssetBundles.
8. Exit Unity

As you can see, this process is insanely poluted with recursive calls of OnPostprocessAllAssets and a naive attempt to duct tape it all together. I had to invest lot of time to get it running kind of stable, but with the slightest change (like the Unity 5.3 update), it's completely broken again.

Any suggestions?

Thanks for your time,
Andreas

 

I got it running again.
1. There was an unrelated bug causing an out of memory exception
2. The behaviour on when/how OnPostprocessAllAssets is called after invoking SaveAssets() must have changed somehow. I had to spend some hours debugging where the thing failed.

Which proves that my build scripts are poorely designed, but as far as I can see Unity doesn't give me alterantives

 

unfortunately no good news here.
I glued up my code, that holds togehter the whole thing as best as I could and it seems to run for now.

 

Yes, we use it in production. At the moment as a one-time process before publishing the content (with manual checks).
But I'd reconsider it for fully automatic requests.

 

I'm building a service like that for internal usage atm. Won't be production ready soon, but PM me if you're interested.

 

Our service is running for internal use atm and still has some way to go