Huge amount of hi-poly meshes => long re-import. AssetBundles/Addressables to the rescue?

Hi,

I have a huge project with many gigabytes of meshes for scientific visualization (fluid simulation datasets of several hundred time step meshes, each very high-poly; reducing the poly count is sadly not an option). Recently, I switched from Unity 5 to 2018.2 after finding out that the limitation of 65k vertices per mesh was removed at some point, speeding up the import process quite noticeably.

However, loading the project in the new version meant re-importing everything, which still took hours and didn't even finish when all meshes were still in the project as Unity ran out of RAM (64GB!).

All this got me thinking that there must be a better way to deal with this mass of meshes. I read that AssetBundles are forward-compatible and thus wouldn't need to be re-built and re-imported in newer versions (right?). I thought about having separate projects per dataset, importing the needed meshes there once and building AssetBundles containing all meshes needed for one dataset / scene in the visualization project. But I'm unsure about a few points:

• when would an AssetBundle be loaded? Is it possible to load it once in the Editor/during build and then have the same/comparable access times as if the contained meshes were imported separately?
• Would there be any way to edit the meshes through the GUI (location, scale, ....) or just through code? Or would I need to make any modifications in the "data projects" and rebuild the related AssetBundle?

I also read about the new AddressableAssets, but couldn't really get a feeling whether or not that feature is already stable enough and if it would fit my purposes better than AssetBundles, does anybody have some insight here?

Thanks in advance!

 

asset bundles are runtime-only: they are compiled for a specific platform and don't work on others.
addressables is built on top of asset bundles.

I think what you are looking for is either the cache server, or importing your stuff a few at a time (maybe scripting it?)

you can (manually) make asset bundles cross-project though: you can have a Unity project for each set, build bundles from it, then load its assets by name.
but you can't have shared dependencies, and used scripts must be exactly the same across the bundle projects AND in the main project (at least: same guids, same names/namespaces, same assemblies, and same serialization layout)

 

Thanks for the answer!
The platform-specific part shouldn't be a problem for me I guess, because everything I described would happen on one single machine.
Importing models in batches helps, but I can't force Unity to do so when it re-imports all assets upon loading an older project in a newer version.

> I'm guessing the scripts you mention would be any that are placed on the models to be put in an AssetBundle and not the ones used to build the bundles - in that case, that wouldn't be a problem, as I would only add meshes to the bundle without any other assets attached.

> That sounds like what I was planning to do, but I'm still not sure when the bundle would be loaded. Or would I load every mesh separately in code by name just when I need it and thus never load the whole bundle at once?

I will have to read about the cache server, haven't stumbled upon it yet.

 

that's not what I mean: you can build asset bundles for e.g. OSX Standalone, then only an OSX Standalone player can load them. editor can simulate them in playmode, but only for playing.
you could try to save them using

AssetDatabase

, but I don't think it can work (except possibly for gameobjects, as they are internal types. all imported assets will break.
for instance, a mesh in Unity is not an fbx/obj/whatever. it is an internal format that gets created from the source (and actually lives in Library/)

you can create a new project, export your scripts and essentials from the old project via .unitypackage, then import your other stuff in multiple batches. you can use .unitypackage to keep references across them, just import the stuff in the new project in the correct order (if any)

if your stuff has no shared dependencies, (i.e. each model has its own textures and materials, or they are isolated) you could automate via script (old project -> find all models -> for each model create a unitypackage and include dependencies -> in the new project import the unitypackages one at a time)

 

Hi, thanks again for the answers!

That limitation still shouldn't apply in my case, because the asset types I would put into AssetBundles would be strictly limited to models/meshes and maybe a single material, but no scripts.

Maybe a more concrete example could help make my needs more comprehensible: Currently, I have one scene in my single monolithic project with an empty GameObject and a script on there to spawn sequential time steps of a fluid simulation. In order to do so, the script has a public GameObject array, through which I specify the ~200 models to be spawned, each representing a point in time of the simulation. The script then loops through these models, spawning a new model from the array after destroying the last one every few milliseconds. Notably, the models themselves are only spawned and destroyed by that script, but have no dependencies on anything else in the project besides their material, which I could easily "extract" to another project / an asset bundle.
There are five more scenes with the same setup but different sets of models. So my hope was, that for each scene, I could have a separate project in which I would import all the models for that scene once, bundle (only) them into an asset bundle and then use that asset bundle in my main project, which would then - in theory - be independent from the models in a sense that it could be updated to new Unity versions without having to reimport every single model or resorting to some (semi-)automated batch import mechanism, as the models themselves would no longer be part of the project. That last point is what I'm unsure about, whether building an asset bundle containing only models "transforms" them into a comparable (performance-wise) representation which can then be loaded quickly in the main project the same as having imported the models there directly as I am doing now. Assuming that this is a workable approach, would I then simply point my spawn script to the asset bundle and have it load the time step models by name when needed?
And also what degree of control I would have over the models once instantiated (things like assigning another material, changing transform values etc)?
Or would using an asset bundle that way require it to first be imported as a whole - in the worst case every time the scene is entering play mode?

The "problem" of asset bundles being platform-specific would be subordinate, as we rarely do standalone builds and mostly just (ab)use the in-editor preview when showing something, as the project evolves all the time.

I hope I'm slowly approaching a point where it's comprehensible what I'm trying to achieve and why, please bear with me. Also sorry if what I'm going on about is over-complicated and hard to understand, I'm having a hard time trying to express what I'm after.