Search Unity

Every day, I hate asset bundles even more..

Discussion in 'Asset Bundles' started by jbooth, Jun 15, 2017.

  1. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    5,461
    So I know the asset bundle system pretty well. I've shipped a game that has over 10,000 of them on Unity 5.2, and have threaded the needle of bugs, race conditions in the threading code, and undocumented limitations to do that.

    After our last game, I spend about a month and a half writing code to automate the hellishly manual process we used to decide what goes into what bundle, along with lots of tools to manage building multiple variant levels without having to have multiple copies of every texture in the project. I pointed out many limitations and issues to Ian when he was writing his massive manifesto on Asset Bundles, and Unity has asked me more than once to talk to other teams having issues with them.

    Today, I open up the profiler on android and notice 200mb being consumed by SerializedFile. So I search the internet/Unity docs to see what SerializedFile actually is, and it turns up nothing. Turns out there's yet another undocumented 'feature' of asset bundles, which is that each of them consumes HALF A MEG of memory on an android device while open. That means we're actually using more memory for these stupid buffers than we are for the rest of the game.

    Please, for the love of all that is right in the world, send this system to hell and replace it with something that actually has a functioning design and doesn't require you to lube up first. This is insanity.
     
  2. Mazak

    Mazak

    Joined:
    Mar 24, 2013
    Posts:
    226
    <sigh>
     
  3. Reichert

    Reichert

    Unity Technologies

    Joined:
    Jun 20, 2014
    Posts:
    63
    a serializedFile is a simple file format for storing serialized unity objects. There are always one or more serializedFiles registered at runtime with the PersistentManager. The PersistentManager generates instance IDs for every object in the file and creates what are essentially weak pointers to each object on disk. When one of these weak pointers is de-referenced, the target object is loaded from disk from the correct location within the correct serializedFile.

    Every player build contains a few serializedFiles for all of the built-in content, and these are registered at startup with the PersistentManager. Each Asset Bundle is an archive that contains a serializedFile, which is registered when the Asset Bundle is loaded.

    Read buffers are created for each serializedFile that is registered with the PersistentManager. They vary in size based on platform, and whether or not they are contained within an Asset Bundle. On Android, these buffers were pretty large at one point - requiring a total of 512kb per Asset Bundle. This was optimized down to 16kb in Unity 5.5.4, and backported to 5.4.6 and 5.3.8. This forum thread has a little more detail on the problem and the fix.

    @jbooth I see we sent you the patch yesterday for your custom build of 5.3. Looking forward to hearing if that works for you.
     
    Last edited: Jun 16, 2017
  4. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    5,461

    Yes, our 80mb of loaded data only constructs 17mb of these buffers after applying the changes.

    Look, I know there's a lot of memory on modern machines, and while 200mb->17mb is a great change, it still seems kind of crazy that there's such a massive undocumented overhead in loading files. (Forum threads are not documentation). I think the only reason this didn't become a bigger issue before this is that there were so many undocumented bugs and limitations that prevented people from having lots of asset bundles in the first place (the non-POSIX file access which limited # of loaded asset bundles on iOS, the insanity involved in managing the system, etc). So they gave up before hitting these issues.