Search Unity

  1. Unity 6 Preview is now available. To find out what's new, have a look at our Unity 6 Preview blog post.
    Dismiss Notice
  2. Unity is excited to announce that we will be collaborating with TheXPlace for a summer game jam from June 13 - June 19. Learn more.
    Dismiss Notice
  3. Dismiss Notice

AssetBundles for dynamic content updates and upgrading Unity versions

Discussion in 'Asset Bundles' started by adamt, Dec 7, 2016.

  1. adamt

    adamt

    Joined:
    Apr 1, 2014
    Posts:
    116
    We use AssetBundles as a remote content delivery system for our mobile game. Like others have mentioned, this was probably the single largest time-sink in our project, and it's still not quite there. One thing I'm still grappling with is how to manage cached AssetBundles when upgrading versions of Unity. It seems like with upgrades from Unity 5.3 to 5.4, and now from 5.4 to 5.5, something has changed (likely shader-related) such that all my sprite assets show up as bright pink rectangles on-device after upgrading Unity but not rebuilding AssetBundles.

    The process (including pitfalls) I used from 5.4 to 5.5 is this:
    1. Call BuildPipeline.BuildAssetBundles (chunk compression, no forced rebuild)
      1. Realize this doesn't actually seem to update any of the AssetBundles, because whatever that method is doing doesn't seem to think the AssetBundles have changed (even though they apparently have). This sounds like it might be a bug.
    2. Delete the AssetBundles on-disk on our build machine and re-run the build process in step 1. This recreates the AssetBundles in the Unity 5.5 format and uploads them to our CDN.
    3. Open our game built with Unity 5.5 on-device.
    4. Our custom content manager class runs and downloads the AssetBundleManifest from our CDN
      1. We iterate over the bundle names in AssetBundleManifest#GetAllAssetBundles()
      2. Using UnityWebRequest.GetAssetBundle(string uri, Hash128 hash, uint crc) with the hash set to AssetBundleManifest#GetAssetBundleHash(), we attempt to download each AssetBundle
        1. This is again troubling because it seems as though the hashes that the AssetBundleManifest file reports for the AssetBundles matches the previously downloaded AssetBundles' hashes (built in Unity 5.4), even though they have been completely rebuilt between Unity 5.4 and 5.5. Shouldn't those hashes be different, since the AssetBundles are seemingly different between Unity versions?
    5. At this point, we realize we need to completely clear the cached (previously downloaded) AssetBundles on-device using Caching.CleanCache(), and have our content management system re-download every AssetBundle built for Unity 5.5.
    Now, I feel like there are at least two bugs here (step 1.1 and step 4.2.1), but it also could just be that I'm misunderstanding the AssetBundle system. The overall workflow issue I have is this: How can I periodically update Unity versions once our game is launched (fortunately, it's still in development) without having our users completely re-download AssetBundles built for each new version? I'd like this process to be iterative and seamless, where users don't have to re-download any AssetBundles, unless their contained assets have been updated by our artists.

    This really seems like the worst of both worlds, because not only do we not have any noticeable changes between the AssetBundles (again, all I've done between 5.4 and 5.5 is delete the AssetBundles from our CDN and regenerate/re-upload them), but our users have to download ALL of the AssetBundles again, and won't really have any idea why.

    Is there some kind of standard process (other than "don't upgrade your tools on a shipped game") I'm missing here? We very well may peg the version of Unity with the launch of our game, but I'd still like to reserve the right to upgrade it if something amazing happens in 5.6, 5.7, and beyond, without our users having to re-download every AssetBundle in our game.
     
  2. Reichert

    Reichert

    Unity Technologies

    Joined:
    Jun 20, 2014
    Posts:
    63
    adamt,

    I'm somewhat surprised, but it's possible that the hashes remained stable between Unity versions if you didn't touch any assets. If shaders needed to be recompiled for whatever reason, unless the source shader also didn't change, I would expect that to cause a hash change as well. So at the very least there is probably a bug related to dependency gathering and hashing with shaders.

    As far as bundle determinism and forwards compatibility in general, I recognize we are not in a good spot with this issue. Besides a handful of known bugs that just need to be fixed, we don't have clear guidelines published for what you should be able to expect from us - this needs to be created. Lastly, while there are mechanisms in place to provide forwards compatibility of bundles to new Unity versions, there are various ways to break or circumvent those mechanisms, so we also have some work to do to detect when that happens so it can either be fixed, or noted as a breaking change in our release notes.

    Something we will be working on in the coming weeks and months.

    -Stephen
     
  3. adamt

    adamt

    Joined:
    Apr 1, 2014
    Posts:
    116
    Thanks for the response, Stephen.

    It very well may be that I have a bug in my hash checking code, although I tried my best to stick to what Unity gives me out of the box (i.e., simply calling UnityWebRequest.GetAssetBundle with AssetBundleManifest#GetAssetBundleHash), but clearly something funky is happening. I can try to reproduce this and file a bug, but it's sort of a time-consuming information gathering process. Maybe this is something that'd be easier for someone on your team to reproduce? I'm happy to provide more context (e.g., code, Jenkins pipeline steps, etc.) if it's something you think you'll a.) ever realistically look at (I'm a pragmatist -- I understand development team priorities don't always mesh with squeaky wheels) and b.) would help if you're committed to looking into it.

    I watched your video from Unite LA yesterday and it gave good information about where you and the team are headed in the coming months. I'm excited about the updated pipeline as well as the tooling that Hiroki has been working on. Maybe there's an opportunity for me to hook into one of the various points in the pipeline to support the kind of upgrade-Unity-without-breaking-existing-users-cached-AssetBundles workflow. Maybe there's even a case to be made for baking in support at a higher level of the API.

    I also wanted to just take a moment to applaud your AssetBundles and Resources guide. I stumbled upon it (keyword: stumbled -- it should be plastered all over your documentation how to find incredible content like this!) while I was building my content delivery system and was an invaluable part of that piece of our game's puzzle. Really understanding what's going on under the hood, especially since so much of the AssetBundle code is a black box, helped me to reason about the organizational structure of my assets and how to solve various issues that had plagued us (e.g., understanding what the pink rectangle sprites portended, even if the solution has been messy).
     
  4. Reichert

    Reichert

    Unity Technologies

    Joined:
    Jun 20, 2014
    Posts:
    63
    adamt,

    Someone on my team is going to reach out to you about getting a repro case together for the hash issue.

    I can't take credit for the Resource Guide - that was put together by our support engineering team a few months ago. But I agree, I point everyone at that as must-read material.

    We are now underway in our effort to completely revamp Asset Bundle documentation, where the Resource Guide will eventually be integrated directly.

    -Stephen
     
  5. nindim

    nindim

    Joined:
    Jan 22, 2013
    Posts:
    130
    Hi adamt,

    Is this the guide you were referring to?

    https://unity3d.com/learn/tutorials/topics/best-practices/guide-assetbundles-and-resources

    I am looking to gain knowledge on this subject as I have just started with AssetBundles. Please see my other post where I am experiencing an issue with the local server in case you have a fix for it:

    https://forum.unity3d.com/threads/assetbundle-manger-local-server-broken-on-windows-10.449268/

    Thanks a lot,

    Niall


     
  6. adamt

    adamt

    Joined:
    Apr 1, 2014
    Posts:
    116
    @nindim

    Yes, "A Guide to AssetBundles and Resources" was the guide I was referring to.

    With regard to the local server included in the AssetBundleManager, I only used it briefly before rolling my own version of Simulation Mode when developing locally (it's basically the same as AssetBundleManager's implementation with a bit of configuration sprinkled in). Building the AssetBundles every time got really cumbersome/slow, so it just wasn't an option for us, unfortunately.

    One way you might be able to circumvent the need for the local server is by referencing the AssetBundles using the file URI scheme (e.g., file:///Users/nindim/project/AssetBundles/Windows/Windows). Both UnityWebRequest and WWW are capable of locating assets using that scheme as if they were remote-hosted assets on your CDN. I'll admit I'm not sure if this will work when it comes to App Slicing and variants, but it might give you a place to start.

    Good luck!