Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. Dismiss Notice

Extremely slow loading

Discussion in 'Addressables' started by RogueCode, Mar 29, 2020.

  1. RogueCode

    RogueCode

    Joined:
    Apr 3, 2013
    Posts:
    230
    My game has about 8000 prefabs (3D models). Currently every one of those assets is in a scene that I load at runtime to preload everything, and then I instantiate copies as needed. I want to move to loading on demand so it doesn't load 90% of stuff it doesn't need (which I can't tell beforehand because there is a level editor in the game).

    I've switched over to Addressables and nothing I've tried has been remotely fast enough. The scene descibed above loads in about 2 seconds on my PC in a Windows build. Loading those same assets with addressables takes about 100 seconds.
    Every asset is in 1 large group, and has a unique address. As a test to load every asset:

    Code (CSharp):
    1.     var loadingOperations = new List<AsyncOperationHandle<GameObject>>();
    2.     foreach (var item in keys)
    3.     {
    4.       loadingOperations.Add(Addressables.LoadAssetAsync<GameObject>(item));
    5.     }
    6.  
    7.     while (true)
    8.     {
    9.       yield return null;
    10.  
    11.       var anyNotDone = false;
    12.       foreach (var item in loadingOperations)
    13.       {
    14.         if (!item.IsDone)
    15.           anyNotDone = true;
    16.       }
    17.  
    18.       if (anyNotDone)
    19.         continue;
    20.  
    21.       break;
    22.     }
    While loading, it is barely doing anything:
    upload_2020-3-29_22-23-40.png

    How would I get this to load faster? I'd be happy if it just completely blocked while loading and finished it quickly.

    Thanks!

    EDIT: I've tried disabling CRC and compression and it didn't seem to make much difference. Also tried packing separately, and that seems a bit quicker but still very slow and increases the build size dramatically.
     
    Last edited: Mar 30, 2020
  2. TreyK-47

    TreyK-47

    Unity Technologies

    Joined:
    Oct 22, 2019
    Posts:
    1,795
    I'll flag this for the dev team to take a look. Which version of the editor and Addressables are you using?
     
  3. RogueCode

    RogueCode

    Joined:
    Apr 3, 2013
    Posts:
    230
    I'm using v1.7.5 (it is the same on 1.6.x), and tried in both 2020.1.0b4 and 2019.1.4f1. It behaves the same everywhere.

    I've done some more tests in a new clean project. I imported the Synty assets FantasyKingdom pack (2130 prefabs), and dragged those into a group. All of the following were loading all 2130 assets, and initializing them.
    All of these were done in non-debug Windows builds, restarting Windows between each just to make 100% sure Windows wasn't caching anything.

    A) Individually loading assets via addressables:
    HDD - 9669ms
    SSD - 7708ms

    B) Making a single prefab with a script with property List<GameObject> referencing all 2130 prefabs, and loading that single prefab by addressables, then instantiating the assets by looping through that list and doing a normal Instantiate:
    HDD - 1868ms
    SSD - 1518ms

    C) Not using addressables. Creating a scene with all assets referenced in a script like above. Then async loading the scene, and when loaded looping through the list and instantiating normally:
    HDD - 4352ms
    SSD - 796ms


    The scene loading way has results exactly like I'd expect, where the HDD is much slower than an SSD. But look at the addressables results - there is barely any gain on an SSD. To me this indicates exactly what I was saying in the first post, that addressables seem to be intentionally loading slowly (or at least they aren't hitting any hardware limit).
    It really seems like each call to addressables has a massive delay where it isn't loading the asset. This means that the size of the thing you're loading doesn't matter at all, but simply the number of different assets you have addressable. Requesting addressables to load a staircase takes as long as it does requesting it to load a prefab that has 1000 different types of staircases in it.

    The results for B) seem promising, except in a real situation I don't think that would help. The whole point of trying to use addressables is so we don't have to load stuff we don't need. And manually grouping prefabs up into meta List<GameObject> prefabs to reduce the addressables calls will mean that once again we're loading tons of stuff we don't need.

    EDIT: Also tried serializing a List<AssetReference> of references to those 2130 assets hoping that the part taking long was the internal mapping lookup between the address and the actual asset, but it didn't get faster (at least not by much).
     
    Last edited: Apr 5, 2020
    Lex4art likes this.
  4. davidla_unity

    davidla_unity

    Unity Technologies

    Joined:
    Nov 17, 2016
    Posts:
    736
    Hey @RogueCode have you tried changing your addressable group to Pack Separately under the Advanced Options? Loading a single asset bundle with 8000 prefabs is bound to take a little time. If you're wanting to load your prefabs on demand I'd suggest changing that group to Pack Separately that way you don't incur that overhead for the entire group when you just need a few prefabs out of the group.
     
  5. RogueCode

    RogueCode

    Joined:
    Apr 3, 2013
    Posts:
    230
    Yes I did, it helped a bit, but still unusable.

    I've switched to straight assetbundles since this last post. Much easier to understand/use and I'm getting great perf (just a tiny bit slower than the scene loading way in some cases, and faster in others).
     
  6. Thermos

    Thermos

    Joined:
    Feb 23, 2015
    Posts:
    148
    ..I thought, in the past few years, by loading an asset bundle into memory means it only load the manifest(or category whatever you call it) , and then you can choose to actually load individual asset, but you can only unload by unload entire bundle or use Resources.UnloadUnusedAssets??...
     
  7. TreyK-47

    TreyK-47

    Unity Technologies

    Joined:
    Oct 22, 2019
    Posts:
    1,795
    Say, @RogueCode, what compression are you using that is straight AssetBundles that solved your issue? If regular AssetBundles are faster, you might file a bug report.

    @Thermos I spoke with the team, and wanted to pass along that we don't use the manifest anymore since our content catalog is basically what keeps track of what is where.

    You all could make sure your compression is LZ4 on the Default addressable group. It should be, but if that was changed it might affect your load times
     
  8. RogueCode

    RogueCode

    Joined:
    Apr 3, 2013
    Posts:
    230
    @TreyK-47 ^
    Tried all the compression types.

    This slowness is really easy to repro. Stick a few thousand assets into addressables and try load them all. Then do the same with a bundle and it is an order of magnitude faster. Regardless of CRC and compression.
     
  9. DoubleLoopAndy

    DoubleLoopAndy

    Joined:
    May 1, 2020
    Posts:
    3
    I ran into a similar problem myself. As it turns out, the AssetDatabase and Simulated Groups providers simulate virtual bandwidth. When requesting assets one-at-a-time it's effectively wasting a large portion of that virtual bandwidth since it doesn't track the unused portion for upcoming requests.
    When I switched things to request *all* assets up front and then wait on all of them to finish afterwards (using Task.WaitAll on the individual tasks), the load time was back to what I expected.
     
  10. RogueCode

    RogueCode

    Joined:
    Apr 3, 2013
    Posts:
    230
    Just for context, this wasn't the issue I was having. Mine was in a build, not using the assetdatabase.

    @TreyK-47 did this issue ever get resolved?
     
  11. bbridgesvb

    bbridgesvb

    Joined:
    Jan 4, 2019
    Posts:
    11
    Hi, was there ever any resolution to this? My team and I are having issues in a WebGL app where loading too many Addressables at once seems to take a considerable amount of time, and we're trying to determine if this issue is related or not.
     
  12. TreyK-47

    TreyK-47

    Unity Technologies

    Joined:
    Oct 22, 2019
    Posts:
    1,795
    Sorry for the delay! Let me ping the team about the status of this.
     
  13. davidla_unity

    davidla_unity

    Unity Technologies

    Joined:
    Nov 17, 2016
    Posts:
    736
    We've made some runtime loading improvements (and continue to do so) in newer versions of the package (1.16.15 is the newest at the time of writing) so it may be worth attempting to upgrade. Otherwise it's likely worth filing a bug using the bug reporter tool so we can check out what's going on.
     
  14. peterneufeld

    peterneufeld

    Joined:
    Feb 6, 2020
    Posts:
    3
    I've also been trying to debug why Addressables' LoadAssetAsync is *so slow*. It's extra frustrating when my black loading screen is just spinning with zero load, while LoadAssetAsync is going at a snail's pace in the background. I recently refactored my project to stop using Resources.Load, and now I find that load performance is *horrible* compared to what I had before (very frustrating!).

    So far, from trying things, the following seems to help:
    1) Application.backgroundLoadingPriority = ThreadPriority.High

    Doing this speeds up loading times quite a bit on my PC, but on some platforms seems to result in random main thread lockups - race condition =( I guess on a slower machine getting CPU starved, and the background thread fastlooping while waiting for something from said main thread.

    2) In project settings, increase Async Upload Time Slice and Async Upload Buffer Size. I don't know what good values for these are, so I just tried 16 and 64 respectively. This might be placebo or something, but this seemed to help LoadAssetAsync in some cases.


    I understand that Unity is trying to make the Addressables interface work for both remote bundles and local, but it's really unacceptable that LoadAssetAsync would take 500ms to load 3kb of prefab data off disk! It looks like forcing Addressables to load asynchronously puts loading into a really low priority side thread, and the lack of an actual synchronous API call means that there's no way to get the *main* thread to do the loading like say during a black loading screen.
     
    DavidZobrist likes this.
  15. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,438
    What Unity and Addressables versions did you use before and use now? On which platforms do you see the performance issue?
     
  16. peterneufeld

    peterneufeld

    Joined:
    Feb 6, 2020
    Posts:
    3
    Switch is the main platform with the biggest issue (where it takes literally seconds to load), but I can see that even on PC my loading times are up 10x. Even 0.5s loading time between very small and simple room prefab is not acceptable, when I had practically instantaneous before when I was loading with Resources.Load

    Unity: 2018.4.29f1
    Addressables: 1.16.15

    (I made the switch to addressables a few days ago)
     
  17. peterneufeld

    peterneufeld

    Joined:
    Feb 6, 2020
    Posts:
    3
    FYI I tried the SyncAddressables Providers examples from the Copenhagen talk, but didn't see a huge improvement. As far as I can tell the slowdown is probably in some sort of polling background code in ResourceManager maybe (but that code isn't accessible so I'm SOL there).

    Anyways, I stuck an interface in front of my LoadAssetAsync class and wrote a class underneath which just uses Resources.Load, and my loading times are back to normal again (milliseconds instead of seconds).
     
  18. Morphus74

    Morphus74

    Joined:
    Jun 12, 2018
    Posts:
    174
    Has that issue ever been resolve, loading without addressable is immediate, loading with Addressable, once download is completed, take 15-25 seconds.

    I use LoadSceneAsync, Scene bundle is about 275MB and a secondary bundle is 172M.
     
  19. gsylvain

    gsylvain

    Joined:
    Aug 6, 2014
    Posts:
    96
    Chiming in as well, we're using a lot of large addressable groups. Loading are surprisingly fast in the editor but as soon as we do a standalone build, loading time are super long while at 260+ fps. The problem gets even worst on console as we are dangerously near the deadly 3min marker (which is a TRC fail).

    Unity 2020.3.0f1 + Addressables 1.16.16
     
  20. RogueCode

    RogueCode

    Joined:
    Apr 3, 2013
    Posts:
    230
    Is this maybe because in the editor you have it set to load from the assetdatabase?
     
  21. xjjon

    xjjon

    Joined:
    Apr 15, 2016
    Posts:
    591
    Did you find any fix with this in 2020.3
     
  22. gsylvain

    gsylvain

    Joined:
    Aug 6, 2014
    Posts:
    96
    Hello xjjon,

    we finally got a solution. Our loading times went from 3 min to 20 sec on xbox one.

    1. AddressableAssetSettings: enable "Contiguous Bundles"
    2. For every addressable group assets, you may want to disable both "Use asset bundle CRC". We use the asset bundle cache.

    We are using the basic "LZ4" compression. I also want to mention that we had a bit of group refactoring to do to prevent having multiples instances of a specific assets across multiple groups.

    I hope these few tweaks will help you!

    Cheers,
     
    DavidZobrist likes this.
  23. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,438
    What's the idea of that CRC anyway? If the CRC check fails, then what? Download the asset bundle again? Display error message and ask user to quit game?
     
  24. gsylvain

    gsylvain

    Joined:
    Aug 6, 2014
    Posts:
    96
    I would say to make sure that the file has not been tampered on the disk (user hack) or during the download (bad packet received). Each project must evaluate how their files are deployed and what is your risk tolerance.
     
  25. jmnel

    jmnel

    Joined:
    Oct 9, 2016
    Posts:
    2
    My game has 2d prefabs only with 500 Mb, every time I change something in the code it takes 3 or 5 minutes to load everything, every time when I hit play it takes 5 or 7 minutes to load, and don't forget to build it takes more than 20 minutes sometimes, I'm using Unity 2020.2.6, this never happened in unity 2018 when, I started to use it I was very enthusiastic with the engine, now I'm rethinking to switch engine due this issue is 2d only, because what will happen when I start to create something in 3d.