Search Unity

  1. Unity 6 Preview is now available. To find out what's new, have a look at our Unity 6 Preview blog post.
    Dismiss Notice
  2. Unity is excited to announce that we will be collaborating with TheXPlace for a summer game jam from June 13 - June 19. Learn more.
    Dismiss Notice

Bug Building addressables with cached elements using ScriptableBuildPipeline 1.21.21 EXTREMELY slow

Discussion in 'Addressables' started by SVC-Games, Nov 2, 2023.

  1. SVC-Games

    SVC-Games

    Joined:
    May 21, 2013
    Posts:
    140
    After upgrading Addressables to 1.21.19 and ScriptableBuildPipeline to 1.21.21 we found that there's a bug building with Build -> New Build -> Default Build Script

    If there's cached elements the process freezes when reaching the point shown here:

    upload_2023-11-2_18-33-13.png

    If we clear the cache it builds with no problem at all.

    Also, when building addressables 'slimWriteResults' are overwritten from true to false. Can this be related since that flag is to improve cache performance?

    upload_2023-11-2_18-33-54.png

    EDIT: This also happens when using 'Update a previous build'
     
    Last edited: Nov 2, 2023
  2. SVC-Games

    SVC-Games

    Joined:
    May 21, 2013
    Posts:
    140
    Further investigation: Tried just updating ScriptableBuildPipeline to 1.21.21 without updating addressables and it happens. Reverting to 1.21.20 (and addressables 1.21.19) work without a hitch
     
  3. SVC-Games

    SVC-Games

    Joined:
    May 21, 2013
    Posts:
    140
    It turns out it doesn't freeze, building just becomes INCREDIBLY slow. What otherwise would took less than a minute takes now several minutes.

    We performed the following test with ScriptableBuildPipeline 1.21.20 and 1.21.21:
    • Clear all addresables build cache.
    • Perform a full build. (New Build -> Default build script)
    • Changed a string in a prefab contained in an asset group (only affecting a bundle)
    • Perform a full build and time it (only 1 bundle should be updated, the other are the previously built versions)

    The results are as follows:
    • With ScriptableBuildPipeline 1.21.20: 42 seconds.
    • With ScriptableBuildPipeline 1.21.21: 18 minutes!!

    Don't know if slimWriteResults might be the culprit here, but its certainly suspicious and it gets overwritten to false everytime we try to build.

    EDIT: Just rebuilding again with New Build -> Default build script after a first clean build, even without having made any changes, also triggers this behavior
     
    Last edited: Nov 3, 2023
  4. Alan-Liu

    Alan-Liu

    Joined:
    Jan 23, 2014
    Posts:
    397
    Looking into the modification of SBP 1.21.21, it seems there is a regression in
    CalculateSceneDependencyData
    .

    upload_2023-11-3_17-26-44.png

    As you can see, the
    break
    was moved inside the
    if
    block.

    If the scene contains multiple Sprites,
    CalculateSceneDependencyData
    would call
    ContentBuildInterface.CalculatePlayerDependenciesForScene
    only once in 1.21.20, but in 1.21.21, it would call once for each Sprite!

    Calling
    ContentBuildInterface.CalculatePlayerDependenciesForScene
    once should be enough.
     
    Last edited: Nov 6, 2023
  5. SVC-Games

    SVC-Games

    Joined:
    May 21, 2013
    Posts:
    140
    Wow, nice catch there!

    Is there any way to report this to the Addressables / ScriptableBuildPipeline team?
     
    Gamedelta likes this.
  6. Alan-Liu

    Alan-Liu

    Joined:
    Jan 23, 2014
    Posts:
    397
    I will submit a bug report to Unity. Before dev team fixes it, you can move SBP to Packages folder and fix it manually.
     
    chrismarch, Gamedelta and SVC-Games like this.
  7. SVC-Games

    SVC-Games

    Joined:
    May 21, 2013
    Posts:
    140
    Thanks for the tip, we got scared there for a second, but after changing the 'break' positioning it works as expected.

    Please, if you submit the bug repot to Unity Issue Tracker share the link so we can vote it

    Regards!
     
    Alan-Liu likes this.
  8. Alan-Liu

    Alan-Liu

    Joined:
    Jan 23, 2014
    Posts:
    397
    The bug report has been submitted: IN-59732. Once Unity QA confirmed the bug, I will share the public link here.
     
    SVC-Games likes this.
  9. CDF

    CDF

    Joined:
    Sep 14, 2013
    Posts:
    1,321
    Unity Regressionables
     
  10. SVC-Games

    SVC-Games

    Joined:
    May 21, 2013
    Posts:
    140
    "Production Ready" is, at least, questionable...
     
  11. CaioRosiscaKG

    CaioRosiscaKG

    Joined:
    Oct 13, 2020
    Posts:
    4
    Any updates on this?
     
    SVC-Games likes this.
  12. Alan-Liu

    Alan-Liu

    Joined:
    Jan 23, 2014
    Posts:
    397
    No, based on my experience, it usually takes days, even weeks for a bug report to be confirmed by Unity QA.
     
  13. studentutu

    studentutu

    Joined:
    Oct 22, 2017
    Posts:
    128
  14. SVC-Games

    SVC-Games

    Joined:
    May 21, 2013
    Posts:
    140
    Do we have any news on this regression?
     
    gooby429 likes this.
  15. BrooklynDiPi

    BrooklynDiPi

    Joined:
    Sep 30, 2019
    Posts:
    4
    Can confirm I experience this with version 2022.3.13 onwards. My last safe version to build from is 2022.3.11f1, which uses Addressables 1.21.18 with Scriptable Build Pipeline 1.21.20. As mentioned above, building takes an hour or so on the broken version where it would normally be 20 minutes.
     
    SVC-Games likes this.
  16. Alan-Liu

    Alan-Liu

    Joined:
    Jan 23, 2014
    Posts:
    397
    The bug report has not been processed by Unity.
     
    SVC-Games likes this.
  17. SVC-Games

    SVC-Games

    Joined:
    May 21, 2013
    Posts:
    140
    Bugger... Maybe @TreyK-47 , @andymilsom , @davidla_unity or @Ryanc_unity can check this out? I've seen a couple post around describing basically the same thing and is quite the regression for people using addressables
     
    mgregoirelds likes this.
  18. joonturbo

    joonturbo

    Joined:
    Jun 24, 2013
    Posts:
    77
    EDIT: turns out Unity automatically finds packages that you move in to the Packages folder, and updated the package manifest. Didn't need to do anything.

    I'm being hit by this as well.
    Can someone guide me to how I can set up the ScriptableBuildPipeline package to be in my project instead of in the package manager?

    I don't have any other custom packages, so I don't know what this should look like after I copy the package folder into UnityProject/Packages

    this is what it looks like now, but how doI change this to point to the new path?
    Code (CSharp):
    1.   "com.unity.scriptablebuildpipeline": {
    2.       "version": "1.21.21",
    3.       "depth": 1,
    4.       "source": "registry",
    5.       "dependencies": {},
    6.       "url": "https://packages.unity.com"
    7.     },
     
    Last edited: Nov 29, 2023
  19. gooby429

    gooby429

    Joined:
    Aug 13, 2019
    Posts:
    140
    any month now it will be fixed :)
     
    forestrf likes this.
  20. KeigoTakamura

    KeigoTakamura

    Joined:
    Mar 8, 2022
    Posts:
    6
    After updating to Unity 2022, Addressable was unusually slow, so I was looking into what was wrong and found this thread.
    Confirmed with 2022 3.14f.
    Continue to follow this news
     
  21. JulenScarecrowStudio

    JulenScarecrowStudio

    Joined:
    Jan 17, 2018
    Posts:
    20
    Is this currently being fixed?
     
  22. Alan-Liu

    Alan-Liu

    Joined:
    Jan 23, 2014
    Posts:
    397
  23. Gluckly

    Gluckly

    Joined:
    Nov 7, 2017
    Posts:
    1
    Using now 2022.3.15f (moved from 2021.3.7f) with this problem on: com.unity.scriptablebuildpipeline": version": "1.21.21"

    I just reinstalled back to: com.unity.scriptablebuildpipeline": version": "1.21.20" from github. Convert zip to tgz and install through Package manager tarball.

    1.21.20 - Good every platforms (PC, Xbox, Switch, PS) - 15 min build time for one platform
    1.21.21 and later - Build each one around 3h

    Works for me.
     
    dnnkeeper likes this.
  24. jawnwee

    jawnwee

    Joined:
    Nov 4, 2019
    Posts:
    33
    Following here -- issuetracker link still doesn't work. I manually added the 1.21.21 package with the change @Alan-Liu pointed out, but still hangs way longer than before.
     
  25. SVC-Games

    SVC-Games

    Joined:
    May 21, 2013
    Posts:
    140
    Check that the change actually remains. In my case I change the break position and when going back to unity y reverts to its original version. If you try again it will stay that way (at least until the package is restored)
     
  26. jawnwee

    jawnwee

    Joined:
    Nov 4, 2019
    Posts:
    33
    If anyone is looking for exact instructions here it is (assuming you're on the latest addressables):

    1. Download 1.21.20 from : https://github.com/needle-mirror/com.unity.scriptablebuildpipeline/tags
    2. Unzip it, and open package.json and change
    "version": "1.21.20" to "version": "1.21.21"
    3. zip/compress the entire com.unity.scriptablebuildpipeline-1.21.20 folder
    4. rename the extension to com.unity.scriptablebuildpipeline-1.21.20.tgz
    5. In your unity project manifest.json file, add the following line with where your tgz file is above

    "com.unity.scriptablebuildpipeline": "file:/Users/XXXX/Downloads/com.unity.scriptablebuildpipeline-1.21.20"'
     
    Last edited: Dec 21, 2023
  27. Alan-Liu

    Alan-Liu

    Joined:
    Jan 23, 2014
    Posts:
    397
    Simpler steps to modify the source code of a package:
    1. Move the entire folder of a package in the Library/PackageCache to Packages and remove the version suffix, e.g. Library/PackageCache/com.unity.scriptablebuildpipeline@1.21.21 -> Packages/com.unity.scriptablebuildpipeline
    2. Open Unity and it will update Packages/manifest.json and Packages/packages-lock.json automatically
    3. Now, you can modify the source code of the package freely
     
  28. unity_shane

    unity_shane

    Unity Technologies

    Joined:
    Jun 3, 2021
    Posts:
    106
    Hi there, I'm not sure what's going on about getting the ticket through QA, but I'm looking at this personally now that I've seen this thread. As for the change in CalculateSceneDependencyData, it seems like it was made to fix a different bug with sprite atlases, but a performance regression of this magnitude seems far too big of a side effect for a sprite atlas fix. I'll see about looking into this and see if there's another way to fix the original issue without causing this level of performance hit. In the meantime, especially if the change isn't causing any issues with your sprite atlases, by all means edit your package using the instructions @Alan-Liu gave. Thanks for isolating the regressed area Alan-liu, it's always a really big help in your bug reports.
     
  29. unity_shane

    unity_shane

    Unity Technologies

    Joined:
    Jun 3, 2021
    Posts:
    106
    Just following up here, I've gone ahead and implemented @Alan-Liu's fix and it should be going out in the next version of addressables 2.0 and 1.21. Thanks for posting about it and for finding a fix! It appears that the regression slipped through the cracks as part of another bug fix, so it's prompted us to consider some ways we can prevent things like that happening again in the future in addressables or otherwise. Thanks.
     
    SVC-Games and samanabo like this.
  30. CaioRosiscaKG

    CaioRosiscaKG

    Joined:
    Oct 13, 2020
    Posts:
    4
    Thank you so much for looking into it, would you have any ETA for when a new version of 1.21 would be available?
     
  31. unity_shane

    unity_shane

    Unity Technologies

    Joined:
    Jun 3, 2021
    Posts:
    106
    We're looking at doing the next release some time by the end of this month assuming there are no blockers. In the meantime, if this issue is causing you a lot of trouble, I highly recommend manually adding the fix as described by Alan-Liu earlier in the thread using the instructions described in the following post:

    From there any edits to the version of the package in the Packages folder will not be deleted on domain reload. Then once we do our next release you can just delete the scriptablebuildpipeline folder in your packages folder, and you'll be back up to date on the regular addressables version. Hopefully this helps in the meantime!
     
  32. cihadturhan_unity

    cihadturhan_unity

    Joined:
    Apr 19, 2021
    Posts:
    72
    I do have an issue but I'm on 1.21.19 so I'm trying to figure out why it takes so much time.
    It used to be fast.
    I have around 10 simple unity scenes (it's a mobile game) with a 10 mp3 files, and scriptable objects. All takes around 160MB so it's not that big.

    I've been waiting more than 15 minutes just for updating the previous build.
     
    Whatever560 likes this.
  33. RaventurnPatrick

    RaventurnPatrick

    Joined:
    Aug 9, 2011
    Posts:
    250
    Do you have an ETA when this bugfix will be available?
    Also can you elaborate why the addressable package was not updated in the last few months? I would hope for faster hotfixes in such a core Unity package
     
  34. DanielF-NQX

    DanielF-NQX

    Joined:
    Jun 22, 2023
    Posts:
    28
    Its not really core since it only wraps the functionality of Asset Bundles, which are core. If the issue was in the Asset Bundles you could have expected a fix a few days later (or whenever a repo sync day would be).

    But, either way. I agree that such a minor fix should have been pushed a lot sooner, rather than attempting to bundle it in with whatever new features are in this next version, because we could be waiting months before the package gets approved for commit.
     
    gooby429 likes this.
  35. amynox

    amynox

    Joined:
    Oct 23, 2016
    Posts:
    178
    @unity_shane is this bug was fixed? i see some package in LTS Release2022.3.18f1

    Many thanks
     
  36. DanielF-NQX

    DanielF-NQX

    Joined:
    Jun 22, 2023
    Posts:
    28
    Where are you seeing this?

    Packages mentioned in that version are
     
  37. amynox

    amynox

    Joined:
    Oct 23, 2016
    Posts:
    178
    The suggest fix was related to com.unity.scriptablebuildpipeline. So may be it was fixed there and not in adressable package. maybe not :D
     
  38. chrismarch

    chrismarch

    Joined:
    Jul 24, 2013
    Posts:
    472
    I followed these instructions, to move the scriptablebuildpipeline folder so that I can move the break outside of the if block, but the Packages/manifest.json file did not change upon opening Unity Editor. We did not previously have scriptablebuildpipeline in our manifest, but it was included as a dependency for Addressables, so maybe this is expected? I hit breakpoints in the modified CalculateSceneDependencyData.cs, so I suppose it works, although I never hit the case where the break was moved, so maybe it doesn't help us.

    I did notice, the first time I opened Unity Editor (2022.3.16f1), it presented a dialog asking if I wanted to update the API of code in the new Packages/com.unity.scriptablebuildpipeline folder. If I select Yes, then the custom scriptablebuildpipeline package appears in the Package Manger under Packages - Other. If I select No, then the custom scriptablebuildpipeline package appears under Packages - Unity:
    upload_2024-2-8_11-47-6.png

    This was the dialog:
    upload_2024-2-8_11-56-39.png
     
    Last edited: Feb 8, 2024
  39. JulenScarecrowStudio

    JulenScarecrowStudio

    Joined:
    Jan 17, 2018
    Posts:
    20

    Could you provide us with an updated ETA?
     
  40. davidla_unity

    davidla_unity

    Unity Technologies

    Joined:
    Nov 17, 2016
    Posts:
    763
    Hey y'all, so we have a fix for this that's landed internally. It is going to be released with Scriptable Build Pipeline 1.21.22,which is currently in our package release pipeline. There have been a couple snag, so it's taken a bit longer than we thought, but it shouldn't too long now
     
  41. SVC-Games

    SVC-Games

    Joined:
    May 21, 2013
    Posts:
    140
    Great to hear. Were the snags related to this issue? Just curiosity
     
  42. yorube

    yorube

    Joined:
    Jan 2, 2017
    Posts:
    4
    @
    Is the calculatedependency issue fixed on Scriptable Build Pipeline 1.21.22?
    I using Addressable 1.21.20 and Scriptable Build Pipeline 1.21.22.
    Still stuck on CalculateSceneDependency for an extremely long time.
    I test downgraded SBP to 1.21.20 it's normal building time.
    Did someone test the new version and still have the same problem?
     
  43. yorube

    yorube

    Joined:
    Jan 2, 2017
    Posts:
    4
    I found because my project was upgraded from Unity 2019.4 to Unity 2022.3.14 the addressable setting "Non-Recursive Dependency Calculation" was not enabled (Unity 2021+ is enabled by default) when enabling this option it's not stuck in calculate dependency(I am not sure if is this option solve the issue ).

    "Non-Recursive Dependency Calculation" option will cause other problems as far as I know but I just enabled it I am not encountering it currently.
    Just remind anyone who wants to enable this option to know about the details.
    Addressable Asset Settings reference | Addressables | 1.21.20 (unity3d.com)
    Update
    "Non-Recursive Dependency Calculation" did not solve my problem. Last time I successfully because when I downgraded SBR that time build had a cache.
    I cleared all build cache and then running the default build again failed:(.
     
    Last edited: Feb 15, 2024
  44. RaventurnPatrick

    RaventurnPatrick

    Joined:
    Aug 9, 2011
    Posts:
    250
    You are bringing up a very good point. For Some reason the addressable system has a plethora of configuration options (for the build pipeline + the bundle settings) but non of those get properly migrated when upgrading to new engine versions. So maybe recreating the whole build configuration would help?
     
  45. SVC-Games

    SVC-Games

    Joined:
    May 21, 2013
    Posts:
    140
    Updated and the fix seems to be applied (it was just moving the break out of the if)

    SBP-Fix.png

    Seems to be working
     
  46. yorube

    yorube

    Joined:
    Jan 2, 2017
    Posts:
    4
    After clearing the Library and Cache and then building, it seems to work. I am not stuck in Dependency Calculation.

    But I am stuck in the write serialized file stage now and have not succeeded in full addressable build once.
    I also found during the build process my memory usage is very high.
    My current hardware memory is 32g and I only run unity addressable build sometimes it causes a failure because out of memory.

    When writing serialized file stage memory usage is about 12g to 14g should it be so high?

    My project uses a lot of sprites so it might be the reason for the memory issues but it's not happen in the old version.

    If still can't solve it by myself unfortunately I still need to downgrade SBP:(.
    Sorry, My issue seems off-topic in the original post. If anyone has a solution I would appreciate it.

    davidla_unity do you have any information?
     
    Last edited: Feb 16, 2024
  47. RaventurnPatrick

    RaventurnPatrick

    Joined:
    Aug 9, 2011
    Posts:
    250
    If you are using sprites and are building for a mobile platform you are most likely using atsc or etc compression. The problem is as I understand on Windows/Mac those compressions are not supported and thus get decompressed into memory (can easily result in a few mbs per sprite). You can try to include some unloading logic that unloads the sprites from editor memory during the addressable build. You can also set the max size to a lower value.
     
  48. yorube

    yorube

    Joined:
    Jan 2, 2017
    Posts:
    4
    I build for PC/Console and work on Windows OS. Finally, to build success I changed to another computer(48g ram) when the archive bundle memory usually was about 20g and till archive bundle finished spent about 40 min then I was stuck in the postevent stage(the stage after all archive bundles)for 2 hours.

    Yes, you are right generally modifying texture in the editor will load the raw data to memory.
    I think building addressable does need to keep data in memory for so long also I built the same bundle in the old version it's fine.
    But like my previous post said, I did change some settings after the upgrade so It's not the same environment.

    I already set my sprite in target size but the sprite count is huge so I think it can't avoid huge total usage in my case.
    If not necessary I don't want to spend too much time manually releasing memory in the editor(To be honest I don't think that should happen).

    Anyway, I already tried my best to check most settings as the document recommends and also researched about pack strategy.
    It shouldn't be so much difficult just in the build bundle stage. Hope my information can help the team find any issues.
     
    Last edited: Feb 16, 2024
  49. Whatever560

    Whatever560

    Joined:
    Jan 5, 2016
    Posts:
    538
    As a workaround, I cloned a mirror repo and checked-out the correct tag, it gives more control but anyone could probably also just add this url as a git package, https://github.com/needle-mirror/com.unity.scriptablebuildpipeline/tree/1.21.20

    [Edit] : The git URL to add in the package manager to target the tag is https://github.com/needle-mirror/com.unity.scriptablebuildpipeline#1.21.20

    You don't need to do anything else, Unity will use this version in place of any other. I can confirm that the built process is wayyyyyy faster.
     
    Last edited: Mar 16, 2024
    Jribs likes this.
  50. Developer1999-1

    Developer1999-1

    Joined:
    Oct 27, 2020
    Posts:
    41
    I would like to add that this issue still exist on Addressable 1.21.20, SBP 1.21.22, Unity 2022.3.26f1, at least on my machine (M2 Mac Mini). I have been trying to build for Android for 2 days now -- to no avail. I have attempted:

    1. Deleting Library
    2. Clearing Addressable Build Cache
    3. Even manually downgrading Addressable and SBP's version.

    All to no avail.

    I literally can not build Addressable to one of the biggest target platforms. It's not even a disappointment anymore; the engine literally doesn't work. I don't even know what the package is good for if we can not perform such a basic task on the latest LTS.