Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. Unite 2023 Registrations are now LIVE! We are thrilled to announce that Unite 2023 Registration is now live and open for all!
    Dismiss Notice
New Forum User Notice Unite 2023 Registrations are now LIVE!

Bug AssetBundle generation from SpriteAtlas is not deterministic.

Discussion in 'Asset Bundles' started by vectorized-runner, Sep 20, 2023.

  1. vectorized-runner

    vectorized-runner

    Joined:
    Jan 22, 2018
    Posts:
    383
    In our project we're using AssetBundle hashes to determine if we should download the AssetBundles or not.
    Also we create Sprite Atlases from code, so we're not caching SpriteAtlas assets in our project.

    The problem: Every time we create a Sprite Atlas from a list of sprites, even though the SpriteAtlas content is same (in file bytes), the generated asset bundles aren't.

    Here's an example code:
    Code (CSharp):
    1.         private static void BuildAssetBundlesAndCompare()
    2.         {
    3.             // Duplicate a SpriteAtlasAsset in your project, they have the same content in bytes
    4.             var path1 = "Assets/test1/a.spriteatlas";
    5.             var path2 = "Assets/test2/a.spriteatlas";
    6.             var fullPath1 = Path.Combine(Application.dataPath, "test1", "a.spriteatlas");
    7.             var fullPath2 = Path.Combine(Application.dataPath, "test2", "a.spriteatlas");
    8.             var bytes1 = File.ReadAllBytes(fullPath1);
    9.             var bytes2 = File.ReadAllBytes(fullPath2);
    10.  
    11.             if (AreEqual(bytes1, bytes2))
    12.             {
    13.                 Debug.Log("AreEqual");
    14.             }
    15.             else
    16.             {
    17.                 Debug.Log("NotEqual");
    18.             }
    19.          
    20.             var bbs = new AssetBundleBuild[2];
    21.             bbs[0] = new AssetBundleBuild
    22.                      {
    23.                          addressableNames = new[] { "1" },
    24.                          assetBundleName = "bundle1",
    25.                          assetBundleVariant = null,
    26.                          assetNames = new[] { path1 },
    27.                      };
    28.             bbs[1] = new AssetBundleBuild
    29.                      {
    30.                          addressableNames = new[] { "2" },
    31.                          assetBundleName = "bundle2",
    32.                          assetBundleVariant = null,
    33.                          assetNames = new[] { path2 },
    34.                      };
    35.          
    36.             const BuildAssetBundleOptions BuildOptions =
    37.             BuildAssetBundleOptions.UncompressedAssetBundle |
    38.             BuildAssetBundleOptions.StrictMode |
    39.             BuildAssetBundleOptions.AssetBundleStripUnityVersion;
    40.  
    41.             BuildPipeline.BuildAssetBundles(Application.dataPath, bbs, BuildOptions, BuildTarget.StandaloneOSX);
    42.  
    43.             fullPath1 = Path.Combine(Application.dataPath, "bundle1");
    44.             fullPath2 = Path.Combine(Application.dataPath, "bundle2");
    45.  
    46.             bytes1 = File.ReadAllBytes(fullPath1);
    47.             bytes2 = File.ReadAllBytes(fullPath2);
    48.          
    49.             if (AreEqual(bytes1, bytes2))
    50.             {
    51.                 Debug.Log("AreEqual");
    52.             }
    53.             else
    54.             {
    55.                 Debug.Log("NotEqual");
    56.             }
    57.         }
    First "AreEqual", then "NotEqual" is printed on the console.

    BuildAssetBundleOptions.DeterministicAssetBundle doesn't change the output.
    Using Unity 2022.3.2f1

    Is this a known issue? Any plans to fix this?
     
    Last edited: Sep 20, 2023
    vectorized-runner, Sep 20, 2023
    #1

  2. George-Ing

    George-Ing

    Unity Technologies

    Joined:
    Jan 14, 2020
    Posts:
    68
    Hey!

    This certainly looks like a bug - it may be tangential to something we discussed this morning incidentally.

    Would you be able to file a bug report per-chance, and we'll make sure it's prioritised alongside the rest of our work.
     
    George-Ing, Sep 20, 2023
    #2

  3. vectorized-runner

    vectorized-runner

    Joined:
    Jan 22, 2018
    Posts:
    383
    I've submitted the bug, IN-55415
     
    vectorized-runner, Sep 21, 2023
    #3

  4. George-Ing

    George-Ing

    Unity Technologies

    Joined:
    Jan 14, 2020
    Posts:
    68
    Legend! Thanks - much appreciated! <3
     
    George-Ing, Sep 21, 2023
    #4

  5. vectorized-runner

    vectorized-runner

    Joined:
    Jan 22, 2018
    Posts:
    383
    Hi, can I get an update on this bug? I'm getting emails about some updates but I can't log into your Jira bug tracking system for some reason.
     
    vectorized-runner, Oct 11, 2023
    #5

  6. George-Ing

    George-Ing

    Unity Technologies

    Joined:
    Jan 14, 2020
    Posts:
    68
    Hey,

    We're just about to have an engineering discussion with the Unity build team this afternoon - this bug is on our agenda to discuss + prioritize.

    To help with prioritization - is this blocking your project?
     
    George-Ing, Oct 18, 2023
    #6

  7. vectorized-runner

    vectorized-runner

    Joined:
    Jan 22, 2018
    Posts:
    383
    Yes its actually blocking multiple of our projects as we're using a shared asset solution for all of them. Currently we don't have a nice workaround as we're generating the sprite atlas on the fly (when building asset bundles) so different bytes produces a different hash every time, this causes users to download the sprite atlas every time when we make an asset update while checking missing assets on the device at startup
    .
     
    vectorized-runner, Oct 18, 2023
    #7

  8. AndrewSkow

    AndrewSkow

    Unity Technologies

    Joined:
    Nov 17, 2020
    Posts:
    78
    Hi vectorized-runner, thanks for submitting your project.

    I notice from your script that you are building two different bundles that contain two different sprite atlases. Even if the .spritealtas files have identical content they are different Assets from the perspective of Unity and assigned different Asset guids.

    I also cracked open the bundles in your project with WebExtract and Binary2Text and confirmed that the differences are small and just associated with the different asset that is provided and different bundle name.

    So this behaviour is expected. If you did two builds in a row, both building the same spriteatlas file and both outputing to the same bundle name then i would expect those two builds to be identical.

    I hope this helps clear it up, we will resolve the ticket but i wanted to explain here.
     
    AndrewSkow, Oct 18, 2023
    #8

  9. AndrewSkow

    AndrewSkow

    Unity Technologies

    Joined:
    Nov 17, 2020
    Posts:
    78
    Here is an example of the diff between the contents of the two bundles.
    upload_2023-10-18_15-32-19.png
     
    AndrewSkow, Oct 18, 2023
    #9

  10. vectorized-runner

    vectorized-runner

    Joined:
    Jan 22, 2018
    Posts:
    383
    Bundle name doesn't matter, I didn't know it affected the resulting bytes but we can use the same bundle names.

    I don't understand why the engine cares about including asset guids or paths when building the asset bundle. It's obviously a bad design as all of the content is already in the asset bundle, the guid and path is irrelevant at that point, I can delete the assets from the project and the asset bundle works.
     
    vectorized-runner, Oct 19, 2023
    #10

  11. Astro75

    Astro75

    Joined:
    Dec 18, 2014
    Posts:
    47
    We also have bundle determinism issues related to atlases. We use Unity 2022.3.8.
    We use Scriptable Build Pipeline to build the bundles.

    Issue #1
    2 identical bundle packages built from same commit on our CI. Different PCs used for these builds. m_DownscaleFallback values for generated atlas textures are different.
    upload_2023-10-24_18-1-38.png

    Issue #2
    Extracted textual data is identical. Except a few .resS files which I assume contain atlas textures. Small part of the binary diff displayed in the screenshot. upload_2023-10-24_18-8-50.png
     
    Astro75, Oct 24, 2023
    #11

  12. AndrewSkow

    AndrewSkow

    Unity Technologies

    Joined:
    Nov 17, 2020
    Posts:
    78
    @vectorized-runner - You are correct that after an AssetBundle is built the original Assets do not need to continue to exist. However that original identity (e.g. path / guid) is used so that the built content can be loaded e.g. AssetBundle.LoadAsset(string).

    And internally we require a unique identification for objects to that references between objects works (including references that go between AssetBundles).

    @Astro75 - thanks for describing this specific case you've hit, including the specifics of the diffs that you see. I've mentioned this post to the Addressables team who take care of the Scriptable Build Pipeline.
     
    AndrewSkow, Oct 26, 2023
    #12

  13. Astro75

    Astro75

    Joined:
    Dec 18, 2014
    Posts:
    47
    Maybe I should create a bug report? Can't produce a repro project, but would be able to give built bundle samples for my issue #2 if that would help.
     
    Astro75, Oct 30, 2023 at 12:46 PM
    #13