Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. Unite 2023 Registrations are now LIVE! We are thrilled to announce that Unite 2023 Registration is now live and open for all!
    Dismiss Notice
New Forum User Notice Unite 2023 Registrations are now LIVE!

How to list which asset bundles are referenced in a catalog.json?

Discussion in 'Addressables' started by Garth, May 28, 2021.

  1. Garth

    Garth

    Joined:
    Apr 2, 2013
    Posts:
    4
    Hello Unity Community!

    How can I tell the difference between two catalogs?

    Example:
    • We currently build Addressables and deploy new asset bundle content via an S3 bucket and CloudFront.
    • catalog_1.json and catalog_1.hash previously exist on the server.
    • We then upload asset bundles and catalog_2.json and catalog_2.hash.
    • When it's time to turn on catalog_2 for our players, we have our Game Server return "catalog_2" instead of "catalog_1"
    We would like to see what changes we are deploying for our players before we actually flip the switch. I would also like to know what changes are being made if something goes wrong so we can debug.

    Unfortunately, when I do a diff between the two JSON files, it looks like all the asset bundle data has been encoded in some long strings for keys m_InternalIds, m_KeyDataString, m_BucketDataString, m_EntryDataString, and m_ExtraDataString. See the screenshot below.

    How can I decode these? I'm wondering is there any way to see differences between these catalogs?

    Thanks,
    - Garth

    AddressablesCatalogDiff.png
     
    Garth, May 28, 2021
    #1

  2. Garth

    Garth

    Joined:
    Apr 2, 2013
    Posts:
    4
    Another reason I want to be able to parse these fields is so we can know which asset bundles are currently in use, and which asset bundles are old and no longer used. We have never deleted an asset bundle and our S3 usage is measured in terabytes.
     
    Garth, May 28, 2021
    #2

  3. Garth

    Garth

    Joined:
    Apr 2, 2013
    Posts:
    4
    We decided we will generate a manifest ourselves whenever we build asset bundles.
    1. Delete all previous asset bundles. We don't want old asset bundles around so we can tell what the new asset bundles will be.
    2. Build asset bundles.
    3. Create a file list of the generated asset bundles were created. Save this next to the catalog json.
    4. Compare this generated file list with the previous file list to see what asset bundles have their hashes changed and which bundles are unchanged.
     
    Garth, Jun 1, 2021
    #3

  4. toto007

    toto007

    Joined:
    Jul 18, 2014
    Posts:
    33
    Hi,
    How do you have generate your "catalog" (or manifest) custom?

    Do you have find a method to decode field m_KeyDataString, etc ?
     
    toto007, Jun 18, 2021
    #4

  5. Kamand0l

    Kamand0l

    Joined:
    May 19, 2014
    Posts:
    24
    That would be very useful indeed
     
    Kamand0l, Jun 16, 2022
    #5

  6. andymilsom

    andymilsom

    Unity Technologies

    Joined:
    Mar 2, 2016
    Posts:
    294
    You will be better off doing what you are for your purpose with generating extra diagnostic data. The catalog has the data it needs for loading content, and removes anything it doesn't need. The results in its use for what you want to be limited.
    You can get access to the Catalog data through code easy enough, here is an example of how you may do so:
    Code (CSharp):
    1. public static void GetChanges()
    2. {
    3.     string assetPath1 = EditorUtility.OpenFilePanel("Addressables Catalog 1", Path.GetDirectoryName(Application.dataPath), "json");
    4.     if (File.Exists(assetPath1) == false)
    5.         return;
    6.     string assetPath2 = EditorUtility.OpenFilePanel("Addressables Catalog 2", Path.GetDirectoryName(Application.dataPath), "json");
    7.     if (File.Exists(assetPath2) == false)
    8.         return;
    9.  
    10.     var json = File.ReadAllText(assetPath1);
    11.     ContentCatalogData d = JsonUtility.FromJson<ContentCatalogData>(json);
    12.     var resourceMap1 = d.CreateLocator();
    13.    
    14.     json = File.ReadAllText(assetPath2);
    15.     d = JsonUtility.FromJson<ContentCatalogData>(json);
    16.     var resourceMap2 = d.CreateLocator();
    17.  
    18.     Dictionary<string, IResourceLocation> map1BundleNameToLocation = new Dictionary<string, IResourceLocation>();
    19.     foreach (KeyValuePair<object, IList<IResourceLocation>> pair in resourceMap1.Locations)
    20.     {
    21.         foreach (IResourceLocation loc in pair.Value)
    22.         {
    23.             if (loc.ProviderId.Contains("AssetBundle"))
    24.             {
    25.                 var options = loc.Data as AssetBundleRequestOptions;
    26.                 map1BundleNameToLocation.Add(options.BundleName, loc);
    27.             }
    28.         }
    29.     }
    30.    
    31.     foreach (KeyValuePair<object, IList<IResourceLocation>> pair in resourceMap2.Locations)
    32.     {
    33.         foreach (IResourceLocation loc in pair.Value)
    34.         {
    35.             if (loc.ProviderId.Contains("AssetBundle"))
    36.             {
    37.                 var options = loc.Data as AssetBundleRequestOptions;
    38.                 if (map1BundleNameToLocation.TryGetValue(options.BundleName, out IResourceLocation otherLoc))
    39.                 {
    40.                     var otherOptions = otherLoc.Data as AssetBundleRequestOptions;
    41.                     // only works if using caching as the hash will be emitted otherwise
    42.                     if (options.Hash.Equals(otherOptions.Hash, StringComparison.Ordinal) == false)
    43.                         Debug.Log($"Content Hashes are different for {loc.InternalId} and {otherLoc.InternalId}");
    44.                 }
    45.             }
    46.         }
    47.     }
    48. }
     
    andymilsom, Jun 23, 2022
    #6