Search Unity

Bug (Case 1343820) Addressables doesn't detect corrupted local cached files prior loading

Discussion in 'Addressables' started by Peter77, Jun 17, 2021.

  1. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,609
    EDIT: https://issuetracker.unity3d.com/is...l-cached-files-are-not-detected-prior-loading

    Using Unity 2019.4.20f1 and Addressables 1.18.4.

    Addressables seems to be unable to detect if earlier downloaded files, cached files, are corrupted prior loading those files. I don't know if this functionality is broken or missing in Addressables. Searching the Addressables manual for "integrity" and "corrupt" doesn't yield any results.

    Cached files can get corrupt for a variety of reasons. Therefore it's important that Addressables provides a mechanism to allow us to run an integrity/validation check in the player prior loading possibly corrupted data. This functionality is either not working or doesn't exist in Addressables.

    Please see the attached video where I go over the problem in detail.


    Reproduce
    1. Open project on Windows
    2. Open Build Settings and enable "Development Mode"
    3. Open "Addressables Profiles" window and activate "Editor Hosted"
    4. Open "Addressables Hosting" window and enable the "Local Hosting" service
    5. Select "Assets/AddressableAssetsData/Menu" and set BuildPath=LocalBuildPath and LoadPath=LocalLoadPath in the Inspector
    6. Click in the Addressables Groups window "Build > New Build > Default Build Script"
    7. Press from main menu "File > Build" and build a Win64 Player
    8. Select "Assets/AddressableAssetsData/Menu" and set BuildPath=RemoteBuildPath and LoadPath=RemoteLoadPath in the Inspector
    9. Click in the Addressables Groups window "Build > New Build > Default Build Script"
    10. Run the earlier built Player, press "Check for Catalog Update" and "Check for new Content" and notice it downloaded new content
    11. Close Player, switch back to Unity
    12. Open "Scenes/Menu.unity" and change the Main Camera background color
    13. Click in the Addressables Groups window "Build > New Build > Default Build Script"
    14. Run earlier built player
    15. In the Player press "Check for Catalog Update"
    16. In the Player press "Check for new Content"
    17. In the Player press "Load" and notice the scene has the same color you just edited in Unity.
    18. At this point you figured that content updates work
    19. Close Player
    20. In Windows Explorer delete the cached content in
      C:\Users\%UserName%\AppData\LocalLow\Unity\DefaultCompany_AddressablesCacheIntegrity
    21. Run earlier built player
    22. In the Player press "Check for Catalog Update". No update available, which is expected.
    23. In the Player press "Check for new Content". New data is being downloaded, which is expected.
    24. Close Player
    25. Open the "__data" file located in a directory under
      C:\Users\%UserName%\AppData\LocalLow\Unity\DefaultCompany_AddressablesCacheIntegrity
      in a HexEditor and change it. This is to simulate a corrupted file.
    26. Run earlier built player
    27. In the Player press "Check for Catalog Update". No update available, which is expected.
    28. In the Player press "Check for new Content". No update is available, which is NOT expected, because the local cached content is corrupt.


    Actual
    Addressables doesn't detect the corrupt cached content.

    Expected
    When asking Addressables for new content it detects if cached content is corrupt.
    An API that we can use to run an integrity / validation step over the cached files and retrieving a list of corrupted files that we can redownload.


    Notes

    The catalog.json file is locally cached at:
    C:\Users\%UserName%\AppData\LocalLow\DefaultCompany\AddressablesCacheIntegrity\com.unity.addressables


    The downloaded content is locally cached at:
    C:\Users\%UserName%\AppData\LocalLow\Unity\DefaultCompany_AddressablesCacheIntegrity


    Deleting the cached content files allows the player to download them again.
     

    Attached Files:

    Last edited: Sep 6, 2021
    DavidZobrist, apkdev and adamgolden like this.
  2. TreyK-47

    TreyK-47

    Unity Technologies

    Joined:
    Oct 22, 2019
    Posts:
    1,820
    Thanks again for the bug report, Peter!
     
    Peter77 likes this.
  3. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,609
    This bug-report was closed as "By Design" with the following note:
    It's against some vendor guidelines to just download data without the users consent, so this implementation is not acceptable from a guidelines point of view.

    That's why I pointed out an API to run an integrity / validation step over the cached files is missing.

    Here is an example how I would like to handle a corrupted bundle error:
    1. Game detects corrupted assets during load
    2. Game displays error "The game detected corrupted files that must be updated. We recommend that you're connected to Wi-Fi and have sufficient free space on your device."
    3. Now the only option for the user is to press the "Update" button to continue playing. Without pressing "Update", the game does not download any data and does not continue.
    4. Game runs an integrity / validation step over all cached files to collect a list of bundles that must be downloaded again. This functionality is missing in Addressables.
    5. Game re-downloads corrupted data
    Can you please re-open the bug-report?
     
    Last edited: Sep 6, 2021
    apkdev likes this.
  4. CineTek

    CineTek

    Joined:
    Jul 12, 2013
    Posts:
    98
    We have not submitted to Apple yet but we have had the same concern when looking at the missing integrity/validation APIs. This is a game breaker for the Addressable system.
     
    Peter77 likes this.
  5. TreyK-47

    TreyK-47

    Unity Technologies

    Joined:
    Oct 22, 2019
    Posts:
    1,820
    I'll reach out to our QA team for an update.
     
    Peter77 likes this.
  6. CineTek

    CineTek

    Joined:
    Jul 12, 2013
    Posts:
    98
    Has there been any update on that? A validation/integrity check API we can trigger ourselves is probably the best here...
     
    Peter77 likes this.
  7. unity_shane

    unity_shane

    Unity Technologies

    Joined:
    Jun 3, 2021
    Posts:
    106
    @Peter77 @CineTek, Hi there, sorry for the delayed response on this one. We think this sounds like a good idea, and are adding it to the list of features we're going to explore for future versions of Addressables, particularly Addressables 2.0 and beyond. Thanks!
     
  8. CineTek

    CineTek

    Joined:
    Jul 12, 2013
    Posts:
    98
    Is there any workaround for this right now? @unity_shane
     
  9. unity_shane

    unity_shane

    Unity Technologies

    Joined:
    Jun 3, 2021
    Posts:
    106
    @CineTek @Peter77

    If you want to stop downloads from happening on cached bundles because of a CRC mismatch, then you should probably just disable CRC checking for cached bundles. While bundles getting corrupted during download is definitely a concern, the likelihood of cached bundles getting corrupted (saving for intentional tampering by an end user) is extremely low. Removing this CRC checking also conveys a performance benefit, for what its worth. Additionally, if CRC checking is off and the bundle is corrupted, attempting to load it will throw an error which you can then catch and use to prompt the user to redownload.

    Another possible solution is to navigate to the BundledAssetGroupSchema of all of the groups that you wish to have not automatically redownload corrupted cached bundles and set their Retry Count to -1. Doing this will make it so that when you try and redownload a cached bundle that is corrupted, it will throw a RemoteProviderException. From here, you can catch the exception and implement the sort of prompt you were talking about manually.
     
  10. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,609
    But this will also affect an user initiated download to fail without a retry when the connection gets shortly interrupted I assume?
     
  11. unity_shane

    unity_shane

    Unity Technologies

    Joined:
    Jun 3, 2021
    Posts:
    106
    @Peter77
    Likely yes, although that's not inherently a problem - the real purpose of doing this is to stop the redownload from happening automatically (i.e. without you, the developer, knowing about it in code). You should be able to catch the exception thrown when they try to redownload and then try and manually redownload it after - the key is that before there would be no way for you to know if a bundle was going to be redownloaded, but using this workaround you should be able to know through the exception.
     
  12. optimise

    optimise

    Joined:
    Jan 22, 2014
    Posts:
    2,129
    @unity_shane Just curious. Is that Addressables 2.0 = dots Addressables?
     
  13. unity_shane

    unity_shane

    Unity Technologies

    Joined:
    Jun 3, 2021
    Posts:
    106
    @optimise, no, DOTS addressables and Addressables 2.0 are two separate things. The details of what both of these will entail are still not completely finalized, but in general Addressables 2.0 will be building off of the existing Addressables API and have many of the same bones as the existing Addressables, while DOTS addressables will be a new content management system built from the ground up.