Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. We have updated the language to the Editor Terms based on feedback from our employees and community. Learn more.
    Dismiss Notice
  3. Join us on November 16th, 2023, between 1 pm and 9 pm CET for Ask the Experts Online on Discord and on Unity Discussions.
    Dismiss Notice
  4. Dismiss Notice

Make stored persistent data location configurable

Discussion in 'WebGL' started by CrazyGamesCom, Feb 17, 2021.

  1. CrazyGamesCom

    CrazyGamesCom

    Joined:
    Nov 11, 2016
    Posts:
    16
    At CrazyGames.com we're big users of the Unity WebGL build.

    One issue that we've found to be a problem for our users, is that when a game is updated, all stored persistent data is lost. This often results in users losing all of their game progress.

    The reason is that the location of the IndexedDB is dependent on the path. But for caching reasons, every game version has a new path.

    Would it be an option to make the location of IndexedDB configurable?

    There are some other reports about this issue, so we're definitely not the only one with this problem:
    https://forum.unity.com/threads/per...s-different-paths-for-differet-builds.526776/
    https://itch.io/t/140214/persistent-data-in-updatable-webgl-games

    Hopefully this can be added to the roadmap! Let us know if you have any further questions on this problem or our use case.

    Thank you in advance.
     
  2. jukka_j

    jukka_j

    Unity Technologies

    Joined:
    May 4, 2018
    Posts:
    944
    Looking through the code implementation for the cached path, the persistent data path is initialized using a MD5 hash of the url of the page, where the search query part (?foo) of the URL is removed.
    The url field comes from initialization code at
    Code (CSharp):
    1.     core::string url = GetStringFromJS(JS_SystemInfo_GetDocumentURL);
    2.     InitializeWebGLPersistentDataPath(url);
    3.  
    which is from JavaScript side in SystemInfo.js:
    Code (JavaScript):
    1.     JS_SystemInfo_GetDocumentURL: function(buffer, bufferSize)
    2.     {
    3.         if (buffer)
    4.             stringToUTF8(document.URL, buffer, bufferSize);
    5.         return lengthBytesUTF8(document.URL);
    6.     },
    So according to this, the IDBFS persistent data path is "/idbfs/<md5hash(document.URL)>", and hence should stay the same on each rebuild. (looking at Unity 2020.2 code)
    I wonder if I missed something here. Do you use the same online URL for the game when you upload a new version, or does that change?
    If you are changing the location, then what you can try to locally work around is to create a file named "DocumentUrl.jslib" in the Assets/ directory of the project, with the contents
    Code (JavaScript):
    1. mergeInto(LibraryManager.library, {
    2.     JS_SystemInfo_GetDocumentURL: function(buffer, bufferSize) {
    3.         var myDocumentUrl = 'https://crazygames.com/mygame';
    4.         if (buffer)
    5.             stringToUTF8(myDocumentUrl, buffer, bufferSize);
    6.         return lengthBytesUTF8(myDocumentUrl);
    7.     }
    8. });
    9.  
    and do a rebuild of the project. Then verify in a Development build that this function can be found in the generated .framework.js file. After that the hash name of the IndexedDB repository should stay the same.
     
  3. CrazyGamesCom

    CrazyGamesCom

    Joined:
    Nov 11, 2016
    Posts:
    16
    Thanks for your quick and in-depth response, Jukka.
    • We always host a new version of a game in a new directory, since both browser caches and CDN caches tend to cause issues. I know this is the case for almost all web gaming platforms (I think for the same reason as us).
    • As for the workaround, unfortunately it's not feasible since we accept the compiled versions from many hundreds of game developers (and asking them individually is not really feasible since their level of expertise varies).
    • Is there a particular reason that an optional config option is not a good idea?
     
  4. jukka_j

    jukka_j

    Unity Technologies

    Joined:
    May 4, 2018
    Posts:
    944
    Oh, this is a gaming portal type of scenario. Gotcha.

    Does every game reside on their own (sub)domain so that they don't stomp on each other's data? If rebuilding in the game is not feasible, how do would such a configuration then be deployed? We could entertain the option of adding a build setting or a field in the .html template, but in any case, the project will need to be modified somehow?
     
  5. sumpfkraut

    sumpfkraut

    Joined:
    Jan 18, 2013
    Posts:
    241
    Wouldn't it be possible to force Unity NOT to use the files from the cache with a new version?
    I also had this problem. Relaod with ctrl + F5 did not help. i had to delete them from the browser settings.
     
  6. jukka_j

    jukka_j

    Unity Technologies

    Joined:
    May 4, 2018
    Posts:
    944
    That sounds like a separate problem from @CrazyGamesCom above. He is asking to be able to share the persistent data when launching from different URLs, you are asking to not share the persistent data when launching from the same URL?
     
  7. sumpfkraut

    sumpfkraut

    Joined:
    Jan 18, 2013
    Posts:
    241
    I may have misunderstood it, I thought he was doing that with the folder because there was no other way. (because the cached files)
     
  8. jukka_j

    jukka_j

    Unity Technologies

    Joined:
    May 4, 2018
    Posts:
    944
    My understanding is that he is using different folders for each game upload to avoid having to configure man-in-the-middle CDNs to flush their HTTP caches (and other HTTP caching problems)?
     
  9. CrazyGamesCom

    CrazyGamesCom

    Joined:
    Nov 11, 2016
    Posts:
    16
    Yes that's correct (sorry for the delay).