Search Unity

WebGL Settings Data caching?

Discussion in 'Web' started by Apollo-Meijer, Apr 30, 2015.

  1. Apollo-Meijer

    Apollo-Meijer

    Joined:
    Mar 30, 2015
    Posts:
    31
    Hey,

    In the WebGL Publishing Settings there is a check for Data caching.
    I couldn't find any documentation on this.
    What does it do exactly?
     
    Apollo-Meijer, Apr 30, 2015
    #1
    liortal likes this.

  2. Schubkraft

    Schubkraft

    Unity Technologies

    Joined:
    Dec 3, 2012
    Posts:
    1,071
    This enables the build to use the browser cache to store its files. So returning users won't have to download your files again.
     
    Schubkraft, Apr 30, 2015
    #2
    liortal likes this.

  3. Apollo-Meijer

    Apollo-Meijer

    Joined:
    Mar 30, 2015
    Posts:
    31
    Does this also effect PlayerPrefs?
     
    Apollo-Meijer, Apr 30, 2015
    #3

  4. jonas-echterhoff

    jonas-echterhoff

    Unity Technologies

    Joined:
    Aug 18, 2005
    Posts:
    1,666
    No. How/why would it?
     
    jonas-echterhoff, Apr 30, 2015
    #4

  5. Mandoz

    Mandoz

    Joined:
    Aug 30, 2013
    Posts:
    9
    If I update an asset file that is already cached by browser, does the browser try to download a new asset file?
     
    Mandoz, Apr 30, 2015
    #5

  6. Apollo-Meijer

    Apollo-Meijer

    Joined:
    Mar 30, 2015
    Posts:
    31
    Was just trying get this confirmed, turns out the testers cleaned there browser data on reload. thanks for the response
     
    Apollo-Meijer, Apr 30, 2015
    #6

  7. fafase

    fafase

    Joined:
    Jul 3, 2012
    Posts:
    163
    fafase, Jun 25, 2015
    #7

  8. Claytonious

    Claytonious

    Joined:
    Feb 16, 2009
    Posts:
    903
    What exactly is cached and when/how does it expire and how do we make sure new builds correctly bust the cache?
     
    Claytonious, Aug 7, 2015
    #8
    Foreman_Dev and mowax74 like this.

  9. fingersbleeding

    fingersbleeding

    Joined:
    Jun 10, 2013
    Posts:
    24
    What Claytonious said, plus can we confirm Data Caching works? Delivering the compressed version of our webgl build (within Facebook Canvas) to Safari and Chrome with Data Caching checked, both browsers crash and fail to reach the default Unity preloader. Uncheck the option and our content loads.
     
    fingersbleeding, Aug 10, 2015
    #9

  10. zzajac

    zzajac

    Joined:
    Feb 20, 2015
    Posts:
    22
    Hi, I have problem with caching of WebGL,
    I do not use Data Caching in PlayerSetting.
    I set Cache Control: public, max-age=3600 for all files.
    When I upload new build many users have problem to load it.
    Browser gets new *.js files but it has problem with loading WebGL.data file(maybe because of diffrent content-type)
    Even after 1 hour still there is error: Content-Length mismatch with file WebGL.data.
    Hard refresh (ctrl+F5) fixes it, but users do not know about it.
     
    zzajac, Oct 2, 2015
    #10

  11. zzajac

    zzajac

    Joined:
    Feb 20, 2015
    Posts:
    22
    Still UnityLoader.js is not able to load file WebGL.data when browser will return it with http status 206 partial ok
     
    zzajac, Feb 1, 2016
    #11

  12. Marco-Trivellato

    Marco-Trivellato

    Unity Technologies

    Joined:
    Jul 9, 2013
    Posts:
    1,654
    is there a url that we can check ?
     
    Marco-Trivellato, Feb 1, 2016
    #12

  13. AIVIK

    AIVIK

    Joined:
    Feb 12, 2015
    Posts:
    14
    Got the same problem, data caching disabled, but need ctrl+f5 (only in chrome)
     
    AIVIK, Feb 2, 2016
    #13

  14. Claytonious

    Claytonious

    Joined:
    Feb 16, 2009
    Posts:
    903
    Whatever issue may or may not be present in individual builds, can we address the broader question? "What exactly is cached and when/how does it expire and how do we make sure new builds correctly bust the cache?"

    Thanks!
     
    Claytonious, Feb 12, 2016
    #14
    mowax74 likes this.

  15. alexsuvorov

    alexsuvorov

    Unity Technologies

    Joined:
    Nov 15, 2015
    Posts:
    327
    Hello Claytonious.

    I believe Data caching build option requires some additional explanation.

    Basically, there are two independent layers of .data caching:

    1) Browser cache.
    This cache layer is fully controlled by the browser. The behavior of the browser cache depends on the browser configuration and the headers provided by the server. Depending on the Last-Modified, ETag, Expires, Cache-Control etc. headers, the browser can either serve the previously cached response without validation, or efficiently reload it using If-Modified-Since or If-None-Match headers. The WebGL loader currently does not make use of the server provided headers, so the server response headers only affect the browser caching layer.

    2) Optional IndexedDB cache (Data caching build option).
    This is an additional cache layer managed by the WebGL loader. Each time you build your project, the .data package is assigned a unique id. This package uuid is hard-coded in the UnityLoader.js in a form of
    var PACKAGE_UUID = '71dc6a9c-a252-4eb9-986a-4468b011729a';
    When Data caching option is enabled, the downloaded .data package with it's associated uuid are stored in the browser IndexedDB/your-origin/EM_PRELOAD_CACHE/ location.
    Now if you reload the build in the browser, UnityLoader.js will first check if the .data package with the specified PACKAGE_UUID is already present in the browser IndexedDB, in which case the .data package will be taken from the database.

    Normally you can rely on the browser cache layer. But in some special cases (like if your server does not handle If-Modified-Since or If-None-Match properly, or in some other way makes browser caching mechanism unusable) you may consider enabling Data caching build option. Note that some browsers may ask the user for permission to store additional data in the IndexedDB if you exceed the IndexedDB quota.
     
    alexsuvorov, Feb 16, 2016
    #15
    bbridgesvb, seanalegra, sdalex and 15 others like this.

  16. Claytonious

    Claytonious

    Joined:
    Feb 16, 2009
    Posts:
    903
    Thanks for that detailed response. It's very helpful. Have a great day!
     
    Claytonious, Feb 16, 2016
    #16
    mowax74 likes this.

  17. JanusAnderson

    JanusAnderson

    Joined:
    Apr 30, 2013
    Posts:
    27
    I am also seeing this issue - if I update the webGL build, I will get out of memory errors, uncaught aborts, and whatnot unless the player manually clears the browser cache. I've tried setting cache-control to "no-cache", expires to "0", and pragma = "no-cache" in the index.html header, but it doesn't seem to have any effect. Caching has been turned off in the player settings.

    Did anyone on this thread who was having these issues find a solution? Thanks in advance.
     
    JanusAnderson, Mar 31, 2016
    #17
    HamidMac and seanalegra like this.

  18. alexsuvorov

    alexsuvorov

    Unity Technologies

    Joined:
    Nov 15, 2015
    Posts:
    327
    Hello JanusAnderson.

    Note that caching option in the player settings controls caching in the IndexedDB, which has very little to do with the browser's response content caching (although IndexedDB can be also optionally cleaned up when user clears the cache).

    If you get multiple weird errors like abort or out of memory etc. after the game update, then it is most probably the issue with your server caching configuration (which means that the client receives old versions of some content files and not the updated ones). Not only you have to ensure that the index.html is being revalidated, but also all the content files from your Release folder (i.e. .data, .js and .mem) and streaming assets (if present) should be revalidated. Note that changing the headers on the server does not necessarily imply immediate changes in the server response, as the content might be already cached by the intermediate proxy software. You can easily check if the server caching configuration is causing the issue if you append a random query to all the requested urls, which will enforce complete revalidation irrespective of the server setup. Check out the following answer for the exact procedure http://forum.unity3d.com/threads/host-update-time-after-upload.377601/#post-2451672
     
    alexsuvorov, Mar 31, 2016
    #18
    seanalegra, Novack, dforstmaier and 1 other person like this.

  19. Foriero

    Foriero

    Joined:
    Jan 24, 2012
    Posts:
    584
    Hi, would it be possible to have JS code that would invalidate unity data for next loading? We think to put it next to "fullscreen" button for WEBGL where IF user encounter some weird issues caused by caching or that we updated new version on server then he/she simply click something like "Force Reload" button. Is there a JS code for that?
     
    Foriero, Oct 31, 2016
    #19

  20. Foriero

    Foriero

    Joined:
    Jan 24, 2012
    Posts:
    584
    Can this be the solution?

    self.clear =function(){
    //location.reload(true); Doesn't work to IE neither Firefox;//also, hash tags must be removed or no postback will occur.
    window.location.href = window.location.href.replace(/#.*$/,'');
    };
     
    Foriero, Oct 31, 2016
    #20

  21. alexsuvorov

    alexsuvorov

    Unity Technologies

    Joined:
    Nov 15, 2015
    Posts:
    327
    Hello Foriero.

    No, this will have no effect, as it seems to be equivalent to simply reloading the page. The discussion above is focused on the following issues:

    a) The problem originates on the server side, where .js and .data files might have different lifetime in the intermediate cache (not in the browser cache). So, when you update the build, you might end up with the old .js file and the new .data file returned from the server. Currently the .data file itself does not contain a version id, so the loader always assumes that the version is correct. If the file has been cached in the indexedDB incorrectly due to improper server setup, the only way to resolve this is to delete the record in the EM_PRELOAD_CACHE database. Reloading the page does not update the indexedDB cache.

    b) Some files have been served with improper Expires/Cache-Control headers, which prevents its revalidation by the browser on subsequent requests. Clearing the browser cache seems to be the only universal solution to this problem, however, there is no explicit way to perform browser cache cleanup directly from JavaScript.


    How to avoid cache-related issues:

    1) If you have full control over your server, make sure to set the Expires/Cache-Control headers appropriately.

    2) If your access to the server configuration is restricted, you can just upload each new build to a new folder on the server (for example, the name of the folder may include a version number or a date, to make it unique). Unique url guarantees that there will be no collisions with previously cached files. The problem here is that your starting .html page address also changes. You can either load the updated build in an iframe from some main html, or adjust the dataUrl, codeUrl and memUrl in the old html appropriately.

    3) The future embedding scheme for WebGL content should resolve most of the caching issues. For example, you will be able to easily change the embedded url, or generate a build with hashified names, or even provide the actual hash of the content so that the response can be explicitly validated. You can check on the details here https://forum.unity3d.com/threads/new-unity-webgl-embedding-api-white-paper.430909/
     
    alexsuvorov, Oct 31, 2016
    #21
    dforstmaier likes this.

  22. dansav

    dansav

    Joined:
    Sep 22, 2005
    Posts:
    510
    I have a project that is about compressed about 8mb. The second load of webgl on safari mobile the same speed as the first load. On desktop the second loading is about 3 seconds because it's cached. I have set data caching in player preferences. This is a webassembly build with gzip. Why so slow. My network speeds are the same because I'm on the same wifi.

    Edit: On further investigation I have this error
    [UnityCache] 'https://whimsica.com/csbwebglmono2017build/Build/csbwebglmono2017build.data.unityweb' successfully downloaded but not stored in the indexedDB cache due to the error: [object Event]

    What does this mean?
    Edit: On futher investigation it's the data caching setting.
    I made a custom loader to get rid of the unitywebgl warning and according to previous posts the indexedDB cache is hardcoded into the UnityLoader file so it can't find it.

    Edit: Okay letting the browser handle it is no better still looks like a full reload each time. Also using the original unity loader.
     
    Last edited: Nov 7, 2019
    dansav, Nov 7, 2019
    #22

  23. IOU_RAY

    IOU_RAY

    Joined:
    Jul 25, 2017
    Posts:
    127
    Anyone have luck on this? We've tried all scenarios given and none work; mostly issues for Firefox users...

    Or do we just sit as Unity comes up with constant excuses that it's on our end, not theirs?
     
    IOU_RAY, Jan 9, 2020
    #23

  24. MFKJ

    MFKJ

    Joined:
    May 13, 2015
    Posts:
    264
    Its 4 years old post, still valid for Unity 2020 or Unity 2019??
     
    MFKJ, Aug 10, 2020
    #24

  25. claytoncurmi

    claytoncurmi

    Joined:
    Jul 6, 2010
    Posts:
    168
    I am also getting these warnings about being unable to load the indexedDB.

    [UnityCache] indexedDB database could not be opened.


    In previous Unity loaders there was a 
    settimeout
    function however this is not the case anymore (we are currently on 2020.1.4f1). We are experiencing a lot of issues with this particular version for WebGL, mostly related to loading and memory exceptions (there were minimal changes when we upgraded from 2018.2.20 to 2020.1.4). Will try to disable caching and rely on the default browser caching as we have hashed filenames.
     
    claytoncurmi, Oct 12, 2020
    #25

  26. AGameByBrad

    AGameByBrad

    Joined:
    Jun 5, 2020
    Posts:
    2
    I was having problems with games crashing or not updating with the changes I made due to cache issues. I can not ask my players to clear their caches so the solution was this:

    Put the build folder into a subfolder on the server which is named with a version number that changes with every build.
    In the subfolder I put a .htaccess with the following
    RewriteEngine off
    ​
    Then in the game root folder I put a .htaccess with the following
    RewriteEngine on
    RewriteRule ^(.*)$ your-website/game-root-folder/sub-folder-version-number [R=301]​

    This way they are redirected to the new folder no matter what version they try and go to.
     
    AGameByBrad, Mar 30, 2021
    #26