Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

UnityWebRequest makes redundant cache-validation requests for cached files despite max-age directive

Discussion in 'Web' started by ThisIsDangerous, Jan 20, 2020.

  1. ThisIsDangerous

    ThisIsDangerous

    Joined:
    Dec 3, 2018
    Posts:
    39
    TLDR: UnityWebRequest makes cache-validation requests (those that yield 304 response code) for already cached files every time. That's probably because of also storing downloaded content in indexedDB. Valid and correct Cache-Control: max-age directive is present and doesn't help while indexedDB is enabled.

    First, sorry if this question is already answered here on the forums, I couldn't check because the search isn't working for some reason. And posts I found manually do not cover my issue.

    Now more in detail.

    We are facing a problem here regarding redundant cache validation requests of static files. But let me first explain the full situation to eliminate resemblance to look-alike issues.

    Say we have a .png file that has to be downloaded at run time by means of UnityWebRequest (or the obsolete WWW, doesn't matter).

    The Cache-Control: max-age=n response header is present.

    For subsequent attempts to get this same file the expected behavior is to grab it directly from the browser cache and not make ANY more requests to the server until n seconds have passed or the file is evicted from the browser cache earlier for some reason.

    But in fact validation request to the server is made every time we try to get the file. Okay, the server returns 304 with zero-length content, but the request is still a request which takes time and in this case is completely redundant.

    Actually we have plenty of such static files thus generating tremendous amounts of redundant latency both serverside and playerside. Our game client load-up times are in fact multiple times longer due to many such requests-to-see-if-the-file-is-modified.

    After some research I came to an understanding that this is all due to the second caching layer, the indexedDB, which is used by Unity to store downloaded content in. Actually, by disabling indexedDB in the browser settings I have reached the desired behavior: the content is requested, downloaded and cached by the browser, obeying Cache-Control headers, subsequent requests are not made.

    But since we cannot tell our playerbase to tinker with their browsers' setting (let alone mention that this setting cannot be managed per site) we still have the problem.

    So, how do we overcome this? It seems to me that the easiest solution would be to have some checkbox in the build settings that will force UnityWebRequest to bypass indexedDB completely (Data Caching checkbox does not do this, it is for other things). The ideal solution, of course, would be to have UnityWebRequest obey caching rules.

    Maybe there are some hacky tricks or a checkbox I didn't happen to notice?