Search Unity

Question Firefox vs Chrome Memory Crashes

Discussion in 'Web' started by robrab2000-aa, Nov 15, 2022.

  1. robrab2000-aa

    robrab2000-aa

    Joined:
    Feb 8, 2019
    Posts:
    117
    We have some assets we're loading into our WebGL build (2019.4) using the addressable system. They seem to load in Firefox but when we load them through Chrome (including Edge, Opera and Brave) they run out of memory and crash.

    The files we're loading are not small, but should be loadable in the browser. The loading seems to be inconsistent across content builds so this leads me to question whether the issue lies with the addressable system perhaps. We're investigating whether there is an issue with how we're creating the source assets too.

    The fact that they consistently load in Firefox but not in Chrome means that its worth finding out whether there are known differences in how they handle memory.
     
  2. robrab2000-aa

    robrab2000-aa

    Joined:
    Feb 8, 2019
    Posts:
    117
    Our build is suspiciously large and inconsistent in memory too. by this I mean that depending on where we look, we see different numbers:

    Here are four different readouts of memory usage taken at the same time:
    (I've used Opera here because it gives me an extra readout, but the results are the same in chrome, brave, and Firefox)

    Graphy (https://github.com/Tayx94/graphy):
    Screenshot 2022-11-15 at 11.14.41.png

    WebGl Stats (https://github.com/kongregate/Unity-WebGL-Utilities):
    Screenshot 2022-11-15 at 11.14.46.png

    Opera Memory Usage:
    Screenshot 2022-11-15 at 11.14.52.png

    Opera Memory Snapshot:
    Screenshot 2022-11-15 at 11.15.27.png
     
  3. CodeSmile

    CodeSmile

    Joined:
    Apr 10, 2014
    Posts:
    5,808
    It's not unusual for WebGL to show different behaviour across browsers, unfortunately.
    Just recently I did memory snippets of Unity's Forma demo in Firefox, Edge, Opera GX and Chrome and the memory usage was all over the place with just this one tab open (except Chrome): https://forum.unity.com/threads/wha...untime-in-a-webgl-build.1359829/#post-8580919
    There's some extra info in there but in general what memory you record inside the Unity app is just what Unity is aware of, of course the browser needs to allocate more memory beyond that for javascript, content cache and so on.

    How "not small" are we talking about? How much memory did you set as the baseline for the app?

    One issue with loading assets at runtime is that it can temporarily occupy a lot more memory even just for a single frame due to decompressing, converting, creating an in-memory copy and what not.

    Another issue with increasing memory beyond the initial capacity is the requirement that only contiguous memory can be allocated, meaning if more than enough free memory is available but most of it is fragmented the allocation might fail. This is why I've come to say that the initial size of the app should be the maximum size the app should use because there's simply no guarantee that any memory in excess of that can be allocated.
     
  4. robrab2000-aa

    robrab2000-aa

    Joined:
    Feb 8, 2019
    Posts:
    117
    Right, thank you.

    I set this for the build: `PlayerSettings.WebGL.emscriptenArgs = "-s WASM_MEM_MAX=2047MB";`
    My understanding is that this should set the memory to the maximum that we can (when I set it to 2048 it just crashes, I'm assuming its a zero index thing or something).
     
  5. robrab2000-aa

    robrab2000-aa

    Joined:
    Feb 8, 2019
    Posts:
    117
    By not small I mean that its about 200-300mb. But we have assets that are 500+ which do seem to load so that's weird
     
  6. robrab2000-aa

    robrab2000-aa

    Joined:
    Feb 8, 2019
    Posts:
    117
    I have witnessed this issue of the build requiring memory to unpack the asset bundle and I definitely think this is where its crashing on chrome. as soon as it finishes downloading in Chrome, it crashes.

    On Firefox however I've managed to take snapshots throughout the process and this is what I found:
    Screenshot 2022-11-11 at 16.55.38.png
    This is taken during the load. there are two 1gb arrays declared here. On the left you can see 2 snapshots I took before loading the asset in question, and then here is what it looked like after it was loaded:
    Screenshot 2022-11-11 at 16.55.47.png

    We go from 1389 before loading, to 3025 during the load (unpacking I presume) and then back down to 1547 after. So it feels like its not reusing the available memory to unpack the asset. instead it declares a whole new array, pushing it over the threshold at which chrome crashes.
     
  7. CodeSmile

    CodeSmile

    Joined:
    Apr 10, 2014
    Posts:
    5,808
    No way! :eek: I think this is an absurd amount. 2GB may be the theoretical limit for a browser tab, but that memory cannot entirely be occupied by the WASM app alone. You may have to leave some headroom for other things the browser may need to allocate memory for in the tab. Perhaps tuning this down will already help preventing those crashes, try making several builds with max mem set to 1.75 or down to 1 gb.

    Moreover, consider how much memory users are willing for a browser tab to allocate. 2GB is extremely high for a single tab. 512 MB is already rather on the high-end side of things. In particular consider users with a machine that has "only" 8 GB of memory (or less) of which roughly 4 GB are in use by Windows most of the time and the browser consuming maybe 1-2 GB as well depending on number of open tabs, plus many other apps that may possibly be running.
     
  8. CodeSmile

    CodeSmile

    Joined:
    Apr 10, 2014
    Posts:
    5,808
    robrab2000-aa likes this.
  9. robrab2000-aa

    robrab2000-aa

    Joined:
    Feb 8, 2019
    Posts:
    117
    Thanks, yeah its a big allocation. The application (3dctrl.com) is not totally public facing so we have a certain amount of control over what hardware the app is accessed on, so having maximum headroom for assets to be loaded is more important than compatibility for our use case. We are usually only loading 1 or 2 assets at a time and they can be quite big.

    I will certainly try reducing it to 1.75 (I tried 1gb yesterday and had the same issue). :)
     
  10. robrab2000-aa

    robrab2000-aa

    Joined:
    Feb 8, 2019
    Posts:
    117
    Cheers for that link, I've tried setting "-s TOTAL_MEMORY=32MB" but this hasn't made any difference. I've also tried setting
    "-s WASM_MEM_MAX=1024MB", "-s WASM_MEM_MAX=1536MB", and "-s WASM_MEM_MAX=1792MB" and none of these have made any difference. By this I mean both that it still fails to load our large asset, but also that the base memory footprint isn't any different.

    Is there a way I can get a better idea of what's in that memory or at least where the 1.3gb allocation is coming from?
     
  11. K_Kuriyama

    K_Kuriyama

    Joined:
    Jul 4, 2020
    Posts:
    66
    @robrab2000-aa
    > max mem set to 1.75 or down to 1 gb.

    These have no basis.
    Changing the value of the WASM_MEM_MAX setting does not solve anything, as it only forces a forced termination in Runtime.
    If you want it to work as much as possible, you should set it as large as possible, specifically 4096. The rationale for this is taken from the survey results in the next section.
    However, this value is limited on the Unity side, so 2 GB seems to be the upper limit. We hope that this will be released with higher versions.

    > 2GB may be the theoretical limit for a browser tab, but that memory
    There is no rationale for this value. From my research, the upper limit is between 500MB and 4GB, depending on each browser. (This is close to the commit size shown in the browser's task manager, not the memory shown in MemoryPlofiler.)
    This is the result of my past investigation of the upper limit of the commit size for each OS and browser combination I have investigated.
    Please refer to the following page for reference as the upper limit varies depending on the OS as well as the browser.
    https://qiita.com/kazuki_kuriyama/items/39a508f7c9ea5d08082b


    > I mean that its about 200-300mb.
    The memory crashes depend on the commit size of the browser and the maximum memory a single tab of the browser can hold.
    However, as mentioned in other posts, the margin to reserve depends on the size of the assets that may be loaded, so 200-300mb may be correct in some cases.

    The difference in commit size, which is directly related to MemoryProfiler and browser crashes, can be somewhat suppressed by calling "System.gc()" in .jslib.
    Note, however, that frequent calls are necessary for performance.
     
    Last edited: Nov 15, 2022
  12. K_Kuriyama

    K_Kuriyama

    Joined:
    Jul 4, 2020
    Posts:
    66
    > We go from 1389 before loading, to 3025 during the load (unpacking I presume) and then back down to 1547 after.
    This value is indeed large, so please consider splitting the load into multiple assets and loading them separately, as this may help to reduce the peak.