Downloading a 250MB File Causes "Out of Memory" Error

I'm trying to port a mobile app to WebGL and I'm struggling a bit with memory/storage. Specifically, I download a 250MB zip file to Application.persistentdatapath, which I understand uses IndexedDB internally.

In Edge, this works fine but in Chrome, I get the "Aw snap" error with the error code "out of memory". At this point, developer tools shut down so I can't be sure, but I don't see anything out of order in the javascript console. Smaller zip files work fine so there's no problem with the logic. I increased the heap size by 250MB and it still crashes, which makes sense since it shouldn't really be in the heap since I'm streaming the download to a file, not loading it all in memory at once.

Now there's a long pause after the file finishes downloading and I don't know if that's relevant. I disabled "data caching" just in case but this doesn't make any difference. I have plenty of free hard drive space so I don't think Chrome is struggling to find enough space for a 250MB file.

I've read through the WebGL memory management article on the blog too but it's very out of date and doesn't offer any insight on this particular problem. At least, not to me.

Now, I *could* upload the contents of the zip to a folder and download each file separately and that might well mitigate or fix the problem. And it's not terribly inconvenient. But I don't want to work around a problem I don't fully understand when I'm still relatively new to WebGL (but not Unity.) I'd much rather get a proper understanding of what's going on when I write a 250MB zip to persistentdatapath, why it causes a problem, why only on Chrome, and how to go about fixing it. In the future, I may need a single file which is 250MB and when I do, not using a zip won't change anything.

 

To unpack a zip, the browser will need to have enough room to hold both the zipped file and its unpacked contents in memory at the same time (probably more because there's other things in your project, like the Unity engine).

Every web browser handles its memory cache a little differently, some are more strict.

 

The error happens before I even think about unpacking it so that's not an issue here. If I just downloaded the contents of the zip separately, an individual file might end up being 250MB. I will still need to know how to download that file without crashing the browser.

I appreciate that the implementation will vary between browsers but I will nevertheless need to know how to negotiate this correctly and what specifically the issue is.

• Does "Aw snap, code "out of memory" always indicate a particular type of memory allocation that's failed? Is there any way to determine which?
  
• Is there any way to determine the available amount of storage available of whichever type of memory it's running out of before downloading the file?
• Assuming, it's not heap memory (which it seems not to be) then is there any way to request higher allocation?
• Are there any memory management/allocation issues with Unity 2018.3 that would make it necessary to upgrade to 2019.x?

 

I have the same issue with writing large files (also around 250 MB) to the persistent data path in WebGL. I use this to cache files downloaded from a server. So any solutions appreciated. Using Unity 2019.4.8f1

 

Same to me. In Unity 2020.1, when unpacking a 150mb asset bundle, it needs around 2gb of ram when unpacking. Maybe more, but then chrome crashes. In firefox there are no problems at all.

So, why does it need way more ram than the unpacked asset size to unpack the asset bundle? And why only in Chrome?

 

Unable to allocate 512.000 MB
Heap size is now: 501.063 MB (525,402,112 bytes)

Using chrome 88.0.4324.93 (Official Build) (32-bit)
On Android 10, Galaxy S10e