Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. Dismiss Notice

Bug Trying to extract a GZipStream in WebGL Release Build crashes the WebGL Player

Discussion in 'WebGL' started by AEonAX, Jul 26, 2023.

  1. AEonAX

    AEonAX

    Joined:
    Apr 17, 2019
    Posts:
    5
    My game relies on gzips to store player replays.
    Reading this gzips in webgl release builds crashes the webgl player.
    Minimal Reproducible Example: Add below script to an object

    Code (CSharp):
    1. using System.Collections;
    2. using System.Collections.Generic;
    3. using UnityEngine;
    4. using System.IO;
    5. public class Gzipper : MonoBehaviour
    6. {
    7.     // Start is called before the first frame update
    8.     void Start()
    9.     {
    10.         Debug.Log("Creating Gz");
    11.         using (FileStream file = new FileStream("test.gz", FileMode.OpenOrCreate, FileAccess.ReadWrite))
    12.         {
    13.             using (var gzip = new System.IO.Compression.GZipStream(file, System.IO.Compression.CompressionMode.Compress))
    14.             {
    15.                 using (var bwMeta = new BinaryWriter(gzip))
    16.                 {
    17.                     bwMeta.Write("aaa");
    18.                 }
    19.  
    20.             }
    21.         }
    22.         Debug.Log("Written Gz");
    23.         Debug.Log("Creating Mem");
    24.  
    25.         var masterMemoryStream = new MemoryStream();
    26.         Debug.Log("Reading Gz");
    27.         using (FileStream file = new FileStream("test.gz", FileMode.Open, FileAccess.Read))//<-- Crashes here I believe########################
    28.         {
    29.             using (var gzip = new System.IO.Compression.GZipStream(file, System.IO.Compression.CompressionMode.Decompress))
    30.             {
    31.                 gzip.CopyTo(masterMemoryStream);
    32.             }
    33.         }
    34.         Debug.Log("Read Gz " + masterMemoryStream.Length);
    35.  
    36.     }
    37. }
    38.  


    On release webgl build I get below error alert in browser
    I am using Unity 2022.3.4f1 Apple Silicon Edition
    Anyone knows a solution for this?
     
  2. CodeSmile

    CodeSmile

    Joined:
    Apr 10, 2014
    Posts:
    3,899
    You don't have filesystem access in the WebGL browser which may be the issue here. I'm not sure if "test.gz" gets redirected like File.WriteAllBytes should. Try just FileStream without the Gzip to see if that works.


    There is also a storage limit for files in the sandbox, I believe it's 10 MB at most but may be even lower than that.
     
  3. AEonAX

    AEonAX

    Joined:
    Apr 17, 2019
    Posts:
    5
    It works in non release builds! In my actual game most of the file based operations are working even in release build, except the above.
     
  4. CodeSmile

    CodeSmile

    Joined:
    Apr 10, 2014
    Posts:
    3,899
    Oh, in release. My immediate thought is: stripping level. Try building for release with stripping none, in case some code gets stripped out that is being needed but stripping couldn't know that.
     
  5. AEonAX

    AEonAX

    Joined:
    Apr 17, 2019
    Posts:
    5
    I solved it by switching to
    dotnetzip
     
    Michael_Swan likes this.
  6. unityruba

    unityruba

    Unity Technologies

    Joined:
    Nov 6, 2020
    Posts:
    233
    Someone else on the team ran into this recently, so I'll share the bug report they're creating for it! Thanks for bringing it up.
     
  7. arjuniscool0204

    arjuniscool0204

    Joined:
    May 16, 2017
    Posts:
    14
    I have the exact same issue.
     
  8. AEonAX

    AEonAX

    Joined:
    Apr 17, 2019
    Posts:
    5
  9. Michael_Swan

    Michael_Swan

    Joined:
    Aug 24, 2021
    Posts:
    49
    Have the same issue. Please fix asap.
     
  10. jukka_j

    jukka_j

    Unity Technologies

    Joined:
    May 4, 2018
    Posts:
    940
    Thank you so much for filing the issue. This came to my table to debug, and the root culprit turned out to be in upstream LLVM LTO optimizer. ( https://github.com/llvm/llvm-project )

    LTO optimizer is a special last-stage optimizer in the native LLVM build chain that looks for additional optimizations across the whole codebase, instead of just optimizing individual functions at a time.

    To mitigate the issue in current Unity, doing a Development build, or a Release build that sets optimization to "Shorter Build Time" should avoid the LTO optimizer from being triggered. The LTO optimizer currently is only enabled for "Runtime Speed" and "Disk Size" builds.

    We will have an update that separates LTO into its own setting (so you can do either "Runtime Speed" or "Runtime Speed with LTO"), since we unfortunately don't have a fix to the LLVM project repository at the time.
     
    Michael_Swan and KamilCSPS like this.
  11. mikaelwallen

    mikaelwallen

    Joined:
    Jun 3, 2016
    Posts:
    56
    Thank you jukka_j for the info.
    We just ran into this issue yesterday as well after switching to Unity 2022.3.8. Here are some followup questions:
    • Is there a way to control the "Code Optimization" flag programmatically?
    • Is it the LTO optimizer that is causing the very large increase in build time? In 2021.3.17 which we used before changing version the build time for release builds was significantly shorter.
    • Can we expect the new setting to build without LTO optimization to result in build times comparable to 2021.3.17?
    Thanks!
     
    KamilCSPS likes this.
  12. jukka_j

    jukka_j

    Unity Technologies

    Joined:
    May 4, 2018
    Posts:
    940
    > Is there a way to control the "Code Optimization" flag programmatically?

    That is a good question. Can you try with

    Code (csharp):
    1.  
    2. using UnityEditor;
    3.  
    4. EditorUserBuildSettings.SetPlatformSettings(BuildPipeline.GetBuildTargetName(BuildTarget.WebGL), "CodeOptimization", x);
    5.  
    where x is a string "BuildTimes", "RuntimeSpeed", or "DiskSize".

    > Is it the LTO optimizer that is causing the very large increase in build time? In 2021.3.17 which we used before changing version the build time for release builds was significantly shorter.

    I believe it is, although I don't have 100% certainty of this. My impression is that the upstream LLVM compiler changed in how it interprets the optimization if code consists of a mix of LTO-enabled and LTO-disabled object files, which is what Unity build system has had.

    > Can we expect the new setting to build without LTO optimization to result in build times comparable to 2021.3.17?

    I hope it would be faster. Especially if you enable targeting the new WebAssembly.Table and BigInt features (in case your project warrants requiring a newer browser version, see https://webassembly.org/roadmap/ for which browser versions are needed for those features):

    upload_2023-8-30_14-38-43.png
     
  13. mikaelwallen

    mikaelwallen

    Joined:
    Jun 3, 2016
    Posts:
    56
    Thanks jukka_j. Great info!
    It did work to change the Code Optimization like you mentioned.
    I am wondering a bit about the resulting application performance if we use "Shorter Build Time" now with 2022.3.8 compared to the "Speed" option which was default in 2021.3.17. Will the application be significantly slower?
     
  14. jukka_j

    jukka_j

    Unity Technologies

    Joined:
    May 4, 2018
    Posts:
    940
    It is hard to say. There is a good chance that the performance will be quite much the same. Or not. The optimizations that are applied depend a lot on whether the game is GPU or CPU bound. If it is GPU bound, then the CPU side optimization differences can be meaningless. Also, some of the more "aggressive" optimizations tend to become a little bit pedantic/academic in nature that the practical difference might not be large, it depends a lot on if there is a specific type of operation that the game code might be repeatedly doing that particularly is impacted by an optimization.

    Release Runtime Speed equals LLVM -O3 optimization level, whereas
    Release Shorter Build Time equals LLVM -O2 optimization level.

    There is a particularly infamous news piece from Linux world a few years back, where Linux dropped using -O3, saying "it is not worth it" and "it is buggy": https://www.phoronix.com/news/No-O3-For-Linux-Kernel . (although it should be clearly noted that in this Unity instance, the bug resides with LTO pass, not with -O3 pass in general, to avoid spreading fud)
     
    mikaelwallen likes this.
  15. Michael_Swan

    Michael_Swan

    Joined:
    Aug 24, 2021
    Posts:
    49
    @jukka_j Thanks very much for looking at it quickly and the speedy workaround! I've been trying to change to dotnetzip, and haven't been able to figure it out, so this might be a quick fix for now. Please let us know when/if the LTO is fixed.I'm assuming it'd be 6 months+, if ever.
    Can confirm that faster build is a lot faster. ie about 30 minutes faster for us.

    Is 2023 webgl better than 2022 atm? Is it worth upgrading to 2023?
     
  16. jukka_j

    jukka_j

    Unity Technologies

    Joined:
    May 4, 2018
    Posts:
    940
    > Please let us know when/if the LTO is fixed.I'm assuming it'd be 6 months+, if ever.

    This bug is not of Unity's doing and unfortunately we don't currently have a minimal repro to offer to the LLVM project about this issue. We are trying to coordinate one to the project, but cannot make any guarantees about what the fix timeline would be. That is why we are offering a separate option into Unity, and depending on how that goes, we are considering disabling LTO support altogether if this is buggy.

    >Is 2023 webgl better than 2022 atm? Is it worth upgrading to 2023?

    Yeah, 2023 WebGL is better than 2022. It is recommended to upgrade!
     
    Michael_Swan likes this.
  17. Michael_Swan

    Michael_Swan

    Joined:
    Aug 24, 2021
    Posts:
    49
    Yep, I realise it's not a unity issue. - And thank you so much for figuring it out finding a work around! It really helps a lot.

    Is there a a feature list of the upgrades from Webgl 2022 to 2023. I'm having a hard time convincing managers to upgrade to a non LTS version.
     
  18. Michael_Swan

    Michael_Swan

    Joined:
    Aug 24, 2021
    Posts:
    49
    Not sure if this is helpful:
    " If you want to take optimization into the extreme, you can tweak in gcc via --param the costs associated with certain optimizations. Additionally note that gcc now has the ability to put attributes at functions that control optimization settings just for these functions, so when you find you have a problem with -O3 in one function (or want to try out special flags for just that function), you don't need to compile the whole file or even whole project with O2."

    https://stackoverflow.com/questions/11546075/is-optimisation-level-o3-dangerous-in-g

    Might it be possible to exclude gzip from O3 optimisations ?
     
  19. bugfinders

    bugfinders

    Joined:
    Jul 5, 2018
    Posts:
    711
    I wonder if this is why a friend on playing my game gets an "angry red box" and gets the same on her computer (which i find thoroughly unusual) when others dont, but maybe she has even more of a potatoe for equipment than I do.. while I am using 2023.1 i think Im possibly on the fastest not the shortest.. Probllem of course is, being a phone i couldnt get more than angry red box out of her, and the fact her pc didnt run it either really screams something environmental.
     
  20. Michael_Swan

    Michael_Swan

    Joined:
    Aug 24, 2021
    Posts:
    49
    FYI, I found this bug testing when and where it crashed and then by writing about 30-40 debug.log commands through the code, seeing which ones got called and put more in closer and closer to the line it broke on. I found the line it broke on, rebuild a webgl with just tests on that code, which worked perfectly. So it had to be something before it got to that line. It was in-cased in a gzip extraction, so I searched for that and found this thread. You might need to do something similar to find the root cause of a release only bug.
     
    bugfinders likes this.