Search Unity

WebGL - RuntimeError: memory access out of bounds

Discussion in 'Web' started by stonstad, Aug 11, 2020.

  1. stonstad

    stonstad

    Joined:
    Jan 19, 2018
    Posts:
    660
    I'm seeing frequent exceptions from Unity's main rendering loop in WASM-compiled WebGL players. I'm using Unity 2020.1. I'll create a support ticket but I want to get this information out there for other developers who may be impacted.

    Behavior: Unity's WebGL player will randomly crash the WASM runtime with a fatal "memory access out of bounds" error. It doesn't happen in the editor or PC/Desktop builds. The stacktrace shows it is internal to a material pass and render.

    ApplyMaterialPassAndKeywordsWithCache

    I suspect the issue can be remedied by turning off dynamic batching. Will try and report back here.
     
  2. ajeet09

    ajeet09

    Joined:
    May 18, 2020
    Posts:
    4
    Did you found any solution for this?
     
  3. stonstad

    stonstad

    Joined:
    Jan 19, 2018
    Posts:
    660
    Partially. See https://forum.unity.com/threads/async-await-and-webgl-builds.472994/page-2#post-6218307 for a few suggestions which may help. I am unable to eliminate all threading issues. WebGL threading, whether in C# or Unity itself, is very broken.

    Recent issues...
    - Builds broken - https://forum.unity.com/threads/can...settings-webgl-threadssupport-is-true.952772/
    - Concurrent collections unsupported - 1243361 (Open) WebGL Blows up on ConcurrentDictionary.Count

    Does anyone know the bug id for general threading being broken/unsupported in C# user code?
     
  4. jukka_j

    jukka_j

    Unity Technologies

    Joined:
    May 4, 2018
    Posts:
    953
    The root issue here is that multithreaded C# is currently not supported even with multithreaded Unity WebGL builds. Unfortunately to add insult to the injury, we got bit by a LLVM toolchain bug that is preventing multithreaded WebGL builds, that we are currently resolving.
     
  5. stonstad

    stonstad

    Joined:
    Jan 19, 2018
    Posts:
    660
    @jukka_j I am only now starting to appreciate the full impact and scope of the statement, "multithreaded C# is currently not supported even with multithreaded Unity WebGL builds".

    There are quite a few third-party assets in the store which provide support for WebGL. Many of these assets are highly rated and widely used. @BestHTTP and UniTask comes to mind... These assets use ConcurrentDictionary. As I understand it, ConcurrentDictionary is unsupported in WebGL (see https://fogbugz.unity3d.com/default.asp?1243361_i2ujptm2ut9qhjai) due to it being considered "multithreaded C#". This is a genuine land mine for compatibility issues! The compiler does not flag use of these classes and the only hint that something is wrong are random out of bounds memory errors that crash the compiled Unity WebGL player. Developers beware!

    *Updated to note that BestHTTP uses its own implementation of concurrent classes (not ConcurrentDictionary). As a result, BestHTTP is stable in WebGL builds (source developer).
     
    Last edited: Oct 6, 2020
    DerrickBarra and KamilCSPS like this.
  6. DerrickBarra

    DerrickBarra

    Joined:
    Nov 19, 2013
    Posts:
    210
    @stonstad : That's a major hint for a lot of WebGL developers here I'm sure! You have to be extra careful with plugins with WebGL to avoid dependencies like that one that'll cause head-scratching bugs down the line.
     
  7. jukka_j

    jukka_j

    Unity Technologies

    Joined:
    May 4, 2018
    Posts:
    953
    Btw, the original crash call stack trace does not read like something that would be related to lack of C# multithreading, but rather some other type of multithreaded rendering crash. If you have a test case you can narrow down, would be great to grab a bug report!
     
  8. stonstad

    stonstad

    Joined:
    Jan 19, 2018
    Posts:
    660
    I'm working to provide a reproduction project. Reproducing threaded and/or random errors can be very challenging.
     
  9. stonstad

    stonstad

    Joined:
    Jan 19, 2018
    Posts:
    660
    When we say C# user threading is unsupported it would help to know what this includes. A list would be helpful for us @jukka_j.

    Supported
    • Async / Await (without threading)
    • Lock
    • Mutex
    • Interlocked

    Unsupported:
    • ConcurrentDictionary
    • ConcurrentQueue
    • Thread / ThreadStart / ParameterizedThreadStart
    • Task.Run
    • ThreadPool
    • Timer (System.Timers.Timer and System.Threading.Timer).

    Updated 10/6/2020 (Unity 2020.1.6f)
     
    Last edited: Oct 6, 2020
    De-Panther likes this.
  10. jukka_j

    jukka_j

    Unity Technologies

    Joined:
    May 4, 2018
    Posts:
    953
    That list does look accurate, although I have to say that I do not have a particularly clear picture about this either. Basically any API that would spawn a thread is known to be unsupported, because C# Thread creation is not possible. Atomics APIs in C# should be fully supported (both locking and lockfree operations), so Lock, Interlocked and Mutex should all work.

    I believe what happened with ConcurrentDictionary is that .NET runtime spawns some kind of background manager thread upon the call to ConcurrentDictionary.Count, which is where the blowup occurred.

    .NET runtime is a large beast, and we have toyed with the idea of modifying its source code for Unity WebGL builds, but have not so far had the chance to tackle.
     
  11. stonstad

    stonstad

    Joined:
    Jan 19, 2018
    Posts:
    660
    I updated the above list. I am wondering if certain timer classes are unsupported?
     
  12. jukka_j

    jukka_j

    Unity Technologies

    Joined:
    May 4, 2018
    Posts:
    953
    Hard to say. I think it comes down to whether .NET implemented timers by spawning a thread that waits for X seconds on a mutex before releasing, or by using an async event that would be singlethreading friendly. Help charting the landscape here would certainly be appreciated.
     
  13. stonstad

    stonstad

    Joined:
    Jan 19, 2018
    Posts:
    660
    System.Threading.Timer uses ThreadPool threads. System.Timers.Timer uses System.Threading.Timer. I'll update the list above.
     
  14. stonstad

    stonstad

    Joined:
    Jan 19, 2018
    Posts:
    660
    @jukka_j OK, I included a reproduction project for random freezes that are happening without an error or stack trace. WebGL Player Freezes (Bug Case 1283169). I wanted to make it as small as possible, and highly repeatable. The best case scenario for causing a freeze is to load the same WebGL player project in two browser tabs at the same time.
     
  15. jukka_j

    jukka_j

    Unity Technologies

    Joined:
    May 4, 2018
    Posts:
    953
    Thanks, I see the bug report, but it was closed - not sure what happened there. It is missing all the usual information around a bug (Unity editor version, priority, QA correspondent, etc.), I wonder if using "Unity Editor > Help > Report A Bug.." will get all of those set up and that is why it was closed?
     
  16. stonstad

    stonstad

    Joined:
    Jan 19, 2018
    Posts:
    660
    Thanks, @jukka_j . Resubmitted through Unity w/ Case #1283357
     
  17. stonstad

    stonstad

    Joined:
    Jan 19, 2018
    Posts:
    660
    The Unity documentation should be updated to reflect which .NET APIs are unsupported in the WebGL player. I understand the breakage might be temporary but a fix doesn't seem likely anytime soon.
     
    Last edited: Oct 13, 2020
  18. jukka_j

    jukka_j

    Unity Technologies

    Joined:
    May 4, 2018
    Posts:
    953
    Thanks, yeah, this is on the TODO list to do while we are able to address the multithreading breakage.