Search Unity

How to Avoid WebGL Thread Usage?

Discussion in 'Web' started by stonstad, Oct 15, 2020.

  1. stonstad

    stonstad

    Joined:
    Jan 19, 2018
    Posts:
    659
    Given that the WebGL player cannot use threads, concurrent collections, or anything that touches System.Threading, how does one ensure this isn't happening in external libraries ... or even those which are part of .NET?

    For example, how would one know if compiled DLLs for Newtonsoft.Json or .NET (HttpClient) uses a concurrent collection within the compiled DLL? We can avoid async/await easily enough, but determining if a concurrent collection is used internally within a library isn't straight-forward. Ironically, even libraries which are created to help Unity (i.e. UniTask) make the simple error of using a concurrent dictionary internally -- knowledge of this issue is not widely known.

    The WebGL build process does not warn against these scenarios at compile and other than a player freeze, there is no indication at runtime. I'm honestly asking -- the current approach of developing, testing, and then hunting down usage of a taboo class that fails 1/20 times is very painful.
     
    stonstad, Oct 15, 2020
    #1
    _geo__ and De-Panther like this.

  2. KamilCSPS

    KamilCSPS

    Joined:
    May 21, 2020
    Posts:
    448
    Do we know how much of that issue is Emscripten compiling vs Unity's C# to C?

    Noticed that Unity uses a 2.5 year old version of Emscripten (v1.38.10: 07/23/2018)
    , since then it has added a lot of support for thread and memory handling, notably: https://github.com/emscripten-core/emscripten/pull/8365.

    But I'll be honest, mostly talking out of my ass on this particular issue.
     
    KamilCSPS, Oct 15, 2020
    #2

  3. jukka_j

    jukka_j

    Unity Technologies

    Joined:
    May 4, 2018
    Posts:
    953
    The issue with not being able to support C# threads come from the inability to examine the execution stack in WebAssembly (due to security design reasons), which is needed for (multithreaded) garbage collection; the issue is not due to Unity's use of Emscripten or due to the Emscripten version that is being used.

    You would not. That is why the workflow is that Unity plugin DLL developers need to design and develop their plugins to be WebGL & WebGL multithreading compatible.

    I believe we have discussed this already before in another thread, and we understand this trouble, but we do not have a fix to offer to this at present time, as painful as it is for you, and for me to write. I am sorry to say that opening another thread will not help this matter.
     
    jukka_j, Oct 16, 2020
    #3
    KamilCSPS likes this.

  4. De-Panther

    De-Panther

    Joined:
    Dec 27, 2009
    Posts:
    589
    Maybe a tool that can check the DLL content and see if it calls unsupported system threads methods/classes?
     
    De-Panther, Oct 16, 2020
    #4

  5. jukka_j

    jukka_j

    Unity Technologies

    Joined:
    May 4, 2018
    Posts:
    953
    Maybe. Such a tool would need to be implemented, and we simply do not have the solution to that today.

    The fact is that there are a lot of issues that we are working through, and we have only so many developers on the web team, that we have to apply prioritization. Supporting non-experimental features over experimental features comes at a priority. We do have marked down a task to audit through the extent of .NET APIs that can be supported in multithreading enabled builds, and we are looking to recover multithreading support for Unity 2020 releases, but unfortunately everything takes time. I wish I could give you more here, but this is where we are currently at.
     
    jukka_j, Oct 16, 2020
    #5
    De-Panther and KamilCSPS like this.

  6. KamilCSPS

    KamilCSPS

    Joined:
    May 21, 2020
    Posts:
    448
    Thank you Jukka, and I feel the frustration (&restrain) in your post. I don't have a horse in this thread specifically but we are ramping up to use WebGL more and more extensively over the next few years here. Consequently, strong WebGL support from Unity is essential for the success of our projects and thus my interest in various technical topics.

    You mentioned the limited ressources; are there channels you recommend that the userbase can use to communicate to Unity the importance of continuing investment in WebGL?
     
    KamilCSPS, Oct 16, 2020
    #6

  7. stonstad

    stonstad

    Joined:
    Jan 19, 2018
    Posts:
    659
    Could you explain why this is causing runtime instability? From your helpful explanation, it sounds like the challenge is threaded garbage collection is unsupported due to an inability to examine the execution stack -- why would this manifest as random freezes or crashes?

    Providing some counterpoint here... I'm not sure how realistic the expectation is, especially considering Unity's own framework devs aren't doing the same within .net. But your point is understood, heard, and appreciated.

    Guilty as charged -- I am definitely generating visibility to the issue and I am continuing to communicate the pain and urgency while pursuing a work-around and solutions. While I do this, I'd like to convey my gratitude for your assistance and advocacy to the community.
     
    Last edited: Oct 16, 2020
    stonstad, Oct 16, 2020
    #7

  8. jukka_j

    jukka_j

    Unity Technologies

    Joined:
    May 4, 2018
    Posts:
    953
    Beyond the usual - communicating on WebGL forums, raising bug reports and upvoting bugs you care about, there are a few channels that come to my mind that people can reach out:

    Unity runs an Advisory Panel program (https://unity.com/advisorypanel) that allows providing non-bug related feedback.

    Larger companies can reach out via Enterprise Support (https://unity.com/pages/unity-enterprise) or via Unity Core Support (https://unity.com/products/unity-core-support) to help WebGL meet their needs.

    Also being vocal in Beta related communication channels (https://unity3d.com/beta/2020.2b) and Beta forums (https://forum.unity.com/categories/betas-experimental-features.86/) about WebGL helps raise the noise.

    Ultimately, keeping building and deploying on the web (Editor collects some amount of analytics of what platforms are used and how much), writing blog posts and videos about games & non-games WebGL applications, success reports, and war stories that find the audience is probably the most important thing. Unity WebGL found a lot of use in Itch.io game jams because of the ease of sharing, and the amount of WebGL games that come out there has opened eyes inside Unity as to how many people use WebGL.

    And maybe as last stretch guerrilla targeting these kinds of feedback directly to Unity people on Twitter (https://unity.com/community/advocates, https://unity.com/our-company ) can make an impact in raising awareness as well :)
     
    jukka_j, Oct 17, 2020
    #8
    KamilCSPS and De-Panther like this.

  9. jukka_j

    jukka_j

    Unity Technologies

    Joined:
    May 4, 2018
    Posts:
    953
    The lack of ability to inspect the stack is not causing instabilities. The lack of stack introspection is causing the inability to enable multithreaded garbage collection, which results in the requirement to disable C# multithreading.

    This disabled C# threads support is what is then causing all these limitations to .NET APIs, in which, unfortunately, we currently can only abort at runtime when their use is attempted, rather than e.g. giving a project build time error about an unsupported API being used.

    I am not sure about the randomness of the crashes, if one attempts to use an unsupported multithreaded .NET API, then it should abort every time (that it goes down that unsupported code path - maybe it internally doesn't do that always, depending on timing or some other factors?).

    If you have a scenario that is exhibiting random crashing instead, and it does not occur in relation to any attempt to do multithreading in a .NET API, then that sounds like a bug that should be reported.
     
    jukka_j, Oct 17, 2020
    #9

  10. KamilCSPS

    KamilCSPS

    Joined:
    May 21, 2020
    Posts:
    448
    All good channels, noted! As for this comment ^ in particular, my experience is that WebGL is very much in use in non-game environments such as immersive online training and learning. Three of the biggest players in that field are military, govt and academia. And I can tell you for a fact that the first two (both contracted out or internally built) -do not- share their experience and have editor-analytics disabled by default... skewing that data set immensely. Food for thought when the marketing department uses that argument against WebGL ;)

    When a target platform is selected in the Editor, I noticed Unity can analyse the calls being done to Unity's API to verify compatibility with said platform. I assume this is only possible because it's to Unity's API right? No such platform-specific verification can be made to .NET's API?
     
    Last edited: Oct 17, 2020
    KamilCSPS, Oct 17, 2020
    #10

  11. jukka_j

    jukka_j

    Unity Technologies

    Joined:
    May 4, 2018
    Posts:
    953
    Yes/no/maybe/I don't know. It depends on many factors, such as whether the code paths are static, or can happen dynamically/driven by data. Or whether function call flows can occur through reflection, which is quite heavily used in .NET. Unfortunately I do not know at this point.

    It is something we may explore, or we may go through a different route of rewriting the garbage collector to use a completely different approach in multithreaded applications.
     
    jukka_j, Oct 18, 2020
    #11
    KamilCSPS likes this.