Search Unity

Multithreading and WebGL

Discussion in 'Web' started by ChristianKoGaMa, Jan 29, 2020.

Page 1 of 3

  1. ChristianKoGaMa

    ChristianKoGaMa

    Joined:
    Jul 11, 2017
    Posts:
    18
    I see that support for this has been added in 2019.3. What is the browser situation currently? Which browsers has enabled multithreading (shared array buffer) fully?
     
    ChristianKoGaMa, Jan 29, 2020
    #1

  2. lightmapper

    lightmapper

    Joined:
    Jan 14, 2015
    Posts:
    27
    lightmapper, Jan 29, 2020
    #2

  3. hsallander

    hsallander

    Joined:
    Dec 19, 2013
    Posts:
    46
    @ChristianKoGaMa Hi! Where did you find the info that support for multithreading has been added in 2019.3? The info in the Unity documentation for WebGL in 2019.3 still states the following:
    • Threads are not supported due to the lack of threading supporting in JavaScript. This applies to both Unity’s internal use of threads to speed up performance, and to the use of threads in script code and managed dlls. Essentially, anything in the System.Threading namespace is not supported.
     
    hsallander, Feb 17, 2020
    #3

  4. Schubkraft

    Schubkraft

    Unity Technologies

    Joined:
    Dec 3, 2012
    Posts:
    1,073
    We added WASM multi-threading as an experimental feature in 2019.1 but due to the CPU exploit issues this has not been spread as far as folks were hoping as the link above shows.
     
    Schubkraft, Feb 17, 2020
    #4
    davtam and hsallander like this.

  5. stonstad

    stonstad

    Joined:
    Jan 19, 2018
    Posts:
    660
    @Schubkraft Is WebGL threading technically possible in 2019.x by setting webGLThreadsSupport to 1 in project settings?
     
    stonstad, Apr 30, 2020
    #5
    Lightning_A likes this.

  6. stonstad

    stonstad

    Joined:
    Jan 19, 2018
    Posts:
    660
    -- @Schubkraft Is WebGL threading technically possible in 2019.x by setting webGLThreadsSupport to 1

    Re: multithreading and WebGL -- you are damaging good will with developers by heavily marketing this feature and then failing to explain the degree to which it is working, isn't working, or even should be expected to work. Is this related to the issue with ConcurrentDictionary.Count causing the WebGL player to crash?
     
    stonstad, May 4, 2020
    #6
    szymongvox and DrViJ like this.

  7. bogdanspatial

    bogdanspatial

    Joined:
    Aug 28, 2019
    Posts:
    3
    @Schubkraft we'd also love to get more info about the extent to which webGL multithreading is functional. It's in FireFox nightly and shipped a while ago with Chrome 74.
     
    bogdanspatial, Jul 3, 2020
    #7

  8. bogdanspatial

    bogdanspatial

    Joined:
    Aug 28, 2019
    Posts:
    3
    Ah, I found some info in the webgl roadmap post:
    I'm interested in knowing whether the experimental multithreading support in 2019.3 & 2020.1 is for the internals of the Unity engine only or if anything in Sytem.Threading.* works yet
     
    bogdanspatial, Jul 3, 2020
    #8
    Rallix and Carrotpie like this.

  9. bogdanspatial

    bogdanspatial

    Joined:
    Aug 28, 2019
    Posts:
    3
    bogdanspatial, Jul 28, 2020
    #9
    deus0 likes this.

  10. deus0

    deus0

    Joined:
    May 12, 2015
    Posts:
    256
    I can confirm multi threading enabled worked on chrome but still displays the error message for firefox both 79 and 80.
    Only worked with a empty cube scene so far. Didnt work with my ecs project though.
    Was tested on 2020.1.1 I believe.
     
    Last edited: Aug 17, 2020
    deus0, Aug 3, 2020
    #10
    xeniaosense, franMx, squarelover and 1 other person like this.

  11. squarelover

    squarelover

    Joined:
    Nov 14, 2012
    Posts:
    31
    interesting
     
    squarelover, Aug 12, 2020
    #11

  12. DrViJ

    DrViJ

    Joined:
    Feb 9, 2013
    Posts:
    163
    I tried to enable WebGLThreadsSupport in 2020.1.2 and it breaks builds, the project does not compile after this setting is enabled. Looks like this feature is broken in 2020.1.2
     
    Last edited: Aug 16, 2020
    DrViJ, Aug 16, 2020
    #12
    stonstad and deus0 like this.

  13. DrViJ

    DrViJ

    Joined:
    Feb 9, 2013
    Posts:
    163
    What unity version do you have? I cant enable this function in 2020.1.2 cause builds start failing :(

    Edit:
    My bug was reproduced with Unity QA Team:
    https://forum.unity.com/threads/can...settings-webgl-threadssupport-is-true.952772/
     
    Last edited: Aug 17, 2020
    DrViJ, Aug 16, 2020
    #13
    stonstad and deus0 like this.

  14. deus0

    deus0

    Joined:
    May 12, 2015
    Posts:
    256
    Hi Dr ViJ, I believe that you need to build the web build with development build settings. I can confirm in 2020.1.2f that I get the same error without development, same with 2020.1.1f.
    Edit: Wait never mind, dev is not working now too! Maybe I set something differently.. I think it's just not working anymore haha.
     
    Last edited: Aug 17, 2020
    deus0, Aug 17, 2020
    #14
    DrViJ likes this.

  15. DrViJ

    DrViJ

    Joined:
    Feb 9, 2013
    Posts:
    163
    Thank you! I will try this way as temporary solution until bug is fixed.

    Edit:
    I tried to compile with Development Build, Unfortunatly the build failed, so I will wait for Unity Technologies fix that. But thank you for support :)
     
    Last edited: Aug 17, 2020
    DrViJ, Aug 17, 2020
    #15
    deus0 likes this.

  16. Claytonious

    Claytonious

    Joined:
    Feb 16, 2009
    Posts:
    904
    Is there any unity version where this currently works?
     
    Claytonious, Sep 10, 2020
    #16

  17. stonstad

    stonstad

    Joined:
    Jan 19, 2018
    Posts:
    660
    2020.1.2f, 2020.1.4f fails. 2020.1.0f works.
     
    stonstad, Sep 12, 2020
    #17
    Claytonious likes this.

  18. deus0

    deus0

    Joined:
    May 12, 2015
    Posts:
    256
    I had difficulties with EntityCommandBuffers inside jobs were breaking. I removed that and I got it to work. But burst didnt work. And my custom animation system uses burst to do bone skinning. Does anyone know if unitys SkinnedMeshRenders work lol
     
    deus0, Sep 15, 2020
    #18

  19. tim_jones

    tim_jones

    Unity Technologies

    Joined:
    May 2, 2019
    Posts:
    287
    @deus0 what error(s) did you get from Burst?
     
    tim_jones, Sep 17, 2020
    #19

  20. stonstad

    stonstad

    Joined:
    Jan 19, 2018
    Posts:
    660
    Surly this can't be true:

     
    stonstad, Sep 22, 2020
    #20

  21. stonstad

    stonstad

    Joined:
    Jan 19, 2018
    Posts:
    660
    WebGL build status (case 1270915)

    2020.1.0f: OK
    2020.1.2f: FAIL
    2020.1.4f: FAIL
    2020.1.6f: FAIL
    2020.1.7f: FAIL
     
    Last edited: Oct 5, 2020
    stonstad, Sep 22, 2020
    #21
    DrViJ likes this.

  22. jukka_j

    jukka_j

    Unity Technologies

    Joined:
    May 4, 2018
    Posts:
    953
    I think we touched on this on another forum thread already, though echoing the summary here:
    - The regression that multithreading builds fail is in our radar - on my personal bug fix queue..
    - Unfortunately C# scripts are not possible to run multithreaded (after the fix to above multithreading build regression) at present - we are looking into how to enable this, due to limitations in WebAssembly it is looking like quite a hard thing to add support to.
     
    jukka_j, Oct 6, 2020
    #22
    codestage, xeniaosense, swProdBdk and 2 others like this.

  23. stonstad

    stonstad

    Joined:
    Jan 19, 2018
    Posts:
    660
    Thank you @jukka_j for keeping us updated. The transparency in communication, even if its bad news, is also appreciated!
     
    stonstad, Oct 6, 2020
    #23

  24. Nick_CR

    Nick_CR

    Joined:
    Jul 15, 2020
    Posts:
    1
    So what exactly will be multi threaded once this regression is fixed? Is there anywhere we can track the progress of the work to enable multithreading in C#?
     
    Nick_CR, Nov 26, 2020
    #24

  25. stonstad

    stonstad

    Joined:
    Jan 19, 2018
    Posts:
    660
    Any updates on either?
     
    stonstad, Dec 21, 2020
    #25
    swProdBdk and moritz_jaeger like this.

  26. LukeGeach

    LukeGeach

    Joined:
    Nov 22, 2019
    Posts:
    8
    Can we have an update on this? We have two projects finished and awaiting this to be fixed before publishing.
     
    LukeGeach, Jan 6, 2021
    #26
    stonstad and moritz_jaeger like this.

  27. swProdBdk

    swProdBdk

    Joined:
    Apr 22, 2020
    Posts:
    10
    Also excited to hear about any updates :)
     
    swProdBdk, Jan 12, 2021
    #27

  28. stonstad

    stonstad

    Joined:
    Jan 19, 2018
    Posts:
    660
    "The regression that multithreading builds fail is in our radar"

    @jukka_j An update would be greatly appreciated, as it would be the minimum Unity could do after screwing us nicely.
     
    stonstad, Jan 12, 2021
    #28

  29. birschtl

    birschtl

    Joined:
    Dec 21, 2013
    Posts:
    3
    @jukka_j @stonstad any news on this multithreading issue? Haven't seen anything change in the latest beta neither (2021.1.0b4.1811). Support for System.Threading would be really great!
     
    birschtl, Feb 2, 2021
    #29
    deus0 likes this.

  30. jukka_j

    jukka_j

    Unity Technologies

    Joined:
    May 4, 2018
    Posts:
    953
    Basically the current task roadmap we have to restore the previous multithreading looks like follows:
    1. Enable support to emsdk for Apple M1 hardware: https://github.com/emscripten-core/emsdk/issues/671
    2. Help coordinate a fix to browser garbage collection issue when multithreading is used: https://github.com/WebAssembly/threads/issues/176
    3. Land part two to fix a regression in upstream Google Emscripten: (part 1 was here and here)
    4. Implement a certain migration change in our internal testing infrastructure (changing virtualization mechanism for test runners)
    5. Update Emscripten toolchain for Unity 2021.2 release
    6. Fix the known regressions with enabling C/C++ multithreading with Unity WebGL builds
    7. Investigate the existing issues with C/C++ multithreading highlighted by @stonstad

    (item 2 is not blocking in this sequence, will done in parallel, but will block any stable release of multithreading)

    In the current stage, like was discussed in more detail in the earlier thread https://forum.unity.com/threads/async-await-and-webgl-builds.472994/ , what we are looking to enable is "C/C++ multithreading" in WebGL, i.e. multithreading the native task threads that are used in WebGL builds.

    To get to C# multithreading with enabling multithreading of user C# code, a complete redesign and reimplementation of how garbage collection works in C# on the web will be needed. This is because one cannot scan the native stack (due to security restrictions) on the web, which is the way that our existing Boehm library based garbage collection design works.

    If all go well, we should get the above 1.-7. resolved by Unity 2021.2 release. It is likely that WebGL multithreading will remain in some kind of pre-release status until the whole C# multithreading story is resolved, so that includes the final "multithreaded C# garbage collection" piece.
     
    jukka_j, Feb 3, 2021
    #30
    De-Panther, adamgolden, codestage and 9 others like this.

  31. deus0

    deus0

    Joined:
    May 12, 2015
    Posts:
    256
    I am hoping by 2022 then sometime. It will be really awesome if our ECS projects can be pushed to webgl (outside of Tiny). Thanks for the details.
     
    deus0, Feb 3, 2021
    #31

  32. Chris9465

    Chris9465

    Joined:
    Nov 3, 2020
    Posts:
    24
    Thank you for the update! <3
     
    Chris9465, Mar 26, 2021
    #32
    deus0 likes this.

  33. EmeralLotus

    EmeralLotus

    Joined:
    Aug 10, 2012
    Posts:
    1,462
    Any recent updates on multithreading and webgl
     
    EmeralLotus, May 17, 2021
    #33
    bdovaz likes this.

  34. Schubkraft

    Schubkraft

    Unity Technologies

    Joined:
    Dec 3, 2012
    Posts:
    1,073
    We are currently at step 5, trying to get emscripten updated for 21.2.
     
    Schubkraft, May 17, 2021
    #34
    LogicFlow, Samasab, DerrickBarra and 7 others like this.

  35. OceanX000

    OceanX000

    Joined:
    Feb 24, 2021
    Posts:
    120
    Then，if emscripten updated, is it possible to use SIMD？
     
    OceanX000, May 18, 2021
    #35

  36. Schubkraft

    Schubkraft

    Unity Technologies

    Joined:
    Dec 3, 2012
    Posts:
    1,073
    If you are talking about WASM SIMD then that is not part of our current efforts.
     
    Last edited: May 19, 2021
    Schubkraft, May 18, 2021
    #36

  37. jukka_j

    jukka_j

    Unity Technologies

    Joined:
    May 4, 2018
    Posts:
    953
    An update since Stefan's last note, it looks like Emscripten 2 is now finally making it in into the release tree! Unity 2021.2 Beta should see the use of Emscripten 2.0.19. This will unblock us forwards for step 6 on restoring native C/C++ multithreading, and we have now had preliminary conversations with the IL2CPP scripting team on how to also start tackling C# multithreading. The Unity release cogs move slowly, but they do move!

    Eventually, yes! I spent about three months of dev time last year contributing SSE compilation support to Emscripten. ( https://github.com/emscripten-core/emscripten/pulls?q=is:pr+is:closed+author:juj+sse ) That work is now finally making its way into Unity tree to use. We have plans to leverage this support both in classic Unity and in Tiny Unity, although this is not something that is super high in our priority list at the moment: improving URP renderer, and unlocking more capabilities for mobile browser support are higher on the list.

    (Super advanced users could already after that change utilize SSE in their native C/C++ plugins for Unity with a little bit of configuration of the Emscripten compiler and linker flags)

    There is a large body of work we are now talking about to go into our testing infrastructure to enable more mobile oriented testing - historically we would be testing in desktop browsers, but that will need to change.
     
    jukka_j, May 20, 2021
    #37
    LogicFlow, OneManArmy3D, Arowx and 5 others like this.

  38. gtk2k

    gtk2k

    Joined:
    Aug 13, 2014
    Posts:
    288
    @jukka_j
    Will emscripten 2.0.19 allow jslib to be written in ES6+?
     
    gtk2k, May 20, 2021
    #38

  39. jukka_j

    jukka_j

    Unity Technologies

    Joined:
    May 4, 2018
    Posts:
    953
    jukka_j, May 20, 2021
    #39
    De-Panther, KamilCSPS and gtk2k like this.

  40. gtk2k

    gtk2k

    Joined:
    Aug 13, 2014
    Posts:
    288
    gtk2k, May 20, 2021
    #40

  41. stonstad

    stonstad

    Joined:
    Jan 19, 2018
    Posts:
    660
    @jukka_j Thank you for the communication and all the effort that goes behind moving this forward.
     
    stonstad, May 20, 2021
    #41
    jukka_j likes this.

  42. De-Panther

    De-Panther

    Joined:
    Dec 27, 2009
    Posts:
    589
    WOW! Great news!
     
    De-Panther, May 20, 2021
    #42

  43. De-Panther

    De-Panther

    Joined:
    Dec 27, 2009
    Posts:
    589
    Tried the 2021.2.0a17 and there's a bug with native plugins in packages.

    Exception message: "Error: ENOENT: no such file or directory, open 'PATH_TO_PROJECT\PROJECT_FOLDER\Packages\com.de-panther.webxr\Runtime\Plugins\WebGL\webxr.jslib'" | Error: ENOENT: no such file or directory, open 'PATH_TO_PROJECT\PROJECT_FOLDER\Packages\com.de-panther.webxr\Runtime\Plugins\WebGL\webxr.jslib'

    It should have loaded it from 'PATH_TO_PROJECT\PROJECT_FOLDER\Library\PackageCache\com.de-panther.webxr@0.10.1-preview\Runtime\Plugins\WebGL\webxr.jslib'
     
    De-Panther, May 20, 2021
    #43

  44. Schubkraft

    Schubkraft

    Unity Technologies

    Joined:
    Dec 3, 2012
    Posts:
    1,073
    Could you put together a quick repro case as a bug report for this and ping the case number here? Looks like something that slipped through when we redid the WebGL build pipeline (this is an additional Unity-wide effort to speed up build/iteration times especially of subsequent builds in 21.2).

    Thank you!
     
    Schubkraft, May 21, 2021
    #44
    De-Panther likes this.

  45. OceanX000

    OceanX000

    Joined:
    Feb 24, 2021
    Posts:
    120
    Great news!
     
    OceanX000, May 21, 2021
    #45

  46. jukka_j

    jukka_j

    Unity Technologies

    Joined:
    May 4, 2018
    Posts:
    953
    2021.2.0a17 already has a new "incremental build pipeline" for doing WebGL builds. This can improve iteration times, and looks like that has regressed somehow. a17 does not just yet have the Emscripten 2.0.19 update, that is currently in the merge queue, meaning that it should be imminently landed. Not sure if the next released version will be a 2021.2.0a18 or a 2021.2.0b1, though unless something bumps the patch from the queue, whichever the next version is, it will have the Emscripten compiler update.

    Nevertheless, a bug report would be very much welcomed, since we must have missed this scenario in the incremental build pipeline work.
     
    jukka_j, May 21, 2021
    #46
    OceanX000 and De-Panther like this.

  47. brendanduncan_u3d

    brendanduncan_u3d

    Unity Technologies

    Joined:
    Jul 30, 2019
    Posts:
    437
    The package jslib bug with the new incremental build pipeline has been fixed and will be in an upcoming release soon.
     
    brendanduncan_u3d, May 21, 2021
    #47
    DrViJ, KnifeOffWork, OceanX000 and 2 others like this.

  48. cowtrix

    cowtrix

    Joined:
    Oct 23, 2012
    Posts:
    322
    Will C# scripts that utilize things like `System.Threading` or `System.Threading.Tasks` ever see support within browser exports?
     
    cowtrix, Jun 26, 2021
    #48
    DrViJ and deus0 like this.

  49. chris_hellsten

    chris_hellsten

    Joined:
    May 5, 2014
    Posts:
    12
    I understand that C# (System.Threading.*) threads are a can of worms but just to get some clarification, when you say C/C++ engine threading, I presume user-written Jobs (Unity Job System) / ECS lambdas leveraging Schedule / ScheduleParallel are also expected to work in background threads when this fix lands?
     
    chris_hellsten, Jun 30, 2021
    #49

  50. stonstad

    stonstad

    Joined:
    Jan 19, 2018
    Posts:
    660
    @jukka_j @brendanduncan_u3d @Schubkraft
    At risk of sounding like a broken record, are there any updates with regard to restoring C# threading support in WebGL? This is an essential feature for platform compat. It is painful building products which leverage multithreading on mobile and then abandons it for single threading in WebGL.
     
    stonstad, Jul 15, 2021
    #50
    Stormer2020, sergeperepel, fxlange and 1 other person like this.
Page 1 of 3