Search Unity

  1. Unity 2019.1 is now released.
    Dismiss Notice

WebGL Streaming Instantiation Support - Errors on Loading in Browser

Discussion in '2019.1 Beta' started by JJJohan, Nov 15, 2018.

  1. JJJohan

    JJJohan

    Joined:
    Mar 18, 2016
    Posts:
    169
    I was excited to see that in Unity 2019.1 Alpha 9, WebGL now has an option to enable Wasm streaming instanation. However it looks like there's a bit more involved than simply ticking the box in the player settings to have it working.

    Simply uploading the output files results in the bellow console output:

    Code (JavaScript):
    1. wasm streaming compile failed: TypeError: Response has unsupported MIME type
    2.  
    3. falling back to ArrayBuffer instantiation
    4.  
    5. on the web, we need the wasm binary to be preloaded and set on Module['wasmBinary']. emcc.py will do that for you when generating HTML (but not JS)
    6.  
    7. failed to asynchronously prepare wasm
    8.  
    9. uncaught exception: abort("abort(\"on the web, we need the wasm binary to be preloaded and set on Module['wasmBinary']. emcc.py will do that for you when generating HTML (but not JS)\")
    Unfortunately the ArrayBuffer fallback appears to simply fail.

    Reading the WebAssembly spec as well as the Emscripten compile instructions, it appears that for streaming instantiation to work the files need to have their Content-Type set on the server as application/wasm. Sadly this doesn't appear to change the outcome.



    @Marco-Trivellato would you have any suggestions?

    I suspect Amazon S3 isn't serving the Content-Type header correctly.. Other headers (e.g. Content-Encoding gzip) behave correctly.
     
    Last edited: Nov 15, 2018
  2. Marco-Trivellato

    Marco-Trivellato

    Unity Technologies

    Joined:
    Jul 9, 2013
    Posts:
    1,650
    Hi @JJJohan,

    To be honest, I only tested it on our self hosted apache server so far, see this build. Basically you need to add .wasm to the mime types and make sure it's configured according to the compression selected in PlayerSettings.WebGL.compression.
    For brotli, the .htacces looks like:
    Code (csharp):
    1.  
    2. <IfModule mod_mime.c>
    3.   AddEncoding br .unityweb
    4.   AddType application/wasm .wasm
    5.   AddEncoding br .wasm
    6.   AddOutputFilterByType DEFLATE application/wasm
    7. </IfModule>
    similarly, for gzip:
    Code (csharp):
    1.  
    2. <IfModule mod_mime.c>
    3.   AddEncoding gzip .unityweb
    4.   AddType application/wasm .wasm
    5.   AddEncoding gzip .wasm
    6.   AddOutputFilterByType DEFLATE application/wasm
    7. </IfModule>
    So for IIS you need to do something similar. We will try to update the docs as soon as possible.

    One known issue is that we are not currently caching the wasm file but this should be fixed in the coming weeks.

    Thanks for being an early adopter :)
     
    JJJohan likes this.
  3. AlexHell

    AlexHell

    Joined:
    Oct 2, 2014
    Posts:
    102
    I can't build webgl in 2019.1 - alpha 12 - it freezes on build

    I need to know - is it work? Is it help with background (un-focused tab) download and initialize webgl, and not freezes for user focus (unity 5.6 and 2017 is freeze before final init, and wait for user focus tab)
     
  4. traden

    traden

    Joined:
    Sep 16, 2009
    Posts:
    59
    I managed to get the mime type error fixed (running Express server). But I'm still getting magic word errors. CompileError: AsyncCompile: Wasm decoding failed: expected magic word.

    This is happening now when I just do a standard ctrl+b local build now as well with WebAssembly streaming. As Alex stated, basic local builds are no longer functional.
     
  5. Marco-Trivellato

    Marco-Trivellato

    Unity Technologies

    Joined:
    Jul 9, 2013
    Posts:
    1,650
    This is a known issue that we aim to fix before releasing 2019.1. Here is the issue tracker.
     
  6. Marco-Trivellato

    Marco-Trivellato

    Unity Technologies

    Joined:
    Jul 9, 2013
    Posts:
    1,650
    This is not a known issue. Unity 2019.1.0a13 is out now, would you mind trying with that?

    If you are referring to the freeze at the very end of loading, it is not fixed by wasm streaming.
     
  7. AlexHell

    AlexHell

    Joined:
    Oct 2, 2014
    Posts:
    102
    I mean 5 sec freeze (in Firefox.. GIF animation is freezed too) (or 10 sec in chrome) at the end, after preInit and till onRuntimeInitialized on Module object (I have write logs and checked this at unity 5.6.)
     
  8. bererton

    bererton

    Joined:
    Jun 20, 2014
    Posts:
    34
    Hi @Marco-Trivellato , We're about to launch a webgl game, and we more or less successfully upgraded to Unity2019.1b5.

    We are having similar problems with getting data streaming to work. I was able to get a local Apache server up and running and with your config it *does* work. I got it to do the brotli compression on both .unityweb files and have the streaming compilation succeed.

    The issue I have is when I want to put it up on our CDN I'm not sure exactly what's breaking where. It's the exact same files as on our local server which work, but break on the CDN. I get the error: wasm streaming compile failed: CompileError: AsyncCompile: Wasm decoding failed: expected magic word 00 61 73 6d, found 6b 8d 00 55 @+0

    However, looking at the headers it seems that the .wasm file does indeed have
    content-encoding: br set. Where I think it's going wrong is that the .wasm file then also assumes that the .unityweb files are also brotli encoded, which they are not (because I haven't convinced the CDN to do it yet ;-). Would that be a possible explanation of the Wasm decode failure?
    (Link to the example that is failing is https://arxarcana.io/streamingtest/index.html)

    Is there a way to check that the wasm file is correctly brotli compressed? Should I write some custom javascript to compare the original file with the one that's downloaded for example?

    Thanks for any help!
     
    Last edited: Mar 7, 2019
  9. bererton

    bererton

    Joined:
    Jun 20, 2014
    Posts:
    34
    A note from the future. I have made progress on this. Here are some notes:
    • The wasm decoding failed expected magic word stuff. All *decompressed* streaming wasm files start with the bytes 00 61 73 6d.
    • The unity build (as of Unity 2019.1.0b5) with either gzip or brotli enabled *compresses* the wasm file. Some CDNs when they see that the content type is application/wasm will compress the wasm file a *second time*, the browser gets it and decompresses it only once. This will lead to it neither doing streaming compilation nor actually working at all.
    • The line that Marco put into his apache config file: AddOutputFilterByType DEFLATE application/wasm will deflate a compressed wasm build as it is delivered to the client over the wire. In our case that means that apache was sending 26megs of uncompressed wasm instead of 7 megs of compressed wasm which may not be what you want in a production environment.
    • The most success I've had is to create an uncompressed build, and have the CDN compress the wasm file for you. With this approach I was indeed able to shave anywhere from 1 second to 6 seconds off our WebGL build. I now need to figure out how to tell our CDN to *also* compressed the uncompressed .unityweb files coming out of an uncompressed unity build, so more time might yet be saved.
    If someone else has more success or thinks my numbers should be better let me know!
     
  10. bigbrainz

    bigbrainz

    Joined:
    Jul 21, 2015
    Posts:
    84
    We are still getting the "unsupported MIME type" and "magic number" errors in 2019.1.0b5. I'm going to request that our Hosting company adjust our server to to handle the Mime issue, but:

    1. Is it ok/possible to set our Apache server to handle both Brotli AND gzip? We use Brotli for our public build, but gzip for our dev builds and rapid prototyping? I know we tried setting it up to use Brotli before and it created a lot of problems so I'm more cautious this time.
    2. Is the magic number bug nearly fixed?
    3. I'm also curious why we have to set the mime to use WebAssembly when we're already using WebAssembly just fine. Is this just for the Streaming aspect?

    I'm very eager to see what kind of increases this might get us for our WebGL builds.
     
  11. jukka_j

    jukka_j

    Unity Technologies

    Joined:
    May 4, 2018
    Posts:
    10
    Checking out https://arxarcana.io/streamingtest/index.html , it looks like everything is in order there: the file https://arxarcana.io/streamingtest/Build/3478eaf2eb1bb591f21fd75b001d480e.wasm is served as brotli-compressed (its compressed network transfer size is 5.6MB, compressed bytes start with 6B 8D 00 55).

    The file is served with Content-Encoding: br, and Content-Type: application/wasm as is proper here. This should trigger the browser to decompress the file upon receiving it, but it looks like it doesn't.

    One thing to try, can you rename the file suffix from .wasm to .wasm.br? (and double-check that Content-Encoding: br, and Content-Type: application/wasm are still being passed), and edit the loader JS files to download the .wasm.br version instead of the .wasm file. I wonder if browsers require that Brotli files are served with file suffix .br in order to trigger the decompression at browser end.

    Decompressing the downloaded wasm brotli file locally with 'brotli' decompressor works out ok, although the command-line decompressor requires that the file ends with suffix .br, otherwise it refuses to process it. Decompressed .wasm file is 27.9MB, and starts with bytes 00 61 73 6D.

    It should be possible, but will need some manual work on the build output that Unity currently produces. The general scheme is to place two files to CDN: a.wasm.gz and a.wasm.br, and have browser JS download file "a.wasm", and depending on what client browser advertises under "Accept-Encoding:" when it is doing the request, the web server should be configured to redirect to serve a.wasm.br instead if "Accept-Encoding: br" had been specified, or a.wasm.gz if "Accept-Encoding: gzip" had been specified.

    Yes, the MIME application/wasm is only needed to tell the browser that the incoming data is wasm, so that it knows to start streaming compilation early. If MIME type is wrong, browser will then not know to do streaming compilation, but will have to wait to get the whole file to compile.
     
  12. bererton

    bererton

    Joined:
    Jun 20, 2014
    Posts:
    34
    Hi, I just saw this reply now, thanks @jukka_j . I should have mentioned that I was able to get this successfully working on our CDN (Cloudflare). In this case I created an *uncompressed* wasm build, and then Cloudflare automatically compresses it. It's not 100% ideal since it doesn't compress it as much as the compression done by Unity as part of the build, but it does work. I was also able to get Cloudflare to compress the .webl files by settings their content type to image-svg which Cloudflare then decides to compress. That got it working well enough. I have heard that people were able to get brotli compressed builds + web streaming working, but it *may* require that you be able to set headers directly using for example Nginx as your source. I was using Amazon S3 as the source for Cloudflare CDN.

    But long story short I did eventually get it to work. I think this problem is more a webhost/CDN issue than a Unity Issue, but it would be great to get a series of solid debugging steps for people trying to figure this out for the first time. Hopefully they'll find this thread.