Search Unity

Changes to the WebGL loader and templates introduced in Unity 2020.1

Discussion in 'WebGL' started by alexsuvorov, Jan 28, 2020.

  1. alexsuvorov

    alexsuvorov

    Unity Technologies

    Joined:
    Nov 15, 2015
    Posts:
    323
    The following changes have been introduced to the WebGL loader and templates in Unity 2020.1:
    • Unity now generates a build-specific WebGL loader, which is stripped and optimized based on the selected player settings.
    • WebGL templates can now use conditional directives and JavaScript macros.
    • Build configuration is no longer stored in an external JSON file, but is embedded directly in the html.
    • The WebGL instantiation function now uses the target canvas element as an argument, which gives the developers full control of the page layout.
    • Unity 2020.1 supports server-friendly naming schemes for the build files.

    Build-specific loader

    In order to improve the loading performance, Unity 2020.1 will now generate a WebGL loader which is specific to the build. Unused code will be stripped from the loader, and the most efficient loading scheme will be chosen, based on the selected player settings.

    Unity versions prior to 2020.1 generated a universal, build independent WebGL loader, which could be used to load other WebGL builds, created with different Unity versions.This approach simplified the embedding of WebGL builds in an html document, and was also very useful for embedding multiple builds, created with different Unity versions, on the same page.

    However, using the universal loader to load just one specific build usually led to significant performance overhead. Firstly, the universal loader contains a lot of code which is never executed when loading a specific build (for example, it contains both gzip and brotli decompression fallbacks, although those compression methods are rarely used together). Secondly, the universal loader lacks optimizations which could be introduced for a build with specific configuration.

    The changes implemented in the updated WebGL loader are intended to alleviate these problems. For comparison, the size of the minified loader in Unity 2019.3 is around 155 KB, while the loader in Unity 2020.1 can be stripped down to just 9 KB.

    Template variables, macros and conditional directives

    In previous Unity versions it was only possible to use %VARIABLE% tags in the html code, which were replaced with corresponding values when the template was preprocessed.

    In Unity 2020.1 it is now possible to use JavaScript macros and conditional directives inside the template files. This allows you to perform advanced preprocessing of the source template files, based on the selected player settings.

    Unity 2020.1 automatically preprocesses all the *.html, *.php, *.css, *.js and *.json files located in the template folder.

    The following preprocessor variables can be used in JavaScript macros and conditional directives:

    • COMPANY_NAME - the value of the Company Name field
    • PRODUCT_NAME - the value of the Product Name field
    • PRODUCT_VERSION - the value of the Version field
    • WIDTH - the value of the Default Canvas Width field
    • HEIGHT - the value of the Default Canvas Height field
    • SPLASH_SCREEN_STYLE - is set to "Dark" or "Light", depending on the selected Splash Style
    • BACKGROUND_COLOR - background color in a form of a hex triplet
    • UNITY_VERSION - current Unity version
    • DEVELOPMENT_PLAYER - is set to true if the Development Build option is enabled
    • DECOMPRESSION_FALLBACK - is set to "Gzip", "Brotli" or an empty string, based on the selected decompression fallback
    • TOTAL_MEMORY - the initial size of the memory heap in bytes
    • USE_WASM - is set to true if the current build is a WebAssembly build
    • USE_THREADS - is set to true if the current build uses threads
    • USE_WEBGL_1_0 - is set to true if the current build supports WebGL1.0 graphics API
    • USE_WEBGL_2_0 - is set to true if the current build supports WebGL2.0 graphics API
    • USE_DATA_CACHING - is set to true if the current build uses indexedDB caching
    • LOADER_FILENAME - the filename of the build loader script
    • DATA_FILENAME - the filename of the main data file
    • FRAMEWORK_FILENAME - the filename of the build framework script
    • CODE_FILENAME - the filename of the WebAssembly module
    • MEMORY_FILENAME - the filename of the memory file
    • SYMBOLS_FILENAME - the filename of the JSON file containing debug symbols
    • BACKGROUND_FILENAME - the filename of the background image

    JavaScript macros

    JavaScript macros are blocks of JavaScript code, included in the template files, which are surrounded by triple curly brackets. When these code blocks are found in preprocessed template files the preprocessor evaluates them and replaces the macros with the result of the code evaluation. Evaluated JavaScript code can use internal preprocessor variables which are assigned at build time according to the values supplied by the Editor.

    As an example, see the following line from the index.html file used in the Default template:

    Code (HTML):
    1. <div id="unity-build-title">{{{ PRODUCT_NAME }}}</div>
    If the value of the Product Name in the Player Settings is set to "My WebGL Game", then the internal preprocessor variable PRODUCT_NAME will be also set to "My WebGL Game" value, and in the output index.html file the mentioned line will be transformed into:

    Code (HTML):
    1. <div id="unity-build-title">My WebGL Game</div>
    Now let’s consider a more complex example from the same index.html template file:

    Code (JavaScript):
    1. canvas.style.background = "url('" + buildUrl + "/{{{ BACKGROUND_FILENAME.replace(/'/g, '%27') }}}') center / cover";
    If the target build folder is called "Let’s try WebGL", and a background image is provided in the Player Settings, then the internal preprocessor variable BACKGROUND_FILENAME will be set to "Let’s try WebGL.jpg" value, and in the output index.html file the mentioned line will be transformed into:

    Code (JavaScript):
    1. canvas.style.background = "url('" + buildUrl + "/Let%27s try WebGL.jpg') center / cover";
    JavaScript macros can be used to preprocess the values supplied by the Editor (that is, escape single quotes as shown in the above example). JavaScript macros are not limited in complexity, they can include multiple operators, loops, functions, and any other JavaScript constructs.

    Conditional directives

    In Unity 2020.1 it is now also possible to use conditional #if, #else, #endif directives in template files.

    Example of a conditional group:

    Code (JavaScript):
    1. #if EXPRESSION
    2.   // this block will be included in the output if EXPRESSION has truthy value
    3. #else
    4.   // this block will be included in the output otherwise
    5. #endif
    JavaScript expressions evaluated in conditional directives are not limited in complexity, they can include brackets, logical operators and other JavaScript constructs. Conditional directives can be nested.

    This is an example conditional directive from the index.html file used in the Default template:

    Code (JavaScript):
    1. #if SYMBOLS_FILENAME
    2.         symbolsUrl: buildUrl + "/{{{ SYMBOLS_FILENAME }}}",
    3. #endif
    If the Debug Symbols option is enabled, the Development Build option is disabled, the target build folder is called "Let’s try WebGL" and Compression Format is set to Disabled, then the internal preprocessor variable SYMBOLS_FILENAME will be set to "Let's try WebGL.symbols.json", and in the output index.html file the mentioned block will be transformed into:

    Code (JavaScript):
    1.         symbolsUrl: buildUrl + "/Let's try WebGL.symbols.json",
    If Debug Symbols option is disabled or Development Build option is enabled, then the internal preprocessor variable SYMBOLS_FILENAME will be set to an empty string, and the mentioned block will be completely stripped from the output index.html.

    Custom user variables

    In previous Unity versions it was necessary to use the "UNITY_CUSTOM_" prefix to mark a custom user template variable. This is no longer necessary in Unity 2020.1, as the template parser automatically finds all the user variables.

    When a WebGL template is selected in the Player Settings window, Unity parses the template files looking for JavaScript macros and conditional directives. JavaScript variables used in macros and conditional directive expressions, which are not declared in the template code and are not internal preprocessor variables, are treated as a custom user variables and are automatically added as editable fields to the Player Settings.

    For example, if you would like to control the title of the generated index.html page directly from the Player Settings you can do this by modifying the <title> line of the index.html in your custom template as shown:

    Code (HTML):
    1. <title>{{{ PAGE_TITLE }}}</title>
    Now, if you reopen the Player Settings window and reselect the template, the template will be re-parsed, a new PAGE_TITLE variable will be found and added as an editable field directly to the Player Settings.

    Instantiation of the build

    In Unity 2020.1, the following important changes have been introduced for the instantiation function:
    • The instantiation function now directly accepts a <canvas> element as an argument. This gives the developers full control of the page layout.
    • The build configuration object is no longer stored in an external JSON file, but is embedded directly in the html code, and is used as an argument for the instantiation function. This increases the size of the JavaScript code which is added to the embedding html, but improves the loading performance, because it eliminates the need to perform an additional server request.
    • Instantiation function now returns a Promise object.
    The new instantiation function can be used in the following way:

    Code (JavaScript):
    1. createUnityInstance(canvas, config, onProgress).then(onSuccess).catch(onError);
    where:
    • canvas is a <canvas> element which will be used to render the game.
    • config is an object which contains build configuration, specifically, code and data urls as well as product and company name and version. Configuration object can be defined using the following code:
      Code (JavaScript):
      1. var buildUrl = "Build";
      2. var config = {
      3.   dataUrl: buildUrl + "/{{{ DATA_FILENAME }}}",
      4.   frameworkUrl: buildUrl + "/{{{ FRAMEWORK_FILENAME }}}",
      5.   codeUrl: buildUrl + "/{{{ CODE_FILENAME }}}",
      6. #if MEMORY_FILENAME
      7.   memoryUrl: buildUrl + "/{{{ MEMORY_FILENAME }}}",
      8. #endif
      9. #if SYMBOLS_FILENAME
      10.   symbolsUrl: buildUrl + "/{{{ SYMBOLS_FILENAME }}}",
      11. #endif
      12.   streamingAssetsUrl: "StreamingAssets",
      13.   companyName: "{{{ COMPANY_NAME }}}",
      14.   productName: "{{{ PRODUCT_NAME }}}",
      15.   productVersion: "{{{ PRODUCT_VERSION }}}",
      16. };
    • onProgress(progress) callback is called every time when the download progress updates. It is provided with the progress argument, which determines the loading progress as a value from 0.0 to 1.0.
    • onSuccess(unityInstance) callback is called after the build has been successfully instantiated. The created Unity instance object is provided as an argument. This object can be used for interaction with the build.
    • onError(message) callback is called if an error occurs during build instantiation. An error message is provided as an argument.

    Decompression fallback and file naming schemes in Unity 2020.1

    The newly introduced Decompression Fallback option, which can be found under the Publishing Settings, has a significant impact on the loading performance. When the Decompression Fallback option is enabled, the build files will have a .unityweb extension, and the loader will include a JavaScript decompressor for the selected compression method. The JavaScript decompressor will automatically be used in cases where the downloaded content fails to be decompressed natively by the browser. This option can be useful if you don't have access to the server configuration or are not able to append specific http headers to the server response. However, using this option will also result in increased size of the loader and inefficient loading scheme for the build files (for example, WebAssembly streaming can not be used together with decompression fallback).

    When the Decompression Fallback option is disabled (which is the default), the build files will have an extension corresponding to the selected compression method (i.e. .gz or .br). Additionally, the loader will try to use WebAssembly streaming by default. When hosting such a build on a server, the following http headers should be added to the server responses in order to make the build load correctly:
    • .gz files should be served with a Content-Encoding: gzip response header.
    • .br files should be served with a Content-Encoding: br response header.
    • .wasm, .wasm.gz or .wasm.br files should be served with a Content-Type: application/wasm response header.
    • .js, .js.gz or .js.br files should be served with a Content-Type: application/javascript response header.
    Note that compressed WebGL builds which don't include decompression fallback can not be loaded from the file:// urls due to the lack of the necessary response headers. In order to run such a build locally, you can use the Build and Run option or a local web server with properly setup response headers.
     
  2. De-Panther

    De-Panther

    Joined:
    Dec 27, 2009
    Posts:
    306
    The template variables, macros and conditional directives looks like a great solution.

    Does the WebGLTemplates folder still must be only in the Assets folder, or can we have templates in subfolders, similar to the Editor and Plugins folders?
     
  3. alexsuvorov

    alexsuvorov

    Unity Technologies

    Joined:
    Nov 15, 2015
    Posts:
    323
    Yes, currently WebGLTemplates folder should be located directly under the Assets folder. However, I agree, in some cases it probably makes sense to have templates in a different place. For example, if a developer wants to store platform-related content in separate subfolders, or distribute a template as a part of a package. This possibility might be considered in the future.
     
  4. De-Panther

    De-Panther

    Joined:
    Dec 27, 2009
    Posts:
    306
    Thanks, I opened a bug(1157275) about it a few months ago, and got an email a few weeks ago that it won't be adressed. I hoped that it would be resolved with such a huge update in the way of how templates works
     
  5. kognito1

    kognito1

    Joined:
    Apr 7, 2015
    Posts:
    331
    By any chance is there a way to make the optional decompression fallback feature a runtime decision? It's not a big deal if not, but currently we have a similar setup for wasm streaming. We host the vast majority of our simulations, but we do have a few custom ones. Unfortunately some of those are on servers where getting approval to modify headers is...quite the undertaking. :) At the moment we build with wasm streaming enabled and then automatically duplicate the [build_name].wasm file and rename it to [build_name].unityweb. Then we have a simple wasm flag in our config file that tells us which file to load at runtime ([build_name].wasm for wasm streaming, [build_name].unityweb for normal).

    I'm very interested in the load time improvements when disabling decompression fallback as this is our use case 95% of the time, but losing the flexibility to disable wasm streaming (and now fallback decompression) in a config file would hurt a little. I suspect though worst case is we'll just have to make two builds (fallback off and on) of every release and switch between them at runtime.
     
  6. alexsuvorov

    alexsuvorov

    Unity Technologies

    Joined:
    Nov 15, 2015
    Posts:
    323
    This situation is rather unusual, because normally capabilities of the hosting server are known in advance. If you create a build before you make decision about the hosting server, then you actually need a universal, unstripped loader, which can cover all possible hosting scenarios.

    Basically, both native browser decompression and WebAssembly streaming can only be used if you are able to modify server response headers, therefore there is no reason to configure these settings separately. If you are able to modify server response headers to properly serve compressed files, then you should also be able to provide response headers, which are necessary for WebAssembly streaming. And vice versa, if you don't have access to the server headers, then WebAssembly streaming won't work, and you will also have to include decompression fallback.

    In your specific case, the loading scenario depends on the capabilities of the server, and not on the capabilities of the client, therefore I see no reason here to make any decision at runtime (maybe I'm missing something). You can indeed just make two builds and upload the appropriate one depending on the hosting server capabilities. This way, each server will serve the type of the loader which is appropriate for that server. By choosing which server to load the build from, you will automatically choose the appropriate loading scheme.

    It might be convenient to create multiple build variations using, for example, the following /Assets/Editor/build.cs script:
    Code (CSharp):
    1. using UnityEditor;
    2. using UnityEngine;
    3. using System.IO;
    4.  
    5. public class BuildPlayerExample : MonoBehaviour
    6. {
    7.     [MenuItem("Build/Build WebGL with and without Decompression Fallback")]
    8.     public static void BuildWebGL()
    9.     {
    10.         BuildPlayerOptions buildPlayerOptions = new BuildPlayerOptions();
    11.         buildPlayerOptions.scenes = new[] { "Assets/Scenes/SampleScene.unity" };
    12.         buildPlayerOptions.target = BuildTarget.WebGL;
    13.         buildPlayerOptions.options = BuildOptions.None;
    14.         buildPlayerOptions.locationPathName = "MyBuild";
    15.      
    16.         PlayerSettings.WebGL.decompressionFallback = true;
    17.         BuildPipeline.BuildPlayer(buildPlayerOptions);
    18.         Directory.Move(buildPlayerOptions.locationPathName, buildPlayerOptions.locationPathName + "_with_decompression_fallback");
    19.      
    20.         PlayerSettings.WebGL.decompressionFallback = false;
    21.         BuildPipeline.BuildPlayer(buildPlayerOptions);
    22.         Directory.Move(buildPlayerOptions.locationPathName, buildPlayerOptions.locationPathName + "_without_decompression_fallback");
    23.     }
    24. }
     
  7. kognito1

    kognito1

    Joined:
    Apr 7, 2015
    Posts:
    331
    I agree our use case is unusual in that we do not know what the server configuration will be when building. That's why I understand if it's not in the cards. :D But you can think of our use case in this way:

    We sell product A. Client X and Y might be fine with us hosting product A for them, but client Z might (often for security reasons) want to keep it within their network. In that circumstance we give them the (already built) files to rehost product A on their network. Unfortunately sometimes they are unwilling to modify headers for us, which as you correctly point out breaks wasm streaming. So it's really convenient for us just to have a simple "disable wasm streaming" flag that we can set when giving our product to a 3rd party.

    You're not really "missing anything" other than I don't want to make more builds. :p We already do something similar to the code you provided; we make three builds (no msaa for mobile, msaa for desktop, development) for every beta/release. So we just need to make 2 more now (no msaa-no fallback, no msaa-fallback, msaa-no fallback, msaa-fallback, development-fallback). Not the end of the world obviously, but will probably add 25-30 minutes to our build times. :(
     
  8. alexsuvorov

    alexsuvorov

    Unity Technologies

    Joined:
    Nov 15, 2015
    Posts:
    323
    If I understand correctly, in this specific case you would like to generate some intermediate version of the loader, which supports both native and JavaScript decompression, but at the same time has all other unnecessary functionality stripped out. Such scenario is currently not supported by default, in fact, there will always be some special corner cases which might force developers to adjust the loader code. Previously, developers tried to patch the generated WebGL loader in a post-build step, in order to make it compatible with their specific hosting environment. This is no longer necessary in Unity 2020.1, because you are now able to use your own WebGL loader as a part of the WebGL template.

    The default WebGL loader is preprocessed using the same exact scheme as all other files in the WebGL template. The behaviour of the loader can be customised using the configuration object passed to the instantiation function, which should be sufficient for most use cases. However, if you are considering some unusual loading scenario, you can fully customise the loader for your specific needs in the following way.

    1) Create a custom WebGL template by copying the contents of the
    <Unity Installation>/PlaybackEngines/WebGLSupport/BuildTools/WebGLTemplates/Default/
    folder into the
    <Your Project>/Assets/WebGLTemplates/MyTemplate/
    folder and select the newly added "MyTemplate" template under Player Settings > Settings for WebGL > Resolution and Presentation > WebGL Template.

    2) Copy the file
    <Unity Installation>/PlaybackEngines/WebGLSupport/BuildTools/UnityLoader/UnityLoader.js
    into your custom template as
    <Your Project>/Assets/WebGLTemplates/MyTemplate/MyLoader.js
    Now you have your own custom WebGL loader as a part of your custom template. The only complication here is that the loader embeds some code from external files using read() function provided by the preprocessor. Preprocessor resolves relative path of the embedded file using the containing folder of the embedding script. Considering that we only copied the "UnityLoader.js" script into our template, but not other supporting files, preprocessor will now fail to find embedded files when preprocessing the custom loader. This can be resolved in the following ways:
    • Recommended way:
      You can precede the agruments of the read() function with "UnityLoader/" subfolder to make them relative to the preprocessor script folder, specifically, in the "MyLoader.js" change:
      read("XMLHttpRequest.js") to read("UnityLoader/XMLHttpRequest.js")
      read("Gzip.js") to read("UnityLoader/Gzip.js")
      read("Brotli.js") to read("UnityLoader/Brotli.js")

    • Hacky way:
      Using the fact that preprocessor needs to evaluate conditional directives before processing the contents of the file, we can override the read() function directly from the evaluated expression of a dummy conditional directive at the very top of the "MyLoader.js" script:
      Code (JavaScript):
      1. #if read = ((read, path) => read("UnityLoader/" + path)).bind(null, read)
      2. #endif // this approach is not recommended, use for testing purposes only

    3)
    Now we just need to load the custom loader instead of the default one. To achieve this, simply replace the line
    Code (JavaScript):
    1. var loaderUrl = buildUrl + "/{{{ LOADER_FILENAME }}}";
    in the "/Assets/WebGLTemplates/MyTemplate/index.html" with
    Code (JavaScript):
    1. var loaderUrl = "MyLoader.js";
    Note: Currently you can't put template files under "Build" subfolder, so your custom loader will be generated outside of the "Build" subfolder.

    If you now build and run your project, then the custom "MyLoader.js" loader will be used instead of the default one.

    Ideally, you could create a custom loader which properly handles all your special hosting scenarios, however, in your specific case it would probably be easier to just generate two separate loaders, with and without decompression fallback (for example, the custom loader can be generated with decompression fallback and the default loader without fallback). Means you should uncheck the Decompression Fallback option in the Player Settings to properly generate the default loader, and adjust the custom loader code to behave as if that option was enabled. This can be achieved in the following ways:
    • Recommended way:
      Go through the "MyLoader.js" code and manually resolve all the preprocessor expressions containing DECOMPRESSION_FALLBACK variable as if it had a value of "Gzip" or "Brotli" (depending on the fallback you are planning to use). The easiest way to achieve this would be to just auto-replace all the DECOMPRESSION_FALLBACK occurrences with "Gzip" or "Brotli".

    • Hacky way:
      You can override the value of the DECOMPRESSION_FALLBACK preprocessor variable using the following dummy conditional directive at the very top of the "MyLoader.js" script:
      Code (JavaScript):
      1. #if DECOMPRESSION_FALLBACK = "Gzip"
      2. #endif // this approach is not recommended, use for testing purposes only
    If you now disable the Decompression Fallback option and build your project, then the custom MyLoader.js loader will be generated with decompression fallback and the default loader under the "Build" subfolder will be generated without fallback. This way you only need to build the project once to generate both variations of the loader. You would also need to perform a runtime check in the index.html to decide which loader should be used in each specific case.
     
    Last edited: Feb 10, 2020
    kognito1 likes this.
  9. hsallander

    hsallander

    Joined:
    Dec 19, 2013
    Posts:
    36
    This all seem very promising @alexsuvorov thanks for this info!

    A bit off topic but still related to WebGL templates: is there any chance we might see support for using custom WebGL templates when building with Unity Cloud Build in the future? Currently we're doing all our production builds for our WebGL project in UCB (and then hosting it on our own servers of course), but we've hacked together our own solution to get our custom template to be used and it would be nice to see official support for custom templates in UCB instead.
     
    andrzej_cadp and De-Panther like this.
  10. kognito1

    kognito1

    Joined:
    Apr 7, 2015
    Posts:
    331
    Sorry for the delayed response, but thank you @alexsuvorov for the solution! Yes this will work for us perfectly. Thanks again for the work and detailed explanation!
     
  11. Johannski

    Johannski

    Joined:
    Jan 25, 2014
    Posts:
    639
    Hi @alexsuvorov
    Sadly I can't quite get the apache configuration right for the newly introduced .br/.gz file endings. What I got so far, is that with Unity 2020.1 the Content Security Policy of worker-src now needs to include blob:.
    Here is what I got so far:
    Code (CSharp):
    1. Header set Content-Security-Policy "worker-src 'self' blob:;"
    2.  
    3. <IfModule mod_mime.c>
    4.   AddEncoding x-gzip .gz
    5.   AddEncoding x-brotli .br
    6.  
    7.   AddEncoding br .unityweb
    8.   AddEncoding gzip .unityweb
    9.  
    10.   AddType text/x-javascript .js .js.br .js.gz
    11.   AddType application/wasm .wasm .wasm.br .wasm.gz
    12.   AddEncoding br .wasm
    13.   AddEncoding gzip .wasm
    14.   AddOutputFilterByType DEFLATE application/wasm
    15. </IfModule>
    With that I get the following error:
    Demo

    Can you point me in the right direction of how to handle the new file naming?
     
unityunity