Search Unity

WebGL build size for empty scene is huge. Optimisation tips and tricks?

Discussion in 'Unity 5 Pre-order Beta' started by sluice, Oct 29, 2014.

  1. sluice

    sluice

    Joined:
    Jan 31, 2014
    Posts:
    416
    Hello all,

    I created a new project, saved the empty scene and made a build with these settings:
    • Optimisation Level: Fastest (very slow builds)
    • Development mode: unchecked
    • Player Settings / Other settings / Stripping Level: Strip ByteCode
    The size of the output folder is a 14Mb.
    Adding a simple cube to the scene and the build size is up to a whooping 22.7Mb.

    Compared to an empty WebPlayer build(~28Kb), the difference is huge. I understand that part of the engine is included in the WebPlayer plugin. Understandably, a part of the engine has to be included in every WebGL build.

    Unfortunately in my real world scenario, I have to keep my file size under 10Mb. Therefore, for my previous WebPlayer and Flash builds, all I did was splitting my scenes as asset bundle and it did the trick.

    Here is the size of the main hub of the project I'm trying to convert:
    • WebPlayer: 10.7Mb
    • Flash: 11.3MB
    • WebGL: 75.2Mb
    I really have to lower that 75.2Mb as much as possible...


    Questions:
    1. How can a primitive cube increase the output size by 8.7MB?
    2. Are there anything else I can do to reduced the overhead size of a WebGL build?
     
  2. RalphH

    RalphH

    Unity Technologies

    Joined:
    Dec 22, 2011
    Posts:
    591
    Hi.

    I think your measuring of size is kind off of; I did the same thing and I can see how you got to those numbers, but they are "all inclusive".

    In a deployment folder, you'll find a index file, a .htaccess file and 2 folders.
    Your measurement is a measurement of all those folders, but that is never what will go over the wire; instead, depending on how your webserver is configured (which we try to help with by providing the .htaccess file), you should measure the index file and the compressed folder.

    That will give you this number, for an empty scene with a cube.
    3.98 MB (4,181,830 bytes)

    Flash already does this compression (actually it does better compression through lzma), and the unity webplayer has compression for the .unity3d files.

    Now, the uncompressed folder can still be served up through your webserver, but using the .htaccess file, the server will gzip for you. The compressed folder is all the files pre-compressed, so the server does not have to do it upon request.

    Regards,
    Ralph.
     
    WendelinReich and sluice like this.
  3. sluice

    sluice

    Joined:
    Jan 31, 2014
    Posts:
    416
    @RalphH, this is great! Getting close to my 10MB limit is all the more possible now!

    On the server we want the converted project to run, we have a problem of MIME Type. Basically the .mem and the .data aren't seen on the server. Until, we are able to add these MIME type (it's a long and complicated process, since are not in complete control of the server), is there a way to change that extension to say .txt or anything else that is seen by the server?

    Wondering if there is a setting or a file somewhere that can be configure. Looking at the principal JS file from the build, I can easily see where the .mem file is being use. Except changing it there, won't affect the change in the Compressed folder.
     
  4. jonas-echterhoff

    jonas-echterhoff

    Unity Technologies

    Joined:
    Aug 18, 2005
    Posts:
    1,600
    The compressed folder is the exact same files, just gzip compressed. The .htaccess file will then tell apache or compatible web servers to serve those when a client requests the files without the gz extension. So you should be able to edit that to match. The question is whether your server will allow that level of setup at all, if you can't even get it to server files with arbitrary extensions. (In that case, maybe you should consider changing your hosting setup).
     
  5. sluice

    sluice

    Joined:
    Jan 31, 2014
    Posts:
    416
    The server should allow that level of setup. It's just not our server, but the client's. For security and obvious reason, we do not have access. It's just a painfully long process.

    I need to figure out the MIME type for the .mem and the .data as these are not standards.
    What are they? text/plain? application/x-javascript? Something else?
     
    Last edited: Oct 30, 2014
  6. lami

    lami

    Joined:
    Jun 17, 2013
    Posts:
    17
    *push*
    We have the same problem and it would be great to know the MIME types of the .mem and .data files!
    @sluice : in the mean time we just set attribute compression="force" to our <Connector/> (using Apache Tomcat 7)
    Maybe this is suitable for you too.
     
  7. sluice

    sluice

    Joined:
    Jan 31, 2014
    Posts:
    416
    @lami
    On my local machine (OSX), I configured Apache, and at first it didn't work. Obviously I had to use .htaccess that comes when you do a WebGL build. Now it works (on my machine).

    My issue is on the clients server, it's using IIS (and I won't be able to a web.config file)...

    But one thing for sure, I really need to figure out those MIME types for the .mem and .data files otherwise the WebGL platform is useless to me.

    Hopefully we get an answer soon.
     
  8. Ostwind

    Ostwind

    Joined:
    Mar 22, 2011
    Posts:
    2,772
    you can use application/octet-stream as both are binary files
     
    Dustin-Horne, lami and sluice like this.
  9. sluice

    sluice

    Joined:
    Jan 31, 2014
    Posts:
    416
    @Ostwind
    Just confirming that using application/octet-stream for both .mem and .data file works!
     
    lami likes this.
  10. unisip

    unisip

    Joined:
    Sep 15, 2010
    Posts:
    269
    Can anyone point me to tutorials on how to configure WAMP to serve Unity WebGL compressed files?

    I know close to nothing about Apache, but I installed WampServer 2.5 on my Windows 7 PC so that I can test WebGL exports locally and not have to upload those huge files to a server every time I make a build. WAMP works ok as is and I can load my uncompressed WebGL projects, but out of curiosity (and also to learn what to tell to my clients when they deploy Unity WebGL apps), I'd like to know what to change in the Apache config to support those compressed files
     
  11. Ostwind

    Ostwind

    Joined:
    Mar 22, 2011
    Posts:
    2,772
    you can google: wampserver gzip

    or other thousands of apache tutorials around the net for case specific instructions
     
  12. tomekkie2

    tomekkie2

    Joined:
    Jul 6, 2012
    Posts:
    481
    I made a comparison of webplayer and webgl builds made from the same project.
    Basically looks like the webgl loads 1,5 longer, as the file's weight is 1,5 bigger.
    But as a reward the webgl loads 3 times faster once the files get cached on the machine. (in Firefox on my desktop computer)
    Here it is in my blog post:
    http://virtualplayground.d2.pl/?p=810
     
    Last edited: Jan 13, 2015
    sluice likes this.
  13. sluice

    sluice

    Joined:
    Jan 31, 2014
    Posts:
    416
  14. diggerjohn

    diggerjohn

    Joined:
    Apr 28, 2009
    Posts:
    26
    Wow ... that is all so cool.
    I am a long time Unity developer but totally new to all this server side stuff. Have always wanted
    to know. And now with gl it looks as though I finally need to.
    Is there a tutorial or other source that someone could point me to that would give me some sort of
    practical overview? A general, not necessarily basic, how too? Something to get me started?
    Thanks .....
     
  15. EvilEliot17

    EvilEliot17

    Joined:
    Apr 12, 2014
    Posts:
    9
    Hi, good to know this i make an empty scene with a cube, i was trying to configure my server on Amazon to serve compressed files, i manage to activate the mod_deflate on my apache server, i use this tool http://www.feedthebot.com/tools/gzip/ to be sure i'm serving gzip files and it says "gzip working compressing 60%".

    but apparently only works on .js (as you can see on the image) but the files .data an .mem files keep serving with full size even i add the application/octet-stream, i´m doing something wrong but im not very good at servers, maibe you can give me an advice at what i´m missing. i leave the code i add to the httpd.conf to activate gzip.



    httpd.conf code

    Code (CSharp):
    1. # mod_deflate configuration
    2. <IfModule mod_deflate.c>
    3. # Restrict compression to these MIME types
    4. AddOutputFilterByType DEFLATE text/plain
    5. AddOutputFilterByType DEFLATE text/html
    6. AddOutputFilterByType DEFLATE application/xhtml+xml
    7. AddOutputFilterByType DEFLATE text/xml
    8. AddOutputFilterByType DEFLATE application/xml
    9. AddOutputFilterByType DEFLATE application/x-javascript
    10. AddOutputFilterByType DEFLATE application/octet-stream
    11. AddOutputFilterByType DEFLATE text/javascript
    12. AddOutputFilterByType DEFLATE text/css
    13. # Level of compression (Highest 9 - Lowest 1)
    14. DeflateCompressionLevel 9
    15. # Netscape 4.x has some problems.
    16. BrowserMatch ^Mozilla/4 gzip-only-text/html
    17. # Netscape 4.06-4.08 have some more problems
    18. BrowserMatch ^Mozilla/4\.0[678] no-gzip
    19. # MSIE masquerades as Netscape, but it is fine
    20. BrowserMatch \bMSI[E] !no-gzip !gzip-only-text/html
    21. <IfModule mod_headers.c>
    22. # Make sure proxies don't deliver the wrong content
    23. Header append Vary User-Agent env=!dont-vary
    24. </IfModule>
    25. </IfModule>
    thanx
     
  16. PrefabEvolution

    PrefabEvolution

    Joined:
    Mar 27, 2014
    Posts:
    225
    After hours of trying to port our project to WebGL i can said: WebGL is veeeery young and unstable to run rich content. For example. After compilation we have 3.5mb stripped mono assemblies. DLL => IL2CPP => Emscripten => JS == 647mb(uncompressed) ~60 mb(compresed). And the problem is: some browsers just cant allocate > 600 mb of RAM to compile to ASM JS and execute. More over its just cant allocate more 512 mb to emulate unmanaged RAM for asmjs runtime... Its very very sad:(...
     
  17. RalphH

    RalphH

    Unity Technologies

    Joined:
    Dec 22, 2011
    Posts:
    591
    PrefabEvolution, that sounds extreme and likely a bug; did you build a development build ? Could you file a bugreport with us ? Preferrably just the entire project, but otherwise (if it's big) please file it after doing a build and zip up Temp/StagingArea/Data/Managed (in the project folder).
     
  18. Ryan.Alcock

    Ryan.Alcock

    Joined:
    Nov 2, 2014
    Posts:
    5
    I'm getting a similar problem trying to get a scene working. It's trying to allocate about 1.6GB of memory but it (obviously) fails. I'm not using a dev build and I've tried some different levels of stripping / optimisation levels to no avail.

    I'll file a bug report tomorrow when I'm at work again.