Search Unity

  1. Get the latest news, tutorials and offers directly to your inbox with our newsletters. Sign up now.
    Dismiss Notice

Unity IL2CPP Build Time Improvements - Seeking Feedback

Discussion in '2021.2 Alpha' started by joncham, Feb 24, 2021.

  1. joncham

    joncham

    Unity Technologies

    Joined:
    Dec 1, 2011
    Posts:
    265
    Hello Alpha Users!

    Multiple teams in Unity have been working on improving iteration time of player builds the past few releases. You should have noticed faster IL2CPP builds during the 2020 release cycle and even more improvements are coming in 2021.

    To this end, we have been working on a new option for IL2CPP that generates much less code (up to 50% less). This allows for both faster iteration times and smaller executable files. There *may* be a small runtime performance impact due to the different code generation.

    This graph shows improvements to compile times with this option enabled:



    We are asking that you try out this new option with your IL2CPP builds. We want to know if you see real world performance issues when running with this option. Also, let us know if the builds are faster and smaller.

    Note: This is available in alpha builds only, from 2021.2.0a5 onward.

    How to Enable: Set the environment variable “IL2CPP_ADDITIONAL_ARGS=--generics-option=EnableFullSharing” before launching Unity. This will run IL2CPP in the new code generation mode. Please be sure to close/kill any Unity and Unity Hub instances after setting this to be sure the environment variable change is picked up.

    Please respond with:
    1. Any difference in FPS or startup your applications see when running with this option enabled.
    2. The profile.json file in the generated IL2CPP C++ output folder (next to build output folder) for builds with and without this option. You can also send for cold and warm (incremental) builds. This allows us to see where build time is being spent and saved.
    3. You can also send stats on generated C++ output size and final binary size.
    Respond to this thread with any questions or findings. Additionally, you can DM me if you have information you don’t want to share in public.

    Thanks!
     
    Last edited: Feb 24, 2021
    Haneferd, mahdi_jeddi, mh114 and 9 others like this.
  2. fherbst

    fherbst

    Joined:
    Jun 24, 2012
    Posts:
    754
    Sounds interesting!
    I'm wondering a bit why you share detailed graphs about the build times but not about the possible performance regression. We're using a lot of generics so it would certainly be interesting which microbenchmarks show what slowdowns, and also how the above examples do in terms of performance. Mind sharing that?
     
    De-Panther, rz_0lento and Ruchir like this.
  3. joncham

    joncham

    Unity Technologies

    Joined:
    Dec 1, 2011
    Posts:
    265
    Hello,

    I shared the build times as the benefits of this option are clear. It is unclear what the negative impacts are for real projects. When running test projects we didn't see negative performance impact beyond ~10% of script time in a frame. That's why we are asking for more feedback, to have a larger sample size for finding regressions.

    Here is an example microbenchmark:

    Code (CSharp):
    1.     public class EqualityComparerEnum
    2.     {
    3.         private MyEnum _enum;
    4.         private EqualityComparer<MyEnum> _comparer;
    5.  
    6.         public EqualityComparerEnum()
    7.         {
    8.             var rand = new Random(12345);
    9.             _enum = (MyEnum)rand.Next(0, 3);
    10.  
    11.             _comparer = EqualityComparer<MyEnum>.Default;
    12.         }
    13.  
    14.         [Benchmark]
    15.         public void CallToEnumEqualityComparerGetHashCode()
    16.         {
    17.             _comparer.GetHashCode(_enum);
    18.         }
    19.  
    20.         enum MyEnum
    21.         {
    22.             A,
    23.             B,
    24.             C,
    25.             D
    26.         }
    27.     }
    This (the call to GetHashCode) runs an order of magnitude slower (~12x) with full generic sharing enabled.
     
  4. RealMTG

    RealMTG

    Joined:
    Jul 27, 2013
    Posts:
    364
    Unfortunately, I don't have any projects to really test the changes on but I just want to add my two cents.
    Personally, I'd rather have longer build times for more performance. The Release setting could be close to final and faster with these new changes while Master could take longer but maximizes performance.
     
    Ruchir likes this.
  5. Kleptine

    Kleptine

    Joined:
    Dec 23, 2013
    Posts:
    135
    Presumably this will be a configurable setting. Especially because there is bound to be corner-case code where sharing generics proves to be very slow (like the micro-benchmark mentioned above).

    Glad to see this! I would try this out if we were on the Alpha. Looking forward to seeing what other people observe.
     
  6. joncham

    joncham

    Unity Technologies

    Joined:
    Dec 1, 2011
    Posts:
    265
    This is partly why we are trying to gather feedback. In order to know if and how it should be configurable.
     
    RealMTG, JoNax97 and Kleptine like this.
  7. Skjalg

    Skjalg

    Joined:
    May 25, 2009
    Posts:
    194
    I observed a shorter build time when trying this out, but I immediately get nullreferenceexception when I run the build (happening when loading an addresable scene). Building with mono works though.

    So I have to say that I'd much rather it works than it being fast hehe :)


    Unity 2019.4.21f1
    Clean -
    Build completed with a result of 'Succeeded' in 1431 seconds (1431279 ms)
    Build completed with a result of 'Succeeded' in 468 seconds (467873 ms)
    Build completed with a result of 'Succeeded' in 462 seconds (462387 ms)

    Unity 2020.2.6f1
    Clean - Build completed with a result of 'Succeeded' in 1052 seconds (1051999 ms)
    Build completed with a result of 'Succeeded' in 136 seconds (135982 ms)
    Build completed with a result of 'Succeeded' in 135 seconds (134927 ms)

    Unity 2021.2.0a6.1011
    Clean - Build completed with a result of 'Succeeded' in 681 seconds (681159 ms)
    Build completed with a result of 'Succeeded' in 102 seconds (102462 ms)
    Build completed with a result of 'Succeeded' in 105 seconds (105499 ms)
     

    Attached Files:

  8. Skjalg

    Skjalg

    Joined:
    May 25, 2009
    Posts:
    194
    I gotta say I am impressed by the speed upgrade when going from 2019.4 LTS to 2020.2. Cant wait for 2020 LTS version, its going to be very good!
     
    joncham likes this.
  9. joncham

    joncham

    Unity Technologies

    Joined:
    Dec 1, 2011
    Posts:
    265
    Do you get a NullReferenceException even without the option enabled? Or only when the option for full generic sharing is enabled?
     
  10. TimHeijden

    TimHeijden

    Joined:
    Mar 5, 2015
    Posts:
    33
    I've just ran some tests and got surprising results, in that building is SLOWER than before!

    (Note: All builds are development builds, with ILC2PP configuration "master")

    Build Time
    Build Size
    I was quite careful in testing, making sure to close unity between each build (& after upgrading before making B.) so this is quite weird. I've also made some profiler snapshots for each build, I'll send you a DM with those & some extra details (like the profile.json) later today.
     
    joncham likes this.
  11. joncham

    joncham

    Unity Technologies

    Joined:
    Dec 1, 2011
    Posts:
    265
    Thanks for your testing of this @TimHeijden ! I took a look at your build profiler output and have a few thoughts.

    1. I see you are building in master/"ReleasePlus" build config. This enables LTO (link time optimization). While this may help performance, it's not something you want to iterate with as it is extremely expensive. It looks like 95% of your entire build time is the link event. I'd suggest Release for your daily work and it's something we should note more clearly to improve iteration.
    2. I see you are using the VS 2017 toolchain. We have found the VS 2019 linker much faster. It may be something else to keep in mind when you can upgrade.
    3. It seems like the "C" case didn't pickup the environment variable. You will want to make sure it is set, and kill/restart Unity and Unity Hub.

    If you have time and are interested in providing more data, I would suggest:
    1. Just doing runs for "B" and "C" cases. This is the most interesting data points for this new option.
    2. Use Release rather than ReleasePlus/master config.
    3. Again, for "C" case make sure environment variable is set and all instances of Unity/UnityHub are killed to pick up new value
    4. Do both clean (which is what your data seems to come from) and incremental builds. Tweak one line in a script and build again, as all things should be faster but we are mainly focused on this incremental use case.

    Thanks again for providing feedback!
     
  12. TimHeijden

    TimHeijden

    Joined:
    Mar 5, 2015
    Posts:
    33
    Hey Jon, thanks for clarifying the main use case you're looking for. I think I'll do another test later today and otherwise next week. I had killed unity & the hub, so its strange that "C" didn't pickup the environment variable... will check more closely. (I had assumed it worked because the build time was so much shorter again)

    As for building "ReleasePlus", I was already fully aware that its not really intended for iteration, but I was also very curious about the possible performance drop and thus thought I'd use the "optimum" setting for that. Note that I'm also very much of the opinion that unity devs should be able to choose for the maximum runtime performance, especially when developing for slower platforms than PC and trying to hit a stable FPS, 10% extra performance cost for scripting is quite a lot!

    In addition, I am indeed using VS2017. This is because some platforms require use of 2017 rather than 2019. Good to know that 2019 is faster though, will keep that in mind!
     
  13. TimHeijden

    TimHeijden

    Joined:
    Mar 5, 2015
    Posts:
    33
    Couldn't help myself, did it right away. This time I used the IL2CPP configuration "release" instead, and also did cold/warm tests. All builds are using Unity 2021.1.0a8

    I hope this time the environment variable stuck properly. Stats may be slightly skewed because I restarted my PC between no-env & env builds.

    Build Time
    I again sent a DM with the other details inside.
     
    LeonhardP and mahdi_jeddi like this.
  14. poprev-3d

    poprev-3d

    Joined:
    Apr 12, 2019
    Posts:
    20
    When trying to Il2CPP build an empty project with 2021.2.0.0a9.1376 i get this error during build time:


    Exception: Unity.IL2CPP.Building.BuilderFailedException: Build failed with 190 successful nodes and 1 failed ones
    Annotation: C_Win64_VS2019Pch y2vq/pch-cpp-3692469889848082439.pch (+obj)
    ...
    Files (x86)\Windows Kits\10\Include\10.0.18362.0\ucrt" /I"C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\cppwinrt" /Fo"y2vq\pch-cpp-3692469889848082439.obj" /D_HAS_AUTO_PTR_ETC=1 /D_SILENCE_ALL_CXX17_DEPRECATION_WARNINGS /D_SILENCE_ALL_CXX20_DEPRECATION_WARNINGS /Zc:__cplusplus /std:c++17
    ExitCode: 1
    Stdout:
    pch-cpp.hpp
    c1xx: fatal error C1083: Cannot open source file: 'Files\Unity\Hub\Editor\2021.2.0a9\Editor\Data\il2cpp\libil2cpp\pch\pch-cpp.hpp': No such file or directory
    pch-cpp.hpp
    c1xx: fatal error C1083: Cannot open source file: 'Files\Unity\Hub\Editor\2021.2.0a9\Editor\Data\il2cpp\libil2cpp\pch\pch-cpp.hpp': No such file or directory
    pch-cpp-3692469889848082439.cpp
    C:\Users\User\Desktop\tessddd\New Unity Project\Library\il2cpp_cache\buildstate\y2vq\pch-cpp-3692469889848082439.cpp: fatal error C1083: Cannot open include file: 'C:\Program': No such file or directory


    It seems that pch-cpp.hpp is missing although it is on my computer.

    Any idea ? I really wish i could try the 2021.2 new features.
     
  15. joncham

    joncham

    Unity Technologies

    Joined:
    Dec 1, 2011
    Posts:
    265
  16. poprev-3d

    poprev-3d

    Joined:
    Apr 12, 2019
    Posts:
    20
    @joncham yup it seems due to the space of "Program Files", thank you !
     
  17. KamilDA

    KamilDA

    Joined:
    May 21, 2020
    Posts:
    104
    @joncham, would these improvement also work when building for WebGL or is it limited to mobile/desktop toolchains?

    Thanks
     
  18. joncham

    joncham

    Unity Technologies

    Joined:
    Dec 1, 2011
    Posts:
    265
    It also applies to WebGL. The build time improvement on WebGL may vary due to the build process for that platform being more complex than most.
     
    Haneferd and KamilDA like this.
  19. polemical

    polemical

    Joined:
    Jun 17, 2019
    Posts:
    827
    I've noticed faster builds lately (I generally update to the latest alpha when I notice it, or soon thereafter). Actually there have even been times I was like did it really finish already? For WebGL that's a pretty big deal :D So yeah.. whatever you're doing, thanks.
     
    LeonhardP and joncham like this.
  20. gsylvain

    gsylvain

    Joined:
    Aug 6, 2014
    Posts:
    55
    @joncham A very long moment is spent waiting for the Microsoft Incremental Linker. I noticed that this process is single thread. On console, we are wasting a considerable amount of time because of this. Would it be possible to analyze if you could use all available CPU cores? Thank you!
    upload_2021-4-16_13-8-25.png
     
    Ruchir and Lars-Steenhoff like this.
  21. MiFrilke

    MiFrilke

    Joined:
    Dec 14, 2016
    Posts:
    23
    So I was mainly interested in checking out these possible performance implications of this option: "We want to know if you see real world performance issues when running with this option"

    Especially the example provided above regarding equality comparision and GetHashCode being much slower piqued my interest.
    We have a pretty sophisticated save/load system at the heart of our game where we use and abuse Dictionaries a lot, so GetHashCode gets called quite often on various objects all over the place.

    Sadly i couldn't even get as far as testing save performance, since the game just straight out crashes during saving when running with full generic sharing enabled.

    I didn't check build time differences or FPS yet, as I'm mainly interested in the saving and loading impact. Is there anything I could share regarding this issue that would be helpful to you to fix this? Sharing the project sadly is no option since it is way too big.

    For now i've attached the profile.json files for both a regular il2cpp build and one with the new option enabled.

    (Tested with Unity 2021.2.0a12)


    Stack trace of crash:
    Code (CSharp):
    1. ========== OUTPUTTING STACK TRACE ==================
    2.  
    3. 0x00007FF8F1181F20 (GameAssembly) [u:\desperados_unity_2021.2a\_unity_install\editor\data\il2cpp\libil2cpp\codegen\il2cpp-codegen-il2cpp.cpp:890] il2cpp_codegen_runtime_constrained_call
    4. 0x00007FF8F2B03208 (GameAssembly) [u:\desperados_unity_2021.2a\library\il2cppbuildcache\windows\x64\il2cppoutput\generics6.cpp:13965] ObjectEqualityComparer_1_GetHashCode_m04F4321254D5C4DB954043E12F6B360683A3FE83_gshared
    5. 0x00007FF8F10C42F1 (GameAssembly) [u:\desperados_unity_2021.2a\library\il2cppbuildcache\windows\x64\il2cppoutput\il2cppinvokertable.cpp:40369] RuntimeInvoker_TrueInt32_t680FF22E76F6EFAD4375103CBBFFA0421349384C_Il2CppFullySharedGenericAny
    6. 0x00007FF8F17EB60B (GameAssembly) [u:\desperados_unity_2021.2a\library\il2cppbuildcache\windows\x64\il2cppoutput\assembly-csharp16.cpp:20356] MiCharacterNPCWeaponHolder_GetHashCode_m217FB843B629933D7B6D64C7C67CA7B8EC3FF25C
    7. 0x00007FF8F107445C (GameAssembly) [u:\desperados_unity_2021.2a\library\il2cppbuildcache\windows\x64\il2cppoutput\il2cppinvokertable.cpp:44473] RuntimeInvoker_TrueInt32_t680FF22E76F6EFAD4375103CBBFFA0421349384C
    8. 0x00007FF8F1181F26 (GameAssembly) [u:\desperados_unity_2021.2a\_unity_install\editor\data\il2cpp\libil2cpp\codegen\il2cpp-codegen-il2cpp.cpp:890] il2cpp_codegen_runtime_constrained_call
    9. 0x00007FF8F2B03208 (GameAssembly) [u:\desperados_unity_2021.2a\library\il2cppbuildcache\windows\x64\il2cppoutput\generics6.cpp:13965] ObjectEqualityComparer_1_GetHashCode_m04F4321254D5C4DB954043E12F6B360683A3FE83_gshared
    10. 0x00007FF8F10C42F1 (GameAssembly) [u:\desperados_unity_2021.2a\library\il2cppbuildcache\windows\x64\il2cppoutput\il2cppinvokertable.cpp:40369] RuntimeInvoker_TrueInt32_t680FF22E76F6EFAD4375103CBBFFA0421349384C_Il2CppFullySharedGenericAny
    11. 0x00007FF8F0FE56A7 (GameAssembly) [u:\desperados_unity_2021.2a\library\il2cppbuildcache\windows\x64\il2cppoutput\assembly-csharp-firstpass12.cpp:249] InterfaceFuncInvoker1<int,Il2CppObject *>::Invoke
    12. 0x00007FF8F2A76737 (GameAssembly) [u:\desperados_unity_2021.2a\library\il2cppbuildcache\windows\x64\il2cppoutput\generics3.cpp:20746] Dictionary_2_FindEntry_mC9A39BDAA66FDC3D9E96031E0EB4D71E51306E46_gshared
    13. 0x00007FF8F10C42F1 (GameAssembly) [u:\desperados_unity_2021.2a\library\il2cppbuildcache\windows\x64\il2cppoutput\il2cppinvokertable.cpp:40369] RuntimeInvoker_TrueInt32_t680FF22E76F6EFAD4375103CBBFFA0421349384C_Il2CppFullySharedGenericAny
    14. 0x00007FF8F2A7ACDE (GameAssembly) [u:\desperados_unity_2021.2a\library\il2cppbuildcache\windows\x64\il2cppoutput\generics3.cpp:21710] Dictionary_2_TryGetValue_mC15984CACD8C4EC727085E5BA025AF6CA0D158DB_gshared
    15. 0x00007FF8F14EC397 (GameAssembly) [u:\desperados_unity_2021.2a\library\il2cppbuildcache\windows\x64\il2cppoutput\assembly-csharp-firstpass6.cpp:24401] SaveLoadSceneManager_iRefToID_mEFC75EA9734E2A7C9B9B2349BA1E91674EAD7FE2
    16. 0x00007FF8F14E1690 (GameAssembly) [u:\desperados_unity_2021.2a\library\il2cppbuildcache\windows\x64\il2cppoutput\assembly-csharp-firstpass6.cpp:21709] SaveLoadSceneManager_castToYamlSerializable_m842964935ECCCE167A2B89B0D41878866DD0B0BC
    17. 0x00007FF8F14EDFBB (GameAssembly) [u:\desperados_unity_2021.2a\library\il2cppbuildcache\windows\x64\il2cppoutput\assembly-csharp-firstpass6.cpp:20443] SaveLoadSceneManager_reflectMiSerializeInfo_mB54930FF38D3506C72FFCFC27DE816CFA2DF1E4F
    18. 0x00007FF8F14EBB4E (GameAssembly) [u:\desperados_unity_2021.2a\library\il2cppbuildcache\windows\x64\il2cppoutput\assembly-csharp-firstpass6.cpp:17684] SaveLoadSceneManager_handleSerializeInfosForClass_m1BCDFE87EA546F5BAD2FF76355441557E898DF52
    19. 0x00007FF8F14E7E49 (GameAssembly) [u:\desperados_unity_2021.2a\library\il2cppbuildcache\windows\x64\il2cppoutput\assembly-csharp-firstpass6.cpp:16509] SaveLoadSceneManager_createValueTypeData_m2AF9AA486E67C808AC63707C7C91E6254F5701BA
    20. 0x00007FF8F1A9F126 (GameAssembly) [u:\desperados_unity_2021.2a\library\il2cppbuildcache\windows\x64\il2cppoutput\assembly-csharp42.cpp:26105] U3CcreateSceneSaveDateAndStoreToMemberU3Ed__85_MoveNext_m8FDC2EDE5F0307820A57F531E1CC45367F15B2B0
    21. 0x00007FF8F107E98C (GameAssembly) [u:\desperados_unity_2021.2a\library\il2cppbuildcache\windows\x64\il2cppoutput\il2cppinvokertable.cpp:45097] RuntimeInvoker_TrueSByte_tFEFFEF5D2FEBF5207950AE6FAC150FC53B668DB5
    22. 0x00007FF8F389C243 (GameAssembly) [u:\desperados_unity_2021.2a\library\il2cppbuildcache\windows\x64\il2cppoutput\unityengine.coremodule1.cpp:19961] SetupCoroutine_InvokeMoveNext_m4C8B79D8A35F69EEECAA16C82793525E74D75CE9
    23. 0x00007FF8F11E3126 (GameAssembly) [u:\desperados_unity_2021.2a\_unity_install\editor\data\il2cpp\libil2cpp\vm\runtime.cpp:583] il2cpp::vm::Runtime::InvokeWithThrow
    24. 0x00007FF8F11E2BF1 (GameAssembly) [u:\desperados_unity_2021.2a\_unity_install\editor\data\il2cpp\libil2cpp\vm\runtime.cpp:568] il2cpp::vm::Runtime::Invoke
     

    Attached Files:

unityunity