Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. Dismiss Notice

Android il2cpp build is too slow compared to iOS

Discussion in 'Android' started by kyubuns, Aug 21, 2019.

  1. kyubuns

    kyubuns

    Joined:
    Aug 6, 2013
    Posts:
    135
    My build machine will complete the ios il2cpp build in 20 minutes, android mono build in less than 10minutes.
    But, Android il2cpp build has over 7 hours. (ARMv7, ARM64, 3.5 hours each)
    Can you think of anything that might have caused it?

    https://docs.unity3d.com/Manual/IL2CPP-OptimizingBuildTimes.html
    This has already been tried.

    ```
    Invoking il2cpp with arguments: --convert-to-cpp --emit-null-checks --enable-array-bounds-check --dotnetprofile="unityaot" --compile-cpp --libil2cpp-static --platform="Android" --architecture="ARMv7" --configuration="Release" --outputpath="/Users/username/.jenkins/jobs/project_name/jobs/project_name.task.android.test/workspace/client/Temp/StagingArea/assets/bin/Data/Native/armeabi-v7a/libil2cpp.so" --cachedirectory="/Users/username/.jenkins/jobs/project_name/jobs/project_name.task.android.test/workspace/client/Assets/../Library/il2cpp_android_armeabi-v7a/il2cpp_cache" --additional-include-directories="/Applications/Unity/Hub/Editor/2018.4.5f1/PlaybackEngines/AndroidPlayer/Tools/bdwgc/include" --additional-include-directories="/Applications/Unity/Hub/Editor/2018.4.5f1/PlaybackEngines/AndroidPlayer/Tools/libil2cpp/include" --tool-chain-path="/Applications/android-ndk-r16b" --map-file-parser="/Applications/Unity/Hub/Editor/2018.4.5f1/Unity.app/Contents/Tools/MapFileParser/MapFileParser" --profiler-report --directory="/Users/username/.jenkins/jobs/project_name/jobs/project_name.task.android.test/workspace/client/Temp/StagingArea/assets/bin/Data/Managed" --generatedcppdir="/Users/username/.jenkins/jobs/project_name/jobs/project_name.task.android.test/workspace/client/Temp/StagingArea/Il2Cpp/il2cppOutput"
    ```

    I upload the profile by "--profiler-report".
    (renamed profile.txt from profile.json)

    Environment:
    - Unity2018.4.5f1
    - .NET 4.x

    Build Machine:
    - Mac Pro (Late 2013)
    - macOS Mojave 10.14.6
    - 3.5GHz 6-Core Intel Xeon E5, 64GB Memory, SSD
     

    Attached Files:

  2. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,773
    Wow, that is a really long time - something is wrong here. The profile indicates that almost all of the time was spent compiling the Il2CppTypeDefinitions.cpp file (221 minutes, or about 3.5 hours). It seems that Il2CppGenericClassTable.cpp is also pretty bad, at 28 minutes.

    Can you share these two files (maybe via private message, if you would like)? They should exist below the Temp/ directory in your Unity project when the editor is open after a build. I'd like to see the content of those generated files.

    It might also be worth comparing them in your iOS build, so see what is different (if anything).
     
  3. kyubuns

    kyubuns

    Joined:
    Aug 6, 2013
    Posts:
    135
    Thanks for your reply!
    I try to collect the files, ios and android each.
    I want to send by private message, because the project is production.

    I send there files to you by conversation.
    Please check it.
     
    Last edited: Aug 22, 2019
  4. rwetzold

    rwetzold

    Joined:
    Jun 21, 2021
    Posts:
    2
    Was there any resolution for this? I am on 2021.2.7 and Android is taking sooooo much longer than other build targets and compared to 2021.1.28 it also is much slower. Is there any way to debug the culprit of this? It is for example stuck at libil2cpp.so for many minutes without any progress until it finally hands it over to gradle.
     
  5. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,773
    I don't recall what, if any, resolution there was for this. If you can provide the profile.json files from the project directory after the build is complete, then we can have a look a them, and might be able to tell what is causing the issue.

    It also might be useful to compare them from 2021.2.7 to 2021.2.28 versions of Unity. You can also inspect these files locally using Google Chrome with chrome://tracing.
     
  6. mahito2932

    mahito2932

    Joined:
    Nov 15, 2020
    Posts:
    89
    I can confirm it's incredebly slow. I tried to build one scene Low-poly it takes me 28 minutes with il2cpp enabled

    I tried many versions of Unity all the same results with the newer version it's even worse
     
  7. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,773
    We do expect builds with IL2CPP to be slower than builds with Mono, as the IL2CPP build has to pre-compile the code, where Mono compiles it on the Android device.

    With that said, our testing shows that with the newest versions of Unity (e.g. 2022.1) the build times are very close for most projects. If you have a project that shows a significant difference like this between Mono and IL2CPP, and you can hare it, we would be happy to have a look at it in a bug report.
     
  8. mahito2932

    mahito2932

    Joined:
    Nov 15, 2020
    Posts:
    89

    I will try to look on 2022 but unfortunately many google services won't work with 2022.

    The problem is that I have another game in il2cpp but much heavier and we are talking about almost 700MB but build only takes 2 minutes instead with a Low-poly scene only 10k vertices takes 28 minutes.

    Update Edit:
    yes currently on 2022 it works well. My project and the scenes look wonderful.
    Well I'll try for now 2022. :)
    How beautiful 2022 looks like. :D

    Build Time first was 28 min... ver (all)2021
    now 37 seconds with 2022 version
     
    Last edited: Feb 9, 2022
  9. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,773
    I'm glad it is that much faster! I did not expect such a significant difference though!
     
  10. mahito2932

    mahito2932

    Joined:
    Nov 15, 2020
    Posts:
    89
    I was shocked too. The only thing I noticed is that the event system no longer works on android Unity 2022. I always used OnMouseDown to check for collider for example
    if (EventSystem.current.IsPointerOverGameObject())
    return;
    is there another method around because i did not find anything actually Pointerdown not work for me either
     
  11. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,773
    Sorry, I don't know enough to help with this aspect.
     
  12. mahito2932

    mahito2932

    Joined:
    Nov 15, 2020
    Posts:
    89
    Okay i come back to old version of unity 2020 because 2022 is unstable... now the build is not anymore 28 minutes but 2 hours :(
     
  13. Badinha

    Badinha

    Joined:
    May 6, 2021
    Posts:
    5
    Sorry for digging this one up, but there's anything specific here that you would like to take a look @JoshPeterson, most because I'm dealing with a similar issue. Our project its a live project with almost 7 years of live service, so this has been upgraded between multiple unity versions and we are in 2019 (on the brink of going 2020) but our il2cpp build times takes a really long time, I'm wondering what you would suggest to understand why
     
  14. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,773
    See the original post in this thread. You case use the https://docs.unity3d.com/ScriptReference/PlayerSettings.SetAdditionalIl2CppArgs.html API method to pass the --profiler-report option to IL2CPP. That will generate a file named profile.json that you can then use with Google Chrome tracing to see where the time in the build is going.
     
  15. pistoleta

    pistoleta

    Joined:
    Sep 14, 2017
    Posts:
    536
    Having this issue on Unity 2022.3.0 . While on iOS the build time on I2LCPP is 3 minutes aprox... for android takes 30-40 minutes.... there's something really wrong here.
    Can't say how much took to build for android in prior versions since this is the first time.
     
  16. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,773
    Do you happen to have the C++ Compiler Configuration option in the Player Settings set to a value of "Master"? This will enable link-time optimization during the build. That can really increase build times, especially on Android.
     
  17. pistoleta

    pistoleta

    Joined:
    Sep 14, 2017
    Posts:
    536
    I have it on Release. I even changed the IL2CPP code generation to Faster (smaller) builds to no avail.
    Anything else i should check ?
    Thanks for your help
     
  18. pistoleta

    pistoleta

    Joined:
    Sep 14, 2017
    Posts:
    536
    Also one of the things that caught my attention is the amount of time the window 'Cloud Diagnostics Symbol Upload' shows up..... probably stays there for 10 min or more... although maybe is not what its internally doing... I thought i should share it.
     
  19. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,773
    The next troubleshooting step is to watch the processes that are running as children of the Unity editor process during the build. I suspect that there may be one C++ compiler process which takes a long time, and the rest of the build is waiting on that. If we can get the full command line for this process, that might help.
     
  20. pistoleta

    pistoleta

    Joined:
    Sep 14, 2017
    Posts:
    536
    Where could I see this children?

    I've realized everytime I make a build, no matter if it's just 1 line of code the 'build and run' or the 'patch and run' do a new build... at least 15 min... on another thread they told me its possible some script is changing dependencies in the Build.IPreprocessBuildWithReport.OnPreprocessBuild call. Do you think this can be the case?

    Thanks a lot for your help!
     
  21. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,773
    What platform are you running the editor on? That will indicate which tool you should use.

    That could be possible, although I would expect the incremental build to lower the build time even then.

    I think the next key is to understand what processes the build is spawning.
     
  22. pistoleta

    pistoleta

    Joined:
    Sep 14, 2017
    Posts:
    536
    Im on MacOS Sillicon Ventura 13.1
     
  23. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,773
    Then I would recommend htop to view processes: https://htop.dev/

    It is a pretty nice command line process viewer that works well on macOS.
     
  24. pistoleta

    pistoleta

    Joined:
    Sep 14, 2017
    Posts:
    536
    Does this tell something?
     
  25. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,773
    The highlighted line there is a compression step to compress the symbol file used by Cloud Diagnostics - if that seems to be taking the longest time, can you try a build with cloud diagnostics disabled?

    Note that in htop, you can use the F5 key to see a tree of processes - that might make it easier to find the process that are child of Unity.
     
  26. pistoleta

    pistoleta

    Joined:
    Sep 14, 2017
    Posts:
    536
    I uninstalled Unity Diagnostics long ago, I've been looking around the project to see if theres still something left but I dont find anything... any idea where this might come from?

    Child view at a later point of the build:



    and another one:
     
  27. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,773
  28. pistoleta

    pistoleta

    Joined:
    Sep 14, 2017
    Posts:
    536
  29. pistoleta

    pistoleta

    Joined:
    Sep 14, 2017
    Posts:
    536
    I got rid of it!!! by installing the package again, disabling, and uninstalling. Now it doesnt appear anymore, incremental builds are faster, not as fast as someone told me should be (10-20 seconds) but now takes maybe 6 minutes aproximately, is this timing inside of what we can consider normal? (This is an incremental build with script debugging on)....

    Curiously, now the step that takes the most time is 'Incrementally deploying to ..." At the beggining i thought maybe it was transferring the build to the device via wifi but now is everything connected by usb and still taking quite some time.

    Thanks again for all your help.
     
    Last edited: Jun 3, 2023
    pixelsprite_unity likes this.
  30. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,773
    Normal really varies per project, unfortunately. I think that 6 minutes is within the realm of possibility though.
     
  31. pistoleta

    pistoleta

    Joined:
    Sep 14, 2017
    Posts:
    536
    Even for an incremental build?

    Just testing and no, not 6 minutes, Incremental building takes already :
    >10 -15 minutes at least 'Building Gradle Project' (does an incremental build need to do so?)
     
    Last edited: Jun 9, 2023
  32. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,773
    There could be a lot of things that go into building the player, but 10-15 minutes seems to be too long.

    I'm not sure how building a grade project fits into this picture though. I''ll ask around and get back to you.
     
  33. Tomas1856

    Tomas1856

    Unity Technologies

    Joined:
    Sep 21, 2012
    Posts:
    3,663
    Out of curiosity, how did you determine that 'Building Gradle Project' is talking 10-15mins? You need to check Editor.log I think it contains real time for that task, if you're determining build time by looking at the progress dialog, it might not be accurate
     
  34. AcidArrow

    AcidArrow

    Joined:
    May 20, 2010
    Posts:
    11,002
    20-25 minute builds have been normal with Unity for a while now (at least with IL2CPP and Master config)

    The same project on the same machine takes around 3 minutes on iOS (and another 2-3 for archiving on XCODE if we want to make things equivalent)

    I don't really think Unity is at fault though, really. When we used to export to a gradle project, exporting from Unity was fairly quick (under 10 minutes maybe closer to 5?), and then android studio took forever.

    I'm just posting here so that you know that from our experience going from Unity to an actual device, takes 5 to 10 times longer on Android than on iOS.