Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Official Android 11 ARM64 native heap allocator issues

Discussion in 'Android' started by Tomas1856, Jan 28, 2021.

  1. Tomas1856

    Tomas1856

    Unity Technologies

    Joined:
    Sep 21, 2012
    Posts:
    3,875
    Hi,

    In Android 11 release notes, there was a change to native heap alloactor https://developer.android.com/about/versions/11/behavior-changes-all#scudo.
    Native heap allocator was changed from jemalloc to scudo. The change affected Unity's native heap allocator, causing Unity based applications to crash on Android 11 when running on ARM 64.

    The error usually manifests itself with "Using memoryadresses from more than 16GB of memory" message in the logcat, followed by crash containing a stacktrace into UnityDefaultAllocator.
    Since UnityDefaultAllocator is/was used by all Unity versions, the problem affects all Unity versions.

    The issue was fixed, by switching UnityDefaultAllocator to DynamicHeapAllocator (an internal Unity native allocator), which works correctly with Scudo allocator:
    • 2018.4.30
    • 2019.4.15
    • 2020.1.14
    • 2020.2.0b12

    Additional notes:
    • As far as we know, there's no way to switch back to jemalloc allocator.
    • The fix won't be backported to Unity versions 2017 or lower, since the support for those versions has ended.
    • To switch back to UnityDefaultAllocator, you need to pass -systemallocator as command line argument
    • If you use adb shell dumpsys meminfo or Android Studio profiler, you'll notice that after switching to DynamicHeapAllocator "Native Heap" memory usage decreased, "Private Other" memory usage increased. That's expected.
    • Some people reported, that after switching their game to Unity version where the issue was supposedly fixed, they were still seeing "Using memoryadresses from more than 16GB of memory" in their crash logs. The underlying issue was with Unity Cloud Diagnostics:
    • When Unity Cloud Diagnostics sends a crash report, it specifies sender app's unity version, in other words the following scenario occurred:
    - Game made with Unity version 2019.4.14 crashes with "Using memoryadresses from more than 16GB of memory", the crash report is not sent immediately, but is saved on the phone
    - Game is updated to 2019.4.15
    - When game is launched, Unity Cloud Diagnostics runtime checks if there are any reports saved on the phone, founds it and sends it to the server, but instead of specifying 2019.4.14 version, it specifies 2019.4.15, so it may look like the crash was still occuring with 2019.4.15, but in reality, it was old report being sent with wrong Unity version.
    The issue was reported to Unity Cloud Diagnostics team.
    - Please check Google's Android Vitals instead, assuming you've uploaded native symbols, you shouldn't see crashes with UnityDefaultAllocator in stacktraces, since the allocator was switched to DynamicHeapAllocator.

    Update 2023-07-10

    UnityDefaultAllocator/SystemAllocator should work again in the following versions (assuming you've passed -systemallocator command line argument) when running on ARM64 on Android 11 or higher:
    • 2021.3.28f1
    • 2022.3.5f1
    • 2023.1.3f1
    • 2023.2.0a22
     
    Last edited: Jul 10, 2023
  2. unwshd_elend

    unwshd_elend

    Joined:
    Mar 21, 2019
    Posts:
    23
    Thanks , can you explain how can I switch from UnityDefaultAllocator to DynamicHeapAllocator?
     
    eskivor likes this.
  3. Tomas1856

    Tomas1856

    Unity Technologies

    Joined:
    Sep 21, 2012
    Posts:
    3,875
    If you're using the Unity versions specified above, they already use DynamicHeapAllocator, in earlier Unity versions it's impossible to switch to Dynamic Heap Allocator.
     
    unwshd_elend likes this.
  4. unwshd_elend

    unwshd_elend

    Joined:
    Mar 21, 2019
    Posts:
    23
    my unity version is 2020.1.2f1 , and we building with target api 29 with ll2cpp and arm64

    I understand the versions you mentioned are using DynamicHeapAllocator but we are still not sure why its happening to us in this unity version , is it using default allocator?
     
  5. Tomas1856

    Tomas1856

    Unity Technologies

    Joined:
    Sep 21, 2012
    Posts:
    3,875
    Yes 2020.1.2f1 is using default allocator, so you would need to switch to 2020.1.14f or higher.

    Also, note: 2020.1.* is no longer supported, so no bug fixes will be done in that version. I would really recommend you to switch to 2020 LTS version, which will be supported until year 2022 + few months.
     
    unwshd_elend likes this.
  6. namvip1998unt

    namvip1998unt

    Joined:
    Jan 8, 2019
    Posts:
    3
    Does 2019.4.30 also use DynamicHeapAllocator ? I used 2019.4.8 and crash on android 11 happened so much then I saw this post and want to upgrade. Idk if I should use 2019.4.15 like this post specified or 2019.4.30 since it's the newest of 2019. Please response asaf. Thank you @Tomas1856
     
  7. kaarloew

    kaarloew

    Joined:
    Nov 1, 2018
    Posts:
    360
    Use the latest LTS version.
     
  8. trooper

    trooper

    Joined:
    Aug 21, 2009
    Posts:
    748
    Is the default allocator used when running armv7 il2cpp with 2020.3.19f1

    I've been trying to figure out why I have so much reserved memory showing up and this warning shows on the latest memory profiler.

    upload_2021-9-28_8-58-56.png
     
  9. Tomas1856

    Tomas1856

    Unity Technologies

    Joined:
    Sep 21, 2012
    Posts:
    3,875
    No, both ARM7 and ARM64 use DynamicHeapAllocator. Also, you can see which allocator is used in the logcat.
     
    Last edited: Sep 28, 2021
  10. trooper

    trooper

    Joined:
    Aug 21, 2009
    Posts:
    748
    Thanks. Logcat does say Dynamic Heap Allocator so memory profiler must just have a bug.
     
  11. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,432
    It does, thanks for raising this, I've taken the liberty of filing that bug myself. :)
     
    Tomas1856 likes this.
  12. sacb0y

    sacb0y

    Joined:
    May 9, 2016
    Posts:
    867
    oh ok good that explains that message
     
  13. 00christian00

    00christian00

    Joined:
    Jul 22, 2012
    Posts:
    1,035
    Hi Tomas , I upgraded to the latest 2020.3.23 and I'm getting tons of crash on scene reload when the garbage collector start. Had to revert to 2020.3.14. I just saw a topic about the gradle upgrade and checked the sdk installed by Unity and it is using build tool 30.0.2 and tried both target api 29 and 31, both crash.
    Did the bug came back?
    If I build for IL2CPP there is no crash, but I can't build every time for IL2CPP just for testing. The crash happen 100% of the times on scene reload. I made a bug report : 1383483
     
  14. Tomas1856

    Tomas1856

    Unity Technologies

    Joined:
    Sep 21, 2012
    Posts:
    3,875
    This thread is regarding allocator changes, looking at the Unity versions you've mentioned, I would say your problem is unrelated.
     
  15. Avi-Anzu

    Avi-Anzu

    Joined:
    Aug 15, 2019
    Posts:
    4
    Hello there.
    If we compile scripts to dll files using mcs from Unity 2017, and then import those compiled scripts files into a project that is using 2018.4.36 ( for example ), which allocator will be used? The default from 2017, or Dynamic?
     
  16. Tomas1856

    Tomas1856

    Unity Technologies

    Joined:
    Sep 21, 2012
    Posts:
    3,875
    The allocator choice is not related to scripts, the code for which allocator to use lives in libunity.so, p.s you cannot move libunity.so from one Unity version to another.
     
  17. Avi-Anzu

    Avi-Anzu

    Joined:
    Aug 15, 2019
    Posts:
    4
    In other words - the editor decides which allocator will be used, right? Good ty.
     
  18. Tomas1856

    Tomas1856

    Unity Technologies

    Joined:
    Sep 21, 2012
    Posts:
    3,875
    Not the Editor, but player runtime which lives in native shared library - libunity.so
     
  19. Avi-Anzu

    Avi-Anzu

    Joined:
    Aug 15, 2019
    Posts:
    4
    Yes, but it is copied from current Unity Editor folder ( for example
    D:\UnityEngines\2020.3.7f1\Editor\Data\PlaybackEngines\AndroidPlayer\Variations\il2cpp\Release\Libs\arm64-v8a\libunity.so 
    ) when building the apk / project. Thus inherently which allocator will be used is decided by which editor the developer uses to build the project.
     
  20. Tomas1856

    Tomas1856

    Unity Technologies

    Joined:
    Sep 21, 2012
    Posts:
    3,875
    If by Editor, you mean Unity version, then yes, it's decided by Unity version.
     
    Avi-Anzu likes this.
  21. AbelChiu

    AbelChiu

    Joined:
    Nov 29, 2016
    Posts:
    12
    Hello, can you tell me the result of your attempt?
     
  22. canyon_gyh

    canyon_gyh

    Joined:
    Aug 15, 2018
    Posts:
    48
    I get the Crash(Using memoryadresses from more than 16GB of memory) 4 Android 11. Use Unity Version 2019-3-7f1
    Thanks
     
  23. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,432
    Hello @canyon_gyh,
    As the message you quoted states:
    So this version
    Is several versions before the fix and you should update to the latest patch release of 2019.4, or at the least, to 2019.4.15.
     
  24. canyon_gyh

    canyon_gyh

    Joined:
    Aug 15, 2018
    Posts:
    48
    Okey,Thank's
     
  25. Jazeera

    Jazeera

    Joined:
    Jun 13, 2017
    Posts:
    1
    Hello everyone, hello @Tomas1856
    Does any Unity editor newer than 2020.3.33 support 'Dynamic Heap Allocator' which is the solution for these crashes?
     
  26. Tomas1856

    Tomas1856

    Unity Technologies

    Joined:
    Sep 21, 2012
    Posts:
    3,875
    Yes, you can check logcat as well, I think it should say something like "Using Dynamic Allocator"
     
  27. Avi-Anzu

    Avi-Anzu

    Joined:
    Aug 15, 2019
    Posts:
    4
    Sorry didn't see your question. Updating the Unity patch version fixed the issue
     
  28. Tomas1856

    Tomas1856

    Unity Technologies

    Joined:
    Sep 21, 2012
    Posts:
    3,875
    Hey,

    we know some of you tried to use native Android tools to track memory issues which was only possible if Unity internally uses UnityDefaultAllocator/SystemAllocator, thus we tried to fix UnityDefaultAllocator issue on ARM64, the fix was applied in:
    • 2021.3.28f1
    • 2022.3.5f1
    • 2023.1.3f1
    • 2023.2.0a22
    Thus passing -systemallocator command line argument should no longer crash the app on Android 11 (or higher) ARM64 devices (Updated top page too).

    Cheers
     
    manugil, JesOb and MartinTilo like this.
  29. nopact_can

    nopact_can

    Joined:
    Feb 10, 2016
    Posts:
    2
    Hi, with Unity 2022.3.9f1, I am still experiencing the same issue. Here is my stacktrace:


    Code (CSharp):
    1. Build fingerprint: 'google/sdk_gphone64_arm64/emu64a:13/TE1A.220922.033/10836537:userdebug/dev-keys'
    2. Revision: '0'
    3. ABI: 'arm64'
    4. Timestamp: 2023-10-19 01:52:29.833305806-0700
    5. Process uptime: 11s
    6. Cmdline: com.nopact.bullethills
    7. pid: 9525, tid: 9704, name: UnityGfxDeviceW  >>> com.nopact.bullethills <<<
    8. uid: 10181
    9. tagged_addr_ctrl: 0000000000000001 (PR_TAGGED_ADDR_ENABLE)
    10. pac_enabled_keys: 000000000000000f (PR_PAC_APIAKEY, PR_PAC_APIBKEY, PR_PAC_APDAKEY, PR_PAC_APDBKEY)
    11. signal 6 (SIGABRT), code -1 (SI_QUEUE), fault addr --------
    12. Abort message: 'Scudo ERROR: invalid chunk state when deallocating address 0x2000073b5cacdc0'
    13.     x0  0000000000000000  x1  00000000000025e8  x2  0000000000000006  x3  0000007259511420
    14.     x4  0000000000808080  x5  0000000000808080  x6  0000000000808080  x7  8080808080808080
    15.     x8  00000000000000f0  x9  000000762762ea00  x10 0000000000000001  x11 000000762766cde4
    16.     x12 0101010101010101  x13 000000007fffffff  x14 000000000000ddee  x15 0000000000000030
    17.     x16 00000076276d1d58  x17 00000076276aec70  x18 00000072560e0000  x19 0000000000002535
    18.     x20 00000000000025e8  x21 00000000ffffffff  x22 0000000000000000  x23 0000007259512000
    19.     x24 00000000000d23c8  x25 b400007485d59ed0  x26 00000072d1100080  x27 000000724000018c
    20.     x28 0000007259512000  x29 00000072595114a0
    21.     lr  000000762765e968  sp  0000007259511400  pc  000000762765e994  pst 0000000000001000
    22. backtrace:
    23.       #00 pc 0000000000051994  /apex/com.android.runtime/lib64/bionic/libc.so (abort+164) (BuildId: 01331f74b0bb2cb958bdc15282b8ec7b)
    24.       #01 pc 0000000000041814  /apex/com.android.runtime/lib64/bionic/libc.so (scudo::die()+8) (BuildId: 01331f74b0bb2cb958bdc15282b8ec7b)
    25.       #02 pc 0000000000041ec0  /apex/com.android.runtime/lib64/bionic/libc.so (scudo::ScopedErrorReport::~ScopedErrorReport()+32) (BuildId: 01331f74b0bb2cb958bdc15282b8ec7b)
    26.       #03 pc 0000000000042220  /apex/com.android.runtime/lib64/bionic/libc.so (scudo::reportInvalidChunkState(scudo::AllocatorAction, void*)+116) (BuildId: 01331f74b0bb2cb958bdc15282b8ec7b)
    27.       #04 pc 0000000000043a20  /apex/com.android.runtime/lib64/bionic/libc.so (scudo::Allocator<scudo::AndroidConfig, &(scudo_malloc_postinit)>::deallocate(void*, scudo::Chunk::Origin, unsigned long, unsigned long)+308) (BuildId: 01331f74b0bb2cb958bdc15282b8ec7b)
    28.       #05 pc 000000000001046c  /vendor/lib64/egl/libEGL_emulation.so (eglDestroySyncKHR+228) (BuildId: 01d06ed28f3e61881aecee6e352da587)
    29.       #06 pc 000000000001cb5c  /system/lib64/libEGL.so (eglDestroySyncKHR+44) (BuildId: f7b27cec8464273f3f930678324abd0d)
    30.       #07 pc 000000000114718c  /data/app/~~s36mj0iluBigFDq4rUezfg==/com.nopact.bullethills-RsdFbknBUtRayPyTuLnuRQ==/lib/arm64/libunity.so (BuildId: 72e040b473990035)
    31.       #08 pc 0000000001147a1c  /data/app/~~s36mj0iluBigFDq4rUezfg==/com.nopact.bullethills-RsdFbknBUtRayPyTuLnuRQ==/lib/arm64/libunity.so (BuildId: 72e040b473990035)
    32.       #09 pc 000000000114792c  /data/app/~~s36mj0iluBigFDq4rUezfg==/com.nopact.bullethills-RsdFbknBUtRayPyTuLnuRQ==/lib/arm64/libunity.so (BuildId: 72e040b473990035)
    33.       #10 pc 0000000000a18e8c  /data/app/~~s36mj0iluBigFDq4rUezfg==/com.nopact.bullethills-RsdFbknBUtRayPyTuLnuRQ==/lib/arm64/libunity.so (BuildId: 72e040b473990035)
    34.       #11 pc 0000000000a18e28  /data/app/~~s36mj0iluBigFDq4rUezfg==/com.nopact.bullethills-RsdFbknBUtRayPyTuLnuRQ==/lib/arm64/libunity.so (BuildId: 72e040b473990035)
    35.       #12 pc 0000000000f0b920  /data/app/~~s36mj0iluBigFDq4rUezfg==/com.nopact.bullethills-RsdFbknBUtRayPyTuLnuRQ==/lib/arm64/libunity.so (BuildId: 72e040b473990035)
    36.       #13 pc 00000000010d99e8  /data/app/~~s36mj0iluBigFDq4rUezfg==/com.nopact.bullethills-RsdFbknBUtRayPyTuLnuRQ==/lib/arm64/libunity.so (BuildId: 72e040b473990035)
    37.       #14 pc 00000000010d1e5c  /data/app/~~s36mj0iluBigFDq4rUezfg==/com.nopact.bullethills-RsdFbknBUtRayPyTuLnuRQ==/lib/arm64/libunity.so (BuildId: 72e040b473990035)
    38.       #15 pc 00000000010d1ba4  /data/app/~~s36mj0iluBigFDq4rUezfg==/com.nopact.bullethills-RsdFbknBUtRayPyTuLnuRQ==/lib/arm64/libunity.so (BuildId: 72e040b473990035)
    39.       #16 pc 00000000008cbec4  /data/app/~~s36mj0iluBigFDq4rUezfg==/com.nopact.bullethills-RsdFbknBUtRayPyTuLnuRQ==/lib/arm64/libunity.so (BuildId: 72e040b473990035)
    40.       #17 pc 00000000000b63b0  /apex/com.android.runtime/lib64/bionic/libc.so (__pthread_start(void*)+208) (BuildId: 01331f74b0bb2cb958bdc15282b8ec7b)
    41.       #18 pc 00000000000530b8  /apex/com.android.runtime/lib64/bionic/libc.so (__start_thread+64) (BuildId: 01331f74b0bb2cb958bdc15282b8ec7b)
     
  30. Tomas1856

    Tomas1856

    Unity Technologies

    Joined:
    Sep 21, 2012
    Posts:
    3,875
    I don't think it's the same issue, you need to resolve stacktrace to know for sure (you didn't resolve libunity.so functions)
     
  31. victornor

    victornor

    Joined:
    Jan 17, 2014
    Posts:
    87
    We are getting scudo errors again
    11-09 14:10:48.828: I/scudo(9296): Scudo ERROR: invalid chunk state when deallocating address 0x200007c229db9a0

    This happens on 2022 LTS builds, when trying to pass the play console checks on arm64 virtual devices.
    Upgrading to 2023 solves this issue. Using the JDK, SDK and NDK from 2023 in 2023 LTS does not solve it.

    No mentions of scudo in robo tests with builds from 2023.
     
  32. Tomas1856

    Tomas1856

    Unity Technologies

    Joined:
    Sep 21, 2012
    Posts:
    3,875
    Are you forcing system allocator ? either way, you might want to report a bug.
     
  33. victornor

    victornor

    Joined:
    Jan 17, 2014
    Posts:
    87
    No, i'm not forcing anything. This is an issue everyone is facing. I have been forced to upgrade to 2023, which is completely unacceptable.

    Here's the post regarding this issue. It's the same error.
    https://forum.unity.com/threads/crash-log-from-the-google-play.1507211/