Search Unity

  1. Unity Asset Manager is now available in public beta. Try it out now and join the conversation here in the forums.
    Dismiss Notice

Diagnosing il2cpp threading crashes

Discussion in '2022.1 Beta' started by Carpet_Head, Feb 2, 2022.

  1. Carpet_Head

    Carpet_Head

    Joined:
    Nov 27, 2014
    Posts:
    258
    We are noticing a lot of crash issues using threads in il2cpp on android mobile vr headsets. The stacktrace makes it seem very internal to unity and threading, but it's really hard to figure out if there is anything we can do about it? this is the desymbolicated stacktrace from the android device. It appears to happen when we starting c# tasks

    Code (CSharp):
    1. #00 0x000000000007f338 /apex/com.android.runtime/lib64/bionic/libc.so (__memcpy+296) (BuildId: 21847aa9757f000b0461310a9f5e6e51)
    2. #01 0x00000000008bae4c /data/app/com.gravitysketch.gravitysketch-d2c0lU9R7nVvoRQmAot9RQ==/lib/arm64/libil2cpp.so (BuildId: 3ff31b57838c6a4d10d7a59c8dc34eace22ac721)
    3.                                                                                                                   std::__ndk1::enable_if<(is_same<std::__ndk1::remove_const<Il2CppInternalThread*>::type, Il2CppInternalThread*>::value) && (is_trivially_copy_assignable<Il2CppInternalThread*>::value), Il2CppInternalThread**>::type std::__ndk1::__move<Il2CppInternalThread*, Il2CppInternalThread*>(Il2CppInternalThread**, Il2CppInternalThread**, Il2CppInternalThread**)
    4.                                                                                                                   G:/android-ndk-r21d\toolchains\llvm\prebuilt\windows-x86_64\bin\..\sysroot\usr\include\c++\v1\algorithm:1856:9
    5.                                                                                                                   Il2CppInternalThread** std::__ndk1::move<Il2CppInternalThread**, Il2CppInternalThread**>(Il2CppInternalThread**, Il2CppInternalThread**, Il2CppInternalThread**)
    6.                                                                                                                   G:/android-ndk-r21d\toolchains\llvm\prebuilt\windows-x86_64\bin\..\sysroot\usr\include\c++\v1\algorithm:1865:0
    7.                                                                                                                   std::__ndk1::vector<Il2CppInternalThread*, std::__ndk1::allocator<Il2CppInternalThread*> >::erase(std::__ndk1::__wrap_iter<Il2CppInternalThread* const*>)
    8.                                                                                                                   G:/android-ndk-r21d\toolchains\llvm\prebuilt\windows-x86_64\bin\..\sysroot\usr\include\c++\v1\vector:1713:0
    9.                                                                                                                   remove_working_thread(Il2CppInternalThread*)
    10.                                                                                                                   C:/SourceTree/GravitySketchVRTest/Build/unityLibrary/src/main/Il2CppOutputProject/IL2CPP/libil2cpp/mono/ThreadPool\ThreadPoolWorkerThread.cpp:27:0
    11. #02 0x00000000008b8908 /data/app/com.gravitysketch.gravitysketch-d2c0lU9R7nVvoRQmAot9RQ==/lib/arm64/libil2cpp.so (BuildId: 3ff31b57838c6a4d10d7a59c8dc34eace22ac721)
    12.                                                                                                                   worker_park()
    13.                                                                                                                   C:/SourceTree/GravitySketchVRTest/Build/unityLibrary/src/main/Il2CppOutputProject/IL2CPP/libil2cpp/mono/ThreadPool\ThreadPoolWorkerThread.cpp:80:9
    14.                                                                                                                   worker_thread(void*)
    15.                                                                                                                   C:/SourceTree/GravitySketchVRTest/Build/unityLibrary/src/main/Il2CppOutputProject/IL2CPP/libil2cpp/mono/ThreadPool\ThreadPoolWorkerThread.cpp:239:0
    16. #03 0x000000000085c724 /data/app/com.gravitysketch.gravitysketch-d2c0lU9R7nVvoRQmAot9RQ==/lib/arm64/libil2cpp.so (BuildId: 3ff31b57838c6a4d10d7a59c8dc34eace22ac721)
    17.                                                                                                                   il2cpp::vm::ThreadStart(void*)
    18.                                                                                                                   C:/SourceTree/GravitySketchVRTest/Build/unityLibrary/src/main/Il2CppOutputProject/IL2CPP/libil2cpp/vm\Thread.cpp:701:21
    19. #04 0x000000000086f4dc /data/app/com.gravitysketch.gravitysketch-d2c0lU9R7nVvoRQmAot9RQ==/lib/arm64/libil2cpp.so (BuildId: 3ff31b57838c6a4d10d7a59c8dc34eace22ac721)
    20.                                                                                                                   il2cpp::os::Thread::RunWrapper(void*)
    21.                                                                                                                   C:/SourceTree/GravitySketchVRTest/Build/unityLibrary/src/main/Il2CppOutputProject/IL2CPP/libil2cpp/os\Thread.cpp:201:9
    22. #05 0x00000000008c1edc /data/app/com.gravitysketch.gravitysketch-d2c0lU9R7nVvoRQmAot9RQ==/lib/arm64/libil2cpp.so (BuildId: 3ff31b57838c6a4d10d7a59c8dc34eace22ac721)
    23.                                                                                                                   il2cpp::os::ThreadImpl::ThreadStartWrapper(void*)
    24.                                                                                                                   C:/SourceTree/GravitySketchVRTest/Build/unityLibrary/src/main/Il2CppOutputProject/IL2CPP/libil2cpp/os/Posix\ThreadImpl.cpp:125:9
    25. #06 0x00000000000e68a0 /apex/com.android.runtime/lib64/bionic/libc.so (__pthread_start(void*)+36) (BuildId: 21847aa9757f000b0461310a9f5e6e51)
    26. #07 0x0000000000084b6c /apex/com.android.runtime/lib64/bionic/libc.so (__start_thread+64) (BuildId: 21847aa9757f000b0461310a9f5e6e51)
     
  2. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,931
    I don't think there is anything you can do to work around this issue. It does look like an internal problem in the IL2CPP thread pool implementation. Is this something you can reproduce locally, if only intermittently? If so, can you submit a bug report?

    Nothing in the code jumps out at me as being obviously wrong, but if we can get a case that causes this crash, we can likely track down the issue.
     
  3. Carpet_Head

    Carpet_Head

    Joined:
    Nov 27, 2014
    Posts:
    258
    we are finding it really hard to reliably reproduce, and are hoping to not submit our entire application as a bug report. It's hard to make a repro project for this, but we can work on it
     
    adbourdages and JoshPeterson like this.
  4. adbourdages

    adbourdages

    Joined:
    Jul 8, 2012
    Posts:
    75
    Resurrecting a dead discussion... but I'm running into the same kind of crash (memcpy) plus some more involving the threads (thread start, thread remove, QuickException). There doesn't appear to be anything else on the forums about this however, which is surprising considering the tentative new build is crashing at 15x the rate of the previous one.

    We can't easily reproduce these crash locally - they show up in the crash reporting services.
     
  5. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,931
    What player platform(s) and Unity version(s) are you seeing this happen with?
     
  6. adbourdages

    adbourdages

    Joined:
    Jul 8, 2012
    Posts:
    75
    Seeing it in 2021.3.2f1 on Android. Mostly Android version 12 but some 11 and 13 across many manufacturers. There is a more in-depth description of one of the crashes on this thread.

    Here are some stack traces from the reporting services. I can provide way more :)

    Exception java.lang.Error: FATAL EXCEPTION [UnityMain]
    Unity version : 2021.3.2f1
    Device model : samsung SM-G975U
    Device fingerprint: samsung/beyond2qltesq/beyond2q:12/SP1A.210812.016/G975USQU6IVE7:user/release-keys
    Build Type : Release
    Scripting Backend : IL2CPP
    ABI : arm64-v8a
    Strip Engine Code : true

    Caused by: java.lang.Error: *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
    Version '2021.3.2f1 (d6360bedb9a0)', Build type 'Release', Scripting Backend 'il2cpp', CPU 'arm64-v8a'
    Build fingerprint: 'samsung/beyond2qltesq/beyond2q:12/SP1A.210812.016/G975USQU6IVE7:user/release-keys'
    Revision: '17'
    ABI: 'arm64'
    Timestamp: 2022-08-21 06:58:05-0400
    pid: 3090, tid: 3444, name: IL2CPP Threadpo >>> com.greencod.pinballdeluxereloaded <<<
    uid: 10321
    signal 11 (SIGSEGV), code 2 (SEGV_ACCERR), fault addr 0x78348a0000
    x0 0000007834840900 x1 000000783489ffc8 x2 fffffffffffa08e8 x3 000000783489ff80
    x4 0000007834840900 x5 00000078348408f8 x6 0000000000000000 x7 0000000000000000
    x8 0000000000000000 x9 0000000000000000 x10 0000000000000000 x11 0000000000000000
    x12 0000000000000000 x13 0000000000000000 x14 00274d55ba041508 x15 0000d68566294b32
    x16 0000007755884a50 x17 0000007b088e8510 x18 00000075fe596000 x19 0000007834840900
    x20 00000078e498b470 x21 0000007834840900 x22 0000000000000000 x23 ffffffffffffffff
    x24 0000007755bceeb8 x25 0000007834840908 x26 00000078e498b470 x27 0000000000000002
    x28 0000007755bce000 x29 00000076503e7c50
    sp 00000076503e7b10 lr 0000007752856950 pc 0000007b088e84c4

    backtrace:
    #00 pc 000000000004d4c4 /apex/com.android.runtime/lib64/bionic/libc.so (__memcpy+292) (BuildId: 73e145ce4e1dfca57a8eaf6d29c1d3fe)
    #01 pc 0000000000c0494c /data/app/~~b4msFz58zlKdARVYUYib_g==/com.greencod.pinballdeluxereloaded-Lk8rzBezJUzXLP7klPk0bw==/lib/arm64/libil2cpp.so (BuildId: 15b9ea11eef6639cc996abdefa2d015db23c3cad)
    #02 pc 0000000000c022e8 /data/app/~~b4msFz58zlKdARVYUYib_g==/com.greencod.pinballdeluxereloaded-Lk8rzBezJUzXLP7klPk0bw==/lib/arm64/libil2cpp.so (BuildId: 15b9ea11eef6639cc996abdefa2d015db23c3cad)
    #03 pc 0000000000bd4c7c /data/app/~~b4msFz58zlKdARVYUYib_g==/com.greencod.pinballdeluxereloaded-Lk8rzBezJUzXLP7klPk0bw==/lib/arm64/libil2cpp.so (BuildId: 15b9ea11eef6639cc996abdefa2d015db23c3cad)
    #04 pc 0000000000c11f4c /data/app/~~b4msFz58zlKdARVYUYib_g==/com.greencod.pinballdeluxereloaded-Lk8rzBezJUzXLP7klPk0bw==/lib/arm64/libil2cpp.so (BuildId: 15b9ea11eef6639cc996abdefa2d015db23c3cad)
    #05 pc 0000000000bf0c1c /data/app/~~b4msFz58zlKdARVYUYib_g==/com.greencod.pinballdeluxereloaded-Lk8rzBezJUzXLP7klPk0bw==/lib/arm64/libil2cpp.so (BuildId: 15b9ea11eef6639cc996abdefa2d015db23c3cad)
    #06 pc 00000000000b6e44 /apex/com.android.runtime/lib64/bionic/libc.so (__pthread_start(void*)+264) (BuildId: 73e145ce4e1dfca57a8eaf6d29c1d3fe)
    #07 pc 0000000000053454 /apex/com.android.runtime/lib64/bionic/libc.so (__start_thread+68) (BuildId: 73e145ce4e1dfca57a8eaf6d29c1d3fe)
    at libc.__memcpy (__memcpy:292)
    at libil2cpp
    at libil2cpp
    at libil2cpp
    at libil2cpp
    at libil2cpp
    at libc.__pthread_start(void*) (__pthread_start:264)
    at libc.__start_thread (__start_thread:68)​

    A quite different crash signature here also involving threads

    Exception java.lang.Error: FATAL EXCEPTION [main]
    Unity version : 2021.3.2f1
    Device model : samsung SM-X706B
    Device fingerprint: samsung/gts8xxx/gts8:12/SP1A.210812.016/X706BXXU2AVE4:user/release-keys
    Build Type : Release
    Scripting Backend : IL2CPP
    ABI : arm64-v8a
    Strip Engine Code : true

    Caused by: java.lang.Error: *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
    Version '2021.3.2f1 (d6360bedb9a0)', Build type 'Release', Scripting Backend 'il2cpp', CPU 'arm64-v8a'
    Build fingerprint: 'samsung/gts8xxx/gts8:12/SP1A.210812.016/X706BXXU2AVE4:user/release-keys'
    Revision: '8'
    ABI: 'arm64'
    Timestamp: 2022-08-19 21:35:31+0200
    pid: 5826, tid: 6808, name: PrewarmService >>> com.greencod.pinballdeluxereloaded <<<
    uid: 10309
    signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0xc2c0e290
    x0 0000000000000011 x1 0000000000000001 x2 8fc04011121e0042 x3 8fc04011121e0042
    x4 fffffffffffffffe x5 221e350000000406 x6 0000000000443c6a x7 221e350000000406
    x8 00000000c2c0e28c x9 0000006b585806b0 x10 0000000000000000 x11 0000008fc0401112
    x12 fffffffffffffffe x13 8fc04011121e0042 x14 8fc04011121e0042 x15 0000000000000001
    x16 0000000000000002 x17 0000000000000000 x18 0000006b5759c000 x19 0000006b58580e00
    x20 00000000712e5a38 x21 0000006b58580b20 x22 0000000000000011 x23 000000007223265c
    x24 000000007223265c x25 0000006b58582000 x26 0000006b58580ca0 x27 0000006b585806bc
    x28 fffffffffffffffe x29 0000006b58580880
    sp 0000006b58580660 lr 0000006dcfcca918 pc 0000006dcfcca960

    backtrace:
    #00 pc 00000000006ca960 /apex/com.android.art/lib64/libart.so (art::CatchBlockStackVisitor::VisitFrame()+216) (BuildId: 34e3dd028e2e682b63a512d6a4f1b5eb)
    #01 pc 00000000002e4d34 /apex/com.android.art/lib64/libart.so (void art::StackVisitor::WalkStack<(art::StackVisitor::CountTransitions)0>(bool)+3244) (BuildId: 34e3dd028e2e682b63a512d6a4f1b5eb)
    #02 pc 000000000036049c /apex/com.android.art/lib64/libart.so (art::QuickExceptionHandler::FindCatch(art::ObjPtr<art::mirror::Throwable>)+436) (BuildId: 34e3dd028e2e682b63a512d6a4f1b5eb)
    #03 pc 000000000035f8dc /apex/com.android.art/lib64/libart.so (art::Thread::QuickDeliverException()+144) (BuildId: 34e3dd028e2e682b63a512d6a4f1b5eb)
    #04 pc 000000000035f848 /apex/com.android.art/lib64/libart.so (art::Thread::QuickDeliverException()) (BuildId: 34e3dd028e2e682b63a512d6a4f1b5eb)
    #05 pc 00000000002c9c58 /apex/com.android.art/lib64/libart.so (art_quick_deliver_exception+72) (BuildId: 34e3dd028e2e682b63a512d6a4f1b5eb)
    #06 pc 000000000023d4a4 /system/framework/arm64/boot-framework.oat (BuildId: e29e1b3299244900ff1a80a052d4d26517d9a003)
    at libart.art::CatchBlockStackVisitor::VisitFrame() (VisitFrame:216)​
     
  7. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,931
  8. adbourdages

    adbourdages

    Joined:
    Jul 8, 2012
    Posts:
    75
    Thanks for the update and checking out the potentially related problem!
     
  9. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,931
    It looks like the existing bug linked above is indeed the one @adbourdages experienced. The bug has been corrected internally, and we're getting the fix out to all Unity versions now.
     
  10. tylearymf_unity

    tylearymf_unity

    Joined:
    May 25, 2022
    Posts:
    12
    Last edited: Sep 9, 2022
  11. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,931
  12. manasgupta435

    manasgupta435

    Joined:
    Jul 11, 2020
    Posts:
    9
    We are also facing similar crashes in Android with IL2CPP build.
    There are 2 different stack-traces we found related to IL2CPP threadpool in the google play services crash reports

    @JoshPeterson Can you please let us know if this issue is related?

    Stacktraces:

    #00 pc 000000000004d384 (libc.so not found) /apex/com.android.runtime/lib64/bionic/libc.so (__memcpy+292) (BuildId: 5df338f27370a6fedaf36a8e8c973d1f)
    #01 pc 0000000000ec1e44 (_ZNSt6__ndk16__moveIP20Il2CppInternalThreadS2_EENS_9enable_ifIXaasr7is_sameINS_12remove_constIT_E4typeET0_EE5valuesr28is_trivially_copy_assignableIS8_EE5valueEPS8_E4typeEPS5_SC_S9_ at C:/Program Files/Unity/Hub/Editor/2021.3.4f1/Editor/Data/PlaybackEngines/AndroidPlayer/NDK/toolchains/llvm/prebuilt/windows-x86_64/bin/../sysroot/usr/include/c++/v1/algorithm:1856) /data/app/~~TCcur0QNICNwts5kMnguhQ==/cricketgames.hitwicket.strategy-zM_tO5_otXAeRjqutX8Wrg==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #02 pc 0000000000ebf900 (worker_park() at C:/Program Files/Unity/Hub/Editor/2021.3.4f1/Editor/Data/il2cpp/libil2cpp/mono/ThreadPool/ThreadPoolWorkerThread.cpp:80) /data/app/~~TCcur0QNICNwts5kMnguhQ==/cricketgames.hitwicket.strategy-zM_tO5_otXAeRjqutX8Wrg==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #03 pc 0000000000ea29e0 (il2cpp::vm::ThreadStart(void*) at C:/Program Files/Unity/Hub/Editor/2021.3.4f1/Editor/Data/il2cpp/libil2cpp/vm/Thread.cpp:701) /data/app/~~TCcur0QNICNwts5kMnguhQ==/cricketgames.hitwicket.strategy-zM_tO5_otXAeRjqutX8Wrg==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #04 pc 0000000000e9501c (il2cpp::eek:s::Thread::RunWrapper(void*) at C:/Program Files/Unity/Hub/Editor/2021.3.4f1/Editor/Data/il2cpp/libil2cpp/os/Thread.cpp:201) /data/app/~~TCcur0QNICNwts5kMnguhQ==/cricketgames.hitwicket.strategy-zM_tO5_otXAeRjqutX8Wrg==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #05 pc 0000000000ebe0f0 (il2cpp::eek:s::ThreadImpl::ThreadStartWrapper(void*) at C:/Program Files/Unity/Hub/Editor/2021.3.4f1/Editor/Data/il2cpp/libil2cpp/os/Posix/ThreadImpl.cpp:125) /data/app/~~TCcur0QNICNwts5kMnguhQ==/cricketgames.hitwicket.strategy-zM_tO5_otXAeRjqutX8Wrg==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #06 pc 00000000000b6d54 (libc.so not found) /apex/com.android.runtime/lib64/bionic/libc.so (__pthread_start(void*)+264) (BuildId: 5df338f27370a6fedaf36a8e8c973d1f)
    #07 pc 0000000000053370 (libc.so not found) /apex/com.android.runtime/lib64/bionic/libc.so (__start_thread+68) (BuildId: 5df338f27370a6fedaf36a8e8c973d1f)
    at libc.__memcpy (__memcpy:292)





    #00 pc 0000000002a2090c (String_get_Chars_mC49DF0CD2D3BE7BE97B3AD9C995BE3094F8E36D3 at C:/Users/Manas/My Files/Hitwicket/Hitwicket_2020/Library/Bee/artifacts/Android/il2cppOutput/cpp/mscorlib1.cpp:33144) /data/app/~~7VCt-yB1V4KWTvESq3aIJQ==/cricketgames.hitwicket.strategy-SeB_xyLYsTXG7JEeJAV6uQ==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #01 pc 0000000002b9055c (StackTrace_AddFrames_mFD2C38B4281C8B800FC463FD96DF39DA57BF13A9 at C:/Users/Manas/My Files/Hitwicket/Hitwicket_2020/Library/Bee/artifacts/Android/il2cppOutput/cpp/mscorlib23.cpp:16421) /data/app/~~7VCt-yB1V4KWTvESq3aIJQ==/cricketgames.hitwicket.strategy-SeB_xyLYsTXG7JEeJAV6uQ==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #02 pc 0000000002b912d4 (StackTrace_ToString_m46323EE6573660B32A02BA5BC774FC1D1F529717 at C:/Users/Manas/My Files/Hitwicket/Hitwicket_2020/Library/Bee/artifacts/Android/il2cppOutput/cpp/mscorlib23.cpp:17267) /data/app/~~7VCt-yB1V4KWTvESq3aIJQ==/cricketgames.hitwicket.strategy-SeB_xyLYsTXG7JEeJAV6uQ==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #03 pc 0000000002bfdd04 (Exception_GetStackTrace_m82E62621A0DB19C4B787314387AB85F612AF479C at C:/Users/Manas/My Files/Hitwicket/Hitwicket_2020/Library/Bee/artifacts/Android/il2cppOutput/cpp/mscorlib6.cpp:22962) /data/app/~~7VCt-yB1V4KWTvESq3aIJQ==/cricketgames.hitwicket.strategy-SeB_xyLYsTXG7JEeJAV6uQ==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #04 pc 0000000002bfe104 (Exception_ToString_m239DECD3A14DD6BC055B3835E623CBE887D5A732 at C:/Users/Manas/My Files/Hitwicket/Hitwicket_2020/Library/Bee/artifacts/Android/il2cppOutput/cpp/mscorlib6.cpp:23190) /data/app/~~7VCt-yB1V4KWTvESq3aIJQ==/cricketgames.hitwicket.strategy-SeB_xyLYsTXG7JEeJAV6uQ==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #05 pc 000000000101f9d0 (VirtualFuncInvoker0<String_t*>::Invoke(unsigned short, Il2CppObject*) at C:/Users/Manas/My Files/Hitwicket/Hitwicket_2020/Library/Bee/artifacts/Android/il2cppOutput/cpp/Assembly-CSharp-firstpass2.cpp:65) /data/app/~~7VCt-yB1V4KWTvESq3aIJQ==/cricketgames.hitwicket.strategy-SeB_xyLYsTXG7JEeJAV6uQ==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #06 pc 0000000002c442b0 (Action_Invoke_m7126A54DACA72B845424072887B5F3A51FC3808E_inline(Action_tD00B0A84D7945E50C2DFFC28EFEE6ED44ED2AD07*, MethodInfo const*) at C:/Users/Manas/My Files/Hitwicket/Hitwicket_2020/Library/Bee/artifacts/Android/il2cppOutput/cpp/mscorlib9.cpp:34500) /data/app/~~7VCt-yB1V4KWTvESq3aIJQ==/cricketgames.hitwicket.strategy-SeB_xyLYsTXG7JEeJAV6uQ==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #07 pc 0000000002c42334 (Task_FinishContinuations_m21B540B556CDABD76F5985DF83079F7F30C78174 at C:/Users/Manas/My Files/Hitwicket/Hitwicket_2020/Library/Bee/artifacts/Android/il2cppOutput/cpp/mscorlib9.cpp:21117) /data/app/~~7VCt-yB1V4KWTvESq3aIJQ==/cricketgames.hitwicket.strategy-SeB_xyLYsTXG7JEeJAV6uQ==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #08 pc 00000000028f5fd0 (Task_1_TrySetResult_mE67D469AB00E1958DECAE0D97B932BC43B9C2D4F_gshared at C:/Users/Manas/My Files/Hitwicket/Hitwicket_2020/Library/Bee/artifacts/Android/il2cppOutput/cpp/Generics92.cpp:21775) /data/app/~~7VCt-yB1V4KWTvESq3aIJQ==/cricketgames.hitwicket.strategy-SeB_xyLYsTXG7JEeJAV6uQ==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #09 pc 0000000001ba7ac8 (AsyncTaskMethodBuilder_1_SetResult_mC4BAC830514CDB105730FF11F7744A4424A4009B_gshared at C:/Users/Manas/My Files/Hitwicket/Hitwicket_2020/Library/Bee/artifacts/Android/il2cppOutput/cpp/Generics10.cpp:16471) /data/app/~~7VCt-yB1V4KWTvESq3aIJQ==/cricketgames.hitwicket.strategy-SeB_xyLYsTXG7JEeJAV6uQ==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #10 pc 0000000002ecb088 (AsyncTaskMethodBuilder_1_SetResult_mC4BAC830514CDB105730FF11F7744A4424A4009B(AsyncTaskMethodBuilder_1_tF0BDBC54D5EB433C1205550D0F9819F43C9D4019*, int, MethodInfo const*) at C:/Users/Manas/My Files/Hitwicket/Hitwicket_2020/Library/Bee/artifacts/Android/il2cppOutput/cpp/System.cpp:6903) /data/app/~~7VCt-yB1V4KWTvESq3aIJQ==/cricketgames.hitwicket.strategy-SeB_xyLYsTXG7JEeJAV6uQ==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #11 pc 0000000002c32a4c (ContextCallback_Invoke_m872CCCD40428B88C2612772491BE5157895B5F61_inline(ContextCallback_tE8AFBDBFCC040FDA8DA8C1EEFE9BD66B16BDA007*, Il2CppObject*, MethodInfo const*) at C:/Users/Manas/My Files/Hitwicket/Hitwicket_2020/Library/Bee/artifacts/Android/il2cppOutput/cpp/mscorlib8.cpp:32302) /data/app/~~7VCt-yB1V4KWTvESq3aIJQ==/cricketgames.hitwicket.strategy-SeB_xyLYsTXG7JEeJAV6uQ==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #12 pc 0000000002ae3d88 (MoveNextRunner_Run_m86395F123FB453F2B5FF89CE1E98532F7426F9DE at C:/Users/Manas/My Files/Hitwicket/Hitwicket_2020/Library/Bee/artifacts/Android/il2cppOutput/cpp/mscorlib17.cpp:28524) /data/app/~~7VCt-yB1V4KWTvESq3aIJQ==/cricketgames.hitwicket.strategy-SeB_xyLYsTXG7JEeJAV6uQ==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #13 pc 0000000002c442b0 (Action_Invoke_m7126A54DACA72B845424072887B5F3A51FC3808E_inline(Action_tD00B0A84D7945E50C2DFFC28EFEE6ED44ED2AD07*, MethodInfo const*) at C:/Users/Manas/My Files/Hitwicket/Hitwicket_2020/Library/Bee/artifacts/Android/il2cppOutput/cpp/mscorlib9.cpp:34500) /data/app/~~7VCt-yB1V4KWTvESq3aIJQ==/cricketgames.hitwicket.strategy-SeB_xyLYsTXG7JEeJAV6uQ==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #14 pc 0000000002c42334 (Task_FinishContinuations_m21B540B556CDABD76F5985DF83079F7F30C78174 at C:/Users/Manas/My Files/Hitwicket/Hitwicket_2020/Library/Bee/artifacts/Android/il2cppOutput/cpp/mscorlib9.cpp:21117) /data/app/~~7VCt-yB1V4KWTvESq3aIJQ==/cricketgames.hitwicket.strategy-SeB_xyLYsTXG7JEeJAV6uQ==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #15 pc 00000000028f8d48 (Task_1_TrySetResult_m2EE766FD3F76F4824990F4A93ED1F7253ECE014C_gshared at C:/Users/Manas/My Files/Hitwicket/Hitwicket_2020/Library/Bee/artifacts/Android/il2cppOutput/cpp/Generics92.cpp:23938) /data/app/~~7VCt-yB1V4KWTvESq3aIJQ==/cricketgames.hitwicket.strategy-SeB_xyLYsTXG7JEeJAV6uQ==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #16 pc 0000000001bab26c (AsyncTaskMethodBuilder_1_SetResult_m0D83195F995F9825D7A6DCDC3835D6917C43B5A6_gshared at C:/Users/Manas/My Files/Hitwicket/Hitwicket_2020/Library/Bee/artifacts/Android/il2cppOutput/cpp/Generics10.cpp:18724) /data/app/~~7VCt-yB1V4KWTvESq3aIJQ==/cricketgames.hitwicket.strategy-SeB_xyLYsTXG7JEeJAV6uQ==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #17 pc 0000000002ec6064 (AsyncTaskMethodBuilder_1_SetResult_mC456CDE030D0195F4C7D57C9CF7CC83EB532D6EA(AsyncTaskMethodBuilder_1_tB85AB2AF366A9742C159DD6FBD3FE38EDD13543B*, AsyncProtocolResult_t9A7197618C6BD851C5A33C65F45B4622C3CF9E05*, MethodInfo const*) at C:/Users/Manas/My Files/Hitwicket/Hitwicket_2020/Library/Bee/artifacts/Android/il2cppOutput/cpp/System.cpp:6497) /data/app/~~7VCt-yB1V4KWTvESq3aIJQ==/cricketgames.hitwicket.strategy-SeB_xyLYsTXG7JEeJAV6uQ==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #18 pc 0000000002c32a4c (ContextCallback_Invoke_m872CCCD40428B88C2612772491BE5157895B5F61_inline(ContextCallback_tE8AFBDBFCC040FDA8DA8C1EEFE9BD66B16BDA007*, Il2CppObject*, MethodInfo const*) at C:/Users/Manas/My Files/Hitwicket/Hitwicket_2020/Library/Bee/artifacts/Android/il2cppOutput/cpp/mscorlib8.cpp:32302) /data/app/~~7VCt-yB1V4KWTvESq3aIJQ==/cricketgames.hitwicket.strategy-SeB_xyLYsTXG7JEeJAV6uQ==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #19 pc 0000000002ae3d88 (MoveNextRunner_Run_m86395F123FB453F2B5FF89CE1E98532F7426F9DE at C:/Users/Manas/My Files/Hitwicket/Hitwicket_2020/Library/Bee/artifacts/Android/il2cppOutput/cpp/mscorlib17.cpp:28524) /data/app/~~7VCt-yB1V4KWTvESq3aIJQ==/cricketgames.hitwicket.strategy-SeB_xyLYsTXG7JEeJAV6uQ==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #20 pc 0000000002c442b0 (Action_Invoke_m7126A54DACA72B845424072887B5F3A51FC3808E_inline(Action_tD00B0A84D7945E50C2DFFC28EFEE6ED44ED2AD07*, MethodInfo const*) at C:/Users/Manas/My Files/Hitwicket/Hitwicket_2020/Library/Bee/artifacts/Android/il2cppOutput/cpp/mscorlib9.cpp:34500) /data/app/~~7VCt-yB1V4KWTvESq3aIJQ==/cricketgames.hitwicket.strategy-SeB_xyLYsTXG7JEeJAV6uQ==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #21 pc 0000000002c42334 (Task_FinishContinuations_m21B540B556CDABD76F5985DF83079F7F30C78174 at C:/Users/Manas/My Files/Hitwicket/Hitwicket_2020/Library/Bee/artifacts/Android/il2cppOutput/cpp/mscorlib9.cpp:21117) /data/app/~~7VCt-yB1V4KWTvESq3aIJQ==/cricketgames.hitwicket.strategy-SeB_xyLYsTXG7JEeJAV6uQ==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #22 pc 00000000028fcbd0 (Task_1_TrySetResult_m80576355963E201DD03E813734ED4CDA847E199E_gshared at C:/Users/Manas/My Files/Hitwicket/Hitwicket_2020/Library/Bee/artifacts/Android/il2cppOutput/cpp/Generics92.cpp:26857) /data/app/~~7VCt-yB1V4KWTvESq3aIJQ==/cricketgames.hitwicket.strategy-SeB_xyLYsTXG7JEeJAV6uQ==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #23 pc 0000000001bace3c (AsyncTaskMethodBuilder_1_SetResult_m46806C460117058C2FEC2AC46948F213CF5B4D11_gshared at C:/Users/Manas/My Files/Hitwicket/Hitwicket_2020/Library/Bee/artifacts/Android/il2cppOutput/cpp/Generics10.cpp:20214) /data/app/~~7VCt-yB1V4KWTvESq3aIJQ==/cricketgames.hitwicket.strategy-SeB_xyLYsTXG7JEeJAV6uQ==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #24 pc 0000000002ec6520 (U3CProcessOperationU3Ed__24_MoveNext_mA73F0FD80CE65D2E3E12390E49E3F34CFF47E98B at C:/Users/Manas/My Files/Hitwicket/Hitwicket_2020/Library/Bee/artifacts/Android/il2cppOutput/cpp/System.cpp:18099) /data/app/~~7VCt-yB1V4KWTvESq3aIJQ==/cricketgames.hitwicket.strategy-SeB_xyLYsTXG7JEeJAV6uQ==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #25 pc 0000000002c32a4c (ContextCallback_Invoke_m872CCCD40428B88C2612772491BE5157895B5F61_inline(ContextCallback_tE8AFBDBFCC040FDA8DA8C1EEFE9BD66B16BDA007*, Il2CppObject*, MethodInfo const*) at C:/Users/Manas/My Files/Hitwicket/Hitwicket_2020/Library/Bee/artifacts/Android/il2cppOutput/cpp/mscorlib8.cpp:32302) /data/app/~~7VCt-yB1V4KWTvESq3aIJQ==/cricketgames.hitwicket.strategy-SeB_xyLYsTXG7JEeJAV6uQ==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #26 pc 0000000002ae3d88 (MoveNextRunner_Run_m86395F123FB453F2B5FF89CE1E98532F7426F9DE at C:/Users/Manas/My Files/Hitwicket/Hitwicket_2020/Library/Bee/artifacts/Android/il2cppOutput/cpp/mscorlib17.cpp:28524) /data/app/~~7VCt-yB1V4KWTvESq3aIJQ==/cricketgames.hitwicket.strategy-SeB_xyLYsTXG7JEeJAV6uQ==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #27 pc 0000000002c442b0 (Action_Invoke_m7126A54DACA72B845424072887B5F3A51FC3808E_inline(Action_tD00B0A84D7945E50C2DFFC28EFEE6ED44ED2AD07*, MethodInfo const*) at C:/Users/Manas/My Files/Hitwicket/Hitwicket_2020/Library/Bee/artifacts/Android/il2cppOutput/cpp/mscorlib9.cpp:34500) /data/app/~~7VCt-yB1V4KWTvESq3aIJQ==/cricketgames.hitwicket.strategy-SeB_xyLYsTXG7JEeJAV6uQ==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #28 pc 0000000002c42334 (Task_FinishContinuations_m21B540B556CDABD76F5985DF83079F7F30C78174 at C:/Users/Manas/My Files/Hitwicket/Hitwicket_2020/Library/Bee/artifacts/Android/il2cppOutput/cpp/mscorlib9.cpp:21117) /data/app/~~7VCt-yB1V4KWTvESq3aIJQ==/cricketgames.hitwicket.strategy-SeB_xyLYsTXG7JEeJAV6uQ==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #29 pc 00000000028ef2b0 (Task_1_TrySetResult_m48D840CA62136D390874D338ED6E57849E33490B_gshared at C:/Users/Manas/My Files/Hitwicket/Hitwicket_2020/Library/Bee/artifacts/Android/il2cppOutput/cpp/Generics92.cpp:16639) /data/app/~~7VCt-yB1V4KWTvESq3aIJQ==/cricketgames.hitwicket.strategy-SeB_xyLYsTXG7JEeJAV6uQ==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #30 pc 0000000001ba13f8 (AsyncTaskMethodBuilder_1_SetResult_m7814370659E8E3550E3A679C1F2F516592198487_gshared at C:/Users/Manas/My Files/Hitwicket/Hitwicket_2020/Library/Bee/artifacts/Android/il2cppOutput/cpp/Generics10.cpp:11944) /data/app/~~7VCt-yB1V4KWTvESq3aIJQ==/cricketgames.hitwicket.strategy-SeB_xyLYsTXG7JEeJAV6uQ==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #31 pc 0000000002ec6c94 (AsyncTaskMethodBuilder_1_SetResult_m7814370659E8E3550E3A679C1F2F516592198487(AsyncTaskMethodBuilder_1_t7BA09CA1BB279AA706D98A618EE11038FD1985F7*, Nullable_1_tCF32C56A2641879C053C86F273C0C6EC1B40BC28, MethodInfo const*) at C:/Users/Manas/My Files/Hitwicket/Hitwicket_2020/Library/Bee/artifacts/Android/il2cppOutput/cpp/System.cpp:6612) /data/app/~~7VCt-yB1V4KWTvESq3aIJQ==/cricketgames.hitwicket.strategy-SeB_xyLYsTXG7JEeJAV6uQ==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #32 pc 0000000002c32a4c (ContextCallback_Invoke_m872CCCD40428B88C2612772491BE5157895B5F61_inline(ContextCallback_tE8AFBDBFCC040FDA8DA8C1EEFE9BD66B16BDA007*, Il2CppObject*, MethodInfo const*) at C:/Users/Manas/My Files/Hitwicket/Hitwicket_2020/Library/Bee/artifacts/Android/il2cppOutput/cpp/mscorlib8.cpp:32302) /data/app/~~7VCt-yB1V4KWTvESq3aIJQ==/cricketgames.hitwicket.strategy-SeB_xyLYsTXG7JEeJAV6uQ==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #33 pc 0000000002ae3d88 (MoveNextRunner_Run_m86395F123FB453F2B5FF89CE1E98532F7426F9DE at C:/Users/Manas/My Files/Hitwicket/Hitwicket_2020/Library/Bee/artifacts/Android/il2cppOutput/cpp/mscorlib17.cpp:28524) /data/app/~~7VCt-yB1V4KWTvESq3aIJQ==/cricketgames.hitwicket.strategy-SeB_xyLYsTXG7JEeJAV6uQ==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #34 pc 0000000002c442b0 (Action_Invoke_m7126A54DACA72B845424072887B5F3A51FC3808E_inline(Action_tD00B0A84D7945E50C2DFFC28EFEE6ED44ED2AD07*, MethodInfo const*) at C:/Users/Manas/My Files/Hitwicket/Hitwicket_2020/Library/Bee/artifacts/Android/il2cppOutput/cpp/mscorlib9.cpp:34500) /data/app/~~7VCt-yB1V4KWTvESq3aIJQ==/cricketgames.hitwicket.strategy-SeB_xyLYsTXG7JEeJAV6uQ==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #35 pc 0000000002c42334 (Task_FinishContinuations_m21B540B556CDABD76F5985DF83079F7F30C78174 at C:/Users/Manas/My Files/Hitwicket/Hitwicket_2020/Library/Bee/artifacts/Android/il2cppOutput/cpp/mscorlib9.cpp:21117) /data/app/~~7VCt-yB1V4KWTvESq3aIJQ==/cricketgames.hitwicket.strategy-SeB_xyLYsTXG7JEeJAV6uQ==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #36 pc 00000000028f5fd0 (Task_1_TrySetResult_mE67D469AB00E1958DECAE0D97B932BC43B9C2D4F_gshared at C:/Users/Manas/My Files/Hitwicket/Hitwicket_2020/Library/Bee/artifacts/Android/il2cppOutput/cpp/Generics92.cpp:21775) /data/app/~~7VCt-yB1V4KWTvESq3aIJQ==/cricketgames.hitwicket.strategy-SeB_xyLYsTXG7JEeJAV6uQ==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #37 pc 0000000001ba7ac8 (AsyncTaskMethodBuilder_1_SetResult_mC4BAC830514CDB105730FF11F7744A4424A4009B_gshared at C:/Users/Manas/My Files/Hitwicket/Hitwicket_2020/Library/Bee/artifacts/Android/il2cppOutput/cpp/Generics10.cpp:16471) /data/app/~~7VCt-yB1V4KWTvESq3aIJQ==/cricketgames.hitwicket.strategy-SeB_xyLYsTXG7JEeJAV6uQ==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #38 pc 0000000002ecb52c (AsyncTaskMethodBuilder_1_SetResult_mC4BAC830514CDB105730FF11F7744A4424A4009B(AsyncTaskMethodBuilder_1_tF0BDBC54D5EB433C1205550D0F9819F43C9D4019*, int, MethodInfo const*) at C:/Users/Manas/My Files/Hitwicket/Hitwicket_2020/Library/Bee/artifacts/Android/il2cppOutput/cpp/System.cpp:6903) /data/app/~~7VCt-yB1V4KWTvESq3aIJQ==/cricketgames.hitwicket.strategy-SeB_xyLYsTXG7JEeJAV6uQ==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #39 pc 0000000002c32a4c (ContextCallback_Invoke_m872CCCD40428B88C2612772491BE5157895B5F61_inline(ContextCallback_tE8AFBDBFCC040FDA8DA8C1EEFE9BD66B16BDA007*, Il2CppObject*, MethodInfo const*) at C:/Users/Manas/My Files/Hitwicket/Hitwicket_2020/Library/Bee/artifacts/Android/il2cppOutput/cpp/mscorlib8.cpp:32302) /data/app/~~7VCt-yB1V4KWTvESq3aIJQ==/cricketgames.hitwicket.strategy-SeB_xyLYsTXG7JEeJAV6uQ==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #40 pc 0000000002ae3d88 (MoveNextRunner_Run_m86395F123FB453F2B5FF89CE1E98532F7426F9DE at C:/Users/Manas/My Files/Hitwicket/Hitwicket_2020/Library/Bee/artifacts/Android/il2cppOutput/cpp/mscorlib17.cpp:28524) /data/app/~~7VCt-yB1V4KWTvESq3aIJQ==/cricketgames.hitwicket.strategy-SeB_xyLYsTXG7JEeJAV6uQ==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #41 pc 0000000002c442b0 (Action_Invoke_m7126A54DACA72B845424072887B5F3A51FC3808E_inline(Action_tD00B0A84D7945E50C2DFFC28EFEE6ED44ED2AD07*, MethodInfo const*) at C:/Users/Manas/My Files/Hitwicket/Hitwicket_2020/Library/Bee/artifacts/Android/il2cppOutput/cpp/mscorlib9.cpp:34500) /data/app/~~7VCt-yB1V4KWTvESq3aIJQ==/cricketgames.hitwicket.strategy-SeB_xyLYsTXG7JEeJAV6uQ==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #42 pc 0000000002c42334 (Task_FinishContinuations_m21B540B556CDABD76F5985DF83079F7F30C78174 at C:/Users/Manas/My Files/Hitwicket/Hitwicket_2020/Library/Bee/artifacts/Android/il2cppOutput/cpp/mscorlib9.cpp:21117) /data/app/~~7VCt-yB1V4KWTvESq3aIJQ==/cricketgames.hitwicket.strategy-SeB_xyLYsTXG7JEeJAV6uQ==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #43 pc 00000000028f5fd0 (Task_1_TrySetResult_mE67D469AB00E1958DECAE0D97B932BC43B9C2D4F_gshared at C:/Users/Manas/My Files/Hitwicket/Hitwicket_2020/Library/Bee/artifacts/Android/il2cppOutput/cpp/Generics92.cpp:21775) /data/app/~~7VCt-yB1V4KWTvESq3aIJQ==/cricketgames.hitwicket.strategy-SeB_xyLYsTXG7JEeJAV6uQ==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #44 pc 0000000001b43c8c (U3CU3Ec_U3C_cctorU3Eb__4_0_mA2CBCE5676C3946BC3ACD492C10FA56F20300DAB_gshared at C:/Users/Manas/My Files/Hitwicket/Hitwicket_2020/Library/Bee/artifacts/Android/il2cppOutput/cpp/Generics.cpp:22412) /data/app/~~7VCt-yB1V4KWTvESq3aIJQ==/cricketgames.hitwicket.strategy-SeB_xyLYsTXG7JEeJAV6uQ==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #45 pc 0000000002c38528 (InterfaceActionInvoker0::Invoke(unsigned short, Il2CppClass*, Il2CppObject*) at C:/Users/Manas/My Files/Hitwicket/Hitwicket_2020/Library/Bee/artifacts/Android/il2cppOutput/cpp/mscorlib9.cpp:171) /data/app/~~7VCt-yB1V4KWTvESq3aIJQ==/cricketgames.hitwicket.strategy-SeB_xyLYsTXG7JEeJAV6uQ==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #46 pc 0000000000e1d1f4 (RuntimeInvoker_FalseByte_t94D9231AC217BE4D2E004C4CD32DF6D099EA41A3(void (*)(), MethodInfo const*, void*, void**, void*) at C:/Users/Manas/My Files/Hitwicket/Hitwicket_2020/Library/Bee/artifacts/Android/il2cppOutput/cpp/Il2CppInvokerTable.cpp:137034) /data/app/~~7VCt-yB1V4KWTvESq3aIJQ==/cricketgames.hitwicket.strategy-SeB_xyLYsTXG7JEeJAV6uQ==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #47 pc 0000000000f04ca8 (il2cpp::vm::Runtime::InvokeWithThrow(MethodInfo const*, void*, void**) at C:/Program Files/Unity/Hub/Editor/2021.3.4f1/Editor/Data/il2cpp/libil2cpp/vm/Runtime.cpp:586) /data/app/~~7VCt-yB1V4KWTvESq3aIJQ==/cricketgames.hitwicket.strategy-SeB_xyLYsTXG7JEeJAV6uQ==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #48 pc 0000000000ebf768 (worker_thread(void*) at C:/Program Files/Unity/Hub/Editor/2021.3.4f1/Editor/Data/il2cpp/libil2cpp/mono/ThreadPool/ThreadPoolWorkerThread.cpp:249) /data/app/~~7VCt-yB1V4KWTvESq3aIJQ==/cricketgames.hitwicket.strategy-SeB_xyLYsTXG7JEeJAV6uQ==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #49 pc 0000000000ea29e0 (il2cpp::vm::ThreadStart(void*) at C:/Program Files/Unity/Hub/Editor/2021.3.4f1/Editor/Data/il2cpp/libil2cpp/vm/Thread.cpp:701) /data/app/~~7VCt-yB1V4KWTvESq3aIJQ==/cricketgames.hitwicket.strategy-SeB_xyLYsTXG7JEeJAV6uQ==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #50 pc 0000000000e9501c (il2cpp::eek:s::Thread::RunWrapper(void*) at C:/Program Files/Unity/Hub/Editor/2021.3.4f1/Editor/Data/il2cpp/libil2cpp/os/Thread.cpp:201) /data/app/~~7VCt-yB1V4KWTvESq3aIJQ==/cricketgames.hitwicket.strategy-SeB_xyLYsTXG7JEeJAV6uQ==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #51 pc 0000000000ebe0f0 (il2cpp::eek:s::ThreadImpl::ThreadStartWrapper(void*) at C:/Program Files/Unity/Hub/Editor/2021.3.4f1/Editor/Data/il2cpp/libil2cpp/os/Posix/ThreadImpl.cpp:125) /data/app/~~7VCt-yB1V4KWTvESq3aIJQ==/cricketgames.hitwicket.strategy-SeB_xyLYsTXG7JEeJAV6uQ==/lib/arm64/libil2cpp.so (BuildId: 3d2007f5358661a549e43a098289f7375a7dafa6)
    #52 pc 00000000000eb6b0 (libc.so not found) /apex/com.android.runtime/lib64/bionic/libc.so (__pthread_start(void*)+264) (BuildId: ddfcb9d4fd7d4d5ecb9d226c1e8e24dd)
    #53 pc 000000000008af4c (libc.so not found) /apex/com.android.runtime/lib64/bionic/libc.so (__start_thread+64) (BuildId: ddfcb9d4fd7d4d5ecb9d226c1e8e24dd)
     
  13. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,931
    Yes, these crashes look like they are caused by the same issue we have been discussing in this thread.
     
  14. Carpet_Head

    Carpet_Head

    Joined:
    Nov 27, 2014
    Posts:
    258
    any nerdy tech details about what caused the issue for those interested?
     
  15. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,931
    It is not too nerdy, but basically the problem was a weakness in the code which removes a thread from the threadpool's list of threads that are available to pick up work. There were two different parts of the code which could have done this. One of them incorrectly assumed that it would always find that thread in the list of threads, if it did not, then it would remove the wrong thread!

    So there is a race condition between the two parts of the code which could do this. Sometimes that triggered the bug, where the wrong thread was removed. Later, that thread would be in trouble because it was incorrectly removed.

    Then finally something recently triggered this to start happening more often, so the weakness was exposed as a bug in real projects. We're not sure what change has exposed this, but we were able to correct the root cause.
     
    eccece likes this.
  16. manasgupta435

    manasgupta435

    Joined:
    Jul 11, 2020
    Posts:
    9
  17. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,931
    Unfortunately no, it looks like the release went out before we were able to get this fix merged. It should make 2021.3.11 though.
     
  18. epernigo

    epernigo

    Joined:
    Sep 24, 2019
    Posts:
    11
    Hi, @JoshPeterson , Is this issue fixed in 2020.3.39f1? thank you
     
  19. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,931
    epernigo likes this.
  20. manasgupta435

    manasgupta435

    Joined:
    Jul 11, 2020
    Posts:
    9
    Hi @JoshPeterson, Will this fix be released in 2021.3.11 for sure? Crash rate is increasing rapidly in our app and now it is way past the threshold. I request you get this fix released as soon as possible.
     
  21. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,931
    It looks like this change won't make 2021.3.11, unfortunately. I think it should be good for 2021.3.12 though.
     
  22. Carpet_Head

    Carpet_Head

    Joined:
    Nov 27, 2014
    Posts:
    258
    eagerly awaiting the fix in 2022.1.x - do you know when it will land?
     
  23. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,931
    My guess now is that it will make 2022.1.21, scheduled to be released around October 19.
     
    Carpet_Head likes this.
  24. tnt1

    tnt1

    Joined:
    Jul 16, 2018
    Posts:
    5
    Is there any news on when we have to expect the fix for this issue? The estimated version is already quite some days overdue. Also eagerly waiting for this.
     
  25. tnt1

    tnt1

    Joined:
    Jul 16, 2018
    Posts:
    5
  26. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,931
    eccece likes this.
  27. tnt1

    tnt1

    Joined:
    Jul 16, 2018
    Posts:
    5
    Can we expect a 2022.x version also soon?
     
  28. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,931
  29. tnt1

    tnt1

    Joined:
    Jul 16, 2018
    Posts:
    5
    How about the version 2022.1.21 mentioned before? Or should we expect a 2022.2.0 non beta to have the fix first in a final release?
     
  30. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,931
    We're still working on landing the fix in a 2022.1 version. I would expect it to be in 2022.1.23, scheduled to be published on November 9.
     
  31. tnt1

    tnt1

    Joined:
    Jul 16, 2018
    Posts:
    5
    Thanks for the answer. Unfortunately we have to find a different solution in meantime than. Our crash rates on google are quite high due to this currently.

    Is there any possible workaround known? We use 2022.1.17 and Unitask plugin.
     
  32. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    6,931
    Yes, I'll ping you directly.
     
  33. mdienersberger

    mdienersberger

    Joined:
    Oct 26, 2022
    Posts:
    2
    @JoshPeterson this is my actual unity account can you please continue the discussion with this one?
     
  34. Carpet_Head

    Carpet_Head

    Joined:
    Nov 27, 2014
    Posts:
    258
    I'm also curious about workarounds
     
  35. mdienersberger

    mdienersberger

    Joined:
    Oct 26, 2022
    Posts:
    2
    This week there was a release of Unity version 2022.1.23 that should have fixed it now.
     
  36. in_2358

    in_2358

    Joined:
    Nov 21, 2019
    Posts:
    1
    Our games have millions of DAUs, and our users experience tens of thousands of crashes related to threading in Unity. I will share some of the stack traces of crashes. Can you verify if there is any solution to these crashes? We are an enterprise customer; our internal support ticket ID is #1701496.

    The thread-related crashes started after we upgraded our game from Unity 2019 LTS to Unity 2021 LTS. Currently, we are using Unity 2021.3.21.

    Crash 1
    - We are making a simple WebClient call in BackgroundWorker thread:

    Code (CSharp):
    1. backtrace:
    2. #00 (LinkedList_1_AddLast_mF5239C871EADC44D51C6B621592A9CAC43449A2E_gshared)
    3. #01 (ServicePointScheduler_SendRequest_m6915605C55AC898BF656A67E50AE2F848C6B1E33)
    4. #02 (ServicePoint_SendRequest_mB271F49FB5B3DB5DA556253585CC150A51E7C9CA)
    5. #03 (HttpWebRequest_SendRequest_m9CFD21B34BCCA09123225ED98ED24DF1984B3130)
    6. #04 (HttpWebRequest_MyGetRequestStreamAsync_mE060EBB783E0B28365D19128AA6ADC552957C49B)
    7. #05 (HttpWebRequest_RunWithTimeout_TisRuntimeObject_mCC3E5591B883EC68AC707F1988456AC714AA0A6C_gshared)
    8. #06 (HttpWebRequest_GetRequestStream_mE365A429EFA4B39073F60BFB522F7B098AF94C3A)
    9. #07 (WebClient_UploadBits_mE8E6CDBAB846A577C6562B4ED649FD197F7972DC)
    10. #08 (WebClient_UploadDataInternal_m2970A799BCDB5458AB8E5558F0937FD37A0B2BFF)
    11. #09 (WebClient_UploadData_m3EBEADD2D37BD7568F14128BB492D30D44E4F601)
    12. #10 (BackgroundWebClientRequest_BackgroundWorker_DoWork_mE0C7734E058CB0FBE1479A73654A38C4A7F0A7BF)
    13. #11 (BackgroundWorker_WorkerThreadStart_mE143779D088D9ACF023342E95084D19CA3F794E1)
    14. #12 (Task_Execute_m4FBCBCE932B924AD239A2448AED5708B6D14A7F1)
    15. #13 (ExecutionContext_RunInternal_mC37E85D6FC972DF219A00FE1EE34329D1D63CC94)
    16. #14 (Task_ExecuteWithThreadLocal_mD3F4E7ADBBC473EAF7C24B3A785EBB452D6A76BE)
    17. #15 (Task_ExecuteEntry_m735BD89474854D6EAA2ED929D6765AF1B4AEDCDF)
    18. #16 (ThreadPoolWorkQueue_Dispatch_mEAAC936B3756DAA856BA3A4D976A98A3184CDC87)
    19. #17 (RuntimeInvoker_FalseByte_t94D9231AC217BE4D2E004C4CD32DF6D099EA41A3(void (*)(), MethodInfo const*, void*, void**, void*))
    20. #18 (il2cpp::vm::Runtime::InvokeWithThrow(MethodInfo const*, void*, void**))
    21. #19 (worker_thread(void*))
    22. #20 (il2cpp::vm::ThreadStart(void*))
    23. #21 (il2cpp::os::Thread::RunWrapper(void*))
    24. #22 (il2cpp::os::ThreadImpl::ThreadStartWrapper(void*))
    25. #23 /system/lib64/libc.so (__pthread_start(void*)+208)
    26. #24 /system/lib64/libc.so (__start_thread+16)
    Crash 2
    - WebClient using in BackgroundWorker:

    Code (CSharp):
    1. backtrace:
    2.   #00  (String_FormatHelper_m9D0E826CCC406EADF91AF3B83FBC5DD012C81192)
    3.   #01  (String_Format_mA8DBB4C2516B9723C5A41E6CB1E2FAF4BBE96DD8)
    4.   #02  (RequestHeaderHelper_GetVersionHeaderString_mCEA631BF664150EBB4C1E80CDE4F8242E8738533)
    5.   #03  (RequestHeaderHelper_AddCommonHeaders_m35335F39498309C0B2CFD4FB14C7B30CE069CD76)
    6.   #04  (BackgroundWebClientRequest_BackgroundWorker_DoWork_mE0C7734E058CB0FBE1479A73654A38C4A7F0A7BF)
    7.   #05  (BackgroundWorker_WorkerThreadStart_mE143779D088D9ACF023342E95084D19CA3F794E1)
    8.   #06  (Task_Execute_m4FBCBCE932B924AD239A2448AED5708B6D14A7F1)
    9.   #07  (ExecutionContext_RunInternal_mC37E85D6FC972DF219A00FE1EE34329D1D63CC94)
    10.   #08  (Task_ExecuteWithThreadLocal_mD3F4E7ADBBC473EAF7C24B3A785EBB452D6A76BE)
    11.   #09  (Task_ExecuteEntry_m735BD89474854D6EAA2ED929D6765AF1B4AEDCDF)
    12.   #10  (ThreadPoolWorkQueue_Dispatch_mEAAC936B3756DAA856BA3A4D976A98A3184CDC87)
    13.   #11  (RuntimeInvoker_FalseByte_t94D9231AC217BE4D2E004C4CD32DF6D099EA41A3(void (*)(), MethodInfo const*, void*, void**, void*))
    14.   #12  (il2cpp::vm::Runtime::InvokeWithThrow(MethodInfo const*, void*, void**))
    15.   #13  (worker_thread(void*))
    16.   #14  (il2cpp::vm::ThreadStart(void*))
    17.   #15  (il2cpp::os::Thread::RunWrapper(void*))
    18.   #16  (il2cpp::os::ThreadImpl::ThreadStartWrapper(void*))
    19.   #17  /system/lib64/libc.so (__pthread_start(void*)+36)
    20.   #18  /system/lib64/libc.so (__start_thread+68)
    Crash 3
    - Using WebClient with async await and Task system:

    Code (CSharp):
    1. backtrace:
    2.   #00 (LinkedList_1_Remove_m6548CB1A0181356C321DB1F226FC0598378F7997_gshared+38580)
    3.   #01 (ConnectionGroup_RemoveConnection_m4E4DFFE07C6E0005F796957B2BA4D5F4D3D6241B+8631)
    4.   #02 (ServicePointScheduler_OperationCompleted_m442DE8063CFDFCF6C7DDEF30EB2C8BF79ED95C00+29387)
    5.   #03 (U3CRunSchedulerU3Ed__32_MoveNext_mA08FEA7A8A1F2C0B268B36676228CFDF37B007C1+31585)
    6.   #04 (ExecutionContext_RunInternal_mC37E85D6FC972DF219A00FE1EE34329D1D63CC94+34502)
    7.   #05 (MoveNextRunner_Run_m86395F123FB453F2B5FF89CE1E98532F7426F9DE+22974)
    8.   #06 (AwaitTaskContinuation_RunOrScheduleAction_mAB1E447337DDFFE6824A12BBEDE6324AA2823800+38117)
    9.   #07 (Task_FinishContinuations_m21B540B556CDABD76F5985DF83079F7F30C78174+14196)
    10.   #08 (Task_1_TrySetResult_m2EE766FD3F76F4824990F4A93ED1F7253ECE014C_gshared+15090)
    11.   #09 (CompleteOnInvokePromise_Invoke_mF5F31F1D9FD06E73EA1239A32B3B14A3D925543E+5674)
    12.   #10 (ThreadPoolWorkQueue_Dispatch_mEAAC936B3756DAA856BA3A4D976A98A3184CDC87+127)
    13.   #11 (RuntimeInvoker_FalseByte_t94D9231AC217BE4D2E004C4CD32DF6D099EA41A3(void (*)(), MethodInfo const*, void*, void**, void*)+150972)
    14.   #12 (il2cpp::vm::Runtime::InvokeWithThrow(MethodInfo const*, void*, void**)+588)
    15.   #13 (worker_thread(void*)+250)
    16.   #14 (il2cpp::vm::ThreadStart(void*)+701)
    17.   #15 (il2cpp::os::Thread::RunWrapper(void*)+201)
    18.   #16 (il2cpp::os::ThreadImpl::ThreadStartWrapper(void*)+125)
    19.   #17 /apex/com.android.runtime/lib64/bionic/libc.so (__pthread_start(void*)+208)
    20.   #18 /apex/com.android.runtime/lib64/bionic/libc.so (__start_thread+64)
    Crash 4
    - We got crashes when using the BestHttp plugin, which creates a new Thread object to manage HTTP connections.
    Code (CSharp):
    1. backtrace:
    2.   #00 (il2cpp::vm::Class::Init(Il2CppClass*)+1548)
    3.   #01 (il2cpp::vm::Type::GetNameInternal(std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char>>&, Il2CppType const*, Il2CppTypeNameFormat, bool)+683)
    4.   #02 (il2cpp::vm::Type::GetName(Il2CppType const*, Il2CppTypeNameFormat)+772)
    5.   #03 (il2cpp::vm::RaiseExceptionForNotFoundInterface(Il2CppClass const*, Il2CppClass const*, unsigned short)+49)
    6.   #04 (il2cpp::vm::ClassInlines::GetInterfaceInvokeDataFromVTableSlowPath(Il2CppObject*, Il2CppClass const*, unsigned short)+98)
    7.   #05 (SecureRandom_NextBytes_mC14C8CB2D4AABEAF9D678AE0901DE3E10D7E4731+56)
    8.   #06 (SecureRandom_GetNextBytes_mE3598835B15F00C81A9AC93E9023B5476457CBB9+21)
    9.   #07 (SecureRandom_CreatePrng_mB042E1D091C307F7AD897CEB631AA63332574187+17549)
    10.   #08 (SecureRandom__ctor_mEB890447CE4F4FC7FFA1B78F545A1318A681852E+17714)
    11.   #09 (DefaultTls13Client__ctor_m9F9E6165C81A0DAD764FE4B3303002BFFB489226+10189)
    12.   #10 (HTTPManager_DefaultTlsClientFactory_mFE5C452F1C3ECDFA58100AC2DC71CC0BE03F1B57+12020)
    13.   #11 (TCPConnector_Connect_mFF2188E178DCCAA9AFE787AAE66D7396A25E6C8B)
    14.   #12 (HTTPConnection_ThreadFunc_mE79EAE3CE7AEE073E71DCC2506B7BF6F34996340+6698)
    15.   #13 (Task_Execute_m4FBCBCE932B924AD239A2448AED5708B6D14A7F1+22)
    16.   #14 (ExecutionContext_RunInternal_mC37E85D6FC972DF219A00FE1EE34329D1D63CC94+34502)
    17.   #15 (Task_ExecuteWithThreadLocal_mD3F4E7ADBBC473EAF7C24B3A785EBB452D6A76BE+16176)
    18.   #16 (Task_ExecuteEntry_m735BD89474854D6EAA2ED929D6765AF1B4AEDCDF+12870)
    19.   #17 (ExecutionContext_RunInternal_mC37E85D6FC972DF219A00FE1EE34329D1D63CC94+34502)
    20.   #18 (il2cpp::vm::Runtime::InvokeWithThrow(MethodInfo const*, void*, void**)+578)
    21.   #19 (il2cpp::vm::Runtime::Invoke(MethodInfo const*, void*, void**, Il2CppException**)+564)
    22.   #20 (il2cpp::icalls::mscorlib::System::Threading::ThreadStart(void*)+64)
    23.   #21 (il2cpp::os::Thread::RunWrapper(void*)+201)
    24.   #22 (il2cpp::os::ThreadImpl::ThreadStartWrapper(void*)+125)
    25.   #23 /apex/com.android.runtime/lib64/bionic/libc.so (__pthread_start(void*)+64)
    26.   #24 /apex/com.android.runtime/lib64/bionic/libc.so (__start_thread+64)

    Crash 5
    - Might be irrelevant, but increased when we are using BackgroundWorker with WebClient.
    Code (CSharp):
    1. backtrace:
    2.   Unknown
    3.   #01 (AGCThread::Run())
    4.   #02 (AGCThread::RunThread(void*))
    5.   #03 (Thread::RunThreadWrapper(void*))
    6.   #04 /apex/com.android.runtime/lib64/bionic/libc.so (__pthread_start(void*)+64)
    7.   #05 /apex/com.android.runtime/lib64/bionic/libc.so (__start_thread+64)
    Summary

    1) We got crashes when using HttpClient using async / await and Task.Run
    2) We got crashes when using WebClient using Task.Run
    3) We got crashes if we use a third-party HTTP connection library called BestHTTP, which utilizes a new Thread.
    4) We got crashes when using HTTPClient using ThreadPoolWorker.
    5) We got crashes when using WebClient using BackgroundWorker. (Least number of crashes, but still there are some)

    Right now, we are trying to solve the issue through Unity support. However, we feel stuck. Does anyone have any solution?

    Thank you
     
    moxisgames likes this.