Search Unity

Bug There is a bug and the crash in libGLESv2_mtk.so on some devices

Discussion in 'General Graphics' started by vladrybak, Apr 2, 2022.

  1. eggbbq

    eggbbq

    Joined:
    Jun 7, 2022
    Posts:
    15
    It still doesn't work. I spent a week to test almost all various combinations.(unity2021~2022, opengl es 2~3, Linear, Gamma, Vulkan...etc)
     
  2. JimmyAndLYF

    JimmyAndLYF

    Joined:
    Mar 10, 2021
    Posts:
    4
    Did you fix this crash?
     
  3. eggbbq

    eggbbq

    Joined:
    Jun 7, 2022
    Posts:
    15
    I tested it, and it's not related to multithreaded rendering. I'm not sure how to fix it. I submitted a bug report to Unity, but they rejected it. All I can do now is provide feedback here and continue testing it with new versions of Unity. I hope Unity will fix the issue soon.
    As a temporary solution, my project falls back to "gamma + OpenGL ES 2". Although this option has worse visual effects, it crashes less frequently.
     
  4. JimmyAndLYF

    JimmyAndLYF

    Joined:
    Mar 10, 2021
    Posts:
    4
    Fk unity
    Thank you very much for your reply
     
  5. JimmyAndLYF

    JimmyAndLYF

    Joined:
    Mar 10, 2021
    Posts:
    4
    Could you share your unity version, urp version with me? BTW, are u using any custom shaders? I am wondering whether custom shader for urp triggered this creepy bug
     
  6. eggbbq

    eggbbq

    Joined:
    Jun 7, 2022
    Posts:
    15
    Thank you for your advice.
    Project environment: Unity 2021.3.13f with URP 12.1.7.
    We have used many custom shaders. I tested this issue again in my project and found that it logs the same error message per frame, but it doesn't crash for hours this time. We have optimized this project a lot in the last few months, which is confusing.
    I will push this build to production and let users test it again.
     
  7. JimmyAndLYF

    JimmyAndLYF

    Joined:
    Mar 10, 2021
    Posts:
    4
    Thanks alot for your sharing. BTW have you tried auto graphic api checked?
     
  8. chisolo

    chisolo

    Joined:
    May 4, 2023
    Posts:
    2
    This result seems a litter weird to me.
    Why it still crash at libglesv2_mtk, when you use vulkan?
     
  9. liuxuan

    liuxuan

    Joined:
    Oct 13, 2014
    Posts:
    7
    I released my game to google play with "Unity 2021.3.5f1 URP + gamma + OpenGL ES 2 + disable Optimized Frame Pacing" 5 days ago, In the google play console statistics, the crash rate has dropped by more than 90% compared with the previous version (auto grahics API + Optimezed Fraem Pacing), but there are still a small number of crash statistics like this:
    backtrace:
    #00 pc 0x000000000019c9c8 /vendor/lib64/egl/libGLESv2_powervr.so
    #01 pc 0x000000000015fa48 /vendor/lib64/egl/libGLESv2_powervr.so
    #02 pc 0x00000000000aefe0 /vendor/lib64/egl/libGLESv2_powervr.so
    #03 pc 0x00000000000af70c /vendor/lib64/egl/libGLESv2_powervr.so (glClear+620)
    #04 pc 0x0000000000802b5c /data/app/~~3HRwJblHAj5FVT_tWjou2A==/com.xxx.zzz-ZCzq1-w-zVpakYzoB1vy1A==/split_config.arm64_v8a.apk
    #05 pc 0x00000000007f8cdc /data/app/~~3HRwJblHAj5FVT_tWjou2A==/com.xxx.zzz-ZCzq1-w-zVpakYzoB1vy1A==/split_config.arm64_v8a.apk
    #06 pc 0x0000000000922504 /data/app/~~3HRwJblHAj5FVT_tWjou2A==/com.xxx.zzz-ZCzq1-w-zVpakYzoB1vy1A==/split_config.arm64_v8a.apk
    #07 pc 0x000000000092a934 /data/app/~~3HRwJblHAj5FVT_tWjou2A==/com.xxx.zzz-ZCzq1-w-zVpakYzoB1vy1A==/split_config.arm64_v8a.apk
    #08 pc 0x00000000009221c4 /data/app/~~3HRwJblHAj5FVT_tWjou2A==/com.xxx.zzz-ZCzq1-w-zVpakYzoB1vy1A==/split_config.arm64_v8a.apk
    #09 pc 0x000000000034bf54 /data/app/~~3HRwJblHAj5FVT_tWjou2A==/com.xxx.zzz-ZCzq1-w-zVpakYzoB1vy1A==/split_config.arm64_v8a.apk
    #10 pc 0x00000000000af97c /apex/com.android.runtime/lib64/bionic/libc.so (__pthread_start(void*)+64)
    #11 pc 0x00000000000500d0 /apex/com.android.runtime/lib64/bionic/libc.so (__start_thread+64)
     
  10. eggbbq

    eggbbq

    Joined:
    Jun 7, 2022
    Posts:
    15
    Sorry!Vulkan caused more crashes, so I skipped that option. Therefore, I did not include any further error logs in the problem description.
     
  11. unity_E5rFZl_dL2dNLg

    unity_E5rFZl_dL2dNLg

    Joined:
    Feb 27, 2020
    Posts:
    1
    Hello, I got this issue too. Crash on Android 11. My Unity version is 2021.3.9f1. How can I fix it ?
     
  12. ZenithGames

    ZenithGames

    Joined:
    Mar 26, 2017
    Posts:
    15
    Any news Unity?
     
  13. ZenithGames

    ZenithGames

    Joined:
    Mar 26, 2017
    Posts:
    15
    Guys, I can provide a repro project but tell me where to put it. We can't fix it, you have to do it on your side. We've tested on multiple versions and it crashes on all of them:
    • 2021.3.23f1
    • 2023.1.0f1
    • 2022.3.2f1
    It happens only with URP.
    It's pretty ridiculous because the only, only workaround is to not to use URP at all and we had to migrate all the projects back. This PowerVR thing apparently affects 10% of the user base (and basically everybody else's user bases)
    We tested on Xiaomi Redmi 9 but I can provide a full list of devices it crashes on.
     
    joshuaFS and eggbbq like this.
  14. eggbbq

    eggbbq

    Joined:
    Jun 7, 2022
    Posts:
    15
    upload_2023-6-21_11-49-16.png
    User-perceived crash rate < 1.09%. It's too hard.
    Is anyone at Unity working on fixing this bug?
     
  15. jperry_oddgames

    jperry_oddgames

    Joined:
    Sep 18, 2017
    Posts:
    62
    Hey @ZenithGames , if you haven't already please submit a bug report through unity with the reproduction project attached and comment the report case number in this thread, tagging the unity staff. Seems you might be the only one with a reproducible so far so it would be great if you could help Unity fix this one.


    Side note: We're also experiencing this crash issue on many devices from builds on unity 2022.3.0f1 & 2022.3.4f1, and I believe also on earlier 2021.3.x versions. I'll post a crash stacktrace below in case it offers any insights but it seems like this thread already has as much info as I'm able to provide

    backtrace:
    #00 pc 0x000000000010f54c /vendor/lib/egl/libGLESv2_mtk.so
    #01 pc 0x000000000004efdd /vendor/lib/egl/libGLESv2_mtk.so
    #02 pc 0x00000000000c104f /vendor/lib/egl/libGLESv2_mtk.so
    #03 pc 0x0000000000085fa9 /vendor/lib/egl/libGLESv2_mtk.so
    #04 pc 0x000000000008653f /vendor/lib/egl/libGLESv2_mtk.so (glClear+474)
    #05 pc 0x0000000000787940 /data/app/~~5ugvJOcF_kHscgwRX9TW3A==/com.projectname/lib/arm/libunity.so (ApiGLES::Clear(unsigned int, ColorRGBAf const&, bool, float, int))
    #06 pc 0x000000000077bf38 /data/app/~~5ugvJOcF_kHscgwRX9TW3A==/com.projectname/lib/arm/libunity.so (gles::ClearCurrentFramebuffer(ApiGLES*, bool, bool, bool, ColorRGBAf const&, float, int, int))
    #07 pc 0x0000000000799804 /data/app/~~5ugvJOcF_kHscgwRX9TW3A==/com.projectname/lib/arm/libunity.so (GfxFramebufferGLES::Clear(GfxClearFlags, ColorRGBAf const&, float, int, bool))
    #08 pc 0x0000000000776150 /data/app/~~5ugvJOcF_kHscgwRX9TW3A==/com.projectname/lib/arm/libunity.so (GfxDeviceGLES::Clear(GfxClearFlags, ColorRGBAf const&, float, unsigned int))
    #09 pc 0x00000000009236db /data/app/~~5ugvJOcF_kHscgwRX9TW3A==/com.projectname/lib/arm/libunity.so (GfxDeviceWorker::RunCommand(ThreadedStreamBuffer&))
    #10 pc 0x00000000009233d1 /data/app/~~5ugvJOcF_kHscgwRX9TW3A==/com.projectname/lib/arm/libunity.so (GfxDeviceWorker::RunExt(ThreadedStreamBuffer&))
    #11 pc 0x000000000092314b /data/app/~~5ugvJOcF_kHscgwRX9TW3A==/com.projectname/lib/arm/libunity.so (GfxDeviceWorker::RunGfxDeviceWorker(void*))
    #12 pc 0x000000000036f28b /data/app/~~5ugvJOcF_kHscgwRX9TW3A==/com.projectname/lib/arm/libunity.so (Thread::RunThreadWrapper(void*))
    #13 pc 0x0000000000080923 /apex/com.android.runtime/lib/bionic/libc.so (__pthread_start(void*)+40)
    #14 pc 0x0000000000039d63 /apex/com.android.runtime/lib/bionic/libc.so (__start_thread+30)
     
  16. joshuaFS

    joshuaFS

    Joined:
    Jul 23, 2018
    Posts:
    25
    Well, we are running into this now too. I'm guessing a ton of developers are and maybe just don't even know it.

    Has anyone tried the early on suggestion to create a native rendering plugin and call glFinish relatively frequently to try and void the graphics driver bug? My guess is Unity is less than included to fix this much more because they see it as a bug in the graphics driver and so "not my problem". I'm tempted to do the native rendering plugin just to see, but was hoping someone had already tried it??
     
  17. joshuaFS

    joshuaFS

    Joined:
    Jul 23, 2018
    Posts:
    25
    We tried this in a beta testing group with some impacted users (at various intervals), and it did NOT solve the issue. It did initially look like it might help some, but if it is helping, it doesn't help much at all.

    I also did some digging through the PowerVR developer support forums, and it would seem similar things aren't uncommon. I even acquired a device with this potential issue (the Nokia C110), but as it turns out, that device is NOT impacted by this and our game runs just fine on it. I'm guessing that PowerVR (Imagination Technologies) has actually fixed this issue with their driver, and newer devices then don't have the problem. However, it is also apparent that older devices are not likely to get this update any time soon, if at all, as manufacturers just don't care. Furthermore, it is clear that not every game that is played on these devices has this problem, and so it seems equally clear that Unity could work around the problem if properly motivated to do so.

    Any chance anyone at Unity actually wants to care about this? Would sure be nice to not write off a non-trivial amount of users just because Unity can't work around something every other game engine seems to not have a problem with. Why do I regularly feel like I made the wrong call with using Unity...
     
    eggbbq and jperry_oddgames like this.
  18. joshuaFS

    joshuaFS

    Joined:
    Jul 23, 2018
    Posts:
    25
    Did you ever provide this to Unity so they can get this fixed?
     
  19. gurbrindersingh

    gurbrindersingh

    Joined:
    Feb 28, 2017
    Posts:
    32
    Do anyone know the fix? Or Unity team and the rest of the community have given up on it?
     
    joshuaFS likes this.
  20. joshuaFS

    joshuaFS

    Joined:
    Jul 23, 2018
    Posts:
    25
    There is no Unity fix, and there is apparently nobody at Unity that cares to address it either. Other game engines seem to run fine on these devices, so solutions and fixes are possible. This, and other issues, make it appear that making an engine that runs reliably is not a priority at Unity. Sorry, I'm a bit bitter here with the other recent fiasco, and several other ongoing open issues that don't appear to ever get fixed.
     
    jperry_oddgames likes this.
  21. optimise

    optimise

    Joined:
    Jan 22, 2014
    Posts:
    2,129
    Hi @Tomas1856, is someone still working on this issue?
     
  22. Tomas1856

    Tomas1856

    Unity Technologies

    Joined:
    Sep 21, 2012
    Posts:
    3,917
    There's no active bug, thus I don't think anyone is fixing this.
     
  23. troylightfox

    troylightfox

    Joined:
    Feb 1, 2023
    Posts:
    30
    This is occurring for us at a very high rate on the PowerVR 8320 devices with the Mediatek driver on Unity 2022.3.7f1:

    #00 pc 00000000001052dc /vendor/lib64/egl/libGLESv2_mtk.so (BuildId: e4df3d68e05df74645dcb8286d896e8b)
    #01 pc 000000000010c9d4 /vendor/lib64/egl/libGLESv2_mtk.so (BuildId: e4df3d68e05df74645dcb8286d896e8b)
    #02 pc 0000000000072a54 /vendor/lib64/egl/libGLESv2_mtk.so (BuildId: e4df3d68e05df74645dcb8286d896e8b)
    #03 pc 0000000000033b18 /vendor/lib64/libsrv_um.so (BuildId: 322a1adcc56edc459a2c8250fa7a2624)
    #04 pc 00000000000ed138 /apex/com.android.runtime/lib64/bionic/libc.so (__pthread_start(void*)+264) (BuildId: e210c4c63224a2c0f652a1c781aab289)
    #05 pc 000000000008b2a0 /apex/com.android.runtime/lib64/bionic/libc.so (__start_thread+64) (BuildId: e210c4c63224a2c0f652a1c781aab289)

    Crashes always occur at low free RAM, possibly consistent with the framebuffer leak mentioned earlier in the thread.

    For us, this is triggered specifically when calling
    commandBuffer.DrawRenderer(toRender, material, i, 0); bound to a RenderTexture.

    We can set up the command buffer and clear it with no issues but as soon as we start calling DrawRenderer we have a very high likelihood of crashing.

    I've reached out to Samsung Developer Support with a A7 tab reproduced crash to see if they can at least symbolicate it.

    Things we've tried that didn't work:
    - Injecting a glFlush at the end of every frame
    - Sharing/re-using RenderTextures
    - Drawing once to a RenderTexture and copying to a regular texture so we don't call CommandBuffer.DrawRenderer repeatedly
    - Limiting our commandBuffer.DrawRenderer usage to just MeshRenderers(not skinned/particles)

    Anyone have any ideas or suggestions for a workaround? We'd like to continue to using CommandBuffers if possible due to the camera stack being very slow/heavy in URP.
     
  24. joshuaFS

    joshuaFS

    Joined:
    Jul 23, 2018
    Posts:
    25
    It is happening to a lot of us, but Unity clearly won't touch it. People have tried opening a bug report with Unity before, but since you will never get a reproducible project that crashes on all hardware (or whatever hardware Unity may be using to run their tests), I doubt they will do anything but just close it.

    If you have a project that can reproduce this, then definitely submit a bug report with that project. You should make a note about what specific device hardware is used to reproduce the issue, to hopefully get Unity to use that hardware and actually test it. They don't seem interested in doing that right now.

    That said, since this does seem to technically be a driver bug on that specific hardware, I don't think Unity is going to be patching around it. We have all been waiting for that for a while now, and they have done absolutely nothing to actually address it. FWIW, It does seem other game engines have either fixed this or don't run into it.
     
  25. troylightfox

    troylightfox

    Joined:
    Feb 1, 2023
    Posts:
    30
    I tried to produce a minimal repro project, but it didn't crash. I'm happy to provide an APK that will crash 95% of the time within seconds on common devices like the Galaxy A7 Tab if it will actually be looked at it.

    Considering this problematic chipset is present on >10% of asian market phones, it's worth fixing. Does Unity's mobile rendering team have contacts in PowerVR/Mediatek/Samsung they could work with to get this thing figured out? At least getting libGLESv2_mtk.so symbols so we can see what is even crashing.

    I have threads open in a few places like the Imgtec developer forums, and a ticket open with Samsung developer support. Will report back if I get more information.
     
  26. joshuaFS

    joshuaFS

    Joined:
    Jul 23, 2018
    Posts:
    25
    If you have one, do it. It can't hurt to try. Who knows if they will work on it, but at least you would have given them the best chance possible to get it fixed.
     
  27. troylightfox

    troylightfox

    Joined:
    Feb 1, 2023
    Posts:
    30
    Update, my colleague Ryan discovered a workaround.

    Specifically, one shader we had this:
    CBUFFER_START(UnityPerMaterial)
    float _Outline;
    float4 _OutlineColor;
    CBUFFER_END

    Removing the CBUFFER_START/CBUFFER_END resulted in these moving in the compiled shader from:

    Constant Buffer "UnityPerMaterial" (48 bytes) on slot 2 {
    Float _Outline at 0
    Vector4 _OutlineColor at 16
    }

    to

    Constant Buffer "$Globals" (2080 bytes) on slot 0 {
    Matrix4x4 unity_MatrixVP at 1216
    Float _Outline at 2032
    Vector4 _OutlineColor at 2048
    }

    I never did get confirmation from Imgtec/Samsung on what the symbolicated callstack was, but we're (for now) crash free.

    What does this mean? It's hard to say for sure, but maybe check your shaders and compiled shader graphs for any usages of "UnityPerMaterial" especially when drawing to a RenderTexture and avoid them on the PowerVR GE8320.
     
    Last edited: Oct 30, 2023
  28. joshuaFS

    joshuaFS

    Joined:
    Jul 23, 2018
    Posts:
    25
    If you could submit a bug report that included both a repro case, and this which "fixes" it as a work around, I'm sure it would help. Did you ever create a bug report? I'll see if we can try this in our project, but it does seem strange that this would prevent the issue.
     
  29. Danielius_Mr_Bee

    Danielius_Mr_Bee

    Unity Technologies

    Joined:
    Jun 7, 2022
    Posts:
    1
  30. nxtboyIII

    nxtboyIII

    Joined:
    Jun 4, 2015
    Posts:
    281
    I'm also facing this issue, except I'm not using URP, using the built-in render pipeline. It also happens on devices with PowerVR GPU
     
  31. e_cheshenkov

    e_cheshenkov

    Joined:
    Nov 28, 2018
    Posts:
    2
    Hello! Maybe this information can help you for finding solution.

    If you add an active cube(or other any mesh) to the scene within the camera's visibility range, the errors stop appearing.
    Turning occlusion on/off on the camera has no effect. The object must be within the camera's view.
    100% reproduced on empty (single scene with only camera and lightning) URP project with Moto G Pure.

    Thank you and look forward to fixes in future versions of Unity!
     
  32. unity_69B8894E87F8C95B9B0E

    unity_69B8894E87F8C95B9B0E

    Joined:
    Jun 22, 2023
    Posts:
    1
    Hello. Would you please describe more? What do you mean by active cube