Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. We have updated the language to the Editor Terms based on feedback from our employees and community. Learn more.
    Dismiss Notice
  3. Join us on November 16th, 2023, between 1 pm and 9 pm CET for Ask the Experts Online on Discord and on Unity Discussions.
    Dismiss Notice
  4. Dismiss Notice

What is "Input dispatching timed out" and how to solve it ?

Discussion in 'Android' started by neufoctobre, Feb 22, 2020.

  1. D_Cergy

    D_Cergy

    Joined:
    Feb 18, 2015
    Posts:
    35
    Based on Dashboard reports, ANR rate of the most problematic device in my case is 5% of their session. The other top 15 problematic devices ranging from 0.5% - 4.5%. And some models also have several variants, which might behave differently. So you need some luck and try 20-200 sessions to reproduce such issue.

    We ended up excluding those problematic devices from Device Catalog, which accounts for 10% of our users. And the ANR rate dropped by 0.15% - 0.2%.
     
  2. Faic

    Faic

    Joined:
    Jun 2, 2015
    Posts:
    19
    Same issue. We wanted to update Unity from 2021.2.13f1 which had no crashes and no ANR to 2022, which failed horribly with tons of crashes, then we wanted to go to 2021 LTS which had major UI glitches, so we have been forced back to 2021.2.13f1 which is the last stable version for us.

    The only change to before was that we also had to upgrade Unity Ad to 4.x.x which introduced a bunch of ANRs including the one of this topic.

    If anyone knows a version that is stable and works with Ads 4.x.x PLEASE let us know. It's frustrating that literally ALL crashes and ANRs come from Unity and we have to pray that some combination of Unity and Ads actually works -.-
     
  3. pradotech

    pradotech

    Joined:
    Oct 17, 2019
    Posts:
    35
    After I upgraded the latest changes (such as AdMob flags), the issue was reduced but not solved. My data still fresh, but the most problematic is Moto G 2nd gen, that's only 1Gb RAM. I removed all devices <= 1Gb RAM, that represented about 9% of installs. I think it's a fair trade if I can see some changes in the near future. This issue came just when my game was featured on Play Store home page, so I'm suffering a little bit by having less visibility than it could have.

    I'm starting to think that the issue is also related to Unity Ads, cause I got one specific ANR that indicates the issue inside Unity Ads package:

    Input dispatching timed out (com.pradostudios.toonchase/com.unity3d.services.ads.adunit.AdUnitActivity, Waiting because the touched window's input channel is full. Outbound queue length: 1. Wait queue length: 53.)


    All started happening after I upgraded both AdMob and UnityAds version. At least AdMob updated their plugin with the flags to optimize Initialization and Ad loading. Maybe Unity also should be doing something regarding this issue.
     
    Last edited: Aug 8, 2022
  4. Tomurtogu

    Tomurtogu

    Joined:
    Nov 30, 2012
    Posts:
    26
    This issue occurred after adding Unity Ads Mediation v3.2.1 and Admob v7.1

    I enabled the optimization flags but that made the situation worse.
    In the test I made on my own device, freezing and fps drops started to occur as if it was sending requests periodically in the background, and after a while, the application crashed.

    Native method - android.os.MessageQueue.nativePollOnce
    Input dispatching timed out
     
    th3z0d1ac likes this.
  5. OmerPlaytika

    OmerPlaytika

    Joined:
    Feb 10, 2022
    Posts:
    4
    Same here .... having about 1% ANR from this issue
     
  6. D_Cergy

    D_Cergy

    Joined:
    Feb 18, 2015
    Posts:
    35
    I think the loss will be compensated when your discoverability goes up. And in fact, my downloads is not getting worse as if there are replacements for the potential loss, but the downloads capped in a certain range.
    That's a pity you missed the potential discoverability from your featuring, I hope you'll get other chances in the future.

    Regarding the RAM, I also confirmed that the lower the RAM, the higher the ANR rate. I'm thinking if the engineer at Admob forgot to release the allocated RAM within their code.

    I also see traces of issue from UnityAds classes, I'm not sure who is the real culprit as I also updated Admob and UnityAds at the same time. Should upload the symbol files next time.
     
  7. pradotech

    pradotech

    Joined:
    Oct 17, 2019
    Posts:
    35
    Thanks, I appreciate your good wishes. I was using both old and new input systems, so I managed to upgrade my code to use only the new input system, cause I saw someone (don't remember where) saying that using both input handlings could contribute to this issue. I also made some changes on my scripts Awake to prevent some performance spikes on the first frames. I pretend to release those changes soon, and I'll come back here to share any improvements.
     
  8. th3z0d1ac

    th3z0d1ac

    Joined:
    May 14, 2014
    Posts:
    15
    My new updated apps with latest Google Mobile Ads doing this crash:

    Native method - android.os.MessageQueue.nativePollOnce

    Any advice or solution for this from someone? I have no idea what to do. When I start my game after init I request rewarded ad and interstitial ad together. Maybe that's causing the problem? I have no idea.

    Optimization flags added, they helped nothing.
     
    DarkCooker likes this.
  9. Tomurtogu

    Tomurtogu

    Joined:
    Nov 30, 2012
    Posts:
    26
    My game has no interstitials but it still crashes. I also created a queue so as not to request ads at the same time. However, that didn't work either.
    I removed Google Mobile Ads and added Applovin instead. I see a lot less crashes but not very good on revenue.
     
  10. pep_dj

    pep_dj

    Joined:
    Nov 7, 2014
    Posts:
    178
    Do you think the problem is only with Unity Ads Mediation? Or the problem is with AdMob plugin?
     
  11. Endahs

    Endahs

    Joined:
    Sep 16, 2014
    Posts:
    94
    Are you still using your ANR watcher? How did it work for you?
     
  12. Endahs

    Endahs

    Joined:
    Sep 16, 2014
    Posts:
    94
    Did you notice an improvement with version 21 and with the flags enabled?
     
  13. Niter88

    Niter88

    Joined:
    Jul 24, 2019
    Posts:
    112
    Can confirm the problem is in AdMobs, I am not using any Unity Ads library.
    I am also using the AdMobs on the Java side, so Unity doesn't have access to it.

    Basically I am having problem with AdMobs and the Billing Library, both from google.
    This is not a Unity bug actually, it's google's

    ...and yes, I couldn't solve it. Using Threads to solve another ANR brought me here...
     
  14. Niter88

    Niter88

    Joined:
    Jul 24, 2019
    Posts:
    112
    Ó um BR aí HUEHUE
    I had to remove the app for "Android Go Devices" and anything 1,5GB ram or less
    In my case I build meshes in realtime, so I cannot suport less than 2GB or crappie processors
     
  15. Ashar01

    Ashar01

    Joined:
    Aug 1, 2014
    Posts:
    6
    Avoid using Awake() too much use Start() instead. Doing this my ANR's dropped from 0.7% to 0.19%

    Use Awake() in just one script per scene.

    Me myself was facing the increasing ANR's issue and I got them reduced from 0.7% to 0.19% just by replacing Awake() by Start()
     
    DarkCooker likes this.
  16. andrew_pearce_

    andrew_pearce_

    Joined:
    Nov 5, 2018
    Posts:
    164
    @Ashar01 that's not the case. I double checked all Awake() methods and in our code we use them only for initialization purposes. The whole logic is in Start() method. Do you want to say that having Awake() method itself causes the problem? Have you tried to compare your app via Profile before and after update?

    The good news is that this ANR is definitely not AdMob related. I recently was forced to upgrade project from Unity 2018 to 2022 and that's when problem started. I hate updates and once project updated, I manually downgraded all plugins back to initial version, with exception of Purchasing which was updated to 4.4.1.

    I strongly believe this is either Unity 2020 bug or Purchasing. Friend of mine has the same issue after migration to 2020 and she is using latest version for all plugins (AdMob, Firebase, Purchasing, Notifications).

    Having using Unity 2020 for a few weeks, we noticed that it's more strict. The code which was working fine on 2018, started to generate minor errors in 2020. I am assuming Unity 2020 requires code optimization on our end. However there is no any special announcement from Unity.

    We also checked Profile both in Editor and on a device but see no difference from previous builds. The average FPS is in range 30-45 even on old Samsung with Android 4.4. There are rare spikes for 0.5sec when new scene gets loaded but that's normal for loading screen.

    We also cannot see ANRs in Crashalitics. What if this is not actual ANRs but some bug in Google Play statistics? We plan to install Android Performance Tuner to find the spots when ANR occur. Once we have some information, I will update the post.

    UPD1: There was a solution suggested in this topic. Basically Google is aware of ANR caused by some ads so they implemented an released an update 5 days ago which has an option to optimize loading. It's easy to check if it helps. The author of the post also has another 100% working solution. However, we doubt that users will be happy =)
     
    Last edited: Oct 15, 2022
  17. Ashar01

    Ashar01

    Joined:
    Aug 1, 2014
    Posts:
    6
    @Andrew_pw yes that's exactly what I'm saying. Don't do initialization in Awake(). Use Start() for that and set the Script Execution sequence accordinlgy. In short, phones with lesser memory, load of initialization hangs the threads at the very beginning of its creation.

    I literally tried everything nothing helped but this solution. Maybe you have other reasons for ANR's but for me and for lot of other people this was the reason.

    I myself had a belief that Unity3D is the core reason behind these rising ANR's, But that does not make any sense I mean people are facing this issue since 2018, do you think Unity development team is unable to fix this issue in past 4 years? I don't think so. Trust me it's either your code or the sdks.

    BTW I'm using Unity 2021.3.10f
     
    Last edited: Oct 16, 2022
  18. andrew_pearce_

    andrew_pearce_

    Joined:
    Nov 5, 2018
    Posts:
    164
    @Ashar01 Thanks for update and additional details. I will definitely try to get rid of all Awake() methods. Extra performance will never be out of place =) I completely agree with you. Unity is really good tool but instead of making it better, they keep releasing a new unstable version and force us to migrate from stable one. We are not users who needs big version numbers (just like Chrome and Firefox did for sake of marketing), we are developers!

    After in-depth analysis, it seems like the problem existed for at least few months. Here is interesting fact, the same app compiled under 2018 generates 0.2/0.3% of ANR (that's why we did not see it earlier) while 2020 - 0.5/0.8% (same plugins). The code of AdMob may not well written but it WAS WORKING fine in 2018... so why should we be forced to migrate to 2020 if it makes code less stable? It means our code will also work less efficiently than it was working under 2018.

    Taking in an account that AdMob released an update with "Optimize initialization and ad loading (Beta)" feature and described it as "...improve the overall responsiveness of ads and help prevent "Application Not Responding" (ANR) errors on your app." We believe updating AdMob to 7.0.2 (requires manual modification of AndroidManifest.xml) or 7.3.0 (checkboxes in editor). BTW the first day shows drop of ANRs!

    I will update my post after few days.

    UPD: We can confirm that updating AdMob and enabling optimization resolved the issue with ANR. However, as @Ashar01 rightly noted, bad coding practices could be an issue in other cases. You need to check what causes ANR under "Crashes and ANRs" of Google Play Console.
     
    Last edited: Oct 18, 2022
    DarkCooker and Endahs like this.
  19. Ashar01

    Ashar01

    Joined:
    Aug 1, 2014
    Posts:
    6
    It's not just about the Unity3D, Android os is also the reason behind this. As you can see on console most of the ANR's are happening either in Android 10 or in Android 11.

    Every Android OS update comes along with the new possibilities of Crashes and ANR's, so in this case I side with Unity3D as they have no choice other than releasing an update.

    You are right Admob SDK was also causing the ANR's and new sdk 7.0.2 had it fixed. But there are lot of people on this post who are still facing the issue even after removing all the sdks. That's because of bad coding practices.
     
    Last edited: Oct 16, 2022
    DarkCooker likes this.
  20. Niter88

    Niter88

    Joined:
    Jul 24, 2019
    Posts:
    112
    My app is a unity lib inside an android studio app. The adMob is only on the android studio part, can confirm that's not a Unity issue. Screwing some things I could reproduce a crash in an activity that doesn't had any adMb functions but I got a crash on an adMob function inside another activity!

    Let me put an example:
    Activity 1 (has admob banner)
    Activity 2 (no admob, screwed function)

    When I got a crash on Activity's 2 method, the log pointed a crahs at the adMob Banner on the 1th Activity instead.
    I know that was not the problem nor the origin of the bug because I've caused it!

    That said, I am getting some errors only possible on some devices, google won't say which device, just the android versions. The error logs are a fake, the actual crash isn't on adMob, but the debugger won't let me have the real culprit.

    I'm on my way to transfer all my code to Unity and just use a Java plugin instead (I have to use java), Android Studio is taking my sanity away. Google should have more care for their libs instead of telling devs they don't know why and blaming them on their sheety libs (not all of them but a lot).

    I will start to avoid using Google's libs from now on.
     
  21. lucbloom

    lucbloom

    Joined:
    Mar 27, 2020
    Posts:
    32
    As I recall, the threaded init & loading fix from Google did not alleviate our ANR rate.

    The ANR Supervisor works, technically. We did see a decrease in ANRs in our live build, but in the end my colleagues found the payoff too small to risk force-closing the game. A few weeks back we switched to another ad provider and now enjoy an increase in revenue and a decrease in ANRs.

    Anyway, I've updated the repo to include that changes that make it more prudent:
    - Only close the application if the thread includes GMS (AdMob) and the status is BLOCKED.
    - Only send the first 3 lines of the stack trace. Saves bandwidth and ensures the report gets sent.
     
  22. andrew_pearce_

    andrew_pearce_

    Joined:
    Nov 5, 2018
    Posts:
    164
    Could you please share the name of that provide @lucbloom? Do you have any idea how they solved the problem? Thanks in advance!

    Theoretically, if Crashalitics were able to record ANR then we can record last shown ad using SetCustomKey() and then simply disable those ads within AdMob. I recall there was a unique shown ad ID somewhere in Firebase Analytics but I am not sure if we have such information via Unity plugin.
     
  23. Dmitriy0188

    Dmitriy0188

    Joined:
    Aug 14, 2022
    Posts:
    1
    Hello. I also encounter a lot of errors and failures ANR. Most of all there was a crash in this openGL 3.2 version. Снимок экрана 2022-11-11 в 01.46.36.png Снимок экрана 2022-11-11 в 01.46.36.png Снимок экрана 2022-11-11 в 01.47.48.png Снимок экрана 2022-11-11 в 01.47.39.png
     

    Attached Files:

  24. Niter88

    Niter88

    Joined:
    Jul 24, 2019
    Posts:
    112
    Endahs likes this.
  25. lorddesu

    lorddesu

    Joined:
    Aug 20, 2014
    Posts:
    31
    What is percent of closing app by that code among all users and all sessions? We tried to track ANRs with this tool https://github.com/SalomonBrys/ANR-WatchDog and it looks way too many false positive. We got 11% of all users has 4.5 sec hang. While ANRs in google console was about only 1%. Is seem not every hang becomes ANR. So If your code has too many false positive(hangs that not actually an ARN) the drawback effect may be huge. Especially if user trying to get IAP and app close.

    Of course I'm speaking about version that do not check only GMS.ADS but trying to catch all ANRs.
     
  26. lorddesu

    lorddesu

    Joined:
    Aug 20, 2014
    Posts:
    31
    For those who are still struggling with anr, I have a way to drop another 0.1-0.2%. Tested on two projects with 300-500k active installations.
    In general, this is memory optimization. Part of the problematic devices are devices with memory less than or equal to 1GB. Quite often, you can simply compress texters, meshes, animations and sounds in a project, without much loss of quality and quite quickly in terms of time. Sometimes this can save 200-300MB of RAM.

    It's even better to use addressables. But this usually requires a rewrite of the project if it was not originally planned.
     
    andreyul, bekici and DarkCooker like this.
  27. DarkCooker

    DarkCooker

    Joined:
    Jan 7, 2015
    Posts:
    119
    ANR Supervisor will damage the organic downloads as it stops sending signal back to Google. We stop using this plugin and downloads come back. So we need another better "less-forceful" solutions for ANR issues.
     
    Brown2Fox and Niter88 like this.
  28. Niter88

    Niter88

    Joined:
    Jul 24, 2019
    Posts:
    112
    For now my solution is not to use Threading because it was hiding the stacktrace and I am now having to completely remove one of the ads because it just crashes on some android 11 and 12 devices (Specially Samsung and Xiaomi).

    If you wanna keep away from those crashes just don't use any google library
     
    DarkCooker likes this.
  29. DmitrySemenychev

    DmitrySemenychev

    Joined:
    Aug 19, 2021
    Posts:
    1
    Good afternoon!

    We encountered an increase in ANR after adding AdVideo to the application. The increase in failures was observed with both UnityAds and IronSource.

    The following examples are the most common types of failures:
    1)Native method - android.os.MessageQueueue.nativePollOnce
    * Input dispatching timed out
    2) Native method - jdk.internal.misc.Unsafe.park
    * Input dispatching timed out
    3) unavailable - com.unity3d.player.UnityPlayer.updateDisplayInternal
    * Input dispatching timed out

    The second and third types of the failure are observed only on Android 11, 12. It appears only in the background.

    The first failure occurs mostly in the background, but approximately 40% of the cases occurred in the foreground.

    We have found that if System.Exit(0) rather than Application.Quit() is called to exit the application, the second and third ANR options go away completely, while the first shows a noticeable decrease. However, the ANR level of android.os.MessageQueueue.nativePollOnce remains quite high.

    We assume that some cases of ANR occur either when the application is paused or when it is resumed. But we can’t be 100% sure.

    In FirebaseCrashlytics, many ANRs occur significantly later (up to several hours) than the last log from the user. Perhaps this is a problem of ANR detection in FirebaseCrashlytics.

    Using OPTIMIZE_INITIALIZATION and OPTIMIZE_AD_LOADING in the AdMob mediator did not lead to the observed changes.

    Attempting to reproduce ANR in synthetic tests also failed. Direct or indirect suspensions of execution flow for tens of seconds were not recorded as ANR.

    Any ideas on how to approach this problem? Is there any reason to suspect that ANR android.os.MessageQueueue.nativePollOnce happens exactly when pausing or exiting?
     
    DarkCooker and KreuzgutGames like this.
  30. cihadturhan_unity

    cihadturhan_unity

    Joined:
    Apr 19, 2021
    Posts:
    51
    upload_2022-12-9_22-23-41.png
    Happens to me as well. Ironsource mediation + admob + fb + unity + applovin + vungle...
     
    DarkCooker likes this.
  31. KreuzgutGames

    KreuzgutGames

    Joined:
    May 16, 2021
    Posts:
    3
    Also the ANR Supervisor version where only AdMob is checked?
     
  32. DarkCooker

    DarkCooker

    Joined:
    Jan 7, 2015
    Posts:
    119
    May I know what google library is it? You mean Google Adapter?
     
  33. DarkCooker

    DarkCooker

    Joined:
    Jan 7, 2015
    Posts:
    119
    Same here. We tried Applovin and Ironsrc mediation both have such issues.
     
    cihadturhan_unity likes this.
  34. DarkCooker

    DarkCooker

    Joined:
    Jan 7, 2015
    Posts:
    119
    One of the top issues but no official comes to fix from either Ironsource, Google Ads or Unity.
     
  35. Niter88

    Niter88

    Joined:
    Jul 24, 2019
    Posts:
    112

    //google play store 1
    def billing_version = "5.0.0"
    implementation "com.android.billingclient:billing:$billing_version"

    //ads
    implementation 'com.google.android.gms:play-services-ads:21.3.0'

    Basically two of two Google libraries I use give me groundbreaking errors. There are lots of reports of errors from those libraries since 2014 or 2017. The same error we are experiencing.
    Those errors of mine are not on an Unity application, so it's not Unity's fault that they're happening.
    Those google errors are mostly pointing to the wrong scripts, it's a mess.
     
  36. KreuzgutGames

    KreuzgutGames

    Joined:
    May 16, 2021
    Posts:
    3

    @DarkCooker does the ANR Supervisor version where only AdMob is checked, also destroy the organic downloads? I don't quite understand how this plugin can break the organic downloads?
     
    Endahs likes this.
  37. Endahs

    Endahs

    Joined:
    Sep 16, 2014
    Posts:
    94
    Can you elaborate? What thread is ANR Supervisor closing that causes Google to stop getting telemetry?
     
    domonyiv likes this.
  38. faisalali1

    faisalali1

    Joined:
    Feb 26, 2020
    Posts:
    1
    anyone solve this issue now ?
    we are facing same anr in our unity game 2021 LTS version
     
  39. Tomas1856

    Tomas1856

    Unity Technologies

    Joined:
    Sep 21, 2012
    Posts:
    3,669
    Could you post full dump of all threads ?
     
  40. andrew_pearce_

    andrew_pearce_

    Joined:
    Nov 5, 2018
    Posts:
    164
    @Tomas1856 is there any chance to locate what causes this ANRs? We get rid of critical amount of ANRs by updating to latest Unity Notification and AdMob plugins and upgrading a project to Unity 2020.3.40f. However, we are still experiencing 0.30% of ANRs.

    Code (csharp):
    1. "main" tid=1 Native
    2.   #00  pc 0x0000000000018cd8  /system/lib/libc.so (syscall+28)
    3.   #01  pc 0x00000000000b7e89  /system/lib/libart.so (art::ConditionVariable::WaitHoldingLocks(art::Thread*)+88)
    4.   #02  pc 0x0000000000282d03  /system/lib/libart.so (art::JNI::NewLocalRef(_JNIEnv*, _jobject*)+298)
    5.   #03  pc 0x0000000000ccf905  /data/app/com.android.chrome-tp2IXxrNtf_d2KkSVY4vew==/base.apk
    6.   #04  pc 0x0000000000bee731  /data/app/com.android.chrome-tp2IXxrNtf_d2KkSVY4vew==/base.apk
    7.   #05  pc 0x0000000000c937b7  /data/app/com.android.chrome-tp2IXxrNtf_d2KkSVY4vew==/base.apk
    8.   #06  pc 0x0000000000bee677  /data/app/com.android.chrome-tp2IXxrNtf_d2KkSVY4vew==/base.apk
    9.   #07  pc 0x000000000011bb6f  /data/app/com.android.chrome-tp2IXxrNtf_d2KkSVY4vew==/base.apk
    10.   #08  pc 0x000000000078ad13  /data/app/com.android.chrome-tp2IXxrNtf_d2KkSVY4vew==/base.apk
    11.   #09  pc 0x00000000005e8ec5  /data/app/com.android.chrome-tp2IXxrNtf_d2KkSVY4vew==/base.apk
    12.   #10  pc 0x00000000005e7669  /data/app/com.android.chrome-tp2IXxrNtf_d2KkSVY4vew==/base.apk
    13.   #11  pc 0x0000000000d82781  /data/app/com.android.chrome-tp2IXxrNtf_d2KkSVY4vew==/base.apk
    14.   #12  pc 0x0000000000d5ccbf  /data/app/com.android.chrome-tp2IXxrNtf_d2KkSVY4vew==/base.apk
    15.   #13  pc 0x0000000000d5cc75  /data/app/com.android.chrome-tp2IXxrNtf_d2KkSVY4vew==/base.apk
    16.   #14  pc 0x0000000000d5cc05  /data/app/com.android.chrome-tp2IXxrNtf_d2KkSVY4vew==/base.apk
    17.   #15  pc 0x00000000000102c9  /system/lib/libutils.so (android::Looper::pollInner(int)+572)
    18.   #16  pc 0x000000000000fff5  /system/lib/libutils.so (android::Looper::pollOnce(int, int*, int*, void**)+32)
    19.   #17  pc 0x00000000000b21c5  /system/lib/libandroid_runtime.so (android::NativeMessageQueue::pollOnce(_JNIEnv*, _jobject*, int)+24)
    20.   #18  pc 0x00000000000bcd55  /system/framework/arm/boot-framework.oat (Java_android_os_MessageQueue_nativePollOnce__JI+92)
    21.   at android.os.MessageQueue.nativePollOnce (Native method)
    22.   at android.os.MessageQueue.next (MessageQueue.java:325)
    23.   at android.os.Looper.loop (Looper.java:142)
    24.   at android.app.ActivityThread.main (ActivityThread.java:6715)
    25.   at java.lang.reflect.Method.invoke (Native method)
    26.   at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run (RuntimeInit.java:449)
    27.   at com.android.internal.os.ZygoteInit.main (ZygoteInit.java:807)
    28.  
    29. "ThreadPoolForeg" tid=19 Runnable
    30.   at java.util.ArrayList.<init> (ArrayList.java:190)
    31.   at CU0.a (chromium-Monochrome.aab-stable-<US_SOCIAL_SECURITY_NUMBER>:95)
    32.   at l8.g (chromium-Monochrome.aab-stable-<US_SOCIAL_SECURITY_NUMBER>:78)
    33.   at i8.run (chromium-Monochrome.aab-stable-<US_SOCIAL_SECURITY_NUMBER>:34)
    34.  
    35. "Signal Catcher" tid=3 Runnable
    36.   #00  pc 0x00000000002edaf7  /system/lib/libart.so (art::DumpNativeStack(std::__1::basic_ostream<char, std::__1::char_traits<char>>&, int, BacktraceMap*, char const*, art::ArtMethod*, void*)+130)
    37.   #01  pc 0x0000000000380dc5  /system/lib/libart.so (art::Thread::DumpStack(std::__1::basic_ostream<char, std::__1::char_traits<char>>&, bool, BacktraceMap*, bool) const+204)
    38.   #02  pc 0x000000000037d2ff  /system/lib/libart.so (art::Thread::Dump(std::__1::basic_ostream<char, std::__1::char_traits<char>>&, bool, BacktraceMap*, bool) const+34)
    39.   #03  pc 0x0000000000395365  /system/lib/libart.so (art::DumpCheckpoint::Run(art::Thread*)+684)
    40.   #04  pc 0x000000000038ee37  /system/lib/libart.so (art::ThreadList::RunCheckpoint(art::Closure*, art::Closure*)+322)
    41.   #05  pc 0x000000000038e8e7  /system/lib/libart.so (art::ThreadList::Dump(std::__1::basic_ostream<char, std::__1::char_traits<char>>&, bool)+614)
    42.   #06  pc 0x000000000038e5b1  /system/lib/libart.so (art::ThreadList::DumpForSigQuit(std::__1::basic_ostream<char, std::__1::char_traits<char>>&)+648)
    43.   #07  pc 0x000000000036b843  /system/lib/libart.so (art::Runtime::DumpForSigQuit(std::__1::basic_ostream<char, std::__1::char_traits<char>>&)+122)
    44.   #08  pc 0x0000000000372e45  /system/lib/libart.so (art::SignalCatcher::HandleSigQuit()+1276)
    45.   #09  pc 0x0000000000371dcd  /system/lib/libart.so (art::SignalCatcher::Run(void*)+240)
    46.   #10  pc 0x000000000004790b  /system/lib/libc.so (__pthread_start(void*)+22)
    47.   #11  pc 0x000000000001add3  /system/lib/libc.so (__start_thread+32)
    48.  
    49. "Jit thread pool worker thread 0" tid=2 Runnable
    50.   #00  pc 0x00000000002edaf7  /system/lib/libart.so (art::DumpNativeStack(std::__1::basic_ostream<char, std::__1::char_traits<char>>&, int, BacktraceMap*, char const*, art::ArtMethod*, void*)+130)
    51.   #01  pc 0x0000000000380dc5  /system/lib/libart.so (art::Thread::DumpStack(std::__1::basic_ostream<char, std::__1::char_traits<char>>&, bool, BacktraceMap*, bool) const+204)
    52.   #02  pc 0x000000000037d2ff  /system/lib/libart.so (art::Thread::Dump(std::__1::basic_ostream<char, std::__1::char_traits<char>>&, bool, BacktraceMap*, bool) const+34)
    53.   #03  pc 0x0000000000395365  /system/lib/libart.so (art::DumpCheckpoint::Run(art::Thread*)+684)
    54.   #04  pc 0x00000000003819b7  /system/lib/libart.so (art::Thread::RunCheckpointFunction()+278)
    55.   #05  pc 0x00000000000c430b  /system/lib/libart-compiler.so (art::HInliner::TryInline(art::HInvoke*)+1090)
    56.   #06  pc 0x00000000000c3d1d  /system/lib/libart-compiler.so (art::HInliner::Run()+412)
    57.   #07  pc 0x00000000000ff74f  /system/lib/libart-compiler.so (art::OptimizingCompiler::MaybeRunInliner(art::HGraph*, art::CodeGenerator*, art::CompilerDriver*, art::DexCompilationUnit const&, art::passObserver*, art::VariableSizedHandleScope*) const+130)
    58.   #08  pc 0x00000000001002d3  /system/lib/libart-compiler.so (art::OptimizingCompiler::RunOptimizations(art::HGraph*, art::CodeGenerator*, art::CompilerDriver*, art::DexCompilationUnit const&, art::passObserver*, art::VariableSizedHandleScope*) const+2002)
    59.   #09  pc 0x0000000000101db5  /system/lib/libart-compiler.so (art::OptimizingCompiler::TryCompile(art::ArenaAllocator*, art::CodeVectorAllocator*, art::DexFile::CodeItem const*, unsigned int, art::InvokeType, unsigned short, unsigned int, art::Handle<art::mirror::ClassLoader>, art::DexFile const&, art::Handle<art::mirror::DexCache>, art::ArtMethod*, bool, art::VariableSizedHandleScope*) const+2312)
    60.   #10  pc 0x00000000001032fd  /system/lib/libart-compiler.so (art::OptimizingCompiler::JitCompile(art::Thread*, art::jit::JitCodeCache*, art::ArtMethod*, bool, art::jit::JitLogger*)+612)
    61.   #11  pc 0x00000000000a09b9  /system/lib/libart-compiler.so (art::jit::JitCompiler::CompileMethod(art::Thread*, art::ArtMethod*, bool)+92)
    62.   #12  pc 0x00000000002678c5  /system/lib/libart.so (art::jit::Jit::CompileMethod(art::ArtMethod*, art::Thread*, bool)+288)
    63.   #13  pc 0x00000000002696bb  /system/lib/libart.so (art::jit::JitCompileTask::Run(art::Thread*)+406)
    64.   #14  pc 0x0000000000395f3d  /system/lib/libart.so (art::ThreadPoolWorker::Run()+44)
    65.   #15  pc 0x0000000000395b6f  /system/lib/libart.so (art::ThreadPoolWorker::Callback(void*)+90)
    66.   #16  pc 0x000000000004790b  /system/lib/libc.so (__pthread_start(void*)+22)
    67.   #17  pc 0x000000000001add3  /system/lib/libc.so (__start_thread+32)
    68.  
    69. "FinalizerDaemon" tid=4 Waiting
    70.   at java.lang.Object.wait (Native method)
    71.   at java.lang.Object.wait (Object.java:422)
    72.   at java.lang.ref.ReferenceQueue.remove (ReferenceQueue.java:188)
    73.   at java.lang.ref.ReferenceQueue.remove (ReferenceQueue.java:209)
    74.   at java.lang.Daemons$FinalizerDaemon.runInternal (Daemons.java:235)
    75.   at java.lang.Daemons$Daemon.run (Daemons.java:106)
    76.   at java.lang.Thread.run (Thread.java:764)
    77.  
    78. "HeapTaskDaemon" tid=5 Blocked
    79.   #00  pc 0x0000000000018cd8  /system/lib/libc.so (syscall+28)
    80.   #01  pc 0x00000000000b7e89  /system/lib/libart.so (art::ConditionVariable::WaitHoldingLocks(art::Thread*)+88)
    81.   #02  pc 0x00000000001cd49d  /system/lib/libart.so (art::gc::TaskProcessor::GetTask(art::Thread*)+284)
    82.   #03  pc 0x00000000001cda7d  /system/lib/libart.so (art::gc::TaskProcessor::RunAllTasks(art::Thread*)+44)
    83.   #04  pc 0x00000000000552eb  /system/framework/arm/boot-core-libart.oat (Java_dalvik_system_VMRuntime_runHeapTasks__+74)
    84.   at dalvik.system.VMRuntime.runHeapTasks (Native method)
    85.   at java.lang.Daemons$HeapTaskDaemon.runInternal (Daemons.java:481)
    86.   at java.lang.Daemons$Daemon.run (Daemons.java:106)
    87.   at java.lang.Thread.run (Thread.java:764)
    88.  
    89. "FinalizerWatchdogDaemon" tid=6 Timed Waiting
    90.   at java.lang.Thread.sleep (Native method)
    91.   at java.lang.Thread.sleep (Thread.java:373)
    92.   at java.lang.Thread.sleep (Thread.java:314)
    93.   at java.lang.Daemons$FinalizerWatchdogDaemon.sleepFor (Daemons.java:345)
    94.   at java.lang.Daemons$FinalizerWatchdogDaemon.waitForFinalization (Daemons.java:367)
    95.   at java.lang.Daemons$FinalizerWatchdogDaemon.runInternal (Daemons.java:284)
    96.   at java.lang.Daemons$Daemon.run (Daemons.java:106)
    97.   at java.lang.Thread.run (Thread.java:764)
    98.  
    99. "ReferenceQueueDaemon" tid=7 Waiting
    100.   at java.lang.Object.wait (Native method)
    101.   at java.lang.Daemons$ReferenceQueueDaemon.runInternal (Daemons.java:181)
    102.   at java.lang.Daemons$Daemon.run (Daemons.java:106)
    103.   at java.lang.Thread.run (Thread.java:764)
    104.  
    105. "Binder:14535_1" tid=8 Native
    106.   #00  pc 0x0000000000048fe0  /system/lib/libc.so (__ioctl+8)
    107.   #01  pc 0x000000000001dc4b  /system/lib/libc.so (ioctl+38)
    108.   #02  pc 0x000000000004267f  /system/lib/libbinder.so (android::IPCThreadState::talkWithDriver(bool)+170)
    109.   #03  pc 0x0000000000042779  /system/lib/libbinder.so (android::IPCThreadState::getAndExecuteCommand()+8)
    110.   #04  pc 0x0000000000042ccb  /system/lib/libbinder.so (android::IPCThreadState::joinThreadPool(bool)+38)
    111.   #05  pc 0x0000000000058175  /system/lib/libbinder.so
    112.   #06  pc 0x000000000000d2a9  /system/lib/libutils.so (android::Thread::_threadLoop(void*)+144)
    113.   #07  pc 0x000000000006e0fd  /system/lib/libandroid_runtime.so (android::AndroidRuntime::javaThreadShell(void*)+80)
    114.   #08  pc 0x000000000004790b  /system/lib/libc.so (__pthread_start(void*)+22)
    115.   #09  pc 0x000000000001add3  /system/lib/libc.so (__start_thread+32)
    116.  
    117. "Binder:14535_2" tid=9 Native
    118.   #00  pc 0x0000000000048fe0  /system/lib/libc.so (__ioctl+8)
    119.   #01  pc 0x000000000001dc4b  /system/lib/libc.so (ioctl+38)
    120.   #02  pc 0x000000000004267f  /system/lib/libbinder.so (android::IPCThreadState::talkWithDriver(bool)+170)
    121.   #03  pc 0x0000000000042779  /system/lib/libbinder.so (android::IPCThreadState::getAndExecuteCommand()+8)
    122.   #04  pc 0x0000000000042ccb  /system/lib/libbinder.so (android::IPCThreadState::joinThreadPool(bool)+38)
    123.   #05  pc 0x0000000000058175  /system/lib/libbinder.so
    124.   #06  pc 0x000000000000d2a9  /system/lib/libutils.so (android::Thread::_threadLoop(void*)+144)
    125.   #07  pc 0x000000000006e0fd  /system/lib/libandroid_runtime.so (android::AndroidRuntime::javaThreadShell(void*)+80)
    126.   #08  pc 0x000000000004790b  /system/lib/libc.so (__pthread_start(void*)+22)
    127.   #09  pc 0x000000000001add3  /system/lib/libc.so (__start_thread+32)
    128.  
    129. "Profile Saver" tid=10 Native
    130.   #00  pc 0x0000000000018cd8  /system/lib/libc.so (syscall+28)
    131.   #01  pc 0x00000000000b7e89  /system/lib/libart.so (art::ConditionVariable::WaitHoldingLocks(art::Thread*)+88)
    132.   #02  pc 0x0000000000279323  /system/lib/libart.so (art::profileSaver::Run()+346)
    133.   #03  pc 0x000000000027b6d7  /system/lib/libart.so (art::profileSaver::RunProfileSaverThread(void*)+50)
    134.   #04  pc 0x000000000004790b  /system/lib/libc.so (__pthread_start(void*)+22)
    135.   #05  pc 0x000000000001add3  /system/lib/libc.so (__start_thread+32)
    136.  
    137. "ScionFrontendApi" tid=12 Timed Waiting
    138.   at java.lang.Object.wait (Native method)
    139.   at java.lang.Thread.parkFor$ (Thread.java:2137)
    140.   at sun.misc.Unsafe.park (Unsafe.java:358)
    141.   at java.util.concurrent.locks.LockSupport.parkNanos (LockSupport.java:230)
    142.   at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos (AbstractQueuedSynchronizer.java:2101)
    143.   at java.util.concurrent.LinkedBlockingQueue.poll (LinkedBlockingQueue.java:467)
    144.   at java.util.concurrent.ThreadPoolExecutor.getTask (ThreadPoolExecutor.java:1086)
    145.   at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1147)
    146.   at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:636)
    147.   at java.lang.Thread.run (Thread.java:764)
    148.  
    149. "Crashlytics Exception Handler1" tid=13 Waiting
    150.   at java.lang.Object.wait (Native method)
    151.   at java.lang.Thread.parkFor$ (Thread.java:2137)
    152.   at sun.misc.Unsafe.park (Unsafe.java:358)
    153.   at java.util.concurrent.locks.LockSupport.park (LockSupport.java:190)
    154.   at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await (AbstractQueuedSynchronizer.java:2059)
    155.   at java.util.concurrent.LinkedBlockingQueue.take (LinkedBlockingQueue.java:442)
    156.   at java.util.concurrent.ThreadPoolExecutor.getTask (ThreadPoolExecutor.java:1087)
    157.   at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1147)
    158.   at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:636)
    159.   at com.google.firebase.crashlytics.internal.common.ExecutorUtils$1$1.onRun (ExecutorUtils.java:64)
    160.   at com.google.firebase.crashlytics.internal.common.BackgroundPriorityRunnable.run (BackgroundPriorityRunnable.java:27)
    161.   at java.lang.Thread.run (Thread.java:764)
    162.  
    163. "GmsDynamite" tid=14 Waiting
    164.   at java.lang.Object.wait (Native method)
    165.   at com.google.android.gms.dynamite.zza.run (com.google.android.gms:play-services-basement@@18.1.0:2)
    166.  
    167. "pool-3-thread-1" tid=15 Waiting
    168.   at java.lang.Object.wait (Native method)
    169.   at java.lang.Thread.parkFor$ (Thread.java:2137)
    170.   at sun.misc.Unsafe.park (Unsafe.java:358)
    171.   at java.util.concurrent.locks.LockSupport.park (LockSupport.java:190)
    172.   at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await (AbstractQueuedSynchronizer.java:2059)
    173.   at java.util.concurrent.LinkedBlockingQueue.take (LinkedBlockingQueue.java:442)
    174.   at java.util.concurrent.ThreadPoolExecutor.getTask (ThreadPoolExecutor.java:1087)
    175.   at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1147)
    176.   at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:636)
    177.   at java.lang.Thread.run (Thread.java:764)
    178.  
    179. "awaitEvenIfOnMainThread task continuation executor1" tid=16 Waiting
    180.   at java.lang.Object.wait (Native method)
    181.   at java.lang.Thread.parkFor$ (Thread.java:2137)
    182.   at sun.misc.Unsafe.park (Unsafe.java:358)
    183.   at java.util.concurrent.locks.LockSupport.park (LockSupport.java:190)
    184.   at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await (AbstractQueuedSynchronizer.java:2059)
    185.   at java.util.concurrent.LinkedBlockingQueue.take (LinkedBlockingQueue.java:442)
    186.   at java.util.concurrent.ThreadPoolExecutor.getTask (ThreadPoolExecutor.java:1087)
    187.   at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1147)
    188.   at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:636)
    189.   at com.google.firebase.crashlytics.internal.common.ExecutorUtils$1$1.onRun (ExecutorUtils.java:64)
    190.   at com.google.firebase.crashlytics.internal.common.BackgroundPriorityRunnable.run (BackgroundPriorityRunnable.java:27)
    191.   at java.lang.Thread.run (Thread.java:764)
    192.  
    193. "com.google.firebase.crashlytics.startup1" tid=17 Waiting
    194.   at java.lang.Object.wait (Native method)
    195.   at java.lang.Thread.parkFor$ (Thread.java:2137)
    196.   at sun.misc.Unsafe.park (Unsafe.java:358)
    197.   at java.util.concurrent.locks.LockSupport.park (LockSupport.java:190)
    198.   at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await (AbstractQueuedSynchronizer.java:2059)
    199.   at java.util.concurrent.LinkedBlockingQueue.take (LinkedBlockingQueue.java:442)
    200.   at java.util.concurrent.ThreadPoolExecutor.getTask (ThreadPoolExecutor.java:1087)
    201.   at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1147)
    202.   at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:636)
    203.   at com.google.firebase.crashlytics.internal.common.ExecutorUtils$1$1.onRun (ExecutorUtils.java:64)
    204.   at com.google.firebase.crashlytics.internal.common.BackgroundPriorityRunnable.run (BackgroundPriorityRunnable.java:27)
    205.   at java.lang.Thread.run (Thread.java:764)
    206.  
    207. "Measurement Worker" tid=27 Runnable
    208.   at android.text.TextUtils.isEmpty (TextUtils.java)
    209.   at cg.d :)com.google.android.gms.dynamite_measurementdynamite@224915059@22.49.15 (110304-0))
    210.   at ke.U :)com.google.android.gms.dynamite_measurementdynamite@224915059@22.49.15 (110304-0):1)
    211.   at jk.run :)com.google.android.gms.dynamite_measurementdynamite@224915059@22.49.15 (110304-0))
    212.   at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:457)
    213.   at java.util.concurrent.FutureTask.run (FutureTask.java:266)
    214.   at iq.run :)com.google.android.gms.dynamite_measurementdynamite@224915059@22.49.15 (110304-0):5)
    215.  
    216. "pool-6-thread-1" tid=18 Timed Waiting
    217.   at java.lang.Object.wait (Native method)
    218.   at java.lang.Thread.parkFor$ (Thread.java:2137)
    219.   at sun.misc.Unsafe.park (Unsafe.java:358)
    220.   at java.util.concurrent.locks.LockSupport.parkNanos (LockSupport.java:230)
    221.   at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill (SynchronousQueue.java:461)
    222.   at java.util.concurrent.SynchronousQueue$TransferStack.transfer (SynchronousQueue.java:362)
    223.   at java.util.concurrent.SynchronousQueue.poll (SynchronousQueue.java:937)
    224.   at java.util.concurrent.ThreadPoolExecutor.getTask (ThreadPoolExecutor.java:1086)
    225.   at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1147)
    226.   at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:636)
    227.   at java.lang.Thread.run (Thread.java:764)
    228.  
    229. "pool-8-thread-1" tid=21 Timed Waiting
    230.   at java.lang.Object.wait (Native method)
    231.   at java.lang.Thread.parkFor$ (Thread.java:2137)
    232.   at sun.misc.Unsafe.park (Unsafe.java:358)
    233.   at java.util.concurrent.locks.LockSupport.parkNanos (LockSupport.java:230)
    234.   at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill (SynchronousQueue.java:461)
    235.   at java.util.concurrent.SynchronousQueue$TransferStack.transfer (SynchronousQueue.java:362)
    236.   at java.util.concurrent.SynchronousQueue.poll (SynchronousQueue.java:937)
    237.   at java.util.concurrent.ThreadPoolExecutor.getTask (ThreadPoolExecutor.java:1086)
    238.   at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1147)
    239.   at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:636)
    240.   at java.lang.Thread.run (Thread.java:764)
    241.  
    242. "pool-9-thread-1" tid=22 Timed Waiting
    243.   at java.lang.Object.wait (Native method)
    244.   at java.lang.Thread.parkFor$ (Thread.java:2137)
    245.   at sun.misc.Unsafe.park (Unsafe.java:358)
    246.   at java.util.concurrent.locks.LockSupport.parkNanos (LockSupport.java:230)
    247.   at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill (SynchronousQueue.java:461)
    248.   at java.util.concurrent.SynchronousQueue$TransferStack.transfer (SynchronousQueue.java:362)
    249.   at java.util.concurrent.SynchronousQueue.poll (SynchronousQueue.java:937)
    250.   at java.util.concurrent.ThreadPoolExecutor.getTask (ThreadPoolExecutor.java:1086)
    251.   at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1147)
    252.   at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:636)
    253.   at java.lang.Thread.run (Thread.java:764)
    254.  
    255. "pool-7-thread-1" tid=23 Timed Waiting
    256.   at java.lang.Object.wait (Native method)
    257.   at java.lang.Thread.parkFor$ (Thread.java:2137)
    258.   at sun.misc.Unsafe.park (Unsafe.java:358)
    259.   at java.util.concurrent.locks.LockSupport.parkNanos (LockSupport.java:230)
    260.   at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill (SynchronousQueue.java:461)
    261.   at java.util.concurrent.SynchronousQueue$TransferStack.transfer (SynchronousQueue.java:362)
    262.   at java.util.concurrent.SynchronousQueue.poll (SynchronousQueue.java:937)
    263.   at java.util.concurrent.ThreadPoolExecutor.getTask (ThreadPoolExecutor.java:1086)
    264.   at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1147)
    265.   at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:636)
    266.   at java.lang.Thread.run (Thread.java:764)
    267.  
    268. "WM.task-1" tid=25 Waiting
    269.   at java.lang.Object.wait (Native method)
    270.   at java.lang.Thread.parkFor$ (Thread.java:2137)
    271.   at sun.misc.Unsafe.park (Unsafe.java:358)
    272.   at java.util.concurrent.locks.LockSupport.park (LockSupport.java:190)
    273.   at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await (AbstractQueuedSynchronizer.java:2059)
    274.   at java.util.concurrent.LinkedBlockingQueue.take (LinkedBlockingQueue.java:442)
    275.   at java.util.concurrent.ThreadPoolExecutor.getTask (ThreadPoolExecutor.java:1087)
    276.   at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1147)
    277.   at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:636)
    278.   at java.lang.Thread.run (Thread.java:764)
    279.  
    280. "WM.task-2" tid=26 Waiting
    281.   at java.lang.Object.wait (Native method)
    282.   at java.lang.Thread.parkFor$ (Thread.java:2137)
    283.   at sun.misc.Unsafe.park (Unsafe.java:358)
    284.   at java.util.concurrent.locks.LockSupport.park (LockSupport.java:190)
    285.   at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await (AbstractQueuedSynchronizer.java:2059)
    286.   at java.util.concurrent.LinkedBlockingQueue.take (LinkedBlockingQueue.java:442)
    287.   at java.util.concurrent.ThreadPoolExecutor.getTask (ThreadPoolExecutor.java:1087)
    288.   at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1147)
    289.   at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:636)
    290.   at java.lang.Thread.run (Thread.java:764)
    291.  
    292. "UnityMain" tid=28 Native
    293.   #00  pc 0x0000000000018cd8  /system/lib/libc.so (syscall+28)
    294.   #01  pc 0x00000000000b7e89  /system/lib/libart.so (art::ConditionVariable::WaitHoldingLocks(art::Thread*)+88)
    295.   #02  pc 0x0000000000281909  /system/lib/libart.so (art::JNI::pushLocalFrame(_JNIEnv*, int)+340)
    296.   #03  pc 0x000000000061c725  /data/app/com.app.name-uG2T4LshnAqPrF1eFaJSzA==/lib/arm/libunity.so
    297.   at com.unity3d.player.UnityPlayer.nativeRender (Native method)
    298.   at com.unity3d.player.UnityPlayer.access$300 (unavailable)
    299.   at com.unity3d.player.UnityPlayer$e$1.handleMessage (unavailable)
    300.   at android.os.Handler.dispatchMessage (Handler.java:102)
    301.   at android.os.Looper.loop (Looper.java:164)
    302.   at com.unity3d.player.UnityPlayer$e.run (unavailable)
    303.  
    304. "queued-work-looper" tid=11 Native
    305.   #00  pc 0x0000000000048ea4  /system/lib/libc.so (__epoll_pwait+20)
    306.   #01  pc 0x000000000001b1cd  /system/lib/libc.so (epoll_pwait+60)
    307.   #02  pc 0x000000000001b1fd  /system/lib/libc.so (epoll_wait+12)
    308.   #03  pc 0x0000000000010103  /system/lib/libutils.so (android::Looper::pollInner(int)+118)
    309.   #04  pc 0x000000000000fff5  /system/lib/libutils.so (android::Looper::pollOnce(int, int*, int*, void**)+32)
    310.   #05  pc 0x00000000000b21c5  /system/lib/libandroid_runtime.so (android::NativeMessageQueue::pollOnce(_JNIEnv*, _jobject*, int)+24)
    311.   #06  pc 0x00000000000bcd55  /system/framework/arm/boot-framework.oat (Java_android_os_MessageQueue_nativePollOnce__JI+92)
    312.   at android.os.MessageQueue.nativePollOnce (Native method)
    313.   at android.os.MessageQueue.next (MessageQueue.java:325)
    314.   at android.os.Looper.loop (Looper.java:142)
    315.   at android.os.HandlerThread.run (HandlerThread.java:65)
    316.  
    317. "Binder:14535_3" tid=29 Native
    318.   #00  pc 0x0000000000048fe0  /system/lib/libc.so (__ioctl+8)
    319.   #01  pc 0x000000000001dc4b  /system/lib/libc.so (ioctl+38)
    320.   #02  pc 0x000000000004267f  /system/lib/libbinder.so (android::IPCThreadState::talkWithDriver(bool)+170)
    321.   #03  pc 0x0000000000042779  /system/lib/libbinder.so (android::IPCThreadState::getAndExecuteCommand()+8)
    322.   #04  pc 0x0000000000042ccb  /system/lib/libbinder.so (android::IPCThreadState::joinThreadPool(bool)+38)
    323.   #05  pc 0x0000000000058175  /system/lib/libbinder.so
    324.   #06  pc 0x000000000000d2a9  /system/lib/libutils.so (android::Thread::_threadLoop(void*)+144)
    325.   #07  pc 0x000000000006e0fd  /system/lib/libandroid_runtime.so (android::AndroidRuntime::javaThreadShell(void*)+80)
    326.   #08  pc 0x000000000004790b  /system/lib/libc.so (__pthread_start(void*)+22)
    327.   #09  pc 0x000000000001add3  /system/lib/libc.so (__start_thread+32)
    328.  
    329. "FMODAudioDevice" tid=30 Native
    330.   #00  pc 0x0000000000018cd8  /system/lib/libc.so (syscall+28)
    331.   #01  pc 0x00000000000b7e89  /system/lib/libart.so (art::ConditionVariable::WaitHoldingLocks(art::Thread*)+88)
    332.   #02  pc 0x00000000002bcc1b  /system/lib/libart.so (void art::JNI::ReleasePrimitiveArray<_jbyteArray*, signed char, art::mirror::primitiveArray<signed char>>(_JNIEnv*, _jbyteArray*, signed char*, int)+346)
    333.   #03  pc 0x00000000000de335  /system/lib/libandroid_runtime.so
    334.   #04  pc 0x000000000020f6fb  /system/framework/arm/boot-framework.oat (Java_android_media_AudioTrack_native_1write_1byte___3BIIIZ+122)
    335.   at android.media.AudioTrack.native_write_byte (Native method)
    336.   at android.media.AudioTrack.write (AudioTrack.java:2184)
    337.   at android.media.AudioTrack.write (AudioTrack.java:2121)
    338.   at org.fmod.FMODAudioDevice.run (unavailable)
    339.   at java.lang.Thread.run (Thread.java:764)
    340.  
    341. "AudioTrack" tid=31 Native
    342.   #00  pc 0x0000000000018cdc  /system/lib/libc.so (syscall+32)
    343.   #01  pc 0x00000000000472a1  /system/lib/libc.so (__pthread_cond_timedwait(pthread_cond_internal_t*, pthread_mutex_t*, bool, timespec const*)+102)
    344.   #02  pc 0x000000000002ff61  /system/lib/libaudioclient.so (android::AudioTrack::AudioTrackThread::threadLoop()+492)
    345.   #03  pc 0x000000000000d327  /system/lib/libutils.so (android::Thread::_threadLoop(void*)+270)
    346.   #04  pc 0x000000000006e0fd  /system/lib/libandroid_runtime.so (android::AndroidRuntime::javaThreadShell(void*)+80)
    347.   #05  pc 0x000000000004790b  /system/lib/libc.so (__pthread_start(void*)+22)
    348.   #06  pc 0x000000000001add3  /system/lib/libc.so (__start_thread+32)
    349.  
    350. "ConnectivityThread" tid=32 Native
    351.   #00  pc 0x0000000000048ea4  /system/lib/libc.so (__epoll_pwait+20)
    352.   #01  pc 0x000000000001b1cd  /system/lib/libc.so (epoll_pwait+60)
    353.   #02  pc 0x000000000001b1fd  /system/lib/libc.so (epoll_wait+12)
    354.   #03  pc 0x0000000000010103  /system/lib/libutils.so (android::Looper::pollInner(int)+118)
    355.   #04  pc 0x000000000000fff5  /system/lib/libutils.so (android::Looper::pollOnce(int, int*, int*, void**)+32)
    356.   #05  pc 0x00000000000b21c5  /system/lib/libandroid_runtime.so (android::NativeMessageQueue::pollOnce(_JNIEnv*, _jobject*, int)+24)
    357.   #06  pc 0x00000000000bcd55  /system/framework/arm/boot-framework.oat (Java_android_os_MessageQueue_nativePollOnce__JI+92)
    358.   at android.os.MessageQueue.nativePollOnce (Native method)
    359.   at android.os.MessageQueue.next (MessageQueue.java:325)
    360.   at android.os.Looper.loop (Looper.java:142)
    361.   at android.os.HandlerThread.run (HandlerThread.java:65)
    362.  
    363. "UnityChoreographer" tid=33 Native
    364.   #00  pc 0x0000000000018cd8  /system/lib/libc.so (syscall+28)
    365.   #01  pc 0x00000000000b7e89  /system/lib/libart.so (art::ConditionVariable::WaitHoldingLocks(art::Thread*)+88)
    366.   #02  pc 0x000000000028544d  /system/lib/libart.so (art::JNI::CallObjectMethod(_JNIEnv*, _jobject*, _jmethodID*, ...)+360)
    367.   #03  pc 0x00000000000974d9  /system/lib/libandroid_runtime.so (android::NativeDisplayEventReceiver::dispatchVsync(long long, int, unsigned int)+20)
    368.   #04  pc 0x000000000003139d  /system/lib/libandroidfw.so (android::DisplayEventDispatcher::handleEvent(int, int, void*)+100)
    369.   #05  pc 0x00000000000102c9  /system/lib/libutils.so (android::Looper::pollInner(int)+572)
    370.   #06  pc 0x000000000000fff5  /system/lib/libutils.so (android::Looper::pollOnce(int, int*, int*, void**)+32)
    371.   #07  pc 0x00000000000b21c5  /system/lib/libandroid_runtime.so (android::NativeMessageQueue::pollOnce(_JNIEnv*, _jobject*, int)+24)
    372.   #08  pc 0x00000000000bcd55  /system/framework/arm/boot-framework.oat (Java_android_os_MessageQueue_nativePollOnce__JI+92)
    373.   at android.os.MessageQueue.nativePollOnce (Native method)
    374.   at android.os.MessageQueue.next (MessageQueue.java:325)
    375.   at android.os.Looper.loop (Looper.java:142)
    376.   at android.os.HandlerThread.run (HandlerThread.java:65)
    377.  
    378. "AdWorker(Default) #1" tid=34 Waiting
     
    Last edited: May 23, 2023
  41. Tomas1856

    Tomas1856

    Unity Technologies

    Joined:
    Sep 21, 2012
    Posts:
    3,669
    ANR happens because "main" tid=1 is waiting on something, probably on UnityMain thread
    Code (CSharp):
    1. "UnityMain" tid=28 Native
    2.   #00  pc 0x0000000000018cd8  /system/lib/libc.so (syscall+28)
    3.   #01  pc 0x00000000000b7e89  /system/lib/libart.so (art::ConditionVariable::WaitHoldingLocks(art::Thread*)+88)
    4.   #02  pc 0x0000000000281909  /system/lib/libart.so (art::JNI::pushLocalFrame(_JNIEnv*, int)+340)
    5.   #03  pc 0x000000000061c725  /data/app/com.*************-uG2T4LshnAqPrF1eFaJSzA==/lib/arm/libunity.so
    6.   at com.unity3d.player.UnityPlayer.nativeRender (Native method)
    7.   at com.unity3d.player.UnityPlayer.access$300 (unavailable)
    8.   at com.unity3d.player.UnityPlayer$e$1.handleMessage (unavailable)
    9.   at android.os.Handler.dispatchMessage (Handler.java:102)
    10.   at android.os.Looper.loop (Looper.java:164)
    11.   at com.unity3d.player.UnityPlayer$e.run (unavailable)
    Try resolving
    Code (CSharp):
    1.  
    2. #03  pc 0x000000000061c725  /data/app/com.*************-uG2T4LshnAqPrF1eFaJSzA==/lib/arm/libunity.so
    3.  
    to see what function are you in, assuming you exported symbols during your build
     
    Last edited: May 23, 2023
    andrew_pearce_ likes this.
  42. andrew_pearce_

    andrew_pearce_

    Joined:
    Nov 5, 2018
    Posts:
    164
    Thank you very much for your reply @Tomas1856!

    When I tried "Stacktrace Utility", added path to libunity.so file, I got the following result:

    Code (csharp):
    1. #0 (?? ??:0)3  pc 0 (?? ??:0)x0 (?? ??:0)0 (?? ??:0)0 (?? ??:0)0 (?? ??:0)0 (?? ??:0)0 (?? ??:0)0 (?? ??:0)0 (?? ??:0)0 (?? ??:0)0 (?? ??:0)61c725  /data/app/com.app.name-uG2T4LshnAqPrF1eFaJSzA==/lib/arm/libunity.so
    Then I tried "nm" tool from "GNU Binutils" but there is no such address there:

    Code (csharp):
    1. .\nm.exe -C libunity.so > libunity.txt
    Do you have any suggestion what I am doing wrong?
     
  43. Tomas1856

    Tomas1856

    Unity Technologies

    Joined:
    Sep 21, 2012
    Posts:
    3,669
    Did you use libunity.so from symbols package? armeabi-v7a folder?
     
  44. andrew_pearce_

    andrew_pearce_

    Joined:
    Nov 5, 2018
    Posts:
    164
    @Tomas1856 I tried both libunity.so from "arm64-v8a" and "armeabi-v7a". Is there a chance that I am doing it wrong?





    This is how symbols were created:

     
  45. Tomas1856

    Tomas1856

    Unity Technologies

    Joined:
    Sep 21, 2012
    Posts:
    3,669
    Open Internal Log window in logcat package and see how the command for resolving stacktrace look like, I am guessing the regex somehow incorrectly parses your line
     

    Attached Files:

    andrew_pearce_ likes this.
  46. andrew_pearce_

    andrew_pearce_

    Joined:
    Nov 5, 2018
    Posts:
    164
    Thanks for all your efforts @Tomas1856. Here is the result from log cat:

    By the way Tomas, is there an artificial method to generate an ANR on a device in order to obtain a dump? If so, I can attempt to extract the function name. If this approach proves successful, we can then conclude that the issue might lie with Google Play, which may be providing an incorrect address or dump.
     
  47. Tomas1856

    Tomas1856

    Unity Technologies

    Joined:
    Sep 21, 2012
    Posts:
    3,669
    It seems there's a problem with regex which finds address from the line, in Stacktrace Utility click Configure Regex, and click Reset button to restore regexes, the first regex should look like this:

    Code (CSharp):
    1. \s*#\d{2}\s*pc\s(?<address>[a-fA-F0-9x-x]+).*\/(?<abi>\S+)\/(?<libName>lib.*)\.so
    then the passed address should be 0x000000000061c725 instead of 0..


    As for causing ANR, try this https://docs.unity3d.com/ScriptReference/AndroidJavaRunnable.html, and cause a delay in
    runOnUiThread, it should be longer than 5 seconds.
     
    andrew_pearce_ likes this.
  48. andrew_pearce_

    andrew_pearce_

    Joined:
    Nov 5, 2018
    Posts:
    164
    Great news, Tomas! We have made significant progress. You were absolutely right about the address not needing to start with "0x". Consequently, we have adjusted our regular expression to align with your suggestion. Let me show you what we have achieved so far:

    BTW Tomas, what in the dump led you to believe that this specific line could trigger the ANR? I had always assumed that we should be looking for a thread with a "Waiting" status. However, the thread containing "60baa5" actually has a "Native" status.

    What should be our next course of action once we have identified the function name? It appears to be a function related to Unity. Additionally, I have examined two other frequent ANRs and obtained the following information:

    We have a class named "AudioManager" in the project. However, it's quite a generic name, making it challenging to identify is it our "AudioManager" class or Unity's or Android's =) Tomas, if my questions seem too basic and wastes your time, please feel free to provide me with a link to any relevant documentation, article which I should read in order to deepen my understanding.

    Thank you in advance for your help!
     
  49. Tomas1856

    Tomas1856

    Unity Technologies

    Joined:
    Sep 21, 2012
    Posts:
    3,669
    Those are mainly guesses from my side, I see that both UI/main thread have WaitHoldingLocks, so I thought there's a deadlock here. It's also possible something's wrong here:

    Code (CSharp):
    1. "FinalizerDaemon" tid=4 Waiting
    2.   at java.lang.Object.wait (Native method)
    3.   at java.lang.Object.wait (Object.java:422)
    4.   at java.lang.ref.ReferenceQueue.remove (ReferenceQueue.java:188)
    5.   at java.lang.ref.ReferenceQueue.remove (ReferenceQueue.java:209)
    6.   at java.lang.Daemons$FinalizerDaemon.runInternal (Daemons.java:235)
    7.   at java.lang.Daemons$Daemon.run (Daemons.java:106)
    8.   at java.lang.Thread.run (Thread.java:764)
    Finalizer thread is trying to remove references, but main and Unity thread are trying to create reference, obviously you cannot create references while there's a deletion in progress. Sadly just from stacktrace it's hard to say. We would need to know what kind of object is being deleted.
     
    andrew_pearce_ likes this.
  50. andrew_pearce_

    andrew_pearce_

    Joined:
    Nov 5, 2018
    Posts:
    164
    Thank you, Tomas! Could you please explain how you inferred from the preceding 8 lines that Unity is attempting to create references?

    Is there a method to detect ANR from the code, allowing us to report it to Crashalytics? We need some information about the occurrence to aid in reproduction, as it is quite rare, with only a 0.3% chance. I recall someone having written an Android plugin that continuously invoked the main thread to detect ANRs, but unfortunately, I have lost the thread.

    p.s. Tomas, I often encounter bugs for which there seems to be no solution. For instance, recently I discovered that in some exceedingly rare cases, properties referencing a prefab could be null, while three other properties holding another implementation of the same type prefab were correctly set. As no one could pinpoint the cause of this issue, I resorted to adding a null check. The second bug pertained to Asset Bundles, which should not be loaded and subsequently unloaded if a scene loading process is underway. This led to textures turning white when attempting to load the bundle again at a later stage =)