Search Unity

  1. We are migrating the Unity Forums to Unity Discussions. On July 12, the Unity Forums will become read-only.

    Please, do not make any changes to your username or email addresses at id.unity.com during this transition time.

    It's still possible to reply to existing private message conversations during the migration, but any new replies you post will be missing after the main migration is complete. We'll do our best to migrate these messages in a follow-up step.

    On July 15, Unity Discussions will become read-only until July 18, when the new design and the migrated forum contents will go live.


    Read our full announcement for more information and let us know if you have any questions.

Bug Unity Analytics (New) in Offline Mode

Discussion in 'Unity Analytics' started by PeachyPixels, Feb 22, 2022.

  1. PeachyPixels

    PeachyPixels

    Joined:
    Feb 17, 2018
    Posts:
    716
    Hello!

    I've assigned this as a bug, but it could equally be argued as implementation.

    So one of my test scenarios with the new Analytics was to run it in offline (airplane) mode.

    When doing so I observed...

    UnityServices.InitializeAsync() initialised after 1 second with a State of Initialised (which is great)

    Events.CheckForRequiredConsents() failed after 10 seconds (fair enough, it needs a geo-lookup call)

    However, when then calling Events.Flush() it throws the following exception (I'm assuming auto flush does the same)...

    ConsentCheckException: The required consent flow cannot be determined. Make sure GeoIP was successfully called.

    So this means all calls to Flush() need to be wrapped with exception handling, which is a pain. Could this not be built into the API please?

    Or isn't there a case for supressing consent checks if required consents could not be determined? Or a fallback is device locale? Or a cached locale from a previous online session? Given it requires a UI and player input, anything other than on app init is not going to work well.

    Basically, the demo code pattern showing await Events.CheckForRequiredConsents() causes knock on effects when in offline mode and I can see this catching a lot of people out.
     
    Last edited: Feb 22, 2022
  2. JeffDUnity3D

    JeffDUnity3D

    Joined:
    May 2, 2017
    Posts:
    14,446
    @PeachPixels I just tested the following code with no Internet, and it displays the following. If this were the case, you would not want to send any events nor call Flush(). We may need to update the documentation.

    Error reason = NoInternetConnection.

    Code (CSharp):
    1. async void Start()
    2.     {
    3.      
    4.         string consentIdentifier = "";
    5.         bool isOptInConsentRequired = false;
    6.  
    7.         try
    8.         {
    9.  
    10.             var options = new InitializationOptions();
    11.             options.SetAnalyticsUserId("my_test");
    12.  
    13.             await UnityServices.InitializeAsync();
    14.             List<string> consentIdentifiers = await Events.CheckForRequiredConsents();
    15.             Debug.Log("Count = " + consentIdentifiers.Count.ToString());
    16.             if (consentIdentifiers.Count > 0)
    17.             {
    18.                 consentIdentifier = consentIdentifiers[0];
    19.                 isOptInConsentRequired = consentIdentifier == "pipl";
    20.             }
    21.             if (isOptInConsentRequired)
    22.             {
    23.                 Events.ProvideOptInConsent(consentIdentifier, false);
    24.             }
    25.         }
    26.         catch (ConsentCheckException e)
    27.         {
    28.             Debug.Log("Error reason = " + e.Reason.ToString());
    29.         }
    30.  
    31.         Debug.Log("Done with Start!");
    32.      }
     
  3. PeachyPixels

    PeachyPixels

    Joined:
    Feb 17, 2018
    Posts:
    716
    In offline mode does the new Analytics not cache events locally? Am sure the documentation states it does.

    UPDATE: From the docs...

    There might be occasions where the SDK fails to send the Event, for example, when the Analytics service is down or the player is offline. The Events are stored in-memory (and to disc) until they can be uploaded.

    If so, it's a fair assumption that if a developer can call Events.CustomData() in offline mode, the same should be for Events.Flush(). Even if it just silently exits and leaves the cached items for the next attempt.

    Of course this can all be handled with a more complete pattern or handling in our own abstracted classes (in my case) but I can see a lot not doing this and getting caught out.
     
    SebT_Unity likes this.
  4. JeffDUnity3D

    JeffDUnity3D

    Joined:
    May 2, 2017
    Posts:
    14,446
    Good point! Again, we will follow up.
     
    PeachyPixels likes this.
  5. PeachyPixels

    PeachyPixels

    Joined:
    Feb 17, 2018
    Posts:
    716
    Hey Jeff!

    With a fresher mind on this today, I've some more findings to report...

    1.

    UnityServices.InitializeAsync() does not need to be wrapped in the exception block. It will initialise successfully in offline mode and State will be set to Initialised

    That's all good and for ease of use, the preferred approach imo.

    2.

    Events.CheckForRequiredConsents() does need to be wrapped in the exception block and will fail in offline mode (which it's clearly been designed to-do)

    For consistency and ease of use, it might be better to follow the same approach as InitializeAsync. That is it should not fail (if offline) and its status determined by a property (much like State). Even better, why not wrap CheckForRequiredConsents() into InitializeAsync() and deal with it (in the app) via a callback?

    In the event of a failed consent process then Events.CustomData() & Events.Flush() should still work (see point 4)

    3.

    As mentioned previously (and without knowing GDPR/CCPA/PIPL & App Store rules inside\out) having fallbacks for Events.CheckForRequiredConsents() offline mode is much needed. Even a fallback of 'assume no consent given' is preferable.

    4.

    I tested two Events.CheckForRequiredConsents() scenarios...

    a) If the device was offline when this call is made, from that point on I could not see any events being cached to storage (Editor\Analytics\ArchivedEvents?) or sent if the device came back online. It seems that if this call fails, Analytics events will not be sent even with auto flush. The exception was that I did see a few standard events, but definitely no custom events.

    b) If the device was online but the call was omitted (on game init) then events are not cached to storage (Editor\Analytics\ArchivedEvents?) but they are sent if using auto flush (every 60 seconds)

    In both situations Events.Flush() fails. The only way it will work is if successful consent has been given.

    Events were observed using Charles Proxy.

    So basically..

    The offline workflow when using Events.CheckForRequiredConsents() is breaking the new Analytics at present. It appears better to ignore the consent process (if offline) and manual flushes (if no consent given)

    These are just my findings and may not be complete. I'm sure there are others scenarios that could be tested. On-top of that, you may see different behaviour.

    But hopefully this all helps! :)
     
    Ohilo likes this.
  6. PeachyPixels

    PeachyPixels

    Joined:
    Feb 17, 2018
    Posts:
    716
    Hey Jeff!

    I'm moving away from the new Analytics for the time being, but just observed the following...

    If the device is offline at init then UnityServices.InitializeAsync() sets the state to Initialised but... I'm not seeing any events being sent if the device then comes back online.

    The above call throws the following exception...

    Curl error 28: Operation timed out after 10000 milliseconds with 0 bytes received

    So assuming this is not being handled & re-thrown, then internally the API is setting State but has left itself in an undetermined state.

    Basically if a device is offline during init then Analytics never appears to cache or send events, even if the device comes back online.

    So...

    Offline
    UnityServices.InitializeAsync()
    Events.CheckForRequiredConsents() <- Optional
    Online
    ...
    Events.CustomData() -> No event sent (or auto flush not being called)
    ...
    Exit Game -> No event sent (or auto flush not being called)

    FWIW legacy Analytics would handle and recover far more gracefully in offline situations, so definitely some changes needed in this area with new Analytics.

    I hope this all helps!
     
    Ohilo and JeffDUnity3D like this.
  7. Laurie-Unity

    Laurie-Unity

    Unity Technologies

    Joined:
    Mar 5, 2020
    Posts:
    236
    Hi David,

    Thanks for all the valuable feedback.

    Can you confirm which build version of the Analytics packager you were using when you experienced these issues. I'm asking because we did release an update to the Analytics package to v3.0.0-pre.4 on Tuesday 22nd. It contained a fix for one known caching issue, but I'm not sure if that will resolve this issue.

    Jeff and I will investigate further along and share your feedback with our engineering colleagues

    Thanks
    Laurie
     
  8. PeachyPixels

    PeachyPixels

    Joined:
    Feb 17, 2018
    Posts:
    716
    Hey @Laurie-Unity

    Thanks for the update.

    I'm running v3.0.0-pre.4 here, installed on Tuesday.

    Certainly for me, the offline issues are the biggest problem right now. That said, when online it's working well and the new dashboard is big step forward :)

    Anything else I can help with, please just ask.
     
  9. Laurie-Unity

    Laurie-Unity

    Unity Technologies

    Joined:
    Mar 5, 2020
    Posts:
    236
    Hi David,

    I'm delighted to hear that you like the new dashboard and that its working well whilst the player is online.

    I have some feedback from my engineering colleagues on the offline issue. For now you will need to perform an additional consent check when the player comes back online, it isn't automatic.

    We are working hard on improvements and new capabilities to simplify the various privacy workflows required by the growing number of legislations that you need to comply with. We will be sure to post on the forum as soon as we have more information that we can share.

    Please continue to share you thoughts, suggestions and any issues that you run into. It is really helpful and we are very grateful.

    Thanks,
    Laurie
     
  10. PeachyPixels

    PeachyPixels

    Joined:
    Feb 17, 2018
    Posts:
    716
    Thanks @Laurie-Unity

    My game brings the various sub-bsystems online during game init in a linear fashion (using various coroutinues). Some (like analytics, ads etc) will try for N number of seconds then continue the init process if it the sub-system didn't initialise.

    Those sub-systems will then keep trying in the background (every N seconds) until they do initialise. None of these sub-systems are mission critical, so they won't stop gameplay if not initialised.

    The new analytics documentation states that events are cached even if the service is not initialised, then flushed when it eventually is. So I'm just testing the process now with various scenarios (offline before init and after etc) to see how well that works.

    The trouble with the consent process as it stands is that ultimately it is very tied to having internet access, which is far from guaranteed with mobile devices. It also requires player input and obviously the consent ui can't popup mid-game (if that's when the device comes online). So for most this would mean adding the process to game init or at the latest, the home menu.

    That was my thinking behind having (non-online) fall-back processes. It means consent can always be determined during game init.

    Of course app store policies or legal may have other ideas on that :)
     
    Last edited: Feb 24, 2022
  11. PeachyPixels

    PeachyPixels

    Joined:
    Feb 17, 2018
    Posts:
    716
    Hi @Laurie-Unity

    So I've worked through a few offline scenarios with my new workflow (which initialises Analytics in the background only when the device has internet access) and the findings are below. It just shows one of my own custom events (FeatureVisited) and ignores standard events.

    Scenario 1 (Device Offline on App Start, Device Online Mid-Game)

    1) Device Offline
    2) App Start
    3) Analytics Init Fail (Background Retry Every N Seconds)
    4) Home Menu
    5) Visit FeatureA (Settings)
    6) Level Start
    7) Visit FeatureB (Pause)
    8) Device Online
    9) Analytics Init Success [FeatureA, FeatureB Events Posted - Seen in Charles Proxy]
    10) Level End
    11) App Exit

    So as the documentation states, the events were cached (in memory?) and sent when the device came online. Which is good.

    But obviously consent shouldn't be requested mid-game (stage 9) so in this scenario stage 3 or 4 are really the best options. Now that's not a difficult feature to build, but it's not made clear in the documentation and I can see others getting caught out by this.

    Scenario 2 (Device Offline for Session 1, Online for Session 2)

    1) Device Offline
    2) App Start
    3) Analytics Init Fail (Background Retry Every N Seconds)
    4) Home Menu
    5) Visit FeatureA (Settings)
    6) Level Start
    7) Visit FeatureB (Pause)
    8) Level End
    9) App Exit
    10) Device Online
    11) App Start
    12) Analytics Init Success [FeatureA, FeatureB Events NOT Posted - Not Seen in Charles Proxy]

    So in that scenario all events are lost. If this is a feature rather than bug, couldn't in memory events be flushed to storage on app exit for sending on the next successful init?

    I've implemented Beta Analytics as best as possible for now, so will be moving onto something else until the next version.

    Please reach out if you need any further info etc though.

    Good luck! :)
     
    Last edited: Feb 24, 2022
    Ohilo likes this.
  12. mylastggeast

    mylastggeast

    Joined:
    Jun 14, 2021
    Posts:
    43
    Thanks for the detailed flows @PeachyPixels!
    Do you know if the above is the current state of the offline use cases for the New Analytics or has any progress been made with later versions? @Laurie-Unity might also know more about this?
    I am using 4.2.0 currently btw!
     
    PeachyPixels likes this.
  13. PeachyPixels

    PeachyPixels

    Joined:
    Feb 17, 2018
    Posts:
    716
    I've no idea to be honest. I converted my abstraction layer to the new Unity Analytics, tested as best as possible, then moved onto other things.

    Current thinking (for my next game) is to actually not support Analytics at all. But if that changes I will re-test (with the current Verified version of Unity Analytics) and post here if there are any significant updates.

    Good luck with your implementation.
     
    mylastggeast likes this.