Search Unity

  1. We are migrating the Unity Forums to Unity Discussions. On July 12, the Unity Forums will become read-only. On July 15, Unity Discussions will become read-only until July 18, when the new design and the migrated forum contents will go live. Read our full announcement for more information and let us know if you have any questions.

Bug Player crash when comparing strings - possible lack of type checking when loading object reference

Discussion in 'Android' started by BDG_Toast, Apr 19, 2024.

  1. BDG_Toast

    BDG_Toast

    Joined:
    Sep 12, 2022
    Posts:
    14
    Hi there,

    Not sure what the best place to report player crashes/engine bugs/il2cpp runtime bugs, so I'll leave it here and hopefully the right people see it.

    Recently we ran into an Android player crash on Quest 2 that would very occasionally happen when the player performed a certain action. Luckily we were able to reproduce it internally and still had the pre-stripped libil2cpp.so handy from that build. After grabbing the tombstone and mapping the stack back to the debug symbols we identified we were encountering a bad pointer deref (sigsegv) in String_Equals_mXXXXX. This was obviously very surprising to see and I thought I might have made a mistake when mapping the symbols at first, but the rest of the stack made perfect sense considering when this crash happens. I took a peek at the disassembly for that symbol and indeed the register it was attempting to load had some garbage data in it.
    The specific instruction was ldrsw x8, [x0, #16] where x0 held 12d6f21e4d416be0. Looks like x0 and x1 hold the input string pointers, based on my very rudimentary understanding of arm.

    At this point it's obvious that the input is somehow malformed, so I identified which strings it was attempting to compare and where they came from. The comparison involved achievement ID strings which we store in achievement data scriptable objects, but we've never had issues with loading these scriptable objects previously. I could see from the stack trace that the asset reference actually came from our user save data, which is not possible since we serialise to JSON and can't serialise asset references... unless?

    From this I discovered that we had recently started serialising more achievement progress data to our user saves, in particular references to achievement data scriptable objects were being serialised to JSON directly, which had unfortunately slipped through our code review process. This obviously doesn't work as intended, and the output JSON only contains the object's instance ID, in effect we were serialising a transient object reference instead of an asset reference. Easy enough fix on our side, but this obviously shouldn't be possible so we've run into some sort of engine bug.

    If the string comes from the asset reference, and the asset reference comes from our saves, that indicates that we have somehow deserialised an instance ID into a "valid" object reference (from the point of view of the il2cpp runtime) without anything else breaking until we try to read read data from it.

    We saw the crash quite rarely, so I suspect most of the time loading the object reference gave us a null back and only gave us a "valid" object reference when there was an object with a matching instance ID in memory that's still "alive" (not destroyed/whatever). We try to read the achievement ID string from this and instead of the pointer to our ID string, we get some garbage data from a completely unrelated object. We then crash when we attempt to deref the garbage pointer. Or maybe there is even a valid string pointer in that address offset occasionally and our string comparison always fails.

    An alternative explanation would be that the loaded asset was replaced/stomped somehow and this has nothing to do with deserialising object references, but that seems less likely considering we've never had issues with instances of scriptable objects being invalidated/corrupted from underneath us before.

    I also don't know how unity object references are actually represented in the il2cpp runtime so I could be way off if it's not simply a pointer, but that would still seem to point towards a lack of type checking at some step in this process.

    This is all obviously wild speculation without source access, but the theory seems consistent with what we've observed. I unfortunately have not been able to replicate the crash conditions with a debugger attached or narrow down an exact set of repro steps, so I don't know if it only happens when we save & load in the same session or load a save from a previous session.

    Unity version 2022.3.16
    User saves (de)serialised using the builtin JsonUtility

    If I get some time I'll put together a test project that loads object references via JsonUtility and see if I can reproduce the crash, but I can't promise anything at this point in time sorry.

    Cheers,
    Brandon