Search Unity

  1. Unity Asset Manager is now available in public beta. Try it out now and join the conversation here in the forums.
    Dismiss Notice

Bug [1263103] Incorrectly summed size in diff view

Discussion in 'Profiler Previews' started by Fishing_Cactus, Aug 11, 2020.

  1. Fishing_Cactus

    Fishing_Cactus

    Joined:
    Nov 8, 2014
    Posts:
    45
    Hi,

    I've reported case 1263103 a while ago.
    Here's the description of the issue, without screenshots, as it's a copy of the bugreport:

    1. What happened
    When taking 2 snapshots, captured in 2 different sessions, and then comparing their values, the sizes reported in the diff view are incorrect.

    Let's load each capture individually, and look at the reported Texture2D value, in the memory map view:

    Snapshot 1 (RC41) - Texture2D: 351.1 MB
    Snapshot 2 (RC64 3) - Texture2D: 348.4 MB
    if we manually compare the difference in size here, we end up with a difference of 2.7 MB (351.1 - 348.4 == 2.7)

    Now let's diff this using the memory profiler diff button
    When diffing these 2 snapshots and comparing the Texture2D sizes between each, the total Owned Size doesn't make sense.
    I group on [Diff] and on [Type] to get the following data:

    Same: 174.1 MB, Deleted: 177.1 MB, New: 122.1 MB.
    If we sum these values, there's a problem.
    Same + Deleted = 351.2 MB (seems to match total value of Snapshot 1)
    Same + New = 296.2 MB --> This value is completely off. The total should be 348.4 MB. We're missing 52.2 MB here.

    2. How we can reproduce it using the example you attached
    * Import the 2 snapshots in the memory profiler
    * Load each snapshot invididually and apply filters: GROUP on Type, MATCH on type with "Texture2D" as value
    * Verify that Owned Size for Texture2D in the snapshots is around 350 MB each (see exact numbers in step 1)
    * Load both snapshots and diff using the button on the bottom left of the Memory Profiler UI.
    * Apply the following filters: GROUP on [Diff], GROUP on Type, MATCH on type with "Texture2D" as value.
    * Inspect Owned Size and observe that the values don't add up. (see step 1 for details)


    QA closed this issue, saying the following:
    Capturing the Snapshots in different game sessions is unfortunately expected to cause this kind of behavior even if the Editor isn't closed and reopened in between them.

    We understand that this is causing some problems and confusion, and so it will be addressed in a future version.

    Could somebody please explain why this is expected? It looks like the needed data is present in each snapshot, but that the diff view has some math operation problems.
    It would feel good to get some extra understanding of this, after all, we take the time to report these bugs and if they get closed without a good explanation, it gets demotivating to spend the time to setup new reproduction projects to report bugs.

    Thank you in advance,

    Pieter
     
    Fishing_Cactus, Aug 11, 2020
    #1

  2. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,456
    Hi Peter
    and thank you again for that bug report. While we did close it, we do want to make this work better in a future version and the bug report inspired an idea on how to solve this, so it was not in vein.

    The main issue is that the Memory Profiler doesn't know that the snapshot came from two sessions. It then tries to diff the snapshot and find similar or changed objects based on their instance IDs and/or addresses, both of which could change between the sessions, leading to wrong identifications of objects. The GUIDs for assets do not exist at runtime and are therefore not captured by the Memory Profiler. So what looks to a human like it's obviously the same object, with the same name and because we know which asset it came from, is un-identifiable to the profiler.

    Besides reworking the way we diff and present the diff views, capturing a session ID with the Memory snapshots* will allow us to be able to say: hold on, those are two different sessions, be very conservative in matching what is the same.

    Additionally the high-level grouping and showing what is different shouldn't use the objects found as same or different but do it's own math, to prevent such confusion.

    I was quite hesitant to us closing the bug but since we can't quite predict when we'll get to reworking this, it felt like the more honest and forthright option. The alternative would have been us sitting on a bug until we build several new features and basically reworked the entire underlying system.

    (* That was the idea we got from this.)
     
    MartinTilo, Aug 11, 2020
    #2
    Fishing_Cactus likes this.

  3. Fishing_Cactus

    Fishing_Cactus

    Joined:
    Nov 8, 2014
    Posts:
    45
    Hi Martin,
    Thank you for going in such detail about this bug. That's refreshing!
    So, if I understand it correctly, when loading each snapshot individually, we know which object corresponds to which type. That's why we see accurate values when viewing each snapshot individually.
    But when we diff 2 snapshots, you don't actually crawl the "type" data from both snapshots, hence you can't know for sure what type a certain object is? Or are objects really misidentified as "the same object" and is that the source of this bug

    Here's a use case of what I was using the tool for when encountering this bug:
    We have issues with memory being to close to the maximum allowed amount.
    To fix this, usually I do the following:
    * Profile my scene with memory profiler
    * Use the memory map to spot areas that can be improved.
    * Take the necessary actions (reduce texture size, remove some dependencies, swap materials, ...)
    * Rebuild
    * Profile scene again with memory profiler
    * Compare the 2 snapshots to quantify how much difference this made, and to make sure I didn't lose memory in unexpecteds places.


    Another use case:
    We saw a big increase in base memory consumption in our game compared to a few months ago.
    I was planning to use the memory profiler to quickly identify what is going on here, so I made a snapshot in the old build, and made a snapshot in the new build, on the same place.
    I was then hoping to use the diff view to figure out which assets were not present in the old build.
    I understand that the tool has big difficulties matching assets accross sessions, so I'm not expecting all assets to show as "same", except for what has been added.
    But I was hoping to group on types to at least be able to see which type of objects got changed. That would point me in a direction for quickly fixing the issues.

    I guess this could be done manually with CSV export, by loading each snapshot individually, exporting to CSV, and grouping by type in a data processor software, but on the other hand, this also seems like it could be core functionality of this tool.

    If you have any feedback on how to tackle those 2 use cases, please let me know.

    Kind regards,
    Pieter
     
    Fishing_Cactus, Aug 11, 2020
    #3

  4. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,456
    No. @alexrvn might have to correct me on this one but the type would be the same, just e.g. Texture 1 in snapshot 1 would be miss-identified as the same as Texture 2 in snapshot 2. Or it might even be the same audio asset that is identified as the same but in one instance it was the first one loaded and therefore got the buffer in the FMod plugin attributed to itself (a known quirk of the audio system).

    Point being, the objects aren't being checked to have the same size (and arguably, between sessions the size of assets could have changed).

    Thank you also for highlighting those two usecases. Yes, the Memory Profiler should be usable to provide some high level comparisons in those cases too, which would be part of an overhauled way to diff the snapshots and just generally providing more high level summaries of the snapshot contents. I'll add them to our feedback tracker :)

    As for know, there's sadly little to go by except for visually diffing, i.e. toggling between comparable views in both snapshots and using the Open snapshot dialog to switch between the snapshots... Which is quite obviously sub-optimal :(
     
    MartinTilo, Aug 11, 2020
    #4

  5. Fishing_Cactus

    Fishing_Cactus

    Joined:
    Nov 8, 2014
    Posts:
    45
    Thank you for your response Martin,
    We'll manage!
     
    Fishing_Cactus, Aug 12, 2020
    #5
    MartinTilo likes this.