Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. We have updated the language to the Editor Terms based on feedback from our employees and community. Learn more.
    Dismiss Notice
New Forum User Notice Update to the Unity Editor Software Terms

Bug Wrong colors in memory Map - False positive fragmentation

Discussion in 'Profiler Previews' started by Fishing_Cactus, Jul 8, 2020.

  1. Fishing_Cactus

    Fishing_Cactus

    Joined:
    Nov 8, 2014
    Posts:
    45
    Hi,

    We've been using the Memory Map to hunt down memory fragmentation problems and to analyze how our game memory is layed out.
    On this project we have some native memory regions (pools) with a size of 16Mb.
    At first glance, those pools seem as good as empty in the memory map view. The dark green is Native Memory (Reserved).
    upload_2020-7-8_10-4-8.png
    Already here we can see that there are some black lines in our 16Mb pool section.
    To the best of my knowledge, this could mean it's Virtual Memory, as that's black in the legend?

    Anyway, we were baffled our memory regions were empty.
    But if we look at the allocation list for this region, it turns out it's not empty at all...
    upload_2020-7-8_10-6-46.png
    If you sum all this, you get 18944Kb of data. with the rounding happening for the MB display, it's fair to say that we are completely using up our memory buffer.
    So, where is the light green "Native Memory (Allocated)" then? Shouldn't it be completely light green?

    Let's drill deeper and look at those black bars:
    We now use rows of 32Kb to try and see what those black bars are.

    It turns out, those black bars seem to be my allocated memory.
    upload_2020-7-8_10-11-20.png
    Everything selected in between the black bars show nothing at all in the allocation list:
    upload_2020-7-8_10-12-30.png
    And then the last thing, the size of those black bars seem to be very wrong.
    As you can see in the previous image, the little black bar is a size of 3.7Kb on a 34Kb row. It should be around 1/10th of the width of the bar.
    But....
    upload_2020-7-8_10-14-53.png
    This small section is 34.1Kb and it takes up about the same amount of space as the 3.7Kb section, while instead it should span more than the entire line.

    The fact that these visuals don't seem to make a lot of sense, makes it very hard for us to actually drill down on our memory problems and we're almost out of time to fix them.

    @MartinTilo could you confirm if my analysis is correct?

    If so, I'm willing to modify the package locally to try and have an accurate visual representation but any help would be appreciated.

    Thank you in advance,

    Pieter
     
    Fishing_Cactus, Jul 8, 2020
    #1

  2. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,247
    Hi Pieter,

    I am, to be honest, not quite sure why there are no lighter green bars in your first two screenshots. This could be a bug in the visualization or with the UI where maybe the Display Filters (top right hand drop down) got into a bad serialization state and the Allocations are filtered out. I guess in that case you shouldn't see the legend item for "Native Memory (Reserved)" though so that seems unlikely.

    Could you please file a bug report with this? Just the snapshot file attached should be enough for us to reproduce but please ping me the bug ID so that our Customer QA doesn't ask you for a full repro project (they can't always make the right judgment calls in these cases as it requires more knowledge of the Memory Profiler system).

    As to the black sections: your selections span the beginning of the next green section and that's the one you are seeing in the table. The black sections are empty. So it would seem the selection only checks if the address is within the range of the selection, not any point between address + size.

    I have to admit: that is UX wise a bit unfortunate. I guess there is a balance to be struck between avoiding that you accidentally overlap your selection into the previous Region/Allocation/Object by a pixel margin (and getting confused about seeing that previous item in your list), and the selected range showing any larger scope items that it overlaps with partially, so that the confusion you experienced here is avoided. I've added an issue tracker item for this one myself.

    Now as to why those black sections are so big, I'm not too sure myself. I measured and made a calculated estimate that this gap could be ~194B (+- pixel and measurement inaccuracies. This was literally me holding a ruler against the screen, dividing the width of the black section by the whole line width and multiplying with 32 KB, ergo, super scientific).

    I'll check back on this one with my colleagues, but for now, I don't believe ~194B gaps of unused virtual address space should have any (significant) impact on the memory usage of you project.
     
    MartinTilo, Jul 9, 2020
    #2

  3. Fishing_Cactus

    Fishing_Cactus

    Joined:
    Nov 8, 2014
    Posts:
    45
    Hi @MartinTilo
    I really appreciate your fast and detailed replies on the forums. Thanks for that!

    Issue submitted with ID: 1261948

    Thanks,
    Pieter
     
    Fishing_Cactus, Jul 9, 2020
    #3

  4. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,247
    Thank you. I've found the main issue with the colors being wrong (linear vs non-linear color space) but still digging a bit further :)

    P.s. Here's the public issue tracker link for this.
     
    Last edited: Aug 3, 2020
    MartinTilo, Jul 28, 2020
    #4

  5. NuclearC00kie

    NuclearC00kie

    Joined:
    Apr 11, 2019
    Posts:
    32
    Hi @MartinTilo,

    Great to hear you've found the main issue with the color being wrong! I can't wait to see that in the memory profiler package.

    Unfortunately the public issue tracker link doesn't seem to be public.

    Kind regards,
    Pieter
     
    NuclearC00kie, Aug 3, 2020
    #5

  6. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,247
    dmn, I was sure I had fixed it :D, now it is.
     
    MartinTilo, Aug 3, 2020
    #6
    NuclearC00kie likes this.

  7. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,247
    FYI, the fix for the Linear Color Space issue has just landed with the release o com.unity.memoryprofiler@0.2.6-preview.1. As noted out on the issue tracker item. We'll continue to look into further issues with colors and Memory Map though.
     
    MartinTilo, Oct 1, 2020
    #7

  8. NuclearC00kie

    NuclearC00kie

    Joined:
    Apr 11, 2019
    Posts:
    32
    Thank you for the update. Glad to see a long awaited update to the memory profiler :)
    I hope the "big internal refactor" went well.
     
    NuclearC00kie, Oct 1, 2020
    #8
    MartinTilo likes this.

  9. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,247
    hah, that's still ongoing :D
    This release is mostly bug fixes and smaller improvements with initial refactor changes under the hood that fit in with these fixes.
     
    MartinTilo, Oct 1, 2020
    #9
    NuclearC00kie likes this.