Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. We have updated the language to the Editor Terms based on feedback from our employees and community. Learn more.
    Dismiss Notice
  3. Join us on November 16th, 2023, between 1 pm and 9 pm CET for Ask the Experts Online on Discord and on Unity Discussions.
    Dismiss Notice
  4. Unite 2023 Registrations are now LIVE! We are thrilled to announce that Unite 2023 Registration is now live and open for all!
    Dismiss Notice
New Forum User Notice Update to the Unity Editor Software Terms Unite 2023 - Ask the Experts Online Unite 2023 Registrations are now LIVE!

Memory Profiler: Memory Map view legend/docs incomplete

Discussion in 'Profiler Previews' started by bgrz, Jun 9, 2020.

  1. bgrz

    bgrz

    Joined:
    Mar 22, 2015
    Posts:
    59
    There's a lot of gray in Memory Map view of our game build's snapshot:

    2020-06-09_18-18-07.png

    I presume it's unallocated/unused memory, but neither the documentation nor legend above says what it is.
    What prompts me to doubt my presumption is that there's so much of it. It's like more than half of memory is unused. I couldn't even call this "fragmentation" because gray chunks are huge. Can anyone clarify?
     
    bgrz, Jun 9, 2020
    #1

  2. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,164
    Hi,
    thank you for pointing out that the manual and legend are lacking in this regard, we'll get that sorted.
    Some info for now:
    The non-gray chunks represent virtual address ranges owned by Unity's memory manager, mapped onto the virtual address space. Virtual address space is local to every app and doesn't correspond 1-to-1 to actually physical address ranges (which Unity doesn't know about because that is obfuscated by the OS). As an app requests memory, it will get virtual address ranges starting at address 0 for the first allocation and the 0+size-of-first-allocation for the second and so forth, only ever moving forward. If memory is freed and unmapped, that virtual address range will now be unused forever. That doesn't mean the physical space it occupied won't be reused though. It might just get mapped to a new virtual range that could even have its chunks split up across different smaller sections of physical memory. So that's not memory fragmentation.

    One other thing to note here is that there might well be native plugins that can request Memory in the same virtual address space, but these plugins can't currently report their allocations to unity's memory manager, so it can't know about the contents of these ranges.

    This means empty spaces here can mean two things: the Memory has been used by Unity at some point and got unmapped, is still used by a native plugin or external library.

    Now regarding fragmentation: if you have mapped ranges of managed or native memory that contain some memory that is still used but that also contain unused space, that might be fragmentation. In case of managed Memory, only the latest address range of managed heap is used to Allocate new managed heap memory, i.e. "Objects". All previous ranges are waiting to be emptied completely before eventually being returned to the OS. So any empty space you see in these is problematic. Or rather, the non empty space is, because whatever is in there is what's keeping the range as reserved memory.

    For the native memory, this is a bit different as the memory ranges could be a that of a pool, buffer or heap that is still actively used for new native allocations. Provided the space in there is fitting for new Allocations there is no problem.
     
    MartinTilo, Jun 11, 2020
    #2
    Fishing_Cactus likes this.

  3. bgrz

    bgrz

    Joined:
    Mar 22, 2015
    Posts:
    59
    Thanks for the quick and detailed reply, it's really informative. What you wrote could very well be just copy pasted to Memory Profiler docs as its own section. For posterity I'll write a bit more about our case here, context might be of use to someone else reading this.

    What prompted me to post here (and really use Memory Profiler in the first place) is that in the build of our game there is a large discrepancy between what's reported by Task Manager's default view "Memory usage" and "Commit Size" column (in this case ~4 vs ~10 GB, but after some time playing it gets to 8 vs 19 GB)

    Taskmgr_2020-05-22_01-20-45.png Taskmgr_2020-05-22_01-21-05.png

    which kind of corresponds to the non-gray to gray ratio in the Memory Map view. Commit size never seems to go down by much, a couple hundred megabytes most, and thankfully doesn't seem to grow indefinitely, the rise slows down around 19-20GB and seems to stay capped there. I didn't do a snapshot in that case (because it'd probably take too long or even crash), the one above was shortly after running the game (takes about 5 minutes to take a snapshot), after one or two calls to Resources.UnloadUnusedAssets have been made (which are also called during regular play). This large "Commit Size" memory usage causes the game to crash for many players, and I wasn't sure if that memory is really put to good use by Unity.
     
    bgrz, Jun 11, 2020
    #3

  4. Fishing_Cactus

    Fishing_Cactus

    Joined:
    Nov 8, 2014
    Posts:
    45
    @MartinTilo I've been looking at a good description of how memory fragmentation can be spotted using the Memory Map view and this is the first time I see a detailed answer like this. I agree with bgrz that this would be a great addition to the memory profiler's documentation.
    Thanks for finally helping me understand how to properly use this.
     
    Fishing_Cactus, Jul 8, 2020
    #4
    MartinTilo likes this.

  5. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,164
    Glad this could help :)

    I'm working on a bigger Documentation update to clear up on ideally all things Memory in Unity. As such, it is probably gonna take a bit longer to write this up but should provide quite some value for anyone, even us internally at Unity (there aren't too many people around that have a complete picture of this, even internally and to be fair, that might be more than people need most of the time. I'm not even sure if I count to these few so this'll be a team effort too ;)).
     
    MartinTilo, Jul 8, 2020
    #5