Search Unity

  1. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice

Help Wanted 4x memory usage in Xcode GPU Frame Capture

Discussion in 'iOS and tvOS' started by joshuacwilde, Sep 4, 2020.

  1. joshuacwilde

    joshuacwilde

    Joined:
    Feb 4, 2018
    Posts:
    169
    I'm looking at the frame capture tool right now. I went to look at these heaps here and noticed they each store one texture, but their allocated size is 4x the contained texture size. For example, a heap will be allocated for 40 MB, but only contain a 10 MB texture. Or it will be allocated to 32 MB, but only contain an 8 MB texture. You can see this in the screenshot attached.

    Any ideas why this could be?

    We are trying to maximize memory use right now and this looks suspicious.
     

    Attached Files:

  2. joshuacwilde

    joshuacwilde

    Joined:
    Feb 4, 2018
    Posts:
    169
    After further testing, this appears to be due to using dynamic resolution. It will allocate the memory required for 100% screen scaling, then the actual texture gets dynamically allocated to be whatever % you set the dynamic scaling at.

    Is there a way I can force the memory to be lowered if I know the dynamic resolution will only ever be 75% or whatever?
     
  3. joshuacwilde

    joshuacwilde

    Joined:
    Feb 4, 2018
    Posts:
    169
    Found a bug in the unity memory profiler. Look at the sizes of the textures in the unity profiler compared to the sizes xcode reports. The sizes unity reports are always smaller, than the ones shown in xcode, thus unity will report a lower memory usage than is correct.

    I have one example selected in the screenshot, but if you study it, you will see a few examples of the texture sizes not lining up in those two screenshots.

    @aleksandrk
     

    Attached Files:

  4. aleksandrk

    aleksandrk

    Unity Technologies

    Joined:
    Jul 3, 2017
    Posts:
    1,660
    What's the texture size and format?
     
  5. joshuacwilde

    joshuacwilde

    Joined:
    Feb 4, 2018
    Posts:
    169
    Lots of varying sizes. Not sure what you mean. Also those formats should for the most part be PVRTC or ASTC
     
  6. joshuacwilde

    joshuacwilde

    Joined:
    Feb 4, 2018
    Posts:
    169
    I mean in that screenshot there are varying formats and sizes. They are all POT though.
     
  7. aleksandrk

    aleksandrk

    Unity Technologies

    Joined:
    Jul 3, 2017
    Posts:
    1,660
    I meant the texture that's highlighted :)
     
  8. joshuacwilde

    joshuacwilde

    Joined:
    Feb 4, 2018
    Posts:
    169
    Ah hmm this one is actually a bit odd. It is 300x300, so not POT. I forgot that our UI textures tend to not be POT. Also PVRTC. But this is supposed to be on an atlas, so odd to me that this is using gpu memory separate from the atlas. Also we are using half res in the quality settings for textures.

    Here is a screenshot:
     

    Attached Files:

  9. aleksandrk

    aleksandrk

    Unity Technologies

    Joined:
    Jul 3, 2017
    Posts:
    1,660
    It looks like it's showing you the original texture memory in the profiler, which is around 350KB for a RGBA 300x300 texture.
     
  10. joshuacwilde

    joshuacwilde

    Joined:
    Feb 4, 2018
    Posts:
    169
    Yeah good point. I noticed that after when I posted the image, but shouldn't this show the size in memory? or else what is the point of using it to profile runtime memory?

    I appreciate your help so far btw, thanks :)
     
    aleksandrk likes this.
  11. aleksandrk

    aleksandrk

    Unity Technologies

    Joined:
    Jul 3, 2017
    Posts:
    1,660
    I guess it's difficult to show per-texture memory usage when a texture is in an atlas :)
    There might be a way to highlight that it's actually part of an atlas, but that's a question to the people working on the profiler :)

    What does the profiler show as total texture memory used? Is it different from what Xcode reports?
     
  12. joshuacwilde

    joshuacwilde

    Joined:
    Feb 4, 2018
    Posts:
    169
    Yeah I could understand that if it didn't list the atlas also. But both Xcode and Unity list both texture memory from the atlas and textures that should be in the atlas.

    Xcode profiler in game shows 1.02 GB total, and texture memory in Xcode is 155.1 MB, however this also includes other textures unity doesn't include, such as some temporary render textures. Unity ends up showing 0.79 GB total memory, with 190.6 MB for texture memory (from the built in unity profiler).

    So it is all kinds of weirdness. This is the part that makes it really hard for us to debug unity memory usage on iOS.
     

    Attached Files:

  13. aleksandrk

    aleksandrk

    Unity Technologies

    Joined:
    Jul 3, 2017
    Posts:
    1,660
    Are those textures marked as readable by chance (the Read/Write Enabled checkbox)?
     
  14. joshuacwilde

    joshuacwilde

    Joined:
    Feb 4, 2018
    Posts:
    169
    The UI textures are not marked Read/Write. The model textures are marked to use Streaming Mipmaps, but also not marked as Read/Write
     
  15. joshuacwilde

    joshuacwilde

    Joined:
    Feb 4, 2018
    Posts:
    169
    Oh wait I take it back, those ui textures actually weren't in an atlas. Now that I put them in an atlas, they aren't shown otherwise. Although the memory discrepancy between textures remains...
     
  16. aleksandrk

    aleksandrk

    Unity Technologies

    Joined:
    Jul 3, 2017
    Posts:
    1,660
    OK, good, one part figured :)

    Regarding the discrepancy, how large is the remaining difference in the stats?
     
  17. joshuacwilde

    joshuacwilde

    Joined:
    Feb 4, 2018
    Posts:
    169
    Memory discrepancy should be about the same as shown in the most recent comment with screenshots.
     
  18. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    934
    The Memory Profiler backend relies on Native UnityEngine.Objects to report their memory size through Profiler.GetRuntimeMemorySizeLong() which needs to be implemented per object and then needs to report the correct size. In some cases this may be doing some slightly wrong calculations of it's size. Whenever such a discrepancy (like here where a platform specific profiler reports a different size than Unity's Memory Manager) comes up, please report them as a bug, with the project (or at least a stripped/empty project with the offending asset with the right import settings getting loaded in in a scene so this can be reproduced).
     
  19. joshuacwilde

    joshuacwilde

    Joined:
    Feb 4, 2018
    Posts:
    169
    Yeah I can definitely do that, but I find it hard to believe this is only from textures. There is a few hundred MB discrepancy in some cases. So it leads me to believe this is a lot bigger problem than some textures being off by 50 KB or so.
     
  20. joshuacwilde

    joshuacwilde

    Joined:
    Feb 4, 2018
    Posts:
    169
    I’ve noticed other things where some textures for post processing aren’t showing up in the unity profiler, but do show up in Xcode. Unity seems to catch most of them, but at least one render texture in particular I couldn’t find a reference to in unity’s profiler, whereas I could in Xcode.

    It’s extremely frustrating as I’m sure you can imagine, because I am forced to look at Xcode instruments to get an idea for where allocations are happening, and then to understand that, I have to have some amount of understanding of unity source code, or at least guess from the c++ method names (which is what I’ve been trying to do as of late).
     
  21. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    934
    Well, I meant you can report the issue in case of these textures. Besides the Total System Used Memory stat that we're getting from the OSs of the particular platforms directly (*), the Memory Manager doesn't currently know about all memory associated with the app, most notably being native memory allocated in plugins. We're mostly aware of these gaps and don't need bugreports on those.

    I know that this is furstrating because it boils down to that gap being a blackbox of sorts which is why we are striving to close it but it's sadly a rather twisted path to get there.

    (*) Case in point: Asking Android on versions from Oreo and up the full app size in memory comes at a ~500ms performance cost per query and we're therefore currently fixing the Total System Used Memory stat on Android to report Residual Shared memory instead to use in Total System Used Memory. Currently it is taking its name to seriously and reports the memory used by all applications on the device, which is obviously not what it was meant to report and being treated as a bug. Once fixed, this and other deviations of what Total System Used Memory means will be properly documented in the Memory Profiler module page of the manual, along with all the other stats. (mentioning this here because this forum post is liable to be out-dated at some point in the future so please check the manual for your specific Unity version. We're doing our best to keep that up-to-date as the go to source of truth.)
     
  22. joshuacwilde

    joshuacwilde

    Joined:
    Feb 4, 2018
    Posts:
    169
    Yeah I see what you are saying, and I appreciate your help on this. It definitely does feel like a blackbox. I am getting pretty good at figuring out some things from xcode instruments, but definitely still ambiguous at some parts.

    On another note, do you know how I can decrease the memory usage shown in the screenshot below? It is basically the original point of this post before I jumped into other memory issues :)

    Basically, because we are using dynamic resolution, it appears Unity creates a heap to hold the whole screen resolution, then dynamically allocates a correctly scaled texture within that heap. However, if we know that dynamic resolution will only ever exceed X%, is there a way we can avoid the heap allocation? Because that wastes about 50% memory in our case.
     

    Attached Files:

  23. joshuacwilde

    joshuacwilde

    Joined:
    Feb 4, 2018
    Posts:
    169
    Here are some more really odd things I am seeing. Look at the size of these buffers, compared to the amount of data they are holding. These buffers are hundreds of times bigger than they should be. By my calculations, that mesh should be about 340 bytes, not 128 KB. Also not sure why there are two of them...

    I'm also seeing this problem with other meshes where they take many many times more memory than they should be.
     

    Attached Files:

    Last edited: Sep 8, 2020
  24. joshuacwilde

    joshuacwilde

    Joined:
    Feb 4, 2018
    Posts:
    169
    Some more weirdness. This texture is 32x32 with ASTC compression and mips. Even uncompressed, my calculation would put it at 5460 bytes. But Xcode is saying it is taking 128 KB!!! That is almost 23 times as much runtime memory as it should be, and even more so because my calculation is based off an uncompressed texture, but the texture is actually ASTC 4x4.

    This is happening with lots of textures as you can see below. The max size of any of those textures in the screenshot should be around 128x128, although most are smaller. And yet most of them take up 128 KB.
     
  25. joshuacwilde

    joshuacwilde

    Joined:
    Feb 4, 2018
    Posts:
    169
    Forgot to post the screenshots for the last comment :
     

    Attached Files:

  26. aleksandrk

    aleksandrk

    Unity Technologies

    Joined:
    Jul 3, 2017
    Posts:
    1,660
    @joshuacwilde My guess: the driver does 128KB allocations on the heap, which can contain several resources at the same time, but Xcode reports them as 128KB each.
     
  27. joshuacwilde

    joshuacwilde

    Joined:
    Feb 4, 2018
    Posts:
    169
    In the screenshot, I only showed the 128 KB ones, but if you scroll down in Xcode, there are indeed smaller sized textures, even though they are the same size in pixels, and same compression type. There are also smaller sizes buffers than 128 KB as well. I just picked the 128 KB items as they are common.

    It also seems odd to me that Xcode would take so much effort to be granular about everything, then just group things together without giving more info.

    It’s possible it is an Xcode bug. I tried on two different Xcode versions on iOS 13 and 14 and got the same results though, so if it is a bug, it would have to be a long standing one.
     
    Last edited: Sep 9, 2020
  28. Alterego-Games

    Alterego-Games

    Joined:
    Jul 13, 2015
    Posts:
    40
    Are you sure this is what is happening? I also have a few of those Heaps, containing a texture 4 times the size of the heap. My dynamic resolution is set to 0.8. I somehow doubt that a dynamic resolution creates a texture 4 times smaller than 1.0.
    Woven heap iphone x.jpg
     
  29. joshuacwilde

    joshuacwilde

    Joined:
    Feb 4, 2018
    Posts:
    169
    Hmm it seemed like it was from my tests, but I’ll double check. Try turning off dynamic res and see what results you get.
     
  30. Alterego-Games

    Alterego-Games

    Joined:
    Jul 13, 2015
    Posts:
    40
    I disabled dynamic resolution on all camera's and now all the heaps are removed, the texture size that was a Heap of 64mb before (with a texture of 16mb in it) is now 21.7mb. So it seems the dynamic resolution is in fact making the texture itself smaller, but reserving a lot more memory than it should need.

    Funny fact:

    The Heapsize on an iPhone XS is 21.8 MB, which is infact the (almost the) exact same format of the texture without dynamic resolution.

    The Heapsize on an iPhone X is 64 MB with dynamic resolution enabled.

    I think there's a bug here.
     
    Last edited: Sep 9, 2020
    joshuacwilde likes this.
  31. joshuacwilde

    joshuacwilde

    Joined:
    Feb 4, 2018
    Posts:
    169
    Oh interesting. Good catch!
     
  32. joshuacwilde

    joshuacwilde

    Joined:
    Feb 4, 2018
    Posts:
    169
  33. joshuacwilde

    joshuacwilde

    Joined:
    Feb 4, 2018
    Posts:
    169
    Last edited: Sep 9, 2020
  34. Alterego-Games

    Alterego-Games

    Joined:
    Jul 13, 2015
    Posts:
    40
    If I check the heapsizes on your first screenshot, I can see that they also are way bigger than they would be on my iPhone XS, my guess is that you could easily save 70mb of vram if that Heap would be the correct size. You are lucky, on my project I have 10 of those heaps that are 3 to 4 times bigger than the original texture would be with dynamic resolution size to 1.0f. It eats 300mb of my vram on an iPhone X and only about 80mb of vram on an iPhone 6s and iPhone XS... I've also reported a bug about it and made a new thread specifically about it here:

    https://forum.unity.com/threads/dynamic-resolution-heap-size-bug-on-some-ios-devices.967417/
     
  35. joshuacwilde

    joshuacwilde

    Joined:
    Feb 4, 2018
    Posts:
    169
    @aleksandrk @MartinTilo I'm sure y'all are really busy, but I think if this is an actual unity bug, and not a problem with xcode, then this would be a pretty big problem, and would benefit all iOS devs. I put together a very simple project for you that illustrates the problem. It is just some meshes with two materials on the screen. This exhibits the same problems as shown in all the screenshots above though (minus the dynamic resolution issue), so it is a good simple repro.

    Here is a link : https://drive.google.com/file/d/1qd14sus-GNqUZROrh-y2WdXQvcGics6p/view?usp=sharing
     
  36. aleksandrk

    aleksandrk

    Unity Technologies

    Joined:
    Jul 3, 2017
    Posts:
    1,660
    @joshuacwilde Can you please report a bug via the bug reporter in this case? This was it will get assigned to the correct team and if it's really a problem with Unity memory tracking, ensure that the fix happens on all relevant versions :)
    Thank you!
     
    MartinTilo likes this.
  37. joshuacwilde

    joshuacwilde

    Joined:
    Feb 4, 2018
    Posts:
    169
  38. aleksandrk

    aleksandrk

    Unity Technologies

    Joined:
    Jul 3, 2017
    Posts:
    1,660
    Please just post the bug number - 1277219 in this case.
    The link is for you only :)
     
  39. joshuacwilde

    joshuacwilde

    Joined:
    Feb 4, 2018
    Posts:
    169
    I created a test project in SpriteKit using the exact same texture. I couldn't get mip maps to work in SpriteKit, but what you are seeing in the attached image is a frame capture of that project using the same texture as the one in Unity.

    Compare/Contrast:

    Unity:
    * 64x64 texture with mip maps - 128 KB in Xcode GPU Frame Capture
    * 64x64 texture without mip maps - 128 KB in Xcode GPU Frame Capture

    SpriteKit:
    64x64 texture without mip maps - 16 KB in Xcode GPU Frame Capture

    The SpriteKit readings here add up as 64*64*4 is 16384 bytes, aka 16 KB (assuming 4 bytes per pixel)
    Unity is allocating 8 times as much memory as required in the case of the texture without mip maps.

    It is becoming more and more obvious to me there are some bugs with GPU memory management in Unity on iOS. And it isn't an OS problem because it works fine with SpriteKit as shown above.

    Can someone please look further into this? This is a major problem and is 100% reproducible with even the most basic of Unity projects. I linked the test project above, and I can link a SpriteKit project as well if desired. I am really surprised a bug this severe is so overlooked. This bug is causing us big problems and is hurting our ability to release on time.
     

    Attached Files:

    chrismarch likes this.
  40. joshuacwilde

    joshuacwilde

    Joined:
    Feb 4, 2018
    Posts:
    169
    @aleksandrk any updates here? is there more info I can provide to speed things along?
     
  41. aleksandrk

    aleksandrk

    Unity Technologies

    Joined:
    Jul 3, 2017
    Posts:
    1,660
    @joshuacwilde I poked relevant people, seems it got stuck a bit in the process. Sorry about that.
     
  42. joshuacwilde

    joshuacwilde

    Joined:
    Feb 4, 2018
    Posts:
    169
    Alright thanks for the update. When would you have more info about this? I'm more than willing to jump on a call or do anything else that might help. This is getting in the way of our release.
     
  43. aleksandrk

    aleksandrk

    Unity Technologies

    Joined:
    Jul 3, 2017
    Posts:
    1,660
    I'll let them know about your situation :)
     
    joshuacwilde likes this.
  44. joshuacwilde

    joshuacwilde

    Joined:
    Feb 4, 2018
    Posts:
    169
  45. joshuacwilde

    joshuacwilde

    Joined:
    Feb 4, 2018
    Posts:
    169
    @aleksandrk an issue got created for one of the problems in this thread, which is great! Thanks for all you did on that end :)

    The more pressing bug still doesn't have a related issue. The issue above is related to the dynamic resolution memory bug, but there is also the bug that I created a sample project for that doesn't have an issue yet. That one is where meshes and textures take up more memory than they should. It is a separate issue from the dynamic resolution memory bug.
     
  46. aleksandrk

    aleksandrk

    Unity Technologies

    Joined:
    Jul 3, 2017
    Posts:
    1,660
  47. joshuacwilde

    joshuacwilde

    Joined:
    Feb 4, 2018
    Posts:
    169
  48. aleksandrk

    aleksandrk

    Unity Technologies

    Joined:
    Jul 3, 2017
    Posts:
    1,660
    Ah, got it :)
    Well, if it's a separate issue, it deserves its own bug report.
     
    joshuacwilde likes this.
  49. VitaSkr

    VitaSkr

    Unity Technologies

    Joined:
    Jul 2, 2019
    Posts:
    14
    Hi @joshuacwilde,

    I see that your problem is not solved, so I've created a new bug report and asked you some questions to ensure a proper interpretation of the problem. Would you find some of your time to answer the questions?

    Here's the bug number: 1280967

    Warmest regards,
    Vita
     
    joshuacwilde likes this.
  50. AcidArrow

    AcidArrow

    Joined:
    May 20, 2010
    Posts:
    7,144
unityunity