Question 4x memory usage in Xcode GPU Frame Capture

I'm looking at the frame capture tool right now. I went to look at these heaps here and noticed they each store one texture, but their allocated size is 4x the contained texture size. For example, a heap will be allocated for 40 MB, but only contain a 10 MB texture. Or it will be allocated to 32 MB, but only contain an 8 MB texture. You can see this in the screenshot attached.

Any ideas why this could be?

We are trying to maximize memory use right now and this looks suspicious.

 

After further testing, this appears to be due to using dynamic resolution. It will allocate the memory required for 100% screen scaling, then the actual texture gets dynamically allocated to be whatever % you set the dynamic scaling at.

Is there a way I can force the memory to be lowered if I know the dynamic resolution will only ever be 75% or whatever?

 

Found a bug in the unity memory profiler. Look at the sizes of the textures in the unity profiler compared to the sizes xcode reports. The sizes unity reports are always smaller, than the ones shown in xcode, thus unity will report a lower memory usage than is correct.

I have one example selected in the screenshot, but if you study it, you will see a few examples of the texture sizes not lining up in those two screenshots.

@aleksandrk

 

Lots of varying sizes. Not sure what you mean. Also those formats should for the most part be PVRTC or ASTC

 

I mean in that screenshot there are varying formats and sizes. They are all POT though.

 

Ah hmm this one is actually a bit odd. It is 300x300, so not POT. I forgot that our UI textures tend to not be POT. Also PVRTC. But this is supposed to be on an atlas, so odd to me that this is using gpu memory separate from the atlas. Also we are using half res in the quality settings for textures.

Here is a screenshot:

 

Yeah good point. I noticed that after when I posted the image, but shouldn't this show the size in memory? or else what is the point of using it to profile runtime memory?

I appreciate your help so far btw, thanks

 

Yeah I could understand that if it didn't list the atlas also. But both Xcode and Unity list both texture memory from the atlas and textures that should be in the atlas.

Xcode profiler in game shows 1.02 GB total, and texture memory in Xcode is 155.1 MB, however this also includes other textures unity doesn't include, such as some temporary render textures. Unity ends up showing 0.79 GB total memory, with 190.6 MB for texture memory (from the built in unity profiler).

So it is all kinds of weirdness. This is the part that makes it really hard for us to debug unity memory usage on iOS.

 

The UI textures are not marked Read/Write. The model textures are marked to use Streaming Mipmaps, but also not marked as Read/Write

 

Oh wait I take it back, those ui textures actually weren't in an atlas. Now that I put them in an atlas, they aren't shown otherwise. Although the memory discrepancy between textures remains...

 

Memory discrepancy should be about the same as shown in the most recent comment with screenshots.

 

Yeah I can definitely do that, but I find it hard to believe this is only from textures. There is a few hundred MB discrepancy in some cases. So it leads me to believe this is a lot bigger problem than some textures being off by 50 KB or so.

 

I’ve noticed other things where some textures for post processing aren’t showing up in the unity profiler, but do show up in Xcode. Unity seems to catch most of them, but at least one render texture in particular I couldn’t find a reference to in unity’s profiler, whereas I could in Xcode.

It’s extremely frustrating as I’m sure you can imagine, because I am forced to look at Xcode instruments to get an idea for where allocations are happening, and then to understand that, I have to have some amount of understanding of unity source code, or at least guess from the c++ method names (which is what I’ve been trying to do as of late).

 

Yeah I see what you are saying, and I appreciate your help on this. It definitely does feel like a blackbox. I am getting pretty good at figuring out some things from xcode instruments, but definitely still ambiguous at some parts.

On another note, do you know how I can decrease the memory usage shown in the screenshot below? It is basically the original point of this post before I jumped into other memory issues

Basically, because we are using dynamic resolution, it appears Unity creates a heap to hold the whole screen resolution, then dynamically allocates a correctly scaled texture within that heap. However, if we know that dynamic resolution will only ever exceed X%, is there a way we can avoid the heap allocation? Because that wastes about 50% memory in our case.

 

Here are some more really odd things I am seeing. Look at the size of these buffers, compared to the amount of data they are holding. These buffers are hundreds of times bigger than they should be. By my calculations, that mesh should be about 340 bytes, not 128 KB. Also not sure why there are two of them...

I'm also seeing this problem with other meshes where they take many many times more memory than they should be.

 

Some more weirdness. This texture is 32x32 with ASTC compression and mips. Even uncompressed, my calculation would put it at 5460 bytes. But Xcode is saying it is taking 128 KB!!! That is almost 23 times as much runtime memory as it should be, and even more so because my calculation is based off an uncompressed texture, but the texture is actually ASTC 4x4.

This is happening with lots of textures as you can see below. The max size of any of those textures in the screenshot should be around 128x128, although most are smaller. And yet most of them take up 128 KB.

 

Forgot to post the screenshots for the last comment :

 

In the screenshot, I only showed the 128 KB ones, but if you scroll down in Xcode, there are indeed smaller sized textures, even though they are the same size in pixels, and same compression type. There are also smaller sizes buffers than 128 KB as well. I just picked the 128 KB items as they are common.

It also seems odd to me that Xcode would take so much effort to be granular about everything, then just group things together without giving more info.

It’s possible it is an Xcode bug. I tried on two different Xcode versions on iOS 13 and 14 and got the same results though, so if it is a bug, it would have to be a long standing one.

 

Are you sure this is what is happening? I also have a few of those Heaps, containing a texture 4 times the size of the heap. My dynamic resolution is set to 0.8. I somehow doubt that a dynamic resolution creates a texture 4 times smaller than 1.0.

 

Hmm it seemed like it was from my tests, but I’ll double check. Try turning off dynamic res and see what results you get.

 

I disabled dynamic resolution on all camera's and now all the heaps are removed, the texture size that was a Heap of 64mb before (with a texture of 16mb in it) is now 21.7mb. So it seems the dynamic resolution is in fact making the texture itself smaller, but reserving a lot more memory than it should need.

Funny fact:

The Heapsize on an iPhone XS is 21.8 MB, which is infact the (almost the) exact same format of the texture without dynamic resolution.

The Heapsize on an iPhone X is 64 MB with dynamic resolution enabled.

I think there's a bug here.

 

Oh interesting. Good catch!

 

Further threads I created from a couple years back that could be related :
https://forum.unity.com/threads/inefficient-mesh-memory-allocation.585643/
https://forum.unity.com/threads/meshes-using-way-more-data-than-expected-calculated.559237/

 

Here is a link to a sample project : https://drive.google.com/file/d/1qd14sus-GNqUZROrh-y2WdXQvcGics6p/view?usp=sharing

It is about as simple as it can get. Just one scene. And it exhibits all the same problems as listed above.

There are some other anomalies in that project if you look at gpu frame capture. The meshes appear to be duplicated in memory.

 

If I check the heapsizes on your first screenshot, I can see that they also are way bigger than they would be on my iPhone XS, my guess is that you could easily save 70mb of vram if that Heap would be the correct size. You are lucky, on my project I have 10 of those heaps that are 3 to 4 times bigger than the original texture would be with dynamic resolution size to 1.0f. It eats 300mb of my vram on an iPhone X and only about 80mb of vram on an iPhone 6s and iPhone XS... I've also reported a bug about it and made a new thread specifically about it here:

https://forum.unity.com/threads/dynamic-resolution-heap-size-bug-on-some-ios-devices.967417/

 

@aleksandrk @MartinTilo I'm sure y'all are really busy, but I think if this is an actual unity bug, and not a problem with xcode, then this would be a pretty big problem, and would benefit all iOS devs. I put together a very simple project for you that illustrates the problem. It is just some meshes with two materials on the screen. This exhibits the same problems as shown in all the screenshots above though (minus the dynamic resolution issue), so it is a good simple repro.

Here is a link : https://drive.google.com/file/d/1qd14sus-GNqUZROrh-y2WdXQvcGics6p/view?usp=sharing

 

bug link here : http://fogbugz.unity3d.com/default.asp?1277219_3jldgu2p2nj4jl9d

 

I created a test project in SpriteKit using the exact same texture. I couldn't get mip maps to work in SpriteKit, but what you are seeing in the attached image is a frame capture of that project using the same texture as the one in Unity.

Compare/Contrast:

Unity:
* 64x64 texture with mip maps - 128 KB in Xcode GPU Frame Capture
* 64x64 texture without mip maps - 128 KB in Xcode GPU Frame Capture

SpriteKit:
64x64 texture without mip maps - 16 KB in Xcode GPU Frame Capture

The SpriteKit readings here add up as 64*64*4 is 16384 bytes, aka 16 KB (assuming 4 bytes per pixel)
Unity is allocating 8 times as much memory as required in the case of the texture without mip maps.

It is becoming more and more obvious to me there are some bugs with GPU memory management in Unity on iOS. And it isn't an OS problem because it works fine with SpriteKit as shown above.

Can someone please look further into this? This is a major problem and is 100% reproducible with even the most basic of Unity projects. I linked the test project above, and I can link a SpriteKit project as well if desired. I am really surprised a bug this severe is so overlooked. This bug is causing us big problems and is hurting our ability to release on time.

 

@aleksandrk any updates here? is there more info I can provide to speed things along?

 

Alright thanks for the update. When would you have more info about this? I'm more than willing to jump on a call or do anything else that might help. This is getting in the way of our release.

 

issue tracker for one of the issues : https://issuetracker.unity3d.com/is...olution-causes-excessive-memory-usage-in-heap

 

@aleksandrk an issue got created for one of the problems in this thread, which is great! Thanks for all you did on that end

The more pressing bug still doesn't have a related issue. The issue above is related to the dynamic resolution memory bug, but there is also the bug that I created a sample project for that doesn't have an issue yet. That one is where meshes and textures take up more memory than they should. It is a separate issue from the dynamic resolution memory bug.

 

Well the bug number here : https://fogbugz.unity3d.com/default.asp?1277219_3jldgu2p2nj4jl9d is the same as here : https://issuetracker.unity3d.com/is...olution-causes-excessive-memory-usage-in-heap

But they are separate issues. Both are GPU memory issues, but they are separate from each other, and they each can be tested in separate projects from each other.

 

How is this different than : https://forum.unity.com/threads/ios-12-large-memory-usage.558550/page-4#post-4271797 ?