Profiler: More info on the "GameObjects in Scene" count?

Hello

For "GameObjects in Scene" via

UnityEngine.GetObjects<GameObject>().Length

For "Total Objects in Scene" via

UnityEngine.GetObjects<GameObject>().Length + UnityEngine.GetObjects<Component>().Length

and "Assets" via

UnityEngine.GetObjects<Object>().Length - (UnityEngine.GetObjects<GameObject>().Length + UnityEngine.GetObjects<Component>().Length)

(... ok, that might be a minor approximation only but, close enough in most instances)

In 2020.3 or newer, you could also use the ProfilerRecorder API and the ProfilerCounter names listed on the Memory Profiler Module documentation page to get these stats. They are also explained there.

Excellent question. If you're profiling the Editor, it could faultily catch prefabs open in an inspector or if it doubles it might add the value from two frames together? somewhat sounds like a bug or a limitation of Profiling the Editor because Memory behaves weirdly in the Editor.

You can swap from Simple to Detailed and take a capture. In the detailed capture, these objects should be listed under Scene Memory.

You might also want to check out the Memory Profiler Package. We just updated it to 0.5 which should make it easier to investigate why these Object are held in memory. As you describe that you are apparently leaking these, just use the package, take two snapshots while their number is growing, open both in the compare mode and group or filter ("Match") the Diff Column by clicking the column header and check out all "New" entries.

 

To that end, you might want to un-hide the "Length" column.

My advice from before still stands on this topic:

You might want to turn on Allocation Call-stacks (might need to be turned on in the profiler as you build & run if you aren't using the latest patch release of whichever major Unity version you're on, as there was a bug that we fixed that wouldn't communicate a later stste change of that setting to the player) or deep profiling to find these GC sources and eliminate them. Others in a build is often down to Debug.Log messages. Those don't get compiled out of release builds either so you might want to look at those. That shouldn't increase the object count though.

Also, just selecting things in the details of the CPU profiler highlights the contribution of the selected sample stack to each of the categories in the chart so you might want to use that to identify the source of that "Other" usage.

 

And I'm happy to see you found and pressumably fixed your Texture2D leak

 

We added more options for Call Stacks to be added to other samples than just GC.Alloc samples. I.e. on 2019.4 that toggle is what activates the feature I linked to.

That is what the "Self Time" column shows. That time is there and accounted for. E.g. your spending 13+6= 19ms on culling. But what you focused on in this screenshot does is too much in the details to show the actual distribution of time spend. The shown section only accounts for ~20% of the entire frame

 

That's a bit too deep in Android specifics for me to comment, maybe something for the mobile subforum?

But: Deep Profiling comes with a small overhead per managed method call, so the more calls, the bigger the overhead.

If incremental GC is on, write barriers are in place to inform it if it needs to redo part of its time-sliced mark and sweep phase. That can add up to 1ms spread out over all managed code executing in a frame. But that would be affecting all frames, though the more managed code executes and the more references and allocations change, the worse the impact.

GC.Alloc samples aren't timed. Thqt would add too much overhead to them. They just have an alibi time attributed to them but could and normally would easily use more time than the profiler shows.

A phone has a tiny power budget to deal with that it needs to dole out to CPU, GPU, Memory, I/O, Network, or other hardware. It usually can't give all of these the same priority so, using more of one system can lead to the OS down clocking the others.

So higher memory access and vibrate could just eat up some of that budget. Transmitting the data for that over ADB or Wifi to the profiler takes out a bit of that too btw so, might be worth measuring the frame time and those samples with UnityEngine.Profiling.Recorders without the profiler and deep profiling attached, just to be sure it's actually a problem

 

If you want to dive deeper on the android specifics and hardware metrics, maybe a Native Profiler like Arm Mobile Studio might get you further on that.

 

Thank you and you're welcome. Happy to help

 

So, it's not scripting code directly, so it's not your C# code (that'd be blue). But it could be related to calls you make into the native backend and those calls executinf something that doesn't fit into the other categories, like Logging Debug messages.

It can also relate to some work which you might be inderectly affecting with the content or settings, and in the Editor it's usually all of the EditorLoop if you're targeting Playmode.

Adding profiler markers is not gonna change the color about if those are placed around the calls to the native backend via the API, then it could help attribute the Other stuff but you're better off starting at the top and just selecting things in the Hierarchy or Timeline view and checking how the selection filters the Chart colors. If what you selected contains time attributed to Other in itself (the self time of brown green markers in Timeline view vor example) or within the self time of its child profiler samples, then the Chart will show you how much of that is categorized as "Other". This way you can narrow it down.

Beyond the above, pleas use Unity.Profiling.ProfiletMarkers instead. They have way less overhead, don't clutter string memory, are Burst compatible and are for all of those reasons mostly ok to leave in your code, even as it goes into Release. (Begin and End calls are entirely compiled out in release, Auto() returns a null object)

Please don't take your information on that sample from the forums, but from the Common Marker documentation page.

The forum threads discussing these are full of stab-in-the-dark, random unity bug or performance regression related issues and confusion mingled together under the most surface level expression of an issue that is almost always destinct for each project hitting it. In fact, these threads are the reason that Documentation page exists in the first place.

Case in point: you are not GPU bound and the marker doesn't mean that either. The "GfxThread" it is waiting on is the Render Thread, which is running on the CPU. In your screenshot you can clearly see that the last frame's Render Thread work overlaps onto this frames WaitForPresentOnGfxThread. So the GPU hasn't even finished getting all the Commands from the Render Thread, but as soon as it has, Gfx.Present actually seems to be done with this frame (or the last one) within 4.5ms.

So you'll likely need to reducing your draw calls batching costs or similar. What exactly I can't tell you based on screenshotsscreenshots alone, as it's rather project specific and requires deeper analysis. The Frames Debugger is going to be your friend and guide through some of that, as is the Profiler Timeline view, with a focus on the Render thread and rendering work happening on the main thread.

 

Not even Player loop or ...DirectorUpdateAnimationB... Or Physics Fixed update? I can see those attribute to Other in your Screenshot of the Timeline view. The chart is basically the cross section of the main thread timeline colors as seen from below, looking up by a 1-dimensional observer.

Alternatively please make sure you're on the latest patch releases for your majorunity version. There where some bugs fixed about this quite a while ago.

You should be seeing a spike in the GC Collect category on the CPU Chart then, but you aren't, so that's not what's happening.

The 3KB allocated in that frame isn't that much either. It's more indicative of something happening that is creating new things, which could affect performance in different ways. And it goes back to power budgets, i.e. touching multiple hardware components simultaneously and thereby reducing how much power each gets.

Performance slowing down after a while with no other change would sound like Heat Throttling, i.e. down clocking because your phone is getting to hot and is only passively cooled. It does seem like your Rendering stats are changing though so I guess it's not a static situation, so it doesn't sound like a super clear case to me.

The time cost for GC.Alloc on the CPU isn't measured to reduce overhead, they only emit a one of sample and not a begin+end like all the others. The samples are then given a minimal duration by the profiler deserializer. Allocating memory can be slow and take longer than that minimal time but 33 allocs at 3KB don't explain a 33ms rendering spike. Just check out what the CPU view tells you about what is going on.

It might be easier to catch the new allocations as they happen by following the GC Alloc Column values on the CPU profiler module. Maybe with Call Stacks data collection turned on (ideally from the start of recording). But yeah, this could work in a pinch.

Yep, lowering Draw Calls helps the CPU a bit because it needs to send less commands but it mostly helps the GPU because there are less context switches. Processing the data into batches to be send to the GPU however comes with a CPU performance cost. It ends up being a balancing act: are you GPU bound? Optimize what work the GPU needs to do by preparing stuff better on the CPU side. Are you Render Thread bound but have time left on the GPU, let it do a couple things that are costly to process in a brute force kinda way, and see if that runs better overall.

But just drawing less in general helps both. Though also Culling and some other factors have an effect on the CPU side of things. Just, please look at what the Render thread samples are telling you is taking the time. And then see if you can find out how to attack that.

 

Thank you, that video Illustrates an issue we do know about with the labels and the cursor potentially getting in the way of things. We have some rough ideas of how to change the UX for that to something more useful and better readable, because yeah, the Other time is also hard to distinguish just by the color of the time label. We're also looking into clearer colors but maybe if you switch to color blind mode using the three dots menu in the window toolbar, it becomes easier.

Also you can rearrange the order of the categories in the legend of the chart, maybe that can help too? Otherwise, timeline view can help because it uses roughly the same color scheme. I say roughly because every color present on the chart maps to Timeline, but Timeline has more colors and all remainders map to Other. Except for grey ones if they have a parent sample, then they take on their parent's category and color.

I've taken a note on that feedback, but for 2022.2 the main (/near only) focus is on the Memory Profiler so it might take us some time to get to that.

I'm not sure. I think there was something about shadows and batching, please use the Frame Debugger or RenderDoc for details. But yes it will probably put a bit of a strain on the Render thread and the GPU. But you seem to still be looking at Hierarchy and you're missing most of the Rendering work from the whole picture that you should be looking at. You can switch Hierarchy view to show the Render thread but some rendering work is spread across the two and you might want to see the timing between them and see the totals for these added up across the threads in Timeline views selection "tool-tip".

On mobile devices you're also always froced into vSync so a slight delay on the CPU to send the work to the GPU could mean that you miss the vBlank and need to wait for 16.66ms on the GPU. That's also an effect that spans across frames so Hierarchy view really gives you too narrow a window into what's actually happening when you're analyzing rendering and GPU work.

Reducing overall workload or being careful about what system you use (downloading stuff from the web, memory and I/O work, GPS, ...) is the most you can do about the power and thermal budget. The more time the CPU and GPU have for idle time, the less heat and the less power they need. A rule of thumb there is to leave about 15-20% of idle time on mobile.

To analyze if that is the problem, you'll need to use a Native profiler like XCode Instruments, Arm Mobile Studio, Android Studio, Perfetto or similar and hope that some indicator for these is represented in these for your device.

 

Also, since you're looking at a phone with a Mali GPU, Arm Mobile Studio and the System Metrics Mali package might help shedding some light on the GPU work for you.

 

I don't know enough about that to really comment. There seems to be some reasonable assumptions there but I can't know for certain. It could also be that the video playback utilized the GPU differently, heated up the phone and/or that after it was played, a new one is requested from the backend to be downloaded, redirecting some of your power budget to the mobile data/wifi chips. There is also a Video Profiler Module that might be able to show more and 2021.1+ has a File Access Profiler Module that might also help if the downloading video file is stored to disk.