Huge amount Garbage Allocation with entities (and Post Processing i think)

Hello guys,

i hope this isnt a newbie-thing since im new... but i searched the web for 2 days straight and have not found a clue.
Garbage Allocation is causing me some trouble with Entities in an extremely Simple Scene. Essentially I spawn in some entities and then i get about 180kb - 350 kb allocated each frame. There is not even an Update Function in the Script.... and performance is good with zero entities, horrible with 5 and only slightly worse with >50k entities. No Jobs or Systems are affecting the entities.
So to test just created a new Project and loaded the spawner-script and viola, 50k entities run smooth with >200 fps.
I was using HDRP for the entire time, but i just recently implemented Post Processing and i think this is where the problem is since this problem arose just recently (disabling the empties with the PP volumes or different Camera settings didn't fix the problem tho...) (occlusion Culling with no effect).
Did some Allocation Callstacks but i dont know how to interpret the results, see Pictures.
Thank you!


Spawner Script:

Spoiler

Code (CSharp):

1. using System.Collections;
2. using System.Collections.Generic;
3. using UnityEngine;
4. using Unity.Transforms;
5. using Unity.Entities;
6. using Unity.Mathematics;
7. using Unity.Rendering;
8.  
9.  
10.  
11.  
12. public class GameHandler : MonoBehaviour
13. {
14.    
15.     [SerializeField] private Mesh levelMesh;
16.     [SerializeField] private Material levelMaterial;
17.  
18.  
19.  
20.     private static EntityManager entityManager;
21.  
22.  
23.     void Start()
24.     {
25.         entityManager = World.Active.EntityManager;
26.  
27.         for (int i = 0; i < 50000; i++)
28.         {
29.             Quaternion rotation = new Quaternion();
30.             rotation.eulerAngles = new Vector3(UnityEngine.Random.Range(0f, 360f), UnityEngine.Random.Range(0f, 360f), UnityEngine.Random.Range(0f, 360f));
31.             SpawnLevelEntity(
32.                 new float3(UnityEngine.Random.Range(-5000f, 5000f), UnityEngine.Random.Range(-20f, 20f), UnityEngine.Random.Range(-5000f, 5000f)),
33.                 rotation,
34.                 new float3(UnityEngine.Random.Range(0f, 15f), UnityEngine.Random.Range(0f, 15f), UnityEngine.Random.Range(0f, 15f))
35.                  );
36.  
37.         }
38.     }
39.  
40.  
41.  
42.  
43.     private void SpawnLevelEntity(float3 position, quaternion rotation, float3 scale)
44.     {
45.         Entity entity = entityManager.CreateEntity(
46.  
47.             typeof(LocalToWorld),
48.             typeof(Translation),
49.             typeof(Rotation),
50.             typeof(NonUniformScale),
51.  
52.             typeof(RenderMesh),
53.             typeof(Level),
54.             typeof(Collider)
55.  
56.             );
57.  
58.  
59.  
60.         SetEntityComponentData(entity, position, rotation, scale, levelMesh, levelMaterial);
61.     }
62.  
63.  
64.  
65.     private void SetEntityComponentData(Entity entity, float3 spawnPosition, quaternion spawnRotation, float3 scale, Mesh mesh, Material material)
66.     {
67.         entityManager.SetSharedComponentData<RenderMesh>(entity,
68.             new RenderMesh
69.             {
70.                 mesh = mesh,
71.                 material = material,
72.             });
73.  
74.         entityManager.SetComponentData<Translation>(entity,
75.            new Translation
76.            {
77.                Value = spawnPosition
78.            });
79.         entityManager.SetComponentData<Rotation>(entity,
80.           new Rotation
81.           {
82.               Value = spawnRotation
83.           });
84.         entityManager.SetComponentData<NonUniformScale>(entity,
85.         new NonUniformScale
86.         {
87.             Value = scale
88.         });
89.  
90.  
91.     }
92. }
93.  
94. public struct Player : IComponentData { }
95. public struct Level : IComponentData { }
96. public struct Collider : IComponentData
97. {
98.     public bool IsClose;
99.     public bool HasCollider;
100. }
101.  
102.  

Pictures:
5 Entities Hierarchy:

Spoiler

50k Entities Hierarchy:

Spoiler

50k Entities TimeLine:

Spoiler

Callstacks1

Spoiler

Callstacks2

Spoiler

 

Hi!
Looking at the callstack the allocation is coming from NativeArray safety system - DisposeSentinel.
The system is used to ensure there are no race conditions when you use NativeArray with JobSystem.
In the Player it is controlled by

ENABLE_UNITY_COLLECTIONS_CHECKS

define which is not defined in Release builds. You can not disable it in the Editor afaik.

 

Thanks for your reply. Not sure how the DisposeSentinel is active without any Jobs being scheduled, since the only Script active was the one i postet. I have other Job-Scripts wich do work with NativeArrays, but i commented them out entirely.
The got around this problem by creating an empty Project and recreating the entire thing...Post Processing is active aswell. Now it runs as it did before with over 150fps, 50k entities and they even get scanned every frame for the closest to the player and shifted back to origin from time to time. DOTS is just awesome.

 

Glad to hear you are not blocked, thanks for following up!

I think you can use Jobs->Burst->Safety Checks menu to dynamically enable/disable collection checks.

 

@alexeyzakharov is it possible to remove the ENABLE_UNITY_COLLECTIONS_CHECKS define from development builds?

It seems to increase CPU costs so much that made our development builds pretty much unplayable on most platforms (PS4, XBOX and Switch; PC builds don't suffer as much).

Changing the options on the Jobs menu and the Burst AOT Settings don't seem to have an effect. We're using Unity 2019.2.20f1 and Burst 1.1.2 (we tried 1.2.2 but it was causing DLL loading issues on PC Standalone builds and we had to roll back).

 

ENABLE_UNITY_COLLECTIONS_CHECKS

is only added automatically for the Editor target. Development players should not have the define set (unless you explicitly add it to script defines in player settings).

Do you have a profiler data showing that

ENABLE_UNITY_COLLECTIONS_CHECKS

is enabled in players? What is slow according to the profiler?

 

Hi Alexey, thanks for the quick response!

I didn't have actual profiler data to back my suspicion - it was based on empyrical data from 2 projects (one using DOTS, the other only using NativeArrays to manipulate texture data), where CPU times of jobs and native collection logic degraded very badly on development builds.

When attempting to generate a test case for you, though, I stumbled upon an interesting finding: the performance difference isn't necessarily related to development builds, but to toggling "Allow Debugging" (BuildOptions.AllowDebugging and EditorUserBuildSettings.allowDebugging), which is something our build scripts were setting automatically on development builds.

This is the profiler information for a PS4 development build with "Allow Debugging" enabled:

(update group names ommitted since they reference an unannounced project)

And this is the same point in the game with "Allow Debugging" disabled:

(update group names ommitted since they reference an unannounced project)

The CPU times on Switch follow roughly the same pattern (seen on both Unity Profiler and Nintendo NX CPU Profiler).


So I think you're right, that define doesn't seem to be related to this. Is there a reliable way to tell whether Burst options might be affecting performance on builds? We're logging Burst options on builds:

var burstCompilerOptions = Unity.Burst.BurstCompiler.Options;

strBuilder.Append( $"Burst Settings:\n" );

strBuilder.Append( $"\tEnabled? {burstCompilerOptions.IsEnabled}\n" );

strBuilder.Append( $"\tOptimizations disabled? {burstCompilerOptions.DisableOptimizations}\n" );

strBuilder.Append( $"\tCompilation enabled? {burstCompilerOptions.EnableBurstCompilation}\n" );

strBuilder.Append( $"\tFast math enabled? {burstCompilerOptions.EnableFastMath}\n" );

strBuilder.Append( $"\tSynchronous compilation? {burstCompilerOptions.EnableBurstCompileSynchronously}\n" );

strBuilder.Append( $"\tSafety checks? {burstCompilerOptions.EnableBurstSafetyChecks}\n" );

But we always get the same log on all platforms, no matter what changes we make on the Jobs Editor menu or the Burst AOT Settings on the Player Settings:

Burst Settings:

   Enabled? True

   Optimizations disabled? False

   Compilation enabled? True

   Fast math enabled? True

   Synchronous compilation? False

   Safety checks? True

Thanks a lot for your help!

 

This enables debugging support for the managed code which basically means the generated code is not really optimized. E.g. in order to support breakpoints there should be some space (nops) between instructions, methods can't be inlined, etc. This is relevant to both Mono and il2cpp afaik. If you see the difference in those methods they are probably not Burst'ed. Let me ask the Burst team about the platform support.

 

Safety check surely affect the performance a bit.
Overwise the seetings lok good.

 

Got an answer about the debug options that it does affect Burst compilation since the assemblies will have been compiled in debug, Burst will be compiling the debug assembly version of the code. Ultimately Burst should still generate the same or very close final output, but any methods guarded by being debug only will now be included etc.