Bug EntityManager.CopyEntitesFrom(EntityManager,NativeArray<Entity>) MemoryLeak iff >10 Entities

Code (CSharp):

1. EntityManager em           = World.DefaultGameObjectInjectionWorld.EntityManager;
2.  
3. EntityArchetype archetype = em.CreateArchetype(ComponentType.ReadOnly<LocalToWorld>(),
4.                                                ComponentType.ReadOnly<RenderMesh>(),
5.                                                ComponentType.ReadOnly<RenderBounds>(),
6.                                                ComponentType.ReadOnly<PhysicsCollider>());
7. if(!currentEntities.IsCreated)
8.   currentEntities = em.CreateEntity(archetype, 1000, Allocator.Persistent);
9. var serializationWorld = new World("Serialization World");
10. var emSerialize        = serializationWorld.EntityManager;
11. emSerialize.CopyEntitiesFrom(em, currentEntities);

Throws referring to CopyEntitesFrom (Line 11):

Code (CSharp):

1. [Error] A Native Collection has not been disposed, resulting in a memory leak. Allocated from:
2. Unity.Collections.NativeArray`1:.ctor(Int32, Allocator, NativeArrayOptions)
3. Unity.Entities.EntityManager:CreateEntity(EntityArchetype, Int32, Allocator) (at Library\PackageCache\com.unity.entities@0.10.0-preview.6\Unity.Entities\EntityManagerCreateDestroyEntities.cs:104)
4. Frameworks.OrderedPointCloud.Cache.PatchHierarchyMpkCache:Load(OpcLodHierarchy, Int32) (at Assets\Scripts\Frameworks\OrderedPointCloud\Cache\PatchHierarchyMpkCache.cs:453)
5. Frameworks.OrderedPointCloud.Scenes.OpcLodHierarchy:Load(Int32) (at Assets\Scripts\Frameworks\OrderedPointCloud\Scenes\OpcLodHierarchy.cs:40)
6. Application.Streaming.<OnSceneChange>d__7:MoveNext() (at Assets\Scripts\Application\Streaming\StreamingSystem.cs:57)
7. Cysharp.Threading.Tasks.CompilerServices.AsyncUniTaskMethodBuilder:Start(<OnSceneChange>d__7&) (at Assets\Plugins\UniTask\Runtime\CompilerServices\AsyncUniTaskMethodBuilder.cs:111)
8. Application.Streaming.StreamingSystem:OnSceneChange(IScene)
9. Application.Lod.<OnSceneChange>d__32:MoveNext() (at Assets\Scripts\Application\Lod\LodController.cs:132)
10. Cysharp.Threading.Tasks.CompilerServices.AsyncUniTaskMethodBuilder:Start(<OnSceneChange>d__32&) (at Assets\Plugins\UniTask\Runtime\CompilerServices\AsyncUniTaskMethodBuilder.cs:111)
11. Application.Lod.LodController:OnSceneChange(IScene)
12. Application.Layers.<Notify>d__9:MoveNext() (at Assets\Scripts\Application\Layers\LayerController.cs:70)
13. Cysharp.Threading.Tasks.CompilerServices.AsyncUniTaskMethodBuilder:Start(<Notify>d__9&) (at Assets\Plugins\UniTask\Runtime\CompilerServices\AsyncUniTaskMethodBuilder.cs:111)
14. Application.Layers.LayerController:Notify(IScene)
15. Application.Layers.<OnSceneChange>d__8:MoveNext() (at Assets\Scripts\Application\Layers\LayerController.cs:65)
16. Cysharp.Threading.Tasks.CompilerServices.AsyncUniTaskMethodBuilder:Start(<OnSceneChange>d__8&) (at Assets\Plugins\UniTask\Runtime\CompilerServices\AsyncUniTaskMethodBuilder.cs:111)
17. Application.Layers.LayerController:OnSceneChange(IScene)
18. Application.Scenes.<SetScene>d__14:MoveNext() (at Assets\Scripts\Application\Scenes\SceneController.cs:67)
19. Cysharp.Threading.Tasks.CompilerServices.AsyncUniTaskMethodBuilder:Start(<SetScene>d__14&) (at Assets\Plugins\UniTask\Runtime\CompilerServices\AsyncUniTaskMethodBuilder.cs:111)
20. Application.Scenes.SceneController:SetScene(IScene)
21. Application.Scenes.<LoadOpc>d__12:MoveNext() (at Assets\Scripts\Application\Scenes\SceneController.cs:51)
22. Cysharp.Threading.Tasks.CompilerServices.AsyncUniTaskMethodBuilder`1:Start(<LoadOpc>d__12&) (at Assets\Plugins\UniTask\Runtime\CompilerServices\AsyncUniTaskMethodBuilder.cs:242)
23. Application.Scenes.SceneController:LoadOpc(String)
24. Application.Scenes.<LoadHomePlate>d__11:MoveNext() (at Assets\Scripts\Application\Scenes\SceneController.cs:43)
25. Cysharp.Threading.Tasks.CompilerServices.AsyncUniTaskMethodBuilder:Start(<LoadHomePlate>d__11&) (at Assets\Plugins\UniTask\Runtime\CompilerServices\AsyncUniTaskMethodBuilder.cs:111)
26. Application.Scenes.SceneController:LoadHomePlate()
27. Presentation.Menus.LoadScene.<OnButtonClicked>d__0:MoveNext() (at Assets\Scripts\Presentation\Menus\LoadScene\LoadHomeplateMenuItem.cs:10)
28. System.Runtime.CompilerServices.AsyncVoidMethodBuilder:Start(<OnButtonClicked>d__0&)
29. Presentation.Menus.LoadScene.LoadHomeplateMenuItem:OnButtonClicked()
30. Infrastructure.NoxRx.UI.<>c__DisplayClass6_0:<Call>b__0(Unit) (at Assets\Scripts\Common\NoxRx\UI\PresenterBindable.cs:23)
31. UniRx.Subscribe`1:OnNext(Unit) (at Assets\Plugins\UniRx\Scripts\Observer.cs:165)
32. UniRx.Operators.RefCount:OnNext(Unit) (at Assets\Plugins\UniRx\Scripts\Operators\RefCount.cs:61)
33. UniRx.Subject`1:OnNext(Unit) (at Assets\Plugins\UniRx\Scripts\Subjects\Subject.cs:62)
34. UniRx.Operators.AsObservable:OnNext(Unit) (at Assets\Plugins\UniRx\Scripts\Operators\AsObservable.cs:29)
35. UniRx.Operators.FromEvent:OnNext() (at Assets\Plugins\UniRx\Scripts\Operators\FromEvent.cs:124)
36. Button.Press() at C:\Program Files\Unity\Hub\Editor\2020.1.0b10\Editor\Data\Resources\PackageManager\BuiltInPackages\com.unity.ugui\Runtime\UI\Core\Button.cs:68
37. 67:       UISystemProfilerApi.AddMarker("Button.onClick", this);
38. -->68:       m_OnClick.Invoke();
39. 69:
40.  

As currentEntities are on purpose beeing allocated with AllocatorPersisent I am wondering which collection it is referring to.

Stepping through shows that it looks like it happens during disposing the srcManagerInstances? That does not make much sense to me:

Edit1: The test with <10 Entities worked, but now with 42 it fails. It must be a bug in the CreateBatchList.

Edit2:Stepping even further into the code shows that I am unable to step over the dispose here

Code (CSharp):

1.     [WriteAccessRequired]
2.     public unsafe void Dispose()
3.     {
4.       if (!UnsafeUtility.IsValidAllocator(this.m_AllocatorLabel))
5.         throw new InvalidOperationException("The NativeArray can not be Disposed because it was not allocated with a valid allocator.");
6.       DisposeSentinel.Dispose(ref this.m_Safety, ref this.m_DisposeSentinel);
7.       UnsafeUtility.Free(this.m_Buffer, this.m_AllocatorLabel);
8.       this.m_Buffer = (void*) null;
9.       this.m_Length = 0;
10.     }

Interestingly it is not the InvalidOperation shown here but the DisposeSentinelDispose or UnsafeUtility.Free cause the error.

Edit forgot to add the manifest.json:

Code (CSharp):

1. {
2.   "dependencies": {
3.     "com.havok.physics": "0.2.2-preview",
4.     "com.unity.build-report-inspector": "0.2.2-preview",
5.     "com.unity.burst": "1.3.1",
6.     "com.unity.collections": "0.9.0-preview.5",
7.     "com.unity.dots.editor": "0.6.0-preview",
8.     "com.unity.editorcoroutines": "1.0.0",
9.     "com.unity.entities": "0.11.0-preview.7",
10.     "com.unity.ext.nunit": "1.0.0",
11.     "com.unity.ide.rider": "1.2.1",
12.     "com.unity.ide.visualstudio": "2.0.2",
13.     "com.unity.inputsystem": "1.0.0",
14.     "com.unity.jobs": "0.2.10-preview.11",
15.     "com.unity.mathematics": "1.1.0",
16.     "com.unity.memoryprofiler": "0.2.3-preview.2",
17.     "com.unity.performance.profile-analyzer": "1.0.0",
18.     "com.unity.physics": "0.3.2-preview",
19.     "com.unity.platforms": "0.5.0-preview.1",
20.     "com.unity.quicksearch": "1.5.4",
21.     "com.unity.render-pipelines.universal": "9.0.0-preview.14",
22.     "com.unity.rendering.hybrid": "0.5.1-preview.18",
23.     "com.unity.scriptablebuildpipeline": "1.7.3",
24.     "com.unity.searcher": "4.3.1",
25.     "com.unity.test-framework": "1.1.14",
26.     "com.unity.test-framework.performance": "2.2.0-preview",
27.     "com.unity.testtools.codecoverage": "0.3.0-preview",
28.     "com.unity.textmeshpro": "3.0.0-preview.14",
29.     "nuget.mono-cecil": "0.1.6-preview",
30.     "com.unity.modules.animation": "1.0.0",
31.     "com.unity.modules.assetbundle": "1.0.0",
32.     "com.unity.modules.audio": "1.0.0",
33.     "com.unity.modules.imageconversion": "1.0.0",
34.     "com.unity.modules.imgui": "1.0.0",
35.     "com.unity.modules.jsonserialize": "1.0.0",
36.     "com.unity.modules.particlesystem": "1.0.0",
37.     "com.unity.modules.physics": "1.0.0",
38.     "com.unity.modules.physics2d": "1.0.0",
39.     "com.unity.modules.terrain": "1.0.0",
40.     "com.unity.modules.terrainphysics": "1.0.0",
41.     "com.unity.modules.ui": "1.0.0",
42.     "com.unity.modules.uielements": "1.0.0",
43.     "com.unity.modules.umbra": "1.0.0",
44.     "com.unity.modules.unitywebrequest": "1.0.0",
45.     "com.unity.modules.unitywebrequestassetbundle": "1.0.0",
46.     "com.unity.modules.unitywebrequestaudio": "1.0.0",
47.     "com.unity.modules.unitywebrequestwww": "1.0.0",
48.     "com.unity.modules.vr": "1.0.0",
49.     "com.unity.modules.xr": "1.0.0"
50.   },
51.   "testables": [
52.     "com.unity.test-framework.performance"
53.   ]
54. }
55.  

 

Confirmed. It works when I just copy the first 10 entities to an array and use that one instead:

Code (CSharp):

1. NativeArray<Entity> tmp = new NativeArray<Entity>(10, Allocator.Temp);
2. for (var i = 0; i < tmp.Length; i++)
3. {
4.     tmp[i] = currentEntities[i];
5. }
6. emSerialize.CopyEntitiesFrom(em, tmp); //Works
7. tmp.Dispose();

but

Code (CSharp):

1. NativeArray<Entity> tmp = new NativeArray<Entity>(currentEntities.Length, Allocator.Temp);
2. for (var i = 0; i < tmp.Length; i++)
3. {
4.     tmp[i] = currentEntities[i];
5. }
6. emSerialize.CopyEntitiesFrom(em, tmp); // Fails
7. tmp.Dispose();

Edit: Also tested all other entities form currentEnties. I can CopyEntitesFrom every single one of them, but only 10 at a time.

 

I think you came across the same bug I posted here: https://fogbugz.unity3d.com/default.asp?1238456_ovkvmpq3hvrf0n6j

I did not have the time to create minimum repro project, so it simply was closed.

My explanation from that bug report:

Edited: Sorry for the confusion, this is probably a different problem, I did not read the last post carefully enough. My problem is a probably unrelated issue with CopyFromEntities. I was too glad, that someone else came across my problem.

 

@s_schoener thank you so much for telling me that. Just knowing it is beeing worked on, helps to de-stress me a lot

 

@cort_of_unity, I'm sure this isn't the first time somebody's compared your portrait to Crocodile Dundee, but my point is this: every time you post, I hear, in Dundee's voice, "that's not a fix; THIS is a fix"! <3