Bug AtomicSafetyHandle:Release crash when using EntityCommandBufferSystems

The project I'm working on reliably crashes the editor (full crash to desktop with an accompanying crash dump) when it's run anywhere from 5 seconds to 15/20 minutes ever since we upgrade com.unity.entities to 1.0.10 (skipped 1.0.8 entirely, though reproduced the crash with 1.0.8 as well). Happens both in Unity 2022.2.21f1 and 2022.3.0f1.

The stack trace is always the same, originating from EntityCommandBufferSystem.cs and ending in AtomicSafetyHandle:Release:

Spoiler

Code (csharp):

1. 2023-05-31T06:19:09.817Z|0x1dd8|Received signal SIGSEGV
2. 2023-05-31T06:19:10.181Z|0x1dd8|Obtained 31 stack frames
3. 0x00007ff6122291b0 (Unity) AtomicSafetyHandle::Release
4. 0x00007ff6117d2847 (Unity) AtomicSafetyHandle_CUSTOM_Release_Injected
5. 0x00000257bc9883fa (Mono JIT Code) (wrapper managed-to-native) Unity.Collections.LowLevel.Unsafe.AtomicSafetyHandle:Release_Injected (Unity.Collections.LowLevel.Unsafe.AtomicSafetyHandle&)
6. 0x00000257bc988323 (Mono JIT Code) Unity.Collections.LowLevel.Unsafe.AtomicSafetyHandle:Release (Unity.Collections.LowLevel.Unsafe.AtomicSafetyHandle)
7. 0x000002577b7a5e5b (Mono JIT Code) Unity.Collections.AllocatorManager/AllocatorHandle:InvalidateDependents () (at ./Library/PackageCache/com.unity.collections@2.1.4/Unity.Collections/AllocatorManager.cs:421)
8. 0x000002577b7a5d23 (Mono JIT Code) Unity.Collections.AllocatorManager/AllocatorHandle:Rewind () (at ./Library/PackageCache/com.unity.collections@2.1.4/Unity.Collections/AllocatorManager.cs:235)
9. 0x000002577b7aa6d3 (Mono JIT Code) Unity.Collections.RewindableAllocator:Rewind () (at ./Library/PackageCache/com.unity.collections@2.1.4/Unity.Collections/RewindableAllocator.cs:312)
10. 0x00000257bdc30d83 (Mono JIT Code) Unity.Entities.EntityCommandBufferSystem:FlushPendingBuffers (bool) (at ./Library/PackageCache/com.unity.entities@1.0.10/Unity.Entities/EntityCommandBufferSystem.cs:292)
11. 0x00000257bdc2f2d3 (Mono JIT Code) Unity.Entities.EntityCommandBufferSystem:OnUpdate () (at ./Library/PackageCache/com.unity.entities@1.0.10/Unity.Entities/EntityCommandBufferSystem.cs:183)
12. 0x00000257bdc2dc93 (Mono JIT Code) Unity.Entities.SystemBase:Update () (at ./Library/PackageCache/com.unity.entities@1.0.10/Unity.Entities/SystemBase.cs:420)
13. 0x00000257bdc2f18c (Mono JIT Code) Unity.Entities.ComponentSystemGroup:UpdateAllSystems () (at ./Library/PackageCache/com.unity.entities@1.0.10/Unity.Entities/ComponentSystemGroup.cs:737)
14. 0x00000257bdc2ebc3 (Mono JIT Code) Unity.Entities.ComponentSystemGroup:OnUpdate () (at ./Library/PackageCache/com.unity.entities@1.0.10/Unity.Entities/ComponentSystemGroup.cs:693)
15. 0x00000257bdc2dc93 (Mono JIT Code) Unity.Entities.SystemBase:Update () (at ./Library/PackageCache/com.unity.entities@1.0.10/Unity.Entities/SystemBase.cs:420)
16. 0x00000257bdc2d9ad (Mono JIT Code) Unity.Entities.ScriptBehaviourUpdateOrder/DummyDelegateWrapper:TriggerUpdate () (at ./Library/PackageCache/com.unity.entities@1.0.10/Unity.Entities/ScriptBehaviourUpdateOrder.cs:528)
17. 0x0000025548b69888 (Mono JIT Code) (wrapper runtime-invoke) object:runtime_invoke_void__this__ (object,intptr,intptr,intptr)
18. 0x00007fff46a2e084 (mono-2.0-bdwgc) mono_jit_runtime_invoke (at C:/build/output/Unity-Technologies/mono/mono/mini/mini-runtime.c:3445)
19. 0x00007fff4696eb84 (mono-2.0-bdwgc) do_runtime_invoke (at C:/build/output/Unity-Technologies/mono/mono/metadata/object.c:3066)
20. 0x00007fff4696ed1c (mono-2.0-bdwgc) mono_runtime_invoke (at C:/build/output/Unity-Technologies/mono/mono/metadata/object.c:3113)
21. 0x00007ff61260ee94 (Unity) scripting_method_invoke
22. 0x00007ff6125ef714 (Unity) ScriptingInvocation::Invoke
23. 0x00007ff612284115 (Unity) ExecutePlayerLoop
24. 0x00007ff612284136 (Unity) ExecutePlayerLoop
25. 0x00007ff61228a985 (Unity) PlayerLoop
26. 0x00007ff61323e48f (Unity) PlayerLoopController::InternalUpdateScene
27. 0x00007ff61324b1dd (Unity) PlayerLoopController::UpdateSceneIfNeededFromMainLoop
28. 0x00007ff6132494e1 (Unity) Application::TickTimer
29. 0x00007ff6136d54ca (Unity) MainMessageLoop
30. 0x00007ff6136da270 (Unity) WinMain
31. 0x00007ff614abe69e (Unity) __scrt_common_main_seh
32. 0x00007fffcf3b26ad (KERNEL32) BaseThreadInitThunk
33. 0x00007fffd090a9f8 (ntdll) RtlUserThreadStart

If I remove all usages of any EntityCommandBufferSystem from the project it no longer crashes. Of course doing a quick removal means that some jobs that used to be run in parallel now get run on the main thread so this makes for a big change to behaviour.

I've reported this to Unity as well as IN-42519, I'm posting here in case anyone else has run into the same issue and might have some insight.

I'm not doing anything odd with the ECB systems, each one is accessed in an ISystem using the related Singleton and they're used to add/remove components.

 

I seem to have the same issue, but on 1.0.14. Have you found any solution?
This is how the stack trace looks for me.

Spoiler: Stack trace

Code (CSharp):

1.  
2.  
3. AtomicSafetyNode has either been corrupted or is being accessed on a job which is not allowed.
4. UnityEngine.StackTraceUtility:ExtractStackTrace ()
5. Unity.Collections.LowLevel.Unsafe.AtomicSafetyHandle:Release (Unity.Collections.LowLevel.Unsafe.AtomicSafetyHandle)
6. Unity.Collections.AllocatorManager/AllocatorHandle:InvalidateDependents () (at ./Library/PackageCache/com.unity.collections@2.1.4/Unity.Collections/AllocatorManager.cs:427)
7. Unity.Collections.AllocatorManager/AllocatorHandle:Rewind () (at ./Library/PackageCache/com.unity.collections@2.1.4/Unity.Collections/AllocatorManager.cs:234)
8. Unity.Collections.RewindableAllocator:Rewind () (at ./Library/PackageCache/com.unity.collections@2.1.4/Unity.Collections/RewindableAllocator.cs:312)
9. Unity.Entities.EntityCommandBufferSystem:FlushPendingBuffers (bool) (at Packages/com.unity.entities@1.0.14/Unity.Entities/EntityCommandBufferSystem.cs:300)
10. Unity.Entities.EntityCommandBufferSystem:OnUpdate () (at Packages/com.unity.entities@1.0.14/Unity.Entities/EntityCommandBufferSystem.cs:182)
11. Unity.Entities.SystemBase:Update () (at Packages/com.unity.entities@1.0.14/Unity.Entities/SystemBase.cs:418)
12. Unity.Entities.ComponentSystemGroup:UpdateAllSystems () (at Packages/com.unity.entities@1.0.14/Unity.Entities/ComponentSystemGroup.cs:739)
13. Unity.Entities.ComponentSystemGroup:OnUpdate () (at Packages/com.unity.entities@1.0.14/Unity.Entities/ComponentSystemGroup.cs:697)
14. Unity.Entities.SystemBase:Update () (at Packages/com.unity.entities@1.0.14/Unity.Entities/SystemBase.cs:418)
15. Unity.Entities.ScriptBehaviourUpdateOrder/DummyDelegateWrapper:TriggerUpdate () (at Packages/com.unity.entities@1.0.14/Unity.Entities/ScriptBehaviourUpdateOrder.cs:526)
16.  
17. [./Library/PackageCache/com.unity.collections@2.1.4/Unity.Collections/AllocatorManager.cs line 427]
18.  
19. The Handle has already been released.
20. UnityEngine.StackTraceUtility:ExtractStackTrace ()
21. Unity.Collections.LowLevel.Unsafe.AtomicSafetyHandle:Release (Unity.Collections.LowLevel.Unsafe.AtomicSafetyHandle)
22. Unity.Collections.AllocatorManager/AllocatorHandle:InvalidateDependents () (at ./Library/PackageCache/com.unity.collections@2.1.4/Unity.Collections/AllocatorManager.cs:427)
23. Unity.Collections.AllocatorManager/AllocatorHandle:Rewind () (at ./Library/PackageCache/com.unity.collections@2.1.4/Unity.Collections/AllocatorManager.cs:234)
24. Unity.Collections.RewindableAllocator:Rewind () (at ./Library/PackageCache/com.unity.collections@2.1.4/Unity.Collections/RewindableAllocator.cs:312)
25. Unity.Entities.EntityCommandBufferSystem:FlushPendingBuffers (bool) (at Packages/com.unity.entities@1.0.14/Unity.Entities/EntityCommandBufferSystem.cs:300)
26. Unity.Entities.EntityCommandBufferSystem:OnUpdate () (at Packages/com.unity.entities@1.0.14/Unity.Entities/EntityCommandBufferSystem.cs:182)
27. Unity.Entities.SystemBase:Update () (at Packages/com.unity.entities@1.0.14/Unity.Entities/SystemBase.cs:418)
28. Unity.Entities.ComponentSystemGroup:UpdateAllSystems () (at Packages/com.unity.entities@1.0.14/Unity.Entities/ComponentSystemGroup.cs:739)
29. Unity.Entities.ComponentSystemGroup:OnUpdate () (at Packages/com.unity.entities@1.0.14/Unity.Entities/ComponentSystemGroup.cs:697)
30. Unity.Entities.SystemBase:Update () (at Packages/com.unity.entities@1.0.14/Unity.Entities/SystemBase.cs:418)
31. Unity.Entities.ScriptBehaviourUpdateOrder/DummyDelegateWrapper:TriggerUpdate () (at Packages/com.unity.entities@1.0.14/Unity.Entities/ScriptBehaviourUpdateOrder.cs:526)
32.  
33. [./Library/PackageCache/com.unity.collections@2.1.4/Unity.Collections/AllocatorManager.cs line 427]
34.  
35.       at <unknown> <0xffffffff>
36.       at Unity.Collections.LowLevel.Unsafe.AtomicSafetyHandle:Release_Injected <0x00089>
37.       at Unity.Collections.LowLevel.Unsafe.AtomicSafetyHandle:Release <0x00022>
38.       at AllocatorHandle:InvalidateDependents <0x000ea>
39.       at AllocatorHandle:Rewind <0x00022>
40.       at Unity.Collections.RewindableAllocator:Rewind <0x00042>
41.       at Unity.Entities.EntityCommandBufferSystem:FlushPendingBuffers <0x00e82>
42.       at Unity.Entities.EntityCommandBufferSystem:OnUpdate <0x00022>
43.       at Unity.Entities.SystemBase:Update <0x002c2>
44.       at Unity.Entities.ComponentSystemGroup:UpdateAllSystems <0x0054b>
45.       at Unity.Entities.ComponentSystemGroup:OnUpdate <0x00042>
46.       at Unity.Entities.SystemBase:Update <0x002c2>
47.       at DummyDelegateWrapper:TriggerUpdate <0x0002c>
48.       at System.Object:runtime_invoke_void__this__ <0x00087>
49.