Bug Valid generic burst job not working in bursted ISystem

With Burst we have some limitations of how generic jobs can be used. We used to them for years already with non bursted schedule context, in pre 1.0 era, that's ok. As stated in docs having generic job called explicitly or have wrapped struct instantiated explicitly and calling job schedule implicitly is completely valid approach, blah blah blah, nothing new. And that's works without issues in non bursted Schedule context (btw there is typo in docs I believe,

aren't

should be

are

?):
https://docs.unity3d.com/Packages/com.unity.burst@1.8/manual/compilation-generic-jobs.html

Spoiler: Docs and typo

There is 3 simple examples of different working burst compiled jobs in both Editor and Runtime, BUT notice that OnUpdate itself is not burst compiled yet.

Code (CSharp):

1. public interface IIncrement
2.     {
3.         public void  Increment(float v);
4.         public float Get();
5.     }
6.  
7.     public struct SimpleData : IIncrement
8.     {
9.         public float Value;
10.         public void  Increment(float v) => Value += v;
11.         public float Get() => Value;
12.     }
13.  
14.     public partial struct SimpleJobSystem : ISystem
15.     {
16.         public void OnUpdate(ref SystemState state)
17.         {
18.             var someContainer = new NativeReference<SimpleData>(state.WorldUpdateAllocator)
19.             {
20.                 Value = new SimpleData()
21.                 {
22.                     Value = 1
23.                 }
24.             };
25.  
26.             // 1) Explicit job usage
27.             state.Dependency = new SimpleGenericJob<SimpleData>()
28.             {
29.                 someContainer = someContainer
30.             }.Schedule(state.Dependency);
31.  
32.             // 2) Explicit job usage through static
33.             state.Dependency = SimpleGenericJob<SimpleData>.Schedule(state.Dependency, someContainer);
34.  
35.             // 3) Implicit job usage through explicit wrapper struct instance
36.             state.Dependency = new SimpleJobWrapper<SimpleData>().Schedule(state.Dependency, someContainer);
37.         }
38.     }
39.  
40.     [BurstCompile]
41.     public struct SimpleGenericJob<T> : IJob where T : unmanaged, IIncrement
42.     {
43.         public NativeReference<T> someContainer;
44.  
45.         public void Execute()
46.         {
47.             var value = someContainer.Value;
48.             // Just simulate workload to easily see job on timeline
49.             for (var i = 0; i < 10000; i++)
50.                 value.Increment(math.cos(math.sin(math.sqrt(i /10000f))));
51.             someContainer.Value = value;
52.         }
53.  
54.         public static JobHandle Schedule(JobHandle dependency, NativeReference<T> someContainer)
55.         {
56.             return new SimpleGenericJob<T>()
57.             {
58.                 someContainer = someContainer
59.             }.Schedule(dependency);
60.         }
61.     }
62.  
63.     public struct SimpleJobWrapper<T> where T : unmanaged, IIncrement
64.     {
65.         [BurstCompile]
66.         public struct SimpleNestedJob : IJob
67.         {
68.             public NativeReference<T> someContainer;
69.  
70.             public void Execute()
71.             {
72.                 var value = someContainer.Value;
73.                 // Just simulate workload to easily see job on timeline
74.                 for (var i = 0; i < 10000; i++)
75.                     value.Increment(math.cos(math.sin(math.sqrt(i /10000f))));
76.                 someContainer.Value = value;
77.             }
78.         }
79.  
80.         public JobHandle Schedule(JobHandle dependency, NativeReference<T> someContainer)
81.         {
82.             return new SimpleNestedJob()
83.             {
84.                 someContainer = someContainer
85.             }.Schedule(dependency);
86.         }
87.     }

Here you can see that this code is valid for Burst in standalone build.



And as soon as we make OnUpdate bursted all this falls apart, and nested generic jobs will stop working (third Schedule example) with error below in editor and crash in build, if we comment third sample, everything will work again, explicitly called generic jobs will work just fine. Also notice how useless and irrelevant stack trace is, it points to EntityCommandBuffer (!)




The main point is why it's confusing with nested struct - Unity NativeContainers (which are generic structs) have Dispose(JobHandle) version which is scheduling job inside, and their code works without issues inside bursted OnUpdate, but our custom NativeContainers with exactly the same Dispose(JobHandle) and similar jobs inside - not.

 

You might understand this problem better if you dump out the TypeSpec tables.

Sometimes, the Roslyn compiler will split your concrete generic job type into two separate typespec entries, and you have to use inference to recognize that a concrete instance is valid. https://forum.unity.com/threads/wil...neric-jobs-going-forward.974187/#post-6409620

When and how this happens isn't totally clear to me, and maybe that's what you are struggling with too. But what I can say is that BurstReflection.cs is able to resolve this by using ResolveType to get the System.Type and that's how the jobs get Burst-compiled. However, the ILPP that does the EarlyJobInit (which has nothing to do with Burst other than that it can't be called from within a Burst context) instead uses TypeReference of CECIL and that struggles to resolve the types properly. This may be a bug in CECIL. This may be a missing step in the ILPP. I'm not really sure yet.

I had a similar issue where RegisterGenericJobType wasn't an option, because the generic arguments could be private. Using reflection to invoke the EarlyJobInit was sufficient. Link which may die in the future: https://github.com/Dreaming381/Lati...ics/Internal/Queries/Layers/BurstEarlyInit.cs

 

Yeah spoke with @elliotc-unity about this in Discord. I've checked typespec and methodspec from assembly metadata yeah that's very confusing, when and how It splits, and just doesn't work like how you expect. Most confusing as mentioned in my initial message

Especially when I clearly see records of my

NativeQuadTree<Entity>.Dispose(JobHandle)

and Unity one for their

NativeList<QuadTreeElement<Entity>>.Dispose(JobHandle)

for example in typespec of assembly, and none of exact jobs calls neither mine or

NativeList<T>.NativeListDisposeJob

(maybe I'm looking at something wrong, as I rarely use disassemblers these days)

 

I'm not sure how you are reading the assembly metadata, but some tools will resolve the types like Burst and others won't. I used monodis to identify the issue in my investigation, as the output makes it very obvious when type information is missing from an entry and has to be resolved by inference. But this was a while ago. I'm not even sure if that tool is even maintained.

I do think it is an issue Unity needs to resolve. Burst has very well-defined rules, and ILPP doesn't.

 

I used JetBrains dotPeek and checked managed assemblies (like Assembly-CSharp), isn't Unity scan them for genetic instances for future burst compilation?

 

Unity has two separate scanners. One is for scheduling jobs from a Burst context (ISystem) using ILPP while the other is for actually compiling jobs. The one that compiles jobs works perfectly. The one for scheduling jobs (ILPP and CECIL for EarlyJobInit), not so much. That latter lives in the Jobs subdirectory of the Collections package. You can compare the two scanners to see the difference.

 

Ah, I’m just dumb. Collections works the same. I’m just completely missed that NativeListDisposeJob lies outside of NativeList struct that’s puts everything on it’s place as it’s not generic by itself