Search Unity

  1. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice

Deprecating IJobForEach, good replacement patterns

Discussion in 'Data Oriented Technology Stack' started by snacktime, Mar 13, 2020.

  1. snacktime

    snacktime

    Joined:
    Apr 15, 2013
    Posts:
    3,008

    The builder pattern I'm ok with sort of. I actually think that will make it easier on noobs especially since it provides defaults. It's all right there. You have to lookup what some things do, but not as much.

    Currently I think the biggest bang for buck feature we could get is an alternative to ForEach. It solves issues related to the lambda expressions. It would allow far more flexible abstractions to be created. Assuming the builder options don't require any local context then you could abstract out the entire thing mostly, leaving OnUpdate nice and clean.

    The alternative could be a job struct, or it could just be another level of indirection behind ForEach to allow it to support delegate reference types for the expression. I like the job struct better but I'd settle for delegates.
     
  2. PhilSA

    PhilSA

    Joined:
    Jul 11, 2013
    Posts:
    1,470
    Here's an attempt to illustrate what I had in mind:
    Code (CSharp):
    1.  
    2. public struct ExampleJobDescription : IJobDescription
    3. {
    4.     public float DeltaTime;
    5.     [ReadOnly]
    6.     public PhysicsWorld PhysicsWorld;
    7.  
    8.     // This is the update loop of our job.
    9.     // This would be used to know how to generate the whole IJobChunk
    10.     [JobUpdate]
    11.     public static void JobUpdate(ref ExampleJobDescription jobData, ref Translation translation, in PhysicsVelocity physicsVelocity)
    12.     {
    13.         translation.Value += physicsVelocity.Value * jobData.deltaTime;
    14.         // ... and pretend we need to do something with jobData.PhysicsWorld too
    15.     }
    16.  
    17.     // This is called to assign en entityQuery to the job when scheduling
    18.     public EntityQueryDesc GetQueryDesc()
    19.     {
    20.         return new EntityQueryDesc
    21.         {
    22.             All = new ComponentType[] { typeof(Translation), typeof(PhysicsVelocity), typeof(PhysicsCollider) }
    23.         };
    24.     }
    25. }
    26.  
    27. public class ExampleSystem : SystemBase
    28. {
    29.     protected override void OnUpdate()
    30.     {
    31.         ExampleJobDescription jobDesc = new ExampleJobDescription
    32.         {
    33.             DeltaTime = system.World.Time.DeltaTime,
    34.             PhysicsWorld = system.World.GetOrCreateSystem<BuildPhysicsWorld>().PhysicsWorld,
    35.         };
    36.         ScheduleJobDescription(jobDesc);
    37.     }
    38. }
    39.  

    Or here's another variant:
    Code (CSharp):
    1.  
    2. public struct JobData
    3. {
    4.     public float DeltaTime;
    5.     [ReadOnly]
    6.     public PhysicsWorld PhysicsWorld;
    7. }
    8.  
    9. public class ExampleSystem : GeneratedSystem<JobData>
    10. {
    11.     // This is the update loop of our job.
    12.     // This would be used to know how to generate the whole IJobChunk
    13.     [JobUpdate]
    14.     public static void JobUpdate(ref JobData jobData, ref Translation translation, in PhysicsVelocity physicsVelocity)
    15.     {
    16.         translation.Value += physicsVelocity.Value * jobData.deltaTime;
    17.  
    18.         // ... and pretend we need to do something with jobData.PhysicsWorld too
    19.     }
    20.  
    21.     // (Optional: only if you have additional job data to pass)
    22.     // This is called to construct the JobData struct that will be passed to the job right before scheduling
    23.     public override JobData PrepareJobData()
    24.     {
    25.         return new JobData
    26.         {
    27.             DeltaTime = World.Time.DeltaTime,
    28.             PhysicsWorld = World.GetOrCreateSystem<BuildPhysicsWorld>().PhysicsWorld,
    29.         };
    30.     }
    31.  
    32.     // (Optional: only if you don't want the default-generated Query desc)
    33.     // This is called to assign en entityQuery to the job when scheduling
    34.     public override EntityQueryDesc GetQueryDesc()
    35.     {
    36.         return new EntityQueryDesc
    37.         {
    38.             All = new ComponentType[] { typeof(Translation), typeof(PhysicsVelocity), typeof(PhysicsCollider) }
    39.         };
    40.     }
    41. }

    Theoretically, a user could write only this, and the entire job scheduling + IJobChunk would be generated from that. Although I haven't thought about it long enough and I don't understand on-the-fly codegen well enough to be sure that there are no pitfalls & limitations to this. What I prefer about this is that it feels more conventional programming-wise than the ForEach, and it's easier to understand what's really going on. There are more lines of code, but to me, it's less confusing

    But in order to make DOTS truly super accessible, perhaps the real solution to push for is bursted main thread code? I think I read you guys were working on this somewhere. It would still probably be much better performance-wise than monobehaviour (especially with all the rendering + physics + transforms being properly multithreaded), and would allow even beginners to get started super quickly. And if people encounter performance issues, they can look into jobifying only the stuff that's really too heavy
     
    Last edited: May 2, 2020
    GliderGuy likes this.
  3. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    5,086
    We are working on making System.Update burstable. However i don't understand how it would solve this. We still need a simple API the iterates over all entities and processes them.
     
  4. PhilSA

    PhilSA

    Joined:
    Jul 11, 2013
    Posts:
    1,470
    now that I think about it... you're right
     
  5. 8bitgoose

    8bitgoose

    Joined:
    Dec 28, 2014
    Posts:
    249
    Should be just like any other good programming practices. Two spots, one for method calls (in this case jobs) and an area to organize and schedule them. If the OnUpdate is 4000 lines long, that is a big pain for manageability.

    I have a question.

    Code (CSharp):
    1. public void Execute(ArchetypeChunk chunk, int chunkIndex, int firstEntityIndex)
    What does firstEntityIndex mean? Say I have three chunks of 100 entities, would it deliver 0, 100, 200 on each execution? If this is true, I can switch all my IJobForEach to chunks without much difficulty. I use entityIndex a lot to modify data in a consistent way with temporary nativeArrays.
     
  6. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    5,086
    Yes, thats what it does.
     
  7. 8bitgoose

    8bitgoose

    Joined:
    Dec 28, 2014
    Posts:
    249
    Thanks @Joachim_Ante, that is great. It wasn't super clear in the documentation, but that was many versions ago. I haven't checked the latest version of the docs to see if that was a bit clearer.
     
  8. Radu392

    Radu392

    Joined:
    Jan 6, 2016
    Posts:
    210
    I agree about ijobchunks. Since I heard about the deprecation news, I started using exclusivelly ijobchunks. I prefer stability and a bit of verbosity over having to constantly change my way of coding and going back to change tons of code just to keep up with all the api changes happening every few months.
     
    Karearea and 8bitgoose like this.
  9. 8bitgoose

    8bitgoose

    Joined:
    Dec 28, 2014
    Posts:
    249
    Seems like IJobChunk is here to stay since it is pretty low level. @Radu392 have you found any differences between that and IJobForEach? Any gotchas?
     
  10. EwieElektro

    EwieElektro

    Joined:
    Feb 22, 2016
    Posts:
    30
    Hey folks,
    I currently try to port some IJobForEach... to Entities.ForEach....
    Now i have a big problem. I need to run a ForEach only as single thread (not on main thread). ScheduleSingle does'nt exist on the new approach :/ Some ideas how to handle that?
    Multi threaded is not an option, because i need to change a Component (kind of cached counter) from other entities

    pseudo:

    counterComponent = GetComponentDataFromEntity<CounterComponent>(false)

    ForEach < EntityLinkComponent eLink>
    counterComponent [ eLink.entitiy ].counter ++
     
  11. joepl

    joepl

    Unity Technologies

    Joined:
    Jul 6, 2017
    Posts:
    71
    @EwieElektro Entities.ForEach().Schedule should indicate single thread scheduling when used in SystemBase (this indicated parallel execution in JobComponentSystem, but you need to explicitly use ScheduleParallel in SystemBase).
     
  12. EwieElektro

    EwieElektro

    Joined:
    Feb 22, 2016
    Posts:
    30
    oh, i still used JobComponentSystem :D thank you!! :)
     
  13. joepl

    joepl

    Unity Technologies

    Joined:
    Jul 6, 2017
    Posts:
    71
  14. maxxa05

    maxxa05

    Joined:
    Nov 17, 2012
    Posts:
    118
    What I will really miss the most is the ability to make a generic system that would do a similar operation, but on a different query. Consider this method:

    Code (CSharp):
    1.         protected T GetComponentWithHighestPriority<T>(T defaultValue)
    2.         {
    3.             T component = defaultValue;
    4.             int currentPriority = int.MinValue;
    5.        
    6.             var foundEntity = Entity.Null;
    7.        
    8.             Entities.WithAll<EnvironmentOverrideSetActive, T>()
    9.                 .ForEach((Entity entity, in EnvironmentOverrideSet set) =>
    10.                 {
    11.                     if (set.Priority >= currentPriority)
    12.                     {
    13.                         currentPriority = set.Priority;
    14.                         foundEntity = entity;
    15.                     }
    16.                 }).Run();
    17.  
    18.             return EntityManager.GetComponentData<T>(foundEntity);
    19.         }
    This code would iterate on a query of OverrideSet that are active and contain T, and return the component with the highest priority. But it does not compile. If we'd be able to at least feed a specific query in the ForEach that would contain T, it could work, but even that is not possible. And that's not even trying to assign the T variable directly in the job instead of doing it with EntityManager.GetComponentData, which would be the best possible way. The only way to do something like that is by rewriting the whole job description and job lambda over and over again in every system that needs this. A bit could be rewritten in a static job, but it would not save a whole lot of code to be honest.

    Before I could at least feed a query when I scheduled a job, with said query being created using the T type. Now it's not possible anymore.
     
    Last edited: May 16, 2020
unityunity