Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. Dismiss Notice

Feedback request: IJobEntity

Discussion in 'Entity Component System' started by joepl, May 11, 2020.

  1. joepl

    joepl

    Unity Technologies

    Joined:
    Jul 6, 2017
    Posts:
    85
    A while back we deprecated IJobForEach. This was done due to make sure that we weren't continuing to support an interface that would have performance issues and could not be supported easily with DOTS Runtime. The two remaining solutions however - Entities.ForEach and IJobChunk - did not provide a middle ground for writing jobs that deal with entities and their components. You either wrote your code in an inline lambda or you wrote all of the per-entity iteration code on your own in an IJobChunk.

    IJobEntity would provide a way to define entity iteration for users wanting similar functionality to IJobForEach. With this interface, it would be possible to define the job and query separate from an individual system and use it in multiple places. It is also built on much simpler codegen than Enties.ForEach. The job type is a separate type that is unmodified by codegen (a surrounding IJobChunk is generated that uses it). This allows better scaffolding to IJobChunk and more understandable generated code.

    The interface has no abstract (or other) methods. Users can define a OnUpdate method that will get called for every entity that matches the query generated for the job. Queries are specified with the method parameters of OnUpdate (along with usage of 'ref', 'in', etc. similar to Entities.ForEach) as well as for any optional attributes placed on the OnUpdate method (`[AllInQuery]`, `[AnyInQuery]`, `[NoneInQuery]`, etc.).

    Code (CSharp):
    1. // EXAMPLE : RotationSpeedSystem
    2.  
    3. // User defined Job Type
    4. [BurstCompile]
    5. struct RotateEntityJob : IJobEntity
    6. {
    7.     public float DeltaTime;
    8.  
    9.     public void OnUpdate(ref Rotation rotation, in RotationSpeed_ForEach speed)
    10.     {
    11.         rotation.Value = math.mul(math.normalize(rotation.Value), quaternion.AxisAngle(math.up(), speed.RadiansPerSecond * DeltaTime));
    12.     }
    13. }
    14.  
    15. // System that schedules the Job to run with parallel execution
    16. public class RotationSpeedSystem : SystemBase
    17. {
    18.    protected override void OnUpdate()
    19.    {
    20.         var rotateEntityJob = new RotateEntityJob() { DeltaTime = Time.DeltaTime };
    21.         Entities.OnUpdate(rotateEntityJob).ScheduleParallel();
    22.    }
    23. }
    Advantages over Entities.ForEach include:
    - Easier code re-use
    - Not built on lambdas (which some users clearly love and some...do not)
    - Generated code looks more similar to user written IJobChunk code (viewable through DOTS Compiler Inspector)
    - Can support methods with any combination of parameters (no need for predefined set of delegates like Entities.ForEach)

    Disadvantages are:
    - No automatic capturing/writing back to local variables (they would need to be manually set on job struct)
    - Some things will still need to be defined with attributes (BurstCompile, optional EntityQuery components and ones not defined in OnUpdate params)
    - No patching of entity access through SystemBase methods (such as how we allow GetComponent/SetComponent inside of Entities.ForEach)
     
  2. MhmdAL1111

    MhmdAL1111

    Joined:
    Oct 25, 2017
    Posts:
    30
    That looks like a great solution. I would use it for 90% of my entity jobs. I think the disadvantages are not really significant and worth the sacrifice. :)
     
  3. Deleted User

    Deleted User

    Guest

    Agreed with the statement above. The disadvantages are very minor and won't affect many users that are still on IJobChunk or IJobForeach.

    One side note IJobEntity name is not as specific as IJobForeach, maybe something like IJobBase, IJobUpdate, or even IJobForeach itself just without generic parameters will be helpful to slowly deprecate generic version of it.
     
  4. DreamingImLatios

    DreamingImLatios

    Joined:
    Jun 3, 2017
    Posts:
    3,984
    I love the interface name. That is about as intuitive of an interface name as you could get, as it compares itself well to IJobChunk.
    I don't like the interface's method being called OnUpdate instead of Execute. OnUpdate is for systems. Similarly, invoking the job also could use a better name, such as reusing ForEach, or maybe Entities.Job or something.
     
    TWolfram, lclemens, JoNax97 and 7 others like this.
  5. PhilSA

    PhilSA

    Joined:
    Jul 11, 2013
    Posts:
    1,926
    This looks fantastic! I love it

    Seems like it's basically an IJobChunk that skips all the boilerplate

    Would it be possible to also have a "OnBeginChunk()" function that we can implement aside from OnUpdate()? To give us an opportunity to do some per-chunk logic before we iterate on all entities of the chunk? I feel like we'd almost never need to use IJobChunks again if we had that (for use cases like these, it would be better than working with a threadIndex at every entity iteration, possibly)
     
    Last edited: May 11, 2020
    Orimay and JesOb like this.
  6. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    2,653
    Nice to see, that Unity heard community :) Disadvantages really negligible
    Couple of small things which will be useful:
    - Explicit query usage (same many people want fot Entities.ForEach), when we can explicitly define query, set types and access rules, and use that query as argument for that jobs (and Entities.ForEach) for reducing WithAll (AllInQuery) WithNone (NoneInQuery) WithAny (AnyInQuery) (and that namings, maybe lead them to one denominator? :) )
    - About
    ref
    , extend it somehow if possible (through codegen and code analysis maybe) for check if we really write to component (like with setter of ComponentDataFromEntity, chunk.GetNativeArray) for bumping chunk version and not just bump it every time just by definition. It will make life easier for change filtering :)

    Can be a bit confusing for newcomers, because we have IJobFor :) IJobEntity IMO best naming for that type of job.

    I also agree with this point. Well Execute (for me) sounds natural and represent behaviour which I expect - execute that block of code.
     
    lclemens, lndcobra, JakesOC and 7 others like this.
  7. snacktime

    snacktime

    Joined:
    Apr 15, 2013
    Posts:
    3,356
    Pretty much what I think most of us wanted. I also don't think OnUpdate is a good name. It has strong historical ties to the gameloop and I think it's best kept reserved for that.
     
    lclemens, Orimay and charleshendry like this.
  8. thelebaron

    thelebaron

    Joined:
    Jun 2, 2013
    Posts:
    825
    Is there a reason it cant reuse the same scheduling as chunk/old ijobforeach, ie Dependency = new Job().ScheduleParallel(myQuery, Dependency)? I personally prefer that over the proposed Entities.WithXXX.OnUpdate(job) etc
     
    Orimay likes this.
  9. PhilSA

    PhilSA

    Joined:
    Jul 11, 2013
    Posts:
    1,926
    A small nitpick:

    instead of
    • Entities.OnUpdate(rotateEntityJob).ScheduleParallel();
    I think I'd prefer something like one of these:
    • Entities.ScheduleParallel(rotateEntityJob)
    • Entities.CreateJob(rotateEntityJob).ScheduleParallel()
    • rotateEntityJob.ScheduleParallel()
    Especially that last one, if the codegen can deal with it... but that is very much a nitpick

    And as @eizenhorn mentioned, an optional entityQuery that we can specify when scheduling would be great
     
    Last edited: May 12, 2020
    andreiagmu, lndcobra, JakesOC and 3 others like this.
  10. psuong

    psuong

    Joined:
    Jun 11, 2014
    Posts:
    118
    I generally prefer Phil's suggestion with `Entities.ScheduleParallel(rotateEntityJob)`, it makes it easier to read and see what it's doing at a glance.

    With IJobEntity's OnUpdate(params), the OnUpdate method kind of picks at me since Execute(...) is so much more familiar, since IJob/IJobParallelFor/IJobChunk still use it.

    I'm not too much of a fan with Attributes in function params, I find that syntax to be difficult to read, so being able to use entity queries alongside scheduling would be very nice addition.
     
    KwahuNashoba and Orimay like this.
  11. jdtec

    jdtec

    Joined:
    Oct 25, 2017
    Posts:
    297
    +1 for:

    1) (and most important) - an optional entityQuery that we can specify when scheduling would be great
    2) Execute instead of OnUpdate
    3) This style: Dependency = new Job().ScheduleParallel(myQuery, Dependency)

    Good work DOTs team!
     
  12. 8bitgoose

    8bitgoose

    Joined:
    Dec 28, 2014
    Posts:
    449
    This is looking pretty good, still needs an OnUpdate function that would give you Entity and iteration Index so same as IJobForEachWithEntity. I use that a lot!

    Thanks for listening to our feedback :).
     
    Orimay likes this.
  13. JoNax97

    JoNax97

    Joined:
    Feb 4, 2016
    Posts:
    611
    Yeah IpI much agree with the above.
    • The name is is clear. If IJobChunk is a job over a chunk, IJobEntity is a job over an entity. Is consistent.
    • Use Execute() consistently with other job types.
    Also I know this is unrelated but I don't think deserves a separate post... I find Job.WithCode to be a pretty bad name. Everything is code so it's a meaningless name. I think Job.Execute ( lambda) would be clearer in intent (execute what I'm passing you) and consistent with other job-related terminology.
     
    Orimay, florianhanke, jdtec and 2 others like this.
  14. joepl

    joepl

    Unity Technologies

    Joined:
    Jul 6, 2017
    Posts:
    85
    Thanks for all the great feedback!

    To consolidate a bit:
    • Sounds like Execute() is much preferred over OnUpdate() for the job method to be called. I think this makes a lot of sense.
    • It is important to be able to supply/reuse an optional query from the scheduling site (this goes along with thinking of an addition to Entities.ForEach as well).
    • Having a OnBeginChunk/OnBegin/OnPost methods would be handy (probably not going to be in first iteration, but noted).
    • We should be able to define Execute methods that take an entity and entity/thread indices (this is a given).
    • Attributes on method params aren't great if we can avoid them and it would be nice to actually check to see if we write to writable params.
    • I think the following is what we are leaning towards as far as what the scheduling invocation would look like:

      Code (CSharp):
      1. // Use Execute method (Fetch from parameters + attributes)
      2. var rotateEntityJob = new RotateEntityJob() { DeltaTime = Time.DeltaTime };
      3. Entities.Execute(rotateEntityJob).ScheduleParallel();
      4. // Allow WithQuery on all Entities.ForEach and Execute etc
      5. var rotateEntityJob = new RotateEntityJob() { DeltaTime = Time.DeltaTime };
      6. Entities.WithQuery(myQuery).Execute(rotateEntityJob).ScheduleParallel();
      7. // Allow WithAll both Entities.ForEach and Execute
      8. var rotateEntityJob = new RotateEntityJob() { DeltaTime = Time.DeltaTime };
      9. Entities.WithAll<AnotherRequiredComponent>.Execute(rotateEntityJob).ScheduleParallel();
      Also, just to be clear, this is still just a design for an API. Internal buy-off needs to happen and the work needs to be complete before this is a reality. From all the feedback, it sounds like this would be a pretty significant improvement however.
     
  15. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    3,627
    I approve
     
  16. iamarugin

    iamarugin

    Joined:
    Dec 17, 2014
    Posts:
    863
    Looks great.
     
  17. DreamersINC

    DreamersINC

    Joined:
    Mar 4, 2015
    Posts:
    130
    If it increase the readability of my code be removing all the WithReadOnly/WithDisable...... boilerplate code involve in Entities.ForEach job, I am in
     
  18. Zec_

    Zec_

    Joined:
    Feb 9, 2017
    Posts:
    148
    I think it looks great! I more or less create these kinds of structs already using Entities.ForEach, but the fact that we need to write the method declaration inside of the ForEach and invoke a method in the struct manually makes it tedious. This would simplify a lot.
     
  19. 8bitgoose

    8bitgoose

    Joined:
    Dec 28, 2014
    Posts:
    449
    Super pumped about this! Thanks for making sure this is implemented. I think it is a good idea to allow the rest of the Lamba stuff on these ForEachEntity structs. I think having the Lambas override attributes on the ForEachEntity would make the most sense, for example if your lambda said to include ComponentA but the ForEachEntity said to exclude ComponentA or something.
     
  20. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    5,203
    I would say they should be exclusive. Either you specify attributes, or you specify WithAll, WithNone.
    Both of them are relative to the parameter list of the Execute method.

    When both attributes & WithAll are defined we give a compile error. I think if both are used. It will be exceptionally unintuitive to understand what it will actually happen. So better prevent it.
     
  21. 8bitgoose

    8bitgoose

    Joined:
    Dec 28, 2014
    Posts:
    449
    @Joachim_Ante A compile error would be even better, hadn't though of that.
     
  22. Orimay

    Orimay

    Joined:
    Nov 16, 2012
    Posts:
    304
    Can we have multiple Execute methods within a job, while having different signatures? For instance, to have a single job for all the transform operation combinations instead of having a job per combination.

    Also, would've been awesome to have some sort of extension methods on the interface in order to be able to use GetComponent, GetBuffer etc within such structs. It's really awesome thing to have
     
    Last edited: May 16, 2020
  23. WAYNGames

    WAYNGames

    Joined:
    Mar 16, 2019
    Posts:
    939
    I am a bit bothered with the Entities. Execute(job).schedule....

    Seems like execute and schedule are a bit contradictory to me.

    I would prefer a Entites.WithJob(job).schedule
    Or
    Etities. ScheduledJob(job)
     
    lclemens, florianhanke and lndcobra like this.
  24. JoNax97

    JoNax97

    Joined:
    Feb 4, 2016
    Posts:
    611
    Yeah I agree with the comment above. Execute is the right name for the method inside the job, but not for the Entities method
     
    lclemens likes this.
  25. iamarugin

    iamarugin

    Joined:
    Dec 17, 2014
    Posts:
    863
    I don't like ScheduledJob(job) because it is inconsistent with Entities.Foreach. And Dependency as a second parameter would look ugly.
     
    KwahuNashoba likes this.
  26. cultureulterior

    cultureulterior

    Joined:
    Mar 15, 2015
    Posts:
    68
    Looks amazing! Would be nice if you paused deprecation of IJobForEach (keeping it deprecated, but not removing it) until this arrived honestly.
     
    MostHated and charleshendry like this.
  27. 8bitgoose

    8bitgoose

    Joined:
    Dec 28, 2014
    Posts:
    449
    Best to depreciate it so there is fewer stuff to maintain. Less maintain code == more new features.
     
  28. snacktime

    snacktime

    Joined:
    Apr 15, 2013
    Posts:
    3,356
    So hit a restriction today that I'm happy to look at alternatives but don't see anything obvious. Which is Job and Entities I can't pass via a method as they can only be used in a complete builder context. So any context that is complex, long lived, and you need multiple of them running concurrently I don't see a good way to do that.

    We have multiple places where we have this paradigm. Navmesh surface/crowd handling for example.

    Something like this is what I would like to do.

    Code (csharp):
    1.  
    2. public abstract class AiSubSystem
    3.     {
    4.      
    5.         protected LambdaSingleJobDescription Job;
    6.         protected ForEachLambdaJobDescription Entities;
    7.  
    8.    
    9.         public void Init(LambdaSingleJobDescription job, ForEachLambdaJobDescription entities)
    10.         {
    11.             Job = job;
    12.             Entities = entities;
    13.         }
    14.  
    15.         public virtual JobHandle OnUpdate(JobHandle Dependency)
    16.         {
    17.             return Dependency;
    18.         }
    19.     }
    20.  
     
    Orimay likes this.
  29. snacktime

    snacktime

    Joined:
    Apr 15, 2013
    Posts:
    3,356
    So the pattern I came up with that works now and hopefully won't break with some future update is putting every Job/Entities into it's own public method that takes the job struct. Then pass the system instance to the code that needs to call those methods.

    Breaks encapsulation in multiple ways that I don't like.

    Job/Entities after a second glance look like they probably create a new instance on each use so the approach I posted above seems like a non starter.

    In any case long term a design that prevents abstracting out jobs seems problematic. That's going to create so much bad code over the long term that it's worth doing right IMO.
     
  30. RogueStargun

    RogueStargun

    Joined:
    Aug 5, 2018
    Posts:
    286
    Personally, I am fine with Entities.ForEach. It's intuitive, and reminds me a bit of SQLAlchemy querying. The main issue isn't that its bad, it's that its hard to remember where to put the semicolons and brackets!
     
  31. iamarugin

    iamarugin

    Joined:
    Dec 17, 2014
    Posts:
    863
    Entities.ForEach is fantastic.
     
  32. 8bitgoose

    8bitgoose

    Joined:
    Dec 28, 2014
    Posts:
    449
    Entities.ForEach is fantastic for short stuff for sure. Good for prototyping which is harder in entity systems. It won't handle very complex stuff. Glad we have both.
     
    MostHated likes this.
  33. zachlindblad

    zachlindblad

    Joined:
    Sep 29, 2016
    Posts:
    39
    @joepl Are there any updates on if/when IJobEntity is being added?
    It looks like IJobForEach has already been removed from the most recent Entities packages, but I don't really want to update till I won't have to move all my code over to lambdas.
    Really hope it's coming soon as I really do like how it fixes the removal of the previous API.
    Thanks!
     
  34. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    5,203
    IJobForEach has not yet been removed. We are currently upgrading all our codegen to C# source generators, since they will:
    * Optimize compile time / iteration speed
    * Simplify creating codegen code
    * make code more debuggable

    This is a bigger effort and we are doing that before we add more job types. Hence the delay. But we are getting close to being able to land that.

    We will also ship IJobEntity as part of this ugprade.
     
    Last edited: Sep 15, 2020
    lclemens, andreiagmu, Ruchir and 12 others like this.
  35. zachlindblad

    zachlindblad

    Joined:
    Sep 29, 2016
    Posts:
    39
  36. jdtec

    jdtec

    Joined:
    Oct 25, 2017
    Posts:
    297
    Ah wow that sounds great. Those are some of my biggest gripes with DOTs. Tried out 2020.2 beta and that seems to have a better iteration speed back too. Things are looking good.
     
  37. Ashkan_gc

    Ashkan_gc

    Joined:
    Aug 12, 2009
    Posts:
    1,102
    This seems very interesting and I somehow like it more than Entities.Foreach but don't know why yet
     
  38. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    10,580
    I am probably a bit late to this train, but I definatelly would be happy see more IJobs over lambda approaches. Specially because recent IJob type depreciation.

    Also, I Agree, that execute is much more suitable, than OnUpdate. Less confusion.
     
  39. lndcobra

    lndcobra

    Joined:
    Jan 28, 2020
    Posts:
    21
    100% Agree with this, I do no think we should be having:

    Code (CSharp):
    1. Entities.Execute(rotateEntityJob).ScheduleParallel();
    My preference

    Code (CSharp):
    1. // Ideal, would need compiler magic so maybe out of scope?
    2. rotateEntityJob.ScheduleParallel();
    3.  
    4. // Would need ability for overloaded methods? Not sure if it is supported in your compile chain
    5. Entities.ScheduleParallel(rotateEntityJob);
    6.  
    7. // If overloads not supported
    8. Entities.ScheduleParallelJob(rotateEntityJob);
    9. Entities.ScheduleParallelEntityJob(rotateEntityJob);
    10.  
    11. // Otherwise my vote would go to an overloaded one of the following:
    12. Entites.Job(rotateEntityJob).ScheduleParallel();
    13. Entites.WithJob(rotateEntityJob).ScheduleParallel();
    14.  
    15. // If overloading not supported
    16. Entites.WithEntityJob(rotateEntityJob).ScheduleParallel();
     
    brunocoimbra and apkdev like this.
  40. lclemens

    lclemens

    Joined:
    Feb 15, 2020
    Posts:
    714
    I like it. I will miss GetComponent() and SetComponent() though - they are very handy and much cleaner than setting up GetComponentDataFromEntity(). Is there any way to somehow sneak those back in?
     
  41. Sinyavtsev

    Sinyavtsev

    Joined:
    Feb 29, 2016
    Posts:
    8
    Hey! Any news on it?
     
    rbehshad, lclemens and bb8_1 like this.
  42. 8bitgoose

    8bitgoose

    Joined:
    Dec 28, 2014
    Posts:
    449
  43. DaxodeUnity

    DaxodeUnity

    Unity Technologies

    Joined:
    Aug 5, 2021
    Posts:
    27
    Hope you guys like it :)
     
  44. 8bitgoose

    8bitgoose

    Joined:
    Dec 28, 2014
    Posts:
    449
    From what I've read, it seems to replicate the old system it replaced but the syntax is way less messy and it has more flexibility! Thanks for adding this and not just focusing on the Lambda systems :).
     
    bb8_1 likes this.
  45. DaxodeUnity

    DaxodeUnity

    Unity Technologies

    Joined:
    Aug 5, 2021
    Posts:
    27
    Yup, it's meant to read like a job, but with way less typing involved, you can even use your own queries, just make sure they match with your execute parameters :)
     
  46. Abbrew

    Abbrew

    Joined:
    Jan 1, 2018
    Posts:
    417
    All my IJobEntity have the following warning

    There is no defined ordering between fields in multiple declarations of partial struct 'DecisionMakerSystem.DecisionMakerJob'. To specify an ordering, all instance fields must be in the same declaration. [Collapse]csharp(CS0282)
     
  47. lclemens

    lclemens

    Joined:
    Feb 15, 2020
    Posts:
    714
    This works pretty well...

    But I really miss GetComponent(). It would be really nice if there was an easy way to get that back in (or access data from other components without the extra boilerplate of GetComponentDataFromEntity.

    Also, there is no easy way to exclude a component (that I know of). I have to maintain a query with 15 parameters that match the 15 parameters of Execute() exactly, and then have an additional ComponentType.Exclude<DeathData>() in the query?

    If these two things were resolved, it would get a 10/10!

    [EDIT: Good news concerning the second issue - although it's not in the documentation for IJobEntity, a clever person in the Turbo Makes Games discord figured out that this works --> [WithNone(typeof(SomeTag))] public partial struct SomeJob : IJobEntity { void Execute(){} } . ]
     
    Last edited: Mar 30, 2022
    JohnnyTurbo and bb8_1 like this.
  48. DaxodeUnity

    DaxodeUnity

    Unity Technologies

    Joined:
    Aug 5, 2021
    Posts:
    27
    This is sadly a limitation of what happens when Roslyn throws one error in unity, they also throw all their warnings (which in this case is irrelevant) Would be nice if you found the odd one out and wrote it here, I'll be happy to help :)

    Thanks, hope it helped :3
     
    lclemens likes this.
  49. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    29,723
    I'm confused what the true north is. I want this syntax but all the stuff I'm reading online doesn't use this syntax. So I've decided to go cry in a corner. I don't like lambdas and boilerplate, so this looks like an appealing syntax to me.

    I just would prefer if Unity updated some examples using this newer, cleaner way to iterate over entities. Would that be possible?

    I suppose I'm left wondering what the right way to write code is.
     
    Daxode likes this.
  50. thelebaron

    thelebaron

    Joined:
    Jun 2, 2013
    Posts:
    825
    If case you haven't seen it already, this page explains its use - https://docs.unity3d.com/Packages/com.unity.entities@0.50/manual/ecs_ijobentity.html

    edit: just read up and saw this already posted, so if there are some specific instances from say the entities repo post them and maybe I or someone else can help with a code example
     
    hippocoder likes this.