Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Most up to date and performant way to do "Non-Job Systems"?

Discussion in 'C# Job System' started by Guedez, Feb 19, 2019.

  1. Guedez

    Guedez

    Joined:
    Jun 1, 2012
    Posts:
    827
    My general understanding is that there are two ways of doing a system loop:
    A) using
    [Inject] MyQueryObject ConsumeItemOperation;
    and then iterating over it's
    ComponentDataArray
    fields
    B) using
    m_MainGroup = GetComponentGroup(typeof(MyIComponentDataA), typeof(MyIComponentDataB));
    and then iterating over
    m_MainGroup.GetComponentDataArray<MyIComponentDataB>()


    So.. what are the pros and cons of each approach? Or they are interchangeable? Is there a 3rd option?

    UPDATE
    If you found this on google, here is a probably complete example:
    Many caveats you probably will want to know in the thread regardless
    Code (CSharp):
    1.  
    2. using Unity.Entities;
    3.  
    4. public class YourSystem : ComponentSystem {
    5.     ComponentGroup m_MainGroup;
    6.  
    7.     protected override void OnCreateManager() {
    8.         m_MainGroup = GetComponentGroup(typeof(YourISharedComponentData), ComponentType.Subtractive(typeof(TheIComponentDataYouDontLike)), typeof(YourIComponentData));
    9.     }
    10.     protected override void OnUpdate() {//ISharedComponentData use ref, IComponentData does not
    11.         ForEach((Entity entity, YourISharedComponentData no_ref_here, ref YourIComponentData notice_the_ref) => {
    12.             Do.Your.Code.Here();
    13.         }, m_MainGroup);// <- this is needed to filter out TheIComponentDataYouDontLike
    14.     }
    15. }
    16.  
     
    Last edited: Feb 20, 2019
    MostHated and Sibz9000 like this.
  2. GilCat

    GilCat

    Joined:
    Sep 21, 2013
    Posts:
    676
    [Inject] is soon to be deprecated so use the B)
     
  3. Guedez

    Guedez

    Joined:
    Jun 1, 2012
    Posts:
    827
    Oh wow, most of my systems are on [Inject]
    I guess there's nothing much to discuss unless there's a 3rd option (that is not using Jobs)
     
  4. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    3,753
    GetComponentDataArray is also depreciated so use C) chunk iteration (or new ForEach if you're just on main thread).
     
    Last edited: Feb 19, 2019
  5. Guedez

    Guedez

    Joined:
    Jun 1, 2012
    Posts:
    827
    Time to get googling
     
  6. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    5,203
    The default recommended way of writing main thread code is to use ForEach().

    Generally we are saying:
    1. ComponentSystem is convenient and easy to use
    2. JobComponentSystem has performance and is easy to use as we can make it but not at the expense of performance

    So generally if you care about performance in any way. Use JobComponentSystem.
     
    Last edited: Feb 19, 2019
    timmehhhhhhh and FROS7 like this.
  7. MostHated

    MostHated

    Joined:
    Nov 29, 2015
    Posts:
    1,235
    I had read before when researching optimizations that it was best to use the standard :

    Code (CSharp):
    1.             for (int i = 0; i < whatever.Length; i++)
    2.             {
    3.                
    4.             }
    due to foreach not being as performant, producing garbage, etc from sites such as these below (I realize they are older, but that is why I am inquiring currently)

    https://jacksondunstan.com/articles/3805
    https://jacksondunstan.com/articles/4573
    https://gamedev.stackexchange.com/q...ore-efficient-than-the-for-each-loop-in-unity

    I was just wondering if that has now changed and foreach is an ok way to go about things, as I have always been sticking with for loops because of that.
     
  8. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    3,753
    you're thinking of a different foreach. There is a new ForEach syntax for component systems.

    Code (CSharp):
    1.     protected override void OnUpdate()
    2.     {
    3.         ForEach((Position position) =>
    4.         {        
    5.        
    6.         });
    7.     }
     
  9. recursive

    recursive

    Joined:
    Jul 12, 2012
    Posts:
    669
    @MostHated - I believe the switch from 3.5 to 4.X .NET fixed the GC allocated foreach (language construct) iterator (for lists and arrays only), but while the iterator is a struct, a raw for loop that doesn't access the Count or Length property every iteration is optimal:

    Code (CSharp):
    1. int len = someArray.Length; //or someList.Count;
    2. for(int i = 0; i < len; i++)
    3. {
    4.    // do work...
    5. }
    That said, @Joachim_Ante's suggestion of the ComponentGroup ForEach function deals with the chunk and element access overhead for you on the main thread, so I'd still probably use that if you're using ComponentSystem and don't need to do something with chunks themselves.

    There's more documentation on the Lambda ForEach here: https://github.com/Unity-Technologi...ed.md#componentsystem---a-step-into-a-new-era

    You can also use a ComponentGroup to take advantage of it's ability to filter types and use subtractive components:

    https://forum.unity.com/threads/lambda-foreach-subtractive-components.617353/
     
    NotaNaN and MostHated like this.
  10. MostHated

    MostHated

    Joined:
    Nov 29, 2015
    Posts:
    1,235
    Oh, that's a "my bad" then. I didn't realize there was even another.

    Thank you much as well for the info.
     
    Last edited: Feb 20, 2019
    NotaNaN and recursive like this.
  11. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    5,203
    Code (CSharp):
    1. protected override void OnUpdate()
    2.     {
    3.         ForEach((ref Position position) =>
    4.         {    
    5.    
    6.         });
    7.     }
    You actually need to use ref, but yes. Thats how that works.
     
    GilCat likes this.
  12. Srokaaa

    Srokaaa

    Joined:
    Sep 18, 2018
    Posts:
    169
    I wonder, won't ForEach cause Allocation when I would like to use something like EntityManager from the current system inside? I think in C# lambdas are allocation free as long as they don't reference anything outside its local scope. I might be wrong though
     
  13. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    5,203
    ForEach can allocate GC in some cases. If you want your code to run fast and have no GC alloc you should use JobComponentSystem.
     
  14. Guedez

    Guedez

    Joined:
    Jun 1, 2012
    Posts:
    827
    How do I use this ForEach with ISharedComponentData?
     
  15. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    5,203
    You can ForEach ((Entity entity, ....))

    And then use EntityManager.GetSharedComponentData<MyData>(entity);
     
  16. Guedez

    Guedez

    Joined:
    Jun 1, 2012
    Posts:
    827
    So even though I know that many entites share the same ISharedComponentData, I will have to retrieve the same ISharedComponentData many times rather than iterating over all ISharedComponentData and then iterating over all entites associated with it?

    I guess it works, but it's very counter intuitive
     
  17. JPrzemieniecki

    JPrzemieniecki

    Joined:
    Feb 7, 2013
    Posts:
    33
    Looking at the available delegate types
    Code (csharp):
    1. ForEach((MySharedComp comp, ref Position pos) =>{});
    seems to be supported as well, the required order of arguments being Entity, shared components, hybrid components, normal components (look at Unity.Entities/Iterators/ForEachIterator.generated.cs for the full list of available delegates).

    P.S. ForEach can filter by component group (passed as a second argument)
    Unfortunately there is no entity-only overload, so we can't do
    ForEach((Entity e) =>{}, group);
     
  18. Srokaaa

    Srokaaa

    Joined:
    Sep 18, 2018
    Posts:
    169
    @Joachim_Ante Maybe we could get another overload of ForEach then so that we may pass some arbitrary type to the function:

    Code (CSharp):
    1.  
    2.         protected delegate void F_SD<S0, T0>(ref S0 s0, ref T0 c0) where T0 : struct, IComponentData;
    3.  
    4.         protected void ForEach<S0, T0>(S0 systemOrData, F_SD<S0, T0> operate, ComponentGroup group = null)
    5.             where T0 : struct, IComponentData
    6.         {
    7.             // Blah, blah, blah..
    8.             operate(ref systemOrData, ref UnsafeUtilityEx.ArrayElementAsRef<T0>(array0, i));
    9.             // Blah, blah, blah..
    10.         }
    11.  
    We could then us it like this:

    Code (CSharp):
    1.  
    2.             ForEach(this, (ref CurrentSystem system, ref Position position) =>
    3.             {
    4.                 system.EntityManage.Foo()
    5.                
    6.             });
    7.  
    This wouldn't allocate 60 (or whatever the framerate is) objects per second
     
    RaL and M_R like this.
  19. Guedez

    Guedez

    Joined:
    Jun 1, 2012
    Posts:
    827
    I attempted with the "ref" preceding the ISharedComponentData and didn't occur to me to test if it would work without the "ref".
    Seems to be working.
    For those finding this thread through google, a example for the answer to the first question in the thread should be something like this:
    Code (CSharp):
    1. using Unity.Entities;
    2.  
    3. [UpdateBefore(typeof(Cooking2HighLimitSystem))]
    4. [UpdateBefore(typeof(Cooking2LowLimitSystem))]
    5. public class Cooking1IdealTemperatureSystem : ComponentSystem {
    6.     ComponentGroup m_MainGroup;
    7.  
    8.     protected override void OnCreateManager() {
    9.         m_MainGroup = GetComponentGroup(typeof(CookingUtensilReference), ComponentType.Subtractive(typeof(CookingIdealTemperature)), typeof(CookPropertiesProcessor.CookProperties));
    10.     }
    11.     protected override void OnUpdate() {
    12.         ForEach((Entity entity, CookingUtensilReference comp, ref CookPropertiesProcessor.CookProperties props) => {
    13.             PostUpdateCommands.AddComponent(entity, new CookingIdealTemperature(props.Ideal_Temperature));
    14.         }, m_MainGroup);
    15.     }
    16. }
    I am unsure if the ", m_MainGroup);" at the end of the ForEach is required or not, as I assume that this is implicit by the fact that this system only runs if those conditions are met. Maybe it's meant to be used if you want one system using many groups?
    Some observations for context:
    CookingUtensilReference 
    is a
    ISharedComponentData 

    CookingIdealTemperature 
    is a
    IComponentData

    CookPropertiesProcessor.CookProperties
    is also a
    IComponentData

    This system whole point is to enable modding (using a untested technique that I theorise to be possible)
    Since
    CookPropertiesProcessor.CookProperties
    is a copy of the ingredient base properties, it's ideal to not change it, therefore a new component data is created to store the value. A modder could then create a new system, which have this attribute:
    [UpdateAfter(typeof(Cooking1IdealTemperatureSystem))]
    , which then changes the
    CookingIdealTemperature 
    value
     
  20. JPrzemieniecki

    JPrzemieniecki

    Joined:
    Feb 7, 2013
    Posts:
    33
    The ForEach by default creates a group based on all the lambda parameters. You pass your own group if you want to apply additional filters (in your case, without the group entities with CookingIdealTemperature would be processed in the ForEach, but the group filters them out.)
     
  21. NotaNaN

    NotaNaN

    Joined:
    Dec 14, 2018
    Posts:
    325
    Pardon my ignorance... But does this mean ForEach should be used over Chunk-Iteration in the case of "unjobified" code? Or does Chunk-Iteration still take the cake in terms of which is more performant? (Cause if the latter is true i think I'll just stick with Chunk-Iteration). :p

    (Plus Chunk-Iteration just sounds cooler than ForEach... Like, if i was going to go around blowing my own horn I would rather want to say: "i know how to do Chunk-Iteration!" than: "i can use ForEach!") :D
     
  22. JPrzemieniecki

    JPrzemieniecki

    Joined:
    Feb 7, 2013
    Posts:
    33
    ForEach is just a thin wrapper around chunk iteration, the allocations Joachim Ante mentions come from allocating a closure for the lambda you pass. There are two cases here:
    Code (CSharp):
    1. //case 1 - closure allocated
    2. var x = GetSomeValue();
    3. ForEach ((ref Comp c) => {c.Value = x;});
    4.  
    5. //case 2 - no allocation
    6. ForEach((ref Comp c) => {c.Value = 5;});
    Notice how in the first case the lambda uses a value from the outside. That value has to have some memory allocated for it, so a new closure object is allocated and passed to the ForEach. Case 2 does not need any additional memory that changes with each invocation, so no allocation is required*.

    The annoying part is that it is very easy to end up in case 1 without noticing - others in this thread gave the example of using EntityManager in the lambda - that's a value from the outside, we need memory to store the reference!

    The solution here is what @Srokaaa suggested a few posts up - have a version of ForEach that passes a user defined context to the lambda, so we can put all our outside state in a struct and pass it by as an argument.
    .
    *The lambda is still allocated, but the compiler automagically stores it in a static field and reuses the same object every time.
     
    MostHated and NotaNaN like this.
  23. NotaNaN

    NotaNaN

    Joined:
    Dec 14, 2018
    Posts:
    325
    Ah, so it isn't "one is more efficient over the other" as it is instead "both are equally as efficient as each other when used properly -- but only Chunk-Iteration works for Jobs". I get it!

    Thanks for the detailed answer and description; it really cleared things up for me!

    (I still think Chunk-Iteration sounds cooler though...) :p
     
    Last edited: Feb 20, 2019
  24. Srokaaa

    Srokaaa

    Joined:
    Sep 18, 2018
    Posts:
    169
    Rider has a special Unity code inspections and warns you if you do this. I wouldn't be smart enough to notice it by myself :p
     
    MostHated and NotaNaN like this.
  25. MostHated

    MostHated

    Joined:
    Nov 29, 2015
    Posts:
    1,235
    xVergilx likes this.
  26. Guedez

    Guedez

    Joined:
    Jun 1, 2012
    Posts:
    827
    I updated the thread opening with what I believe to be the right answer. But there is still a question
    Does the filter group only ever serves as a filter now? Previously it would signal that the System requires X, Y and Z components or that it prohibits W component. Does it still works as it is there (Having only the subtractive type)? Or the group still have to have both the subtractive and the "additive" ComponentTypes?
     
  27. JPrzemieniecki

    JPrzemieniecki

    Joined:
    Feb 7, 2013
    Posts:
    33
    ComponentGroups are always filters (at least the way I look at them), and yes, you still need the additive components.
    ForEach defaults to constructing a group from the lambda argument types, the group you provide overrides that default.
    For Example, you can have a group with {X, Y, Subtractive<Z>} and do ForEach((ref X x) => {}, grp); to iterate over X components on entities that also have Y, but don't have Z.
     
  28. Micz84

    Micz84

    Joined:
    Jul 21, 2012
    Posts:
    447
    If you ask if group can prevent system from running if there is no entity matching group then yes you can registered group as required for systems to run.
     
  29. Guedez

    Guedez

    Joined:
    Jun 1, 2012
    Posts:
    827
    Updated the OP accordingly
     
  30. kork

    kork

    Joined:
    Jul 14, 2009
    Posts:
    280
    I wonder if there is a break/even point for this. E.g. if all you do in your system lets say create or modify a few entities, does spawning a job for this make things faster than just doing it on the main thread? Or asked differently, what is the overhead of spawning a job for everything?
     
    NotaNaN likes this.
  31. Rotary-Heart

    Rotary-Heart

    Joined:
    Dec 18, 2012
    Posts:
    813
    I compared the ForEach approach with what I've been using and it seems like using the ForEach for shared components is slower than mine. Which makes sense since it's iterating them over instead of just using the different ones. So here's how I'm doing it:
    Code (CSharp):
    1. ComponentGroup m_queryGroup;
    2.  
    3. protected override void OnCreateManager()
    4. {
    5.     m_queryGroup = GetComponentGroup(new EntityArchetypeQuery()
    6.     {
    7.         Any = System.Array.Empty<ComponentType>(),
    8.         None = System.Array.Empty<ComponentType>(),
    9.         All = new ComponentType[] { typeof(YourSharedComponent) }
    10.     });
    11. }
    12.  
    13. protected override void OnUpdate()
    14. {
    15.     ArchetypeChunkSharedComponentType<YourSharedComponent> SharedType = GetArchetypeChunkSharedComponentType<YourSharedComponent>();
    16.     int lastIndex = -1;
    17.  
    18.     using (NativeArray<ArchetypeChunk> chunks = m_queryGroup.CreateArchetypeChunkArray(Allocator.TempJob))
    19.     {
    20.         int chunkCount = chunks.Length;
    21.  
    22.         for (int iChunk = 0; iChunk < chunkCount; iChunk++)
    23.         {
    24.             ArchetypeChunk chunk = chunks[iChunk];
    25.             int sharedIndex = chunk.GetSharedComponentIndex(SharedType);
    26.             bool indexChanged = sharedIndex != lastIndex;
    27.  
    28.             if (indexChanged)
    29.             {
    30.                 lastIndex = sharedIndex;
    31.                 //Your code here
    32.                 YourSharedComponent sharedComponent = EntityManager.GetSharedComponentData<YourSharedComponent>(sharedIndex);
    33.             }
    34.         }
    35.     }
    36. }
    Of course it all depends on what you are doing inside your system.
     
  32. jwvanderbeck

    jwvanderbeck

    Joined:
    Dec 4, 2014
    Posts:
    825
    I'm still learning all this and trying to keep up with the changes, so I'm not sure how much value my feedback has, but I really feel like the API is getting uglier each time it gets revised in the name of performance. That ForEach with the embedded lambda makes me weep for the days of elegant code and gives me Javascript flashbacks.

    I get that the goal here is to provide more performant code by default and that to do so requires some sacrifice of elegant code, but after the necessary murders of my OOP children, which I understand, I'm trying to find at least in my own mind some sensible middle ground. I know everyone has a different opinion of "elegant" code but are we really going to be forced into these archaic ways of writing code even when not trying to squeeze every drop of blood out of the CPU?

    The code here https://github.com/Unity-Technologi...blob/master/Documentation~/component_group.md I was perfectly ok with. It was a necessary change that I understood but the code was still very standard C# and felt like code I would write anyway. The JobComponentSystem also looks ok, but I feel a little bit like my hands are tied on that because I don't get to build my ComponentGroup myself, but rather have to rely on generics which feels limiting and gets really ugly with complex combinations of components. Then you also seem to lose the ability to filter completely since you don't have a ComponentGroup.

    /rant
     
  33. Micz84

    Micz84

    Joined:
    Jul 21, 2012
    Posts:
    447

    But you can still use ComponentGroup if you want. You do not have to use ForEach.
     
  34. JPrzemieniecki

    JPrzemieniecki

    Joined:
    Feb 7, 2013
    Posts:
    33
    You can use component groups with JobComponentSystem, use job.ScheduleGroup(group, dependencies);

    Code (CSharp):
    1.  
    2. class GravitySystem : JobComponentSystem
    3. {
    4.     struct Job : IJobProcessComponentData<MoveVelocity>
    5.     {
    6.         public float DeltaTime;
    7.  
    8.         public void Execute(ref MoveVelocity v) => v.Value.y -= DeltaTime * 9.8f;
    9.     }
    10.     ComponentGroup _grp;
    11.  
    12.     protected override void OnCreateManager() => _grp = GetComponentGroup(typeof(MoveVelocity), typeof(UsesGravity));
    13.     protected override JobHandle OnUpdate(JobHandle inputDeps) => new Job()
    14.     {
    15.         DeltaTime = Time.fixedDeltaTime
    16.     }.ScheduleGroup(_grp, inputDeps);
    17. }
     
    Last edited: Feb 21, 2019
  35. jwvanderbeck

    jwvanderbeck

    Joined:
    Dec 4, 2014
    Posts:
    825
    That example is confusing to me. You are creating a ComponentGroup with components Position and UsesGravity. Then you are defining a JobSystem that uses the component MoveVelocity, and you aren't doing anything with the ComponentGroup at all.
     
  36. JPrzemieniecki

    JPrzemieniecki

    Joined:
    Feb 7, 2013
    Posts:
    33
    I use the group when scheduling the job (line 16). This way the job only executes for entites with both MoveVelocity and UseGravity.
     
  37. jwvanderbeck

    jwvanderbeck

    Joined:
    Dec 4, 2014
    Posts:
    825
    But those component aren't in the signature for the job itself. How would you access them in the job?
     
  38. JPrzemieniecki

    JPrzemieniecki

    Joined:
    Feb 7, 2013
    Posts:
    33
    I'm confused, MoveVelocity is used (line 8), and UseGravity is just a tag component in this example.
     
  39. MostHated

    MostHated

    Joined:
    Nov 29, 2015
    Posts:
    1,235
    Why not just struct Job : IJobProcessComponentData<MoveVelocity, Position, Gravity> instead?
     
  40. JPrzemieniecki

    JPrzemieniecki

    Joined:
    Feb 7, 2013
    Posts:
    33
    Uh, a typing error in the example, the group was supposed to include typeof(MoveVelocity), not typeof(Position). Edited. Apologies, @jwvanderbeck

    To show that you can use ComponentGroups with jobs.
     
  41. MostHated

    MostHated

    Joined:
    Nov 29, 2015
    Posts:
    1,235
    Oh, *derp*. That is my bad, I missed that part some how, lol.
     
  42. jwvanderbeck

    jwvanderbeck

    Joined:
    Dec 4, 2014
    Posts:
    825
    Ok then it makes a bit more sense, but since you are including the component in the job signature again I still don't see how the component group is actually being used. I could take your example and remove the ComponentGroup completely and it would still operate the same, since the component is being specified in the generic signature.

    What I am looking for is how to access the components specified from a ComponentGroup in the job WITHOUT specifying the Component query in the IJobProcessComponentData<> signature.
     
    Last edited: Feb 21, 2019
  43. jwvanderbeck

    jwvanderbeck

    Joined:
    Dec 4, 2014
    Posts:
    825
    I realize that I've derailed this thread with this line of inquiry so I apologize. I am going to start a new thread focused on the topic.
     
  44. timmehhhhhhh

    timmehhhhhhh

    Joined:
    Sep 10, 2013
    Posts:
    157
    i had no idea the ForEach existed until today, such a pleasant way to work with component systems!
     
    MostHated likes this.