Search Unity

Rewriting a system using chunks/queries instead of to-be-deprecated injected group structs

Discussion in 'Data Oriented Technology Stack' started by bac9-flcl, Oct 4, 2018.

  1. bac9-flcl

    bac9-flcl

    Joined:
    Dec 5, 2012
    Posts:
    779
    Since injected struct fields with component groups would be deprecated (according developer comments from "API usability" thread), I wanted to learn how to write systems without relying on them. To that end, I have three very simple systems handling removal, change or addition of instanced mesh renderer component based on tag components:

    https://www.hastebin.com/fiwadulayi.cs

    Rewriting these into a single query/chunk based system seemed like a good idea, especially since that was exactly what Unity developers did with the MeshInstanceRendererSystem they provided as a sample. So, following that example, I tried to rewrite the three systems above into one system accomplishing exact same thing with three queries.

    https://www.hastebin.com/qakujicido.cs

    I quickly encountered a few issues:

    • I'm not sure how to get entities from a chunk, but I assumed that chunk.GetNativeArray (entityType) is the correct way to do that.
    • I'm not sure how to get components from a chunk, but I assumed that chunk.GetNativeArray (componentType) is the correct way to do that.
    • I'm not sure if I understand the concept of chunks correctly - am I understanding right that to work with individual entities and components, I have to use two loop levels, one for chunk collection, and one for content of a chunk?
    • One of these assumptions here is wrong, or I'm making some other mistake in the code, because the system linked above fails to do anything (BlockRendererAddition components remain attached to entities forever with no apparent consequences). I'm not sure where the issue is, because there are no errors or warnings
    Getting past all these details, the biggest question I have is simple: is chunk/query use supposed to be so verbose, or am I doing something wrong? Comparing this system to 3 old injection based systems, the difference in verbosity is huge. I really dislike magical attributes and I'm all for a shift away from injected fields, but the three systems linked at the top of this post are vastly easier to maintain and less error prone than the query/chunk based system I link. I'm inclined to believe I'm just doing something wrong, incorrectly complicating the implementation, so I'd love to hear any advice on the right way of using chunks/queries.
     
    Last edited: Oct 4, 2018
    rigidbuddy likes this.
  2. julian-moschuering

    julian-moschuering

    Joined:
    Apr 15, 2014
    Posts:
    350
    Correct
    Correct
    Correct. ComponentDataArray and co basically do the same in the background.
    Some issues I see:
    * GetArchetypeChunkEntityType/GetArchetypeChunkComponentType has to be executed in OnUpdate as this call is responsible for dependency tracking.
    * There currently is nothing like Injection for this, you actually have to execute your queries:
    Code (CSharp):
    1. var chunks = EntityManager.CreateArchetypeChunkArray(query, Allocator.TempJob);
    Currently it is but I think they are working on it.
     
    TZ-, eizenhorn and rigidbuddy like this.
  3. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,515
    You not getting chunks in your code, you must use something like this:
    NativeArray<ArchetypeChunk> chunks = EntityManager.CreateArchetypeChunkArray(someEntityArchetypeQuery, someAllocatorType);

    as @julian-moschuering says
     
  4. bac9-flcl

    bac9-flcl

    Joined:
    Dec 5, 2012
    Posts:
    779
    @julian-moschuering @eizenhorn
    Thanks for quick replies! I have changed a few things based on your advice:
    • Archetype chunk entity/component types are no longer fetched at manager creation and are instead filled on every update
    • Chunks are retrieved in every update using CreateArchetypeChunkArray
    Additionally:
    • I'm not sure about this since I have little experience with native types, but I think the result of CreateArchetypeChunkArray has to be disposed, so I added a Dispose call for it at the end of each update
    • Looking at MeshInstanceRendererSystem, PostUpdateCommands is not used at all there - instead, a temporary EntityCommandBuffer is allocated, modified in the nested loop for each entity in a chunk, and disposed. I replicated the same
    Here is the updated code:
    https://www.hastebin.com/datenopeve.cs

    This updated version seems to work, but it triggers an avalanche of warnings which I'm unable to pinpoint. Each warning goes like this:

    Code (csharp):
    1. A Native Collection has not been disposed, resulting in a memory leak.
    2. It was allocated at C:\Users\username\AppData\Local\Unity\cache\packages\
    3. packages.unity.com\com.unity.entities@0.0.12-preview.15\
    4. Unity.Entities\Iterators\ArchetypeChunkArray.cs:280.
    This is not a very useful error message since it points at an internal ECS method as opposed to part of my code, but I suspect it has something to do with either a native array I get from CreateArchetypeChunkArray call or an EntityCommandBuffer. I dispose both, though, so I don't know what's causing this error.
     
  5. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,515
    upload_2018-10-4_15-4-2.png
     
  6. bac9-flcl

    bac9-flcl

    Joined:
    Dec 5, 2012
    Posts:
    779
    Yeah, that's what I do, unless I'm misunderstanding something. Check each Process* method:

    Code (csharp):
    1. entityCommandBuffer.Dispose ();
    2. chunks.Dispose ();
     
  7. julian-moschuering

    julian-moschuering

    Joined:
    Apr 15, 2014
    Posts:
    350
    Empty arrays have to be disposed too. You only dispose if length > 0.
     
    eizenhorn likes this.
  8. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,515
    Ah you fastest than me :p

    @bac9-flcl yes you allocate NativeArray<ArchetypeChunk> but not dispose if length == 0.
     
    Last edited: Oct 4, 2018
    julian-moschuering likes this.
  9. bac9-flcl

    bac9-flcl

    Joined:
    Dec 5, 2012
    Posts:
    779
    Oh dang, that was such a rookie mistake! :D

    Thanks for pointing this out, disposing the chunk array in a case where I bail out at length == 0 solved the error. Final version (I hope?): https://www.hastebin.com/opajurohil.cs
     
  10. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,515
    Also I not see where you dispose "entities" and "additionComponents" (and other in other
    Process*)

    upload_2018-10-4_15-19-17.png
     
  11. julian-moschuering

    julian-moschuering

    Joined:
    Apr 15, 2014
    Posts:
    350
    Those actually don't have and don't should be disposed. Only the Top chunk array must be disposed. GetNativeArray returns a wrapper around the actual chunk data.
     
    eizenhorn likes this.
  12. bac9-flcl

    bac9-flcl

    Joined:
    Dec 5, 2012
    Posts:
    779
    So, the correct steps for a basic system meant to do component additions/removals/changes are these, right?:
    • Get native array with chunks from a query
    • Allocate a new entity command buffer
    • For each chunk
      • Get a native array with entities from the chunk array based on chunk index
      • Get native arrays with relevant components from the current chunk
      • For each entity
        • Select entity and components from native arrays based on entity index
        • Call removal/addition/setting methods on entity command buffer using entity and components
    • Execute entity command buffer
    • Dispose entity command buffer
    • Dispose the chunks array
    Edit: Removed disposal of entity and component arrays as @julian-moschuering suggested.
     
  13. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,515
    Yes you are right, I confused this piece with a manually allocated values array from the documentation which is then passed to the job :) My bad :)
     
  14. bac9-flcl

    bac9-flcl

    Joined:
    Dec 5, 2012
    Posts:
    779
    Could you elaborate a bit more on how this "wrapper" works and can be identified? Both ArchetypeChunk.GetNativeArray and EntityManager.CreateArchetypeChunkArray return exact same type (native array) - what makes one require disposal and another not require it? I mean, now I know what to dispose in this particular case, but I don't know how to identify the difference in the future across the rest of the ECS API.

    One guess I have is that EntityManager.CreateArchetypeChunkArray creates an entirely new native array for you, which makes disposal your responsibility, while ArchetypeChunk.GetNativeArray gives you an existing array that was created without your input and is disposed by the chunk holding it when the time comes. So, no type difference or some internal state difference on the native array itself, just the context matters.
     
  15. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,515
    Because they not allocated and reuse allocated memory in chunk. (ArchetypeChunk.GetNativeArray)
    upload_2018-10-4_16-13-26.png

    and EntityManager.CreateArchetypeChunkArray allocates memory (wich reused in ArchetypeChunk.GetNativeArray above)

    upload_2018-10-4_16-14-59.png
     
    julian-moschuering likes this.
  16. julian-moschuering

    julian-moschuering

    Joined:
    Apr 15, 2014
    Posts:
    350
    There currently is no way to see it without looking at the source but you can't really do it incorrectly.
    Not disposing the ChunkArray will throw an exception and disposing the array returned by GetNativeArray will throw one also (The NativeArray can not be Disposed because it was not allocated with a valid allocator.)
     
  17. bac9-flcl

    bac9-flcl

    Joined:
    Dec 5, 2012
    Posts:
    779
    Good to know, thanks!
     
  18. davenirline

    davenirline

    Joined:
    Jul 7, 2010
    Posts:
    497
    I'm sad that injected structs are going to be deprecated. I have build a system around it.
     
  19. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,515
    Just move to Chunk Iteration, it's not hard and clean, and you can write your own [Inject] in future :)
     
  20. noio

    noio

    Joined:
    Dec 17, 2013
    Posts:
    100
    I'm confident that the engineers at Unity are striving for a system that will as simple to use as Injection, but maybe with less magic involved. One example was explored by @unity_WMhs-DVznwl60Q in the API Usability topic, where the required components are specified to a generic class, and then just the inner part of the iteration is exposed to the user. (Similar to IJobProcessComponentData)

    I think you could just as well stick with [Inject] until the new API design comes along.

    I don't think the plan is to have users deal with Chunk iteration in the 'final' API, it feels like one too many thing to iterate over that the user does not necessarily need to know/care about.
     
    melian likes this.
  21. julian-moschuering

    julian-moschuering

    Joined:
    Apr 15, 2014
    Posts:
    350
    Chunk iteration is still very important for optimization. As soon as you have SCD's we need a way to do per Chunk operations once which IJobProcessComponentData does not offer yet.
     
    noio likes this.
  22. noio

    noio

    Joined:
    Dec 17, 2013
    Posts:
    100
    That's just a matter of API, right?

    As a user I just want to do stuff with "all components with a certain SharedComponentData". As long as the API is able to let me iterate over those directly, then I don't really care which Chunks they are in. Or am I missing something? (Apart from the fact that this obviously is not possible yet)

    In other words: is there ever a use case for fiddling with Chunks directly? Would I need something like "Entities with Component A and B, but in a chunk that also has Component C", then I would just require "A + B + C" and not worry about Chunks?
     
  23. Adam-Mechtley

    Adam-Mechtley

    Unity Technologies

    Joined:
    Feb 5, 2007
    Posts:
    191
    Don't worry. There is no plan to remove the low-level API. It should always be available. The plan is only to adjust what the high-level API looks like for cases where users "just want to do stuff with "all components with a certain SharedComponentData"", as @noio put it
     
    FROS7, wobes, noio and 2 others like this.
  24. julian-moschuering

    julian-moschuering

    Joined:
    Apr 15, 2014
    Posts:
    350
    Something like this:

    foreach chunk
    {
    do some preprocessing/conversion/lookups with SCD
    foreach instance in chunk
    {
    process instance using preprocessed data
    }
    }


    Also using Chunk iteration some stuff is actually less verbose. Eg having slightly different processing based on component existance can be done with per chunk branching. Without chunk iteration this would require additional groups with different Jobs/Functions as a per instance branch isn't feasible.
     
    eizenhorn and noio like this.
  25. noio

    noio

    Joined:
    Dec 17, 2013
    Posts:
    100
    Good point! I hadn't thought of it that way. When you want to do stuff dynamically depending on component existence or SharedComponent values then Chunks of course become the appropriate level of abstraction.
     
  26. bac9-flcl

    bac9-flcl

    Joined:
    Dec 5, 2012
    Posts:
    779
    I thought the current state of my system was stable, but apparently I'm still making a mistake somewhere. Whenever I'm processing new additions, I get a set of errors about native array deallocation/disposal that don't make a whole lot of sense:

    https://www.hastebin.com/iluhofulez.rb

    This doesn't happen the first time I set up the level (when up to 20000 entities receive instanced model components through that system), but whenever I punch a hole in the level and about 4-8 new model components have to be added, I immediately get that error. Here is the full code of the system, with the offending (according to error messages) line being line 150:

    https://www.hastebin.com/irocebamug.cs

    The line does this:

    Code (csharp):
    1. var entities = chunk.GetNativeArray (entityType);
    Not sure how it causes an error in a small task later in the game while not causing an error in the first few frames when thousands of entities are processed, and not sure why it encounters issues with deallocation at all - I thought arrays fetched through GetNativeArray are supposed to be automatically disposed.
     
  27. julian-moschuering

    julian-moschuering

    Joined:
    Apr 15, 2014
    Posts:
    350
    I think this error is thrown when entityType was not recreated this frame, eg when it is fetched in
    OnCreateManager or maybe it is not fetched under certain circumstances in your code.
     
  28. bac9-flcl

    bac9-flcl

    Joined:
    Dec 5, 2012
    Posts:
    779
    But I do fetch the type every update:

    Code (csharp):
    1. protected override void OnUpdate ()
    2. {
    3.     entityType = GetArchetypeChunkEntityType ();
    4.     removalChunkType = GetArchetypeChunkComponentType<BlockRendererRemoval> ();
    5.     changeChunkType = GetArchetypeChunkComponentType<BlockRendererChange> ();
    6.     additionChunkType = GetArchetypeChunkComponentType<BlockRendererAddition> ();
    7.  
    8.     ProcessRemovals ();
    9.     ProcessChanges ();
    10.     ProcessAdditions ();
    11. }
     
  29. julian-moschuering

    julian-moschuering

    Joined:
    Apr 15, 2014
    Posts:
    350
    Sorry, didn't see the code.

    EntityCommandBuffer.Playback might invalidate your handles when doing structural changes. I guess it should work when refetching the types after ProcessRemovals and ProcessChanges. It would be better to do all structural changes in a single place thought eg by using a single EntityCommandBuffer or by using a BarrierSystem. This will make later jobification easier too.
     
  30. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,515
    Try create ONE ECB before all loops in OnUpdate and playback it after ProcessAdditions for test. It works first time, because you not have changes or removals which call playback and invalidate arrays.
    upload_2018-10-8_12-58-53.png
     
  31. bac9-flcl

    bac9-flcl

    Joined:
    Dec 5, 2012
    Posts:
    779
    Oh wow, that's a great catch! The level damage changes meshes on some objects and adds meshes to some points which previously had no meshes, in contrast with initial level setup that only caused additions to happen, so it makes total sense that addition would encounter invalidated handles.
     
  32. illinar

    illinar

    Joined:
    Apr 6, 2011
    Posts:
    597
    Hmm.. I got used to injection now. In 90% of cases so far in small systems, I just use one line generic injections.

    Can anyone please give me an example of this system with the new API:

    Code (CSharp):
    1. public class ApplyTransformToGameObjectSystem : ComponentSystem
    2. {
    3.     [Inject] ReadonlyComponentInjection<Transform, LocalToWorld, KeepGOTransformUpdated> _data;
    4.  
    5.     protected override void OnUpdate()
    6.     {
    7.         for (int i = 0; i < _data.Length; i++)
    8.         {
    9.             var transform = _data.Components[i];
    10.             var matrix = _data.Data1[i].Value;
    11.             transform.localPosition = new Vector3(matrix.c3.x, matrix.c3.y, matrix.c3.z);
    12.         }
    13.     }
    14. }
    I hope it's not gonna add like 5-7 more lines...
     
  33. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,515
    Read https://github.com/Unity-Technologi...ster/Documentation/content/chunk_iteration.md
     
    illinar likes this.
  34. bac9-flcl

    bac9-flcl

    Joined:
    Dec 5, 2012
    Posts:
    779
    It would require more lines, yeah. :) I'm not familiar with what ReadonlyComponentInjection is and I have no experience with influencing MonoBehaviours from systems so I'm going to use a Position data component instead of Transform, and the system would probably go like this:

    Code (CSharp):
    1. public class ApplyTransformToGameObjectSystem : ComponentSystem
    2. {
    3.     EntityArchetypeQuery query;
    4.  
    5.     protected override void OnCreateManager ()
    6.     {
    7.         query = new EntityArchetypeQuery
    8.         {
    9.             Any = Array.Empty<ComponentType> (),
    10.             None = Array.Empty<ComponentType> (),
    11.             All = new ComponentType[] { typeof (Position), typeof (LocalToWorld), typeof (KeepGOTransformUpdated) }
    12.         };
    13.     }
    14.  
    15.     protected override void OnUpdate ()
    16.     {
    17.         var chunks = EntityManager.CreateArchetypeChunkArray (query, Allocator.TempJob);
    18.         int chunkLength = chunks.Length;
    19.  
    20.         if (chunkLength == 0)
    21.         {
    22.             chunks.Dispose ();
    23.             return;
    24.         }
    25.  
    26.         var entityCommandBuffer = new EntityCommandBuffer (Allocator.Temp);
    27.         var entityChunkType = GetArchetypeChunkEntityType ();
    28.         var matrixChunkType = GetArchetypeChunkComponentType<LocalToWorld> ();
    29.  
    30.         for (int i = 0; i < chunkLength; i++)
    31.         {
    32.             var chunk = chunks[i];
    33.             var entities = chunk.GetNativeArray (entityChunkType);
    34.             var matrices = chunk.GetNativeArray (matrixChunkType);
    35.  
    36.             int entitiesLength = entities.Length;
    37.             for (int e = 0; e < entitiesLength; e++)
    38.             {
    39.                 var entity = entities[e];
    40.                 var matrix = matrices[e].Value;
    41.                 entityCommandBuffer.SetComponent (entity, new Position { Value = new float3 (matrix.c3.x, matrix.c3.y, matrix.c3.z) });
    42.             }
    43.         }
    44.  
    45.         chunks.Dispose ();
    46.         entityCommandBuffer.Playback (EntityManager);
    47.         entityCommandBuffer.Dispose ();
    48.     }
    49. }
    It's a more verbose way of writing systems, but it has its benefits and I like how there is less hidden magic going on. As others have said, it might not be worth it to rewrite everything to use chunks, since a similarly simple replacement for injection would probably be in by the point injection is phased out. I'm mainly interested in learning it to understand more about ECS and to have another tool at my disposal.
     
    Last edited: Oct 9, 2018
    llMarty, Creepgin, wobes and 4 others like this.
  35. illinar

    illinar

    Joined:
    Apr 6, 2011
    Posts:
    597
    You gotta be #$%* kidding me! :D

    49 lines vs 14... Well okay, I'm kinda cheating there the ReadonlyComponentInjection is one of the custom generic classes I use for injection.

    Code (CSharp):
    1. public struct ReadonlyComponentInjection<Component, ComponentData1, ComponentData2> where Component : UnityEngine.Component where ComponentData1 : struct, IComponentData where ComponentData2 : struct, IComponentData
    2. {
    3.     public readonly ComponentArray<Component> Components;
    4.     public readonly ComponentDataArray<ComponentData1> Data1;
    5.     public readonly ComponentDataArray<ComponentData2> Data2;
    6.     public readonly EntityArray Entities;
    7.     public readonly int Length;
    8.     public static implicit operator bool(ReadonlyComponentInjection<Component, ComponentData1, ComponentData2> a) => a.Length > 0;
    9. }
    Still, that would be 20 lines vs 45-49.

    Btw thanks a lot for the example and the link, guys.
     
    Last edited: Oct 9, 2018
  36. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    4,715
    For the above example IJobProcessComponentData gets exact same performance and that is definitely here to stay.

    Chunk based processing is useful when performing operations where you can cache things on a per chunk basis or you want custom per chunk filtering.

    But its definately useful to learn chunk based API because that makes it very clear what is actaully happening in higher level scaffolding like IJobProcessComponentData.
     
    illinar likes this.
  37. davenirline

    davenirline

    Joined:
    Jul 7, 2010
    Posts:
    497
    I'm in the process of transforming my systems to chunk iteration. I use jobs a lot. When is the best time to dispose the chunk returned by EntityManager.CreateArchetypeChunkArray?

    Code (CSharp):
    1. protected override JobHandle OnUpdate(JobHandle inputDeps) {
    2.     this.chunks = this.EntityManager.CreateArchetypeChunkArray(this.query, Allocator.TempJob);
    3.  
    4.     Job job = new Job() {
    5.         Chunks = this.chunks
    6.     };
    7.  
    8.     // Should I dispose chunks here?
    9.     // I'm not sure because the job hasn't been scheduled yet
    10.  
    11.     return job.Schedule(this.chunks.Length, 64, inputDeps);
    12. }
    Edit: I got it. I used the DeallocateOnJobCompletion attribute on the job struct.
     
    Last edited: Oct 14, 2018
  38. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    4,715
    Using this attribute on the Chunks field in the job:
    [DeallocateOnJobCompletion]
     
    eizenhorn likes this.
  39. davenirline

    davenirline

    Joined:
    Jul 7, 2010
    Posts:
    497
    I have a case where I chain jobs like this:

    Code (csharp):
    1. protected override JobHandle OnUpdate(JobHandle inputDeps) {
    2.     JobHandle lastHandle = inputDeps;
    3.  
    4.     for(...) {
    5.         Job job = new Job() { ... };
    6.  
    7.         JobHandle currentHandle = job.Schedule(..., lastHandle);
    8.         lastHandle = currentHandle;
    9.     }
    10.  
    11.     return lastHandle;
    12. }
    My problem is how do I cache a NativeArray<ArchetypeChunk> such that it can be set on all jobs but deallocate at the last one. I could create another job struct that uses [DeallocateOnJobCompletion] and use this on the last one but this creates duplicate code.

    I tried creating the chunks inside the loop. It means that it's being deallocated per job. It ran very slow. I suppose I should cache the chunk, but how? ComponentDataArray does not have this problem.
     
  40. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    4,715
    An empty additional job dependent is the official pattern in this case.

    Do note, we have upcoming ArchetypeChunk API's that avoid the allocation... It will be a few weeks, so for now this pattern is fine.
     
    eizenhorn likes this.
  41. davenirline

    davenirline

    Joined:
    Jul 7, 2010
    Posts:
    497
    Oh right. That's clever. However, the chunk iteration version is still very slow compared to the ComponentDataArray version. This is the chunk iteration version:

    Code (CSharp):
    1.     public class UpdateVerticesSystem : JobComponentSystem {
    2.      
    3.         private readonly List<SpriteManager> managers = new List<SpriteManager>(1);
    4.         private readonly List<int> managerIndeces = new List<int>(1);
    5.  
    6.         private EntityArchetypeQuery query;
    7.      
    8.         [ReadOnly]
    9.         private ArchetypeChunkComponentType<Sprite> spriteType;
    10.  
    11.         [BurstCompile]
    12.         private struct Job : IJobParallelFor {
    13.             [ReadOnly]
    14.             public ArchetypeChunkComponentType<Sprite> spriteType;
    15.          
    16.             [ReadOnly]
    17.             public NativeArray<ArchetypeChunk> chunks;
    18.  
    19.             [ReadOnly]
    20.             public Entity spriteManagerEntity;
    21.          
    22.             [NativeDisableParallelForRestriction]
    23.             public NativeArray<Vector3> vertices;
    24.          
    25.             [NativeDisableParallelForRestriction]
    26.             public NativeArray<Vector2> uv;
    27.          
    28.             [NativeDisableParallelForRestriction]
    29.             public NativeArray<Vector2> uv2;
    30.          
    31.             [NativeDisableParallelForRestriction]
    32.             public NativeArray<Color> colors;
    33.  
    34.             public void Execute(int index) {
    35.                 for (int i = 0; i < this.chunks.Length; ++i) {
    36.                     ArchetypeChunk chunk = this.chunks[i];
    37.                     Process(chunk);
    38.                 }
    39.             }
    40.  
    41.             private void Process(ArchetypeChunk chunk) {
    42.                 NativeArray<Sprite> sprites = chunk.GetNativeArray(this.spriteType);
    43.                 for (int i = 0; i < chunk.Count; ++i) {
    44.                     Sprite sprite = sprites[i];
    45.                     Process(sprite);
    46.                 }
    47.             }
    48.  
    49.             private void Process(Sprite sprite) {
    50.                 if (sprite.spriteManagerEntity == Entity.Null) {
    51.                     // No manager specified. Maybe not yet added to one.
    52.                     return;
    53.                 }
    54.  
    55.                 if (sprite.spriteManagerEntity != this.spriteManagerEntity) {
    56.                     // Not the same sprite manager
    57.                     return;
    58.                 }
    59.              
    60.                 this.vertices[sprite.index1] = sprite.transformedV1;
    61.                 this.vertices[sprite.index2] = sprite.transformedV2;
    62.                 this.vertices[sprite.index3] = sprite.transformedV3;
    63.                 this.vertices[sprite.index4] = sprite.transformedV4;
    64.  
    65.                 this.uv[sprite.index1] = sprite.uv_1;
    66.                 this.uv[sprite.index2] = sprite.uv_2;
    67.                 this.uv[sprite.index3] = sprite.uv_3;
    68.                 this.uv[sprite.index4] = sprite.uv_4;
    69.  
    70.                 this.uv2[sprite.index1] = sprite.uv2_1;
    71.                 this.uv2[sprite.index2] = sprite.uv2_2;
    72.                 this.uv2[sprite.index3] = sprite.uv2_3;
    73.                 this.uv2[sprite.index4] = sprite.uv2_4;
    74.  
    75.                 this.colors[sprite.index1] = sprite.color;
    76.                 this.colors[sprite.index2] = sprite.color;
    77.                 this.colors[sprite.index3] = sprite.color;
    78.                 this.colors[sprite.index4] = sprite.color;
    79.             }
    80.         }
    81.  
    82.         [BurstCompile]
    83.         private struct DeallocateJob : IJob {
    84.             [ReadOnly]
    85.             [DeallocateOnJobCompletion]
    86.             public NativeArray<ArchetypeChunk> chunks;
    87.  
    88.             public void Execute() {
    89.                 // Does nothing. It just deallocates the chunks.
    90.             }
    91.         }
    92.  
    93.         protected override void OnCreateManager() {
    94.             // All entities with Sprite
    95.             this.query = new EntityArchetypeQuery() {
    96.                 Any = Array.Empty<ComponentType>(),
    97.                 None = Array.Empty<ComponentType>(),
    98.                 All = new ComponentType[] {
    99.                     typeof(Sprite)
    100.                 }
    101.             };
    102.         }
    103.      
    104.         protected override JobHandle OnUpdate(JobHandle inputDeps) {
    105.             this.managers.Clear();
    106.             this.managerIndeces.Clear();
    107.             this.EntityManager.GetAllUniqueSharedComponentData(this.managers, this.managerIndeces);
    108.  
    109.             this.spriteType = GetArchetypeChunkComponentType<Sprite>();
    110.          
    111.             JobHandle lastHandle = inputDeps;
    112.          
    113.             NativeArray<ArchetypeChunk> chunks =
    114.                 this.EntityManager.CreateArchetypeChunkArray(this.query, Allocator.TempJob);
    115.          
    116.             // Note here that we start counting from 1 since the first entry is always a default one
    117.             // In this case, SpriteManager.internal has not been allocated. So we get a NullPointerException
    118.             // if we try to access the default entry at 0.
    119.             for (int i = 1; i < this.managers.Count; ++i) {
    120.                 SpriteManager manager = this.managers[i];
    121.  
    122.                 if (manager.Owner == Entity.Null) {
    123.                     // Note that the owner of a SpriteManager might be set in SetOwnerToSpriteManagerSystem
    124.                     // If it's null, then it hasn't been assigned yet
    125.                     continue;
    126.                 }
    127.  
    128.                 if (!(manager.AlwaysUpdateMesh || manager.VerticesChanged || manager.UvChanged || manager.ColorsChanged)) {
    129.                     // Nothing changed. No need to process.
    130.                     continue;
    131.                 }
    132.  
    133.                 Job job = new Job() {
    134.                     spriteType = this.spriteType,
    135.                     chunks = chunks,
    136.                     spriteManagerEntity = manager.Owner,
    137.                     vertices = manager.NativeVertices,
    138.                     uv = manager.NativeUv,
    139.                     uv2 = manager.NativeUv2,
    140.                     colors = manager.NativeColors
    141.                 };
    142.  
    143.                 JobHandle currentHandle = job.Schedule(chunks.Length, 64, lastHandle);
    144.                 lastHandle = currentHandle;
    145.             }
    146.  
    147.             DeallocateJob deallocateJob = new DeallocateJob() {
    148.                 chunks = chunks
    149.             };
    150.             JobHandle deallocateHandle = deallocateJob.Schedule(lastHandle);
    151.             lastHandle = deallocateHandle;
    152.          
    153.             return lastHandle;
    154.         }
    155.  
    156.     }
    This is the old ComponentDataArray version (the faster one):
    Code (CSharp):
    1. public class UpdateVerticesSystem : JobComponentSystem {
    2.  
    3.         private struct SpriteData {
    4.             public readonly int Length;
    5.  
    6.             [ReadOnly]
    7.             public ComponentDataArray<Sprite> Sprite;
    8.         }
    9.  
    10.         [Inject]
    11.         private SpriteData spriteData;
    12.  
    13.         private struct ManagerData {
    14.             public readonly int Length;
    15.  
    16.             [ReadOnly]
    17.             public EntityArray Entity;
    18.  
    19.             [ReadOnly]
    20.             public SharedComponentDataArray<SpriteManager> SpriteManager;
    21.         }
    22.  
    23.         [Inject]
    24.         private ManagerData managerData;
    25.  
    26.         [BurstCompile]
    27.         private struct Job : IJobParallelFor {
    28.             [ReadOnly]
    29.             public SpriteData spriteData;
    30.  
    31.             public Entity spriteManagerEntity;
    32.            
    33.             [NativeDisableParallelForRestriction]
    34.             public NativeArray<Vector3> vertices;
    35.            
    36.             [NativeDisableParallelForRestriction]
    37.             public NativeArray<Vector2> uv;
    38.            
    39.             [NativeDisableParallelForRestriction]
    40.             public NativeArray<Vector2> uv2;
    41.            
    42.             [NativeDisableParallelForRestriction]
    43.             public NativeArray<Color> colors;
    44.  
    45.             public void Execute(int index) {
    46.                 Sprite sprite = this.spriteData.Sprite[index];
    47.  
    48.                 if (sprite.spriteManagerEntity == Entity.Null) {
    49.                     // No manager specified. Maybe not yet added to one.
    50.                     return;
    51.                 }
    52.  
    53.                 if (sprite.spriteManagerEntity != this.spriteManagerEntity) {
    54.                     // Not the same sprite manager
    55.                     return;
    56.                 }
    57.                
    58.                 this.vertices[sprite.index1] = sprite.transformedV1;
    59.                 this.vertices[sprite.index2] = sprite.transformedV2;
    60.                 this.vertices[sprite.index3] = sprite.transformedV3;
    61.                 this.vertices[sprite.index4] = sprite.transformedV4;
    62.  
    63.                 this.uv[sprite.index1] = sprite.uv_1;
    64.                 this.uv[sprite.index2] = sprite.uv_2;
    65.                 this.uv[sprite.index3] = sprite.uv_3;
    66.                 this.uv[sprite.index4] = sprite.uv_4;
    67.  
    68.                 this.uv2[sprite.index1] = sprite.uv2_1;
    69.                 this.uv2[sprite.index2] = sprite.uv2_2;
    70.                 this.uv2[sprite.index3] = sprite.uv2_3;
    71.                 this.uv2[sprite.index4] = sprite.uv2_4;
    72.  
    73.                 this.colors[sprite.index1] = sprite.color;
    74.                 this.colors[sprite.index2] = sprite.color;
    75.                 this.colors[sprite.index3] = sprite.color;
    76.                 this.colors[sprite.index4] = sprite.color;
    77.             }
    78.         }
    79.        
    80.         protected override JobHandle OnUpdate(JobHandle inputDeps) {
    81.             JobHandle lastHandle = inputDeps;
    82.             for (int i = 0; i < this.managerData.Length; ++i) {
    83.                 SpriteManager manager = this.managerData.SpriteManager[i];
    84.  
    85.                 if (!(manager.AlwaysUpdateMesh || manager.VerticesChanged || manager.UvChanged || manager.ColorsChanged)) {
    86.                     // Nothing changed. No need to process.
    87.                     continue;
    88.                 }
    89.                
    90.                 Entity managerEntity = this.managerData.Entity[i];
    91.  
    92.                 Job job = new Job() {
    93.                     spriteData = this.spriteData,
    94.                     spriteManagerEntity = managerEntity,
    95.                     vertices = manager.NativeVertices,
    96.                     uv = manager.NativeUv,
    97.                     uv2 = manager.NativeUv2,
    98.                     colors = manager.NativeColors
    99.                 };
    100.  
    101.                 JobHandle currentHandle = job.Schedule(this.spriteData.Length, 64, lastHandle);
    102.                 lastHandle = currentHandle;
    103.             }
    104.            
    105.             return lastHandle;
    106.         }
    107.  
    108.     }
    I can't find where I did wrong. In the profiler, it says UpdateVerticesSystem:Job takes around 15-20ms in worker threads. The main thread waits for these to finish.
     
  42. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    4,715
    You want to use the ComponentGroup based API to get the ArchetypeChunk Array instead... Thats much faster, the other one will get removed once all internal restructuring related to it is done.
     
  43. pcysl5edgo

    pcysl5edgo

    Joined:
    Jun 3, 2018
    Posts:
    45
    Code (CSharp):
    1. public void Execute(int index) {
    2.   for (int i = 0; i < this.chunks.Length; ++i) {
    3.     ArchetypeChunk chunk = this.chunks[i];
    4.     Process(chunk);
    5.   }
    6. }
    Why don't you use parameter 'index' in IJobParallelFor job?
     
  44. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    4,715
    Yeah remove the for loops and use chunks[index];

    You are essentially executing all the works multiple times...
     
    Zoey_O likes this.
  45. davenirline

    davenirline

    Joined:
    Jul 7, 2010
    Posts:
    497
    Ah! Okay, that was stupid.
     
  46. davenirline

    davenirline

    Joined:
    Jul 7, 2010
    Posts:
    497
  47. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,515
  48. davenirline

    davenirline

    Joined:
    Jul 7, 2010
    Posts:
    497
    Met another obstacle. How do I use TransformAccessArray now? I can get it with Inject but I have no idea how to get it when using chunk iteration.
     
  49. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,515
    You can get TransformAccessArray from ComponentGroup.
    group.GetTransformAccessArray();
     
  50. davenirline

    davenirline

    Joined:
    Jul 7, 2010
    Posts:
    497
    Cool! Thanks!