200k dynamic animated sprites at 80fps

That is what I'm doing - the only per instance data to display the sprite is a single index. I'm animating by just changing that index and the shader will automatically update the sprite.

In most cases one type of data will probably change nearly every frame - whether it's the uv cell or the position or the color. But right now if any one thing changes all the buffers need to update, if I split it out properly then they should be able to update independently without affecting each-other.

Aside from that I'm now trying to get rid of more unnecessary copies. Right now when I'm moving the data from components to dynamic buffers I do EntityQuery.ToComponentDataArray. This gives me a nice list I can push into the buffers. I'd like to use an IJobForEach or an IJobChunk instead using the filtered query, but Dynamic Buffers don't support parallel writing. I've tried using a queue to read and write the data but the copy has to be run once for every shared component data, and trying to manage queues with dynamic filtering based on SCD seems like a nightmare.

I suspect this is a pretty common problem, I'm pretty new to parallel coding so my brain hurts trying to solve this stuff. I'm thinking a NativeMultiHashMap<SCDFilterValue,DataType> might work, not sure yet.

 

Thanks for giving such a good explanation, that really helps me out. I had started poking around with ECS but wasn't able to understand much of it at the time but I think I'm going to spend a good bit of this weekend trying to go through your code line by line and understand what's going on exactly.

 

NMHM is not good if you have big data sets in my experience - it takes time to clear and also accessing the data is not that fast (for what you want to use it for)

I can take a look once you are a step further or this whole thing is integrated somewhere (we have now 2 parallel developments, right?). Do you need dynamic buffers or can you use nativearrays straight away (maintain them in a list or so). You can simple .ScheduleSingle to write to the buffer, if you have a few .ScheduleSingle that you can schedule in parallel (i.e. for filtered SCD) that's better as you dont have to deal with .concurrent but you still fill up all worker threads.

Have you looked into the renderer Joachim posted recently? I did not have a chance to check it out, but one can assume that this is pretty optimized, maybe some ideas there...

 

You're definitely right about NMHM, it slowed everything way down. For now I reverted to just Copying and cleaned everything up a bit.

I completely forgot about ScheduleSingle since I've never had to use it before - I think that's exactly what I need! I will try that. And I didn't even know about the GPUAnimation thing, I will definitely give it a look, thanks.

Edit: Switching to using IJobForEachWithEntity and ScheduleSingle helped a ton. It simplified the code a lot and it runs faster without having to do unecessary array copies. Now up to ~60fps with 350k entities in the editor.

 

Yes that is the bottleneck. To overcome that a double buffer of the components must happen but that can be also an issue. I would try to get a hash of the data and double buffer it based on that.

 

I had a quick look at @Sarkahn repro, because it seems to have been updated last.
1- burst the animation job
2- do you have to double buffer the ComputeBuffers, do you have to clean them? Can you not simply overwrite them and only renew / resize them if the length (i.e. entities) change? this would allow you to use changefilter and only update if components have really changed (i mentioned this before, i.e. assume you have individual color but it does not change every frame, no need to copy anything, same applies to matrix - here you can separate all combinations of pos, rot, scale, etc.)
3- I have to download this to better understand what is going on, but do you need in the generaterenderbuffersystem, the else branch? or do you only need to do this if the count != bufferlength
4- i find the base class harder to read than just having individual / independent systems (this is a style point only + maybe also because i read on mobile)
5- various small optimizations, i.e. why removeat(0) and not simply for (int i = 1;....) --- but it looks like this is now soon down to profiling and micro optimizations

congrats! I cant wait to download and replace my old system with this one

 

I just downloaded this and had a real peek --- seems complicated or I did not understand something.

1- simplest way would be to use .ToComponentDataArray of all the attributes and just keep the SpriteRenderSystem
2- if one assumes, not all the attributes change each frame, instead of .ToComponentDataArray, use IJFEWE with ChangeFilter and copy the values directly into a permanent NativeArray (edit: this can be in parallel, disableparallelforrestriction)
3- this would require a .Complete call in the SpriteRenderSystem or separating the systems to beginning of frame and end of frame and passing a reference

DrawMeshInstancedIndirect looks wonderful, no more 1023 restriction...

I might create a version of this next weekend, but I think, I would use the Unity Transform system (even if it is 3D) to use their parenting.... @Joachim_Ante it would really be nice if we could add a "Subtractive Tag" for the HybridRenderer so those systems can coexist.

 

I'm poking at it right now, you may be right about just ditching the dynamic buffers altogether. But if we're dealing with potentially hundreds of thousands of entities, most of which will realistically be changing in some way each frame, that's a LOT of allocations. Probably way too much but maybe I'm missing something.

I'm working on caching the compute buffers but keeping them in sync with the dynamic buffers and the shared component data is pretty nontrivial.

As of right now though the biggest bottleneck seems to be "Gfx.ProcessCommands":

I'm not familiar enough with how this all works to know exactly what that means. Is it from doing buffer.SetData? material.SetBuffer? Is it just the state of the buffers themselves causing it, meaning I would have to compress the data that's being sent to the gpu?

 

The buffers are fine, but in this particular case maybe not required (they overcome the coupling I suggested in point 3 above, i.e. a reference between systems). But there is no difference memory wise. But check if you really need to hold on to a copy. Also, double check if you can write to them in parallel (the way you write to them, it could work with disableparallelforrestriction, if you were to use a NativeArray, you could do it)

Edit: You would allocate the NativeArray permanent and only resize it, if the entity count changes.

EDIT: It's late here... not sure if all I said above is correct....I have to test it myself...I hope I find time next weekend, it's a nice litte project....I will watch were you get to....and maybe all the work is already done....

 

Alright I've refactored the renderer and now I'm caching the compute buffers. It's now way simpler, easier to add new buffer types and I got a nice performance boost. It can now do ~60fps with 400k entities rendering in the editor. The GPU continues to be the bottleneck, I think the only way to continue scaling it is to only write when things changed.

I'm trying to do that but not having much luck. I thought after reading the EntityQuery manual that it would work like this:

I've verified that I don't access my SpriteSheetColor as ReadWrite anywhere. But this query always returns entities. Does filtering on SharedComponentData overwrite the SetFilterChanged that I did when I created it? If so how are you supposed to filter changed while also filtering on shared component data?

 

Add [ChangedFilter] - it's per chunk, but pretty much free and likely the only way to do this in a meaningful way.

public void Execute(Entity e, int index, [ReadOnly, ChangedFilter] ref SpriteSheetColor c0)

 

Thank you, that did work for my component to dynamic buffer jobs. But my renderer needs to stay on the main thread to work with compute buffers. Is there really no way to do this outside a job filter?

If not I'll just have to add an intermediate job to set a flag on any changed buffers. Not the end of the world but seems unnecessarily complicated.

 

Did another big rewrite using @sngdan 's suggestions - got another massive performance boost. Frame rate starts to dip around 500k entities

@sngdan You were pretty much dead on - I'm absolutely able to write to the dynamic buffers in parallel with the way I'm doing it. I split out all the data as well, including TRS, and was able to simplify the streaming classes a lot since they were then limited to one component for one system.

The filters work on the Components->Buffers stream but I couldn't come up with a decent way to make it work for dynamic buffers -> compute buffers. I cannot for the life of me figure out how to filter both the shared component data AND changed components from the main thread. At least not in a way that can scale and doesn't make the code terrible.

Still seeing the really long "gfx.ProcessCommands". Since I'm still writing to compute buffers every frame I'm assuming I could cut that down a ton if I could properly filter the unchanged buffers from the render system.

Thanks again @sngdan for all the advice, I learned a ton thanks to you. I'm probably going to clean the code up and take a break from this unless someone can help me solve the filtering problem. Code is here if anyone is interested:
https://github.com/sarkahn/SpriteSheetRenderer

 

Nice - I just had a quick look at your source...need to check it properly next weekend, but it looks you put a lot of effort...and it was worth it - great performance boost. Congrats!

 

Sorry for not being active on this post, but I had to enjoy the Italian sun for a while.
Soo, I decided to adopt the DynamicBuffers, path and the performance boost was massive, in the screenshot, there are 700k sprites on the screen, at 85fps.
I tried to write the code as simple as possible without over-engineering, but I still have to polish it a bit, that's why I'm yet to commit and push my changes.

 

Is that on your latest commit?

EDIT: Oh i see it is in another branch.

 

@belva1234

I only had a very quick look, Questions:
* bounds2d - changefilter?
* colorbuffersystem, matrixbuffersystem,spriteSheetUVsystem - why not IJFEWE like you did for the rest?

Spoiler: IFFEWE

Code (CSharp):

1. public class SpriteSheetUvJobSystem : JobComponentSystem {
2.     //todo menage multiple materials
3.     //todo menage destruction of buffers
4.     [BurstCompile]
5.     struct UpdateJob : IJobForEachWithEntity<SpriteIndex> {
6.         [NativeDisableParallelForRestriction] public DynamicBuffer<SpriteIndexBuffer> indexBuffer;
7.        
8.         public void Execute(Entity e, int i, [ReadOnly, ChangedFilter] ref SpriteIndex data){
9.             indexBuffer[data.bufferID] = data.Value;
10.         }
11.     }
12.  
13.     protected override JobHandle OnUpdate(JobHandle inputDeps) {
14.         inputDeps = new UpdateJob() {
15.             indexBuffer = DynamicBufferManager.GetIndexBuffer()
16.         }.Schedule(this, inputDeps);
17.        
18.         inputDeps.Complete();
19.        
20.         return inputDeps;
21.     }
22. }

For worst case testing, the chnagedfilters should be disabled, they are a nice to have to save time in case not everything gets updated per frame.

 

I didn't use IJFEWE because I didn't know yet how to deal with different material since there is a different "BufferEntity" for each different material.
IJFEWE is surely the way to go, and once I figure out how to deal with the materials, I'll change to them.

Code (CSharp):

1. inputDeps = new UpdateJob() {
2.             indexBuffer = DynamicBufferManager.GetIndexBuffer()
3.         }.Schedule(this, inputDeps);

right now, the index buffer is the same, but it should change based on different materials.

They are still W.I.P and not used atm, since I am working on a ComputeShader to dynamically occlude stuff.

 

Would you not just create one entity with the buffer for each material? You can use the SCD index as the index into the entity array (the ones that have the buffer) with index 0 being unused / dummy (due to how SCD works)

I have not looked into how you buffer yet. How do you do it?

 

I' am 70% done, this weekend I will finish everything.
In the end, IJFE was the correct choice, so I do not create NativeArray with all the datas every time I instantiate a job, this leads to another performance boost, I easily rendered 1 million sprites at around 75-80 fps, of course, they are static now(except the animation), I will later tell you how slow it gets after they are all actually moving.

 

I'd love to know.

 

With DrawMeshInstancedIndirect you can draw more than 1 Million sprite easily on the target machine in question.
The problem is when you have dynamic properties like transforms, animations and so on.
The only way i have found to minimize this is to double buffer the dynamic components and then only set the compute buffer indices that have changed.
Still there must always be a trade off when you double buffer because that will also slow things down.
I'm double buffering per chunk which makes it faster but in the other hand if an entity changes in a chunk only that chunk will update the compute buffer.
Tried double buffer per entity but the cost of that is too high.

 

I did my test.
1 milion sprites with dynamic position,scale,rotation,color,animation run at about 55fps..
800k sprites are rendered at 75fps

The biggest bottleneck now as @GilCat said is to only update the ComputeBuffers with the data that that is actually changing, without passing every frame the whole data for matrices, colors, etc..

 

Yep, and I doubt it gets better than chunk filtering. The only way to optimize is I guess to pre-organize your sprites into different groups (i.e. have an enum on the scd for static, every few frames, every frame) so that you have a higher chance to do the chunk filtering.

The only other thing I saw, is that there seems to be an improvement in the latest alpha to the Graphics API, where you can pass native arrays directly...

@GilCat how do you keep the indices in order? does your system allow runtime creation/destroy of sprites?

Would be great if @GilCat, @belva1234 and @Sarkahn could explain a little their final approaches - I only looked at this mostly from mobile out of interest and lost a bit track of who did what in the end.

 

Yes i do that.

Basically i keep a native array of the buffer values that i want to push into the compute buffer. That native array is a direct match to all the sprites. I then extract those values per chunk, i do double buffer so i only extract the chunks that have changed. Finally i push only those changed values into the indices that have changed.
And yes i can destroy/add sprites as much as i want because those arrays will rebuild if the overall sprite count change.

Code (CSharp):

1. ComputeBuffer.SetData(m_values, startIndex, startIndex, valueCount);

Right now my bottleneck is having to double buffer the components so i know which one changed. I actually do per chunk double buffer by calculating the hashcode of the whole chunk (it's faster than doing it per entity). That said it means i can only detect that the whole chunk has changed which cause that if a position of only one sprite changes i will push the indices of the whole chunk it belongs to.

 

@GilCat

Yes, this is exactly, how I would have done it (but never actually did). I was wondering if there was a way to avoid the rebuild if you add/destroy sprites like sorting or keeping another index, but never thought it through. I don't know exactly how you double buffer, the way I was thinking of doing it is

Entity per material

• with DynamicBuffers (i.e. colors, pos, etc.) -> keep those updated leveraging changefilter
• with DynamicBuffers maintaining the copy ranges of each -> as above (they update in parallel and keep startindex + length)

As I described before, if the game allows grouping the sprites into update frequency categories, I would split them further into chunks via i.e. an enum on the SCD material. So basically you reduce the chance that you have a mix of sprites in a chunk that are static and update every frame (to avoid copying chunks where many entities have not actually changed values)

 

I have one entity per sprite here. I'm also using the existing Transform System so i pass LocalToWorld matrix into the compute buffer.
I use Unity builtin SpriteAtlas to extract the sprite (UV, Scale and pivot) using a quad for a mesh when SpriteAtlas is present and the sprite own mesh if it's a sprite without atlas. I plan to combine meshes in the future so i can use the actual tight mesh with atlas.
I'm also extracting the animated sprites from an AnimationClip (Similar to what is used in Vertex shader based skinned character animation).
I will make my code available to anyone interested when i think it meets my performance requirements.

 

I create one buffer entity per material. Each buffer entity holds one dynamic buffer for all the different types of per instance data. In the renderer I keep a dictionary of compute buffer lists keyed by material that's kept in sync with the dynamic buffers. The original idea being I would only write to dynamic buffers from source data that has changed, and in turn the dynamic buffers would only be pushed to the compute buffers when they have changed. Since there's a one to one relationship between instance data and compute buffer it would in theory be very good at only doing work when necessary.

Unfortunately, as I mentioned in my last post, I couldn't come up with a way to filter out changed buffers, so all my dynamic buffers get pushed to their respective compute buffers every frame, even if the source data hasn't changed. I need to be able to filter an EntityQuery by both SharedComponentData AND ChangedFilter, but I couldn't figure out how.

 

@GilCat When I said 1 entity per material, that was for buffering only. If you don’t hold the buffers in a system, i have not understood your buffering strategy.

I would be interested in your code, as I like the approach of leveraging Unities tooling - I doubt you will be able to achieve the same level of performance but it’s more practical for real development (did you manage your tenderer to co-exist with unities V2 based on a tag or something?)

@Sarkahn my post above explains the idea how to do it, did you try and it did not work? I did not fully understand your filtering requirements, but did you try to filter the EQ by SCD and then schedule your job with change filter and the filtered EQ (ie .Schedule(EQ-filteredSCD, ...)

 

I was about to write my version of this, but life came in the way. Sadly, I simply don't have time currently...other than reading on commute... this is the buffering system I had in mind, not really tested much...maybe it helps someone here.

edit: reminder for self - memcopy approach

- for each material 1 entity
- this entity holds a DynamicBuffer (basically a native array) of each value to be passed to the ComputeBuffer (this is a copy of whats in the ComputeBuffer) - only changes get updated per frame
- this entity also holds a DynamicBuffer with a start index and a length of all the values that changed in the last frame (this is per chunk)

Spoiler: concept only

Code (CSharp):

1. using System;
2. using System.Collections.Generic;
3. using UnityEngine;
4. using Unity.Entities;
5. using Unity.Collections;
6. using Unity.Burst;
7. using Unity.Jobs;
8. using Unity.Mathematics;
9.  
10. namespace ForumRenderer
11. {
12.     public struct MySCD : ISharedComponentData
13.     {
14.         public int myMatieral;                        // for testing not using a material, when using a reference type, index 0 becomes default -> ignore later.
15.         public int myUpdateFrequency;    // idea - further split chunks to make change filter more effective 0-low update frequency, 1-med , 2-high
16.     }
17.  
18.     public struct MyColor : IComponentData
19.     {
20.         public float4 Value;
21.     }
22.  
23.     public struct MyDummy : IComponentData
24.     {
25.         public NativeString4096 Dummy1;    // to fill up chunk fast
26.     }
27.  
28.     [InternalBufferCapacity(0)]
29.     public struct ColorBuffer : IBufferElementData
30.     {
31.         public float4 Value;
32.     }
33.  
34.     [InternalBufferCapacity(16)]
35.     public struct ColorBufferChanges : IBufferElementData
36.     {
37.         public int2 Value;        // x = startPos, y = Length
38.     }
39.  
40.  
41.     public class SpriteUpdateSystem : JobComponentSystem
42.     {
43.         int materialCount = 3;
44.         int updateGroupCount = 1;
45.         int spriteCountMin = 10;
46.         int spriteCountMax = 21;
47.  
48.         protected override void OnCreate()
49.         {
50.             CreateSprites();
51.         }
52.  
53.         protected override JobHandle OnUpdate(JobHandle inputDeps)
54.         {
55.             return inputDeps;
56.         }
57.  
58.         void CreateSprites()
59.         {
60.             int counter = 0;
61.             for (int i = 0; i < materialCount; i++)
62.             {
63.                 for (int u = 0; u < updateGroupCount; u++)
64.                 {
65.                     var spriteCount = UnityEngine.Random.Range(spriteCountMin, spriteCountMax);
66.                     for (int j = 0; j < spriteCount; j++)
67.                     {
68.                         var e = EntityManager.CreateEntity(ComponentType.ReadOnly<MySCD>(), ComponentType.ReadOnly<MyColor>(), ComponentType.ReadOnly<MyDummy>());
69.                         EntityManager.SetSharedComponentData<MySCD>(e, new MySCD {myMatieral = i, myUpdateFrequency = u});
70.                         var color = UnityEngine.Random.ColorHSV();
71.                         EntityManager.SetComponentData<MyColor>(e, new MyColor {Value = new float4 (color.r, color.g, color.b, counter)});
72.                         counter++;
73.                     }
74.                 }
75.             }
76.         }
77.     }
78.  
79.     public class BufferUpdateSystem : JobComponentSystem
80.     {  
81.         EntityQuery myRenderChunks;
82.         EntityQuery myBufferEntities;
83.         List<MySCD> sharedComponentValues = new List<MySCD>();
84.         List<int> sharedComponentIndices = new List<int>();
85.         NativeList<int> bufferOffsets = new NativeList<int>(Allocator.Persistent);
86.  
87.         [BurstCompile]
88.         public struct BufferUpdateJob : IJobChunk
89.         {
90.             [ReadOnly] public ArchetypeChunkComponentType<MyColor> MyColorType;
91.             [ReadOnly] public ArchetypeChunkSharedComponentType<MySCD> MySCDType;
92.             [ReadOnly, DeallocateOnJobCompletion] public NativeArray<Entity> ColorBufferEntities;
93.             [ReadOnly] public NativeList<int> BufferOffsets;
94.             [WriteOnly, NativeDisableParallelForRestriction] public BufferFromEntity<ColorBuffer> ColorBuffers;
95.             [WriteOnly, NativeDisableParallelForRestriction] public BufferFromEntity<ColorBufferChanges> ColorBuffersChanges;
96.             public uint LastSystemVersion;
97.  
98.             public void Execute(ArchetypeChunk chunk, int chunkIndex, int firstEntityIndex)
99.             {
100.                 if (chunk.DidChange(MyColorType, LastSystemVersion))
101.                 {
102.                     var chunkColors = chunk.GetNativeArray(MyColorType);
103.                     var bufferIndex = chunk.GetSharedComponentIndex<MySCD>(MySCDType);
104.                     var colorBuffer = ColorBuffers[ColorBufferEntities[bufferIndex]].Reinterpret<float4>().AsNativeArray();
105.                     var bufferOffset = BufferOffsets[bufferIndex];
106.                     var bufferStart = firstEntityIndex - bufferOffset;
107.              
108.                     for (int i = 0; i < chunk.Count; i++)
109.                     {
110.                         var index = bufferStart + i;
111.                         colorBuffer[index] = chunkColors[i].Value;
112.                     }
113.                     ColorBuffersChanges[ColorBufferEntities[bufferIndex]].Add(new ColorBufferChanges {Value = new int2 {x = bufferStart, y = chunk.Count}});
114.                 }
115.          
116.                 // Repeat with other data to buffer
117.             }
118.         }
119.  
120.         protected override void OnCreate()
121.         {
122.             myRenderChunks = GetEntityQuery(ComponentType.ReadOnly<MySCD>(), ComponentType.ReadOnly<MyColor>(), ComponentType.ReadOnly<MyDummy>());
123.             myBufferEntities = GetEntityQuery(ComponentType.ReadWrite<ColorBuffer>(), ComponentType.ReadWrite<ColorBufferChanges>());
124.         }
125.  
126.         protected override void OnDestroy()
127.         {
128.             bufferOffsets.Dispose();
129.         }
130.  
131.         protected override JobHandle OnUpdate(JobHandle inputDeps)
132.         {
133.             // INITIATE BUFFERS - NOT IMPLEMENTED (use systemstatecomponent -> update buffers if structural changes)
134.             // for testing purpose only - no structural changes
135.             if ( (sharedComponentValues.Count == 0) || (sharedComponentValues.Count != myBufferEntities.CalculateLength()) )
136.             {
137.                 InitBuffers();
138.             }
139.  
140.             var ColorBufferEntities = myBufferEntities.ToEntityArray(Allocator.TempJob);            // this could be cached at buffer initialization
141.             inputDeps = new BufferUpdateJob
142.             {
143.                 MyColorType            = GetArchetypeChunkComponentType<MyColor>(true),
144.                 MySCDType            = GetArchetypeChunkSharedComponentType<MySCD>(),
145.                 ColorBufferEntities    = ColorBufferEntities,
146.                 BufferOffsets        = bufferOffsets,
147.                 ColorBuffers        = GetBufferFromEntity<ColorBuffer>(false),
148.                 ColorBuffersChanges    = GetBufferFromEntity<ColorBufferChanges>(false),
149.                 LastSystemVersion    = this.LastSystemVersion
150.             }.Schedule(myRenderChunks, inputDeps);
151.      
152.             // to-do: job that consolidates bufferchange ranges in case adjacent chunks change at the same time
153.      
154.             // for testing
155.             inputDeps.Complete();
156.      
157.             return inputDeps;
158.         }
159.  
160.         void InitBuffers ()
161.         {
162.             bufferOffsets.Clear();
163.             sharedComponentValues.Clear();
164.             sharedComponentIndices.Clear();
165.      
166.             EntityManager.GetAllUniqueSharedComponentData<MySCD>(sharedComponentValues, sharedComponentIndices);
167.      
168.             int offset = 0;
169.             for (int i = 0; i < sharedComponentValues.Count; i++)
170.             {
171.                 // create one entity per material, to hold all buffers for this material
172.                 var e = EntityManager.CreateEntity(ComponentType.ReadWrite<ColorBuffer>(), ComponentType.ReadWrite<ColorBufferChanges>());
173.                 var b = EntityManager.GetBuffer<ColorBuffer>(e);
174.          
175.                 // resize the buffers to the number of entities with this material
176.                 myRenderChunks.SetFilter(sharedComponentValues[i]);
177.                 var length = myRenderChunks.CalculateLength();
178.                 b.ResizeUninitialized(length);
179.                 bufferOffsets.Add(offset);
180.                 offset = offset + length;
181.  
182.                 #if UNITY_EDITOR
183.                 var name = "Buffer mat: " + sharedComponentValues[i].myMatieral + " freq: " + sharedComponentValues[i].myUpdateFrequency;
184.                 EntityManager.SetName(e, name);
185.                 #endif
186.             }
187.             myRenderChunks.ResetFilter();
188.         }
189.     }
190.  
191.     //[UpdateAfter(typeof(BufferUpdateSystem))]
192.     //public class SpriteSheetRenderer : ComponentSystem
193.     //{
194. // TO-DO: ONCE THE BUFFER IS COPIED, IT HAS TO BE CLEARED
195.         //}
196.  
197. }
198.  

 

My approach is similar to the @Sarkahn one.
For each material, I have an entity that contains all the buffers(Matrix, Color, SpriteIndex, UvMapping).
Each sprite has a BufferHook component that helps me to get the entity with all the buffers for a specific material and the index of the buffer that corresponds to that sprite.
In this way I can easly update the buffers using IJFE, and than copy them inside the Render function reinterpreting the buffer.

Code (CSharp):

1.  
2. public class MatrixBufferSystem : JobComponentSystem {
3.   [BurstCompile]
4.   struct UpdateJob : IJobForEach<RenderData, BufferHook> {
5.     [NativeDisableParallelForRestriction]
6.     public DynamicBuffer<MatrixBuffer> indexBuffer;
7.     [ReadOnly]
8.     public int bufferEnityID;
9.     public void Execute([ReadOnly, ChangedFilter] ref RenderData data, [ReadOnly] ref BufferHook hook) {
10.       if(bufferEnityID == hook.bufferEnityID)
11.         indexBuffer[hook.bufferID] = data.matrix;
12.     }
13.   }
14.  
15.   protected override JobHandle OnUpdate(JobHandle inputDeps) {
16.     var buffers = DynamicBufferManager.GetMatrixBuffers();
17.     NativeArray<JobHandle> jobs = new NativeArray<JobHandle>(buffers.Length, Allocator.TempJob);
18.     for(int i = 0; i < buffers.Length; i++) {
19.       inputDeps = new UpdateJob() {
20.         indexBuffer = buffers[i],
21.         bufferEnityID = i
22.       }.Schedule(this, inputDeps);
23.       jobs[i] = inputDeps;
24.     }
25.     JobHandle.CompleteAll(jobs);
26.     jobs.Dispose();
27.     return inputDeps;
28.   }
29. }
30.  
31.  

Code (CSharp):

1. matrixBufferQuery.SetFilter(material);    
2. var entities = matrixBufferQuery.ToEntityArray(Allocator.TempJob);
3. var bufferEntity = entities[0];
4. entities.Dispose();
5. renderInfos[renderIndex].matrixBuffer.SetData(EntityManager.GetBuffer<MatrixBuffer>(bufferEntity).Reinterpret<float4>().AsNativeArray());
6.  

 

I do filter per RenderMesh shared component (Mesh + Material) and i hold a buffer per RenderMesh.

Real life comes first. Hope all gets better for you soon.

I do something like that but not with Dynamic Buffers.

 

Would be wonderful if ComputeBuffer could receive and array of indices to be updated.

 

Thank - no all is good, it's just work / family thats keeping me away from the hobby...

I wanted to use this
- public void SetData(NativeArray<T> data, int nativeBufferStartIndex, int computeBufferStartIndex, int count);

And I think a better approach than the IJobChunk I was using is to schedule IJobs that process sequentially through the data to consolidate the copy ranges on the fly, also instead of keeping a full copy of the data (i.e. color) it should be enough to only keep the changes (first frame = full copy).

BufferJobComponentSystem (The jobs run in parallel and are not depending on each other -> still good thread utilization)
Foreach material
- Schedule IJob ColorBufffer (copies the changed values + keeps track of the ranges)
- Schedule IJob xxxBuffer
- ...

RenderComponentSystem
Foreach material
- Foreach copyrange
-- SetData(as above)
- Graphics.DrawMeshInstancedIndirect


edit: i have not worked with computebuffers before, so I am not sure if this is actually worth the effort, or if the expensive call is SetBuffer

 

So here is my work on sprite rendering under DOTS
https://github.com/GilbertoGojira/DOTS-Stackray

Feel free to use it and abuse it. There are still work to do.
Sorry for the lack of a README but i will do that after vacations.
For now load the SpriteRendererExample project and open the several scenes in there.

 

This is all great, but will there be any official implementation from Unity any time soon? Where can I check the roadmap for implementing rendering side of the engine for ECS?

 

Probably not. They're probably going to work with 3D first then 2D. We made our own DOTS 2D rendering as well. It's not hard.

 

Might not be hard for some, but I can't blame anyone for not wanting to write boilerplate code that Unity provided a game designer before DOTS.

DOTS still has a long way to go when it comes to rendering. With their current Hybrid renderer I can't do want I want with DOTS compared to the old renderer. Namely material property block support and if I remember right my Texture Arrays may have been causing a problem, can't recall it's been awhile.

 

Just saying that if you want to use DOTS for performance but for 2D, might as well use these open source options or make your own because Unity won't get there soon enough.

 

I haven't tried yet to be honest, but anyone knows, if discussed approaches will be equally effective for 3D than 2D?
I don't see reason why not (other than poly counts), but I may miss something less obvious?

 

yeah they will be, 2D solution is just 3D with only quads so they are not really different.

I tested this https://github.com/sarkahn/SpriteSheetRenderer
but i'm getting only ~12 fps in editor (with burst on) any ideas why?

 

Sorry about that, seems I left some test code in there last time I was messing with it. I had an empty query running on every entity in the scene every frame.

I've removed that it's working again - better than before it seems with the latest package updates. I was getting around 120 fps with the same scene using the latest beta and up to date entities package.

 

@Sarkahn I started reading this thread, and had my doubts, but after testing this on 19.3b I am more than amazed, had to make small changes to the shader (added #pragma instancing_options procedural:setup and a dummy void setup() {}) to remove the error in console, not sure if that is 19.3b specific but I finally ended up with the following results...





thats 1.5m sprites in the editor...

 

Yeah it definitely works well. It's a far cry from being a generic tool that anyone could use for any kind of real-life scenario, but it's a great proof of concept for how hard you can push ECS and graphics instancing if your use case is simple enough.

When I tried to run it with that many sprites it seized up on me. I have a pretty beefy PC too, so I'm not sure why everyone else seems to get such better performance, hahah.

Sadly Unity's hard pivot away from being able to work with entities directly in the editor has kinda killed any interest I had in continuing to mess with ECS. The conversion workflow is just so awkward right now, especially with the lack of proper documentation. Hopefully we get some promising news from Unite.

 

@Sarkahn is there any catch to change sprite sheet?

ArgumentException: Attempting to create a zero length compute buffer
Parameter name: count
​

I'm getting this error when trying to apply my spritesheet with your shader. Any clues?

EDIT. i found out that i need to place my texture in resource folder but still i have strange resoults,
CachedUVData.GetCellCount(mat); gives me 10 when i have only 5 sprites

EDIT2. Below you can see what my texture is and how it looks like in game

Spoiler

Spoiler

 

...and months later, I finally had a moment to put a quick proof of concept together...

[Updated 04 Oct 19]

• Minor code clean-up
• Still uses the built-in transform system, which can be optimized for 2D (i.e. pass 16 bytes to gpu for 2Dpos, rot, scl & calculate local to world in shader instead of passing ltw matrix with 64 bytes and cpu ltw calculation)
• Supports different materials & meshes (i.e. you can also put 3D meshes)
• Optimized for non-stress test scenarios, i.e. not all values change per frame, it only copies changes to the GPU not the full buffer
• Only proof of concept, misses some functionality like re-init buffers in case of structural changes, etc.
• In my view much simpler than other implementations

Spoiler: Buffer System

Code (CSharp):

1.  
2. public class SimpleBufferUpdateSystem : JobComponentSystem
3.     {
4.         private readonly List<int> _sharedComponentIndices = new List<int>();
5.         private readonly List<SimpleRenderMesh> _sharedComponentValues = new List<SimpleRenderMesh>();
6.         private Dictionary<SimpleRenderMesh, Entity> _simpleRenderMeshBufferEntityLookup = new Dictionary<SimpleRenderMesh, Entity>();
7.         private EntityQuery _bufferQuery;
8.         private EntityQuery _renderQuery;
9.    
10.        [BurstCompile]
11.         private struct BufferUpdateJob : IJobChunk
12.         {
13.             [ReadOnly] public ArchetypeChunkComponentType<LocalToWorld> LocalToWorldType;
14.             [NativeDisableParallelForRestriction, NativeDisableContainerSafetyRestriction, NativeDisableUnsafePtrRestriction] public NativeArray<float4x4> LtwBuffer;
15.             [NativeDisableParallelForRestriction, NativeDisableContainerSafetyRestriction, NativeDisableUnsafePtrRestriction] public NativeArray<int2> LtwBufferChunkChanges;
16.             [ReadOnly] public ArchetypeChunkComponentType<SimpleColor> SimpleColorType;
17.             [NativeDisableParallelForRestriction, NativeDisableContainerSafetyRestriction, NativeDisableUnsafePtrRestriction] public NativeArray<float4> ColorBuffer;
18.             [NativeDisableParallelForRestriction, NativeDisableContainerSafetyRestriction, NativeDisableUnsafePtrRestriction] public NativeArray<int2> ColorBufferChunkChanges;
19.             public uint LastSystemVersion;
20.  
21.             public void Execute(ArchetypeChunk chunk, int chunkIndex, int firstEntityIndex)
22.             {
23.                 var value = new int2(firstEntityIndex, -1);
24.                 if (chunk.DidChange(LocalToWorldType, LastSystemVersion))
25.                 {
26.                     unsafe
27.                     {
28.                         var sourcePtr = chunk.GetNativeArray(LocalToWorldType).GetUnsafeReadOnlyPtr();
29.                         var destinationPtr = (byte*) LtwBuffer.GetUnsafePtr() + UnsafeUtility.SizeOf<LocalToWorld>() * firstEntityIndex;
30.                         var copySizeInBytes = UnsafeUtility.SizeOf<LocalToWorld>() * chunk.Count;
31.                    
32.                         UnsafeUtility.MemCpy(destinationPtr, sourcePtr, copySizeInBytes);
33.                         value.y = chunk.Count;
34.                     }
35.                 }
36.                 LtwBufferChunkChanges[chunkIndex] = value;
37.  
38.                 value = new int2(firstEntityIndex, -1);
39.                 if (chunk.DidChange(SimpleColorType, LastSystemVersion))
40.                 {
41.                     unsafe
42.                     {
43.                         var sourcePtr = chunk.GetNativeArray(SimpleColorType).GetUnsafeReadOnlyPtr();
44.                         var destinationPtr = (byte*) ColorBuffer.GetUnsafePtr() + UnsafeUtility.SizeOf<SimpleColor>() * firstEntityIndex;
45.                         var copySizeInBytes = UnsafeUtility.SizeOf<SimpleColor>() * chunk.Count;
46.  
47.                         UnsafeUtility.MemCpy(destinationPtr, sourcePtr, copySizeInBytes);
48.                         value.y = chunk.Count;
49.                     }
50.                 }
51.                 ColorBufferChunkChanges[chunkIndex] = value;
52.             }
53.         }
54.    
55.         protected override void OnCreate()
56.         {
57.             _renderQuery = GetEntityQuery(ComponentType.ReadOnly<LocalToWorld>(), ComponentType.ReadOnly<SimpleColor>(), ComponentType.ReadOnly<SimpleRenderMesh>());
58.             _bufferQuery = GetEntityQuery(ComponentType.ReadWrite<SimpleLtwBuffer>(), ComponentType.ReadWrite<SimpleLtwBufferChunkChanges>(), ComponentType.ReadWrite<SimpleColorBuffer>(), ComponentType.ReadWrite<SimpleColorBufferChunkChanges>(), ComponentType.ReadOnly<SimpleRenderMesh>());
59.         }
60.  
61.         protected override void OnDestroy()
62.         {
63.             var bufferEntityArray = _bufferQuery.ToEntityArray(Allocator.TempJob);
64.             foreach (var e in bufferEntityArray)
65.             {
66.                 var simpleRenderMesh = EntityManager.GetSharedComponentData<SimpleRenderMesh>(e);
67.                 var rb = simpleRenderMesh.renderBuffers;
68.                 rb.ReleaseBuffers();
69.             }
70.             bufferEntityArray.Dispose();
71.         }
72.  
73.         protected override JobHandle OnUpdate(JobHandle inputDeps)
74.         {
75. //            INITIATE BUFFERS FOR STRUCTURAL CHANGES NOT IMPLEMENTED, should be enough to check if all entities of a scd type < corresponding buffer length
76.             if (_sharedComponentValues.Count == 0 || _sharedComponentValues.Count != _bufferQuery.CalculateEntityCount()) InitBuffers();
77.  
78.             var colorBufferEntities = _bufferQuery.ToEntityArray(Allocator.TempJob); // this could be cached at buffer initialization
79.             var parallelJobHandles = new NativeArray<JobHandle>(_sharedComponentValues.Count, Allocator.TempJob);
80.        
81.             for (int i = 0; i < _sharedComponentValues.Count; i++)
82.             {
83.                 _renderQuery.SetFilter(_sharedComponentValues[i]);
84.                 var bufferEntity = colorBufferEntities[i];
85.            
86.                 parallelJobHandles[i] = new BufferUpdateJob
87.                 {
88.                     LocalToWorldType = GetArchetypeChunkComponentType<LocalToWorld>(true),
89.                     LtwBuffer = GetBufferFromEntity<SimpleLtwBuffer>()[bufferEntity].Reinterpret<float4x4>().AsNativeArray(),
90.                     LtwBufferChunkChanges = GetBufferFromEntity<SimpleLtwBufferChunkChanges>()[bufferEntity].Reinterpret<int2>().AsNativeArray(),
91.                     SimpleColorType = GetArchetypeChunkComponentType<SimpleColor>(true),
92.                     ColorBuffer = GetBufferFromEntity<SimpleColorBuffer>()[bufferEntity].Reinterpret<float4>().AsNativeArray(),
93.                     ColorBufferChunkChanges = GetBufferFromEntity<SimpleColorBufferChunkChanges>()[bufferEntity].Reinterpret<int2>().AsNativeArray(),
94.                     LastSystemVersion = LastSystemVersion
95.                 }.Schedule(_renderQuery, inputDeps);
96.             }
97.        
98.             inputDeps = JobHandle.CombineDependencies(parallelJobHandles);
99.             parallelJobHandles.Dispose();
100.             colorBufferEntities.Dispose();
101.             // inputDeps.Complete();
102.             return inputDeps;
103.         }
104.  
105.         private void InitBuffers()
106.         {
107.             Debug.Log("Init Buffers");
108.             // get unique list of srRenderMesh SCD indices & values
109.             _sharedComponentValues.Clear();
110.             _sharedComponentIndices.Clear();
111.             EntityManager.GetAllUniqueSharedComponentData(_sharedComponentValues, _sharedComponentIndices);
112.             _sharedComponentValues.RemoveAt(0);
113.        
114.             // iterate through all unique srRenderMesh SCD values
115.             for (var i = 0; i < _sharedComponentValues.Count; i++)
116.             {
117. //              get or create one entity per srRenderMesh SCD value, to hold all buffers for this material
118.                 if (!_simpleRenderMeshBufferEntityLookup.TryGetValue(_sharedComponentValues[i], out var e))
119.                 {
120.                     e = EntityManager.CreateEntity(
121.                         ComponentType.ReadWrite<SimpleLtwBuffer>(),
122.                         ComponentType.ReadWrite<SimpleLtwBufferChunkChanges>(),
123.                         ComponentType.ReadWrite<SimpleColorBuffer>(),
124.                         ComponentType.ReadWrite<SimpleColorBufferChunkChanges>(),
125.                         ComponentType.ReadWrite<SimpleRenderMesh>());
126.                     _simpleRenderMeshBufferEntityLookup.Add(_sharedComponentValues[i], e);
127.                 }
128.  
129. //                filter query to current SCD value
130.                 _renderQuery.SetFilter(_sharedComponentValues[i]);
131.            
132. //                resize buffers to the number of entities with this material
133.                 var entityCount = _renderQuery.CalculateEntityCount();
134.                 var chunkCount = _renderQuery.CalculateChunkCount();
135.                 EntityManager.GetBuffer<SimpleLtwBuffer>(e).ResizeUninitialized(entityCount);
136.                 EntityManager.GetBuffer<SimpleLtwBufferChunkChanges>(e).ResizeUninitialized(chunkCount);
137.                 EntityManager.GetBuffer<SimpleColorBuffer>(e).ResizeUninitialized(entityCount);
138.                 EntityManager.GetBuffer<SimpleColorBufferChunkChanges>(e).ResizeUninitialized(chunkCount);
139.            
140.                 // update the Component Buffers
141.                 var scd = _sharedComponentValues[i];
142.                 var rb = scd.renderBuffers;
143.                 rb.args[0] = (uint) scd.mesh.GetIndexCount(0);
144.                 rb.args[1] = (uint) entityCount;
145.                 rb.args[2] = scd.mesh.GetIndexStart(0);
146.                 rb.args[3] = scd.mesh.GetBaseVertex(0);
147.                 rb.ltwBuffer?.Release();
148.                 rb.ltwBuffer = new ComputeBuffer(entityCount, 4 * 4 * sizeof(float));
149.                 rb.colBuffer?.Release();
150.                 rb.colBuffer = new ComputeBuffer(entityCount, 4 * sizeof(float));
151.                 EntityManager.SetSharedComponentData(e, scd);
152.  
153. #if UNITY_EDITOR
154.                 var name = "Buffer mat: " + _sharedComponentValues[i].material.name;
155.                 EntityManager.SetName(e, name);
156. #endif
157.             }
158.             _renderQuery.ResetFilter();
159.         }
160.     }
161.  
162.  

Spoiler: Render System

Code (CSharp):

1.  
2. [UpdateInGroup(typeof(PresentationSystemGroup))]
3. public class SimpleRenderSystem : ComponentSystem
4. {
5.     private readonly Bounds _renderBounds = new Bounds(Vector3.zero, 1000000 * Vector3.one);
6.     private readonly MaterialPropertyBlock _mpb = new MaterialPropertyBlock();
7.     private HashSet<Entity> _initializedBufferEntities = new HashSet<Entity>();
8.  
9.     protected override void OnUpdate()
10.     {
11.         Entities.WithAll<SimpleLtwBuffer, SimpleColorBuffer, SimpleRenderMesh>().ForEach((Entity e) =>
12.         {
13.             var simpleLtwBuffer = EntityManager.GetBuffer<SimpleLtwBuffer>(e);
14.             var simpleLtwBufferChunkChanges = EntityManager.GetBuffer<SimpleLtwBufferChunkChanges>(e);
15.             var simpleColorBuffer = EntityManager.GetBuffer<SimpleColorBuffer>(e);
16.             var simpleColorBufferChunkChanges = EntityManager.GetBuffer<SimpleColorBufferChunkChanges>(e);
17.             var simpleRenderMesh = EntityManager.GetSharedComponentData<SimpleRenderMesh>(e);
18.        
19.             if (!_initializedBufferEntities.Contains(e))
20.             {
21.                 simpleRenderMesh.material.SetBuffer("ltwBuffer", simpleRenderMesh.renderBuffers.ltwBuffer);
22.                 simpleRenderMesh.renderBuffers.ltwBuffer.SetData(simpleLtwBuffer.AsNativeArray());
23.                 simpleRenderMesh.material.SetBuffer("colBuffer", simpleRenderMesh.renderBuffers.colBuffer);
24.                 simpleRenderMesh.renderBuffers.colBuffer.SetData(simpleColorBuffer.AsNativeArray());
25.                 simpleRenderMesh.renderBuffers.argBuffer.SetData(simpleRenderMesh.renderBuffers.args);
26.                 _initializedBufferEntities.Add(e);
27.             }
28.             else
29.             {
30.                 UpdateRenderBuffer(simpleRenderMesh.renderBuffers.ltwBuffer, simpleLtwBuffer.AsNativeArray(), simpleLtwBufferChunkChanges.Reinterpret<int2>().AsNativeArray());
31.                 UpdateRenderBuffer(simpleRenderMesh.renderBuffers.colBuffer, simpleColorBuffer.AsNativeArray(), simpleColorBufferChunkChanges.Reinterpret<int2>().AsNativeArray());
32.             }
33.        
34.             Graphics.DrawMeshInstancedIndirect(simpleRenderMesh.mesh, 0, simpleRenderMesh.material, _renderBounds, simpleRenderMesh.renderBuffers.argBuffer, 0, _mpb, simpleRenderMesh.castShadows, simpleRenderMesh.receiveShadows);
35.         });
36.     }
37.  
38.     private void UpdateRenderBuffer<T>(ComputeBuffer computeBuffer, NativeArray<T> dataBuffer, NativeArray<int2> bufferChunkChanges) where T : struct
39.     {
40.         var start = 0;
41.         var def = new int2(-1,-1);
42.    
43.         for (int i = 0; i < bufferChunkChanges.Length; i++)
44.         {
45.             var cur = bufferChunkChanges[i];
46.             var pre = i == 0 ? def : bufferChunkChanges[i - 1];
47.             var nxt = (i == bufferChunkChanges.Length - 1) ? def : bufferChunkChanges[i + 1];
48.  
49.             if (pre.y == -1 && cur.y != -1) start = cur.x;
50.             if (nxt.y == -1 && cur.y != -1)
51.             {
52.                 var count = cur.x + cur.y - start;
53.                 computeBuffer.SetData(dataBuffer, start, start, count);
54.             }
55.         }
56.     }
57. }
58.  

 

Running the latest version of GitHub only gives me 12fps unless I disable the following code in EntitySpawner.cs:

Code (CSharp):

1.   bool blocked = false;
2.   protected override void OnUpdate() {
3.     if(Input.GetKey(KeyCode.Space)) {
4.       MakeSpriteEntities.SpawnEntity(PostUpdateCommands, "emoji");
5.     }
6.     if(Input.GetKey(KeyCode.U)) {
7.       blocked = false;
8.     }
9.     Entities.WithAll<SpriteSheetAnimation>().ForEach((Entity e, ref SpriteSheetAnimation anim) => {
10.       if(Input.GetKeyUp(KeyCode.R)) {
11.         blocked = true;
12.         MakeSpriteEntities.DestroyEntity(PostUpdateCommands, e, "emoji");
13.         return;
14.       }
15.     });

If I do I run 250000 sprites @ 110fps. The ForEach on 250000 components every frame is the issue there.

 

I poked around a bit, but it's been a long time since I've done anything with this code. My version is most definitely over-engineered in the end, even after all my attempts to simplify it. On top of that I've never been very good with shaders. I can try to explain how the original works and maybe you can fix your problem.

In the original example the cached uv data gets pushed into a compute buffer based on Unity's "sprite.uv". A separate compute buffer stores the "uv cell" - aka whichever sprite in the sprite sheet the entity wants to appear as. The shader then gets each of the four vertices' uv data for it's cell via SV_VertexID and sends that to the fragment shader.

The sprites themselves need to be set up in Unity's sprite editor. In my case all my sprites are squares, they're all the same size and laid out sequentially on the same sprite sheet. Based on your image it seems like yours are as well but I can't tell for sure. Maybe double check that they are actually sliced out properly in the sprite editor based on my criteria.

If that doesn't work you'll have to rework the uv caching or the shader to fit your needs. It was never meant to be a general purpose solution so it's bound to break if you try to drop new things in.

 

Any difference which I see is that you have squares and I have rectangles (100x116) because of hexes.
I'm assuming it doesn't care about size and shape, just take sprite made in sprite editor and take its uvs.
So I'm wondering what's the difference? Or maybe should ask: what are your criteria?

EDIT. And so it seems that CachedUVData gets wrong UVs from my texture but i'm unsure why. It takes only existing UVs but it seems that it somehow gets more data than my 5 sprites. Cell count is 10 and UV count is respectively 40. UV values are gibberish. I have clean 5x4 cell texture size yet again I see that in code it doesn't care.

EDIT2. I resolved my issue. I was wondering about double count of UV so I change mesh type in my spritesheet from "tight" to "full rect" and it worked. It seems that "tight" option is cutting out all alpha area from sprite so UV ends jumbled up and "full rect" leaves everything as is. So there was no problem in example spritesheet because of no alpha channel there.

Spoiler

 

What an obscure bug, good on you for figuring it out. I guess making assumptions about how Unity was storing the Sprite uv data was not a good idea on my part. Thanks for explaining the problem either way.