Question: How to do Animations in ECS\Dots or best workarounds?

I am making a top down controller in Dots for fun and i have kind of hit a road block around best practice around animations in Dots\ECS. I have created a really messy solution to get around this problem, and i was hoping to query the community about what you guys are doing with dots now.

At the moment all my ECS components do not have any "mesh\Animations". I have identical Game objects for each Component that are mono-behaviors . The Mono Game Object has an update script which it uses to take the translation and rotation of the Entity & queries the entities State in order to get details like "Is grounded" and "Speed". This is then used by the mono behavior to animate and move a mesh right where the entity is.

The reason why I have the animator component separate from the Entity is because animators do not work on entities once converted, or i don't know how to convert them.


Question:
I was wondering if anybody knows the best way to animate in ECS\Dots? Is this approach the best we have until unity releases support for animation in ECS?

 

I believe the conversion will work now (it was updated in december).
I ended up going further and did the skinning in another systembase system since systems are everywhere. However there is a big CPU cost on processing the bones / verts every frame. I believe processing and uploading the deformed mesh cost something like 2-4 ms per frame, for one character, which is alot.. That's just uploading the mesh every frame. I cache the vertex data and then copy it directly from a component memory (unsafe blitablearrays). I also did baking the animations to the textures initially, but wanted to go with something more dynamic. But that method is still really great for simpler minions.

 

There are samples here showing some cases on the how to use dots animation

Unity-Technologies/Unity.Animation.Samples: Repository of projects that showcase the new DOTS animation package (com.unity.animation). (github.com)


Animation is built on this the DataFlowGraph package and imo its essential that you also import the DataFlowGraph packages' samples("Guided tour in code") from the package manager, to assist in understanding how to digest how animation is setup
Make sure you enable show preview packages and also show package dependencies. Then in the package manager under dataflowgraph there should be an import samples button.

 

Yes I checked those samples first, before asking. Those samples are specific to using old game object data. I'm building a pure ECS solution. I don't want all that bloat, I just want a way to access the Shaders Deformation functionality (nodes), since that's all I really need. The above solution is good for speeding up a project that uses a lot of the old animation systems, and kind of mediates between them.
So just reiterating, I already checked through samples but did not find a way to simply pass bone data into the mesh. I'm using ECS packages and rendering things as entities. I don't want to suddenly throw lots of game objects into the mix. Maybe I have miss read the samples, but it seems that was the case.

 

The examples are using sub scenes. It's using the skinned meshes and animation clips as source data, and bakes it all out into entities at design time. There are no gameobjects involved at runtime. I suggest going through the examples until you understand the flow as sub scenes are pretty much required here. Partly because the data involved is just way too heavy to be converting to blob assets at runtime. And partly becomes some of the api's needed for the conversion are editor only (or were last I looked).

 

I found a solution that doesn't involve adding alot more to my project then needs to be. It is just a bit more technically harder. It mostly involves using a custom shadergraph node and doing bone deformations myself. Then just updating the ComputeShader data every time entity transforms update.
https://catlikecoding.com/unity/tutorials/basics/compute-shaders/
The conversion workflows to me seem to be for improving runtime performance of a normal project, but I'm doing mine with pure dots as it's a procedural game by nature.

 

Just out of curiosity, have you tried playing with the SkinMatrix dynamic buffer that gets added to converted SkinnedMeshRenderers when you do not have the animation package installed?

 

When I looked through the animation package and examples, I saw several animation (Rig, etc) components added to the entity, and I believe a lot of the data was based on the old animation systems. I think there was a lot more then I needed it to be. I guess I didn't want to add more data, and hoping to have thousands of characters at different resolutions. Wonder if anyone's tested scaling of the animation system? I've seen a lot of rendering scaling lately, particularly the SpriteSheetRenderer https://github.com/sarkahn/SpriteSheetRenderer
The reason I prefer this scaling is because i can build game code that will last a lot longer then any intermediate solution, in ten years time it will still run wonderfully with more details. I wonder if the SkinMatrix dynamic buffer can just get pushed to the shadergraph directly? (I mean adding the component and setting the bone matrices) I still don't like to have to grab the entire package though. As my project is pure ECS, i'de even prefer to remove all the old unity packages. But Input package relies on a lot of it.

 

Sorry. I should have been a little more verbose. Whether you have the animation package or not changes which conversion system creates SkinMatrix. SkinMatrix is actually defined in the Entities package under Unity.Deformations. If you do not have the animation package installed, the conversion creates a much simpler representation that just adds the buffer. The hybrid renderer picks up the SkinMatrix buffer, so it is completely possible to utilize the hybrid renderer's skinning support without having the animation package installed. Modifying the buffer elements did change things when I last played with it back in October. But I had issues getting the matrices to behave. I'm not sure if should blame Blender, Unity, or myself for the odd results.

 

Oh Wow, I just looked through hybrid package again, 'SkinningDeformationSystemBase' shows the data being put into the compute buffer. SkinningComputeShader.compute should be something like the deformation functions used in the shader node. And there is also 'SkinMatrixBufferIndex' but I couldn't find the SkinMatrix.
But doesn't seem very use-able for me as I wouldn't know how to use it all. I am better at writing my own code then using what's there, and rely a lot on examples that are testable. I think someone should be able to get it working though. Thank you for all the suggestions though.

 

The examples were updated an hour ago.

 

None of the scenes work for me using Unity 2021.1.0b3, URP Examples.

No errors, the meshes just don't show up.

Update: HDRP works

 

anyone know how to set the SkinMatrix buffer? where we find the matrix data for the current animation playing?
i am thinking to use an animator monobheaviour and update the matrices from there

 

I went with a custom shader, inserting the bone data manually using compute shader buffers and StructuredBuffer in the shader.
(This is my response to the first question posed about a general ECS solution to animations)

Code (CSharp):

1. Shader "zoxel/characterAnimated"
2. {
3.     SubShader
4.     {            
5.         //Tags { "RenderType"="Transparent" "Queue"="Transparent" }
6.         Tags { "RenderType"="Opaque" }
7.         LOD 100
8.  
9.         Pass
10.         {
11.             //Blend SrcAlpha OneMinusSrcAlpha
12.  
13.             CGPROGRAM
14.             #pragma vertex vert
15.             #pragma fragment frag
16.             #pragma multi_compile_instancing
17.             #pragma multi_compile_fog
18.  
19.             #pragma target 4.5
20.  
21.             #include "UnityCG.cginc"
22.  
23.             struct VertInput
24.             {
25.                 float4 vertex : POSITION;
26.                 float3 color : COLOR;
27.             };
28.  
29.             struct VertOutput
30.             {
31.                 float4 vertex : SV_POSITION;
32.                 float4 color : COLOR;
33.                 UNITY_FOG_COORDS(1)
34.             };
35.  
36.             //#ifdef UNITY_PROCEDURAL_INSTANCING_ENABLED
37.             uniform StructuredBuffer<float4> baseColors; // : register(t1);
38.             uniform StructuredBuffer<float4> additionColors; // : register(t2);
39.             uniform StructuredBuffer<float4x4> currentBones; // : register(t2);
40.             uniform StructuredBuffer<float4x4> originalBones; // : register(t3);
41.             uniform StructuredBuffer<uint> boneIndexes; // : register(t4);
42.             //#endif
43.  
44.             VertOutput vert(VertInput v, uint id : SV_VertexID, uint instanceID : SV_InstanceID)   //  SV_Vertex
45.             {
46.                 VertOutput vertOutput;
47.                 uint boneIndex = boneIndexes[id];
48.                 float4 color = float4(v.color, 1);
49.                 color *= baseColors[instanceID];
50.                 color += additionColors[instanceID];
51.                 vertOutput.color = color;
52.                 float3 vertex = v.vertex.xyz;
53.                 float4x4 originalInverseMatrix = originalBones[boneIndex];
54.                 float4x4 currentMatrix = currentBones[boneIndex];
55.                 float4 vertex4 = float4(vertex, 1);
56.                 // draws it using the bounds or matrix suppled by Graphics class
57.                 float4 boneVertex = mul(currentMatrix, mul(originalInverseMatrix, vertex4));
58.                 vertOutput.vertex = UnityObjectToClipPos(boneVertex);
59.                 UNITY_TRANSFER_FOG(vertOutput, vertOutput.vertex);
60.                 return vertOutput;
61.             }
62.  
63.             fixed4 frag (VertOutput vertOutput) : SV_Target
64.             {
65.                 float4 color = vertOutput.color;
66.                 UNITY_APPLY_FOG(vertOutput.fogCoord, color);
67.                 return color;
68.             }
69.             ENDCG
70.         }
71.     }
72. }

And in a system to upload the data to the shader:

Code (CSharp):

1. using Unity.Entities;
2. using Unity.Mathematics;
3. using Unity.Collections;
4. using Unity.Transforms;
5. using Unity.Jobs;
6.  
7. namespace Zoxel.Voxels
8. {
9.     [UpdateInGroup(typeof(ZenderSystemGroup))]
10.     public class ModelRenderBoneSystem : SystemBase
11.     {
12.         protected override void OnUpdate()
13.         {
14.             // .WithChangeFilter<SkeletonAnimationLink>() - doesnt work as uses memory link to skeleton
15.             Entities
16.                 .WithoutBurst()
17.                 .WithNone<InitializeChunkRender, ChunkRenderBuilder, DestroyChunkRender>()
18.                 .ForEach((Entity e, in ChunkMeshAnimation chunkMeshAnimation, in SkeletonAnimationLink skeletonAnimationLink,
19.                     in RenderChunkMesh renderChunkMesh, in ZoxID zoxID) =>
20.             {              
21.                 if (chunkMeshAnimation.disabled == 1 || chunkMeshAnimation.boneIndexes.Length == 0 || !ModelRenderSystem.drawers.ContainsKey(zoxID.id))              
22.                 {                  
23.                     return;
24.                 }
25.                 var buffer = ModelRenderSystem.drawers[zoxID.id];    
26.                 // if resized buffers
27.                 if (buffer.SetBoneIndexesSize(chunkMeshAnimation.boneIndexes.Length))              
28.                 {
29.                     var boneIndexes = buffer.boneIndexes.BeginWrite<uint>(0, chunkMeshAnimation.boneIndexes.Length);
30.                     for (int i = 0; i < chunkMeshAnimation.boneIndexes.Length; i++)
31.                     {
32.                         boneIndexes[i] = chunkMeshAnimation.boneIndexes[i];
33.                     }
34.                     buffer.boneIndexes.EndWrite<uint>(chunkMeshAnimation.boneIndexes.Length);              
35.                     renderChunkMesh.material.SetBuffer("boneIndexes", buffer.boneIndexes);              
36.                 }          
37.                 if (buffer.SetBoneSize(skeletonAnimationLink.originalMatrixes.Length))              
38.                 {                  
39.                     // original bones - if size changed
40.                     var originalBones = buffer.originalBonesBuffer.BeginWrite<float4x4>(0, skeletonAnimationLink.originalMatrixes.Length);
41.                     for (int i = 0; i < skeletonAnimationLink.originalMatrixes.Length; i++)
42.                     {
43.                         originalBones[i] = skeletonAnimationLink.originalMatrixes[i];
44.                     }
45.                     buffer.originalBonesBuffer.EndWrite<uint>(skeletonAnimationLink.originalMatrixes.Length);
46.                     renderChunkMesh.material.SetBuffer("originalBones", buffer.originalBonesBuffer);              
47.                 }              
48.                 // current bones              
49.                 var currentBones = buffer.currentBonesBuffer.BeginWrite<float4x4>(0, skeletonAnimationLink.boneMatrixes.Length);
50.                 for (int i = 0; i < skeletonAnimationLink.boneMatrixes.Length; i++)
51.                 {
52.                     currentBones[i] = skeletonAnimationLink.boneMatrixes[i];
53.                 }
54.                 buffer.currentBonesBuffer.EndWrite<float4x4>(skeletonAnimationLink.boneMatrixes.Length);
55.                 renderChunkMesh.material.SetBuffer("currentBones", buffer.currentBonesBuffer);
56.                 //ModelRenderSystem.drawers[zoxID.id] = buffer;
57.             }).Run();
58.         }
59.     }
60. }