Converting a OOP system to DOTS - suggestions please..

Hi i currently have a UnitTask system written using typical oop inheritance etc, and currently looking how to transfer this over to a DOTS/ECS solution..

Currently i have a Task class, with different task types inheriting from this - GotoTask, WaitTask, etc. I queue up tasks for each unit, so each unit could have say a Goto and a Wait task - when Goto is complete, it starts the Wait task.

Obviously this type of inheritance isn't possible with structs so i'm looking for ways to refactor this system in ECS. Currently my thought is i could have a GotoTaskData dynamic buffer and a WaitTaskData dynamic buffer, along with a TaskSchedule component on each unit entity. The TaskSchedule would have a simple NativeArray of TaskType to specify the order of execution. So the TaskSystem would loop each entity TaskSchedule, check what the first TaskType is in the Taskchedule.tasks array, and then pass the first task from the relevant GotoTaskData buffer or WaitTaskData buffer into a static method which processes the task ( and if task is complete removes the element from the buffer and also the task list ).

Does this sound like a good approach and/or how else could i approach this? Bear in mind i'm pretty new to Unity DOTS implementation so learning as i go here..

Here's my rough idea in code form:

Code (CSharp):

1.  
2. protected override void OnUpdate()
3. {
4.        Dependency = Entities
5.               .ForEach((
6.                      ref TaskSchedule taskSchedule,
7.                      ref DynamicBuffer<TaskData_WaitTask> waitBuffer,
8.                      ref DynamicBuffer<TaskData_GotoTask> gotoBuffer
9.               ) => {
10.                      switch (taskSchedule.tasks[0])
11.                      {
12.                             case TaskType.GOTO:
13.                                    TaskData_GotoTask gotoData = gotoBuffer[0];
14.                                    gotoBuffer[0] = ProcessGoto(ref taskSchedule, gotoData);
15.                                    break;
16.                             case TaskType.WAIT:
17.                                    TaskData_WaitTask waitData = waitBuffer[0];
18.                                    waitBuffer[0] = ProcessWait(ref taskSchedule, waitData);
19.                                    break;
20.                      }
21.               }).ScheduleParallel(Dependency);
22. }
23. public static TaskData_GotoTask ProcessGoto(ref TaskSchedule taskSchedule, TaskData_GotoTask gotoData)
24. {
25.        // do some stuff //
26.        return gotoData;
27. }
28. public static TaskData_WaitTask ProcessWait(ref TaskSchedule taskSchedule, TaskData_WaitTask waitData)
29. {
30.        // do some stuff //
31.        return waitData;
32. }
33.  

 

(apologies in advance for the immense wall of text, this is a big complex subject!)

You've landed on exactly the kind of problem that tends to be tricky to implement in an ECS architecture: calling functions of varying types in a sequence, where the sequence of things is arbitrary & cannot be pre-determined by archetype (or would be too expensive to arrange by dynamically changing archetypes). And unfortunately, this problem isn't that uncommon in games

There are a few approaches I can suggest, but no clear winner:

----------------------------

Spoiler: Approach A : TaskWorld (another switch-case approach)

One solution for this use case could look like this:

• Each task is an Entity with a TaskId component (remembers what kind of task it is), and a component meant to hold the specific data of that task type.
• Create a "TaskWorld" struct, which is a struct that's meant to be passed to a bursted job and whose role is to contain all the data & logic needed to update tasks. Somewhat similar in concept to "PhysicsWorld". Its end goal is to be able to update a task by simply passing the task entity to a method.
• "TaskWorld" has a ComponentDataFromEntity<T> of all possible task components, and one for the TaskId component as well
• "TaskWorld" also holds ComponentDataFromEntity<T> for all possible components that tasks in the game could need read/write access to
  
• "TaskWorld" has a ProcessTask(Entity taskEntity) function, which gets the TaskId component on the task entity, and then does a switch case on that Id, and gets all the right components to do the task processing update

So with this approach, a job that updates tasks for units would look like this:

• Each unit has a DynamicBuffer<TaskEntity> to represent its ordered queue of tasks
• Build a TaskWorld struct and pass it to the UnitTaskUpdateJob before scheduling
• In the job, for each unit, get the DynamicBuffer<TaskEntity> and call TaskWorld.ProcessTask(tasksBuffer[0])

If it weren't for all the boilerplate code & manual management necessary to create the TaskWorld struct, this would be the most "hassle-free" solution because it does not rely on any sort of structural change or sync point, and can be used to update all possible task types in a single job. If we eventually get SourceGenerators to help us codegen stucts like this "TaskWorld", this kind of problem will become much easier to deal with in DOTS

----------------------------

Spoiler: Approach B : Structural changes

Another alternative, which could be either much better or much worse for performance depending on your specific game's needs, is to use structural changes to determine which tasks should be updating right now:

• Each Task is an entity with these components:
  • Task (holds a "TargetEntity" field and a "IsComplete" field)
  • the specific task Component (GotoTask, WaitTask, etc...)
• A unit has a DynamicBuffe<TarskEntity> which represents its task queue
• When you want the unit to start a task, you add an "ActiveTask" component on the task entity
• Each task type has a job that iterates on entities that have the "Task" component, the specific task component (GotoTask, WaitTask, etc...), as well as the "ActiveTask" component. This job updates the task, and remembers to set "IsComplete" to true in the Task component
• After all specific task jobs, a general task job iterates on all entities that have the "Task" + "ActiveTask" component. This job checks if "IsComplete" is true, and if yes, it Destroys self entity, removes itself from the DynamicBuffer<TaskEntity> on the target entity, and adds a "ActiveTask" component to the next task entity in the queue

This alternative is more efficient if you're okay with having a maximum of 1 task that gets completed per unit per frame. But it could become waaay less efficient if you have queues of Tasks that are instantaneous, and you want all subsequent instantaneous tasks in the queue to be processed in one frame. On each frame, you'd have to reschedule all of your task jobs & recreate sync points for as many times as there are Tasks in your queue, and the performance hit of this would just be ridiculous. You'd basically need a system that does the following:

Code (CSharp):

1.  
2. protected override void OnCreate()
3. {
4.     // Sync point
5.  
6.     bool hasAnyRemainingTasks = true;
7.     while (hasAnyRemainingTasks)
8.     {
9.         // Schedule the jobs of all possible Task types in your game
10.  
11.         // Sync point. Playback the EntityCommandBuffers that are supposed to handle task entity destruction and "ActiveTask" component adding
12.  
13.         // Iterate on all DynamicBuffer<TarskEntity> and set "hasAnyRemainingTasks" to true if at least one buffer has a Length > 0. Set to false otherwise
14.     }
15. }
16.  

Depending on the max quantity of tasks in your queues (Q), the quantity of task types in the game (G), and the variety of different types of tasks that are currently needed for processing, this could result in having to schedule up to Q*G amount of jobs and create Q amount of sync points every frame. It could very quickly become too heavy.

----------------------------

Spoiler: Approach C : OOP within the ECS project

Another solution is to just use OOP within an ECS project. You can accomplish this without even requiring GameObjects/Monobehaviours; by using class-based IComponentData components. These components live on entities but are allowed to contain managed types.

That means you can do full OOP programming on the main thread with them, but they are still assigned to an Entity, and you can still read/write data to components on those entities from your OOP code (using the EntityManager). It is OOP, but it's still playing nice with the ECS structure of the game. It doesn't create this whole other structure that exists in addition to the ECS like a hybrid GameObjects approach would

Your units would, for example, have a UnitTasks managed component containing a List<Task>, where "Task" is an abstract class. You'd have a system that creates a sync point, and updates all entities that have a "UnitTasks" component on the main thread with a job launched with .Run()

----------------------------

Spoiler: Approach D : Generalized Task BufferElement

• Create a new IBufferElementData that is simply named "Task". This is a struct that can function as any different kind of task in the game, depending on the data that you give it
• "Task" has an enum/int field to determine what type of task it is
• "Task" has fields to accomodate all the possible data that all tasks in the game might need
• "Task" has a ProcessTask(ref data) method, which does a switch-case on the task id & does the task processing logic accordingly. The "ref data" represents all the data that all possible task types might need read/write access to (there could be a bunch of ComponentDataFromEntity<> in there, for instance)
  • If you don't like having methods inside components, ProcessTask(ref Task, ref data) could also just be a static utility function

This approach is similar to what I've described here: DOTS Polymorphic Components

----------------------------

Spoiler: Rejected approaches

There are other approaches that could work, but have some serious pitfalls.

Tasks as serialized byte arrays
Any kind of attempt at representing your task queue as a DynamicBuffer<Byte> or NativeStream that can be deserialized (much like receiving network messages from a server and deserializing them to their respective types based on an ID) will make it impossible for tasks to safely keep references to entities. This is because Entity fields have special handling when structural changes are done. They can be updated and "remapped". If you serialize your Entities as a series of bytes, I doubt that entity remappers will be able to update them

Task function pointers
I haven't used DOTS function pointers much, but the docs seem to suggest that you can't pass native collections as parameters to function pointers. This would severely limit what tasks are allowed to do in their processing, if each task type had its function pointer. For instance: no ComponentDataFromEntity<> access, no PhysicsWorld access, etc... (although I'd need confirmation on that to be sure)

It seems it would accept being passed an entire job as parameter, but then your tasks would be tied to that specific job only. Besides, the function pointer approach doesn't really have notable advantages compared to approaches A or D

-----------------------------------------

Spoiler: Conclusion & comparisons

The conclusion is that each solution has pros & cons

TaskWorld/switch-case approach

• Pros:
  • No sync points or expensive structural changes needed
  • Can process all task types immediately in a single job (this can help keep your code simple)
  • Could have superior performance if you need to process more than one task per queue in a frame
• Cons:
  • Lots of random memory access
  • If TaskWorld happens to have a ComponentDataFromEntity of the same type as one that is already present in the job where you use the TaskWorld, burst will give you aliasing errors
  • Without codegen, all tasks kinda have to be implemented in the same file to some extent. It can be a pain to maintain this code

Structural changes approach

• Pros:
  • Updates tasks in a proper high-performance DoD fashion
  • Each task is implemented individually in its own system
• Cons:
  • Could require way too many sync points & jobs to schedule if you need to process more than one task per queue in a frame
  • The cost of structural changes & scheduling could end up making the performance of this approach worse than the TaskWorld approach, even though TaskWorld does random memory access all over the place

Main Thread OOP approach

• Pros:
  • Easiest to use & setup
• Cons:
  • Stuck to main thread, unless you maybe come up with your own multithreading that doesn't use the Job system
  • Can't use Burst, and things that want to add tasks to units can't be bursted either
  • Probably has the worst performance in most cases... (we'd have to profile and see, though). But it could still be worth it if your tasks update isn't a huge portion of the frame cost anyway

Generalized Task BufferElement approach

• Pros:
  • Same pros as TaskWorld, except it would have better performance because we need less ComponentDataFromEntity<>, and we also don't have to create Entities when adding tasks
• Cons:
  • Could be even more tedious to maintain & work with than TaskWorld. Especially because the fields of the "Task" struct need to be some kind of union of all possible fields that all tasks might need. Moreover, entity remapping makes it difficult for tasks to hold Entity fields, if you want to solve this using union structs with explicit struct layouts
  • Also a bit more tedious than TaskWorld because when using this approach, you have to pass all the ComponentDataFromEntity to the function manually. With TaskWorld, it all comes built-into the struct

Among the ECS approaches, I'd say approaches A and D might be considered the safest default choices, because even though they don't give you top possible performance in the best-case scenario, they do perform better by a large margin in the worst-case scenario

 

Thanks for taking the time to reply Phil. Going to take some time to digest what you've said as i'm pretty sure you're thinking two or three levels above me in the Unity dots world..

Actually i've been following your DOTS work on and off for quite a while looking for an entry point into using it myself, one of the things in particular being your work on skinned animation. Rival is superb i've successfully used it to DOTSify my AI allowing me to spawn in several thousand where previously i was limited to 1-200 with oldschool SkinnedMeshRenderer and Animator, which is just incredible. So this is me literally now seeing if i can refactor more of my systems to DOTS and fully integrate it into my project. It's quite a big step however, i've written as much of my code as i can using as few mono's as possible, everything is run with managers ( projectile manager, fx manager, ai movement, ai logic, all of it ) to try and mimic some of the DoD ideas and get some of the benefits. Now that i'm actually looking at writing DOTS code it's a whole other level..

 

Best approach for AI so far for me was to go hybrid.

Evaluate AI tree / tasks on the MonoBehaviour side (or you can use non-bursted system), and apply components to the related entities for performing actions. Components can be added via ECB's so this do not block your main thread as much.

In general, it shouldn't take much of CPU time, even in large quantities of units, if you update them only X times per second. As well as spreading load equally throughout multiple frames.

 

Thanks for the reply Vergil yeah tbh my existing manager class system is fairly quick already, and i think i've jumped in at the deep-end by choosing to DOTSify the AI task system, but i guess it's a good way to learn and if i can get blazing-quick as opposed to just quick then happy days

 

So i've managed to get this up and running. I've pretty much ran with the idea i had with a couple of Phils suggestions rolled in. I'll definitely be referring back to Phils comments moving forward as there's a wealth of stuff in there..

I wanted to avoid structural changes to Units as tasks are changed/replaced pretty often so the basic structure is this:

Each Unit entity has:

• a TaskType buffer, simple enum array that holds the order of task types, first in list is always the active task.
• a dynamic buffer for each Task type - for now it's just GotoTask and WaitTask - these just hold the basic data required for the task and can be stacked up to chain together tasks in coordination with order of the TaskType buffer
• a PathData component - this holds current pathfinding waypoint data for Unit movement independent of any tasks
• I have a single PathRequest entity which holds a buffer of PathRequests for all Unit entities. Goto tasks add new requests to this when a Goto task is initialized.
• I have an AIPathSystem which processes the path requests through my 3rd party pathfinding plugin, and when completed paths are received by the system it updates the Entity's PathData component with the waypoints. The AIPathSystem then runs an update on all entity paths to traverse waypoints and push the movement vectors into the Unit MovementInput system.

Here is how some of that code looks:

First job in the UnitTaskSystem initializes any tasks that have arrived in the buffer, sending requests to the global PathRequests buffer if needed:

Code (CSharp):

1.  
2.            
3. var pathRequestEntity = GetSingletonEntity<PathRequestEntityComponent>();
4. var pathRequestBuffer = EntityManager.GetBuffer<PathData_PathRequests>(pathRequestEntity);
5.  
6. Dependency = Entities.
7.         ForEach((Entity e, ref DynamicBuffer<TaskData_TaskType> taskTypes,
8.                     ref UnitPathData pathData, ref DynamicBuffer<TaskData_GotoTask> gotoBuffer,
9.                             ref DynamicBuffer<TaskData_WaitTask> waitBuffer, in Translation t) =>
10. {
11.     if (taskTypes.Length > 0)
12.     {
13.         switch (taskTypes[0].TaskType)
14.         {
15.             case (TaskType.GOTO):
16.                 TaskData_GotoTask gotoData = gotoBuffer[0];
17.                 if (!gotoData.inited)
18.                 {
19.                     switch (gotoData.targetData.positionChooserType)
20.                     {
21.                         case PositionChooserType.EXACTPOSITION:
22.                             gotoData.targetPosition = gotoData.targetData.targetPosition;
23.                             break;
24.                         case PositionChooserType.DISTANCEFROM:
25.                             gotoData.targetPosition = GetPositionAtDistanceFromTarget(t.Value, gotoData.targetData.targetPosition, gotoData.targetData.idealDistance);
26.                             break;
27.                         case PositionChooserType.DISTANCEAROUND:
28.                             gotoData.targetPosition = GetPositionAtDistanceAroundTarget(t.Value, gotoData.targetData.targetPosition, gotoData.targetData.idealDistance);
29.                             break;
30.                     }
31.                     pathRequestBuffer.Add(new PathData_PathRequests() { entity = e, startPos = t.Value, targPos = gotoData.targetPosition, allowNearest = gotoData.targetData.allowPathfindingToNearestPosition });
32.                     pathData.HasPath = false;
33.                     pathData.AwaitingPath = true;
34.                     pathData.PathFail = false;
35.                     gotoData.inited = true;
36.                     gotoBuffer[0] = gotoData;
37.                 }
38.                 break;
39.             case (TaskType.WAIT):
40.                 TaskData_WaitTask waitData = waitBuffer[0];
41.                 if (!waitData.inited)
42.                 {
43.                     // Init Wait //
44.                 }
45.                 break;
46.         }
47.     }
48. }).Schedule(Dependency);
49.  
50.  

Second job in UnitTaskSystem processes active Tasks:

Code (CSharp):

1.  
2.            Dependency = Entities.
3.                       ForEach((ref DynamicBuffer<TaskData_TaskType> taskTypes,
4.                                  ref DynamicBuffer<TaskData_WaitTask> waitBuffer,
5.                                             ref DynamicBuffer<TaskData_GotoTask> gotoBuffer,
6.                                                        ref UnitPathData pathData, in Translation t) =>
7.             {
8.                 bool taskFail = false;
9.                 if (taskTypes.Length > 0)
10.                 {
11.                     switch (taskTypes[0].TaskType)
12.                     {
13.                         case TaskType.GOTO:
14.                             gotoBuffer[0] = ProcessGoto(ref pathData, gotoBuffer[0], t.Value);
15.                             if (gotoBuffer[0].complete)
16.                             {
17.                                 taskTypes.RemoveAt(0);
18.                                 gotoBuffer.RemoveAt(0);
19.                             }
20.                             else if (gotoBuffer[0].fail) taskFail = true;
21.                             break;
22.                         case TaskType.WAIT:
23.                             waitBuffer[0] = ProcessWait(waitBuffer[0]);
24.                             if (waitBuffer[0].complete)
25.                             {
26.                                 taskTypes.RemoveAt(0);
27.                                 waitBuffer.RemoveAt(0);
28.                             }
29.                             else if (waitBuffer[0].fail) taskFail = true;
30.                             break;
31.                     }
32.                 }
33.                 if (taskFail)
34.                 {
35.                     taskTypes.Clear();
36.                     gotoBuffer.Clear();
37.                     waitBuffer.Clear();
38.                 }
39.             }).ScheduleParallel(Dependency);
40.  

It's running pretty well so far and i'm quite happy i've been able to get it up and running. Any suggestions appreciated as i'm pretty new to Unity DOTS/ECS at this point!