Suggestions for maintainable AI logic?

I'm diving into AI, and I am running into trouble splitting different behaviors into different systems. I cannot use tagging; I've tried but I have too many units updating too frequently. Now I am just using a switch statement. But its ugly and I am left with a "God job" which processes all behaviors. Normally in OOP I would have different behaviors as classes, but we can't do that in ECS.

I am working in an event system which helps alot with setting behaviors/reading player commands, but my problem is I don't know how I can filter out different units with different behaviors when the time comes to process them. Using one job is an issue for me because as my list of behaviors grows, I am needing more and more component fields..

I also tried tagging "higher level" behaviors, and just polling sub-states within those, but many of these sub-states still use a large variety of components so it doesn't really fix my problem. Any suggestions or threads welcome!

Here's an example of what the job looks like right now.

Code (CSharp):

1. [BurstCompile]
2. private struct MeleeUnitBehaviourJob : IJobEntityBatch
3. {
4.     [ReadOnly] public float deltaTime;
5.  
6.     [ReadOnly] public ComponentTypeHandle<Translation> translationHandle;
7.     [ReadOnly] public ComponentTypeHandle<Velocity> velocityHandle;
8.     [ReadOnly] public ComponentTypeHandle<TargetPosition> targetPositionHandle;
9.     [ReadOnly] public ComponentTypeHandle<FormationForward> forwardHandle;
10.  
11.     public ComponentTypeHandle<UnitSpeedData> speedHandle;
12.     public ComponentTypeHandle<Acceleration> accelerationHandle;
13.     public ComponentTypeHandle<Rotation> rotationHandle;
14.     public ComponentTypeHandle<UnitAnimState> animationStateHandle; // dont need
15.     public ComponentTypeHandle<UnitBehaviourState> behaviourStateHandle;
16.  
17.     // target enemy
18.     // enemy stats
19.     // unit stats
20.     // potentially many more
21.  
22.     public void Execute( ArchetypeChunk batchInChunk , int batchIndex )
23.     {
24.         var batchTranslation = batchInChunk.GetNativeArray( translationHandle );
25.         var batchVelocity = batchInChunk.GetNativeArray( velocityHandle );
26.         var batchTargetPosition = batchInChunk.GetNativeArray( targetPositionHandle );
27.         var batchForward = batchInChunk.GetNativeArray( forwardHandle );
28.  
29.         var batchSpeed = batchInChunk.GetNativeArray( speedHandle );
30.         var batchAcceleration = batchInChunk.GetNativeArray( accelerationHandle );
31.         var batchRotation = batchInChunk.GetNativeArray( rotationHandle );
32.         var batchAnimState = batchInChunk.GetNativeArray( animationStateHandle );
33.         var batchBehaveState = batchInChunk.GetNativeArray( behaviourStateHandle );
34.  
35.         for ( int i = 0; i < batchInChunk.Count; i++ )
36.         {
37.             var position = batchTranslation[ i ].Value;
38.             var forward = batchForward[ i ].Value;
39.             var target = batchTargetPosition[ i ].Value;
40.             var velocity = batchVelocity[ i ].Value;
41.  
42.             var rotation = batchRotation[ i ].Value;
43.             var acceleration = batchAcceleration[ i ].Value;
44.             var speed = batchSpeed[ i ];
45.             var behaveState = batchBehaveState[ i ];
46.             var animState = batchAnimState[ i ];
47.  
48.             // many of these states rely on similar components in order to run, and I can have 100+ states, and there would be dozens of component type handles
49.             switch ( behaveState.MainState )
50.             {
51.                 case ( int ) UnitMainState.Form:
52.                     switch ( behaveState.SubState )
53.                     {
54.                         case ( int ) UnitFormState.Idle:
55.                             UnitBehaviourLogic.ProcessFormIdle( position , forward , target , ref rotation , ref speed , ref behaveState , ref animState );
56.                             break;
57.                         case ( int ) UnitFormState.Rotate:
58.                             UnitBehaviourLogic.ProcessFormRotate( deltaTime , position , forward , ref rotation , ref behaveState , ref animState );
59.                             break;
60.                         case ( int ) UnitFormState.Shift:
61.                             UnitBehaviourLogic.ProcessFormShift( position , target , ref acceleration , ref behaveState , ref animState );
62.                             break;
63.                     }
64.                     break;
65.                 case ( int ) UnitMainState.March:
66.                     switch ( behaveState.SubState )
67.                     {
68.                         case ( int ) UnitMarchState.Start:
69.                             UnitBehaviourLogic.ProcessMarchStart( position , target , velocity , ref acceleration , ref rotation , ref behaveState , ref animState );
70.                             break;
71.                         case ( int ) UnitMarchState.Rotate:
72.                             UnitBehaviourLogic.ProcessMarchRotate( deltaTime , position , target , ref rotation , ref behaveState , ref animState );
73.                             break;
74.                         case ( int ) UnitMarchState.SpeedUp:
75.                             UnitBehaviourLogic.ProcessMarchSpeedUp( deltaTime , position , target , speed , ref rotation , ref acceleration , ref behaveState , ref animState );
76.                             break;
77.                         case ( int ) UnitMarchState.March:
78.                             UnitBehaviourLogic.ProcessMarchMarch( deltaTime , position , target , speed , ref acceleration , ref rotation , ref behaveState , ref animState );
79.                             break;
80.                         case ( int ) UnitMarchState.SlowDown:
81.                             UnitBehaviourLogic.ProcessMarchSlowDown( velocity , position , target , ref acceleration , ref behaveState , ref animState );
82.                             break;
83.                     }
84.                     break;
85.             }
86.  
87.             batchRotation[ i ] = new Rotation { Value = rotation };
88.             batchAcceleration[ i ] = new Acceleration { Value = acceleration };
89.             batchSpeed[ i ] = speed;
90.             batchBehaveState[ i ] = behaveState;
91.             batchAnimState[ i ] = animState;
92.         }
93.     }
94. }

 

For anyone interested, I've decided to keep a list of lists which can be indexed by an integer state. This way I can run a job on each list representing a state, and this allows easy adding of states.

 

I'm yet to see any good alternative to Utility AI for DOTS (in particular Infinite Axis Utility AI)

I would highly recommend reading into it.

 

Would you mind sharing how many agents you had active while you were trying a tagging approach?

We have been using tagging in our ai solution. We end up tagging thousands of agent each frame, and it’s been stable up above 50k agents (near 100k agents, other parts of DOTS start to break down, so we haven’t been able to test above that).

Honestly, tagging like this is something we never expected to work. We were surprised when it did, but it’s been reliable so far.

Hopefully we aren’t about to run into an unforeseen problem that you encountered.

 

I am stress testing around 50k units as well. My issue is that sometimes all 50k (or rather 25k for one team) may receive an order at once, for example telling the entire army to move, and that's 25k tags.

Otherwise on average there would be around a few hundred to five thousand state changes per frame, and that causes spikes of 1-5 ms.

95% + of my game I have managed to get fully multithreaded so far, so I don't really have anything to fill that gap caused by the single threaded tagging.

Also, my entities are pretty large in memory so maybe that is why as all the components have to be moved as well.

Oh also, I am using tagging for other parts of my game, such as using tagging for unit formations AI, projectiles, etc, so I already have a few ms taken up by that. No room for more.

 

My game is structured where individual units are grouped into larger formations. I am looking to use many ai approaches including utility AI on those, but right now I am talking about the individual units which have a simpler ai model better represented by states.

 

I'll second Utility AI. I haven't started coding it just yet, but I have the framework planned out and it is the next thing on my list. The design seems to fit very well with ECS. I only have a few minor concerns about minimizing component additions and removals.

 

Audio is a good candidate to fill this gap. Sampling can get pretty expensive when you have thousands of audio sources.

You might want to consider utility trees. The idea is that each behavior leaf node is evaluated and generates its own unique payload of actions and utility values. Then parent nodes evaluate multiple payloads and export their own payload, either propagating relevant data from the chosen child, or perhaps even mixing the children. This continues up to the root which then outputs the actions the AI should take. While this approach doesn't generalize well since you have to write code for all the nodes, it does map very cleanly to components and systems, and is consequently very easy to get up and running for simple AI models. It also can model a state machine as you just need a parent node to propagate the output of the correct child state.

But if you want to use a state machine with structural changes, then using entities as states and instantiating and destroying those state entities to switch states will save you on structural change costs.

 

Thanks for the reply. A few thoughts:

A brute force approach for 50,000 agents like that might be a bit of an ask for even the most powerful game ai on the planet. At least today, on a home computer with good framerates. Just saying - worth considering that this is boundary-pushing territory.

It might be worth it to consider some AI optimizations to yield the same visual result. For example:

If your units are grouped into formations, perhaps the formations themselves could be agents. The formation agents are the ones with the “move” behavior, and the individual units always take direction queues from whatever formation they are grouped under.

in that scenario, your individual soldiers don’t need to get new behaviors - just their formations. If you have 50k units, and each formation has 100 units, that’s down to 500 agents receiving the new behavior at once.

 

I have decided to go with a sort of hybrid approach. Thoughts on what I came up with?

So I have an enum of states. Per frame, one job runs for each state, processing the current behavior. If an end condition is met, a flag is set.

Then periodically, a system checks all entities to see if their current state is finished. If so, a utility tree runs to see what the next best state should be. Units can have lists of states to complete for more complex behaviors, and if the current sequence takes precedence over other actions, the current state is set to the next one in the list. Otherwise, the list is cleared and a new state or sequence is added.

Also, states can be overridden by events per frame, such as taking a hit or dying. I also have "soft" events, which are factored into the utility tree as extra parameters, such as nearby enemies. These soft events can also break the unit out of a state even if conditions aren't met.

 

Actually without doing structural changes 50,000 agents is just fine for me.

I do have formation behaviors, which are more complex as well. The reason I still have per agent ai, is because let's say a formation is in battle. Many units are not fighting, and so are in a "ready" state, not "combat". Also units can get separated from their formations, so those agents would have a "marching" state to catch up to the formation which may be idle. Having only the formations as agents makes the game feel "gamey" if that makes sense.

I posted the approach I think I am going with right above if you're interested.

 

This makes it sound like you’ve standardized the data layout for all of your behaviors, so that they can all be represented by a single struct type. Is that correct?

 

It sounds like you came up with something that matches your mental model of what you want and is also potentially ECS-friendly. If it does what you want and improves performance, then it is a definite win.

 

My data layout is like this so far.

Code (CSharp):

1. public struct UnitAIData : IComponentData
2. {
3.     public int CurrentState;
4.     public float StateTimer;
5.     public bool Loop;
6.     public bool FollowingSequence;
7.     public bool ConditionMet;
8. }
9.  
10. public struct UnitStateSequence : IBufferElementData
11. {
12.     public DynamicBuffer<UnitBehaviourStates> States;
13. }
14.  
15. public struct UnitBehaviourEvents : IBufferElementData
16. {
17.     public DynamicBuffer<UnitBehaviourEventType> Events;
18. }
19.  
20. public struct UnitInteractionEvent
21. {
22.     public Entity Entity;
23.     public Entity OtherEntity;
24. }
25. public struct UnitBehaviourEvent
26. {
27.     public Entity Entity;
28.     public UnitBehaviourEventType Type;
29. }

CurrentState references an enum, holding all states. ConditionMet is determined in the jobs which process each state. It can rely of the state timer, some other condition, or both. The buffer allows me to represent an "action", for example attack another unit can be the states "Rotate, March, Enter Combat". Or an action can simply be a single state. With the event system, I found this allows basically anything to be modeled.

For example, if an agent has the AttackUnit sequence, but the higher level system detects another enemy very close, a "NearbyEnemy" event will be triggered. This is a soft event, so the unit will finish its "Rotate" state, then clear the sequence and update to combat.

A "hard" event like being attacked will immediately break the current state.

An event like being shot can insert a state into the current sequence, so it will play "Hit" animation then return to whatever agent was doing.

All the higher level decision making is done by the tree and events, so it neatly separates states and decision making. The only downside so far is each state is very specific, so around 100 states, and I need a separate job for each. Similar to tagging but instead its just a NativeList<UnsafeList<Entity>>, and each UnsafeList represents one state holding all entities in that state.

On state change, the entity is simply removed from current list and added to appropriate list.

 

I appreciate the breakdown, thank you!

It took me a second to grok, since you’re using vocabulary here different from how I typically hear it used.

It sounds like in your model, a behavior can have one or more states. I’m used to FSMs being used in which states have one or more behaviors, and those behaviors become active when the agent transitions to that state.

For instance, if an agent is in a “defensive” state, it would respond differently to events than it would if it were in an “offensive” or “retreat” state, etc. I’m used to states being used to switch up how an agent is currently behaving at a high level. It sounds like you’re using that word to describe the different phases, or sub-tasks, of an action.

But it sounds like a semantic difference. There’s no right or wrong way to build a state machine, as long as it does what you need. I’m glad this is working for you, and that you found a solution!

 

Apologies I am kinda mixing and matching terms here!

A state is what the agent is doing right now, while a behavior can be comprised of one or more states in sequence. Any state can switch to any other state, but for convenience certain behaviors are set sequences of states. But an agent can break out of that sequence into another behavior regardless of its current state.

 

Tiny suggestion: If it’s always true that the ‘currentState’ represents an action the agent is currently performing, then it might be worth simplifying your terminology by renaming 'currentState' to 'currentAction'.

...And in general, replacing the word "state" with "action" in all of your variable and type names.

Conceptually, the agent's "state of being" is always that they're performing an action. And the 'currentAction' variable describes what that action is.

I'm used to this concept being described as "macro" and "micro" behaviors. Or "tasks" and "subtasks". For example: "Completing a macro behavior can involve completing one or more micro behaviors."

A neat design trick is that there doesn't really need to be a structural difference between a macro and micro behavior. They can both just be considered the same type of thing - a 'behavior'.

For example: AttackBehavior's functionality could just be to schedule a new series of upcoming behaviors:

1. RotateBehavior
2. MarchBehavior
3. EnterCombatBehavior

...and then complete itself.

What a change like this accomplishes is to remove the need for there to be a separate "sub-task" type of thing in your ai model. Now, every behavior is just one type of thing - a 'behavior'.

The code doesn't need to care that 'AttackBehavior' represents a higher level of granularity than the other behaviors it schedules. All the code needs to know about AttackBehavior is that it's a behavior, which means it does something, and then finished based on a condition - just like all other behaviors. It's probably fine if only the humans are aware of a concept like the 'level of granularity' of a behavior - knowing that isn't going to make the program run any faster, but it will add complexity.

I find that whenever you can flatten out a model by reducing the # of different types, the new simplicity is often a benefit.

Of course, ignore that if I'm off base, or if I'm misunderstanding your model.

 

Excellent suggestion! I will do that; it does make the model simpler and more generic.

 

Good luck!

 

This is likely something that you've already considered/implemented but I'll mention it just in case - if by tag components you mean zero-data components you could take advantage of the

.AddComponent()

and

.RemoveComponent()

overloads which take

EntityQuery

instead of individual entities. As I understand it this is much more performant as only the chunk's archetype will change and no entities need to be moved.

If the entities in your armies are grouped in such a way that they have their own queries then you could take advantage of this to bring down those spikes.

 

You know, I’ve never found a good use for that API. It’s very rare that I know ahead of time that I’ll want to add the same tag to every entity in a given chunk. It seems like an unusual edge case, unless you manually group your entities in specific chunks just to take advantage of it.

 

You pretty much hit the nail on the head with the second part. I find that designing with the intention of using the EntityQuery overload pretty much always introduces chunk fragmentation. I most typically find a use for it after I've decided to split up entities into different archetypes and even then, as you mention, its pretty rare that you know that every single entity in a chunk will be moved

 

Forgive me for bringing this thread back up, but I have been reflecting a bit. I looked into Utility AI but it does not seems to match what you are suggesting. What I get from your suggestion is to have multiple behavior trees and run a utility AI to pick the most appropriate tree, then run the tree. Is this correct?

 

Utility trees are a pretty uncommon type of utility AI as they don't really generalize well and are better suited for when you have a very specific AI model in mind. A behavior tree uses a root-first depth-first traversal and do not evaluate every node every tick. In contrast, a utility tree is a leaf-first breath first traversal in which every node is evaluated every tick.

 

I see, thanks for the clarification!

 

I made a state machine that takes into consideration the time it takes for a task to complete (with an override for interruptions), so that given time NPCs will not have to do an AI update as frequently, and preferably not on or near the same frame range. The minimum time between tasks is 250ms so a queue can have a maximum of four tasks per second. There is no need to do AI checks faster than (within the suspension of disbelief) the NPCs can react.

 

You could try having the decision-making be done by a Monobehaviour Asset while the agent logic and persistence done in ECS. I wrote a post about this strategy at https://forum.unity.com/threads/gui...monobehavior-decision-making-modules.1094935/

 

Upon trying out a behavior tree for fun in DOTS, and trying state machines as well, I found out Utility AI really does seem the way to go. However, I still do not understand why I would use a utility tree over a normal utility AI. Sorry for necro..

 

You might not.

I have found them pretty useful for breaking down purpose-built AI into more manageable subfunctions, but besides that and being easy to implement in ECS, there's not much going for them. If more traditional utility AI works better for you, by all means go with that.