Enemy AI / Controllers and Delegates/States - help please c#

Hello.

I'd like to start adding enemies in my project however I don't really know where to start.

Almost all the tutorials I've seen and read use States, coroutines and Delegates to perform the AI however it is all extremely confusing and very hard to apply to my situation because;

-I don't have the character models and animations which they have implemented already through code.
-My project game is 2D constrained to the X and Z axis.
-There seems to be a massive leap into states/delegates/coroutines - or at least I havn't found a decent starting point.

So with all those of you who know about this stuff, I'd appreciate it greatly if someone could point me in the right direction, where to start, whether I need them, what I need them for etc.

Ideally if I need to use them I would like to start simple, and apply Animation among other things when I actually have the assets.

Thanks for reading.

 

A lot of the stuff I've seen being done for AI seems like it could be done with lots of normal functions. For a 2D game with basic AI I can't see how states are necessary.

Can someone explain please?

 

States and a State Machine decouple AI agent behavior while also making it re-usable. It also provides excellent separation of duties which makes debugging your AI much easier. For example - if your agent isn't moving around your world correctly then you know you need to look in only one place - your move state. What's more, if you decouple steering from move state then you can use the move state for any AI agent that moves in your entire game. It also gives you a finite start and end point to a state which means attaching handlers to those events is much simpler.

A state machine is also very extensible. If you need to add more functionality to your AI it's just a matter of creating new states and telling your agent when to use them. You don't have to touch any existing state code.

 

Thanks KelsoMRK - I realise that I will most likely need to use them in my game.

However I still don't know where to start, I've read about some FSM frameworks, which I believe is basically the functionality behind switching states and such. However lets say I have a framework. Can I then create all these types of states :

-Walking
-Dead
-Attack
-Waiting
-Guarding
-Chasing

They are self explanatory. But I really don't know where to start and what framework to use or if I have to make my own etc.

Any suggestions?

 

I execute my state logic as an IEnumerator that gets wrapped in an agent's coroutine. This is mainly because Unity doesn't let you execute Coroutines in classes that don't extend MonoBehaviour. So - a very basic example to get started:

Every state implements a common interface with these methods:

Code (csharp):

1.  
2. public interface IState
3. {
4.     void Enter();
5.     System.Collections.IEnumerator Execute();
6.     void Exit();
7. }
8.  

The state machine:

Code (csharp):

1.  
2. public class StateMachine
3. {
4.     private Agent owner;
5.     private IState currentState;
6.  
7.     public StateMachine(Agent owner)
8.     {
9.         this.owner = owner;
10.     }
11.  
12.     public void ChangeState(IState newState)
13.     {
14.         if (this.currentState != null)
15.         {
16.             this.currentState.Exit();
17.         }
18.         this.currentState = newState;
19.         this.currentState.Enter();
20.     }
21. }
22.  

The Agent that owns an instance of a state machine. It has a coroutine that wraps the current state's execution

Code (csharp):

1.  
2. public class Agent : MonoBehaviour
3. {
4.     public StateMachine FSM = new StateMachine(this);
5.  
6.     public IEnumerator StateWrapper(IEnumerator state)
7.     {
8.         while (state.MoveNext())
9.             yield return state.Current;
10.     }
11. }
12.  

A simple state that implements our interface

Code (csharp):

1.  
2. public class SomeState : IState
3. {
4.     protected Agent owner;
5.  
6.     private int counter;
7.  
8.     public SomeState(Agent owner)
9.     {
10.         this.owner = owner;
11.     }
12.  
13.     public void Enter()
14.     {
15.         Debug.Log("Entering Test State");
16.         owner.StateCoroutine("StateWrapper", this.Execute());
17.     }
18.  
19.     public IEnumerator Execute()
20.     {
21.         while (counter < 10)
22.         {
23.             Debug.Log("Executing State");
24.             counter++;
25.             yield return null;
26.         }
27.         this.Exit();
28.     }
29.  
30.     public void Exit()
31.     {
32.         Debug.Log("Exiting Test State");
33.     }
34. }
35.  

We can change state in the agent by doing

Code (csharp):

1.  
2. this.FSM.ChangeState(new SomeState(this));
3.  

If we call the above line before SomeState had finished counting to 10 you would still see the "Exiting State" statement in the console because Enter and Exit are always called when changing states. I chose to also call it when Execute had finished but there's nothing saying you have to. You could switch to a different state and Exit would still run.

 

Maybe you want to start out a bit more simple..

Code (csharp):

1.  
2.  
3. public enum Behavior
4. {
5.     idle, search, confused, moveToPlayer, wander, attack, useAbility
6. }
7.  
8. public class EnemyAI : MonoBehaviour
9. {
10.     public Behavior currentState = Behavior.idle;
11.     private Behavior nextState = Behavior.idle;
12.     private Transform playerTarget;
13.  
14.     public int playerLayer = 1<<8;          //Just saying the player layer is on 8
15.     public float viewDistance = 5;          //Distance they can see in front of them.
16.     public float viewAngle = 45;            //Angle they can see a player infront of them
17.     public float awareDistance = 1;         //Aware of player if they are in this distnace
18.  
19.     public float attackDistance = 2;        //Distance at which the player can attack
20.  
21.     private bool isStateFinished = true;
22.  
23.     //Maybe the player stunned the enemy..
24.     private bool interuptState = false;
25.     public bool InteruptState
26.     {
27.         get{return interuptState;}
28.         set{interuptState = value;}
29.     }
30.  
31.     public void Update()
32.     {
33.         if(currentState != nextState  (isStateFinished || interuptState))
34.         {
35.             currentState = nextState;
36.             isStateFinished = false;
37.             switch(currentState)
38.             {
39.                 case Behavior.idle:
40.                     Idle();
41.                     break;
42.                 case Behavior.search:
43.                     Search();
44.                     break;
45.                 case Behavior.confused:
46.                     Confused();
47.                     break;
48.                 case Behavior.MoveToPlayer:
49.                     MoveToPlayer();
50.                     break;
51.                 case Behavior.attack:
52.                     Attack();
53.                     break;
54.                 case Behavior.useAbility:
55.                     UseAbility();
56.                     break;
57.             }
58.         }
59.        
60.     }
61.  
62. #region Behavior methods
63.     private void Idle()
64.     {
65.         interuptState = true;
66.         List<GameObject> viewedPlayers = CheckIfPlayerInView();
67.         if(viewedPlayers.Count > 0)
68.         {
69.             playerTarget = viewedPlayers[0];    //Grab the first.. Just for an example.. Maybe it would check distance and get the closest.
70.             currentState = Behavior.Move();
71.         }
72.         isStateFinished = false;
73.     }
74.  
75.     private void Search()
76.     {
77.         //Randomly move around and search?
78.     }
79.  
80.     private void Confused()
81.     {
82.         //Could be enabled by player attack/ability
83.     }
84.    
85.     private void MoveToPlayer()
86.     {
87.         //Move to the player's location
88.         //Check if the player is in range
89.     }
90.  
91.     private void Wander()
92.     {
93.         //Move to a random legal location
94.     }
95.  
96.     private void Attack()
97.     {
98.         // animate attack..
99.         // apply damage to the player
100.     }
101.  
102.     private void UseAbility()
103.     {
104.         //Enable some particle effect and apply dmg to the player.
105.     }
106. #endregion
107.  
108. #region Behavior Checking methods
109.     private List<GameObject> CheckIfPlayerInView()
110.     {
111.         List<GameObject> resultList = new List<GameObject>();
112.         hit = Physics.OverlapSphere(this.transform.position, viewDistance, playerLayer);
113.  
114.         foreach (Collider c in hit)
115.         {
116.             if (Vector3.Angle(this.transform.forward, c.transform.position - from.position) <= viewAngle)
117.             {  
118.                 //Check if alive.. or whatever else before adding.
119.                 resultList.Add(c.gameObject);
120.             }
121.            
122.         }
123.         return resultList;
124.     }
125. #endregion
126. }
127.  

 

Thanks for this example Kelso its helping my understanding of all this. so if you had a state called Dead you would call it at some point like;

Code (csharp):

1.  
2. if(PlayerHealth <= 0)
3. {
4. // all the stuff that happens when an entity dies in the game will happen in the State Dead instead of normally being here
5. this.FSM.ChangeState(new Dead(this));
6. }
7.  
8.  public IEnumerator Dead()
9.  {
10. // something died, go through all the timed things and events that are supposed to happen when something dies
11.  
12.  
13. }
14.  
15.  

is something like that correct ?

and the IEnumerator Dead() .. if I kept this separate from all my enemy scripts that means I could use the same script that handles death for any Enemy in the game right?

 

Just reading through what you posted now, thanks Eiznek.

 

@Eiznek- Simpler - but not a state machine and if your logic got complex that code would become a maintenance disaster. You're also polling state every frame regardless of what it is.
@Ovredon- Exactly.

 

I'm kind of getting it, but to see it working in action with that example would be great Eiznek, atm its got 4 errors, I think 2 of them are typos but I can't figure out the other 2. I'd like to get this in unity working to help me understand it better, I just don't get how any of those functions or where they are called atm though :\

Unless that is the framework and the states are called from a different class?

 

KelsoMRK, I'm trying to use your example, however I'm getting an error that says 'this' is not available in the current context

public StateMachine FSM = new StateMachine(this);

Code (csharp):

1.  
2. using UnityEngine;
3. using System.Collections;
4.  
5. public class Agent : MonoBehaviour
6.  
7. {
8.  
9.     public StateMachine FSM = new StateMachine(this);
10.  
11.  
12.  
13.     public IEnumerator StateWrapper(IEnumerator state)
14.  
15.     {
16.  
17.         while (state.MoveNext())
18.  
19.             yield return state.Current;
20.  
21.     }
22.  
23. }
24.  

line 8

 

Sorry. Move it to Awake.

Code (csharp):

1.  
2. public class Agent : MonoBehaviour
3. {
4.     public StateMachine FSM;
5.  
6.     void Awake()
7.     {
8.         this.StateMachine = new StateMachine(this);
9.     }
10. }
11.  

The StateMachine actually doesn't have to know about its owner - only the individual states do. So including it is up to you.

 

Okay thanks!

I appreciate the help you've gave me Kelso cheers.

I just want to say that I read up tons of stuff before coming asking questions on the forums, I must have read about 6 articles on States/Delegates/AI and how they all tie in, and watching about 1 and a half hours of video tutorials, but they havn't seemed to give me the stepping stone I need.

Where should I go from here?

If I have a game object, and attach those scripts to it, do I need another script which is specific for that type of enemy (for example) ?

 

Ahh dam, with the change you suggested I get the error
" Type 'Agent' does not contain a definition for 'StateMachine' and no extension method 'StateMachine' of type 'Agent' could be found (are you missing a using directive or assembly reference?)"

 

Derp - should be this.FSM not this.StateMachine.

If an enemy attacks in a specific way then that enemy type should use a unique attack state that (optionally) derives from a base attack state.

 

I "THINK" I might be getting it now...

Okay... So I need to attach the agent onto a game object, or 'enemy'.

I have a script attached to the enemy which would communicate with the Agent and call a function to enter the enemy into the correct state he needs entering, OnCollisionEnter or something would be a good example of a way of saying put enemy in Attack state if something is colliding with it, put it in chase state if something is near it and put it in sleep / wander if nothing is near it and nothing is colliding.

those 3-4 example states above would be attached to the enemy, and would again switch between them by using this.FSM.ChangeState(new SomeState(this));

Okay so.

I think I am starting to understand now however there's one last thing II think.

If I want to have Attack/Move/Wander/Sleep those kind of states, surely the variable of Speed or Damage the enemy inflicts will be nested inside the State, if that is the case since the states don't inherit from Monodevelop is it easy to throw over variables for that enemy to change values such as speed/ damage.

 

You can choose to either hold variables in the state or on the agent (another reason for the state to know about its owner). For statistic type things like speed and damage I would hold them on the agent. Something like a calculated path for pathfinding I would hold in a move state because there is no point in the agent carrying it around when it only has a lifecycle associated with a particular state and is of no use otherwise.

 

Ok.

That means that the Agent is going to have to hold a "lot" of variables which aren't always going to be used. If that Agent is to be used on all enemies there will be redundant variables for some enemies as there might be a bool or statistic that only some need as they pertain to the unique functionality behind the enemy.

Doing it this way does however allow me to include things like the TakeDamge function on the Agent.

However it means that the variables have to be public and assigned through the inspector so they pertain only to that particular enemy and not globally for anything using that class. Unless I want to find out what enemy it is (via name or tag etc), and then have a SetStats() function if its this enemy set these stats, etc etc...

Does this seem like an acceptable way of doing it, do you think Kelso, I'm asking because I don't want to do it like this and all along it wont work and if only I'd have asked, heh

I wish there was like a +reputation thing on this website Kelso, you deserve it

Oh and also, I don't know if this is the proper way of doing it, just let me know please but if I'm in the state class and I need to get a function or variable and access its properties of that the agent is attached to I would just do

owner.gameObject

and have a GameObject variable and find the type/name of the GameObject that Agent is attached to if I needed to destroy it etc ?

 

Sorry for not responding sooner - I'm doing the Ludum Dare 7DRTS this week so my ability to post in the evening is pretty much nil.

There are a lot of options when it comes to how you hold your data. My state's 'owner' is typically a manager class that derives from MonoBehaviour and includes "actions"- so things like TakeDamage, Heal, Move, Attack, Pursue, etc etc. A lot of these actions kick off state changes in and of themselves. Having an owner in the state also exposes that GameObject to the state so you're free to do the things you talked about like checking tags and such.

One option if you need to maintain vital stats is to make a container class that is serialized so you can still assign via the Inspector but you don't glob up your manager class with a bunch of members. You also don't have to make stuff public. If you want good encapsulation you can make the members private and use [SerializeField()] to expose them to the Inspector. A good example is health, which you wouldn't want to manipulate directly.

Code (csharp):

1.  
2. public class Agent : MonoBehaviour
3. {
4.     [SerializeField()]
5.     private int maxHealth; //assign via inspector but can't be changed outside this class
6.     private int currentHealth;  // hidden from inspector because we assign in Awake
7.  
8.     void Awake()
9.     {
10.         this.currentHealth = this.maxHealth;
11.     }
12.  
13.     public void TakeDamage(int amount)
14.     {
15.         this.currentHealth -= amount;
16.         if (this.currentHealth <= 0)
17.         {
18.             this.FSM.ChangeState(new UnitDieState(this));
19.         }
20.     }
21.  
22.     public void Heal(int amount)
23.     {
24.         this.currentHealth += amount;
25.         if (this.currentHealth > this.maxHealth)
26.             this.currenHealth = this.maxHealth;
27.     }
28. }
29.  

If you want to go the container class route

Code (csharp):

1.  
2. [System.Serializable()]
3. public class AgentStats
4. {
5.     public int MaxHealth;
6. }
7.  
8. public class Agent : MonoBehaviour
9. {
10.     public AgentStats Stats; // this gets exposed in Inspector as a collapsible prop drawer
11.  
12.     private int currentHealth;
13.    
14.     void Awake()
15.     {
16.         this.currentHealth = this.Stats.MaxHealth;
17.     }
18.  
19. // etc....
20. }
21.  

 

Great thanks, that's exactly what I needed, I am much clearer today than I was 2 days ago on all of this. I am just sorting ouit take damage functions and stuff and organising variables as we speak.

Thanks again kelso.

Though I'm probably going with the first option you showed. I'll see if I like the serialize thing. its probably best to do it that way anyway

 

any way of using
[SerializeField()]
to apply to multiple variables without having to do this:

[SerializeField()]
private int Health;

[SerializeField()]
private int Ammo;


so for example everything after

[SerializeField()]

is exposed

[SerializeField()]
private int Health
private int Ammo
private int Shield
etc..

 

No problem - I try to post in these types of topics because they generally aren't well covered in a specific Unity context and I spent a fair amount of time working on a system that I think works pretty well (others may disagree of course).

Now if you can hook events into your state machine implementation you can start to do some pretty gross stuff.

Edit: To your serialize question - no, it's a decorator on that member only.

Double Edit: You had asked about re-usability. I would make each distinct agent type into a prefab so that you only have to set vitals once in the Inspector. Then you can kick off initial stats from Start so they look smart as soon as they are instantiated.

 

Yeah. I think that I am going to do it that way (your double edit).

What did you mean about doing gross stuff?

thanks.

 

Hooking up events allows you to affect other aspects of your game without adding a bunch of code to an agent and making him track abstract systems that he wouldn't otherwise care about. It's the codification of "this thing happened in the game".

So for example you could create an event that is fired when an enemy dies. Your UI could hook that event to display a message to the player. Your score system could hook the same event to increase the player score. All the enemy does is fire the event, it doesn't care who is listening or what those listeners do.

 

Sounds really useful going to have to read up on that at some point now haha

 

I am very grateful for your help getting this system set up however I think I might only need to use it for bosses or complex enemies.

Simple seeking enemies or random movement enemies I can't see needed states. unless you can tell me otherwise I'll probably end up using the state system for the bosses who needs multiple AIs for health % or other situations.

 

How you use it is up to you. You could certainly make the case that an enemy that does only a single thing it's entire lifetime does not need something like this (ie - if it never changes state then why does it need a state machine). On the flip side you could argue for doing a consistent implementation across all entities and it would make life easier if you ever changed that enemy's behavior in the future.

 

Sorry for necroing. Good discussions were going on here. I learnt a lot.
I have some questions to ask,
1. How to you actually exit a state in Exit() when a state is finished; stopcourotine can't be called right?
for example an enemy is in "Idle" state for a while, then he needs to go to "Patrol" state
2. For solving first question, if we add ChangeState() in the Exit() of Idle State. It seems like working!
3. If we do 2, when we want to change the state in FSM instead of transiting in state itself, we call
currentState.Exit();
But in this Exit(), we actually added ChangeState() in 2.
turns out 2 messed up everything.

how do we solve this brother. can u give me some direction

 

I feel like I get it solved, but my thought is still not that clear..

Code (CSharp):

1. public IEnumerator Execute()
2.     {
3.         Debug.Log("alert!!");
4.  
5.         yield return null;
6.  
7.         this.ExitToNextState();
8.     }
9.  
10.     public void Exit()
11.     {
12.         aiEntity.StateTerminator(this.Execute());
13.         Debug.Log("exit alert state.");
14.     }
15.  
16.     public void ExitToNextState()
17.     {
18.         aiEntity.FSM.ChangeState(new AttackState(aiEntity, target), false);
19.     }

Code (CSharp):

1. public void ChangeState(AIState newState, bool interruptCurrentState)
2.     {
3.         if (interruptCurrentState)
4.         {
5.             if (this.currentState != null)
6.             {
7.                 this.currentState.Exit();
8.              
9.             }
10.         }
11.  
12.         this.currentState = newState;
13.         this.currentState.Enter();
14.     }

Code (CSharp):

1. protected void Update()
2.     {
3.         if (Vector2.Distance(transform.position, target.position) < 1)
4.         {
5.             if (!(FSM.currentState is AlertState) && !(FSM.currentState is AttackState))
6.             {
7.                 FSM.ChangeState(new AlertState(this, target), true);
8.             }
9.         }
10.     }
11.  
12. public void StateTerminator(IEnumerator state)
13.     {
14.         StopCoroutine(state);
15.     }

and code feels redundant as well....
Gota go to bed, hopefully come up with something better tomorrow

​

 

Code (CSharp):

1. public class AlertState : MonoBehaviour, AIState{
2. }

states are inherited from monobehavior for now in order to use console output.

 

We ran into a bunch of issues with blended coroutines that we stopped using them and now just execute in an Update method.

Also - you can call Debug.Log() from anywhere - doesn't have to inherit from MonoBehavior

Lastly - your interruptState argument is puzzling. Finite State Machines should only use a single state so you should always be calling Exit on the previous state.

 

I know I'm late here but using the pattern above would I see a significant performance loss when having hundreds of active agents in a level?

Right now each of my agents use a single 2000 lines switch(state) in their update. The nice thing is there is no allocation on the heap. The bad thing is it's become increasingly difficult to add or modify states.

By using the above I can see myself creating a lot of states and I wonder how bad it would be on the GC to do so.

 

I'm of the opinion that the readability and maintainability of individual states far outweighs whatever GC you might (but probably won't) incur. It's more of a concern for un-managed languages where you need to handle disposal yourself. In this case, some people will make their states singletons that get an agent passed to them for processing. The downside is that your agent needs to hold stateful data on itself instead of letting the state manage it (ie - a destination for a move state).

I'm also of the opinion that if you do anything that ends up with a 2000 line switch statement then you've done something very bad

 

Thank you for your insight I'll start working on something. You're absolutely right, when I first started I had three states it was good enough and I didn't know any better. I just kept on adding and adding because it was easy then and allowed me to prototype fast.

Hey it's good experience and will make for a very good refactoring weekend

 

@KelsoMRK
Sorry for reviving an 8 year old thread.
I have questions if you don't mind:
1. Did you manage to implement coroutines, if not, how do you manage delays for automatic transitions in states?
e.g Transition from idle to patrol after 10 seconds of being idle automatically.
2. For example, if I have 3 different types of enemy behaviors that have different conditions for transitions: e.g. Enemy 1 switches to chase player from patrol based on 10 distance between them. Enemy 2 switches to chase from patrol based on 20 distance and Enemy 3 has a whole different transition which is attack from patrol based on 2 distance. Do I have to create for instance, Enemy 1 Patrol State, Enemy 2 Patrol State and Enemy 3 Patrol State with different conditions for transitions to next states or is there a better way?

 

Things worked so much nicer using Update that we never looked back. If you want something to happen after X number of seconds you can just count down to 0 (or up to X) in your Execute method because it runs every frame.

If the only difference is the distance at which they transition from patrol to attack then put that data on the agent (preferably via Inspector variable) and have the state use it

 

Thank you