Discussion State machine using all Monobehaviours, bad idea?

Edit: For whoever is reading this in the far future, the answer I was looking for is using [serializeReference] check it out:

Hi guys, I am working on creating a finite state machine for the AI of the enemies in my game. I first had all the transitions checked by the AIController, but I felt that this was hard to maintain and expand.

I try to split things up like this:

AIController - Monobehaviour: Holds the current state and handles transitions
State - Monobehaviour: Contains state logic and a list of transitions
Transition: script: Holds a decision and a state to which is switched if the condition is met
Decision: Monobehaviour: Checks a certain condition and returns true or false

I add the AIController and the different State and Decision monobehaviours to the enemy gameObjec:


Then I add the transitions in a list on the States like this:


I choose monobehaviours for the states and decisions because they often have to contain state (The not FSM kind) and contain parameters. I find it more convenient to tweak the parameters on the object itself than in a separate scriptable object. The only downside I have so far is that you end up with a lot of monobehaviours as part of the ai. I honestly can't find anyone else that has it set up this way so I am feeling like I am kinda crazy. Are there any obvious pitfalls with this approach?


Here is the code of my base classes:

Code (CSharp):

1. public class AIController : MonoBehaviour
2. {
3.     public AI_State initialState;
4.     private AI_State currentSate;
5.  
6.     void Start()
7.     {
8.         SwitchState(initialState);
9.     }
10.  
11.     void FixedUpdate()
12.     {
13.         currentSate.PerformState();
14.         currentSate.CheckTransitions(this);
15.     }
16.  
17.     public void SwitchState(AI_State newState)
18.     {
19.         if (currentSate != null)
20.         {
21.             currentSate.StopState();
22.             currentSate.DeInitializeDecisions(this);
23.         }
24.         newState.StartState();
25.         newState.InitializeDecisions(this);
26.         currentSate = newState;
27.     }
28. }

Code (CSharp):

1. public class Transition
2. {
3.     public Decision decision;
4.     public AI_State newState;
5.  
6.     public void CheckTransition(AIController aIController)
7.     {
8.         if (decision.Decide(aIController))
9.             aIController.SwitchState(newState);
10.     }
11. }

Code (CSharp):

1. public abstract class Decision : MonoBehaviour
2. {
3.     public virtual void InitializeDecision(AIController aiController) { }
4.     public virtual void DeInitializeDecision(AIController aiController) { }
5.     public abstract bool Decide(AIController aiController);
6. }

Code (CSharp):

1. public abstract class AI_State : MonoBehaviour
2. {
3.     public List<Transition> Transitions = new List<Transition>();
4.  
5.     public abstract void StartState();
6.  
7.     public abstract void PerformState();
8.  
9.     public abstract void StopState();
10.  
11.     public void InitializeDecisions(AIController aiController) {
12.         foreach (Transition transition in Transitions)
13.         {
14.             transition.decision.InitializeDecision(aiController);
15.         }
16.     }
17.     public void DeInitializeDecisions(AIController aiController)
18.     {
19.         foreach (Transition transition in Transitions)
20.         {
21.             transition.decision.DeInitializeDecision(aiController);
22.         }
23.     }
24.  
25.     public void CheckTransitions(AIController aiController) {
26.         foreach (Transition transition in Transitions)
27.         {
28.             transition.CheckTransition(aiController);
29.         }
30.     }
31. }

 

It'll be a mess to maintain in editor, and it will fail the moment you forget to add one of the behavior (which is definitely going to happen).

I'd recommend to take a look at coroutines as they can be used for ai, similar to decision trees.

 

Yeah but you will always have to define your states and transitions somehow, if you forget to add it in code it will also fail. It's the tradeoff between flexibility and having to set things up. I think you can easily group similar AI in a prefab with maybe some prefab variants to tweak parameters.

I'll have a look into coroutines because I haven't seen them used for this application before.

 

If you "forget to add code", that will result in a compile time error. If you forget to add a component, that will result in a runtime error, that you will not be able to catch until you run the project. Compile time error is preferable, as you'll waste less time detecting it.

Additionally, you do not benefit from behaviors being components, because to switch to another state, they'll have to be aware of the other state existing. They also do not support nested logic.

Ther's a reason why I suggested coroutines - because they can result in a very clean and concise code.

Code (csharp):

1.  
2. while(amAlive()){
3.      if (havePatrolRoute())
4.           yield return doPatrol();
5.      else
6.           yield return doIdle();
7. }
8.  

The interesting thing is that you can design certain situation as "sub functions". For example, there could be a coroutine called doCombat(), and both doPatrol() and doIdle() could call it if a combat situation occurs.
Logically this behavior would be equivalent to decision trees.

This will not really be possible with your setup, as components cannot be nested. Meaning you cannot implement a state as a statemachine.

 

There are ways you can mitigate this but they're still not as ideal as relying on the compiler. OnValidate will execute whenever a change is made in the inspector as well as when you enter play mode. If the error is severe enough for you play mode can be interrupted.

Code (csharp):

1. using UnityEngine;
2.  
3. public class Foo : MonoBehaviour
4. {
5.     public GameObject go;
6.  
7.     private void OnValidate()
8.     {
9.         if (go == null)
10.             Debug.LogError("You forgot to set <color=#00FF00>go</color>.");
11.     }
12. }

Odin Inspector has a required attribute that will trigger an error message.

Code (csharp):

1. public class Foo : MonoBehaviour
2. {
3.     [Required]
4.     public GameObject go;
5. }

Both of these solutions have a glaring limitation though. A scene has to be loaded to be validated. If you want to validate everything you would have to load every scene to let the validators run. You can do this with the editor's scene manager.

https://docs.unity3d.com/ScriptReference/SceneManagement.EditorSceneManager.html

Alternatively you can use a third party asset designed for this kind of thing.

https://assetstore.unity.com/packages/tools/utilities/odin-validator-227861

 

One problem with using coroutines for this is that this would be considered hot paths, and yield return someSubEnumerator() will allocate


Edit: might not be a problem with the semi new incremental GC feature

 

You can make custon editors or PropertyDrawers to show non-MonoBehaviour stuff in the Inspector, so they don't have to be MonoBehaviours if this is the reason.

Even if they were MBs, I'd consider a custom editor anyway. This isn't just tweaking mostly-independent properties, as the default Inspector is designed for. I probably wouldn't stop at just the Inspector for this, either.

Also note that you can write some self-test code to motigate some of the other issues raised.

 

Thank you, I think this is probably the best solution for the Decision MonoBehaviours at the very least.

 

Use interfaces instead that way you retain the ability to define the states in whatever way you need. (ex. as mono behaviours/ scriptable objects/ simple c# class)

As for which way is better just go with what makes more sense in your project and in each use case.
At the end you gonna use the editor to setup references one way or another unless you want to battle the engine design to the core so don't stress too much over it.

 

I looked into this, but I feel like I am missing something. I can't find anything that would allow me to both assign a script to the "decision" reference and also contain state without using a monobehaviour. A scriptable object doesn't seem work because the state is global and when I make it plain C# object I don't know how to assign an instance through the inspector even with a custom editor.

 

Well, you can't do that. And I'm unsure why you'd want to do that.

You can, however, use unity events. And assign methods to them. Keep in mind that overuse of unity events makes things harder to debug, as it becomes hard to track which object calls which function.

 

Have you marked your relevant classes as being serializable? [System.Serializable], I think it is.

From memory that'll allow all content which Unity can deal with (i.e. serialization compatible) to show in the Inspector.

 

I am just looking for an easy way to fill in the specific implementation of my transition:



This is how it's with mono behaviours. But I would prefer not having to drag in all the behaviour monobehaviours into the object.

 

Have you looked at a behaviour tree solution? Like node canvas or similar

 

Yeah, looking into it, a behaviour tree looks more suitable in the long run, thanks guys!

 

A behaviour tree is a state machine.

 

But not all state machines are a behaviour tree. And my current implementation sure as hell ain't one.

 

Oh don't confuse people... "Behavior tree" is well defined enough so one can easily distinguish it from what people claim they have implemented when they just say "statemachine".

 

Your current implementation isn't one because your current implementation, as mentioned in the critique in this thread, is just kind of unmanageable and so it can't really be easily converted into one. When I say "easily" I do mean that too, as you just make the decision point to different behaviours and make it serializable, generally.

Incidentally, as also mentioned before, you shouldn't be using Monobehaviours for decisions or actions. This is something that can easily be done with ScriptableObjects by just passing relevant data into functions in the SO itself.

It literally is a state machine. The only thing that separates it from this very implementation is the fact that decision types aren't serialized classes that point to transitions. In any reasonably maintainable state machine implementation, this change is a matter of changing maybe half a dozen lines of code.

 

Why would it it better to use SO? You go to an even greater space to select your decisions from.

 

SOs are neater and easier to use in a modular context, and inherently more easily serializable when compared to Monobehaviours. If you are making behaviours you constantly need to swap out/replace but that don't require update loops, you might as well be using SOs for them.

 

Ask yourself this, does a state machine need a transform?

 

Is it even using a hierarchical node tree structure? In every It is missing crucial parts like control flow nodes (sequence, parallel), various decorators and I also don't see strictly limitated states "running", "failure" and "success".

Sure I'm sounding like from a lecture, but that's what is usually referred to by behavior tree...

 

If you serialize your transitions in the way I explained you immediately have access to a hierarchical structure. That's how simple it is. Control flow can now be handled by the decision system. The reason I know these things is because I have literally made this rework in my own state machines in the past before moving away from traditional FSM and behaviour tree models because of the way both end up suffering greatly from maintainability issues once scale becomes involved.

 

If you're sticking to the default Editor displays, yes. It's well worth taking a while to learn about making custom Inspectors, Property Drawers, Editor windows, and using the SerializedObject API.

For example, even without writing a custom Inspector, a Property Drawer can let you provide a picker rather than dragging / selecting your SOs from the Project tab.

 

You can't drag a script into inspector, because it is not an object. Meaning it is not derived from UnityEngine.Object, it is not a prefab and it is not ScriptableObject. Well, technically you could, but that wiould result in a useless textasset and not a callable code. You could, however, use UnityEvents, but that will make things even messier.

Additionally, if you really want to, you can abuse animator controller to control your AI.
You see, Animator Controller already implements state machines, and you can attach scripts to animation nodes, using StateMachineBehavior.
https://docs.unity3d.com/ScriptReference/StateMachineBehaviour.html