When you're designing a game, it's easy to come up with what enemies should do. When to trigger behaviors properly is a bit more complex though. You don't want your enemies to run into walls like bumbling idiots, nor do you want them to hunt the player down rapidly and kill them with pinpoint accuracy. Apart from niche gameplay functions that might have a place in modern games, AI that behaves too stupid or too competent is just not much fun to play against. Nor it is fun to fight glitchy, badly programmed enemies that plainly don't function properly.
In order to drive behaviors logically, smart sages of the past came up with the idea of a State Machine. While not the most modern of approaches to enemy AI, it's a tried and true method that works exceptionally well to drive consistent, predictable behavior by ensuring that:
I won't go to deep in the why of things, but you can imagine that if you cover the above logic, behavior becomes quite consistent. I'm also not going to pretend my code is the best way to program state machines, it's one way to program state machines. I blatantly worked my way through a tutorial on the subject, modified the code to do what I needed it to do, and wanted to share some work in progress. The code is in C+ but the basic idea is applicable in any language or program with some sort of update cycle. Let's delve in!
The base class for any of my state machines is actually quite lightweight. The protected variable _currentState keeps the current state. Because it's private no external code can change it, which will come in handy later. We can retrieve it via the getter currentState. For the same reason I'll explain later, it also needs a constructor. For now, just play along with the idea that having a constructor is what happens when a class reaches adolescence.
It's first method InitState() is the only explicit way to externally set another state. Yeah yeah, I know I'm already breaking the rules! We're not going to break with the intent though. While we could optionally add a null check to prevent it being used twice, the reality of the matter is that forcing the initial state in a constructor simply isn't that desirable in reality. What if I told you we now have a public method with which to set an initial state at our own discretion, for instance when a level is loaded? Or when an enemy is resurrected? For now, we'll go with the option that gives us the most flexibility as swapping a state externally shouldn't lead to programmatic behavior, it just breaks with the expected, state driven behavior we'll code next. Let's agree on using it with caution, out of the players sight.
The same goes for it's next public method, SetState(). This method is the key to state machine behavior. Every time it's invoked, it calls the OnStateExit() method on the state we're leaving, and the OnStateEnter() method of the state we are entering. As long as we swap states via SetState(), no programmatic errors should occur because those methods make sure of the following:
Then there's the Run method. It has the simplest task of all: to just keep the OnStateUpdate() method running each frame or whatever unit of time/progress your game uses.
using System;
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
[Serializable]
public class StateMachine
{
[SerializeField] protected State _currentState;
public State currentState { get { return _currentState; } }
public StateMachine()
{
}
public void InitState(State state)
{
_currentState = state;
_currentState.OnStateEnter();
}
/*
* Before we switch states, we fire the OnStateExit method
* of the current State. Then we swap them out, and fire the
* OnStateEnter method of the new State. States should 'clean
* themselves up' before another State takes over control.
*/
public void SetState(State state)
{
if (_currentState != null)
{
_currentState.OnStateExit();
}
_currentState = state;
_currentState.OnStateEnter();
}
public void Run()
{
currentState.OnStateUpdate();
}
}
Ah, the slender meat of our class package! Think of a single state as a continuous action, such as "running" or "climbing". They can also be actions with a set duration. The constructor requires a StateMachine instance and stores it in a field so it can run it's public methods. In this example I also set a name field but that's entirely optional, I just use it for serialization purposes. It helps debugging if the name of the current state is displayed in the Editor.
OnStateEnter(), OnStateUpdate() and OnStateExit() are all virtual methods that don't contain any code. These methods are there for more concrete classes to extend. The idea is that we're going to write our StateMachines with an object oriented approach where we can extend existing behavior. That might sound complex, but later on we'll see that the complexity is mainly in satisfying the compiler and not in the state logic itself.
I've added an example of how this class could be extended below. A typical workflow would be to extend the class, add a constructor, and override the three signature methods as required. The virtual methods do nothing: they are programming guides for future extensions.
using System;
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
[Serializable]
public class State
{
[NonSerialized] public StateMachine machine; // our "machine" variable
[SerializeField] public string name;
public State(StateMachine machine)
{
this.name = this.GetType().ToString();
this.machine = machine;
}
/*
* Fired by State Machines when a State is first entered.
*/
virtual public void OnStateEnter()
{
//Debug.Log(this + " OnStateEnter");
}
/*
* Fired continiously during an Update cycle.
*/
virtual public void OnStateUpdate()
{
//Debug.Log(this + " OnStateUpdate");
}
/*
* Fired by State Machines when the State is left.
*/
virtual public void OnStateExit()
{
//Debug.Log(this + " OnStateExit");
}
}
/*
* Here are some example states, most of them actually do jack shit,
* they just serve as an example of how each of the On.. methods can
* be overridden, based, etc.
*/
/*
* The virtual methods are all overridden in this example.
*/
public class StateIdle : State
{
public StateIdle(StateMachine machine) : base(machine) { }
override public void OnStateEnter()
{
base.OnStateEnter();
}
override public void OnStateUpdate()
{
base.OnStateUpdate();
}
override public void OnStateExit()
{
base.OnStateEnter();
}
}
This is the base class of any enemy that moves around using a NavMeshSurface. You'll probably recognize it extends MonoBehavior. The other components it requires are actually custom MonoBehaviors that are only pertinent to later examples. I made sure the names of their public methods are descriptive so you can follow along. To give you an idea: the PlayerSensor gathers information about the player's current position, direction and rotation, while the AimSensor is used to calculate line of sight. Consider them the "eyes and ears" of your enemy, containing data for States to evaluate. Because these are added to the base class enemy along with a NavMeshAgent, every state is assured to have scope on this data.
Notice that the state machine we are adding here is called StateMachineNavMeshEnemy, and that it's constructor has an argument. Let's take a look at this extended class in the next section now that we have an idea how our enemies are structured. In this game, every enemy is stored in a Prefab that has the aforementioned components added and configured for the specific enemy. While I don't like storing configuration for enemies in Prefabs, that issue is beyond the scope of this article and simply how Unity stores concrete "things" you can spawn, despawn and pool to your liking.
using System;
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.AI;
[RequireComponent(typeof(NavMeshAgent))]
[RequireComponent(typeof(AimSensor))]
[RequireComponent(typeof(PlayerSensor))]
public class NavMeshEnemy : MonoBehaviour
{
public NavMeshAgent agent;
public PlayerSensor playerSensor;
public AimSensor aimSensor;
public StateMachine stateMachine;
public Animator animator;
virtual protected void AddStateMachine()
{
stateMachine = new StateMachineNavMeshEnemy(this);
stateMachine.SetState((stateMachine as StateMachineNavMeshEnemy).idle);
}
private void Awake()
{
agent = GetComponent<NavMeshAgent>();
playerSensor = GetComponent<PlayerSensor>();
aimSensor = GetComponent<AimSensor>();
}
private void Start()
{
AddStateMachine();
}
private void Update()
{
if (stateMachine != null)
{
stateMachine.Run();
}
}
public Quaternion GetPlayerRotation()
{
Vector3 direction = playerSensor.direction;
direction.y = 0;
return Quaternion.LookRotation(direction, Vector3.up);
}
}
Extending the StateMachine class for a specific enemy or enemy type is quite easy. The one thing we need to keep in mind is that now things become concrete, we need some way to retrieve data for our States to evaluate. This is why we modify the constructor to accept a new parameter of the NavMeshEnemy class. After storing it in a field we instantiate some states. These are very rudimentary, and only cover two behaviors: idle and approach. In truth they are not even required, and mainly serve as an example to show:
In practice, this enemy already works. It will remain idle until it establishes line of sight with the player. When this happens, it switches behavior to approaching the player. Should the player move out of line of sight, it moves to it's last known position. It's very basic, but it covers the basics of what an enemy designed to move around NavMeshSurfaces can do.
using System;
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
[Serializable]
public class StateMachineNavMeshEnemy : StateMachine
{
// enemy
public NavMeshEnemy enemy;
// states
public StateNavMeshEnemyIdle idle;
public StateNavMeshEnemyPursue pursue;
// constructor
public StateMachineNavMeshEnemy(NavMeshEnemy enemy)
{
this.enemy = enemy;
idle = new StateNavMeshEnemyIdle(this);
pursue = new StateNavMeshEnemyPursue(this);
}
}
public class StateNavMeshEnemyIdle : StateNavMeshEnemy
{
public StateNavMeshEnemyIdle(StateMachineNavMeshEnemy machine) : base(machine) { }
public override void OnStateEnter()
{
base.OnStateEnter();
}
public override void OnStateUpdate()
{
base.OnStateUpdate();
if (stateMachine.enemy.aimSensor.hasLOS)
{
stateMachine.SetState(stateMachine.pursue);
}
}
public override void OnStateExit()
{
base.OnStateExit();
}
}
public class StateNavMeshEnemyPursue : StateNavMeshEnemy
{
public StateNavMeshEnemyPursue(StateMachineNavMeshEnemy machine) : base(machine) { }
public override void OnStateEnter()
{
base.OnStateEnter();
}
public override void OnStateUpdate()
{
base.OnStateUpdate();
if (!stateMachine.enemy.aimSensor.hasLOS)
{
stateMachine.SetState(stateMachine.idle);
stateMachine.enemy.agent.SetDestination(stateMachine.enemy.aimSensor.lastKnownPosition);
}
else
{
bool dest = stateMachine.enemy.agent.SetDestination(stateMachine.enemy.playerSensor.player.transform.position + new Vector3(0, -1, 0));
if (!dest)
{
stateMachine.SetState(stateMachine.idle);
}
}
}
public override void OnStateExit()
{
base.OnStateExit();
}
}
We will extend our NavMeshEnemy class to define our first real concrete enemy, the BruteNavMeshEnemy or Brute for short. Imagine a lumbering hulk of a man, hell bent on your destruction. It's base class ensures we have eyes and ears, as well as a NavMeshAgent. If that doesn't sound dangerous enough, we will also write a far more intelligent state machine for it. Remember that virtual AddStateMachine() method? This is where we override it, set the StateMachine for it, and fill the stateMachine variable by casting it to the expected extension to satisfy our compiler. The variable itself doesn't care, because StateMachineBrute extends StateMachineNavMeshEnemy anyway.
using System;
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
public class BruteNavMeshEnemy : NavMeshEnemy
{
public StateMachineBrute stateMachineBrute;
protected override void AddStateMachine()
{
stateMachineBrute = new StateMachineBrute(this);
stateMachineBrute.SetState(stateMachineBrute.idleBrute);
stateMachine = stateMachineBrute as StateMachine;//in order to catch Run command
}
}
We are one step away from unleashing the power of state machines upon the unsuspecting players of our game. This is the finalized StateMachineBrute class, including it's States. These classes follow the same pattern by defining additional fields on top of existing logic. The StateMachineBrute class itself is extremely simple, but has a small catch. It has states named idleBrute and pursueBrute, but the idle and pursue states are still inherited, not overwritten! It is here I had to make a decision regarding gameplay possibilities I have not yet fully explored. Consider the following:
I chose the latter option and decided on a simple naming convention.
using System;
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
[Serializable]
public class StateMachineBrute : StateMachineNavMeshEnemy
{
// enemy
public BruteNavMeshEnemy brute;
// states
public StateBruteIdle idleBrute;
public StateBrutePursue pursueBrute;
public StateBruteFlank flank;
public StateBruteCombat combat;
public StateBruteSwipe swipe;
public StateBrutePunch punch;
// constructor
public StateMachineBrute(BruteNavMeshEnemy brute) : base (brute)
{
this.brute = brute;
idleBrute = new StateBruteIdle(this);
pursueBrute = new StateBrutePursue(this);
flank = new StateBruteFlank(this);
combat = new StateBruteCombat(this);
swipe = new StateBruteSwipe(this);
punch = new StateBrutePunch(this);
}
}
public class StateBrute : State
{
public StateMachineBrute stateMachine;
public StateBrute(StateMachineBrute machine) : base(machine) { stateMachine = machine; }
}
public class StateBruteIdle : StateBrute
{
public StateBruteIdle(StateMachineBrute machine) : base(machine) { }
public override void OnStateEnter()
{
base.OnStateEnter();
stateMachine.brute.agent.SetDestination(stateMachine.brute.transform.position);
}
public override void OnStateUpdate()
{
stateMachine.brute.agent.SetDestination(stateMachine.brute.transform.position);
if (stateMachine.brute.playerSensor.distance > stateMachine.brute.agent.stoppingDistance + 0.1f)
{
if (stateMachine.brute.aimSensor.hasLOS)
{
stateMachine.SetState(stateMachine.pursueBrute);
}
} else
{
stateMachine.SetState(stateMachine.combat);
}
if (stateMachine.brute.animator)
{
stateMachine.brute.animator.SetBool("walking", false);
}
}
}
public class StateBrutePursue : StateBrute
{
public StateBrutePursue(StateMachineBrute machine) : base(machine) { }
public override void OnStateEnter()
{
base.OnStateEnter();
stateMachine.brute.animator.SetBool("walking", true);
}
public override void OnStateUpdate()
{
base.OnStateUpdate();
stateMachine.brute.agent.updateRotation = true;
stateMachine.brute.animator.SetBool("walking", true);
if (stateMachine.brute.playerSensor.distance < stateMachine.brute.agent.stoppingDistance + 0.1f)
{
stateMachine.SetState(stateMachine.combat);
}
if (!stateMachine.brute.aimSensor.hasLOS)
{
if (Vector3.Distance(stateMachine.brute.transform.position, stateMachine.brute.aimSensor.lastKnownPosition) < stateMachine.brute.agent.stoppingDistance - 0.1f)
{
stateMachine.brute.agent.SetDestination(stateMachine.brute.aimSensor.lastKnownPosition);
} else
{
stateMachine.SetState(stateMachine.idleBrute);
}
}
else
{
bool dest = stateMachine.brute.agent.SetDestination(stateMachine.brute.playerSensor.player.transform.position + new Vector3(0, -1, 0));
if (!dest)
{
stateMachine.SetState(stateMachine.idleBrute);
}
}
// NOT GREAT!
if (stateMachine.brute.agent.velocity.magnitude < 0.001f)
{
if(stateMachine.brute.playerSensor.distance < stateMachine.brute.aimSensor.flankingDistance)
{
stateMachine.SetState(stateMachine.flank);
}
}
//Debug.Log(stateMachine.brute.name + " : " + stateMachine.brute.agent.velocity.magnitude);
}
}
public class StateBruteFlank : StateBrute
{
public StateBruteFlank(StateMachineBrute machine) : base(machine) { }
public Vector3 destination;
public override void OnStateEnter()
{
base.OnStateEnter();
if(stateMachine.brute.aimSensor.flankingPositions.Count > 0)
{
int rand = UnityEngine.Random.Range(0, stateMachine.brute.aimSensor.flankingPositions.Count);
destination = stateMachine.brute.aimSensor.flankingPositions[rand];
stateMachine.brute.agent.SetDestination(destination);
stateMachine.brute.animator.SetBool("walking", true);
} else
{
stateMachine.SetState(stateMachine.idleBrute);
}
}
public override void OnStateUpdate()
{
base.OnStateUpdate();
if (Vector3.Distance(stateMachine.brute.transform.position, destination) < stateMachine.brute.agent.stoppingDistance - 0.1f)
{
stateMachine.SetState(stateMachine.pursueBrute);
}
// NOT GREAT!
if (stateMachine.brute.agent.velocity.magnitude < 0.001f)
{
stateMachine.SetState(stateMachine.pursueBrute);
}
}
}
public class StateBruteCombat : StateBrute
{
public StateBruteCombat(StateMachineBrute machine) : base(machine) { }
public override void OnStateEnter()
{
base.OnStateEnter();
int rand = UnityEngine.Random.Range(0, 2);
switch(rand)
{
case 0:
stateMachine.SetState(stateMachine.swipe);
break;
case 1:
stateMachine.SetState(stateMachine.punch);
break;
}
}
}
/* Note: I started extending because I thought calculating the rotations would be useful, but on this enemy the animations are nicer with constant lerping. I kept the extension anyway. */
public class StateBruteAttack : StateBrute
{
protected Quaternion rotationEnter;
protected Quaternion rotationExit;
protected float timer;
public StateBruteAttack(StateMachineBrute machine) : base(machine) { }
public override void OnStateEnter()
{
base.OnStateEnter();
rotationExit = stateMachine.brute.GetPlayerRotation();
rotationEnter = stateMachine.brute.transform.rotation;
timer = 0;
}
}
public class StateBruteSwipe : StateBruteAttack
{
public StateBruteSwipe(StateMachineBrute machine) : base(machine) { }
public override void OnStateEnter()
{
base.OnStateEnter();
stateMachine.brute.agent.updateRotation = false;
stateMachine.brute.animator.SetBool("walking", false);
stateMachine.brute.animator.SetTrigger("swipe");
}
public override void OnStateUpdate()
{
base.OnStateUpdate();
// not used
timer += Time.deltaTime * Time.timeScale;
float lerp = Mathf.Clamp(timer / 2, 0, 1);
//this technique does not allow for exit times... perhaps add the clip(s) to the Machine?
if (stateMachine.brute.animator.GetCurrentAnimatorClipInfo(0)[0].clip.name == "Brute_Swiping")
{
Quaternion rotation = Quaternion.Lerp(stateMachine.brute.transform.rotation, stateMachine.brute.GetPlayerRotation(), 0.2f);
stateMachine.brute.transform.rotation = rotation;
}
else
{
stateMachine.SetState(stateMachine.idleBrute);
}
}
public override void OnStateExit()
{
base.OnStateExit();
}
}
public class StateBrutePunch : StateBruteAttack
{
public StateBrutePunch(StateMachineBrute machine) : base(machine) { }
public override void OnStateEnter()
{
base.OnStateEnter();
stateMachine.brute.agent.updateRotation = false;
stateMachine.brute.animator.SetBool("walking", false);
stateMachine.brute.animator.SetTrigger("punch");
}
public override void OnStateUpdate()
{
base.OnStateUpdate();
//not used
timer += Time.deltaTime * Time.timeScale;
float lerp = Mathf.Clamp(timer / 2, 0, 1);
if (stateMachine.brute.animator.GetCurrentAnimatorClipInfo(0)[0].clip.name == "Brute_Punching")
{
Quaternion rotation = Quaternion.Lerp(stateMachine.brute.transform.rotation, stateMachine.brute.GetPlayerRotation(), 0.2f);
stateMachine.brute.transform.rotation = rotation;
}
else
{
stateMachine.SetState(stateMachine.idleBrute);
}
}
public override void OnStateExit()
{
base.OnStateExit();
}
}
Let's done one more example. I found the Brute was a bit messy. This StateMachineNinja class is slightly more refined. It follows the same pattern, but instead of adding a whole State for every single animation I allowed the meleeNinja state to decide upon an animation randomly, while meleeNinjaPunch handles returning to any other states. This is just an example of how not every state needs to be overridden, leading to less boilerplate then had I used an Interface to safisfy dependencies. In this example, OnStateExit is not required, because I had no timers to reset: I simply check whether the expected animationclip Character_Idle has been reached on OnStateUpdate().
using System;
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
[Serializable]
public class StateMachineNinja : StateMachineNavMeshEnemy
{
// enemy
public NinjaNavMeshEnemy ninja;
// states
public StateIdleNinja idleNinja;
public StatePursueNinja pursueNinja;
public StateMeleeNinja meleeNinja;
public StateMeleeNinjaPunch meleeNinjaPunch;
// constructor
public StateMachineNinja(NinjaNavMeshEnemy ninja) : base(ninja)
{
this.ninja = ninja;
idleNinja = new StateIdleNinja(this);
pursueNinja = new StatePursueNinja(this);
meleeNinja = new StateMeleeNinja(this);
meleeNinjaPunch = new StateMeleeNinjaPunch(this);
}
}
public class StateNinja : State
{
public StateMachineNinja stateMachine;
public StateNinja(StateMachineNinja machine) : base(machine) { stateMachine = machine; }
}
public class StateIdleNinja : StateNinja
{
public StateIdleNinja(StateMachineNinja machine) : base(machine) { }
public override void OnStateEnter()
{
base.OnStateEnter();
stateMachine.enemy.agent.updateRotation = true;
}
public override void OnStateUpdate()
{
base.OnStateUpdate();
stateMachine.ninja.animator.SetBool("walking", false);
if (stateMachine.enemy.playerSensor.distance > stateMachine.enemy.agent.stoppingDistance + 0.1f)
{
stateMachine.SetState(stateMachine.pursueNinja);
}
else if (stateMachine.enemy.playerSensor.distance < stateMachine.enemy.agent.stoppingDistance + 0.1f)
{
stateMachine.SetState(stateMachine.meleeNinja);
}
}
}
public class StatePursueNinja : StateNinja
{
public StatePursueNinja(StateMachineNinja machine) : base(machine) { }
public override void OnStateEnter()
{
base.OnStateEnter();
stateMachine.enemy.agent.updateRotation = true;
stateMachine.enemy.animator.SetBool("walking", true);
}
public override void OnStateUpdate()
{
base.OnStateUpdate();
if (stateMachine.enemy.playerSensor.distance < stateMachine.enemy.agent.stoppingDistance + 0.1f)
{
stateMachine.SetState(stateMachine.meleeNinja);
}
Vector3 pos = stateMachine.ninja.playerSensor.player.transform.position + new Vector3(0, -1, 0);
bool dest = stateMachine.ninja.agent.SetDestination(pos);
//decide by distance?
//stateMachine.ninja.playerSensor.distance
}
}
public class StateMeleeNinja : StateNinja
{
public StateMeleeNinja(StateMachineNinja machine) : base(machine) { }
public override void OnStateEnter()
{
base.OnStateEnter();
int rand = UnityEngine.Random.Range(0,4);
switch (rand)
{
case 0:
stateMachine.enemy.animator.SetTrigger("punch");
break;
case 1:
stateMachine.enemy.animator.SetTrigger("hook");
break;
case 2:
stateMachine.enemy.animator.SetTrigger("elbow");
break;
case 3:
stateMachine.enemy.animator.SetTrigger("combo");
break;
}
stateMachine.ninja.animator.SetBool("walking", false);
stateMachine.SetState(stateMachine.meleeNinjaPunch);
}
}
public class StateMeleeNinjaPunch : StateNinja
{
public StateMeleeNinjaPunch(StateMachineNinja machine) : base(machine) { }
public override void OnStateEnter()
{
base.OnStateEnter();
stateMachine.enemy.agent.updateRotation = false;
}
public override void OnStateUpdate()
{
base.OnStateUpdate();
Quaternion rotation = Quaternion.Lerp(stateMachine.enemy.transform.rotation, stateMachine.enemy.GetPlayerRotation(), 0.2f);
stateMachine.enemy.transform.rotation = rotation;
if (stateMachine.enemy.animator.GetCurrentAnimatorClipInfo(0)[0].clip.name == "Character_Idle")
{
stateMachine.SetState(stateMachine.idleNinja);
}
}
}