AI in ECS. What approaches do we have?

In the beginning, we had spaghetti hard-coded If/Else AI.
Then, we had finite state machines.
Behaviour trees were born.
Goal Oriented Action Planning gave us new possibilities.
Unity is working on their own AI Planner which seems to be GOAP focused.

Now, it seems the future holds the promise of performance by default via Data Oriented Technology Stack with ECS and friends. But as I play with ECS, I feel as though I'm thrust back into the dark ages of AI. I struggle to reconcile the concept of a Behaviour Tree with with Systems, Components, and linear data access. I wonder how the pieces fit together, how I could promote rapid iteration of AI design, how to generally create an intelligently acting agent that's not a nightmare to change and improve on.

How do you approach AI for your ECS entity agents? Have you found anything that works for you? What options do we have?

 

Thanks man this sounds like some really good advice. I suppose I was constraining my thinking too much by thinking about linear data access. I didn't quite realise that you could expect such a speed up even with random memory access. I'll go over my previous thought processes without the artificial linear constraint, and a more liberal use of entities and entity references and see what I come up with.

Cheers!

 

Can give some insight as I have been working on a 3rd person shooter w/ ECS for some months. I'm using Utility AI and plan to explore implementing a HTN planner next. Units make decisions based on scores for score in a UtilityPlanningSystem, when their ActorBrain currentDecision != lastDecision the system will add appropriate components to make them perform action.

EX: Flee, Pursue, Wander all give a Moving tag. This tag allows the entities to start the pathfind proccesse that looks like this:

QueryPositionsIndividualJob : IJobProcessComponentDataWithEntity<MovementRequest,ActorBrain,Target>


The PositionQueryJob reads ActorBrain to determine the specifications of the PositionQuery based on current action.l When a position is found another PathRequest component is added and their MovementRequest.ready flag is set to 0. With a successful position found a PathRequest tag is added

With the PathRequest tag on, the unit will begin pathfinding and if the path fails the MovementRequest's sample size is increased to allow more eligible points.

With the PathRequest and a Moving tag that was placed based on if the action is moving they will follow their path.

If the action changes to something like Shoot or Reload which are both stationary, the Moving tag is removed and they won't follow their current path. The PathRequest is removed if a PathRequest exist but the entity is not moving.

Actions like reload and shoot are handled in two different ways which I will alter when I have more actions. Shooting is like this and there are UnInterruptible Actions that need to be finished before another one can be selected.

For handling damage the shooting job adds the Entity held in the Target component to a NativeMultiHashMap with some information about the damage they attacker is inflicting. This is a nice solution because we avoid collisions that would come from writing to the same Health component. Using this queue also lets me simulate AI accuracy as we can use the stats from the job to simulate misses and hits.

Code (CSharp):

1.  
2. public struct EndActionJob : IJobProcessComponentDataWithEntity<UnInteruptableAction>{
3.         [ReadOnly]public EntityCommandBuffer buffer;
4.         public void Execute(Entity entity, int index, [ChangedFilter]ref UnInteruptableAction actionLock){
5.             if(actionLock.finished != 1)
6.                 return;
7.                
8.            
9.             buffer.RemoveComponent<Reload>(entity);
10.            
11.            
12.             buffer.RemoveComponent<UnInteruptableAction>(entity);
13.         }
14.     }
15.  
16.     [RequireComponentTag(typeof(Shoot))]
17.     public struct EndShootingJob : IJobProcessComponentDataWithEntity<ActorBrain>{
18.         [ReadOnly]public EntityCommandBuffer buffer;
19.  
20.         public void Execute(Entity entity, int index, [ChangedFilter][ReadOnly] ref ActorBrain actor){
21.             if(actor.currentAction != DecisionFlag.Shoot){
22.                 buffer.RemoveComponent<Shoot>(entity);
23.                 buffer.RemoveComponent<Audible>(entity);
24.             }
25.         }
26.  

 

In my game I use something that looks similar to Goal Oriented Action Planning, althought I am not quite sure if it is exactly the same since I didn't hear of it before.
[EDIT]: in fact I think it is not like GOAP since I don't really have a global planner

I have several "goals" and "actions" that can be performed simulteously or not (in that it differs from a behavioural tree I guess).

Goals are generally represented by a single component attached to an entity, for instance an Attack goal component would be added to ask an entity to attack something, but it will not specify in which way. The Escape goal component can also be added to tell an AI that it should escape a danger, but it doesn't specify how to escape. The jobs that process these components can choose which action to perform to achieve this goal depending on the situation. I also have something like an Farm component to ask the agent to farm.

If an agent has no goal, then the IdleJob (a job component system), will assign one depending on the current situation. For instance it will ask an agent to attack by adding the Attack component to it. I also have another job, the ShouldEscapeJob that add the Escape goal at any time. That means that an agent can be at the same time in the attack and escape states, which allows him to escape and still attack another agent if he has this opportunity. The EscapeJob is then in charge of making any agent with the Escape component well... escape. When escaping is a success, it can then have a fully agressive strategy by removing the Escape component since it is not required anymore. The jobs handling goals can also cancel their goals or planify another depending on the situation, still by adding/Removing goal components.

Some goals however cannot be achieved while the agent has some other goals. These priorities are simply handled by adding [ExcludeComponent] tags to the jobs. For instance I don't want an agent to try to farm while escaping. Thus the FarmJob will have [ExcludeComponent(typeof(Escape))]. Once the escape is done, it can go back to farming again.

Then I have actions. Actions are also represented by components, such as ReachAction, FireAction, PickAction, etc. and they each have corresponding jobs that perform the action, or cancel it if it cannot be done anymore (this is simply done by removing the action component). The goal systems use actions to achieve their intended goal by adding different actions depending on the situation. Dependencies between systems can help to prioritize some over others.

On top of that, I have a pathfinder job that uses costs computed from game-specific heuristics to also help the AI choose more "intelligent" paths when moving arround (this is allows to have some kind of "passive" actions like picking things on the way or avoiding danger areas when possible).

I don't think this is the best approach but in my case it is working quite well and provides very interesting behaviour.

 

Both very interesting posts! I've been imagining a system similar to flobocs, but also trying to think of understand how some other tools come into play like riskparitygawd's solution.

I've been considering modelling stretegic/tactical/operational layers where strategic might run a behaviour tree once every couple of seconds to make a decision which gets applied as a goal or decision component, tactical would run more often also probably utilising tags but requiring less data perhaps, and operational would just be your if/else in systems, running every frame as data oriented as possible.

I think in the end changing structure with tags seems like the ecs way for the most part, but maybe with these layers archetypes can stay relatively stable, and I can throw in random memory access structures on the higher layers.

 

Like @riskparitygawd I too have been thinking about implementing Utility AI in ECS, following Dave Mark's "Infinite Axis" model.

There will probably be a lot of random memory access unless I first do a "data gathering" phase and then process the data and calculate scoring.

Good comment here by Dave himself with links to some of his lectures if you are interested:
https://www.reddit.com/r/gameai/comments/5paxt2/utility_based_ai/

Oh and he also wrote this excellent book on the subject
https://www.amazon.com/gp/product/B00B7REBDW/

 

My response is not regarding ECS directly, but I created other day Utility AI thread and there is simple Monobehaviour based Utility AI script.

I suggest check out.
Utility AI (Discussion)
https://forum.unity.com/threads/utility-ai-discussion.607561/#post-4072711

Then you can based on that, try to create ECS version.

Just a thought.

 

Great post Antypodish,

I'll move some of my utility-specific discussion to that thread.

I've been working on a utility system in our MMO sim game Seed (https://seed-project.io/) for around a year now. While it works pretty well we've been dealing with some perf/scalability issues I'm hoping to address with a slightly different architecture and leveraging DOTS.

We also made the mistake of relying too much on dynamic ranks instead of using the Inifnite Axis model for scoring (basically each action can have a varying number of scoring factors).

 

My implementation is also based on Dave's Infinite Axis, I'm enjoy its performance and extensibility right now for individual behaviors. I implemented Archetypes/DecisionSets with a monobehaviour containing a scriptable object that links the Actor Entity to an entity that has a buffer of eligible decisions as enums. Whole thing uses a static class and is very fast. Well worth the time.

 

Very cool

I'm curious to hear how you structured your "discovery and scoring flow". ie when an agent is looking for possible decisions, I assume you start by running some query in a job?

An ECS query could give you a set of all currently possible activities, but then they need to be individually scored for each agent (who can have different stats, locations etc).

Could you give me a rough outline of your decision flow?

 

There's a Detection System that keeps track of eight eligible targets in DynamicBuffers. All actors score decisions in a single IJobProccessComponentDataWithEntity (IJobForEach now?). If an action should look at multiple targets it will ScoreDecision for each target.

Decisions look like this and there's a switch case to get the right raw score then transform it with a response curve.

Code (CSharp):

1.  
2. public static float ScoreDecision(int decision,
3.         Entity unitEntity, Entity target, float bonus, float min,
4.         UnitStats unitData, Health health, Weapon weapon, ComponentDataFromEntity<Translation> positions, int visibility = 0
5.     ){
6.         switch(decision){
7.             case (int) DecisionFlag.Shoot:
8.                 var array = new NativeArray<int>(4,Allocator.Temp);
9.                 array[0] = (int) ConsiderationFlag.HasLineOfSight;
10.                 array[1] = (int) ConsiderationFlag.TargetInRange;
11.                 array[2] = (int) ConsiderationFlag.HealthHigh;
12.                 array[3] = (int) ConsiderationFlag.RemainingAmmoHigh;
13.                 var score = Score(array, unitEntity, target, bonus, min, unitData, health, weapon, positions,visibility);
14.                 array.Dispose();
15.                 return score;
16.  

The scoring method.

Code (CSharp):

1.    
2. public static float Score(
3.         NativeArray<int> considerations, Entity unitEntity, Entity target, float bonus, float min,
4.         UnitStats unitData, Health health, Weapon weapon, ComponentDataFromEntity<Translation> positions, int visiblity
5.     ){
6.         float finalScore = bonus;
7.         foreach(int consideration in considerations){
8.             if((0.0f > finalScore || (finalScore < min)))
9.                 return 0.0f;
10.            
11.             float score = GetConsiderationScore(consideration, unitEntity, target, unitData, health, weapon, positions, visiblity);
12.             float response = ComputeResponseCurve(consideration,score);
13.  
14.             ///<remark> All consideration scores  are multiplied together to get a single Decisions utility. </remark>
15.             finalScore *= math.clamp(response,0.0f,1.0f);
16.         }
17.         ///<remark> A compensation factor is added for the varying lengths of considerations per decision. </remark>
18.         var modF = ((1-finalScore)*bonus) * (1 - (1/considerations.Length));
19.         return finalScore + math.abs((modF * finalScore));
20.     }
21.  

 

Switching from foreach to for in NativeArray<T> will save you ton of performance.

 

Does it? I thought it doesn't matter anymore, while in past foreach was indeed slower.
Does ECS compilator and burst treats it differently?

 

https://jacksondunstan.com/articles/4713

 

This is good reading. Thx.
I mean, I always used FOR loop instead FOREACH whenever I could.
I haven't tested myself. But I would thought, burst will be clever enough, to reduce FOREACH, into FOR.
Good lesson anyway.

 

In that particular code the array is just unnecessary indirection and a performance hit. I would take all that code put it into a struct with an initializer that takes all the data except the decision id. Then a method that takes the decision id and calls an eval method for each consideration flag.

GetConsiderationScore is also suspect, I wonder if there is work being done there that's the same per consideration flag, that could be moved higher up in the logic chain.

 

Also, for evaluating curves generally you should look at caching. Ai is not as sensitive precision wise as say things related to rendering. Rounding inputs to some value that makes caching memory efficient can give big gains in performance here.

 

I was not looking into the implementation. I just gave a suggestion to everyone who uses NativeArrays in their code.

 

Are you familiar with the UtilityAI? I don't see a more efficient way to score a decision, you are still going to need to have an identifier for each Action, the point of the NativeArray is that at some point Decisions can be created inside of the editor with modular actions. The goal is to represent Decisions as structs containing ConsiderationDefinitions that let designers create behaviour as in Mark Dave's talks, but these would still be in a NativeArray.

GetConsiderationScore is a switch case that just gets the right part of the state and score's it, then uses a static function depending on what the consideration is to get the raw score.

Code (CSharp):

1.  
2. case (int) ConsiderationFlag.DoesNotHaveLineOfSight:
3.                 return visiblity < 2 ? 1.0f:0.0f;
4. case (int) ConsiderationFlag.TargetInRange:  
5.                 return Consideration.TargetInRange(unitData, positions[unitEntity].Value, positions[target].Value);
6.  

Curves

Code (CSharp):

1.  
2. public static class ResponseCurve{
3.     public static float Exponentional(float x, float power=2){
4.         return math.pow(x,power);
5.     }
6.  
7.     public static float Linear(float x,float slope=1){
8.         return x*slope;
9.     }
10.  
11.     public static float Decay(float t, float mag){
12.         return math.pow(mag,t);
13.     }
14.  
15.     public static float Sigmoid(float t, float k){
16.         return k*t/ (k - t + 1);
17.     }
18. }
19.  

Code (CSharp):

1.  
2. public static float TargetInRange(UnitStats unit, float3 unitPos, float3 enemyPos){
3.             return math.clamp((math.distance(unitPos,enemyPos) - unit.min_range) / (unit.max_range - unit.min_range), 0.01f, 1.0f);
4. }
5.  

This is definitely not the optimized version, I could see calculating the raw % for ammo and health as great optimizations but, optimizations like that are not the goal when first implementing something like this. Now that it's mentioned I'll definitely get around to it one day. The bonus factor and minimum score generally discard most decisions so we aren't doing that many calculations.

In terms of caching it could be helpful if you elaborate but there is a momentum bonus given to actions and prioritization to what Decisions are scored first so that we just skip the least important actions, nothing is allocated for them. Running the system on a timer would help too but it's pretty fast too. I'm dumping mad NativeArrays in the position query system too and it runs pretty fast. All insights are appreciated though.

Thank you for the reminder, that was actually there in order to test whether or not it was slower but I never changed it back.

 

Yes I'm very familiar with utility ai. And my points stand, you would probably increase performance by a factor of at least 5 with my suggestions. None of which prevent making it all available in the editor either, there is more then one way to skin a cat.

 

Replying to a different thread by @kstothert, condensing here
https://forum.unity.com/threads/ecs-utility-ai.676993/

When you say "CAN perform these tasks" you mean an agent can perform that task on the entity with the components?

In general I see it this way:

What tasks can be performed on an entity is not static, it can differ on things like gameplay state. There are two approaches here:

1) Declare any possible action that can be performed to an entity, and filter invalid ones out in selection. This has the downside of increasing the dataset you need to process. It's also hard to predict everything that can be done on an entity, since some actions can be attached at runtime (ie Cure task on a status effect).

2) Expose a dynamic buffer that is the current set of actions that can be performed on the entity. This more easily allows for a variable number of actions, and the entity itself would have some component that tracks conditions for what is exposed.

Of these I think the dynamic buffer is the better choice.

Then when your agents look for actions to score they could do so with a query that gathers upp all entities with the dynamic buffer (I don't think we can query directly on a dynamic buffer yet, so probably need a tag component as well).

In your ActiveTask component you don't need to store the whole component, just a reference to it? Create some basic struct to identity the action/task being executed, ie an entity reference to the task definition, an entity ref to the issuer of the task and so on.

Just curious, how are you authoring your task/actions? How is the data stored?

 

I am considering Utility AI in ECS as following, which for me seams simplest solution.

Entity is an AI agent. I can have as many of them as I like. And that's only entity type, my Utility need to worry about.
This Entity Agents has buffer array, which stores utility I'd tasks. That can be either enumerator hard coded based, or flexible as data driven tasks. Doesn't matter, as only stores ids to tasks. Now you could consider it as single group per agent, which you test against.

For example one agent will store tasks group eat, sleep, work.
Other independent agent, can store drive, turn left, right, break, fire.
Now you can simply enable / disable agents with component tag.

Additionally, I would store in buffer array for each agent entity, curves ids. So ID 0 can be flat linear, 2 diagonal, 3 curve, 4 x^3 etc. Again, I can either hardcode values per I'd, or have them flexible read from data file. And as many I like.

Just store their curves behaviours corresponding to ids in Native array, or dedicated entity, or fixed enumerator if you like.

That's it really to it.
You could further expand upon agents, by adding flag pair, for each task type ids, which can tell, which task to ignore. So you could disable sleep in a group of eat, sleep, work, for that particular agent. Or disable eat.

Single system which does utility checks, can have switch, or if statements, with task logics. Downside to it is, you would have single system, to store all AI logic. But if that is an issue for any reason, see my comment below.

Now you got fairly powerful single system of Utility AI.


Of course, you can make utility AI more hard coded, using tags for eating, sleep, work etc,and match corresponding system. Now you split AI logic in separate systems. Which may be cleaner in some respect, to do so. But you get flooded potentially with components, to filter AI behaviors.

But that approach may be a bit faster, since you don't need load ids.


However, I like myself flexibility, since I am working along with modding features. Hence I go with my initial solution, bases on ids.

 

I've been looking into this a lot more since first reading about it here. Had a play and implemented a basic decision. Balance seems like it might be tough to get right with many considerations, and I'm still not entirely sure how to factor 'time' or 'momentum' into this. It seems that even with a tiny probably to pick an action/choice, if you run the sim enough times it will be picked, which makes the rate of decision changes somewhat 'tick rate' based? I wonder how to adjust probabilities based on time, eg. "chance to pick this within 1 second". Could set a tick rate per second, but adjusting it would change the 'change your mind' frequency. There is probably a mathematical solution based on delta time in here somewhere.

I'm considering having AI components on my entities that need behaviour. Thing is they would need access to a lot of other components where the data/facts are stored, I suppose this is primarily OK, but was interesting to see the approaches you guys are taking. I'll have to consider what benefits they have. Do you see a flaw in running an AI component directly on the entity then changing up tags?

Also, check out this post: https://forum.unity.com/threads/utility-ai-discussion.607561/#post-4533874
I found an Infinite Axis Utility UI Editor on github you may be interest in looking at.

 

If considerations don't change, then it shouldn't matter how often you run a sim.. the result should be mostly deterministic (there's usually a tiny random factor to choose between equally important tasks).

If you need that then you'd have a scoring consideration that would go up with time ie something like it stores the last time it was "run" and normalizes the difference between now and then against a value for how often you want the action run.

Yep this is a concern I have as well, I'm going between "just do the random access" or doing a "data gathering" phase.. but I'm just not convinced if the gains would be enough to justify it.

 

If you have any randomness in there at all, then with enough evaluations, you'll eventually pick the low probability thing, and I like the random factor. One potential idea is to have the random factor only change once per 'x' ms like its own fixed timestep, thus making the 'randomness' independent of framerate.

 

Well if your truly want randomness then sure. I've only been using a tiny tiny randomfactor on top of scores atm just to prevent two actions from having the exact same score.

 

Hello all. Apologies for bumping an old thread but I'm really intrigued by this discussion! I learned about GOAP while poking around on the internet the other night and have been learning about ECS/Jobs the past few days. Every time I think I've seen all the videos or read all the forums, I find a little goldmine and have to keep looking. I was contemplating the compatibility of something like GOAP with something like ECS when I found this post.

First off, I wondered if you guys had played around with this demo before:
https://gamedevelopment.tutsplus.co...d-action-planning-for-a-smarter-ai--cms-20793

https://github.com/sploreg/goap

It's old but it gives a simple working example of a basic GOAP system. As some of the comments point out, the pattern appears to solve forward from the start state to the end state, rather than more efficiently backtracking via something like a*. So, it's not optimized, but it helped me understand the structure of a GOAP system. (I still have yet to find a good diagram that really makes me think "oh, now I could implement my own GOAP system." I've tried drawing some of my own but they always end up leading me down rabbit holes).

One such rabbit hole led me to find a paper about LGOAP, which essentially tries to allow for high-to-low level goals so that more complex strategies can be computed in real time, accounting for short and long-term consequences:
http://eldar.mathstat.uoguelph.ca/dashlock/CIG2013/papers/paper_4.pdf

I couldn't find a link but another idea mentioned how the goals/plans could involve cooperation with other agents to achieve a very big goal.

Also, I had no idea that Unity was already working on some sort of GOAP-like implementation! Is there any news on it? I can't find another reference to it anywhere.

 

On a related note, I saw a talk on YouTube where some AAA devs said that the average GOAP plan in FEAR and Shadow of Mordor was 2-3 actions long (eg: run to table, flip table, use table as cover).

My theory is that the power of GOAP would be a lot more impressive if actions happened little more closely together, and plans were a little more intricate (eg: block right punch, dodge left kick, knock opponent A off balance, turn to opponent B, then maybe opponent B interrups the plan with his own attack, so now our goal includes getting around opponent B). Some plans would be getting interrupted more often than others, but in a way you'd have a dynamic way of choreographing something that is normally scripted. If I were to add some props in the scene to add the possibility that someone could use them as a melee weapon, the complexity increases as the number of items increases, but then so has our computational efficiency.

Having a global planner doesn't really align with the ECS paradigm so well, but just spitballing, is there a reason someone couldn't implement a parallel GOAP by having local planner entities for each NPC? The NPCs would only be acting upon the facts of which they're aware, stored in their own local cache. Then, because the caches are separate, the planning process could be jobified without concern for concurrency issues when accessing data about the world. One could cap the size of the NPC's local cache to manage space.

In my mind, this pseudo design doesn't seem impossible, though it's certainly above my skill to experiment with ECS at the moment.

 

Jaw is on the floor. Did not know about this. Thanks!

 

GOAP has been tried many time in game, with little success relative to the much more flexible behavior tree, which can be seen as a kind of planning (it's a search or action through a tree) and currently planning is mostly reserved to strategy game which use a varient called HTN, which closely resemble behavior tree anyway. The thing is that you could probably has an action queue to a behavior tree to get a full planning system. FEAR has been shown by the creator to be more successful due to how it encode special movement in the level, but more importantly by the richness of the audiovisual feedback to the player so she can understand the state of the action (for some reason that's the less linked paper of the creators of the game, I wonder why, I had linked it somewhere on the thread).

But a working planning system has been successfully attempted and documented for games, in the ground breaking game "Versu".
https://versu.com/about/how-versu-works/
Still the absolute reference in ai in games.

EDIT:
IN fact, the modern discussion around goap, has move toward finding way to bake the planning tree toward a behavior tree. Ie just set the actions and let the system build the BT procedurally.

 

Just a few games using GOAP:

• F.E.A.R
• Condemned
  
• S.T.A.L.K.E.R.
• Fallout 3
• Tomb Raider (2013)
  
• Empire: Total War
  
• Middle-earth Shadow of Mordor
• Just Cause 2
  
• Deus Ex: Human Revolution

Yes, behavior trees are more popular but GOAP is actually more flexible and easier to use in terms of setting up AI reactions. You do not need to write every possible interaction the AI can have. The planner automatically figures out all that for you. This is however a trade-off since it means you have less control over exactly what happens. But you do not have to use just one AI system or the other in your game. You can actually combine approaches in all kinds of interesting ways.

I do not understand your point here. Are you saying it's more successful in F.E.A.R because it looks good? That it is somewhat superficial? In any case I think you are greatly undermining it's power and utility. It's fine if you don't like GOAP or feel it doesn't work for your designs. But it has it's place and usefulness and has been quite successfully used.

 

Every post mortem I read about the stuff are basically smear of disappointment, it's never use twice in the same game series generally, and Alex J Champaniard has show (aigamedev.com, Killzone) has shown that BT is generalization of all 3 major game ai (state machine, script and planner), it's basically a directed search. The case of an enemy going to cover by flipping a table can be done with BT. Planner fair better in complex game like strategy or simulation.

I don't think it's superficial, I was paraphrasing the dev of FEAR in one of their post mortem paper (which I should link back but I'm lazy now). Basically my intention isn't to say GOAP is bad, I'm mostly trying to convey a cautious YAGNI. As in it's not magical, don't give it too much mystic. I redirected at another planning implementation with versu too, which have a planner that can anticipate future consequence and reason lightly on them too. I'm nerdy enough I can make a compilation of all talk and post mortem that mention planning, I don't have tume now, but maybe in the future.

But ultimately, it's not GOAP or planning that is the problem. Generally the problem is game design. You have to be cautious about any fancy AI solution, because like procedural, it's used as an excuse to let an algorithm design your game. Good design will trump fancy algorithm everytimes, because if you have the smartest shooter enemy but level are corridor, you won't see them do fancy flanking maneuver. That is, in before the algorithm got done, you need the proper level design to support it, as an example. For example, the main reason GOAP was generally a disapointment in most post mortem, is the nature of the game it is used, action enemy only need reactive reasoning, and also don't live long enough to carry long plan, and it is more efficient to use other AI to get the same result, and the need for action to be readable mean you will have a lot of boiler plate to prevent the planner to get seemingly random.

Also GOAP is myopic, it's hard to handle coordination with it, and it also solving the design experience from the point of view of the agent not of the player, which is the main thing fear agent does well, they bark their state everytime, emote visibly and tells their plan out loud, it's about the player experience. And time and time again, every post mortem tells you how much more important that feedback to the player is the real key (AI dev remember soldier of fortune AI that were able to exchange ammo between them, but behind cover silently, and the player couldn't tell, so it was useless, or that MMO I forget the name (conan?) where npc had full ecologic and schedule, but the nature of the player loop make it indistinct from any other mmo that just spawn stuff. Every milestone in game AI was about that feedback, especially in shooter (golden eye, half life, halo, fear) good feedback to the player is enough to gave an illusion of an ai way smarter that it actually is, and also smart level design that gave ai more thing to do.

 

Well it was used in F.E.A.R and it's sequel F.E.A.R 2, Tomb Raider and it's sequel Rise of the Tomb Raider, Shadow of Mordor and it's sequel Shadow of War, etc. It's also been used in most of the games at various studios that have implemented it for years, without the need to really even change it much between projects. So it would seem you're being highly selective and bias in your research on the topic to say the least. I would suggest reading some actual success stores on the topic sometime too.

That's not the impression you gave at all! But maybe it's just a language barrier issue. I would also never claim GOAP is magic. It's simply one of many tools, and you should of course use the right tool for the job. However, you said in your last two posts that GOAP was pretty much a failure in most projects it's been used in, and that's simply not true.

In any case you oblivious don't like GOAP and that's totally fine but you seem to be criticizing something you've never personally used or understood. Or at least criticizing it for things that have nothing to do with it! Like incorrect usage. Which I think is completely unfair.

 

I'm curious, could you help the system avoid some of the drawbacks of GOAP that have been mentioned by using ML agents? Surely some decisions warrant thorough search space coverage every time a new plan is computed (planning tactics in a firefight) but maybe the most repeated ones can be learned and optimized (leaving ammo in covered zones is a good strategy)?

Part of the dev process would be your classic 3D world building process, but part of it would become taking the characters you made and teaching them how they work! You would design training scenarios to compute the most lifelike ensemble of ML models and planning systems, and then you would give it time to run all these scenarios.

Heck, maybe you use an imitation learning model to duplicate how you play, then you get a bunch of NPCs to work out the best plans to defeat your strategies, and you run that simulation in parallel for awhile. Like baking lightmaps but you're baking AI instead!

 

@growling_egg reheating this old thread again!
As far as I could judge (3 years stint in an ML startup but not very close to core dev; that and looking at Unity's ML resources) what you described can already be realized... however...

The effort right now to setup the training correctly, run it, and put the result back in your game, does not make this a pragmatic option today.
You'd do it experimentally, because it's interesting, and forward looking. In the same way was reading a recent article about some band who baked a new album by training AIs on their past songs... very clunky, very awkward process; good result (according to the musicians themselves and some critics) but workflow wise, experimental is what it's at.

GOAP isn't as inflexible as it seems in terms of directing AIs towards certain results/play styles. You can adjust preconditions and cost functions to bias towards what you're interested in... In fact viewed a few talks about procedural/AI design this summer, and there was a lot of convergence around "want AI/procedural? Frame it as well as you can, work closely with designers, have a set of sample output that you want your system to be compatible with.

Having said that GOAP as far as I could judge specifically refers to a very contrived planner used in FEAR, where the planning aspect didn't interest them per se, and they were (like BT and Halo 2) trying to develop their design while avoiding some of the pitfalls of FSM.

The beauty of GOAP is that planning-wise it really is a toy. Great for learning and experimenting with planning. Scalable not so much; extensible? tons of possibilities.

 

Do you guys know much about this repo by DreamersIncStudios?

https://github.com/DreamersIncStudios/ECS-IAUS-sytstem/tree/Alpha-0.7-new

The last commit was only 3 days ago and the author seems has been pretty active with it.

 

I haven't looked at this repo, but if I was going to recommend any AI approach in DOTS it would be Infinite Axis Utility System which is from the readme what this implements. It fits the concept of ECS perfectly and what we use in our project.

-edit-

to add onto this. it scales really well. This is 8,000 AI controlled entities updating in a frame (decision system group), 0.5ms main thread, about 1ms in jobs [in editor]. As you can see it has decent thread utilization.

I didn't write this but did review the implementation and believe there are ways to speed this up considerably if required (inverting some logic to remove branching and allow vectorization). However we just don't need it (we have a lot less entities updating than this.)



So yeah if you're looking for an AI solution, without having actually read the above repo, I'd highly recommend looking into it as it's a great approach.

 

I'm working on an IAUS, and in order to avoid a single blob of a system with every possible consideration, I've made each action an entity, and each consideration a component. Normally this could be memory hungry, so I store considerations as a chunk component with a BlobAssetReference, storing each consideration once per chunk.

This allows flexible/extendible consideration logic, where all I need to do is add a system, at the cost of the job needing to be an IJobChunk(no ForEach chunk component access yet). I can even order the systems from least to most costly, and once per component enable/disable lands, disable the action before the rest process.

 

I've only been studying this for a day now and even after a few hours it became obvious to me that IAUS (or just Utility AI in general) is the way to go. I read about the other methods (finite state machines, behavior trees, goap, etc) and I was getting discouraged, and then suddenly Utility AI appeared and I think I actually said "This is so cool!" out loud. Even with OOP I would be using it, but with ECS it seems like a perfect fit. It's good to know that some of the heavy-hitters in the forums like you are recommending it - it means I'm on the right track.

I tried compiling the dev version of that DreamerIncStudios repo, but it had a lot of errors - I think he is in the middle of a rewrite.

Of all the Utility AI approaches I've seen so far, I like the solution that Antypodish proposed... my AI is going to be pretty basic and his idea seems like it would be easy to implement. I like the idea of storing each curve by ID - I could even put m, k, b, and c in there as well if I want to take the Infinite Axis approach. Storing actions on agents by ID is nice and lightweight. Having a single system doesn't seems so bad because I can always break it up into multiple functions for different logistics.

 

Is there any more detailed docs on how is it used? Like how to add considerations, how to add actions? How to implement the actions?

 

Not that I know of. The author has a wordpress blog here: https://dreamerincblog.wordpress.com/ , with a tiny bit of information and he's in these forums under the name @DreamersINC .

 

I really do need to do a tutorial write and update blog post. They are major things on the list of to do. In Alpha-0.7, I have pretty much did major restart. The design of the system is for each Action State Component to be added individual and the appropriate considerations are add at running. For example, for the patrol action, you would drag and drop the patrol component, which uses the Generate Authoring Component, and assign the Response Variable or create your own authoring script. At runtime, IAUSBrainSetupSystem runs and calls AddPartolState Job which adds which adds the distance consideration if it has not already been added. I am working on trying to make it more user friendly as well as performant. I have run into a few issue with performance on console as well as slow down with SMT on with my 1800x. If you find bug or have concerns, submit an issue on the repo.

 

What compile errors have you run into ? FYI I am on using 2019.4.18 with Entities 11.2. I just got tried of refactoring frequently and overall bugs and issues with non LTS versions. I will possibly update the project to 2020 LTS and latest version of DOTS at the time.

 

I'm currently pondering ways to avoid unnecessary calculations. With IAUS, the scores all need to be added up periodically, which means that the algorithms to generate these scores need to be run periodically. However, while it may be obvious to the developer that a certain expensive algorithm can be avoided due to the score of other variables, the score summing system does not account for this. GOAP and Behavior Trees do not have this issue.

A quick example:
There is MoveToNearestEnemy action which depends on 2 considerations:

• as health increases, utility increases
• if a path to a nearby enemy > 1, utility increases.

For this example, let's pretend that the path search is very computationally slow. If it is known that health is super low, and there is an alternative action that score highly, then we should be able to avoid calculating the search path. But with standard Utility AI, that path needs to be calculated even if there is no way that action will be chosen as the winner. CPU cycles get wasted.

I'm sure there are a lot of other examples where an expensive operation could be avoided.

1. One option is to just ignore it and do the expensive extra calculations anyway. It's not efficient, but maybe this is the standard approach?
2. Another option might be to put some fancy logic into the scoring system so that it performs the action scoring calculations in order of least complex to most complex. It would also goes through each consideration in order of least complex to most complex and short out if a consideration score is so low that no matter what the other considerations are for the current action, they can't possibly beat the current winning action score.
3. Something else?

Has anyone run across any literature on solving this issue? Ideas?

 

I could see a two-layer approach where the first pass calculates easy evaluations and generates a min-max range for action, and then any challenging axis that is only an input to actions that aren't viable can have its evaluation skipped in the second pass.

However, a downside to this technique is that it can be prone to frame spikes if not tuned well. So personally, I would first try to rely on change filters or expiring evaluation shortcut caches and see if those resolve things first.

 

If you spatialize your world and the features in it (interests), then you can always get away with a simple comparison:
- do I have a high interest action in the same spatial volume? if yes, save the computation and score that high
- if I don't have a high interest action in the vicinity, or there are possibly higher scored actions in "N" distance, then let's run the pathfinding for those actions which are in "N" distance and score them

You're basically trading on-the-fly pathfinding for some memory and maintaining another quadtree. I like to think of it as pathfinding mipmapping. When you zoom out, you use the simplified quadtree, when you zoom in, run the more detailed pathfinding.
Of course, there are other considerations too: if you have unreachable territories, it is worth to maintain separate signal for them (store the fact they are unreachable and don't include in the "possible pool of actions" at the first place).

 

Thanks for all the answers!

Just for clarification on terminology - An agent can have 1 or more actions (which are essentially the same as decisions). Each action is based on 1 or more considerations (Dave Mark sometimes calls them axis as well). A scoring system loops through each action, sums up their considerations, and then presents a final winning score for all actions, which is the action that will be taken.

@DreamingImLatios - That two pass approach seems reasonable (making two categories: easy and hard). Is the "expiring evaluation shortcut caches" idea is similar to the #2 in my options? I've never been a fan of the change-filter stuff - it has so many limitations and idiosyncrasies that I typically end up getting frustrated and ditching it. It only works per-chunk, if you write to one all are invalidated, etc.

@Lurking-Ninja - I think that pathfinding mip-mapping proposal is a clever solution for the example I gave, but the path navigation I mentioned was just an example of an algorithm that has a high time complexity. I need a solution to handle any algorithm that is slow. For example, algorithms such as "calculating morale" or "finding the best cover" could be really slow and we would want to minimize how often they are executed.

@RecursiveEclipse - That is an interesting architecture for sure. Mine is simpler than that (it's mostly arrays and lookups instead of separate components for each consideration). At this time I'm not so much interested in the architecture as I am concerning the issue of minimizing the running of expensive algorithms by eliminating the need to score certain actions. #1 in your list addresses that. It means designating a field, perhaps a byte or something that specifies a complexity value for an action and/or consideration. Obviously a way to enable/disable certain scoring algorithms goes along with that. It is similar to the #2 option in my list.

------------

I'm still bouncing around a few different ideas right now, but I haven't settled on anything concrete yet. I went back to some of Dave Mark's talks on IAUS, but I don't see anywhere that he addresses the issue of minimizing the running of complex algorithms. If all calculations needed to score an action or consideration were identical in complexity, what he describes would work quite easily.

 

Well, given that any one AI is running infrequently at the first place, I don't think they would setup any mitigation against "slow" calculations. Also you can distribute this load over many frames, if your AI is running in every second (the evaluation, not the execution), you have at least 60 frames to distribute your heavy calculations for the given AI. It is very rare when you need to run your evaluation in every frame.
But you also can decouple these things from each other, you run your AI and query the distances from actions array. But you calculate the distances from actions array separately and just update it once in a while. Obviously, just replace the distance example with any expensive calculation in this.