AI Planner

The AI Planner was released as a preview package during GDC 2019 and was mentioned in the GDC Talk: Unity AI and Machine Learning Tools for Behavior Creation

The sample project for the planner is located here: https://github.com/Unity-Technologies/otto

Additionally, you can install the package separately in your project by selecting "Show preview packages" under Advanced drop-down in the Package Manager and searching for "AI Planner".

Please use this new thread for any questions you have about Otto or the AI Planner. You're also welcome to start new threads. We will try to monitor this forum as best we can.

 

I went to create some sort of hello world for this. I just want to create one action with one effect which just displays a Debug.Log(). I don't get creating the action part.

I created a domain with one trait named Condition, which has a field named Value which is just a bool.


But in action creation, I couldn't get Condition.Value to display in either precondition or effect:


My intent was for the action to have a precondition Agent.Condition.Value = false, then have an effect Agent.Condition.Value = true.

 

Nevermind. So you have to click Generate Classes for them to be displayed on the editor.

 

So I generated my TestAction. One thing I didn't know how to do is how to set the goal. For the TestAction to be invoked, I have to set a goal where Condition.value = true, right? But I can't find where to do this.

Anyways, so I made a GameObject and attached my agent class (the one deriving from BaseAgent) then set the Plan Definition. However, when I played the scene with just this GameObject, I get the following error:

ArgumentException: SimpleDomain.Condition is an IComponentData, and thus must be blittable (No managed object is allowed on the struct).
Unity.Entities.TypeManager.BuildComponentType (System.Type type, System.Int32* writeGroups, System.Int32 writeGroupCount) (at Library/PackageCache/com.unity.entities@0.0.12-preview.24/Unity.Entities/Types/TypeManager.cs:705)
Unity.Entities.TypeManager.BuildComponentType (System.Type type) (at Library/PackageCache/com.unity.entities@0.0.12-preview.24/Unity.Entities/Types/TypeManager.cs:680)
Unity.Entities.TypeManager.CreateTypeIndexThreadSafe (System.Type type) (at Library/PackageCache/com.unity.entities@0.0.12-preview.24/Unity.Entities/Types/TypeManager.cs:643)
Unity.Entities.TypeManager.GetTypeIndex (System.Type type) (at Library/PackageCache/com.unity.entities@0.0.12-preview.24/Unity.Entities/Types/TypeManager.cs:502)
Unity.Entities.TypeManager.GetTypeIndex[T] () (at Library/PackageCache/com.unity.entities@0.0.12-preview.24/Unity.Entities/Types/TypeManager.cs:493)
Unity.Entities.ComponentType.Create[T] () (at Library/PackageCache/com.unity.entities@0.0.12-preview.24/Unity.Entities/Types/ComponentType.cs:31)
SimpleDomain.SimpleDomainUpdateSystem.OnCreateManager () (at Assets/AI.Planner/Generated/SimpleDomain/SimpleDomain.cs:49)
Unity.Entities.ScriptBehaviourManager.CreateInstance (Unity.Entities.World world) (at Library/PackageCache/com.unity.entities@0.0.12-preview.24/Unity.Entities/ScriptBehaviourManager.cs:40)
Unity.Entities.World.AddManager[T] (T manager) (at Library/PackageCache/com.unity.entities@0.0.12-preview.24/Unity.Entities/Injection/World.cs:244)
Unity.Entities.World.CreateManagerInternal (System.Type type, System.Object[] constructorArguments) (at Library/PackageCache/com.unity.entities@0.0.12-preview.24/Unity.Entities/Injection/World.cs:131)
Unity.Entities.World.GetOrCreateManagerInternal (System.Type type) (at Library/PackageCache/com.unity.entities@0.0.12-preview.24/Unity.Entities/Injection/World.cs:147)
Unity.Entities.World.GetOrCreateManager (System.Type type) (at Library/PackageCache/com.unity.entities@0.0.12-preview.24/Unity.Entities/Injection/World.cs:227)
Unity.AI.Planner.PlannerSystem.GetBehaviourManagerAndLogException (Unity.Entities.World world, System.Type type) (at Library/PackageCache/com.unity.ai.planner@0.0.1-preview.2/Runtime/Planner/Planner.cs:549)
UnityEngine.Debug:LogException(Exception)
Unity.AI.Planner.PlannerSystem:GetBehaviourManagerAndLogException(World, Type) (at Library/PackageCache/com.unity.ai.planner@0.0.1-preview.2/Runtime/Planner/Planner.cs:553)
Unity.AI.Planner.PlannerSystem:CreateBehaviourManagersForMatchingTypes(Boolean, IEnumerable`1, World) (at Library/PackageCache/com.unity.ai.planner@0.0.1-preview.2/Runtime/Planner/Planner.cs:541)
Unity.AI.Planner.PlannerSystem:Initialize(World) (at Library/PackageCache/com.unity.ai.planner@0.0.1-preview.2/Runtime/Planner/Planner.cs:411)
UnityEngine.AI.Planner.Agent.BaseAgent`1:Start()

The following is my test action:

Code (CSharp):

1. public class TestAction : SimplePlannerAction {
2.     public override void BeginExecution(Entity stateEntity, ActionContext action, SimplePlanner agent) {
3.         Debug.Log("TestAction.Begin");
4.     }
5.  
6.     public override void ContinueExecution(Entity stateEntity, ActionContext action, SimplePlanner agent) {
7.         Debug.Log("TestAction.Continue");
8.     }
9.  
10.     public override void EndExecution(Entity stateEntity, ActionContext action, SimplePlanner agent) {
11.         Debug.Log("TestAction.End");
12.     }
13.  
14.     public override OperationalActionStatus Status(Entity stateEntity, ActionContext action, SimplePlanner agent) {
15.         return OperationalActionStatus.Completed;
16.     }
17. }

 

So I haven't given up yet and made Condition.Value into an int so that the resulting IComponentData would be blittable. When I played the scene again, I got this error:

ArgumentException: The entity does not exist
Unity.Entities.EntityDataManager.AssertEntityHasComponent (Unity.Entities.Entity entity, Unity.Entities.ComponentType componentType) (at Library/PackageCache/com.unity.entities@0.0.12-preview.24/Unity.Entities/EntityDataManager.cs:369)
Unity.Entities.EntityDataManager.AssertEntityHasComponent (Unity.Entities.Entity entity, System.Int32 componentType) (at Library/PackageCache/com.unity.entities@0.0.12-preview.24/Unity.Entities/EntityDataManager.cs:377)
Unity.Entities.EntityManager.GetBuffer[T] (Unity.Entities.Entity entity) (at Library/PackageCache/com.unity.entities@0.0.12-preview.24/Unity.Entities/EntityManager.cs:831)
SimpleDomain.SimpleDomainUpdateSystem.StateEquals (Unity.Entities.Entity lhsStateEntity, Unity.Entities.Entity rhsStateEntity) (at Assets/AI.Planner/Generated/SimpleDomain/SimpleDomain.cs:87)
Unity.AI.Planner.PolicyGraphUpdateSystem.LookupState (Unity.AI.Planner.HashCode stateHash, Unity.Entities.Entity stateEntity, Unity.Entities.Entity& matchedStateEntity) (at Library/PackageCache/com.unity.ai.planner@0.0.1-preview.2/Runtime/Planner/Planner.cs:711)
Unity.AI.Planner.PolicyGraphContainer.UpdateRoot (Unity.Entities.Entity stateEntity, System.Int32 searchHorizon) (at Library/PackageCache/com.unity.ai.planner@0.0.1-preview.2/Runtime/Planner/Planner.cs:235)
Unity.AI.Planner.PolicyGraphContainer.UpdatePlan (Unity.Entities.Entity stateEntity) (at Library/PackageCache/com.unity.ai.planner@0.0.1-preview.2/Runtime/Planner/Planner.cs:246)
Unity.AI.Planner.Agent.Controller`1[TAgent].CompleteAction () (at Library/PackageCache/com.unity.ai.planner@0.0.1-preview.2/Runtime/Agent/Controller.cs:123)
Unity.AI.Planner.Agent.Controller`1[TAgent].Update () (at Library/PackageCache/com.unity.ai.planner@0.0.1-preview.2/Runtime/Agent/Controller.cs:76)
UnityEngine.AI.Planner.Agent.BaseAgent`1[TAgent].Update () (at Library/PackageCache/com.unity.ai.planner@0.0.1-preview.2/Runtime/Agent/BaseAgent.cs:161)

I tried adding GameObjectEntity to my agent, but it's still shows the same error. What am I missing?

 

Did you set up your initial state on your Agent script?

 

What do I set it with? I only have one trait so far:

 

Hmm. Looks okay, but have you tried removing the trait from the state and only having it on the agent?

 

I know that modding for unity is in general a highly discussed topic, but do you see any opportunity in this AI planner that would allow some kind of AI modding.

I'm just thinking if there would be the possibility of having a "definition language" for "Plan Definition files" that modders could manipulate to add traits, effects, pre-conditions in different combinations which could lead to completely new behavior based on the actions written by the developer?

 

Can you share a bit more about what you're looking to do? Plan Definition files are currently just a yaml asset, but there's no reason why we couldn't serialize that out to a json or other format, too. Or, are you looking for a way to programmatically generate plan definitions at runtime?

 

Tried removing the Initial State Trait Data. It's still the same error:

 

Could you post a small .unitypackage or make it available somewhere with your domain/plan definitions and possibly the sample scene?

 

1. an off-topic question: how will AI-Planner work with ml-agents ? seems a bit functionality overlapping?

IMHO, AI-Planner is more suitable for high-level behavior, maybe use it to change ml-agent's brain?

2. how do i enable virtual studio interp(syntax highlight) with csc.rsp's define (like WORKAHOLICDOMAIN_GENERATED in otto demo)?

 

An example would be that Traits and Enumerations could be defined in XML, that will then be used to generate the actual classes in runtime, for example. Like in the example of the package documentation, a modder could extend the ConsumableType and add new values to it. Instead having Apple and Bottle, it could contain Apple, Bottle and Bread.

Regarding action in an FPS game, I could create the Action "Open Door". A precondition would be "has Key", to open the door. Also here, when able to define everything in an external file, a modder could add the precondition "has Weapon", this way the Agent could only open the door when having a key and a weapon.

These are very basic examples with not such a great use-case, but depending on how much actions you think of for your agent, some other developers could come up with completely new combinations of actions and traits that could lead to brand new behavior.

Also if you think about chaning rewards or costs. You could simply mod a FSP to melee only, because the costs for ranged attacks are too high.

I had the X Series in mind when thinking about that. From what I know they have a lot of AI logic in XML files and there are quite some extensive mods out there that drastically change the behavior and therefor the game.

Of course XML is just an example, but the idea is to have an easily modifiable file format for modders.

 

Do you have a Bitbucket account? I just added my test scenes in Otto project. It's still big when it's stripped off. I'll add you to my Bitbucket repository where I stored my project.

 

Had a couple of questions to see if this was worth exploring.

First question is how well does it integrate with external data, and can it handle loading/unloading larger (several hundred mb in memory) datasets at runtime?

Second question is how does it perform with a few hundred agents and is the main work done in jobs? I saw at a glance some ECS in there but didn't dig very deep.

 

For 1. - You should watch the talk when it is available I'll update the main message in this thread to point to it when it becomes available in the vault. I'm not sure if it will be a free talk though. However, to answer your question roughly, yes, it will be suitable for more higher level behavior that is more discrete in nature in terms of decision-making. ML-Agents work particularly well for character control and other problems as you move towards continuous problem domains.

For 2. - I'm not entirely sure as I see the same behavior in Rider. This isn't likely our final solution for this -- it was mainly a convenience, so if I delete the generated directories or the csc.rsp file that the project will still compile correctly and compile out the parts that are dependent on the generated classes. It's mainly there for when I need to clean and re-generate all files.

 

Hmm.. If you're using the Otto project, how about just forking it in github and then send me the link to your fork/branch and I can take a look?

 

To your first question - this is untested currently, but we did experiment at one point with serializing plans / policy graphs that had been pre-computed. I can possibly answer better if you share more about what these larger data sets look like.

Re: your second question - we are currently in the process of jobifying the planner (as of now it runs on the main thread); We're creating performance tests to verify progress as we jobify. Our plan is to support multiple planners / agents running in a single game with possible caps for memory usage/computation for each planner.

 

We'll likely support XML and json as formats for import. Our system currently requires code generation for performance reasons and to work well with DOTS/ECS. The idea is that as more people use the system that there can be domain definitions defined that can get shared around and that many people could define plan definitions that operate within those domains to get specific behavior. It is possible currently to modify the reward function. See https://github.com/Unity-Technologi...nner/WorkaholicDomain-Custom/Work.Extra.cs#L7

 

Our current ai system is largely based on rather dense environment data. Some we pre bake and then stream in and out as the player moves around, as it's too large to fit into memory all at once. Example of this is height data for the map. Some is semi dynamic like best cover/los data that is rebuilt when structures are added/remove at runtime. Some is completely dynamic and rebuilt every frame. Spatial maps of npc's by type/faction and damage received/done lists.

Most of this is currently used for target selection, fight or flight triggers, and best location to move to. We use a fairly brute force approach right now with all of the data in a 2d grid structure, and for every npc iterate over nearby cells in the grid. Which with burst actually scales fairly well.

 

Well, for the purposes of the planner this opens up the question of data modeling in the domain. Ideally, you'd minimize your state to the minimum amount of data needed to make the decisions you need to make. For example, if you simply throw a bunch of locations into the planner and have a simple Travel action with no constraints, then at every step of the planner it's going to be exploding on traveling between locations. Now, if that is the purpose of the domain and you want to reward optimal travel, then so be it. However, if travel is a minor action among other actions, then it's possible that you might want to abstract that out further in your domain definition. Ideally, you'll model your data in such a way that the planner is making important decisions at each step in the graph rather than ones that are trivial.

This showed up for us when we were working with the Overcooked game and broke down the actions into all the atomic things you can do in the game. There were something like 35 steps necessary to cook a single pot of soup (e.g. individual actions for traveling, gathering ingredients, chopping, etc.), which although the planner could converge on that solution wasn't actually that impressive because we already knew the steps. What was more interesting when we remodeled the domain was that it would figure out efficiencies of where to put the pot to optimize the preparation and delivery of soup.

 

The otto sample project has been updated: https://github.com/Unity-Technologies/otto

Includes: com.unity.ai.planner@0.0.1-preview.4 and the latest entities package
Release notes: https://docs.unity3d.com/Packages/com.unity.ai.planner@0.0/changelog/CHANGELOG.html

 

Thank you for this. Before I dive in, what are the drawbacks if any (runtime allocations, perf etc)?

 

We haven't done an in-depth analysis, but if you run the profiler you'll see stable memory usage. There's still some allocations that I'm seeing (~3-5K a frame) that we'll have to look into at some point. Using DOTS/ECS has reduced heap allocations by quite a lot over our older unreleased, non-ECS version.

 

Thanks, do you believe that at some point, the runtime allocations will be removed? I'm not able to use anything in production that has ongoing allocations.

And to be honest, I really want to embrace this in production, if it's anywhere near as cool as I think it is. It would allow us to simulate a little world, which is our use-case. But memory pressure is something we could do well to avoid.

Our title is not dots based at the moment. Can you advise? thanks.

 

Try this one: https://bitbucket.org/marnel_estrada/aiplannertest. Open the HelloPlanner scene.

 

Hi, I tried to open the project in Unity 2019.1.0b10, I get a lot of errors related to the fbx files

ImportFBX Errors:
Couldn't read file <project location>/otto-master3/otto-master/Assets/Models/Otto/Otto.FBX.
Unexpected file type


UnityEngine.GUIUtilityrocessEvent(Int32, IntPtr)

followed by

Failed to load 'C:/Users/RICHARD/Documents/otto-master3/otto-master/Assets/Models/Otto/Animation/Item_PocketDrink.anim' because it was serialized with a newer version of Unity. (Has a higher SerializedFile version)
UnityEngine.GUIUtilityrocessEvent(Int32, IntPtr)

And I see this error on the import setting


And I get this for all the animation files :
Failed to load 'C:/Users/RICHARD/Documents/otto-master/otto-master/Assets/Models/Otto/Animation/Item_PocketEat.anim' because it was serialized with a newer version of Unity. (Has a higher SerializedFile version)

And Skinned mesh rendered for yolo does not have a mesh

 

Hey, i had the same problem when i downloaded the project as [zip].
I got it to work after cloning the whole repo and opening that instead.

 

Yes, we'd like to remove any managed heap allocations that can be removed or at least create caches where we can, so we are not allocating every frame.

It's okay that your title is not DOTS-based. Although our solution uses DOTS it does not require you to. The sample otto project is a standard MonoBehaviour-based project.

 

Hmmm.. I just cloned a fresh copy of https://github.com/Unity-Technologies/otto and opened it in 2019.1.0b10 and the only thing I get are warnings from the shaders. Did you update an existing working copy? Have you tried reimporting all / deleting your Library folder?

 

Ahh...Now I see - this is probably the same issue for you @richard141289. You can't just download a zip file because large files will not be included, which are pretty much any of the FBX files and textures, etc.

 

the talk is up on the vault on this page https://www.gdcvault.com/browse/sponsored-all ( just search for text Unity AI and Machine Learning Tools for Behavior Creation )

looking forward to using the AI Planner, happy to see that it is a planner and not just a behaviour tree or similar. I've worked with GOAP planners in the past from from a level design/combat point of view in FPS games (FEAR series) and think they are great for emergent AI and interesting situations.

 

Thanks, @XRA. I've updated the original post. To others, here is the link for convenience: https://www.gdcvault.com/play/1026172/

 

I'm not able to view the video of gdcvault. Is it available elsewhere?

 

Unfortunately, no. Are you outside of the country or are you behind a vpn/corporate internet that may block domains?

Here is the underlying video link when I view page source:
https://cdnapisec.kaltura.com/html5..._enxteqnv&videoid=0ca9bcd966c144ee76fb1026172


The only video that we have on the AI Planner prior to this is from Unite LA from 2018:

 

I've tried with and without NordVPN (and also with private browsing). I can enable vpn and pretend to be from another country (I tried connecting to U.S. servers on VPN but that too didn't work) .

https://cdnapisec.kaltura.com/html5..._enxteqnv&videoid=0ca9bcd966c144ee76fb1026172 also suffers from same issue

 

Importing the AI Planner works perfectly in a new 2019.1 project, but when I try to use it in a project which I've just upgraded to 2019.1, I get the following error:

Library/PackageCache/com.unity.ai.planner@0.0.1-preview.4/Runtime/Planner/Planner.cs(590,16): error CS0246: The type or namespace name 'ScriptBehaviourManager' could not be found (are you missing a using directive or an assembly reference?)

The page in the Unity documentation for ScriptBehaviourManager seems to have been moved recently and returns a 404.

Grateful for any help.

 

Even I get the unauthorized domain message and I'm here in the Bay Area, California. I mean the kaltura.com domain. I can see the video on the GDCVault website.

 

@PeterB Just ran into this issue too. Turns out ScriptBehaviourManager was removed from the Entities package in preview.30 and samples 0.0.27. So you need to downgrade Entities package to preview.29 until they update the AI planner package to work with this newer version. Unfortunately this also means you can't use the new hybrid renderer and AI planner at the same time yet.

 

@Lurking-Ninja, Just to clarify -- did you mean that https://www.gdcvault.com/play/1026172/ works for you?

 

Yepp.

 

We'll be coming out with an update soon that will use the latest Entities package.

 

Survey:
For those of you who are using the planner already, what is the one thing that would help you most?
For those of you who tried the planner, but are not going to use it yet -- what would unblock you?

 

I'm really looking foward to using this system on a new project.

For me, editor/UI stability is my biggest concern. I realize it's early in development, but I can't seem to use the system in its current state.
Some core issues:

1. Accidentally hitting "generate classes" without any domain traits or enums defined freezes the editor indefinitely
2. in this example project, adding a precondition to the "Buy" action, then selecting the "buyer" as the first part of the expression will cause a null ref exception and break the editor window.
3. After generating classes, I'm consistently getting an error that HashCode is inaccessible due to its protection level, as well as some other errors (listed a the bottom)

This is all using Unity 2019.1.0f2

I haven't been able to get much further so far.



Other errors, copied from visual studio:


Severity Code Description Project File Line Suppression State
Error CS0534 'DomainDefUpdateSystem' does not implement inherited abstract member 'PolicyGraphUpdateSystem.HashState(Entity)' Assembly-CSharp C:\projects\ai_planner_error_sample\Assets\AI.Planner\Generated\DomainDef\DomainDef.cs 46 Active
Error CS0122 'HashCode' is inaccessible due to its protection level Assembly-CSharp C:\projects\ai_planner_error_sample\Assets\AI.Planner\Generated\DomainDef\DomainDef.cs 72 Active
Error CS0115 'DomainDefUpdateSystem.HashState(Entity)': no suitable method found to override Assembly-CSharp C:\projects\ai_planner_error_sample\Assets\AI.Planner\Generated\DomainDef\DomainDef.cs 72 Active
Error CS0122 'HashCode' is inaccessible due to its protection level Assembly-CSharp C:\projects\ai_planner_error_sample\Assets\AI.Planner\Generated\DomainDef\DomainDef.cs 75 Active
Error CS0122 'HashCode.Value' is inaccessible due to its protection level Assembly-CSharp C:\projects\ai_planner_error_sample\Assets\AI.Planner\Generated\DomainDef\DomainDef.cs 82 Active
Error CS0122 'HashCode.TraitMask' is inaccessible due to its protection level Assembly-CSharp C:\projects\ai_planner_error_sample\Assets\AI.Planner\Generated\DomainDef\DomainDef.cs 87 Active
Error CS0122 'HashCode.Value' is inaccessible due to its protection level Assembly-CSharp C:\projects\ai_planner_error_sample\Assets\AI.Planner\Generated\DomainDef\DomainDef.cs 105 Active
Error CS0122 'HashCode' is inaccessible due to its protection level Assembly-CSharp C:\projects\ai_planner_error_sample\Assets\AI.Planner\Generated\DomainDef\DomainDef.cs 129 Active
Error CS0122 'HashCode.TraitMask' is inaccessible due to its protection level Assembly-CSharp C:\projects\ai_planner_error_sample\Assets\AI.Planner\Generated\DomainDef\DomainDef.cs 186 Active

 

Have been trying to get the basics working for a couple of evenings now. A few things that I got stuck on were:

There's no way to add custom values in the conditions, how were 3 and 7 set in the otto example? The dropdown makes you pick from the pre-defined values. I had to edit the unity YAML file to change it.



For some reason my actions were not running, it turned out after much trial and error that they do not work if inside a namespace:

Code (CSharp):

1. namespace MYNAMESPACE
2. {
3.     public class NavigateAction : IOperationalAction<PlayerAgent>
4.     {
5.         public void BeginExecution(Entity stateEntity, ActionContext action, PlayerAgent agent)
6.         {
7.             // is never called
8.             Debug.Log("testing");
9.         }
10.  
11.         ...
12.     }
13. }

Finally, in a new project I couldn't figure out why the preview "View plan" wasn't showing anything. After reverse engineering the Otto project it turned out to be the Controller<Otto> m_Controller; that needs to be set:

Code (CSharp):

1. if (m_Controller == null) {
2.     m_Controller = m_Otto.Controller;
3.     return;
4. }

It was in the UIManager which is dealing with the game GUI so was pretty confusing. It should just work with the Gameobject that has the BaseAgent attached.

I'm hoping to do some performance tests next, it would be interesting to see if it can scale with thousands of (ECS) agents.

 

I've just noticed this tech now. My main question is, does this integrate well with an ECS focused project, where I have many ECS entities as my agents?

Additionally, what kind of performance impact does this system have over reactive agents like state machines and behaviour trees?

Right now I'm just trying to figure out how to approach AI for an ECS game with thousands of agent entities.

 

Having trouble getting more than one agent to use the same plan and domain definition. A single agent is working fine, as soon as I duplicate the game object the second one doesn't work.

ArgumentException: A component with type:HashCode has not been added to the entity.
ArgumentException: A component with type DomainObjectReference has not been added to the entity.

After disabling the first one the second one begins to work. Not sure how to have multiple agents use the same plan and domain?

 

This looks interesting but I'm getting "403 Forbidden" site-wide on GDC so am unable to access the 2019 talk. Could you kindly upload the video to YT?

Thanks.

 

Same here, I don't know what happened to the GDC Vault website, it always returns a 403 error.