Apex Utility AI

Yes that was very inspiring. Thank you to always take time to respond Jakob, it's very appreciated !

I think I have grasped the pattern now. Next I need to earn the know-how by indeed begin testing for real =)

I guess I have one last question : does the Apex editor let you draw a curve scorer, or do you have to mimic this yourself with, let's say, a multiplier ?

 

@MehO

In the Utility AI there are several options:

#1 You can use Unity animation curves to draw the curve with the tools available in Unity's out of the box animation curve class.
#2 You can use any of the curves that ships with the Utility AI's utility scripts, such as exponential, logistics, linear etc.
#3 You can extend with your own curves - you can look in the utility scripts how Apex has done this.

We are planning to create a utility curve editor that looks like the graphs in this thread, so it will be easier to edit curves in parallel and compare them. This will be part of future general UI upgrade.

Best Regards,

Jakob

 

Ok that's perfect !

 

OK I take note. Like you said earlier, each game will have its own implementation based on the design choices made. Even more for me as I go 3D instead of a 2D cell-grid in the RTS demo. Anyway, as writing actions and scorers is pretty quick and straightforward, I guess I can quickly tweak the code to match the demo. It would also help me validate them, ensuring I write standard and easily reusable code.

And everyone could (and should !) share some interesting actions and scorers, if they add value. You could also motivate people so we all build a library, and gather a community at the same time

As a programmer myself, and because I just need to read code to quickly understand, I can do without it. Not a priority for me. But sure, if more people show interest, go ahead =)
I guess I could need such courses for advanced topics (like the ones used in AAA industry you can't learn by yourself), but people may have demand for getting-started topics first.

Have a nice day !

 

Can this handle airplane fighting AI ?
I want airplanes that defend themselves or attack you/chase you, but of course they have to look realistic. I don't see any option of this on 3d or anything similar.

 

I guess it depends on what you want from airplane combat? Is it WWI, WWII, modern day, space / future, steam punk?

We are currently making an AI for 3d combat on another project, which involves several special maneuvers.

There are of course two dimensions to this: the actual movement of the airplanes, where you can use e.g. steering behaviors such as hunt, evade, separation, velocity alignment, formations etc. You can use Apex Steer for this as the platform, but you will need to extend the behaviors to 3d.

The other dimension is decision-making - i.e. what maneuver to follow, the angle of attack, whether to attack or regroup, when to break-off, when to stay with your wing man, defensive maneuvers etc. This is where you would use the Utility AI. Depending on the type of planes and type of combat, you need to plan several things where the Utility AI makes sense such as angle of attack, angle to target (depending on e.g. whether cannons fire forward, backwards, or can e.g. rotate), how to best approach enemies, given that they might try and outsmart you etc.

We found that the Utility AI evaluations are so much easier to work with as the number of scenarios explode. Hence, using scorers and utility curves give pretty good behavior for both hunters and prey.

If you have specific requirements, let us know.

Best Regards,

Jakob

 

im interested in the plugin as im going to work on a "Sims" like AI, is the trial still availlable? does the plugin have been tested on IOS device?

 

Hi Damien,

We are currently working on a demo with a Sims-based AI, so you can look forward to that.

There is a trial available here:

http://apexgametools.com/shop/?v=dd65ef9a5579

I am also writing on a tutorial for desires, which will be implemented in the Sims-based demo.

Hope you will enjoy it!

Best Regards,

Jakob

 

Finally ! PayPal and the Asset Store processed my payment weeks later after my purchase... Anyway, can't wait to get my hands on the Utility AI.

@Jakob Rasmussen, where is the best place to follow the news about Apex Game Tools ? Nothing fresh on your website, facebook page or Youtube channel these times...

Cheers

 

Can you show a quick example of how a nonlinear curve would be used in the workflow of Apex Utility AI?

 

Hi Everyone,

We just returned from a great Unite Europe 2016 in Amsterdam.

It was really fantastic meeting all of you using the Apex products. We realy appreciate all the feedback, use cases etc. It was fantastic to see all the cool games build with Apex AI inside.

There are a couple of tweets here where you can get a sense of the atmospshere:

https://twitter.com/apexgametools

The Apex talk was video recorded, and I will provide a link once Unity releases the video (in a couple of weeks we were told).

Best Regards,

Jakob

 

Fantastic - hope you enjoy it. And let us know if there are specific usecases you need some input on.

The best place for news is currently our Twitter account, but we'll make sure we become better at making updates on progress and news on our main site, facebook etc as well..

Best Regards,

Jakob

 

There are couple of options regarding non-linear curves.

1/

You can overwrite the scoring methods and implement any script for making non-linear curves. Such as:

if (input < 10f)
{
return 0f;
}
else if (input < 20f)
{
return 1f;
}
else
{
return 2f;
}

2/

You add a non-linear classifier, such as a neural network, and execute it from the scoring methods.

3/

You can execute an entire script from the scoring method, that calculates and return a value, such as look up tables, databases, etc.

4/

You can extend the utilities library (or add your own) with non-linear curver functions.

Let me know if this is what you need.

Best Regards

Jakob

 

Thanks for the reply.

I hope you will prioritize the visual curve editor you mentioned in an earlier post. The options you suggest would be challenging to hand over to a designer to tinker with. Even as a programmer who understands the math, there's really no substitute for seeing the curve and tweaking its shape by hand.

 

I'm in the process of building an RPG Adventure game and I currently own Behavior Designer, Playmaker, ICE Creature controller, as well as Apex Path. I have yet to fully commit to any AI solution and have only done sparse testing with some of my owned tools. I had heard about Apex Utility AI when it first came out, but was very much against the subscription licensing model at that time. I see now that there is a 1 month free trial as well as a personal edition. Both are positive steps and I am investigating the product further. However, before even investing any time in a free trial I want to have a better understanding of just what this product does and what it can do for me.

I have to admit that I don't have a grasp on what this product gives me. And I don't have a clue how it would integrate with 3D characters and the environment. I see that there are various demos, some that have already been released, but this gives me the impression that Apex Utility AI is only a framework out-of-the-box and isn't going to get me from point A to point B without a lot of coding on my part. What I'm trying to understand is just what Apex Utility AI gives me out-of-the-box?

For Example:

1. Can I use it to get characters wandering around, patrolling, attacking, etc with built-in actions? Or does the asset simply allow me to do a high-level design of behavior decisions and I have to implement all the guts of carrying out those decisions? If so, what is involved in doing that? For example, are there built-in tasks within Utility AI that can drive character animation controllers? Or are we required to write custom code to bridge the gap?

2. I see that this integrates with other Apex products, but how tight is that integration? For instance, I have Apex Path but I don't have Apex Steer. Would Utility AI work with just Apex Path or does it also require Apex Steer?

3. How does Utility AI handle actually controlling the characters? For instance, if my AI logic tells the character it needs to reload, how does one go about interfacing to the character to have it play a reload animation, update its inventory, etc.? I would hope for a very clean interface to tie these things together.

Thanks for any information you can provide.

 

Has anyone visiting this thread used Utility AI yet? If so, can you share your experiences? Especially if you can contrast your experiences with using other AI techniques such as behavior trees or FSMs. Thanks.

 

I have it but did not have too much time with it.
Hope I can really get to grips with it this weekend.

in the meantime you could look at the surival shooter tutorial video they have on youtube

I have behavior designer too as well as opsives TPC so I will use the code of the specific actions (like reload etc from opsives AI integration (BD+TPC)) in the actions of apex ai. I currently use a* pathfinding project pro so i will see how easy it is to integrate.
(but using the tutorial video mentioned above it should not be too hard)

 

Hi Magique,

The Apex Utility AI is a very powerful AI tool that let's you do everything Finite State Machine and Behavior Trees do, and much, much more.

I describe why the Utility AI is so powerful and easy to use in this article on Gamasutra:

http://www.gamasutra.com/blogs/Jako...88/Are_Behavior_Trees_a_Thing_of_the_Past.php

I also did a recent talk about AI on Unite 2016 in Amsterdam - it will be out soon - where I describe a number of things you can do with Utility AI, such as dynamic dialogues and dynamic missions generation for RPGs.

1/

The key problem of making good AI in games is to be able to easily design an AI, both when you start developing your game and have few use cases, and also later, when your game becomes more complex, and you need to support more advanced use cases. You don't want your AI to break later in development, or using an undue amount of time debugging it.

Finite State Machines are rarely a good idea, as they don't scale very well. I.e. it's easy to make an AI with a few states such as fire, load, patrol. But once you progress to needed tens or hundreds of states (which I guess you would need for an RPG), Finite State Machines can become very painful to debug.

The Utility AI a more flexible decision-making system, as all decisions are based on the same principle of utility, and you rarely have to redesign existing behaviors when adding new behaviors, as you often have to do with Behavior Trees.

2/

When you design AI in Unity, you often want to break them down into different layers. One architecture is the following:

a/ High-level decision making (Utility AI)
b/ Actions (Game code)
c/ Animation API (e.g. a class that collects all calls to the Animation system e.g. Mecanim)
d/ Animation System (Mecanim)

Ad a/

The Utility AI decides what the AI should do, based on the context (situation, memory, personality, etc). Options could be e.g. patrol, reload, fire, run for cover, move to tactical position, throw grenade, change weapon, take aim, kick in door, use radio, sound alarm etc.

Selecting actions are often not binary, which is why Behavior Trees don't really work very well. Instead, the Utility AI will score each action available, and select the highest action. Such as if there a multiple enemies coming towards the AI, there are few bullets left in the magazine, and nearby covers are not very good, the decision to fire, reload or run for cover requires better AI. By scoring each action based on e.g. distance to enemies, number of enemies, the type of weapons the enemies are carrying, the quality and distance to cover, the chance of hitting the enemies etc the AI can select the best action dynamically. This is Utility AI

The AI can be separated into multiple parallel AIs, such as one for targeting, one for firing, and once for moving. You can see how this is done in the Survival Shooter demo.

Ad b/

Actions are game code, e.g. classes or methods that can be called why the AI.

transform.GetUnitFacade.MoveTo(Vector3 destination) can trigger Apex Path for example to move the AI to the desired position. You will use this command for any action that involves movement, such as patrol, flee, advance, move to cover etc. So often you end up with few actual actions as the AI reuses many of them.

The Utility AI comes with a number of classes for this, and the Survival Shooter adds additional classes for e.g. combat, such as reloading, throwing grenades etc.

There are many ways you can implement Action, and the Survival Shooter demo shows this in Apex quality code, and you cut & paste this into your project. You can also see how we handle reloading and updating inventory for the AI.

There is actually so many cool classes in that project, such as game pools etc that you can use.

Ad c/

The Animation API is the link between the Utility AI and e.g. Mecanim. Is basically controls the parameters that Mecanim requires to change animation state. The animation are different from game to game, you I guess you have already set this up, or are in the process of doing it. You can download the Apex Mecanim Tutorial project to see how this is done. It also has a nice Animation API class you can use. You can also look at the Survival Shooter project where we do the same, but with fewer animations.

You would have to adapt the Animation API class to your Mecanim setup or your custom animation system, as you might use different animations for your project.

Ad d/

This is setting up Mecanim or using your custom animation system to play your animations.

3/

You don't need Apex Steer, unless you want to have local avoidance as well - i.e. characters not bumping into each other.

Also, if you have specific use cases, let me know and I will help you solve them. We have used the Utility AI on many projects now, and the easy of use and powerful AI we get out it continues to amaze us. That was also the feedback we got at Unite.

Also, be sure to check out the tutorials at apexgametools.com, we have loads of examples for specific use cases there.

Best Regards,

Jakob

 

Yes - we have that on the roadmap. It will be really nice to just draw the AI curves next to each other.

Best Regards,

Jakob

 

I am also really interested on feedback from real use. We love it - but we have also developed it

Best Regards,

Jakob

 

@Jakob Rasmussen Thanks for all the detailed information. I was just going to stick with my current tools for my first mini-game project, but now I'm considering getting the free trial and giving it a go. The reasons you give for using Utility AI over behavior trees and FSMs is very compelling.

 

Yeah, that's kind of the problem I think for a lot of people. Initially, your biggest issue was licensing so it must have really stunted sales. Now that licensing models are reasonable, you still don't have any reviews or even enough ratings to give prospective buyers any idea of how good the product is. So, aside from user testimonies, which are non-existent right now, has Apex Game Tools itself developed anything beyond demo scenes for this? Having at least one real-world game testimony would really benefit Apex Game Tools.

 

Hi Magique,

The Mandate http://www.mandategame.com/ is developed with the Apex Utility AI - as well as other Apex tools.

There is a pre-alpha video of tactical combat from the game here:

You can see use cases such as intelligent formations, tactical movement, planning, and multiple teams collaborating in this video. Needless to say, many of these aspects are hard to pull off well in games, but we only used around 20 days to develop the complete AI.

We don't get information from Unity on the games that are using our technologies, but at Unite approximately half the game development companies we spoke with used Apex products.

We are of course looking forward to getting the first reviews on the Utility AI.

The challenge is that Utility AI is rather unknown to game developers. At our talk at Unite, 70% of the 300 attendants knew what a Behavior Tree is, only 4-ish knew what a Utility AI is. However, after the talk where I explaining the advantages of the Apex Utility AI, our booth was visited non-stop. We lost our voices and we ran out of business cards. So hopefully we will see more games soon with a completely new level of AI based on Apex Utility AI.

The early adopters of this technique are the most likely to use this gain a competitive advantage in the depth and richness of their AIs relative to other games in development, as well as the efficiency gains from less debugging and faster workflow.

Best Regards,

Jakob

 

looks very nice

as I watched the presentations/papers from Kevin Dill
http://www.ccs.neu.edu/course/cs5150f13/readings/dill_designpatterns.pdf
I saw that there some additional "actions"/considerations for the AI:

1. implementing a timer/counter so that certain decisions cannot be taken several times a row
2. a kind of lock-in of decisions so that wild decision switching between options of near same importance does not happen
3. decision inertia
4. repeat penalty
5. one-time bonus

how would I use/implement these in apex ai?

 

one thing i notice was some time gun man in the game floating around instead of walking around that was happening with the unity navmesh you have address that issue

 

Hi Arganth,

Thanks for the link to the article. Below are recommendations based on our experience on how to solve the above points.

1/

You would store the AI's memory about recent actions in the Context. You can then implement a Scorer that scores actions that haven't been executed. You can either give all actions that haven't been executed as the last action a positive score, or you can use several Scorers that also take into account things like when the last action was performed, so that if enough time has elapsed, the AI can repeat the action.

You can also make a separate AI that cleans up the memory of actions when they are e.g. more than 5 seconds old.

The ideas in the article with e.g counters are rather primitive. In practice, you could use AI evaluations to take into account multiple criterion such as time elapsed, the seriousness of the situation, what type of action etc. to evaluate whether or actions should be executed.

In general, you want to avoid mathematical algorithms to control AI as they are opaque to the developer. Instead, you should implement AI explicitly in a human-readable way to improve debugging. This is also easier when you have teams, as your colleagues don't have to try and understand your algorithms, but can instead read the AI literally such as "Add points to action if it is not the last action executed". This is one of the true powers of the Apex Utility AI.

2/ + 3/

We often see this happen in use cases such as e.g. evaluating tactical positions. We often use Scorers such as scoring the current option with e.g. 5 points, to ensure that only tactical position with significantly better scores make the AI change position. You can do something similar to ensure that e.g. tactical plans don't change to often. This is the same as creating decision inertia (point 3/) i. e. you want to change plan if the situation requires it, but only if the new plan is significantly better.

4/

A repeat penalty is again a very primitive way of avoiding repetition. For example, you might want to have the AI keep firing multiple times in a row, whereas moving in and out of the same cover should not be repeated multiple times in a row. However, often you resolve this by having higher-order AI, such as tactical plans, that resolves this. You can also often use biases, such as bias the AI toward e.g. the enemy, to ensure that when it returns to cover, it tries to select a cover closer to the enemy, thereby not oscillating in and out of the same cover. In practice this works very well. We use this in e.g. the Survival Shooter demo to bias the AI towards power-ups. It will thus try and move in the direction of the nearest power-up to ensure it can use any opportunity where there are no zombie teddies to pick up the power-up.

5/

You can store the action in a list in the Context. If the list is empty or if the action is not in the list, make a Scorer that assigns this action e.g. an additional 100 points.

Best Regards,

Jakob

 

Hi Unitywlp,

Where do you notice this?

Best Regards,

Jakob

 

2 observations about the video demo. First, often times it looks like the squad members are attached to each other. If the leader moves, all the others suddenly almost seemed to be sucked in that direction unnaturally. Second, the squad members are often running into each other, which looks bad. Other than that, it's pretty cool.

 

great ideas

 

I read this paper on Utility AI too, a few weeks ago. Very interesting. I guess it's one of the first results you get on the topic when doing a Google search =)

I'm gonna try to give you a pre-answer before Jakob comes back. All the 5 bullet points you describe can be achieved with custom scorers

1. CountdownScorer.
Takes 2 parameters in input : a score (like always) and a number of seconds for the timer/cooldown.
Add a property like a lastExecutionTime to the context of the AI, in which you store the value of Time.time (time in seconds since the start of the game) each time the action is executed. Then as soon as Time.time of the current frame is greater than lastExecutionTime + number of seconds in input, you've consumed your cooldown, so you can return the score. Otherwise you need to wait more, so return 0 as score. This is enough for a lonely timer, if you need more, you would want to work with a 2 dimensional array (or List in C#)
I don't explain very well, I guess I could share the exact code if this explanation doesn't help. But that's really 4 or 5 lines of code for the scorer, and 1 more property for the context.

2. I guess you mean smthg like Commitment to the current action ?
They do something similar in the Shooter tutorial demo, where they add a little bit of score to the current target when 2 enemies at the same distance score the same priority, to avoid switching back and forth randomly. Again, it's just about a few lines of code, you really do what you want with your scorers. So you can do just a quick check if enemy == currentTarget, return the value of score, otherwise score 0.
Same with their move to the best position scorer if I remember well : each frame there are multiple safe position, but if they have the same score as the current one, its useless to move, so add some points to the current position to stay in place.

3. Not sure I understand this one. Is it the oppsite of the previous one ?

4. Very similar to the countdownn. You store a list of already executed actions in the context, if it's called again, apply a malus to the returned score. At some point, maybe after many iterations, the addition of malus will push this action behind another one in the list of priority. There you go, repeat penalty in less than 10 lines of code.

5. I don't remember what a one-time bonus involve, but again, you could go with a stronger repeat penalty. First time you return the full score, then you always return 0 as a score... You might eventually be able to destroy the scorer during runtime maybe (don't know about that).

It's very raw but I hope it helps you understand the power of the UTility AI.

 

Oh sorry Jakob answered while I was writing. I'm gonna read his post, should be better =)

 

Hi Magique,

If you are interested, you could give me time-stamps on the bumping etc you are seeing from the video - then I can explain exactly what's going on

Best Regards,

Jakob

 

Hi MehO,

Thanks - your answers are actually pretty similar to mine

Best Regards,

Jakob

 

Hi Everyone,

Just wanted to cross post that we are launching our ducation & Research program beta to support students and universities to make fantastic games with great AI:

http://forum.unity3d.com/threads/apex-education-research-program-beta.410248/

Hence, we are giving our assets free to education and research purposes to advance great AI. So if you know students or universities that should benefit from our assets let us know or feel free to contact them.

Sign up link here:

http://forum.unity3d.com/threads/apex-education-research-program-beta.410248/

Best Regards,

Jakob

 

Hi! I have been instrumental in pushing the Rainworld developer into Utility (and occupancy map) I'm familiar to the technique. So I will raise the stake a bit higher Can you implement an AI like the one in VERSU with your solution

Now that would be state of the art human like interaction for game. And it will significantly differentiate yourself from the competitor.

 

That sounds like a cool challenge. I'll study how it's done, and then we talk about how this can be solved. The dynamic dialogue part of it is not a problem, and the personality part is also covered. However, there might be nuances to the approach - so I will dig into the research published around this.

Ideally, the challenge should not be to replicate, but to produce something that is better - ideally way better.

So, I would appreciate if you could outline the main shortcomings you see in the current approach. Then we can overcome these as well.

Best Regards,

Jakob

 

Here are some:

:36-:40

 

Hi Magique,

Yes - so this is actually standard behavior in Apex Steer. The units will push other units if needed to get by - similar to how Zerglings flood fill in Starcraft; moving units push units that are standing still. However, it was also one of the things that we wanted to change after the GamesCom demo. I.e. marines that pass by other marines, should not push them if there is amble space to pass by.

So, nice spotted.

Was there also another thing you noticed?

Best Regards,

Jakob

 

Yes, I noticed that units following a leader always seemed to follow unnaturally. When a leader moves, it's almost like all the other units are attached to each other on a line and get pulled in the new direction. It just looks very obvious that they are all being controlled by an external force. The best way to describe it is as if you attached the objects to mouse movement and as the mouse moves they units move precisely when the mouse moves and in the mouse direction. They don't appear to have independent thought or reactions.

 

Hi Magique,

This is actually interesting. Each unit have they own AI that updates their position in the formation. They update this every second, and use prediction relative to the leader to stay in formation. The use case in this example was to have very tight formations, for a veteran group of soldiers. I agree that it is a question about the aesthetics you want from the formation.

If you want more natural behavior, you might actually want to scale down on the precision of the position of each marine in the formation. This is something you can build into the AI as well.

Best Regards,

Jakob

 

Yes, scale down precision and/or add randomness =)

 

I think all the paper about versu are on this page
https://versu.com/about/how-versu-works/

I must note it suffer to the game AI paradox, when it work perfectly it don't looks like AI it looks like authored content. The creator is both thrill and sad about that paradox, her next attempt will include "tell" that shows it's AI and that there is infinite choices, to make it more blatant it's not authored content (as in linear set of script with poor branching factor).

There is also Prom week as state of art social game ai.

 

More state of art
http://www.gdcvault.com/play/1015613/Beyond-Eliza-Constructing-Socially-Engaging

http://www.gdcvault.com/play/1017818/Turning-Comedy-of-Manners-into

 

Very interesting. I am going through it, and will check out how this works in game. We have done something similar for AIs in dynamic worlds, but here we added a strong element of personality. The idea is that to have real intelligence you need intent. I.e. the AI should have desires, goals, and reasons to act in certain ways to reach these goals, even though it defies the immediate logic as interpreted by the player. E.g. just like a human would have.

Best Regards,

Jakob

 

Here are some examples on different fidelity in the forming of formations:

Supreme Commander flowfields (0:40+):

The Supreme Commander flow fields are to some almost too perfect, and give the idea of the units almost floating on a plate, instead of being single units.

A little randomness in the formation see e.g. 7:56+ (you can also check out other Warhammer videos)

I don't know whether this is by design, but it looks better than perfect formations. By the way, look at the yellow arrow markers that show how the units will arrive when setting formation destinations.

You can tweak formations both ways in e.g. flow field formations Apex Steer. The reason you sometimes want to use Utility AI based formations instead is that you might want more AI on top of the formations, so that each unit can for example break formation, change position, take actions such as suppressing side-corridors etc. and this is easier to control via AI than via steering algorithms, although it can be done both ways. For a squad, each squad member might also have different roles such as guarding the rear, guarding sides etc.

Best Regards,

Jakob

 

Hi Everyone,

A short update on our work.

We are currently working on a couple of full demos for the Utility AI; an RTS demo and a Sims-like demo.

I just wanted to share a little mid-time research.

The following images are the same AI implemented using a Finite-State Machine / Behavior Tree methodology and the Utility AI methodology. Basically, the AI is a perpetual AI for people oscillating between sleeping and working ... almost like life

The difference in complexity for solving the same tasks is quite significant. We actually implemented the FSM first. However when we were about to extend the number of desires from two (2) to four (4) the pain of updating conditions for the state machine / tree was simply unbearable. So the full features AI will be Utility AI only .

During the development we also experienced the pain of debugging an FSM. We had forgot to set the score on a Qualifier. Consequently, the default action kept getting triggered, leading to a failure of the people picking up work tasks as they were supposed to do. We thought we had checked all scores on all Qualifiers, but due to the size of the tree we had missed one. This would not have happened in a Utility AI.

Finite-State Machine / Behavior Tree :




Utility AI:



Enjoy.

The demos are available soon.

Best Regards,

Jakob

 

I'll wait till you have versu like demo

 

Looks amazing. Will you be releasing a video to accompany the demo?

 

What platforms does Utility AI support? Can it be deployed to Universal Windows Platform? Android or iOS? Consoles?

 

nice to see rts demo will apex utility come with built in support for formations or we have to write our own ?