Question Is ML Agent a good approach for my game?

Hello Everyone,

I'm making a moba game based on many abilities, where players have endless combinations of abilities/spells.
I use DOTS to get acceptable performance for mobile platforms.
So :
1) is ML Agent the right approach for my game?
2) If so, what type of behavior should I train? (imitation...)
3) what is the right approach to get agents to behave decently for any combination of abilities?


Thank you!

 

I tried that albeit years ago on a much older version of ML Agents without any luck for a game I launched a few years ago. What I wanted was to use ML Agents for the tower control AI(s) and also for the enemy units AI(s)
https://store.steampowered.com/app/720270/Defense_Task_Force__Sci_Fi_Tower_Defense/

After about 6 months eventually gave up and went with behavior trees using a combination of custom coded behavior trees and the Node Canvas asset.

If anything can do it I guess it would be ML Agents. However getting reinforcement learning to work for a complex game like what you mention seems to be bit of a black art.

If you can create a data model which can describe each spell using a fixed set of attributes then maybe. However if each spell and what it does is unique to itself and you have lots of spells then that would be a very large extremely sparse one hot input tensor.

 

Yes the abilities can be described with a fixed data model.
But I think my game can be easier with an approach like the behavior tree or ai planner.

 

Behavior Tree based AI:
Pros: straightforward implementation
Cons: Predictable and gets boring after a while. Meaning for any given situation it will always do the same thing so its less of a challenge

ML Agents base AI:
Pros: Generative and not easy to predict. Meaning the AI will eventually learn generative behavior / strategies as well as very deep strategies way beyond what a behavior tree can implement. Especially if you can train it using both curiosity and memory modules enabled.
Cons: Harder to implement.


If the abilities of any given spell can be described as a fixed set of attributes. Then ML Agents is a lot more feasible.

Your Input data for example could contain the attributes of the spell casted by the opponent rather than a 1 hot encoding of the spell itself. Of course along with available elixir and whatever other relevant game play attributes.

Likewise the action output data could contain the desired attributes of the spell to cast and your code could use heuristics to try and find the closest matching spell to the requested attributes. However that attribute to spell matching heuristic will need to be 100% deterministic. Otherwise it is very confusing for the AI cause it would be like. Hey one minute I request this set of attributes and the impact is X, Next time I request the same set of attributes the impact is Y. What's hapenning here. Then the curiosity module goes off on a mission to investigate the discrepancy instead of learning howto play the game.

 

Thank you very much, it's very constructive!

I'm sorry if my question is primitive, because at the moment I haven't gone deep into ML Agent.

Is it possible to give a hard-coded behavior to certain agents like a Familiar that follows a Player (Summoner) or a Monster that Protects a Region of the Map and only attacks if attacked?

Is it possible to get multiple versions of ML agents for different difficulties, like applying some sort of intelligence based on Real Players levels?

 

Re:"Is it possible to give a hard-coded behavior to certain agents..."
Sorry question does not make much sense in the context of ML Agents :

Don't train an AI unless you need one. Simply hard code that entities behavior or use a behavior tree, etc.... Don't hookup ML Agents to it.

For a familiar that follows a player you could simply use AStar Pathfinding Project and set the seeker target of the Familiar to be the players transform. Then sent the end reached distance to a bigger value like 1 or 2 meters etc...
AStar: https://assetstore.unity.com/packages/tools/ai/a-pathfinding-project-pro-87744

Behavior Designer Movement Pack might work for the Familiar game entity:
Behavior Designer: https://assetstore.unity.com/packag...or-designer-behavior-trees-for-everyone-15277
Movement Pack Addon: https://assetstore.unity.com/packages/tools/ai/behavior-designer-movement-pack-16853

Polarith might also do a good job for the Familiar game entity:
Polarith: https://assetstore.unity.com/packages/tools/ai/polarith-ai-pro-movement-with-3d-sensors-71465

And there are a lot more on the asset store under Tools -> AI


Re: "Is it possible to get multiple versions of ML agents for different difficulties.."
Thats a tough one as it all depends on the game itself.

You could abort training early to create an easier AI.

In some games (depending on game mechanics) like FPS you could increase the decision interval. So AI is trained at say 1 second per decision. To make a dumber version of the AI you could run it at say 1.5 seconds per decision.

In some games you could add a noise factor to the input sensors (kind of like a dropout layer) where you randomly zero out some sensor values to make the AI less effective.

All depends on your game mechanics.

 

Thanks !
That’s a lot of informations and good practices
I think the best thing I can do is trying ML Agent kit and see you if it fits my needs, otherwise I’ll go for behavior trees.

 

If you do take the behavior trees path for part or all of your project:
For behavior trees the first three assets stand out on the asset store. Each one takes a different approach:

Node Canvas and Flow Canvas
Node Canvas: https://assetstore.unity.com/packages/tools/visual-scripting/nodecanvas-14914
Flow Canvas: https://assetstore.unity.com/packages/tools/visual-scripting/flowcanvas-33903

Behavior Designer with Movement Pack:
Behavior Designer: https://assetstore.unity.com/packag...or-designer-behavior-trees-for-everyone-15277
Movement Pack Addon: https://assetstore.unity.com/packages/tools/ai/behavior-designer-movement-pack-16853

Decision Flex:
https://assetstore.unity.com/packages/tools/ai/decisionflex-8967

Non Behavior Tree possibilities:

Apex Utility AI:
Take a look at the free demo
https://assetstore.unity.com/publishers/6827

NEAT for Unity:
This is an implementation of the NEAT algorithm which is evolving a neural network using genetic programing. Very few hyper parameters to tune. Pretty much leave it to learn. The negative is that it is all CPU based so training is NOT fast.
https://assetstore.unity.com/packages/tools/ai/ann-tools-138836

AStar PathFinding:
Or maybe code your own logic of where to set the seeker target, waypoint systems etc... on top of A Star Pathfinding Project
https://assetstore.unity.com/packages/tools/ai/a-pathfinding-project-pro-87744

 

I asked for help here https://servreality.com/technologies/unity-3d-development/ Unity 3d development company helps to reduce the development time of a game or application several times, which means you can get a finished project very quickly. Besides, in Servreality takes a very responsible attitude towards meeting the deadlines of the project, and your job will be ready in record time.

 

@AnnyMe
Might as well have said:

Iggg Far Zug Zug Ferr Gumber Groof Fer Vrooom Fra Zug Zug

or maybe
"Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua."

LOL

How come you didn't mention upwork and fiverr as well ?

 

XD

 

Have you considered Utility AI / Infinite Axis Utility System (IAUS)? It's all data driven and I think it fits in perfectly with DOTS - no fancy tree structures or graphs or anything. You won't get any learning like ML, but you can get some interesting behaviors in different scenarios that you might not have anticipated. It's also very fast, especially in DOTS. If you're interested I can send you the three main videos about it (or you can look them up) - a guy named Dave Mark is the primary IAUS guru.

 

Thanks @lclemens !

I've never seen this repository before, but it seems like a good approach for DOTS games.

yes, Thanks!!

 

This one starts at 33:50 and is a good intro to utility AI --> https://www.gdcvault.com/play/1018040/Architecture-Tricks-Managing-Behaviors-in

This one gives a few more details --> https://www.gdcvault.com/play/1012410/Improving-AI-Decision-Modeling-Through

This one dives into the code and has all the implementation details --> https://archive.org/details/GDC2015Mark

I actually just finished my implementation a couple of days ago. It was a lot easier than I thought it would be and so far it I'm quite happy with it. If you decide on utility AI, let me know and I can give you a head start.

 

thanks a lot man !
i am actually working on other tasks:
* Finishing Go Backend,
* Finishing my Custom Hybrid Renderer and Editor Utils,
* Converting Unreal's Gameplay Ability System to Unity Dots and make networking compatible by default using Dots Netcode,
*Add AI to the Game. (For the moment Utility AI seems to be a very good approach)
I will contact you as soon as I am in the AI phase.

Thanks Again @lclemens !