Question Can ML be used to determine if levels are dead ends?

Hi there,

Apologies in advance as this is a very general line of querying, I have done a little ML reading and watched a few tutorials but I am unsure about some things still. I have a game idea in which I'd like to use ML to aid specifically with level design so wanted to get a sense of it on here before I get in too deep.

The main things I was wondering if ML might be good at, are ~

• determine if the level is winnable with the finite resources provided to the player
• if winnable
  • determine the optimum route and therefore least number of resources required to win
  • record the win path i.e. the moves
  • can it do this across the whole game like playing sequentially through all 100 levels? or if it finds an unwinnable level will it get stuck trying to find a win path indefinitely for that level

There will be mechanic changes as the levels proceed so I really wasn't sure if I the ML needs to learn on a single set of conditions i.e. per level, or if it is smart enough to complete a level then move on and learn the next one as well and upgrade the learning as a whole

The dream would be to be able to easily create new levels by passing it specific resources and parameters of difficulty, but I'd be happy for now if it could just tell me I have not created dead end levels

Thanks,
LS

 

I believe that you would be slightly better off allowing some type of dynamic programming algorithm determine the level layout, and whether it can be completed or not. I think you could train an agent which attempts to solve these levels as well, but ultimately if an agent fails does not mean the level is insurmountable, and so, you may spend longer troubleshooting an agent and its rewards than actually using the tool you're describing - even in the simplest instances of agents. That would be my advice.

However, agents can certainly scale their knowledge with Curriculum Learning if your primary concern is that the levels would become too difficult, but it would depend on the types of tasks you'd require it to solve.

This being said, you could use agents such as intrinsically motivated Curiosity Driven agents to search for unintended or unique solutions to your levels.

You might also consider how a demonstration such as this, which was made with ML-Agents would scale to the difficulty of your levels. All food for thought.

 

Thanks for the feedback, I guess I had an interest in trying a little ML to see what it was like so I went ahead and built out a model. I've learnt a lot since, its a very interesting topic, currently I'm at the stage of having a stable basic model on a scaled back simplified level, next step is to scale it out horizontally and see what happens. Due to the turn based nature of the game, and using tweens to move components I am finding that I generally have to use time-scale to reduce the speed it runs at.

I'm learning the value of picking the correct observations and being conservative with rewards, its good though hopefully with horizontal scaling I can get somewhere.

Will keep you posted on it.

 

> determine if the level is winnable with the finite resources provided to the player

ML can, but this typically is not the domain of reinforcement learning, but of supervised learning.

Simply, you'd need at least several thousand pairs of levels and whether winnable or not. Ideally millions of such pairs.

I feel that it's hard to prove that a level is not winnable. What you could do is train a model with reinforcement learning to solve levels, and then feed it a level, and see if it solves it. This won't ever tell you that a level is not winnable, but it will at least be able to give you levels which are in fact winnable.

Similarly:

• determine the optimum route and therefore least number of resources required to win
• record the win path i.e. the moves
• can it do this across the whole game like playing sequentially through all 100 levels? or if it finds an unwinnable level will it get stuck trying to find a win path indefinitely for that level

The RL agent won't necessarily follow the optimimum route. For this you need things like A* search, which can only be used for certain relatively simple environments. The RL agent will give you one winning route, but definiteely not guaranteed to be optimal.