Can I train ML-Agent Brains within Unity? (During gameplay)

I have only recently started using Unity ML-Agents and I'm still learning a lot.
I was wondering, is it was possible to train a Brain from scratch while a game is running? (Without having to build and train externally)

Example:
The goal is to have two agents face off against each other.
Generator Agent - that will generate perfect mazes.
"Player" Agent - that will attempt to solve these mazes.

The Generator will be trained on how to generate a maze of a specific size, depending on the current player score.
The Player agents, some trained at different levels and some not trained at all, need to attempt to solve these mazes.
As the game progresses these Player agents must learn how to solve these mazes during the progress of the game. The idea is, that the better the player agents become at playing the game, the larger and more complex the mazes will become.

Note:
Player agent isn't human controlled.

 

I'd like to know this as well!
Is there a way to train agents with normal gameplay?

 

no python exists outside of Unity, and unity cannot interact with tensorflow directly. You need to train your agents externally, and take the files that the python setup exports into your unity project, and drop them onto your agents/policies in the Inspector.

 

basically, environment design and final execution happens in Unity, but training happens externally

 

This is one of the reasons, why I don't like Unity-python approach. It may be perfect for some, but I used Neural Network approach, which can learn at run time, by any potential player, without needing for Unity.

 

Not 100% sure what you mean by "neural network approach" ... code everything inside Unity in C#, without any external libraries? This will be both veeeery slow and really difficult.
Tensorflow makes the whole thing way easier and that means that you can actually train a decently working agent in reasonable time.

 

Here's the thing: most ML algos are very complicated to implement. On-top of that they're very difficult to make computationally efficient. Libraries like Tensorflow use optimized functions for tensor operations that allow people to do deep learning at a sane speed.

So by using Tensorflow, ml-agents gets access to existing and refined implementations of complicated algos like PPO. That's no small advantage given it would be very difficult to meet the same overall performance in C# (it also has given us access to much more than PPO, so it's not really that bad of a deal (that said, it does have its setbacks)).

But most of our agents aren't exactly "deep". We can in-fact implement our own Ml algos directly in C# (and we can do it with somewhat optimized methods by using unity.mathematics and an external library like numSharp. It's just tedious as F*** to actually do this (although a great learning experience).

 

I've recently been trying to combine Tensorflow and Tensorflow.net into Unity (2018.3), which seems to work fine. I haven't tested the abilities in a (Windows) build yet, but that should also work. Training can be fairly slow though and it's still limited to only some platforms, since tensorflow is a compiled dll for specific platforms.

The advantage of the Tensorflow with Tensorflow.net approach is that you can actually train in the Unity Editor at least.

 

I am also interested in this topic. My goal is to enable adaptive behavior to a player's actions. Imagine playing a game where you face a boss that learns how you like to move and counters it the next time you face it. This would increase replayability by many folds.

Seems like the best way is right now might be to gather and run it externally, then reupload it as a resource that the game would redownload at startup.

 

Hi guys. Well, you can write / use simple home made NN in C#. Like this one:
https://github.com/Blueteak/Unity-Neural-Network
(it also includes nice examples)
However Tensorflow (external python training approach) is by far more advanced and better. But maybe the boss can change between different weapons based on the player behaviour in game or other very simple stuff (DarkGate)