Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. We are updating our Terms of Service for all Unity subscription plans, effective October 13, 2022, to create a more streamlined, user-friendly set of terms. Please review them here:
    Dismiss Notice
  3. Have a look at our Games Focus blog post series which will show what Unity is doing for all game developers – now, next year, and in the future.
    Dismiss Notice
  4. Join us on Thursday, September 29, for a day with Unity's SRP teams here on the forum or on Reddit, and discuss topics around URP, HDRP, and the Scriptable Render Pipeline in general.
    Dismiss Notice

Can I train ML-Agent Brains within Unity? (During gameplay)

Discussion in 'ML-Agents' started by DarkDungeonStudio, Jun 18, 2019.

  1. DarkDungeonStudio


    Oct 31, 2013
    I have only recently started using Unity ML-Agents and I'm still learning a lot.
    I was wondering, is it was possible to train a Brain from scratch while a game is running? (Without having to build and train externally)

    The goal is to have two agents face off against each other.
    Generator Agent - that will generate perfect mazes.
    "Player" Agent - that will attempt to solve these mazes.

    The Generator will be trained on how to generate a maze of a specific size, depending on the current player score.
    The Player agents, some trained at different levels and some not trained at all, need to attempt to solve these mazes.
    As the game progresses these Player agents must learn how to solve these mazes during the progress of the game. The idea is, that the better the player agents become at playing the game, the larger and more complex the mazes will become.

    Player agent isn't human controlled.
  2. astearon


    May 9, 2016
    I'd like to know this as well!
    Is there a way to train agents with normal gameplay?
  3. gmmachine


    Dec 30, 2019
    no :( python exists outside of Unity, and unity cannot interact with tensorflow directly. You need to train your agents externally, and take the files that the python setup exports into your unity project, and drop them onto your agents/policies in the Inspector.
  4. gmmachine


    Dec 30, 2019
    basically, environment design and final execution happens in Unity, but training happens externally :)
    ailuropoda0 likes this.
  5. Antypodish


    Apr 29, 2014
    This is one of the reasons, why I don't like Unity-python approach. It may be perfect for some, but I used Neural Network approach, which can learn at run time, by any potential player, without needing for Unity.
  6. andrzej_


    Dec 2, 2016
    Not 100% sure what you mean by "neural network approach" ... code everything inside Unity in C#, without any external libraries? This will be both veeeery slow and really difficult.
    Tensorflow makes the whole thing way easier and that means that you can actually train a decently working agent in reasonable time.
  7. ChrissCrass


    Mar 19, 2020
    Here's the thing: most ML algos are very complicated to implement. On-top of that they're very difficult to make computationally efficient. Libraries like Tensorflow use optimized functions for tensor operations that allow people to do deep learning at a sane speed.

    So by using Tensorflow, ml-agents gets access to existing and refined implementations of complicated algos like PPO. That's no small advantage given it would be very difficult to meet the same overall performance in C# (it also has given us access to much more than PPO, so it's not really that bad of a deal (that said, it does have its setbacks)).

    But most of our agents aren't exactly "deep". We can in-fact implement our own Ml algos directly in C# (and we can do it with somewhat optimized methods by using unity.mathematics and an external library like numSharp. It's just tedious as F*** to actually do this (although a great learning experience).
  8. jvo3dc


    Oct 11, 2013
    I've recently been trying to combine Tensorflow and into Unity (2018.3), which seems to work fine. I haven't tested the abilities in a (Windows) build yet, but that should also work. Training can be fairly slow though and it's still limited to only some platforms, since tensorflow is a compiled dll for specific platforms.

    The advantage of the Tensorflow with approach is that you can actually train in the Unity Editor at least.
  9. DarkGate


    Jan 26, 2016
    I am also interested in this topic. My goal is to enable adaptive behavior to a player's actions. Imagine playing a game where you face a boss that learns how you like to move and counters it the next time you face it. This would increase replayability by many folds.

    Seems like the best way is right now might be to gather and run it externally, then reupload it as a resource that the game would redownload at startup.
  10. BlackKvader


    Oct 24, 2017
    Hi guys. Well, you can write / use simple home made NN in C#. Like this one:
    (it also includes nice examples)
    However Tensorflow (external python training approach) is by far more advanced and better. But maybe the boss can change between different weapons based on the player behaviour in game or other very simple stuff ;) (DarkGate)