Search Unity

Machine Learning (Tensorflow + CUDA toolkit) with Unity?

Discussion in 'Scripting' started by trappist-1, Apr 5, 2017.

  1. trappist-1

    trappist-1

    Joined:
    Mar 26, 2017
    Posts:
    436
    I was wondering if it would be possible to incorporate machine learning into my Unity project? However I am only familiar with Tensorflow. Would it be possible to build some prototype nets and train them using Python/Tensorflow while running the Unity GE? One possible approach I have considered is simple UDP loopback connection however I was hoping for a method that Unity C# and Python could call each other directly.

    If Tensorflow is a bad approach then are there any machine learning libraries for Unity C#?

    Also I suspect that using the CUDA toolkit while the GE is running might cause it to crash or lag (untested). I do have two 1070 GFX cards and can only hope they will share.

    Thanks!
     
    Last edited: Apr 5, 2017
  2. Bantaru

    Bantaru

    Joined:
    May 11, 2016
    Posts:
    59
    There is plugin called UnityNEAT. It is C# library.
     
  3. wbknox

    wbknox

    Joined:
    Aug 1, 2016
    Posts:
    11
    You might consider Accord.NET.

    Also, Arthur Juliani at Unity recently wrote, "In the next few weeks we will release an interface with a set of algorithms and example projects to allow for the training of similar Deep Reinforcement Learning agent in Unity games and simulations."

    Lastly, TensorFlow now runs on a lot of platforms (including iOS and Android). I haven't examined the feasibility of creating a Unity plugin, but I hope someone will have the time and inclination to give it a try.
     
    trappist-1 likes this.