Search Unity

Question Can I integrate python into unity mlagents?

Discussion in 'ML-Agents' started by Zylkowski_a, Jun 20, 2020.

  1. Zylkowski_a

    Zylkowski_a

    Joined:
    Jul 27, 2019
    Posts:
    157
    I am coming from pytorch background in case of machine learning. I would like to make simple project and create whole environment in Unity. The thing is I would like to utilize python if it's possible. So, is it possible to use python/pytorch with mlagents. If yes, then how can I make it work together?
     
    Zylkowski_a, Jun 20, 2020
    #1

  2. awjuliani

    awjuliani

    Unity Technologies

    Joined:
    Mar 1, 2017
    Posts:
    69
    awjuliani, Jun 20, 2020
    #2

  3. janimator

    janimator

    Joined:
    Apr 5, 2013
    Posts:
    52
    I've been using MLAgents 0.15.1 and have hard a very difficult time getting my Python Pytorch model to work with my project. One of the main issues was getting the model file to re-run back onto Unity. I had to format the model to be a barracuda model and use using barracuda to get anything to load (could not plug my model back into mlagents).
    I'm continuing to deal with many issues.

    Is the latest version of MLAgents any different? Was something changed? Is there an example showing how to train agents with Pytorch then plug the model back into unity?
     
    janimator, Jun 24, 2020
    #3

  4. TreyK-47

    TreyK-47

    Unity Technologies

    Joined:
    Oct 22, 2019
    Posts:
    1,822
    Hey there @janimator - which version of Python are you using?
     
    TreyK-47, Jun 25, 2020
    #4

  5. ervteng_unity

    ervteng_unity

    Unity Technologies

    Joined:
    Dec 6, 2018
    Posts:
    150
    ervteng_unity, Jun 29, 2020
    #5

  6. pedroveloso13

    pedroveloso13

    Joined:
    Mar 10, 2020
    Posts:
    1
    Hi, I am trying something similar. I trained a model in PyTorch and I want to use it as a back-end. I want to be able to run the ML and the environment in Python and communicate back and forth with unity to support user-interaction.

    What is the best way to enable this type of communication? Is mlagents or a custom socket communication between Python and C# more appropriate?
     
    pedroveloso13, Sep 2, 2020
    #6