Search Unity

  1. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

How to transfer the ML-Agent model trained in simulation to a real robot?

Discussion in 'ML-Agents' started by OrSimhon, Mar 29, 2022.

  1. OrSimhon

    OrSimhon

    Joined:
    Nov 23, 2020
    Posts:
    16
    Hello,
    I was trained a robot-car in unity using ml-agent with PPO algorithm.
    Now I want to take the final policy and test it on a real robot.
    For that, I trying to load the final network to python somehow and from there I will send commands to my robot online.
    There is some guidance on how to do that? or some related recommendations?
    Until now I tried to load the .pt file with PyTorch, but I think I must have the model definition to create the model instance and then use state_dict().
    please help me with it, thanks
     
  2. x_ddddd

    x_ddddd

    Joined:
    Sep 24, 2021
    Posts:
    3
    Same question.Looking forward to replies.
     
  3. WaxyMcRivers

    WaxyMcRivers

    Joined:
    May 9, 2016
    Posts:
    59
    I haven't done this myself, though, it sounds super fun. If you go to the post in the sticky section "post your ml-agents project" - a user has shared this video about training in unity and deploying inference in a physical robot.

    If I were going into this project, the primary thing I would consider is how to accurately simulate the physicality of each component of the robot regarding control (stepper motor updates for example) and sensor input (shutter speeds of camera/upload latency, etc) within unity.

    Unity has also released a github for robotics and perception which will likely help you in your experimentation.

    Lastly, I frequently notice people asking duplicate questions on this forum - don't hesitate to use the search functionality in Unity Forums and sort for the ml-agents specific posts. There's lots of good information hidden away; and I vaguely recall some others asking this question a while ago.

    Cheers and best of luck! Would love to see more real world deployment of RL!
     
    OrSimhon likes this.
  4. OrSimhon

    OrSimhon

    Joined:
    Nov 23, 2020
    Posts:
    16
    Thanks for your detailed response!
    My clearest question is how to use the policy network generated in unity and load it in python, such that I could send an observation-like and get back the compitable action?
     
    WaxyMcRivers likes this.
  5. WaxyMcRivers

    WaxyMcRivers

    Joined:
    May 9, 2016
    Posts:
    59
  6. x_ddddd

    x_ddddd

    Joined:
    Sep 24, 2021
    Posts:
    3
    I extract the onnx file after ML agents training. Load the ONNX file with Python, the input is the original input in unity,and action_mask is all set to 0, but the output result is very strange. Can anyone answer it? :) Thanks.