Search Unity

  1. We are migrating the Unity Forums to Unity Discussions by the end of July. Read our announcement for more information and let us know if you have any questions.
    Dismiss Notice
  2. Dismiss Notice

Inference with exported .ONNX model in python

Discussion in 'ML-Agents' started by unity_NFo0Ygq0Gj8sOg, Apr 22, 2020.

  1. unity_NFo0Ygq0Gj8sOg


    Apr 21, 2020
    Hi, I want to run inference from a .onnx model I created in ml-agents in a python script using onnxruntime.

    It is a simple model with discrete action space (branch size 1x3), and runs correctly in Unity. But when I run it in python, it gives an array of three numbers as output, instead of a single integer. Do you know what are those numbers and how can I convert them into the correct vectorAction?

    - If I do inference in Ml-agents, and set the action space to continuous, I also get the same numbers as results. Therefore I think these numbers are what the network should output, and then they are somehow preprocessed before they are passed on to the agent.

    - My Tensorflow version is 1.13.1 and Ml-Agents version is 0.15.1

    Code (Python):
    1. import onnxruntime
    2. import numpy as np
    3. model = "model.onnx"
    4. sess = onnxruntime.InferenceSession(model)
    5. #vector_observation:0
    6. x = np.array([[-4.141203 , -0.8933127 , -3.927535 , -1.150026]])
    7. x = x.astype(np.float32)
    8. #action_masks:0
    9. y = np.array([[-1.031152 , -1.114622 , -1.154025]])
    10. y = y.astype(np.float32)
    11. result =[output_name], {"vector_observation:0": x, "action_masks:0": y})
    [array([[-1.0638015, -1.1055297, -1.1275567]], dtype=float32)]
    the vector action in Unity for the same input was "0"

  2. celion_unity


    Jun 12, 2019
    Those are the log probabilities of each branch being chosen. If you raise 'e' to each one, you'll see that they sum to 1.0:

    Code (Python):
    1. >>> import math
    2. >>> log_probs = [-1.0638015, -1.1055297, -1.1275567]
    3. >>> probs = [math.pow(math.e, lp) for lp in log_probs]
    4. >>> sum(probs)
    5. 1.0000002336510594
    You can see how this is used in C# in this part of the code: - the will contain the log probabilities, then we convert them to normal probability space, and select an index based on the probability weight.
  3. unity_NFo0Ygq0Gj8sOg


    Apr 21, 2020
    Thank you! So I should be able to select the index of the largest number, and that should be the agentAction. But from the numbers I get, the first is always the largest, for example:
    .onnx inference result (Python) correct agent action (from inference in Unity)
    [-1.0044976, -1.1231222, -1.176004] 0
    [-1.0019577, -1.1218884, -1.180334] 0
    [-1.0027038, -1.1222707, -1.179038] 0
    [-1.0023875, -1.1221073, -1.1795887] 1
    [-1.0007201, -1.1212393, -1.1825048] 2

    I have guess, that it has to do with the action masks. I don't use any action masks in my agent and I try to override the default action mask collector:
    Code (CSharp):
    1.     public override void CollectDiscreteActionMasks(DiscreteActionMasker actionMasker)
    2.     {
    3.         List<int> actionIndices = new List<int>() { };
    4.         actionMasker.SetMask(0, actionIndices);
    5.     }
    But the model still expects an array of three for "action_masks:0" to run. For that always gave [1,1,1] as input.