Search Unity

Data Collection

Discussion in 'ML-Agents' started by Yeisonlop10, Sep 22, 2020.

  1. Yeisonlop10

    Yeisonlop10

    Joined:
    Jan 10, 2018
    Posts:
    19
    Hi guys.

    I'm exploring the code and the documentation of the MLAgents package, and I would like to know if there is any way to display the lists, tables etc that our agents are collecting during runtime. Thank you
     
    Yeisonlop10, Sep 22, 2020
    #1

  2. ruoping_unity

    ruoping_unity

    Unity Technologies

    Joined:
    Jul 10, 2020
    Posts:
    134
    Currently we don't display a list of them, but you can see what observations are being collected by looking at the components in the agent.
     
    ruoping_unity, Sep 23, 2020
    #2

  3. Yeisonlop10

    Yeisonlop10

    Joined:
    Jan 10, 2018
    Posts:
    19
    Hi, thank you for your answer.

    Do you mean in CollectObservations(VectorSensor sensor) from the agent?. Is there any way that we can get access to the data?
     
    Yeisonlop10, Sep 23, 2020
    #3

  4. ruoping_unity

    ruoping_unity

    Unity Technologies

    Joined:
    Jul 10, 2020
    Posts:
    134
    Agents can collect different observations by various types sensors, and VectorSensor is one of them. The agent can have other sensor component (e.g. CameraSensorComponents, RenderTextureComponent) depending on what type of observation you want the agent to collect. These sensor components all present in the agent.

    > Is there any way that we can get access to the data?​
    By saying this, do you mean you want to visualize the observations or are you trying to do something with them?
     
    ruoping_unity, Sep 23, 2020
    #4

  5. Yeisonlop10

    Yeisonlop10

    Joined:
    Jan 10, 2018
    Posts:
    19
    Yes. I would like to visualize them and perhaps exploring my own algorithms and make them interact with the ml-agents system.
     
    Yeisonlop10, Sep 24, 2020
    #5

  6. ruoping_unity

    ruoping_unity

    Unity Technologies

    Joined:
    Jul 10, 2020
    Posts:
    134
    If you just want to know the size/shape or type of observation that the agent is collecting, you can easily find all these information in the agent's sensor components.

    If what you're asking is the real-time sensor output, for sensors that use visual observations you can look at the camera display from the camera of the sensor. For others you might need to customize the visualizations since they are a bunch of numerical values and are not inherently visual data. You can access the observations from the output of each sensors.
     
    ruoping_unity, Sep 24, 2020
    #6

  7. Yeisonlop10

    Yeisonlop10

    Joined:
    Jan 10, 2018
    Posts:
    19
    Thank you. I'm currently reviewing the documentation. It seems that I can visualize the info with TensorBoard correct?. The idea is visualizing the data the agent is collecting for example the distance between the enemy and the player in the form of vector 3, or quaternions in the case of rotations, and evaluate if the agent is learning and making the right decisions based in what is sensing.
     
    Yeisonlop10, Sep 26, 2020
    #7

  8. ruoping_unity

    ruoping_unity

    Unity Technologies

    Joined:
    Jul 10, 2020
    Posts:
    134
    The existing code does not log those customized information, only general training metrics like loss, reward, etc. But yes you can log your custom metrics from C# into tensorboard, see this. It's usually only logging single value though, so it might not work with vectors or tensors.
    The current tensorboard also provide you with various statistical information, which are commonly used metrics to evaluate how your agent is learning
     
    ruoping_unity, Sep 26, 2020
    #8