Search Unity

  1. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Preprocessing visual observations

Discussion in 'ML-Agents' started by ludomitch198, Jul 10, 2020.

  1. ludomitch198

    ludomitch198

    Joined:
    May 23, 2020
    Posts:
    10
    Hello,

    I want to use computer vision techniques to extract bounding boxes from the visual observations and use these as vector observations inputted to my agent rather than the raw pixels.

    Extracting the bounding boxes from the raw pixels already works, my question is where would be the ideal place to plugin this functionality in the python ml-agents code and how should I configure the agents' vector observations in Unity to accommodate this? I can zero pad the bounding box information to ensure consistent input to the agent.

    Thanks!