Search Unity

  1. Unity 2019.4 has been released.
    Dismiss Notice
  2. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice
  3. Ever participated in one our Game Jams? Want pointers on your project? Our Evangelists will be available on Friday to give feedback. Come share your games with us!
    Dismiss Notice

Visual observations

Discussion in 'ML-Agents' started by andrzej_, Jun 26, 2020.

  1. andrzej_

    andrzej_

    Joined:
    Dec 2, 2016
    Posts:
    74
    Quite a few people are asking about using visual observations with ML-agents and even how to train the example environments with visual obs (Hallway and Pyramids). I've spend some time expanding on the idea in the Hallway example and even managed to get some proper generalization.


    The agent was trained with a set of 8 different symbols and is then able to recognize new, unseen symbols (30 total used in testing), with varying success rate (but definitely better than random). It even manages to recognize the Unity logo, which the agent haven't seen before.

    As for the hyperparameters and RL model I've used PPO with LSTM memory and the most significant change is using the ResNet backbone for the visual observations.
    I've also used curriculum learning to ramp up the number of simultaneous symbols and some other environments properties (not shown in the video, but some domain randomization).

    I definitely haven't done a full hyperparameter search, so there's definitely room for improvement, but it took on the order of 10-15M steps to get decent results.
     
    celion_unity likes this.
  2. mbaske

    mbaske

    Joined:
    Dec 31, 2017
    Posts:
    229
    That's awesome! Would you be ok with sharing the project files?
     
  3. andrzej_

    andrzej_

    Joined:
    Dec 2, 2016
    Posts:
    74
    Sure, I'd have to do a lot of clean up there, but except the random placement of the symbols where I used poisson disc sampling (which I very lazily implemented - once in a while throws an error when it can't get coordinates matching the requirements) there isn't anything that special in the project. In terms of the ML-agent relevant parts I'd say are very similar to the Hallway example.
    One more new thing that I have there, and didn't use in the end, was experience recorder working with Unity's navigation system. My idea was to randomly spawn obstacles, record playthroughs with an agent avoiding those objects and then I could use that data with GAIL. But in the end I've left it for another project, as matching the possible actions of the agent to the navigation based system might be a bit problematic.
     
  4. ervteng_unity

    ervteng_unity

    Unity Technologies

    Joined:
    Dec 6, 2018
    Posts:
    48
    This is fantastic work - thanks for sharing!
     
    andrzej_ likes this.
unityunity