Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. Join us on Thursday, June 8, for a Q&A with Unity's Content Pipeline group here on the forum, and on the Unity Discord, and discuss topics around Content Build, Import Workflows, Asset Database, and Addressables!
    Dismiss Notice
New Forum User Notice Content Pipeline Dev Blitz Day

Question How do I get visual observations in headless environment?

Discussion in 'ML-Agents' started by lin_cheng_TDSE, Jul 8, 2020.

  1. lin_cheng_TDSE

    lin_cheng_TDSE

    Joined:
    Apr 16, 2020
    Posts:
    7
    I am using azure VM for ml-agents, it works fine when I doing normal observations.
    But when I added visual observations in my agent, I got the Environment Connection Timeout.

    I have found an official document suggested that
    Also, I found this which explained how I should install the xvfb.
    But after following the instruction above, I still got the connection timeout error.

    Am I missing something? Or there are some detail instructions which I can follow?
     
    lin_cheng_TDSE, Jul 8, 2020
    #1

  2. TreyK-47

    TreyK-47

    Unity Technologies

    Joined:
    Oct 22, 2019
    Posts:
    1,769
    Let me flag this for the team to have a look, and pass along any guidance they have.

    Which version of ML Agents, C#, and Python are you using?
     
    TreyK-47, Jul 9, 2020
    #2
    lin_cheng_TDSE likes this.

  3. MrNoodleUnity

    MrNoodleUnity

    Joined:
    May 30, 2020
    Posts:
    1
    I have the same problem (Environment Connection Timeout). I'm trying to use 
    mlagents-learn
    to perform training on a server (i.e. headless) build of the Visual Pyramids task with visual observations. It'd be great to get some guidance on how to set up Xvfb!
     
    MrNoodleUnity, Jul 9, 2020
    #3
    lin_cheng_TDSE likes this.

  4. lin_cheng_TDSE

    lin_cheng_TDSE

    Joined:
    Apr 16, 2020
    Posts:
    7
    mlagents==0.17.0
    mlagents-envs==0.17.0
    python==3.7.7 or 3.6.7     (since that I use azure VM for data science, I can use both by conda environment)

    And I am not quite sure how to check C# version in ubuntu, seems like it is along with the version of .NET.
    Here's my .NET version.

    .NET Core SDK (reflecting any global.json):
    Version: 3.1.202
    Host (useful for support):
    Version: 3.1.4


    Not sure if this is the information you want.
     
    lin_cheng_TDSE, Jul 10, 2020
    #4

  5. vincentpierre

    vincentpierre

    Unity Technologies

    Joined:
    May 5, 2017
    Posts:
    160
    Hi,
    We do not officially support running ml-agents headless on Linux.
    The issue you pointed (https://github.com/Unity-Technologies/ml-agents/issues/1786#issuecomment-551280552) to contains instructions on how to install xvfb on Ubuntu 18.04. First, you need to make sure you have the correct version of Ubuntu as different versions of Ubuntu require different versions of xvfb. Make sure you did not get any error during the installation of xvfb.
    You then need to run mlagents-learn with xvfb. To do so, you need to call "xvfb-run mlagents-learn" to run the mlagents command with xvfb.
     
    vincentpierre, Jul 31, 2020
    #5