Resolved Ml-envs Python API UnityEnvironment Load Fail

Hi. I have a very simple script, one cube agent, one capsule target, on a plane floor with some cube walls it has to navigate around. I've been able to train the environment using the mlagents-learn command on my local device with the built-in PPO option no problem. Everything is fine, training great. I tested inference, that worked too.

Okay now I want to connect to the ml-envs python API so I can start using my own learning algorithm. As per the official docs and Colab tutorial, I used the following code to load the environment from my built binary file:

import mlagents

import mlagents_envs

from mlagents_envs.environment import UnityEnvironment



!chmod -R 755 /content/name_of_my_binary_file.x86_64

env = UnityEnvironment(file_name="name_of_my_binary_file", seed=1, side_channels=[])

I got the following error:

UnityTimeOutException: The Unity environment took too long to respond. Make sure that :

    The environment does not need user interaction to launch

    The Agents' Behavior Parameters > Behavior Type is set to "Default"

    The environment and the Python interface have compatible versions.

- I do not have any code (not even heuristic) that requires user input.

- My agents behavior is set to default and no neural net is loaded into the behavior parameters model.

- On my local, where I built the environment, my specs are:
Ubuntu 18.04.5 LTS os
Python 3.6.9
mlagents==0.20.0

- Where I am trying to connect to the ml-envs API, my specs are:
Google Colaboratory "os"
Python 3.6.9
mlagents==0.20.0

- In Colab, I put the binary x86_64 file in the upload file section.

What is the problem? I also considered building the environment in "headless" mode but I need camera observation, so I don't think that will work. I also made sure when I built my environment, I selected the target platform as Linux, x86_64, and selected my scene.

Totally lost on how to recover, any help is appreciated! I attached a file of my agent's script and important component settings.

 

Update:
So I can reproduce this error in my local python environment on my host machine (which I was previously training the unity using mlagents-learn part of the api). But, the interesting thing is that after some banging my head against my computer I noticed the environment seems to be loaded even though it is throwing the error. So potential bug?

Edit:
I should add more details here. When I call UnityEnvironment(file_name="file", seed=1), it tries to open the build file and start playing the scene but crashes early on. Then if I try to access anything about the env I get the error above or null return.

 

Update 2:
I reproduced the same error using the 3DBall example. So the issue is not how I made my environment. My python API is 1.1.0 and the communicator API is 1.0.0. But this shouldn't be an issue right since the main version is same? I also tried using unity v.2019.4.11f1 and v.2020.1.7f1. I'm using the ml-agents package version 1.5.0 in the environment but also tried switching that with same error persisting.

A concerning note on the docs found at https://github.com/Unity-Technologies/ml-agents/blob/release_7_docs/docs/Learning-Environment-Executable.md. Under how to change the player settings to build the executable file, it says "Display Resolution Dialogue" should be set to disable. However, I cannot find this option anywhere in player settings.

 

Hi! Thank you. Yes, it mlagents-learn works fine. I want to use my own learning algorithm though with the python API. So I don't want to use mlagents-learn right? I need to use mlagents_envs. This is my understanding, is it correct?

Specifically, I am referencing this doc: https://github.com/Unity-Technologies/ml-agents/blob/master/docs/Python-API.md

And I want to use the section 'Loading a Unity Environment' please.

 

Hi again, thank you very much for confirming this. I did run it in python terminal directly and received the same error so it is not a Colab dependent issue. I want to add that when I run the code you provided, after I call UnityEnvirontment it tries to open the game, crashes upon loading, and either produces the error in the terminal or gets stuck on the function itself.

Really appreciate your help.

 

Hi, thanks for getting back to me. It appears after many trials I was able to get it working if I build in debug mode only.

Thanks again!

 

Hi, I was following this thread and I am getting a similar problem. I have built my exe file and I was able to execute using python. But, after running for sometime it says:
INFO:mlagents_envs:Environment timed out shutting down. Killing...
mlagents_envs.exception.UnityTimeOutException: The Unity environment took too long to respond. Make sure that :
The environment does not need user interaction to launch
The Agents are linked to the appropriate Brains
The environment and the Python interface have compatible versions.

Can someone help me out?

 

F:\Works\ml-agents>pip show mlagents
Name: mlagents
Version: 0.27.0
Summary: Unity Machine Learning Agents
Home-page: https://github.com/Unity-Technologies/ml-agents
Author: Unity Technologies
Author-email: ML-Agents@unity3d.com
License: UNKNOWN
Location: f:\works\ml-agents\ml-agents
Requires: grpcio, h5py, mlagents_envs, numpy, Pillow, protobuf, pyyaml, tensorboard, attrs, pypiwin32, cattrs
Required-by:

F:\Works\ml-agents>pip show mlagents-envs
Name: mlagents-envs
Version: 0.27.0
Summary: Unity Machine Learning Agents Interface
Home-page: https://github.com/Unity-Technologies/ml-agents
Author: Unity Technologies
Author-email: ML-Agents@unity3d.com
License: UNKNOWN
Location: f:\works\ml-agents\ml-agents-envs
Requires: cloudpickle, grpcio, numpy, Pillow, protobuf, pyyaml
Required-by: gym-unity, mlagents

unity version : 2021.1.17f

I am also having the same issue. Please help!

 

Is there any solution for this? I also experienced the same problem and I can't figure out a solution.

Here's my [pip show mlagents]

Name: mlagents
Version: 0.29.0
Summary: Unity Machine Learning Agents
Home-page: https://github.com/Unity-Technologies/ml-agents
Author: Unity Technologies
Author-email: ML-Agents@unity3d.com
License: UNKNOWN
Location: d:\user'sdatabase\user project\projects\unity studies\unity machine learning\venv\lib\site-packages
Requires: attrs, cattrs, grpcio, h5py, mlagents-envs, numpy, Pillow, protobuf, pypiwin32, pyyaml, tensorboard
Required-by:

Here's my [pip show mlagents-envs]

Name: mlagents-envs
Version: 0.29.0
Summary: Unity Machine Learning Agents Interface
Home-page: https://github.com/Unity-Technologies/ml-agents
Author: Unity Technologies
Author-email: ML-Agents@unity3d.com
License: UNKNOWN
Location: d:\user's database\user project\projects\unity studies\unity machine learning\venv\lib\site-packages
Requires: cloudpickle, filelock, grpcio, gym, numpy, numpy, pettingzoo, Pillow, protobuf, pyyaml
Required-by: mlagents

I'm using Python Version:
Python 3.9.0 (tags/v3.9.0:9cf6752, Oct 5 2020, 15:34:40) [MSC v.1927 64 bit (AMD64)] on win32

Error Message:
mlagents_envs.exception.UnityTimeOutException: The Unity environment took too long to respond. Make sure that :
The environment does not need user interaction to launch
The Agents' Behavior Parameters > Behavior Type is set to "Default"
The environment and the Python interface have compatible versions.
If you're running on a headless server without graphics support, turn off display by either passing --no-graphics option or build your Unity executable as server build.

I don't seem to find any solution for this PLEASE HELP!

Also, the Machine Learning module runs and the script can work however the module do not safe because of the UnityTimeOutException.

 

Hi, same problem here, am using macos, and tried both our custom agent, as well as 3dball. Both timed out. Same setup as @Proandrewmants

Notebook