Resolved Best way to train in parallel?

Better to parallelize within the same Unity instance or use multiple Unity instances?

The latter would be easy for me to plug into my existing reinforcement learning code, but Unity yells at me if multiple instances try to open the same project:

Aborting batchmode due to failure:
Fatal Error! It looks like another Unity instance is running with this project open.

Multiple Unity instances cannot open the same project.

Working within the same Unity instance seems like it would require less overhead since there is only one engine instance running but wouldn't this be worse in the following ways?
* not able select the number of training environments at configuration time
* harder to deal with reset logic
* need to maintain separate scenes for training and inference

Does anyone have any thoughts / insights? Seems like ml-agent examples parallelize across the same Unity instance.

 

Thanks for the reply. Posted my command below for context. I'll look for "Force Single Instance".

What I mean by "harder to deal with reset logic" is the following. If each Unity instance runs a single training environment, then things are pretty easy and reset() just resets the initial state of the environment. If multiple training environments are running in the same instance, then you would need a reset_index(i: int) to reset the specific training environment that has reached a terminal state. It's not a huge deal I guess, just a bit more complexity in the code.

EDIT: In regards to differences between training/inference. Let's say I'm training an agent to play tennis. And I have 16 tennis boards in my Unity scene to parallelize training. When I want to play against the trained agent (inference), I need to have a separate Unity scene with just a single tennis board.

In contrast, if each training environment runs in its own Unity instance, then the Unity scene is always just a single tennis board whether training 1, 16, or however many agents.

(shade) joe@xps:~/Documents/ml-agents$ python ml-agents/mlagents/run/learn.py config/rollerball_config.yaml --run-id=RollerBall3 --num-envs=2 --env=$UNITY_PATH --force --no-graphics

Version information:
ml-agents: 0.24.0,
ml-agents-envs: 0.24.0,
Communicator API: 1.4.0,
PyTorch: 1.7.0
2021-03-03 15:56:17 INFO [learn.py:245] run_seed set to 6665
2021-03-03 15:56:17 INFO [torch.py:59] default Torch device: cuda

Aborting batchmode due to failure:
Fatal Error! It looks like another Unity instance is running with this project open.

Multiple Unity instances cannot open the same project.

Project: /home/joe/Documents/RollerBall


2021-03-03 15:56:23 INFO [subprocess_env_manager.py:220] UnityEnvironment worker 1: environment stopping.
^C2021-03-03 15:56:41 INFO [trainer_controller.py:187] Learning was interrupted. Please wait while the graph is generated.
2021-03-03 15:56:41 INFO [subprocess_env_manager.py:220] UnityEnvironment worker 0: environment stopping.
2021-03-03 15:56:41 INFO [trainer_controller.py:81] Saved Model

 

Maybe I'm misusing the command. I'm new to Unity, sounds like I need to build an executable instead of provide the path to the Unity executable?

 

Fixed:

Had to build the scene (File -> Build settings). Then "Build". And provide the path of the generated executable to the training script.

python -m mlagents.trainers.learn config/rollerball_config.yaml --run-id=RollerBall --num-envs=2 --env=<path to executable>