Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.

Question Agent is not learning due to PyTorch error?

Discussion in 'ML-Agents' started by theriser777, Oct 27, 2022.

  1. theriser777

    theriser777

    Joined:
    Feb 2, 2020
    Posts:
    11
    This is my first time working with MLAgents and machine learning in general, and I thought I did everything right but the agent has been going for a while now and I'm not seeing any improvements, I'm also getting a lot warnings that I'm not sure what to make of.
    (venv) D:\Projects\Games\Warsh>mlagents-learn --force

    ┐ ╖
    ╓╖╬│╡ ││╬╖╖
    ╓╖╬│││││┘ ╬│││││╬╖
    ╖╬│││││╬╜ ╙╬│││││╖╖ ╗╗╗
    ╬╬╬╬╖││╦╖ ╖╬││╗╣╣╣╬ ╟╣╣╬ ╟╣╣╣ ╜╜╜ ╟╣╣
    ╬╬╬╬╬╬╬╬╖│╬╖╖╓╬╪│╓╣╣╣╣╣╣╣╬ ╟╣╣╬ ╟╣╣╣ ╒╣╣╖╗╣╣╣╗ ╣╣╣ ╣╣╣╣╣╣ ╟╣╣╖ ╣╣╣
    ╬╬╬╬┐ ╙╬╬╬╬│╓╣╣╣╝╜ ╫╣╣╣╬ ╟╣╣╬ ╟╣╣╣ ╟╣╣╣╙ ╙╣╣╣ ╣╣╣ ╙╟╣╣╜╙ ╫╣╣ ╟╣╣
    ╬╬╬╬┐ ╙╬╬╣╣ ╫╣╣╣╬ ╟╣╣╬ ╟╣╣╣ ╟╣╣╬ ╣╣╣ ╣╣╣ ╟╣╣ ╣╣╣┌╣╣╜
    ╬╬╬╜ ╬╬╣╣ ╙╝╣╣╬ ╙╣╣╣╗╖╓╗╣╣╣╜ ╟╣╣╬ ╣╣╣ ╣╣╣ ╟╣╣╦╓ ╣╣╣╣╣
    ╙ ╓╦╖ ╬╬╣╣ ╓╗╗╖ ╙╝╣╣╣╣╝╜ ╘╝╝╜ ╝╝╝ ╝╝╝ ╙╣╣╣ ╟╣╣╣
    ╩╬╬╬╬╬╬╦╦╬╬╣╣╗╣╣╣╣╣╣╣╝ ╫╣╣╣╣
    ╙╬╬╬╬╬╬╬╣╣╣╣╣╣╝╜
    ╙╬╬╬╣╣╣╜


    Version information:
    ml-agents: 0.28.0,
    ml-agents-envs: 0.28.0,
    Communicator API: 1.5.0,
    PyTorch: 1.11.0+cu115
    [INFO] Listening on port 5004. Start training by pressing the Play button in the Unity Editor.
    [INFO] Connected to Unity environment with package version 2.0.1 and communication version 1.5.0
    [INFO] Connected new brain: WarshAgent?team=0
    [WARNING] Behavior name WarshAgent does not match any behaviors specified in the trainer configuration file. A default configuration will be used.
    [WARNING] Deleting TensorBoard data events.out.tfevents.1666961496.DESKTOP-EL954HL.5368.0 that was left over from a previous run.
    [INFO] Hyperparameters for behavior name WarshAgent:
    trainer_type: ppo
    hyperparameters:
    batch_size: 1024
    buffer_size: 10240
    learning_rate: 0.0003
    beta: 0.005
    epsilon: 0.2
    lambd: 0.95
    num_epoch: 3
    learning_rate_schedule: linear
    beta_schedule: linear
    epsilon_schedule: linear
    network_settings:
    normalize: False
    hidden_units: 128
    num_layers: 2
    vis_encode_type: simple
    memory: None
    goal_conditioning_type: hyper
    deterministic: False
    reward_signals:
    extrinsic:
    gamma: 0.99
    strength: 1.0
    network_settings:
    normalize: False
    hidden_units: 128
    num_layers: 2
    vis_encode_type: simple
    memory: None
    goal_conditioning_type: hyper
    deterministic: False
    init_path: None
    keep_checkpoints: 5
    checkpoint_interval: 500000
    max_steps: 500000
    time_horizon: 64
    summary_freq: 50000
    threaded: False
    self_play: None
    behavioral_cloning: None
    D:\Projects\Games\Warsh\venv\lib\site-packages\mlagents\trainers\torch\networks.py:91: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ..\torch\csrc\utils\tensor_new.cpp:210.)
    enc.update_normalization(torch.as_tensor(vec_input))
    [WARNING] Restarting worker[0] after 'Communicator has exited.'
    [INFO] Listening on port 5004. Start training by pressing the Play button in the Unity Editor.
    [INFO] Exported results\ppo\WarshAgent\WarshAgent-2076.onnx
    [INFO] Copied results\ppo\WarshAgent\WarshAgent-2076.onnx to results\ppo\WarshAgent.onnx.
    Traceback (most recent call last):
    File "C:\Users\Rabiie\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
    File "C:\Users\Rabiie\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
    File "D:\Projects\Games\Warsh\venv\Scripts\mlagents-learn.exe\__main__.py", line 7, in <module>
    File "D:\Projects\Games\Warsh\venv\lib\site-packages\mlagents\trainers\learn.py", line 260, in main
    run_cli(parse_command_line())
    File "D:\Projects\Games\Warsh\venv\lib\site-packages\mlagents\trainers\learn.py", line 256, in run_cli
    run_training(run_seed, options, num_areas)
    File "D:\Projects\Games\Warsh\venv\lib\site-packages\mlagents\trainers\learn.py", line 132, in run_training
    tc.start_learning(env_manager)
    File "D:\Projects\Games\Warsh\venv\lib\site-packages\mlagents_envs\timers.py", line 305, in wrapped
    return func(*args, **kwargs)
    File "D:\Projects\Games\Warsh\venv\lib\site-packages\mlagents\trainers\trainer_controller.py", line 176, in start_learning
    n_steps = self.advance(env_manager)
    File "D:\Projects\Games\Warsh\venv\lib\site-packages\mlagents_envs\timers.py", line 305, in wrapped
    return func(*args, **kwargs)
    File "D:\Projects\Games\Warsh\venv\lib\site-packages\mlagents\trainers\trainer_controller.py", line 234, in advance
    new_step_infos = env_manager.get_steps()
    File "D:\Projects\Games\Warsh\venv\lib\site-packages\mlagents\trainers\env_manager.py", line 124, in get_steps
    new_step_infos = self._step()
    File "D:\Projects\Games\Warsh\venv\lib\site-packages\mlagents\trainers\subprocess_env_manager.py", line 420, in _step
    self._restart_failed_workers(step)
    File "D:\Projects\Games\Warsh\venv\lib\site-packages\mlagents\trainers\subprocess_env_manager.py", line 328, in _restart_failed_workers
    self.reset(self.env_parameters)
    File "D:\Projects\Games\Warsh\venv\lib\site-packages\mlagents\trainers\env_manager.py", line 68, in reset
    self.first_step_infos = self._reset_env(config)
    File "D:\Projects\Games\Warsh\venv\lib\site-packages\mlagents\trainers\subprocess_env_manager.py", line 446, in _reset_env
    ew.previous_step = EnvironmentStep(ew.recv().payload, ew.worker_id, {}, {})
    File "D:\Projects\Games\Warsh\venv\lib\site-packages\mlagents\trainers\subprocess_env_manager.py", line 101, in recv
    raise env_exception
    mlagents_envs.exception.UnityTimeOutException: The Unity environment took too long to respond. Make sure that :
    The environment does not need user interaction to launch
    The Agents' Behavior Parameters > Behavior Type is set to "Default"
    The environment and the Python interface have compatible versions.
    If you're running on a headless server without graphics support, turn off display by either passing --no-graphics option or build your Unity executable as server build.

    (venv) D:\Projects\Games\Warsh>

    What am I doing wrong exactly?
     
    Last edited: Oct 28, 2022