Search Unity

  1. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Question Agent is not learning due to PyTorch error?

Discussion in 'ML-Agents' started by theriser777, Oct 27, 2022.

  1. theriser777

    theriser777

    Joined:
    Feb 2, 2020
    Posts:
    12
    This is my first time working with MLAgents and machine learning in general, and I thought I did everything right but the agent has been going for a while now and I'm not seeing any improvements, I'm also getting a lot warnings that I'm not sure what to make of.
    (venv) D:\Projects\Games\Warsh>mlagents-learn --force

    ┐ ╖
    ╓╖╬│╡ ││╬╖╖
    ╓╖╬│││││┘ ╬│││││╬╖
    ╖╬│││││╬╜ ╙╬│││││╖╖ ╗╗╗
    ╬╬╬╬╖││╦╖ ╖╬││╗╣╣╣╬ ╟╣╣╬ ╟╣╣╣ ╜╜╜ ╟╣╣
    ╬╬╬╬╬╬╬╬╖│╬╖╖╓╬╪│╓╣╣╣╣╣╣╣╬ ╟╣╣╬ ╟╣╣╣ ╒╣╣╖╗╣╣╣╗ ╣╣╣ ╣╣╣╣╣╣ ╟╣╣╖ ╣╣╣
    ╬╬╬╬┐ ╙╬╬╬╬│╓╣╣╣╝╜ ╫╣╣╣╬ ╟╣╣╬ ╟╣╣╣ ╟╣╣╣╙ ╙╣╣╣ ╣╣╣ ╙╟╣╣╜╙ ╫╣╣ ╟╣╣
    ╬╬╬╬┐ ╙╬╬╣╣ ╫╣╣╣╬ ╟╣╣╬ ╟╣╣╣ ╟╣╣╬ ╣╣╣ ╣╣╣ ╟╣╣ ╣╣╣┌╣╣╜
    ╬╬╬╜ ╬╬╣╣ ╙╝╣╣╬ ╙╣╣╣╗╖╓╗╣╣╣╜ ╟╣╣╬ ╣╣╣ ╣╣╣ ╟╣╣╦╓ ╣╣╣╣╣
    ╙ ╓╦╖ ╬╬╣╣ ╓╗╗╖ ╙╝╣╣╣╣╝╜ ╘╝╝╜ ╝╝╝ ╝╝╝ ╙╣╣╣ ╟╣╣╣
    ╩╬╬╬╬╬╬╦╦╬╬╣╣╗╣╣╣╣╣╣╣╝ ╫╣╣╣╣
    ╙╬╬╬╬╬╬╬╣╣╣╣╣╣╝╜
    ╙╬╬╬╣╣╣╜


    Version information:
    ml-agents: 0.28.0,
    ml-agents-envs: 0.28.0,
    Communicator API: 1.5.0,
    PyTorch: 1.11.0+cu115
    [INFO] Listening on port 5004. Start training by pressing the Play button in the Unity Editor.
    [INFO] Connected to Unity environment with package version 2.0.1 and communication version 1.5.0
    [INFO] Connected new brain: WarshAgent?team=0
    [WARNING] Behavior name WarshAgent does not match any behaviors specified in the trainer configuration file. A default configuration will be used.
    [WARNING] Deleting TensorBoard data events.out.tfevents.1666961496.DESKTOP-EL954HL.5368.0 that was left over from a previous run.
    [INFO] Hyperparameters for behavior name WarshAgent:
    trainer_type: ppo
    hyperparameters:
    batch_size: 1024
    buffer_size: 10240
    learning_rate: 0.0003
    beta: 0.005
    epsilon: 0.2
    lambd: 0.95
    num_epoch: 3
    learning_rate_schedule: linear
    beta_schedule: linear
    epsilon_schedule: linear
    network_settings:
    normalize: False
    hidden_units: 128
    num_layers: 2
    vis_encode_type: simple
    memory: None
    goal_conditioning_type: hyper
    deterministic: False
    reward_signals:
    extrinsic:
    gamma: 0.99
    strength: 1.0
    network_settings:
    normalize: False
    hidden_units: 128
    num_layers: 2
    vis_encode_type: simple
    memory: None
    goal_conditioning_type: hyper
    deterministic: False
    init_path: None
    keep_checkpoints: 5
    checkpoint_interval: 500000
    max_steps: 500000
    time_horizon: 64
    summary_freq: 50000
    threaded: False
    self_play: None
    behavioral_cloning: None
    D:\Projects\Games\Warsh\venv\lib\site-packages\mlagents\trainers\torch\networks.py:91: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ..\torch\csrc\utils\tensor_new.cpp:210.)
    enc.update_normalization(torch.as_tensor(vec_input))
    [WARNING] Restarting worker[0] after 'Communicator has exited.'
    [INFO] Listening on port 5004. Start training by pressing the Play button in the Unity Editor.
    [INFO] Exported results\ppo\WarshAgent\WarshAgent-2076.onnx
    [INFO] Copied results\ppo\WarshAgent\WarshAgent-2076.onnx to results\ppo\WarshAgent.onnx.
    Traceback (most recent call last):
    File "C:\Users\Rabiie\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
    File "C:\Users\Rabiie\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
    File "D:\Projects\Games\Warsh\venv\Scripts\mlagents-learn.exe\__main__.py", line 7, in <module>
    File "D:\Projects\Games\Warsh\venv\lib\site-packages\mlagents\trainers\learn.py", line 260, in main
    run_cli(parse_command_line())
    File "D:\Projects\Games\Warsh\venv\lib\site-packages\mlagents\trainers\learn.py", line 256, in run_cli
    run_training(run_seed, options, num_areas)
    File "D:\Projects\Games\Warsh\venv\lib\site-packages\mlagents\trainers\learn.py", line 132, in run_training
    tc.start_learning(env_manager)
    File "D:\Projects\Games\Warsh\venv\lib\site-packages\mlagents_envs\timers.py", line 305, in wrapped
    return func(*args, **kwargs)
    File "D:\Projects\Games\Warsh\venv\lib\site-packages\mlagents\trainers\trainer_controller.py", line 176, in start_learning
    n_steps = self.advance(env_manager)
    File "D:\Projects\Games\Warsh\venv\lib\site-packages\mlagents_envs\timers.py", line 305, in wrapped
    return func(*args, **kwargs)
    File "D:\Projects\Games\Warsh\venv\lib\site-packages\mlagents\trainers\trainer_controller.py", line 234, in advance
    new_step_infos = env_manager.get_steps()
    File "D:\Projects\Games\Warsh\venv\lib\site-packages\mlagents\trainers\env_manager.py", line 124, in get_steps
    new_step_infos = self._step()
    File "D:\Projects\Games\Warsh\venv\lib\site-packages\mlagents\trainers\subprocess_env_manager.py", line 420, in _step
    self._restart_failed_workers(step)
    File "D:\Projects\Games\Warsh\venv\lib\site-packages\mlagents\trainers\subprocess_env_manager.py", line 328, in _restart_failed_workers
    self.reset(self.env_parameters)
    File "D:\Projects\Games\Warsh\venv\lib\site-packages\mlagents\trainers\env_manager.py", line 68, in reset
    self.first_step_infos = self._reset_env(config)
    File "D:\Projects\Games\Warsh\venv\lib\site-packages\mlagents\trainers\subprocess_env_manager.py", line 446, in _reset_env
    ew.previous_step = EnvironmentStep(ew.recv().payload, ew.worker_id, {}, {})
    File "D:\Projects\Games\Warsh\venv\lib\site-packages\mlagents\trainers\subprocess_env_manager.py", line 101, in recv
    raise env_exception
    mlagents_envs.exception.UnityTimeOutException: The Unity environment took too long to respond. Make sure that :
    The environment does not need user interaction to launch
    The Agents' Behavior Parameters > Behavior Type is set to "Default"
    The environment and the Python interface have compatible versions.
    If you're running on a headless server without graphics support, turn off display by either passing --no-graphics option or build your Unity executable as server build.

    (venv) D:\Projects\Games\Warsh>

    What am I doing wrong exactly?
     
    Last edited: Oct 28, 2022