Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

[help wanted]I want to train agent using dqn in gym-unity wapper

Discussion in 'ML-Agents' started by hestia_p, Jul 12, 2021.

  1. hestia_p

    hestia_p

    Joined:
    Aug 11, 2020
    Posts:
    5
    hello I want to train agent using dqn.
    I already know it is possible using a gym-unity wapper, but I meet a problem.

    when I train the agent in pushblock env, it happened this error.

    Tensor("deepq/observation:0", shape=(?, 210), dtype=float32)

    Traceback (most recent call last):

    File "train_unity.py", line 35, in <module>

    main()

    File "train_unity.py", line 28, in main

    dueling=False

    File "/Users/opt/anaconda3/envs/unity_ML/lib/python3.7/site-packages/baselines/deepq/deepq.py", line 208, in learn

    param_noise=param_noise

    File "/Users/opt/anaconda3/envs/unity_ML/lib/python3.7/site-packages/baselines/deepq/build_graph.py", line 377, in build_train

    act_f = build_act(make_obs_ph, q_func, num_actions, scope=scope, reuse=reuse)

    File "/Users/opt/anaconda3/envs/unity_ML/lib/python3.7/site-packages/baselines/deepq/build_graph.py", line 184, in build_act

    q_values = q_func(observations_ph.get(), num_actions, scope="q_func")

    File "/Users/opt/anaconda3/envs/unity_ML/lib/python3.7/site-packages/baselines/deepq/models.py", line 12, in q_func_builder

    latent = network(input_placeholder)

    File "/Users/opt/anaconda3/envs/unity_ML/lib/python3.7/site-packages/baselines/common/models.py", line 111, in network_fn

    return nature_cnn(X, **conv_kwargs)

    File "/Users/opt/anaconda3/envs/unity_ML/lib/python3.7/site-packages/baselines/common/models.py", line 22, in nature_cnn

    **conv_kwargs))

    File "/Users/opt/anaconda3/envs/unity_ML/lib/python3.7/site-packages/baselines/a2c/utils.py", line 50, in conv

    nin = x.get_shape()[channel_ax].value

    File "/Users/opt/anaconda3/envs/unity_ML/lib/python3.7/site-packages/tensorflow_core/python/framework/tensor_shape.py", line 870, in __getitem__

    return self._dims[key]

    IndexError: list index out of range.

    I think the conv_kwargs param raises an error, so I change cnn input data format NHWC=>NCHW.

    anything not works.
    this problem same in Gridworld env.

    I using masos, unity mlagent code releases 14,TensorFlow 1.15.0.

    what can I do?