Search Unity

  1. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Resolved No success with tutorial on making new environment - agent doesn't receive signal from python

Discussion in 'ML-Agents' started by mjh-sakh, Oct 14, 2020.

  1. mjh-sakh

    mjh-sakh

    Joined:
    Oct 12, 2020
    Posts:
    10
    Hi community. Have a beginner question on this tutorial (very happy with documentation btw). I set up everything closely to the guide, I can control ball agent using Heuristics, but when I run
    mlagents-learn
    following message is recieved:
    [stats.py:118] RollerBall. Step: 10000. Time Elapsed: 105.998 s. No episode was completed since last summary. Training.

    And yes, ball is standing still. Ball has decision requester script assigned. What else should I check?

    Edit: btw, I successfully run demo scenes/environments from original repository; I use new folder for new environment, though.

    PS
    I'm sorry if it has been discussed already thousands time. I was not able to narrow down search to ML-Agents sub-forum so got lot's of unrelated results and gave up.
     
    Last edited: Oct 15, 2020
  2. vincentpierre

    vincentpierre

    Joined:
    May 5, 2017
    Posts:
    160
    Hi.

    The fact that you see the message
    Code (CSharp):
    1. [stats.py:118] RollerBall. Step: 10000. Time Elapsed: 105.998 s.
    means that the RollerBall is communicating properly with Python. The issue is that No episode was completed since last summary. It is likely that there is nothing in the Game code that terminates the episode of the Agent. This means that the Agent would keep playing the game without ever ending the task (either by dying or by succeeding). I would advise setting up a maximum number of steps for your Agent through the "max step" property of the Agent or by manually ending the Agent's episode (through the "EndEpisode()" method on the Agent). For example, you should end the episode if the Agent falls off the platform or when it successfully picks an item.
     
  3. mjh-sakh

    mjh-sakh

    Joined:
    Oct 12, 2020
    Posts:
    10
    Hi Vincent! Thank you for the reply. I have
    EndEpisode()
    conditions as per the tutorial:
    this.transform.localPosition.y < 0
    and
    distanceToTarget < 0.9f
    . And it works perfectly when I run it in Heuristics. However, I do not see ball moving at all when I start learning, it sits still at same place.
     
    Last edited: Oct 15, 2020
  4. vincentpierre

    vincentpierre

    Joined:
    May 5, 2017
    Posts:
    160
    Have you tried "controlling" the ball during training? I wonder if the Heuristic is somehow overriding the Python actions.
    It could also be due to the action of moving to be very small and un-perceptible, maybe increasing the decision period of the decision requester or increasing the speed change of the actions of the Agent would help diagnose it.
     
  5. mjh-sakh

    mjh-sakh

    Joined:
    Oct 12, 2020
    Posts:
    10
    Tried both. No luck.

     
  6. vincentpierre

    vincentpierre

    Joined:
    May 5, 2017
    Posts:
    160
    Can you send your code so I can try this on my end? I do not know what is wrong and I cannot reproduce this issue without further information.
     
  7. mjh-sakh

    mjh-sakh

    Joined:
    Oct 12, 2020
    Posts:
    10
    Attaching Assets folder (code, materials, scene). Really appreciate your effort, Vincent!
     

    Attached Files:

  8. vincentpierre

    vincentpierre

    Joined:
    May 5, 2017
    Posts:
    160
    Hi,

    I looked at the assets you sent. First of all, the BehaviorType is set to Heuristic only, I think you changed it to Default when training but I just wanted to make sure. The issue is that in your Behavior Parameters, your VectorAction SpaceType is set to discrete. It should be continuous given the implementation you have of OnActionReceived.
    If you have 2 discrete actions with branch size of 1, it means there are 2 actions but only one option for each (option 0) hence the action picked was always the only one available : (0, 0)
     
    mjh-sakh likes this.
  9. mjh-sakh

    mjh-sakh

    Joined:
    Oct 12, 2020
    Posts:
    10
    It worked! You're my savior, Vincent! Thank you a lot!