Search Unity

  1. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Resolved Switching and Null Observations

Discussion in 'ML-Agents' started by justkittenaround, Oct 5, 2020.

  1. justkittenaround

    justkittenaround

    Joined:
    Sep 28, 2020
    Posts:
    21
    Hi.

    I have a very simple environment with a sphere agent. I want to do several things that I am thinking are not possible. Can someone please confirm or deny this?

    1. I want to switch between RayPerceptionsSensor and Camera observations between different training runs.
    - I am assuming this won't work because Ray Cast and Camera, once placed on the agent, are automatically added to the observations state vector. I suppose I could write a fancy script to swap the two, but may be too complicated for me at the moment.

    2. I want to have an agent sometimes have no observations, but still require decision.
    - I am assuming this is throwing an error because the PPO/SAC algorithms need some vector as input. Perhaps I can feed a zero or random vector, but this doesn't feel as good as a true null observation.
     
    Last edited: Oct 5, 2020
  2. henrypeteet

    henrypeteet

    Unity Technologies

    Joined:
    Aug 19, 2020
    Posts:
    37
    Thanks for the questions, please let me know if I missed your points.
    1. If you want to swap sensors then you are essentially training 2 different behaviors (one with camera, one with ray cast). You will want 2 behaviors and likely 2 agents that you train separately. Please let me know if I misunderstood and you for some reason need one agent to have one sensor intermittently for some reason.
    2. If you want to train a network it needs inputs. If you want your agent to be blind then I suppose you could supply constants or random noise for those inputs but I don't see it being useful for much.
     
  3. justkittenaround

    justkittenaround

    Joined:
    Sep 28, 2020
    Posts:
    21
    Thank you for your timely and thorough input henry!

    1. I did not think of them as separate behaviors, so this is a plausible work around. However, it may be interesting in some cases to try swapping within a single agent. Of course this would only be useful if the computation of swapping is less than always having them on-board.

    2. I know it's a weird request. It doesn't seem very useful at first. Random noise will probably suffice.

    With these questions, I am mostly trying to find the limitations of what you can and can't do so I can better understand and implement my designs. I am also thinking about inference stages in which observational failure occurs (e.g. the camera fails and agent must switch to ray casts or nothing), although these are extreme cases and I probably need to think more about inference decision making processes. Must say though, the ml-agents package is phenomenal!