Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.

Is it possible to add observation afterwards

Discussion in 'ML-Agents' started by WalkLearner, Apr 25, 2020.

  1. WalkLearner


    Mar 12, 2020
    As the title of the post, is there any possibility to add observation later? because sometimes the model can be trained for a while, retraining a model from scratch due to the new added observations can be very costly. but simply adding observation seems to break the resume of the training. I guess it is because of the input shape is different from the initial.
  2. mbaske


    Dec 31, 2017
    Adding to WalkLearner's question, I'd also like to ask for suggestions on how to design observations to be more flexible in the first place. Sometimes an agent has to keep track of a variable number of items in an environment. Here are the approaches I'm aware of:
    1) First person visual observations with a high enough resolution to reliably detect items at some distance. Requires a ton of compute and training time.
    2) Third person visual observations, like a top-down minimap type of view. Again, this seems to require a high resolution. I've tried this with relatively low resolutions, but in that case, convolution & pooling seem to strip out critical details.
    3) Lots and lots of raycasts, enough to discern objects at some distance.
    4) A detection method like OverlapSphere with an upper limit of detectable objects. Reserve a large enough vector observation space for the maximum number of items. Set neutral values, if less objects are present. This can result in sorting issues though, e.g. if enemies are constantly moving and change their positions and distances relative to the agent.

    None of these solutions seem ideal. Are there any tricks, workarounds or data structures one might utilize here? Thanks.
    Roboserg likes this.
  3. TreyK-47


    Unity Technologies

    Oct 22, 2019
    I'll flag this for the team to have a look! Thanks!
  4. christophergoy


    Unity Technologies

    Sep 16, 2015
    The short answer is no. The observation space is static as it determines the input shape of the network. Symmetrically, the action space is also static as it determines the shape of the outputs of the network.

    There are other things you can do to help with a dynamic number of things you want to detect. If could have a "cognitive load" on your agent that is set in the begging of training so that it can only keep track of the top 100 things of interest. How you track these things is up to you. A relative vector and a normalized distance, ray casts, etc. There is no way to dynamically reshape the network at runtime unfortunately so you'll have to get creative with how you deal with this.
    WalkLearner and mbaske like this.