Search Unity

  1. Unity 6 Preview is now available. To find out what's new, have a look at our Unity 6 Preview blog post.
    Dismiss Notice
  2. Unity is excited to announce that we will be collaborating with TheXPlace for a summer game jam from June 13 - June 19. Learn more.
    Dismiss Notice

Obversation struggle - Observing an 2 dimensional array int's with 5 statuses

Discussion in 'ML-Agents' started by blackfox-, Aug 10, 2021.

  1. blackfox-

    blackfox-

    Joined:
    Oct 31, 2020
    Posts:
    3
    Hey there!

    So I have a question concerning the ML Agent's observation of an 2d array of integers.
    As can be seen in the screenshot I created a set up for training an agent containing some randomly positioned "rooms" and the agents task is supposed to move and order them in a certain way, that they are arranged and neighbouring as asked (seen in visually represented by colors).
    However my question is concerning the Observations, not the Reward. Because I was using ML Agents 8 for a long time without updating up to newer versions I wasn't aware about the introduction of the Grid Observations.
    So I basically came up with my "own" grid I was planning to also manage the observations with.

    My plan was to "simply" feed the int[,] of the "voxles/pixels" as observations in the network. Reading the documentation I stumbled across the AddOneHotObservation() function, which seems fitting for that.


    So my approach was to simply iterate over all the values of the int[,] and add them on after an other:

    Code (CSharp):
    1.     public override void CollectObservations(VectorSensor sensor)
    2.     {
    3.         //observation of rooms "color"-status
    4.  
    5.         for (int i = 0; i < field.VoxelPositionArray.GetLength(0); i++)
    6.         {
    7.             for (int j = 0; j < field.VoxelPositionArray.GetLength(1); j++)
    8.             {
    9.                 sensor.AddOneHotObservation(field.VoxelPositionArray[i, j].Status, statusCount);
    10.             }
    11.         }
    12.  
    13.     }

    But there is still one sentence in the documentation irritating me:

    "Both Agent.CollectObservations() and ObservableAttributes produce vector observations, which are represented at lists of floats. ISensors can produce both vector observations and visual observations, which are multi-dimensional arrays of floats."

    Later in the Documentation for the Grid Sensor the following is stated:

    "During observations, the sensor detects the presence of detectable objects in each cell and encode that into one-hot representation. The collected information from each cell forms a 3D tensor observation and will be fed into the convolutional neural network (CNN) of the agent policy just like visual observations."

    https://github.com/Unity-Technologi...nts.md#isensor-interface-and-sensorcomponents

    So basically the question is, if it does make any difference for the Network, if I iterate over all observations and feed them in in a "List", respectively an 1DArray via the "one hot Observation" function or somehow collect the observations with one of the ISensors functions?
    Or asked diffently: can I feed in an two dimensional array in the Network or did I understand it correctly, that it is anyways transormed into a "list"/1D array, when fed in?

    And there is an other porblem: with my current Setup I would have 8000 observations and when I use them with the "normal" Vector observations they are all directly fed into a fully connected Network instead of a CNN I guess? Does that matter a lot? Is 8000 Inputs for a network without convolutions to many?

    So what would be the best way in ML Agents in Unity to collect Observations in this particular case without fully rewriting the Setup for using the Grid Sensor or without using a Camera Sensor?
    I could of course use a visual Observation with a Camera Sensor, but on one side I would like to extent that "experiment" later on also in 3 spatial dimensions which exludes using a Grid sensor and so I also don't really see a point in rendering an Image of it, although I already have the "raw" data I would need for it.

    If anyone can help me with that, that would be really great!

    Unity_Board_Question.PNG
     
    Last edited: Aug 10, 2021
  2. mahon123

    mahon123

    Unity Technologies

    Joined:
    Feb 9, 2021
    Posts:
    13
    Hi @blackfox-,
    in the grid sensor case you will have a tensor in shape of
    [width, height, #detectable_tags]

    IIUC, your solution with AddOneHotObservation(), is squashing one dimension to
    [width*height, #detectable_tags]

    This way you're losing a key advantage of CNNs, proximity of features. CNNs are good at picking up on adjacent pixels of a eg. 2d image, if you put all pixels in a 1d array the adjacent feature are not beside each other anymore and it becomes much harder to train with a fully connected network.
     
    Last edited: Aug 16, 2021
    tjumma likes this.
  3. tjumma

    tjumma

    Joined:
    Sep 30, 2019
    Posts:
    12
    @mahon123 How can one implement a two-dimensional tensor with .CollectObservations? I want to try to play with a custom grid sensor instead of using a default GridSensor component