Search Unity

  1. Unity 2019.4 has been released.
    Dismiss Notice
  2. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice
  3. Ever participated in one our Game Jams? Want pointers on your project? Our Evangelists will be available on Friday to give feedback. Come share your games with us!
    Dismiss Notice

Can I change the number of channels for a visual observation?

Discussion in 'ML-Agents' started by MarkTension, Feb 21, 2020.

  1. MarkTension

    MarkTension

    Joined:
    Aug 17, 2019
    Posts:
    30
    Hi there!

    for my project I'd like to change the number of channels to 5 for a visual observation.
    Is this in any way supported?
    I thought the float visual observations, introduced in version 14, could maybe be of use if I can pass a multi array instead of a rendertexture?

    Any ideas?
     
  2. celion_unity

    celion_unity

    Unity Technologies

    Joined:
    Jun 12, 2019
    Posts:
    147
    Hi @MarkTension - I think that the float visual observations will work, although I'll admit that I haven't tried it and there might be a limitation on the python side.

    There's some example code in https://github.com/Unity-Technologi...ts/Editor/Sensor/FloatVisualSensorTests.cs#L6 that shows how you could set this up. It uses a 2-D float array, but you can use whatever storage makes most sense to you; the main thing you'd need to modify is in the Write() method; you'd need an additional loop over the channels, like this:

    Code (CSharp):
    1.  
    2. for (var h = 0; h < Height; h++)
    3. {
    4.   for (var w = 0; w < Width; w++)
    5.   {
    6.     for (var c=0; c< numChannels; c++)
    7.     {
    8.       adapter[h, w, c] = floatData[h, w, c];
    9.     }
    10.   }
    11. }
    12.  
    Let me know if you hit any problems, or need more details on how to implement it.
     
  3. MarkTension

    MarkTension

    Joined:
    Aug 17, 2019
    Posts:
    30
    Thank you, that sounds doable! Will give it a try and will reply if it works. Cheers
     
    celion_unity likes this.
  4. MarkTension

    MarkTension

    Joined:
    Aug 17, 2019
    Posts:
    30
    Heya!

    In what context should this script be used?
    Should it be attached to the agent gameobject just like the other sensors?
    Or should it be called from the agent's CollectObservations() method?

    Also, when making a new c# script, and copying floatVisualSensorTest code to it, I get an error with
    using MLAgents.Sensors;
    Doesn't find the Sensors subclass.

    (I'm working in ml-agents14)
     
  5. celion_unity

    celion_unity

    Unity Technologies

    Joined:
    Jun 12, 2019
    Posts:
    147
    Re: MLAgents.Sensors - that should be in the 0.14.1 release (https://github.com/Unity-Technologi...Tests/Editor/Sensor/FloatVisualSensorTests.cs). We briefly removed it on master but brought it back today.

    We don't have a great example of adding a custom Sensor at the moment, but I'm working on converting one of our examples to do that. The basics are that you'll want to:
    * Create a subclass of ISensor and override the Write() method (sounds like you have that part already).
    * Create a subclass of SensorComponent, and return an instance of your ISensor implementation in the CreateSensor method.

    You can see the work-in-progress here: https://github.com/Unity-Technologi...8/files#diff-0fdc9d63b7f92730e2e53734846ee3dd although it's using SensorBase instead of ISensor; hopefully that's enough to get you going.
     
    MarkTension likes this.
  6. wwaero

    wwaero

    Joined:
    Feb 18, 2020
    Posts:
    28
    @MarkTension Were you able to get this going, thinking about giving it a try but I don't want to move my project to 14.1 unless it does work. Maybe you can share how you did it?
     
  7. MarkTension

    MarkTension

    Joined:
    Aug 17, 2019
    Posts:
    30
    Didn't try yet, but still interested to see how it works. Will try it this week and I'll post here if successful.
     
  8. wwaero

    wwaero

    Joined:
    Feb 18, 2020
    Posts:
    28
    @celion_unity
    I'm a little confused on the following part of the code in the visualsensortests code. I don't quite understand where the actual observation is written. I've duplicated the code in that file and I'm calling the code below after writing to the sonsor.floatData array. However when running the ml agents it's telling me that fewer observations are being made than expected. i.e the size of the sensor array. I usually add observations by AddVectorObs is this a step I'm missing? I'm confused on how to make this work. Do i need to add this floatvisualsensortests.cs to a gameobject instead of instantiating the class?

    Code (csharp):
    1.  
    2. var output = new float[12];
    3. var writer = new WriteAdapter();
    4. writer.SetTarget(output, sensor.GetObservationShape(), 0);
    5. sensor.Write(writer);
    6.  
    This appears to write to the adapter
     
  9. celion_unity

    celion_unity

    Unity Technologies

    Joined:
    Jun 12, 2019
    Posts:
    147
    Sorry, you're right, the VisualSensorTests is probably missing a few pieces. What you'll want to do to add this to an Agent is
    1) Create a new SensorComponent implementation that returns your ISensor implementation
    2) Attach this SensorComponent to your Agent.

    The newest version of the Basic scene has an example of this: https://github.com/Unity-Technologi...xamples/Basic/Scripts/BasicSensorComponent.cs

    You shouldn't need to call SetTarget or Write at all, and this doesn't contribute to the number of vector observations.
     
  10. MarkTension

    MarkTension

    Joined:
    Aug 17, 2019
    Posts:
    30
    Hi @celion_unity thanks so far for the help. I didn't get to it last months, but still very interested to try. Finding it difficult to get a grasp of what to do exactly though.

    As I understand so far:
    I should forget about the visualsensortest scrip, because now I can make a subclass of the Sensorcomponent class, which is done as well by The Basic Example.

    When copying the BasicSensorComponent c# into my project and changing the code to suit my environment I run into a problem.

    - cannot resolve symbol SensorBase. For a while I didn't get an error by having the line 'using MLAgentsExamples;' , but for some reason this only worked one time.
    - will this method automatically use the visual observation encoding (convolutions instead of one dense encoding)?
    - do i need to do more than just adapting the basisSensor script to my environment, or am I still missing something
     
  11. celion_unity

    celion_unity

    Unity Technologies

    Joined:
    Jun 12, 2019
    Posts:
    147
    SensorBase is only in the example code. It's not necessary for writing sensors, but it might make some things simpler.

    Yes, "visual" observations (we need a better name for non-rendered multi-dimensional observations) will be treated convolutionally.

    I hope that's all you need to do. I'm going to try to adapt our GridWorld example to use a custom sensor like this that will have 4 channels (empty, agent, goal, pit) as a better example of how to do with.
     
    MarkTension likes this.
  12. celion_unity

    celion_unity

    Unity Technologies

    Joined:
    Jun 12, 2019
    Posts:
    147
    MarkTension likes this.
  13. MarkTension

    MarkTension

    Joined:
    Aug 17, 2019
    Posts:
    30
    Thank you, this is very clear now!
    Hope it improves my agent. I guess it should.
    Until now I was doing a sort of a clunky bit-encoding of 6 channels into into RGB, which sort of worked, but could be harder to learn I guess. Might also improve training speed without rendertextures etc. This is a nice feature.
     
    Last edited: Jun 19, 2020
  14. MarkTension

    MarkTension

    Joined:
    Aug 17, 2019
    Posts:
    30
    Btw, I can confirm that this is working well now*
     
  15. celion_unity

    celion_unity

    Unity Technologies

    Joined:
    Jun 12, 2019
    Posts:
    147
    Great, glad it's working.

    For Hackweek, one of our engineers did some prototyping with segmentation masks as observations, and came up with a pretty good way of using multiple PNG images (one image for each 3 channels). Hopefully we'll have that cleaned up as an example soon (it takes some changes on the python side too).
     
    MarkTension likes this.
unityunity