Can I change the number of channels for a visual observation?

Hi there!

for my project I'd like to change the number of channels to 5 for a visual observation.
Is this in any way supported?
I thought the float visual observations, introduced in version 14, could maybe be of use if I can pass a multi array instead of a rendertexture?

Any ideas?

 

Hi @MarkTension - I think that the float visual observations will work, although I'll admit that I haven't tried it and there might be a limitation on the python side.

There's some example code in https://github.com/Unity-Technologi...ts/Editor/Sensor/FloatVisualSensorTests.cs#L6 that shows how you could set this up. It uses a 2-D float array, but you can use whatever storage makes most sense to you; the main thing you'd need to modify is in the Write() method; you'd need an additional loop over the channels, like this:

Code (CSharp):

1.  
2. for (var h = 0; h < Height; h++)
3. {
4.   for (var w = 0; w < Width; w++)
5.   {
6.     for (var c=0; c< numChannels; c++)
7.     {
8.       adapter[h, w, c] = floatData[h, w, c];
9.     }
10.   }
11. }
12.  

Let me know if you hit any problems, or need more details on how to implement it.

 

Thank you, that sounds doable! Will give it a try and will reply if it works. Cheers

 

Heya!

In what context should this script be used?
Should it be attached to the agent gameobject just like the other sensors?
Or should it be called from the agent's CollectObservations() method?

Also, when making a new c# script, and copying floatVisualSensorTest code to it, I get an error with

using MLAgents.Sensors;

Doesn't find the Sensors subclass.

(I'm working in ml-agents14)

 

Re: MLAgents.Sensors - that should be in the 0.14.1 release (https://github.com/Unity-Technologi...Tests/Editor/Sensor/FloatVisualSensorTests.cs). We briefly removed it on master but brought it back today.

We don't have a great example of adding a custom Sensor at the moment, but I'm working on converting one of our examples to do that. The basics are that you'll want to:
* Create a subclass of ISensor and override the Write() method (sounds like you have that part already).
* Create a subclass of SensorComponent, and return an instance of your ISensor implementation in the CreateSensor method.

You can see the work-in-progress here: https://github.com/Unity-Technologi...8/files#diff-0fdc9d63b7f92730e2e53734846ee3dd although it's using SensorBase instead of ISensor; hopefully that's enough to get you going.

 

@MarkTension Were you able to get this going, thinking about giving it a try but I don't want to move my project to 14.1 unless it does work. Maybe you can share how you did it?

 

Didn't try yet, but still interested to see how it works. Will try it this week and I'll post here if successful.

 

@celion_unity
I'm a little confused on the following part of the code in the visualsensortests code. I don't quite understand where the actual observation is written. I've duplicated the code in that file and I'm calling the code below after writing to the sonsor.floatData array. However when running the ml agents it's telling me that fewer observations are being made than expected. i.e the size of the sensor array. I usually add observations by AddVectorObs is this a step I'm missing? I'm confused on how to make this work. Do i need to add this floatvisualsensortests.cs to a gameobject instead of instantiating the class?

Code (csharp):

1.  
2. var output = new float[12];
3. var writer = new WriteAdapter();
4. writer.SetTarget(output, sensor.GetObservationShape(), 0);
5. sensor.Write(writer);
6.  

This appears to write to the adapter

 

Sorry, you're right, the VisualSensorTests is probably missing a few pieces. What you'll want to do to add this to an Agent is
1) Create a new SensorComponent implementation that returns your ISensor implementation
2) Attach this SensorComponent to your Agent.

The newest version of the Basic scene has an example of this: https://github.com/Unity-Technologi...xamples/Basic/Scripts/BasicSensorComponent.cs

You shouldn't need to call SetTarget or Write at all, and this doesn't contribute to the number of vector observations.

 

Hi @celion_unity thanks so far for the help. I didn't get to it last months, but still very interested to try. Finding it difficult to get a grasp of what to do exactly though.

As I understand so far:
I should forget about the visualsensortest scrip, because now I can make a subclass of the Sensorcomponent class, which is done as well by The Basic Example.

When copying the BasicSensorComponent c# into my project and changing the code to suit my environment I run into a problem.

- cannot resolve symbol SensorBase. For a while I didn't get an error by having the line 'using MLAgentsExamples;' , but for some reason this only worked one time.
- will this method automatically use the visual observation encoding (convolutions instead of one dense encoding)?
- do i need to do more than just adapting the basisSensor script to my environment, or am I still missing something

 

SensorBase is only in the example code. It's not necessary for writing sensors, but it might make some things simpler.

Yes, "visual" observations (we need a better name for non-rendered multi-dimensional observations) will be treated convolutionally.

I hope that's all you need to do. I'm going to try to adapt our GridWorld example to use a custom sensor like this that will have 4 channels (empty, agent, goal, pit) as a better example of how to do with.

 

Here's (hopefully) a better example.
https://github.com/Unity-Technologi...ples/GridWorld/Scripts/GridSensorComponent.cs

I still need to do some more cleanup, and make sure that it actually trains before merging it to master, but I think the general idea is there.

 

Thank you, this is very clear now!
Hope it improves my agent. I guess it should.
Until now I was doing a sort of a clunky bit-encoding of 6 channels into into RGB, which sort of worked, but could be harder to learn I guess. Might also improve training speed without rendertextures etc. This is a nice feature.

 

Btw, I can confirm that this is working well now*

 

Great, glad it's working.

For Hackweek, one of our engineers did some prototyping with segmentation masks as observations, and came up with a pretty good way of using multiple PNG images (one image for each 3 channels). Hopefully we'll have that cleaned up as an example soon (it takes some changes on the python side too).

 

Just an update, the changes to support multiple PNGs just landed on master as part of a larger PR (https://github.com/Unity-Technologies/ml-agents/pull/4399). You'll still need to do some wrangling to get you channel packed into textures then to PNGs and concatenate the PNG bytes, but hopefully there's a net savings in training time by reducing the data transferred between Unity and the trainer.

 

Cool! If significantly faster I'd be interested to try

 

I haven't done a comparison yet for this particular case. But I benchmarked compressed vs uncompressed on GridWorld a few months ago and it made a 2x or 4x difference.

 

Sounds great! I went ahead and found this unofficial docs release for Grid Sensor (Couldn't find Grid-Sensor in master branch's docs/ folder)

My current approach is copying my state list of int[channels] into the the ISensor write function.
Since I have it already encoded, I understand from your post and the communication section, I'll need to
- convert this state list into Unity's 2D texture
- call (gridsensor. ?)SetPixels on it
- call (gridsensor. ?)EncodeToPNG to encode all to PNG encoding

I'm not sure how to exactly go about it. Is there a more detailed explanation with code examples? Or a gridworld example that I can use for reference?

 

I'm having trouble understanding if/how PNG compression impacts training and inference, besides faster Unity to Python communication.

For a current project, I need a bare-bones grid sensor that does nothing more than wrapping a grid data structure. I tried implementing my own, because although Jaden Travnik's sensor is great, it parses colliders, tags and so on which is a bit overkill for what I want to do right now.

Here's my sensor implementation
https://github.com/mbaske/grid-sensor/tree/master/GridSensor

It works fine with uncompressed data. When I PNG-encode observations, training proceeds just as well, but inference behaviour is erratic. I can't figure out why that's the case. If I had messed up encoding somehow, wouldn't that affect training too?
https://github.com/mbaske/grid-sensor/blob/master/GridSensor/GridSensor.cs#L65

Code is here https://github.com/Unity-Technologi...ents.extensions/Runtime/Sensors/GridSensor.cs

 

Never mind - I forgot that SetPixels32 actually expects pixel data to start at the bottom row. Therefore my training and inference observations were vertically flipped. Fixed in https://github.com/mbaske/grid-sensor/blob/master/GridSensor/PixelGrid.cs#L71

 

Thanks for sharing this! I'm trying to use it but I've got a few questions:
To get it to work I've imported the scripts into my project.
Then from the agent script I initialize the pixelgrid and gridsensor with something like this:

Code (CSharp):

1.  
2. PixelGrid pixelgrid1 = new PixelGrid(9,AlloSize,AlloSize,"grid1");
3. GridSensor sensor1 = new GridSensor(pixelgrid1, SensorCompressionType.PNG, "sensor1");

For storing my float array with different channels I understand that I can loop through my input data and store into pixelgrid1 like so:

Code (CSharp):

1. pixelgrid1.Write(channel,w,h,value);
2.  

What is the next step? I guess I need to call call the sensor each step like so?

Code (CSharp):

1. sensor1.Write();

If so, What should i enter as an argument to Write??

Also, will this work with the lastest stable release? :Or should I update to the latest release on the master branch? (for some reason that one doesn't like

using Unity.MLAgents

) ...

Thanks in advance

 

Sorry, I should add a readme. Still need to build a good working example project for the GridSensor.

Please use the GridSensorComponent to wrap the sensor, I don't think the sensor will work when it's instantiated this way.
https://github.com/mbaske/grid-sensor/blob/master/GridSensor/GridSensorComponent.cs
EDIT: Just to be clear: don't instantiate GridSensor at all. Just add the component which will take care of creating the sensor.

Add the GridSensorComponent to the same transform as your agent. The agent class has to implement the IPixelGridProvider interface.
https://github.com/mbaske/grid-sensor/blob/master/GridSensor/IPixelGridProvider.cs

Meaning it needs to provide a Pixelgrid instance for the component/sensor to work with via

Code (CSharp):

1. public PixelGrid GetPixelGrid()
2. {
3.     return yourPixelgrid;
4. }

Everything else is handled by ML-Agents, you don't need to invoke any methods on the sensor, like sensor1.Write();

Yes, just write your values to the PixelGrid with the Write method.

Yes, it works for me with 1.0.4

 

I don't get this. Do you mean this automatically happens when I add the sensorComponent script to the agent gameobject?
Or do I manually have to call this from my agent script? If so, how do I do that exactly? Sorry, not a champion with C# ..

In any case, I added two GridSensorComponents to the agent gameobject,
and instantiated two pixelgrids

Code (CSharp):

1. _pixelgrid_x = new PixelGrid(9,AlloSize,AlloSize,"gridX");
2. _pixelgridy_y = new PixelGrid(9,AlloSize,AlloSize,"gridY");

(not sure how these two separate pixelgrids can be linked to each separate gridSensorComponent)

This error results:
NullReferenceException: Object reference not set to an instance of an object
MBaske.GridSensorComponent.GetObservationShape () (at Assets/Clay2D_feb21/scripts/sensors/GridSensor/GridSensorComponent.cs:53)

Code (CSharp):

1. NullReferenceException: Object reference not set to an instance of an object
2. MBaske.GridSensorComponent.GetObservationShape () (at Assets/Clay2D_feb21/scripts/sensors/GridSensor/GridSensorComponent.cs:53)

How do I fix this? Guess it has to do with the iPixelGrid provider?

 

Yes, you're getting this error because the GridSensorComponent is looking for script that implements the IPixelGridProvider interface. Normally, that would be your agent.

Code (CSharp):

1. public class YourAgent : Agent, IPixelGridProvider
2. {
3.     /// <summary>
4.     /// Implements the <see cref="IPixelGridProvider"/> interface.
5.     /// </summary>
6.     /// <returns><see cref="PixelGrid"/> instance.</returns>
7.     public PixelGrid GetPixelGrid()
8.     {
9.         return m_Grid;
10.     }
11. ...

I haven't really thought about handling more than one grid yet, to be honest. If your two grids have the same width/height, you might just combine them into a single one with more channels. Or you could check "Use Child Sensors" and place multiple GridSensorComponents in the hierarchy below your agent. Again, each of them would expect a script implementing IPixelGridProvider next to them. Well, I should come up with a better way of assigning the PixelGrid to the sensor - it's not a serializable object and can't be dragged&dropped in the inspector.

There is still an issue with PNG compression if the grid has more than 3 channels. In that case, I'm getting a python error saying observation shapes don't match, since the compressed data has only 3 color channels. However, I'm having the same problem with the ML-Agent's grid sensor. https://github.com/Unity-Technologies/ml-agents/issues/4456

 

Sorry, I haven't been keeping up with this thread.

Correct, that's the only difference. I haven't profiled this in a while, but when I first experimented with it on our GridWorld scene, compression sped up training by a factor of 2x or 4x. But the end model should be the same, and inference is unaffected (we always take the uncompressed path for inference).

I'm working on some utilities for Match 3 games; this branch is still very much a work-in-progress, but this utility might be helpful: https://github.com/Unity-Technologi...h3/Scripts/Match3SensorComponent.cs#L150-L185

 

I should also add that your mileage may vary based on image complexity, etc. A usecase like image segmentation should have large blocks of identical "color" that would compress well, and I hope one-hot encoding for game boards will work well too, but the images might be too small for it to matter.

 

For anyone who cares, using the new sensor also increased my training speed x2. Also having multiple sensors was implementable using child sensors, thanks @mbaske

 

@mbaske @MarkTension @celion_unity Thank you for having this public discussion on grid sensors with more than three channels!

I have created a custom tilemap-sensor in my game where it correctly finds data on my tilemap. But i was having problems wrapping my head around how i would feed data to my model it a way where CNNs were used.

Reading through your discussion, and realising i have to create a new sensorcomponent etc, really helps!

looking forward to try it out!