Can I change the number of channels for a visual observation?

Hi there!

for my project I'd like to change the number of channels to 5 for a visual observation.
Is this in any way supported?
I thought the float visual observations, introduced in version 14, could maybe be of use if I can pass a multi array instead of a rendertexture?

Any ideas?

 

Thank you, that sounds doable! Will give it a try and will reply if it works. Cheers

 

Heya!

In what context should this script be used?
Should it be attached to the agent gameobject just like the other sensors?
Or should it be called from the agent's CollectObservations() method?

Also, when making a new c# script, and copying floatVisualSensorTest code to it, I get an error with

using MLAgents.Sensors;

Doesn't find the Sensors subclass.

(I'm working in ml-agents14)

 

@MarkTension Were you able to get this going, thinking about giving it a try but I don't want to move my project to 14.1 unless it does work. Maybe you can share how you did it?

 

Didn't try yet, but still interested to see how it works. Will try it this week and I'll post here if successful.

 

@celion_unity
I'm a little confused on the following part of the code in the visualsensortests code. I don't quite understand where the actual observation is written. I've duplicated the code in that file and I'm calling the code below after writing to the sonsor.floatData array. However when running the ml agents it's telling me that fewer observations are being made than expected. i.e the size of the sensor array. I usually add observations by AddVectorObs is this a step I'm missing? I'm confused on how to make this work. Do i need to add this floatvisualsensortests.cs to a gameobject instead of instantiating the class?

Code (csharp):

1.  
2. var output = new float[12];
3. var writer = new WriteAdapter();
4. writer.SetTarget(output, sensor.GetObservationShape(), 0);
5. sensor.Write(writer);
6.  

This appears to write to the adapter

 

Hi @celion_unity thanks so far for the help. I didn't get to it last months, but still very interested to try. Finding it difficult to get a grasp of what to do exactly though.

As I understand so far:
I should forget about the visualsensortest scrip, because now I can make a subclass of the Sensorcomponent class, which is done as well by The Basic Example.

When copying the BasicSensorComponent c# into my project and changing the code to suit my environment I run into a problem.

- cannot resolve symbol SensorBase. For a while I didn't get an error by having the line 'using MLAgentsExamples;' , but for some reason this only worked one time.
- will this method automatically use the visual observation encoding (convolutions instead of one dense encoding)?
- do i need to do more than just adapting the basisSensor script to my environment, or am I still missing something

 

Thank you, this is very clear now!
Hope it improves my agent. I guess it should.
Until now I was doing a sort of a clunky bit-encoding of 6 channels into into RGB, which sort of worked, but could be harder to learn I guess. Might also improve training speed without rendertextures etc. This is a nice feature.

 

Btw, I can confirm that this is working well now*

 

Cool! If significantly faster I'd be interested to try

 

Sounds great! I went ahead and found this unofficial docs release for Grid Sensor (Couldn't find Grid-Sensor in master branch's docs/ folder)

My current approach is copying my state list of int[channels] into the the ISensor write function.
Since I have it already encoded, I understand from your post and the communication section, I'll need to
- convert this state list into Unity's 2D texture
- call (gridsensor. ?)SetPixels on it
- call (gridsensor. ?)EncodeToPNG to encode all to PNG encoding

I'm not sure how to exactly go about it. Is there a more detailed explanation with code examples? Or a gridworld example that I can use for reference?

 

I'm having trouble understanding if/how PNG compression impacts training and inference, besides faster Unity to Python communication.

For a current project, I need a bare-bones grid sensor that does nothing more than wrapping a grid data structure. I tried implementing my own, because although Jaden Travnik's sensor is great, it parses colliders, tags and so on which is a bit overkill for what I want to do right now.

Here's my sensor implementation
https://github.com/mbaske/grid-sensor/tree/master/GridSensor

It works fine with uncompressed data. When I PNG-encode observations, training proceeds just as well, but inference behaviour is erratic. I can't figure out why that's the case. If I had messed up encoding somehow, wouldn't that affect training too?
https://github.com/mbaske/grid-sensor/blob/master/GridSensor/GridSensor.cs#L65

Code is here https://github.com/Unity-Technologi...ents.extensions/Runtime/Sensors/GridSensor.cs

 

Never mind - I forgot that SetPixels32 actually expects pixel data to start at the bottom row. Therefore my training and inference observations were vertically flipped. Fixed in https://github.com/mbaske/grid-sensor/blob/master/GridSensor/PixelGrid.cs#L71

 

Thanks for sharing this! I'm trying to use it but I've got a few questions:
To get it to work I've imported the scripts into my project.
Then from the agent script I initialize the pixelgrid and gridsensor with something like this:

Code (CSharp):

1.  
2. PixelGrid pixelgrid1 = new PixelGrid(9,AlloSize,AlloSize,"grid1");
3. GridSensor sensor1 = new GridSensor(pixelgrid1, SensorCompressionType.PNG, "sensor1");

For storing my float array with different channels I understand that I can loop through my input data and store into pixelgrid1 like so:

Code (CSharp):

1. pixelgrid1.Write(channel,w,h,value);
2.  

What is the next step? I guess I need to call call the sensor each step like so?

Code (CSharp):

1. sensor1.Write();

If so, What should i enter as an argument to Write??

Also, will this work with the lastest stable release? :Or should I update to the latest release on the master branch? (for some reason that one doesn't like

using Unity.MLAgents

) ...

Thanks in advance

 

Sorry, I should add a readme. Still need to build a good working example project for the GridSensor.

Please use the GridSensorComponent to wrap the sensor, I don't think the sensor will work when it's instantiated this way.
https://github.com/mbaske/grid-sensor/blob/master/GridSensor/GridSensorComponent.cs
EDIT: Just to be clear: don't instantiate GridSensor at all. Just add the component which will take care of creating the sensor.

Add the GridSensorComponent to the same transform as your agent. The agent class has to implement the IPixelGridProvider interface.
https://github.com/mbaske/grid-sensor/blob/master/GridSensor/IPixelGridProvider.cs

Meaning it needs to provide a Pixelgrid instance for the component/sensor to work with via

Code (CSharp):

1. public PixelGrid GetPixelGrid()
2. {
3.     return yourPixelgrid;
4. }

Everything else is handled by ML-Agents, you don't need to invoke any methods on the sensor, like sensor1.Write();

Yes, just write your values to the PixelGrid with the Write method.

Yes, it works for me with 1.0.4

 

I don't get this. Do you mean this automatically happens when I add the sensorComponent script to the agent gameobject?
Or do I manually have to call this from my agent script? If so, how do I do that exactly? Sorry, not a champion with C# ..

In any case, I added two GridSensorComponents to the agent gameobject,
and instantiated two pixelgrids

Code (CSharp):

1. _pixelgrid_x = new PixelGrid(9,AlloSize,AlloSize,"gridX");
2. _pixelgridy_y = new PixelGrid(9,AlloSize,AlloSize,"gridY");

(not sure how these two separate pixelgrids can be linked to each separate gridSensorComponent)

This error results:
NullReferenceException: Object reference not set to an instance of an object
MBaske.GridSensorComponent.GetObservationShape () (at Assets/Clay2D_feb21/scripts/sensors/GridSensor/GridSensorComponent.cs:53)

Code (CSharp):

1. NullReferenceException: Object reference not set to an instance of an object
2. MBaske.GridSensorComponent.GetObservationShape () (at Assets/Clay2D_feb21/scripts/sensors/GridSensor/GridSensorComponent.cs:53)

How do I fix this? Guess it has to do with the iPixelGrid provider?

 

Yes, you're getting this error because the GridSensorComponent is looking for script that implements the IPixelGridProvider interface. Normally, that would be your agent.

Code (CSharp):

1. public class YourAgent : Agent, IPixelGridProvider
2. {
3.     /// <summary>
4.     /// Implements the <see cref="IPixelGridProvider"/> interface.
5.     /// </summary>
6.     /// <returns><see cref="PixelGrid"/> instance.</returns>
7.     public PixelGrid GetPixelGrid()
8.     {
9.         return m_Grid;
10.     }
11. ...

I haven't really thought about handling more than one grid yet, to be honest. If your two grids have the same width/height, you might just combine them into a single one with more channels. Or you could check "Use Child Sensors" and place multiple GridSensorComponents in the hierarchy below your agent. Again, each of them would expect a script implementing IPixelGridProvider next to them. Well, I should come up with a better way of assigning the PixelGrid to the sensor - it's not a serializable object and can't be dragged&dropped in the inspector.

There is still an issue with PNG compression if the grid has more than 3 channels. In that case, I'm getting a python error saying observation shapes don't match, since the compressed data has only 3 color channels. However, I'm having the same problem with the ML-Agent's grid sensor. https://github.com/Unity-Technologies/ml-agents/issues/4456

 

For anyone who cares, using the new sensor also increased my training speed x2. Also having multiple sensors was implementable using child sensors, thanks @mbaske

 

@mbaske @MarkTension @celion_unity Thank you for having this public discussion on grid sensors with more than three channels!

I have created a custom tilemap-sensor in my game where it correctly finds data on my tilemap. But i was having problems wrapping my head around how i would feed data to my model it a way where CNNs were used.

Reading through your discussion, and realising i have to create a new sensorcomponent etc, really helps!

looking forward to try it out!