Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

3D Grid Sensor, Update for ML-Agents 2.1

Discussion in 'ML-Agents' started by mbaske, Feb 8, 2021.

  1. mbaske

    mbaske

    Joined:
    Dec 31, 2017
    Posts:
    473
    Hi, I've seen this issue come up in the forum before and also struggled with it myself - namely how to observe a variable number of objects. The grid sensor (https://github.com/Unity-Technologi...ents.extensions/Runtime/Sensors/GridSensor.cs) is great for 2D, but if you need to track objects that aren't on a plane, it's either cameras or (lots of) raycasts.

    Now, I wrote my own grid sensor a while back, because I wanted a minimal solution that does nothing more than encoding float values as visual observations. The plan was to later extend it for specific use cases. One such extension is the Spatial Grid Sensor, it detects colliders via OverlapSphere and projects their polar coordinates onto the observed grid.
    I haven't done any extensive testing yet, like comparing how well the sensor does compared to using a ton of raycasts. However, if you want to check it out, you can find the code, description and two example projects here: https://github.com/mbaske/grid-sensor

     
  2. awjuliani

    awjuliani

    Unity Technologies

    Joined:
    Mar 1, 2017
    Posts:
    69
    Hi mbaske. This seems like a very cool extension of the GridSensor. Thanks for sharing it!
     
    mbaske likes this.
  3. mbaske

    mbaske

    Joined:
    Dec 31, 2017
    Posts:
    473
    I've updated the sensor and generalized it for 2D and 3D detection. You'll find an in-depth description of how it works and how to use it at https://github.com/mbaske/grid-sensor

    I also added a comparison with the Eidos grid sensor. It's apples and oranges, really. My sensor uses a different detection approach, which was designed for 3D and is probably a bit overkill for 2D. It's faster though ;)
     
  4. mbaske

    mbaske

    Joined:
    Dec 31, 2017
    Posts:
    473
    My grid sensor(s) have been updated for ML-Agents 2.1 - Changes include:
    • New minimalistic GridSensorComponent and accompanying Maze Explorer example.
    • Component inspector settings are grouped into foldouts.
    • Integrated visualization and debugging options.
    • There are no longer separate 2D and 3D variants of the DetectableGameObject component.
    • Easier handling of observables, 'Distance' and 'One-Hot' are set depending on detection type.
    • Improved shape scanning algorithm and interface. Now supports concave colliders and context sensitive levels of detail.
    • The 'Scan-At-Runtime' option has been removed in favour of explicit method calls.
    • Frustum culling for GridSensorComponent3D with convex FOVs increases detection speed.
    • For stacked observations, the grid sensor is now being wrapped in ML-Agent's StackingSensor.
    • Optional detectors and encoders implement the IDetector and IEncoder interfaces, rather than extending abstract classes.
    • Various classes have been renamed, e.g. the former ChannelGrid and PixelGrid are now called GridBuffer and ColorGridBuffer.
    • Code refactoring and comments.
    https://github.com/mbaske/grid-sensor

     
    Last edited: Jun 29, 2021
  5. Z_Doctor

    Z_Doctor

    Joined:
    Jun 22, 2018
    Posts:
    6
    Looks very cool and well done. I'll keep this in mind if I ever find the need
     
    mbaske likes this.
  6. nosferathoo

    nosferathoo

    Joined:
    Sep 20, 2012
    Posts:
    5
    mbaske likes this.
  7. WaxyMcRivers

    WaxyMcRivers

    Joined:
    May 9, 2016
    Posts:
    59
    Legend
     
    mbaske likes this.
  8. mohdafzal20299

    mohdafzal20299

    Joined:
    Dec 1, 2020
    Posts:
    7
    Hi I am trying to use your scripts to make a grid sensor. I have done everything as done in your DogFight scene, but I am getting the following error-
    ArgumentNullException: Value cannot be null.
    Parameter name: key
    System.Collections.Generic.Dictionary`2[TKey,TValue].FindEntry (TKey key) (at <695d1cc93cca45069c528c15c9fdd749>:0)
    System.Collections.Generic.Dictionary`2[TKey,TValue].TryGetValue (TKey key, TValue& value) (at <695d1cc93cca45069c528c15c9fdd749>:0)
    MBaske.Sensors.Grid.DetectionResult.TryGetItems (System.String tag, System.Collections.Generic.IList`1[MBaske.Sensors.Grid.DetectionResult+Item]& items) (at Assets/Scripts/Sensors/Grid/Basic/Detection/DetectionResult.cs:109)
    MBaske.Sensors.Grid.GridSensorComponentBaseGO+<GetDetectedGameObjects>d__22.MoveNext () (at Assets/Scripts/Sensors/Grid/GameObject/Sensor/GridSensorComponentBaseGO.cs:165)
    SimpleInputControls.CollectObservations (Unity.MLAgents.Sensors.VectorSensor sensor) (at Assets/SimpleInputControls.cs:104)

    Please let me know if you have any solution else let me know what other details you will require to solve this. P.S. - The error takes me here-
     

    Attached Files:

  9. mbaske

    mbaske

    Joined:
    Dec 31, 2017
    Posts:
    473
    Hi @mohdafzal20299
    I haven't looked at this for a while... just did a clean install of Unity Editor 2021.3.151f (Windows), cloned my grid-sensor repo, re-imported the three items in /Examples/!Reimport/ folder and things look ok. I can run the example scenes without any issues.
    From the error you're getting, it looks like the 'tag' argument is null, causing the exception when trying to look up an associated dictionary item. Not sure where that null tag might come from though... Could you check your detectable gameobjects have valid tags and sensor settings are referencing them?