Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Question Generate Collider Shape, Location & Size from a Mask PNG

Discussion in '2D' started by LukeGeach, Jan 4, 2023.

  1. LukeGeach

    LukeGeach

    Joined:
    Nov 22, 2019
    Posts:
    8
    Hi All,

    I am reaching burnout on trying to find the best way of satisfying a clients requirement.
    In this training scenario, the user is required to draw rectangles around objects if they meet a certain criteria and they are then scored on their accuracy.

    As you can see in this image, on the real system the user draws a red box around an object.
    Screenshot 2023-01-04 at 11.34.13.png

    In the simulator i am building my thought process is this, a mask is layered on top of the image, and an area is defined for a collider to be generated, when the user then draws an area, cuboid game object is created with a collider allowing to calculate the accuracy of the user. Sounds pretty simple.

    Here is the mask (white area to be where a collider is generated)
    Screenshot 2023-01-04 at 11.30.39.png

    And here is the mask layered over top the original image, with the white area still visible just to visualise the area.

    Screenshot 2023-01-04 at 11.30.25.png

    The reason masks are being considered is because the source of these images (there will be thousands) is capable of generating these black and white masks as well as exporting the images.

    The confusion / issue / hair pulling stems from using these masks to generate colliders.
    At the moment I can apply a collider to the mask layer (its a simple Rect Transform / Image gameobject)...
    Screenshot 2023-01-04 at 11.40.02.png
    ...but i'm struggling to scale the size of any collider to the size of the white area and not just the full dimensions of the game object.

    If anyone has any suggestions on how to solve this, or approach it in a different way I would be forever grateful!
     
  2. LukeGeach

    LukeGeach

    Joined:
    Nov 22, 2019
    Posts:
    8
    As an addition, there are 2 views the user can choose from, a front or a top view of the scan.
    My brain thinks if these two views can be used to generate the size and position of a new cuboid game object that would be the solution.
    Screenshot 2023-01-04 at 12.07.15.png
     
  3. MelvMay

    MelvMay

    Unity Technologies

    Joined:
    May 24, 2013
    Posts:
    11,333
    Well this peaked my curiosity because I was the senior imaging dev for a company using 3D XRay technology using crenelated multi-energy detectors. Also involved in the US TIP (Thread Image Projection) stuff too injecting fake threats for training. ;)

    That said, the above is UI not 2D sprites and I'm not quite sure what the issue is here. I'm not sure what you're using for the 2D collider nor how you're identifying the area in question. Nor do I follow why you need a collider at all. To note, the UI system and 2D/Physics don't have any interaction at all.

    Happy to help if there's a specific problem in some script etc.
     
  4. LukeGeach

    LukeGeach

    Joined:
    Nov 22, 2019
    Posts:
    8
    Qu'plah Melv,

    You got my number ;)

    So in the final version the user will not be able to see the threats (red outlines) and will be asked to manually identify them (this is a certification process). To do this they must draw a rectangle over each threat they identify (it will appear as a red outline as seen in the first image).

    The complexity comes that these images are random sets of images loaded in from an online database, and the certification app its self will be hosted on an LMS and use xAPI. This LMS part is the easy part.

    So, it needs, at runtime, to load the images and know what parts of the images the user needs to annotate.

    To do this my idea was to also load in masks over the top of the image, not visible to the user.
    Those masks then could generate a game object collider, the white area would define the dimensions and location of the 2d collider.

    The user would then draw over this which would generate another gameobject with a collider, the app knows the user has identified the threat through a collision and the user passes this scenario.

    I guess simply what i need help with is the best way to place colliders at the points where there are threats.
     
  5. LukeGeach

    LukeGeach

    Joined:
    Nov 22, 2019
    Posts:
    8
    OH MY GOD, that simple realisation where you alerted me to UI v's 2D sprites solved everything.

    Sprite renderer instead of Image Sprite, jesus I feel foolish lol
    Screenshot 2023-01-04 at 12.50.44.png
     
    MelvMay likes this.
  6. MelvMay

    MelvMay

    Unity Technologies

    Joined:
    May 24, 2013
    Posts:
    11,333
    Glad you got it working. Thank you for the massive nostalgia trip I just had. I lived and breathed this kind of stuff for about 8 years at 3DX-Ray.
     
  7. LukeGeach

    LukeGeach

    Joined:
    Nov 22, 2019
    Posts:
    8
    Haha my pleasure, i'm now starting to live and breathe it myself. It's providing a super unique challenge compared to making digital product twins.
     
  8. MelvMay

    MelvMay

    Unity Technologies

    Joined:
    May 24, 2013
    Posts:
    11,333
    Good luck with it! Ping me if you need anything.