Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Feedback Train models from images?

Discussion in 'Unity Simulation' started by mcbauer, Oct 23, 2020.

Thread Status:
Not open for further replies.
  1. mcbauer

    mcbauer

    Joined:
    Oct 10, 2015
    Posts:
    524
    I've been very successful so far with ml-agents and the 3d game world, but now my agent needs image detection. I have an ml-agent drone that is able to fly around and navigate its world, but I want it to recognize whether a target is familiar or unfamiliar. If it is familiar it will return to a default monitoring mode. However, if the target is unfamiliar, it will track its movement and follow it.

    How do I start teaching it this new skill?

    My initial thought was to have it recognize simple colors as familiar: red, blue, green, and then cyan as unfamiliar, but I wondered if I am going about it all wrong...since my real goal is face or image detection, shouldn't I just go in that direction? But again....I'm not how to do that with the current plugin.

    This lead me to considering how to do image detection at all in unity which lead me to the Perception engine--I love the idea of generating synthetic data. I'm just not sure what to do with the labeled data in Unity.

    Once I have a few thousand or so labeled images, how can start training models?

    Is this even available inside Unity yet?

    I read this blog: https://blogs.unity3d.com/2020/09/1...-synthetic-data-using-unity-perception-tools/

    ...hoping to gain some insight but it seems it's all 3rd party tools to actually conduct the training.

    Is there something I'm missing, or is this just something not yet integrated in ml-agents?

    Any guidance would be appreciated, thanks!
     
  2. jamesfort

    jamesfort

    Unity Technologies

    Joined:
    Feb 3, 2020
    Posts:
    7
    Hello, I'm the product manager for Unity's Perception solution. You found the right blog post! I'm happy to discuss the easiest path to model training with you in a meeting. There's nothing directly available for computer vision training natively in Unity today but we are working on things that will make this progressively easier. Please email computer-vision@unity.com and we can setup a conversation.

    Thanks,
    James
     
    Last edited by a moderator: Mar 1, 2021
  3. stevenke_unity

    stevenke_unity

    Joined:
    Nov 13, 2018
    Posts:
    30
    Last edited by a moderator: Mar 1, 2021
Thread Status:
Not open for further replies.