Search Unity

Question Semantic tags generation with script.

Discussion in 'Unity MARS' started by hmkn, Nov 30, 2020.

  1. hmkn

    hmkn

    Joined:
    Nov 7, 2019
    Posts:
    23
    Hello. I'm a biggner of Unity MARS.
    I'd like to automate in generating semantic tags traits.

    Situation: I have positions and label data list that collected from 3d space using deep learning inferernce.
    What I need to: I'd like to convert these position data into semantic tag. Is it possible to generate traits by script?

    Thanks.
     
    hmkn, Nov 30, 2020
    #1

  2. jmunozarUTech

    jmunozarUTech

    Unity Technologies

    Joined:
    Jun 1, 2020
    Posts:
    297
    hello @hmkn

    Yes you can generate traits through scripting, to use your type as a trait you must compile your type into its assembly and generate some code; to make this happen move your generated code into 
    Assets/MARS/Extension Types
    folder. A recompile will happen, and then your trait type is ready to use.

    To create your own trait check Providing Traits via Scripts from our docs https://docs.unity3d.com/Packages/com.unity.mars@1.2/manual/Traits.html#semantic-tags
     
    jmunozarUTech, Nov 30, 2020
    #2

  3. unity_andrewm

    unity_andrewm

    Unity Technologies

    Joined:
    Jun 25, 2015
    Posts:
    73
    If the semantic tag is just a string label you won't need to create a new type - the existing tag type will work.
     
    unity_andrewm, Nov 30, 2020
    #3
    jmunozarUTech likes this.

  4. hmkn

    hmkn

    Joined:
    Nov 7, 2019
    Posts:
    23
    Thanks for the quick response.
    I assume tag is just a string label. So I use existing tag type.
    Could you show me a sample implementation to generate traits by script?
     
    hmkn, Dec 1, 2020
    #4

  5. hmkn

    hmkn

    Joined:
    Nov 7, 2019
    Posts:
    23
    I'm trying to create traits via scripts. What is the data IDs and is it possible to add them via scripts?

    "To create a data ID in the first place, use AddOrUpdateData, which comes from IUsesMARSTrackableData." Is there a sample code for it?
     
    hmkn, Dec 8, 2020
    #5

  6. unity_andrewm

    unity_andrewm

    Unity Technologies

    Joined:
    Jun 25, 2015
    Posts:
    73
    Data IDs are what MARS uses to associate different sources of data together.

    You probably want to use a reasoning API for your use case.
    You can find them in: Packages\com.unity.mars\Runtime\Scripts\Reasoning APIs

    FloorReasoningAPI and InViewReasoningAPI are good examples of dynamically applying semantic tags to detected or generated data.
     
    unity_andrewm, Dec 8, 2020
    #6

  7. hmkn

    hmkn

    Joined:
    Nov 7, 2019
    Posts:
    23
    hmkn, Dec 8, 2020
    #7

  8. hmkn

    hmkn

    Joined:
    Nov 7, 2019
    Posts:
    23
    I'm still not familiar with the concepts of Reasoning APIs and Synthetic Data, so please let me know.
    SynthesizedSemanticTag has AddTrait and Reasoning APIs IProvidesTraitsMethods also has methods like AddOrUpdateTrait, but how can Unity.MARS.Data.Reasoning and Unity.MARS.Data.Synthetic be used properly?
    For example, if I want to use a certain position (Pose in this case) as Traits on MARS, do you use SynthesizedPose in Synthetic or TraitNames.Pose in Reasoning API?
     
    hmkn, Dec 9, 2020
    #8

  9. hmkn

    hmkn

    Joined:
    Nov 7, 2019
    Posts:
    23
    IMG_1DBB7E72FC28-1.jpeg
    Is my image correct?
     
    hmkn, Dec 11, 2020
    #9

  10. hmkn

    hmkn

    Joined:
    Nov 7, 2019
    Posts:
    23
    2020-12-11 16.02.03.png
    I understand that I attach synthetic object which consists of synthesized object and synthesized traits for MARS database. I would like to write Semantic Tag by script. However it seems private value. (SynthesizedSemanticTag.cs in line 16.) What should I do to insert value?
     
    Last edited: Dec 14, 2020
    hmkn, Dec 11, 2020
    #10

  11. unity_andrewm

    unity_andrewm

    Unity Technologies

    Joined:
    Jun 25, 2015
    Posts:
    73
    Hi - A few answers to your questions

    Your image is close. MARS has a general concept of 'data' that can come from devices, synthetic objects, or reasoning APIs. Synthetic Objects is our system of translating Unity GameObject data to MARS data that can match queries. This is why all of those systems have methods of adding and removing traits :)

    Synthetic Objects/Traits

    With the currently released version of MARS, The 'trait' aspect of Synthesized Objects is mostly meant to be unchanging from author time (while the value is dynamic). The reason for this is that it gives us a good idea of the capabilities the application needs from the device.. We are working on an update that offers a straightforward way to change these dynamically via script.

    That being said, if you want to dynamically modify a synthetic semantic tag in the current version of MARS, I suggest you make your own synthesized bool trait (Just copy the SynthesizedSemanticTag) and give the trait name a setter as well. You will need to disable/reenable a synthesizedObject to ensure the changed data properly populates the MARS database.

    Reasoning APIs
    These are much more set up for dynamic trait modification. If you are creating and managing where a series of poses are that don't need to go through the game object loop, I suggest you use a reasoning API (and yes, use TraitNames.Pose)
     
    unity_andrewm, Dec 15, 2020
    #11

  12. hmkn

    hmkn

    Joined:
    Nov 7, 2019
    Posts:
    23
    Hi @unity_andrewm . Thank you for the answer!
    Now I understand the design of MARS, synthetic objects and traits.
     
    hmkn, Dec 15, 2020
    #12
    jmunozarUTech likes this.