Search Unity

ARFoundation 2 image tracking with many ref images and many objects

Discussion in 'Handheld AR' started by SpiderJones, May 18, 2019.

  1. SpiderJones

    SpiderJones

    Joined:
    Mar 29, 2014
    Posts:
    129
    pwshaw and sroq like this.
  2. Dennisrudolph

    Dennisrudolph

    Joined:
    Aug 24, 2017
    Posts:
    1
    I am having the same problem here. Any help would be much appreciated.
     
    pwshaw likes this.
  3. PTW_AR

    PTW_AR

    Joined:
    Jan 13, 2019
    Posts:
    13
    I'm really confused, why they haven't included that feature in the first place; image tracking is kinda useless if you cant generate different prefabs for each reference image. Maybe I'm just too spoiled from the ARKit...

    Anyway, I followed the little guide by @Skeketor23 ( https://forum.unity.com/threads/arfoundation-and-2d-image-target-tracking.538062/page-3#post-4556173 ), but couldn't make it work.

    I simply put my prefabs in the ressources folder and named theme accordingly to their respective image in the referenceImageLibrary.
    Working with the trackableID might be better, but I'm not entirely sure how to use them in that case (are they known a priori? Meaning before building my project?). That would not solve the issue though, I would still face the same problem:
    I can't find a way to assign the prefabs to their respective image. The documentation isn't really helping either in that regard.

    (It's hard to tell what's actually going on because of the lack of a remote for the editor.)
     
    pwshaw and sergiovillakramsky like this.
  4. SpiderJones

    SpiderJones

    Joined:
    Mar 29, 2014
    Posts:
    129
    I use the reference image name and instantiate a prefab

    Code (CSharp):
    1. private void OnTrackedImagesChanged(ARTrackedImagesChangedEventArgs eventArgs)
    2.         {
    3.             for (int i = 0; i < eventArgs.added.Count; i++)
    4.             {
    5.                 eventArgs.added[i].referenceImage.name;
    6. // use that to get the gname
    7. }
    8. }
    Here's an example of using the tracked image event -> https://github.com/Unity-Technologi.../ImageTracking/TrackedImageInfoManager.cs#L97
     
    PTW_AR likes this.
  5. drewfjacobs

    drewfjacobs

    Joined:
    Mar 12, 2019
    Posts:
    4
    I too am trying to add this functionality.
    So Spider did you end up making your own or amending the Tracked Image Manager? Would you mind posting the whole class? Thanks!
     
  6. SpiderJones

    SpiderJones

    Joined:
    Mar 29, 2014
    Posts:
    129
    You should never add code to a framework, you should create classes that encapsulate and adapt the framework. Otherwise you make it impossible to easily update the framework. So yes, I created my own class. My class has a lot of code that is specific to my project so I'm not going to share it. I just added a listener to the event, just like here -> https://github.com/Unity-Technologi.../ImageTracking/TrackedImageInfoManager.cs#L97

    Then, iterated over the other Lists like I said.

    Another tip, when I instantiate my GameObject, I am adding the ARTrackedImage as the parent.
     
  7. drewfjacobs

    drewfjacobs

    Joined:
    Mar 12, 2019
    Posts:
    4
    Thanks, Spider. I am fairly new to C# and OOP as I'm more of a tech artist being tasked with implementing AR in Unity. I'm concurrently going through these tuts: http://rbwhitaker.wikidot.com/c-sharp-tutorials as I try to dev this app. Wish me luck! :D
     
  8. SpiderJones

    SpiderJones

    Joined:
    Mar 29, 2014
    Posts:
    129
    Maybe this will work for you ->

    Code (CSharp):
    1.  
    2. private void OnTrackedImagesChanged(ARTrackedImagesChangedEventArgs eventArgs)
    3.         {
    4.             ARTrackedImage trackedImage = null;
    5.  
    6.             for (int i = 0; i < eventArgs.added.Count; i++)
    7.             {
    8.                 trackedImage = eventArgs.added[i];
    9.                 // instantiate AR object, set trackedImage.transform
    10.                 // use a Dictionary, the key could be the trackedImage, or the name of the reference image -> trackedImage.referenceImage.name
    11.                 // the value of the Dictionary is the AR object you instantiate.
    12.             }
    13.  
    14.             for (int i = 0; i < eventArgs.updated.Count; i++)
    15.             {
    16.                 trackedImage = eventArgs.updated[i];
    17.                 if (trackedImage.trackingState == TrackingState.Tracking)
    18.                 //if (trackedImage.trackingState != TrackingState.None)
    19.                 {
    20.                     // set AR object to active, use Dictionary to get AR object based on trackedImage
    21.                     // you can also include TrackingState.Limited by checking for None
    22.                 }
    23.                 else
    24.                 {
    25.                     // set active to false
    26.                 }
    27.             }
    28.  
    29.             for (int i = 0; i < eventArgs.removed.Count; i++)
    30.             {
    31.                 // destroy AR object, or set active to false. Use Dictionary.
    32.             }
    33.         }
    34.  
     
    pwshaw and Edisyo like this.
  9. PTW_AR

    PTW_AR

    Joined:
    Jan 13, 2019
    Posts:
    13
    Sorry to dig this thread up over and over, but I think it might be helpful for other people working with ImageTracking in the future:

    trackedImage.trackingState (in your example): Does it ever change from "tracking" to "none" for you guys? Although the camera loses the image, the TrackingState remains "tracking" (which is really weird to me).

    I managed to instantiate different prefabs for different images quite easily after having the name avialable (referenceImage.name), load them from the resources folder and assign them their respective parent, but they remain active, even though the imageAnchor is nowhere to be found.

    Do I misunderstand the concept of the TrackingStates or is it simply buggy?
     
    dnnkeeper likes this.
  10. SpiderJones

    SpiderJones

    Joined:
    Mar 29, 2014
    Posts:
    129
    use this to deactivate or hide you AR Objects:

    Code (CSharp):
    1. for (int i = 0; i < eventArgs.removed.Count; i++)
    2.             {
    3.                 // destroy AR object, or set active to false. Use Dictionary.
    4.             }
     
    pwshaw likes this.
  11. SpiderJones

    SpiderJones

    Joined:
    Mar 29, 2014
    Posts:
    129
    TrackingState has three states:
    TrackingState.Limited
    TrackingState.None
    TrackingState.Tracking
     
    pwshaw likes this.
  12. PTW_AR

    PTW_AR

    Joined:
    Jan 13, 2019
    Posts:
    13
    Thanks for the quick reply! I am aware of the three TrackingStates. Even with your attempt, it doesn't want to work for me (my guess is that it's never changing to the trackingState none (or limited)):

    Code (CSharp):
    1. for (int i = 0; i < eventArgs.removed.Count; i++)
    2.         {
    3.             trackedImage = eventArgs.removed[i];
    4.             NamePrefab = trackedImage.referenceImage.name;
    5.             TempPrefab = trackedImage.transform.Find(NamePrefab).gameObject;
    6.             Destroy(TempPrefab);
    7.         }
    8.  
     
  13. SpiderJones

    SpiderJones

    Joined:
    Mar 29, 2014
    Posts:
    129
    The code you shared is not using the "updated" List. You are using the "removed" List. So the "none" state not working is irrelevant, right? I have not tested the "none" state, but I do know that for me the "limited" state does work, and that is why I'm ignoring it, because my ARObjects would do strange things if they were visible and the TrackingState was "limited". Also, using Find is cpu intensive. You should store a local reference to your AR GameObject. Use a Dictionary with the referenceImage.name as the key, and the AR GameObject as the value. and maybe there is an issue with how you are trying to find the AR GameObject, are you using the referenceImage.name as the name for your AR GameObject? When you instantiate a Prefab it adds [clone] (or something like that) to the end of the name. Regardless, I recommend that you do not use this approach, use a Dictionary.
     
  14. SpiderJones

    SpiderJones

    Joined:
    Mar 29, 2014
    Posts:
    129
  15. PTW_AR

    PTW_AR

    Joined:
    Jan 13, 2019
    Posts:
    13
    First of all: Thanks for the idea with the dictionary. For me, this only solved my issue to an extend, because it will only switch to trackingState.limited, if you lay down the device on a table and wait for ~2 seconds (found it out through pure chance with some debug logs). The problem is: only then the prefab.setactive is set to be false. The ARKit directly changes the state directly after losing its ImageAnchor, 2 seconds seem to be quite long (besides the annoying laying down part, which is extremely weird). I assume you didn't have the same experience?

    I created a video to further show my problem:
    https://drive.google.com/open?id=1XP4dkKymXubgTILautDcOvTu1eNBtijf

    The debug log shows the following:
    "we are in the updateCycle with trackingstate Tracking"
    or
    "we are in the updateCycle with trackingstate!= Tracking"

    2nd one only appears after laying down the device. Holding it still without image in the camera view doesnt result in a trackingState change to limited... -.-

    Your app looks very clean, good job!
     
    Last edited: Jun 23, 2019
  16. SpiderJones

    SpiderJones

    Joined:
    Mar 29, 2014
    Posts:
    129
    Thank you for your compliment. I'm sorry, but I don't fully understand your issue. In my app, when users show the camera the image marker (playing card), the AR object shows, when the card is hidden, the object doesn't show. There are 13 AR objects in my app that show for each type of card. My app works like Vuforia (on iOS only because ARKit tracks moving images while ARCore does not). I am using the approach I shared, and I'm ignoring the "Limited" state. When it's "Tracking" I show the ARObject, when it's "None" or "Limited" I hide it. And I use the "remove" list to hide AR Objects. And it works perfectly. I do not know what else I can do to help you. Good luck!
     
    PTW_AR likes this.
  17. brunno159

    brunno159

    Joined:
    Oct 24, 2014
    Posts:
    14
    Code (CSharp):
    1. using System.Collections;
    2. using System.Collections.Generic;
    3. using UnityEngine;
    4. using System;
    5. using UnityEngine.XR.ARSubsystems;
    6. using UnityEngine.XR.ARFoundation;
    7.  
    8. public class ARPerImage : MonoBehaviour
    9. {
    10.    
    11.             Dictionary<string, GameObject> myPrefabsDictionary =
    12.             new Dictionary<string, GameObject>();
    13.            
    14.             List<GameObject> o = new List<GameObject>();
    15.            
    16.     // Start is called before the first frame update
    17.     void Start()
    18.     {
    19.         myPrefabsDictionary.Add("Rafflesia", Resources.Load("Rafflesia") as GameObject);
    20.         myPrefabsDictionary.Add("Logo", Resources.Load("Logo") as GameObject);
    21.         myPrefabsDictionary.Add("QRCode", Resources.Load("QRCode") as GameObject);
    22.     }
    23.  
    24.     // Update is called once per frame
    25.     void Update()
    26.     {
    27.        
    28.     }
    29.    
    30.         private void Awake()
    31.     {
    32.         var arTrackedImageManager = gameObject.GetComponent<ARTrackedImageManager>();
    33.         arTrackedImageManager.trackedImagesChanged += OnTrackedImagesChanged;
    34.     }
    35.    
    36.     private void OnTrackedImagesChanged(ARTrackedImagesChangedEventArgs eventArgs)
    37.     {
    38.         ARTrackedImage trackedImage = null;
    39.         // Check the new tracked images
    40.         for (int i = 0; i < eventArgs.added.Count; i++)
    41.         {
    42.             trackedImage = eventArgs.added[i];
    43.             string imgName = trackedImage.referenceImage.name; // this is the name in the library
    44.             GameObject prefab = myPrefabsDictionary[imgName];
    45.             o.Add( Instantiate(prefab, trackedImage.transform));
    46.         }
    47.        
    48.         for (int i = 0; i < eventArgs.updated.Count; i++)
    49.         {
    50.             trackedImage = eventArgs.updated[i];
    51.             if (trackedImage.trackingState == TrackingState.Tracking)
    52.                 o[i].active=true;
    53.             else
    54.                 o[i].active=false;
    55.                
    56.         }
    57.        
    58.         for (int i = 0; i < eventArgs.removed.Count; i++)
    59.         {
    60.                 o[i].active=false;
    61.                
    62.         }
    63.     }
    64.  
    65. }
    66.  
    can anyone help me figure this out? the object is spawning alright, but it keeps active event if i point to another target.....
     
    Edisyo likes this.
  18. PTW_AR

    PTW_AR

    Joined:
    Jan 13, 2019
    Posts:
    13
  19. BMythes

    BMythes

    Joined:
    Feb 20, 2018
    Posts:
    6

    Hi Bruno,

    I tried running your code on my project (changing the Gameobject names in the Dictionary and such to match my own) but I wasn't able to even spawn any objects at all with my images.

    I have all the Objects I want to spawn inside a folder named Resources (to be able to use the Resources.Load, but I wasn't able to call the Gameobjects and I can't seem to get a console log when running it on my iOS to check for errors..

    I imagine you are using Android but could you give some light to this?

    Thank you!
     
  20. jpvanmuijen

    jpvanmuijen

    Joined:
    Aug 23, 2012
    Posts:
    6
    I've been struggling with this, but managed to stitch something together (for iOS) using the different topics on this forum.
    I've taken a slightly different approach, using a serialized array to store the connections between image targets and prefabs via the Inspector. I'm actually using GameObjects already in the scene instead of instantiating prefabs from Project, because I want changes in the objects (e.g. animations, user interactions) to stay in tact between tracking sessions, so to speak.

    Code (CSharp):
    1.  
    2. using System.Collections;
    3. using System.Collections.Generic;
    4. using UnityEngine;
    5. using UnityEngine.UI;
    6. using UnityEngine.XR.ARSubsystems;
    7. using UnityEngine.XR.ARFoundation;
    8.  
    9. [System.Serializable]
    10. public class MarkerPrefabs
    11. {
    12.     public string marker;
    13.     public GameObject targetPrefab;
    14. }
    15.  
    16. public class SwitchPrefab : MonoBehaviour
    17. {
    18.     /* Insepctor array */
    19.     public MarkerPrefabs[] markerPrefabCombos;
    20.     ARTrackedImageManager m_TrackedImageManager;
    21.  
    22.     void Awake()
    23.     {
    24.         m_TrackedImageManager = GetComponent<ARTrackedImageManager>();
    25.     }
    26.  
    27.     void OnEnable()
    28.     {
    29.         m_TrackedImageManager.trackedImagesChanged += OnTrackedImagesChanged;
    30.     }
    31.  
    32.     void OnDisable()
    33.     {
    34.         m_TrackedImageManager.trackedImagesChanged -= OnTrackedImagesChanged;
    35.     }
    36.  
    37.     private void OnTrackedImagesChanged(ARTrackedImagesChangedEventArgs eventArgs)
    38.     {
    39.         foreach (var trackedImage in eventArgs.updated)
    40.         {
    41.         /* If an image is properly tracked */
    42.         if (trackedImage.trackingState == TrackingState.Tracking) {
    43.              
    44.                 /* Loop through image/prefab-combo array */
    45.                 for(int i = 0; i < markerPrefabCombos.Length; i++)
    46.                 {
    47.                     /* If trackedImage matches an image in the array */
    48.                     if (markerPrefabCombos[i].marker==trackedImage.referenceImage.name) {
    49.  
    50.                         /* Set the corresponding prefab to active at the center of the tracked image */                    
    51.                         markerPrefabCombos[i].targetPrefab.SetActive(true);
    52.                         markerPrefabCombos[i].targetPrefab.transform.position = trackedImage.transform.position;
    53.                     }
    54.                 }            
    55.             /* If not properly tracked */
    56.             } else {
    57.  
    58.                 /* Deactivate all prefabs */
    59.                 for(int i = 0; i < markerPrefabCombos.Length; i++) {
    60.                     markerPrefabCombos[i].targetPrefab.SetActive(false);
    61.                 }
    62.             }
    63.         }
    64.     }
    65.  
    66. }
    67.  
     
    pwshaw likes this.