Search Unity

  1. Unity 2019.2 is now released.
    Dismiss Notice

ARFoundation and 2D image target tracking

Discussion in 'Handheld AR' started by jarileon, Jun 27, 2018.

  1. LorenzoValente

    LorenzoValente

    Joined:
    May 2, 2019
    Posts:
    16
    Thank you Unity team, I started an AR project just yesterday and Image Tracking in AR Foundation is just what I needed.
    Is there any documentation about this feature? I can't find any class or component that I could use. In the changelist you talk about a "Tracked Image Manager" but I couldn't find it
     
    MrLeggo likes this.
  2. LorenzoValente

    LorenzoValente

    Joined:
    May 2, 2019
    Posts:
    16
    Ok I got it: open your "Packages/manifest.json" and make sure it looks like this:

    {
    "dependencies": {
    ...
    "com.unity.xr.arfoundation": "2.1.0-preview.2",
    ...
    },
    "registry": "https://staging-packages.unity.com"
    }
     
    Last edited: May 14, 2019
    MrLeggo likes this.
  3. Trifun

    Trifun

    Joined:
    Feb 23, 2018
    Posts:
    1
    Hello i guys,
    I'd like to know if the image tracking feature works on Arcore disabled devices. In other words, can i build a small app that doesn't require Arcore nor ARKit using AR Foundation ?
     
  4. mende

    mende

    Joined:
    Jun 11, 2013
    Posts:
    28
    Hello everybody,
    So I managed to run the sample for image tracking available in tzhe project from here: https://github.com/Unity-Technologies/arfoundation-samples/tree/master .
    Its working:) but I want to do some changes. I want to reference a different prefab for every image.
    At this time there will only be instantiated the same prefab for every tracked image.
    I have set up a list with different prefabs in the "ARTrackedImageManager.cs" , now I want to have a reference to the images of the "XRReferenceImageLibrary".I could not manage to get these tracked Images by an id or name:(
    How to do this in a simple way ? Any idea?
     
  5. polytropoi

    polytropoi

    Joined:
    Aug 16, 2006
    Posts:
    674
    In TrackedImageInfoManager.cs file, check out the UpdateInfo method - gives you name and guid of detected pic.
     
    mende likes this.
  6. polytropoi

    polytropoi

    Joined:
    Aug 16, 2006
    Posts:
    674
    Image tracking seems to work great so far, congratulations to the ARF team! Any pointers on how best to dynamically create or update an XRReferenceImageLibrary at runtime?
     
    LorenzoValente, Basic50 and mende like this.
  7. Basic50

    Basic50

    Joined:
    Dec 30, 2013
    Posts:
    5
    First, thanks for your work on this new release. We were waiting for this new feature (Image tracking) for a while and are excited to play with. However we notice that it's not possible to update the image references at runtime, all the APIs are in UnityEditor namespace. Did we miss something or it's not available yet? And in that case, do you plan to work on it?
     
    Last edited: May 20, 2019
    LorenzoValente, polytropoi and Blarp like this.
  8. Blarp

    Blarp

    Joined:
    May 13, 2014
    Posts:
    166
    This technique will be real handy for choosing a point cloud map to localize against. Map IDs are set to Image Targets that are generated through screenshot (read pixels). So when the image target is recognized, we can programatically set which map to localize against.

    The problem that solves, is you cant iterate between point cloud maps until they match and localize, you need to specify the exact map you want to localize against to start the session.

    Image targetting bridges that cap between what you are looking at to what map to load.
     
  9. Skeketor23

    Skeketor23

    Joined:
    Apr 6, 2019
    Posts:
    8
    Check the function ARTrackableManager::CreateGameObject(). There the Prefab is loaded. You need to adjust the functions in order to pass the name to the ARTracedImageManger::GetPrefab()).
    Then you can choose the Prefab by the Name/ID.

    Update: You will need to embed the ARFoundation package into your project. Otherwise changes to the ARTrackedManager classes cannot be stored permanently:
    https://forum.unity.com/threads/is-it-possible-to-edit-tmp-unity-package-source-code.571420/

    For example:
    upload_2019-5-18_21-52-28.png

    And you need another overload of GetPrefab:
    upload_2019-5-18_21-53-53.png

    And then you can choose the right prefab by its name here:
    upload_2019-5-18_21-55-4.png
     
    Last edited: May 19, 2019
    mende and MrLeggo like this.
  10. Skeketor23

    Skeketor23

    Joined:
    Apr 6, 2019
    Posts:
    8
    Can anybody explain what exactly happens once I scan an image and a game object is instantiated at that position?
    I have some trouble with the alignment of the gameobjects.
    I mean, how is the gameobject internally translated to the anchor position? How is it rotated? Does it make use of ARSession::MakeContentAppearAt function?
    Can anybody give some more background about how this happens?
     
  11. polytropoi

    polytropoi

    Joined:
    Aug 16, 2006
    Posts:
    674
    Skeketor23 likes this.
  12. LorenzoValente

    LorenzoValente

    Joined:
    May 2, 2019
    Posts:
    16
    I'm also struggling with the dynamic update of the XRReferenceImageLibrary. I've no solution yet, I posted my experience in this thread.
     
    polytropoi likes this.
  13. loganfive

    loganfive

    Joined:
    Nov 10, 2016
    Posts:
    1
    nothing new about 2d image tracking?
     
  14. SpiderJones

    SpiderJones

    Joined:
    Mar 29, 2014
    Posts:
    127
  15. MrLeggo

    MrLeggo

    Joined:
    Feb 16, 2018
    Posts:
    17
    So I have the basics working but have some weird scaling issues. For instance if I place a standard cube under the base prefab, it seems to scale the y up, so the cube is now very long!

    I can't figure where or why this is happening. It seems to related to the tracked image size.

     
    Last edited: Jun 7, 2019
  16. SpiderJones

    SpiderJones

    Joined:
    Mar 29, 2014
    Posts:
    127
    I suggest you look here for help -> https://github.com/Unity-Technologies/arfoundation-samples/issues
     
    MrLeggo likes this.
  17. MrLeggo

    MrLeggo

    Joined:
    Feb 16, 2018
    Posts:
    17
  18. dg3

    dg3

    Joined:
    Sep 20, 2016
    Posts:
    3
    Did the same test and having the same problem. Did you get anywhere to resolving? I saw someone else having the same problem here https://github.com/Unity-Technologies/arfoundation-samples/issues/142, but their solution didn't work for me. Thanks.
     
    MrLeggo likes this.
  19. MrLeggo

    MrLeggo

    Joined:
    Feb 16, 2018
    Posts:
    17
    I did the same in that link and added a public vector 3, so I could adjust the values as I needed. It stopped the plane from being the same size as the image, but the objects did become the correct scale. For a while...

    Now it has sopped working again. Not sure what I changed! I need to dive in and figure it out. I have some more pressing projects, so not sure when I will be able to look again.

    I'm sure more people must be having this same issue?
     
  20. NicklasAugsos

    NicklasAugsos

    Joined:
    Sep 26, 2018
    Posts:
    1
    Hi all.
    I am looking through the code of TrackedImageInfoManager and ARTrackedImageManager in ARFoundations Image Tracking.

    My problem is that i am trying to recieve the same functionality as in vuforia image tracking. (when camera is on the target = show content, when camera is off the target = dont show content) when using the demo scene from arfoundation the content stays on the screen instead of being disabled, I am stil fairly new to coding so i have a big problem understanding what is happening, i played around with this standard code from the TrackedImageInfoManager script:
    //Disable the visual plane if it is not being tracked
    if (trackedImage.trackingState != TrackingState.None)
    {
    planeGo.SetActive(true);

    // The image extents is only valid when the image is being tracked
    trackedImage.transform.localScale = new Vector3(trackedImage.size.x, 1f, trackedImage.size.y);

    //// Set the texture
    var material = planeGo.GetComponentInChildren<MeshRenderer>().material;
    material.mainTexture = (trackedImage.referenceImage.texture == null) ? defaultTexture : trackedImage.referenceImage.texture;
    }
    else
    {
    planeGo.SetActive(false);
    }

    My problem is that even when the trackingstate is None - the content is still showing..what can i do to make the stuff disapear, and only show when i scan my images?

    please help, have been stuck for too long :D
     
    LorenzoValente likes this.
  21. StarCoop

    StarCoop

    Joined:
    Nov 26, 2016
    Posts:
    16
    +1 Bump this question!
    Even with half a day of research, there is no clear answer to be found in the Unity pages.
    Anybody can confirm or deny if ARCore is necessary for Image Target detection?
     
  22. polytropoi

    polytropoi

    Joined:
    Aug 16, 2006
    Posts:
    674
    ar foundation requires arcore or arkit support, respectively, for android or ios. If you want to target older devices, stick with vuforia, or opencv, or 8thwall, etc... It would be cool for Unity to implement or integrate their own platform-independent SLAM kind of functionality, but that's not part of ar foundation right now afaik.
     
  23. SpiderJones

    SpiderJones

    Joined:
    Mar 29, 2014
    Posts:
    127
    I believe polytropoi is correct. Also, you can ask questions here -> https://github.com/Unity-Technologies/arfoundation-samples/issues
     
  24. HidingGlass

    HidingGlass

    Joined:
    Apr 29, 2015
    Posts:
    17
    Has anyone had any luck with any kind of Remote (phone-to-unity) or being able to test ARFoundation functionality in-editor? I'm having some issues with Image Targets on build that I can't seem to replicate in editor. It works as intended the first go-around but after restarting the ARSession my scripts are having trouble initializing.

    My guess is it's to do with my understanding of how ARFoundation is creating and destroying ImageTargets but I can't see what's happening unless I DebugLog my entire hierarchy.
     
    Last edited: Jun 28, 2019
    Alexis-Dev likes this.
  25. LucasRizzotto

    LucasRizzotto

    Joined:
    Dec 11, 2015
    Posts:
    8
    Having the same issues as well. Even with all default settings, the image tracking system seems to distort the Y values of any prefab I give it.
     
  26. LMVRINFO

    LMVRINFO

    Joined:
    Oct 17, 2018
    Posts:
    1
    Any update on image tracking for AR Foundation?
     
  27. blackhawk_001

    blackhawk_001

    Joined:
    Jan 25, 2019
    Posts:
    7
    do you manage to make prefab world scale? my project requirement are 2meter image size and prefab spawn same size like image
     
  28. orgelvarg

    orgelvarg

    Joined:
    May 27, 2019
    Posts:
    6
    Any ideas on when it will be added for the android users of ARFoundation?
     
  29. SpiderJones

    SpiderJones

    Joined:
    Mar 29, 2014
    Posts:
    127
    ARFoundation does support image tracking for iOS and Android. I have an app in both stores. I'll share some images soon so you can see the tracking working.
     
  30. adrian-taylor09

    adrian-taylor09

    Joined:
    Dec 22, 2016
    Posts:
    11
    I noticed in the blog post about AR Foundation and ARKit 3.0 that they mentioned improved image tracking:

    Both image tracking and object detection features include significant accuracy and performance improvements

    Just wondering if anyone has noticed any improvements to the Image tracking at all? My observations of the image tracking in ARKit and ARFoundation apps is not as stable and reliable as in Vuforia apps on the same hardware

     
  31. orgelvarg

    orgelvarg

    Joined:
    May 27, 2019
    Posts:
    6
    Ok, @SpiderJones please do. Would be cool to see an example. What is the name of the app?
     
  32. SpiderJones

    SpiderJones

    Joined:
    Mar 29, 2014
    Posts:
    127
    Voronoi and MrLeggo like this.
  33. Ga2z

    Ga2z

    Joined:
    Jul 27, 2019
    Posts:
    1
    Hi, I wanted to test it but it says that my device is not compatible. I have a Samsung Galaxy S8, is up to date and ARcore works for other apps (including some ARFoundation testings I've been doing)
     
  34. SpiderJones

    SpiderJones

    Joined:
    Mar 29, 2014
    Posts:
    127
  35. Voronoi

    Voronoi

    Joined:
    Jul 2, 2012
    Posts:
    250
    Thanks for posting these example apps. I am about to build out a cross-platform AR app and would really like to use the image tracking to place the object more precisely in a room. Your apps conveniently have the 'card-mode' so that I can test how tracking works on a plane versus on an image, using an iPhone 7 plus.

    Your app shows what I have also found – plane tracking is much smoother and more steady when compared to the image tracking. I don't think this is ARFoundation, but the way image tracking is implemented, based on this statement from Apple's documentation :

    My images will be fixed in place, so I don't really need the functionality of tracking a moving image, but I would like the 3D object to lock tightly and smoothly onto the image. Trying to get my head around the best way to approach this, is it possible to use the image tracking to create a plane where the image is oriented? So that, the tracking is smoother like when using planes?
     
  36. SpiderJones

    SpiderJones

    Joined:
    Mar 29, 2014
    Posts:
    127
    @Voronoi That's a good question. I'm working on another app where image tracking is used to start the AR planar experience (Image tracking is used to trigger a message asking the user to point their phone at a surface, then image tracking scene unloads and planar scene loads). But I am curious to see if I could do as you describe, use the position of the image to place the object and then use planar AR to keep it in place, and turn off image tracking. I've posted a question here (feel free to contribute to the question) -> https://github.com/Unity-Technologies/arfoundation-samples/issues/233
     
  37. Voronoi

    Voronoi

    Joined:
    Jul 2, 2012
    Posts:
    250
    Thanks, I added my thoughts to the Git discussion. It seems to me that image tracking should have an option to hand off the image plane to plane tracking if the number of moving image targets is set to zero. Meaning, it's assumed the image is not moving, so once the marker is detected, a plane could take it's place and allow the object to sit cleanly on the plane. If the person moves, the plane tracking should update, and it seems that plane tracking is much more stable.
     
    SpiderJones likes this.
  38. yogi_kenobi

    yogi_kenobi

    Joined:
    May 24, 2017
    Posts:
    9
    This is exactly behaviour and technique I've been looking around lately how to achieve (AR Foundation/wikitude/Vuforia/anything). Super interested to hear if you or anybody finds working work flow for this.
     
  39. yogi_kenobi

    yogi_kenobi

    Joined:
    May 24, 2017
    Posts:
    9
    Aaand I should have followed the link @SpiderJones provided before posting. This github comment and this seems to provide good starting point.
     
  40. Voronoi

    Voronoi

    Joined:
    Jul 2, 2012
    Posts:
    250
    I have been trying out the ideas from Git and I was able to get the behavior that I wanted by creating a new Reference Point at the center of the tracked image. One thing that was throwing off my tests is that you need to quit the app on your phone, or you get drifting because it is resuming the old AR session I think.

    The script below will allow you to place an object at the center of the tracked image, it then stops looking for new images. In my tests, it appears to work as well as using planes, the object more or less stays where it was placed.

    Code (CSharp):
    1. //Script to place an object at the center of a tracked image. Once an image is tracked, we wait a bit before adding a Reference Point
    2. //so that it is more likely to be placed accurately.
    3. //Instructions:
    4. //
    5. //1. Add this script to a 'registration' prefab that is used in the ARTrackedImageManager from the ARFoundation samples
    6. //2. The ARTrackedImageManager is usually on the AR Session Origin gameobject in the scene.
    7. //3. Add the ARReferencePointManager script to the AR Session Origina gameobject as well.
    8. //4. Put the prefab that you want to stick to the image on the ARReferencePointManager Reference Point Prefab
    9. //
    10. //After an image is found, ARTrackedImageManager creates the 'registration' prefab - I used a cube that is the same size as the image.
    11. //The 'registration' prefab has this script and starts to count each time the image tracking is updated. Once delayCount is reached,
    12. //a new ReferencePoint is added.  The ARReferencePointManager takes care of instantiating the object. The ARTrackedImageManager is
    13. //then disabled, which also deletes the 'registration' prefab (I believe).
    14.  
    15. using System.Collections;
    16. using System.Collections.Generic;
    17. using UnityEngine;
    18. using UnityEngine.XR.ARFoundation;
    19.  
    20. public class StickToImage : MonoBehaviour
    21. {
    22.     public int delayCount = 200;
    23.  
    24.     ARTrackedImageManager trackedImageManager;
    25.     ARReferencePointManager referencePointManager;
    26.  
    27.     int count = 0;
    28.  
    29.     void Awake()
    30.     {
    31.         trackedImageManager = FindObjectOfType<ARTrackedImageManager>();
    32.         referencePointManager = FindObjectOfType<ARReferencePointManager>();
    33.     }
    34.  
    35.     void OnEnable()
    36.     {
    37.         trackedImageManager.trackedImagesChanged += OnTrackedImagesChanged;
    38.     }
    39.  
    40.     void OnDisable()
    41.     {
    42.         trackedImageManager.trackedImagesChanged -= OnTrackedImagesChanged;
    43.     }
    44.  
    45.     void UpdateTrackedImage(ARTrackedImage trackedImage)
    46.     {
    47.         count++;
    48.  
    49.         if (count >= delayCount)
    50.         {
    51.             //Make sure your desired prefab is the m_ReferencePointPrefab on the ARReferencePointManager in the Editor
    52.             referencePointManager.AddReferencePoint(new Pose(trackedImage.transform.position, trackedImage.transform.rotation));
    53.             Debug.Log("Placed Object at " + Time.time);
    54.             trackedImageManager.enabled = false;
    55.             Destroy(this);
    56.         }
    57.     }
    58.  
    59.     void OnTrackedImagesChanged(ARTrackedImagesChangedEventArgs eventArgs)
    60.     {
    61.         foreach (var trackedImage in eventArgs.added)
    62.             UpdateTrackedImage(trackedImage);
    63.  
    64.         foreach (var trackedImage in eventArgs.updated)
    65.             UpdateTrackedImage(trackedImage);
    66.     }
    67. }
    68.  
     
    Last edited: Aug 2, 2019
    SpiderJones likes this.
  41. SpiderJones

    SpiderJones

    Joined:
    Mar 29, 2014
    Posts:
    127
    Great work Unity community!
     
  42. vincentfretin

    vincentfretin

    Joined:
    Jul 4, 2019
    Posts:
    3
    @Voronoi I think you should remove the previous reference point before adding a new reference point in your UpdateTrackedImage, otherwise tracked referent points will increase and it may hurt performance?
     
  43. HidingGlass

    HidingGlass

    Joined:
    Apr 29, 2015
    Posts:
    17
    Does anyone have a method for reliably telling when an image target has started/stopped tracking? We are doing some functionality for floating the augmentation to the users screen when they look away from the target.

    Checking for TrackingState.Limited or TrackingState.None works great on iOS but neither event gets triggered in ARTrackedImageManager.trackedImagesChanged on Android unless you cover the camera with your hand.

    Basically, the following code works on iOS and not Android. As a workaround we are starting a timer whenever tracking state stops sending .Tracking but we would like to cut out any "#if ANDROID" code when possible.

    Thanks!

    Code (CSharp):
    1. private void OnTrackedImagesChanged(ARTrackedImagesChangedEventArgs eventArgs)
    2.     {
    3.         foreach (var trackedImage in eventArgs.updated)
    4.         {
    5.             //if the updated image:
    6.             //has the same name as triggerMod
    7.             //and the tracking state is 'tracking' or 'limited'
    8.             if (trackedImage.referenceImage.name == triggerMod.triggerName && (trackedImage.trackingState == TrackingState.Tracking))
    9.             {
    10.                 myTrackedImage = trackedImage;
    11.                 ImageFound(triggerMod.triggerName);
    12.             }
    13.  
    14.             //if the updated image:
    15.             //has the same name as triggerMod
    16.             //and the tracking state is 'none'
    17.             if (trackedImage.referenceImage.name == triggerMod.triggerName && (trackedImage.trackingState == TrackingState.Limited || trackedImage.trackingState == TrackingState.None))
    18.             {
    19.                 ImageLost("~event~" + triggerMod.triggerName);
    20.             }
    21.         }
    22.     }
    23.  
     
  44. SpiderJones

    SpiderJones

    Joined:
    Mar 29, 2014
    Posts:
    127
  45. HidingGlass

    HidingGlass

    Joined:
    Apr 29, 2015
    Posts:
    17
    Awesome. Thanks for putting in that legwork! I had been staying away from 2.2.0 because it was using a preview version of ARKit but maybe that's a non-issue now. You were able to submit your CRISPR app with 2.2.0? (Cool subject matter by the way).
     
  46. SpiderJones

    SpiderJones

    Joined:
    Mar 29, 2014
    Posts:
    127
    You just need to update the ARCore XR plugin, it doesn't update ARKit. CRISPR-3D uses: ARCore XR 2.2.0 Preview 2, AR Foundation 2.1.1, and ARKit XR Plugin 2.1.1. There's a release in both stores (Google Play and Apple App store). It's not using the preview version of ARKit (ARKit XR 2.2.0).

    Thank you for your kind words about CRISPR-3D
     
    HidingGlass likes this.