Search Unity

  1. Unity 2019.2 is now released.
    Dismiss Notice

ARKit support for iOS via Unity-ARKit-Plugin

Discussion in 'ARKit' started by jimmya, Jun 5, 2017.

  1. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
  2. domdev

    domdev

    Joined:
    Feb 2, 2015
    Posts:
    288
  3. DanTaylor

    DanTaylor

    Joined:
    Jul 6, 2013
    Posts:
    72
    Hi there.
    I have a quick, and probably fairly dumb, question.

    The AR in my app works fine... except on iPhone X where it has mirrored borders at the edge of the screen like this:



    I am guessing that all I need to do is update the ARKit plug-in?
    Would somebody be able to confirm this before I start messing with something that already works pretty well on all other devices? :)

    Cheers,
    Dan
     
    yuliwei likes this.
  4. Aiursrage2k

    Aiursrage2k

    Joined:
    Nov 1, 2009
    Posts:
    4,822
    does ar kit work with unity 2017?
     
  5. ph_

    ph_

    Joined:
    Sep 5, 2013
    Posts:
    143
  6. Aiursrage2k

    Aiursrage2k

    Joined:
    Nov 1, 2009
    Posts:
    4,822
    Okay the problem I guess is that I am trying to run arkit and yuforia. Does anyone know if they work together?
     
  7. HaimBendanan

    HaimBendanan

    Joined:
    May 10, 2016
    Posts:
    26
    When I choose the build architecture in xcode, can i set it to arm64 only? Until now I used "$(ARCHS_STANDARD)" but I am wondering if using arm64 would make my build smaller while keeping the support for all supported devices
     
  8. domdev

    domdev

    Joined:
    Feb 2, 2015
    Posts:
    288
    how to check if device dont support ARkit?
     
  9. suxinren

    suxinren

    Joined:
    Jan 13, 2017
    Posts:
    14
    yes,I use unity 2017.2.0f3
     
  10. suxinren

    suxinren

    Joined:
    Jan 13, 2017
    Posts:
    14
    iOS device that supports ARKit(iPhone 6s or later,iPad(2017) or later)
     
  11. tomtong

    tomtong

    Joined:
    Aug 5, 2013
    Posts:
    14
    Code (CSharp):
    1. extern "C" UnityARUserAnchorData SessionAddUserAnchor(void* nativeSession, UnityARUserAnchorData anchorData)
    2. {
    3.     // create a native ARAnchor and add it to the session
    4.     // then return the data back to the user that they will
    5.     // need in case they want to remove it
    6.     UnityARSession* session = (__bridge UnityARSession*)nativeSession;
    7.     ARAnchor *newAnchor = [[ARAnchor alloc] initWithTransform:matrix_identity_float4x4];
    8.    
    9.     [session->_session addAnchor:newAnchor];
    10.     UnityARUserAnchorData returnAnchorData;
    11.     UnityARUserAnchorDataFromARAnchorPtr(returnAnchorData, newAnchor);
    12.     return returnAnchorData;
    13. }
    14.  
    Hello. I find that anchorData has no use in adding SessionAddUserAnchor method. Do I miss something? Thx.
     
  12. CManzione

    CManzione

    Joined:
    May 11, 2015
    Posts:
    7

    Was this ever resolved? I have the same issue.
     
  13. planetmatrix

    planetmatrix

    Joined:
    Dec 8, 2015
    Posts:
    36
    Very happy to share, Sahi Labs shortly coming on ARKit,
    All Thanks to the great efforts by Team Unity for creating the plugin, it made it very easy.
     
    jimmya likes this.
  14. mikenowlin

    mikenowlin

    Joined:
    Jan 11, 2017
    Posts:
    3
    Hello, everybody! Just wanted to share what I created last weekend at the VR Austin Jam using the new face tracking APIs in the Unity ARKit Plugin for the iPhone X. The "floating grey mask" from the example scene reminded me of Andross from Starfox (SNES) so I went about recreating that boss battle with the user attempting to defeat Starfox. Open your mouth to fire and close your eyes to turn invisible.

    You can download the Xcode project from my itch.io page if you want to build it out and give it a go.

     
  15. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    If you choose to target ios 11, you should choose ARM64 instead of Universal architecture in your PlayerSettings.
     
    HaimBendanan likes this.
  16. sam598

    sam598

    Joined:
    Sep 21, 2014
    Posts:
    49
    You have to download the latest beta of 2017.3, use the native Vuforia integration, and set Device Tracking to positional and set the Vuforia camera pose to Device. You only need to use the Vuforia 7 plugin, not the ARKit plugin. Think of it more like Vuforia is using ARKit, instead of both are working at the same time.

    Be aware this is not for the faint of heart. It's buggy, slow, doesn't appear to allow the planar or pointcloud features of ARKit, and not well documented yet (very few of Vuforia's newer features are).

    But it is pretty awesome.
     
    Burglecut likes this.
  17. apelab

    apelab

    Joined:
    Apr 22, 2013
    Posts:
    1
    Hi There. I am using the IphoneX blendshape capabilities.An experimenting to have the TrueDepth camera for retrieving expressions with the blendshapes but I want to show the front camera for normal arkit usage and place assets. Is it possible to have both activated at the same time and working together ? I tried but it seems like when I launch the front facing arkit camera by using the plugin, the blendshape are deactivated. Thanks a lot for your help.
     
  18. Aiursrage2k

    Aiursrage2k

    Joined:
    Nov 1, 2009
    Posts:
    4,822
    When I was trying to use xcode + vufuria + arkit it was giving an error about UIView render. What we wanted to do was have ARkit project on the surface unless vuforia actually found the image target. I dont know why they cant work together especially now that unity has actually intragrated it into its own editor
     
  19. sam598

    sam598

    Joined:
    Sep 21, 2014
    Posts:
    49
    They can't work together because both Vuforia 6 and ARKit require complete control of the camera.

    Vuforia's solution (I'm guessing) is to run ARKit inside of it's own code, then do their planar tracking on top of ARKit's video feed and camera intrinsics.

    If you run the early beta of Vuforia 7, it uses some of ARKit for extended tracking. But doing it this way will require Vuforia to write additional wrapper functions for ARKit features like pointcloud and floor plane detection.
     
  20. RyanYN

    RyanYN

    Joined:
    Aug 13, 2013
    Posts:
    17
    iOS11.1 + Unity2017.1 + Vuforia6.2 + ARKit1.0.11
    i have 3 scenes:
    a. main scene: two button to switch arkit and vuforia;
    b. vuforia scene(can go back to a)
    c. arkit scene(can go back to a)
    (y: the camera works fine. n:the camera doesn't work.)
    1. a -> b(y) -> a -> c(y) -> a -> c(y) -> a -> b(y) -> a -> c(n)
    2. a -> c(y) -> a -> b(y) -> a -> c(n)

    the camera doesn't work after c->b->c, i've no idea whether vuforia or arkit cause the problem.

    any one help, thanks!
     
    Last edited: Nov 19, 2017
  21. NumesSanguis

    NumesSanguis

    Joined:
    Nov 9, 2017
    Posts:
    6
    How is your progress on this example? I would love to see how to use the blendshape coefficients to animate a head in Unity. Hence, I also made this question: https://forum.unity.com/threads/ark...hape-facegeometry-to-modify-face-mesh.504789/
     
    Last edited: Nov 20, 2017
  22. arturmandas

    arturmandas

    Joined:
    Sep 29, 2012
    Posts:
    197
    Hi Dan,
    I have "white bars" issue. Did the update work for you?

    Best!
     
  23. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
  24. Aiursrage2k

    Aiursrage2k

    Joined:
    Nov 1, 2009
    Posts:
    4,822
    Okay do you know how they did that?
     
  25. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    Not too sure what specifically you're talking about. Seems like a standard ARKit app.
     
  26. suxinren

    suxinren

    Joined:
    Jan 13, 2017
    Posts:
    14
    Hi jimmya @jimmya, Are there any arkit learning materials or documentation?
     
  27. hawken

    hawken

    Joined:
    Aug 22, 2013
    Posts:
    488
    does blackscreen on the iPhone 6 mean it's not supporting the 6? (just running the standard asset as vanilla)
     
  28. suxinren

    suxinren

    Joined:
    Jan 13, 2017
    Posts:
    14
    iOS device that supports ARKit(iPhone 6S or later,iPad(2017) or later)
     
  29. pojoih

    pojoih

    Joined:
    Mar 30, 2013
    Posts:
    176
    1) I only have one directional light and the ARLightmanager to adjust it and the light probes and the holograms seem to be off still :(. It looks like the light gets adjusted to the conditions I see with my eyes and not the exposure corrected image from the camera on the screen.
    2) Nope, its an IPad Pro. But seems I compared the image to the wrong camera mode. Photo mode from stock camera app looks much better but slow motion video has the same preview quality as the unity app so nothing to worry about.
     

    Attached Files:

  30. DanTaylor

    DanTaylor

    Joined:
    Jul 6, 2013
    Posts:
    72
  31. DanTaylor

    DanTaylor

    Joined:
    Jul 6, 2013
    Posts:
    72
    Hi there.
    I updated the ARKit Plugin to the latest version, as part of an iPhone X update for Tsuro. AR seems to work OK on my test device with one strange error.

    If I pause the session...

    Code (CSharp):
    1. m_session = UnityARSessionNativeInterface.GetARSessionNativeInterface();
    2. m_session.Pause();
    ...and then return to it using...

    Code (CSharp):
    1. m_session = UnityARSessionNativeInterface.GetARSessionNativeInterface();
    2. Application.targetFrameRate = 60;
    3. ARKitWorldTrackingSessionConfiguration config = new ARKitWorldTrackingSessionConfiguration();
    4. config.planeDetection = UnityARPlaneDetection.Horizontal;
    5. config.alignment = UnityARAlignment.UnityARAlignmentGravity;
    6. config.getPointCloudData = true;
    7. config.enableLightEstimation = true;
    8. m_session.RunWithConfigAndOptions (config, UnityARSessionRunOption.ARSessionRunOptionRemoveExistingAnchors | UnityARSessionRunOption.ARSessionRunOptionResetTracking);
    ...the framerate from the camera drops so much that it's unusable.
    If I pause and restart a second time, the framerate is so low that the image is stationary.

    Did I miss some kind of change in the code that might cause this? Or is there a bug?
    Is anyone else seeing this?

    Any tips would be most welcome, thanks!
     
  32. 4wall

    4wall

    Joined:
    Sep 16, 2014
    Posts:
    69
    Anyone else experiencing a "Screen position out of view frustum" error?
     
  33. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    This happens if you are trying to run in editor with a camera that has a view frustum or projection matrix that is invalid. To prevent it, see what is done in UnityARCameraManager on startup - look for " //put some defaults so that it doesnt complain"
     
  34. pluszak

    pluszak

    Joined:
    Jul 17, 2016
    Posts:
    2
    Hello, im trying to improve the spawning system to make it better than arkit default generate planes,
    and i was wondering how to do this :



    This is from Magnolia Application.

    is there any tutorial / topic that can guide me ?

    Any tips would be most welcome, thanks!
     
  35. re-cheid

    re-cheid

    Joined:
    Apr 10, 2017
    Posts:
    9
    I have basically the same problem right now.
    Instead of Vuforia, i have a scene which utilizes the standard WebcamTexture.

    It seems like the following scenario causes the problem:
    ARKit (works) > any other usage of the Camera (works) > ARKit (broken)


    I understand that ARKit takes over the camera, but is there a proper way to free the camera again and reinitialize ARKit at a later point?
    The problem definitely lies in not freeing the hardware resource from ARKit after usage. Using WebcamTexture first and then ARKit does not cause any problems.

    // EDIT:
    @Ryan2015 I got it working now!
    I changed the following in UnityARCameraManager.cs:

    Changed Start() to OnEnable(), since start is only called once.
    In that Method changed m_session.RunWithConfig(...) to
    Code (CSharp):
    1. m_session.RunWithConfigAndOptions (config, UnityARSessionRunOption.ARSessionRunOptionResetTracking);
    and added the following OnDisable:
    Code (CSharp):
    1. private void OnDisable()
    2. {
    3.     if (m_session != null)
    4.     {
    5.         m_session.Pause();
    6.     }
    7. }
    //UPDATE:
    Still had a minor aspect ratio problem.
    Changed UnityARVideo:
    Replaced OnDestroy by:
    Code (CSharp):
    1. void OnDisable()
    2. {
    3.     if (m_VideoCommandBuffer != null)
    4.     {
    5.         m_Camera.RemoveCommandBuffer (CameraEvent.BeforeForwardOpaque, m_VideoCommandBuffer);
    6.         m_VideoCommandBuffer.Clear();
    7.         m_VideoCommandBuffer = null;
    8.         UnityARSessionNativeInterface.ARFrameUpdatedEvent -= UpdateFrame;
    9.         bCommandBufferInitialized = false;
    10.     }
    11. }
    and changed Start() to OnEnable()

     
    Last edited: Nov 23, 2017
    RyanYN and jimmya like this.
  36. jason_sone

    jason_sone

    Joined:
    Jul 7, 2017
    Posts:
    8
    I'm working on an original iPad Pro 12.9" (model ML0Q2LL/A) with 128GB of RAM, running iOS 11.2 Beta (15C5110b). I've built several ARKit capable test apps in Xcode and Unity and I've tested my scenes in all kinds of environments (sunny outdoors, flourescent indoors, moderately lit indoors), and I cannot get the plane detection to lock in quite like I see in demo apps around the internet. Is there something I can do in the code or on my iPad to improve plane detection and get my objects to "stick" to the ground better than what I'm getting? Most of the time, my objects are jumping all over the screen, and other times, no matter what I do or how long I let the iPad detect the room, it won't even detect a horizontal plane for me to place objects. I understand that ARKit is in it's early stages and will only improve with time, but based on what I'm seeing other developers create, it seems like I'm missing something.

    Any help would be most appreciated.
     
  37. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    Look at FocusSquare example maybe?
     
  38. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    Have a look at our Unite Austin talk for some pointers:
     
  39. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    We're now supporting this with the addition of a settings file and an option therein:

    The settings file for doing this is located at "Assets/UnityARKitPlugin/Resources/UnityARKiPlugin/ARKitSettings.asset"


    Selecting that file will allow you to select in the Inspector whether ARKit is required in your app:



    If the ARKitSettings asset is missing, you can create a new one from the "Assets/Create/UnityARKitPlugin/Settings" menu:


    Thanks to user Patrik Buckau on BitBucket for the PR.
     
    Last edited: Nov 25, 2017
    roddles and daniilkorotin like this.
  40. hwhsr

    hwhsr

    Joined:
    Jul 30, 2017
    Posts:
    12
    @jimmya Not to interrupt the thread, but I’m hoping you can help me. I need to create user touch input for my ar app in Unity. Could you point me to a tutorial for this or give a quick general explanation to help me get started? I need the user to be able to touch my game object in their phone and interact with it. Any help would be greatly appreciated.
     
  41. LondonStevey

    LondonStevey

    Joined:
    Jun 26, 2015
    Posts:
    33
    Hey, I've just release a little ArKit app to the AppStore called PaPe AR:
    If you've got time, download and let me know what you think.
    Maybe even give it some stars if you like! ;)

    https://itunes.apple.com/gb/app/pape-ar/id1312657639?mt=8
    Its the usual placing, resizing type app, but with a few moving animatable objects.

    Thanks so much Unity.. I'm loving the plugin!
    One thing I found useful was creating a grid button, so I could switch between the debug and shadow material. So for general release you can see the shadows, but if you ever want to see whats been detected it just replaces the plane materials with the debug material.
    Happy ARing everyone!
     

    Attached Files:

    jimmya and re-cheid like this.
  42. domdev

    domdev

    Joined:
    Feb 2, 2015
    Posts:
    288
  43. hwhsr

    hwhsr

    Joined:
    Jul 30, 2017
    Posts:
    12
    When I build out my project and test it on my iPhone 7s, the model appears and animation works, but the screen is now blue. This has never happened. I have all of the latest software, hardware, etc...The camera feed seems to have an issue. Anyone ever had this happen before? I don't have a clue on how to fix this. Thanks for reading.
     
  44. a_d_69

    a_d_69

    Joined:
    Jun 7, 2017
    Posts:
    8
    Formacar app coming on ARKit!
    Thanks Unity Team for plugin!
     
  45. phits

    phits

    Joined:
    Aug 31, 2010
    Posts:
    41
    I had the same issue and updated the AKKit plugin and that solve the problem.
     
    DanTaylor likes this.
  46. Crashthatch

    Crashthatch

    Joined:
    Sep 25, 2017
    Posts:
    8
    UnityARKitPlugin\ARKitRemote\ObjectSerializationExtension.cs and UnityARKitPlugin\ARKitRemote\SerializableObjects.cs define a top-level "Utils" namespace. This is bad practice as it causes other code to prefer it over the Utils package in their own included namespace. In my case, it breaks OpenCVForUnity because Utils.MyMethod() now refers to the global Utils namespace rather than OpenCVForUnity.Utils class, breaking many many files.

    Reported error in Unity console is:
    Assets/OpenCVForUnity/Examples/Advanced/ComicFilterExample/ComicFilterExample.cs(196,23): error CS0234: The type or namespace name `matToTexture2D' does not exist in the namespace `Utils'. Are you missing an assembly reference?

    Neither of these packages are my own, but it seems like ARKit plugin defining a top-level namespace named "Utils" is being very presumptive (Who's to say .NET won't introduce a Utils namespace in a later version).

    Sorry for posting in whatever thread this is, but your bitbucket doesn't seem to have an "issues" tab and I don't really know where else to post.
     
  47. Gametyme

    Gametyme

    Joined:
    May 7, 2014
    Posts:
    558
    Hello, Im having trouble spawning my go on the generated plate automatically. The positions appears to be correct with a 3.2 unit offset on the y axis but the rotation is wrong. Ive tried a few things but heres my latest code.

    Code (CSharp):
    1. public void SetScene()
    2.         {
    3.             List<ARPlaneAnchorGameObject> arpags = unityARAnchorManager.GetCurrentPlaneAnchors ();
    4.             if (arpags.Count >= 1) {
    5.                 ARPlaneAnchor ap = arpags [0].planeAnchor;
    6.  
    7.                 Instantiate (testObject, Vector3.zero, Quaternion.identity);
    8.  
    9.                 // From UnityARUtility Class
    10.                 testObject.transform.position = UnityARMatrixOps.GetPosition (ap.transform);
    11.                 testObject.transform.rotation = UnityARMatrixOps.GetRotation (ap.transform);
    12.  
    13.                 Debug.Log ("Setting Scene");
    14.             }
    15.         }
     
  48. mikeyt0078

    mikeyt0078

    Joined:
    Nov 20, 2017
    Posts:
    8
    Hi all,
    Just wondering if anyone could point me in the right direction.

    I am running ARkit Remote and I have an image and plane detection in the editor within unity when connected to an iPhone. However, I have no image on the iPhone in the app. Is this normal? It doesn't seem to be when I have looked at the example videos for setup etc?

    I have created the app with the UnityArKitRemote scene, which seems to be correct from other forum posts.

    Any ideas?

    Thanks.
     
    Last edited: Nov 26, 2017
  49. Burglecut

    Burglecut

    Joined:
    Jun 27, 2015
    Posts:
    8
    I think he is wondering how they do the mapping? They either use some kind of marker or they place the Campus manually. Maybe the have some kind of recenter-mechanism?
     
  50. km30

    km30

    Joined:
    Jun 28, 2017
    Posts:
    5
    Hello,

    I am trying to find out how I can periodically "reset" the coordinate system. If I change the direction and position of the camera via some other means (say, a teleportation game mechanic), how could I then reset ARkit so that it would not try and "warp" the camera back?

    I have found this code, which does not seem to work:
    "
    public void ResetScene()
    {
    ARKitWorldTrackingSessionConfiguration sessionConfig = new ARKitWorldTrackingSessionConfiguration (
    UnityARAlignment.UnityARAlignmentGravityAndHeading, UnityARPlaneDetection.Horizontal,true,true);
    UnityARSessionNativeInterface.GetARSessionNativeInterface ().RunWithConfigAndOptions (
    sessionConfig, UnityARSessionRunOption.ARSessionRunOptionRemoveExistingAnchors |
    UnityARSessionRunOption.ARSessionRunOptionResetTracking);

    }

    "

    Thanks!
     
    gamescorpion likes this.