Search Unity

  1. Read here for Unity's latest plans on OpenXR.
    Dismiss Notice

ARKit support for iOS via Unity-ARKit-Plugin

Discussion in 'AR' started by jimmya, Jun 5, 2017.

  1. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    Please read the FAQ: https://forum.unity.com/threads/unity-arkit-plugin-faq.495191/
    Also, see the TUTORIAL.TXT and SCENES.TXT in the package.
     
  2. Nimdanet

    Nimdanet

    Joined:
    Jan 19, 2013
    Posts:
    21
    Hi!

    With the new release of iPhone X I want to test my game on iPhone X simulator. So I changed on Unity Target SDK to Simulator SDK. I'm getting 5 errors when building on Xcode this way:

    Screen Shot 2017-11-01 at 21.01.03.png

    Screen Shot 2017-11-01 at 21.01.09.png

    I just updated everything hoping the error was gone, no luck. Didn't find anything useful on google as well.

    When I try to build to physical device (not simulator) everything works well.

    Using:
    Unity 5.6.4f1
    Unity ARKit Plugin 1.0.10
    Xcode 9.1

    I run out of ideas, please help!

    Thanks!
     
  3. thylaxene

    thylaxene

    Joined:
    Oct 10, 2005
    Posts:
    709
    You can't use the simulator to test Unity iOS stuff. Been like that since the beginning. Mind I don't keep up to date on these things. But I know in the beginning you couldn't so ever since I don't try. If it works on your test devices and your UI is built to react to different screen sizes then it will more then likely work on iPhone X.
     
    jimmya likes this.
  4. thylaxene

    thylaxene

    Joined:
    Oct 10, 2005
    Posts:
    709
    Yep I'd like to know this as well. I'm doing similar hacky thing. I'm counting points returned (as I'm not using planes) and enabling my placement tools once a threshold is reached.

    So is there a better way of doing? How does the Ikea app do it?

    Cheers.
     
  5. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    The most reliable way would be to hook the ARSessionTrackingChangedEvent and get the camera.trackingState and camera.trackingReason when the event gets fired. See the values and what they mean here: https://developer.apple.com/documentation/arkit/artrackingstate?language=objc and https://developer.apple.com/documentation/arkit/artrackingstatereason?language=objc
     
  6. thylaxene

    thylaxene

    Joined:
    Oct 10, 2005
    Posts:
    709
    Thanks for the reply. As usual I should've RTFM. ;)
     
  7. Numa

    Numa

    Joined:
    Oct 7, 2014
    Posts:
    81
    Is anyone able to record videos using the iOS screen recording while using ARKit on an iPad? iPhone works fine, but on iPad the image freezes as soon as the AR camera starts.
     
  8. Mobgen-Lab

    Mobgen-Lab

    Joined:
    May 23, 2017
    Posts:
    15
    Same problem here
     
    Numa likes this.
  9. natwales

    natwales

    Joined:
    Dec 5, 2012
    Posts:
    10
    FYI - I think there is a bug in the ARSessionTrackingChangedEvent. I've added the following code:

    private void ARSessionTrackingChanged(UnityARCamera arCamera) {
    Debug.Log ("-----");
    Debug.Log ("tracking state is : " + arCamera.trackingState);
    Debug.Log ("tracking reason is :" + arCamera.trackingReason);
    }

    and in my console under good tracking conditions I'm seeing:

    ARTrackingStateLimited with ARTrackingStateReasonNone

    and if I cover up the camera I get:

    ARTrackingStateNormal with ARTrackingStateReasonInsufficientFeatures

    I do not see this behavior natively in ios. The reason seems to be accurate, but somehow the state is not.
     
  10. Numa

    Numa

    Joined:
    Oct 7, 2014
    Posts:
    81
    Interesting problem: Last week we pulled the latest repo (hadn't updated since before the folder structure changes).

    Since then a small number of our users have reported that even under perfect tracking conditions arkit is unable to pick up a surface no matter what they do. However each time, restarting their phone has fixed the problem (restarting the app did not). It happens with fresh installs too so I don't think it could be something in our code lingering from a previous install or previous run. We haven't been able to reproduce it.

    Has anyone seen this happen? Any ideas what could cause this? Maybe some links broke in the Unity scene after moving the plugin scripts around? It seems to work perfectly fine for 97% of people.
     
  11. RyanYN

    RyanYN

    Joined:
    Aug 13, 2013
    Posts:
    18
    hi, can you let me know what's the difference between:
    UnityARSessionNativeInterface.ARAnchorAdded
    UnityARSessionNativeInterface.ARUserAnchorAdded
    UnityARSessionNativeInterface.internal_ARAnchorAdded
    UnityARSessionNativeInterface.internal_ARUserAnchorAdded
    thanks!
     
  12. detolox

    detolox

    Joined:
    Aug 15, 2013
    Posts:
    19
    Hi!.. I have a problem with ARKITRemote cannot connect to iPhone and I try with 2 differents devices, iPhone 6s and 6 plus with the last ios version via USB.

    I build the scene ARKITRemoteConnection in debug mode and Development Build checked. I deploy it into my device and open the app.

    In unity I open the scene EditorTestScene I run the scene in the editor, in the console y select my device and I only get this error "Failed to connect to player"..

    I am using the last version of ARKit plugin and Unity 2017.1 Personal edition. So I dont know what I am doing wrong. Sorry for my english. Thanks in advance.
     
  13. r3000

    r3000

    Joined:
    Aug 2, 2017
    Posts:
    12
    Did you resolve this?
     
  14. natwales

    natwales

    Joined:
    Dec 5, 2012
    Posts:
    10
    no, but since TrackingStateReason seems to be working, I am just switching on ARTrackingStateReason instead. Effectively ARTrackingStateReasonNone = ARTrackingStateNormal as far as I can tell.
     
  15. kushG

    kushG

    Joined:
    May 25, 2017
    Posts:
    18
    Can I get the resolution of textureY and textureCbr ? Right now in UnityARVideo it is just getting the textureHandles and then creates a texture using Screen.currentResolution but I want to create a texture using the actual resolution of textureY and textureCbr in native code. Is there a straight forward way of getting this or I have to write some native code to get these values ?
     
  16. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    iPhone 6 plus will not work (does not have A9).
    You should be building the Assets/UnityARKitPlugin/ARKitRemote/UnityARKitRemote.scene to your device.
     
  17. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    CreateExternalTexture does not create a new texture, but just creates a Texture2D wrapper around the native texture object, so the height and width of the native texture it points to does not change. The height and width passed in will be used to populate the metadata for the Texture2D wrapper. E.g. we should really be passing height/2 and width/2 for the CbCr texture
     
  18. kushG

    kushG

    Joined:
    May 25, 2017
    Posts:
    18
    Thank you for the quick response Jimmya ! I might have been not so clear with my question. I get the point that you create a Texture2D wrapper for the metadata and maybe to fit the native texture to screen but all I want to know is the resolution of native texture. I only get the pointers to these textures, is there a way I can use these pointers to get their resolution ?
     
  19. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    There is not a way via scripting. Why do you want to know it? You stated your reason was to create smaller textures, but as I said we do not create new textures, rather just use the already created native textures directly.
     
  20. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    well for one, the internal ones are not supposed to be public. we need to fix that. they are actually the callbacks from the native code.
     
  21. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    You're right it appears as if TrackingState is returning the wrong value. Let me investigate.
    Update: found the bug - will push a fix to BitBucket.
    Update: fixed - in this commit.
     
    Last edited: Nov 3, 2017
  22. football928

    football928

    Joined:
    Sep 24, 2015
    Posts:
    17
    So I have AR working it's just that my object just floats it's not on a surface. I have the script on that detects the surfaces but how would I use it to put my environment on the surface.
     
  23. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    Did you change the version of XCode 9 used (and therefore ARKit SDK)? What version of XCode are you using now, and what were you using before if so?
     
  24. r3000

    r3000

    Joined:
    Aug 2, 2017
    Posts:
    12
    Thanks Jimmya for this plugin. Saves so much time :)

    Got 2 questions, big thanks to anyone who knows the answers!
    1. The UnityARKitScene example enables you to conjure a single object (cube) and even reposition it but tapping and dragging. What if you want to, with each new tap, add more objects (cubes) and even other objects (spheres, etc)? Where does on learn how to do this in unity?
    2. Has anyone got real time capture and storage of 3D environments (via photogrammetry, etc) into Unity? As in you record an ARKitSession? (See "City Mapping" at 6:14 for eg into this Tango video?
    PS If there's a better place to post such questions, please enlighten me :)
    Thanks!
     
  25. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    For question 1, you should learn how to use the basics of Unity first - do some tutorials, etc.
    For question 2, that tech does not exist yet on ARKit
     
  26. daniilkorotin

    daniilkorotin

    Joined:
    Oct 8, 2017
    Posts:
    4
    Hey, Jimmya. First of all, thanks for the great plugin, well documented and works excellent!

    I'm new to Unity, so need a bit of a general advice here, but related to this particular plugin: how should I update it? At this time I see the flow as following: clone the Bitbucket repo somewhere, take the files I need and replace the ones used in my project (e.g., in the .../NativeInterface folder). Is that correct? Thanks in advance!
     
  27. phits

    phits

    Joined:
    Aug 31, 2010
    Posts:
    41
    Does anyone know how to get Unity3D ARKit to go full-screen? I am getting the camera background video on the right and left hand sides. See attached screenshot. Thanks

    Image-1.jpg
     
    shmyt likes this.
  28. shmyt

    shmyt

    Joined:
    Dec 9, 2016
    Posts:
    5
    I got the same on iPhone X, it's no problem on the other iPhones.
     
    phits likes this.
  29. phits

    phits

    Joined:
    Aug 31, 2010
    Posts:
    41
    Yes, it is full-screen on my iPhone 6S
     
    shmyt likes this.
  30. SkaZonic

    SkaZonic

    Joined:
    Jun 20, 2017
    Posts:
    23
    Hey Guys,

    I have a few questions, so very quickly can someone tell me how I can efficiently stop arkit tracking, the only idea I have is to set config.detectplanes to something??.
    Secondly, is it possible to detect feature points/planes inside a building all the way to a set destination (for example 20m in a straight line) and then afterwards stop arkit plane detection/feature point detection + stopping elements of arkit tracking, maybe? and just use the cameras transformation. However the twist is, that it would be in a scenario such as in smoke, when your navigating back to that destination (with arkit plane detection and feature point detection disabled etc). How would it work? Or is there another approach to have ARkit work in scenarios such as smoke and bad tracking quality. Thanks a lot to the community with supporting me and my project throughout my journey ;)
     
  31. hanger102

    hanger102

    Joined:
    Jan 3, 2013
    Posts:
    13
    UPDATE: Nevermind, Just rebuild it and didn't fix the warnings and that seem to work for some reason. But it seems when running the face tracking scenes it's using the wrong camera with the remote. How do I switch the camera?

    Trying the remote and ARkit for the first time here. I have an iPhone X with xcode 9.1. It seems to connect fine but when I hit the start session button my iPhone closes the remote app. No errors, has anyone gotten this before? Thanks in advance.

    Autoconnected Player</i> connected
    UnityEngine.XR.iOS.ConnectToEditor:EditorConnected(Int32)
    UnityEngine.Events.UnityEvent`1:Invoke(T0)

    (Filename: /Users/builduser/buildslave/unity/build/artifacts/generated/common/runtime/DebugBindings.gen.cpp Line: 51)
     
    Last edited: Nov 5, 2017
  32. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    Did you guys get the latest code from Bitbucket? I have tested on iPhone X and it was fine. There was a fix for that some time ago (Oct 3)
     
    shmyt likes this.
  33. tomtong

    tomtong

    Joined:
    Aug 5, 2013
    Posts:
    14
    Hello. If I want to update the position of anchored object, do I need to clone the object each time or are there any anchor update method for me to call? Thx
     
  34. HaimBendanan

    HaimBendanan

    Joined:
    May 10, 2016
    Posts:
    28
    Hey @jimmya, is it currently possible to apply the face expressions recorded by arkit on iphone x to a humanoid face?
     
    phits likes this.
  35. phits

    phits

    Joined:
    Aug 31, 2010
    Posts:
    41
    Nope, I will try it. What is the best way to upgrade an existing app? Thanks!
     
  36. phits

    phits

    Joined:
    Aug 31, 2010
    Posts:
    41
    I downloaded and installed the latest and it is now full-screen on the iPhone X. Thank you!
     
    shmyt and HaimBendanan like this.
  37. HaimBendanan

    HaimBendanan

    Joined:
    May 10, 2016
    Posts:
    28
    If not - I am looking itno making it happen. Any suggestions?
     
  38. Numa

    Numa

    Joined:
    Oct 7, 2014
    Posts:
    81
    I was testing on the 9.1 beta at the time, then built for the store using the public 9.0 version. Maybe I used the "append" button in Unity and some files carried across from 9.1 to 9.0? If it's a matter of building the app with the wrong SDK, why would it affect some people and not others?
     
  39. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    Yeah I'm not sure what's happening with this, that's why I'm asking questions - my theory is that there was a fix in a later version (maybe 9.1) of XCode that fixed an existing issue in the SDK, and you built with the older version of the SDK and you're still seeing it. AFAIK, there's nothing on Unity side that would affect just a small number of people.
     
  40. krisventure

    krisventure

    Joined:
    Mar 24, 2016
    Posts:
    54
    There's a serious memory leak that's only showing on the iPhone when I run Unity's performance Profiler I can see in the Memory part that the textures as well as the Total object count continuously increasing with time, whether it's just the Remote ARkit app running nothing but the camera input forwarded to my Mac or the external built of my entire ARKit app, in both cases the memory is leaking, the textures and the Object count both go up to the 10s of thousands in less than a minute! It's hard to imagine that this would be somehow an inherent issue with just the camera rendering, more likely base code of ARkit and the way it renders the camera image, probably not cleaning up the rendertexture. It could also be related to iOS11.1 because I just updated to that version and I'm not sure if this already happened before.
     
  41. suxinren

    suxinren

    Joined:
    Jan 13, 2017
    Posts:
    14
    Hi @christophergoy ,I encountered the following issues during my use of Arkit.

    Undefined symbols for architecture arm64:
    "_MTLCreateSystemDefaultDevice", referenced from:
    -[UnityARSession setupMetal] in ARSessionNative.o
    ld: symbol(s) not found for architecture arm64
    clang: error: linker command failed with exit code 1 (use -v to see invocation)

    I've already included the Arkit.framework in the project. How can i solve this problem?

    I use Unity 2017.2.0f3 ,
    XCode 9.2 beta ,
    and I'm using the latest version of the ARKit plugin.
     

    Attached Files:

    Last edited: Nov 6, 2017
  42. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    Do you have the latest code for the plugin? There was a leak that was fixed earlier: https://bitbucket.org/Unity-Technol...mits/0db81286dee40ad34d3c9eb8072cd56f593b5734
     
  43. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    Could be a deprecation in XCode 9.2 beta - could you check same project with XCode 9.1?
     
  44. suxinren

    suxinren

    Joined:
    Jan 13, 2017
    Posts:
    14
    more problems in 9.1 , so i use 9.2
     
  45. suxinren

    suxinren

    Joined:
    Jan 13, 2017
    Posts:
    14
    Hi jimmya,then I tested on XCode 9.0 (9A235),still got the same problems。
     
  46. smallbit

    smallbit

    Joined:
    Oct 3, 2013
    Posts:
    55
    How do I change field of view of AR camera?
     
  47. taplando

    taplando

    Joined:
    May 4, 2017
    Posts:
    1
    Hey guys,

    So I have been using ARKit for a while now, everything works great. But now I have to use some tall objects (that don't fit in the camera view completely unless you move far back) in my scene. The issue is that as soon as the object's base in out of the view it starts drifting randomly.

    Is this a bug or a limitation?
     
  48. Mobgen-Lab

    Mobgen-Lab

    Joined:
    May 23, 2017
    Posts:
    15
    Is it possible have the same session of ARKit for different scenes?

    My problem is that I don't want to wait again to detect the surfaces when I change of scene.

    Thanks
     
  49. Burglecut

    Burglecut

    Joined:
    Jun 27, 2015
    Posts:
    10

    You must use the same session for different scenes. Put everything related to ARkit under the same parent object and set it to don't destroy on load and you can use it in every scene you like :)
     
  50. JelmerV

    JelmerV

    Joined:
    Nov 29, 2012
    Posts:
    219
    Hi, In my game AR Airplanes I experience stuttering motion of the gameobject:

    I have experienced this before when a camera moving script would be running in Update(), while a GameObejct's controlscript would run in LateUpdate().

    But with ARKit and my airplane, I can't find a problem like this (everything is on Update()).
    I can't be due to bad tracking either, as stationary objects appear smooth during tracking.
     
unityunity