Search Unity

  1. New Unity Live Help updates. Check them out here!

    Dismiss Notice

ARKit support for iOS via Unity-ARKit-Plugin

Discussion in 'ARKit' started by jimmya, Jun 5, 2017.

  1. haydex

    haydex

    Joined:
    Jun 16, 2017
    Posts:
    6
    I am actually using an iPhone 6s Plus and the remote app is not saying that it is waiting for connection, it is immediately start when I tap it, I see the camera with the box immediately. But in Unity everything seems to be working correctly, the player displays connected but when I click the start remote ARKit session nothing happens
     
  2. RentificUnity

    RentificUnity

    Joined:
    Aug 11, 2017
    Posts:
    3
    Some followup on this.. it turns out it was iOS11beta4 on my early iPad Pro 12". When I updated to the final seed for iOS 11 (GM), and then used Xcode 9 GM and then reverted to Unity 2017.1.0f3, it ran on my iPad flawlessly. For those who are having issues with build errors, esp linking errors in Xcode consider updating your OS to the latest on your device with the prerelease Xcode 9. I did it so I can submit my project to the iTunes..
     
  3. haydex

    haydex

    Joined:
    Jun 16, 2017
    Posts:
    6
    upload_2017-9-17_16-47-31.png upload_2017-9-17_16-47-31.png
     

    Attached Files:

  4. haydex

    haydex

    Joined:
    Jun 16, 2017
    Posts:
    6
    It turns out that I have been building every scene within the Remote app, I have uncheck all but UnityArkitRemote and now it is working :)

    upload_2017-9-17_16-59-0.png
     
  5. haydex

    haydex

    Joined:
    Jun 16, 2017
    Posts:
    6
    Thank you zacware, I really appreciate any help from you in the future, I am sure that I would face so much more issues
     
  6. dberroa

    dberroa

    Joined:
    Dec 16, 2012
    Posts:
    141
    Think if someone in real life waved their hand and hit the 3D object in AR, how would I detect that? I basically want to know when something crosses over the AR object so I can know that something "hit" it and react.

    Can this be done? If so how?
     
    4NDDYYYY likes this.
  7. wyattdesouza

    wyattdesouza

    Joined:
    Aug 14, 2015
    Posts:
    4
    I can't get the remote working:
    I can get my iPhone 7 plus connected. When i click 'Start Remote Session' i get a very laggy camera feed (1 frame every 10 seconds or so until it freezes), no assets from the scene are on the phone and my unity editor doesn't seem to receive the camera feed except sometimes the first frame. What am I doing wrong?
     
  8. cheliotK

    cheliotK

    Joined:
    Mar 6, 2013
    Posts:
    3
    Hey all,

    I'm having some issues with the ARKit remote, similar to what others have reported in this thread. My aim is to get the ARKit Remote working on an iPad (the deployment device) connected to a Windows 10 machine (the main development device), to avoid installing each new test build on the iPad just for testing. Unfortunately I can't get that to work, the iPad is refusing to connect to the Unity Profiler reliably. I have also tested connecting to a macbook pro, which almost works (more info on this later).

    specs:
    Unity 2017.1.1f1 (on Windows PC)
    Unity 2017.1.0f3 (on macBook)
    Xcode v9.0 beta 6
    ARKit Plugin v1.0.9
    iPad Pro 10.5 Wi-Fi running iOS 11.0

    setup:
    On the macbook:
    I have successfully built the demo 'UnityARKitRemote' Scene and deployed to the iPad.
    With the ARKitRemote app running on the iPad and the 'EditorTestScene' running in Unity on the mac, and with the iPad connected over usb cable, I can remote connect fine, everything works as it should (pic 'mac-usb.png').
    mac-usb.png

    With the ARKitRemote app running on the iPad and the 'EditorTestScene' running in Unity on the mac, and with the iPad connected over WiFi, I can remote connect using the iPad's local address, it connects successfully, but runs extremely slow, ~400ms delay per frame (pic 'mac-wifi.png'). Essentially unusable.
    mac-wifi.png

    On Windows:
    With the ARKitRemote app running on the iPad and the 'EditorTestScene' running in Unity on the PC, and with the iPad connected over WiFi, I can remote connect using the iPad's local address, it again connects successfully, but runs very slow, ~500ms delay per frame (pic 'win-wifi.png'). Same as the macbook attempt, essentially unusable.
    win-wifi.png

    With the ARKitRemote app running on the iPad and the 'EditorTestScene' running in Unity on the PC, and with the iPad connected over usb cable, I can not connect at all, with a 'Failed to connect to player' error (pic 'win-usb.png').
    win-usb.png

    Things to note:
    On both cases connection over wifi works, but seems to be throttled by a camera render process, specifically 'UnityRemoteVideo. OnPreRender()' (see profiler in wifi screenshots).

    At first I thought it might be an issue with iOS not being compatible with Windows and remote profiling only being available on macOS, but seeing as I can connect over WiFi to the Windows machine, I'm thinking there is no issue on OS compatibility.

    So to wrap it up: Is it possible to use ARKit Remote with an iPad on a Windows machine? If so, how?
     
    BL1TZ likes this.
  9. DerekLerner

    DerekLerner

    Joined:
    Jun 12, 2017
    Posts:
    19
    Hi Chris, Wondering if I did this correctly in Xcode?

    Screen Shot 2017-09-18 at 10.58.50 AM.png


    Also curious if I overlooked a way to specify this requirement (or isSupported) in the player settings on the Unity side? If not, would be a nice inclusion to the app or plugin if possible.
     
    Last edited: Sep 18, 2017
  10. aas395

    aas395

    Joined:
    Aug 26, 2017
    Posts:
    4
    Somehow I wasn't understanding from the Unity directions that you were only supposed to build the remote scene to the iOS device in order to get the remote to work. In case anyone else is hung up on this step, you need to build only the UnityARKitRemote scene to the iOS device. After that, you can change scenes in the Unity editor and play those and you'll be able to see them in the Unity game view window running using the device camera.
     
    purplehaze90 likes this.
  11. christophergoy

    christophergoy

    Unity Technologies

    Joined:
    Sep 16, 2015
    Posts:
    513
    Hey,
    This looks right to me based on what I see here: https://developer.apple.com/library...tibilityMatrix/DeviceCompatibilityMatrix.html

    The IsSupported method exists as an instance method on the ARConfiguration objects.
     
    DerekLerner likes this.
  12. justintheartist

    justintheartist

    Joined:
    Aug 11, 2017
    Posts:
    2
    Was searching the forum for this but can't seem to find an answer -
    1. How do I make the GeneratePlane script generate a plane that extends into infinity?
    2. How do I disable tracking once a surface has been detected and a plane has been generated?
     
    DigitalNerds80 likes this.
  13. 4NDDYYYY

    4NDDYYYY

    Joined:
    Jan 28, 2016
    Posts:
    13
    tl;dr no, you need a depth sensor.

    Not with ARKit alone. Think this through. You want to compare the volumes occupied by a virtual and a real object to see if they overlap. You need to know where they are with respect to each other. All you know about the real world is extracted from camera images.

    If you know the details of a real object (its dimensions and orientation) then you can infer its position from how it looks in those camera images. (This is kind of the crux of Vuforia and marker-based tracking.) If you don't know those details, you can be fooled by similar representations at larger or smaller scales.

    Usually, you need a depth camera capable of high density point clouds to do 3D touch interactions.
     
  14. bnorden

    bnorden

    Joined:
    Sep 18, 2017
    Posts:
    1
    I was able to get this to work with Xcode 9 GM Seed by adding in ARKit Framework to Linked librarires and frameworks in the general tab of Xcode for the project settings. Then these all went away
     
  15. danbfx

    danbfx

    Joined:
    Feb 22, 2011
    Posts:
    40
    Hey thx for helping we are really stuck. Yes we are building the ARKit Remote in this same version of Unity and using the same version of the plugin.

    Attempting to connect to player ip: 127.0.0.1 with a 1 ms timeout
    UnityEditorInternal.ProfilerDriver:DirectURLConnect(String)
    UnityEditor.AttachProfilerUI:SelectProfilerClick(Object, String[], Int32)

    Player connected
    UnityEditor.Networking.PlayerConnection.EditorConnection:ConnectedCallbackInternal(Int32)
    UnityEditorInternal.ProfilerDriver:DirectURLConnect(String)
    UnityEditor.AttachProfilerUI:SelectProfilerClick(Object, String[], Int32)

    Failed to connect to player

    Player disconnected
    UnityEditor.Networking.PlayerConnection.EditorConnection:DisconnectedCallback(Int32)
     
    conradHink likes this.
  16. joeld42

    joeld42

    Joined:
    Oct 3, 2016
    Posts:
    6
    Hey quick question, I know this isn't directly supported by ARKit or the Unity-Plugin but I've seen some demos doing this, does anyone have any tips or example code for grabbing a frame and calculating screenspace UVs to "stick" texture from the scene onto an object?

    thanks!
     
  17. danbfx

    danbfx

    Joined:
    Feb 22, 2011
    Posts:
    40
    Hi Jimmya,

    you are correct. Under the profiler the same device is also failing to connect. It breaks of very quickly just like ar remote. How can we fix this? Its an iPhone 6S with the latest iOS11 and it's runnign arkit remote fine.

    Thanks again
     
  18. lancehall10

    lancehall10

    Joined:
    Jul 13, 2015
    Posts:
    6
    I have the ARkit remote working on my Mac, but I came across another problem. No touch input. I touch on iPhone, nothing. I add the get mouse down method, nothing. I know why touching on iPhone does nothing, but why not computer screen?
     
    paulkre likes this.
  19. lancehall10

    lancehall10

    Joined:
    Jul 13, 2015
    Posts:
    6
    Use something like IKEA did. I like it, but haven't learn how to implement it yet.
     
  20. danbfx

    danbfx

    Joined:
    Feb 22, 2011
    Posts:
    40
    Hey Christopher,

    we are building latest UnityARKit scene in latest GM Xcode from fresh project and get


    Showing Recent Issues

    "_OBJC_CLASS_$_ARWorldTrackingConfiguration", referenced from:
    objc-class-ref in ARSessionNative.o

    "_OBJC_CLASS_$_AROrientationTrackingConfiguration", referenced from:
    objc-class-ref in ARSessionNative.o

    "_OBJC_CLASS_$_ARAnchor", referenced from:
    objc-class-ref in ARSessionNative.o

    "_OBJC_CLASS_$_ARPlaneAnchor", referenced from:
    objc-class-ref in ARSessionNative.o

    "_OBJC_CLASS_$_ARSession", referenced from:
    objc-class-ref in ARSessionNative.o

    ld: symbol(s) not found for architecture arm64

    clang: error: linker command failed with exit code 1 (use -v to see invocation)


    Thanks!
     
  21. danbfx

    danbfx

    Joined:
    Feb 22, 2011
    Posts:
    40
    Hi JimmyA thx again, Can't connect the profiler to the device you are correct. Now how would we troubleshoot? Its an iPhone 6S+ and it shows up in the device list. Thx so much been stuck on this for a while...
     
  22. cheliotK

    cheliotK

    Joined:
    Mar 6, 2013
    Posts:
    3
    I believe that's how it is supposed to work regarding the iPhone, i.e. arkit remote lets you have the ARKit capabilities in the editor, and interaction can only happen in the editor window, not the iPhone. If you get the ARKit remote running with the example scenes you should be able to see a plane in the editor game window, clicking on that plane in the editor game window should generate new objects on the plane. Does the example scene not work for you?
     
  23. Raf_230

    Raf_230

    Joined:
    Mar 24, 2016
    Posts:
    2
    Hi
    I want to combine my unity ARkit project with my IOS project . But i got this error every time .

    typeinfo for boost::archive::detail::basic_pointer_iserializer", referenced from unity xcode


    i have combined two project without arkit and it works. Each project run individually also.

    I have updated ARKit and IOS 11 with Xcode Beta 5.
    Here is the error
    Screen Shot 2017-09-19 at 3.53.58 PM.png
     
  24. lancehall10

    lancehall10

    Joined:
    Jul 13, 2015
    Posts:
    6
    Nope nothing. If I run the same code normally, it’s worked.
     
  25. 4NDDYYYY

    4NDDYYYY

    Joined:
    Jan 28, 2016
    Posts:
    13
    When I run my app, I occasionally get messages like this in the xcode console:

    2017-09-19 10:44:29.341562-0400 appname[481:22403] [Technique] World tracking performance is being affected by resource constraints [1]

    Where can I start figuring out whats up?
     
  26. slava_pankratov

    slava_pankratov

    Joined:
    Jan 17, 2015
    Posts:
    42
    Hi guys! Can I use low quality ARKit configuration ( ARKitSessionConfiguration) with iphone 6 and lower or is not relevant ? I get an error at the start. Thank you!
     
  27. HaimBendanan

    HaimBendanan

    Joined:
    May 10, 2016
    Posts:
    27
    My guess is that you need to install the macOS GM. You can find it in the download section of your apple developer account
     
  28. wrutkowski

    wrutkowski

    Joined:
    Aug 9, 2017
    Posts:
    8
    I had problem with my Python configuration. I reinstalled macOS (still to 10.12.6) and it fixed the problem. As per Stack Overflow accepted answer. Thanks for the suggestion though!
     
  29. 4NDDYYYY

    4NDDYYYY

    Joined:
    Jan 28, 2016
    Posts:
    13
    Can we talk seriously about efficiency? Would reducing the Application.targetFrameRate help? What other steps can be taken to reduce energy use?
     
  30. petey

    petey

    Joined:
    May 20, 2009
    Posts:
    1,306
    Hi!
    Just thought I'd post this (rough) little test up, I'm using the scaled content to change the size of the character (@christophergoy) and it's working great!


    I still have to offset the scene by attaching everything to a empty and moving that about though which breaks a few things. Did anyone find a good solution to that?

    Thanks,
    Pete
     
    KwahuNashoba and christophergoy like this.
  31. Mike-B

    Mike-B

    Joined:
    Nov 18, 2012
    Posts:
    13
    hi

    how would I disable the video background (and put an opaque black background instead)?
    I tried disabling the UnityARVideo component but it did not work.

    thanks
     
  32. smallbit

    smallbit

    Joined:
    Oct 3, 2013
    Posts:
    54
    I have a question about initializing the tracking
    So I know I can access tracking trackingState and trackingReason from UnityARCamera but I was wondering how they evaluate the init progression in Ar Dragon https://itunes.apple.com/au/app/ar-dragon/id1270046606?mt=8

    Is it just some arbitrary number based on cloud points or ArKit exposes some quantitative data about initialization progression?
     
  33. mrbroshkin

    mrbroshkin

    Joined:
    Aug 14, 2012
    Posts:
    53
    Just add ARKit.framework to Linked Frameworks and Libraries in XCode
     
  34. dreamplanet

    dreamplanet

    Joined:
    Jun 17, 2017
    Posts:
    7
    Would love to see how you got this to work, Elliott-Mitchell .. please give it to Chris so he can update the branch, as a lot of us are tripping over the same problem. Also, congratulations on your store submission! What's the title, so we can all check it out?
     
    Burglecut likes this.
  35. glenrhodes

    glenrhodes

    Joined:
    Oct 25, 2013
    Posts:
    55
    Is it possible to reset the position and Y rotation of the AR Camera? I have a project where they want to spawn a portal with a world behind it. The problem is, after the scene is loaded, AR is initialized, and the world position is set and rotation is set, and camera matrix is set, the user could then turn, and walk a bit, and then tap the ground to spawn the portal at any arbitrary position.

    This would mean I'd have to spawn an entire world (which includes navigation meshes) at any arbitrary position / rotation. What would be much easier would be on Tap, to reset the AR camera to position 0,0,0 and rotation 0,0,0, and then enable the portal and world. Is this possible?

    Sorry if this has already been answered. I've been reading through, but didn't see anything and the thread is 35 pages long now. :)
     
  36. christophergoy

    christophergoy

    Unity Technologies

    Joined:
    Sep 16, 2015
    Posts:
    513
    I guess the first place to start is profiling with instruments. I you can use the time profiler and some other instruments to see what is taking the most time. Apple doesn't provide any good thermal tracking tools yet, but you can dig deep and find out what sort of things may be causing your device to be throttled.
     
    4NDDYYYY likes this.
  37. DerekLerner

    DerekLerner

    Joined:
    Jun 12, 2017
    Posts:
    19
    Is it possible to add a Sphere Collider to the Main Camera and have Box Collider with Ridged Body objects move/react to the position of the camera when it collides with them in AR? I'm assuming it's not as simple as I'm thinking, but would enjoy any fingers pointed in a good direction about how to implement such a scenario as I'm not having much luck.
    . . . .
    Edit: Could this be accomplished by having a Sphere w/ a Sphere Collider follow/match the AR camera position? Maybe all I'm seeking is a method/script to do this sort of thing, because I did have luck w/ physics & collisions w/ an animated prim colliding w/ a static mesh object in AR. I'm just not yet finding a way for the camera/user to be the "bowling ball".
     
    Last edited: Sep 20, 2017
  38. stuartlangfield

    stuartlangfield

    Joined:
    Jul 21, 2017
    Posts:
    13
    Has anyone tried creating a function that allows you to call the HitTest to place an object, then the HitTest switches off once the object is in place?

    I.e. I'm creating an AR platformer as a test, building off the ARKitScene. Currently the HitTest is interfering with my touch screen controls because every tap repositions the character. I'd love to be able to add a new button that switches on HitTest, allows me to place my scene, then turns off so taps no longer reposition the character etc.
     
  39. christophergoy

    christophergoy

    Unity Technologies

    Joined:
    Sep 16, 2015
    Posts:
    513
    One approach could be that once you placed your character, you turn off the functionality of placing them.
     
  40. stuartlangfield

    stuartlangfield

    Joined:
    Jul 21, 2017
    Posts:
    13
    Thanks Christopher

    Are you able to point me towards any resources where I could learn how to do that? Still getting to grips with a lot of areas in Unity/ARKit.
     
  41. idorurei

    idorurei

    Joined:
    Sep 20, 2017
    Posts:
    7
    Has anyone been able to properly attach a Network Transform to the Camera/Parent? Wondering if there was a way to get a mirrored transform on a host.
     
  42. ClintChil

    ClintChil

    Joined:
    Jun 17, 2016
    Posts:
    1
    I'm noticing a big difference in the lighting between native ARKit apps and apps built in Unity with the ARKit plugin.

    In a brightly lit room during the day, the shadow side of my models are very difficult to see. In a ARKit project made with SceneKit the model is evenly lit and all sides of the model are easy to see. My project is all about showing the detail of these objects and allowing the user to see all sides of the object clearly.

    Please forgive me if this is a naive question, Lighting and 3D modeling is not my strong suit, but is there a way to increase the overall lighting or use a more diffused lighting in a Unity ARKit scene? Thanks for your help. Lighting.png
     
  43. Rich0

    Rich0

    Joined:
    Aug 9, 2013
    Posts:
    49
    Is anyone using ReplayKit with ARKIT?

    I'm getting an error 5804 - recording failed when attempting to start a recording. Using latest patch version of 2017.1.
     
  44. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    In ARKit, they use a mathematical function to hittest against an infinite plane - you cannot actually create an infinite plane as a GameObject in Unity, so you can maybe scale the prefab to be a really really large plane.
    You don't disable tracking, otherwise your camera will not move in scene, but instead you restart the session and in the new configuration, don't detect planes anymore.
     
  45. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    Just adjust lights in your scene, add more lights, or add more ambience https://unity3d.com/learn/tutorials/topics/graphics/lighting-overview
     
  46. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
  47. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
  48. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    In Unity there are many different ways to achieve this. Easiest may be to add a SphereCollider GameObject (Create a Sphere and remove its MeshFilter component and MeshRenderer, leaving you with just the SphereCollider) .Then put it under to your camera in the hierarchy, and since the camera moves around using your device, all its children move with it. Depending on how you want to interact with other colliders in the scene, you can add Rigidbody to the colliders (including the camera sphere). Read https://docs.unity3d.com/Manual/CollidersOverview.html
     
    DerekLerner likes this.
  49. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    ARKit does not expose such quantitative data - they are faking it based on normal init time and possibly other data that they get like feature points detected and planes detected.
     
  50. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    World tracking is an expensive operation, and whatever other stuff you have running on the CPUs are going to affect its performance if it exceeds a certain threshold. You can profile your code to to find bottlenecks using profilers like Unity profiler or Instruments in XCode.
     
    4NDDYYYY likes this.
unityunity