Search Unity

  1. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Resolved ARKit support for iOS via Unity-ARKit-Plugin

Discussion in 'AR' started by jimmya, Jun 5, 2017.

Thread Status:
Not open for further replies.
  1. jimmya

    jimmya

    Joined:
    Nov 15, 2016
    Posts:
    793
    It's probably got a cached connection or something. Turn off "Autoconnect to Profiler" if you have that turned on in Build Settings when you build UnityARKitRemote app. Then restart unity to get rid of any cached connections, and retry instructions.
     
  2. Arvin6

    Arvin6

    Joined:
    Jun 2, 2017
    Posts:
    27
    I have problems with stereoscopic view on ARkit. I followed an old post in this forum where someone used GVR 0.85 sdk to enable VR mode and I got it working too. The problem is, that method uses rendertexture to get the main camera's feed and draw it on a quad which will be in the GVRMain camera's view. This approach quite works, but the drift is pretty bad and there is also protruding images since the quad actually exists in virtual world. I get a slightly cylindrical image from the quad which doesn't work well for Mixed reality.

    Is there any way to just add left and right eye cameras by modifying the Viewport as we usually do? ARKit camera gets the background camera texture quite differently and I'm pretty confused.

    I would love to hear some idea to deal with this on ARkit. I'd like to have something like this
    https://forum.unity3d.com/threads/a...unity-arkit-plugin.474385/page-2#post-3099320
     
  3. steviehutch

    steviehutch

    Joined:
    Jul 7, 2017
    Posts:
    4
    Hi, I've upgraded to Mac 10.13 beta 4, High Sierra, X Code 9 beta 4, and Unity 2017.1.0f3 and ios 11 beta 4 ( on the ios device to test ), however I just can't deploy to the ios device to test.
    When I follow the instructions to deploy to the ios device, and click on the Unity.iPhone.xcodeproj file I get an error telling me that the .xcodeproj file cannot be opened because the project file cannot be parsed. I apologise if this is a really basic issue not related to ARKit, however it is blocking me at the moment. I could deploy fine before I upgraded.
     
  4. jimmya

    jimmya

    Joined:
    Nov 15, 2016
    Posts:
    793
    Could you file a bug (Unity Editor Help Menu last item) with your xcode project included? Put the case number here, so we can investigate.
     
  5. jimmya

    jimmya

    Joined:
    Nov 15, 2016
    Posts:
    793
    @MSFX @Grimps Tried this with iphone 6S, macbook core i7/2.8ghz/16gb, Unity 2017.1 release and Unity ARKit Remote looks pretty good. Did you guys solve your issues on this? Would like to try and solve if lots of people are getting this. Also, if others are able to use this and get decent performance, we'd like to hear about that too.
     
  6. darthbator

    darthbator

    Joined:
    Jan 21, 2012
    Posts:
    169
    I seem to be having issues getting the remote connected. I just updated to the newest version of arkit from the store. When playing I get a gui box telling me to connect to the player in the console menu. I do so and then see the following

    Code (CSharp):
    1. --Attempting to connect to player ip: 127.0.0.1 with a 1 ms timeout
    2. UnityEditorInternal.ProfilerDriver:DirectURLConnect(String)
    3. UnityEditor.AttachProfilerUI:SelectProfilerClick(Object, String[], Int32)
    4. --Failed to connect to player
    The remote appears to be running but I can't seem to get any AR-Kit functionality from it. Am I missing a step in here?
     
    ina likes this.
  7. Eugene_Trufanov

    Eugene_Trufanov

    Joined:
    Dec 8, 2014
    Posts:
    5
    I added simple model on ARKit scene, but this model with scale (1.0f, 1.0f, 1.0f) very big. I see that all prefabs in project contain scale < 1. For example in UnityARBallz BallPrefab have 0.03f scale. How to set small sizes all game objects in scene without changing their scale? I need this for correct calculation of physics.
    Thanks.
     
    yassuu likes this.
  8. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,962
    I havent tested this one and it's just an idea, but I think it might work: Scale your CameraParent object to greater value. This should work with simple scene tracking. But I think it won't work with plane detection.
     
  9. amb3317

    amb3317

    Joined:
    Jun 24, 2016
    Posts:
    4
    Hi there @christophergoy! Quick question: I'm using an iPad Pro, and I notice that the camera feed is "zoomed in" / lower-res than it is with camera applications on the iPad (note that the iPhone 7 we're using doesn't suffer from this issue). See attached photos for comparison b/w ARKit and the native camera app.

    I found this thread from Vuforia that seems to articulate the issue: https://developer.vuforia.com/forum/ios/sdk-version-3-0-5-retina-resolution-problem. Is there any way to get the full camera resolution from the iPad? Without it, the quality of the AR experience is significantly worse :(
     

    Attached Files:

    olix4242 likes this.
  10. mmortall

    mmortall

    Joined:
    Dec 28, 2010
    Posts:
    89
    I have crash on device constantly on initialization of Unity ARKit. xCode 9 beta 3 on iPhone7. unity 5.6.1. Deployment target iOS 11. ARKit.framework added to the project build target.

    Fatal Exception: NSInvalidArgumentException
    *** setObjectForKey: key cannot be nil

    Code (CSharp):
    1.     [nativeSession->_classToCallbackMap setObject:anchorCallbacks forKey:[ARPlaneAnchor class]];
    It seems like [ARPlaneAnchor class] return null. How it can be? Could you help please?
     
  11. MSFX

    MSFX

    Joined:
    Sep 3, 2009
    Posts:
    116
    are you using the beta version of Unity? RC won't work...
     
  12. mmortall

    mmortall

    Joined:
    Dec 28, 2010
    Posts:
    89
    Unity 5.6.1 is not a beta.
     
  13. zhbjvd

    zhbjvd

    Joined:
    Dec 17, 2013
    Posts:
    12
    Thanks .. I will try that. Plus I am building a test scene. I want to have a functionality to reload the scene. Also due to which I want to reload AR Kit Too. I noticed there was only PauseSession Function present in UnityARSessionNativeInterface.cs. I linked a stop function from native side into UnityARSessionNativeInterface.cs. But when tried it is getting the error attached as a screenshot.
     

    Attached Files:

    olix4242 likes this.
  14. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
    Hey,
    We are scaling the camera texture to fit the screen at the moment based on the aspect ratio of the camera feed vs. the aspect ratio of the device.
    Cheers,
    Chris

    Edit: I wanted to add that this was at the recommendation of the ARKit team and I could be curious to know if you see the same thing with their SceneKit example app.
     
    Last edited: Jul 26, 2017
    developersice, lourd and 4NDDYYYY like this.
  15. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
    It could be the device you are using. If you are using anything older than an iPhone 6s, i'm not sure what happens with ARKit. Can you tell us which device you are using?
    Cheers,
    Chris

    Edit: We should account for this in the plugin, so I will make some changes to make sure there is a clear Error message in the log if ARKit is not supported at runtime.
     
    Last edited: Jul 26, 2017
  16. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
    The stop function tears down the texture cache. If you call stop, you will have to mirror that with a call that sets up the texture cache again. ARKit doesn't necessarily have a concept of "stopping." You can pause it, and call RunWithConfigAndOptions to reset tracking data if you wish.

    The stop function in native code is sort of an orphan and should most likely be removed.
    Let me know if this helps.
    Cheers,
    Chris
     
    zhbjvd likes this.
  17. geoffa

    geoffa

    Joined:
    Apr 27, 2015
    Posts:
    2
    I have searched this thread looking for examples of how people are addressing the issue of scale and physics. The GO HitCubeParent in the UnityARKitScene has an x, y, and z scale of 0.1. Which, as I've been reading in this thread, is the correct approach. Using physics approaches I've become accustomed to, for things like detecting collisions and applying velocity, doesn't really work at this scale. My game scene is a child of this GO.

    I saw a mention in this thread that said to change physics settings to make this work. However everything I've tried just ends up being wacky. Would love to hear any thoughts and approaches about how to get Unity's physics engine to work as expected at this smaller scale.

    Thanks and forgive me if this has been answered already. If it has I'd be beyond thankful for a link to it. Really enjoying playing around with ideas using this plugin!
     
    Last edited: Jul 26, 2017
  18. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
    Hi,
    I think you need to select the player to connect to in the editor, in this case it looks like it's trying to connect to localhost (127.0.0.1) which would be incorrect.
    Cheers,
    Chris
     
  19. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
    Hi,
    These seems to be a very common question and I am consulting with our physics team to see if we can come up with a workflow that will allow this to happen.
    Cheers,
    Chris
     
    KwahuNashoba and geoffa like this.
  20. akhella

    akhella

    Joined:
    Apr 14, 2015
    Posts:
    9

    This is super valuable! Thank you for creating this.

    I am having an issue with touch input not being sent to Unity via ARKit Remote. Do I need to rewrite the touch functions with mouse input? Or is there a way to send touch input to Unity?
     
  21. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
    Touch input is not currently working with this remote app. We are looking into putting the functionality from the ARKit remote directly into the regular Unity Remote. Stay posted.
    Cheers,
    Chris
     
    lourd and akhella like this.
  22. amb3317

    amb3317

    Joined:
    Jun 24, 2016
    Posts:
    4
    Thanks for the prompt reply, @christophergoy. To clarify, is the scaling something I can modify? Also, I'll check out the SceneKit example app and let you know!

    Best,
    Avery
     
  23. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
    You could modify it, but it will have an effect on the integrity of experience. For example, it will look like your 3D virtual objects are not anchored in the world because the scaling of the camera is off. If you do scale it, you would modify the YUVShader file to do so. Right now we center it and scale it by a number that is set in the ARVideo.cs script.

    I would strongly recommend on leaving it alone, but I think you will quickly see for yourself what I mean :)
    Cheers,
    Chris
     
  24. darthbator

    darthbator

    Joined:
    Jan 21, 2012
    Posts:
    169
    I'm just selecting the player from the console dropdown? Should I be manually connecting to the LAN IP address of the phone player? I figured it was just routing through localhost because I have the device plugged in via a cable. I would love to get the remote working!
     

    Attached Files:

  25. lingoded

    lingoded

    Joined:
    Aug 22, 2015
    Posts:
    8
    @jimmya thanks for the editor support! But how to turn it off? :D

    My bad.. Lol. Build the demo scene after updating project rather than my own.. All is good :cool:
     
    christophergoy likes this.
  26. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
    If you are selecting the player from the console dropdown, i'm not sure why you would be having an issue. Are you running the ARKit Remote app before connecting?
     
  27. amb3317

    amb3317

    Joined:
    Jun 24, 2016
    Posts:
    4
    Haha, got it. I'll try modifying it just so I can gain an intuition for the process :). Have you had an opportunity to test this out on an iPad Pro? I'm curious what the solution to this problem may eventually be/whose boat the solution falls in (i.e., is this an Apple thing?)

    Best,
    Avery

     
  28. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
    If you remove the ARKit remote component, it should turn off the functionality. Are you running into issues?
     
  29. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
    We have been testing on iPad Pros. And we did bring this up with apple, and it is expected behavior. The solution would be to make cameras that produce a video stream that has the same resolution as the device. This way would wouldn't have to rescale it at runtime.
     
  30. phil-harvey

    phil-harvey

    Joined:
    Aug 27, 2014
    Posts:
    75
    So got the remote set up and it works well. A bit laggy, but I think that is also mainly our wifi network in the office.
    Looking forward to getting touch / Hit Detection in.
     
    christophergoy likes this.
  31. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
    Hi all,
    Just a heads up, I'm ripping off the bandaid and moving all of the files in the top level of the ARKit Plugin into a folder named "ARKitPlugin" so we don't ruin your directory structure. This will force you to delete your Library folder underneath your project folder in order for Unity to not cry about missing .meta files. Which in turn means all of your assets will need to be re-imported. If this is going to interrupt your workflow I recommend updating later tonight. I will post again once the bitbucket version is updated, and again when the Asset Store package is updated.
    Cheers,
    Chris
     
    Arvin6, 4NDDYYYY and mimminito like this.
  32. darthbator

    darthbator

    Joined:
    Jan 21, 2012
    Posts:
    169
    Indeed I am!
     
  33. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
    Are you running the ARKit remote app on the phone or the Unity Remote app on the phone?
     
  34. darthbator

    darthbator

    Joined:
    Jan 21, 2012
    Posts:
    169
    I'm using the ARKit remote that I built out of the plugin scene.
     
  35. mimminito

    mimminito

    Joined:
    Feb 10, 2010
    Posts:
    780
    I am seeing a lot of talk about scale in here. Am I right in assuming 1 unit in Unity will be 1 meter in reality, when using ARKit? So a standard Unity cube when placed into reality would be 1m wide x 1m high x 1m deep?
     
  36. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
    Yes, that is correct
     
  37. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
    Hello all,
    The bitbucket repo has been updated to restructure the ARKit Plugin directory as to leave a smaller footprint when importing it. The package has been submit and is awaiting approval. If you are using the asset store for getting this plugin, you may need to delete the old ARKit files to take advantage of the restructuring. Otherwise your files will just be updated as usual. Let me know if you run into any issues
    Cheers,
    Chris
     
    DavidErosa likes this.
  38. dpentecost

    dpentecost

    Joined:
    Apr 3, 2009
    Posts:
    136
    Hi - Trying to get UnityARKitRemote built and running. Using Latest Xcode 9 beta 4, tried both latest Unity 2017.2 beta and 2017.1. High Sierra beta, 9.7" iPad Pro iOS 11, latest (I think) UnityARKit and plugin. I got a simple build with UnityARKit working easily a couple of weeks ago, but this build of the remote fails with this Xcode message at the bottom:

    dyld`__abort_with_payload:

    0x106a0976c <+0>: mov x16, #0x209

    0x106a09770 <+4>: svc #0x80

    -> 0x106a09774 <+8>: b.lo 0x106a0978c ; <+32>

    0x106a09778 <+12>: stp x29, x30, [sp, #-0x10]!

    0x106a0977c <+16>: mov x29, sp

    0x106a09780 <+20>: bl 0x106a08de4 ; cerror_nocancel

    0x106a09784 <+24>: mov sp, x29

    0x106a09788 <+28>: ldp x29, x30, [sp], #0x10

    0x106a0978c <+32>: ret

    The Remote gets built to the iPad, opens up to a black screen, but never shows "Waiting for editor connection"

    In Unity I run a scene, I find my device in the console, and I get:

    Sending message header failed
    UnityEditorInternal.ProfilerDriverDirectURLConnect(String)
    UnityEditor.AttachProfilerUI:SelectProfilerClick(Object, String[], Int32)

    Sending message header failed

    Failed to connect to player


    Any ideas? Am I using the wrong version of one of these components? This (not 2017) iPad Pro not suitable?

    I'm following the readme for the remote carefully, tried it several times.

    THANKS!
     
  39. petey

    petey

    Joined:
    May 20, 2009
    Posts:
    1,806
    Out of interest, does anyone have a good solution for scaling up and down scenes? I feel like linking everything to a empty game object is just asking for trouble.
     
  40. dpentecost

    dpentecost

    Joined:
    Apr 3, 2009
    Posts:
    136
    Okay, disconnected from Xcode, tried reopening the app. This time got waiting for editor connection. So far so good.
    Went back to Unity, tried with both USB and WiFi. At times seemed to connect, but never get the camera background - only green.

    Getting closer.
     
    christophergoy likes this.
  41. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
    Hey,
    Do you have anything else from the logs that could give more information? If you call IsSuppported on the ARkitSessionConfiguration that you are using it should tell you if the device you are using is supported. If your device doesn't have an A9 processor, ARKit won't work https://developer.apple.com/documen...nfiguration/2867255-issupported?language=objc

    Cheers,
    Chris
     
  42. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
    I'm trying out some magic with applying a scale to the projection matrix to scale up the camera. I'll let you know how it goes.
     
    mikejohnstn, petey and mimminito like this.
  43. dpentecost

    dpentecost

    Joined:
    Apr 3, 2009
    Posts:
    136
    Chris - Thanks for jumping in. iPad Pro has A9X processor. And I think I just got it working!

    I had been working from a project someone posted today. Others got it to work. But I dropped back, started a fresh project and downloaded the package from the Asset Store. All the ARKit files looked the same, same folders, etc. but this time it came together. I'll come back if I still have issues. But this is enough for one night.

    Thanks again!
     
  44. ipstatic

    ipstatic

    Joined:
    Jul 27, 2017
    Posts:
    2
    Hey,

    I'm trying to add some kind of persistence to ARKit. My goal is to create a persistent world that you load when you put your device on a specific dock, so you have the same frame of reference. My current project is succesfully saving and loading the planes gameObject at their right position.

    My problem is, I can't use these loaded planes as anchors for my game objects and they are not updated.
    The gameobjects are here, but it feels they are not recognized as a planeAnchor for the native program.

    Right now i'm using the "AddAnchor" Function from the UnityARAnchorManager but I feel that its not enough.
    My guess is that it's not really added to the anchor list of the native session or the identifier is wrong when loaded.

    Do you have any idea on how to solve this ? Maybe I could use UnityUserAnchor but it feels pretty limited...

    Thanks and have a good day !
     
    Last edited: Jul 27, 2017
  45. zhbjvd

    zhbjvd

    Joined:
    Dec 17, 2013
    Posts:
    12
    Doesn't seem to work. I was getting anchor update logs. It should have added the anchors again.
     
  46. Blarp

    Blarp

    Joined:
    May 13, 2014
    Posts:
    269
    Last edited: Jul 27, 2017
    lingoded, sama-van, idspe and 2 others like this.
  47. P4blo

    P4blo

    Joined:
    Dec 12, 2012
    Posts:
    18
    Hi!

    I just updated my app to use the latest version of the plugin and I'm having the
    'Unrecognized ARTrackingStateReason: 1' error in Xcode.

    Right now I'm using:
    Xcode Version 9.0 beta 4 (9M189t)
    Unity 2017.1.0f3 Pro (64 bits)
    Latest Pull from Mercurial Repository for Unity ARKit Plugin (pulled today 27 July )
    Latest iPad Pro (iOS 11 beta 4)


    In Xcode I'm getting the error:
    'UnrecognizedARTrackingStateReason', reason: 'Unrecognized ARTrackingStateReason: 1'

    The code in ARSessionNative.mm file looks like that:

    Code (CSharp):
    1. static inline UnityARTrackingReason GetUnityARTrackingReasonFromARTrackingReason(ARTrackingStateReason trackingReason)
    2. {
    3.     switch (trackingReason)
    4.     {
    5.         case ARTrackingStateReasonNone:
    6.             return UnityARTrackingStateReasonNone;
    7.         case ARTrackingStateReasonExcessiveMotion:
    8.             return UnityARTrackingStateReasonExcessiveMotion;
    9.         case ARTrackingStateReasonInsufficientFeatures:
    10.             return UnityARTrackingStateReasonInsufficientFeatures;
    11.         default:
    12.             [NSException raise:@"UnrecognizedARTrackingStateReason" format:@"Unrecognized ARTrackingStateReason: %ld", (long)trackingReason];
    13.             break;
    14.     }
    15. }
    I can solve the problem just commenting the Exception in the 'default' case and returning something correct.
    But, my question is: looks this method updated? Is the correct code or do I need some new 'case' statement.

    I have deleted the Xcode project and re-build it again from Unity, so it should be the correct version with the latest update from Unity ARKit Plugin repository.

    Thanks for the great job
     
    Last edited: Jul 27, 2017
  48. steviehutch

    steviehutch

    Joined:
    Jul 7, 2017
    Posts:
    4
    Hi Jimmya, Thank you for your reply.I suspect it is my provisioning profile / inconsistency in the build settings that is at fault, and nothing to do with ARKit, as i have the same issue with a project without ARKit included. I've raised an issue as instructed.

    https://fogbugz.unity3d.com/default.asp?935532_a71mtdkar7lki48b
     
  49. mmortall

    mmortall

    Joined:
    Dec 28, 2010
    Posts:
    89
    Where is "Privacy - Camera Usage Description AR BABY" added to plist? I cannot find it in ARKit project. Unity adds it somewhere by default? That was the reason of my crash I had.
     
  50. DavidErosa

    DavidErosa

    Joined:
    Aug 30, 2012
    Posts:
    41
    It's on the iOS Player Settings -> Other Settings -> "Camera Usage Description" (under the configuration header)

     
    mmortall and christophergoy like this.
Thread Status:
Not open for further replies.