Unity ARKit Plugin FAQ

Posting a FAQ in the hopes that people do not repost the same questions over and over again. Please look through this, and if possible, through the forums before you ask your questions!

Unity ARKit Plugin FAQ

Q: What is the Unity ARkit Plugin?

A: Unity ARKit plugin will provide developers with friendly access to ARKit features like world-tracking, live video rendering, plane estimation and updates, hit-testing API, ambient light estimation, and raw point cloud data.


All the features exposed by the ARKit are available from the C# scripting API within Unity. There are also Unity Components for easy integration of these features with existing Unity game projects. Please see the README.md file in the bitbucket repository for more info.


Please join us at https://forum.unity3d.com/threads/arkit-support-for-ios-via-unity-arkit-plugin.474385 to get started.



Q: What are the prerequisites for making the Unity ARKit Plugin work?

A: This is an experimental preview plugin and it works with beta software, and it should not be used in its current form for production use. It requires the following:

• iOS 11 latest version
  
  
• XCode 9.4+ with latest iOS SDK that contains ARKit Framework
  
  
• Unity version 2017.1.0 or later
  
  
• iOS device that supports ARKit (iPhone 6S or later, iPad (2017) or later)

Q: Where can I get the Unity ARKit Plugin from?

A: Unity ARKit Plugin (experimental) and its example projects are released as open-source in the bitbucket repository https://bitbucket.org/Unity-Technologies/unity-arkit-plugin . It is also updated on the asset store: https://www.assetstore.unity3d.com/#!/content/92515



Q: Is there any Step-by-Step Tutorial written to create ARKit Project for Unity (from scratch) yet so we can quickly grasp basic concepts?


A: You could start with TUTORIAL.txt that is in the bitbucket.org repo and follow the links there.


Q: Is there a description of the various examples included with the Unity ARKit Plugin?

A: See SCENES.txt included with the plugin distribution.

Q: I’m trying to get the included project to load, but I get compile errors like these: “Assets/Plugins/iOS/UnityARKit/UnityARVideo.cs(59,31): error CS0117: `UnityEngine.TextureFormat' does not contain a definition for `R8’
Assets/Plugins/iOS/UnityARKit/UnityARVideo.cs(66,31): error CS0117: `UnityEngine.TextureFormat' does not contain a definition for `RG16'”
What have I done wrong?


A: You need to upgrade your Unity version. The minimum Unity versions are:

Unty 5 - version 5.6.2 or newer.

Unity 2017 - version 2017.1.0f3 or newer


Q: When running, I get this error message in XCode console:


“2017-06-05 17:22:43.726156-0700 ProductName[757:217524] [Session] Unable to run the session, configuration is not supported on this device: <ARWorldTrackingSessionConfiguration: 0x1c429b7b0 planeDetection=Horizontal worldAlignment=Gravity lightEstimation=Enabled>

2017-06-05 17:22:43.788207-0700 ProductName[757:217524] AR FAIL”


What is wrong?


A: Your device must have a minimum of an A9 processor (so iPhone 6s and later, iPad (2017) or later). If in doubt, check your processor in this table


Q: But the documentation says that there is a lower level of AR (3DOF) that should work for devices older than iPhone 6S. But it doesn’t work when I try it?


A: There was a little note on iOS 11 beta release notes: “ARSessionConfiguration is only supported on iOS devices with an A9 or later processor and will be enabled for all iOS 11 devices in a future beta.”


Q: Cannot see the content of assets folder in project inspector. It seems that unity cannot read assets folder and library folder. How to solve it?


A: You may have upgraded to macOS 10.13 and chosen to use the APFS filesystem. There is currently an incompatibility between Unity and APFS filesystem. Please check this link for updates.


Q: Will there also be support for the new Vision framework in iOS 11? How about CoreML?

A: This plugin exposes only the ARKit functionality that is currently available.


Q: If I want to use front facing Camera, for tracking is it possible to do that using the ARKit?

A: This is a question for Apple, but the answer currently is that ARKit only works with rear facing camera.


Q: How do I record a video of my awesome ARKit app?

A: iOS 11 has functionality to record video of your screen. See this link.


Q: Can I use Unity ARKit plugin with Unity Vuforia support?

A: Currently, not simultaneously. Both of them require exclusive use of the camera that is on the device, so when one is active, the other cannot be.


Q: I get a linker error like "Geo::GetiOSAppDocumentsDir()", referenced from:” . How can I fix it?

A: If you see this, make sure the Build Setting in Xcode for Dead Code Stripping is set to Yes


Q: I have an app that I want to support ARKit with, but I also want to support devices that do not have ARKit support. How do I do that?

A: Create an ARKitWorldTackingSessionConfiguration just as you would normally, but before using it to RunWithConfig, query IsSupported on that struct. Then you could decide on a path through your game based on the result of that query.


Q: I have created a gameobject in the scene, but it seems to move with the device camera movement. I cannot seem to get any closer to it.

A: This could be because you created it really far away and with a large scale, and it is really moving closer to you, but you don’t notice it due to perspective. It could also be that you have started ARKit with 3DOF, which will allow you to rotate, but not move in the world.


Q: I got the plugin, and when I export it to my iOS device, it gives me a blank screen with “Waiting for connection..” on it.

A: Go to your Build Settings and specify the “UnityARKitScene” as the scene to export.


Q: The resolution of the video image I get on my app seems lower than that produced by the camera app on the same device?

A: That is correct, ARKit uses a lower resolution camera feed that we upscale to fit your screen. It’s possible they do this to speed up all the processing needed for AR.

Q: I'm trying to get the Unity ARKit Remote to work, but it doesn't succeed in connecting Editor to Remote with messages like "Failed to connect"?

A: Read and follow the instructions carefully. The most common issue is forgetting to make the Remote app use "Development Build" in step 2 of the instructions. Another common problem is people don't realize they need to only build one scene to device: UnityARKitRemote.scene, then the rest of the scenes they can play from editor.


Q: I got the plugin, and I have the correct scene, but when I build and run on my device, it gives me a blue screen.

A: This could happen if you have a device that is not supported (e.g not A9 or better cpu), or your provisioning profile or signing credentials are not setup correctly in XCode. See the XCode console log for clues on what it might be.

Q: Can I use the Stimulator when I build in XCode or I have to connect with a real iphone with iOS11?

A: Simulator does not support ARKit at this time - please use a real device.

Q: When I use an iPhone X, there are sections of the screen on bottom and top with messed up video showing - how do I fix it?
A: This was a bug that was fixed in the plugin Oct 3, 2017. Please get version 1.0.10 or later of the plugin. Version 1.0.10 can be downloaded as a zip from https://bitbucket.org/Unity-Technologies/unity-arkit-plugin/get/1.0.10.zip

Q: When I try to run my ARKit app it crashes with the following log message "This app has crashed because it attempted to access privacy-sensitive data without a usage description. The app's Info.plist must contain an NSCameraUsageDescription key with a string value explaining to the user how the app uses this data." What's the problem?

You have probably changed the ProjectSettings asset distributed with the plugin and removed the PlayerSettings field for "Camera Usage Description" for the build. You need to put in some string in this field for ARKit to work.

Q: When using ARKit Face Tracking, my app crashes right at the beginning with "Terminating with uncaught exception of type Il2CppExceptionWrapper" . What could be the problem?

This usually happens if you have not enabled FaceTracking in the settings and you try to run a face tracking configuration.

Q: I have read everything above, but this plugin is not working as expected. What should I do?

A: Most issues have been solved already and there are hundreds of developers using this plugin to develop apps, so more than likely, your problem is one of the ones in this FAQ, or someone has posted on the forums about it already. If you think you have a legitimate new issue and want to post a question about it, please include the following information:

Xcode logs attached in a txt file (not screenshots of Xcode)

Name of Scene built with unity

Device info (iPhone 7, iPad Pro gen 2, etc.)

iOS beta version

Xcode beta version

Plugin version and whether from bitbucket or asset store

 

iPad (2017) or later is incorrect. Because the first gen of ipad pros (late 2015 and 2016) have A9X chips and so work with ARKit.

 

Limited Tracking:

• It could also be that you have started ARKit with 3DOF, which will allow you to rotate, but not move in the world.
• 3DOF is a configuration, or is this selected by the Plugin?

Reduce Drift:
To reduce Drift of Scene Placed 3d objects is the following the recommended workflow:
1. Config:

• Unity AR Alignment Gravity [ I've found my scene opened horizontally without gravity setting ] and / or heading

2. AnchorPoints

• do they assist in stablising drift of objects in a scene
• should an AnchorPoint be added every metre to build a map space in Apple

3. Adding 3D object

• should 3D object be added to the anchor, or is an object part of the unity 3d coordinates and hence the anchor is irrelevant.

4. Change Config and Run WithConfig to "disable plane detection" as per :

• UnityARPlaneDetection planeDetection = UnityARPlaneDetection.None;
• m_session.RunWithConfig ()
• should this be done once the initial object placement has been completed or after all objects have been placed.

5. Apple:

• ARCamera class provides tracking state reason information, which you can use to develop UI that tells a user how to resolve low-quality tracking situations
• any thoughts on how to best achieve this.

6. Any other thoughts or comments on stabilisation?

 

Can you comment on the following:

• https://medium.com/@dawson.mark/hi-...m-a-shared-3d-co-ordinate-system-7fde757c857f

When the plane updates this transform value remains the same but the planes center point changes from 0,0,0 to some other values like maybe X1, 0 ,Z1 — so the plane location is a combination of those two values. The center point always has Y == 0, the Y value of the plane comes from the ARPlaneAnchor transform values.

So this means the initial Height of Y never changes; it will always be on the same plane height.

The reason i ask is i often find the ARPlaneAnchor's are located too high.

In the editor i have changed the Y location and found the whole scene is scaled rather than moved. To move 1M in real world space requires approximately 20M in Unity space.

 

The planes generated in plugin are based on the prefab that is in the plugin. Don't just look at the top level transform, but one level down, where the updates might change the local position of that node or scale or rotation based on updates from ARKit.

 

Is it possible use the same ARKit camera for 2 different scenes? I mean, instead of changing from one scene to another and reset all what ArKit detected , keep the same camera and all data from one scene to another one. Doing this I could avoid wait the time to detect the surfaces??

Thanks

 

Yes you should be able to do that. You can put all the ARKit related stuff under a "Root" GameObject which sets the "DontDestroyOnLoad" flag, and you can also put this in a separate scene. When you load the next scene, anything not under the Root will get destroyed, but the ARKit related stuff should still exist.

 

Is there any good reason why the ARKitRemote only works once and then at every new play it isn't working anymore?

 

Trying out the new face tracking stuff on iPhone X and it works really well. What we would love is if you can run ARKit with a scene in front of you in AR but still have the back camera FaceTracking working as an input to read your facial expression and the lighting. Does anyone know if that is possible? I don't see anything in the plugin code dealing with turning on the camera modes.

 

That is not currently possible - the WorldTrackingConfiguration uses the back camera exclusively, and the FaceTrackingConfiguration front camera exclusively (and only on iPhone X).

 

Do you still need to use iOS 11 Beta 7 download to get it to work, or will it work on the latest 11 release? My ipad has been lagging something fierce, but I'm not sure if it's due to not having the beta. And, I can't find where to download the beta anywhere! It's not listed as such in the apple developer download section, so if that is the magic answer to my lag problems, please point me to where I can actually get iOS 11 beta 7. If that's not the issue, what is the deal with the lag and where do I need to start looking for answers? Thanks in advance!

 

Latest version of ios 11 should work better than any Beta. Could you describe more on what you're trying to do and what your lag is?

 

Why was the bitbucket repo deleted?

 

It wasn't - are you not able to see it? https://bitbucket.org/Unity-Technologies/unity-arkit-plugin/overview

 

That was weird. The links work now. It used to say it was deleted in the history. None of the links worked before. Thanks, I am glad is there now.

 

Is it possible to change the scene when colliding with a model? I already tried marking the model as a trigger but it didn't work

Thanks

 

If I place environment and look closely, environment repositions itself with the device movement. There is a lot of this repositioning if the camera is too close.

Thanks.

 

Is that supposed to be working? To clean up scene hierarchy, i parented all the ARKit related objects and on builds, the app crashes at start. Fixed when they're all left in root of the scene.

 

Hi there,

is the integration of Apple's Core ML planned?
If not (or not in near future), could you list or compare the features of the Unity ML agents vs. the Core ML features? Like ease of using trained models for image or object recognition, etc.

Thanks

 

I

I'm wondering the same thing too.

 

Hello,
I want to ask about arkit run on Ipad,
While I develop on ipad(ipad Pro), the ar camera zoomed in and the resolution are very low, is there any solution to check the camera resolution and adjust?
Thank you

I stand at the same position to take the photo.

 

Any simple method to know if arkit camera tracking (like AR thing) got initialized and is working!

 

I would like to add a load screen until AR is working!

 

Hi Zyshaq,

For an example, take a look at UnityARCameraManager.cs:
https://bitbucket.org/Unity-Technol...OS/UnityARKit/Helpers/UnityARCameraManager.cs

In the UnityARCameraManager.Start() method, you should see the following code:

Code (CSharp):

1.         if (config.IsSupported) {
2.             m_session.RunWithConfig (config);
3.             UnityARSessionNativeInterface.ARFrameUpdatedEvent += FirstFrameUpdate;
4.         }

which adds an event listener to the ARFrameUpdatedEvent event. Then, in the UnityARCameraManager.FirstFrameUpdate() method, this event listener is removed because the session has processed its first frame.

Todd

 

ARKit 1.5 vs 2.0.
I need to build for iOS 11. Can I use Unity ARKit 2.0 but limit my features to what iOS 11 supports, or do I need to use ARKit 1.5?
Is ARKit 1.5 still being maintained for developers building to iOS 11?
I would use AR Foundation, but I need image targets. This is a cross platform app.

 

Could anyone use ARKIT Remote without errors? I cannot even connect an iOS device to Unity IDE 2019.1 with ARKIT Plugin v2.0.

Thanks in advance.