Resolved ARKit support for iOS via Unity-ARKit-Plugin

Crash at start

Hello everyone! May be someone got same problem like me.

I created my game with ARKit plugin and started this game successful.
But after some time my game has can't start. Usually after 1 day or after 2 days.
I created another projects but behavior as in above example

And i see in setting of device that the camera permission was removed

iPhone 6s, iOs 11.2.5, Unity 2017.2.0.p2, xCode 9.2

Please help!

 

Have you integrated any unity project having ARKitPlugin into native ios?

 

I'm trying to get the image detection to work, but when i try to add the AR Resource Group to unity's create Image.xcassets, the group can't be found:
[General] No resource group with name "AR Resources" found

why is that? Does the xcassets have to be in a specific folder? It's included in the Copy Bundle Resources Build Phase.

Code (CSharp):

1.     NSSet<ARReferenceImage*>* refImages = [ARReferenceImage referenceImagesInGroupNamed:@"AR Resources" bundle:nil];
2.     appleConfig.detectionImages = refImages;
3.     for (ARReferenceImage* img in refImages) {
4.         NSLog(@"Reference Image: %@", img);
5.     }

Edit: Deployment Target was not set to 11.3, so the asset bundle compiler didn't include the AR Resource Group -__-

 

iPhone X face-tracking apps built with Unity freezes randomly maybe every 3-4 minutes. Any way to solve this? Someone else had reported the same problem in Bitbucket.

After that we get this kind of Xcode logs:

Execution of the command buffer was aborted due to an error during execution. Discarded (victim of GPU error/recovery) (IOAF code 5)

 

I bought the suburban house asset off the store. The house will scoot away from me. How can I make the object settle?

 

How are you handling the field of view? The film gate i am seeing is custom and I dont know how to replicate the ratio. I am trying to have stereoscopic view and this is throwing it off. I dont want to use render textures.

 

You get the camera projection matrix from ARKit: https://developer.apple.com/documen...tionmatrix?changes=latest_minor&language=objc

 

I created a sample scene for image detection, if you want to test that:
https://bitbucket.org/mrmatthias/unity-arkit-plugin-image-recognition

Drag the ARResources.xcassets folder from the ImageRecognition example folder to Xcode after building.
I disabled the updating of the ARImageAnchors because the positions/rotations didn't make sense.

 

Thank you for this! The demo is working for me and its super useful.

 

Oh that was super easy. Thanks for the info jimmya, I appreciate your work on this.

 

Awesome work jimmya and co!

https://blogs.unity3d.com/2018/02/16/developing-for-arkit-1-5-update-using-unity-arkit-plugin/

 

The method that we implemented uses a more flexible way of adding images and image sets to the level rather than messing with the XCode project: https://blogs.unity3d.com/2018/02/16/developing-for-arkit-1-5-update-using-unity-arkit-plugin/

 

The problem is that the AR camera is driven through the scene based on the real world position and rotation of the device you hold. So you may be able to push the other virtual objects around, but the position of the camera in the scene will still be relative to the device.

 

Just an FYI and a tip for everyone trying out image detection.

As of right now ARKit only tracks the position and orientation of an image anchor for the first frame it's detected.

It appears that Apple's intended use case is a statically placed image that spawns the AR scene relative to the detected image, as opposed to continuously tracking a la Vuforia.

Fortunately you can reset the tracking for an image anchor by removing it from the native ARKit interface. The Unity ARKit plugin has a built in function for removing user anchors, and it also works for image anchors.

Code (CSharp):

1. UnityARSessionNativeInterface.GetARSessionNativeInterface().RemoveUserAnchor(arImageAnchor.identifier);

This is probably a bad idea, but I decided to try it out and see how much of a bad idea it is. If you remove the anchor the frame after it's detected, this is about how fast ARKit can "continuously" track an image.

 

This is something that Apple specifically talk about: see the last section of this page: https://developer.apple.com/documentation/arkit/recognizing_images_in_an_ar_experience

Anyways, nice to see you got it up and running

 

new plugin works great wondering if you see any capability to load new images for recognition after a publish to device, say as an asset bundle?

 

On this page it says the ARKit API allows target images to be created programmatically using a CVPixelBuffer:
https://developer.apple.com/documentation/arkit/arworldtrackingconfiguration/2941063-detectionimages

But elsewhere in the documentation it says images have to be preprocessed. I don't think it's clear if it's XCode that is doing the preprocessing, or if it can be as easy as just passing a native texture pointer and image size during runtime.

 

It might be possible, but I haven't worked out the details. Might be able to use something like https://developer.apple.com/documentation/arkit/arreferenceimage/2942252-init

 

Check with ios 11.3 beta - if it's still happening, submit a bug report to http://bugreport.apple.com/ and post the report link here.

 

I'm using ARInterface on an iOS app, but I can't get a value for ARTrackingState. It returns that it's not available.
How do I get it?
And is ARInterface maintained? And/or are there other cross platform (ARKit/ARCore) solutions?

 

Hi,

I've just updated to 1.5 and now get this error message ..

UnityEngine.XR.iOS.UnityARCamera' does not contain a definition for `lightEstimation'

Can someone please tell me why, thanks ..

 

Can you give more details? Is this in the editor? If so, which file is complaining about this? Could you capture more of the messages around the error? It's more than likely that you have some code in your project that is referencing a much older version of the plugin.

 

Hi,

Sorry it was my mistake, I'm using the UnityARInterface and didn't update it, found your link to the latest version and all seem well again, thanks.

 

I'm getting the same error with 2017.2.1f1. Just found out in release notes for 2017.3 this: "Fixed various issues in PlayerConnection and EditorConnection, player connect and disconnect should work properly now." Now I hope this is my issue.

Dunno about face tracking data but you need to have "ARKitRemoteConnection" prefab in the scene to connect. However, on 2017.2.1f1 it gets me to above error.

 

Had the same error on 2017.3.1. Went back to manual and realized that I missed one step. Turn on "Development Build" but I also turned on "Autoconnect Profiler" before hitting "Build & Run".

No more connection error! All working now!

 

hey @jimmya ,thanks for the great work! Is there a reason for discontinuing support for Unity 5 with ARKit 1.5, or is it just discarding old versions of Unity? I ask because I would love to update the game with latest ARKit version, however I do not want to move the project to the new version of Unity at this moment, as I should produce this update very quickly.

 

Hey is anyone using the new bitbucket update? The script UnityARBuildPostprocessor - I have to add :
Using UnityEditor.iOS.Xcode;

Also I cannot get the old UnityArkitRemote app to work with the new update. When trying to connect my phone to unity with remoteConnection i get: Invalid log type 1769104723.

I also cannot build through Xcode with the new update. Getting many errors repeatedly showing undeclared identifier and unknown name type, basically for anything along the lines starting with AR - i.e. ARReferenceImage
I have tried to build both the UnityARKitRemote and UnityArVideoFormats; getting same errors.

I am using Unity 2017.3.1.1f1

----
Am I missing something causing these errors?

 

I have the same errors after using bitbucket spring apple branch.

I use
-2017.3.1.1f1
-mac os 10.13.3
-xcode 9.3 beta2

 

can I know how did you place it on top of the table without changing its position and without touching the screen? thank you

 

Hi Jimmy - can you also put in release notes which versions of Unity you have tested the latest unitypackage to work with? There are now many versions of even just Unity 2017 and even a ton of patch versions. Have you tested it to work with 2017.3.1f1?

 

how did you stop changing plain place after detection? and how you place the object without touch the screen?

 

Hi there -
We were hoping to use the truedepth front facing camera to snap a grey-scale depth picture (see attached)
Haven't been able to figure it out from the mesh data in the facetracking example
Any directions you would recommend?
Thanks so much!

 

hi @jimmya

UnityArkitRemote scene is empty in the latest branch(spring). Can you check it please?

 

It hasn't changed since last December: https://bitbucket.org/Unity-Technol...yARKitPlugin/ARKitRemote?at=spring2018_update

 

I have a problem with testing vertical plane detection. I'm using "spring2018_update" branch.
- Unity2017.3.0f3
- Xcode 9.3 Beta
- MacOS 10.13(High Sierra)
- iOS 11.2.6 + iPhone 6s



When I test on device, app crashes and I can see this error in Xcode. Anyone have an idea about this issue?
Do I have to update iOS version as 11.3(beta)?

 

@believedream817 Yes you need to use 11.3 beta to run the spring update branch.

 

does anyone have a favorite plugin in the asset store to use touch for pinch to resize / move. Ive tried a few like "Lean" and I cant seem to get them to work with my single 3d asset

 

i am using UnityARimageAnchor from spring2018_update. why object not spawn on the center of marker?

 

I just started with ARKit and I have recently updated to 2017.3 from 2017.2.

I updated because Point Cloud Particle Example wasn't loading. Updating Unity also was a mistake.

I get error in every frame when debugging with XCode

IndexOutOfRangeException: Array index is out of range.

at PointCloudParticleExample.Update ()

Edit: I found out the more error info after very hard run with ARKitRemote

IndexOutOfRangeException: Array index is out of range.
PointCloudParticleExample.Update () (at Assets/UnityARKitPlugin/Plugins/iOS/UnityARKit/Helpers/PointCloudParticleExample.cs:36)


After I updated Unity to 2017.3 the ARKit remote is not working. I keep getting

Screen position out of view frustum (screen pos 453.000000, 0.000000, 1000.000000) (Camera rect 0 0 453 806)

When I run the project which was working with 2017.2

I did reimport all also but still same.

 

Did you specify the size of the marker correctly? That also determines how it detects the marker; see https://developer.apple.com/documentation/arkit/recognizing_images_in_an_ar_experience

"Enter the physical size of the image in Xcode as accurately as possible. ARKit relies on this information to determine the distance of the image from the camera. Entering an incorrect physical size will result in an ARImageAnchor that’s the wrong distance from the camera."

In our case, we specify the physical size on the ARReferenceImage.asset

 

Yes thank you jimmya

 

Is your Menu Scene having camera usage?

 

Hi, I've searched for the problem I'm facing but haven't managed to find anything regarding it in this thread. Basically I want to augment the same virtual reality for two devices in the same real world space.

• I have each device detecting a plane
• When I click on the plane that is detected I switch scene to a network lobby.
• Once both players are ready I switch to the GameScene.

I manage to save a plane for each device (locally) but what I really want to do is to sync them up in someway. I have watched the "So you think you can augment reality?"-talk and want to do something similar in ARKit without using ARInterface.

 

I just found out that in order to block users with devices < iPhone6S from downloading an ARApp on the AppStore I need to set an flag with value "arkit" in "Required device capabilities" in the Info.plist file in XCode. Is there a way to do that in Unity directly? Would be much appreciated, because otherwise I'm kinda scared I will overlook that flag again, which then leads to users with wrong devices downloading the App and experiencing crashes.

 

It wouldnt pass app store certification if you didnt tick the flag anyway.

 

https://forum.unity.com/threads/ark...nity-arkit-plugin.474385/page-43#post-3297582

 

arkit1.5 when publish to asset store

 

Not until it is released - you can get the plugin as a zip from https://bitbucket.org/Unity-Technologies/unity-arkit-plugin/get/spring2018_update.zip

 

It already did pass, and is available on the Appstore. That's how I found out about that flag in the first place. The thing is that there are Apps which use ARKit as an Add-on rather than as a core functionality, so users should still be able to download such an App even if their phone doesn't support ARKit. But for fully ARKit-based Apps the App just crashes as soon as ARKit tries to initialise.

 

As mentioned elsewhere, you can support ARKit mode in an app that runs on non-ARKit supported phones by making sure you first check the ARKitWorldTrackingSessionConfiguration.IsSupported() before you initialize ARKit.