Resolved ARKit support for iOS via Unity-ARKit-Plugin

Please read the FAQ: https://forum.unity.com/threads/unity-arkit-plugin-faq.495191/
Also, see the TUTORIAL.TXT and SCENES.TXT in the package.

 

Hi!

With the new release of iPhone X I want to test my game on iPhone X simulator. So I changed on Unity Target SDK to Simulator SDK. I'm getting 5 errors when building on Xcode this way:





I just updated everything hoping the error was gone, no luck. Didn't find anything useful on google as well.

When I try to build to physical device (not simulator) everything works well.

Using:
Unity 5.6.4f1
Unity ARKit Plugin 1.0.10
Xcode 9.1

I run out of ideas, please help!

Thanks!

 

You can't use the simulator to test Unity iOS stuff. Been like that since the beginning. Mind I don't keep up to date on these things. But I know in the beginning you couldn't so ever since I don't try. If it works on your test devices and your UI is built to react to different screen sizes then it will more then likely work on iPhone X.

 

Yep I'd like to know this as well. I'm doing similar hacky thing. I'm counting points returned (as I'm not using planes) and enabling my placement tools once a threshold is reached.

So is there a better way of doing? How does the Ikea app do it?

Cheers.

 

The most reliable way would be to hook the ARSessionTrackingChangedEvent and get the camera.trackingState and camera.trackingReason when the event gets fired. See the values and what they mean here: https://developer.apple.com/documentation/arkit/artrackingstate?language=objc and https://developer.apple.com/documentation/arkit/artrackingstatereason?language=objc

 

Thanks for the reply. As usual I should've RTFM.

 

Is anyone able to record videos using the iOS screen recording while using ARKit on an iPad? iPhone works fine, but on iPad the image freezes as soon as the AR camera starts.

 

Same problem here

 

FYI - I think there is a bug in the ARSessionTrackingChangedEvent. I've added the following code:

private void ARSessionTrackingChanged(UnityARCamera arCamera) {
Debug.Log ("-----");
Debug.Log ("tracking state is : " + arCamera.trackingState);
Debug.Log ("tracking reason is :" + arCamera.trackingReason);
}

and in my console under good tracking conditions I'm seeing:

ARTrackingStateLimited with ARTrackingStateReasonNone

and if I cover up the camera I get:

ARTrackingStateNormal with ARTrackingStateReasonInsufficientFeatures

I do not see this behavior natively in ios. The reason seems to be accurate, but somehow the state is not.

 

Interesting problem: Last week we pulled the latest repo (hadn't updated since before the folder structure changes).

Since then a small number of our users have reported that even under perfect tracking conditions arkit is unable to pick up a surface no matter what they do. However each time, restarting their phone has fixed the problem (restarting the app did not). It happens with fresh installs too so I don't think it could be something in our code lingering from a previous install or previous run. We haven't been able to reproduce it.

Has anyone seen this happen? Any ideas what could cause this? Maybe some links broke in the Unity scene after moving the plugin scripts around? It seems to work perfectly fine for 97% of people.

 

hi, can you let me know what's the difference between:
UnityARSessionNativeInterface.ARAnchorAdded
UnityARSessionNativeInterface.ARUserAnchorAdded
UnityARSessionNativeInterface.internal_ARAnchorAdded
UnityARSessionNativeInterface.internal_ARUserAnchorAdded
thanks!

 

Hi!.. I have a problem with ARKITRemote cannot connect to iPhone and I try with 2 differents devices, iPhone 6s and 6 plus with the last ios version via USB.

I build the scene ARKITRemoteConnection in debug mode and Development Build checked. I deploy it into my device and open the app.

In unity I open the scene EditorTestScene I run the scene in the editor, in the console y select my device and I only get this error "Failed to connect to player"..

I am using the last version of ARKit plugin and Unity 2017.1 Personal edition. So I dont know what I am doing wrong. Sorry for my english. Thanks in advance.

 

Did you resolve this?

 

no, but since TrackingStateReason seems to be working, I am just switching on ARTrackingStateReason instead. Effectively ARTrackingStateReasonNone = ARTrackingStateNormal as far as I can tell.

 

Can I get the resolution of textureY and textureCbr ? Right now in UnityARVideo it is just getting the textureHandles and then creates a texture using Screen.currentResolution but I want to create a texture using the actual resolution of textureY and textureCbr in native code. Is there a straight forward way of getting this or I have to write some native code to get these values ?

 

iPhone 6 plus will not work (does not have A9).
You should be building the Assets/UnityARKitPlugin/ARKitRemote/UnityARKitRemote.scene to your device.

 

CreateExternalTexture does not create a new texture, but just creates a Texture2D wrapper around the native texture object, so the height and width of the native texture it points to does not change. The height and width passed in will be used to populate the metadata for the Texture2D wrapper. E.g. we should really be passing height/2 and width/2 for the CbCr texture

 

Thank you for the quick response Jimmya ! I might have been not so clear with my question. I get the point that you create a Texture2D wrapper for the metadata and maybe to fit the native texture to screen but all I want to know is the resolution of native texture. I only get the pointers to these textures, is there a way I can use these pointers to get their resolution ?

 

There is not a way via scripting. Why do you want to know it? You stated your reason was to create smaller textures, but as I said we do not create new textures, rather just use the already created native textures directly.

 

well for one, the internal ones are not supposed to be public. we need to fix that. they are actually the callbacks from the native code.

 

You're right it appears as if TrackingState is returning the wrong value. Let me investigate.
Update: found the bug - will push a fix to BitBucket.
Update: fixed - in this commit.

 

So I have AR working it's just that my object just floats it's not on a surface. I have the script on that detects the surfaces but how would I use it to put my environment on the surface.

 

Did you change the version of XCode 9 used (and therefore ARKit SDK)? What version of XCode are you using now, and what were you using before if so?

 

Thanks Jimmya for this plugin. Saves so much time

Got 2 questions, big thanks to anyone who knows the answers!

1. The UnityARKitScene example enables you to conjure a single object (cube) and even reposition it but tapping and dragging. What if you want to, with each new tap, add more objects (cubes) and even other objects (spheres, etc)? Where does on learn how to do this in unity?
2. Has anyone got real time capture and storage of 3D environments (via photogrammetry, etc) into Unity? As in you record an ARKitSession? (See "City Mapping" at 6:14 for eg into this Tango video?

PS If there's a better place to post such questions, please enlighten me
Thanks!

 

For question 1, you should learn how to use the basics of Unity first - do some tutorials, etc.
For question 2, that tech does not exist yet on ARKit

 

Hey, Jimmya. First of all, thanks for the great plugin, well documented and works excellent!

I'm new to Unity, so need a bit of a general advice here, but related to this particular plugin: how should I update it? At this time I see the flow as following: clone the Bitbucket repo somewhere, take the files I need and replace the ones used in my project (e.g., in the .../NativeInterface folder). Is that correct? Thanks in advance!

 

Does anyone know how to get Unity3D ARKit to go full-screen? I am getting the camera background video on the right and left hand sides. See attached screenshot. Thanks

 

I got the same on iPhone X, it's no problem on the other iPhones.

 

Yes, it is full-screen on my iPhone 6S

 

Hey Guys,

I have a few questions, so very quickly can someone tell me how I can efficiently stop arkit tracking, the only idea I have is to set config.detectplanes to something??.
Secondly, is it possible to detect feature points/planes inside a building all the way to a set destination (for example 20m in a straight line) and then afterwards stop arkit plane detection/feature point detection + stopping elements of arkit tracking, maybe? and just use the cameras transformation. However the twist is, that it would be in a scenario such as in smoke, when your navigating back to that destination (with arkit plane detection and feature point detection disabled etc). How would it work? Or is there another approach to have ARkit work in scenarios such as smoke and bad tracking quality. Thanks a lot to the community with supporting me and my project throughout my journey

 

UPDATE: Nevermind, Just rebuild it and didn't fix the warnings and that seem to work for some reason. But it seems when running the face tracking scenes it's using the wrong camera with the remote. How do I switch the camera?

Trying the remote and ARkit for the first time here. I have an iPhone X with xcode 9.1. It seems to connect fine but when I hit the start session button my iPhone closes the remote app. No errors, has anyone gotten this before? Thanks in advance.

Autoconnected Player</i> connected
UnityEngine.XR.iOS.ConnectToEditor:EditorConnected(Int32)
UnityEngine.Events.UnityEvent`1:Invoke(T0)

(Filename: /Users/builduser/buildslave/unity/build/artifacts/generated/common/runtime/DebugBindings.gen.cpp Line: 51)

 

Did you guys get the latest code from Bitbucket? I have tested on iPhone X and it was fine. There was a fix for that some time ago (Oct 3)

 

Hello. If I want to update the position of anchored object, do I need to clone the object each time or are there any anchor update method for me to call? Thx

 

Hey @jimmya, is it currently possible to apply the face expressions recorded by arkit on iphone x to a humanoid face?

 

Nope, I will try it. What is the best way to upgrade an existing app? Thanks!

 

I downloaded and installed the latest and it is now full-screen on the iPhone X. Thank you!

 

If not - I am looking itno making it happen. Any suggestions?

 

I was testing on the 9.1 beta at the time, then built for the store using the public 9.0 version. Maybe I used the "append" button in Unity and some files carried across from 9.1 to 9.0? If it's a matter of building the app with the wrong SDK, why would it affect some people and not others?

 

Yeah I'm not sure what's happening with this, that's why I'm asking questions - my theory is that there was a fix in a later version (maybe 9.1) of XCode that fixed an existing issue in the SDK, and you built with the older version of the SDK and you're still seeing it. AFAIK, there's nothing on Unity side that would affect just a small number of people.

 

There's a serious memory leak that's only showing on the iPhone when I run Unity's performance Profiler I can see in the Memory part that the textures as well as the Total object count continuously increasing with time, whether it's just the Remote ARkit app running nothing but the camera input forwarded to my Mac or the external built of my entire ARKit app, in both cases the memory is leaking, the textures and the Object count both go up to the 10s of thousands in less than a minute! It's hard to imagine that this would be somehow an inherent issue with just the camera rendering, more likely base code of ARkit and the way it renders the camera image, probably not cleaning up the rendertexture. It could also be related to iOS11.1 because I just updated to that version and I'm not sure if this already happened before.

 

Hi @christophergoy ,I encountered the following issues during my use of Arkit.

Undefined symbols for architecture arm64:
"_MTLCreateSystemDefaultDevice", referenced from:
-[UnityARSession setupMetal] in ARSessionNative.o
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

I've already included the Arkit.framework in the project. How can i solve this problem?

I use Unity 2017.2.0f3 ,
XCode 9.2 beta ,
and I'm using the latest version of the ARKit plugin.

 

Do you have the latest code for the plugin? There was a leak that was fixed earlier: https://bitbucket.org/Unity-Technol...mits/0db81286dee40ad34d3c9eb8072cd56f593b5734

 

Could be a deprecation in XCode 9.2 beta - could you check same project with XCode 9.1?

 

more problems in 9.1 , so i use 9.2

 

Hi jimmya，then I tested on XCode 9.0 （9A235），still got the same problems。

 

How do I change field of view of AR camera?

 

Hey guys,

So I have been using ARKit for a while now, everything works great. But now I have to use some tall objects (that don't fit in the camera view completely unless you move far back) in my scene. The issue is that as soon as the object's base in out of the view it starts drifting randomly.

Is this a bug or a limitation?

 

Is it possible have the same session of ARKit for different scenes?

My problem is that I don't want to wait again to detect the surfaces when I change of scene.

Thanks

 

You must use the same session for different scenes. Put everything related to ARkit under the same parent object and set it to don't destroy on load and you can use it in every scene you like

 

Hi, In my game AR Airplanes I experience stuttering motion of the gameobject:

I have experienced this before when a camera moving script would be running in Update(), while a GameObejct's controlscript would run in LateUpdate().

But with ARKit and my airplane, I can't find a problem like this (everything is on Update()).
I can't be due to bad tracking either, as stationary objects appear smooth during tracking.