Resolved ARKit support for iOS via Unity-ARKit-Plugin

Thank you - we like to see people putting this to good use. Overhead is pretty low: the video texture is actually made up of two textures (Y and UV), and they both exist in native Metal land - they never get transferred to C#, but instead we use the native pointers to the textures to create unity wrappers around them, and they get fed into a YUV shader which combines them (again on the GPU), to render as the video background you see.

 

Want to use 3D scanned video on your ARKit app? Give them 5 mins to show you how:

 

Hi All,

I've been enjoying working with ARkit so far and I'm curious if there is a way to enable geolocation functionality. Ideally I'd like to place 3D objects/models at specific GPS Coordinates. Is this possible?

Thanks!

 

Hi all, I really need help. In the process of working with arkit I encountered the following questions: what WorldConfiguration really does. In the test scene
When changing parameters, absolutely nothing changes, but I would like to see the binding of the scene or objects to the real direction (azimuth). How can this be realized so that the objects in my scene orient themselves around the world? And how to arrange the objects in space, I mean how to arrange the object code for example 10 meters north of the starting position? thank you in advance)

unity 5.6.2f1 , iphone 6s, xcode beta

 

Hi Jimmya,

May I ask you another question regarding units? Is there a way to change Unity scene units conversion to real world distance in AR? We're trying to add ARKit to our app which relies on quite complex world matrix calculations and 1 unit is our basic size which we defined like this a year ago for simplicity. The thing is that 'our' 1 unit should be visible as something like 0.1m in the real (AR) world and now all our objects render 10x bigger than we want them to. Is there a way to somehow manipulate this unit correlation, so we could remain our scenes unchanged but render them in AR smaller? Or maybe there's a another camera trick which comes to your mind to achieve the same effect?

Any help would be appreciated
Thanks!
Olaf

 

Hi!
I have a crash when I try to switch between AR and simple scenes. Crash occurs only when I moving my phone ( in the moment when tracking work and creating new anchor point or plane. If the plane was detected and phone not moving -everything is fine).

My resume and pause methods: https://pastebin.com/DBD0W0ai
My DontDestroyOnLoad objects: https://gyazo.com/3468a386f52c56637ac2c355da20291f
CrashLog: https://pastebin.com/03ReangP
https://gyazo.com/4d6ddeac569e1e3d40988b01ac272af4
StackTrace: https://pastebin.com/1jUB2pMd

 

I have been trying to use Apple's ARKit and it seems that when I build anything, various tutorials for example, my augmentation doesn't seem to be locked down to anything. For lack of a better explanation, it seems like the object is floating around. The object often appears to be slightly above the surface as well. Any suggestions why this might be happening?

 

Have a look at "gravityAndHeading" at https://developer.apple.com/documentation/arkit/arsessionconfiguration.worldalignment to see if it solves your problem.

 

Check answer above, and use https://docs.unity3d.com/ScriptReference/LocationService.html (maybe?).

 

Could be your GeneratePlanes (which keeps track of anchors from outside ARKit) and ARKit itself is out of sync.
[edit: I could maybe explain it better. The UnityARGeneratePlane script uses UnityAnchorManager to create a unity GameObject for each anchor that ARKit reports that it found. So when you reset ARKit, you should take care to remove the existing planeanchor gameobjects (otherwise it could be referencing a planeanchor that does not exist any more). Not sure this is your exact problem, but something along those lines.]

 

Thanks to help from @jimmya

This script works to reload/retrack the objects in AR with a UI onclick button.

Code (CSharp):

1. using UnityEngine;
2. using UnityEngine.UI;
3. using System.Collections;
4. using UnityEngine.XR.iOS;
5.  
6. public class Reload : MonoBehaviour
7. {
8.     public void ReTrack()
9.     {
10.         ARKitWorldTackingSessionConfiguration sessionConfig = new ARKitWorldTackingSessionConfiguration ( UnityARAlignment.UnityARAlignmentGravity, UnityARPlaneDetection.Horizontal);
11.         UnityARSessionNativeInterface.GetARSessionNativeInterface ().RunWithConfigAndOptions (sessionConfig, UnityARSessionRunOption.ARSessionRunOptionRemoveExistingAnchors | UnityARSessionRunOption.ARSessionRunOptionResetTracking);
12.     }
13. }

 

@jimmya do you all have anything on the road map for incorporating the Vision Framework APIs into this plugin (or in a future plugin)? There's a ton of great use cases for object and marker tracking in tandem with ARKit's functionality, so I'd love to see a package that includes both

 

One simple way is to have a "root transform gameobject" under which you create all your gameobjects. Then set the scale for the transform on the root gameobject to be 0.1,0.1, 0.1

 

I tried running the UnityARKit scene from the unitypackage bundle in Unity 5.6.1p, and built it on xcode 9 beta. The app insta-crashed because it throws the exception UnrecognizedARTrackingStateReason. Any ideas?

 

I believe this may have to do with our plugin needing a fix to work with latest beta iOS and XCode released today. Please bear with us while we spin a fix, or keep with older iOS beta /XCode9 beta.

 

There's no way to downgrade from iOS 11 beta 3 to beta 2 that I know of, but I can downgrade my Xcode version. Would that be sufficient?

 

Just as an update, if anybody else is in this situation, downgrading to Xcode 9 Beta 2 did in fact allow the scene to run.

 

Hi Jimmya, this is not possible if the objects are static, like in a typical scenario.

 

I need to place the object to the scene and store its coordinates. But when I start the application again (the device has the same position and rotation) and I load the object with the stored coordinates so the object is 1m beside to the original position. One in five attempts is OK, but it is not reliable solution.

 

Yes, I already studied this type of positioning, but as I said, when you start the finished examples in arkit and configure this type of positioning nothing changes. Maybe you need to implement objects through a special coordinate system? In the example there is a collision check in the space of arkit and there was a code for the location of the objects, but how to adapt it to simply place the object in their coordinate system through Vector3. Any suggestions?

 

Hi Jimmya,

Unfortunately we can't do this because all our world coordinate matrix calculations won't work. Are there any other solutions which come to your mind?

Thanks,
Olaf

 

Thank you, but app falls exactly when I call
ArKitCameraParent.I.ToggleCamera(false); m_session.Pause(); + Moving device
Not when I reset ARKit

 

Hi!

Is it fake or real vertical tracking ? I think he is just draws in the air

 

Tell me please, How can I make a geolocation target?

 

I tried to start making a waypoint app but it was simply unusable... I think you need to use something like a geolocation.

Anyone have any idea how they did this? Although I guess it could have been alot of smoke and mirrors

 

Hi! Thanks for your plugin! But i have a impotant question.
I need to create a hole in the floor, about the same as in the portal example. Upon completion of the output of the main camera, I fill the depth buffer and then output the geometry of the hole by cutting the buffer. Then I take out the geometry under the floor. This technique works fine in the Unity3d but does not work on the device, the buffer does not fill out.

 

Hey!

I'm currently trying to figure out how to make the most accurate object placement on the floor. Im currently building off the UnityARKitScene with both Generate Planes and PointClouds. Currently i'm using the Generate Planes for mapping out the shadows, and also as a floor reference. When i add my custom model as a child of the HitCubeParent it usually resides to gliding a bit ontop of the surface instead of connecting with the shadow on the floor, which kinda ruins the object placement illusion. My current fix is to reach the transform from the raw instance of the floor and setting my transform to the same y value as it. As the data pretty much ends up unprocessed it leads up to the object readjusting everytime it updates the generate planes instance, or everytime the plane reaches a table or higher surface.

What´s your tip for the most accurate placement of objects on top of surfaces?

Thanks in beforehand ,
Elliot!

 

Hey guys,

Cheers for all the info. I'm trying to see the potential of ARKit for iOS as an alternative platform for our HoloLens-based spatial navigation system. What interests me a lot is whether Anchors will have any form of persistence. Or perhaps I will be able to save the AR session? What I'm practically interested in is ARKit's equivalents of WorldAnchorStore (for saving Anchors per app) and WorldAnchorTransferBatch for saving anchors into a binary file to pass it around and especially to re-use it on other iOS devices so they can pick up the anchors and stabilize them against own tracking data.

Shot!

 

Thanks for the plugin, I managed to put this prototype for a few things I will be working on once ARKit is released:

A few questions:

• Will the focus be fixed? Are we going to have auto focus available for ARKit?
• Will this plugin be incorporated to Unity eventually?
• Will ARKit support back and front cameras?

I know some may be Apple question, but your opinion would be nice.

Thanks again for this plugin.

 

I ran into the same issue. I finally got things working by changing the collider on the Plane from a MeshCollider to a BoxCollider. I'm not clear on why this fixed the issue.

A few things I noticed:

1. As you mentioned collisions work fine in the editor
2. Collisions work fine on the device if the prefab is created in the editor
3. Collisions aren't hit on the device for planes that come from the AnchorManager

Did you ever find a solution?

 

Thanks a lot for the awesome support guys!

 

Hey team!

Super happy to have this product. What's the best way to get what the camera sees as an image? I need to do some additional processing on the camera image.

WebCamTexture doesn't work, because I assume there can be only one way to access the camera. I've been trying to get the "_textureY" Texture from the video's 'm_ClearMaterial', but I keep getting the error "Unable to retreive image reference."

What's the best way to get this? Either RGB or a grayscale image would be appropriate.

 

For the focus, I have read in this forum that the focus is fixed around 1 meter, this explains why the plugin detects planes when you point the camera to the floor that is around 1 meter away, but it will have difficulties finding planes from far away things because those things look blury and it is not autofocusing.

If ARKit would support autofocus but by changing the focus then the camera intrinsics and projection matrix would change too, so ARKit would need to adapt to this change I guess.

Do you know anything about this?

 

This question has been answered before on this forum. Summary is : it currently does not do this, and if there is any change, Apple would be the one to ask about it.

 

Check this post.

 

hey!
I am getting an error while running an app. It automatically shuts down after some time of inactivity throwing this message.

Terminating app due to uncaught exception 'UnrecognizedARTrackingStateReason', reason: 'Unrecognized ARTrackingStateReason: 3'

*** First throw call stack:

(0x187507e58 0x186a18528 0x187507da0 0x10221b864 0x1a1d5c780 0x186e89e04 0x186e89dc4 0x186e96f6c 0x1874b1cd8 0x1874af7f0 0x1873d1bf0 0x101ec1648 0x101ec16f0 0x1047d8cc0 0x18b4fa804 0x1877700fc 0x187498084 0x1874b26f0 0x1874b1e98 0x1874af8a8 0x1873d1bf0 0x18922bfac 0x190fa9ec4 0x101ebb48c 0x186ef61e0)

Can you help me out?
Thanks!

 

Hi,
I followed the above procedures but when i run the app on iphone7+ with iOS11..it crashed with the following error

-- Terminating app due to uncaught exception 'UnrecognizedARTrackingStateReason', reason: 'Unrecognized ARTrackingStateReason: 1' ---

any idea why?

 

Update your ARKit, you're building with XCode beta 3.

 

Which is the best Unity version to be building with? 2017.1 ok? I had issues the other day with Xcode latest beta and one of the newer Unity 5.6.2 patches.

 

XCode 9 Beta 3 caused our plugin to crash. There is a fix already up there in the bitbucket repo and asset store package. 2017.1 is the recommended version of Unity to use. Unity 5.6.2 works as well.

[edited to show that asset store package is now live]

 

AR kit can Support metal(Graphic API) rendering?

 

I can build the demo scenes to my studio's iPhone 7 plus, the scenes are always just blue, we can't seem to get the camera feed from the iPhone's camera. Where can I look for the solution?

 

We've been testing some camera effects, and since they're applied to the camera feed from ipad itself, it causes some issues, jittering movement, low fps, even though there isn't that much going on in the scene, about 15k verts, and still 5 fps when using just ambient occlusion.
Tested two different real-time post-processing assets.

Is it possible to get post-processing to work? Even color-grading was causing some weird flashes. Tested Post-Processing stack from asset store, and SSAO PRO.

 

Hello,

I have find at UnityARAnchorManager.cs the function
"public void AddAnchor(ARPlaneAnchor arPlaneAnchor)"

I want to add a anchor manuel at postion 0,0,0 and scale 1,1,1

But i don't know how to use this function.
I hope someone can help me.

 

Hi! What's the reason for converting the ARKit plane anchor ID to a string? Seems like this could produce really slow results when working with many planes.

 

Hi,
many thanks for a plugin, great stuff! I can't find any way to check if device supports AR Kit. Any advice where to dig for such function?