Resolved ARKit support for iOS via Unity-ARKit-Plugin

Hi, I am currently working on incorporating ARKit into my Unity project and generally it works fine. However, in my Unity project the y-axis is used for depth and the z-axis is used for height as opposed to the ARKit asset in which the y-axis represents real-world height and z-axis represents depth.

Is there a way to flip the y-axis and z-axis in the ARKit asset? Thanks

 

What's the recommended way to disable ARKit for re-enabling later, ie., for user switching from rear facing camera to front facing (and back again)

Also, is there a reason why front facing via unity webcamtexture and UnityARVideo aren't working?

 

For question number one, I would either look to see if Apple provide an API to get the battery consumption of the device or the current temperature, and if they do you could use that information to tactically display a warning. Otherwise just have it on a timer which asks for the users to stop playing after a while.

Shaders wise, I would say on mobile always go for mobile oriented shaders. I believe Unity integrates with Metal quite well, so this is something you wont have to worry about too much. Custom shaders are always an option to squeeze performance out and achieve the look your after.

 

Looks like it. All iPad Pro's I think are included (so ensure you have all of those listed) and all 2017 iPad's are included as well I believe.

You can force the app to only work on supported devices using the following technique:
https://developer.apple.com/documentation/arkit

 

I was wondering how do i receive ARCamera (not UnityArCamera) information. I need euler angles any idea ?

 

has it been updated yet

 

Thanks!

 

Thanks for the input!

 

There have been other suggestions, but one other thing you could try: you can revert to a lower cost tracking configuration once you have established surface to use e.g. do not detect planes anymore, or use 3DOF tracking. These will use lower amounts of energy.

 

I would not rely on listing all the supported devices in your code, but instead use ARKit framework's isSupported call on the worldtracking configuration to figure out whether this device is supported.

 

This is going to be a pain - we already had to figure out how to convert from ARKit's right handed coordinate system to Unity's left handed coordinate system, and it was not trivial. I would try and change your project to use Unity's coordinate system.

 

Front facing camera is not supported by ARKit at this time.

 

We dont send the euler angles directly from ARKit, but you can get it from the camera transforms rotation. https://docs.unity3d.com/ScriptReference/Quaternion-eulerAngles.html

 

Sorry, let me try to clarify my question.

ARKit works with rear facing camera. However, does this disable using front facing camera (webcamtexture) in the same ARKit app (of course, not using ARKit but using opencv or other tracking)? This was what I meant about disabling ARKit when user selects front facing and re-enabling it when user selects rear facing

 

a great feature request for 2017.2 would be a matrix that seamlessly converts left handed to right and back again, for easy compatibility with this and other SDKs

 

Hey all this might be a questions that has already been answered in a form or another but i'm getting a bit stumped by it currently. I'm currently working on adding markers into Arkit as a way to get the initial position of the where i specific item would be placed in a given scene and then have Arkit deal with keeping the object in the given location rather than constantly looking for the marker.

Currently I've got opencv's 2D marker texture tracking successfully getting the coordinates based on the image i'm feeding which is coming from the Arkit's supplied camera feed. When i go to place this an object at said location it pops up into the correct spot in the scene, but unlike the objects that I placed into the scene in the editor which tend to keep their relative position when I move around the objects that i'm placing at on demand tend to shift and drift all over when I walk around.

Is there something trivial i'm missing and or overlooking?

 

Or is there a way to pause and later resume the ARKit or Video from ARKit? I notice that OnResume() and such are not provided

 

I had my Unity ARKit iOS demo working great as of last week, but when I updated Unity to 2017.2.0b6 and Xcode to 9.0 Beta 5 (yesterday, August 8), my app now crashes every time. I did, of course, update the ARKit plugin to the latest.

I tried building a plain-vanilla version of the scene "UnityARKitScene" and it crashes with the same error (log attached). I also noticed that in Xcode, I'm seeing "libiconv.2.dylib" in RED in my project hierarchy. I don't think that was there before my update. [My Mac is running Sierra 10.12.6]

Is anyone else building ARKit demos with Unity 2017.2.0b6 with Xcode 9.0 Beta 5 successfully?

 

Thanks for supporting this effort - it's a little better than the days when this kind of stuff would be a volunteer-based community effort. But... a few days of downtime is a long time in arkit land...

Is there a way that you guys can work closer with Apple to have the Unity plugin be up to speed with their updates, especially when things would break?

Or, simply, auto-update everyone on the thread not to update to the latest iOS 11 update (yet)?

 

This question has been asked a few times on the forum. The short answer is that it isn't possible to relocalize the camera after pausing a session. I felt this was a bug, as the `resetTracking` option makes it sound as if you can resume tracking. @jimmya and now Apple's developer relations have told me this is by design.

 

Hi!

Any solution to scale a NavMesh?
I have an entire level to scale depending of the Anchor size, but do not success to access//scale the Static Nav Mesh for the ground generated from Editor....

More detail in a new thread I opened :

- https://forum.unity3d.com/threads/navigation-mesh-arkit-dynamic-scale.487473/

Thank you ! ^_^

 

Thanks for all you're doing Chris, I was also directly affected by the Beta 5 issue (I hadn't seen your note to avoid the upgrade). However, I just updated the plugin and tried building and running the UnityARKitScene scene, but I'm getting several errors in XCode beginning with these few in ARSessionNative.mm:

211: Unknown type name 'ARWorldTrackingConfiguration'; did you mean 'ARWorldTrackingSessionConfiguration'?
218: Unknown type name 'ARConfiguration'
255: No visible @interface for 'ARCamera' declares the selector 'projectionMatrixForOrientation:viewportSize:zNear:zFar:'
417: No visible @interface for 'ARFrame' declares the selector 'displayTransformForOrientation:viewportSize:'


Can you advise on whether this looks like something I'm not doing properly, or whether the updated plugin has bugs in it?

Thanks much

 

Hey y'all,

Here's a video I just recorded of what I've made so far. Comments & critique welcome — I'm a Unity beginner, I'll take all the feedback I can get.

Apologies for the lack of audio. YouTube & Google Photos seem to be stripping the audio from videos made with the new iOS 11 screen recording feature ¯\_(ツ)_/¯. I actually talked through the whole thing explaining what I'm doing, I would have made it shorter if I'd known it would be muted.

This is about a week's worth of work. I'm really happy with progress so far. I made the models in VR using Google's Blocks program. I'm happily surprised with how good they look with just 1 tweak in Unity. I got the pop particle effect working today, which was a really satisfying milestone.

The arrows are being shot when swiping the screen. Touching down and swiping "pulls back" the arrow. Releasing your finger releases the arrow. I'm working on the animations for that so it's visual.

A couple of my biggest questions right now are:

• How does Unity define y=0 for the AR scene? Is it the origin of the device when the AR world tracking starts? Is it the first anchor detected? I have every balloon being created at y=0, but many of them look like they appear at eye level. I'm misunderstanding something.
• Should I leave ARKit plane detection on after getting the first anchor? Does leaving it on decrease the chance of scene/localization drift? In the video I'm turning it off after getting the first anchor (using the NativeSessionInterface.runWithConfig, with the plane detection parameter as UnityARPlaneDetection.None). It seems to have a little more drift. I've read Apple's docs through several times but don't understand if the precision of world tracking is coupled with having plane detection on.
• I want to swap out the debug plane for a custom one. I want to program the plane so that when the arrows hit it, they stick in place, like an actual arrow. I'd like for the plane to receive shadows, appear infinite to the player, and occlude any virtual objects underneath it, i.e. you wouldn't be able to see the tip of an arrow stuck in the ground a few centimeters. How would I go about that? This is kind of related to my first question as well — would I assume that the first anchor is the ground?

Thank you in advance!

 

I know the machine learning API is outside the scope of AR kit, but it seems to me that the vision APIs don't rely on learning data? I think face and object tracking capabilities in Unity would be extremely useful in building out an AR app...

 

is there a way to pause and later resume the ARKit or Video from ARKit? I notice that OnResume() and onPause() and such are not provided

why? one example - to switch to front facing camera via webcamtexture. afterwards, being able to switch back to the arkit video stream.

 

For remote control: can I use Android device (Windows Unity). If not, can I use an older pre-iOS 11 device like iPad 2 (of course this time on OSX Unity).

Best!

 

Hi,

I think we found a bug. Last iOS 11 beta, xCode beta 5.
App Crashes, in device, when we unload or destroy the scene containing the ARKit. xCode log error:

libc++abi.dylib: terminating with uncaught exception of type Il2CppExceptionWrapper

How to test:
Have an ARKit scene (for example, the ARKit "EditorTestScene"), and add a UI button to load another scene (any scene, it can be empty). Run the app on device, the ARKit scene starts, press the button to load the new scene, and the app crashes.
It also crashes if we manually destroy the gameobjects containing the ARKit stuff.

See attached project.

As anyone encountered the same problem? It did not happen with xCode beta 4.

Thanks!

 

It seems the problem occurs when de component "UnityARAmbient" is destroyed.

 

Does anyone have a memory issues with Unity ARPlugin. It seems that the example project (UnityARKitScene) uses more and more memory and never releases it. I've debugged this in Xcode and Unity profiler and both show the same results.

Any ideas how to fix this?

 

what is visual inertial odometry ?
why use visual inertial odometry ?
how it works?

 

In the mean time the following will work:

Code (CSharp):

1. s_CurAffineTransform = [frame displayTransformForOrientation:[[UIApplication sharedApplication] statusBarOrientation] viewportSize:nativeSize];

Also you may have to comment out the following:

Code (CSharp):

1. [NSException raise:@"UnrecognizedARTrackingStateReason" format:@"Unrecognized ARTrackingStateReason: %ld", (long)trackingReason];

 

Did anyone figure out how to make the objects always appear in the same spot. it seems odd that this is such so complex, if I wanted to use it do pathfinding, multiplayer or whatever, is it better to use location services?

 

Prompt how to start the application without the need for connection to the editor. What wouldn't be at start of the message of "Waiting for editor connection"

 

Is this supposed to work with deferred rendering path?

 

There are ideas? That it was possible to depart from the computer far

 

Good idea - I guess the only reason I added the list was to support older / other devices with ARKit disabled. The game has AR mode and non-AR mode. Does that make sense? I have yet to test out this on demo devices.

 

I noticed that the Unity ARKit Plugin available on the Asset Store is submitted as a complete project. Why is this? It makes it a little bit more difficult to upgrade the plugin in my existing project. It tries to overwrite my projects settings every time I import it.

 

Upgraded to Xcode Beta 5/Unity ARKit Plugin for Beta 5 and now I am getting a crash on device as well as in the editor right when I enable AR. Here is the Xcode log:

2017-08-10 17:30:24.910858-0700 shotpro[558:114813] -[ARFrame displayTransformWithViewportSizerientation:]: unrecognized selector sent to instance 0x1c82fed80

2017-08-10 17:30:24.927467-0700 shotpro[558:114813] Uncaught exception: NSInvalidArgumentException: -[ARFrame displayTransformWithViewportSizerientation:]: unrecognized selector sent to instance 0x1c82fed80

(

0 CoreFoundation 0x0000000183530d74 <redacted> + 148

1 libobjc.A.dylib 0x0000000182a44528 objc_exception_throw + 56

2 CoreFoundation 0x000000018353e218 <redacted> + 0

3 CoreFoundation 0x0000000183536708 <redacted> + 1380

4 CoreFoundation 0x000000018341c35c _CF_forwarding_prep_0 + 92

5 shotpro 0x0000000103afa02c -[UnityARSession session:didUpdateFrame:] + 1168

6 ARKit 0x000000019e073b54 <redacted> + 132

7 libdispatch.dylib 0x0000000182eb546c <redacted> + 24

8 libdispatch.dylib 0x0000000182eb542c <redacted> + 16

9 libdispatch.dylib 0x0000000182ec1e70 <redacted> + 1016

10 CoreFoundation 0x00000001834d9138 <redacted> + 12

11 CoreFoundation 0x00000001834d6d14 <redacted> + 2012

12 CoreFoundation 0x00000001833f7558 CFRunLoopRunSpecific + 436

13 GraphicsServices 0x000000018527af84 GSEventRunModal + 100

14 UIKit 0x000000018d0c2984 UIApplicationMain + 208

15 shotpro 0x00000001028d50b4 main + 228

16 libdyld.dylib 0x0000000182f19db4 <redacted> + 4

)

 

Navigation (and other systems) start breaking at very small scales. You'll find some discussion earlier in this thread. @jimmya mentioned they may have a way to scale the camera instead of the world so we can preserve 1m = 1 unit while making the world appear much smaller. This is the same trick lots of people use in VR.

Anyone have any updates on scaling the camera with ARKit?