Resolved ARKit support for iOS via Unity-ARKit-Plugin

I met this problem when I was packing arkit out. May I ask what's wrong with me?

 

Hi guys! Many thanks for plugin and good support!

I'm stucked with problem of small objects physics & navigation mesh. I want to make my level and all objects that small so it fits the table, but I see no way to generate navigation mesh from square-like level of size ~ 1m x 1m. Also physics like navigation agent or character controller (jump feature) works quite odd. I undestand that to make physics work, it is important for objects have "real-world" sizes, but for AR RTS game thats not what I want.

I quickly read latest comments in this thread, and found out about scale-content branch of plugin, but haven't found any, at least short, howtos or descriptions. Is it suitable for my situation? If so, how does it work and how to setup it im my own project properly (at least few words)? To be more specific, let's say, my level have a root object RenderingScene and this is the only object which scale, I suppose, I should care of.

Here is Yoda shopping and watching TV for you made with your plugin

 

Hi. Open UnityARMatrixOps class and replace method GetPosition

namespace UnityEngine.XR.iOS
{
public class UnityARMatrixOps
{
static float _matrixARWorldSize = 100.0f; // 1.0f Default Ar matrix

public static float MatrixARWorldSize
{
get { return _matrixARWorldSize; }
set { _matrixARWorldSize = value; }
}

public static Vector3 GetPosition(Matrix4x4 matrix)
{
// Convert from ARKit's right-handed coordinate
// system to Unity's left-handed
Vector3 position = matrix.GetColumn(3);
position.z = -position.z;

return position* _matrixARWorldSize;
}
...

P.S. Scale - MatrixARWorldSize

 

Hi Unity team.
When there is a support from two cameras, for VR glasses?

 

Hi, how do I make when downloading a new scene, AR tracking again? (Clean old result of tracking)

 

Does anyone know how to make efficient fog for ARKit use? I´ve been toying with FogVolume3 but it is really heavy.
The demo with the widowmaker-model seems to do a nice job, but i have no idea how!

 

I dont think ARKit gives you a ground, per se. It will begin to estimate horizontal surfaces for you... In my app, I'm assuming a starting height of an average adult. I think the Y starts at zero.

 

I think you just start a new session. See UnityARKitControl.cs for some session lifecycle examples.

 

This doesn't seem like it will work - it will mess up all the planes you generate and hit tests, and any other functionality that uses GetPosition that you may not want to scale. Have you actually got this working?

 

Cody, who made that demo app says: "
just used fog from the standard asset
if I remember correctly. made it a while ago
I think the look of the fog is more dependent on the lighting happening in the camera"

 

I'm getting this exception. Any help would be appreciated!

[Session] Session did fail with error: Error Domain=com.apple.arkit.error Code=102 "Required sensor failed." UserInfo={NSLocalizedFailureReason=A sensor failed to deliver the required input., NSUnderlyingError=0x1c0650ef0 {Error Domain=AVFoundationErrorDomain Code=-11819 "Cannot Complete Action" UserInfo={NSLocalizedDescription=Cannot Complete Action, NSLocalizedRecoverySuggestion=Try again later.}}, NSLocalizedRecoverySuggestion=Make sure that the application has the required privacy settings., NSLocalizedDescription=Required sensor failed.}

I get this when I start ARKit for the 2nd time, or rather resume it. The flow is like this: start QR code reader, stop QR code reader, start ARKit (everything works fine), then pause ARKit, start QR code reader, stop QR code reader, and then resume ARKit (then exception thrown). It doesn't happen immediately upon resuming ARKit, but within seconds, it'll throw up that exception, and the camera feed will freeze.

The way I am "pausing" and "resuming" ARKit is by calling:

m_session.Pause();

And then:

m_session.RunWithConfig(config);

where config is the stashed off config object that was used to start ARKit initially.

config = new ARKitWorldTackingSessionConfiguration();
config.planeDetection = UnityARPlaneDetection.Horizontal;
config.alignment = UnityARAlignment.UnityARAlignmentGravity;
config.getPointCloudData = true;
config.enableLightEstimation = true;
m_session.RunWithConfig(config);

 

Looks like you might be dealing with the ARKit and QR code reader interaction not being nice about acquiring and releasing the camera feed resource. Not sure how you would solve - you could try using another QR core reader library, or checking in that QR code reader code to see how they acquire and release camera feed.

 

I agree, however, it seems to work "the first time". That is, QR code reader stops, and ARKit starts up and runs initially the first time. But when user goes back in to QR code reader again, we need to remember the ARKit session, so we pause, and then want to resume. And that's when it fails. So I thought it had more to do with ARKit improperly resuming from a paused state, rather than QR code reader not releasing the camera properly (or perhaps it's not pausing properly). I will try to investigate what QR code reader does to release the camera, however, if there is any more information on properly resuming ARKit that would be helpful. Thanks.

 

So, when you do m_session.Pause(), does the camera feed get released from ARKit? How does one hand off properly the camera for an ar session, without destroying the session, with the intent that you get it back again?

 

Well, I'm not sure whose fault this is. However, when I add a 2 second delay from the time I call "stop" and "destroy" on the QR code reader, until the time I call "resume" on ARKit, then I do not hit this exception, and ARKit seems to be working fine again. I'd still like to know the true cause ...

 

Any updates on this?

 

Hey evstinik95, here's a link to the scaling branch, it should work for your scene and the demo scene is pretty simple.
#1120
The only thing I had a little trouble with was that it would be nice to be able to offset the position also so your asset's transform could stay completely untouched but appear to move to the point in space you have placed them in.
I haven't found a nice way to do that yet, if anyone knows a way please let us know.

 

Hi, I have a situation like this: I have turned on both particle and the blueish plane prefabs, which will show on the screen when the camera is detecting the surface. But sometimes there are so many blueish planes showing on the screen. I sometimes accidentally tap on one of them, which will reset the floor and reset the position of my models.

Is there any way to disable the camera continuously detecting after the first detect? I mean when it detects the surface successfully on the first time, it stops.

Thanks!

 

Hi!.. this asset is really cool!!. but I made a small game and ARKit is killing my battery.. about 20% in about 5 minutes.. I have an iphone 6s plus. Anyone is having the same problem??

 

This was fixed on 06/08 - use UnityARCameraNearFar.cs as in UnityARKitScene.

 

Restart the ARSession with a config that does not detect planes. i.e.
ARKitWorldTackingSessionConfiguration sessionConfig = new ARKitWorldTackingSessionConfiguration ( UnityARAlignment.UnityARAlignmentGravity, UnityARPlaneDetection.None);
UnityARSessionNativeInterface.GetARSessionNativeInterface ().RunWithConfig(sessionConfig);

 

Ok guys, after struggling with the ARKIT for a whole day I post here my configuration that is working TODAY if anyone interested...

First: Look at your OSX version, you have to update the system to version 10.12.6 as minimum.

Second: install Xcode 9 beta 6. You can install this version aside of your old Xcode. No problem.

Third: Install iOS 11 on your iPhone, minimum iPhone for using the ARKIT -> iphone6s.

Fourth: Install Unity 5.6.2 (this is the minimum Unity version for working without patches..), as before, you can have multiple Unity versions only changing the name of the unity directory before doing the installation of the new Unity.

Fifth: Download the ARKIT plugin from the asset store.

Sixth: Let the fun begins!

That's all folks!

 

@jimmya I get "Apple mach-o linker error" when I disable strip engine code. How do I get around this?
I want to disable code stripping since I'm facing crashes on my ARKit app when I load asset bundles with animations. It says missing classID 90 and classID 137. I heard classID 90 was related to Avatar but not sure what classID 137 is. It'll be useful for me to disable stripping and check again to see if this is related. I believe its not mandatory to enable stripping in order to run ARKit app. I'm using Xcode 9 beta 6, Unity 2017.2.0b8, ARKit plugin version 1.0.1.

Unity console after connecting to iphone.

Xcode Log

 

Have you tried combining ARKit plugin with Google Cardboard SDK? I suppose one could set up an AR scene, remove UnityARVideo and use cardboard camera instead? You would have to switch off cardboard camera's headtracking I suppose.

 

christophergoy sorry for replying you late.

[UPDATE: I did not notice Xcode 9 Beta 6 was released since it won't autoupdate or notify me for updates, all those errors are gone! now debugging Match-O-Linker and will post updates]


Since that date, new builds are working on unity but not deploying the code through Xcode onto Iphone gives me the following errors:

Note: I am using the latest unity build 2017.1.0f3, The unity apple arkit as of today, and xCode Version 9.0 beta (9M136h)


/Users/zyshaq/Dropbox/PyramidEscape Apple/appleBuild/Libraries/ARSessionNative.mm:189:14: Use of undeclared identifier 'ARTrackingStateReasonInitializing'; did you mean 'UnityARTrackingStateReasonInitializing'?

/Users/zyshaq/Dropbox/PyramidEscape Apple/appleBuild/Libraries/ARSessionNative.mm:191:14: Duplicate case value: 'UnityARTrackingStateReasonInitializing' and 'ARTrackingStateReasonExcessiveMotion' both equal '1'

/Users/zyshaq/Dropbox/PyramidEscape Apple/appleBuild/Libraries/ARSessionNative.mm:211:134: Unknown type name 'ARWorldTrackingConfiguration'; did you mean 'ARWorldTrackingSessionConfiguration'?

/Users/zyshaq/Dropbox/PyramidEscape Apple/appleBuild/Libraries/ARSessionNative.mm:218:108: Unknown type name 'ARConfiguration'

/Users/zyshaq/Dropbox/PyramidEscape Apple/appleBuild/Libraries/ARSessionNative.mm:687:5: Unknown type name 'ARWorldTrackingConfiguration'

/Users/zyshaq/Dropbox/PyramidEscape Apple/appleBuild/Libraries/ARSessionNative.mm:687:45: Use of undeclared identifier 'ARWorldTrackingConfiguration'

/Users/zyshaq/Dropbox/PyramidEscape Apple/appleBuild/Libraries/ARSessionNative.mm:700:5: Unknown type name 'ARWorldTrackingConfiguration'

/Users/zyshaq/Dropbox/PyramidEscape Apple/appleBuild/Libraries/ARSessionNative.mm:700:45: Use of undeclared identifier 'ARWorldTrackingConfiguration'

/Users/zyshaq/Dropbox/PyramidEscape Apple/appleBuild/Libraries/ARSessionNative.mm:710:5: Unknown type name 'ARConfiguration'

/Users/zyshaq/Dropbox/PyramidEscape Apple/appleBuild/Libraries/ARSessionNative.mm:710:32: Use of undeclared identifier 'AROrientationTrackingConfiguration'

/Users/zyshaq/Dropbox/PyramidEscape Apple/appleBuild/Libraries/ARSessionNative.mm:723:5: Unknown type name 'ARConfiguration'

/Users/zyshaq/Dropbox/PyramidEscape Apple/appleBuild/Libraries/ARSessionNative.mm:723:32: Use of undeclared identifier 'AROrientationTrackingConfiguration'

/Users/zyshaq/Dropbox/PyramidEscape Apple/appleBuild/Libraries/ARSessionNative.mm:884:12: Use of undeclared identifier 'ARWorldTrackingConfiguration'

/Users/zyshaq/Dropbox/PyramidEscape Apple/appleBuild/Libraries/ARSessionNative.mm:889:12: Use of undeclared identifier 'AROrientationTrackingConfiguration'

ARC Semantic Issue Group

/Users/zyshaq/Dropbox/PyramidEscape Apple/appleBuild/Libraries/ARSessionNative.mm:255:48: No visible @interface for 'ARCamera' declares the selector 'projectionMatrixForOrientation:viewportSize:zNear:zFar:'

/Users/zyshaq/Dropbox/PyramidEscape Apple/appleBuild/Libraries/ARSessionNative.mm:417:35: No visible @interface for 'ARFrame' declares the selector 'displayTransformForOrientation:viewportSize:'

 

So just saw this.

https://www.engadget.com/2017/08/29/google-arcore/

Will this SDK be expanded to support boths ARKit and ARCore?

--Update. Well I see there is a SDK already https://developers.google.com/ar/

But would be nice if there was a common interface.

 

is it plan to support OpenGLES2/3? because we are using iVidCapPro for video capturing, it will not work if the Graphics API are only select Metal.

 

Instead of trying not to strip, you could try to add the missing classes to your link.xml: https://docs.unity3d.com/Manual/IL2CPP-BytecodeStripping.html

The classes can be found from ID here: https://docs.unity3d.com/Manual/ClassIDReference.html

 

Anyone knows how to solve this on Xcode: I updated mac OS, unity, Xcode, unity apple arkit sdk today to ensure no workflow related issues:

Apple Mach-O Linker (ld) Error Group
"_OBJC_CLASS_$_ARWorldTrackingConfiguration", referenced from:

"_OBJC_CLASS_$_AROrientationTrackingConfiguration", referenced from:

"_OBJC_CLASS_$_ARAnchor", referenced from:

"_OBJC_CLASS_$_ARPlaneAnchor", referenced from:

"_OBJC_CLASS_$_ARSession", referenced from:

clang: error: linker command failed with exit code 1 (use -v to see invocation)

 

Can you use the built in recording in iOS for video capture: https://docs.unity3d.com/ScriptReference/Apple.ReplayKit.ReplayKit.html

 

Your XCode is not up to date.

 

Since its ARZombing... here is our ARZombie:
The articles below include my 2 years of research and self experiment in this field and will point out major flaws and how to deal with them. I wish everyone who is working on such a project to succeed because my ARZombie is not longer the center of my interest:

Never the less, am interested to know how @Grislymanor was able to build an interest and do successful marketing-Accolade!




Articles:

• https://www.vrfitnessinsider.com/interview-google-tango-ar-zombies/
• https://forum.unity3d.com/threads/ar-zombies-for-tango-indie-game.445623/

Videos:

 

Chris, I think I asked the question improperly. I want to scale a scene to the size of a tabletop. If one unit is one meter, I would have to build on a 1x1 terrain. That presents some problems, as trees can scale to only so small, for example and 0.001 seems to be a hard limit for many options .. not to mention the fact that collisions need to be detected at that same precision, so lots of existing AI stuff also starts breaking down. It's just messy. What I want to be able to do is switch the 1 unit = 1 meter ARKit correspondence to something like 1 unit = 1 centimeter. Possible?

 

Its xCode 9.0 beta 6

 

Also, is there a way to change the "Unity-iPhone" naming everywhere? Couldn't find in plist, forums, etc.

 

Easy, look at the example I'll be posting now, just go to AR manager, scroll to the bottom and multiply the transform.position by a factor.

 

Our current project is not in ARZombie, nor in the AR field, but the one I work on the side is pyramid escape:

 

If you're using the correct version, you should not be getting those linker errors. Do you maybe have an older version of XCode that is actually being run?

 

Will return to you on this soon once I confirm. Its a possibility.

 

I looked twice at my project and I was forgetting to not destroy UnityARAmbient. Don't destroying it as well worked for me. Thanks!

 

Oh cool, could you please let me know where I should place this code in? I guess is in a C# script named UnityARSessionNativeInterface. If true, which line should I paste it in?

Thanks

 

just quick review ReplayKit. Is it able to get the video file, because I have customize UI for preview and share.

 

Can anyone explain what the ARKitControl actually does. I activated it and cant really tell what the buttons do. Please see screenshots. I also cant find any documentation on it.

Thanks in advance.

 

Problem solved USE ReplayKit, https://docs.unity3d.com/ScriptReference/Apple.ReplayKit.ReplayKit.html

 

What's everyone's best approach on cool surface detection indicators? Like more of a cursor than the plane generator!

 

I tried ARkit remote, but it seems touches on the phone will not work?

 

It seems I cannot get plane events? I tried to add debug log in AddPlaneAnchor(), UpdatePlaneAnchor(), RemovePlaneAnchor(), but it is never called, only UpdateCameraFrame() is called.

 

Hi @jimmya and everyone,

I'm trying to figure out how to get the raw pixel data from the YUV color space and use them for image processing in OpenCV. I know that the YUVShader in the Unity ARKit plugin's Shader folder does some kind of conversion from planar YCbCr format to RGB but I'm not quite sure where the raw pixel bytes can be retrieved.

Any help on how to get them to work with OpenCV?

 

I only want to record the video with some cameras. e.g, I want to exclude GUI layer.

 

I have two scenes. Map and AR. How to make a new session start when the AR starts? So that EVERYTHING cleared as if I had run the application? HOW full clean ? all plane, all anchor, ALL. HELP plz!