Resolved ARKit support for iOS via Unity-ARKit-Plugin

https://forum.unity.com/threads/unity-arkit-plugin-faq.495191/#post-3278478

 

got different case.. I'n menu i got normal scene.. was thinking that arkit did not stop after go back and forth to arscene kit.. cause when it got plane the app crashed.. cause I point the camera in the same location when there was a plane detected

 

Hi there.
I have a quick, and probably fairly dumb, question.

The AR in my app works fine... except on iPhone X where it has mirrored borders at the edge of the screen like this:



I am guessing that all I need to do is update the ARKit plug-in?
Would somebody be able to confirm this before I start messing with something that already works pretty well on all other devices?

Cheers,
Dan

 

does ar kit work with unity 2017?

 

@Aiursrage2k yes

 

Okay the problem I guess is that I am trying to run arkit and yuforia. Does anyone know if they work together?

 

When I choose the build architecture in xcode, can i set it to arm64 only? Until now I used "$(ARCHS_STANDARD)" but I am wondering if using arm64 would make my build smaller while keeping the support for all supported devices

 

how to check if device dont support ARkit?

 

yes,I use unity 2017.2.0f3

 

iOS device that supports ARKit(iPhone 6s or later,iPad(2017) or later)

 

Code (CSharp):

1. extern "C" UnityARUserAnchorData SessionAddUserAnchor(void* nativeSession, UnityARUserAnchorData anchorData)
2. {
3.     // create a native ARAnchor and add it to the session
4.     // then return the data back to the user that they will
5.     // need in case they want to remove it
6.     UnityARSession* session = (__bridge UnityARSession*)nativeSession;
7.     ARAnchor *newAnchor = [[ARAnchor alloc] initWithTransform:matrix_identity_float4x4];
8.    
9.     [session->_session addAnchor:newAnchor];
10.     UnityARUserAnchorData returnAnchorData;
11.     UnityARUserAnchorDataFromARAnchorPtr(returnAnchorData, newAnchor);
12.     return returnAnchorData;
13. }
14.  

Hello. I find that anchorData has no use in adding SessionAddUserAnchor method. Do I miss something? Thx.

 

Was this ever resolved? I have the same issue.

 

Very happy to share, Sahi Labs shortly coming on ARKit,
All Thanks to the great efforts by Team Unity for creating the plugin, it made it very easy.

 

Hello, everybody! Just wanted to share what I created last weekend at the VR Austin Jam using the new face tracking APIs in the Unity ARKit Plugin for the iPhone X. The "floating grey mask" from the example scene reminded me of Andross from Starfox (SNES) so I went about recreating that boss battle with the user attempting to defeat Starfox. Open your mouth to fire and close your eyes to turn invisible.

You can download the Xcode project from my itch.io page if you want to build it out and give it a go.

 

If you choose to target ios 11, you should choose ARM64 instead of Universal architecture in your PlayerSettings.

 

You have to download the latest beta of 2017.3, use the native Vuforia integration, and set Device Tracking to positional and set the Vuforia camera pose to Device. You only need to use the Vuforia 7 plugin, not the ARKit plugin. Think of it more like Vuforia is using ARKit, instead of both are working at the same time.

Be aware this is not for the faint of heart. It's buggy, slow, doesn't appear to allow the planar or pointcloud features of ARKit, and not well documented yet (very few of Vuforia's newer features are).

But it is pretty awesome.

 

Hi There. I am using the IphoneX blendshape capabilities.An experimenting to have the TrueDepth camera for retrieving expressions with the blendshapes but I want to show the front camera for normal arkit usage and place assets. Is it possible to have both activated at the same time and working together ? I tried but it seems like when I launch the front facing arkit camera by using the plugin, the blendshape are deactivated. Thanks a lot for your help.

 

When I was trying to use xcode + vufuria + arkit it was giving an error about UIView render. What we wanted to do was have ARkit project on the surface unless vuforia actually found the image target. I dont know why they cant work together especially now that unity has actually intragrated it into its own editor

 

They can't work together because both Vuforia 6 and ARKit require complete control of the camera.

Vuforia's solution (I'm guessing) is to run ARKit inside of it's own code, then do their planar tracking on top of ARKit's video feed and camera intrinsics.

If you run the early beta of Vuforia 7, it uses some of ARKit for extended tracking. But doing it this way will require Vuforia to write additional wrapper functions for ARKit features like pointcloud and floor plane detection.

 

iOS11.1 + Unity2017.1 + Vuforia6.2 + ARKit1.0.11
i have 3 scenes:
a. main scene: two button to switch arkit and vuforia;
b. vuforia scene(can go back to a)
c. arkit scene(can go back to a)
(y: the camera works fine. n:the camera doesn't work.)
1. a -> b(y) -> a -> c(y) -> a -> c(y) -> a -> b(y) -> a -> c(n)
2. a -> c(y) -> a -> b(y) -> a -> c(n)

the camera doesn't work after c->b->c, i've no idea whether vuforia or arkit cause the problem.

any one help, thanks!

 

How is your progress on this example? I would love to see how to use the blendshape coefficients to animate a head in Unity. Hence, I also made this question: https://forum.unity.com/threads/ark...hape-facegeometry-to-modify-face-mesh.504789/

 

Hi Dan,
I have "white bars" issue. Did the update work for you?

Best!

 

This has been answered many times before: https://forum.unity.com/threads/ark...nity-arkit-plugin.474385/page-42#post-3287608

 

Okay do you know how they did that?

 

Not too sure what specifically you're talking about. Seems like a standard ARKit app.

 

Hi jimmya @jimmya, Are there any arkit learning materials or documentation?

 

does blackscreen on the iPhone 6 mean it's not supporting the 6? (just running the standard asset as vanilla)

 

iOS device that supports ARKit(iPhone 6S or later,iPad(2017) or later)

 

1) I only have one directional light and the ARLightmanager to adjust it and the light probes and the holograms seem to be off still . It looks like the light gets adjusted to the conditions I see with my eyes and not the exposure corrected image from the camera on the screen.
2) Nope, its an IPad Pro. But seems I compared the image to the wrong camera mode. Photo mode from stock camera app looks much better but slow motion video has the same preview quality as the unity app so nothing to worry about.

 

Thanks!

 

Hi there.
I updated the ARKit Plugin to the latest version, as part of an iPhone X update for Tsuro. AR seems to work OK on my test device with one strange error.

If I pause the session...

Code (CSharp):

1. m_session = UnityARSessionNativeInterface.GetARSessionNativeInterface();
2. m_session.Pause();

...and then return to it using...

Code (CSharp):

1. m_session = UnityARSessionNativeInterface.GetARSessionNativeInterface();
2. Application.targetFrameRate = 60;
3. ARKitWorldTrackingSessionConfiguration config = new ARKitWorldTrackingSessionConfiguration();
4. config.planeDetection = UnityARPlaneDetection.Horizontal;
5. config.alignment = UnityARAlignment.UnityARAlignmentGravity;
6. config.getPointCloudData = true;
7. config.enableLightEstimation = true;
8. m_session.RunWithConfigAndOptions (config, UnityARSessionRunOption.ARSessionRunOptionRemoveExistingAnchors | UnityARSessionRunOption.ARSessionRunOptionResetTracking);

...the framerate from the camera drops so much that it's unusable.
If I pause and restart a second time, the framerate is so low that the image is stationary.

Did I miss some kind of change in the code that might cause this? Or is there a bug?
Is anyone else seeing this?

Any tips would be most welcome, thanks!

 

Anyone else experiencing a "Screen position out of view frustum" error?

 

This happens if you are trying to run in editor with a camera that has a view frustum or projection matrix that is invalid. To prevent it, see what is done in UnityARCameraManager on startup - look for " //put some defaults so that it doesnt complain"

 

Hello, im trying to improve the spawning system to make it better than arkit default generate planes,
and i was wondering how to do this :

This is from Magnolia Application.

is there any tutorial / topic that can guide me ?

Any tips would be most welcome, thanks!

 

I have basically the same problem right now.
Instead of Vuforia, i have a scene which utilizes the standard WebcamTexture.

It seems like the following scenario causes the problem:
ARKit (works) > any other usage of the Camera (works) > ARKit (broken)


I understand that ARKit takes over the camera, but is there a proper way to free the camera again and reinitialize ARKit at a later point?
The problem definitely lies in not freeing the hardware resource from ARKit after usage. Using WebcamTexture first and then ARKit does not cause any problems.

// EDIT:
@Ryan2015 I got it working now!
I changed the following in UnityARCameraManager.cs:

Changed Start() to OnEnable(), since start is only called once.
In that Method changed m_session.RunWithConfig(...) to

Code (CSharp):

1. m_session.RunWithConfigAndOptions (config, UnityARSessionRunOption.ARSessionRunOptionResetTracking);

and added the following OnDisable:

Code (CSharp):

1. private void OnDisable()
2. {
3.     if (m_session != null)
4.     {
5.         m_session.Pause();
6.     }
7. }

//UPDATE:
Still had a minor aspect ratio problem.
Changed UnityARVideo:
Replaced OnDestroy by:

Code (CSharp):

1. void OnDisable()
2. {
3.     if (m_VideoCommandBuffer != null)
4.     {
5.         m_Camera.RemoveCommandBuffer (CameraEvent.BeforeForwardOpaque, m_VideoCommandBuffer);
6.         m_VideoCommandBuffer.Clear();
7.         m_VideoCommandBuffer = null;
8.         UnityARSessionNativeInterface.ARFrameUpdatedEvent -= UpdateFrame;
9.         bCommandBufferInitialized = false;
10.     }
11. }

and changed Start() to OnEnable()

 

I'm working on an original iPad Pro 12.9" (model ML0Q2LL/A) with 128GB of RAM, running iOS 11.2 Beta (15C5110b). I've built several ARKit capable test apps in Xcode and Unity and I've tested my scenes in all kinds of environments (sunny outdoors, flourescent indoors, moderately lit indoors), and I cannot get the plane detection to lock in quite like I see in demo apps around the internet. Is there something I can do in the code or on my iPad to improve plane detection and get my objects to "stick" to the ground better than what I'm getting? Most of the time, my objects are jumping all over the screen, and other times, no matter what I do or how long I let the iPad detect the room, it won't even detect a horizontal plane for me to place objects. I understand that ARKit is in it's early stages and will only improve with time, but based on what I'm seeing other developers create, it seems like I'm missing something.

Any help would be most appreciated.

 

Look at FocusSquare example maybe?

 

Have a look at our Unite Austin talk for some pointers:

 

We're now supporting this with the addition of a settings file and an option therein:

The settings file for doing this is located at "Assets/UnityARKitPlugin/Resources/UnityARKiPlugin/ARKitSettings.asset"


Selecting that file will allow you to select in the Inspector whether ARKit is required in your app:



If the ARKitSettings asset is missing, you can create a new one from the "Assets/Create/UnityARKitPlugin/Settings" menu:


Thanks to user Patrik Buckau on BitBucket for the PR.

 

@jimmya Not to interrupt the thread, but I’m hoping you can help me. I need to create user touch input for my ar app in Unity. Could you point me to a tutorial for this or give a quick general explanation to help me get started? I need the user to be able to touch my game object in their phone and interact with it. Any help would be greatly appreciated.

 

Hey, I've just release a little ArKit app to the AppStore called PaPe AR:
If you've got time, download and let me know what you think.
Maybe even give it some stars if you like!

https://itunes.apple.com/gb/app/pape-ar/id1312657639?mt=8
Its the usual placing, resizing type app, but with a few moving animatable objects.

Thanks so much Unity.. I'm loving the plugin!
One thing I found useful was creating a grid button, so I could switch between the debug and shadow material. So for general release you can see the shadows, but if you ever want to see whats been detected it just replaces the plane materials with the debug material.
Happy ARing everyone!

 

is there a way the my 3d Object size will be base in the size of the generated plane? like if the normal plane change the size or scale my 3d Object will change too..like in this game I saw in the ios store
https://itunes.apple.com/us/app/arcade-plane/id1288528129?mt=8

 

When I build out my project and test it on my iPhone 7s, the model appears and animation works, but the screen is now blue. This has never happened. I have all of the latest software, hardware, etc...The camera feed seems to have an issue. Anyone ever had this happen before? I don't have a clue on how to fix this. Thanks for reading.

 

Formacar app coming on ARKit!
Thanks Unity Team for plugin!

 

I had the same issue and updated the AKKit plugin and that solve the problem.

 

UnityARKitPlugin\ARKitRemote\ObjectSerializationExtension.cs and UnityARKitPlugin\ARKitRemote\SerializableObjects.cs define a top-level "Utils" namespace. This is bad practice as it causes other code to prefer it over the Utils package in their own included namespace. In my case, it breaks OpenCVForUnity because Utils.MyMethod() now refers to the global Utils namespace rather than OpenCVForUnity.Utils class, breaking many many files.

Reported error in Unity console is:
Assets/OpenCVForUnity/Examples/Advanced/ComicFilterExample/ComicFilterExample.cs(196,23): error CS0234: The type or namespace name `matToTexture2D' does not exist in the namespace `Utils'. Are you missing an assembly reference?

Neither of these packages are my own, but it seems like ARKit plugin defining a top-level namespace named "Utils" is being very presumptive (Who's to say .NET won't introduce a Utils namespace in a later version).

Sorry for posting in whatever thread this is, but your bitbucket doesn't seem to have an "issues" tab and I don't really know where else to post.

 

Hello, Im having trouble spawning my go on the generated plate automatically. The positions appears to be correct with a 3.2 unit offset on the y axis but the rotation is wrong. Ive tried a few things but heres my latest code.

Code (CSharp):

1. public void SetScene()
2.         {
3.             List<ARPlaneAnchorGameObject> arpags = unityARAnchorManager.GetCurrentPlaneAnchors ();
4.             if (arpags.Count >= 1) {
5.                 ARPlaneAnchor ap = arpags [0].planeAnchor;
6.  
7.                 Instantiate (testObject, Vector3.zero, Quaternion.identity);
8.  
9.                 // From UnityARUtility Class
10.                 testObject.transform.position = UnityARMatrixOps.GetPosition (ap.transform);
11.                 testObject.transform.rotation = UnityARMatrixOps.GetRotation (ap.transform);
12.  
13.                 Debug.Log ("Setting Scene");
14.             }
15.         }

 

Hi all,
Just wondering if anyone could point me in the right direction.

I am running ARkit Remote and I have an image and plane detection in the editor within unity when connected to an iPhone. However, I have no image on the iPhone in the app. Is this normal? It doesn't seem to be when I have looked at the example videos for setup etc?

I have created the app with the UnityArKitRemote scene, which seems to be correct from other forum posts.

Any ideas?

Thanks.

 

I think he is wondering how they do the mapping? They either use some kind of marker or they place the Campus manually. Maybe the have some kind of recenter-mechanism?

 

Hello,

I am trying to find out how I can periodically "reset" the coordinate system. If I change the direction and position of the camera via some other means (say, a teleportation game mechanic), how could I then reset ARkit so that it would not try and "warp" the camera back?

I have found this code, which does not seem to work:
"
public void ResetScene()
{
ARKitWorldTrackingSessionConfiguration sessionConfig = new ARKitWorldTrackingSessionConfiguration (
UnityARAlignment.UnityARAlignmentGravityAndHeading, UnityARPlaneDetection.Horizontal,true,true);
UnityARSessionNativeInterface.GetARSessionNativeInterface ().RunWithConfigAndOptions (
sessionConfig, UnityARSessionRunOption.ARSessionRunOptionRemoveExistingAnchors |
UnityARSessionRunOption.ARSessionRunOptionResetTracking);

}

"

Thanks!