Resolved ARKit support for iOS via Unity-ARKit-Plugin

ARKit uses real world scale - so if you want to show buildings on your tabletop, you will not be able to show real sized building on your table top. If you want to show a building as a model, then you will need to adjust other systems to correspond to that.

 

Hey just wondering is there a way to manually adjust the scale? I just have a scene I’d like to test out with ARkit but the scale is completely wrong. It’d be nice to be able to globally hack it back to a good scale.
( sorry this is been mentioned but my search was playing up )

Thanks
Pete

Hey, if anyone has a good approach for this I'd love to know! I've been messing around with my project to try and scale it down to a size that could fit on a table or something reasonable and it is breaking the physics pretty bad. I think it would need a full overhaul to get it working.
Is there a way to scale the camera or incoming data for the track somehow? It would be nice to be able to change this on the fly to accommodate for the real world area that the user has to play with.

 

Just would like to report that Post-processing stack causes problems when applied to the camera, jittering movement and low fps, so we had to remove it from our project.

It is amazing technology nonetheless.

 

hi chris,

i see one very important use case for manually added anchors (if ARKit's anchors are comparable to anchors in other AR solutions like google tango or hololens) which is getting refined transformations every time ARKit's SLAM algorithm does corrections after it learned more about the environment. but apple's documentation at https://developer.apple.com/documentation/arkit/aranchor?language=objc is very limited on this topic, so until i can try this out, i can only suspect and hope that we can get this functionality. so i'm looking forward to the next unity-arkit-plugin update!

let me illustrate the key point with this scenario:
for example with hololens you could set a 1st anchor point exactly on the right corner of your table, then turn right about 300 degrees, look at the left corner of your table, set a 2nd anchor point there, let's say the real table is 100 cm wide and you might get the virtual anchor points 1 and 2 with a little error, let's say 97 cm distance. now when you look more to the right it will recognize the right corner of the table again and apply corrections to all anchors it learned and the camera pose. after the correction, anchors 1 and 2 would end up at a distance of much closer to 100 cm. anchors are finely updated all the time to give the best result in relation to the current knowledge about the environment on hololens.

i very much hope, that session:didUpdateAnchors: of ARSessionDelegate will notify us about corrections for manual anchors https://developer.apple.com/documentation/arkit/arsessiondelegate/2865624-session?language=objc
compare to hololens: https://docs.unity3d.com/Manual/windowsholographic-anchors.html
compare to google tango: https://developers.google.com/tango/apis/unity/unity-howto-area-learning#relocalizing (corrections using timestamps)

thanks for the quick release of your plugin!
robert

 

Ahhh that's a shame I feel like it's something people are going to run into quite a bit.
With my project for example, I was careful to build it to real world scale from the start so as to get the best physics (and lighting) result possible, but that won't fit on a table

 

Hi! Does the ARKitSessionConfiguration work on on devices <A9 or not? Have iphone 5s and start session with
ARKitSessionConfiguration but nothing work's. Nevertheless have ARTrackingStateNormal state in
m_session.GetARTrackingQuality()

 

Hi There,

Could someone please clarify if there is any difference in how ARkit is supported in 5.6.1p v 2017.1.b10 - thought I saw I post but can't find it now.

 

Hi everyone !

Could someone tell me, is it possible for the ARkit to track the movement of a real object and move next to it a virtual object ?
I guess that is unreal , because I have not seen any such example or documentation.

 

ARKit does not do this currently, but there are other frameworks released with ios11 that may do this. We have not tried them together and we do not have support in plugin form for those other frameworks yet.

 

Unity ARKit plugin should work on any Unity 5.6 version past 5.6.1p1, or also on any 2017 version past 2017.1.0.b5. Updated main post with this info.

 

If there are other people dealing with these problems, you might be able to find discussions on them. Another thing to note: you should probably design and author your project specifically for AR instead of trying to turn an existing project into AR - it might work out easier.

 

Please read earlier posts in this forum.

 

It will be great feature!

 

hi!
i replaced the cube under the hit cube parent with my own object. the only issue i have with this touch to place is that it is almost TOO sensitive, and difficult to reposition. how can i change it so that perhaps the object can be repositioned by 'swiping the screen'? if that makes sense. so like instead of tapping the screen to keep reconfiguring, id tap the screen once to put the cube in the game and then after that use the swiping motions to reposition it.

 

This is not really an ARKit question - please refer to Unity tutorials or other forums to get answers to questions such as these. Or you could get a plugin on the asset store that might solve some of this for you.

 

Hi folks,
The Unity ARKit Plugin is now available on the Asset Store.

 

thanks for your view on the topic. but at least for the comparision to hololens world anchors i can confirm, than these often change their coordinates by little amount to fulfill the property "[...] to be fixed in a physical location" even better. i did just set a world anchor in a test scene and logged the position of an anchor over several seconds and got these position updates (no changes by the user, all from the tracking; values in meters):
calibrationPoint1.transform.position = (-0.07617, -0.27336, 0.81071)
calibrationPoint1.transform.position = (-0.07611, -0.27360, 0.81081)
calibrationPoint1.transform.position = (-0.07628, -0.27347, 0.81114)
calibrationPoint1.transform.position = (-0.07530, -0.27256, 0.81130)
calibrationPoint1.transform.position = (-0.07533, -0.27117, 0.81127)
calibrationPoint1.transform.position = (-0.07523, -0.27142, 0.81178)
calibrationPoint1.transform.position = (-0.07505, -0.27189, 0.81274)
calibrationPoint1.transform.position = (-0.07494, -0.27218, 0.81332)
calibrationPoint1.transform.position = (-0.07483, -0.27244, 0.81386)
calibrationPoint1.transform.position = (-0.07489, -0.27364, 0.81355)
these corrections are just a few millimeters, but they make a real difference for a good alignment of real and virtual objects.

 

Hi all,
Sorry but I just have to bring this up once more because I really can’t believe I’m the only one who finds this a bit of a massive roadblock... Scale!

We’ve been taught to use real world scale for physics (and lighting) so if you are making a car you make it roughly the size of a car in unity scale.
So what now? Do we have to make everything to coffee table scale!?
Unity’s implementation of it is really cool, but that just makes it all the more frustrating that it is so inflexible in this way. There must be a way to scale ithe incoming tracking camera data.

Real world scale is great if you are ikea and want to AR a sofa in a room but it’s not great for us!
Rant over,
Pete

 

Hi!
How to use switching between scenes? if I run for example ArKitDemoScene than switch to my custom scene and then back again AR scene - plane detection has stopped, despite the fact that the RunTracking method call on start. If I apply dont destroy to UnityARSessionNativeInterface I have a crash after 30 second after the start.

 

Just wanted to say hi from the guys at Aryzon!

ARKit really is awesome! We make a Cardboard AR headset that works with iPhone. Doing things on screen of course is great but I have had the honour to try it in a headset with ARKit and must say it really is next level. It's getting close to Hololens quality thank you Apple!

 

I would make all the ARKit related GameObjects add a component DontDestroyOnLoad - this includes ARCameraManager, MainCamera (it has UnityARVideo), GeneratePlanes, etc.

 

Is there a material or shader we can use on spatial meshes that are detected that can be used to occlude virtual objects? E.g. if I place a virtual cube behind a wall, when I am behind that wall I would not want it to be rendered.

 

That would be useful we were looking into something like this. But it can basically be a shader that writes to the zbuffer but not to the framebuffer, and is rendered before any other virtual objects.

 

Hi! How to properly delete all planes and clear all tracking data (anchors )after the scene reloads ?

is it right way?
foreach (string key in planeAnchorMap.Keys)
{
RemoveAnchor(planeAnchorMap[key].planeAnchor);
}

 

Use RunOptions:
ARKitWorldTackingSessionConfiguration sessionConfig = new ARKitWorldTackingSessionConfiguration ( UnityARAlignment.UnityARAlignmentGravity, UnityARPlaneDetection.Horizontal);
UnityARSessionNativeInterface.GetARSessionNativeInterface ().RunWithConfigAndOptions (sessionConfig, UnityARSessionRunOption.ARSessionRunOptionRemoveExistingAnchors | UnityARSessionRunOption.ARSessionRunOptionResetTracking);

 

Thank you, jimmya!

 

Now available on the asset store. https://www.assetstore.unity3d.com/en/#!/content/92515

 

Is there a way to do state dependency with this, i.e, a way to remember where the "cube" was last placed (or in the last session)

also i'm still not certain regarding real world scale and arkit / unity scale - is it 1:1 where 1 Unity meter = 1 Real World meter, or is it relative

 

These questions have already been answered in this forum. But to reiterate, every time you start up a session, the ARKit has no information about the real world and it has to build up the information based on what it sees via the camera. So it does not have any state that persists between sessions. You can certainly save the positions of your virtual objects, and they will exist in the virtual world when you come back and load them, relative to the start position of the device in the previous session.
And the scale on ARKit vs Unity is 1:1. The start position of the device is the difference though - wherever you start a session is considered (0,0,0) in the Unity scene.

 

The issue of scale that others mention is affecting us as well, and re-designing the project or scaling the entire world isn't an option for various good reasons (physics, lighting, navigation, cross-platform support...)

In VR, the solution for this issue is to scale the Transform on the camera. For example, a camera with localScale of 10,10,10 on the Vive makes you feel like a giant. With that one simple change, the world becomes effectively 1/10th the size while preserving the 1 unit = 1m correspondence of all GOs in the scene. All physics, lighting, navigation, etc. continue to work great without further modification. Similarly, a camera with localScale of 0.2,0.2,0.2 makes you feel tiny (and the world feels huge).

For AR, camera scales greater than 1 will be extremely helpful for situating small versions of scenes on different surfaces while preserving unit correspondence and accuracy.

@jimmya can you please help us all make camera scale work with ARKit in this way? I'm not familiar with the code being used, but it could simply be a matter of extending the custom projection logic to take the camera's scale into account.

Supporting scaled cameras will make things so much easier, for example we can display the same scene on a large table or a small table just by editing the scale of the camera.

 

Hi! Is it possible to get reflection information(at least an approximate environment color ) from real world and apply it to AR object? Or we can only use the lighting intensity? Thank you .

 

Can I do for example (10,0,0)?

 

This information is not directly available from ARKit, but you might be able to use the video texture as an approximate environment map to achieve this effect.

 

I don't think this will work, as the real device camera and the virtual unity camera are pretty closely tied together via the ARKit camera updates. Let me think about it some more.

 

Yes

 

From the script can be done? I tried in UnityARVideo.cs, but everything was broken
(camM is parent mainCamera)

Code (CSharp):

1.    public void OnPreRender()
2.         {
3.         camM.eulerAngles = new Vector3 (0f, Input.compass.trueHeading  , 0f);
4. }

 

Please read the answer that was linked.

 

I've asked around a bit, and have found no-one who has had to deal with this in AR - it could just mean that AR is new in this regard, and not much effort has been put into solving this. I don't see an easy way to do this generically, but if you are able to find one, please let me know. Here are a few suggestions: (1) scale it in the shader or renderer - use a scale parameter to pass to the shader that is used in conjunction with MVP matrix to scale the object. Disadvantage is that your physics, interaction and other models will be happening in a different space from your rendering, and that could cause problems with lighting, raycasts, etc. (2) render using a separate camera, using the main camera rendertexture as the background. Disadvantage is that you will have to render the ARKit generated primitives (anchorPlanes) in the main camera.

I still think adjusting your physics for your new scale would be the easiest solution that doesn't mess with a lot of other systems. But I am willing to hear any other good solutions to this.

 

Hello, I'm having issues with Instantiated cubes not colliding properly on a plane prefab created from the UnityARGeneratePlane Script. They seem to be bouncing around when I move the camera. I've attached a video link showing the problem, any idea what's going on here?

Video link:

 

This is expected - the planes that are generated by GeneratePlanes are GameObjects that represent the planeAnchors found by ARKit using plane estimation. As you move around, ARKit sees more feature points and is able to provide a better estimate of the plane and will possibly move, rotate and scale the planeAnchor (and thus the GameObject that mimics it) based on its new estimate. If you would like more stable surface to place rigidbodies, you can create/instantiate a plane gameobject that does not update itself based on anchor updates from ARKit.

 

Are there plans to try to unify this with the Unity HoloLens meshing / plane-finding implementation? (There is state dependency there...)


P.S. Also, for recurring questions, consider a FAQ?

 

@jimmya I dont know if this is just my inadequate programming skills, but I have found that when scaling objects bigger causes them to drift more, or then it is just more easily noticeable. Smaller objects stay fairly well fixed in world.pos. I am using the HitCubeExampleScript.

edited: My bad. My GUI cam was interfering the depth buffer.

Another question.. What would be best settings for a UICamera? UI objects on the Main Camera interferes with anchor detection. Likewise using another camera seems to have effects to the accuracy as well. This puzzles me. Ticking culling for UI Layer off from Main Camera and also no depth clearing in the UICamera setting it to only on the UI Layer. Still there seems to be some interfering....

 

Just reiterating what others have said , it would be a massive boost to development speed / porting if there was a way to scale things / project things without scaling the actual game world. If the game is written for ARKit to begin with, it would still be nice, as a way to scale / project the game onto a user-configurable space.

 

Jimmy, first of all it's so awesome Unity released the plugin! Thanks to all of the team!!!

I'm currently in the process of reading this thread carefully)

But can't wait to ask about arkit simulator of some sort! Compiling every time is killing the fun)
Have you considered it? What about Unity Remote app sending arkit data to the Editor?)

 

I am risking of sounding stupid, but: is the ARKit plugin compatible with Windows version of Unity?
I am looking into workflow where I would develop on Windows and either deploy via iOS Project Builder plugin (ideally) or only using Mac remotely to do the XCode deployments to device. Anyone trying a similar set up?

 

I've been attempting to use iTunes Connect/TestFlight to test an AR app made w/ Unity version 5.6.1p1

For anyone running into the "Missing Marketing Icon" error...

"Missing Marketing Icon - iOS Apps must include a 1024x1024px Marketing Icon in PNG format. Apps that do not include the Marketing Icon cannot be submitted for App Review or Beta App Review.".​

The App Icon upload field on iTunes Connect -> App Store -> General App Information, is not the above mentioned missing Marketing Icon. I learned from Apple Dev Support today that as of WWDC this Marketing Icon will need to be included on the Xcode side of things.

Please correct me if I'm wrong... Unity version 5.6.1p1 is not providing a way to include this required 1024x1024px png Marketing Icon.

Xcode-beta 9 within the AppIcon directory at the very bottom of the list of icon assets is where you will find the field to add this missing Marketing Icon.

I was able to upload via Xcode and not receive the missing Marketing Icon error after adding the needed file to Xcode. I also was able to add an external test user to the build.

I'm still receiving the "Missing Push Notification Entitlement" email even after installing the "Missing Push Notification Entitlement Warning Remover" Unity add on. Anyone know of a good solution for this issue?

 

Hello Chris. My apologies for being a bit off topic. I sort of thought this was not just general iOS but related to building something specifically utilizing ARkit with Unity, due to the need to use beta versions of Unity, iOS11, macOS, Xcode, etc., so I thought it would be relevant to anyone working with Unity and ARkit. Sorry about that