A Unity ID allows you to buy and/or subscribe to Unity products and services, shop in the Asset Store and participate
in the Unity community.
Discussion in 'Handheld AR' started by ina, Jun 5, 2019.
Is ARKit 3 fully supported by ARFoundation?
Or is the existing ARKit plugin getting updated soon?
ARKit 3 was announced yesterday, and will release in iOS 13, which hasn't been released yet.
iOS 13 beta has already been released to developers. And Unity team can work on the integration already. I wonder how long will it take to integrate features like people occlusion and motion capture (and possibly with support for Mecanim). And ETA from the team would be greatly appreciated.
I don't expect it to be a small task for the team to "just" implement these things and it's only 2 days old news. However, it would be really nice to be kept informed on when/if and to what degree these tools could be expected. Many ongoing projects could benefit from them and I would rather not optimize for issues that are automatically fixed very soon with the ARkit3
Oh! I just realised that the ARkit plugin has been deprecated and we should now use ARFoundation?? Isn't ARFoundation only catering to the features that the platforms have in common? Something like body tracking, body occlusion etc. are currently unique to iOS. Does that mean that Unity will down prioritize those features until ARcore catches on?
I hope not, that would be a huge bummer for Unity Devs around the world. My guess is they will implement ARKit3's new features in order to be a state-of-the-art dev platform for which Unity is known for.
We seriously hope that’s not the case too. Maybe they would have platform specific features on top of the cross platform functionality. Hoping the devs would clarify more on this matter.
No, AR Foundation also features ARKit-only functions (for instance object tracking).
But what worries me is that the ARKit plugin has always been updated very quickly. In 2017, ARKit was available in Unity within days (I suppose they worked on it before the beta release). But with AR Foundation I've seen that things takes much longer time. Only recently image tracking was added, while this has been around for ages in ARKit and ARCore.
edit: Unity support for ARKit3 https://blogs.unity3d.com/2019/06/06/ar-foundation-support-for-arkit-3/
We were wondering too if we could get our assets out of Unity into RealityKit through USDZ for example. RealityKit has many cool features that the current Unity struggles with (using image effects like motion blur in AR without affecting up the tracking and severely impacting the performance, accurate camera based depth of field, raytraced soft shadows and many more)
There is a USDZ preview package for Unity that allows you to export assets to USDZ I think. Check the package manager
And yes, I'm also going to look into RealityKit, it looks very promising. Motion tracking, depth of field blur, HDR environment maps, camera grain, good looking realtime soft shadows on mobile, recording an AR session for in-editor testing, continues mapping&relocalization across multiple devices, easy multiplayer.
Mostly also possible in Unity, but with RealityKit it's by default it seems
PS: Unity ARKit3 announcement
That was fast! Exciting!
I can’t see a mention of improvements to the post processing workflow in AR to match RealityKit level. I hope the devs address that soon.
Wow. "people occlusion features are available only on iOS devices with the A12 Bionic chip and ANE." Anyone know which specific devices these are?
Also, just to confirm
This repo contains latest ARkit 3 features? Not perfectly clear either from the blog post or reading the manifest.json file.
Also am wondering about LWRP + ARkit3 support.
@eco_bach A12 chip is available in iPhone Xs
wow, that was really fast, so excited!!
Has anybody tried the samples already?
We published a version of ARFoundation today that supports the new features in ARKit 3. Please see our blog post for more details.
The samples repo has now been updated with new samples for ARKit 3 features.
iPhone XS and XS Max
iPad Mini (2019)
iPad Air (2019)
edit: whoops also all A12X devices of course:
11-inch iPad Pro
Third-generation 12.9-inch iPad Pro
from https://en.wikipedia.org/wiki/Apple_A12 & https://en.wikipedia.org/wiki/Apple_A12X
Thank you for the update!! Looks like I'm getting the following error when trying to deploy the ARFoundation 1.5 LWRP sample to iPhone X running iOS 12.1.2:
dyld: Symbol not found: _OBJC_CLASS_$_ARMatteGenerator
The latest ARFoundation 1.5 (and also 2.2) currently requires Xcode 11 beta. That linker error is expected if using Xcode 10 (or less).
Cool. I wonder when ARCore 1.9 will be supported....
The device also needs to run iOS13
Watching the ARkit3 deep dive am wondering how people occlusion is handled under the hood in ARFoundation.
-Scenekit + ARSCNView
- Custom compositon thru ARMattegenerator
Does anybody know what kind of impact "Human Segmentation Stencil Mode" and "Human Segmentation Depth Mode" has?
Yep that was it, device needed to be on iOS 13. Thank you!
Apple mentioned in their WWDC talk they now can also detect white walls (feature-less surfaces) using some AI. Is this also available in ARFoundation? I can't find any documentation on that.
from 41:00: https://developer.apple.com/videos/play/wwdc2019/604/
They call it ML-enhanced plane estimation
I am playing around with with the simultaneous world and face tracking feature (RearCameraWithFrontCameraFaceMesh sample). It is implemented as a toggle, toggling between viewing the back- or front-facing camera's feed, while the tracking data from both managers are accessible.
However, what happens exactly to the position and rotation of the face when we are using the front-facing mode?
The face position seems to be relative to the device's starting position (0,0,0) as all other trackables, but what about the face rotation? It seem to be broken, where just certain face rotations seem to be tracked (e.g. only y-axis, while the other two do not change). To what is this rotation relative to?
It's described in the Unity Blog post (https://blogs.unity3d.com/2019/06/06/ar-foundation-support-for-arkit-3/). Basically, stencil mode tells you if a pixel contains a human or not, depth mode also estimates the distance of that human pixel from the device. I guess stencil mode is much lighter in terms of performance impact, as it only requires one ML network to run, while depth estimation requires (at least) an additional one. So depending on your use-case, stencil mode might be useful to optimise performance if you only need segmentation, not distance-based occlusion.