AR Foundation 2.1 Now Available

Hi everyone. After a long wait we are pleased to announce the availability of AR Foundation 2.1.0 as a preview package.


What’s New In AR Foundation?
This version is compatible with Unity 2019.1 and 2019.2 and contains support for the following new features:

2D Image Tracking
This feature lets you detect specific 2D images in the environment. The Tracked Image Manager automatically creates GameObjects that represent all recognized images. You can change an AR experience based on the presence of specific images. (ARCore & ARKit)

Face Tracking
You can access face landmarks, a mesh representation of detected faces, and blend shape information, which can feed into a facial animation rig. The Face Manager takes care of configuring devices for face-tracking and creates GameObjects for each detected face. (ARCore & ARKit)

3D Object Tracking
You can import digital representations of real-world objects into your Unity experiences and detect them in the environment. The Tracked Object Manager creates GameObjects for each detected physical object to enable experiences to change based on the presence of specific real-world objects. This functionality can be great for building educational and training experiences in addition to making games. (ARKit)

Environment Probes
This detects lighting and color information in specific areas of the environment, which helps enable 3D content to blend seamlessly with the surroundings. The Environment Probe Manager uses this information to automatically create cubemaps in Unity. (ARKit)

What About Unity 2018.3 Users?
Users of Unity 2018.3 must use AR Foundation 1.5.0 which is functionally equivalent to 2.1.0 but has different package dependencies. AR Foundation 2.1.0 is not compatible with Unity 2018.3.

New And Updated Platform Support Packages
In addition to the two new versions of AR Foundation there are updates to the packages to provide AR Foundation support for ARCore and ARKit.

• ARCore XR Plugin (compatible with 2018.3 and later)
• ARKit XR Plugin (compatible with 2018.3 and later)

A new package in the AR Foundation family is:

• ARKit XR Plugin (compatible with 2018.3 and later)

This new face tracking package for ARKit should only be installed in projects that intend to use the Face Tracking capability in AR Foundation. Including this package in your project without indicating your intention to use Face Tracking in your ARKit app’s metadata can result in an App Store certification failure.

How To Get Started
As always, links to the documentation are available from the Package Manager window (Window > Package Manager) for all of these packages. The AR Foundation documentation is the best place to start for learning about the new workflows associated with these new features.

We have also added samples for the new functionality to the AR Foundation Samples repo on GitHub. Check out the new 2.1-preview branches for use with Unity 2019.1 and later. Unity 2018.3 users should use the the 1.5-preview branch.

Migrating An Existing AR Foundation 1.0 Project
The new release of AR Foundation has breaking API changes and automatic upgrade of AR Foundation 1.0 projects is unfortunately not feasible. However, we have created a guide to help make the transition from AR Foundation 1.0 to 2.x easier.

Let Us Know What You Think
This release was a long time coming because we wanted to get it right. We worked extensively with our UX team on the new workflows and went through several rounds of QA prior to publishing these packages.

But there will inevitably be bugs that you find and even desired functionality that we missed. We will be watching the forums closely to answer questions and help where we can. But please feel free to submit bugs via the Unity Issue Tracker as well.

Thank You
Thanks for being Unity customers, for your patience, and for the passion you have for creating great AR experiences with AR Foundation. We are busy adding workflow improvements, support for new platforms, and new functionality to existing platforms. We are heavily invested in the future of AR Foundation and hope it meets your needs as creators.

 

Awesome, will check it out! How do we handle LWRP now? Is the workaround of setting it to 5.6.1 still valid? Is it working with higher versions?

 

Great, glad more of AR Core is now integrated!

 

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

 

The same workaround still applies.

Good question! Check out these two branches if you are interested in using LWRP with this new version of AR Foundation:
https://github.com/Unity-Technologies/arfoundation-samples/tree/lwrp-support-2019.1-with-2.1
https://github.com/Unity-Technologies/arfoundation-samples/tree/lwrp-support-2018.3-with-1.5

And, yes: the workaround using the 5.6.1 version of the LWRP is still in play. We are working with our core graphics team to address that and will update the samples once the workaround is no longer necessary.

 

@mdurand Could we get a working link to the documentation? Also, when will ARExtensions come back out? Extremely important for most things.

 

Super excited about this! Thanks!

I just don't know what documentation I should be reading.
None of the links in the package manager are working in 2019.1.2f1.

Should I read the documentation for version 2.0 instead?
eg.
https://docs.unity3d.com/Packages/com.unity.xr.arfoundation@2.0/manual/index.html
https://docs.unity3d.com/Packages/com.unity.xr.arcore@2.0/manual/index.html
https://docs.unity3d.com/Packages/com.unity.xr.arkit@2.0/manual/index.html

Sorry for my confusion.

 

This is exciting news but both the documentation (error 404 as of May 16 !)am) https://docs.unity3d.com/Packages/com.unity.xr.arfoundation@2.1/manual/index.html
and the migration guide need work.
For the latter it seems the only option is to scrap your existing 2.0 project and start from scratch.

 

How to deal with missing XR.ARExtensions? We used it very often for camera permission checks.

 

@Saicopate Same. I downloaded the last one from : https://bintray.com/unity/unity-staging/com.unity.xr.arextensions/1.0.0-preview.8 dropped the assets into my project and changed a few name spaces to be UnityEngine.XR from UnityEngine.Experimental.XR and it seems to work

 

Excited!!!!! Is there any planned update of the foundations remote so it works?

 

We have requested the docs publish but it always does lag behind the package publish a bit. Thanks for your patience.

ARExtensions has been factored away and will not be coming back. Now that the subsystem declarations are in C# and distributed via packages there is no need for declaring extension methods in ARExtensions. All of the functionality that had previously been added to ARExtensions has been folded into the subsystems themselves.

Makes sense? Please let me know if there is some aspect of ARExtensions you depend on that didn't get included in the refactor.

-Best,
Mike

 

I'm curious what functionality you needed from ARExtensions. All of the functionality previously in that package is now folded into the refactored subsystem definitions. We don't plan to ever publish another version of ARExtensions.

-Best,
Mike

 

We are working on the refactor of remoting and simulation but it is a large undertaking and will not be available in the near future.

-Best,
Mike

 

The camera permission checks are now part of the CameraSubsystem base class itself and not expressed through ARExtensions. You will not need and should not use ARExtensions with any version of ARFoundation 1.0.

Once the API docs are published it will be more clear but we will also modify the migration guide to make it clear that ARExtensions is no longer applicable.

-Mike

 

@mdurand Unity 2019.2 beta, running 2.1.0- preview -- None of the extension classes were working. Just downgraded to 2.0.2 and my extension methods are working fine now

 

Hi folks,

Just a couple clarifying points:

• The ARExtensions package is no longer necessary (nor would it do anything with current ARFoundation). ARExtensions provided extension methods to the XR Subsystems (which ARFoundation talks to). Those extension methods are now simply part of the API, so ARExtensions is no more. In some cases you can simply remove

  using UnityEngine.XR.ARExtensions;

  and continue on. Some extension methods are now properties (they were methods in ARExtensions because C# does not support extension properties). Some type names also changed slightly (prefixed with XR, for instance).
• Multithreaded rendering is now supported by the ARCore package. This was an ask by lots of teams and we forgot to mention it in the original post.

 

Looks like the documentation sites are live now

https://docs.unity3d.com/Packages/com.unity.xr.arfoundation@2.1/manual/index.html
https://docs.unity3d.com/Packages/com.unity.xr.arcore@2.1/manual/index.html
https://docs.unity3d.com/Packages/com.unity.xr.arkit@2.1/manual/index.html

 

Yes; however, some of the links are broken, and some landing pages are incorrect:

ARKit: https://docs.unity3d.com/Packages/com.unity.xr.arkit@2.1/manual/com.unity.xr.arkit.html
ARSubsystems: https://docs.unity3d.com/Packages/c...ems@2.1/manual/com.unity.xr.arsubsystems.html
ARKit Face Tracking: https://docs.unity3d.com/Packages/com.unity.xr.arkit-face-tracking@1.0/manual/index.html

We're working on fixing those.

 

Oh, god finally! Thank you.

 

This is awesome news, thanks Unity Team, great work!

One thing that would give it even more power is, youtube content. A lot more quick tips and short intro's in video format from the Unity Team would be great please, then keep the videos updated please.

 

It's great news! But I found one problem - when I use ARSession.Reset() method, then camera is frozen. It occurs not only in my project but in AR Foundation Samples too.

 

Hi guys, How to create a prefab from «baked lights», exactly?!

I saw a lot of tutorials that they used realtime lights in their scenes and I know they have no idea how to solve this damn basic problem!!
(same question link)

 

Ya I can *definitely* imagine its a large undertaking ( I even began trying to write my own, but gave up as soon as I realized just how many layers of difficulty it woudl be :/ )! I would say that for me ( and the large majority of other AR / XR developers I chat with ) a working, stable remote is by far the most important feature for Foundations, as it takes iteration cycles from minutes to seconds! If theres any way to move this up the list of priorities, I would love to know!

Anyways, great work / congrats to you and the team on all the ARFoundations stuff. I'm very excited to see whats next and thank you so much for your work and effort!

 

So can we easily test in the editor now, like we could back in 2017 with ARInterface? Its frustrating to watch you guys pour so much energy into new features, while leaving core usability fixes to the way side... Maybe have someone on the team actually try and ship a game, so you can fix some of more realworld issues devs have (like not being able to quickly iterate in the editor). There's no reason we should be deploying to device all the time when its just a camera in space, and some planes, that is all very easily simulatable in the editor.

 

Quick question
Should the Environment Probe demo have a black background? Or should we see the camera image in the background? Just the former for me right now.

2 related questions
1. For the Camera should a Custom Material be enabled in the ARCameraBackground component?

and
2. Should the LWRPAdditionalData component be added to the Camera?

 

Where is the documentation about object tracking?
How do I generate object tracking models? (ARKitReferenceObjectEntry?)
Or should I install the ARKit Plugin from Bitbucket and then use the object scanner from there??

EDIT: I suppose I should use Apple's XCode project to make a scanner: https://developer.apple.com/documentation/arkit/scanning_and_detecting_3d_objects

 

Some feedback on the new object tracker
- It seems like ARTrackedObjectManager.trackedObjectsChanged gets called every frame? it doesn't respect if there actually has been any change?
- ARTrackedObjectManager.trackedObjectsChanged -> removed doesn't get called when I remove the physical object?
- Is there a way to set the tracked object as the scene's root? I'm creating a multiplayer experience around an object, and syncing the location of user's avatars would be much easier with the tracked object as root location..
I can't just move ar session origin to the tracked object's location can I?

 

Some docs links are broken, so I'll provide direct links here:
ARFoundation: https://docs.unity3d.com/Packages/com.unity.xr.arfoundation@2.1/manual/tracked-object-manager.html
ARSubsystems: https://docs.unity3d.com/Packages/com.unity.xr.arsubsystems@2.1/manual/object-tracking.html
ARKit: https://docs.unity3d.com/Packages/com.unity.xr.arkit@2.1/manual/arkit-object-tracking.html

No, the Bitbucket plugin is different and not compatible with ARFoundation.

That's correct. This is documented in the ARKit link I mentioned above.

That's true, though the lists of added, updated, and removed will be empty.

I believe this is ARKit's behavior. It does not track moving objects and (I think) assumes they are persistent. The same is true for planes -- ARKit doesn't track moving surfaces.

 

For those looking for a workaround for LWRP, I have put up this project https://github.com/StigOlavsen/LightweightARFoundation that works with the latest versions of the Lightweight render pipeline (project uses 5.13). Only tested with ARKit, but I see no reason it shouldn't also work with ARCore.

 

Apolgies if I'm repeating my self but struggling to customize how the environment probes work
2 issues.
1. I need to control when and where the probes are updated, it seems to happen at arbitrary intervals right now
and
2. I need to soften the transition between maps when using the LWRP.


The official Apple ARkit documentation suggests both options are possible. ie manually placing of probes and softening the transitions. see here
https://developer.apple.com/documentation/arkit/adding_realistic_reflections_to_an_ar_experience

 

@steego Thanks for sharing but AFAIK, no issues getting LWRP 5.6.1 working with ARFoundation 2.1. I find simply updating the manifest works if it loads with latest 5.71

 

The problem is that 5.6.1 is a bit outdated already, this project works with LWRP 5.13 (should work with 5.10+ I think). I did this because I needed some features in the newer versions of LWRP.

 

Random info.

I've found a lot of "UE4 only" artists are starting to get more acquainted with Unity.

Why? Because of the shadergraph.

That one thing, is driving a lot of artists to unity and they wield the shadergraph like Poseidon's Trident.

praise be

 

On the other hand, it takes 2 seconds to transfer your 3d model and materials from a DCC app to UE4 via datasmith and everything gets converted for you for example from a 3ds Max scene with VRay materials.

Unity's current solution for this is either to import fbx (and loose any lighting and materials), or use the Pixyz solution, (and loose any lighting and materials). I'm my line most people use V-Ray or Redshift in 3ds Max. Until Unity comes up with a solution to this, more UE4 guys will stick to it and even some Unity guys move over to UE4 sadly

Some Artist are also coders, and those may choose to stick to Unity since C# is like childsplay compared to C++

 

@steego Thanks. Misread as 5.1.3 Will look into what new features were added between 5.6.1

At the very least would like to dynamically stop the updating of the environment probe.
Wish someone from Unity would chime in...

 

Hi,

Is there a chance that raycast in 2.1 is missing the option to specify the filter angle when detecting feature points?
This was working in the previous version and now raycast has no overload for this.

 

I've got the same problem...any solution?

 

Further update.
So I've attempted to stop automatic updating of Environment Probes via a double tap gesture.
Tried 3 ways so far
By
1 setting automaticPlacement in AREnviromentProbeManager to false;
and by
2 calling Stop on XREnvironmentProbeSubsystem.
and
3 calling Stop on ARKitEnvironmentProbeSubsystem

All result in crashing my application with
System.UnhandledExceptionEventHandler:Invoke(Object, UnhandledExceptionEventArgs)
(Filename: currently not available on il2cpp Line: -1)

I've modified the Script Execution order to make sure that this script runs last.

Code (csharp):

1.  
2. using System.Collections;
3. using System.Collections.Generic;
4. using UnityEngine;
5. using UnityEngine.XR.ARSubsystems;
6. using UnityEngine.XR.ARFoundation;
7. using UnityEngine.UIElements;
8.  
9. public class DoubleTapDetect : MonoBehaviour
10. {
11.  
12. XREnvironmentProbeSubsystem probeSubsystem;
13. AREnvironmentProbeManager probeManager;
14. ARKitEnvironmentProbeSubsystem arKitSubSytem;
15. public GameObject Panel;
16.  
17. void Update()
18. {
19. for (var i = 0; i < Input.touchCount; ++i)
20. {
21. if (Input.GetTouch(i).phase == TouchPhase.Began)
22. {
23. if (Input.GetTouch(i).tapCount == 2)
24. {
25. // double tap :)
26. //stop env probes from updating;
27. probeManager.automaticPlacement = false;
28. // probeSubsystem.Stop();
29. //arKitSubSytem.Stop();
30. }
31. }
32. }
33.  
34. }
35. }
36.  

 

Any news about a working remote control?

 

@steego Noticed a custom LightEstimator script in the sample session Scene.
What are the advantages over the ARCameraManager component with its Light Estimation Mode?

 

Oh I thought I didn't include that, it's just for setting the ambient lighting and stuff to the values read from the camera manager.

 

I want to modify ARPlaneMeshVisualizer,but finally i find it can not import the namespace of UnityEngine.UI,other scripts like AR Plane Manager has the same problem,How can i do if i want to modify some of those scripts ?? thank you .

 

How can i do if i want to hidden the detected plane ?

 

@steego Forgot to ask, is there any change in how Environment Probes are automatically updated?

 

Would appreciate some comments/clarifications here please:
https://forum.unity.com/threads/ar-foundation-2-1-now-available.678979/#post-4555579

What is your teams process when creating AR Applications? You deploy to device, every_single_time you need to test? Or am I missing something?

Previously w/ ARInterface, I would do 99% of development in the editor, using simple RTS style camera controls to move the camera, it was extremely rapid and effective.

Deploying to device can take upwards of 30s, whereas editor playmode is usually <5. Not to mention the hassle of actually having to pick up a physical device, and find a plane each time. Now you'r close to 60s / test cycle., This is a major difference in a real production project, vastly inflating the amount of time required to build your app.

 

Hi, I use AR Foundation 2.1 in my image tracking application, but tracking only works on Android, and iOS doesn’t catch tracking, doesn’t work in general.
Unity 2019.1.0f2

 

ARPlaneMeshVisualizer

and

ARPlaneManager

are not meant to be modified. What are you trying to do?

 

What does that mean exactly? Does it crash? Is there any relevant output in the Xcode console?