Question Can't send "ARPoseDriver.cs" position/rotation to separate GameObject

Hey all. Long time reader, first time posting!

I'm trying to modify AR Foundation's "ARPoseDriver.cs" to send the resulting AR device position/rotation to a separate game object and I can't understand why it's not working.

Ultimately I want to just disable position/rotation on the "AR Camera" and have it be managed by the separate object (more on that below), but for now I'm not even able to send the resulting values to a dummy Vector3 and Quaternion for testing.

I've declared 3 values for testing in lines 17-19:
===

Code (CSharp):

1. using System;
2. using System.Collections;
3. using System.Collections.Generic;
4. using System.Runtime.CompilerServices;
5. using UnityEngine;
6. using UnityEngine.XR;
7.  
8. namespace UnityEngine.XR.ARFoundation
9. {
10.     /// <summary>
11.     /// The ARPoseDriver component applies the current Pose value of an AR device to the transform of the GameObject.
12.     /// </summary>
13.  
14.     [HelpURL(HelpUrls.ApiWithNamespace + nameof(ARPoseDriver) + ".html")]
15.     public class ARPoseDriver : MonoBehaviour
16.     {
17.         public Transform DigitalPlayer;
18.         public Vector3 CameraPosition;
19.         public Quaternion CameraRotation;

===
Here's an image of how that shows in Unity Inspector (look at the public variables now displaying under "AR Pose Driver script):



And later starting at "PerformUpdate()" (which I believe overrides the regular "Update()" I'm trying to update the position value to that dummy "CameraPosition" Vector3 I've got set up to at least visualize if it's receiving values (line 15 and the 2 comments are what I added):
====

Code (CSharp):

1. #endif // UNITY_UNITY_2020_1_OR_NEWER
2.         }
3.         void Update() => PerformUpdate();
4.         void OnBeforeRender() => PerformUpdate();
5.         void PerformUpdate()
6.         {
7.             if (!enabled)
8.                 return;
9.             //This line also does not affect pose data
10.             var updatedPose = GetPoseData();
11.             //Even commenting out the below two lines have no effect on the camera tracking so line 70-74 has no effect on the camera location
12.             if (updatedPose.position.HasValue)
13.             {
14.                 transform.localPosition = updatedPose.position.Value;
15.                 CameraPosition = updatedPose.position.Value;
16.             }
17.             if (updatedPose.rotation.HasValue)
18.                 transform.localRotation = updatedPose.rotation.Value * updatedPose.rotation.Value;
19.         }

====
The rest of the file is just what's in AR Foundation 4.1.7.

But no matter what I try (no errors in the code) I can't get that "CameraPosition" value to show anything other than 0,0,0 when I run the editor.

I tried a different route of "breaking" the code to see where exactly the position/rotation gets called, but even when I comment out the line "var updatePose = GetPoseData();" the camera STILL gets all of the position/rotation data from ARkit.

I can't understand where in this "ARPoseDriver.cs" code that the actual location/rotation gets returned so I can send the values to a separate GameObject.

Anyone able to help?

For context, I've had this method running in Unity 2017 for the last few years without a problem, but since I want to take advantage of the newer ARkit features I'm trying to get this working now in Unity 2021.1.22.
I'm doing this all for location-based games with "QR code" tracked images on the walls, so in my Unity 2017 version rather than use "QR codes" to move the entire scene to the tracked image I had it reverse-engineered so that when a "QR code" on the floor gets seen in the physical room it's the PLAYER that gets moved around the 3D environment (rather than the 3D environment getting moved around the player). This way their 3D position always corresponds to the world position in the actual room, which makes it easy to quickly set up the game for multiplayer by just dropping a QR code on the ground and also allows me to use NavMeshes (since the 3D room never actually moves).

Sending the iPhone's Camera location to a separate object and having the "AR Camera" location calculated separately is the first step to making this all work, which has worked great in my Unity 2017 build.

AR Foundation has added a ton of functionality that I've really loved, but I can't find a way to "break" it to send the camera position/rotation data to a separate GameObject.

PS, I'm also using the "AR Foundation Remote 2.0" plugin and it's been a dream for AR prototyping: https://assetstore.unity.com/packages/tools/utilities/ar-foundation-remote-2-0-201106

 

Shoutout to @KirillKuzyk for "AR Foundation Remote 2.0", ps: https://assetstore.unity.com/packages/tools/utilities/ar-foundation-remote-2-0-201106

Just bought it yesterday and even getting to this point of failure/asking for help probably would have taken me an extra week of testing if I had to run iOS builds each time and send the Debug.Log to the iPhone display like I used to.
Anyone working with AR Foundation should definitely buy this!

 

This is one of the limitations of the AR Foundation Remote plugin: the plugin doesn't implement the native XRInputSystem. Instead, it finds the ARPoseDrive component sitting on the ASSessionOrigin.camera and applies position and rotation directly to the camera's game object.

METHOD 1
There is a simple way to write a custom ARPoseDriver that will work in the Editor for AR Foundation <= 4.1.7 and Unity < 2020.1:

1. Make a copy of the existing ARPoseDriver.cs script and rename the class inside the copied script.
2. Add this line on top of your copied script:
   

   using InputTracking = ARFoundationRemote.Runtime.InputTracking;

METHOD 2
There is another way to get the camera pose that will work regardless of AR Foundation or Unity version. I tested it with Unity 2021.2.6 and AR Foundation 4.2.1, but there is a chance that this code will stop working in future Unity releases because AR Foundation is currently making a transition to New Input System.
You can also wrap the

using InputTracking...

line with a version define to decouple your project from the plugin.

Code (CSharp):

1. using System.Collections.Generic;
2. using UnityEngine;
3. using UnityEngine.XR;
4. using InputTracking = ARFoundationRemote.Runtime.InputTracking;
5.  
6.  
7. public static class ARFoundationRemoteUtils {
8.     public static Pose? GetCameraPose() {
9.         var states = new List<XRNodeState>();
10.         InputTracking.GetNodeStates(states);
11.         foreach (var state in states) {
12.             if (state.nodeType == XRNode.CenterEye) {
13.                 if (!state.TryGetPosition(out var position)) {
14.                     return null;
15.                 }
16.  
17.                 if (!state.TryGetRotation(out var rotation)) {
18.                     return null;
19.                 }
20.  
21.                 return new Pose(position, rotation);    
22.             }
23.         }
24.  
25.         return null;
26.     }
27. }
28.  

 

Thanks for this link, just purchased!

 

Ah, thank you. I was wondering if it had something to do with your plugin. I can't expect Unity Desktop to perfectly reproduce iOS, nor do I expect your remote plugin to work for all fringe use cases like mine. Thanks for your support and I'm glad you just got another sale from this thread via @blamejane !

So you're saying I need to downgrade to 2020.1 for this simple method you posted to work? Would a further iteration like 2020.1.17 work just 2020.1.0, exactly? Or are you saying by "<" that I need to use the last version of 2019?

I might also just try upgrading to Unity 2021.2.6 and AR Foundation 4.2.1, since you said that worked.
I'd cut the "ARPoseDriver.cs" and put this modded "CameraPoseTest.cs" code on the same "AR Camera" GameObject with the stock "ARCameraManager.cs", right?

 

I'm sorry for the small confusion caused by my answer. Please see the edited response above.
Please use your current Unity and AR Foundation. The main philosophy of the plugin is that it should not influence your existing project in any way.

If you use AR Foundation <= 4.1.7 AND Unity < 2020.1, then use METHOD 1.
Else, use METHOD 2.

So if your Unity is 2020.1.17 (which is > than 2020.1), please use METHOD 2.

It depends on what you're trying to achieve. If you just want to manipulate some GameObject's position, you can call the code above at any point to get the camera pose and apply it to the game object. For example:

Code (CSharp):

1. using UnityEngine;
2.  
3.  
4. public class ApplyCameraPoseToThisGameObject : MonoBehaviour {
5.     void Update() {
6.         var cameraPose = ARFoundationRemoteUtils.GetCameraPose();
7.         if (cameraPose.HasValue) {
8.             transform.position = cameraPose.Value.position;
9.             transform.rotation = cameraPose.Value.rotation;
10.         }
11.     }
12. }
13.  

 

@KirillKuzyk Thanks so much! I'll dig into this over the next 1-2 weeks and see how it goes!

 

UPDATE:
The latest version of the AR Foundation Remote plugin now supports the native XRInputSystem in this setup: Unity >= 2020.1, AR Foundation >= 4.0. This means that the ARPoseDriver.cs now works exactly the same way it works on real devices and the aforementioned workarounds are no longer needed.

 

Hey @KirillKuzyk , sorry for my delayed reply! I actually thought I wrote you already!

Well I HAD been busy the last few months shifting to some other priorities, but now I'm writing you at your Gmail address about a few possible paid projects involving ARkit, including this one.

I'll send you the email shortly. Hope you've been well!

 

Hey @KirillKuzyk ,

Just sent you an email to the gmail address on your GitHub repo.

You could also just schedule a 15-30min meeting with me if that's faster than thinking through a reply (sometimes I find it's faster to talk through heavy spec-related projects like this than to try to email back and forth to clarify points, but up to you).

Hope you're having a good day!

-J