Bug Iphone Depth Image for hand stencil does not line up with camera feed vs Ipad

When testing the AR Foundation Samples repo on an ipad and iphone (as well as using the depth images in our own apps), we noticed that the Hand stencil never lines up with the camera feed in AR properly on Iphones, but on Ipad the hand stencil matches the camera feed perfectly.

I have made a video of this below, you can easily see this if you overlay the hand stencil texture with the AR Camera background itself or if you simply pay attention to where my fingers are near the edges of the screen, the stencil is quite far off on Iphone and it is causing us issues in our apps.

https://drive.google.com/file/d/1WGngAOJJz9dz-FjtnCaweb78nPT8ulYW/view?usp=sharing

Github Bug: https://github.com/Unity-Technologies/arfoundation-samples/issues/796

 

I'm almost sure this is related:
https://forum.unity.com/threads/arbackgroundcamera-raw-missalignment.1008955/
https://forum.unity.com/threads/arfoundation-camera-intrinsic-params.603385/#post-6711673

 

Did you have any suggestions on a good workaround? I currently have a shader graph where I nab the AR Camera Background renderer texture using:

Code (CSharp):

1. Graphics.Blit(null, targetRenderTexture, m_ARCameraBackground.material);

Then separately, using the DisplayDepthImage.cs script from ARF samples, I save out that onscreen hand stencil texture to the same shader graph and overlay them buuut... obviously things do not line up on iphone. It works fantastic on ipad which is annoying.

 

You'll have to replicate this line from the ARKitBackground.shader in Shader Graph:

Code (CSharp):

1. // Remap the texture coordinates based on the device rotation.
2. float2 texcoord = mul(float3(v.texcoord, 1.0f), _UnityDisplayTransform).xy;

I assume that the iPad has a similar aspect ratio to the camera video, so the displacement is unnoticeable.

 

Thanks, am bogged down in new functionality work for this app I am working on but I will post back next week if I find a way to recreate that, I did see that line in the background camera shader but it is not very clear how to translate that from shader code to a shader graph

I have seen there is a node for the value but I have no idea what the values in a matrix 4x4 actually mean!
https://docs.unity3d.com/Packages/com.unity.shadergraph@6.9/manual/Matrix-4x4-Node.html

 

As the AR Foundation team didn't seem to have a good explanation of how to do this and advised me to do so, I am chasing this up more in this separate thread in the Shader Graph forum: https://forum.unity.com/threads/how...on-in-shader-graph-displaymatrix-4x4.1094767/