Question iOS back cam zoomed

I'm not sure where to post this. I'm trying to use the back cam on iOS, specifically iPads, as a background in my app. I found some code on YouTube which does what I generally need. but the feed is zoomed in compared to the iOS camera.

I just needed to use the webcam image as a background which is why I opted not to use ARkit. Would this issue be resolved if I do use arkit? I don't don't any of the AR functions.

The code I'm using is below.

Code (CSharp):

1. using System.Collections;
2. using System.Collections.Generic;
3. using UnityEngine;
4. using UnityEngine.UI;  
5.  
6.  
7. public class PhoneCamera : MonoBehaviour
8. {
9.  
10.     private bool camAvailable;
11.     private WebCamTexture backCam;
12.     private Texture defaultBackground;
13.  
14.     public RawImage background;
15.     public AspectRatioFitter fit;
16.  
17.     private void Start()
18.     {
19.         defaultBackground = background.texture;
20.         WebCamDevice[] devices = WebCamTexture.devices;
21.  
22.  
23.         // Check if any Camera is available
24.         if (devices.Length == 0)
25.         {
26.             Debug.Log("No Camera detected");
27.             camAvailable = false;
28.             return;
29.  
30.         }
31.  
32.         // Look for back camera
33.         for (int i = 0; i < devices.Length; i++)
34.         {
35.             if (!devices[i].isFrontFacing)
36.             {
37.                 backCam = new WebCamTexture(devices[i].name, Screen.width, Screen.height);
38.  
39.             }
40.         }
41.  
42.         if (backCam == null)
43.         {
44.             Debug.Log("Unable to find back camera");
45.  
46.         }
47.  
48.         backCam.Play();
49.         background.texture = backCam;
50.  
51.         camAvailable = true;
52.  
53.  
54.     }
55.  
56.     private void Update()
57.     {
58.         if (!camAvailable)
59.             return;
60.  
61.         float ratio = (float)backCam.width / (float)backCam.height;
62.         fit.aspectRatio = ratio;
63.         float scaleY = backCam.videoVerticallyMirrored ? -1f: 1f;
64.         //background.rectTransform.localScale = new Vector3(1f, scaleY, 1f);
65.         background.rectTransform.localScale = new Vector3(1f * ratio, scaleY * ratio, 1f * ratio);
66.  
67.         int orient = -backCam.videoRotationAngle;
68.         background.rectTransform.localEulerAngles = new Vector3(0, 0, orient);
69.     }
70. }

 

I got the camera to display in full resolution but there is letterboxing. I'm trying to play around with the canvas and scaler but no solution yet. This seems like a simple thing since the iPad Camera app can display the feed in full res without letterboxing... What am I misunderstanding?

 

When you use ARCameraBackgroud.cs, this script slightly crops the actual camera image by applying the ARCameraFrameEventArgs.displayMatrix to the image.
Here is what you can try:
1. Write a custom AR camera shader that will not apply the displayMatrix to the camera image.
2. Or access the camera image on the CPU.

Some threads on this topic:
https://forum.unity.com/threads/arbackgroundcamera-raw-missalignment.1008955/
https://forum.unity.com/threads/what-exactly-is-the-displaymatrix.1053050/
https://forum.unity.com/threads/dif...-from-cameramanager-trygetlatestimage.740123/
https://forum.unity.com/threads/arf...ge-to-viewport-screenspace-conversion.817545/

 

how did you solve the problem with the camera? I have a problem. In portrait mode, the image is enlarged, but in landscape mode everything is fine, I think the screen resolution is a problem

 

Way 1:

If we use:

Code (CSharp):

1. backCam = new WebCamTexture(devices[i].name)

...without width and height settings, the zoom effect is going away, but the feed resolution becomes very low.

Way 2:

Considering Aspect Mode as Envelope Parent, we can move Raw Image Z position to fix the zoom.

The question is that how we can calculate this shift for different Mobile Devices.
So, we need to calculate when Raw Image Height is fitted to the Viewport.

P.S. Testing in Unity Editor with a Laptop camera shows different results compared to Mobiles.
So, you can use Runtime Inspector and Hierarchy free plugin to see some values in Mobile Runtime.

Way 3 (Solved with Game Object with RawImage!):

Code (CSharp):

1. rectTransform.localScale = new Vector3(
2.                 1f / currentAspectRatio,
3.                 1f / currentAspectRatio,
4.                 1f / currentAspectRatio
5.                 );

------------------------------------------------------------------------------------------------------------

Update: I released Camera Feed on Background (docs) — Unity Asset which operates WebCamTexture class and does all the work for you.