Ar Foundation Camera Resolution

Hey!

Currently I'm trying to change the camera resolution in ar foundation. The accessible ar background image is pretty small (640x480) and basically only useful as reference. I'm aiming for a "okayish" video quality, at least 1280x720, the goal would be 1920x1080.

I can imagine that it causes problems to process a high-resolution image in ar foundation, but maybe there is a way to feed it to some subsystem before compressing and processing it?

Thanks a lot in advance!
Best regards

 

Please see this documentation:
https://docs.unity3d.com/Packages/com.unity.xr.arfoundation@4.0/manual/cpu-camera-image.html

As well as the sample scene here:
https://github.com/Unity-Technologi...s/blob/master/Assets/Scenes/CameraImage.unity

 

I'm accessing the camera Image already, sadly the captured image has the described resolution.
It's about setting the camera configuration to get a better one.

Ar Core (while using ArFoundation?) seems to usually use 640x480 - preview resolution which gets upscaled to 1920x1080. That leads to a blurry image. The camera resolution seems to be changeable using ArCore. In the ArCameraManager class we can get the current camera configuration but I don't know how to set it.

Edit:
I think I'm pretty close now, what I need to access is the native array in the ArCameraManager. Most likely I'm able to get a list of XRCameraConfigurations which can be set (even if not recommended).

Code (CSharp):

1. public NativeArray<XRCameraConfiguration> GetConfigurations(Allocator allocator)
2.  

What's the allocator to turn in? Does anyone know how to access the list?

 

I think I solved it myself, just not sure if it's the right allocator, can somebody confirm?

Code (CSharp):

1. allocator = Allocator.Temp;
2. NativeArray<XRCameraConfiguration> configurations = new NativeArray<XRCameraConfiguration>();
3.  
4. private void GetAvailableConfig()
5.         {
6.             configurations = arCameraManager.GetConfigurations(allocator: allocator);
7.  
8.             foreach(XRCameraConfiguration config in configurations)
9.             {
10.                 Debug.Log(config);
11.             }
12.         }

 

Hey @suitor , how did you manage to use this to set the configuration? When I try to set the ARCameraManager.currentConfiguration = configurations[0]; I get an error: ARCore session is not valid

 

Hey,

you can change the resolution when ARCameraManager's first frameReceived event fires:

Code (CSharp):

1.     public ARCameraManager cameraManager;
2.     bool init = false;
3.  
4.     void OnEnable() {
5.         cameraManager.frameReceived += OnCameraFrameReceived;
6.     }
7.  
8.     void OnDisable() {
9.         cameraManager.frameReceived -= OnCameraFrameReceived;
10.     }
11.  
12.     void OnCameraFrameReceived(ARCameraFrameEventArgs eventArgs) {
13.         if (!init)
14.         {
15.             cameraManager.subsystem.currentConfiguration = cameraManager.GetConfigurations(Allocator.Temp)[1]; //In my case 0=640*480, 1= 1280*720, 2=1920*1080
16.             init = true;
17.         }

 

as nagybalintgyorgy has written, after you received the frame you can change the resolution

Code (CSharp):

1.  
2. private void Start()
3.         {
4.             arCameraManager = this.gameObject.GetComponent<ARCameraManager>();
5.             allocator = Allocator.Temp;
6.         }
7.  
8. public void SetCameraConfig(int configIndex)
9.         {
10.             configurations = arCameraManager.GetConfigurations(allocator: allocator);
11.  
12.             if (arCameraManager.subsystem.currentConfiguration != configurations[configIndex])
13.             {
14.                 arCameraManager.subsystem.currentConfiguration = configurations[configIndex];
15.                 Debug.Log("Set camera configuration: " + configurations[configIndex]);
16.             }
17.  
18.             else
19.             {
20.                 Debug.Log("Configuration didn't change, choosing same configuration");
21.             }
22.         }

the previous snippet shows how to get the available configs, hope that helps!

 

Hey, when I change the arCameraManager configuration, TryAcquireLatestCpuImage return false... I still get the first frame at the default resolution, but false everytime afterward.

At first I was requesting non supported light estimation, but now I get those logs which looks fine :

AndroidPlayer(ADB@127.0.0.1:34999) Configuration Descriptor 0x1 (rank 1): World Facing Camera, Rotation and Orientation, Plane Tracking, Image Tracking, Auto-Focus, Light Estimation (Ambient Intensity), Light Estimation (Ambient Color)
Configuration Descriptor 0x2 (rank 0): World Facing Camera, Rotation and Orientation, Plane Tracking, Image Tracking, Environment Probes, Auto-Focus, Light Estimation (Spherical Harmonics), Light Estimation (Main Light Direction), Light Estimation (Main Light Intensity)
Configuration Descriptor 0x3 (rank 2): User Facing Camera, Rotation Only, Face Tracking, Auto-Focus, Light Estimation (Ambient Intensity), Light Estimation (Ambient Color)

AndroidPlayer(ADB@127.0.0.1:34999) Using session configuration 0x1
Requested Features: World Facing Camera, Rotation and Orientation, Plane Tracking
Supported Features: World Facing Camera, Rotation and Orientation, Plane Tracking
Requested features not satisfied: (None)

Is there any way to know why I can't use higher resolution picture ? 640*480 is quite low in my usecase. Thanks

 

Could you please share your code of setting the camera configuration? Here is a correct example of how to do it:
https://forum.unity.com/threads/ar-...me-rate-defaulting-to-30.934680/#post-6108225

This is also can be related:
https://forum.unity.com/threads/arc...es-not-report-resolution.951876/#post-6217008

 

Okay I tried to make a simple test case and it worked, so after some trials on my whole project I got it working by switching from async conversion of the XRCpuImage to a sync conversion.
I'm not sure why this happens.

 

I too was facing the same issue, it seems that its not the sync / async conversion. You just need to have a delay after you change the configuration and call TryAcquireLatestCpuImage /
TryGetIntrinsics. Both of these methods return false if you instantly call them after changing the camera config

 

Hi everyone, I'm facing the same issue and don't know if I'm doing something wrong. Getting another configuration is not working for me.

I debugged all configurations from GetConfigurations() and checked the dropdown from AR Foundation Samples about it. It seems like I don't have configurations other than 640 x 480 while AR Camera is set to use the front camera.

Am I missing something? Do I have to set up a configuration somewhere else?

I also tried to force the current configuration to a bigger resolution, but that lead to an error on TryAcquireLatestCpuImage (stops working).

I'm looking forward to later applying a machine-learning model to it. So any other method to get the camera image on a higher resolution would work as well.