Resolved How to get raw image from Unity AR Foundation

I am developing an AR Foundation App, where I need to send each frame (raw image) to OpenCV; after computation, OpenCV will send me the coordinates where I can place my prefab.
I would appreciate it if someone could put some light on this. If possible, do post an example code. Many thanks in advance for considering my request.

 

https://github.com/Unity-Technologi...04be47da1cc1/Assets/Scripts/CpuImageSample.cs

 

Thank you so much for your response, but the problem I am facing now is that I cannot convert NativeArray<byte> to jpg.

I tried to convert to NativeArray -> bytes[] -> base64. But the problem is I am getting some garbage images, + the resolution of the image is also low.

If possible, could you help me retrieve a high-quality image and convert it to jpg/base64 (an example code of retrieving and converting the image to jpg)? Then I can send that to OpenCV to do the remaining image processing.

Code (CSharp):

1. unsafe public void ChangeTransformation()
2.         {
3.  
4.             if (!cameraManager.TryAcquireLatestCpuImage(out XRCpuImage image))
5.             {
6.                 Debug.Log("Capture AR Buffer returns nothing!!!!!!");
7.                 return;
8.             }
9.  
10.             var conversionParams = new XRCpuImage.ConversionParams(image, TextureFormat.RGB24, XRCpuImage.Transformation.None);
11.  
12.             int size = image.GetConvertedDataSize(conversionParams);
13.  
14.             var buffer = new NativeArray<byte>(size, Allocator.Temp);
15.  
16.             Debug.Log("Image Convertion");
17.             image.Convert(conversionParams, new IntPtr(buffer.GetUnsafePtr()), buffer.Length);
18.             image.Dispose();
19.  
20.             // Code to use this image
21.  
22.             buffer.Dispose();
23.         }

 

Thank you so much

 

Hi, I am trying out something similar. When I capture the image using the TryAcquireLatestCpuImage, the image captured covers more feed than what is actually shown on the screen. So when the CV models returns the coordinates, the prefab gets placed at a different position than expected.

How do I get the exact camera raw image that is displayed on the screen?
Can you please help here.