Search Unity

Accessing the Camera Image on the CPU - AR Foundations 2.1 (Old Documentation)

Discussion in 'Handheld AR' started by gdediego, Sep 5, 2019.

  1. gdediego

    gdediego

    Joined:
    Apr 30, 2019
    Posts:
    3
    I'm currently trying to access the camera image on the CPU via AR foundations 2.1 and I came across the following documentation however, it seems as if this is not compatible with ARF 2.1. Some errors encountered include ARSubsystemManager Does Not Exist on the following line:

    ARSubsystemManager.cameraFrameReceived += OnCameraFrameReceived;


    Original Documentation: https://docs.unity3d.com/Packages/com.unity.xr.arfoundation@2.2/manual/cpu-camera-image.html
     
  2. vertis_winters

    vertis_winters

    Joined:
    Jun 8, 2019
    Posts:
    2
    gdediego likes this.
  3. gdediego

    gdediego

    Joined:
    Apr 30, 2019
    Posts:
    3
    Thanks! Was able to refactor the code to fit the naming conventions found in ARCameraManager.cs and all my errors are now gone. I'll definitely take a look at the sample repo but in case anyone stumbles on this issue again here's the code that works for me..

    Code (CSharp):
    1.  
    2. using System;
    3. using Unity.Collections;
    4. using Unity.Collections.LowLevel.Unsafe;
    5. using UnityEngine;
    6. using UnityEngine.XR.ARFoundation;
    7. using UnityEngine.XR.ARSubsystems;
    8.  
    9.  
    10. public class Test : MonoBehaviour
    11. {
    12.     public ARCameraManager cameraManager;
    13.  
    14.     Texture2D m_Texture;
    15.  
    16.     void OnEnable()
    17.     {
    18.         cameraManager.frameReceived += OnCameraFrameReceived;
    19.     }
    20.  
    21.     void OnDisable()
    22.     {
    23.         cameraManager.frameReceived -= OnCameraFrameReceived;
    24.     }
    25.  
    26.     unsafe void OnCameraFrameReceived(ARCameraFrameEventArgs eventArgs)
    27.     {
    28.         XRCameraImage image;
    29.         if (!cameraManager.TryGetLatestImage(out image))
    30.             return;
    31.  
    32.         var conversionParams = new XRCameraImageConversionParams
    33.         {
    34.             // Get the entire image
    35.             inputRect = new RectInt(0, 0, image.width, image.height),
    36.  
    37.             // Downsample by 2
    38.             outputDimensions = new Vector2Int(image.width / 2, image.height / 2),
    39.  
    40.             // Choose RGBA format
    41.             outputFormat = TextureFormat.RGBA32,
    42.  
    43.             // Flip across the vertical axis (mirror image)
    44.             transformation = CameraImageTransformation.MirrorY
    45.         };
    46.  
    47.         // See how many bytes we need to store the final image.
    48.         int size = image.GetConvertedDataSize(conversionParams);
    49.  
    50.         // Allocate a buffer to store the image
    51.         var buffer = new NativeArray<byte>(size, Allocator.Temp);
    52.  
    53.         // Extract the image data
    54.         image.Convert(conversionParams, new IntPtr(buffer.GetUnsafePtr()), buffer.Length);
    55.  
    56.         // The image was converted to RGBA32 format and written into the provided buffer
    57.         // so we can dispose of the CameraImage. We must do this or it will leak resources.
    58.         image.Dispose();
    59.  
    60.         // At this point, we could process the image, pass it to a computer vision algorithm, etc.
    61.         // In this example, we'll just apply it to a texture to visualize it.
    62.  
    63.         // We've got the data; let's put it into a texture so we can visualize it.
    64.         m_Texture = new Texture2D(
    65.             conversionParams.outputDimensions.x,
    66.             conversionParams.outputDimensions.y,
    67.             conversionParams.outputFormat,
    68.             false);
    69.  
    70.         m_Texture.LoadRawTextureData(buffer);
    71.         m_Texture.Apply();
    72.  
    73.         // Done with our temporary data
    74.         buffer.Dispose();
    75.     }
    76. }
    77.  
     
  4. gdediego

    gdediego

    Joined:
    Apr 30, 2019
    Posts:
    3
    vertis_winters likes this.