Search Unity

  1. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice
  2. Ever participated in one our Game Jams? Want pointers on your project? Our Evangelists will be available on Friday to give feedback. Come share your games with us!
    Dismiss Notice

ARFoundation; mirror/flip horizontally ARCamera image with virtual objects?

Discussion in 'Handheld AR' started by neutrum_unity, Jun 14, 2020.

  1. neutrum_unity


    Sep 4, 2019
    I am struggling with how can I flip/mirror screen to create "real mirror" while iPhone is in landscape mode (or portrait doesn't matter at this stage) and connected to TV.

    I am successfully using this code:
    Code (CSharp):
    1. // Attempt to get the latest camera image. If this method succeeds,
    2.         // it acquires a native resource that must be disposed (see below).
    3.         XRCameraImage image;
    4.         if (!cameraManager.TryGetLatestImage(out image))
    5.         {
    6.             return;
    7.         }
    9.         // Once we have a valid XRCameraImage, we can access the individual image "planes"
    10.         // (the separate channels in the image). XRCameraImage.GetPlane provides
    11.         // low-overhead access to this data. This could then be passed to a
    12.         // computer vision algorithm. Here, we will convert the camera image
    13.         // to an RGBA texture and draw it on the screen.
    15.         // Choose an RGBA format.
    16.         // See XRCameraImage.FormatSupported for a complete list of supported formats.
    17.         var format = TextureFormat.RGBA32;
    19.         if (m_Texture == null || m_Texture.width != image.width || m_Texture.height != image.height)
    20.         {
    21.             m_Texture = new Texture2D(image.width, image.height, format, false);
    22.         }
    24.         // Convert the image to format, flipping the image across the Y axis.
    25.         // We can also get a sub rectangle, but we'll get the full image here.
    26.         var conversionParams = new XRCameraImageConversionParams(image, format, CameraImageTransformation.MirrorY | CameraImageTransformation.MirrorX);
    28.         // Texture2D allows us write directly to the raw texture data
    29.         // This allows us to do the conversion in-place without making any copies.
    30.         var rawTextureData = m_Texture.GetRawTextureData<byte>();
    31.         try
    32.         {
    33.             image.Convert(conversionParams, new IntPtr(rawTextureData.GetUnsafePtr()), rawTextureData.Length);
    34.         }
    35.         finally
    36.         {
    37.             // We must dispose of the XRCameraImage after we're finished
    38.             // with it to avoid leaking native resources.
    39.             image.Dispose();
    40.         }
    42.         // Apply the updated texture data to our texture
    43.         m_Texture.Apply();
    45.         // Set the RawImage's texture so we can visualize it.
    46.         //m_RawImage.texture = m_Texture;
    47.         material.SetTexture(MainTexId, m_Texture);
    I practically converted the latest image from ARCameraManager and set it to the custom material texture on ARCameraBackground. It does work but there are no virtual objects. It makes sense because as I understand it I am taking the frame from live camera feed.

    So how can I make it flipped/mirrored and show the virtual objects? Any pointers, suggestions?

    ad: I tried to use .blit() with ARCameraBackground.material and RenderTexture and then set it to RawImage stretched on Canvas, but that was really expensive.

    iPhone X iOS 13.5.1
    ARFoundation Preview.4 - 3.0.0
    Unity 2019.3.14f1