Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Resolved Using Graphics.Blit to copy background texture is not worked with ARKit

Discussion in 'AR' started by cloudending, Jan 26, 2022.

  1. cloudending

    cloudending

    Joined:
    Mar 25, 2019
    Posts:
    15
    I try to copy background in ARCameraBacground using Graphics.Blit.
    Code (CSharp):
    1. void UpdateFrameData(int index, ARCameraFrameEventArgs eventArgs)
    2. {
    3. /*Other code*/
    4. Graphics.Blit(null, frameData.texture, material);
    5. }
    frameData.texture is a cached texture create by:
    Code (CSharp):
    1. frameData.texture = new RenderTexture(eventArgs.textures[0].width, eventArgs.textures[0].height, 0, GraphicsFormat.R8G8B8A8_SNorm);
    the material's shader is the same as ARFoundation Unlit/ARKitBackground.shader.
    I call UpdateFrameData on every
    Code (CSharp):
    1. void OnCameraFrameReceived(ARCameraFrameEventArgs eventArgs)
    and I try to set frameData.texture to a RawImage.texture. But the RawImage is dark.
    This situation only happens with ARKit. But Copy texture with ARCore and using ARCoreBackgroud.shader works well.
    Is Graphics.Blit not working or something
     
  2. KyryloKuzyk

    KyryloKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    1,128
    Here is my best understanding of what's happening when you call Graphics.Blit(Texture sourceTexture, RenderTexture destRT, Material material). More clarification from someone who has more expertise in rendering would be great.
    1. Unity sets the sourceTexture to the _MainTex property of the material's shader.
    2. Unity copies the sourceTexture to the destRT using the material.

    The issue with step 1 is that ARKitBackground.shader doesn't have the _MainTex property, so the result will be undefined.

    This method works both on Android and iOS:
    Code (CSharp):
    1. using JetBrains.Annotations;
    2. using UnityEngine;
    3. using UnityEngine.Assertions;
    4.  
    5. public static void BlitCameraBackgroundTexture([NotNull] Texture2D sourceTexture, [NotNull] RenderTexture destRt, [NotNull] Material materialForTextureBlit) {
    6.     Assert.IsNotNull(sourceTexture, "sourceTexture != null");
    7.     Assert.IsNotNull(destRt, "destRt != null");
    8.     if (materialForTextureBlit.HasProperty("_MainTex")) {
    9.         Graphics.Blit(sourceTexture, destRt, materialForTextureBlit);
    10.     } else {
    11.         Graphics.Blit(sourceTexture, destRt);
    12.     }
    13. }
     
    Last edited: Jan 27, 2022
  3. todds_unity

    todds_unity

    Joined:
    Aug 1, 2018
    Posts:
    324
    KyryloKuzyk likes this.
  4. cloudending

    cloudending

    Joined:
    Mar 25, 2019
    Posts:
    15
    thanks. But I don't know what the sourceTexture is. In ARKitBackground.shader there has
    _textureY and _textureCbCr
     
  5. cloudending

    cloudending

    Joined:
    Mar 25, 2019
    Posts:
    15
    thanks. below is my code. but RawImage is still black.
    Code (CSharp):
    1.            
    2.             Material material = customMaterial;
    3.             int id = latest_frame_index % _videoTextureCached.Length;
    4.             var frameData = _videoTextureCached[id];
    5.             frameData.index = index;
    6.  
    7.             if (material != null)
    8.             {
    9.                 var count = eventArgs.textures.Count;
    10.                 for (int i = 0; i < count; ++i)
    11.                 {
    12.                     if(frameData.texture == null){
    13.                         frameData.texture = new RenderTexture(eventArgs.textures[0].width, eventArgs.textures[0].height, 0, GraphicsFormat.R8G8B8A8_SNorm);
    14.                     }
    15.                     material.SetTexture(eventArgs.propertyNameIds[i], eventArgs.textures[i]);
    16.                 }
    17.  
    18.                 if (eventArgs.displayMatrix.HasValue)
    19.                 {
    20.                     material.SetMatrix(k_DisplayTransformId, eventArgs.displayMatrix.Value);
    21.                 }
    22.  
    23.                 SetMaterialKeywords(material, eventArgs.enabledMaterialKeywords, eventArgs.disabledMaterialKeywords);
    24.  
    25.                var commandBuffer = new CommandBuffer();
    26.                 commandBuffer.name = "AR Camera Background Blit Pass";
    27.                 // var texture = !m_ArCameraBackground.material.HasProperty("_MainTex") ? null : m_ArCameraBackground.material.GetTexture("_MainTex");
    28.                 Graphics.SetRenderTarget(frameData.texture.colorBuffer, frameData.texture.depthBuffer);
    29.                 commandBuffer.ClearRenderTarget(true, false, Color.clear);
    30.                 commandBuffer.Blit(null, BuiltinRenderTextureType.CurrentActive, material);
    31.                 Graphics.ExecuteCommandBuffer(commandBuffer);
    32.  
    33.                 if(m_RawImage){
    34.                     m_RawImage.texture = frameData.texture;
    35.                 }
    36.             }
    37.             }
     
  6. cloudending

    cloudending

    Joined:
    Mar 25, 2019
    Posts:
    15
    Finally.I use
      new RenderTexture(eventArgs.textures[0].width, eventArgs.textures[0].height, 24, GraphicsFormat.R8G8B8A8_SNorm);

    change depth to 24 instead of 0. It works