Resolved Getting background subtraction from AR Foundation with front camera

I'm trying to get a whole body without environment background from front-facing camera in AR Foundation and send it through Agora sdk. All I want to figure out here is how to correctly convert a rendertexture with stencil to a texture 2D, keeping the alpha value. I modified the "HumanStencil" shader in the AR Foundation samples, and it set the pixel alpha value to 0 depanding on the stencil image from AR Occlusion.

Here is my logic:
1. Get Camera image from AR Camera Manager, and set it to the modified HumanStencil shader's "_CameraTex" property
2. Get Stencil image from AR Occlusion Manager.
3. Use Graphics.Blit to process the stencil image with the modified HumanStencil shader.
4. Convert the new rendertexture to texture 2D
5. Send it to Agora

The step 1 and 2 I just use the CpuImage.cs in AR Foundation samples.

Code (CSharp):

1. unsafe void UpdateCameraImage()
2.     {
3.         if (!cameraManager.TryAcquireLatestCpuImage(out XRCpuImage image))
4.         {
5.             return;
6.         }
7.  
8.         // Choose an RGBA format.
9.         // See XRCpuImage.FormatSupported for a complete list of supported formats.
10.         var format = TextureFormat.RGBA32;
11.  
12.         if (m_CameraTexture == null || m_CameraTexture.width != image.width || m_CameraTexture.height != image.height)
13.         {
14.             m_CameraTexture = new Texture2D(image.width, image.height, format, false);
15.         }
16.  
17.         var conversionParams = new XRCpuImage.ConversionParams(image, format, XRCpuImage.Transformation.MirrorY);
18.         var rawTextureData = m_CameraTexture.GetRawTextureData<byte>();
19.         try
20.         {
21.             image.Convert(conversionParams, new IntPtr(rawTextureData.GetUnsafePtr()), rawTextureData.Length);
22.         }
23.         finally
24.         {
25.             image.Dispose();
26.         }
27.  
28.         m_CameraTexture.Apply();
29.         m_CombineMat.SetTexture("_CameraTex", m_CameraTexture);
30.     }

And here is my Stencil RenderTexture process:

Code (CSharp):

1. void UpdateHumanStencilImage()
2.     {
3.         if (m_RawHumanStencilImage == null)
4.             return;
5.  
6.         if (occlusionManager && occlusionManager.TryAcquireHumanStencilCpuImage(out XRCpuImage image))
7.         {
8.             using (image)
9.             {
10.                 UpdateRawImage(m_RawHumanStencilImage, image);
11.             }
12.         }
13.         else
14.         {
15.             m_RawHumanStencilImage.enabled = false;
16.         }
17.     }
18.  
19.     void UpdateRawImage(RawImage rawImage, XRCpuImage cpuImage)
20.     {
21.         var texture = rawImage.texture as Texture2D;
22.  
23.         if (texture == null || texture.width != cpuImage.width || texture.height != cpuImage.height)
24.         {
25.             texture = new Texture2D(cpuImage.width, cpuImage.height, cpuImage.format.AsTextureFormat(), false);
26.             rawImage.texture = texture;
27.         }
28.  
29.         var conversionParams = new XRCpuImage.ConversionParams(cpuImage, cpuImage.format.AsTextureFormat(), XRCpuImage.Transformation.MirrorY);
30.  
31.         var rawTextureData = texture.GetRawTextureData<byte>();
32.  
33.         // Make sure the destination buffer is large enough to hold the converted data (they should be the same size)
34.         Debug.Assert(rawTextureData.Length == cpuImage.GetConvertedDataSize(conversionParams.outputDimensions, conversionParams.outputFormat),
35.             "The Texture2D is not the same size as the converted data.");
36.  
37.         // Perform the conversion.
38.         cpuImage.Convert(conversionParams, rawTextureData);
39.         texture.Apply();
40.  
41.         BufferTexture.graphicsFormat = UnityEngine.Experimental.Rendering.GraphicsFormat.R8G8B8A8_SNorm;
42.         Graphics.Blit(texture, BufferTexture, m_CombineMat);
43.     }

These are called after each camera frame from ARCameraManager

Code (CSharp):

1. void OnCameraFrameReceived(ARCameraFrameEventArgs eventArgs)
2.     {
3.         UpdateCameraImage();
4.         UpdateHumanStencilImage();
5.         ShareRenderTexture();
6.     }

The BufferTexture is the RenderTexture with only the whole body and no environment background.

My problem is here. To test, I apply the BufferTexture to a RawImage on the screen and the BufferTexture works fine with the graphicsFormat GraphicsFormat.R8G8B8A8_SNorm. But when I try to convert it to a texture 2D and display in a RawImage on the screen, it doesn't work. Here is my code:

Code (CSharp):

1. private void ShareRenderTexture()
2.     {
3.         if (BufferTexture == null) // offlined
4.         {
5.             return;
6.         }
7.  
8.         Texture2D tex = new Texture2D(BufferTexture.width, BufferTexture.height, UnityEngine.Experimental.Rendering.GraphicsFormat.R8G8B8A8_SNorm, UnityEngine.Experimental.Rendering.TextureCreationFlags.MipChain);
9.         RenderTexture.active = BufferTexture;
10.         tex.ReadPixels(new Rect(0, 0, BufferTexture.width, BufferTexture.height), 0, 0);
11.         tex.Apply();
12.  
13.         m_agoraImage.texture = tex;
14.  
15.         byte[] bytes = tex.GetRawTextureData();
16.         int size = Marshal.SizeOf(bytes[0]) * bytes.Length;
17.  
18.         // sends the Raw data contained in bytes to Agora
19.         StartCoroutine(PushFrame(bytes, (int)tex.width, (int)tex.height,
20.             () =>
21.             {
22.                 bytes = null;
23.             }));
24.         RenderTexture.active = null;
25.     }

I tried GraphicsFormat.R8G8B8A8_SNorm the same as the BufferTexture. This gave me a result of a solid white, shown in the file

I also tried TextureFormat.RGBA32 and TextureFormat.BGRA32 with a result of semi-transparent gray. And when I tried TextureFormat.RGBAHalf, I got a full transparent image, no human body in it.
Did I do it wrong anywhere? I still think it is the format issue. Another question is why the camera image is rotated 90 degree? How can I make it the right rotation?

 

@TreyK-47 Wow, this was golden information, I couldn't find this information anywhere else!

I was wondering if any advancements here have been made since - is there an easy way to extract the front-facing stencil image using ARFoundation and use it directly for background removal? In my use case, I would be using face tracking simultaneously.

@wuy420 Did you manage to separate the body from the background eventually? I'm desperately trying to achieve this as well.

 

Yes, I finally made it work. The problem was the texture format being used. I ended up setting the RenderTexture format as GraphicsFormat.R8G8B8A8_UNorm and the Texture2D format as TextureFormat.RGBA32. If you just want to use the background removal locally, then the rendertexture is enough. The Texture2D is used for sending image through Agora.