Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. We have updated the language to the Editor Terms based on feedback from our employees and community. Learn more.
    Dismiss Notice

Question Blur camera feed (ARCamera)

Discussion in 'AR' started by timbokoppers, Apr 26, 2023.

  1. timbokoppers


    Nov 14, 2016
    I'm using ARFoundation to do facetracking on iOS with blendshapes. I want to slightly blur the camera feed you see in the background, so your actual face (and environment) is not recognisable.
    I know you can use 'Custom Material' on the ARCamera component. Which on iOS, it feeds the material 2 textures (textureY and textureCbCr).

    Can anybody help out either to write a shader which blurs those textures or do it in another way (renderpass? renderfeature? blit?)

    Thanks in advance!
  2. g1zeonik


    Aug 9, 2022
    did you ever figure out a solution to this? Looking for something similar
  3. timbokoppers


    Nov 14, 2016
  4. Cery_


    Aug 17, 2012
    It depends on the renderpipeline that you are using.
    It is easier with the builtin render pipeline but URP is the current way to go.

    We were doing a selective background blur in combination with additional postprocessing so i haven't tested the shorter method that i will describe.
    The tricky part is at which moment you draw the camera image.
    With the default behaviour you can choose between before opaques and after opaques. This is just an optimisation to reduce the amount drawn when for example the whole camera is blocked by a 3D object. Since we are not going for high performance anyways with the blur i would suggest just working back to front.
    1. Draw the camera feed before opaques, you might be able to even use the default material for that
    2. Blur the image
    3. Draw the 3D elements

    For the blur i went for a separable blur shader with a downsampling step and then a horizontal and a vertical pass.
    Here is the render feature setup. (If you are going for the builtin pipeline you have to add the command buffers directly)
    Code (CSharp):
    2. public class BlurPass : ScriptableRenderPass
    3.     {
    4.         private string _passName = "Camera Blur Pass";
    5.         public Material blurMaterial;
    6.         public RenderTexture mask; // ignore the mask stuff
    8.         public BlurPass()
    9.         {
    10.             // This needs to be before opaques
    11.             renderPassEvent = RenderPassEvent.AfterRenderingTransparents;
    12.         }
    14.         public override void Execute(ScriptableRenderContext context, ref RenderingData renderingData)
    15.         {
    16.             if (blurMaterial == null) return;
    17.             var renderer = renderingData.cameraData.renderer;
    18.             var cmd = CommandBufferPool.Get(_passName);
    19.             cmd.BeginSample(_passName);
    20.             RenderTexture tmpRT = RenderTexture.GetTemporary(mask.descriptor);
    21.             cmd.Blit(renderer.cameraColorTarget, tmpRT, blurMaterial, 0); // horizontal pass
    22.             cmd.Blit(tmpRT, renderer.cameraColorTarget, blurMaterial, 1); // vertical pass
    23.             cmd.EndSample(_passName);
    24.             context.ExecuteCommandBuffer(cmd);
    25.             CommandBufferPool.Release(cmd);
    26.         }
    27.     }
    What this does is basically draw the camera image that is already drawn into the current render target into a smaller temporary texture. During this step we also perform the horizontal blur.
    Then in the next operation the vertical blur is performed and drawn back into the render target.

    In our case the process was a little different due to the desired postprocessing and the masked blur. The trick there is basically draw the 3D Stuff, copy transparency, do postprocessing and then compose the camera feed with the transparency.
    KingOfSnake and timbokoppers like this.