Search Unity

Question How to grab real world texture ?

Discussion in 'AR' started by StanWithers, Jul 12, 2021.

  1. StanWithers

    StanWithers

    Joined:
    Aug 31, 2020
    Posts:
    1
    Hi everyone,
    I am trying to replicate the "face in the wall" effect made by MattB from twitter
    Screen Shot 2021-07-12 at 4.22.40 PM.png

    Basically, it's a plane distorted by face mesh.

    The procedure I'm taking is :
    1. place a face game object on a plane (wall);
    2. for each vertex in face mesh, calculate its foot of perpendicular on the plane;
    3. get the screen position of foot, divided by screen size, so I get the "UV" of each foot in screen texture
    4. replace the uv of each face mesh vertex with the UV in procedure 3;
    5. use screen texture (pure camera texture) as the mainTex to render the face.

    The important question is: Am I in the right path ?
    Is there any better ways to grab the real world texture of a plane ?

    If I am in the right path, I'm encountering two problems:
    1. how to get the pure camera texture (without any virtual objects) in each frame ?
    I'm rendering the ARCamera (arcam) to a RenderTexture (rt) in Update() , but I still get the texture with virtual object rendered.

    Code (CSharp):
    1.  
    2.  
    3. RenderTexture.active = rt;
    4. arcam.targetTexture = rt;
    5. arcam.Render();
    6. arcam.targetTexture = null;
    7. RenderTexture.active = null;

    2. I wrote following vertex shader code the get the face mesh "UV" relative to screen texture, but I get really weird result.
    (I am using a half sphere instead of face for testing purpose)

    Code (CSharp):
    1.  
    2. float4 _PlanePoint;
    3. float4 _PlaneNormal;
    4.  
    5. v2f vert (appdata v)
    6. {
    7.   v2f o;
    8.  
    9.  
    10.   float4 worldPos=mul(unity_ObjectToWorld, v.vertex);
    11.   float3 toVertexDir=worldPos.xyz-_PlanePoint.xyz;
    12.   float dist=dot(toVertexDir, normalize(_PlaneNormal.xyz));
    13.   float3 perpendicular=dist*normalize(_PlaneNormal.xyz);
    14.   float3 footWorld=worldPos.xyz-perpendicular;
    15.  
    16.  
    17.   float4 footLocal=mul(unity_WorldToObject, float4(footWorld, 1));
    18.   float4 footInClip=UnityObjectToClipPos(footLocal);
    19.   float4 screenPos=ComputeScreenPos(footInClip);
    20.   float2 footUV=screenPos.xy / screenPos.w;
    21.  
    22.   o.vertex = UnityObjectToClipPos(v.vertex);
    23.  
    24.   o.uv=footUV;
    25.  
    26.   return o;
    27. }
    28.  
    Screen Shot 2021-07-12 at 5.58.54 PM.png


    Could any one pointing out where am I going wrong ?
    Many Thx