Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.

Question Deformable surface tracking

Discussion in 'AR' started by Michele_P_LF, Jan 3, 2023.

  1. Michele_P_LF

    Michele_P_LF

    Joined:
    Oct 18, 2021
    Posts:
    9
    Hello and happy new year!

    I am looking for a way to perform deformable surface tracking, similar to what Disney do in this video when tracking pages of a book being turned or partially deformed. I have also seen some nice videos on linkedin recently where pillow surfaces can be tracked and materials applied to them at runtime.

    Does anybody know of a tool that could be available to use in AR foundation or would have any ideas/knowledge on how to achieve that kind of tracking?

    Many thanks

    Michele
     
    Michele_P_LF, Jan 3, 2023
    #1

  2. andyb-unity

    andyb-unity

    Unity Technologies

    Joined:
    Feb 10, 2022
    Posts:
    525
    AR Foundation is a scripting interface that allows you to deploy cross-platform AR apps to Android, iOS, and OpenXR devices. Unity does not implement AR features-- we simply support a cross-platform interface to platform-level AR implementations.

    Neither ARCore nor ARKit support deformable surface tracking, so you would need a bespoke solution for this use case. (I am not aware of one, but good luck!)
     
    andyb-unity, Jan 3, 2023
    #2

  3. Michele_P_LF

    Michele_P_LF

    Joined:
    Oct 18, 2021
    Posts:
    9
    Thanks @andyb-unity!
    Would it be possible to use the point cloud and something like the below to achieve the tracking?

    Code (CSharp):
    1.  
    2.         var mesh = new Mesh();
    3.         // Set the vertices and triangles of the mesh from the point cloud data
    4.         mesh.vertices = pointCloud.positions;
    5.         mesh.triangles = GenerateTriangles(pointCloud.positions.Length);
    6.         // Recalculate the normals of the mesh
    7.         mesh.RecalculateNormals();
    8.  
    9.         return mesh;
    Or is there anything in the meshing examples that could help? Sorry I haven't had time to look at the code in there yet, I will do so asap.

    Many thanks
     
    Michele_P_LF, Jan 4, 2023
    #3

  4. Neto_Kokku

    Neto_Kokku

    Joined:
    Feb 15, 2018
    Posts:
    1,629
    If it were this simple to do Disney wouldn't need an entire research group to achieve it.
     
    Neto_Kokku, Jan 4, 2023
    #4

  5. andyb-unity

    andyb-unity

    Unity Technologies

    Joined:
    Feb 10, 2022
    Posts:
    525
    @Michele_P_LF That code simply generates a mesh from a point cloud. It does not achieve image tracking. To be clear, there is nothing currently in ARCore, ARKit, or AR Foundation that will help with this use case.
     
    andyb-unity, Jan 4, 2023
    #5