Search Unity

Question Deformable surface tracking

Discussion in 'AR' started by Michele_P_LF, Jan 3, 2023.

  1. Michele_P_LF

    Michele_P_LF

    Joined:
    Oct 18, 2021
    Posts:
    9
    Hello and happy new year!

    I am looking for a way to perform deformable surface tracking, similar to what Disney do in this video when tracking pages of a book being turned or partially deformed. I have also seen some nice videos on linkedin recently where pillow surfaces can be tracked and materials applied to them at runtime.

    Does anybody know of a tool that could be available to use in AR foundation or would have any ideas/knowledge on how to achieve that kind of tracking?

    Many thanks

    Michele
     
    Michele_P_LF, Jan 3, 2023
    #1

  2. andyb-unity

    andyb-unity

    Unity Technologies

    Joined:
    Feb 10, 2022
    Posts:
    1,062
    AR Foundation is a scripting interface that allows you to deploy cross-platform AR apps to Android, iOS, and OpenXR devices. Unity does not implement AR features-- we simply support a cross-platform interface to platform-level AR implementations.

    Neither ARCore nor ARKit support deformable surface tracking, so you would need a bespoke solution for this use case. (I am not aware of one, but good luck!)
     
    andyb-unity, Jan 3, 2023
    #2

  3. Michele_P_LF

    Michele_P_LF

    Joined:
    Oct 18, 2021
    Posts:
    9
    Thanks @andyb-unity!
    Would it be possible to use the point cloud and something like the below to achieve the tracking?

    Code (CSharp):
    1.  
    2.         var mesh = new Mesh();
    3.         // Set the vertices and triangles of the mesh from the point cloud data
    4.         mesh.vertices = pointCloud.positions;
    5.         mesh.triangles = GenerateTriangles(pointCloud.positions.Length);
    6.         // Recalculate the normals of the mesh
    7.         mesh.RecalculateNormals();
    8.  
    9.         return mesh;
    Or is there anything in the meshing examples that could help? Sorry I haven't had time to look at the code in there yet, I will do so asap.

    Many thanks
     
    Michele_P_LF, Jan 4, 2023
    #3

  4. Neto_Kokku

    Neto_Kokku

    Joined:
    Feb 15, 2018
    Posts:
    1,751
    If it were this simple to do Disney wouldn't need an entire research group to achieve it.
     
    Neto_Kokku, Jan 4, 2023
    #4

  5. andyb-unity

    andyb-unity

    Unity Technologies

    Joined:
    Feb 10, 2022
    Posts:
    1,062
    @Michele_P_LF That code simply generates a mesh from a point cloud. It does not achieve image tracking. To be clear, there is nothing currently in ARCore, ARKit, or AR Foundation that will help with this use case.
     
    andyb-unity, Jan 4, 2023
    #5