Search Unity

Projecting a 3D object on 2D image using worldToCameraMatrix

Discussion in 'AR' started by elos94, Jun 20, 2019.

  1. elos94

    elos94

    Joined:
    Apr 27, 2019
    Posts:
    1
    Hello there!
    I am trying to perform some 2D reprojection starting from a 3D object, based on the information extracted from a Unity scene. I have a scene with the object I need to reproject located in the origin (0, 0, 0), and I am recovering the camera pose (with mainCamera.worldToCameraMatrix), that should encode my 3d rotation matrix and translation 3d vector required for reprojection. The reprojection is done by the classic formula
    2dpoints = K* [R|t] * 3dpoints

    From the camera HORIZONTAL FOV I am recovering the focal length in pixels with:

    Code (CSharp):
    1.  
    2. var half_fov = hfov / 2.0f;
    3. var focal_length = (imageSize / 2) / Math.Tan(half_fov * Math.PI / 180);
    4.  
    which seems correct to me

    The reprojection I am doing is in python, mainly because I have to, so I project the point cloud corresponding to my model.
    But, after I reproject with intrinsic parameters matrix K (created with camera central point and focal_length), the translation seems wrong, because the point cloud I am reprojecting is not overlapping the object in the screenshot of the scene I am reprojecting to. I tried fixing it "manually" and I saw that putting my z axis instead of -3.xxx to -4.xxx seems to be a bit better, but of course is not a correct way to deal that problem and the other axis (x and y) do not have an high value so I don't know if those are also wrong.
    BUT the strange thing is that rotation seems ok, so I don't really know what I am doing wrong.
    I also tried to recover the position of the object by doing mainCamera.transform.InverseTransformDirection of the vector connecting camera and the object of interest, but with no result.

    I am not very confident with these themes, so I may miss something trivial in the visual computing area.
    Any suggestion is really appreciated, even with formulas I am maybe missing/doing wrong!!
    Thank you very much!

    P.S. I hope this is the correct section for this question, I don't know where to post it otherwise. In such case sorry, it's my first post here