Search Unity

  1. Read here for Unity's latest plans on OpenXR.
    Dismiss Notice

Help Wanted OpenCV for Unity - Aruco pose position does not match real-space relative to camera

Discussion in 'AR' started by kukewilly, Sep 16, 2021.

  1. kukewilly

    kukewilly

    Joined:
    Jan 3, 2019
    Posts:
    23
    Hey All,

    I am having an issue getting accurate aruco pose estimation. The transform of the instantiated AR object relative to camera is a factor ~2 smaller than it should be relative to the camera position. For example, when the camera is z=.40m away from the marker in real-space, the AR object is only ~.20m away in virtual space.

    I thought it might be something with the length parameters I measured off the chessboard or aruco markers?

    I used these functions and corresponding measurements:

    Aruco.estimatePoseSingleMarkers (corners, markerLength, camMatrix, distCoeffs, rvecs, tvecs);

    where markerLength = 0.122m and is the distance across the 4x4 aruco (not including the black boarder)

    Code (CSharp):
    1. private void EstimatePoseCanonicalMarker (Mat rgbMat)
    2.         {
    3.             Aruco.estimatePoseSingleMarkers (corners, markerLength, camMatrix, distCoeffs, rvecs, tvecs);
    4.  
    5.             for (int i = 0; i < ids.total (); i++) {
    6.                 using (Mat rvec = new Mat (rvecs, new OpenCVForUnity.CoreModule.Rect (0, i, 1, 1)))
    7.                 using (Mat tvec = new Mat (tvecs, new OpenCVForUnity.CoreModule.Rect (0, i, 1, 1))) {
    8.                     // In this example we are processing with RGB color image, so Axis-color correspondences are X: blue, Y: green, Z: red. (Usually X: red, Y: green, Z: blue)
    9.                     Calib3d.drawFrameAxes (rgbMat, camMatrix, distCoeffs, rvec, tvec, markerLength * 0.5f);
    10.  
    11.                     // This example can display the ARObject on only first detected marker.
    12.                     if (i == 0) {
    13.                         UpdateARObjectTransform (rvec, tvec);
    14.                     }
    15.                 }
    16.             }
    17.         }




    CalcChessboardCorners (patternSize, squareSize, objectPoint, markerType);

    where squareSize = .0245m and is the distance(m) between 1 square on the chessboard

    Code (CSharp):
    1.         private void CalcChessboardCorners (Size patternSize, float squareSize, MatOfPoint3f corners, MarkerType markerType)
    2.         {
    3.             if ((int)(patternSize.width * patternSize.height) != corners.rows ()) {
    4.                 Debug.Log ("Invalid corners size.");
    5.                 corners.create ((int)(patternSize.width * patternSize.height), 1, CvType.CV_32FC3);
    6.             }
    7.  
    8.             const int cn = 3;
    9.             float[] cornersArr = new float[corners.rows () * cn];
    10.             int width = (int)patternSize.width;
    11.             int height = (int)patternSize.height;
    12.  
    13.             switch (markerType) {
    14.             default:
    15.             case MarkerType.ChessBoard:
    16.             case MarkerType.CirclesGlid:
    17.                 for (int i = 0; i < height; ++i) {
    18.                     for (int j = 0; j < width; ++j) {
    19.                         cornersArr [(i * width * cn) + (j * cn)] = j * squareSize;
    20.                         cornersArr [(i * width * cn) + (j * cn) + 1] = i * squareSize;
    21.                         cornersArr [(i * width * cn) + (j * cn) + 2] = 0;
    22.                     }
    23.                 }
    24.                 corners.put (0, 0, cornersArr);
    25.  
    26.                 break;
    27.             case MarkerType.AsymmetricCirclesGlid:
    28.                 for (int i = 0; i < height; ++i) {
    29.                     for (int j = 0; j < width; ++j) {
    30.                         cornersArr [(i * width * cn) + (j * cn)] = (2 * j + i % 2) * squareSize;
    31.                         cornersArr [(i * width * cn) + (j * cn) + 1] = i * squareSize;
    32.                         cornersArr [(i * width * cn) + (j * cn) + 2] = 0;
    33.                     }
    34.                 }
    35.                 corners.put (0, 0, cornersArr);
    36.  
    37.                 break;
    38.             }
    39.         }

    I've tried doubling some of these values and recalibrating to see if it makes the z value close to .40m but it doesn't seem to change. Which makes me feel kind of clueless. I take about 50 calibration photos and they're pretty diverse. I try to capture a range of depth and angle on the camera.

    Any ideas?

    Thanks and let me know if you need more info.
     
unityunity