Question OpenCV for Unity - Aruco pose position does not match real-space relative to camera

Hey All,

I am having an issue getting accurate aruco pose estimation. The transform of the instantiated AR object relative to camera is a factor ~2 smaller than it should be relative to the camera position. For example, when the camera is z=.40m away from the marker in real-space, the AR object is only ~.20m away in virtual space.

I thought it might be something with the length parameters I measured off the chessboard or aruco markers?

I used these functions and corresponding measurements:

Aruco.estimatePoseSingleMarkers (corners, markerLength, camMatrix, distCoeffs, rvecs, tvecs);

where markerLength = 0.122m and is the distance across the 4x4 aruco (not including the black boarder)

Code (CSharp):

1. private void EstimatePoseCanonicalMarker (Mat rgbMat)
2.         {
3.             Aruco.estimatePoseSingleMarkers (corners, markerLength, camMatrix, distCoeffs, rvecs, tvecs);
4.  
5.             for (int i = 0; i < ids.total (); i++) {
6.                 using (Mat rvec = new Mat (rvecs, new OpenCVForUnity.CoreModule.Rect (0, i, 1, 1)))
7.                 using (Mat tvec = new Mat (tvecs, new OpenCVForUnity.CoreModule.Rect (0, i, 1, 1))) {
8.                     // In this example we are processing with RGB color image, so Axis-color correspondences are X: blue, Y: green, Z: red. (Usually X: red, Y: green, Z: blue)
9.                     Calib3d.drawFrameAxes (rgbMat, camMatrix, distCoeffs, rvec, tvec, markerLength * 0.5f);
10.  
11.                     // This example can display the ARObject on only first detected marker.
12.                     if (i == 0) {
13.                         UpdateARObjectTransform (rvec, tvec);
14.                     }
15.                 }
16.             }
17.         }

CalcChessboardCorners (patternSize, squareSize, objectPoint, markerType);

where squareSize = .0245m and is the distance(m) between 1 square on the chessboard

Code (CSharp):

1.         private void CalcChessboardCorners (Size patternSize, float squareSize, MatOfPoint3f corners, MarkerType markerType)
2.         {
3.             if ((int)(patternSize.width * patternSize.height) != corners.rows ()) {
4.                 Debug.Log ("Invalid corners size.");
5.                 corners.create ((int)(patternSize.width * patternSize.height), 1, CvType.CV_32FC3);
6.             }
7.  
8.             const int cn = 3;
9.             float[] cornersArr = new float[corners.rows () * cn];
10.             int width = (int)patternSize.width;
11.             int height = (int)patternSize.height;
12.  
13.             switch (markerType) {
14.             default:
15.             case MarkerType.ChessBoard:
16.             case MarkerType.CirclesGlid:
17.                 for (int i = 0; i < height; ++i) {
18.                     for (int j = 0; j < width; ++j) {
19.                         cornersArr [(i * width * cn) + (j * cn)] = j * squareSize;
20.                         cornersArr [(i * width * cn) + (j * cn) + 1] = i * squareSize;
21.                         cornersArr [(i * width * cn) + (j * cn) + 2] = 0;
22.                     }
23.                 }
24.                 corners.put (0, 0, cornersArr);
25.  
26.                 break;
27.             case MarkerType.AsymmetricCirclesGlid:
28.                 for (int i = 0; i < height; ++i) {
29.                     for (int j = 0; j < width; ++j) {
30.                         cornersArr [(i * width * cn) + (j * cn)] = (2 * j + i % 2) * squareSize;
31.                         cornersArr [(i * width * cn) + (j * cn) + 1] = i * squareSize;
32.                         cornersArr [(i * width * cn) + (j * cn) + 2] = 0;
33.                     }
34.                 }
35.                 corners.put (0, 0, cornersArr);
36.  
37.                 break;
38.             }
39.         }

I've tried doubling some of these values and recalibrating to see if it makes the z value close to .40m but it doesn't seem to change. Which makes me feel kind of clueless. I take about 50 calibration photos and they're pretty diverse. I try to capture a range of depth and angle on the camera.

Any ideas?

Thanks and let me know if you need more info.

 

Hi @kukewilly

I am having the exact same issue with Aruco markers, did you figure it out ?Any insights would be of great help.