Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. We have updated the language to the Editor Terms based on feedback from our employees and community. Learn more.
    Dismiss Notice

Question Setting the physical camera parameters to simulate AR glasses

Discussion in 'AR' started by serhanguel, Sep 10, 2021.

  1. serhanguel


    Jan 7, 2019
    Hi, I would like to simulate something similar to the Timewarping in Oculus or late stage reprojection in HoloLens. For this, I captured camera poses from the AR glasses and loaded them in Unity. Then, I move the camera from one pose to another and capture the resulting images. My goal is to warp the image resulting from the first pose to obtain the second image (assuming that both camera poses are known, like in Timewarping or LSR).

    In order to apply such an image warp, I need to compute the homography for which I need the camera intrinsics (calibration) matrix. As a first step, I set the physical camera parameters as in the attached image and computed the focal lengths in pixels using the formula: fx[pixels] = fx[mm] * img_width[pixels] / sensor_width[mm] where I set fx[mm]=25 and sensor_width[mm]=36 in Unity (similarly for fy). I set the image resolution to 1920x1080, so I capture images of this size for my simulation. These are just representative values, I have currently no idea how to apply in this in a real system but for the simulation I just want to capture the images from Unity and apply a perspective transformation using OpenCV.

    However, I discovered that depending on how the Game view is scaled, the captured images look different. It seems that this depends on the Gate Fit parameter of the physical camera. My questions are:
    1. What is the right way to think about gate fit? Depending on how this is set and how the game view is scaled, I get different images, so I can't be sure if my transform is wrong or the capture.
    2. Is it correct to set a calibration matrix using the parameters from the Unity physical camera? For optical center (the principal point, in pixels) in the matrix, I use cx=img_width/2 and cy=img_height/2.


    Attached Files: