Search Unity

Question How are motion vectors (AOV) actually computed?

Discussion in 'General Graphics' started by wind1050, May 30, 2023.

  1. wind1050

    wind1050

    Joined:
    Apr 6, 2023
    Posts:
    1
    In my application I am required to record the motion vectors of a scene for ground truth optical flow in HDRP. I was wondering how motion vectors are actually computed inside unity if they are an approximation calculated by the 2d Images or actually take account the depth of the scene and other available information.

    I was not able to find concret implementation details thats why I'm asking here. Thank you for your help!
     
  2. c0d3_m0nk3y

    c0d3_m0nk3y

    Joined:
    Oct 21, 2021
    Posts:
    666
    It's not comparing 2d images. AFAIK, it works like this:

    For objects with MotionVectorGenerationMode.Camera, it renders a fullscreen pass, reconstructs the NDC position from the depth buffer, transforms it to the last frame's NDC position with help of the current frame's inverse view-projection matrix and last frame's view-projection matrix. The motion vector is the difference of the positions in NDC space.

    For objects with MotionVectorGenerationMode.Object, it renders the mesh in a separate pass and transforms both the current and last object-space position to the current and last clip space with the current and last model-view-projection matrices. The last object space position is passed as a vertex attribute (there can be skinned objects).

    You can take a look at the implementation
    C:\Program Files\Unity\Hub\Editor\2022.1.15f1\Editor\Data\Resources\PackageManager\BuiltInPackages\com.unity.render-pipelines.high-definition\Runtime\RenderPipeline\RenderPass\MotionVectors\CameraMotionVectors.shader
    C:\Program Files\Unity\Hub\Editor\2022.1.15f1\Editor\Data\Resources\PackageManager\BuiltInPackages\com.unity.render-pipelines.universal\Shaders\ObjectMotionVectors.shader

    Not sure why ObjectMotionVectors.shader is only in URP and not HDRP, though.

    There is also C:\Users\Main User\Downloads\builtin_shaders-2022.1.10f1\DefaultResourcesExtra\Internal-MotionVectors.shader for BiRP.
     
    Last edited: May 30, 2023