Search Unity

Camera not updated with latest hmd pose if rendering to a RenderTexture

Discussion in 'AR/VR (XR) Discussion' started by tspk91, Oct 22, 2019.

  1. tspk91

    tspk91

    Joined:
    Nov 19, 2014
    Posts:
    131
    In our game, we have a stacked camera setup (for near/far mono rendering) where we have a camera that renders to a rendertexture and with a custom projection matrix. The problem is that it does not follow the VR headset correctly (its view matrix is slightly off), unlike the camera without a rendertexture. This makes the overlayed image jittery and needs correction. Also, any camera that renders to texture is forced to render before normal cameras regardless of depth value. Is this to be expected?

    Another "quirk" is that when you call camera.Render() inside OnRenderImage, OnPostRender, etc, that camera ignores its stereoTargetEye and inherits the settings from the event camera.

    While we have been able to work around this limitation, it would be much better if it worked right out of the box.
    This is tested with 2018.3. All info welcome.
     
  2. Matt_D_work

    Matt_D_work

    Unity Technologies

    Joined:
    Nov 30, 2016
    Posts:
    202
    Are you relying on the implicit camera tracking for movement? As its rendering to a texture, the implicit camera tracking wont be applied to it. Use the Tracked Pose Driver found in the `com.unity.xr.legacyinputhelpers` package to drive the camera from the HMD instead.
     
  3. tspk91

    tspk91

    Joined:
    Nov 19, 2014
    Posts:
    131
    With Oculus Rift using 2019.1.14f1, the headtracking is jittery even when using Tracked Pose Driver.

    The current setup is this:
    -Camera renders to rendertexture, with a trackedposedriver for center eye set to update and before render.
    -Second camera renders to backbuffer, with implicit tracking.
    -Composite rendertexture from the first camera into the second with a blit.
    -Result: the content from the second camera tracks perfectly, content that was composited from the first camera is jittery.
     
    Last edited: Nov 7, 2019