Search Unity

Question How to render single pass instanced with a custom stereo camera to a render texture

Discussion in 'EditorXR' started by cecarlsen, Jan 26, 2024.

  1. cecarlsen

    cecarlsen

    Joined:
    Jun 30, 2006
    Posts:
    862
    I want to render Single-Pass Instanced using a camera with custom stereo view-projection matrices to a RenderTexture so I can output to a passive stereoscopic 3D-system that is recognized by the Windows machine as two separate displays.

    What I normally do, is just use two mono (regular) cameras and set their output to display 1 and 2. That works. But performance is critical in this project, so I need Single-Pass Instanced rendering. I would then render left and right image to a single RenderTexture, and then display using two Canvases (Unity UI) that are set to display 1 and 2, each with a RawImage with appropriate uv offset.

    I tried using the MockHMD XR Plugin with the XR Plugin Management system. But when I change the projection and view matrix of the stereo camera, it throws up a bunch of red errors about culling and failed assertions. MockHMD is made to mimic the HTC Vive HMD, and it does not seem to allow custom matrices.

    I thought about writing my own XR plugin, so I copied the com.unity.xr.mock-hmd package and hacked away. But didn't get far ... I couldn't find the source code of the UnityMockHMD.dll that it relies on.

    Is there a public example of how to implement a XR plugin? Or how does one go about this?

    EDIT1: I'm not alone.
    https://forum.unity.com/threads/cre...o-output-dual-display-instead-of-sbs.1465205/
    https://forum.unity.com/threads/xr-plugins-and-subsystems.693463/page-8#post-6846146

    EDIT2: I can see that the Unity plugin of stereo display maker Leia (former Dimenco) resorts to using two cameras, so they also did not figure this out. It's not the system I'm currently targeting, but anyways relevant.

    EDIT3: Even setting the view matrices to what they already are results in a whooole bunch of red errors when using Mock HMD. Also, setting physical camera properties have no effect, I guess because Mock HMD overrides it somewhere from within the dll.
    Code (CSharp):
    1. _camera.SetStereoViewMatrix( Camera.StereoscopicEye.Left, _camera.GetStereoViewMatrix( Camera.StereoscopicEye.Left ) );
    2. _camera.SetStereoViewMatrix( Camera.StereoscopicEye.Right, _camera.GetStereoViewMatrix( Camera.StereoscopicEye.Right ) );
    Errors be like: "Can't calculate the eye texture aspect ratio. Culling/Shadows are likely to break", "Screen position out of view frustum (screen pos 0.000000, 0.000000, 50.000000) (Camera rect 0 0 0 0)", and "Assertion failed on expression: 'std::abs(det) > FLT_MIN'" from UnityEngine.GUIUtilityProcessEvent.

    EDIT4: There is a plugin for Acers SpatialLabs stereoscopic displays and it relies on OpenXR. Unfortunately the secret sauce is in a dll.
     
    Last edited: Jan 26, 2024