Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Question Projection of skymap to objects in VR (Single Pass Stereo)

Discussion in 'Shaders' started by zulu79, Jul 14, 2020.

  1. zulu79

    zulu79

    Joined:
    Oct 8, 2019
    Posts:
    14
    Hello,

    for my open source RC helicopter game, where the pilot is standing at the constant position in the middle of a 360° skybox photo-image, and only the camera orientation changes, I used a custom shader to project the skybox photo-images onto the “ground object” (ground, trees, houses,…). Therefore a second camera is rendered into a “render texture” which is used in the shader: If the helicopter flies behind trees for examples, then it gets covered. The “ground object” catches also shadows.

    VR_test_scene_001.jpg

    Using the old non VR shader and deactivating XR leads to correct results:
    VR_test_scene_002.jpg

    I would like to change my existing shader to work in VR with the Single Pass Stereo rendering method. But due to lack of knowledge and experience I don’t get it work.

    I followed the explanation in the documentation “Single Pass Instanced rendering” --> Custom shaders (https://docs.unity3d.com/Manual/SinglePassInstancing.html)

    Here I’m not sure, if my “GPU instancing” settings are right, I don’t want to have different parameters for the mesh instances. Therefore I only added “#pragma multi_compile_instancing” and activated it in the material settings.

    Continuing in the doku I added the macros at the suggested positions:

    UNITY_VERTEX_INPUT_INSTANCE_ID, UNITY_VERTEX_OUTPUT_STEREO, UNITY_SETUP_INSTANCE_ID(), UNITY_INITIALIZE_OUTPUT(v2f, o) UNITY_INITIALIZE_VERTEX_OUTPUT_STEREO().

    But the projection isn't working (see image below) and also only the left eye shadow is at the correct position.

    VR_test_scene_003.jpg

    Could please give me somebody an advice, what I have to do to fix the shader?


    Shader for non VR mode (working fine) :

    Code (CSharp):
    1. // source:
    2. // https://www.ronja-tutorials.com/2019/01/20/screenspace-texture.html
    3. // https://forum.unity.com/threads/transparent-shader-that-recieves-shadow-kinda-works-already.398674/?_ga=2.69737753.803013357.1578766070-2133810121.1564613509#post-2606783
    4.  
    5. Shader "Custom/ground"
    6. {
    7.     //show values to edit in inspector
    8.     Properties{
    9.         _Color("Tint", Color) = (0, 0, 0, 1)
    10.         _MainTex("Texture", 2D) = "white" {}
    11.         _ShadowStrength("Shadow Strength", Range(0, 1)) = 1
    12.     }
    13.  
    14.     SubShader
    15.     {
    16.  
    17.         //the material is completely non-transparent and is rendered at the same time as the other opaque geometry
    18.         Tags{ "RenderType" = "Opaque" "Queue" = "Geometry" }
    19.  
    20.         Pass
    21.         {
    22.             CGPROGRAM
    23.  
    24.             //include useful shader functions
    25.             #include "UnityCG.cginc"
    26.  
    27.             //define vertex and fragment shader
    28.             #pragma vertex vert
    29.             #pragma fragment frag
    30.  
    31.             //texture and transforms of the texture
    32.             sampler2D _MainTex;
    33.             float4 _MainTex_ST;
    34.  
    35.             //tint of the texture
    36.             fixed4 _Color;
    37.  
    38.             //the object data that's put into the vertex shader
    39.             struct appdata
    40.             {
    41.                 float4 vertex : POSITION;
    42.             };
    43.  
    44.             //the data that's used to generate fragments and can be read by the fragment shader
    45.             struct v2f
    46.             {
    47.                 float4 position : SV_POSITION;
    48.                 float4 screenPosition : TEXCOORD0;
    49.             };
    50.  
    51.             //the vertex shader
    52.             v2f vert(appdata v)
    53.             {
    54.                 v2f o;
    55.                 //convert the vertex positions from object space to clip space so they can be rendered
    56.                 o.position = UnityObjectToClipPos(v.vertex);
    57.                 o.screenPosition = ComputeScreenPos(o.position);
    58.                 return o;
    59.             }
    60.  
    61.             //the fragment shader
    62.             fixed4 frag(v2f i) : SV_TARGET
    63.             {
    64.                 fixed4 col = tex2Dproj(_MainTex, i.screenPosition);
    65.                 col *= _Color;
    66.                 return col;
    67.             }
    68.  
    69.             ENDCG
    70.         }
    71.  
    72.  
    73.  
    74.         Pass
    75.         {
    76.             Blend SrcAlpha OneMinusSrcAlpha
    77.             Tags{ "LightMode" = "ForwardBase" }
    78.             ZWrite Off
    79.  
    80.             CGPROGRAM
    81.             #pragma vertex vert
    82.             #pragma fragment frag
    83.             #pragma multi_compile_fwdbase
    84.  
    85.             #include "UnityCG.cginc"
    86.             #include "AutoLight.cginc"
    87.  
    88.             struct v2f
    89.             {
    90.                 float4 pos : SV_POSITION;
    91.                 SHADOW_COORDS(0)
    92.             };
    93.  
    94.             fixed _ShadowStrength;
    95.             v2f vert(appdata_img v)
    96.             {
    97.                 v2f o;
    98.                 o.pos = UnityObjectToClipPos(v.vertex);
    99.                 TRANSFER_SHADOW(o);
    100.                 return o;
    101.             }
    102.  
    103.             fixed4 frag(v2f i) : COLOR
    104.             {
    105.                 fixed shadow = SHADOW_ATTENUATION(i);
    106.                 fixed shadowalpha = (1.0 - shadow) * _ShadowStrength;
    107.                 return fixed4(0.0, 0.0, 0.0, shadowalpha);
    108.             }
    109.  
    110.             ENDCG
    111.         }
    112.  
    113.  
    114.  
    115.         Pass
    116.         {
    117.             Tags{ "LightMode" = "ShadowCaster" }
    118.  
    119.             CGPROGRAM
    120.             #pragma vertex vert
    121.             #pragma fragment frag
    122.             #pragma multi_compile_shadowcaster
    123.  
    124.             #include "UnityCG.cginc"
    125.  
    126.             struct v2f
    127.             {
    128.                 V2F_SHADOW_CASTER;
    129.             };
    130.  
    131.             v2f vert(appdata_base v)
    132.             {
    133.                 v2f o;
    134.                 TRANSFER_SHADOW_CASTER_NORMALOFFSET(o)
    135.                     return o;
    136.             }
    137.  
    138.             float4 frag(v2f i) : SV_Target
    139.             {
    140.                 SHADOW_CASTER_FRAGMENT(i)
    141.             }
    142.             ENDCG
    143.         }
    144.  
    145.  
    146.     }
    147.     //FallBack "Standard"
    148. }
    149.  


    Shader edited for VR:

    Code (CSharp):
    1. // source:
    2. // https://www.ronja-tutorials.com/2019/01/20/screenspace-texture.html
    3. // https://forum.unity.com/threads/transparent-shader-that-recieves-shadow-kinda-works-already.398674/?_ga=2.69737753.803013357.1578766070-2133810121.1564613509#post-2606783
    4.  
    5. Shader "Custom/ground2"
    6. {
    7.     //show values to edit in inspector
    8.     Properties{
    9.         _Color("Tint", Color) = (0, 0, 0, 1)
    10.         _MainTex("Texture", 2D) = "white" {}
    11.         _ShadowStrength("Shadow Strength", Range(0, 1)) = 1
    12.     }
    13.  
    14.     SubShader
    15.     {
    16.  
    17.         //the material is completely non-transparent and is rendered at the same time as the other opaque geometry
    18.         Tags{ "RenderType" = "Opaque" "Queue" = "Geometry" }
    19.  
    20.         Pass
    21.         {
    22.             CGPROGRAM
    23.  
    24.             //include useful shader functions
    25.             #include "UnityCG.cginc"
    26.  
    27.             //define vertex and fragment shader
    28.             #pragma vertex vert
    29.             #pragma fragment frag
    30.             #pragma multi_compile_instancing  // ##### ADDED #####
    31.  
    32.             //texture and transforms of the texture
    33.             sampler2D _MainTex;
    34.             float4 _MainTex_ST;
    35.  
    36.             //tint of the texture
    37.             fixed4 _Color;
    38.  
    39.             //the object data that's put into the vertex shader
    40.             struct appdata
    41.             {
    42.                 float4 vertex : POSITION;
    43.                 UNITY_VERTEX_INPUT_INSTANCE_ID // ##### ADDED #####
    44.             };
    45.  
    46.             //the data that's used to generate fragments and can be read by the fragment shader
    47.             struct v2f
    48.             {
    49.                 float4 position : SV_POSITION;
    50.                 float4 screenPosition : TEXCOORD0;
    51.                 UNITY_VERTEX_OUTPUT_STEREO // ##### ADDED #####
    52.             };
    53.  
    54.             //the vertex shader
    55.             v2f vert(appdata v)
    56.             {
    57.                 v2f o;
    58.                 // UNITY_SETUP_INSTANCE_ID() calculates and sets the built - in unity_StereoEyeIndex and unity_InstanceID Unity shader variables
    59.                 // to the correct values based on which eye the GPU is currently rendering.
    60.                 // It must be used at the very beginning of a vertex Shader
    61.                 UNITY_SETUP_INSTANCE_ID(v); // ##### ADDED #####
    62.  
    63.                 // initializes all v2f values to 0
    64.                 UNITY_INITIALIZE_OUTPUT(v2f, o); // ##### ADDED #####
    65.  
    66.                 // UNITY_INITIALIZE_VERTEX_OUTPUT_STEREO tells the GPU which eye in the texture array it should render to,
    67.                 // based on the value of unity_StereoEyeIndex.This macro also transfers the value of unity_StereoEyeIndex from the vertex shader
    68.                 // so that it will be accessible in the fragment shader only if UNITY_SETUP_STEREO_EYE_INDEX_POST_VERTEX is called in the fragment shader frag method.
    69.                 UNITY_INITIALIZE_VERTEX_OUTPUT_STEREO(o); // ##### ADDED #####
    70.  
    71.                 // necessary only if you want to access instanced properties in the fragment Shader.
    72.                 //UNITY_TRANSFER_INSTANCE_ID(v, o);
    73.  
    74.                 //convert the vertex positions from object space to clip space so they can be rendered
    75.                 o.position = UnityObjectToClipPos(v.vertex);
    76.                 o.screenPosition = ComputeScreenPos(o.position); // ComputeNonStereoScreenPos(o.position);
    77.                 return o;
    78.             }
    79.  
    80.             //the fragment shader
    81.             fixed4 frag(v2f i) : SV_TARGET
    82.             {
    83.                 fixed4 col = tex2Dproj(_MainTex, i.screenPosition);
    84.                 col *= _Color;
    85.                 return col;
    86.             }
    87.  
    88.             ENDCG
    89.         }
    90.  
    91.  
    92.         Pass
    93.         {
    94.             Blend SrcAlpha OneMinusSrcAlpha
    95.             Tags{ "LightMode" = "ForwardBase" }
    96.             ZWrite Off
    97.  
    98.             CGPROGRAM
    99.             #pragma vertex vert
    100.             #pragma fragment frag
    101.             #pragma multi_compile_fwdbase
    102.  
    103.             #include "UnityCG.cginc"
    104.             #include "AutoLight.cginc"
    105.  
    106.             struct v2f
    107.             {
    108.              
    109.                 float4 pos : SV_POSITION;
    110.                 SHADOW_COORDS(0)
    111.                 UNITY_VERTEX_OUTPUT_STEREO // ##### ADDED #####
    112.             };
    113.  
    114.             fixed _ShadowStrength;
    115.             v2f vert(appdata_img v)
    116.             {
    117.                 v2f o;
    118.                 UNITY_SETUP_INSTANCE_ID(v); // ##### ADDED #####
    119.                 UNITY_INITIALIZE_OUTPUT(v2f, o); // ##### ADDED #####
    120.                 UNITY_INITIALIZE_VERTEX_OUTPUT_STEREO(o); // ##### ADDED #####
    121.  
    122.                 o.pos = UnityObjectToClipPos(v.vertex);
    123.                 TRANSFER_SHADOW(o);
    124.                 return o;
    125.             }
    126.  
    127.             fixed4 frag(v2f i) : COLOR
    128.             {
    129.                 fixed shadow = SHADOW_ATTENUATION(i);
    130.                 fixed shadowalpha = (1.0 - shadow) * _ShadowStrength;
    131.                 return fixed4(0.0, 0.0, 0.0, shadowalpha);
    132.             }
    133.  
    134.             ENDCG
    135.         }
    136.  
    137.  
    138.  
    139.         Pass
    140.         {
    141.             Tags{ "LightMode" = "ShadowCaster" }
    142.  
    143.             CGPROGRAM
    144.             #pragma vertex vert
    145.             #pragma fragment frag
    146.             #pragma multi_compile_shadowcaster
    147.  
    148.             #include "UnityCG.cginc"
    149.  
    150.             struct v2f
    151.             {
    152.                 V2F_SHADOW_CASTER;
    153.                 UNITY_VERTEX_OUTPUT_STEREO // ##### ADDED #####
    154.             };
    155.  
    156.             v2f vert(appdata_base v)
    157.             {
    158.                 v2f o;
    159.                 UNITY_SETUP_INSTANCE_ID(v); // ##### ADDED #####
    160.                 UNITY_INITIALIZE_OUTPUT(v2f, o); // ##### ADDED #####
    161.                 UNITY_INITIALIZE_VERTEX_OUTPUT_STEREO(o); // ##### ADDED #####
    162.                 TRANSFER_SHADOW_CASTER_NORMALOFFSET(o)
    163.                 return o;
    164.             }
    165.  
    166.             float4 frag(v2f i) : SV_Target
    167.             {
    168.                 SHADOW_CASTER_FRAGMENT(i)
    169.             }
    170.             ENDCG
    171.         }
    172.  
    173.     }
    174.     //FallBack "Standard"
    175. }
    176.  

    Setup:

    Win10 / Unity 2020.2.a13 / OpenVR XR Plugin Version 1.0.0-preview.2 / HTC Vive Cosmos, Per Eye 1440 x 1700

    Second Camera (as child of Main Camera under XRig):
    Culling Mask --> nothing / Field of View --> 93.59882 (same as Main Camera) / Depth --> 0 / Target Eye -->None (Main Display)

    Project Settings --> XR Plug-in Management --> OpenVR:
    App Type: Scene / Stereo Rend. Mode: Single Pass Instanced / Mirror View Mode: None
     
  2. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    12,329
    There are a number of complications you're going to run in to here.

    In non-VR, just using the screen position is useful. In your non-VR test case you have a big quad in space and you have the screen aligned to it. You can get those to line up because the screen position is centered and symmetric.

    For VR it's almost never useful since for most VR headsets the eye projections are asymmetric, usually showing slightly more to the sides than the middle, meaning a "screen position" projection won't look like it's infinitely far away because the center points won't match up with "straight ahead" in both eyes. Plus screen space will never match a quad that's anywhere in the world space, even one attached to your face, because by the nature of stereoscopic rendering anything that's not at infinity is going to be at a different position in each eye.

    Skybox rendering for VR works by rendering the sky geometry centered on each individual eye's camera position, meaning the relative angles are identical for both eyes. It's not using screen space.

    For the main directional light's shadows, Unity uses a screen space texture for the shadows. However it's a unique texture per eye, rendered explicitly from that eye's point of view, using that eye's projection. It's not trying to render the same texture for both eyes. The issue you're seeing above is for some reason it's reusing the same screen space texture for both eyes. There's also no need to do the screen space texture and shadow in separate passes. It'll greatly simplify things if you do both in one pass. Though I don't entirely understand why it's broken for you.
     
  3. zulu79

    zulu79

    Joined:
    Oct 8, 2019
    Posts:
    14
    Bgolus, thank you for your explanations.

    I’m having a hard time to understand everything, maybe even if it might be basic stuff.

    Starting from your posting I would like to divide the problem in three topics:

    1.) Asymmetric projection
    2.) Screen space will never match a quad
    3.) Directional shadows

    , whereby I only focused so far on the asymmetric projection.

    First, I would like to understand the theory, because I still don't see, how to solve the problem. What I think about is to apply somehow maybe a projection matrix and eye-offset/position/orientation matrix to the target texture?

    Using additional sources (
    http://paulbourke.net/stereographics/stereorender/ ,
    https://docs.unity3d.com/ScriptReference/Camera-projectionMatrix.html?_ga=2.11977201.412277658.1594574603-2133810121.1564613509,

    https://answers.unity.com/questions/1359718/what-do-the-values-in-the-matrix4x4-for-cameraproj.html
    ) I created a drawing, with following thoughts:

    - There is an offset from the screen center to the projection center due to the asym. proj.
    - An infinite far centered object on the skybox (e.g. sun) should appear for both cameras in the projection center.
    - Four parallel rays starting at the screen corner, point in the camera view towards the projection center

    VR_test_scene_001.png

    The theory does not seem to fit to the next test image. Here the screen center seems to align with the projection center. But it is hard to tell, where the screen center is.

    VR_test_scene_002.png

    Therefore I tested the game-view in the unity editor. Here it seems that the projection center aligns for the right camera only, but not for the left. VR_test_scene_004.png

    Next image tries to interpret the last image.

    VR_test_scene_005.png

    Now I'm really confused.
     

    Attached Files:

  4. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    12,329
    That's not the center line. Screen's center line is the middle of the texture in your screen space shader.
    upload_2020-7-16_10-30-51.png

    The "center" X geometry you have is presumably centered on the in scene camera. That should be inset towards the middle because your eyes are separated and that geometry isn't infinitely far away. There's also slightly more distance from the per eye forward to outside screen edge than the middle screen edge, so if you were to use a cubemap skybox that uses the grid texture for all 6 faces, the center line for that also won't line up with the screen space center ... because screen space center isn't useful for VR.

    edit: Wait, the big quad in the background is a cube map skybox, isn't it, and the X are boxes that are just stretching out as far as they can to the far plane. So yeah, that's showing exactly what I'm talking about then. It's inset towards the center of the view because that's the actual centered forward for that eye. And that does not match the screen space center, because the HMD is using asymmetric projections.

    You've set the OpenVR settings to right eye only, but the game view is set to only show the left view... I honestly have no idea what that'll do. I would expect it to render the non-stereo view at that point, but without seeing exactly how your scene is setup and aligned, I couldn't say for sure what's happening there. Generally, ignore the in editor game view for VR. It's not the view being rendered to the eye, that's just a debug view being rendered for the editor.
     
    Last edited: Jul 16, 2020
  5. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    12,329
    So. Solution time.

    If your skybox is a cubemap already. The solution is to use the same view direction math to sample the skybox cubemap in the shader instead of trying to use screenspace.

    That's really easy.
    Code (csharp):
    1. struct v2f {
    2.     float4 pos : SV_Position;
    3.     float3 camRelativeWorldPos : TEXCOORD0;
    4. };
    5.  
    6. v2f vert(appdata_full v)
    7. {
    8.     v2f o;
    9.  
    10.     o.pos = UnityObjectToClipPos(v.vertex);
    11.  
    12.     float3 worldPos = mul(unity_ObjectToWorld, float4(v.vertex.xyz, 1.0)).xyz;
    13.     o.camRelativeWorldPos = worldPos - _WorldSpaceCameraPos;
    14.  
    15.     return o;
    16. }
    17.  
    18. samplerCUBE _SkyCube;
    19.  
    20. half4 frag (v2f i) : SV_Target
    21. {
    22.     half4 col = texCUBE(_SkyCube, i.camRelativeWorldPos);
    23.     return col;
    24. }
     
  6. zulu79

    zulu79

    Joined:
    Oct 8, 2019
    Posts:
    14
    Thank you again for the explanation.

    Yes the "quad" is an image of a sykbox (but no from a cube map, but from the 6 sided version). The "X" are, as you said, parallel lines showing forward.

    The OpenVR "right eye" / "left eye" setting is the only way to change the editor game view for VR. The game view settings show no effect.

    The projection works now great! Thank you!!!

    I removed the separate shadow passes (see code below), but what remains is the offset of the shadows in the right camera. (If I change the Stereo Rendering Mode from "Single Pass Instanced" to "Multi Pass", than everything is ok.)

    IUnity_Forum_001.jpg

    Code (CSharp):
    1.  
    2. Shader "Custom/ground2"
    3. {
    4.     //show values to edit in inspector
    5.     Properties{
    6.         _Color("Tint", Color) = (0, 0, 0, 1)
    7.         _SkyCube("Cubemap", CUBE) = "" {}
    8.         _ShadowStrength("Shadow Strength", Range(0, 1)) = 1
    9.     }
    10.      
    11.     SubShader
    12.     {
    13.         Tags{ "LightMode" = "ForwardBase" }
    14.  
    15.         Pass
    16.         {
    17.             CGPROGRAM
    18.  
    19.             //include useful shader functions
    20.             #include "UnityCG.cginc"
    21.             #include "AutoLight.cginc"
    22.  
    23.             //define vertex and fragment shader
    24.             #pragma vertex vert
    25.             #pragma fragment frag
    26.             #pragma multi_compile_fwdbase
    27.             //#pragma multi_compile_instancing
    28.  
    29.             //texture and transforms of the texture
    30.             //sampler2D _MainTex;
    31.             //UNITY_DECLARE_SCREENSPACE_TEXTURE(_MainTex);
    32.             //float4 _MainTex_ST;
    33.             samplerCUBE _SkyCube;
    34.  
    35.             //tint of the texture
    36.             fixed4 _Color;
    37.             fixed _ShadowStrength;
    38.  
    39.             struct appdata
    40.             {
    41.                 float4 vertex : POSITION;
    42.                 UNITY_VERTEX_INPUT_INSTANCE_ID
    43.             };
    44.  
    45.             struct v2f
    46.             {
    47.                 float4 pos : SV_POSITION;
    48.                 float3 camRelativeWorldPos : TEXCOORD0;
    49.                 SHADOW_COORDS(1) // put shadows data into TEXCOORD1
    50.                 UNITY_VERTEX_OUTPUT_STEREO
    51.             };
    52.  
    53.             v2f vert(appdata_full v)
    54.             {
    55.                 v2f o;
    56.                 UNITY_SETUP_INSTANCE_ID(v);
    57.                 UNITY_INITIALIZE_OUTPUT(v2f, o);
    58.                 UNITY_INITIALIZE_VERTEX_OUTPUT_STEREO(o);
    59.  
    60.                 o.pos = UnityObjectToClipPos(v.vertex);
    61.  
    62.                 float3 worldPos = mul(unity_ObjectToWorld, float4(v.vertex.xyz, 1.0)).xyz;
    63.                 o.camRelativeWorldPos = worldPos - _WorldSpaceCameraPos;
    64.            
    65.                 TRANSFER_SHADOW(o);
    66.  
    67.                 return o;
    68.             }
    69.  
    70.             half4 frag(v2f i) : SV_Target
    71.             {
    72.                 half4 col = texCUBE(_SkyCube, i.camRelativeWorldPos);
    73.  
    74.                 fixed attenuation = SHADOW_ATTENUATION(i);
    75.                 col.rgb *= (1.0f - (1.0 - attenuation) * _ShadowStrength);
    76.                 col *= _Color;
    77.                 return col;
    78.             }
    79.  
    80.             ENDCG
    81.         }
    82.  
    83.         // shadow casting support
    84.         UsePass "Legacy Shaders/VertexLit/SHADOWCASTER"
    85.  
    86.     }
    87.     //FallBack "Standard"
    88. }
    89.  

    A "cosmetic" question arises from using cubemaps in this sheader. In my game I use a (slightliy modified) 6 sided skybox shader for the Skybox material, because all available game sceneries are based on 6 separate images. I can use the Legacy Cubemap Asset (https://docs.unity3d.com/560/Documentation/Manual/class-Cubemap.html) to generate a cubemap, which is unfortunately limited to a "face size" of 2048. I think, I can also generate the cubemap-texture myself with c#, but is it possible to use the 6 separate images in the shader, without the cubemap?
     
  7. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    12,329
    Possible? Yes.

    But I'd highly recommend you use a cube map instead. It'll save you a huge amount of hassle.

    A trick you could use is to create a cube map of the resolution you want, and then use CopyTexture to copy the individual textures into the faces of the cube. Should be quite cheap to do, and then you can use the cube map for both the sky and the geometry.

    Otherwise, you'll have to implement something like a cube map lookup function that'll get you the face and local UV from the direction vector.
     
  8. zulu79

    zulu79

    Joined:
    Oct 8, 2019
    Posts:
    14
    I have used with success Cubemap's SetPixels() to set the six textures.

    Next problem is, that the alpha channel appears in the shader black and not transparent, but I think, that's a limitation of the Texture2D.LoadImage function. (In the PNG's every pixel has also a color, and the alpha is used in other similar games to set the ground object partially transparent again. This way trees with a lot of details as leafs for example cover the helicopter only where leafs have alpha == 1)}


    Do you have an idea how to solve in Stereo Rendering Mode the shadow offset problem?
     
    Last edited: Jul 17, 2020
  9. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    12,329
    Not entirely sure. Possibly missing the
    UNITY_SETUP_INSTANCE_ID(i);
    at the start of the fragment shader before calling the SHADOW_ATTENTUATION macro. The screen space shadow texture gets handled differently on different VR platforms / rendering modes. Most don't need anything more than what you have above, but some do need the instance ID in the fragment shader to be setup, so that might be it.
     
  10. zulu79

    zulu79

    Joined:
    Oct 8, 2019
    Posts:
    14
    Adding UNITY_SETUP_INSTANCE_ID(i); at the beginning of the fragment shader leads to following error messages:

    1.)
    Shader error in 'Custom/ground': 'UnitySetupInstanceID': no matching 1 parameter function at line 89 (on d3d11)

    Compiling Vertex program with DIRECTIONAL SHADOWS_SCREEN LIGHTPROBE_SH INSTANCING_ON
    Platform defines: UNITY_ENABLE_REFLECTION_BUFFERS UNITY_USE_DITHER_MASK_FOR_ALPHABLENDED_SHADOWS UNITY_PBS_USE_BRDF1 UNITY_SPECCUBE_BOX_PROJECTION UNITY_SPECCUBE_BLENDING UNITY_ENABLE_DETAIL_NORMALMAP SHADER_API_DESKTOP UNITY_COLORSPACE_GAMMA UNITY_LIGHT_PROBE_PROXY_VOLUME UNITY_LIGHTMAP_FULL_HDR UNITY_PASS_FORWARDBASE
    Disabled keywords: SHADOWS_SHADOWMASK DYNAMICLIGHTMAP_ON LIGHTMAP_ON LIGHTMAP_SHADOW_MIXING DIRLIGHTMAP_COMBINED VERTEXLIGHT_ON UNITY_NO_DXT5nm UNITY_ENABLE_NATIVE_SHADOW_LOOKUPS UNITY_METAL_SHADOWS_USE_POINT_FILTERING UNITY_NO_SCREENSPACE_SHADOWS UNITY_PBS_USE_BRDF2 UNITY_PBS_USE_BRDF3 UNITY_NO_FULL_STANDARD_SHADER UNITY_HARDWARE_TIER1 UNITY_HARDWARE_TIER2 UNITY_HARDWARE_TIER3 UNITY_HALF_PRECISION_FRAGMENT_SHADER_REGISTERS UNITY_LIGHTMAP_DLDR_ENCODING UNITY_LIGHTMAP_RGBM_ENCODING UNITY_VIRTUAL_TEXTURING UNITY_PRETRANSFORM_TO_DISPLAY_ORIENTATION UNITY_ASTC_NORMALMAP_ENCODING


    2.)
    Shader error in 'Custom/ground': invalid subscript 'instanceID' at line 89 (on d3d11)

    Compiling Vertex program with DIRECTIONAL SHADOWS_SCREEN LIGHTPROBE_SH INSTANCING_ON
    Platform defines: UNITY_ENABLE_REFLECTION_BUFFERS UNITY_USE_DITHER_MASK_FOR_ALPHABLENDED_SHADOWS UNITY_PBS_USE_BRDF1 UNITY_SPECCUBE_BOX_PROJECTION UNITY_SPECCUBE_BLENDING UNITY_ENABLE_DETAIL_NORMALMAP SHADER_API_DESKTOP UNITY_COLORSPACE_GAMMA UNITY_LIGHT_PROBE_PROXY_VOLUME UNITY_LIGHTMAP_FULL_HDR UNITY_PASS_FORWARDBASE
    Disabled keywords: SHADOWS_SHADOWMASK DYNAMICLIGHTMAP_ON LIGHTMAP_ON LIGHTMAP_SHADOW_MIXING DIRLIGHTMAP_COMBINED VERTEXLIGHT_ON UNITY_NO_DXT5nm UNITY_ENABLE_NATIVE_SHADOW_LOOKUPS UNITY_METAL_SHADOWS_USE_POINT_FILTERING UNITY_NO_SCREENSPACE_SHADOWS UNITY_PBS_USE_BRDF2 UNITY_PBS_USE_BRDF3 UNITY_NO_FULL_STANDARD_SHADER UNITY_HARDWARE_TIER1 UNITY_HARDWARE_TIER2 UNITY_HARDWARE_TIER3 UNITY_HALF_PRECISION_FRAGMENT_SHADER_REGISTERS UNITY_LIGHTMAP_DLDR_ENCODING UNITY_LIGHTMAP_RGBM_ENCODING UNITY_VIRTUAL_TEXTURING UNITY_PRETRANSFORM_TO_DISPLAY_ORIENTATION UNITY_ASTC_NORMALMAP_ENCODING


    I have added the UNITY_VERTEX_INPUT_INSTANCE_ID macro in the v2f struct,
    whereby both error messages disappeared, but the shadow in the right camera are still on the wrong position.

    Code (CSharp):
    1. // source:
    2. // https://www.ronja-tutorials.com/2019/01/20/screenspace-texture.html
    3. // https://forum.unity.com/threads/transparent-shader-that-recieves-shadow-kinda-works-already.398674/?_ga=2.69737753.803013357.1578766070-2133810121.1564613509#post-2606783
    4. // https://forum.unity.com/threads/projection-of-skymap-to-objects-in-vr-single-pass-stereo.931332/add-reply
    5.  
    6. Shader "Custom/ground"
    7. {
    8.     //show values to edit in inspector
    9.     Properties{
    10.         _TintColor("Tint Color", Color) = (.5, .5, .5, 0.5)
    11.         [Gamma] _Exposure("Exposure", Range(0.0, 2)) = 1.0
    12.         _SkyCube("Cubemap", CUBE) = "" {}
    13.         _ShadowStrength("Shadow Strength", Range(0, 1)) = 1
    14.     }
    15.  
    16.     SubShader
    17.     {
    18.         Tags{ "LightMode" = "ForwardBase" }
    19.         //Tags{ "Queue" = "Background" } //  instructs Unity to render this pass before other objects are rendered.
    20.  
    21.         Pass
    22.         {
    23.             CGPROGRAM
    24.  
    25.             //include useful shader functions
    26.             #include "UnityCG.cginc"
    27.             #include "AutoLight.cginc"
    28.  
    29.             //define vertex and fragment shader
    30.             #pragma vertex vert
    31.             #pragma fragment frag
    32.             #pragma multi_compile_fwdbase
    33.             #pragma multi_compile_instancing
    34.  
    35.             //texture and transforms of the texture
    36.             samplerCUBE _SkyCube;
    37.  
    38.             //tint of the texture
    39.             half4 _TintColor;
    40.             half _Exposure;
    41.             fixed _ShadowStrength;
    42.  
    43.  
    44.             //the object data that's put into the vertex shader
    45.             struct appdata
    46.             {
    47.                 float4 vertex : POSITION;
    48.                 UNITY_VERTEX_INPUT_INSTANCE_ID
    49.             };
    50.  
    51.             //the data that's used to generate fragments and can be read by the fragment shader
    52.             struct v2f
    53.             {
    54.                 float4 pos : SV_POSITION;  // This is the (x, y) position of the pixel in normalized coordinates in the range (-1, -1) to (1, 1). The z is the depth position (used for the depth buffer) in the normalized range 0 to 1.
    55.                 float3 camRelativeWorldPos : TEXCOORD0;
    56.  
    57.                 UNITY_VERTEX_INPUT_INSTANCE_ID // necessary only if you want to access instanced properties in fragment Shader.
    58.                 UNITY_VERTEX_OUTPUT_STEREO
    59.                 SHADOW_COORDS(1) // put shadows data into TEXCOORD1
    60.             };
    61.  
    62.             v2f vert(appdata_full v)
    63.             {
    64.                 v2f o;
    65.  
    66.                 //UNITY_TRANSFER_INSTANCE_ID(v, o); // necessary only if you want to access instanced properties in the fragment Shader.
    67.                 UNITY_SETUP_INSTANCE_ID(v);
    68.                 UNITY_INITIALIZE_OUTPUT(v2f, o);
    69.                 UNITY_INITIALIZE_VERTEX_OUTPUT_STEREO(o);
    70.  
    71.                 o.pos = UnityObjectToClipPos(v.vertex);
    72.  
    73.                 float3 worldPos = mul(unity_ObjectToWorld, float4(v.vertex.xyz, 1.0)).xyz;
    74.                 o.camRelativeWorldPos = worldPos - _WorldSpaceCameraPos;
    75.  
    76.                 TRANSFER_SHADOW(o); // o._ShadowCoord = mul(unity_World2Shadow[0], mul(_Object2World, v.vertex));
    77.  
    78.                 return o;
    79.             }
    80.  
    81.             half4 frag(v2f i) : SV_Target
    82.             {      
    83.                 half4 col = texCUBE(_SkyCube, i.camRelativeWorldPos);
    84.                 col *= _TintColor * _Exposure *2.0;
    85.  
    86.                 // compute shadow attenuation (1.0 = fully lit, 0.0 = fully shadowed)
    87.                 UNITY_SETUP_INSTANCE_ID(i); // necessary only if any instanced properties are going to be accessed in the fragment Shader.
    88.                 fixed attenuation = SHADOW_ATTENUATION(i);
    89.                 col.rgb *= (1.0f - (1.0 - attenuation) * _ShadowStrength);  
    90.                 return col;
    91.             }
    92.  
    93.             ENDCG
    94.         }
    95.  
    96.         // shadow casting support
    97.         UsePass "Legacy Shaders/VertexLit/SHADOWCASTER"
    98.  
    99.     }
    100.     //FallBack "Standard"
    101. }
    102.  
     
    dpcactus likes this.
  11. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    12,329
    Not really sure what else to do. Using your shader exactly as it is above works correctly for me in VR. Adding the
    UNITY_SETUP_INSTANCE_ID(i)
    didn't change anything since it already worked, but also didn't cause an error. And I did check that they were rendered using single pass instancing, though not using OpenVR.
     
    zulu79 likes this.
  12. zulu79

    zulu79

    Joined:
    Oct 8, 2019
    Posts:
    14
    Thank you for taking time and testing the shader. I have only a HTC VIVE Cosmos headset but I got the answer from a user, that his Oculus Rift does not have any shadow problems in my demo version (https://ufile.io/761sbz59). So it seams to be exclusively an OpenVR problem.
     
  13. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    12,329
    Yeah, I don’t have a Cosmos to test with. I do have a Vive, but I only tested it using the Rift.

    One option you could try would be to use a surface shader with a custom lighting model, then if that doesn’t have any issues, look at the generated code and try to simplify it down to see what’s needed.
     
  14. zulu79

    zulu79

    Joined:
    Oct 8, 2019
    Posts:
    14
    I did as you suggested and found that
    - in the vertex shader UNITY_TRANSFER_INSTANCE_ID(v, o); has to come after UNITY_INITIALIZE_OUTPUT(v2f, o);
    Now the shadows are correct!

    I also changed your o.camRelativeWorldPos calculation, because this creates in the game a very disturbing effect: The ground - where the cubmap is projected onto - seams to be several meters below the shadow.

    1.) The first change is, that "float3 worldPos = mul(unity_ObjectToWorld, float4(v.vertex.xyz, 1.0)).xyz;" is not necessary, because the ground is stationary and has no position or orientation offset.
    2.) The second change is, as mentioned above, that I do not use the direction vector o.camRelativeWorldPos, but the under 1.) simplified worldPos. This is a projection of the cubmap onto the ground object from single point for both eyes. Now the ground fits perfectly into the environment. One drawback has to be mentioned: If the camera moves, the texture appears doubled behind the ground (for example behind a tree). Therefore I have to rework the 3D sceneries and create several layers with several slightly modified cubemaps.

    figure.png

    Unfortunately the single instanced version does not reduce the GPU-load. Following figure shows two compiled runs of the game (OpenVR / VIVE Cosmos, GTX1080Ti). On the left side the GPU-load of the game with "Multi Pass", on the right side the "Single Pass Instanced" version.

    GPU-Z_log.png

    The initial problem is solved now, thank you very much bgolus!
     
    Last edited: Jul 30, 2020
  15. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    12,329
    You may be interested in this:
    https://developers.google.com/vr/discover/seurat