Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. We have updated the language to the Editor Terms based on feedback from our employees and community. Learn more.
    Dismiss Notice
New Forum User Notice Update to the Unity Editor Software Terms

360 video as skybox with HDRP

Discussion in 'High Definition Render Pipeline' started by bjerken, Jun 1, 2020.

  1. bjerken

    bjerken

    Joined:
    Dec 24, 2015
    Posts:
    11
    Hi!
    Has anyone been able to successfully the Video player rendering a 360 video to a skybox with HDRP in Unity 2019? It is very easily done with the old Standard pipeline and a render texture set to a panoramic skybox. But it seems a bit more tricky to accomplish with the HDRP. The old workflow can be found here:


    Thanks in advance
     
    bjerken, Jun 1, 2020
    #1
    GeorgeAdamon likes this.

  2. Gokcan

    Gokcan

    Joined:
    Aug 15, 2013
    Posts:
    289
    I wonder why u need hdrp just for 360 video content?
     
    Gokcan, Jun 1, 2020
    #2

  3. bjerken

    bjerken

    Joined:
    Dec 24, 2015
    Posts:
    11
    @Gokcan I am only using the 360 video as a backdrop. I need HDRP for the rest of the content sitting inside the scene itself.
     
    bjerken, Jun 2, 2020
    #3

  4. GeorgeAdamon

    GeorgeAdamon

    Joined:
    May 31, 2017
    Posts:
    48
    +1
    Can someone at Unity please provide an answer. The old workflows no longer seem to work, as HDRP doesn't use Skybox materials, and Volume settings don't seem to accept RenderTextures.
     
    GeorgeAdamon, May 31, 2021
    #4

  5. antoinel_unity

    antoinel_unity

    Unity Technologies

    Joined:
    Jan 7, 2019
    Posts:
    237
    Hello,

    We added RenderTexture support in the HDRISky component in Unity 2021.1 (HDRP 11) so it should be possible to have a video rendering as the sky texture with this version.

    The HDRISky component requires a cubemap, but unfortunately, the video player doesn't work really well when you render the video directly into a cubemap render texture, so instead you'll need to render it to a temporary 2D RenderTexture like this: upload_2021-6-1_16-29-25.png

    Then to convert this render texture into a cubemap, the simplest is to use a CustomRenderTexture with this shader:

    Code (CSharp):
    1. Shader "CustomRenderTexture/RadialCoordinateConversion"
    2. {
    3.     Properties
    4.     {
    5.         _VideoTexture("VideoTexture", 2D) = "white" {}
    6.      }
    7.  
    8.      SubShader
    9.      {
    10.         Lighting Off
    11.         Blend One Zero
    12.  
    13.         Pass
    14.         {
    15.             CGPROGRAM
    16.             #include "UnityCustomRenderTexture.cginc"
    17.             #pragma vertex CustomRenderTextureVertexShader
    18.             #pragma fragment frag
    19.             #pragma target 3.0
    20.  
    21.             sampler2D   _VideoTexture;
    22.  
    23.             inline float2 ToRadialCoords(float3 coords)
    24.             {
    25.                 float3 normalizedCoords = normalize(coords);
    26.                 float latitude = acos(normalizedCoords.y);
    27.                 float longitude = atan2(normalizedCoords.z, normalizedCoords.x);
    28.                 float2 sphereCoords = float2(longitude, latitude) * float2(0.5/UNITY_PI, 1.0/UNITY_PI);
    29.                 return float2(0.5,1.0) - sphereCoords;
    30.             }
    31.  
    32.             float4 frag(v2f_customrendertexture IN) : COLOR
    33.             {
    34.                 float2 uv = ToRadialCoords(IN.direction);
    35.                 return tex2D(_VideoTexture, uv );
    36.             }
    37.             ENDCG
    38.             }
    39.     }
    40. }
    41.  
    And setup the custom render texture to use this shader (with a material) and assign the render texture output from the video player in the "VideoTexture" field like so:
    upload_2021-6-1_16-32-15.png

    Note that the CRT is in Relatime Update Mode, otherwiser the video doesn't update.

    Then, the rest is simple, you just have to assign the custom render texture in the HDRI Sky component upload_2021-6-1_16-34-18.png

    And here's the result :)

     
    antoinel_unity, Jun 1, 2021
    #5
    pierred_unity, Ruchir, valarus and 1 other person like this.

  6. GeorgeAdamon

    GeorgeAdamon

    Joined:
    May 31, 2017
    Posts:
    48
    Thanks a lot for the heads-up @antoinel_unity!

    Since I needed this for a 2020.3 project I tried a hack soon after posting this. I ended up modifying a copy of the source code for HDRISky.cs and HDRISkyRenderer.cs, changing the 
    CubemapParameter
    to 
    RenderTextureParameter
    , removing some unused properties in my case, and letting the renderer keep using the same HDRISky.shader.

    This quick hack worked actually, in terms of seeing the spherical video playing in the background. A real-time reflection probe was able to succesfully capture the background, but I had trouble getting the skybox reflect on objects (either via the probe, or via RTX screen space reflection).

    I will make sure to test your solution. Do you thinkyour solution is portable to 2020.3 ?


     
    GeorgeAdamon, Jun 1, 2021
    #6

  7. antoinel_unity

    antoinel_unity

    Unity Technologies

    Joined:
    Jan 7, 2019
    Posts:
    237
    Be sure to set the Update Mode to "Realtime" for the static lighting sky to be updated (used for reflections) upload_2021-6-1_18-30-56.png

    Also for the scene view, you may need to enable the always refresh option for the realtime update to work

    Yes, as long as you manage to have a render texture cubemap in parameter of the sky working, it'll be fine :)

    But on our side, we don't plan to backport the feature as it's a breaking change
     
    antoinel_unity, Jun 1, 2021
    #7
    GeorgeAdamon likes this.

  8. GeorgeAdamon

    GeorgeAdamon

    Joined:
    May 31, 2017
    Posts:
    48
    Awesome, thanks for your clear and detailed explanations, I will test in 2020.3 and report here for posterity.

    Apologies for my slightly aggravated tone on the original message yesterday, under severe stress. :(


     
    GeorgeAdamon, Jun 1, 2021
    #8
    antoinel_unity likes this.

  9. toftof

    toftof

    Joined:
    Apr 12, 2014
    Posts:
    5
    Hi, in the STANDARD RP I could use the Skybox-Panoramic.shader (https://github.com/Unity-Technologies/SkyboxPanoramicShader) to display 3D video inside of Unity as a Skybox. Is it possible to do the same thing with HDRP? I tried your solution, but since the video file contains data for both eyes, it does not render properly.

    Thank you!
     
    toftof, Oct 13, 2021
    #9

  10. GuillaumeLe

    GuillaumeLe

    Joined:
    Nov 11, 2021
    Posts:
    1
    GuillaumeLe, Jul 19, 2022
    #10

  11. herefortriangles

    herefortriangles

    Joined:
    Dec 14, 2021
    Posts:
    3
    Hi @antoinel_unity, thank you for posting the above solution many years ago. I am trying to replicate your solution and results but I am unable to do so. I've tried debugging a lot and am lost. Can you please take a look at the screenshot below and tell me why my 'Cubemap Video (Custom Render Texture)' seems to calculate only one face of the cubemap and not all the 6 faces? Just in case this is relevant, I am using Unity version 2021.3.8f1.
     

    Attached Files:

    herefortriangles, Sep 8, 2022
    #11

  12. antoinel_unity

    antoinel_unity

    Unity Technologies

    Joined:
    Jan 7, 2019
    Posts:
    237
    antoinel_unity, Sep 8, 2022
    #12

  13. herefortriangles

    herefortriangles

    Joined:
    Dec 14, 2021
    Posts:
    3
    @antoinel_unity, Thanks for taking the time to reply! Actually this isn't a VR application. I'm trying to get the skybox to play a panoramic video. I was able to get this working in URP pipeline, but HDRP is proving quite challenging but I need to use HDRP because the rest of the project is using HDRP.

    The shader is exactly the same as the shader you shared above.
     
    herefortriangles, Sep 8, 2022
    #13