Question Trying to get an HDRP portal to work on VR

NEWS : This subject is under an unity issue. Please vote here if you also want portals systems in VR https://issuetracker.unity3d.com/is...position-node-does-not-work-when-xr-is-active

Hello there !

I've made the project free to access here : https://gitlab.com/bouchiermonodtristan/hdrpportalvr
So the main goal is to have a working HDRP VR portal (And potentially URP later), since I havn't found any on the asset store nor on the internet.
I managed to get a working HDRP portal system.

I've been using this youtube video :

Sebastian Lague do share it's portal project with us on github : https://github.com/SebLague/Portals/tree/master

So basically I used shadergraph with the Unlit node and I converted the shader he's using to have his code working with HDRP. I also had to put his render function for the portals in a LateUpdate loop, since the OnPreCull event was removed from HDRP as seen here : https://forum.unity.com/threads/fee...e-render-pipelines.470095/page-8#post-3408481

The Shader in shadergraph is very simple. I just connected the screen position as UV and declared a texture2D as an entry for the texture sampled.


A camera for each portal follow the player camera location and the render will be captured on renders textures to apply them on the portals materials, using the shader above.
As it can be seen, in this video : https://gyazo.com/338782343c0992fc2fcbbf2cb9c1975c the cameras follow the player location.
I also had to remove the post processing from this cameras because it stacked with the player post processing camera and ended up showing different results on the portal.

Then, our portal.cs script apply at runtime a texture2D from the camera attached to the portal to the portal screen material. It's done here : https://hastebin.com/qewevarati.csharp in the void render() function. This function is called as I said above in late update, as the last script in the script execution order.
The script also do other things like taking care of cutting the mesh when traveling the portal but it doesn't matter for now.

Here is the result : a portal properly working in HDRP https://gyazo.com/8ad09a3aec9d4762d678017c508954e4

Now the problem is : Whenever I try to plug my VR headset on (Oculus CV1), everything turn wrong. The result can be seen here : https://gyazo.com/79ce55f67240dd13056e33b66b95a25d
It seem to put both player eyes on the screen as a render texture at the same time.

I'd like to be able to apply the left eye of the render camera on the left eye render of the portal and the right eye of the render camera on the right eye render of the portal.

I do not really know where to start, and if it require to change the shader in shader graph if there is even a way to do it. Since we are on HDRP we cannot really handwrite the shaders.

Thanks if you have any start of answers !

 

It looks like someone over here ran into the same problem as me : https://answers.unity.com/questions/1785666/creating-portal-effect-in-vr-using-a-render-textur.html

 

https://forum.unity.com/threads/getting-screen-coordinates-for-single-pass-stereo-vr.672562/

In this link, the author suggest to do a custom node in shader graph with this piece of code

Code (CSharp):

1. OUT = IN;
2. float4 scaleOffset = unity_StereoScaleOffset[unity_StereoEyeIndex];
3. //check that stereo is enabled
4. if (scaleOffset.x > 0){
5.     OUT.x /= 2;
6.     OUT.x += scaleOffset.z;
7. }

Which end up looking like this :


Unfortunately, the shader is pink :

With this message : Shader error in 'Shader Graphs/PortalSheyneVR': undeclared identifier 'unity_StereoScaleOffset' at line 2254 (on d3d11)

The unity documentation suggest me to enable the single pass in stereo rendering method : https://docs.unity3d.com/2019.3/Documentation/Manual/SinglePassStereoRendering.html
But it is obsolete, since I am using the XR plugin management :

And using Multi Pass or Single Pass Instantied doesn't change anything :


It do seem like I have to find a way to replace the "unity_StereoScaleOffset". Does someone have an idea ?

 

Ok, after futher investigation it looks like
unity_StereoEyeIndex
and
unity_CameraProjection
cannot be used neither in the custom shadergraph node for some reason. I start to think that something is missing in my shadergraph files.

Edit: I had undeclared identifier errors in shadergraph with HDRP 7.3.1, I still have this problem in HDRP 7.5.2 and rollbacked in 7.3.1

 

I uploaded a version of this project here : https://gitlab.com/bouchiermonodtristan/hdrpportalvr if someone want to give it a try.

The scene is under the scene folder. The vr device HMDMock should be on by default so you can try the scene even without any headset, and the shader is actually in portal/Scripts/Core/SheyneVRPortal it doesn't work on vr tought.

 

Ok it seem that the VR camera flipped bug came from the HDRP 7.3.1 as said here : https://forum.unity.com/threads/vr-additional-cameras-render-to-texture-issue.872815/
I'm updating to 7.4.1

EDIT : It ended up on an Issue, the same as here : https://forum.unity.com/threads/steamvr-doesnt-work-with-hdrp-7-4-1.908483/

I'm updating to 7.4.2

Doesn't working.

EDIT 2 : It is finally working on HDRP 7.4.3 and Unity 2019.4.16f

But the portal ScreenPosition is all F***ed up in the shader thought : https://gyazo.com/1bc7fb14259ed924fde49d8a28c7dc4c

So to resume :
Now I need
1) to force render my portals cameras in VR mode,
2) to capture the left render on a render texture, and the right render on another render texture
3) to change my shader so it take the left texture on left eye and the right texture on right eyes (I'm not even sure I can do that in HDRP)
4) to hope using screenPosition will be enough in the shader to put the textures on the plane without artefact, or to find a way to compute the right position of the portal screen on the player view.

 

So it's said here : https://forum.unity.com/threads/how-to-create-stereo-rendertextures-and-cameras.925175/
That unity doesn't actually support multiple vr camera for performance reason, which mean i'll have to uses 1 vr camera and 4 cameras, two per portal. It start to seem crazy.

 

So far I havn't been able to make it work. I am still looking for it. I feel like i need to use a written shader instead of shadergraph. I'm trying to hire people to make one. So if you feel interested and feel like you can do it drop a PM.

To resume : It works in HDRP non VR.
It doesn't work on HDRP VR

 

Hello,I think it is because the normal portal shader is using screen position to remap the texture.But in vr the camera would render twice in different position and combine them to one.And I'm working on these effect too,maybe I will post my further progress soon.

 

Hello,
I'm trying the same on URP and have the same problem with VR. Could anyone find a solution?

 

I am going to give this a new try this evening. Hopefully since 6 months elapsed Unity may have debuged shadergraph to allow the use of VR on the screen position node.
@sunsi12138 any news from your side ?

 

So I updated the project to unity 2021.2, added two camera per portals, changed the script, made good use of the Eye Node from shadergraph 12.0.0. So far so good. Only one limitation : Shadergraph screenposition node still not take account the fact you are using a VR headset. So the render textures from the cameras doesn't fit the portal screen properly.

I don't seem to be able to do any call to unity keywords in the custom node from shadergraph. So i'm blocked. It seem just impossible to do a proper portal without hand writing a shader. And unity provide no documentation about a way to do that outside of shadergraph. The shader is Unlit, so it should not be too much to handle for a skilled shader person.

I am willing to pay someone to solve this issue. So feel free to contact me in DM if you are interested in making this portal system to work in VR.

 

Out of despair I opened a report issue, asking if it was possible to fix the ScreenPosition node for use in VR, or at least to give me a workaround

 

So here is the link with the issue, if anyone want vr portals, please upvote this https://issuetracker.unity3d.com/is...position-node-does-not-work-when-xr-is-active

 

I've been following the same tutorial (best portal tutorial I've seen so far tbh), sticking to non-VR (for now) but still using HDRP. I'd solved the OnPreCull issue (differently, yours is simpler and best I think) but couldn't get past the SetTexture one. Sadly the hastebin link seems to be dead, would mind telling me how you got past this, pretty please? Or fixing the link? That'd be much, MUCH appreciated

Edit: there also seems to be an issue with recursive rendering, I had to comment out the portalCam.Render() bit from Render() (my script is still in an early state though since I couldn't get past the part I mentionned above)

Edit2: solved my problems, nevermind!

 

It's been two more months now and so far we are still waiting for Unity to fix this issue to be able to create a VR portal. with shadergraph https://issuetracker.unity3d.com/is...position-node-does-not-work-when-xr-is-active
If you are also looking for VR shaders, feel free to upvote it so it gets more priority

 

I voted. Did they reply? They normally reply pretty quick to my bug reports.

 

They actually replied two day after I summited the bug report, so they created the ticket properly. But now it is in a stage where it's on their todo. And since two months it's been locked in this state with no news.
Does unity usually reply after the bug is in the issuetracker ? Cause they havn't to me

 

Same problem here, I'm also willing to pay a bit if someone can figure it out.

For me, it seems the right eye's view is in the spot of the left eye on the render texture and vice versa, causing you to see double. I'm not too savvy with shaders, and this problems is over my head. I'm on URP.

My scene is like this:
A secondary camera capturing the view through the portal, then using its render texture in a shader graph to paint in the pixels on the other end of the portal in screen space.

 

Here's an example. In this scene, the portal camera is a child of the main camera, so if everything works, the portal should basically be invisible besides its little border. You can see that in the portal, the left eye is seeing what appears to be the right eye's view, and the right eye is seeing the left eye's view.



For example, imagine cutting out the portal from the left eye's screen and pasting it in the right eye's box. It would then be correct I think.

Know any way to swap the eyes in the shader?

 

No I think they'd normally only reply once/if it's fixed. Best bet is to keep posting here until a Unity employee notices and post the issue elsewhere to get more votes.

 

Hi, I'm a little new to bug reporting to Unity, do you guys know if this would be fixed for a 2019.4.X version of Unity? Right now for good desktop VR on valve index, 2020 and 2021 are not the best (the new VR systems don't even have finger tracking so I have to use OpenVR and weird stuff) so if they fix it for a 2021 or 2022 version... I would be in a quite bad position even if it is fixed.

I would also be glad to pay something if this has some fix :/

 

I am also tracking this issue. I have everything set up for my portal to work but no joy.. I am going to need to rethink how I can get this working.. cant be a case of "im too early" in wanting a feature. Im sure I can code a work around.. my shader is a little different mind you

 

Saw this on the /Graphics/ GitHub page, it claims to be a fix to the screen position node, but I can't seem to get it working with the provided setup.

Link: [ShaderGraph] [2022.1] [Ruby] Fix screen position behaviors by cdxntchou · Pull Request #6040 · Unity-Technologies/Graphics (github.com)

Hoping someone gets this to work.

 

So I got this answer :

Code (CSharp):

1. Hello,
2.  
3. Thank you for submitting this issue. Our Shadergraph team has evaluated it and provided the following feedback:
4.  
5. "The ShaderGraph they are using is set up assuming their "alternate world" camera has the same aspect ratio and FOV as the main rendering camera, which is the requirement to be able to just do a "blit" between the cameras for the portal effect.
6.  
7. However, when XR is enabled in their project, the alternate world camera does not render as an XR camera; it is still rendering at the original non-XR resolution and aspect ratio (and not multi eye I believe).  So the blit ShaderGraph ends up stretching the result inappropriately.
8.  
9. What they are trying to do is definitely going to require much more manual intervention on their part to handle the multiple eyes and the resolution differences.  The blit approach could still work as long as their alternate world camera uses the exact same layout (i.e. XR split eye with same aspect ratio and FOV), but I think their shadergraph would have to take care to sample from the correct eye from the alternate world camera."
10.  
11. The XR Team is evaluating XR specific Shadergraph nodes as a future feature that should handle cases like this.
12.  
13. Please let us know if this information helps you resolve the issue.
14.  
15. Thank you,
16. Mario
17. XR QA

This does not help to resolve the issue. The ticket is still open so far, but from this text we learned that they may consider adding a node in the future to solve this issue, but that we still have to wait for a while. So it's not that bad of a news !

 

New update here.
Apparently Unity doesn't intend to fix ShaderGraph, making it impossible to create any VR mirror and portals on the render pipelines.

https://issuetracker.unity3d.com/is...position-node-does-not-work-when-xr-is-active

I'm super sad and disappointed to have reached this conclusion after one year and half of expectation, but I guess that's it. If I want to make my game idea, it does mean that I cannot do it on Unity Engine...

 

Trying to reach out to the XR team is probably the only hope now.

 

Hi, we are trying to do same, we are using HDRP 12. Do you think any way to just use simple stencil in shader graph?
please really need help.

 

same S*** here

 

I am still working on it. I think I have it working but using a yet different shader. Trying my best to get it on urp and will update here.
Of course we have two renders, blip texture could work.. but wont be revisitng this for another month