Custom Render Pipeline suited specifically for VR

I am thinking of making a thesis project in my university and the subject i am thinking of is basically something like the title of this post. I wanted to ask from people of greater experience and knowledge than me, if it is possible to make a custom render pipeline using the SRP in order to optimize the rendering process so that VR applications can achieve photorealistic visuals. The concept behind is that VR apps limit the visuals because it needs double the render time. So what if it is possible to design a pipeline from scratch(reprogramming each stage or adding and assembling different stages to achieve this)

 

Some ideas:
https://developer.oculus.com/blog/introducing-stereo-shading-reprojection-for-unity/
https://developer.oculus.com/blog/hybrid-mono-rendering-in-ue4-and-unity/

 

thank you so much, i really appreciate it. Any resources or information are welcome

 

Afaik, both LWRP and HDRP already support stereo instancing on VR, you don't have to reinvent the wheel.

 

apparently not all features are supported

 

It takes time to implement the support for all features, these will be more featured in the future. It would still take more time to build SRP for VR from scratch though. Is there something specifically missing right now that you'd need?

 

Its not something that i need. Its something i need to find and make for my thesis more likely. Like i said i am thinking of making some optimizations for VR. The goal is to achieve ultra visuals in the most optimal way possible. I believe that with graphics programming and SRP i can achieve this. In one year and less i will have graduated from university as electrical engineer. Right now i need to do a thesis project in computer graphics (VR,Unity and Game Dev have to be included in this project) so that i can have good chances to apply for Phd in the same field. So if you have any ideas that i could do and has VR + Unity in graphics programming then i will appreciate it

 

Your best resources on this would probably be LWRP and HDRP both as they both do implement what you've asked here. Could also check what Valve did with The Lab Renderer for Unity but I think LWRP already uses similar concepts to it.

 

Yes i am thinking if there is a way of combining them, or at least make features that are prohibited for VR right now, possible for it

 

You only need to use LWRP and bake. This will be photographic. Use proper PBR values and it should look same as a photo. This is known as a pure baked pipeline.

For everything dynamic you look at HDRP.

The real problem here is where you contest what is solved. What are you *really* asking for?

 

I'm still curious what these features that aren't there, would be?

 

Didnt quite understand that

 

Come on, dont tell me that u can build a AAA game with so realistic-heavy visuals and port it to VR straight on

 

Visuals on VR are more about optimizing your content once the renderer part optimizations are done. Both LWRP and HDRP both implement VR optimizations, they also implement many of the regular rendering features, but you haven't answered the question: what are the features that are missing from the LWRP and HDRP VR (that makes them nonviable solutions)?

 

Well if i knew i wouldnt make the post would I. I am not saying that they are not viable solutions, they are really impressive actually. I am asking the simple question to whomever may know whether there is more to improve to optimize. And if there is anything that is already implemented on non VR games and not on VR cause of the constraints it adds.

 

I am about to start a project and i am trying to comprehend whats worth researching for and implementing or improving. It just has to be in Unity and VR for real-time. One idea might be the ray tracing feature NVIDIA implemented on the RTX series

 

From what I gather from your posts, you have the wrong idea about where HDRP and LWRP fit, and you also don't have any clue about VR in general or you wouldn't be making comments about raytracing.

Also you should know HDRP has support for RTX. And support for VR. It's also made by some of the smartest AAA developers on the planet. You need to probably just sit down and start using it for a few weeks, see how far you get.

After all, shouldn't you know what is there and what is a problem, before you try to fix it?

 

I have some basic idea maybe not in the same level as you or as anyone here, well hey i am a student and i am trying to understand. And i would appreciate if you help me understand. Whats the right idea about HDRP and LWRP.?And why VR and raytracing shouldnt be put in the same sentence?

Couldn't agree more, i am not trying to do in some months what serious smart and professional developers took years to do. What I am trying to do is work in something that is worth working for, has some potential and can be evaluated as serious work.Not copying some dude's code or "playing" with those new Pipelines without customizing something and achieving something at least in basic level. I am already working with SRP since i have some intermediate knowledge with OpenGL and GLSL. And by the way mr.Hippocoder nobody obliged you to answer my post and any questions. I didn't intend to insult anyone or underestimate the hard work of people who made those improvements and render pipelines. For that reason i don't understand why you are so aggressive towards me after all i didn't force you to answer or post. Either way any suggestions, ideas, guidance from experienced professionals or anyone would be appreciated. Thanks in advance for your answers.

 

@psomgeorg I think it would help to define what specifications are needed for photoreal (triangle count, texture density etc), the idea of photorealism & technical features required to achieve is subjective. Also the target hardware should be specified for this to be really beneficial.
SRP is customizable & great, but you may end up hitting a wall with Unity as an engine, when it comes down to how the draw calls are issued and handled on the C++ side, when and how the camera matrices are updated with VR tracking, when async reprojection happens, how much GPU work you can schedule in between the core engine stuff etc... Unity can do VR, but it isn't a VR-only engine, so just that alone can be a disadvantage if comparing to writing an engine which was entirely focused for VR and all the inner workings are engineered in a way that is strictly for VR.
So I'm sure you can experiment with rendering techniques for VR in SRP and likely be successful with getting it working, but it is still at the mercy of whatever Unity's core engine foundation is internally, and you may end up wanting more low level access comparable to your experience in OpenGL.

 

If you are looking at VR one interesting area could be VR post processing effects. For example other than the expensive CTAA asset I'm not aware of any TAA for Unity that works well in VR.

 

Mostly because realtime raytracing at interactive framerates has been a very longterm holy grail of computer graphics. Nvidia adding some hardware acceleration for this, and Microsoft supporting it in DirectX are notable steps in the right direction, but it is important to understand the difference between the hype and marketing and the reality. Just study some game review benchmarks of how performance is affected when RTX-based effects are switched on in the handful of released games that support this stuff. Even for non-VR applications, it can be questionable as to whether the visual improvements are worth the decrease in framerate at this stage of GPU raytracing power.

I say this as someone who is very enthusiastic about DXR and RTX. But I'm in it for the long-haul, and for situations where I can get away with targeting framerates such as 60 or 30, not VR that really wants 90.

Now it may be possible that novel approaches to using accelerated raytracing may be imagined and implemented that are actually applicable to VR in the next few years, rather than the next few decades. But the devil is in the detail and requires a background in graphics research that I do not have. Its not where I would look, more fertile ground is likely to be found elsewhere.

 

I like this suggestion because it keeps the scope of work within some kind of realistic bounds. Especially when contrasted with reimagining the entire pipeline, which sounds like a very broad and complex mission, and I would not know where to even begin talking about it without the technical detail of what the alternative pipeline would do differently being clearly understood.

 

I suppose if I were in this situation I would be more likely to be looking to add something to LWRP, because:

Its more mature, as opposed to HDRP which is still a moving target (HDRP under heavy development, VR support not finished yet).

Its much simpler, so easier to grasp how its code works.

It deliberately has a lot of missing features to keep it light, so its probably going to be far easier to find missing things that people want.

For those reasons, I consider LWRP a possible starting point for making a MWRP (medium weight render pipeline) whilst still maintaining performance necessary for VR. With the added bonus that feature(s) you would be looking to add are already well understood and implemented elsewhere, as opposed to being found only in the realms of graphics research papers or someones imagination.

 

i appreciate your answers and thank you for explaining. I hope it wont mind asking here and cant find a suitable area to ask to. What are the most unsolved problems in computer graphics(lets forget for a sec SRP, Unity or even VR). I mean what needs more research, better solutions, more optimized implementations when it comes for instance in volumetric lighting, ambient occlusion, deferred shading or any post process effect?

 

Well, visit this site if you are looking forward to solving problems in real time rendering:
https://openproblems.realtimerendering.com/

However, you do need proper knowledge about graphics and rendering to begin with those presentations.

 

2017 its kinda outdated but thx

 

Lol, Outdated does not mean that they are solved. They are still a problem in the field of graphics.

 

i will check it out thank you

 

My suggestion would be look forward to
Anti-Aliasing: Are We There Yet? - from 2015
and
A Certain Slant of Light -- from 2017.

as they are something that Unity(in general all game engines) lack and try to implement them in Hdrp/Lwrp and then probably share details/ problems you faced in Hdrp/Lwrp here

 

I'm currently appraising these render pipelines for VR.
It would be great to get an update on how this project progressed and any conclusions drawn...