Is Unity looking into 4k rendering pipelines?

My understanding is deferred rendering has very high bandwidth limitations so it makes it harder to hit 4k+ resolution at high refresh rates.

So are you looking into alternative rendering pipelines like Clustered Forward Rendering?

 

For instance there is Forward+ rendering an approach talked about in the Ogre3D development forum and covered here -> http://www.3dgep.com/forward-plus/

There is Visibility Buffering mentioned in a recent GDC conference -> http://www.conffx.com/Visibility_Buffer_GDCE.pdf

There is Clustered Forward Rendering used in Doom (latest version) -> http://www.adriancourreges.com/blog/2016/09/09/doom-2016-graphics-study/

And probably a lot more approaches to reducing the load/bandwidth for rendering on a GPU at > 4k resolution.

 

Given the high resolutions used in VR, I imagine this would be of benefit there too. I know a few other VR devs who are itching for a Forward+ solution.

 

Check out the Clustered Forward rendering using in Doom link above.

Have any games/demos/benchmarks used Forward+ yet, it would be interesting to see what it can do?

Edit: Apparently this demo by AMD uses a form of Forward+ rendering (Light Culling).

 

Good as Sharp has unveiled an 8k monitor (7,680-by-4,320)

 

Forward+ would be really exciting to see in Unity! On a slightly tangential note (sorry for going a bit off topic), it would be awesome if you guys could enable MSAA in forward rendering with HDR when using VR. I can force it to use MSAA when NOT using VR, but trying the same trick with VR results in a black screen.

 

That's fantastic news. Hopefully it works with VR too, that'd be the best possible combination of things

 

Well speed + high quality option is welcome on all platforms not just VR... deferred is the high quality option but sometimes you just have too much gpu pressure, so choices are welcome

 

I got a chance to look at the new HDR RenderLoop Unity is working and definitely it will bring a lot of speed to Unity. To give you an idea, it's similar to The Lab Renderer but on roids! More dynamic lights (32) and a maximum of 16 shadowed lights at once on screen (6 shadows = 1 point shadows) in a single pass. Although not quite sure why 16, having 18 will give 3 point lights at once, maybe to avoid running out of textureinterpolators/instructions? I have to investigate more on that.
Anyway, instead of being segmented, shadowmaps are all atlased in a huge giant depth texture, which is again a lot better.
The ID Tech 666 renderer (the one used for the new DOOM) is definitely the way to go.

 

It does seem like the best of all worlds, it's not as though we actually need more than 32 lights! in fact I struggle to use 8 thanks to old habits - reusing them cleverly, so 32 is a luxury. I have concerns Unity might miss some optimisations taken for granted with deferred ie buffers etc, we'll have to see how it all plays out with their new post in performance vs deferred too.

Basically I just want "fast as hell next gen visuals on console bro".

Yeah - this one is just plain weird... it's not very useful unless spotlights take less than 6.

 

Forward+ (or the hybrid approach) is the new way to go.
Tbh, 32 lights for me is more than enough, unless you want to simulate GI only with dynamic lights (good luck with that though). ^^
The new HDR renderer does a depth prepass for a more efficient light culling. This will double your drawcalls as your objects needs to be rendered first in a depth pass and then final color pass but with the benefit of having a depth buffer for special depth effects. It should be a lot faster than traditional forward/defferred rendering when many dynamic lighting and shadows are rendered (with a limitation of 32 dynamic lights, 16 shadows and 32 ref probes) in one pass.
But If you need a basic forward renderer without any depth pre-pass, you can make it. I got one running on SM 2.0 hardware that does up to 3 lights with shadows on it and SM3 with more lights and shadows. If you target SM5, you can have a lot more lights with shadows decode in one pass. Not with the new ScriptableRenderLoop but it's running on C#.

Instead of doing cubedepth textures, they lay out point light faces in the same depth texture atlas, that's why 1 point shadows = 6 spot shadows. A good optimization is to update faces of that point shadow only if needed, for instance if some side of the light don't render anything in its frustum just skip that one and reuse previous ones rendered depth frames. I think the 16 shadows limits have something to do with the way the shadow atlas grid is splitted (gotta have to check that out more in depth btw).

Other benefits of this new renderer is MSAA + HDR, unified lighting (opaque and transparent does the exact same lighting), different objects having different lighting models at once is less tricky, less GPU memory bandwidth, etc. Plus Deferred is known to be very bad with lot of small objects on the screen.
Sticking to traditional Deferred or Forward is not good anymore. Making hybrid approaches is the best way now.

 

Unreal has added a Forward renderer for VR, that sounds very similar to Doom's rendered as it divides the frustum into a grid, and apparently provides a 22% performance boost.

http://www.roadtovr.com/unreal-engine-4-14-forward-renderer-vr-optimized-improvements/

RoboRecall the VR game that uses the new system, quite amusing robotic dialogue and looks good.

 

Unity's experimental Scriptable Render Loop (unknown ETA), the code for which is public here, is implementing "Cluster FPTL", which is likely based on Morten Mikkelsen's "Fine Pruned Tiled Lighting" as mentioned here. Mikkelsen is currently a Unity employee (at least he was last I knew), though he does not appear to be working on this himself.

Note, the C# and shader code for the Scriptable Render Loop is public, but it uses engine features and APIs that don't exist in any public build so it's not really publicly usable yet.

 

Very excited, been a bit slow on certain platforms unmentionable...

 

FPTL sounds interesting (read the link) and im curious to know more details about the implications this will have and how it will be used?

 

Seeing that it's not part of the first public beta of 5.6, does it mean it'll be released in a future beta release or as a separate experimental build?

 

So FPTL is a version of whats called Forward+?

 

Forward rendering using FPTL would be a type of "Forward+", though it looks like FPTL is specifically only implemented for deferred in Unity's current Scriptable Render Loop. This isn't necessarily a bad thing as they implemented Clustered Lighting for forward rendering instead. Clustered can be a bit easier to use with forward rendering than FPTL and most Tiled Lighting implementations since they require a depth pre-pass for optimal rendering performance, whereas Clustered Lighting does not. If you're rendering deferred you get the depth pass "for free" during the gbuffer creation.

How the original papers defined Forward+ vs many modern implementations of what people call Forward+ don't really match, but any kind of Tiled Lighting or Clustered Lighting combined with forward rendering could be called Forward+ at this point, though generally only Tiled Lighting techniques actually fit the original definition.

The easiest way to understand "traditional" forward vs forward+ is in a traditional forward renderer you calculate what lights effect each object individually and either render those lights in multiple passes of the object (like Unity's forward render path) or pass a few of only the brightest lights for each object and render in one pass (like Unity's vertex lit render path). Forward+ techniques instead split up the screen into tiles and make lists of which lights can be seen in each tile. When objects are rendered they check their screen position and get the list of lights to render for that pixel. Clustered extends this by also grouping lights by distance from the camera.

 

Thanx for the great explanation!
Looking at some pages it seems like "Forward+" is quite a lot faster than deferred for many small lights, which would make it ideal for VR. I wonder when this is coming out...

 

https://twitter.com/aras_p/status/808694318332542982

 

I see on gitHub that Scriptable rendering loop works now with 5.6, would be nice to see some experiments around that : ) I suck to much on this kind of thing to try it out myself : p

 

I've not had time to explore the scriptable render loop stuff beyond cursorily digging through the depot, but does any of this work with VR yet? I mean the tech itself conceptually does of course, similar stuff is already being used in other engines for VR and I'm using a custom naive clustered approach in a project already, but I haven't had time to see if what's implemented currently in 5.6 and the depot works with VR.

 

Good to know. The inability to get script access to the per eye projection matrices with single pass stereo has made doing efficient frustum based tiled or clustered lighting implausible within the current (5.5 and prior) systems. I've contemplated doing a single, oversized, super frustum and fudging the projection matrix to encompass both eyes, but have avoided that rabbit hole in favor of waiting for your work. I'm very excited to get in and play with it eventually.

 

Just wondering if this was rolled into the new rendering pipelines and which rendering setups is the lowest in bandwidth for higher resolution displays?

 

That would be LWRP.
LWRP will be good for everything if you want mainly performance over visuals.
HDRP also supports VR now so you can try that in either Unity 2019.1 or 2019.2a