Official What is next for us at Unity with Scriptable Render Pipelines

Of shader graph missing a register/portal node, how can anybody work like this?


Meanwhile, in ASE, a bit uglier looking , but it is far ahead SG. Why can't unity just copy some good parts if they are already proven to work?


Edit: Guess how many people are working on ASE and how fast any issues are solved.

 

Since we are on the topic , this is what bothers me

 

To me it makes sense to have One Render Pipeline. And then have a very big highly capable set of customizable surface shaders to make the render pipeline look like what ever type of graphics you want for your game.

It would make sense that one render pipeline would be far less likely to introduce another set of problems in some other area of the engine. Some how it almost feels like a house of cards that really does not need to be taking the chance, to introduce more errors then could be necessary. Advantages ? Disadvantages ?

I can hear the words " hallelujah " coming from dev's that make assets only have to deal with one RP, that will probably break far less in successive versions of Unity. Same thing could be from people that want to code surface shader's to sell on the store. ( This could free up a lot of game dev , work time ) ..

I guess one would have to add up all the pro's and con's of just having one RP .. but I have a feeling where it would end up.

Another rhetorical question. IF* a company could hire, one of the " Best at any cost " worldwide person to code the render pipeline and be in charge and have final say in that development.... IF* that company could know with a high degree of certainty, they would only have to do that once ... and be set for good .. who would not do that ?

Just an observation and guessing, but I would say, let the professionals do their job's the best they know how.. and keep the " suits " out of development of the real engine. Good code , should have no politics.

 

Personally, these are some of the issues with the SRP API that were exposed for me when I was working on a custom SRP for a project:

• No access to lower-level C# and C++ code, cannot tinker with culling code and jobs.
• Cannot access object data from CullingResults.
• Monolithic lights, cannot do custom/semi-custom culling for light shadows (UE5's virtual SM sounds tasty).
• Inability to draw shadows with a custom SRP mask as a parameter for ScriptableRenderContext.DrawShadows(). This can be useful for situations where you want to skip some renderers on some cascades for example. In my own SRP I've repurposed the newly introduced StaticShadowcaster bool property for this feature.
• Poor scheduling of culling jobs and their control, has to do a lot of multithreaded code before returning on the main thread, wasting tons of performance in some cases (would've been great if I could Complete() on some jobs before accessing specific data like VisibleLights, but ensuring that ScriptableRenderContext.Cull() wastes as little of the main-thread time as possible).
• No GL.InvertCulling equivalent for CommandBuffer. GL.InvertCulling still works, but it sticks out like a sore thumb.
• CommandBuffers not being safe as UT claims, main-thread checks are everywhere, cannot fill command buffers off the main-thread even if I can guarantee that all the UnityObjects aren't null on the C++ side or if I'm not using UnityObjects as arguments at all. Also considering that CommandBuffers get used a lot of times per frame with an SRP, the safety checks can add up to the performance overhead.
• Somewhat slow submit of the ScriptableRenderContext, wasting quite a bit of performance on the main thread.
• No Async Compute on PC Vulkan.
• No SM6.0 intrinsics on PC DX12 yet (being worked on with the introduction of the DXC compiler, but it's a one-way ticket because DXC only compiles to DX12, no mechanism of targeting DX11 and DX12 with the DXC being a primary target yet IIRC).
• No mechanism or helper methods to maintain material keywords and disabled passes, need to code those myself. Absolutely need those because SRP Batcher differentiates keywords by the keyword string array (not by keywords in the shader) and need to disable unused material passes.
• Inability to have additional per-object properties (through MaterialPropertyBlock, perhaps) while also using SRP Batcher (SRP batcher not used in that case). Can create dedicated material instances at runtime to mitigate that, but this method is not usable in the editor (dirties the scene because of the necessity to modify SharedMaterials). Also cannot do, say, custom light probes with this method, as the number of various materials can become silly. I'm not jumping on board to Hybrid Renderer for this.

Gotta reiterate though, that despite these issues I still think that SRP API was quite a good and enjoyable experience for me. Creating a custom SRP was a relatively smooth experience and it allowed us to tailor rendering specifically to our needs while introducing features that cannot be done in BIRP. The benefits most definitely outweigh the drawbacks in our case. Stings all the more knowing that it can get better, but probably won't.

 

This isn't limited to SRP, but: I do not understand why individual components aren't excellent. Why isn't ShaderGraph amazing? Why isn't it the kind of shader node editor that other engines and software take notes from, the kind that Unreal users look and start pestering Epic to copy features from? Why doesn't it make the Amplify Shader Editor look like amateur hour in comparison?

Especially if the plan was to kill hand written shaders.

Why is it instead a weirdly limiting, sub-par hack job, that got to 70% of the way there and then stopped, because I guess they can check the "we have a shader editor" checkbox, so why bother.

All new Unity features feel like they were made by someone that really didn't want to create any new features, but they kept being pestered, so they half heartedly made something that barely passes as the requested features so they can be left alone.

 

This

 

My guess is the issue is a lack of talent and resources and more things to work on than ever. They have the cash to do these things, but there's a rather limited number of people in the world who are top-flight rendering engine talent. Most of those people work... well... on Unreal Engine. Not that there are no smart folks at Unity, but there are not enough of them to go around, I imagine.

The situation seems to be exasperated by the fact they have such focus drift. UE5 picked a lane and a target and threw all their brain cells at it. Unity tries to be all things to all people, with an ever-increasing pile of use-cases and needed tech pieces such that it's impossible to make any one coherent thing anymore. Multiple rendering pipelines... tons of half-completed tools and experimental features... it just keeps digging a deeper and deeper hole.

Just look at the unity.com website compared to unrealengine.com and you can easily see that Unity has a real identity crisis on its hands, even if they are reluctant to admit it.

 

For more than a year I'm crying over this one.
Also they got custom function node in shade graph but don't have it in VFX graph .

This are all low effort high usefulness features so WTF!

 

I think this is the problem.

I don't buy the lack of talent argument. You need maybe one really talented person with clear vision per team to take lead. They can poach people from Epic, they have money, or they can entice them with a more exciting working place, or something.

 

It's drifting off topic again, best keep the discussion to SRP. If the discussion is about Unity as a business it would belong in General discussion so long as it's not about the people.

 

They would be very hard-pressed to poach talent from Unreal. There is quite a disparity in the two organizations, not to mention where people like that would rather deploy their talents. FWIW: This is related to SRP, as that's the specific talent I am referring too.


I find it amazing, though, that the last big release of URP talked about point lights having shadows as if that was a major advancement. Just stunning that something as fundamentally basic as that is considered newsworthy to Unity. Compare to what Unreal rolls out in its announcements.

Further, the shadows are not even done as well as Unreal. No virtual shadow mapping, just the same old inefficient standard pipeline style implementation. And they first talked about it in May of 2018. It's 2021... for those counting. If this doesn't demonstrate the level of dysfunction, I don't know what does.

Our license agreement came up for our studio, and instead of renewing it for 3 years, we've renewed it for one (forgoing the price break we've taken advantage of for a long time). I just cannot rely on Unity being a competitive engine, at least in the end player-facing product, anymore.

 

My problems with SRP are:

HDRP doesn’t scale down enough.
SRPs in general are not easily extensible, either I own the whole SRP or I use an existing one as is.
Neither SRP is a superset of built in.

And URP is the least exciting thing ever.

It is decade+ old rendering tech (even with the latest cutting edge additions of point light shadows and box projected cube maps)

Supporting tools are weirdly limiting and undercooked (Shadergraph).

Performance is eh. For how slow built in is supposed to be, Unity surely has trouble coming up with scenarios where URP clearly beats built in in performance.

 

Maybe instead of waiting for Unity to fix things we do it ourselves?
Aka. Community Made SRP, created by the community for the community with are needs in mind.
If something like that got going with some momentum and was maintained then users could even start making Asset on the asset store for it.Think of what other community's have been able to achieve and do for so long? (Quake, HalfLife, Emulators, etc.) With the amount off talent frequenting these forums there is no way any one can tell me its not possible.

Plus a community maintained SRP would shift the weight of responsibility off of one single person which is a big reason why no one has wanted to bother making there own.

 

I would do that if:
1- there was clear documentation on SRP (there isn't. You have to reverse engineer URP or HDRP and improvise)
2- SRP was stable. (as long as I know, it isn't, and it's changing with URP and HDRP)
3- Surface Shaders was compatible with SRP in general. (it isn't. And I don't even know if we could make Unity's shader graph compatible with a custom SRP)

I have been waiting for years for SRP to stabilize and get full documentation.
Until that happens, I'm not even touching it with a stick.

 

I can say that the SRP API is quite stable IMO. My own SRP started off on Unity version 2018.3, now the project is on 2021.1, we've been using 2018.3, 2019.2, 2019.3, 2020.1, 2020.2 and 2021.1. Not once our SRP broke down or had new issues because of changes to the SRP API, new features and changes were non-intrusive at all.

 

That's good to know, thanks!
What about the documentation though? Did I miss it somewhere, or does it still require reverse engineering the URP or HDRP?
In your experience, do you think it is possible to port the built-in forward render pipeline to SRP and keep compatibility in some way?

 

Reverse engineering was required, but only minimally. The API is fairly easy to understand, CommandBuffers do not require any special treatment and you work with them as you always have, only submitting them to ScriptableRenderContext. 80% of an SRP is CommandBuffer code, the rest is what SRP API directly provides.

Some documentation is available here (https://docs.unity3d.com/Manual/ScriptableRenderPipeline.html) and here (https://docs.unity3d.com/Packages/com.unity.render-pipelines.core@11.0/manual/index.html).

I don't think that completely porting BIRP forward to SRP would be possible. BIRP's Forward draws objects once per pixel light that affect them, which isn't possible in SRP (though this method of drawing lights is highly questionable for modern rendering IMO). Grabpass isn't supported as well.

However, theoretically, porting the bare bones of it (considering that you only have directional light) should be possible.

 

Unfortunately, but there is one serious caveat - asset store guys will complain a lot about the support for one more renderPipe.

 

That completely missis the point, why would every one work on a SRP that would not be compatible with current assets? That would literally be making the same mistake Unity took. You could make a SRP that was compatibly with BIRP-URP and to some extent HDRP (its a beast on a different level). Look at BetterShaders by JasonBooth. He made a asset that lets you write shaders that work with all the pipelines and he did it in a few weeks. Unity just in the latest versions added BIRP support and it's taken them a few years.

 

The solution for "too much render pipelines" will never be "another render pipeline".
The solution should come in the form of full retro-compatibility and a cohesive ecosystem.
Passing from a render pipeline to the other should be completely transparent for the user and should never break old shaders.

Surface Shaders accomplished this in an elegant and worry-free manner for the user: you can write a shader once and make it work from Unity 3 up to Unity 2021 without issues. (on built-in, of course)

If SRP can't do that, then it's dead on arrival:
Unity is not only an "engine", it's above all an ECOSYSTEM built by a large community of people over many many years.
You can't throw that out of the window just because you feel like making a "better" or "faster" renderer, because in doing so, you are throwing away what made Unity great in the first place and you are removing any incentive to keep using it in the future.

As I said earlier: if I have to redo all my graphical programming from scratch, I may as well redo it somewhere else.
Specially considering that after deprecating the built-in render pipeline, Monobehaviours will be next in favor of the new and shiny DOTS. (it's not official, but I can see them coming)

 

I don't think that this can ever work. The point of URP and HDRP is that they offer different features, so they will never be fully compatible. If we forced them to be fully compatible with each other we would be restricting the development of each.

I agree that there should be a basic common standard surface shader between the two, and a standard interface offering extensibility points which they're both able to provide, but they will need to differ in some significant ways and the switch will never be seamless.

The question is whether there should be multiple pipelines at all.

 

I don't necessarily agree. You're not meant to freely switch between them - that's not what they were designed for. I think that as many assets as possible should be able to be written in an SRP agnostic way so that they work with both over a common shading and extensibility API, but it's okay for there to be some differences. You're meant to commit to one at the start of a project and stick with it. I understand that somebody may wish to switch halfway through development, but I think it's okay if that takes some effort to transition from one pipeline's features to the other.

 

I can't find one single person, on these forums, that does not want a single pipeline that scales from mobile to DXR, bouncing off low and high end XR along the way.

 

I completely agree that that is the ideal thing to work against. The obvious question is how feasible it is to make a single pipeline that fully scales to both of those extremes and actually offers a best in class experience at each of the extremes. If you look at Unreal for example, they're good at high end rendering but they don't scale down that well.

When you have a large list of separate rendering features that can be swapped in and out you get a combinatoric explosion problem when they interact. Sometimes the high performance way to do something is fundamentally different from the high quality way. You either need to add support for weird feature combinations that are unlikely to ever be used, or you have a mess of potential configurations that report "feature X is incompatible with feature Y". The more combinations you allow, the less you can optimise. It's a bit like building against arbitrary hardware vs optimising for a known console. Ultimately I don't have the expertise to say how solvable this combinatoric problem is.

I can see the appeal of wanting to avoid these problems, but the multi pipeline "fix" is a bad user experience.

I do wonder if there was a potential world where they went the two pipeline route but designed things cleanly enough that usability was much better, and we're all very happy about it because of how fantastically the engine performance can scale across devices.

 

Why not? They scale from PS5 to mobile, how much lower do you want?

Also Built in does the same. Now imagine a built in that wasn’t abandoned 4 years ago and that got 4 more years of performance improvements and new features.

 

I'm basically stuck on URP because it's the only thing that can scale down that far. Fair enough, I can live with that, a single pipeline. It's a shame I had to say good-bye to HDRP but I guess that is their design.

Only a complete idiot is going to actually attempt converting between them including all their scenes and lighting. It's part of Unity's plan to make us idiots though because that is why shadergraph is crippled FUBAR.

Problem is URP still isn't anywhere near as fast or usable as it obviously could be, and that's confusing. I'll probably hold out a little longer for surface shaders but if it doesn't come in hard and optimised and finished in under one year from this date, then it never will.

And I think a lot of people will move on if they fail that, because it's flogging a dead horse.

 

Seriously, none of this was an issue with built-in.
We're going backwards, creating issues to fix instead of fixing issues.
The ONLY thing that built-in forward needed was single-pass lighting, instead of multi-pass.
And Valve actually fixed that when they made "the lab renderer" for the HTC Vive VR headset.
Why didn't Unity integrate this solution instead of creating all this mess?

 

This might seem off topic but I was thinking of this earlier:
https://blog.unity.com/technology/behind-the-scenes-speeding-up-unity-workflows

The reason why I don't think having surface shaders alone will help me feel better about being wedded to Unity is because as others have mentioned the problems feel systemic. Using that link as an example a performance increase of 90% is celebrated.

"This is because they swapped from an ordered linked list to an array for managing your scene hierarchy". What worries me is why the hell was it like this in the first place. A 90% increase is Huge, that is terribly written code that should never have made it live because it sounds like a simple change to fix it.

Then later on there is a 1400x speed up for selecting lot of items. 1400x is just nuts, again only very bad code can be at fault for this.

Same story with the Burst compiler as a whole. The 10x performance number was touted to show how good it is. Now Burst is awesome, but a 10x increase is Very significant and only comes from having a lot of bad code in the first place (E.G Unity teaching bad habbits and not inlining methods etc etc). What is a better number is seeing how burst compares to say some C++ code. I would hope/suspect its along the lines of a 1.5>2x speedup, the rest is Unity dead weight.

The one in the link that says we tweaked threads, reduced their memory cost and got a 8.7% speed up. That is what I want to hear more of because it shows the original code was good and now its been gone over with a fine tooth comb to make some improvements.

 

To be fair, I prefer for them to recognize and fix the errors than to simply sweep them under the rug, like every other company does.
In example: I can't believe HOW SLOW modern compilers are when I'm using a 12 core CPU with 64GB of super fast RAM on an M.2 NVME SSD x4 on PCIe-4!
If compilers had been like that in the past, 8bit games would have never end compilation and Microsoft would still be compiling Windows 95...

Or the uber mess that Microsoft Office is: how can you make a word processor that eats up SO MUCH RAM and CPU and takes SO MUCH TIME to start up??

So, for the Unity editor to have been optimized, I am glad.

Actually, the Unity Editor has only got better and better, so I not only can't complain about it, but I actually compliment them for a good job, so THANK YOU FOR YOUR WORK UNITY EDITOR TEAM!

I am however very much concerned about the state of the render pipeline and the complete chasm it has generated in the lovely ecosystem that Unity was.

 

OK don't get soft though. We wont get that if Unity thinks it's all chef's kiss. I want to see proper movement and I'm entitled ask for that as I have given them a lot of time and money.

 

This I disagree with, its been getting Really sluggish lately to the point of being borderline unaccapable which I suspect is what prompted that investigation I linked in the first place.

However, this is a quick response to fix it compared to SRP issues. Also a good example of Unity listening to the user base and acting so I will give them credit there.

Still worries me it needed fixed in the first place though

 

Well you make it suck at the beginning so later when you got nothing to brag about you can make this basic change and show people how much improvement you have made.

 

Let's not get into tin foil hat mode now. I don't think they had any interest in make it suck from the start.

I don't know, maybe it depends on each others hardware or specific projects. I have been having a great experience with the editor except for the ASTC compression taking ages to complete and the occasional full reimport triggered when updating Unity.
So I'm VERY happy to see the ASTC compression sped up and the build target switch boosted.
Overall I like the Unity Editor very much.

But as lovely as the editor could be, if the built-in pipeline is removed, I'm done.

 

I figure URP and HDRP will have good times with DOTS / Hybrid Renderer, so I doubt the future is built-in but we shall have to wait and see what Unity decides.

 

We all know unity is designed to make small indie lofi designer brand games, anyway.

 

Naw, you can make real stuff. This is what we're building with Unity (Built-In RP):



Making this work in Unity is.... tough. We look longingly at Unreal Engine 5... but that's not a thing yet. The SRP options are not sufficiently fleshed out to do what we are trying to accomplish yet. They keep getting closer, but it is such a slow slog.

 

How many systems did you either write from scratch or use bought tools to solve problems that really shouldn't have been problems in the first place though?

This is us:




I've lost track of how many systems I've wrote/bought and thought to myself, why did I have to do this?

 

Well I've done my fair share of 3D engine development. I was the principal engineer/designer of HeroEngine, which is an MMO engine. We sold that division off in 2009. The 3D engine is dated by today's standards, obviously, but was good enough for the era. The engine was used in games like Star Wars: The Old Republic MMO. To this day, no game engine or tool (or addon) has a collaborative editing environment like we created despite so many attempts.

And of course, I'm doing a lot of custom stuff in GALAHAD 3093 (the screenshot I put there). I use compute shaders and some crazy ways of using lights and particle systems to make them performant in the built-in pipeline. Most features seem to fall apart at scale and require us to do a lot of fast dancing to keep things performant at any level.

Unity's scriptable pipelines, in one way or another, don't even meet that bar yet. I had to throw a fit to get the HDRP team to realize that alpha in render targets needed to actually... you know... work! They don't eat their own dog-food, like Unreal does, and so you get so many unworkable deadends. I wish they understood just how detrimental this is to their efforts.

 

This is very important feedback for all developers using SRP https://forum.unity.com/threads/survey-multi-platform-development.1140547/#post-7324267

 

It's not. Feedback is only as important as the willingness of the one receiving it to act on it.

Also, maybe it's only me, but aren't all recent Unity surveys super weirdly worded / structured and slightly off-topic?

I don't feel like I'm conveying the feedback I want to convey through them.

 

By design.
Will not fix.

 

So you are telling me that you made that in unity, while:
- having years of experience making commercial renderer used in big name game.
- and it was tough
- and you still had to throw a fit at unity for an obscure use.

I mean weird flex but thanks for making my point. Because even with the skills you had to karen to the management. Imagine average dev who is a designer artist but not an advance tech artist... he won't.

 

Not flexing, just describing my experience.

We do get some attention from Unity, but we try not to abuse it. Sometimes we have intractable Unity-issues and don't bring them up, so as not to become (too) annoying. For example, we are now experiencing significant memory leaks that seem engine-related (they do not show in the memory profiler, for example, but crash the server and clients). I might have to start yammering on that if we can't fight through it first.

But my goal is always to figure it out ourselves first... if it is at all possible.

But yeah, anything worthwhile is hard to do. Unity makes it amazingly easy to get started on things, but if you set your vision high... like AA or (fret) AAA, then the engine strains and falters. The SRPs don't solve this issue in any way yet. HDRP is targeted there but is still not a viable solution as much as I want it to be.

So... we are stuck in Built-In RP land for the foreseeable future. And I definitely wish it was otherwise.

 

"portals" / remote edges, set/get variable nodes should have been a feature from the the very beginning and ought to be implemented sooner than later but fwiw, it is in the works. once the graph tools foundation api is finished, shader graph, visual effect graph and most or all other visual programming languages (VPL) in unity will have their front ends ported to graph tools foundation and share many common features and workflows such as "portals" to unify (pun not intended lol) the disjointed mess that we have right now where every VPL in unity is littered with its own idiosyncrasies and limitations that the other VPLs could / should be aligned on, as most of this divergence is not domain specific but rather general purpose / foundational. some related information:
portals in graph tools foundation
https://docs.unity3d.com/Packages/c...n@0.11/manual/graphview-overview.html#portals
graph tools roadmap announcement https://forum.unity.com/threads/visual-scripting-roadmap-update-september-2020.978732/
unofficial shadergraph "portals" variable custom nodes by @Cyanilux https://github.com/Cyanilux/ShaderGraphVariables

 

I never have a spaghetti problem with shadergraph but then I do collapse a lot to sub graphs, so it's all neat. I totally recommend it to people. Find out about them here (anyone): https://docs.unity3d.com/Packages/com.unity.shadergraph@7.1/manual/Sub-graph.html

 

I take a different approach with the shaders and I found it to be the best workflow, at least for me and I think for the users too.

I have a big base function in Ase (sub graph) with a lot of exposed options so users can customize their shaders without the user needing to have any shader knoledge.
Here is an example:

Internally, I use a lot of subgraphs as well for things I reuse in other graphs, but for the rest, I like to keep everything in one place and I just couldn't work without register/get variables, this particular graph is quite big.

Some other really cool things Ase has would be a switch by pipeline node and the functions switches which allows you to customize the output of a subgraph via toggles or enums.

 

Graph foundations looks quite nice! It would be awesome for other graph based assets to adopt it once it is ready for production. Now everybody uses a custom solution. Even unity uses different solutiins for their graphs.