[Official] Removing some fixed function shader functions for 5.0?

Why is there even still ANY fixed-function stuff in Unity ShaderLab... ditch all of it, I say. Before Unity 5 is out next year, even the last GL-ES 1.x or OpenGL 1.x or DX7 devices remaining somewhere deep in the steppes of Mongolia will finally have died.

 

I see no direct reason to keep them alive, given that (unless I forgot one) Unity 4.3 removed the last FFP graphics API from support anyway.
For some of the stuff, a simple shader macro would be welcome though depending on what you add to the 'to remove' list on top of what you have there (cubemap UVs, ...)


What I consider somewhat harsh though is calling dot3 obscure, given its relevance on OpenGL ES 1.0 platforms (though that was partially neutred due to Unitys lack of stencil buffer support back then and lack of inbuilt shader that used it, which I personally never understood)

 

If one targets a low end mobile gadget, he should take things in hand and use cg shaders to do specific optimisation as much as possible.
I personally never used fixed function shaders with unity.
So, ditch them all.

 

I'm not a shader master, but I find ShaderLab syntax pretty painful. I'd love to remove most of the ShaderLab syntax to be honest. My recommendation would be to put more of the functionality of the shader onto the material/shader inspectors. Also automatically propagating material properties into the CG program would be nice as well.

3 ways to do the same thing is just too much (surf and vert/frag should be enough).

Eventually, I expect that most of this stuff would be handled by the über shader.

 

Yeah, nuke them.

I've never used fixed function shaders so far.

 

Now this is some pretty cryptic stuff.. I'll have to think long and hard on this one *stupefied*

 

Since fixed function is gone, I would favor stripping all fixed function out and providing a simple upgrade guide with Cg snippets for the more "tricky" things fixed function does for you. Things like TexGen CubeReflect/SphereMap to assist people who were relying on those.

Yes, for an asteroid shader for a cold-call project pitch. This was before OpenGL ES 2 was a good option, so it's irrelevant now.

Code (CSharp):

1. Shader "iPhone/Dot3" {
2. Properties {
3.     _MainTex ("Base (RGB)", 2D) = "white" {}
4.     _BumpMap ("Bump", 2D) = "white" {}
5. }
6. SubShader {
7.     Fog {
8.         Mode Linear
9.         Range 270, 400
10.     }
11.  
12.     Pass {
13.         SetTexture[_BumpMap] {
14.             constantColor [_DotLightDirection]
15.             combine texture dot3 constant
16.         }
17.         SetTexture[_MainTex] {
18.             combine previous * texture double
19.         }
20.     }
21. }
22. }
23.  

-Jon

edit: Err.. I'm a new member now? I registered 3rd!

 

@metaleap Hmm, I wrote that at 3am but what I was trying to convey not make sense?

What I meant to say is most of the settings that I use (Queue, ZTest/Write, BlendFunc, Stencil) could honestly be served by a custom shader inspector that auto-generates code and then double clicking a shader could just bring you to the auto-generated Shader allowing you to tweak the auto-generated shader.

 

To be honest, I'd remove surface shaders too. Not only are they slow and inefficient but they're also another source of confusion. Instead I would introduce more macros for Unity-CG. I would also kill off GLSL too (but obviously you compile to it), and leave one single clean shader language with maximum speed and flexibility, with the focus being on tools like shader forge.

I want Unity to focus on *one* shader language, and one single unified api for it. This will enable tools like Shader Forge to get more optimised, ultimately matching UE4's node shader approach or going beyond.

So in my fantasy unity world there would be just Unity shaders, and Unity Compute Shaders.

Think big, be bold with your vision.

I don't want old stinky poo lying around in Unity 5. I want a lean, mean fast and clean shader pipeline. People will mostly use Universal shader anyways, and there's shader forge. Far better to clean up the docs with as well. Hell, some platforms don't even work with fixed function shaders, and fixed function means people *think* they're doing less typing. If you're doing less typing and that's the problem - use shader forge for gods sake.

Unity needs to mature and sometimes that means not spending time with old toys.

 

Actually - recently I was thinking of using sprite color as the light direction so that I could have a lot of normal mapped sprites without necessarily incurring a massive overhead (think DX7), but it was so klunky and frankly a bit ugly, I dropped it. I'm not sure it would be faster really? The concept was to avoid pixel lights.

In any case whenever unity asks the question "should we do this for speed?" the answer is always yes.

If anyone's on the fence then next time your game is slow don't come crying

 

Yes, please remove them.

 

Where's your latest benchmark for this assertion? I'm just wondering because as far as I know either the GPU driver (on desktop) or Aras' shader optimizer (for mobile / GLSL in general) would ultimately strip unused variables, inline functions, remove unreachable code etc..

I think Surface Shaders are pretty great.. I mean I do custom vert/frag throughout myself, but I'm decidedly a freak.. for people who want to get a project Done instead of tinker away somewhat aimlessly for years like I admittedly love doing, Surface Shaders are pretty damn cool!

Big fat nope from me. And again a case where I'm not myself a user of the feature in question but still have what I think is well-reasoned opinion: generally in terms of philosophy, I think Unity has to remain highly "hackable". This is what keeps attracting me to it and stops from another futile and ultimately doomed attempt of writing an own renderer. There are indeed fringe use-cases where targeting only GL and handwritten GLSL may be imperative. I would even go further and say, give us much broader "#pragma -at-my-own-hackety-risk" access flags, not just pragma glsl but even pragma GL4-with-this-glsl-version-and-those-extensions. Some projects do totally warrant this, but wouldn't quite warrant writing a completely separate engine (with parallel renderloop, cullings, etc. all the other non-shading-related machinations that Unity does well enough).

Guess what, people still can use ShaderForge, they still can use Surface Shaders, they still can use the new "standard shader". But why artificially close already-half-open doors to some hacky fringe needs that no CryEngine or UnrealEngine would enable.. so that newbies are less confused? That's part and parcel of the "newbie learning ladder" we all need to climb. They are clearly instructed to use surface/standard shaders or else face the abyss of learning as you go

 

PS Vita and Mobile. Take your pick. If you've released much you'll just know it's night and day in performance. I wish it wasn't so, but it is, but that's logical because surface shaders do a lot of extra work. It's nothing to do with optimisation. And your comment of tinker away aimlessly for years is redundant, it means you completely didn't understand what Macro meant. It means you can pretty much code exactly the same as surface shaders, just with a clearly defined frag and vert. But then, that's kinda what surface shaders are. But wrapping with magic code has limits, and ultimately that's more overhead for Unity, instead of Unity just focussing on speed.

If you're one of those people who believes coding a shader for longer than 10 mins is tinkering aimlessly, then perhaps you'd get a lot better results with shader forge (which doesn't generate surface shaders).

So you don't in fact, know what you're commenting on? They are to all intents and purposes identical at the end, so one of them is pretty much redundant. In fact as far as I know, nobody actually uses GLSL directly any more. There's no point. In this case cg->glsl is really nicely optimised. Given that cg is cross platform and GLSL isn't, Unity should adopt and expand one of them to maturity and compile to the other.

I'm all for flexibility but when you're so open minded your brain falls out, there's no focus there and certainly no time to bring them all to an optimised maturity. I'm personally tired of how Unity is spread thin, and would like a sharp sword instead of a collection of rusty knives.

 

RRRRIGHT... time for me to bugger off from here for a while..

 

I didn't mean to sound harsh sorry! I should use more smilies. I mean, I personally love tinkering but understand it's not really productive.

I don't disagree with what you're saying, but I don't think its mutually exclusive from what I'm suggesting. Sorry if that came out bad.

 

Stay on topic!

 

It is on topic.

 

"We noticed significant performance increases when we removed lighting, and started using the same color for every surface. So we optimized this approach by using a single screen space quad, and drawing that color once. Unity 6 will feature drastically reduced drawcalls, and significantly faster shader compile times."

 

Glad people have a sense of humour

But yeah, whenever the question comes up "should we do x for speed" then yes they should, at cost of dumbing down Unity, not at cost of visual fidelity.

The fact is, if people can't get their heads around shaders and are not willing to, there's answers for that - node based shader editors and Universal shaders

When you're making an ease of use decision for something that has easy alternatives to (shader forge etc) and it sacrifices performance or efficiency for everyone else, then that's a bad decision IMHO.

The amount of people tinkering with shaders without much of a clue (ie fixed function) is purely down to the lack of an integrated shader editor that performs well.

 

Yeah, I think you could probably ditch the entire fixed function pipeline without much issue.

Not surface shaders, though. As much of a pain as they are, they can be really handy.

 

I guess it makes more sense for them to actually optimise surface shaders. I've just found them dead slow each time when compared to cg.

 

Even when you use parameters such as noambient, novertexlights, nolightmap and nodirlightmap?

 

Hmm, not found them too bad. But for quickly knocking out something that just works with the lighting system, it's far easier to do.

Generally I'll redo it in CG afterwards so it does exactly what I want and no more, but it's like testing an idea in Python before doing it properly in C++.

And I'm happy to do that, others aren't.

 

Yes.

Have you done much with mobile or console? Those things appreciate fast shaders, and cg is the fastest you can get, with surface being less than optimal. For desktop I don't notice any difference, but then I do roll with a 780 ti and not intel integrated

If you look at the source, you'll see redundant operations going on with surface shaders, presumably to err on the side of caution, plus precision goes out of the window.

 

No, that's why I asked

 

Before I can vote on this I would like some education (from anyone patient enough to answer this) since I'm still learning how to write shaders properly. Yes, I'm ignorant, and perhaps this is not the place to ask, but...

Am I correct in assuming that anything done in the fixed pipeline, like the matrix operation for textures and the dot3 operation, has equivalent or better ways to do this in the programmable pipeline?

 

That's correct.

 

All the fixed-function shaders are nowadays actually rewritten into programmable-pipeline shaders on-the-fly by Unity anyway IIRC.

 

I don't like them but I see no reason why to remove them, on the first view they look easier for beginners and hence they can be a reason for people to start writing shaders which is a good thing. It's comparable to your (obvious favorite) node based stuff. But yes, it can be a bad thing and raise confusion but there are a lot of people who use surface shaders so let them continue using them.

There are people who don't want to support platforms that don't use GLSL at all - using CG which will be translated to GLSL anyway would be pretty idiotic then. To be honest CG and GLSL are almost identical it's not hard to write in GLSL first and rewrite the same shader to CG after that if you really think you need it.

If people have problems to write shaders and want to use toys like Shader Forge then fine let them do it. But don't destroy the flexibility in favor of such toys just because some people are too lazy to learn.

All this PlayMaker and Shader Forge stuff goes into the wrong direction and has nothing to do with optimization.

If Unity slowly goes into this direction (instead of coexistence) then I would prefer that they would make that clear and do the transition fast because it would be the top reason for me to leave. I don't have time to waste it on a slowly dying horse.

If you want fast shaders then you need to write them. Optimization takes time and is far from being intuitive. A tool will never do that for you. It's terrible to see all those people selling their totally unoptimized stuff on the asset store which usually can be twice as fast by changing some lines of code in the shaders just because they have no idea about what makes things worse. The real optimization starts after you've eliminated all those obvious things.

That's wrong. I use it.
What makes you think you could know that at all??? The forums are a bad indicator for that because you will usually find those people having problems with something here. I think people who use both might be more experienced than the beginners who run into all these more simple shader problems here.


The best thing they could do is to hire someone like Martin Kraus and let him write a giant shader manual with lot of examples how to do things from standard stuff to complex things and with and without macros. A manual that grows almost every day even with weird experiments. Something that is inspiring. People need to realize that writing shaders is SIMPLE and FUN.

 

It may be simple and fun for a programmer, but I'm fairly convinced it can be done by artists and should be done by artists. I'd rather get on with doing code that matters, and I think shader editors can be optimised enough to the point where they outperform 99% of written shaders.

If that wasn't the case we'd all still be using asm. Shader compilation optimisation is in it's infancy.

 

We clearly have a different basic opinion here and it starts with our ideas about artists and programmers.

For games I would never hire a pure artist or a pure programmer, I just would not trust them do be able to do the job because of either the lack of "classic" creativity or the lack of creative and complex thinking - anyway both would prove with their lack of skills that they are just not really interested in the whole thing and the wrong persons. The only programmer without any art skills that I would ever hire would be a physicist with programming skills or someone who is specialized in networks.
When it comes to shaders it is usually MUCH more than just art, shaders and programming goes hand in hand, both is directly connected and can't be split from each other unless you are talking about simple shaders. A pure artist just can't do this job and a programmer without any art skills would be lost. I think there is not a single shader in my current project that really works or makes sense without other code.

It's obvious that we both seem to have a dramatically different approach on how things basically work.

 

Sure. But I hand-code all our shaders. I don't think that's sustainable for large projects involving tonnes of VFX.

 

For me that's the only way to keep lot of things visually working together.

But the point is both ways are possible with Unity and both ways should be possible in the future. That's why I must protest against your ideas to reduce features in (imaginary) favor of other things (that I don't use at all).

I doubt that (when talking about a really optimized shader) and It's way too complicated to explain that.

 

Leave GLSL and Surface Shaders alone

They are not obsolete and their existence helps many people.

Also, wear sunscreen.

 

Can you provide some examples of this? I'd also love to see one of your improved hand written shaders that replace any of the default Unity shaders as I really don't believe surface shaders are as bad as you make out, at least not when taking in the context of what they do. Just saying it would be really interesting to compare and see where and what the savings are.

That is not to say I don't believe you, I'm pretty sure one could easily produce far better shaders than surface shaders produce, but only in terms of the developer having 'expert knowledge' - i.e. optimising for specific uses, platforms, needs and requirements (not simply their level of shader expertise).

However if trying to emulate exactly what surface shaders do, then whilst i'm pretty sure one could shave off a few instructions here and there, i'd find it debatable as to how much performance that would really gain. Which really is the point of me asking, I don't like seeing unsubstantiated claims, that aren't backed up with some stats and most importantly 'scope' in order to accurately judge the differences.

For example it would, in my opinion, be completely unfair to compare an optimised forward rendering shader that natively support 4 pixel lights (without forward Add), using SM3 or above to a surface shader as they are fundamentally different approaches, one that I feel Unity is probably not in a position to follow.

I will completely agree that surface shades can be far from optimal, but I feel this is due far more to the self imposed restrictions/requirements from Unity than them simply being poorly constructed. So while I could easily see you providing a list of things that are non-optimal, from my point of view I suspect I would argue that they would frequently come down to being practical answers to complex issues.

For example forward rendering is non-optimal due to the decision to have a pass per pixel light, an approach that puzzled me for the longest time. That obviously impacts the design of Unity shaders surface or not. However looking at in context of providing shaders that would have to; fit in SM2 while supporting a wide range of features from TBN normalmaps, lightmaps, shadowmapping , allow dynamic changes to the number of runtime pixel lights supported etc, I can see the advantages of such an approach. At least that's why I assumed this approach was taken.

Sadly of course performance is not one of those advantage, but that's were your graphic programmers earn their money, as frankly it should be expected in performance critical apps that the default surface shaders are rewritten for optimal performance. By which I mean for say forward rendering that they are re-written to support a fixed number of pixel lights without having additional passes. That alone would provide huge savings in performance, far more than shaving off a few instructions here and there, due to the massive savings in drawcalls.

Now one might argue that Unity should be doing that, but honestly I think there are likely to be far too many custom requirements to make it practical. Hence why we had shaderlab and surface shaders in the first place. Unity have to provide a somewhat generic system in order to cover the vast range of platforms, capabilities and features. Perhaps those systems are getting a little dated (hence this thread proposing the removal of FFP ) but its a delicate balancing act for Unity to get right.

I'm not to sure about the validity of redundant operations, particularity if seen in the intermediate shaderlab/debug code as those should be optimised out by the respective 3D API shader compilers. Its hard to know without seeing some examples, but its also the sort of thing I regularly saw out from shader editors and while untidy ultimately I don't think it had any real performance implications.

Then you have cases about precision and certainly its possible to use lower precision for performance gains in shaders, but that can also be very much a case by case basis. If it was good across the board then I would have expected Unity to have already done more than they have improving their current shaders. Again this all goes back to it being difficult, in my opinion, for Unity to maintain a shader codebase that is somewhat generic so it can work across the board, compared to an individual working on specific target hardware who can easily optimise the code.

So I don't want to see surface shaders dropped, it feels like a backwards step, all those Macro's you mentioned are what we had to use and still can ( surface shaders just hide more of it) in the past. No doubt you make full use of Macros for you hand written shaders, but I would have to wonder just how generic or applicable those could be for other developers? I.e. how specific have they become to your target hardware? What sacrifices have you made for some optimisations that might be acceptable for you game but not for some-one else.

What I would like to see is more time given over to documentation for shaders and opening up the source for the shaderlab parses/compiler so that developers can leverage it their own needs - e.g. i'm having a horrible time trying to integrate Geometry shaders with surface shaders, some of which might be alleviated by having access to the the shaderlab parse source (hard to know for sure though).

 

Well that's the thing, it doesn't emulate *exactly* what surface shaders do because you end up not wanting everything surface shaders do. But precision is important, and removing redundant things you don't want is also important - on mobile. On desktop I've not seen a blind bit of difference. I'm crunched at the moment, but when I get time I'd love to explore this topic fully.

Yeah I backtracked on getting rid of surface shaders earlier in the thread. But ultimately if surface shaders are unity's future (they probably will be) then we need to ensure they're going to be the same efficiency and are able to do less where it matters.

I shaved off a few ms where it really counted just changing unity's surface shader precision where used within the shadow system.

It's time consuming chasing performance and I feel those ms make a difference on some devices.

 

You should work as a screenwriter for horror movies. You know how to scare people.

 

Uninteresting side-fact: being a writer is one of my side jobs

 

Great, then I'll guess your horror sentence was just a kind of training to add more drama. It works.

 

Well i'm sure the devs at Unity would love to optimise mobile more, but as they often say too little time too much to do.

As for exploring the topic further I agree completely.

Shaders are so important for real-time 3D and yet are poorly documented, often black-boxed and along with parts of the rendering process have a huge amount of depth and breadth to understand. Every time I write a shader I find I learn something new about them, the 3d API or Unity.

Just not sure how such a topic could be explored. I don't feel forums or IRC are really up to the task, to much to discuss that takes too long to type out. A meet-up would be cool, but doubt I could make it. Voice/video chat/conference would be great, but not sure there is anything around that could support more than a few people at once?

I'm not suggesting this should be done, just your comment got me thinking about it a bit more and that personally it would be great to get a bit of a brain dump between interested parties, asking questions and getting a vareity of answers, etc.

Honestly i'm just not sure that's a realistic goal and I doubt a shader editor would be much better either, but your claim about shader precision for shadows sounds very interesting, so maybe there are a few special case areas that could be improved.

But that's half the fun of it, isn't it?

 

You're right. I changed my mind just after I'd typed it but it was too late by then. Surface shaders are infinitely nicer to use, and in any case cg as nvidia knows it, is entirely redundant. Or out of date right now. So, if Unity is abstracting without performance loss, then I'm going to have to support that. Sounds like surface shaders. We need to make these identical speed to equivalent cg. Hows that done?

 

Just wanted to chime in and say that as an artist, I love surface shaders. They provide nice input/output structure for me where I can grab inputs and textures, experiment with them and feed results in diffuse/normal/emission or whatever output and do not have to worry that shader will not compile properly on different platforms or won't work with certain features of unity ecosystem etc etc.

I know node based shader editors are best bet for artists. However, since unity does not have one I'd have to rely on third party which is not something I feel confident about in this particular case.