Search Unity

  1. Unity Asset Manager is now available in public beta. Try it out now and join the conversation here in the forums.
    Dismiss Notice
  2. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  3. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Official Block Shaders: Surface Shaders for SRPs and more (Public Demo Now Available!)

Discussion in 'Graphics Experimental Previews' started by dnach, Oct 19, 2022.

  1. aleksandrk

    aleksandrk

    Unity Technologies

    Joined:
    Jul 3, 2017
    Posts:
    2,983
    We do have an idea about that, but I don't think we're ready to share this publicly just yet. When we know better how things are converging, we'll post an update.
     
  2. Edy

    Edy

    Joined:
    Jun 3, 2010
    Posts:
    2,497
    While that would be nice, don't forget that the whole point is that BlockShaders should work in BiRP as well. So we could also write our new shaders as BlockShaders and everything would keep working in BiRP.

    ShaderLab supports BiRP, so I can't understand why support for BiRP can't be provided straight way or needs some discussion around adding it or not...
     
    VIC20 likes this.
  3. aleksandrk

    aleksandrk

    Unity Technologies

    Joined:
    Jul 3, 2017
    Posts:
    2,983
    Don't get us wrong, BlockShaders could work with BiRP. The system doesn't prevent that.

    It's always a question of priorities and time involved. If it's two weeks of work, it's easy enough to include. But if it adds months to development, pushes back the release and adds a lot of extra support burden? It's a way tougher decision :)

    Again, we're not saying "this won't happen". We have been talking about this internally and will continue doing so.
    Thank you for the feedback, it very valuable in our internal discussions. We'll keep you updated with our roadmap :)
     
    Edy likes this.
  4. Edy

    Edy

    Joined:
    Jun 3, 2010
    Posts:
    2,497
    ....and then resolves the situation many of us have been stuck into for way too many years? Allowing us to finally use BiRP, URP and HDRP interchangeably? Well, that's what I'd expect Unity to do :) Don't know the actual "cons", but for sure the "pros" (making Unity "unity" again) should make the decision much easier ;)
     
  5. VIC20

    VIC20

    Joined:
    Jan 19, 2008
    Posts:
    2,687
    Seriously, what the hell could be more important?
     
  6. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    5,461
    Any word on a mac build?
     
  7. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    5,461

    If supporting BiRP is months of work and support, your system is badly designed and broken.

    I managed cross compatible shaders with a block like system (stackables) in three weeks as a solo developer with no documentation on shaders or support from Unity, having to fully reverse engineer all 3 pipelines- your a 6000 employee company with billions in resources who wrote the very pipelines you're trying to abstract and inflicted us with this mess in the first place and are now making shoddy excuses for why you might not be able to actually solve the problem.

    Unity literally took my business, and a bunch of others, and said "Reverse engineer these two extra pipelines or you'll be out of business", and then provided no actual help in doing that, which would have been trivial to do. If this was Epic, they'd have canceled the royalties we pay for asset sales and had engineers working with us on it. Unity was unbelievably callous to its asset store developers and users with this move, and then acted incredibly arrogant about it for years, denying it was a problem and even going so far as to say it couldn't be done.

    If you can't write a template for BiRP in a day, something is very wrong and perhaps you should buy a system that can actually do that instead of writing your own. Seriously, I'm trying to be helpful here, but how would you feel if you were forced to do several years of unpaid labor?
     
    Jaimi, Shushustorm, TieSKey and 19 others like this.
  8. JesOb

    JesOb

    Joined:
    Sep 3, 2012
    Posts:
    1,106
    May be then better to make tech for recent apis and only then create additional workaround just for old devices with warning that we can not use Feature 1, 2, 6, and 18 if we target OpenGL ES
     
    joshcamas and Saniell like this.
  9. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    5,461
    Seriously, Unity complaining about the potential support burden of supporting BiRP is about the richest thing I've ever heard. Try rewriting some of the most complex shaders in Unity for all 9 pipelines I now support URP2018, URP2019, URP2020, URP2021, BiRP, HDRP2018, HDRP2019, HDRP2020, HDRP2021, then fielding every bug in those pipelines because people don't know if it's the render pipeline, the abstraction layer, or something in the shader that's causing the bug. SRP issues are still the largest part of supporting my products. Do it all by yourself, with what can only be described as resentment from Unity for calling them out on the issue after warning them repeatedly about the mess they were going to make before they even made it.

    So yeah, I'm really feeling for your potential support burden here. Until you have the entire engineering team at Unity on it, it's not matching mine.
     
    Jaimi, Shushustorm, ElliotB and 13 others like this.
  10. joshcamas

    joshcamas

    Joined:
    Jun 16, 2017
    Posts:
    1,276
    THIS. The entirety of Unity, including HDRP being held back because URP needs to support an ancient device is asinine.
     
  11. fherbst

    fherbst

    Joined:
    Jun 24, 2012
    Posts:
    802
    Exciting! But also scary! I second some of the comments in here regarding "yet another layer" being added.

    So, as a result of that, some questions:
    1. will URP/Lit etc., HDRP/Lit etc. be built on Block Shaders?
    2. will ShaderGraph run on top of Block Shaders?
    3. will the underlying SRP core shaders be "patchable" with blocks as well?

    As an example of (3): "make custom fog for one shader" was never the problem, "change how fog works across shaders in this project" is the problem. Right now one has to modify URP core and change the code for how fog mixing works. Having a way to replace the "fog block" in a project would be desired, same as for other systems. I don't want to implement custom fog calculations for each shader.

    This is another scary point – so is it extensible or not? One big goal of bringing back text-based shaders, in my opinion, should be that one can "do whatever", e.g. the full set of "what shaders can do" becomes unlocked in the right hands.
    The comment makes it sound like it will be another system that needs work from Unity's end (thus, blocking innovation) to implement advanced things. E.g. I'm able to do that with ShaderGraph, but need to modify core; my hope would be that a text-based system makes that easier, not harder.

    I also second everyone else here that either you should support BiRP or remove BiRP entirely. That's a black-and-white decision – either you keep BiRP and support it or you remove it. It's not acceptable having to write shaders in yet another system. I'm happy that finally we can use "one ShaderGraph for everything" in some projects and not have pink everywhere all the time when switching pipelines... don't regress that please :)
     
  12. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    5,461
    A quick google search says it's still 30% of android phones that don't support vulcan. Would be a shame if this greatly impacts the API because that number will be reduced over time, but it's not an ignorable percent right now. In a year or three when this is out and games start shipping with it though?
     
    transat and joshcamas like this.
  13. JesOb

    JesOb

    Joined:
    Sep 3, 2012
    Posts:
    1,106
    Slightly more precise stats
    • From this page, you can see that on 01 July 2022 Vulkan was supported on 77% devices
    • This is not gamers this is all users around the globe with phone
    • From mobile GPU specs, you can know that all devices shipped from 2016 support vulkan (adreno 5xx etc)
    • You can find that phone with adreno 306 onboard shipped in 2017 but I'm sure you will not support it in 2024+
    • If you see how Vulkan support changes over time you will notice that it increases by 3% each 4 months
    • This means that for today Vulkan support already is 80% in 1.5 years when we can potentially ship games with unity 2023.2 Vulkan support will be around 92% for the entire phone market
    • Average gamer auditory that plays some demanded games will have 100% Vulkan support. For example, we have FPS game and auditory of it already have 100% Vulkan support
    • in 2024 any phone that don't support vulkan will be 8 years old. What games want to support that phones? If such games exists why they can not just stick to Unity 2022.3 and forget about Block Shaders at all?


    Vulkan 2022 03 Mar .png Vulkan 2022 07 Jul .png
     
    Last edited: Oct 20, 2022
    wellmor, OBiwer, VirtusH and 3 others like this.
  14. sacb0y

    sacb0y

    Joined:
    May 9, 2016
    Posts:
    867
    Personally I think the community needs to decide what they want as well.

    Is it enough to just make a converter so people can move on, or does it need full support?

    I don't think people will really know until we see this "coexistence" effort in action.

    That said I get the vulkan demand it did perform better on android for me, but in my experience it's not ready yet as far as unity goes. Just not remotely stable.
     
  15. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    5,461
    My experience with f2p mobile was that supporting the low end users was not worth the effort - they accounted for more bad reviews and spent little money. But there may be other platforms and stats at play here than just android. Personally I find your reasoning sound.

    I think a converter is a totally different ask than what is being proposed here - doing conversion from surface shaders seems like the kind of thing you could sink a ton of time into and still have things not work, since stuff like Grab Pass isn't available and surface shaders allowed for some very wacky stuff. And for many of us, we've already had to rewrite everything 7-9 times, so getting one system we can write to instead of having to backwards engineer things every few months would be a huge win.

    As for how it works out, if done right it shouldn't be much different than what Better Shaders does right now, just with a different format/syntax and some different edge cases that arise from its design. You should be able to write a shader that goes to any pipeline, and have any extra bits for a specific pipeline just get stripped in the other ones. You should be able to take a shader in this system and easily add additional shader functionality to it without having to modify the original shader. That should be automatically updated when Unity changes the render pipeline in some way, and 'just work' through multiple versions of Unity and the render pipeline without breaking. You should be able to write custom templates for custom pipelines or lighting models. If it doesn't do this, the effort has failed. To test this, my plan is to port all of Better Lit Shader's stackables to the new system, seeing what is missing along the way, what's easier, what's harder, etc.

    Now, there are lots of cool stretch goals possible - like having users be able to combine shader blocks created with text and shader graph via the interface system, etc. Or being able to convert from a surface shader. But all of these require this pipeline to be built first.
     
    VirtusH, noio and Edy like this.
  16. marcte_unity

    marcte_unity

    Unity Technologies

    Joined:
    Sep 24, 2018
    Posts:
    17
    @fherbst said,

    1. Yes, we are coordinating with the SRP teams to ensure this system is sufficient for their needs and users can build with the same Shader Blocks we use internally.
    2. This is the long-term plan, yes. We have a prototype working already but we're not sure of the release date yet.
    3. You can extend/inherit from the URP Template with your own Template to override any or all of its functionality. Then any shader you base on that Template will pick up your override but still get base URP function when you haven't.
     
    OCASM likes this.
  17. fherbst

    fherbst

    Joined:
    Jun 24, 2012
    Posts:
    802
    Thanks, good to know – how long is long-term as layed out in (2) if I may ask?

    - that's not quite what I asked - my question is overriding the existing shaders. If I modify URP core today to change how fog works that changes for all URP shaders, ShaderGraphs, presumably Block Shaders. Will I be able to do that with Block Shaders / changing the core blocks without having to modify the URP package?
     
  18. marcte_unity

    marcte_unity

    Unity Technologies

    Joined:
    Sep 24, 2018
    Posts:
    17
    @jbooth we're looking forward to the results of your porting effort! That will be very valuable feedback to ensure the Shader Foundry meets the needs of Asset Store developers.

    Regarding fully open vs more restricted systems, here's my take. As a gamedev on a deadline, you're always going to want full control to modify whatever, we all get that. I've been there. But as a platform owner, we need to build something that works today and hopefully 10 years from now. If we leave things fully open, you all will get up to your usual shenanigans doing crazy (beautiful!) stuff, and we'll be locked out from critical changes to avoid breaking you. Creating an interface that we can work behind gives us that future-proofing and lets us continue to evolve this system so we don't end up with Yet Another Unity System (tm).

    A big part of the reason behind releasing early alphas is to get feedback on what kinds of use cases you're concerned won't fit in the Foundry design. We've gotten some good callouts already, so please keep 'em coming.

    Finally, we're not removing plain Shader Lab. For those last-minute emergencies, you can always write that directly, or take the output of a Block Shader and tweak it if you need full control.
     
    Gooren, CiroContns, noio and 3 others like this.
  19. Edy

    Edy

    Joined:
    Jun 3, 2010
    Posts:
    2,497
    BiRP support from the very beginning. Don't let this remain as "we'll see" or "in our backlog for later". It's already supported by Shader Graph. Please just add it here as well so we could provide support from the first alphas.
     
  20. aleksandrk

    aleksandrk

    Unity Technologies

    Joined:
    Jul 3, 2017
    Posts:
    2,983
    It may be better. But there's still a large share of devices that Unity currently supports (yes, based on Adreno 3xx), and we can't cut the from the equation yet.
    Such a layer could be built, for sure. It's a non-trivial thing to build, given that definitions of what "separate sampler state" means are so different between GL and everything else. We evaluated it at some point, and we may end up doing that. It just has to be prioritised against other things as well :)

    Unfortunately, they are not so ancient. The hardware is, for sure, but there were new devices released even in 2021 with it. That's just 3 years old in 2024.

    There's multiple approached to that. One is to make a custom fog block and use it in all shaders that need it (likely a one-liner change in each). Another option is to extend the default template and add your custom fog block there, although this requires your shaders to work with this new template instead of the default one.
    The good old "modify the block in the package" would also work and be as cumbersome as it is right now :D

    We'll evaluate if allowing to override some blocks would be something we should support. A big part of the equation is how to let SRPs change the internal implementation without breaking your code immediately.

    If you need to provide "everything what regular shaders can do, without Unity intervention", you're back to regular shaders (although, arguably, the shader compiler may need to be updated on our side as well to support all the shiny new things available in the latest and greatest shader model in DX or other APIs).
    With this in mind, this system is a layer on top of ShaderLab, and things that are outside of HLSLPROGRAM or those that are specific to ShaderLab will need support from our side, like pragmas that specify stage entry points.

    That's "devices that say they support Vulkan". This doesn't mean Unity would select Vulkan on those - many early GPU drivers were a mess, and we blacklist some of them.
     
  21. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    5,461
    Any news on a working mac build?
     
  22. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    5,461
    This is all a very valid argument- I guess the common worry here is that Unity will pull people off the project and stick it in stasis like so many projects before it; long before it has all the features that are needed. For instance, tessellation support in shader graph was like 4 years later (and maybe not in URP yet?). Shader Graph's API for user created nodes was made internal with the promise that it would be added later, and isn't even on the roadmap. Audio Mixer has not had an update since Unity 5.0. This is a common thing with Unity, and to some extend any system which cannot be easily extended by users. A large point of handwritten shaders is not being held up by these types of issues. For instance, you could easily add tessellation support to Better Shaders without modifying its source if I had not added it. But I totally get the idea that you want everything in your system so you can refactor it or optimize it's output behind the scenes for new technology, etc.

    That said, I suspect a lot more of this is being built in C# and available via a package vs. the old surface shader system, so as undesirable and not valid for the asset store as this would be, it's another form of "out" in a worst case.
     
    Le_Tai, goncalo09, JesOb and 3 others like this.
  23. jessebarker

    jessebarker

    Unity Technologies

    Joined:
    Dec 13, 2016
    Posts:
    6
    I updated the quickstart guide with a link to a Unity.pkg for Mac, as well as some instructions that might come in handy around getting it installed (like telling macOS to open the package and run the installer, and telling Hub to add the Editor to your list of installs - as this isn't a real release). I'm running with the build from that package now, but it's possible I left out a nuance of the install steps, so please reply back if it doesn't work as described.
     
    jbooth likes this.
  24. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    5,461
    works!
     
    dnach and aleksandrk like this.
  25. bac9-flcl

    bac9-flcl

    Joined:
    Dec 5, 2012
    Posts:
    829
    I really want to add another voice to the need for out of the box support for BIRP. I understand skipping that for an initial demo but it is something that should be possible to add in a reasonable time. Please prioritize adding it. You mention there is nothing about the design that prevents it, which is great! I hope adding support for BIRP goes smoothly, but if doesn't or ends up taking excessive time, there is something off about the design and it's best to discover that early and course correct the design to address it. BIRP not being under active development anymore should be a benefit here: if it's not actively being changed anymore, then the bulk of an adapter that allows Blocks to output to it would only have to be written once, and would hardly need updates over time, in contrast to URP/HDRP adapters.

    I think that projects that have to support multiple pipelines are a rarity, but almost every single user of Unity benefits from the rich third-party asset ecosystem, such as third party shaders. If it continues to be hard to ship third-party shaders or assets depending on custom shaders (vegetation etc.) to all Unity users, the asset store offerings will shrink over time, and ultimately Unity would be losing one of its huge selling points. Needing to support BIRP+URP+HDRP is not something an end user developing one specific game will necessarily encounter, which might make it seem it's not an important priority, but I feel that the knock on effects of not prioritizing that are really problematic for the ecosystem we all rely on.
     
    Last edited: Oct 22, 2022
  26. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    5,461
    Ok, so starting to dig into this a little bit to understand the examples. Kinda thinking out loud and also asking questions:

    - OffsetPositionAlongNormal is called from a lot of different shaders, but there's no particular syntax I see bringing that function into the shader - is it just globally scoped? What happens if someone else has a function of the same name?
    - The interfaces allow us to call the apply function in another block, and remap data between blocks if they don't share the same interface. Can I call internal functions embedded in that block? Like public float Foo() { return _Bar*2; } can I myBloc.Foo()? Does a block act as more of a glorified include or as a scope for functions and data?
    - What happens if multiple blocks have the same property name within one shader?
    - What happens if multiple of the same block appear in the same shader? Either to do things like add multiple copies of the same effect, or just because multiple blocks use the same sub-block for some reason?
    - Where is the C preprocessor available? In Better Shaders, adding the same block multiple times mutates the defines and property names, but I suspect that is not the case here. So in an example of wanting several texture layers available, you'd have to declare the sets of properties it needs multiple times and remap them to the values used in the actual block function. When one of these blocks has 30 properties and 30 shader keywords, this can be both time consuming and a potential for bugs. So being able to use the C preprocessor to macro a definition which keeps them all in sync would be useful.
    - It annoys me that the standard for functions is lowercase first camel case, but that's a personal thing.
    - Can interfaces be included in instead of just copied between blocks?
    - Are there going to be some standard interfaces for (all/individual) pipelines? For instance, if I rewrite Better Lit Shader in this system, I'd want to be able to add modules from that to whatever shaders unity writes. Basically the surface output structure just gets passed along between all the blocks and modified.
    - Fundamentally a given template will expect a certain interface - it would be nice if these were easy to find, and when you new up a shader it gives you the expected output of your shader so you know what you are writing to (albedo, etc).
    - Another personal taste thing, but I think standardizing some common stuff was super nice in Better Shaders. Knowing that I can always get the tangentSpaceViewDir by that exact name instead of it being arbitrarily up to every shader to name things whatever they want made coding much nicer. While obviously custom stuff will be made better by custom names, I think being able to rename the vertex position in clip space, etc, actually only hurts the quality of the shaders. So perhaps this is something a given template can provide?
    - Are there pass specific defines I can check? For instance, to prevent shader stripping errors sometimes I need to #if _PASS_DEPTHNORMAL some things.
    - Same for HDRP/URP/BiRP defines, and version numbers of those systems.

    I'll start writing some code with it when I can and I expect some of this will be more clear, and I'll have a bunch of new questions..
     
  27. LaireonGames

    LaireonGames

    Joined:
    Nov 16, 2013
    Posts:
    705
    I've been bedridden with COVID this past week and only just noticed all this. I'm glad to see progress being shared!

    Everything is still a bit fuzzy so I don't trust myself to look into the nitty gritty details but here are some things that stood out to me from this thread etc:

    +1, this annoys me so much and I much prefer uppercase first camel case (is that normal camel case?)

    On a similar vein, what is up with "metalness"? Feels like bad english and doesn't match existing wording of metallic. Why reinvent the wheel, it it just to match roughness? If you want to improve clarity, sure, but it doesn't feel like it does that. Please preserve existing wording where it still makes sense to do so. A huge point of this endeavour is to get people moving onto SRP so you should be looking at pain points and ease of transition during the whole process.

    With this in mind, what utter madness is Unity thinking by not supporting BIRP as a standard?

    The SRP blitz day (or whatever it was called) stated 70% of users still use BIRP. Considering how much time, effort and frankly, deception has been spent trying to get users to convert that number has huge significance. It's not just resistance to change or nostalgia or whatever reasons I've been quoted by Unity staff.

    Not supporting BIRP from even with this announcement feels like yet another attempt to force users to Unity's will. Lets hint at this carrot being only in SRP and see how they respond....

    Well you forget, I don't actually want this. I imagine most people don't actually want yet another way to write shaders, why would we possibly?

    What we want about this is an end to insanity. A way for us to be on modern graphics pipelines without rewriting years of work. That is the only reason to be excited about this and it looks like again, Unity have utterly missed the point. If the end goal is to really move us over to URP then this system should have been viewed as the tool to help with the transition and this decision shows its not been viewed that way.

    Which means I have serious reservations about it.

    @marcte_unity "If we leave things fully open, you all will get up to your usual shenanigans doing crazy (beautiful!) stuff, and we'll be locked out from critical changes to avoid breaking you."

    This topic keeps coming up.... Unity needs to do a study to fully understand what should and shouldn't be allowed to break. If only there was a fully fledged game you could test these things on...... If I have wrote a low level hack or done something very specific in a shader and it breaks between versions because of a critical update then I am fine with it.

    However what annoys the heck out of me is hearing this excuse thrown in our faces constantly as a reason to justify constant luke warm, half baked, products yet at the same time LTS are an utter joke of stability. Here is a thread of people complaining about uGUI being basically broken for most of this LTS:

    https://forum.unity.com/threads/can...-broken-after-upgrade-to-unity-2021-3.1308642

    Its like we are constantly getting the worst of both worlds!
     
    Noisecrime, atomicjoe and goncalo09 like this.
  28. AcidArrow

    AcidArrow

    Joined:
    May 20, 2010
    Posts:
    11,631
    Thanks for saying that, it was interesting to read, and I am happy that these matters also worry Unity employees.

    Having said that I'm having a really hard time believing that Unity is able to create a system that continues to evolve and become better / more capable after it hits 1.0.

    My jaded heart thinks this will release as a 1.0, will have an ok reception (kinda promising, but still a lot of issues), and then, apart from the minorest of features and critical bug fixes, nothing will change.

    So with that in mind, to me it's CLEAR: the system needs to be super hackable, so once again the users can do Unity's job for them and somehow bend this into a usable system.

    But, I don't work at Unity. I don't know how things are there these days. I'm just sharing how it looks to me from the outside. If you can see futures that my jaded heart cannot dream, then sure, go ahead, and make the first Unity feature that keeps evolving.

    Just make sure you remember that a feature that isn't open / hackable and does not evolve post 1.0, is a dead feature.
     
  29. recursive

    recursive

    Joined:
    Jul 12, 2012
    Posts:
    669
    If this is eventually going to handle supporting shadergraph, is there something that can be done for simplifying precision support for custom HLSL nodes? It's either write a hlsl file with 2x the function calls for both
    float
    and
    half
    or use the
    $precision
    specifier in shadergraph nodes, which reduces the redunancy but is annoying to remember.

    I was trying to write a shader library asset, and this was the most annoying thing next to Multi-pipeline support (and I don't feel like I have anything to say that others haven't already pointed out with SRP issues).
     
  30. fherbst

    fherbst

    Joined:
    Jun 24, 2012
    Posts:
    802
    I think this nails the problem with the approach: for years now, advanced users complain that new shiny things are developed for the least common denominator (which is easier to support) with no path towards proper control as with the layers below the new shiny thing.

    ShaderGraph still has no way to do custom pragmas. Of course people found workarounds - but that should not be necessary. "Old" Unity core has a great separation between "here's public things that are supported" and "here's public things that are unsupported, don't rely on those for long term support". New systems should do the same. Users getting "creative" should be embraced, not frowned upon and wherever possible prevented...
    (this goes a bit beyond the core thread topic, happy to discuss elsewhere - in summary, please provide attachment points to do complex things. Even when they are explicitly unsupported.)
     
    cecarlsen, vertxxyz and Wokky like this.
  31. aleksandrk

    aleksandrk

    Unity Technologies

    Joined:
    Jul 3, 2017
    Posts:
    2,983
    In the prototype, yes, for simplicity. In the real thing you'll have include-like directives for block libraries.
    A block acts more like a glorified struct with members. That said, we'll provide a robust way to call another block from HLSL. The "apply" function and the interface is the public API of a block, everything else is conceptually internal to it.
    IIRC we have some way to resolve this in the template or in the block shader. I'll double-check.
    Multiple copies of a block are fine. The blocks will have a way to specify, which properties are unique to a block instance and which are the same across all; the invocation in the block sequence will have a special attribute to specify whether to make a new instance of the properties or not.
    Imagine a block that implements Perlin noise - you may want to sample it several times in the shader, with different settings.
    Not on the interface. You can use it inside the HLSL in the block.
    Not at the moment. Maybe in the future :)
    Yes, we plan to have a set of interfaces that are shared between the pipelines. We'll support writing a shader that works across multiple pipelines, and allow to specify a separate implementation per pipeline where needed.
    Are you referring to the usage of separate samplers where you're using albedo's sampler but that can get stripped by the compiler?
    Technically, these should always be available even if the templates do not provide them. You can always simply include the file with the version directly in the block that needs it.

    Yes, but I'm not sure it will be too far away from what SG offers with
    $precision
    .
     
    recursive and JoNax97 like this.
  32. BOXOPHOBIC

    BOXOPHOBIC

    Joined:
    Jul 17, 2015
    Posts:
    506
    Long-awaited, thanks for making it happen, finally! Like many people said already, BIRP is a must-have if we want to support all render pipelines with one shader, especially for us, asset store developers!

    I just had a quick look at the blocks and while I don't write many shaders (I much prefer graphs), I have a few notes (some of which might be irrelevant maybe):

    1. I would separate the properties from the inputs, this would allow using the old properties as before
    Code (CSharp):
    1. // New semantics
    2. [Property(defaultValue = "(1, 1, 1)")][Color] in float3 AlbedoValue;
    3. // Could use the old semantics and a separate input
    4. AlbedoColor("AlbedoValue", Color) = (1,1,1,1)
    5. in float3 AlbedoColor;

    2. How do I add a custom display name?
    Code (CSharp):
    1. [Property(defaultValue = "(1, 1, 1)")][Color] in float3 AlbedoValue;

    3. The block should be able to parse defaultValue = black instead of "\"black\" {} :rolleyes:
    As far as I know, the {} are not even used anymore, why keep them
    [Property(defaultValue = "\"black\" {}")]


    4. Custom attributes work, but having spaces will break the shader:
    Code (CSharp):
    1. // Works
    2. [Header(MyTitle)][Property(defaultValue = "(1, 1, 1)")][Color] in float3 AlbedoValue;
    3. // Breaks: Error on line 25 (27, 32): Expected token type 'EndParenthesis'
    4. [Header(My Title)][Property(defaultValue = "(1, 1, 1)")][Color] in float3 AlbedoValue;

    5. Do you plan to implement something like this to dynamically show/hide properties without requiring a custom shader GUI?
    Code (CSharp):
    1. #if defined(_HIGH_QUALITY)
    2.     [Space(10)][Property(defaultValue = "(1, 1, 1)")][Color] in float3 AlbedoValue;
    3. #else
    4.     [Property(defaultValue = "(1, 0, 0)")][Color] in float3 AlbedoValue;
    5. #endif
     
    Edy likes this.
  33. aleksandrk

    aleksandrk

    Unity Technologies

    Joined:
    Jul 3, 2017
    Posts:
    2,983
    [Property(displayName = "MyAwesomeProp")]


    Yes, this is a limitation of the prototype :)
    We'll make it proper later.

    I would expect a string there, TBH:
    [Header("My Title")]
    . Do you need such arbitrary things inside attributes?

    Not at the moment. As stated somewhere above, we're not doing anything that ShaderLab doesn't allow. We may add this later to both systems.
     
  34. BOXOPHOBIC

    BOXOPHOBIC

    Joined:
    Jul 17, 2015
    Posts:
    506
    Thanks for the answers! Shaderlab uses a string without "": [Header(My Title)]. Some special characters are not supported as attribute parameters, even if the parameter is a string, here are some other examples:

    [StyledBanner(Motion Orientation Element)]_Banner("Banner", Float) = 0
    [StyledCategory(Render Settings)]_RenderCat("[ Render Cat ]", Float) = 0
     
    Last edited: Oct 24, 2022
    aleksandrk likes this.
  35. aleksandrk

    aleksandrk

    Unity Technologies

    Joined:
    Jul 3, 2017
    Posts:
    2,983
    @BOXOPHOBIC I think we will require a string and not necessarily match ShaderLab syntax 1:1.
     
    BOXOPHOBIC likes this.
  36. burningmime

    burningmime

    Joined:
    Jan 25, 2014
    Posts:
    845
    The problem of specifying per-pixel an albedo/normal/smoothness/metallic/emission and then letting the pipeline light it with some standard BRDF... IMO, that's a solved problem a dozen times over. For Unity, we have surface shaders, better shaders, shader graph, amplify... It can be trivially baked by a hundred different programs when creating assets (assuming it doesn't change per frame). The only thing porting the old surface shaders to URP/HDRP would do is allow some old assets from the asset store to work.

    But, be me, a solo dev (occasionally working w/ artists) in URP.
    • All my characters' hair looks flat. I can buy Lux from the asset store or write my own shader from scratch which will be incompatible next time Unity upgrades.
    • Oh no, one of my careless characters just spilled a bunch of gasoline all over the concrete. Now how do I make it look like an iridescent gas spill? Guess I'm writing my own shader from scratch.
    • How do I make glass look realistic? I have a thin window, a thick window, a beer mug, and a wine glass. Guess it's reading through undocumented URP sources and writing a custom shader from scratch.
    • I got a tree pack from the asset store, a bush from quixel megascans, and made some flowers in blender. Now how would I integrate them together and make them handle translucency? Luckily, Vegetation Engine solves that really nicely, but that's just passing the buck onto BOXOPHOBIC to write a ton of custom shaders and hope they're performant/pretty and work with your art style.
    • What if I wanted a toon or non-PBR lighting model
    • ....
    Now, I know that 5-10 years ago the idea of customizing lighting models without customizing the pipeline was a complete anathema because of deferred rendering. I get it; deferred was a big jump in quality and performance. And PBR was still a buzzword rather than the default. So there was a proliferation of tools that let you customize a surface and use a single lighting model for everything that looked reasonable. And when it didn't, for photorealisitic rendering, engines (Unreal, HDRP) just provided a set of alternatives that covered 90% of use cases and hoped it was enough. URP never provided this besides clear coat*.

    But now we're in an age where the low end (phones and VR) want to use pure forward rendering for MSAA, while AAA has already moved to tile-based forward. "Forward+" is being added to URP soon and is the default for HDRP. Deferred rendering is going the way of stencil shadows. Allowing users to customize the lighting model without writing the whole shader from scratch is a real possibility. And it's something that Block Shaders *might* be able to provide while surface shaders and shader graphs can't.

    * Arguably, this type of thing is more important for URP than HDRP since URP is commonly used for NPR/toon shaded games. HDRP can get away with providing separate shaders for skin, hair, eyes, foliage, and a few other specific things and using bog-standard PBR for the rest, since the assumption is users want realism. That's not really a valid assumption in URP. For NPR, you're basically stuck writing your own shaders, buying an asset ,or hacking it into Shader Graph in a way that's as difficult to maintain as option 1 except now you have a graph file to manage too).
     
    Last edited: Oct 26, 2022
  37. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    5,461
    Well the ideal end result of all this effort should be:

    - surface style shaders can be written which feed A lighting model
    - surface style shaders can be composed from shader components (blocks, stackables), like adding a snow system
    - lighting models can be written which take data from A surface shader and light it
    - shaders can be written as graphs or text
    - Anyone can update their aspect without breaking the other persons work
    - Upgrades to unity versions have little breakage, because of the above point.

    This should open up the possibility space for shaders on the asset store substantially compared to 5 years ago because you can just focus on the area you want to modify without having to, say, handle every surfacing case because you added a new lighting model. And it will be way better than it is now, where the support cost of writing a shader and maintaining it through SRP versions is beyond insane.

    Now if I can just stop traveling long enough to play with it ;)
     
  38. atomicjoe

    atomicjoe

    Joined:
    Apr 10, 2013
    Posts:
    1,869
    I'm doing just that with Surface Shaders right now, although, yes, it implies modifying the shader code to put an include in it and change the reference to the lighting function used, but that's how block shaders work anyway: you still need to modify the code to do that.
    So yes, Surface Shaders of course support that. That's the whole reason they were invented in the first place, to decouple lighting and surface functions and be able to customize or change both separately.

    Seriously, Block Shaders are reinventing the wheel while breaking compatibility with everything and still managing to do less than Surface Shaders already do. All because the guy who wrote the Surface Shaders solution in the first place isn't in Unity anymore and nobody there seems to be up to the task to upgrade his code.
    I could understand that on a small private company, but not on a big multinational like Unity.

    Unity, stop the excuses, just update the old systems instead of rewriting everything because it's easier for you while worse for the user.
     
    Last edited: Oct 27, 2022
  39. sacb0y

    sacb0y

    Joined:
    May 9, 2016
    Posts:
    867
    While this is all true (cause I do it too) I think the mentality is changing from "URP is NPR/Toon" to "URP is mobile and low end". I was surprised to find "I am Fish" was HDRP when I thought it was URP.

    I imagine the reason for this change in mentality is:
    1. Maybe URP getting the high end features people want access to makes it less mobile friendly and doesn't make sense to split the work on those features.(e.g. to versions of raytracing one on URP and one on HDRP)
    2. URP will probably focus more on performance than features since that's the biggest problem people have with it.
    3. Now "coexistence" will let people use HDRP and apparently switch the game to URP for mobile or low end targets.
    I imagine we'll see a little more NPR/Toon in HDRP from unity. It's already long been a thing, but it just never made sense cause of platform limitations of HDRP.

    If the coexistence effort is as useful as I hope most of these problems will be mitigated. And maybe things will feel less fragmented as platforms serve a technical purpose over a visual one. No longer forcing people to be stuck between a performance focused SRP that's multiplatform, and a features based SRP that's more limited.

    But I'm only guessing.
     
    OCASM likes this.
  40. dnach

    dnach

    Unity Technologies

    Joined:
    Mar 9, 2022
    Posts:
    89
    The problem is that this would not necessarily help with unifying the shader tools development and shader authoring workflow across pipelines, and between graph based and text based shaders.

    To that end, a more generic and lower level approach was needed in the form of the Shader Foundry API, as well as its higher level application that is Block Shaders. As @jbooth mentioned above, Shader Foundry will allow us to provide a more powerful, extensible and stable shader authoring workflow - across text and graph based shaders alike. Beyond authoring Surface Shaders, asset creators and tools developers could utilize the Shader Foundry API to create and extend templates in a text-based or programmatic fashion, and reliably support these across pipelines and versions.

    There are many additional exciting possibilities, one example being future support for compute shaders to unlock many interesting and generic use cases. This announcement and demo is focused on the Block Shaders workflow, but we will share more information on Shader Foundry soon!

    Regarding custom lighting, Block Templates could indeed provide "customization points" to override the lighting model used (like BiRP surface shaders), as well as expose additional customization points not currently supported by BiRP surface shaders.

    You can experiment with this in the demo we provided by adding a lighting customization point to one of the existing templates! One of the examples we included uses a "FinalColor" customization point to implement simple toon directional lighting, though the official templates may end up providing a specific CP for lighting override.
     
    Gooren, sacb0y, OCASM and 2 others like this.
  41. atomicjoe

    atomicjoe

    Joined:
    Apr 10, 2013
    Posts:
    1,869
    Then reimplement Surface Shaders using the Shader Foundry API instead of doing a whole new incompatible system like Block Shaders.
    Again, it shows you just don't want to support the systems that are already there and prefer redoing everything from the ground up just because it's easier than managing and updating other people's code.
    It's been like that for many years now and yet you fail to impress the users, which still prefer old systems because they are actually better, more flexible and they just work.
    I have been reading the excuse of the "growing pains" for so long now that it's not even funny anymore. Just stop this non-sense already.
     
  42. OCASM

    OCASM

    Joined:
    Jan 12, 2011
    Posts:
    328
    So, you're just going to ignore every valid reason they give for the existence of this system?
     
    Gooren likes this.
  43. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    5,461
    Honestly I think a new system is actually needed - surface shaders have a lot of funky behavior because they were kind of designed on the fly, and they were designed around Unity 3 or so, and the performance characteristics/concerns of the times don't match that of modern shading. What should have happened is that this should have been done before SRPs launched, and we should have all transitioned to a new system over the last 5 years instead of having to manually roll our own solutions, or at least we should have had a lot of help from Unity making that transition instead of having to reverse engineer it all while Unity pretended it wasn't an issue and provided no support. But that didn't happen.

    So now, with a solution finally being worked on, it feels like another round of being forced to do a bunch of transition work instead of something that is a positive improvement, which is unfortunate because if this goes well it will be a big help.

    That said, I do think it would be worth considering what would make this transition easier for people than the last ones. For instance, the property syntax changing is a larger change that I would have expected. I can likely write a sort of template for better shaders to output into this new system, but it will mean parsing and translating the property definitions as well, which will be much harder because my system currently doesn't understand anything about properties. Or it will require every user rewriting all their shaders to use the new syntax, which is annoying. Further, since the new system kind of merges CBuffer declarations and properties into one thing, it's even messier. So using this example, being able to declare the old style of properties/cbuffer entries would significantly make the transition easier for m to make automatic for my users and my shaders.

    Second, the cost/benefit on many of these changes are not clear right now, and the limitations aren't either. This also makes the 'ask' of rewriting everything less clear. From my perspective, is rewriting MicroSplat and Better Lit Shaders into block form even possible? Is it less work to continue to maintain low level SRP versions of everything? It requires a bit of a leap of faith to attempt such a thing, and is going to be a TON of work if I can't write a wrapper around the system to translate the existing code and am forced to really rewrite it all. (For instance, MicroSplat is still effectively a surface shader. My pipeline wrappers simply generate the needed structs and pass them to the internal functions like it was a surface shader, then map that back into the pipeline at the end).

    So I guess my question for everyone involved is essentially this: Assuming we buy into the idea that we need a new format, what parts of this system would make it difficult to port your current work to it. And from Unity's end, what are the gains around changing things like the property syntax, and which bits of the old system could be supported to make migrations easier - after all, there are many, many projects with shaders written in Better Shaders or ASE, and the ultimate goal is really to get them all writing to this format and not having to reverse engineer templates every few months, which lead to a bad user experience. And finally, what does Unity plan to do to help the community make this transition this time?
     
    Noisecrime, Gooren, Prodigga and 6 others like this.
  44. atomicjoe

    atomicjoe

    Joined:
    Apr 10, 2013
    Posts:
    1,869
    You put a lot of faith in a company that has repeatedly and consistently ignored all our concerns and issues with the SRPs for years.
    They clearly DON'T WANT to support current code base. What will happen when block shaders will be abandoned at half bake in two years?

    Imagine what would happen if Microsoft decided to completely change how you program in C++ and did so unilaterally without asking anyone and tried to force everyone into adopting it.
    I bet the Unity Team wouldn't be happy AT ALL that Microsoft decided to screw them and all their code base because it's now considered "legacy code".
    That's the same here.

    You can't construct a building on shaky foundations, and yet the Unity Team keeps changing everything under our feet because it's more convenient for them to maintain or to sell.

    I see a lot of ass kissing in general over here, while we should be furious for how things have been handled and how they keep the same S***ty mindset for new developments, simply ignoring everything that has come before to build yet another Unity half-baked disappointment instead of updating the systems that already work.

    We asked for Surface Shader support for SRPs since DAY ONE. They ignored us FOR YEARS.
    Then it reached a point of SRP non-adoption rate that they felt forced to throw us a bone, so they said "we've heard you and we're porting Surface Shaders to SRPs! Rejoice, peasants!" and now it turns out that it will be yet another incompatible layer of systems over the already incompatible layers we have.
    But I'm being SO NEGATIVE about it all!
    I should rejoice that they even did SOMETHING in that direction! Yay!

    I get that the majority of people around here are either asset store devs just wanting an unified solution to shader programing across render pipelines to make maintenance easier or just average consumers of asset store products that don't really understand what is the difference between a Surface Shader and a Block Shader but think the later must be better because it's New and Shiny™.

    I have a feeling that Unity has become less and less a Game Engine and more a consumer oriented front-end aimed at selling assets in the asset store as if it were micro-transactions.
    And I guess that if you fall in one of these two cases, then everything is fine for you and Block Shaders.
    But it's NOT if you pretend to do something that lasts with Unity.

    Maybe the problem is me for beating a dead horse and expect it to resurrect somehow, but I got that email telling me that the feature I was so much interested in was finally being implemented and I came here like a sucker only to find the Unity Team doubling down on the same path of incompatibility and messiness that got us here in the first place.

    I guess I should have learned my lesson by now.
     
    Last edited: Oct 27, 2022
  45. sacb0y

    sacb0y

    Joined:
    May 9, 2016
    Posts:
    867
    Yeah, imagine if they F***ed up again. We'll find out 2023. But for now, at the very least, it seems there's a decent change of course in progress. And if this course correction goes well, it might finally remedy those problems people have been complaining about. To me it seems like all this stuff like block shaders is a direct result of the coexistence effort, so I imagine Unity deems it important rather than just appeasement.

    You're not wrong for being mad but at least give proper feedback, I can't tell what your exact problem is. It reads like "Unity does new thing when they could just update old thing" but you don't explain what that means for you as a developer.

    For asset store devs it means their assets can be more easily maintained
    For solo devs it means assets they buy are better (and cross SRP)
    For people who make shaders it seems like in a "coexistence" future this will simplify the workflow.

    What hurdle this is creating for you besides general unity issues?
     
  46. atomicjoe

    atomicjoe

    Joined:
    Apr 10, 2013
    Posts:
    1,869
    - no retrocompatibility with older Unity versions (which many projects are stuck in because of breaking changes across versions)
    - no retrocompatibility with more than a decade of Unity shaders, including my own shaders of course
    - no BiRP support
    - worse and higher level than Surface Shaders (no tessellation, lack of low level control)
    - yet another system to take care of, learn and debug
    - deflection of responsibilities: by making this half-assed implementation, they don't have to support Surface Shaders (which are better) and will blame any lack of features on "growing pains" while parity with Surface Shaders will never be really reached.

    The only thing Block Shaders was meant to fix was the lack of a text based approach to write shaders for URP and HDRP, and since upgrading Surface Shaders was a pain in the ass, they just made a new thing and broke compatibility with everything because who cares.

    My point is that Block Shaders is a LOW EFFORT solution to a problem that has been stated over and over since the very introduction of the SRPs.
    That the PROPER solution would be to update the Surface Shader system to make it compatible with the SRPs and fix any inconsistency or funkyness it could have as part of a proper support program for an already established system.

    That Unity should build on what's already there instead of destroying what others did before them because "other people's code stinks, mine is better".
    That they should create an UNITY in the engine development and forget fads to grow what's already there to be better.

    I don't buy the idea that old systems can't be upgraded because their foundation is outdated.
    That's bullshit.
    There is a new trending fad in systems development every 5 or 6 years, and seminars are full of people wanting you to believe the newest system is much better than the old one just because that's the system they use.

    Forward Plus rendering has been here for AGES. AMD did a tech demo like 10 years ago about it, but everybody was too busy with the latest fad to notice: Deferred Rendering.
    Now people are talking about Forward Plus as if it was the bee's knees.

    Same for DOTS: let's break everything and make an entire new way of programming because we have seen 2 TED talks about that and it's exciting and we like to do New and Shiny Things!
    I would much rather prefer to have multithreading support in the Unity API, thank you very much!
    Then I'll organize things in a data oriented way if I want to, thanks.
    But NO, Unity, a multi-million dollar publicly traded corporation can't make it's API multi-thread safe, because that would be a lot of work. Let's reinvent the wheel instead and spend years in development hell and F*** everything that has been coded before!
    PERFECTION.

    I'm tired of FADS, just support the things that ALREADY WORK, expand them and make them better:

    - upgrade the Built-in Render Pipeline Forward Renderer to Forward Plus and implement the URP batcher in it.
    - upgrade the Surface Shaders System to be compatible with HDRP and keep it up to date as the back bone of all shaders
    - deprecate URP and BiRP Deferred and make BiRP Forward Plus the standard renderer instead.
    - deprecate DOTS and make the current Unity API Multithread safe
    DONE
    EVERYBODY HAPPY
    LEGACY RESTORED, Easy of use restored, user fragmentation fixed, performance through the roof!

    I can't believe the lack of vision the Unity Team has proved to have since they had the brilliant idea of breaking everything with DOTS and the SRPs.
     
    Last edited: Oct 28, 2022
    TieSKey and funkyCoty like this.
  47. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    5,461
    If you think I'm "ass kissing" Unity developers, you clearly are new to this conversation, and haven't even read this whole thread, let alone the hundreds of others. I was the first voice to call them out on this issue, before they even implemented SRPs. I've been screwed over by this issue more than perhaps anyone, and have done more to get it fixed as well. So if you think you have some righteous position over the people in this thread, get in the back of the line.
     
  48. aleksandrk

    aleksandrk

    Unity Technologies

    Joined:
    Jul 3, 2017
    Posts:
    2,983
    Since properties are backed by uniforms and uniforms can be declared in different CBuffers, it makes sense to specify all of this in one place. "Here's a field in the block interface; I want it to be backed by a uniform called foo that is part of CBuffer X; expose to UI as a range with name Foo."
    At the same time, the property syntax in ShaderLab has certain oddities and limitations. We want to provide a more robust system, and it's a good opportunity to do so.
    I also think it will be quite tricky if we want to fully support the old syntax in the new system. At the very least, the uniform declarations have to be directly on the fields.
    That said, I understand your concern about having to rewrite too many things. We'll definitely think about supporting the transition better. Keep in mind that we're at the stage of a public prototype, so it's a bit early for such things.

    You could write a single block that wraps calls into your existing HLSL code (and the code can be in a traditional include file). It can consume the inputs from the pipelines if you need them (and we'll make sure they have a clearly defined space!) and write to a subset of outputs.

    @atomicjoe There are cases when making a new system is justified. This is exactly one of those. The new system will be more flexible than Surface shaders, on both engine side and user side and will support more things as well.
     
    Gooren and SonicBloomEric like this.
  49. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    5,461
    Yes, this is how I basically wrap microsplat for SRPs. What I'd like to do is write some kind of block conversion for stackables in better shaders, which will mean translating the existing property/cbuffer data into the new format, then calling into the various functions that exist in my syntax with the data from yours.
     
    aleksandrk likes this.
  50. atomicjoe

    atomicjoe

    Joined:
    Apr 10, 2013
    Posts:
    1,869
    Dude, I was answering to OCASM with this, not you. Reread my post.
    And if you think I'm new to this conversation, you clearly haven't been paying atention.