Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Official What is next for us at Unity with Scriptable Render Pipelines

Discussion in 'General Graphics' started by natashat, Jul 2, 2020.

Thread Status:
Not open for further replies.
  1. colin299

    colin299

    Joined:
    Sep 2, 2013
    Posts:
    181
    Sorry I absolutely can't agree with this.
    To me: "Theres just no reason to create shaders by shadergraph for 95% of all things."
    I like shadergraph when the job is about vfx/prototype, but for anything related to custom lighting/computebuffer/multi_compile/putting work in vertex shader/version control friendly......, I need text shader to do the job, and only text shader can do the job.
     
  2. fherbst

    fherbst

    Joined:
    Jun 24, 2012
    Posts:
    802
    I think a major pain point was/is that these aren't very extensive when combining features (e.g. URP + Android + ShaderGraph + VR) to the point of not happening at all.

    I offered it in the past, happy to send you 1-2 of our heavy Quest projects for testing.

    Also to the previous point, please use the semi-official Boat Attack demo and other Unity-released demos as examples as well. Boat Attack still doesn't work on Quest at all last time I checked.

    Regarding not distributing as packages anymore, I think this is a step backwards and doesn't take into account how previous versions went. Imagine how broken everything would be if people would be forced to use 7.0.0 on 2019.3.3, for example. Upgrading to a newer Unity dot release isn't always possible, and "It will just work out of the box" hasn't been true for any URP version so far. This might change at the end of all you outlined, but it tells a lot that you had to not only publish new URP versions regularily, but unpublish the previous ones since they were so broken nobody should use them.

    This will require you to actually listen to bug reports on upgradeability. Everytime I mentioned how easy it is to break upgrading to URP QA said "this is by design". Example? Case 1199280, super easy to repro - create a cube, upgrade to URP or HDRP. Result is a pink cube. 90% of AssetStore packages are affected by this. Closed as "by design".
    I see this very problem is explicitly mentioned, but it feels very sour that you haven't listened so far even when people shout at you (same as @jbooth mentioned).

    All official blogposts and marketing pretend HDRP is production-ready. Will there be an official stance on this outside of the Graphics team, or will you continue to say it is production ready?

    Regarding the choices about Shader Graph and abstractions, I think a pain point for many people is that e.g. Amplify allows a lot of control while Unity does not. Unity tries to hold hands with its users (Shader Graph scopes are still crazy broken). Amplify feels like it gives power. At least that's the feeling.
    To me, this answer by @Coroknight sums it up best: "seems like their strategy to achieve this is to limit customizability ...Just let us mess things up if we want".
     
    Last edited: Jul 6, 2020
  3. a436t4ataf

    a436t4ataf

    Joined:
    May 19, 2013
    Posts:
    1,924
    This is about the AssetStore (and any bigger teams that run their own tools team/department that they need to maintain across multiple projects): without the version locking, you can say goodbye to the assetstore (for graphics). Maintenance and support of code that itself uses / builds on top of SRPs is a much bigger problem than the stuff you're (rightly) describing - and locking editor versions to SRPs is (currently) the only viable way to fix that problem.

    So - if I understand correctly - it's not about what's perfect (*) here, but about dealing with a blocking problem that has no other workaround, and is blocking asset-authors from operating at all.

    On the plus side: the "doesnt work out of the box" will change very rapidly with this new approach (they'll have no choice!).

    (* - "perfect" might be for a company-wide overhaul of the concept of "Asset" and "dependency" and "unitypackage" and "packagemanager package" and "version control" and ... rewriting the whole of the Editor to deal with the changes ... and ... I think the problem is clear :). Unity has never had a real, production-quality, system for import/export/management of chunks of assets + code, and fixing that hole is going to take YEARS more (the packagemanager is a good start, but it's got a lot of catching-up to do). We can't wait that long, we we need something viable today)
     
    GoGoGadget likes this.
  4. RoughSpaghetti3211

    RoughSpaghetti3211

    Joined:
    Aug 11, 2015
    Posts:
    1,697
    I think I’m the only other person that feel that removing SRP as a package is going backwards. I wish there was some middle ground where packages were locked to a LTS editor version and asset store asset only support LTS.. idk I feel nervous but I will trust others with more experience in this problem space.
     
  5. DreamingImLatios

    DreamingImLatios

    Joined:
    Jun 3, 2017
    Posts:
    4,200
    Well you managed to bring me out of the shadows again! Thanks for the awesome update!

    I'm very excited to see a proper shader API on the roadmap. I understand why you decided to focus on artist tooling first. I just wish you were more open about this from the beginning that you planned to do this after you got the artist tooling side of things where you wanted it to be. But better late than never! I still have no idea if HDRP has view-biased normals (stretched reflections and hallway light reflections). Hopefully once this API comes out I will be able to figure that out.

    At some point I will need a renderer that mixes realistic rendering (HDRP-style) with NPR deferred lighting. I have no idea yet if this will involve writing a custom SRP or extending HDRP. I would appreciate guidance and insight into this at some point.

    In HLSL, every function in a class that implements an interface (either directly or indirectly) is virtual. I can't remember if this also applies to functions of a class which don't come from an interface. If not, it is not a big deal as a class can implement more than one interface anyways. You can make a class abstract by simply not implementing a function of an interface it implements. Polymorphism is resolved at compile time along with aggressive inlining optimizations. You can do runtime polymorphism using dynamic linking, but that is exclusively shader model 5 and isn't that useful in practice, whereas classes and interfaces work with most shader models and are useful for setting up contracts with complex algorithms, like a light loop.
     
    SenseEater, FROS7 and phobos2077 like this.
  6. Korindian

    Korindian

    Joined:
    Jun 25, 2013
    Posts:
    584
    I think it's pretty clear from the feedback on this thread so far that both text-written and visual shader graphs are equally important to different people. I've used both and they're both equally important to me. Unity, please make them both awesome and make them work well together, ok? Thanks :)

    In addition to what others have written, here's what I would like to see for shaders, after having spent many weeks converting a lot of my shaders written in the built-in pipeline to the Shader Graph for HDRP:

    Text/code shaders:
    • Extensive documentation and examples. Shader coding is already a little more difficult then regular coding, and having stuff all over the place and not explained clearly takes the difficulty to a whole new level. Had there been a surface shader-like abstraction for HDRP from the beginning or at least any documentation at all, all those weeks of porting shaders could have just taken days at the most as I could have just left it as code. I'm hoping Unity will deliver well on their documentation promise.

    Shader Graph:
    • As many options as possible from coding shaders to be available as options in Shader Graph. Many shader options are hidden in Shader Graph that are exposed in Amplify Shader Editor for example.

    • I agree that Unity's included Lit/Stacked/Unlit shaders should be done in Shader Graph as well (just like @richardkettlewell did for the particle shaders), to show not only that it can be done, but that Unity is also confident in the performance of Shader Graph's generated code. If certain custom features are not possible in Shader Graph yet, then focus should be on getting those features implemented in Shader Graph as soon as possible.

    • Have Shader Graph output more organized and readable code, like Amplify Shader Editor does. I feel this would also help a lot of artists to become more comfortable with coding shaders. As it is now, whenever I generate shader code from the Shader Graph, I'm hit with a big "what the heck is all this" looking at all that repeated code.

    While I appreciate all the hard work done by Unity devs on the SRPs, and Unity in general has allowed me to pursue a career I probably wouldn't have been able to otherwise, I've got to say that recently it's been a pretty sour experience from a user point of view for many reasons, some of which Unity addressed in the OP. I've invested a lot of time and money into Unity's ecosystem for quite a while, but in the past year, I've been so frustrated with Unity's decisions in general that I came very close to deciding that the time spent learning a different game engine and porting my project over would be worth it rather than continue with Unity in the current confusion it's in.

    I have a son who's very interested in game development, but I can't recommend him to learn Unity right now even with all the knowledge I can pass on to him, because it currently feels like a mess. I don't know if what I'm teaching him is going to be deprecated soon, or is going to change a lot in the near future, so I've decided he'd be better off learning a different engine for now.

    Everything you (Unity) wrote in the original post sounds great, but since your timelines for some of these things are so far in the future, I honestly don't know if I can trust that Unity will see all of it through. I really hope you will.

    Edit: Oh one more piece of feedback, I also agree that if Unity could try developing an entire game/finished project themselves (and not just samples/showcases), and try to keep it updated through various versions, you'd really be able to see and feel what your users go through, rather than just relying on feedback. I think Unity would prioritize things a lot differently if you did this.

    I have a lot of doctors in my family, and they say that once they become patients themselves and have to go through the system, it gives them a whole different perspective and understanding than just listening to patients' feedback.
     
    Last edited: Jul 3, 2020
  7. Tanner555

    Tanner555

    Joined:
    May 2, 2018
    Posts:
    78
    If integrating all the SRP packages into Core Unity is Successful, I'd also like to see many other essential packages be integrated into Unity once they are ready for production. The Burst Compiler has been ready for production for ages, as well as ProBuilder, ProGrids (is this abandoned?), all the 2D packages, Timeline/Cinemachine, and the IDE integrations. Once Entities, the Job System, all the UI Toolkit packages, and Visual Scripting become production ready, they should be integrated into Unity as well.
     
    soft_sound, OCASM and Korindian like this.
  8. RoughSpaghetti3211

    RoughSpaghetti3211

    Joined:
    Aug 11, 2015
    Posts:
    1,697
    Doesn’t this lead back us back to the original problem where bug fixes could only be pushed out with the editor ?

    I honesty feel package manager dropped the ball here.
     
    Last edited: Jul 3, 2020
    cultureulterior and a436t4ataf like this.
  9. fherbst

    fherbst

    Joined:
    Jun 24, 2012
    Posts:
    802
    @a436t4ataf I am running a team with tools department :) Moving graphics back to core will reduce the need of support on Unity even more, as now even upgrading to an official URP release (in contrast to forking it directly) will not be supported by Unity if I read that correctly, unless you're willing to upgrade the entire editor. It removes ALL advantages of the package idea. This is just reducing the support surface if you ask me.

    I believe this is just wishful thinking. The SRP teams were always very confident that this time they got it right and then the avalanche of bugs hit hard. Since at least two years I and a lot of other people on here are fighting against that. It might work if all the nice talk about upgradability and regression testing is turned into practice, which it only can if (and only if) a stable state is reached first - otherwise you're just verifying that something is still broken (the Graphics repo has multiple test cases that have pink results and stayed that way for years).
    Try BoatAttack on Quest if you dare.
     
    rz_0lento likes this.
  10. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    5,457
    I believe this is an incorrect reading- you will still be able to download a copy of whatever version you want from the Git depot and install it into your project. The only change here is to make it not available via packages, so the average user doesn't get confused about which version they should be using..
     
    Martin_H likes this.
  11. Noisecrime

    Noisecrime

    Joined:
    Apr 7, 2010
    Posts:
    2,035
    I don't know if I like this. I really like the concept of the package manager and would much rather use it to access these packages than going to github. So why not make the package manager more useful and build on what its meant to be doing in the first place?

    Let's add a 'LTS only' ( or major versions ) setting, on by default for each new project ( yeah would love to be able to have defaults become a setting devs can create for themselves, but lets face it that ain't happening soon if at all ) that will enforce downloads of LTS only versions of packages ( where they exist for a feature ). It will also only show LTS versions in the package manager.

    There you go! By default URP/HDRP packages are locked to LTS versions, a user cannot accidentally get a preview version. That ensures unification between the editor and asset store packs, but as an 'advanced' user can disable the LTS setting, they are not locked to using those versions and instead of going off to github they can use the package manager as its meant to be used and grab a different version of the package should they need to.


    EDIT - The Package Manager

    BTW the package manager itself needs some love. I just got pinged on an old thread about lack of being able to decide where the package cache is stored and out of interest checked my cache size - 7GB! When we were already talking about the issue of not being able to relocate the cache people were at 1-2GB. The size wouldn't be too bad if I could relocate the folder ( which you can be have to do so at the OS level ) but honestly it shouldn't really get to this stage.

    The problem is that switching between multiple versions of the editor, creating multiple projects a month and downloading experimental features/demos means the cache quickly fills up with massive numbers of preview versions of packages and there is no good way to clear them out.

    I do propose some ideas, though I realize a fundamental issue is that if i re-open an old test project using say an old version of burst, Unity will go off and re-download th npm package, then decompress it before I can switch to an updated version. I'm not sure what can be done here. Perhaps an option to upgrade packages on opening a project.

    As I mention in that thread, I feel the package manager needs some options for clearing the cache as its only going to grow more over time.
     
    Last edited: Jul 4, 2020
  12. harry_js

    harry_js

    Joined:
    Jan 22, 2020
    Posts:
    139
    Really interesting thread. Somewhat comforting to see others on here that have encountered similar issues and/or have similar worries too!

    A lot of my current concerns have already been brought up in previous posts but I'll dump a few random other thoughts below. Outside of those, the Helping the Community to Be Successful with SRP section probably contains my biggest immediate wants - docs, docs, docs, best practices, yes please.


    For context - we have an increasingly modified HDRP fork, adding
    • an additional, bespoke stylised shading model
    • full support for dynamic (+indirect) instancing and DrawProcedural()
    • ShaderGraph code injection
    • a growing pile of ShaderGraph custom nodes
    .. which means we're lucky in some respects because we've just hacked straight through some of the walls we hit. If this wasn't the case, I would def be more vocal about the lack of SG extensibility API and/or a path to lower-level shader customisation.


    First some good stuff
    • This is a really nicely written rendering framework. Although boy there is a lot of it, I've found it surprisingly intuitive to modify and extend (and learned a bunch of interesting techniques in the process).

      Hot-reloading a renderer live in Editor is a w e s o m e.
    • The built-in visual debugger for materials, tile load, GBuffer values etc is GREAT and the fact I can have my custom GBuffer + extensions also integrated into it with almost no effort, perfect. Def appreciate that you've taken the time to build this kind of deep debugging in
    • Deploying via UPM is very useful for us. I can merge with upstream/platform HDRP versions and ship our fork via custom internally versioned packages and it all just works (although we've had to blow semver rules out of the water to do so, but that's not really your fault)
    • There are some useful architectural choices when it comes to managing quality and performance across platforms (ie. the Quality-config-via-HDRP-asset system) but I have some confusion as to where this is going, explained in a moment

    Platform Gripes

    > We already run extensive automated testing in our build system for supported platforms

    This is reassuring, certainly.

    However, there are certain platforms that are either outside of this system or the tests aren't comprehensive enough, because I've run into very obviously broken or missing things (or some severe performance issues) with even basic test scenes.

    > bleeding edge graphics functionality for GPU-compute-capable, high-end platforms

    Hopefully there can still be some optimisation investment in certain platforms that might not be considered "high-end" but that are on the support table in your SRP guidance page.


    Configuration Complexity Management

    I'm ready to learn perhaps there is already a way to manage this that I'm ignorant of, but:

    The centralised quality/perf management via HDRP asset is a great move towards managing the complexity of organising a scalable project that has to ship on a broad range of platforms.

    However - it feels like there is 'mixed enforcement' of this pattern across the HDRP components.

    Prime example (and I realise I'm ragging on a pre-release thing here) - the new TAA in HDRP 9 has all the quality setup baked into the camera. Is this intentional? IMHO a camera component should simply 'subscribe' to TAA, or at least a modular "TAA Config" whose finer quality config details are set centrally and therefore can seamlessly vary by platform / graphics tier.

    Other examples off the top of my head - (raytraced) shadow settings on lights. On one hand you have the asset-data-driven shadow map resolution (yay) and then in the next moment we're setting ray samples directly in the light (boo)?

    ... Performance tuning options in post-processing filters (eg. filtering and resolution in DOF).
    ... Basically anything that substantially changes the performance profile vs the core aesthetic result.

    The reason this stuff irks me is because I feel we were circling around a powerful mechanism for managing the explosion of "tuneables" but it already feels inconsistent. ScriptableObject all the things! Right?


    If one does not exist, I'd love to see a Unity-maintained HDRP project that demonstrates the workflow for
    • A scaled quality setup shifting from a - say - 8-year old PC through to a current-gen DXR-capable rig
    • Configurable macro-level settings (eg. scaling "Shadowing" from High to Medium might involve triggering all but the most crucial point-shadow-casters have their shadowing disabled .. or scaling Raytracing shadows' sampling levels .. or shunting all lights' from being raytraced except for the key directional)
    • How to establish and maintain order across multiple scenes WRT lights, post-processing, etc.
    (and to be clear, this can be a bunch of boxes on other boxes across a pile of scenes, I just want to see the workflow demonstrated)

    How simple can you make that authoring, enforcement and test process? I know none of this is sexy cutting-edge graphics work but it's the plumbing that can help actually final a title without killing the TAs in the process


    We've already hit some hurdles here and unless I'm missing something fundamental, we're going to have to write a bunch of code - and likely start adding more edits to the HDRP source - to effectively work around keeping a lid on this stuff at scale.


    Vague Batchery

    I wish the SRPB clustered objects into single dispatch instancing calls (like UE does, for example) as it's still producing a fair amount of work in scenes with repeated geometry - I presume the reason is "DOTS will fix all this / obsolete that work eventually". If that is the reason (even if you can't say it is), please pour one out for those of us that have to build a game right now rather than in 2 years

    We also see some curious sorting / ordering behaviour in the batches via renderdoc - I take it there isn't any further tech docs beyond the page in the manual?

    .. also it's weird that I have to download a C# script off a blog post to be able to profile what the batcher is doing. Last I checked that's still the case in 2019.4.1. Shouldn't this migrate into the [stats] window et al properly?


    Delicious Dogfood

    I'll +1 Jason's comment

    > Your teams should be taking demo's like the URP boat demo, porting it to both SRPs, and keeping it as a live working example through changes. Your team needs to feel the same pain we do.


    Plus - find the teams trying to build and ship games using SRPs and get on a Zoom call. I can only speak for ourselves but we'll happily screen-share our pain points if it will help*

    ( * .. get all our complaints fixed :p )
     
  13. fherbst

    fherbst

    Joined:
    Jun 24, 2012
    Posts:
    802
    @jbooth @Tim-C I think there is some clarification needed on the "graphics moves back to core" idea, around the user flows for that.

    Let's just suppose this same principle had been applied during the SRP 7.x cycle (I'm making some numbers up here).
    2019.1 comes out, with 7.0. People start using it, but quickly there's bugs detected.
    It takes a while to get these fixed, the first ones land in 2019.1.10 which releases with 7.0.3. Not all bugs get fixed here. 2019.2 releases, with 7.1.0. Since 2019.2 is out, there's no updates to 2019.1 anymore, essentially forcing everyone to update 2019.2. Sounds familiar? Yeah, it's the same as if a bug had been found in Built-in. That's what I mean with losing the advantage of packages.

    Additionally, this will lead developers (e.g. AssetStore publishers, Unity themselves) to assume people always use a specific version of Unity + specific version of SRP. Nobody will make proper URP version defines anymore, just use #if UNITY_2020_X.
    This means that people who have to/need to work with a custom fork will have an even harder time, because now making sure a specific version of Unity now works with it is on the user! Same for all AssetStore content! There will even be cases where the Core now makes incorrect assumptions about which URP is used, because nobody feels inclined to test newer URP releases with existing Unity versions, "that's not supported anyways".

    So in my opinion this will just result in more work on top of people who need to use a custom fork, making customization even harder and more unlikely to happen. Plus, it enforces much more regular Unity updates, which come with costly library reimports and all kinds of nice regressions that nobody wants to care about when all they need is get a shader bug fixed in VR rendering for Quest. (This exact thing was what packages allowed in the first place).

    I understand the idea of making it easier for AssetStore Publishers, but I don't think breaking packages is the right way to do this. I'd like a middle ground more; use SemVer properly, don't break API in a minor version, make sure there's a consistent set of shader and code defines from the SRPs that people can build against. Otherwise I can already see this decision/solution sounding nice at first and then causing major trouble.
     
  14. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    5,457

    I guess i just don't see what the big deal is. Right now, if you install 2019.3, you will get a version of the SRP depending on what patch version of unity you are installing, automatically, if you create a new project and select a render pipeline. If you want a different version, you can go to the package manager and download it. In the future, you will go to GitHub and download the version you want. I don't see why changing where you download it from is such a big deal.
     
    GoGoGadget and Tanner555 like this.
  15. fherbst

    fherbst

    Joined:
    Jun 24, 2012
    Posts:
    802
    Wow, yeah, that's part of the issue: this "user flow" goes way beyond the reality of working with any team in Unity where not everyone is a graphics dev or even a developer. There's a HUGE gap between "people who are fine updating to a newer URP or HDRP version in package manager" and "people who will maintain their own forks of the Graphics repo".

    That would mean dealing with submodules, patching stuff manually, and, as I hope I outlined, being completely on your own in unsupported territory, just because you want to use the advantages of packages. Nobody will test this workflow of using e.g. URP 7.3.2 on Unity 2019.3.12 (if the latter released with 7.3.1), neither asset store developers nor Unity themselves.

    We have > 10 projects going on simultaneously, we would probably have to hire a new engineer if suddenly we have to decide between maintaining a custom SRP on all 10 of them or upgrading all these projects to a new Unity version every week (currently we maintain custom SRPs on 2 of them). As said, upgrading to a newer version of Unity is not always possible.
     
  16. valarnur

    valarnur

    Joined:
    Apr 7, 2019
    Posts:
    436
    I would like that URP/HDRP replace built-in pipeline fully.

    Also, some existing workflows should be replaced. For example, ProGrid snapping is superior to Unity default snapping.

    I have installed 5.7GB of Unity engine and on top of that I have 4GB of URP/HDRP with ProGrid and Polybrush.
    There is no need to have this additional 4GB of packages when it can be default and better.

    For 2021+ would be appericiated one unified and modern workflow.
     
    Tanner555, FROS7 and tspk91 like this.
  17. AcidArrow

    AcidArrow

    Joined:
    May 20, 2010
    Posts:
    11,594
    Because...?
     
    Ruslank100 likes this.
  18. valarnur

    valarnur

    Joined:
    Apr 7, 2019
    Posts:
    436
    Two render pipelines are more than enough.
     
  19. AcidArrow

    AcidArrow

    Joined:
    May 20, 2010
    Posts:
    11,594
    If neither is able to replace the existing one in terms of functionality then why should it be replaced? New features should replace older ones when they are a superset of features and performance of the old ones, not because of design purity or because "Two render pipelines are more than enough". Because the answer to that is "One is enough actually, we don't need 2".
     
    SMHall, Ruslank100, JBR-games and 5 others like this.
  20. tspk91

    tspk91

    Joined:
    Nov 19, 2014
    Posts:
    131
    Shadow cascades are still hardcoded in SRPs (most of the shadowmap framework really). In some cases you want specific control over what renderers are drawn in each cascade (LODs, layers, impostors), more cascades (f.example Elite: Dangerous uses six or seven cascades due to the huge distance ranges), and rendering custom cascades. All of this is still opaque to the developer in Unity.
     
    Ruslank100 and OCASM like this.
  21. Neto_Kokku

    Neto_Kokku

    Joined:
    Feb 15, 2018
    Posts:
    1,751
    How about officially supporting HDRP on Switch? Yes, it's slower than the other consoles, but the GPU supports everything HDRP requires plus some, not unlike modern low end PC GPUs.

    Developers should have the choice to at least try to target it using lower settings, instead of either maintaining a parallel URP version of their project or giving up HDRP entirely.

    There's also a bunch Nvidia Maxwell features that are not used at all by any rendering pipeline, leaving performance on the table (like being able to render all shadow cascades in a single pass). These features aren't even exposed by the engine, requiring native rendering plug-ins to be used.

    Just to make it more salty: there are UE4 Switch games that could not be realized 1:1 with SRPs out of the box, and some of them even run at 60fps.
     
    Last edited: Jul 4, 2020
  22. iddqd

    iddqd

    Joined:
    Apr 14, 2012
    Posts:
    501
    But that's perhaps the reason why it's good to bind a big URP release to a Unity release. Because at least that one will be tested extensively by Unity and it will put more pressure on them to fix critical bugs and get that SRP release in a good state for that Unity release. If newer package versions are not 100% compatible with various Unity versions, it kind of defeats the purpose of this modularity and as you say, nobody tests this and it leads to this:
    "This is an HDRP package. Compatible with 2018.4. Due to changes in HDRP, this package currently is not supported in 2019.x."
    from https://assetstore.unity.com/packages/templates/packs/snaps-art-hd-sci-fi-military-base-158105

    Ideally I would prefer to not even have to know what a render pipeline is, I just want to install Unity and develop games.
     
    protopop, OCASM, GoGoGadget and 2 others like this.
  23. Jasper-Flick

    Jasper-Flick

    Joined:
    Jan 17, 2011
    Posts:
    957
    Do not forget third-party SRPs. Whatever you add to make URP and HDRP better, make sure that it works for any SRP, not just your own. For example, make shader graph work with any SRP via a public and documented API. If you make surface shaders 2.0, do it via a public API and don't hard-code it for URP/HDRP. Work as if you're going to introduce a third RP later, made by a completely new team.
     
  24. Neto_Kokku

    Neto_Kokku

    Joined:
    Feb 15, 2018
    Posts:
    1,751
    This x1000. The original intent of SRP goes down the drain if all tooling is hard-coded to work only with Unity's own SRPs.
     
    OCASM, Tanner555 and JoNax97 like this.
  25. fherbst

    fherbst

    Joined:
    Jun 24, 2012
    Posts:
    802
    @iddqd the issue with Unity's own AssetStore content would still be same under the new release strategy. This is just unwillingness (or lack of time, or whatever) to commit to supporting their own assets, something no other AssetStore publisher is allowed to do to this degree of broken-ness.
     
    SMHall likes this.
  26. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,586
    I still don't understand how URP updates go along with Unity LTS.

    I have an Unity 2019.4 project where I would like to use URP eventually. However, the project uses deferred rendering, which isn't supported in URP at the time of writing, this feature is currently in progress.

    My question is:
    When you completed deferred rendering in URP, will it make it into 2019.4 LTS?

    The LTS definition says "only cover usability fixes aimed at improving the stability". So per definition, a new feature wouldn't be added to LTS, does this apply to URP too?
     
  27. DeathRace26

    DeathRace26

    Joined:
    May 26, 2019
    Posts:
    75
    If you didnt make 3 (or whatever that count of render pipelines is) and do one main instead like with unreal, you wouldnt have to fix every update with ductaped code, where every RP release has completely different naming of variables for all unity versions. If someone wants to make low poly looking art games, they should be able to degrade rendering capabilities themselves in editor. By this, you literally wasted your own workload. This is more of a organisation failure and that is why were here complaining all the time....
     
    Ruslank100 likes this.
  28. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    5,457
    downloading a folder and dropping it into your project is no harder than installing anything else. They don’t have to fork it, merge, use sub modules, etc. and if they can’t handle copying a folder they likely can’t handle actually developing a game anyway. You have to know how to copy files to ship a game. Your turning this into some massive thing it isn’t. If they can’t handle cttrl-c, ctrl-v, then they shouldn’t be using the bleeding edge unverified SRP anyway.
     
  29. a436t4ataf

    a436t4ataf

    Joined:
    May 19, 2013
    Posts:
    1,924
    Yes. Because the support-surface is currently making AssetStore development impossible. The support cost is insane right now and grows at an utterly non-supportable rate (go read some of jbooth's posts to get a feel for the numbers - it's combinatorially expensive). The idea of doing SRP's via packages was a pretty dream but never viable: if they'd wanted to do SRP's as packages, they needed either a fully-mature packaging system (which has never existed in Unity and is years away), or they needed to plan everything out in advance, and make heavy use of stable, locked-in-stone, APIs, etc, with full forwards/backwards compatibility (which is totally non-existent).

    Instead, they did extreme agile development, which is great (I've been in dev teams doing this for years), but not supportable in Unity today: Unity has no concept of file versioning, and the package manager only has very limited concepts of package/dependency management (apt was better than this 20 years ago - that's how far behind the curve the package manager is right now: decades).
     
    transat, Ruslank100, OCASM and 5 others like this.
  30. RoughSpaghetti3211

    RoughSpaghetti3211

    Joined:
    Aug 11, 2015
    Posts:
    1,697
    I feel the same way, SRP is not the only tech in our projects. The other issues is upgrading a Unity editor has to go through a internal process. This take time. Packages is a way around that.
     
  31. RoughSpaghetti3211

    RoughSpaghetti3211

    Joined:
    Aug 11, 2015
    Posts:
    1,697
    Legal would lose their mind if we download ctrl-c ctrl-v. Tech teams would freak out too. At least packages sells it more as an official process
     
  32. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    5,457
    Both of these are an example of you having bad processes in your company which you are trying to circumvent. The vetting for a new unity version should be no different than a new SRP version- in fact, I'd be more worried about breakage from an SRP version, especially to an unverified beta one, than a Unity version. I also find it unlikely that your company would rather have you using unstable beta versions of things installed without vetting than they would official versions, or that your legal team even knows what a package manager is.

    Y'all are being silly and looking for an issue that doesn't exist, stretching the truth to fit some worry or anxiety you have. No capabilities have been taken away, simply the way in which you download the files is changing. If you need to run custom versions, use bleeding edge and buggy beta's, you can still do all of that stuff. The only difference is that you'll do those things intentionally, instead of some person randomly clicking buttons in the interface to see if they can get something to work. I can't tell you how many times the first reaction to any issue someone has is "Oh, there's a newer version of URP, let me try that" - literally before I even have a chance to answer their question.

    As example, the built in shaders have used this model for years, and I haven't heard a single complaint about it - ever.
     
  33. RoughSpaghetti3211

    RoughSpaghetti3211

    Joined:
    Aug 11, 2015
    Posts:
    1,697
    I can see from your point of view how frustrating it must be as an asset store dev. An I know I’m part of a very small minority that like packages so I trust the collective but I wanted to highlight where it did work well in my previous comments.I still feel package manager dropped the ball here and of packages took the fall.
     
  34. fherbst

    fherbst

    Joined:
    Jun 24, 2012
    Posts:
    802
    @jbooth please stay nice and keep the bad words for other discussions.
    We're in not quite the same, but similar boats here.

    You, as an AssetStore publisher of many packages, want the burden of support on users ("it's your fault if you updated to a SRP version that didn't ship with that exact version of Unity!");
    whereas I, as someone working on cutting-edge graphics projects for customers, want the burden of support on AssetStore publishers and Unity ("just because there's a single shader that Unity finally fixed I now need to do what?!").

    I'm not talking about "some anxiety" but the reality of business.
     
    Last edited: Jul 5, 2020
    Noisecrime likes this.
  35. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    5,457
    not quite; I want the burden of support to be removed by having a proper abstraction layer instead of a constantly breaking and fragile system. With surface shaders, I don’t have to worry about which version of Unity and it’s renderer your using. Right now I can only support a single version of and SRP because the support burden of supporting multiple is too great.

    i still fail to see how where you download files from affects your use case. Are your teams really unable to do anything if it’s not in the package manager? How did they do things before? How did they use asset store assets, since they have only been in the package manager since this year? This insistence that this is somehow important seems disingenuous and like arguing semantics for the sake of arguing. There are real issues worth discussing, and despite several back and forth’s I have not heard a single thing that you won’t be able to do because of this change. So please, explain what you won’t be able to accomplish with this change?
     
    Last edited: Jul 4, 2020
  36. fherbst

    fherbst

    Joined:
    Jun 24, 2012
    Posts:
    802
    Sorry, I really think the whole discussion around "should Graphics move back into Core right now" is super relevant, otherwise I wouldn't write so much about it. It's not for the sake of arguing :)

    I'm not talking about AssetStore packages, I'm talking about maintaining different SRP versions for different projects.
    I'm speaking of experience with the current state of URP and HDRP. Maybe numbers can show you what I'm talking about:
    - I have reported > 100 bugs in URP and HDRP over the last year or so
    - Many of these are showstoppers, and often those get fixed for the next URP or HDRP release
    - Currently I maintain, say, 10 projects as lead graphics engineer (which means it's my decision to update Unity versions or update package versions).
    - of those, 8 use "some URP/HDRP version from package manager" which means I can test + decide whether to update relatively quickly (say, takes me 1 hour to figure out if a new version is better than the current, and it's just a couple clicks of actually selecting a version).
    - the remaining 2 have custom SRP versions where I was unable to wait for Unity to fix issues and had to fix them myself. These forks of the Graphics repo take at least 10x more time to maintain, check which version to update to, ... even when the actual changes are just a few lines of code.
    - Unity version updates for these projects happen much less regular than SRP version updates because of the much higher maintenance cost of figuring out what new regressions these new versions bring in other areas.

    Now, when graphics stuff moves from packages back to core, this will essentially either force me to
    - update Unity all the time, majorly disrupting everyone's workflow (download new editors, update library, ...)
    - have forks of SRP in each of the projects, essentially requiring 10x maintenance time

    That's why I keep insisting that killing graphics packages is not a good idea - I'm afraid it will kill productivity for us, or force us back to a time where we had to wait a couple of months for Core fixes. The SRPs don't have the stability yet to move back to that.

    Proposal: put the Graphics packages on another registry. Keep them away from regular users. Don't keep them away from people who actually want to update SRP versions while staying on the same Unity version.
    Note that this will STILL keep the issue I mentioned above, neither AssetStore publishers nor Unity will test these combinations, which will (again) result in more maintenance time on our end.
     
  37. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    5,457
    I still don’t see how this affects you much. Yes, clicking on a download button in the package manager is slightly easier than downloading a folder from git and copying it into the project. But otherwise everything is exactly the same. If you put a specific version into your project it will be used instead of the built in version, same as if you install a non-verified version into your project. Don’t want to upgrade unity versions but want that patch release of the SRP? Download, copy, check it in and your done.

    now, the only potential difference I see is if Unity starts changing the c++ side more often than they would now, causing fewer versions of the SRPs to work with each version of Unity. But there’s currently no sign they are doing this. In fact, the change they are doing is essentially the exact same you suggested, but instead of a separate package manager they use GitHub.
     
    a436t4ataf likes this.
  38. fherbst

    fherbst

    Joined:
    Jun 24, 2012
    Posts:
    802
    What exactly doesn't affect me if I can tell you from experience (with packages & with forks) that it's at least 10x more maintenance work? ...
     
    sand_lantern likes this.
  39. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    5,457
    package version-
    1. Open package manager
    2. Select version
    3. Select upgrade

    Git version:
    1. Open repository
    2. Select version label
    3. Press download
    4. Copy folder to project.

    your managing 10 projects and doing custom SRPs, and that’s your bottleneck?
     
  40. Coroknight

    Coroknight

    Joined:
    Jul 10, 2012
    Posts:
    26
    I agree, this seems like a silly thing to argue about.

    Sure, it would be nice if you could override the built-in SRP with a package but don’t you guys think there are more important things to discuss here?
     
    jbooth likes this.
  41. fherbst

    fherbst

    Joined:
    Jun 24, 2012
    Posts:
    802
    You forgot to incorporate
    a) the fact that for each project and each potential update of SRP this has to be done X times to figure out if an update is "good" or not
    b) the fact that with version control, the above turns into submodule management instead of package.json management which is a big time factor in a mixed team with artists with many ways for it to break alone

    I think we'll leave it at that, this already has too much space here — to me the whole package situation is an important point, it hope this shortsighted "solution" of "hey let's drop packages again howaboutthat" will be revisited or explained better, and ultimately we all want to have better stability and cleaner APIs.
     
  42. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    5,457
    This is the same regardless of packages or git. Compared to the testing, etc, it’s an extra click or two.Even times 50 it’s less than dealing with a single bug.

    You’d only do this if you plan to modify the code - in which case you couldn’t use packages anyway. If you don’t plan to modify the code, why would you set all this up?

    if we end up with a unified API, a shader abstraction layer we can use to maintain forward compatibility, and feature parity with standard, and systems that allow us to plug into the renderer rather than having to modify it, I suspect the amount of work you have to do for SRPs would drop drastically, making a lot of this moot.
     
    GoGoGadget likes this.
  43. With all due respect, you clearly haven't tried to work with bleeding edge packages too much.

    First, I find it gruesome that the people, who are "clicking randomly in the editor" get the comfortable in-editor package management and those, who are trying to get ahead get the external/text-based package management.

    I'm working with at least a dozen packages (ECS project with various things in it).
    These have a constellation of versions which work together. Sometimes they don't. When they don't, I just try another constellation before I come for support.
    Now, if this whole stuff goes down as it looks, I will have to hunt for dependencies by hand. If I change one version in my manifest, I will need to look up, what is the expected version regarding this package among the several other packages and update those as well. BY HAND. I don't have to tell you how error-prone is this... and how quickly will end up with the "eff this whole thing".

    While the people who are "randomly clicking in the editor window" considered "real developer" and get the treatment of a - at this point completely useless - package manager.
     
    JBR-games and fherbst like this.
  44. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    5,457
    I'm quite familiar with the package manager and supporting bleeding edge technology, thank you. Been doing this for 25 years, have written game engines from the ground up, and have supported LWRP/URP/HDRP for well over a year. And my work for the last year has primarily consisted of optimizing people's games with early versions of Burst/Jobs. The dependency chain in the SRPs is not deep- in fact, it's entirely contained in a single depot. If you were talking about ECS, you'd be more correct, as it has dependencies on many different packages stored in many different places. But we're not, we're talking about the render pipelines, which have a dependency on the SRP framework, all of which are in a single repository, along with the tools which depend on them (shader graph, vfx graph, etc). So drop the "theoretical cases" and debate based on actual case please. No one is talking about removing the package manager from Unity.

    I mean, isn't that how it should be? Being on the bleeding edge means you will encounter more bugs and generally have a worse experience. That should not be the average users experience. Unity doesn't make the alpha version of Unity the one that downloads by default, for instance. Yes, that would be easier for people who want the alpha version, but what most people want is stability and reliability.

    I just find it so funny that out of all the issues there are with SRPs the "download button" is the hill you guys are choosing to die on.
     
    SenseEater and OCASM like this.
  45. neoshaman

    neoshaman

    Joined:
    Feb 11, 2011
    Posts:
    6,492
    It is not about randomly clicking the editor, it's about reducing clutter.
     
  46. Actually, I was talking about that.

    Because this decision wasn't made in isolation, it doesn't affect only the SRP, it is across the board. If it was only the SRPs, I wouldn't be happy, but it would be infinitely less painful. Although all of this fails as soon as you try to integrate the Hybrid Renderer into your shiny new ECS-based project. It took quite a upgrading/degrading session to figure out that actual 0.5.2-preview 4 Hybrid Renderer V2 is only working properly if you have the v9-preview14 URP package (with the Core v9-preview 13 and the Shader Graph v9-preview 14). And this was just the Renderer + SRP stuff. We didn't mention the other parts, which also will be moved, thanks to this decision. You said you extensively worked with DOTS, then you know how many packages we're talking about and how fragile they are in terms of version-matching.

    (edit: PS. obviously I'm on-board with all the other stuff, regarding the SRPs, although I don't see how can it be any better than the built-in after cramping the URP and the HDRP together... but well, I'm not planning to build any HDRP any time soon, so whatever from me)

    Well, the clutter is already reduced. You can find the "show preview" checkbox in the settings. If you don't want to see "the clutter", turn it off. It should be this simple. But no, because the randomly clicking people randomly clicking and then whining on the forums that the package manager is too confusing. Well...
     
    Last edited by a moderator: Jul 5, 2020
    sand_lantern and Noisecrime like this.
  47. Flow-Fire-Games

    Flow-Fire-Games

    Joined:
    Jun 11, 2015
    Posts:
    305
    Ah what I forgot about the mess with unity standard shaders (which should be entirely killed and replaced with graph shaders)

    - Unity plugins like Polybrush using a custom shaders for vertex painting etc is also something that totally has to be killed of and set on a taboo list. Nobody should be searching around how to make vertex painting work with shader graph while the unity plugin uses some dead-end diffuse only custom vertex paint shader from 2015 bloating the project instead of taking 15 minutes and making the proper vertex paint setup in Graph for people to use as a base and learn from. Its things like that on every corner making the entirety look like a total mess with no hand or foot.
     
  48. Kolyasisan

    Kolyasisan

    Joined:
    Feb 2, 2015
    Posts:
    397
    I'm a tad bit concerned about all the instancing features. We have the regular instancing from 5.4 I believe, SRP Batcher rendering path (which doesn't use instancing) and also the upcoming DOTS instancing for Hybrid Renderer V2, which is very unclear to me, especially how it ties with SRP Batcher and why it even needs instancing in the first place. I think this needs some major clarifications as well.
     
    OCASM likes this.
  49. AcidArrow

    AcidArrow

    Joined:
    May 20, 2010
    Posts:
    11,594
    Reading this thread, it seems like community opinions on SRP are more fragmented than Unity's render pipelines themselves!

    *rimshot*
     
    SMHall, Ruslank100, JBR-games and 3 others like this.
  50. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    5,457
    This was only for the SRPs - I haven't seen any announcement from Unity that they are getting rid of the package manager and just including everything in Unity again.

    One of the advantages of having some kind of surface shader replacement is that this stuff should "Just work", in that whatever code is needed for instancing in whatever version you are using should just be there. But yeah, the term is heavily overloaded at this point- we also have "Draw Instanced" on the terrain, which is technically correct but requires extra shader code, because it also means "Draw with new terrain system which uses textures to store vertex height and normals instead of being computed on the CPU, which happens to also draw using instancing".
     
    OCASM likes this.
Thread Status:
Not open for further replies.