Question How to ship render pipeline modifications as assets?

I've got a prototype of a custom GI system. It's currently implemented just as a custom shader that needs to be applied to objects in order to consume the GI. I would like to instead have it automatically work with all existing lit shaders. I'm targeting URP.

I think that forking the render pipeline and patching

SAMPLE_GI

may get me what I need, although I've not tried it yet. A few more edits may be required.

The problem that I have is that even if this works it seems like a very bad workflow. I have to maintain a fork of URP to just change one

#DEFINE

, and I have no idea how I would go about shipping this as an asset. Would I have to ship a whole custom render pipeline and tell users to switch to it? This is broken for a number of reasons:

• I now have to maintain my fork against all future URP changes and put out asset updates to maintain compatibility, even though I'm changing only one small thing that won't usually be touched by URP updates.
• This is incompatible with any other asset that also makes a small tweak to a render pipeline. There's no way to do modular interplay between two assets that each make a modification.
• I'm shipping a whole render pipeline worth of code and assets in order to patch ~10 lines.

One thing that I've found incredibly frustrating about SRPs is that they're not setup to allow this type of module replacement. They could be designed around having a standardised interfaces for modules so that I could create a GI Module asset that would only override that particular area of behaviour, but they're not. For all of the talk about being customisable, they're not designed to be customised.

One hack might be to create a tool that edits the shader source files to patch them as I need. The user would have the standard URP, but the tool would edit in the patch. This feels incredibly brittle and broken, but it might just about work.

What's the proper way to go about changing one small area of render pipeline functionality, and shipping that change as an asset?

 

There isn't a proper way. As you said, they're not designed with this in mind. Unity's original expectation seems to be that people would "simply" put out entirely custom SRPs for people to use, not modify the URP / HDRP. It's one of the frustrations many of us long time users have had with Unity and the SRPs, as easy modification was one of the big features we've loved about Unity.

 

Thanks, this is the answer that I feared. Almost every day of working with Unity ends with me frustratedly saying "maybe I should just build my own engine".

It would be a large project and I can't just do it on a whim, but I'm temped to attempt to build a custom SRP designed for modularity. Do you know if anybody has attempted this or whether there would be major barriers? I'm imagining cleanly separated modules with standard interfaces with some clean way to drop in custom implementations of functions. Maybe the existing render pipelines could be refactored in this pattern without having to redo and test the actual implementations...

 

If you just want to add a pass to URP forward renderer, Make your ScriptableRenderPass like this.
And your ScriptableRenderFeature like this. You can set the renderFeature to the forward renderer asset, and that rendererFeature insert the pass to the current URP forward renderer.
I think you can add #DEFINE in your renderPass, and it is valid in the pass after that rendePass. But if that GI define is not from URP but from Unity C++ side( based on MeshRenderer Monobehavior settings), you cannot override it from SRP, anyway.

You can customize some #DEFINE from Unity C++ side at context.DrawRenderers with your DrawingSettings that has PerObjectData options. So you may have to customize forward renderer's standard passes.
If you need to customize forward renderer's standard passes, you have to make your ScriptableRenderer. Maybe you need to copy,rename then modify the ForwardRenderer.cs. You can set the renderer to the URP asset( and select it as a default or per camera).

These all can be done without modifing URP package. But ForwardRenderer.cs is actively updating, so providing the custom ForwardRenderer asset will be hard.. Providing my renderPasses and/or simple original rendereres works with multi URP versions, for my case( in-house projects).

 

Thanks for the response! This is interesting, but I don't quite see how it addresses my use case yet. That may just be me not understanding!

It looks to me like you're talking about

#DEFINE

s in terms of toggles. That's not what

SAMPLE_GI

is.

SAMPLE_GI

is defined in the shader library in

GlobalIllumination.hlsl

and is used by other library functions as well as specific shaders. It currently looks like this:

Code (CSharp):

1. // We either sample GI from baked lightmap or from probes.
2. // If lightmap: sampleData.xy = lightmapUV
3. // If probe: sampleData.xyz = L2 SH terms
4. #if defined(LIGHTMAP_ON) && defined(DYNAMICLIGHTMAP_ON)
5. #define SAMPLE_GI(staticLmName, dynamicLmName, shName, normalWSName) SampleLightmap(staticLmName, dynamicLmName, normalWSName)
6. #elif defined(DYNAMICLIGHTMAP_ON)
7. #define SAMPLE_GI(staticLmName, dynamicLmName, shName, normalWSName) SampleLightmap(0, dynamicLmName, normalWSName)
8. #elif defined(LIGHTMAP_ON)
9. #define SAMPLE_GI(staticLmName, shName, normalWSName) SampleLightmap(staticLmName, 0, normalWSName)
10. #else
11. #define SAMPLE_GI(staticLmName, shName, normalWSName) SampleSHPixel(shName, normalWSName)
12. #endif

The patch that I would need to make is to modify it to point to a custom function which samples GI data from my own data structure, so that everything else in the shader library, and all of the lit shaders, use my GI data instead of Unity's. Another way to do this might be to replace the

SampleSHPixel

function instead, as my GI solution basically provides its own probes in a different format.
.
I don't think that any method that injects some code later in the pipeline will work. The edit needs to happen exactly there in the shader library code so that it gets used correctly by the rest of the library. We can't inject the custom

#DEFINE

before the library gets included, because the library will replace it. We can't inject it after, because the library will have already used the wrong one. It really does need to be an edit to the existing library code.

At least, that's the conclusion that I've reached so far.

 

For the record, I'm aware that there is sort of a clean way to do this custom GI, which is to target deferred and apply it to the GBuffer. That's not a route I'm interested in going down at this time, although I do have it in the back of my mind as a last resort.

 

Another problem with SRPs is that you can t modify the included files without having a local copy of urp/hdrp. And not many people want to go this way.

I had a similar problem with a terrain detail grass shader, where you would need to have a local copy of urp. Because of this, I just never even bother releasing it for urp and kept it only for builtin rp.

 

Yeah, There are no proper way to edit those shader functions without custom URP package. Provide a custom function node for Shader Graph that redefine Baked GI is a way, but Shader Graph itself is not practical at this point, so..

 

Well your individual shaders can #undef those macro's and redefine them. However, it means you have to ship custom shaders for everything.

Ideally Unity would separate #defines like this from implementation as they did in BIRP, but they tend to be conflated in URP. For instance, I wanted to set the unity_ColorSpaceDielectricSpec to 0 to prevent blown out specular on edge on terrain. In BIRP, this is defined in one file, then used in another, so simply redefining it before the second include works. But on URP it's defined and used in the same file, so you'd have to replace that entire file to change it.

Most of the SRPs end up like this- they favor top down control and branching of millions of lines of code over being modular and allowing people to adjust things.

 

Yeah, that wouldn't be a perfect drop in replacement, but it might be something.

It's interesting that you say that Shader Graph isn't practical. I've never actually used it yet, because I'm comfortable writing shaders and I don't enjoy visual scripting. What's wrong with Shader Graph? I thought that it was considered ready.

 

Yeah, if I'm only trying to get this to work in my own project then there are various ways that I can hack that. I expect that I will end up with a forked URP - or a forked HDRP as I may transition.

The situation that I'm in is that I have a prototype of a custom built fully realtime GI engine that's proving surprisingly strong in a range of circumstances. In light of Unity's shocking lack of proper realtime GI, I've been seriously considering polishing my solution up and publishing the asset. The problem is that I can't find a way to do that that offers a good experience to the asset customer. If I provide a hacked together solution, all that will do is increase the size of the support job. I don't want to be in the business of engine hacking just so I can ship a lighting algorithm which already works.

Yes. It's infuriating. Every time that I try to do something in Unity I run into either a broken design or a broken implementation. One of these days I'm going to crack and write a whole commercial engine from scratch. I should be able to get that done in a weekend, right?

 

Then you don't want to do anything with shaders or SRPs. Though it's of note that in your case all you have to do is provide replacement shaders. Bakery, for instance, does this, and I recently added a Bakery stackable to Better Shaders that makes it easy to add bakery support to any shader written in Better Shaders. So it's a viable solution to ship your own shaders for something like this- just annoying that you have to.

 

You're not wrong. Well, I like writing shaders themselves, but making them play nicely with the whole of Unity in a modular fashion has been a nightmare. I appreciate that you're one of the people trying to make that better! Your assets are certainly a step in the right direction.

I'll take a look at the Bakery replacement shader approach, thanks for the tip!

My concern is that people may have lots of custom lit shaders - either that they've written themselves, or from other assets - that would need to be rebuilt to accommodate my GI, so just providing replacements of the standard lit shaders wouldn't cut it. You deal in this area, do you have a sense of how true that is?

 

I mean it's an issue, but varies from project to project. Most people using SRPs only have the Lit shader or shader graph shaders, because anything else is basically under attack from the Unity devs. For assets, everything is built in Better Shaders or Amplify, or only runs on a sub section of SRPs. This is one reason I came up with Stackables for Better Shaders, since it means if you have better shaders installed you can just stack integrations from 3rd party assets onto whatever shaders you have and it all just works. (it really should be the base of Unity's shader system).

Maybe one day Unity will win it war against people writing shaders and then we'll all be forced to use theirs, seems to be what they are going for.

 

To clarify my position, I've actually got this project mostly built on the BIRP right now and am just future proofing on my planned transition to SRP. The actual SRP ecosystem is one that I'm clearly not experienced in yet, so this is all good information, thanks!

On the BIRP side of things, I personally have a custom volumetric steam/fog/clouds/smoke shader that samples GI, a parallax window shader that samples GI, some volumetric translucent crystals that sample GI, a couple of compute shaders that sample GI, and probably some other things that I'm forgetting at this moment. Perhaps I'm unusual in this regard.

Oh well, maybe one day you'll get the acquisition email from Unity.

Forgive me, I've not looked at Better Shaders in much detail yet. How does your stackable concept handle just adding additional light to a surface - is there a clean API for that? It sounds like your Bakery integration achieves it.

 

Code (CSharp):

1. BEGIN_DEFINES
2. #define _OVERRIDE_BAKEDGI
3. END_DEFINES
4.  
5. BEGIN_CODE
6.  
7. void SurfaceFunction(inout Surface o, ShaderData d)
8. {
9.    o.DiffuseGI = 1;
10.    o.BackDiffuseGI = 1;
11.    o.SpecularGI = 1;
12. }
13.  
14. END_CODE
15.  

Where 1 is whatever value you compute.

 

Very nice!