How do I write a normal decal shader using a newly added (Unity 5.2) "finalgbuffer" modifier?

Can someone explain to me how alpha blending is supposed to work for a shader that outputs to an RGBA GBuffer (specifically, deferred RT2 with specular in RGB and smoothness in A)? This is a hugely puzzling thing to me.

• Traditional alpha blending (Blend SrcAlpha OneMinusSrcAlpha and output.a = o.Alpha) - you are completely destroying anything you put in the fourth channel of the output (in this case, smoothness)
• Premultiplied alpha blending (Blend One OneMinusSrcAlpha and output *= o.Alpha) - you are invariably making the smoothness wrong by multiplying it by an unrelated factor, and smoothness intensity is used for blending itself, which makes it impossible to override high smoothness with low smoothness

Let's say underlying surface has specSmoothness.w value of 0.8. How can you ever write a value of, say, 0.2 into specSmoothness.a if that very same value of 0.2 is used to weight your contribution to specSmoothness, resulting in smoothness close to 0.8?

 

You don't need to go all out with the vertex coloring when using something vertex colors like e.g. make your decal Color.red. You could use a subtle vertex coloring to give a little variance to an area of repeating debris decals to make it look less copy-pasted. I'm sure good usecases can be found if the functionality is there. So if it isn't a big performance or implementation issue I'd also welcome the feature very much.

 

Yeah it has pretty much no performance cost and is easy to add.

But as I've said, stuff like debris isn't exactly depicting a deformation or wear of underlying surface, it's depicting an entirely separate object. So it has to output albedo, smoothness, specular, normals and emission at full intensity, overriding whatever was underneath. And if that's the case, you can just use a standard transparent shader with Offset -1, -1. There is no need for finalgbuffer trickery I do here.

 

To rephrase the question:

Is it possible to do blending using a completely separate float? With Blend One OneMinusSrcAlpha and Blend SrcAlpha OneMinusSrcAlpha you have to explicitly modify all four RT outputs in the finalgbuffer functions with o.Alpha (with One it's *= o.Alpha, with SrcAlpha it's .a = o.Alpha), which breaks outputs that are supposed to write to RGBA RTs, specifically smoothing.

Basically, I'm wondering whether it's possible to do something like Blend Src:myFloat OneMinusSrc:myFloat or whether there is some other way to keep specSmoothness (or any other RGBA output) .w completely untouched while controlling the blending with other float.

 

I'm pretty sure that's unfortunately not possible. You can't both use alpha for controlling the transparency and also treat it as a sensible value. It's the reason why engines shy away from putting important shading values in alpha channels of their g-buffers. The best you can do is not touch them at all and use a separate blending mode for the alpha: Blend SrcAlpha OneMinusSrcAlpha, Zero One

 

@Dolkar

So I take it the only way to sidestep this is to modify the deferred rendering itself to use one more render target for smoothing (with 2 channels left unused, potentially for some other changes)? If I go that route, I have to ask where exactly finalgbuffer signature is defined, because I was not able to find it in any of the files included with 5.2 shader source archive.

P.S.: Also, are Logical blend operations introduced into Unity with DX11.1 support relevant in this case at all? Can you use them to set up blending based on a custom value, or are those logical added for something else?

 

@Dolkar

Another idea - is it possible to write a two-pass surface shader that will use alpha blending for RT1, RT3 and RT4 in the first pass (while outputting full transparency to RT2, causing no contribution to happen there), and then do a second pass that will output nothing to RT1, RT3 and RT4 while using a custom float based clipping (alpha test) to output to RT2 (specSmoothness)?

 

I don't think you're able to add an additional buffer, because Unity has a limit of up to 4 buffers bound at once. Logical blend operations are mostly made for using binary operations with uint buffers. I suppose what you're suggesting in the second post would work, if alpha testing smoothness contribution is acceptable for you.

 

If there is no other way to blend it, then I sure as hell will take this. So how can I go about writing such a shader? I can selectively remove the contribution to RT2 in a traditional alpha blended or premultiplied alpha blended pass, but I have no idea how to remove the contributions to RT1, RT3 and RT4 in an alpha tested pass.

I probably need to use two clip operations in a row, but I don't think I understand how.

 

@Dolkar
Am I on the right track with this?
http://hastebin.com/uqokusezoz.avrasm

Not sure which pass should be the first and how to clip diffuse/normals/emission independently from specSmoothness output in the alphatested pass.

 

I think this is the best way to go around it:
Your first pass uses the suggested Blend SrcAlpha OneMinusSrcAlpha, Zero One and renders to all render targets as usual, including the RT2. That means you get alpha blended normals, but the target alpha values of all render targets will be left untouched. The goal of the second pass would then be to just write alpha tested smoothness.

To trick is in making the second pass leave the other attributes untouched. I see two options here: You might notice that the alpha channel of all other render targets besides n.2 is unused, so we don't need to care about overwriting those. All we need is to turn off writing to the rgb channels, which is easily done by adding ColorMask A. The shader would then output all zeros except for the smoothness channel.
The second option is basically the above, but further optimized. There's a more efficient way than blending mechanics to turn off rendering into other targets: By only specifying those you want in the fragment shader output. Unfortunately, you can't do that with surface shaders. You have to use the traditional vert/frag approach and define your fragment function as such: float4 frag(v2f IN) : Color2 {} which returns just a single vector that gets written into the second render target.
Luckily, you only need to sample the main texture and extract the smoothness for it in the fragment shader. That's nothing you're not already doing.

 

@Dolkar
You're saving me again! I owe you a beer at the very least, this is another bit of incredibly useful advice!




This works very well! You can notice the harsh transition sometimes, but tweaking cutoff to match the smooth alpha blended specular/diffuse/normals edge hides it among more noticeable high-frequency detail eye can focus on, especially if your edge smoothness value isn't too jarringly different from the smoothness of an underlying surface. And I guess I can dither/break up the smoothness edge with noise later.



Here I'm using ColorMask A - am I correct in assuming that your second idea (with Color2 {}) will also have to be alpha tested, only being different performance-wise?

 

I'm glad to help! Especially so since deferred decals is something I'd have to look into sooner or later and now I know what are the limitations of it in Unity. Alpha tested smoothness really isn't as noticeable as I thought it would be.
And yes. The only thing that changes is that instead of needlessly writing the alpha to all render textures and wasting performance, it just truly only affects the smoothness channel.

 

Great to hear, I'll definitely make a fragment version then.

By the way, do you think it might be possible to make smooth smoothness (har har) blending by using three passes? First one will be the same, second one will be additive (Blend One One) with 0 in all RGB channels (therefore making no effect) and top half of the alpha range (remapped to 0-1), and third one will be multiplicative (Blend DstColor Zero) with 1 in all RGB channels (therefore making no effect) and bottom half of the alpha range (remapped to 0-1). Not sure if that setup will allow you to output explicit smoothness values like you can do now, but you can get highlighting, you can get darkening, and you can get smooth edges with it. Probably not very performance friendly, though.

Another question, about the old first pass, by the way - I'll probably need to remove some independent control over output to RTs from the shader to avoid incorrect combinations. Which outputs are tied together and should never be tweaked separately? I guess albedo + specular must absolutely be linked, because depending on metalness value, your diffuse texture goes into one or another. But I don't know how exactly emission is linked to albedo and specular RTs, and I don't know whether emission should be blended when you only want a normal contribution, and... Err, I guess I can sum all the cases I'm worried about like this:

• If I want to output only normals - do I need to also push emission contribution to 1 in that case? I guess emission contains reflections, and reflections are affected by normals, so I do want to set it to 1, but emission from ambient also depends on albedo color (due to energy conservation, I guess), which makes albedo come through your normal-only decal if you include emission. Same deal if I want to output only ambient occlusion or normals and AO. I guess the nicest solution possible is to output RT3 (emission), but mask it not just by alpha but also by occlusion map to keep as much of underlying emission intact as possible. Won't be physically correct, unless you fine tune directly unused albedo and spec values of your decal material, but it will be the least destructive way to blend in AO and reflections, right?
• If I want to override specular or diffuse with a decal, I need to output to both RT0 and RT1, especially with metalness workflow, because albedo map can go into either of those, correct?
• If I want to override only smoothness, I can drop the whole first pass altogether - smoothness should have zero effect on emission result, correct?

 

Actually, now that I'm thinking about it... What if the first pass multiplied the existing smoothness value by one minus the actual alpha. Then all the second pass would have to do is just output (smoothness * alpha) with additive blending! It'd basically become premultiplied alpha, where the target multiplication is done in a previous pass.
So, for the first pass you'd have Blend SrcAlpha OneMinusSrcAlpha, Zero OneMinusSrcAlpha with outputs the same as they are now... and the second pass has just Blend One One but now delivers the second part of the equation, smoothness * alpha in its alpha output. And there you go, true, alpha blended smoothness!

The emission output is tricky, yes. The only situation when it becomes correct is when the decal actually modifies everything. Pretty much all the material attributes have an effect on the emission value, including smoothness, which makes reflections look rougher. It's hard to say what compromises you can take without introducing noticeable artifacts... Experimentation is probably the safest way to figure that out here.

 

Sorry, I'm not sure why you're saying that outputs should be "the same as they are now" when previous pasted version contained premultiplied alpha setup while blending mode you're quoting is not Blend One OneMinusSrcAlpha; and I'm not sure I follow where exactly is "existing smoothness value" should be "multiplied by one minus the actual alpha" and how exactly should second pass look like. Is this what you had in mind?

http://hastebin.com/xumomizuzu.coffee

P.S.: Also, whoops. To be honest, I have also never used the comma sign in the blend mode lines before. I have not actually used Blend SrcAlpha OneMinusSrcAlpha, Zero One you have suggested in your previous post, I used Blend SrcAlpha OneMinusSrcAlpha and assumed that the later part it was just a typo on your part. Those alpha test screenshots above were taken with just Blend SrcAlpha OneMinusSrcAlpha, nothing after that. I wonder what exactly is two-part blend mode supposed to do, because this is the first time I'm ever seeing such a definition. Documentation on SL contains no examples of blending modes with two pairs of sources and destinations.

As about emission, thanks for the explanation! I'll keep it fully blended or blended based on a separate slider moved at an artist's discretion for now, I guess.

 

My bad, I should have explained further. The comma in between blending modes signifies separate blending mode for the alpha channel. The SL documentation actually does briefly mention it under Syntax.

Also, I haven't realized you sticked with premultiplied alpha, so I just posted how the version with classic alpha would look like. Either one works. But yes, both the passes you now posted look correct to me, with the exception that the second pass should also have the alpha multiplied by _ContributionDiffuseSpecular. The two alphas, one that is used directly in the first pass and the other multiplied with the new smoothness need to match.

 

I made that change and unfortunately, the shader is still is not giving us the desired affect. Here is a short gif with everything but diffuse-specular-smoothness contribution disabled:



As you can see, smoothness is not being rendered at all, with the only remaining bit being an alpha edge artifact. Here is an up to date version, just in case:

http://hastebin.com/deficaquze.avrasm

Edit: Another attempt, this time I multiply smoothness by 1 - o.Alpha in the first surface shader, not in the first finalgbuffer function. Different result, but still not what we need - all smoothness is now black:



http://hastebin.com/ipecagiwuk.coffee

 

The 1 - o.Alpha won't do... The second shader needs to output specSmoothness.a *= o.Alpha * _ContributionDiffuseSpecular. The smoothness component needs to be multiplied with the same alpha as in the previous pass. Which, as I just noticed, also means you need to actually define those values in the surface function.
Here's my attempt: http://hastebin.com/uyuwefuqir.coffee

 

Hell yeah, this works! I was confused by simultaneous use of .a and *=, but this resolves it!

 

Hi, so I was trying this out, and I am getting syntax errors that make no sense to me when just taking the shader code from the hastebin post above.
Any ideas? It seems to start complaining at ln 26 (Offset -1,-1) and then also complains at lines 87,96, and 104. Is there something I am missing from that second pass? It seems to not like seeing input redefined or something like that? This is in 5.2.1f

 

I absolutely love this.

Is there a way to have an only ambient occlusion mesh decal being multiplied to a material.

That would be a great way to fake global illumination along complex shapes while using very little memory.

 

I still can't get the final shader to compile, but the old alpha test version seems to work, and I saw some issues:

When placing a quad with a normal map in the editor I am noticing some quirks with the orientation affecting the apparent normals (ie, I rotate the quad which is just a simple divot, and the specular highlight rotates with it). I am wondering if the normals need to be transformed from tangent space to world space before going into the Gbuffer (because aren't the ones in the gbuffer in world space?)
EDIT: as seen below this can be ignored, but to avoid this issue make sure you are baking your normals properly.

 

Shader failing to compile-SOLVED
If anyone else has this problem, for some reason the Offset -1,-1 was pasting in with a space between the - and the 1. In addition, in the code on hastebin, ln85 is missing a semicolon after sampler2D _MainTex.

EDIT: Removed the mention of issues with world normals, as it was false. Apparently that was ocurring because I was using a normal map made from the nVidia photoshop filter on the Unity Quad. When I properly baked out the map, it behaved correctly, so YAY!

 

Great work! Is it possible to adjust that shader for curved surfaces?

 

So there is an issue with this still. For some reason, light that comes from global illumination in Realtime GI(ie an emissive surface), is not affected by the normals in here. Doesn't make any sense to me really. You can see the problem in this image:


EDIT: Some more info. This is specifically affecting Directional Specular Lightmaps. The specular from this type is what is affected. Which is too bad, because they add alot when they work.

Further EDITS, this may be impossible to solve. It appears that the specular from Directional Specular Lightmaps is applied in the emissive buffer, and is not actually deferred. So it may be impossible for the gbuffer to affect it (from looking at where it pops up in Frame Debugger).

 

Posted cleaned up and updated versions of that shader to the Asset Store along with some sample content, it's awaiting approval now. As promised, all free.



@ruj, you are correct, for some weird reason GI is not deferred, so this is impossible to solve. Well, you can overwrite the emission of the underlying surface to alleviate that somewhat, but emission is heavily dependent on albedo, specular, smoothness and occlusion outputs of your decal, so you'll have to tweak them to match the underlying surface or blend them straight in if you want natural-looking blending. Another thing that can somewhat mask this issue is reflection differences from smoothing values - the jump in reflection sharpness and highlight radius distracts the eye from some missing GI quite nicely.

@justtime

Not sure what you mean. The shader works very well on the curved surfaces:

 

So yeah, you can get a pretty convincing result by partially blending the normals and then fully blending the emissive. Makes a pretty solid mixture of the two. Although I am sure it varies based on use case.

 

Thanks a lot, you are the hero this forum needs!

 

I mean some cube-shape form(with plane) which can wrap meshes like in http://blogs.unity3d.com/ru/2015/02/06/extending-unity-5-rendering-pipeline-command-buffers/

 

Ah, planar projection using depth buffer and a box. Err, technically that should be possible, but I have absolutely no use for projection like that so I never tried to make a shader handling it. I'm interested in technique used in Star Citizen and Alien: Isolation, which takes advantage of pre-modeled decals that are UV mapped manually.

I can understand why you want it to work without Command Buffers (they are a royal pain with all those additional components and other clutter), and I'll see if it can work without Command Buffers if I'll have the time, but can't really promise anything.

 

It's would be great for many people i think : blood dynamic decals, in editor design things, may be something else =)

 

Isn't there a limitation for that technique that makes it unsuitable for meshes that deform during animation?

 

Ooh can you also create a repo for this, so people can submit variant of these shaders?

 

Tried out a way to break up seams, since you can be using lots and lots of seams when going with that hardsurface style. Works great (don't mind some weird edges between surface materials, I was too lazy to detail them properly there):



Pretty simple idea. Works best if your 3d software has instancing and good snapping and rotation tools, since with that placing reusable decal pieces becomes an absolute breeze.

P.S: Another idea: as packed map layout I'm using already includes height for parallax in one of the channels, I can probably reuse it for height-based blending change, ignoring the underlying surface albedo/spec/gloss past certain depth. That will allow me to hide the edges between materials and remove the need to author them carefully and precisely. Basically, I'll have one mask (usual alpha) for blending normals, and another mask (derived from height) to blend albedo/spec/smoothness. I already do it here to some extend with dirt mapping based on AO, but using height will give me more precise control.

 

Just wanted to say how impressed I am with this technique, it clearly has huge potential when used in the right circumstances. Having tracked down the various threads on polycount and elsewhere, its been an enjoyable afternoons reading. Actually I found polycount thread to be much more informative than here with all animated gifs to get an understanding of the process, but this thread is great for learning from the actual code.

So many thanks to back9-flcl for pursuing this and to those that assisted in getting the shader working. Looking forward to grabbing the asset from the asset store once approved.

 

Yeah I already have it in a private repo, but it's shared with some other stuff I'm working on. I'll have to clean up some stuff I can't distribute or set up another repo. I'll look into it a bit later.

 

And here is the final result. I think I have that workflow with modular seams pretty much nailed. Made some improvements to decal shader and the process itself:



This illustrates only the approach to the seams, but I already posted examples of isolated details, which are a lot easier to use.

 

I was wondering if it is even possible to author this type of shader using the vertex:vert, fragment:frag method of writing shaders.

I have been able to get it writing to the proper buffers this way, but there seems to be no way to force the decal objects to render at the end of the deferred g-buffer pass, so they often have their values overwritten. The "Queue" tag seems to have no effect when using the deferred path. Any ideas on that front?

 

Of course it's possible. They are vert-frag shaders already, just copy the code Unity generates from the surf shader inspector.

Not sure what you mean by queue issues. I haven't had issues with that, geometry + 5 queue seems to work fine for me.

 

Hmm, interesting, I was trying +1, maybe the number just has to be much higher, will check that out. At +1 the objects were just rendering right in the middle of all the others so they got completely overwritten. Will test that.

EDIT: Okay, great, now that works, not sure why +1 didn't do it, but hooray.

 

Would it be possible to move the quads in-game? Maybe with a skinned mesh for instance.

 

Hey @bac9-flcl, where was the link to the asset store item for this awesome shader? Can't seem to find it in this thread anywhere.

 

So, yeah, where we can find it?

 

Any update on the store status?

 

@bac9-flcl
Very impressive work! This problem has been bugging me for ages. I can't wait to add an implementation of this technique to Alloy.

 

@n00body @Andrew_atl @id0 @cowtrix @ruj @Noisecrime @rea @justtime

Hey folks, sorry for the delay, asset store submission got rejected because I forgot to fill out publisher info, and I haven't had the time to set up a repo before. But I do now, so here it is:

https://bitbucket.org/bac9/shading/overview

The repository includes the shaders themselves, some sample scenes with set up objects, and some meshes/materials/textures for those objects. I also added some packed surface shaders useful for objects under the decals. That green box from screenshots above is in there.

License is basic MIT, full text included in the repo. Though, if you end up using anything, I'd appreciate a message showing your results - I'm always curious about other's results, that's how this whole thing got going (from looking at Star Citizen and Alien Isolation art workflows)

 

Thank you so much for all your hard work and for sharing this!

I still get: "You do not have access to this repository." but I assume that's not intentional?

 

Should be open now!

 

This shader is fantastic. Massive kudos. I do have a question though. I would like to try and use this with an albedo map as well, but I am having an issue in both finding a way to use a albedo map with the shader as is, and in trying to add an albedo slot to the shader itself. Any help would be greatly appreciated.