Using vertex colors for texture blending

I'm a total newbie to shader language, but this seems like a pretty trivial thing to do. I'd like to combine these two scripts (TerrainFourLayer and VertexColor) so that the RGBA vertex colors of the model are used to blend the textures in place of the mask layer. Can I get some help please?

Thanks!

 

Those are two different beasts. Do you want to blend four textures, or just two? I can give you a shader, if you only need to use the alpha channel of the vertex colors. Otherwise, it requires Cg or GLSL, and I don't know them yet.

Also, TerrainFourLayer doesn't use lighting. Do you need it? If so, what kind?

 

This shader seems to combine vertex lighting with vertex colors.

--Eric

 

Oops. I totally skipped that because there were no parameters for it in the Properties. (As in, the other shader uses _Color as a tint, independent of lighting.)

Only for specular highlights. (And SeparateSpecular doesn't even work with Unity iPhone. :x)

 

It doesn't matter too much to me which kind of lighting this uses.

The reason I want to use vertex colors instead of a mask image is because I'm generating meshes procedurally which means it's impossible to (seamlessly) UV project any better than XZ=UV, so I would not be able to have two different textures at different Y values. My current implementation can generate vertex colors trivially so it would be best to be able to use the vertex colors for the texture masking.

So, is this possible? What would need to be done to make this work?

 

Yes.

It depends. Please figure out what you need, or else nobody can help you.

Figure out what you really need in terms of lighting. If you don't need lighting, the shader is faster. And it works on more cards.

 

This shader will need lighting of some sort. There will be a lot of objects using this shader. I would prefer to be able to blend four textures, but I might be able to work with just two if that's all that's possible.

 

However, if you use ColorMaterial AmbientAndDiffuse, or ColorMaterial Diffuse, the alpha channel of the vertex colors is unaltered. (Otherwise, all vertices have their alpha channel overwritten to white.) As such, you can use this shader, and have vertex lighting, but you only get to blend two textures.

Now, I never knew about that ColorMaterial thing until just now. What would be a lot more useful, I think, is to be able to use ColorMaterial Specular, so you could use vertex colors as a specular map. However, like I said, that overwrites the alpha value. You think this is something UT could fix, or do you think/know if it is just a GPU design issue?

Code (csharp):

1. Shader "Vertex Color w/Vertex Alpha Blend" {
2.    
3. Properties
4. {
5.     _SpecColor ("Specular Color", Color) = (1,1,1,1)
6.     _Shininess ("Shininess", Range (0,1) ) = 0.7
7.     _Emission ("Emmisive Color", Color) = (0,0,0)
8.     _MainTex ("Texture 1 (white vertices)", 2D) = ""
9.     _Texture2 ("Texture 2 (black vertices)", 2D) = ""
10. }
11.  
12. SubShader{Pass
13. {
14.     Material
15.     {
16.         Specular [_SpecColor]
17.         Shininess [_Shininess]
18.         Emission [_Emission]
19.     }
20.                        
21.     ColorMaterial AmbientAndDiffuse
22.     Lighting On
23.     SeperateSpecular On
24.    
25.     SetTexture[_MainTex]
26.     SetTexture[_Texture2] {Combine previous Lerp(primary) texture}
27.     SetTexture[nothing] {Combine previous * primary Double}
28. }}
29.  
30. }

 

Then would it be possible to do a pass with the vertex colors as the diffuse color with no lighting, and use that for the texture blending in another pass with the lighting? If this was possible it would reduce the number of layers possible to 3 but it would still be useful.

I've been experimenting with it myself but unfortunately I don't know the shader language so it's just been a matter of copying or removing code.

 

Yes, but I'm sure that there's a better way. It would just involve Cg or GLSL, and I won't bother to learn either until Unity iPhone supports it. If nobody else contributes anything soon, and you still need something in a few days, I'll write the shader you're talking about for you. (I think it's a waste of each of our time if someone more knowledgeable can help.)

 

Well, unfortunately, nobody else has chimed in. This is the best I can do for you for the time being; it's a horrible shader, in that it takes more passes than I think it should (it doesn't even batch with Unity iPhone), it doesn't run on 2-texture cards, and I can't work SeparateSpecular-type highlights in without another pass. That said, it lets you blend three textures, using vertex colors, and it uses the full capabilities of fixed-function vertex lighting. :?

Code (csharp):

1. Shader "2-Part Vertex Blend" {
2.  
3. Properties
4. {
5.     _Color ("Main Color", Color) = (1,1,1)
6.     _SpecColor ("Specular Color", Color) = (1,1,1)
7.     _Shininess ("Shininess", Range (0,1) ) = 0.7
8.     _Emission ("Emmisive Color", Color) = (0,0,0)
9.     _MainTex ("Texture 1 (white RGB)", 2D) = ""
10.     _Texture2 ("Texture 2 (white Alpha)", 2D) = ""
11.     _Texture3 ("Texture 3 (black Alpha)", 2D) = ""
12. }
13.  
14. SubShader
15. {
16.     BindChannels
17.     {
18.         Bind "vertex", vertex
19.         Bind "texcoord", texcoord
20.         Bind "color", color
21.         Bind "normal", normal
22.     }
23.    
24.     Pass{SetTexture[_MainTex] {Combine texture * primary} }
25.    
26.     Pass
27.     {
28.         Blend One One
29.        
30.         SetTexture[_Texture2]
31.         SetTexture[_Texture3] {Combine previous Lerp(primary) texture}
32.         SetTexture[_] {Combine previous * one - primary}
33.     }
34.    
35.     // Same as using Double, with SeparateSpecular Off.
36.     Pass
37.     {
38.         Blend DstColor SrcColor
39.        
40.         Material
41.         {
42.             Ambient [_Color]
43.             Diffuse [_Color]
44.             Specular [_SpecColor]
45.             Shininess [_Shininess]
46.             Emission [_Emission]
47.         }
48.         Lighting On
49.     }
50. }
51.  
52. }

 

Thank you! It pretty much works, but not quite. The attached image sums it up pretty well. The only way to get Tex 3 is to use 0 alpha and some color with max saturation. However, any areas of the textures that are almost black are tinted toward the vertex colors, as you can see in the upper left with the blue, and in the middle with the red.

The middle ball should be orange + black + white but instead it's orange + red + white (or whatever color I set).

Here's a page I found in the docs when I was trying to kludge together this shader that might help:
http://unity3d.com/support/documentation/Components/SL-VertexProgramInputs.html

It uses Cg though. I tried fudging that together with the TerrainFourLayer shader but didn't get very far without knowing any Cg.

I found another shader that uses vertex colors and no Cg: BakedVertexColorBumpSpec2

Thank you for your help! I've been struggling with this few a few days, and getting nowhere.

[EDIT]Whoops! I seem to have completely overlooked that using 0,0,0,0 gives Tex 3! Nevermind all that. This ought to be good enough. Thank you yet again!

 

I'm not sure we were on the same page with your previous post...

With the fixed function pipeline, you don't have the ability to separate out individual channels, and use them as grayscale blending devices. Only the alpha channel has that ability. So your RGB vertex colors all need the same value if you don't want psychedelia. As such, I put the way to use your vertex colors into the name of the properties. (white RGB, white Alpha, and black Alpha)

So what you do first, is decide how much of Texture 1 you want in the blend. That's what RGB is. Then, you decide how much Texture 2 you want in the remaining portion, which is a blend between textures 2 and 3. That's the Alpha.

Texture 1 = (1,1,1,0), as you showed
Texture 2 = (0,0,0,1), as you showed
Texture 3 = (0,0,0,0), as you wrote.

Halfway Blends:
T1 + T2 = (.5, .5, .5, 1)
T1 + T3 = (.5, .5, .5, 0)
T2 + T3 = (0, 0, 0, .5)

Equal blend between all three = (1/3, 1/3, 1/3, .5)

(And yes, the result is really 0-255 if you have a 24-bit screen.)

What's nice about this method, is that you can never accidentally get an overbrightened version of the textures; your blending values will always add up to 1. That's not the case with the Cg/RGBA method. Still, that's a tradeoff that's worth the performance benefit of not needing another pass.

 

I was just confused since when I was trying to use CGPROGRAM I was able to access the components of a float4 using .x .y .z .w, or .r .g .b .a. I didn't know you couldn't do that like this.

There's more I want to add to this, so I'm sure by the time I'm done with it, it will be the shader from hell!

 

I'm in an identical situation: Terrain can be manipulated dynamically, so need to use vertex color as basis for texture masking.

I also tried out the TerrainFourLayer to no avail in Unity 4.

You mention that vertex-lit isn't possible simultaneous to vertex color-based operations. What about diffuse? Lighting is certainly necessary in my case, and shadows, too. Is this possible in a surface shader?

It seems to me that you're using 1110, 0001, and 0000 instead of R G and B (100, 010, and 001) so that the vertex color doesn't "infect" the texture. Would this remain a problem with a different lighting approach such as diffuse?

Does anyone know what sort of changes might be necessary to make this work with Unity 5's new physically-based lighting?