Combining/merging multiple alpha textures into a new one

I have a simple shader which has 2 alpha textures (_Alpha1 & 2) that work together to make the final alpha. I have another empty slot (_AlphaCombined) for an alpha texture. I would like to generate a new alpha texture and assign it to the empty slot, by combining the existing 2 alpha textures.

I have also looked at doing this via C# instead of in the shader, but there appears to be some difficulty in combining the alphas via SetPixel.

What would be the best way to do this? I would like to do something like _AlphaCombined = _Alpha1 * _Alpha2; but that of course doesn't work.

Here is the shader:

Code (CSharp):

1. Shader "Test Alpha Shader" {
2. Properties {
3.     _SpecColor ("Specular Color", Color) = (0, 0, 0, 1)
4.     _Shininess ("Shininess", Range (0.01, 1)) = 0.078125
5.     _Color ("Overall Color", Color) = (0.8,0.8,0.8,1)
6.     _FirstTex ("First Texture (RGB) TransGloss (A)", 2D) = "white" {}
7.     _AlphaCombined ("Transparency Combined", 2D) = "white" {}
8.     _Alpha1 ("Transparency 1", 2D) = "white" {}
9.     _Alpha2 ("Transparency 2", 2D) = "white" {}
10.     _Cutoff ("Alpha cutoff", Range(0,1)) = 0.1
11. }
12.  
13. SubShader {
14.     Tags {"Queue"="AlphaTest" "IgnoreProjector"="True" "RenderType"="TransparentCutout"}
15.     LOD 400
16.  
17.     // Alpha test depth only pass
18.         Pass {
19.             // default ZTest, here to make sure it's not overriden by the later one
20.             ZTest LEqual
21.             // only render to depth buffer
22.             ColorMask 0
23.             CGPROGRAM
24.             #pragma vertex vert
25.             #pragma fragment frag
26.             #include "UnityCG.cginc"
27.             struct v2f {
28.                 float4 pos : SV_POSITION;
29.                 float2 uv : TEXCOORD0;
30.             };
31.             sampler2D _AlphaCombined;
32.             sampler2D _Alpha1;
33.             sampler2D _Alpha2;
34.             float4 _MainTex_ST;
35.             fixed _Cutoff;
36.             fixed4 _Color;
37.             // super basic vertex shader
38.             void vert (appdata_base v, out v2f o)
39.             {
40.                 o.pos = UnityObjectToClipPos(v.vertex);
41.                 o.uv = TRANSFORM_TEX(v.texcoord, _AlphaCombined);
42.             }
43.             // frag shader with no return value! it only renders to depth so we don't need one!
44.             void frag(v2f i)
45.             {
46.                 half4 c = tex2D(_Alpha1, i.uv);
47.                 half4 d = tex2D(_Alpha2, i.uv);
48.                 //_AlphaCombined = c * d;  //What I would like to do
49.                 clip(tex2D(_AlphaCombined, i.uv).a * c.a * d.a - _Cutoff);
50.             }
51.             ENDCG
52.         }
53.  
54. FallBack "Legacy Shaders/Transparent/Cutout/Bumped Specular"
55. }

 

You cannot create new textures inside a shader. It can read existing textures, or it can render into a render texture as it’s final output. But if you want that texture to be read by a shader afterward you’ll need to use scripting to some extent. (Or use a grab pass, which if the goal here is to make things faster, you should not use.)

In what way? There shouldn’t be any issue with this. If you’re trying to multiply two

Color32

values together directly you’re going to have a bad time, but converting the alpha values to floats to multiply and back to a byte works fine, as does using

Color

instead.

The real question is are you trying to generate new content, or is this something you want to do at runtime?

 

Oh ok I had thought that perhaps this could be done via the legacy texture combiners, though I must admit I haven't used them yet.

I'll try the C# route otherwise but just wanted to check first if that's the best way since there is a bit involved. This is my understanding of what I would need to do:

1. GetPixels
2. SetPixels
3. SetTexture

If so there is just one part which I am not clear on, and that is how do I combine the alpha textures to get the desired result? For example in the shader I am doing this by multiplying them together:

clip(tex2D(_AlphaCombined, i.uv).a * c.a * d.a - _Cutoff);

How do I achieve this combining in C#?

 

Those have been deprecated for years now. Plus those old fixed function shaders work the same as modern shaders: you're just sampling two textures in the shader and multiplying them together. You're not creating a new texture.

Again, it depends a little on if you're looking to do this at runtime or at edit time. However the basics are create a new

Texture2D

with the size & format you want (must be an uncompressed format, so

Alpha8

or

RGBA32

for alpha). Call

GetPixels

or

GetPixels32

on the two existing textures to get their pixel color data. Iterate over the

Color

or

Color32

arrays and combine. Call

SetPixels

or

SetPixels32

on the new

Texture2D

to assign the colors and

Apply

to generate mip maps & upload to the GPU. Then call

SetTexture

on your material to assign the combined alpha texture. That material should not have the two alpha texture properties and

tex2D

calls, otherwise you're always sampling 3 textures which defeats the purpose of making the combined one.

With a

*

just like in the shader?

Code (csharp):

1. for (int i=0; i<alpha1Colors.Length; i++)
2.     newColors[i].a = alpha1Colors[i].a * alpha2Colors[i].a;

The reason to use

Get/SetColors32

vs

Get/SetColors

is the

Color32

version is a little faster because it avoids the conversion from the byte the colors are actually stored as to float. The difference is relatively minor though, and you'd need to do the conversion here for the multiply anyway. If this is something you're going to do every frame, you'd probably want to use the

Color32

version.

Code (csharp):

1. for (int i=0; i<alpha1Color32s.Length; i++)
2.     newColor32s[i].a = (alpha1Color32s[i].a / 255.0f) * (alpha2Color32s[i].a / 255.0f) * 255.0f;

 

Thank you very much. I have worked out pretty much everything except for this part. I am doing the following to get the pixels of the alpha textures and then trying to combine them:

Code (CSharp):

1.         skinnedMeshRenderer = GetComponent<SkinnedMeshRenderer>();
2.         skinnedMeshMaterial = skinnedMeshRenderer.material;
3.         skinnedMeshMaterial.EnableKeyword("_MainTex1");
4.         texture1= skinnedMeshMaterial.GetTexture("_MainTex1") as Texture2D;
5.         skinnedMeshMaterial.EnableKeyword("_MainTex2");
6.         texture2= skinnedMeshMaterial.GetTexture("_MainTex2") as Texture2D;
7.  
8.         //Get texture pixels & combine
9.         int x = Mathf.FloorToInt(sourceRect.x);
10.         int y = Mathf.FloorToInt(sourceRect.y);
11.         int width = Mathf.FloorToInt(sourceRect.width);
12.         int height = Mathf.FloorToInt(sourceRect.height);
13.         Color[] colors1 = texture1.GetPixels(x, y, width, height);
14.         Color[] colors2 = texture2.GetPixels(x, y, width, height);
15.         Color[] colorsCombined = colors1 * colors2;
16.  
17.         //Create new texture & set
18.         TransparencyCombinedTexture = new Texture2D(width, height);
19.         TransparencyCombinedTexture.SetPixels(colorsCombined);
20.         TransparencyCombinedTexture.Apply();
21.         skinnedMeshMaterial.EnableKeyword("_MainTex");
22.         skinnedMeshMaterial.SetTexture("_MainTex", TransparencyCombinedTexture);

Of course line 15 where I try to combine the colors is wrong, which is the part I am not sure how to do. This is executed in the Start function.

 

The bit of code I posted is almost literally the code you should have a line 15.

GetPixels

returns an array of values. In c# you can't just multiply two arrays together, you have to iterate over the individual elements of the array. However you also have to create an array before you can assign values to it, so you can't just stick my code in and have it work.

Code (csharp):

1. Color[] colors1 = texture1.GetPixels(x, y, width, height);
2. Color[] colors2 = texture2.GetPixels(x, y, width, height);
3. int numColors = colors1.Length;
4.  
5. // creates a new array the same length as colors1
6. Color[] colorsCombined = new Color[numColors];
7.  
8. // iterate over all elements of the arrays and multiply the color values in both color1 and color2
9. for (int i=0; i<numColors; i++)
10.     colorsCombined[i] = colors1[i] * colors2[i];

 

Ok sorry, thank you that helps a lot, I wasn't sure about that part.

Unfortunately I am getting an "Array size must be at least width*height" error on line 24. I suspect it is because I have not set the sourceRect variable correctly, since I have set the x & y to 0 which I'm not sure if it should be? The width and height are both 2048 which is the same as the texture.

Or am I missing something else?

Here is the code as it is now:

Code (CSharp):

1.         skinnedMeshRenderer = GetComponent<SkinnedMeshRenderer>();
2.         skinnedMeshMaterial = skinnedMeshRenderer.material;
3.         skinnedMeshMaterial.EnableKeyword("_MainTex1");
4.         texture1= skinnedMeshMaterial.GetTexture("_MainTex1") as Texture2D;
5.         skinnedMeshMaterial.EnableKeyword("_MainTex2");
6.         texture2= skinnedMeshMaterial.GetTexture("_MainTex2") as Texture2D;
7.  
8.         //Get texture pixels & combine
9.         int x = Mathf.FloorToInt(sourceRect.x);
10.         int y = Mathf.FloorToInt(sourceRect.y);
11.         int width = Mathf.FloorToInt(sourceRect.width);
12.         int height = Mathf.FloorToInt(sourceRect.height);
13.         Color[] colors1 = texture1.GetPixels(x, y, width, height);
14.         Color[] colors2 = texture2.GetPixels(x, y, width, height);
15.         int numColors = colors1.Length;
16.         // creates a new array the same length as colors1
17.         Color[] colorsCombined = new Color[numColors];
18.         // iterate over all elements of the arrays and multiply the color values in both color1 and color2
19.         for (int i=0; i<numColors; i++)
20.               colorsCombined[i] = colors1[i] * colors2[i];
21.  
22.         //Create new texture & set
23.         TransparencyCombinedTexture = new Texture2D(width, height);
24.         TransparencyCombinedTexture.SetPixels(colorsCombined);
25.         TransparencyCombinedTexture.Apply();
26.         skinnedMeshMaterial.EnableKeyword("_MainTex");
27.         skinnedMeshMaterial.SetTexture("_MainTex", TransparencyCombinedTexture);

 

Are the two source textures always the same resolution and do you always want to combine the full texture from both?

If so, there's no need to call that override of the

GetPixels

function. Just call

texture1.GetPixels();

which gives you the pixel values of the full texture.

 

Ok I worked out the issue. I was testing it with one of the textures not assigned since there will be times when one or both textures will need to not have any transparent parts. This was causing the color array to be 0 of course since there was no texture. I will need to make sure that if one of the texture don't need any transparent parts that I still put a fully white texture in the slot so that it doesn't cause the error.

Thank you for your help, really appreciate your time and patience

 

If one texture is missing maybe just skip the creation of a new one and just assign the existing texture.

 

Is such a C# loop the fastest way to combine two textures at runtime?

It's what I'm doing now, but this is running on Oculus Quest and it's causing a significant lag spike (hundreds of milliseconds) when we build the character by layering several skin textures and the clothing. And this happens whenever a player joins, i.e. in the middle of a match, so it's not good.

Is there any faster approach? I've thought about setting up a special camera, loading these textures onto a series of quads, and rendering to a RenderTexture. But that seems like a lot of work, which I'd prefer to avoid if there is any solution which is faster, easier, and/or more off-the-shelf.

EDIT: I've just discovered Graphics.Blit. Perhaps this is the solution?

 

Yes. Use Blit.

 

Yep. In this case I was able to refactor my code to take a RenderTexture rather than a Texture2D, and it was a clear win.

In other projects I sometimes have a need to both copy things from one texture to another (as fast as possible), and to do pixel-level manipulations. And in this case I seem to be in a bind, because Blit only draws into a RenderTexture, and RenderTextures don't have GetPixels or SetPixels.

Is there any way to get these two bits of functionality together? Some efficient way to convert a RenderTexture to a Texture2D, or vice versa, so that I can (at various times) use something like Blit, and something like Get/SetPixels, on the same pixel buffer?

 

Use a shader. A Blit is really just rendering a full quad mesh to a render texture with a built in shader that doesn't do much more than sample and output the input texture. You can supply a material with whatever shader and settings you want instead.

 

It took me a minute to understand your suggestion. But I think I've got it: you're saying, whatever pixel-level manipulations I'm currently doing now with Get/SetPixels, scrap that and instead do it with a custom shader.

But I haven't found any documentation on how to know, for example, the pixel location that shader is writing to. I suppose some experimentation could work it out, though.

Hmm. It might make a nice asset for somebody (with more time than me) to make, to just provide a suite of drawing methods that draw directly into a RenderTexture using this method. Draw line, draw ellipse, draw a Texture2D (or some portion thereof), etc. Similar to some of what my PixelSurface asset does, but leveraging Blit and the GPU.