Question Multiple Textures Overlaying A Base Texture (Character Shader)

Hello I'm new to shader writing and want some advise on how the best / faster way to render this would be.

My character has 11 textures. That includes 1 base texture at 512x512 and 10 texture to overlay the base texture:

Base Texture at 512x512


Here, 10 smaller images are overlaid on top of the base texture. While not visible here, some of these smaller overlays may be transparent PNGs... for example pants with holes in the knees.


These 10 smaller images will never overlay each other... they are broken down piece by piece to fit perfectly into the 512x512 area.


Therefore, the only "blending" that needs to be done is the Base Texture with each of the 10 overlays. If the overlay is not transparent, the base texture should not be visible.

I guess my question is probably broken down into two parts:
- How to render the smaller images at exact locations? (128 pixels down, 64 pixels to the right)
- How do I blend each small texture with the base texture, but not with each other?

And overall, what's the best/fastest method to render this character in a single pass?

 

I've broken this forum post down into 2 different Unity Answer questions:

- How to modify XY position of texture?
https://answers.unity.com/questions/1800292/shader-modify-xy-position-of-texture.html

- How to render a transparent texture over a base texture?
https://answers.unity.com/questions/1800296/shader-1-base-texture-overlaid-by-1-transparent-te.html

 

The best way to do this is to not use a shader. Or, perhaps more specifically, not to use the shader on the character to draw all of the textures.

Instead the way to approach this is you want to merge those textures into a single texture before applying it to your character. There are several ways to approach this, but the fastest and most direct approach is to use

CopyTexture()

to merge several textures of the same format together. This only works if all of the textures are the same GPU texture format, the regions you're copying are opaque, and the regions are grid aligned to a 4x4 grid, as that's how most GPU texture formats are stored. You mentioned you need to support optional transparent overlays, so that approach can't be used for that. The next best approach is to

Blit()

all of your textures into a render texture one by one, and then either use the resulting render texture as the character's texture. Or you can copy the render texture back to a

Texture2D

and, if you're not targeting mobile platforms, compress the results using run time texture compression.

Now to your specific questions:

In a shader, you would scale & offset the UV position. UVs are a 0.0 to 1.0 range for the x and y so that 0.0, 0.0 is the bottom left corner and 1.0, 1.0 is the top right of a texture, regardless of the resolution or aspect ratio of the texture. So if you're looking to position a 256x64 texture inside a 512x512 texture, you'd need to scale it by (0.5, 0.125), aka (256/512, 64/512), to match the scale of the 512x512 texture, and offset it by (0.25, 0.75) to get it 64 to the right and 128 "down" (actually 384 up from the bottom). Most Unity shaders have a scale and offset value for texture properties, which is what you could be using. This is the same technique you could use for

Blit()

, as the scale and offset values that function takes just applies to the same scale and offset in the default internal blit shader. The

Blit()

function just renders a "full screen" quad to the current render target with whatever material & texture(s) you tell it to use. In this case the render target would be a render texture that's the same dimensions as the base texture, which you've "blitted" in first before doing the subsequent textures.

However there remains the issue of the fact textures by default tile or repeat outside of the 0.0 to 1.0 range, so if you were to just render a texture over another, the area outside of the scaled and offset 0.0 to 1.0 range would just be more of that texture rather than showing the original. To solve that for a shader, you'd have to test for UV values outside of the 0.0 to 1.0 range and skip those. For

Blit()

you could also use a custom shader that calls

clip()

outside the UV range. There's also fun stuff like using a viewport or scissor rect to confine the area you want to draw to, which would probably work better here.

For

CopyTexture()

you'd be using pixel integer values, again in multiples of 4 if you're using GPU friendly compressed texture formats. Note, the format of the source texture on disc is irrelevant. If it is a texture that's in the asset folder, it's being converted into some other GPU friendly format, and by default that's a compressed format. If you read the original image file from a resources folder it'll be loaded as an uncompressed image. GPUs do not support PNG, JPG, or any of the formats you're likely familiar with.

In a shader that's on an object, using the

lerp()

function. Sample the base texture, sample the transparent texture, then do:

Code (csharp):

1. half3 newColor = lerp(base.rgb, transparent.rgb, transparent.a);

With a

Blit()

using a custom transparent shader. As I mentioned above, this is just rendering a "full screen" quad to a render target. In this case it'd be rendering that quad with a transparent blend.

Here's some sample code that uses something like the

Blit()

approach mentioned above:

Spoiler: Hidden-BlitTransparent.shader

Code (csharp):

1. Shader "Hidden/BlitTransparent" {
2.     Properties
3.     {
4.         _MainTex ("Texture", any) = "" {}
5.     }
6.     SubShader {
7.         Pass {
8.             ZTest Always Cull Off ZWrite Off
9.             Blend SrcAlpha OneMinusSrcAlpha, One OneMinusSrcAlpha
10.  
11.             CGPROGRAM
12.             #pragma vertex vert
13.             #pragma fragment frag
14.             #include "UnityCG.cginc"
15.  
16.             UNITY_DECLARE_SCREENSPACE_TEXTURE(_MainTex);
17.             uniform float4 _MainTex_ST;
18.  
19.             struct appdata_t {
20.                 float4 vertex : POSITION;
21.                 float2 texcoord : TEXCOORD0;
22.                 UNITY_VERTEX_INPUT_INSTANCE_ID
23.             };
24.  
25.             struct v2f {
26.                 float4 vertex : SV_POSITION;
27.                 float2 texcoord : TEXCOORD0;
28.                 UNITY_VERTEX_OUTPUT_STEREO
29.             };
30.  
31.             v2f vert (appdata_t v)
32.             {
33.                 v2f o;
34.                 UNITY_SETUP_INSTANCE_ID(v);
35.                 UNITY_INITIALIZE_VERTEX_OUTPUT_STEREO(o);
36.                 o.vertex = UnityObjectToClipPos(v.vertex);
37.                 o.texcoord = TRANSFORM_TEX(v.texcoord.xy, _MainTex);
38.                 return o;
39.             }
40.  
41.             fixed4 frag (v2f i) : SV_Target
42.             {
43.                 UNITY_SETUP_STEREO_EYE_INDEX_POST_VERTEX(i);
44.                 return UNITY_SAMPLE_SCREENSPACE_TEXTURE(_MainTex, i.texcoord);
45.             }
46.             ENDCG
47.  
48.         }
49.     }
50.     Fallback Off
51. }

Spoiler: CharacterTextureSplatter.cs

Code (CSharp):

1. using System;
2. using System.Collections;
3. using UnityEngine;
4. using UnityEngine.Rendering;
5.  
6. public class CharacterTextureSplatter : MonoBehaviour
7. {
8.     // base texture
9.     [SerializeField] Texture2D baseTex;
10.  
11.     // material for doing the transparent blit
12.     [SerializeField] Material blitMat;
13.  
14.     // the texture layers, with an offset from the bottom left
15.     [Serializable]
16.     public class TextureSplat {
17.         public Texture2D tex;
18.         public Vector2Int offset;
19.     }
20.     [SerializeField] TextureSplat[] splats;
21.  
22.     // just for debug
23.     [SerializeField] Texture2D generatedTex;
24.  
25.     Texture2D GenerateCharacterTexture(Texture2D baseTex, TextureSplat[] splats)
26.     {
27.         if (splats == null || splats.Length == 0)
28.             return null;
29.  
30.         if (blitMat == null)
31.             blitMat = new Material(Shader.Find("Hidden/BlitTransparent"));
32.  
33.         // create render texture based off of base texture
34.         // same width and height, no depth buffer
35.         // ARGB32 format, since that’s what basically any imported texture is going to be
36.         RenderTexture tempRT = RenderTexture.GetTemporary(baseTex.width, baseTex.height, 0, RenderTextureFormat.ARGB32, RenderTextureReadWrite.sRGB);
37.  
38.         // blit base texture to render texture to fill it in
39.         Graphics.Blit(baseTex, tempRT);
40.  
41.         // set things up to do a manual blit
42.         // below is recreating what Blit() already does, but lets us set the viewport
43.  
44.         // copy off the active render target
45.         RenderTexture currentActiveRT = RenderTexture.active;
46.  
47.         // set the temp rt to be the active render target
48.         RenderTexture.active = tempRT;
49.  
50.         // stores the current rendering matrices for later
51.         GL.PushMatrix();
52.  
53.         // set the projection to orthographic
54.         GL.LoadOrtho();
55.  
56.         // iterate through splat textures
57.         for (int i=0; i<splats.Length; i++)
58.         {
59.             if (splats[i].tex == null)
60.                 continue;
61.  
62.             // set the current splat texture on the material, and set it as the current material to use for rendering
63.             blitMat.SetTexture("_MainTex", splats[i].tex);
64.             blitMat.SetPass(0);
65.  
66.             // magic!
67.             // sets the pixel area we're rendering to within the render target
68.             // basically lets us set a pixel accurate offset and size
69.             GL.Viewport(new Rect(splats[i].offset.x, splats[i].offset.y, splats[i].tex.width, splats[i].tex.height));
70.  
71.             // draw a full screen triangle
72.             // Blit() uses a full screen quad, but this is slightly faster, and less code
73.             // the viewport confines it to a rectangular area
74.             GL.Begin(GL.TRIANGLES);
75.             GL.TexCoord2(0, 0);
76.             GL.Vertex3(0, 0, 0.1f);
77.  
78.             GL.TexCoord2(2, 0);
79.             GL.Vertex3(2, 0, 0.1f);
80.  
81.             GL.TexCoord2(0, 2);
82.             GL.Vertex3(0, 2, 0.1f);
83.             GL.End();
84.         }
85.  
86.         // remove the reference to the last splat texture in case we want to unload it
87.         blitMat.SetTexture("_MainTex", null);
88.  
89.         // restore the original matrices
90.         GL.PopMatrix();
91.  
92.         // a real Blit() would set the active target back now, but we need it set to the tempRT for a little longer
93.  
94.         // create a new Texture2D to copy the contents of the render texture to
95.         // same width and height as the baseTex
96.         // RGB24 instead of RGBA32 since we presumably don't need the alpha
97.         // has mip maps
98.         // not linear (aka is a sRGB color texture)
99.         Texture2D newBase = new Texture2D(baseTex.width, baseTex.height, TextureFormat.RGB24, true, false);
100.  
101.         // could maybe reuse the existing texture if a generated one already exists, but we'll skip that for now
102.  
103.         // get the contents of the render texture and copy them to the new texture
104.         newBase.ReadPixels(new Rect(0, 0, baseTex.width, baseTex.height), 0, 0, true);
105.  
106.         // we don't need the tempRT anymore, so set the active render target back and release the tempRT
107.         RenderTexture.active = currentActiveRT;
108.         RenderTexture.ReleaseTemporary(tempRT);
109.  
110.         // compress the texture
111.         // uses high quality mode, but if the source textures are already DXT1 or DXT5, there are going to be artifacts
112.         // kind of like compressing a low quality jpg a second time
113.         // you can skip this if you can afford uncompressed textures, or you're on mobile since you can't do runtime compression
114.         newBase.Compress(true);
115.  
116.         // do this instead if you don't want to or can't do the runtime compression
117.         // newBase.Apply();
118.  
119.         // returns the new texture with all layers applied
120.         // use this on your character's material
121.         return newBase;
122.     }
123.  
124.     // for testing the above code
125.     [ContextMenu("Generate Texture")]
126.     void GenerateTexture()
127.     {
128.         if (generatedTex != null)
129.             DestroyImmediate(generatedTex);
130.  
131.         generatedTex = GenerateCharacterTexture(baseTex, splats);
132.     }
133. }

This setup probably isn't exactly how you'd want to do it, like obviously something else should call the

GenerateCharacterTexture

with the list of textures and offsets, but it's self contained for simplicity.

 

This needs to be nominated for Best Forum Post Replies of 2020. Thank you very much!

 

There we go