Resolved Stuck trying to copy textures into bigger textures

I'm prototyping some tools to let users customize each face of a die individually with an image.

This means that for an eight-sided die I'd have eight smaller textures that I want to put in specific places in a combined texture to match the UV map created for the die models. (I plan to support many types of dice)

The below example I made by hand as proof of concept:

Calculating the offsets for each texture isn't a problem, but actually combining them is.

I've been experimenting with Blitting into RenderTextures, but have also had my eye on generating texture atlases. I haven't used these methods before though and I've been finding it hard to find good information or code examples for them. I keep getting stuck on errors about mip levels and graphics formats.

That doesn't bode well for allowing user input.
I don't care about optimization, but it does have to be tolerant to different formats, mip levels, compression and other texture settings. So far everything I've seen seems very fragile and single-purpose.

Am I best off creating shaders with a texture parameter per side of the dice and doing all the combination work there? Or should texture atlases or blitting into render textures be able to do this somehow?

 

Have you experimented with using a Texture2D and myTexture.SetPixels? I generate spritesheets at runtime with this approach. Tex.Compress & Tex.Apply(updateMipmaps: true)

 

Yeah, I started by using SetPixels, but it kept complaining that the textures I was copying from and to weren't the same size. You don't have that problem?

If not, it's possible I was misunderstanding the error, I'll give it another shot. Thanks.

 

Check the dimensions of the texture and be sure you're pasting in subsections of your source texture(s) into the generated texture correctly. My sprites in the spritesheet are all manner of sizes and shapes so although it may take a bit of difficulty to figure out what the API wants from you I can assure you that it is possible.

 

I would use a RenderTexture over fiddling with sprite sheets. You don't need any special shaders; the built-in Unlit/Transparent is enough. Blit your base texture then if you have the bounds for each side, use the GL immediate mode API to draw quads in the right place.

You are working in normalized texture space so textures will be sampled/interpolated at the correct mip-level regardless of the size of the input image.

Then use ReadPixels to get the pixel data back into a Texture2D, and then Compress it. Runtime compression is limited to DXT in 2020, but supports ETC for mobile platforms in 2021.

 

Thanks guys, I tried three things, and they all allowed me to write to the texture (must have been inputting some garbage before), but each one of them has it's own downsides, apparently:

SetPixels
Calling SetPixels once for each of the smaller textures does allow me to copy them at the locations I want in the bigger texture, BUT it doesn't alpha blend, which I do need.


SetPixel
I can iterate through each pixel of the smaller textures, do my own alpha blending and apply the resulting colours. It's inelegant and kind of slow, but I could live with it if I need to.


RenderTexture
I'm not sure I understand how to use it.
While I don't get any errors while Blitting the smaller textures into the big one, it seems like it will only ever retain the last texture I blitted into it. And not alpha blend.

If I use the scale and offset I used in SetPixel(s):


By reducing the scale and removing the offset I can get the last applied texture to be correct, but none of the others show up:

I'm using

Code (CSharp):

1. Graphics.Blit(faceTexture, rendTex, scale, offSet);

 

By default the shader used for Blit just overwrites the values with new ones with an opaque blend. This means the alpha isn't ignored, it's still being written, it just isn't being used to do a blend.

To use blit to do compositing of multiple textures you need to use a different shader. Preferably one using an alpha blend. The built in Unlit/Transparent for example should work. To use a different shader than the default one, you have to use the

Blit()

method that takes a material, and thus you need to create a material with that shader and use that. There's no form that takes a material and a scale & offset, but that's okay. The scale and offset values where being used to set the

material.mainTextureScale

and

material.mainTextureOffset

properties on an internal hidden material. So in your code you'd instead be setting those values yourself, like this:

Code (csharp):

1. blitMat.mainTextureScale = scale;
2. blitMat.mainTextureOffset = offset;
3. Graphics.Blit(faceTexture, rendTex, blitMat);

Minor follow up: you might wonder why the default blit is an opaque "copy". The reasons is because most of the time if you're calling

Blit()

without a custom shader it is to just copy a texture from one to another. If you want a different behavior you'd use a custom shader. The most common use case for

Blit()

is for post processing, in which case a custom shader, or potentially several, are guaranteed.

 

That sounds promising. I´ll try that once I can get back to it

 

Finally got back to it and I really don't understand why I can't get it to work. :/

The code sets the material to the scale and offset that I want, but the blitting doesn't seem to use it at all, but writes every texture on top of each other at the same scale, so I only see the final one.




I create the render texture and blit in the background image

Code (CSharp):

1.         RenderTextureDescriptor desc = new RenderTextureDescriptor(targetDimensions, targetDimensions);
2.         desc.useMipMap = true;
3.         desc.autoGenerateMips = false;
4.         desc.depthBufferBits = 2;
5.         desc.colorFormat = RenderTextureFormat.ARGB32;
6.         desc.enableRandomWrite = true;
7.  
8.         RenderTexture rendTex = new RenderTexture(desc);
9.         rendTex.Create();
10.         Graphics.Blit(backGroundTexture, rendTex, blitMat);

(I've tried with and without most of the settings above, thought I'd include them here for posterity)

Then inside a loop that goes through all the face textures I blit each one individually at a certain scale and offset:

Code (CSharp):

1.             Vector2 scale = new Vector2(1f / (float)textureSubdivisions, 1f / (float)textureSubdivisions);
2.             Vector2 offSet = new Vector2(centeredX, centeredY);
3.  
4.             blitMat.mainTextureScale = scale;
5.             blitMat.mainTextureOffset = offSet;
6.  
7.             Graphics.Blit(faceTexture, rendTex, blitMat);

And finally read from the render texture into the final composited texture:

Code (CSharp):

1.      
2.         rendTex.GenerateMips();
3.         targetTexture.ReadPixels(new Rect(0,0,rendTex.width, rendTex.height), 0, 0);
4.         targetTexture.Apply();
5.  

 

The material Offset is in a normalized UV range of 0.0 to 1.0, not pixels.

 

This won't break anything, but there's no point in having a render texture with mip maps if you're going to copy it back to the CPU side texture.

ReadPixels()

only grabs the top mip level, so you'll need to recalculate the mip maps on the

Texture2D

anyway. The

ReadPixels()

function has an option to do just that.

I also wanted to go over some of your render texture settings real quick.

desc.useMipMap = true;
desc.autoGenerateMips = false;
As mentioned above, you do not need this, unless you plan on using the render texture directly on the object's material. In which case this, along with the

GenerateMips()

call are good ideas.

desc.depthBufferBits = 2;
The only valid values are 0, 16, 24, or 32. Anything else I would expect to be throwing an error, or at least a warning. But in your case where you're just doing blits, you want 0, as you don't want or need a depth buffer. The depth buffer exists only for rendering opaque 3D geometry to make sure the surfaces sort properly per pixel.

desc.enableRandomWrite = true;
This is mainly for compute shaders, or other very specific shader setups which you are not doing and is unnecessary. The short version is it allows a compute shader or fragment shader to write to any random pixel in the render texture from the shader code, rather than as part of the shader's output. Since you're not doing that, don't include this setting.

 

Thank you.
That's could have been clearer in the documentation, for sure. Changed it to UV range.

A lot of those rendertexture settings I had been toying around with to see if they had any effects, but I've set them to what you recommend, just to prevent them from causing any problems.

However, the final texture still doesn't respect the offset nor the scaling, but just has the last blitted texture fill the entire space.

So I get this:


Whereas what I'd expect is this:

 

Played around with this locally. The behavior of

Blit()

with a material seems to change every few years in subtle ways, and the current behavior appears to be that the values of the

_MainTex_ST

(the vector that the

mainTextureScale

and

mainTextureOffset

actually get passed to the shader as) always gets stomped on by the

Blit()

.

Basically this means you have to use a custom shader and cannot use the built in scale and offset values.

I also led you astray a little bit with my comments about the offset. It is still in normalized UV space, but it's in the UV space scaled by the texture scale. So for what you're trying to do, which is blit a grid of images, the offsets are in whole integers for how many scaled tile widths you want. And you want them to be negative values.

Here's the shader I quickly threw together along with the test script.

Code (csharp):

1. Shader "Blit AlphaBlend"
2. {
3.     Properties {
4.         [NoScaleOffset] _MainTex ("", 2D) = "white" {}
5.         _MainTex_ScaleOffset ("Scale (xy) Offset (zw)", Vector) = (1,1,0,0)
6.     }
7.     SubShader {
8.         Pass {
9.             Blend SrcAlpha OneMinusSrcAlpha
10.             ZWrite Off
11.             ZTest Always
12.             Cull Off
13.  
14.             CGPROGRAM
15.             #pragma vertex vert
16.             #pragma fragment frag
17.  
18.             #include "UnityCG.cginc"
19.  
20.             struct v2f {
21.                 float4 pos : SV_Position;
22.                 float2 uv : TEXCOORD0;
23.             };
24.  
25.             sampler2D _MainTex;
26.             float4 _MainTex_ScaleOffset;
27.  
28.             v2f vert(appdata_img v) {
29.                 v2f o;
30.  
31.                 o.pos = UnityObjectToClipPos(v.vertex);
32.                 // basically the same thing as the main texture scale and offset values would have done
33.                 o.uv = v.texcoord * _MainTex_ScaleOffset.xy + _MainTex_ScaleOffset.zw;
34.  
35.                 return o;
36.             }
37.  
38.             half4 frag(v2f i) : SV_Target {
39.                 return tex2D(_MainTex, i.uv);
40.             }
41.             ENDCG
42.         }
43.     }
44. }

Code (csharp):

1. using System.Collections;
2. using System.Collections.Generic;
3. using UnityEngine;
4.  
5. public class DiceBlit : MonoBehaviour
6. {
7.     public Vector2Int resolution = new Vector2Int(256, 256);
8.     public Texture2D background;
9.     public Texture2D[] sides;
10.  
11.     public Material blitMat;
12.  
13.     public RenderTexture rt;
14.  
15.     [ContextMenu("Do It")]
16.     public void GenerateTexture()
17.     {
18.         if (blitMat == null)
19.         {
20.             blitMat = new Material(Shader.Find("Blit AlphaBlend"));
21.         }
22.  
23.         bool newRT = false;
24.         if (rt == null || rt.width != resolution.x || rt.height != resolution.y )
25.         {
26.             if (rt != null)
27.                 DestroyImmediate(rt);
28.  
29.             rt = new RenderTexture(resolution.x, resolution.y, 0, RenderTextureFormat.ARGB32);
30.             newRT = true;
31.         }
32.  
33.         if (background != null)
34.         {
35.             Graphics.Blit(background, rt);
36.         }
37.  
38.         int numSideTextures = sides.Length;
39.         int gridDim = Mathf.CeilToInt(Mathf.Sqrt((float)numSideTextures));
40.  
41.         Vector2 scale = Vector2.one * (float)gridDim;
42.         Vector2 offsetStep = -Vector2.one / (float)gridDim;
43.         int index = 0;
44.         for (int y=0; y<gridDim; y++)
45.         {
46.             for (int x=0; x<gridDim; x++)
47.             {
48.                 index = x + y * gridDim;
49.  
50.                 if (index >= numSideTextures)
51.                     break;
52.  
53.                 Vector4 scaleOffset = new Vector4(scale.x, scale.y, -(float)x, -(float)y);
54.                 blitMat.SetVector("_MainTex_ScaleOffset", scaleOffset);
55.                
56.                 Graphics.Blit(sides[index], rt, blitMat);
57.             }
58.  
59.             if (index >= numSideTextures)
60.                 break;
61.         }
62.  
63.         if (newRT)
64.         {
65.             newRT = false;
66.             var rend = GetComponent<Renderer>();
67.             if (rend != null)
68.             {
69.                 var matBlock = new MaterialPropertyBlock();
70.                 matBlock.SetTexture("_MainTex", rt);
71.                 rend.SetPropertyBlock(matBlock);
72.             }
73.         }
74.     }
75.  
76.     void Update()
77.     {
78.         // updates the render texture every frame for testing purposes
79.         // don't do this for real!
80.         GenerateTexture();
81.     }
82. }

And here's the output. Yes the numbers are in the "wrong" order, because the UV 0,0 starts at the bottom left. You'd have to do a little extra math to offset so it starts at the top, which I'm too lazy to do.

 

Thank you so much for doing that.
It was extremely helpful, and there's no way I could have figured this out for myself.

I'm surprised how complicated the solution is. Isn't this a fairly common task?

I updated my code to work the same way, (+the up/down math) and I finally have something that makes sense.


The one remaining issue is if there's any non-transparent pixels on the edges of one of the textures it seems like the blitting smears them all over the combined texture. (if I clamp the textures, but repeating or mirroring them isn't exactly an improvement)

If my face textures look like this:


Then when the script has finished running, the line on the fifth face will smear up and down like this:


But if I use SetPixel it comes out like this:


Now obviously I don't need the fifth face to look like it does here, but since there's texture seams between the faces I'll sometimes want to hide them by putting a solid colour around the edges of each face.

 

Yeah, my test example I use a 2 pixel wide transparent border around each of those textures.

It can be fixed by adding some code in the blit shader's frag function.

I think this will do it.

Code (csharp):

1. clip(0.5 - abs(i.uv - 0.5));

 

Hah! Clever.
That did the trick indeed.

Finally, a fully blitted combined texture with no artifacts.



That was a lot more painful than expected.

Thanks again for all your help, it has been very generous of you to take the time for it.

 

@bgolus thank you that you exist man!