Resolution and anti-aliasing when reading alpha values from a compute buffer

Hello All,

I have a very unique use case and I hope some one here can help me out.

I coded a shader to display a single texture (much like the unlit texture shader).
It also contains a compute buffer which is an int array. At run time I feed an array of alpha values (1 - 255) to this buffer. For each pixel the alpha of the texture is read from the buffer.

The result if a portion of the texture being visible.

Here is the fragment function (whole code is at the bottom of the post)

Code (CSharp):

1.  
2. fixed4 frag (v2f i) : SV_Target
3. {
4. int colorWidth = (int)(i.uv.x * (float)640);
5. int colorHeight = (int)(i.uv.y * (float)480);
6. int colorIndex = (int)(colorWidth + colorHeight * (float)640);
7.  
8. fixed4 col = tex2D(_MainTex, i.uv);
9.  
10. // Get the alpha value from the buffer
11. col.a = alphaValues[colorIndex];
12.              
13. return col;
14. }
15.  

I am facing two issues (perhaps related):
1) The alpha values are sent with a fixed resolution of 640x480. When the shader is applied to a larder Quad (e.g. 1280 x 768) The resulting area is blown up; thus causing pixelation on the edges.

2) Even at the given resolution there is need for anti-aliasing on the edges. I could not find how to address this.

I'd appreciate any help pointing me to correct direction.
Cheers,
Doruk

Here is the full code:

Code (CSharp):

1. Shader "Unlit/CutOutShader"
2. {
3.     Properties
4.     {
5.         [NoScaleOffset] _MainTex ("Texture", 2D) = "white" {}
6.  
7.     }
8.     SubShader
9.     {
10.         Tags { "Queue"="Transparent" "RenderType"="Transparent" }
11.  
12.         ZWrite Off
13.         Blend SrcAlpha OneMinusSrcAlpha
14.  
15.         Pass
16.         {
17.             CGPROGRAM
18.             #pragma vertex vert
19.             #pragma fragment frag
20.  
21.             // vertex shader inputs
22.             struct appdata
23.             {
24.                 float4 vertex : POSITION; // vertex position
25.                 float2 uv : TEXCOORD0; // texture coordinate
26.             };
27.  
28.             // vertex shader outputs ("vertex to fragment")
29.             struct v2f
30.             {
31.                 float2 uv : TEXCOORD0; // texture coordinate
32.                 float4 vertex : SV_POSITION; // clip space position
33.             };
34.  
35.             // vertex shader
36.             v2f vert (appdata v)
37.             {
38.                 v2f o;
39.                 // transform position to clip space
40.                 // (multiply with model*view*projection matrix)
41.                 o.vertex = UnityObjectToClipPos(v.vertex);
42.                 // just pass the texture coordinate
43.                 o.uv = v.uv;
44.                 return o;
45.             }
46.          
47.             sampler2D _MainTex;
48.             StructuredBuffer<int> alphaValues;
49.  
50.  
51.             // pixel shader; returns low precision ("fixed4" type)
52.             fixed4 frag (v2f i) : SV_Target
53.             {
54.                 int colorWidth = (int)(i.uv.x * (float)640);
55.                 int colorHeight = (int)(i.uv.y * (float)480);
56.                 int colorIndex = (int)(colorWidth + colorHeight * (float)640);
57.  
58.                 fixed4 col = tex2D(_MainTex, i.uv);
59.  
60.                 // Get the alpha value from the buffer
61.                 col.a = alphaValues[colorIndex];
62.              
63.                 return col;
64.  
65.             }
66.             ENDCG
67.         }
68.     }
69. }

 

First question.
Why aren't you using a texture to hold your data? That gets you "free" bilinear sampling, and as long as you're using an Alpha8 texture format with no mip maps is going to be the same amount if data transfer between the CPU to the GPU.

If you want to stick to the structured array you'll need to do bilinear filtering on your own. Get the point you're "on" as well as the next nearest 3 points, calculate how close you each you are, and interpolate.

To make it look nicer you could do bicubic interpolation. That requires getting 16 values with the structured array, or 4 if you're using the texture approach.

If you need better anti-aliasing than the results you get with bicubic you can apply an FXAA-like pass or look at pixel art upscaling techniques. See this article for an example of using the later to anti-alias a low resolution fog of war texture.
https://engineering.riotgames.com/news/story-fog-and-war

 

Thank you very much for the reply.
One remark: That is the first time I am writing the shader myself. So I might be missing some obvious stuff.

The array updates around 25 times per second. I can feed it to a texture then use the texture functions to resize it. My assumption is that it would be very heavy to run this 25 times per second. Is that a correct assumption?

Not sure how the mip maps work though. Am I missing some obvious solution?

I will read thoroughly and try to apply. (if I cant get away with the "using texture" suggestion) Do you happen to have code to go with it? Not for copy/paste but for understanding how to apply it in code.

Cheers,
Doruk

 

Updating a texture vs updating the array is the same amount if data. A texture ends up just being an array passed to the GPU. That's why I said Alpha8 format and no mip maps as that's exactly the same single byte per pixel.

The main thing is you're not resizing the texture on the CPU, you're passing the same 307k bytes you would be before, but in the shader you only need to do tex2D(_AlphaTex, i.uv).a and you get a nicely bilinearly sampled value. Nothing else really need to be done. The "resizing" happens on the GPU as part of that bilinear sampling.


Mip maps aren't needed here at all. Those are only for when you plan on displaying the data at a lower resolution.

 

Thank you very much for your help.
My code ended up being as follows.

On the C# side I have this:

Code (CSharp):

1. using System.Collections;
2. using System.Collections.Generic;
3. using UnityEngine;
4. using System;
5.  
6. public class ByteArrayToShader : MonoBehaviour {
7.  
8.     public Material targetMaterial;
9.     private Texture2D alphaTexture;
10.  
11.     void Start(){
12.  
13.         alphaTexture = new Texture2D (640, 480, TextureFormat.Alpha8, false);
14.  
15.         targetMaterial.SetTexture("_AlphaTex", alphaTexture);
16.     }
17.  
18.     // This is called 25 times per second to pass the byte array to the shader.
19.     public void convertToTexture(byte[] _bytes){
20.         alphaTexture.LoadRawTextureData (_bytes);
21.         alphaTexture.Apply ();
22.     }
23. }

Do you think it is the correct way to pass the data to the shader?

And the shader:

Code (CSharp):

1.  
2. Shader "Unlit/CutOutShader"
3. {
4.     Properties
5.     {
6.         [NoScaleOffset] _MainTex ("Texture", 2D) = "white" {}
7.         _AlphaTex ("Texture", 2D) = "white" {}
8.     }
9.     SubShader
10.     {
11.         Tags { "Queue"="Transparent" "RenderType"="Transparent" }
12.  
13.         ZWrite Off
14.         Blend SrcAlpha OneMinusSrcAlpha
15.  
16.         Pass
17.         {
18.             CGPROGRAM
19.             #pragma vertex vert
20.             #pragma fragment frag
21.  
22.             // vertex shader inputs
23.             struct appdata
24.             {
25.                 float4 vertex : POSITION; // vertex position
26.                 float2 uv : TEXCOORD0; // texture coordinate
27.             };
28.  
29.             // vertex shader outputs ("vertex to fragment")
30.             struct v2f
31.             {
32.                 float2 uv : TEXCOORD0; // texture coordinate
33.                 float4 vertex : SV_POSITION; // clip space position
34.             };
35.  
36.             // vertex shader
37.             v2f vert (appdata v)
38.             {
39.                 v2f o;
40.                 // transform position to clip space
41.                 // (multiply with model*view*projection matrix)
42.                 o.vertex = UnityObjectToClipPos(v.vertex);
43.                 // just pass the texture coordinate
44.                 o.uv = v.uv;
45.                 return o;
46.             }
47.          
48.             // texture we will sample
49.             sampler2D _MainTex;
50.             sampler2D _AlphaTex;
51.  
52.             fixed4 frag (v2f i) : SV_Target
53.             {
54.  
55.                 fixed4 col = tex2D(_MainTex, i.uv);
56.                 col.a = tex2D(_AlphaTex, i.uv).a;
57.              
58.                 return col;
59.  
60.             }
61.             ENDCG
62.         }
63.     }
64. }

I will test the performance on the live system and add my remark on that later as well.

Thanks!
Doruk

 

That looks correct to me. I can't guarantee the format for Alpha8 is a straight byte value per pixel as I haven't tried it before (though it certainly seems like it should be). The slightly uglier (& slower) alternative would be to use SetPixels() or potentially SetPixels32() and an array of Color values.

 

Hello Again,

It took me some time to iterate and test on the real system. Hence the later post.
I used the above code from my previous post almost as-is. It worked and performed very good.

Thank you for the help.
Cheers,
Doruk