Creating a shader that takes a texture made out of many small images

So what I'm trying to do is load satellite images from an SQL table and wrap them around a sphere to create a globe. As far as I can tell, this process involves loading the images into a Texture2DArray and applying this new mega-image to the material's main texture. But it doesn't seem like Unity's standard shader is cooperating with this TextureArray. Here's the shader code I've written so far:

Code (CSharp):

1. Shader "Custom/Tiling"
2. {
3.     Properties
4.     {
5.         _MainTex("Texture", 2DArray) = "white" {}
6.     }
7.         SubShader
8.     {
9.         Tags { "RenderType" = "Opaque" }
10.         Lighting Off
11.         ZWrite Off
12.         Pass
13.         {
14.             CGPROGRAM
15.             #pragma vertex vert
16.             #pragma fragment frag
17.             #pragma target 5.0
18.             #include "UnityCG.cginc"
19.  
20.             struct appdata
21.             {
22.                 float4 vertex : POSITION;
23.                 float3 uv : TEXCOORD0;
24.                 float4 color:COLOR;
25.             };
26.  
27.             struct v2f
28.             {
29.                 float3 uv : TEXCOORD0;
30.                 float4 vertex : SV_POSITION;
31.                 float4 color:COLOR;
32.             };
33.  
34.             UNITY_DECLARE_TEX2DARRAY(_MainTex);
35.  
36.             float4 _MainTex_ST;
37.             int _Index;
38.             SamplerState sampler_linear_repeat;
39.             float lightlevel;
40.             v2f vert(appdata v)
41.             {
42.                 v2f o;
43.                 o.vertex = UnityObjectToClipPos(v.vertex);
44.                 o.uv.xy = v.uv.xy;
45.                 o.uv.z = v.uv.z;
46.                 o.color = v.color;
47.                 return o;
48.             }
49.  
50.             //I found this code online that's supposed to separate the images into an appropriate grid, but it just makes a white texture.
51.             int _COLUMNS = 50;
52.             int _ROWS = 50;
53.             half4 frag(v2f i) : SV_Target
54.             {
55.                 float3 uv = float3(i.uv.x % _COLUMNS, i.uv.y % _ROWS, 0);
56.                 uv.z = floor(i.uv.x / _COLUMNS) + _COLUMNS * floor(i.uv.y / _ROWS);
57.                 return UNITY_SAMPLE_TEX2DARRAY(_MainTex, uv);
58.             }
59.             ENDCG
60.         }
61.     }
62. }
63.  

Any shader experts out there that can help a poor fool out?

 

Check your Texture2DArray creation code. Not in shader, but in the C# script you are using to then send the texture array to this shader. Usually people forget to use the Apply() method after SetPixels() on the texture array slices, which prevents the texture array from being created correctly. If this isn't it just let me know, I have a shader based tile map renderer which basically does what you want so I'll take a look at my code to see if I can spot what might be missing in yours, as I also remember experiencing this issue with the texture array not working properly.

By the way, since we are mapping square tiles around a sphere there is a lot of distortion near the poles so your images will need to match this distortion in order to properly project on the sphere.

 

Thanks for the help. I know I'm not missing the apply method, so I'm not quite sure where my issue is. Here's my tiler class.

Code (CSharp):

1. MapTiler tiler = new MapTiler("Assets/viewing-ready_s2cloudless-2018_4326_v1.0.1_sample_l0-4.sqlite");
2.     public Material mat;
3.     Texture2DArray texArr;
4.     Texture2D[] textures;
5.     int size = 2048;
6.     int count = 0;
7.     // Start is called before the first frame update
8.     void Start()
9.     {
10.         textures = new Texture2D[size];
11.             for (int x = 0; x <= 31; x++) //The x and y values in their tables go up to 31 and 15 respectively
12.             {
13.                 for (int y = 0; y <= 15; y++)
14.                 {
15.                     textures[count] = tiler.Read(x, y, 1); //The z determines the zoom level, so I wouldn't want them all loaded at once
16.                     Debug.Log(count);
17.                     if (textures[count] != null) TextureScale.Bilinear(textures[count], 256, 256);
18.                     count++;
19.                 }
20.             }
21.  
22.      
23.         texArr = new Texture2DArray(256, 256, textures.Length, TextureFormat.RGBA32, true, true);
24.         texArr.filterMode = FilterMode.Bilinear;
25.         texArr.wrapMode = TextureWrapMode.Repeat;
26.         for (int i = 0; i < textures.Length; i++)
27.         {
28.             if (textures[i] == null) continue;
29.             texArr.SetPixels(textures[i].GetPixels(0), i, 0);
30.         }  
31.         texArr.Apply();
32.         mat.SetTexture("_MainTex", texArr);
33.     }

Here's the Read function that I use to actually get the images

Code (CSharp):

1. public Texture2D Read(int x, int y, int z)
2.     {
3.         SqliteCommand cmd = new SqliteCommand(m_dbConnection)
4.         {
5.             CommandText = String.Format(
6.                 "SELECT data FROM tiles where tileset = 's2cloudless-2018' AND grid = 'WGS84' AND x = {0} AND y = {1} and z = {2}",
7.                 x, y, z)
8.         };
9.         Texture2D thing = null;
10.         byte[] data = (byte[])cmd.ExecuteScalar();
11.         if (data != null)
12.         {
13.             if (data.Length > 5)
14.             {
15.                 // Copy JPEG data to Texture
16.                 thing = new Texture2D(x, y);
17.                 thing.LoadImage(data);          
18.             }
19.          
20.          
21.         }
22.         return thing;
23.     }

 

If I were you I would first check if that image fetching code works fine perhaps by logging the color data of a pixel in console. If you get pure white from this code you will know immediately that way. Another thing to do would be to manually create color textures in script instead of fetching and trying to render that as a tile. Start by eliminating that part of your program from being the possible cause of the issue.

Once you are sure this is a shader problem, you could see if it works when you start from the example code at the bottom of this page, it could be some part of your shader is incompatible with the arrays in some way. That's all I can think of for now I hope it helps!

 

Alright, so I think I determined that my images are loading correctly. After applying the shader code that you linked, my globe looks like this:
Which actually is substantial progress, so thank you.
Any idea how I can edit that shader code to un-jumble the images?

 

Great! Now, the shader code I linked contains some vertex shader based coordinate generation which I'm quite unfamiliar with. Maybe you need to play with the UVScale and Slices parameters? You could also ditch that and instead return to transforming UV coordinates in the fragment shader. Just grab the Texture2DArray related code from the working shader and put them into your original shader as is, texture name and everything, to see if it solves the original issue of white textures with your own code.

However the uv.z part of your first shader code is most likely not properly creating a set of grid cell coordinates over your texture. UV coordinates go from 0 to 1 and you are flooring the result of a division from a number within that range, which will always either return 0 or 1. You most likely need to scale the UV up by the amount of grid cells you want, to first generate a grid cell coordinates that can return say 0 to 10 as values.

Also, remember that the coordinates you finally use to sample the array should be going from 0 to 1 otherwise you will only draw part of your textures, but instead of doing this over the whole image it needs to happen over the size of your grid cell. I think you can use the fractional component of the scaled UV grid cell values you generated BEFORE the flooring ( because that always removes the fractional part ) to accomplish this. It might cause some odd seams that you'll need to play around with to fix.

 

OK, here's where we're at right now:

The texture size and images change if we change the Row and Column values, like so:
I did my best to combine aspects of both shaders and follow your advice, and this is what I came up with:

Code (CSharp):

1. Properties
2.     {
3.         _MainTexArray("Tex", 2DArray) = "" {}
4.         _SliceRange("Slices", Range(0,32)) = 6
5.         _UVScale("UVScale", Float) = 1
6.         _COLUMNS("Columns", Range(0, 1)) = 1
7.         _ROWS("Rows", Range(0, 1)) = 1
8.     }
9.         SubShader
10.         {
11.             Pass
12.             {
13.                 CGPROGRAM
14.                 #pragma vertex vert
15.                 #pragma fragment frag
16.                 // texture arrays are not available everywhere,
17.                 // only compile shader on platforms where they are
18.                 #pragma require 2darray
19.  
20.                 #include "UnityCG.cginc"
21.  
22.                 struct v2f
23.                 {
24.                     float3 uv : TEXCOORD0;
25.                     float4 vertex : SV_POSITION;
26.                 };
27.  
28.                 float _SliceRange;
29.                 float _UVScale;
30.  
31.                 v2f vert(float4 vertex : POSITION)
32.                 {
33.                     v2f o;
34.                     o.vertex = UnityObjectToClipPos(vertex);
35.                     o.uv.xy = (vertex.xy + 0.5) * _UVScale;
36.                     o.uv.z = (vertex.z + 0.5) * _SliceRange;
37.                     return o;
38.                 }
39.                 float _COLUMNS; //Columns and rows only go between 0 and 1
40.                 float _ROWS;
41.                 UNITY_DECLARE_TEX2DARRAY(_MainTexArray);    
42.                 half4 frag(v2f i) : SV_Target
43.                 {
44.                     float3 uv = float3(i.uv.x % _COLUMNS, i.uv.y % _ROWS, 0);
45.                     uv /= 16; //Create fractional component before floor
46.                     uv.z = floor(i.uv.x / _COLUMNS) + _COLUMNS * floor(i.uv.y / _ROWS);
47.                     uv *= 16;//UV is multiplied by number of tiles I expect
48.                     return UNITY_SAMPLE_TEX2DARRAY(_MainTexArray, uv);
49.                 }
50.                 ENDCG
51.             }
52.         }

Can you tell me if there's anything obviously wrong? I may have been confused on how to "use the fractional component of the scaled UV grid cell values you generated BEFORE the flooring."
Thanks again for your help. Sorry, I'm still somewhat inexperienced with shaders.

 

Sorry I'm really not sure what's wrong with it, I'm also not the best with shader programming but went through a similar issue recently so I thought I could give out some general tips. I also realized that the modulo thing done to UV coordinates might be doing the "scaling" part of the process on its own.

Basically, all you want to do here is split a space that goes from 0 to 1 into equally sized tiles and for each of those tiles you need to be able to gather a unique value in your texture array that ends up placing textures in the right order. When you originally get your UV parameter in the fragment shader it goes from 0 to 1, so if you multiply this coordinate value by say 20 you basically get a texture made up of 20x20 sections that go from 0 to 1.

The texture array still requires you to input coordinates from 0 to 1 in order to correctly fetch the whole texture. Going beyond this range will likely induce some odd behavior. In HLSL we get the frac() function, which takes in a floating point number and will return only the decimal part. Basically, when your scaled uv.x value is of 10.5, the frac() function would return .5. This works to create a "local" UV space for each cell in your grid that go from 0 to 1 and allow correct texture array fetching.

I don't know if that's what you need to do. I'm just throwing some information on how I personally did it out there so it can hopefully give you the right idea of what has to be done. Here is a screen shot of the code I generally use to partition UV space into a grid, you can see the distinct colors of each cell. The important bit is at the top. The multiplication is to essentially duplicate UV space a bunch of times and the floor() is to make sure values returned by this function are whole numbers representing the X and Y indices of the cell the input UV coordinates currently reside within. And by using a cellid.x + cellid.y * columns operation you can easily get the correct single row array index of the texture you should be drawing at the current cell position.

​

That's all I can really think of, hopefully this helps you out a bit!