Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Question Dynamic texture indexing

Discussion in 'Shaders' started by sewy, Jan 16, 2023.

  1. sewy

    sewy

    Joined:
    Oct 11, 2015
    Posts:
    150
    Continuation of https://forum.unity.com/threads/how-does-in-shader-define-actually-work.1010953/

    TLDR; So my goal is to switch from Texture2DArray to series of Texture2D to support mip streaming and crunch compression = Idealy I need to emulate array of Texture2Ds

    I have succesfully implemented Mesh combining as an offline alternative to a static batching to lighten CPU bottleneck.
    1. Lets say, the scene is devided into many chunks.
    2. Each chunk has single material
    3. While combining, each vertex in a chunk is containing indices into Texture2DArray (Albedo, Spec, Normal), which is created by copying material textures into respective array.
    4. In a fragment shader, I can sample specific Texture2DArray slice according to vertex index.
    This was good enough, but because it is a copy of the texture it takes additional memory, doesn't support mip streaming and chunk compression, textures in array have to share resolution etc.

    Due to this, I've decided to switch to series of Texture2D, but having trouble with indexing.
    1. While combining, each vertex in a chunk is containing index into list of Texture2D (which are then assigned into multiple shader Textures)
    2. In a shader, I need to decide, which Texture should be sampled according to given index. This if where the trouble begins.
    I'm looking for more optimal solution as in my shader below, because with getTexture(), which would be optimal, I get compiler crush, and with sample(), there are many unnecessary texture samples.
    • As guru @bgolus mentioned here I could make optimisations using multi_compile variants, but that still leads to many unnecessary per pixel sampler.
    • Another option would be to choose texture in vertex shader, but I cannot pass Texture2D reference from vertex to fragment (or can I?)
    I believe in some elegant solution, as everything is known at offline (at build time). Idealy in vertex shader (flat shaded VR)
    Note that in example below there are only 10 textures, but this is only an example (sample limit is 64 and texture limit is 128).

    Code (CSharp):
    1. Shader "Custom/MeshCombine-Texture2D"
    2. {
    3.     SubShader
    4.     {
    5.         Tags { "Queue"="Geometry" }
    6.  
    7.         Pass
    8.         {
    9.             CGPROGRAM
    10.                 #pragma target 3.0
    11.                 #pragma vertex vert
    12.                 #pragma fragment frag
    13.                 #include "UnityCG.cginc"
    14.              
    15.                 UNITY_DECLARE_TEX2D(_Texture1);
    16.                 UNITY_DECLARE_TEX2D_NOSAMPLER(_Texture2  );
    17.                 UNITY_DECLARE_TEX2D_NOSAMPLER(_Texture3  );
    18.                 UNITY_DECLARE_TEX2D_NOSAMPLER(_Texture4  );
    19.                 UNITY_DECLARE_TEX2D_NOSAMPLER(_Texture5  );
    20.                 UNITY_DECLARE_TEX2D_NOSAMPLER(_Texture6  );
    21.                 UNITY_DECLARE_TEX2D_NOSAMPLER(_Texture7  );
    22.                 UNITY_DECLARE_TEX2D_NOSAMPLER(_Texture8  );
    23.                 UNITY_DECLARE_TEX2D_NOSAMPLER(_Texture9  );
    24.                 UNITY_DECLARE_TEX2D_NOSAMPLER(_Texture10 );
    25.  
    26.                                            
    27.                 struct appdata              
    28.                 {                          
    29.                     float4 vertex : POSITION;
    30.                     float4 texcoord : TEXCOORD0; // UVs
    31.  
    32.                     float4 color : COLOR; // Albedo Color
    33.                     float4 texcoord1 : TEXCOORD1; // Texture indices - x = albedo, y = specular, z = normal, w = emission
    34.                 };
    35.  
    36.                 struct v2f
    37.                 {
    38.                     float4 pos : SV_POSITION;
    39.                     float4 albedoColor : COLOR;
    40.                     float2 uv : TEXCOORD0;
    41.                     int4 indices : TEXCOORD1;
    42.                 };
    43.              
    44.                 v2f vert (appdata v)
    45.                 {
    46.                     v2f o;
    47.                     UNITY_INITIALIZE_OUTPUT(v2f, o);
    48.                              
    49.                     o.pos = UnityObjectToClipPos(v.vertex);
    50.                     o.uv = v.texcoord;
    51.                     o.albedoColor = v.color;
    52.                     o.indices = v.texcoord1;
    53.  
    54.                     return o;
    55.                 }
    56.  
    57.                 Texture2D getTexture(int index)
    58.                 {
    59.                     UNITY_BRANCH
    60.                     if(index > 9)
    61.                         return _Texture10;
    62.                     else if(index > 8)
    63.                         return _Texture9;
    64.                     else if(index > 7)
    65.                         return _Texture8;
    66.                     else if(index > 6)
    67.                         return _Texture7;
    68.                     else if(index > 5)
    69.                         return _Texture6;
    70.                     else if(index > 4)
    71.                         return _Texture5;
    72.                     else if(index > 3)
    73.                         return _Texture4;
    74.                     else if(index > 2)
    75.                         return _Texture3;
    76.                     else if(index > 1)
    77.                         return _Texture2;
    78.  
    79.                     return _Texture1;
    80.                 }
    81.  
    82.                 float4 sample(float2 uv, int index)
    83.                 {
    84.                     UNITY_BRANCH
    85.                     if(index > 9)
    86.                         return UNITY_SAMPLE_TEX2D_SAMPLER(_Texture10, _Texture1, uv);
    87.                     else if(index > 8)
    88.                         return UNITY_SAMPLE_TEX2D_SAMPLER(_Texture9, _Texture1, uv);
    89.                     else if(index > 7)
    90.                         return UNITY_SAMPLE_TEX2D_SAMPLER(_Texture8, _Texture1, uv);
    91.                     else if(index > 6)
    92.                         return UNITY_SAMPLE_TEX2D_SAMPLER(_Texture7, _Texture1, uv);
    93.                     else if(index > 5)
    94.                         return UNITY_SAMPLE_TEX2D_SAMPLER(_Texture6, _Texture1, uv);
    95.                     else if(index > 4)
    96.                         return UNITY_SAMPLE_TEX2D_SAMPLER(_Texture5, _Texture1, uv);
    97.                     else if(index > 3)
    98.                         return UNITY_SAMPLE_TEX2D_SAMPLER(_Texture4, _Texture1, uv);
    99.                     else if(index > 2)
    100.                         return UNITY_SAMPLE_TEX2D_SAMPLER(_Texture3, _Texture1, uv);
    101.                     else if(index > 1)
    102.                         return UNITY_SAMPLE_TEX2D_SAMPLER(_Texture2, _Texture1, uv);
    103.  
    104.                     return UNITY_SAMPLE_TEX2D_SAMPLER(_Texture1, _Texture1, uv);
    105.                 }
    106.  
    107.                 float4 frag (v2f i) : SV_Target
    108.                 {        
    109.                     float4 albedoColor = i.albedoColor;
    110.                     albedoColor *= sample(i.uv, i.indices.x);
    111.  
    112.                     // This leads to:
    113.                     // Shader Compiler IPC Exception: Terminating shader compiler process
    114.                     // and:
    115.                     // Internal error communicating with the shader compiler process.
    116.                     //albedoColor *= UNITY_SAMPLE_TEX2D_SAMPLER(getTexture(i.indices.x), _Texture1, i.uv);
    117.  
    118.                     return albedoColor;
    119.                 }
    120.  
    121.             ENDCG    
    122.         }
    123.     }
    124. }
     
    Last edited: Jan 16, 2023
  2. joshuacwilde

    joshuacwilde

    Joined:
    Feb 4, 2018
    Posts:
    725
    What I have done before is to use a texture atlas array (just a texture array, but each texture is a big 4K texture [for example] and each texture is itself a texture atlas) then stream textures into those atlases. That way you get the best of both worlds.

    I have tried that method before of lots of if statements, but it just wasn't very performant. That was on mobile.
     
  3. sewy

    sewy

    Joined:
    Oct 11, 2015
    Posts:
    150
    Texture atlas would work with streaming and crunching, but it is also duplication of the original texture (which in my case can be used by other non-combined meshes, as I only combine object smaller than 10m). Another point is, that some of my textures are already 4k, so as I could make atlasses up to 16k, but that would introduce another problems with the memory.

    What do you mean by "stream textures into atlasses"? If they are part of Texture2DArray, they cannot be streamed.
     
  4. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    12,329
    Unless you’re combining the textures into an atlas or array, you’re not going to get any savings combining the meshes. Yes, you’ll get fewer draw calls which will save on CPU cost, but you’re going to make the draws themselves much more expensive. When you have long lists of if statements like that (or any other way of choosing between multiple Texture2D samples in a shader), and the index for which to sample from is coming from the vertex info and not a material property, the GPU is generally just going to sample all of the textures all of the time.

    The real solution to this kind of thing is atlas everything. If a texture is being used on a mesh that isn’t being combined, you’ll still want to update that mesh’s UVs so it can use the texture atlas version.
     
  5. sewy

    sewy

    Joined:
    Oct 11, 2015
    Posts:
    150
    Reason for combining was the CPU bottleneck, so moving the work to GPU was the point. Acording to profiler, it was caused by static batching (for dozens of small objects, which are now combined).

    Is there a possibility to move the indexToTexture into vertex shader e.g. choose a texture from index and pass the reference to pixel?

    Some of our textures are already 4k so to make 16k atlas, we can combine only 16 of those textures - so there still will be texure size(count) limit. Also mip streaming will not be that performant. Atlasses would need to be per scene, not to duplicate textures, which will again introduce said problem of multiple textures per shader.

    As our world is static and baked, everything we do is known before playmode (e.g. offline). I have the textures and indices, is it realy that impossible to combine the two together? Array of textures is what I need, but Texture2DArray is unfortunately not the same.
     
  6. burningmime

    burningmime

    Joined:
    Jan 25, 2014
    Posts:
    845
    There are options between a single mega-mesh and tons of small meshes. For example, I have this (from the Japanese School Art Room on the Asset Store) which I made into a prefab. It has tons of GameObjects, some of which are very small:

    upload_2023-1-17_6-22-35.png

    After merging, it looks like this. There's a single mesh for all opaque geometry, with 3 submeshes:

    upload_2023-1-17_6-22-25.png

    The submeshes are divided based on the texture atlas they sample from

    upload_2023-1-17_6-40-21.png

    And the atlases themselves are 2048x2048 or 4096x4096:

    upload_2023-1-17_6-39-39.png

    So instead of ~900 draw calls for the original objects, it's 3 draw calls. Which is more than one, but it doesn't require any special shader at all.

    I can share my code that merges the meshes/textures if it would help, although it's very messy and dependent on some internal stuff. I've been meaning to pull it together and open source it, but I don't want to step on the toes of the Mesh Baker folks, since that does something similar.
     
    sewy likes this.
  7. Invertex

    Invertex

    Joined:
    Nov 7, 2013
    Posts:
    1,539
    Why not use the Sparse textures (mega texturing) approach then to avoid dupes?
     
  8. sewy

    sewy

    Joined:
    Oct 11, 2015
    Posts:
    150
    That is the way to try it for now, I am already doing this for SkinnedMeshes to avoid multiple CPU skinning instructions (mesh itself is skinned on GPU).

    Didn't know that those exists, it seems like better alternative to atlassing. Unfortunately according to docs
    Sparse textures only support non-compressed texture formats.
     
  9. Invertex

    Invertex

    Joined:
    Nov 7, 2013
    Posts:
    1,539
    Well how many textures do you need and what is the target platform? A 16k sparse texture would be 792MB of VRAM. And sections of that are getting swapped out for the textures needing to be viewed currently.

    Unity does have experimental Virtual Texturing though, compatible with older hardware and similar to the techniques other games (like RAGE) used.
    https://docs.unity3d.com/2022.2/Documentation/Manual/svt-streaming-virtual-texturing.html
    https://docs.unity3d.com/2022.2/Documentation/Manual/svt-use-in-shader-graph.html
    With Virtual Texturing the textures should still be compressed.
    But there are drawbacks in workflow and ability to use other features that you wouldn't have with individual textures per-material, like losing the ability to repeat texture or use addressables.

    Just ensuring your shaders are SRP batchable by bundling the per-mat properties in the CBBUFFER and just going with separate textures may be the most ideal here.
     
    Last edited: Jan 18, 2023
  10. burningmime

    burningmime

    Joined:
    Jan 25, 2014
    Posts:
    845
    Here's my mesh baking thing for URP. I copied it with all its dependencies into a single file, so should be easy to use.

    Create a blank scene, choose a prefab GameObject in the asset window, pick it from the menu, and it will work its magic. If you're using Builtin, just change the SHADER_NAME to "Standard" (I think that's it, could be wrong). HDRP will require you to tweak all your material parameter names.

    The main gotcha It requires that your mask maps already be packed (R=metalness, G=occlusion, A=smoothness). If the green channel of your metallic/gloss maps is empty, then everything will show up black because it assumes that most textures have occlusion.
     

    Attached Files: