How does "##" in shader #define actually work?

I am trying to use #define to make the texcoord number a parameter to pass in, and I am testing around with Unity's shader code and found the results very confusing. The results are as follows:

Code (CSharp):

1. //works
2. #define SHADOW_COORDS(idx1) float4 _ShadowCoord : TEXCOORD##idx1;
3.  
4. //Also works - Why??
5. #define SHADOW_COORDS(idx1) float4 _ShadowCoord : TEXCOORD##idx1idx1;
6. #define SHADOW_COORDS(idx1) float4 _ShadowCoord : TEXCOORDidx1;
7. #define SHADOW_COORDS(idx1) float4 _ShadowCoord : TEXCOORD#a;
8. #define SHADOW_COORDS(idx1) float4 _ShadowCoord : TEXCOORDa;

How does the syntax actually work?? Where can I learn about ShaderLab's syntax?

 

Nothing to do with ShaderLab, this is purely an HLSL compiler thing.
https://docs.microsoft.com/en-us/windows/win32/direct3dhlsl/dx-graphics-hlsl-appendix-pre-define-2

HLSL macros are modeled after c preprocessor macros. The

##

operator is a concatenation operator, intended to merge two strings into a single variable name.
https://gcc.gnu.org/onlinedocs/cpp/Concatenation.html

Now, as for why the above variations all work, that has almost nothing to do with the macros, but rather because the official Microsoft HLSL compiler will ignore any semantics that aren’t valid and magically replace them with working ones.

 

what about merging string with passed value?

Code (CSharp):

1. #define MERGE_TEX_NAME(tex, num) tex##num
2.  
3. int i = 5;
4. MERGE_TEX_NAME(_Texture, i)

this returns "_Texturei", but I would need it to return "_Texture5"

The idea is to have an index as variable passed to the shader (reason is not needed).

 

You can't dynamically get texture names like that. Even if it were possible to do that as a macro, it still wouldn't compile. You are better off just passing the right texture to the shader, not the name of the texture to the shader. Or you could use a texture array if you want dynamic indexing, but that might not be within your constraints.

 

Thanks joshuacwilde,

I was using texture arrays, but then we need to make texture MIP streaming and crunch compressing enabled and sadly, Texture2DArray does not support either. So I am looking for another solution. It is part of my custom mesh combine system, and with multiple textures we can use both, are not restricted by resolution per array etc.

Another option is to use if-else statement, but due to shaders compute both branches, this can possibly lead to 100 unnecessary texture samples.

In my current solution I have per combined chunk material with bunch of textures and one int4 with indices, but now I need to somehow connect it together to sample correct texture with as low overhead as possible.

 

Not possible. Macros are a pre-process, meaning they run before any code is evaluated. The macro has no idea that

i

is a variable with a value of 5, it just sees it as the character

i

. Also because pre-processes happen before the code is run, the passed in values can't be dynamic. They must be hard coded in the shader itself.

Then, as @joshuacwilde noted, texture names can't be dynamic*. And those texture references must be bound to the shader when it runs on the GPU or the GPU will crash!

For handling some arbitrary number of textures in a shader, there are usually two ways to go about this.

One is to have the shader only handle some small fixed number of textures, and run the shader multiple times. For example, if your shader only handles 8 textures at a time, and you have 22 textures to go through, you run the shader 3 times and have it output into the same render target, or ping pong between two render targets.

The other is to have a couple of variants setup with some number of max textures in each variant. For example have variants that handle 32, 16, 8, or 4 textures, and run which ever variant can handle all of the textures you need.

Sometimes both of these techniques are used together.

* DX12 and Vulkan allow for bindless textures, which allows you to grab an arbitrary texture from a list of textures passed to the GPU. But Unity still uses DX11 style texture binding so this isn't something that can be used yet.

 

Shaders don't always compute branches. They run in chunks of 32 or 64 threads at once. If all 64 threads want to take one branch, then it just computes that branch. If 63 of the threads want to take one branch and 1 of the threads wants to take the other branch, then it needs to compute both branches. For compute shaders, you have some control over how these threads are grouped together; for pixel shaders it's done by screen location (often 8x4 or 8x8 blocks of pixels).

In the case of texture fetches in a pixel shader, there's one other thing that can make it compute both sides. It's super random, not documented well, and even if you know all the parts involved, it still might be surprising:

Code (CSharp):

1.  
2. float2 uv = IN.uv
3.  
4. // this might issue a sample even if the branch is coherent
5. if(splat.x > .04f) {
6.     albedo += SAMPLE_TEXTURE2D(_Texture0, sLinearWrap, uv);
7. }
8.  
9. // this will force it to branch
10. float2 dpdx = ddx(uv), dpdy = ddy(uv); // need to calculate these OUTSIDE of the branch
11. UNITY_BRANCH if(splat.x > .04f) {
12.     albedo += SAMPLE_TEXTURE2D_GRAD(_Texture0, sLinearWrap, uv, dpdx, dpdy);
13. }

Note also the

UNITY_BRANCH

before the

if

which will give you an error if it can't branch, which can aid in debugging.

This is one of those strange esoteric knowledge things that you won't find in most shader tutorials or even whole courses on shader programming.

EDIT: More info https://interplayoflight.wordpress.com/2014/03/25/branches-and-texture-sampling/

 

Thank you @joshuacwilde @bgolus @burningmime, I've created a new thread to continue, as this no longer relates to the original Q.

https://forum.unity.com/threads/dynamic-texture-indexing.1387200/