Texture sampling using derivatives in vertex program

Has anyone had any luck sampling from a texture using derivatives in a vertex program?
I know this is possible in openGL using texture2DGrad and I have had this working on my graphics card before (not in Unity though)
Im guessing cg would support this as well.

For example

Code (csharp):

1.     float2 uv_dx = ddx(uv.xy);
2.         float2 uv_dy = ddy(uv.xy);
3.     float4 texsample = tex2D( _MainTex, uv, uv_dx, uv_dy );

This code works in the frag program but not in the vert program.

I dont actually need the ddx and ddy functions as I can calculate them manually so no big deal if they are not supported in a Unity vert program but I still need the tex2D function taking the derivatives.

Is only the tex2Dlod function supported in Unity vert programs?

 

Yeah, for vertex texture fetch you need to use tex2Dlod.

 

I know I posted this a while ago but I was just looking into it again and found a solution.

While this is something few people would ever need to do there is very little information around on the topic so I thought I would make this post to provide the solution. It may save someone a lot of time.

Basically this issue is as follows...

In a fragment program you would sample a texture like so.

Code (csharp):

1. float4 texel = tex2D(sampler, uv);

But what the GPU is really doing is this...

Code (csharp):

1. float4 texel = tex2D(sampler, uv, dx, dy);

Where dx and dy are the derivatives of the uv's and are used by the GPU to select what mip map level to use. This is extremely important to reduce aliasing issues.

You can of course provide your own derivatives as the tex2D functions is overloaded to do so (look here to see how).

Now the issue I had is that I needed to do this in the vertex shader. As there are some limitations and performance issues in doing texture look ups in vertex shaders the only provided function to do so in Unity is tex2Dlod.

This allows you to manually select what mip map level to use but if I had the derivatives how do I convert these to the mip map level?

The answer is like so...

Code (csharp):

1. float4 Texture2DGrad(sampler2D tex, float2 uv, float2 dx, float2 dy, float2 texSize)
2. {
3.     //Sampling a texture by derivatives in unsupported in vert shaders in Unity but if you
4.     //can manually calculate the derivates you can reproduce its effect using tex2Dlod
5.     float2 px = texSize.x * dx;
6.     float2 py = texSize.y * dy;
7.     float lod = 0.5 * log2(max(dot(px, px), dot(py, py)));
8.     return tex2Dlod(tex, float4(uv, 0, lod));
9. }

Where size is the textures dimensions (width, height).

This allows you to sample a texture in a vert shader in a equivalent manner to the tex2D function in cg or the texture2DGrad function in glsl as long as you can manually determine the derivatives.

 

It's not so much that automatic derivatives are not supported in a vertex shader in Unity.

These derivatives are calculated by taking the difference with neighbouring pixels and this is only possible because the GPU will always render at least a 2x2 grid of pixels. In the vertex shader there is no per pixel difference available, since the pixels are yet to be rasterized. So that's why the ddx and ddy instructions and all instructions that rely on it are not available.

So, it has nothing to do with Unity. It's also not really unsupported. The whole ddx instruction just makes no sense in a vertex shader. Which in term means that tex2D makes no sense in a vertex shader. This might seem a bit weird at first, but knowing how graphics cards work, it is actually logical that only tex2Dlod is available in vertex shaders.

 

My situation was that I did not need the ddx and ddy functions in the vert shader as I had a way to manually determine the derivatives. All I needed was a was to convert the derivatives to a lod level.

Some shader languages (like glsl) do have texture look ups beside tex2Dlod in the vert shader (like texture2DGrad). The catch is that you need to provide the function the derivatives yourself.

 

HLSL indeed only supports tex2Dgrad in the pixel shader. With GLSL the texture2DGrad instruction is added later and the documentation doesn't really mention whether it works in the vertex shader or not.

I'm guessing that anisotropic filtering is not supported from a vertex shader and microsoft didn't want to create false hope by letting you supply a ddx and ddy. With just one lod parameter it is clear that anisotropic filtering is not possible.