Billboarding a mesh using _WorldSpaceCameraPos

Hello!

I'm trying to modify a billboard shader I have for VR so that meshes are billboarded towards the camera's position instead of the direction the camera is facing, and I'm trying to understand what I'm doing wrong. I'm using the NO_ROTATE_Z def of this code:

Code (CSharp):

1. float4 CalcBillboardPos(float3 vertex, float xScale, float yScale)
2. {
3. #ifdef NO_ROTATE_Z
4.     const float3 local = float3(vertex.x, vertex.y, 0);
5.     const float3 offset = vertex.xyz - local;
6.  
7.     const float3 upVector = half3(0, 1, 0);
8.     const float3 forwardVector = _WorldSpaceCameraPos - mul(unity_ObjectToWorld, vertex); //UNITY_MATRIX_IT_MV[2].xyz;
9.     const float3 rightVector = normalize(cross(forwardVector, upVector));
10.     float3 position = 0;
11.     position += local.x * rightVector * xScale;
12.     position += local.y * upVector * yScale;
13.     position += local.z * forwardVector;
14.     float3 world = mul(UNITY_MATRIX_M, float4(0, 0, 0, 1));
15.     return mul(UNITY_MATRIX_VP, float4(world + position + offset, 1));
16. #else
17.     float3 viewSpace = UnityObjectToViewPos(float3(0.0, 0.0, 0.0));
18.     return mul(UNITY_MATRIX_P, float4(viewSpace.xyz + float3(vertex.x * xScale, vertex.y * yScale, 0), 1));
19. #endif
20. }
21.  

In my mind replacing UNITY_MATRIX_IT[2].xyz with _WorldSpaceCameraPos - mul(unity_ObjectToWorld, vertex) would make it so that forwardVector is now pointing towards the camera instead pointing the way the camera is facing, and that's sort of what happens, but only if the mesh is placed at 0,0,0 in the world.

​

In this example the center mesh would be at 0,0,0, and would properly billboard towards the camera's position, but the other two meshes on either side are billboarded in the same direction. I thought using mul(unity_ObjectToWorld, vertex) would account for the mesh's position but it doesn't seem like it. I've tried it with and without batching enabled, but that doesn't seem to make a difference.

I'm not very (or at all) familiar with shaders so it's entirely possible that I'm just not understanding what this code is doing. Any help would be appreciated! For reference this is what I'm trying to achieve:

​

 

The main problem is that you're multiplying local space vertex position with world matrix, and use THAT to calculate billboard vector. It will result in each vertex having different right/up vectors and will negate effect completely.

Don't do this:

Code (csharp):

1.  
2. _WorldSpaceCameraPos - mul(unity_ObjectToWorld, vertex);
3.  

You need to use model world position and not vertex world position. Right now you're using vertex world position. The model world position can be extracted directly from object world matrix (should be the last row), or you could keep using this, I suppose:

Code (csharp):

1.  
2. mul(UNITY_MATRIX_M, float4(0, 0, 0, 1));
3.  

In hopes the shader compiler optimizes the operation.

1. Calculate world position of the model. (extract it from UNITY_MATRIX_M or multiply)
2. Calculate "forward" vector (modelPos - cameraPos)
3. Calculate "right" using cross product between forward and (0, 1, 0)
4. RE calculate "up" using cross product between forward and right (otherwise objects below line of sight won't tilt)
5. This forms new rotation matrix of the model. with "Right" being xAxis, "up" being yAxis and "forward" being zAxis.
6. For world position use "world Position"(from step 1) + local.x * right + local.y * up + local.z * forward. This is a full equivalent of matrix transform.
7. Normals and tangetns need to be transformed into world space in the same way. "normal = origNormal.x * right + origNormal.y * up + origNormal.z * forward". This is also an equivalent of a rotation-only matrix transform.
8. Feed calculated world position into view/projection matrix.

By the way, this scheme will fail to work if you tilt head 90 degrees to the left or right. At this point the billboards will fail to calculate right vector properly.

To fix that instead of (0, 1, 0), use camera's "up" vector which can be extractd from UNITY_MATRIX_V which represents camera transform. Camera transform is an inverse of object transform (i.e. if you build a matrix from camera OBJECT, and inverse it, you'll get the matrix of the camera being used in the shader), and given that camera transform is rotation/position, you can extract vectors from COLUMNS of camera matrix, and "up" vector will be the second column. If you do that, even if you tilt camera, the billboards will remain vertical relative to the screen, which is how they usually work, but which may not be what you want, depending on circumstances.

 

Hey! Thank you so much for your help! I've got it working now.

One thing that was tripping me up was the 'camera position', since using _WorldSpaceCameraPos seems to take the position of each eye individually when in VR, which works when at a distance but up close causes problems. I eventually came across this resource which shows several Unity built-in shader variables that oddly aren't covered in the documentation? Specifically the one I needed was unity_StereoWorldSpaceCameraPos.

Anyway, thanks again for the help!