Question Finding transform's screen space position using a different field of view.

I have a first person setup, in which the world's field of view is dynamic while the first person model is static, so that the user can tweak the FOV to their liking while keeping the arms unchanged.

However as a result, spawning something (eg: a projectile) in world space in front of a muzzle can lead to undesired results, like the projectile looking disconnected from the gun.

The solution i've been thinking of would be to find where on the near clipping plane the muzzle is rendered. If i now draw a line that goes through the camera's position and the coordinate on the near clipping plane and extend it as far as the distance between the camera and muzzle, i should now find the exact point in which to spawn my projectile. Should the two FOVs be identical, the line should end up in the same position of the muzzle.


The issue i'm facing is that

Camera.WorldToViewportPoint(muzzle.position)

doesn't seem to take into account the different field of views.
Here for example i'm attempting to draw a gizmo around a cube (which has fixed fov) and the camera doesn't match:

Is there any way to essentially say:
World FOV is 100, first person FOV is 60.
Transform has position A, resulting in coordinates (X,Y) of the near clipping plane, if i take the camera's position, then move in the direction of the coordinates found by the magnitude of A, i'll find point B, which is the muzzle's position if the camera had a FOV of 100.
However, i want to find a point C, which is the muzzle's position if the camera had a FOV of 60.
Which would look akin to this (the cyan cube looks correctly in the same place as the red one in camera space):

I've spent the day trying to figure this out but my brain doesn't deal with quaternions and transformation matrixes very well (or at all).

Thank you all in advance. ^^

 

Are bumps a thing? ^^
I've made a small gif to further showcase the issue, my goal is to have the gizmo match the red cube's position. The issue occurs when i change the field of view of the arm using the RenderObjects' camera override.

 

Additional information, this should be the code that is tweaking the field of view of the arm/cube:

Code (CSharp):

1. Matrix4x4 projectionMatrix = Matrix4x4.Perspective(m_CameraSettings.cameraFieldOfView, cameraAspect,
2.                             camera.nearClipPlane, camera.farClipPlane);
3. projectionMatrix = GL.GetGPUProjectionMatrix(projectionMatrix, cameraData.IsCameraProjectionMatrixFlipped());
4.  
5. Matrix4x4 viewMatrix = cameraData.GetViewMatrix();
6. Vector4 cameraTranslation = viewMatrix.GetColumn(3);
7. viewMatrix.SetColumn(3, cameraTranslation + m_CameraSettings.offset);
8.  
9. RenderingUtils.SetViewAndProjectionMatrices(cmd, viewMatrix, projectionMatrix, false);

Taken from here: https://github.com/Unity-Technologi...universal/Runtime/Passes/RenderObjectsPass.cs

What i'm trying to do is:
I have a transform t (P1 in the first image i attached).
1. Use Camera.WorldToScreenPoint (or WorldToViewportPoint, whichever is handier) to find the coordinate (x,y) that tells me where t is on the screen. This would be P1'
2. Figure out where (x,y) would be if the field of view was different (eg: 100 instead of 60), finding P2' (this is where i'm stuck)
3. Extend a line from the camera and through P2' to find P2

I'm really not sure how i'm supposed to do this or where else i can ask for help.
If anyone knows the answer to either, please let me know!

 

Hi iv2b,

I have a similar problem. Did you figure out a way to get the worldpoint which takes the fov into account? Or do you have another solution altogether? Or still unsolved?

 

Hey, i completely forgot i asked here, thank you for reminding me.
This is the function i ended up making (with support from Wokarol from Brackeys' server), that worked (it only draws gizmos):

Code (CSharp):

1.     void OnDrawGizmos()
2.     {
3.         Gizmos.color = Color.yellow;
4.  
5.         // draw line and sphere towards object, real fov
6.         Gizmos.DrawWireSphere(target.position, 0.02f);
7.         Gizmos.DrawLine(target.position, c.transform.position);
8.  
9.         Gizmos.color = Color.red;
10.  
11.         float distance = c.nearClipPlane;
12.         float halfFOV = c.fieldOfView * 0.5f * Mathf.Deg2Rad;
13.         float aspect = c.aspect;
14.  
15.         float height = Mathf.Tan(halfFOV) * distance;
16.         float width = height * aspect;
17.  
18.         // draw small red spheres for screen perimeter and center
19.         Gizmos.DrawWireSphere(c.transform.position + c.transform.forward * distance + c.transform.right * width, 0.015f);
20.         Gizmos.DrawWireSphere(c.transform.position + c.transform.forward * distance - c.transform.right * width, 0.015f);
21.         Gizmos.DrawWireSphere(c.transform.position + c.transform.forward * distance + c.transform.up * height, 0.015f);
22.         Gizmos.DrawWireSphere(c.transform.position + c.transform.forward * distance - c.transform.up * height, 0.015f);
23.         Gizmos.DrawWireSphere(c.transform.position + c.transform.forward * distance, 0.015f);
24.  
25.         Vector3 targetViewportPoint = c.WorldToViewportPoint(target.position);
26.         targetViewportPoint.x = targetViewportPoint.x * 2f - 1f;
27.         targetViewportPoint.y = targetViewportPoint.y * 2f - 1f;
28.  
29.         // draw red sphere, real fov, on near clip plane
30.         Gizmos.DrawWireSphere(c.transform.position + c.transform.forward * distance + c.transform.right * width * targetViewportPoint.x + c.transform.up * height * targetViewportPoint.y, 0.02f);
31.  
32.         ////// this is the magic sauce
33.  
34.         Matrix4x4 normalToScreen = Matrix4x4.Perspective(targetFov, c.aspect, c.nearClipPlane, c.farClipPlane) * c.worldToCameraMatrix;
35.         Matrix4x4 modifiedToScreen = Matrix4x4.Perspective(c.fieldOfView, c.aspect, c.nearClipPlane, c.farClipPlane) * c.worldToCameraMatrix;
36.  
37.         Matrix4x4 m = Matrix4x4.Inverse(modifiedToScreen) * normalToScreen;
38.  
39.         Vector3 pos = m.MultiplyPoint(target.position);
40.         Gizmos.color = Color.magenta;
41.        
42.         // draw line and sphere towards object, desired fov
43.         Gizmos.DrawLine(c.transform.position, pos);
44.         Gizmos.DrawWireSphere(pos, 0.02f);
45.  
46.         targetViewportPoint = c.WorldToViewportPoint(pos);
47.         targetViewportPoint.x = targetViewportPoint.x * 2f - 1f;
48.         targetViewportPoint.y = targetViewportPoint.y * 2f - 1f;
49.        
50.         // draw white sphere, desired fov, on near clip plane
51.         Gizmos.color = Color.white;
52.         Gizmos.DrawWireSphere(c.transform.position + c.transform.forward * distance + c.transform.right * width * targetViewportPoint.x + c.transform.up * height * targetViewportPoint.y, 0.02f);
53.  
54.     }

This is the end result:

Camera fov set to 120, desired fov set to 60, the purple line shows the position in world space with the desired fov, keeping the same distance from the camera, while the white sphere shows the point on the near clip plane of the desired position.

Now if i change my camera fov to my desired fov (60), this is the result:

 

Also, this is the end result of using this logic alongside render pass shenanigans:

In both animations, the hand grips the object correctly, however the hand is drawn at a fov of 30 (using render passes).
Notice how at the start i can change the real fov, without affecting the hand (which is a major advantage).
As the hand reaches out, its fov changes to 85, which is the real fov.
Once the item is grabbed, both are returned to 30.

In case where this isn't possible (there are a number of caveats and limitations), the script from earlier allows you to position the hand on top of the object, making the transition easier to hide not nearly as jarring.

 

Hey,

thank you for this in depth answer and congratz on your result!
I think this is amazing!

Actually, I found that Camera.main.ViewportToWorldPoint() actually does respect the fov of - in this case - the main camera. I've rewritten my code and now it works. No clue why it didn't before.