Question Screen space to World space. Canvas - Screen Space Camera. Camera is orthogonal.

Hi,

I have been having issue with a using `Camera.main.ScreenToWorldPoint` so I created a sample project to test out my theories.

My sample scene has a Camera set up as orthogonal, a Canvas set up as Screen Space - Camera with all default values. In this canvas I have UI elements that I want to highlight during a tutorial with a shader.

I have a shader that takes a world space position and highlights an area of the screen. This shader lives on a 3d quad that sits between the camera and the canvas, it darkens the screen and highlights an area. Here's what the quad looks like


Here is a script that controls what thing to highlight.

Code (CSharp):

1. public class HighlightContoller : MonoBehaviour
2. {
3.  
4.     public Material Material;
5.     public Transform Transform;
6.     public float Z;
7.     public ColorTag Tag;
8.     public bool UseCanvasPosition;
9.     public Camera Camera;
10.  
11.     private void OnValidate()
12.     {
13.        
14.         GameObject go = GameObject.FindWithTag(Tag.ToString());
15.         if (go != null)
16.         {
17.             if (UseCanvasPosition)
18.             {
19.                 HighlightCanvasTransform(go.transform);
20.             }
21.             else
22.             {
23.                 HighlightTransform(go.transform);
24.             }
25.         }
26.     }
27.  
28.     public void HighlightTransform(Transform transform)
29.     {
30.         Vector3 worldPos = transform.position;
31.         Material.SetVector("_HighlightAreaCenter", worldPos);
32.     }
33.    
34.     public void HighlightCanvasTransform(Transform transform)
35.     {
36.         Vector3 worldPos = Camera.main.ScreenToWorldPoint(new Vector3(transform.localPosition.x, transform.localPosition.y, Z));
37.         Material.SetVector("_HighlightAreaCenter", worldPos);
38.     }
39. }
40.  
41. [Serializable]
42. public enum ColorTag
43. {
44.     green,
45.     red,
46.     blue,
47.     pink
48. }

Here's a look at the entire scene


Using this code, I was able to test different combinations of transformations, but I could never something working that called

Camera.main.ScreenToWorldPoint

. Whats maddening is I dont think that

HighlightTransform(Transform transform)

should work but it does. So, here are some of my questions

1. Why does

HighlightTransform(Transform transform)

work?
2. Why doesn't

HighlightCanvasTransform(Transform transform)

work?

Here is the project:
https://github.com/billykatz/TestScreenToWorld
Unity version 2020.3.26f1


I have read many forum post but most are asking about a perspective camera, my camera is orthogonal.
I read through this one and it doesnt seem like any of those things help me. https://forum.unity.com/threads/overlay-canvas-and-world-space-coordinates.291108/

 

So I answered my questions with the help of reddit and a relaxing weekend away from the code:

1. This works because the world space position of a object in the canvas in camera mode is just transform.position
2. Calling Camera.main.ScreenToWorldPoint doesnt really make sense when the canvas mode is set to camera. It doesnt provdie any useful information.

Also I was able to fix the issue in my actual project which seems like a timing bug. I had a hunch that maybe the position of the UI element was changing at some point during scene loading.
I originally was grabbing the transform on Start which didnt work. I moved the code to right after the first frame on Update that also didnt work. Now I grab the transform right before I need it, and that seems to work.