Get mouse coordinates in UI-element

Hi,

i have big troubles with getting the mouseposition inside a ui-element,
what i need to do is mark a spot on a RawImage.

I've tried it for days now, and searched the web of course, but it looks like i'm the only one who tries something like this.

I already have a BoxCollider2D on my RawImage.
Also tried to create a Rect with the same size etc. and try to get the coordinates inside the rect.
Tried a bunch of different things now, but none of them will work.

My last try was to use the screen coordinates of the mouse, then look where the rawImage coordinates are and subtract them, but the coordinates of my rawimage are always 0.0, 0.0 but that's surely not right..

Please, give me a little hint bout this.. Thank you in advance.

 

something here ?
http://answers.unity3d.com/questions/799616/unity-46-beta-19-how-to-convert-from-world-space-t.html

 

Thank you,

i now get a number and not just 0.0.
But didn't work either, it's now 0.5, 0.5 or other coordinates which also didn't make sense to me.
I thought reading the coordinates on an image will be a trivial thing, and now i'm stuck.

Should i change the Object where i lay the image on to something different than RawImage?

 

Code (CSharp):

1. using UnityEngine;
2. using UnityEngine.UI;
3.  
4. public class TMP : MonoBehaviour {
5.     void Update () {
6.         Vector3[] corners = new Vector3[4];
7.         GetComponent<RawImage>().rectTransform.GetWorldCorners(corners);
8.         Rect newRect = new Rect(corners[0], corners[2]-corners[0]);
9.         Debug.Log(newRect.Contains(Input.mousePosition));
10.     }
11. }
12.  

Just checked, it works. That was interesting digging in manuals. This checks if mouse is currently in rectangle, formed by corners of RawImage frame.
Do I need to say that to get exact coordinate inside image you need to compare Input.mousePosition with corners[0] and corners[2]? I don't know if you need absolute (pixel) coordinates or relative so I leave you to finish solution.

P.S. oh, and if coordinates aren't overlay you should've said so. It will become trickier. Maybe.

 

Hi Teravisor,

thank you, i now get the coordinates that i need.
What i need is the down left corner in pixel coordinates on the screen:

Code (csharp):

1.  
2. Vector3[] corners = newVector3[4];
3. rawImage.rectTransform.GetWorldCorners(corners);
4. RectnewRect = newRect(corners[0], corners[2]-corners[0]);
5. Debug.Log("newRect pixel coordinates: " + cam.WorldToScreenPoint(corners[0]));
6.  

The rest is simple now, just subtracting this from the mouse position on the screen.

 

Hello, I am looking at this too, except my use-case is for a minimap camera (using a rendertexture on the uGUI). But I'm confused as to how to convert the world space coordinates or the rendertexture raw image into a camera's viewport system which Unity says is normalized and relative to the camera.

Even if I manage to get the correct coordinates on my raw image element, I can't think of how to convert that to the camera's viewport coordinates.

 

Let's say we're making minimap with rendertexture draw on rawimage+its camera and you need to click on it to select something in world you would need to make several transformations:

1. From mousePosition into absolute coordinates of rawImage(Input.mousePosition - corners[0] from above);
2. From absolute coordinates of rawImage to relative coordinates of rawImage(divide each coordinate by absolute size. Size is corners[2]-corners[0]);
3. From relative coordinates in rawImage (assuming they are same as relative coordinates on rendertexture, so same as relative coordinates in camera that draws minimap) into ray of camera that draws minimap(Depends on how you render your minimap. Most likely, something like multiply by (width,height) vector and use Camera.ScreenPointToRay);
4. From ray of camera that draws minimap into world coordinate you clicked(in orthogonal - just position of ray, in perspective a little calculation on line-to-plane intersection).

If you need to transform world coord->rendertexture camera->rawimage->screen coords on UI, then reverse all 4 steps.
I'm not about to write code here as it's offtopic to OP.

 

Hi Teravisor,

Thanks for reply. Actually I almost have it. Except that, because I'm using a canvas scaler (I think), the raw image gets resized depending on the resolution or aspect ratio of the camera (I don't know which one actually).

Here is my code that works (except I'm using a static number for my raw image size right now -- and it somehow needs to be dynamic):

Code (CSharp):

1. Vector3[] corners = new Vector3[4];
2.                 minimapRawImage.rectTransform.GetWorldCorners(corners);
3.                 Rect newRect = new Rect(corners[0], corners[2] - corners[0]);
4.  
5.                 float xPositionDeltaPoint = Input.mousePosition.x - newRect.x;
6.                 float yPositionDeltaPoint = Input.mousePosition.y - newRect.y;
7.  
8.                 //Debug.Log("The x position delta is: " + xPositionDeltaPoint);
9.                 //Debug.Log("The y position delta is: " + yPositionDeltaPoint);
10.  
11.                 //The value "170" is the raw image size currently
12.                 float xPositionCameraCoordinates = (xPositionDeltaPoint / 170);
13.                 float yPositionCameraCoordinates = (yPositionDeltaPoint / 170);

For some reason, this code works if I don't maximize my game in the Unity editor. If I maximize the game though, then I'm off by the amount that the game was maximized by.

The size of my raw image right now is 170x170. Which, again, works if I don't resize my screen in any way. Do you know how to compensate for the size of the raw image? I think I need to somehow manually get the size of the raw image no matter what the aspect ratio or screen width and height are......but I'm not sure.

 

Are you by any chance doing that calculation only one time? Because after resizing it should give different result.

 

Well the calculations run every time the user clicks on the minimap, so that shouldn't be a problem. So every mouse click it runs.

I'm fairly certain it has to do with the screen resolution actually. So that "170" value number in my code needs to fluctuate depending on the pixel width and height of the raw image. I just can't think of the math formula to convert it based on screen resolution (or if there's a way to reference the image size). I'm also not sure how the "Canvas Scaler" might play a role in it.

For example, currently in my Unity editor my screen resolution is somewhere around 800 x 600. The code works at this resolution (with a raw image width and height value of 170). Now, I click to maximize my screen, the resolution increases to around 1920 x 1080. Now when I click on my minimap, the location is off diagonally to a point (and relative amount) to a location to the upper-right of where the user clicked on the minimap.

I'm still experimenting with how to adjust for that.

 

Ok I solved it

Thanks for your help Teravisor, or at least for letting me think through it on the forums. My base code does work (and works with aspect ratio as well due to the uGUI doing it automatically), however it does need to be adjusted heavily based on screen resolution. And a big chunk of the problem did have to do with the Canvas Scaler script i'm using on my canvas. I'm posting my solution, but anyone who uses this should know that it depends heavily on your settings in the "Canvas Scaler" script on your uGUI Canvas, if you're even using one.

Code (CSharp):

1. //Get the 4 corners (in world space) of the raw image gameobject's rect transform on the GUI
2.                 Vector3[] corners = new Vector3[4];
3.                 minimapRawImage.rectTransform.GetWorldCorners(corners);
4.                 Rect newRect = new Rect(corners[0], corners[2] - corners[0]);
5.  
6.                 //Get the pixel offset amount from the current mouse position to the left edge of the minimap
7.                 //rect transform.  And likewise for the y offset position.
8.                 float xPositionDeltaPoint = Input.mousePosition.x - newRect.x;
9.                 float yPositionDeltaPoint = Input.mousePosition.y - newRect.y;
10.  
11.                 //Debug.Log("The x position delta is: " + xPositionDeltaPoint);
12.                 //Debug.Log("The y position delta is: " + yPositionDeltaPoint);
13.  
14.                
15.                 //The value "170" is the raw image size.
16.                 float compensateForScalingX = 170 * mainCanvasObjectCanvasScript.scaleFactor;
17.                 //"600" is the current reference resolution height on the Canvas Scaler script.
18.                 float compensateForScalingY = 170 * (Screen.height / 600) * mainCanvasObjectCanvasScript.scaleFactor;
19.  
20.                 //If the game screen height resolution and the canvas scaler script's "y reference resolution" are
21.                 //exactly the same, then the division value will be zero.  Since you can't divide by zero, I need
22.                 //to check for this here.
23.                 if (compensateForScalingY == 0)
24.                 {
25.                     compensateForScalingY = 170;
26.                 }
27.  
28.                 //The value "170" is the raw image size currently
29.                 float xPositionCameraCoordinates = (xPositionDeltaPoint / compensateForScalingX);
30.                 float yPositionCameraCoordinates = (yPositionDeltaPoint / compensateForScalingY);

 

You need a tool to translate coordinates I did one called "Rect Transform Extended" you can convert coordinates from / to: Screen space, canvas, anchors, rect, match objects coordinates located at different containers, etc, take a look:https://www.assetstore.unity3d.com/#!/content/41927

 

Old thread, but it still shows up in google. Just use RectTransformUtility class. It has useful methods like this one:
https://docs.unity3d.com/ScriptRefe...ility.ScreenPointToLocalPointInRectangle.html