Cross-canvas interaction

Hi all,
I'm trying to embed an interactable map in another canvas. I've set up the map to be a camera with a worldspace camera projected onto RawImage. I'm trying to figure out how to get clicks from the main Overlay canvas to the worldspace canvas.
If anyone has suggestions, it'd be greatly appreciated.

 

If you only need clicks, I imagine you could have some kind of click handler on the RawImage that raycasts out from your camera to find the corresponding object in your other canvas. But that sounds like a fair amount of work, and it wouldn't easily extend to events like pointer-enter.

Why are you using a second camera and rendering it to a RawImage instead of using a single camera with two canvasses? Then all the pointer events would just work without any special effort. I haven't done extensive work on this kind of thing, so there might be obvious reasons that I'm missing, but I would have thought that was the most natural way to do this. A screenspace overlay canvas will continue to appear in the same location on the screen even if the camera moves (that's the point), so you can pan around and look at different worldspace things without the overlay canvas needing to care.

 

The main camera is beings used as the character's view, where the 2nd camera is providing the top-down. I could, if I got desperate, re-use the first camera to provide the top-down, but I'd really rather not.

 

I'll take the opportunity to pimp my Datasacks package again. It's the politically-acceptable global variable package that plays nice with the Unity UI and editor.

Specifically for your use, you would use a datasack (or datasacks) to send clicks from a button (or slider or whatever) on one canvas and have it affect something else, in your case an item on another canvas. Datasacks don't care what canvas is where.

Datasacks is presently hosted at these locations:

https://bitbucket.org/kurtdekker/datasacks

https://github.com/kurtdekker/datasacks

https://gitlab.com/kurtdekker/datasacks

Here's the executive overview of it:

 

I have to admit, I feel a tad intimidated by that, and not entirely sure how it would even work.

A work-around, I supposed, would be if there's another way to display another camera's view in a Overlay canvas.

 

Oh, you are trying to show two different worldspace views simultaneously. I don't have any great ideas in that case.

I can see some solutions in the "rather unpleasant but technically possible" category, including:

• Move all the UI elements from your secondary (worldspace) canvas onto your primary (overlay) canvas, doing a bunch of complex math so that they "line up" and look like they're part of the overhead view, but they're actually not.
• Dynamically swap the cameras so that the overhead view becomes the "main" camera and the first-person view is (temporarily) rendered to a RawImage on it, instead of the other way around.
  
• Create your own simulated mouse, and your own mouse event system, and run the entire UI of your secondary canvas using your own system.

My understanding of the problem is that Vedrit has embedded a picture of a canvas inside of another one, so the canvas the user is actually clicking on doesn't have any of the buttons, sliders, etc. that they are seeing--those are located on the secondary canvas, which can't be seen by the primary camera, and so the regular UI raycast logic has no way of knowing which elements you are pointing at or clicking on.

I don't see how global variables help with that problem.

 

I'm currently trying to figure out how I might do something similar to your third suggestion. I'm trying to get the mouse position within the RawImage and send it to the second canvas/camera. The issue I'm running into is that most OnPointer events return position within the entire screen

 

If the hardest thing you need to do when building this system is to derive the math to perform that coordinate conversion manually, I will be rather surprised.

But you might be able to use something like ScreenPointToWorldPointInRectangle to help out.

 

So, ScreenPointToWorldPointInRectangle was returning the same coordinates as IPointerDown pressEvent.position was returning.
However, ScreenPointToLocalPointInRectangle gave me the results I was looking for. Now I can just adjust the local point coordinates to match the dimensions of the 2nd camera/canvas. Now I need to figure out how to trigger a click in that other space...

 

I don't think "triggering a click" is a thing you can do, exactly. Unity processes mouse input automatically in a bunch of complex ways, and it isn't designed to let you give it simulated inputs.

You can call functions on objects, though, and you can ask Unity to propagate your attempted function-calls up the object hierarchy using SendMessageUpwards; e.g.

Code (CSharp):

1. gameObject.SendMessageUpwards("OnPointerClick", eventData, SendMessageOptions.DontRequireReceiver);

That requires you to first calculate which GameObject should receive the click, though. And it isn't the same as an actual click; it's just looking for functions named "OnPointerClick". It won't call OnPointerDown or OnMouseUpAsButton or any other click-related functions, unless you explicitly call them separately. On the other hand, it will call any function named "OnPointerClick" even if the component doesn't implement the IPointerClickHandler interface.

You could potentially call a bunch of functions like OnPointerDown, OnPointerUp, OnPointerClick, etc. and you'd probably get pretty good simulation of clicks (especially if you design your UI specifically to rely on the functions you are actually calling, and not on other ones).

However, as I noted upthread, this doesn't extend very well to events like OnPointerEnter, because those are related to boundaries in your secondary canvas that don't exist in your primary canvas, so calculating when to call them is much more complex than with the click events. (This might be important, for example, if you want to have rollover highlights on your buttons.) You'd basically need to check the mouse position every frame, raycast what it would be pointing at in your secondary canvas every frame, and compare the result to the previous frame to figure out what functions to call.

If you extend this to all kinds of input and build a robust system to handle everything, I think you're essentially designing and implementing your own input event system.

 

That's fair enough, thanks for being honest. And thanks to @Antistone for further clarification too.

I am not sure I fully understand your use case. Are you trying to tap on that minimap rendering and then know where in the real world you touched?

 

That... sounds like a lot of mess.
Maybe there's a different way to do this that will achieve the desired end-result?

 

Click in the "mini-map", yes, but not so much know where in the world I clicked, but more know where in the other canvas I clicked

 

That's why the list was titled "rather unpleasant but technically possible".

 

Ok, I got a sort of approximation of a mouse click. I first get the relative position of the mouse within the frame by using ScreenPointToLocalPointInRectangle, then use InverseLerp to get it as a float, then use the float for lerp to get the relative position in the worldspace canvas, and then use GraphicRaycast to see what is being clicked on.
I would like to figure out mouse hovering, but I think this is enough messiness for now.

For anyone curious, here's the OnPointerDown method

Code (csharp):

1. public void OnPointerDown(PointerEventData data)
2.     {
3.         Vector2 worldPoint = new Vector2();
4.         RectTransformUtility.ScreenPointToLocalPointInRectangle(rect, data.pressPosition, null, out worldPoint);
5.         float lerpX = Mathf.InverseLerp(-250f, 250f, worldPoint.x);
6.         float lerpY = Mathf.InverseLerp(-250f, 250f, worldPoint.y);
7.         worldPoint.x = Mathf.Lerp(0, otherCanvas.eventCamera.scaledPixelHeight, lerpX);
8.         worldPoint.y = Mathf.Lerp(0, otherCanvas.eventCamera.scaledPixelWidth, lerpY);
9.         List<RaycastResult> hitList = new List<RaycastResult>();
10.         PointerEventData pointer = new PointerEventData(events);
11.         pointer.position = worldPoint;
12.         otherCanvas.Raycast(pointer, hitList);
13.         for(int i = 0; i < hitList.Count; i++)
14.         {
15.                 //Do the stuff
16.         }
17.     }