Question Camera Stacking and Mouse Input and UI Transforms

I'm working on an inventory system for my game. Among such things, I had a UI Tooltip element that would follow the cursor and raycast to find out what the cursor was hovering over, and find out if it could display the gameObject's (custom) tooltip component. It worked as easily as

Code (CSharp):

1. TooltipUI.position = cursorPos;

Instead of using sprites to display all my items, I wanted to use 3D models rendered in the respective item slots. I previously had it set to sprites, so I disabled all those parts of my code where it would update the slots with the images from the sprites the items had referenced. Looking how to do the 3D overlay shenanigans, I learned about Camera Stacking, and thus I now have a main camera and a UI camera. I have their projection matrices tied together so I don't have to bother managing them separately (at least a demo project showed that, so I just followed suit). So I changed the canvas settings on my "PlayerUI" gameObject, and thats where the issues started. Screen Space - Overlay swapped to Screen Space - Camera, I targeted the Render Camera to the new camera I'm using to manage the UI, gave the cameras the correct layers to render, and the code for moving the tooltip around broke. After messing with it extensively for a few hours, I got this result.
As shown, I've managed to edit and change the code so that it follows the cursor...but it has some weird scaling action going with it and will not simply go 1:1 with the cursor position.

Code (CSharp):

1. PlayerInput myInput = Player.GetComponent<PlayerInput>();
2.             Vector2 cursorPos = myInput.actions["UI/Point"].ReadValue<Vector2>();
3.             var screenPoint = new Vector3(cursorPos.x,cursorPos.y,100.0f);
4.             TooltipUI.position = Camera.main.ScreenToWorldPoint(screenPoint);

Far as I can tell, this is where the problem rises up from. I'm using the Unity Input module for my input, and its been working pretty alright so far. cursorPos does in fact give me the correct cursorPos, at least before the swap in canvas render mode, so I'm pretty sure thats not the issue. I've changed "ScreenToWorldPoint" around from ScreenToViewport, ViewportToWorld, etc, and while I haven't specifically checked the trial/error list, I figured I'd come here for advice and try and understand it further rather than try my luck on another 100 iterations of code.
Not sure if its related, but additionally whenever I try and directly change the position of the Rect Transform for the TooltipUI component, half the time it doesn't actually modify the position of the component.

How can I get it so that it once more will follow my cursor correctly? And furthermore, if anyone has any help/tip/advice for rendering 3D objects in the overlay, I'd appreciate it and any tutorial links that may be available.

 

Just move the tooltip into its own overlay canvas. It can actually be performant sometimes to break certain elements into their own canvas to minimize layout recalculations.

 

Also, you're probably want to get the Rect transform of your tooltip and set anchoredPosition instead of transform.position

 

As far as 3D objects in UI, do you only need a single frame? Or are they animated? Either way, I'd probably dedicated one camera to the job. I would render each item out to a render texture then display the result in the UI.

 

So I took your advice and made the tooltip go to its own Canvas-Overlay, but setting anchoredPosition was doing some very odd things. However, doing it how I previously had it,

Code (CSharp):

1. PlayerInput myInput = Player.GetComponent<PlayerInput>();
2.             Vector2 cursorPos = myInput.actions["UI/Point"].ReadValue<Vector2>();
3.             TooltipUI.position = cursorPos;

worked just fine. So, that works now, thanks!

As for your comments on the 3D objects in UI; the idea was have each "item" drawn in its own item slot, and to shortcut it I was going to just draw them all on the one camera overlay, but it sounds like your suggestion is to have them render to a texture and draw that instead. Sounds good in theory, but... does that mean I should have 1 camera per item, rendering a texture for each? That sounds like a lot of processing that shouldn't need to happen. I could very well be wrong though, so could you clarify your suggestion?

 

The reason for this is probably because anchoredPosition respects your pivot/anchor. So you'd have to set the proper pivot on the tooltip to get the desired behavior.

No, just one camera for all items. One texture for each item. This technique assumes you are okay with all items sharing the same perspective projection. If your items don't animate, you can just render out one frame. If you need to animate, then you would updated you render textures every frame. If all is kept on the GPU, should not be a performance hit.

 

So is there any issue with me still using transform.position instead of anchoredPosition + setting the proper pivot? Or is this more of a "best practices" situation, where I should but I don't need to?

Additionally, if you have links to the technique you describe or examples thereof, I'd be very grateful as far as the camera > texture jazz. I'm not sure how I should go about setting up such things. That said, I've found that just rendering the items in front of the player while they're tied to the camera works as well. And, in fact, lets lighting affect the inventory... Which I think is cool and I'm going to make a feature lol.

 

Best practice, you can still use transform.Position.

I don't have any resources immediately available for rendering to render targets. I'm sure some quick googling would yield some result. Sounds like you may already have a solution though