Bug Canvas Raycast Hitbox doesn't match Screen Resolution when using Camera with Render Texture

Edit: Bug is on the issue tracker!
https://issuetracker.unity3d.com/is...n-camera-when-ui-camera-targets-rendertexture

Bandaid workaround:

Original post below.
_____

I'm trying to set up a UI that accepts mouse interactions, in a game with a low internal resolution. Our setup requires our UI and in-game cameras to output to a render texture, so that we can apply visual effects as a final pass over the entire screen via the main camera.

The problem: The raycasts for the UI don't scale to fit the actual game window's resolution. Our internal resolution is 384x216, and that's the size of the region where all the UI raycasts are being calculated in the bottom-left of the window.

Here's a gif demonstration:



To replicate, just make a scene that features:

• Two cameras - Main and UI
  
• A render texture with a low resolution, to be attached to the UI cam
  
• A canvas that renders to Screen Space - Camera using the UI camera, and uses a Graphic Raycaster and a Canvas Scaler component
• A regular ol' Button gameobject in that canvas, added via GameObject > UI > Button
• An Event System gameobject, to make that Button functional
  
• And a tiny script that blits the render texture to the Main camera, with the render texture added.

Code (CSharp):

1. using UnityEngine;
2. using System.Collections;
3.  
4. [ExecuteInEditMode]
5. public class BlitRenderTexture : MonoBehaviour
6. {
7.     public RenderTexture BlitTex;
8.  
9.     void OnRenderImage(RenderTexture src, RenderTexture dest)
10.     {
11.         if (BlitTex != null)
12.             Graphics.Blit(BlitTex, dest);
13.     }
14. }

Am I trying to attempt this the wrong way, or have I run into a bug?
Is there a way to rescale the canvas raycast hitbox so that it matches the screen / an arbitrary region?

If anyone knows a workaround or a way to control this, I'd greatly appreciate it.

 

Just some ideas

• It could be that there is one component on top of the button? on the hierarchy
• You have 2X apply on the scale?

 

There shouldn't be! In the GIF I posted, the button is inside a GameObject with a Vertical Group Layout component, but that shouldn't affect how the raycasts are scaled to fit the game window's resolution. In any case, even if the button is simply on its own inside a canvas, the problem persists.

There's no extra components on the Button either. Every component on it is exactly what you get when a button is created from the GameObject menu.

I'm not sure I understand the question. Have I altered the scale of the Canvas or Button gameobjects?
If that's what you mean: No, I haven't. When the Canvas "Render Mode" is set to "Screen Space - Camera", the RectTransform is completely greyed out, so I can't alter any part of it. And the scale of the Button is (1,1,1).

I will note, the UI works correctly when the Canvas "Render Mode" is set to "Screen Space - Overlay", but this bypasses the UI Camera, preventing the UI from being captured by the render texture. So unfortunately, the Overlay mode is functionally useless to me.

 

In the editor, there is a scale slider to make zoom. And I notice that the zoom is a scale to 2x.(next to "Maximize On Play"). Can it be that that is somehow triggering problems?
Does this happen also on the game build?

 

Oh, no, that's not the cause. Unity automatically switches to 1x scale after I select 16:9. It also doesn't matter if it starts at 1x or 4x scale, the outcome is the same.

And yes, it does occur in a build.

 

Can you put a screenshot of your Hierarchy with the button selected?
Usually, I put the anchors on the corner.

Another tip for debugging the issue:

• I use to make a copy of the project.
• I open the copy as a new project
• I stake out all that is not related
• Until I hit the game object that triggers the bug
• later I undo
• delete the component that can trigger the bug

In this way, I found that the game object that triggers the bug is far away in the hierarchy.

 

Here, I've created a wholly new project from scratch. This is the hierarchy of the entire scene.


I've attached a UnityPackage to this post, containing this entire project. This was created with Unity 2020.1.12f1, but the issue exists on earlier and later versions of Unity. You can import this package into a new project and see it for yourself.

To reproduce the issue, using the package at the end of this post:

• Create a custom game resolution, with the same dimensions as the render texture being used by UI Camera. In this project, the resolution is: 320px Wide, 180px High. (This is a 16:9 image.)

• Press Play with this custom resolution ticked.
• Move the mouse over the button to check that it changes correctly.
• Change the game resolution to Free Aspect.
• Try the button again, as you see it on-screen.
• Move your mouse around the bottom-left region of the screen, until the button reacts.

 

Some ideas

• Do you need to scale the render texture to fit the resolution?
• One camera is 2D the second is 3D. And if both are 2D or 3D?
• Draw on the background, a UI image texture showing pixels and lines
• Get the actual resolution
• Submit a bug report

 

Make a bug report.
I was not able to understand what is triggering this bug.
The two cameras probably.

 

Just submitted a bug report today.
I've also updated the .unitypackage in my post above, with the exact scene that I've sent Unity.

From what I can tell, the problem begins with the UI Camera: It is targeting a render texture, which uses custom dimensions.

When the UI Canvas is set to "Screen Space - Camera", the canvas ends up using the dimensions of the UI Camera's target texture (the render texture) in order to perform raycasts. This raycast region is anchored to the bottom-left corner of the screen, (0,0) in screen space.

For some reason, this raycast region is not being scaled to fit the resolution of the game window.

 

Wow! I just ran against this problem and found this thread through google, I was about to ask if you ever found a solution before I saw the date on that last post!

The hacky bandaid I have found is to use one camera for raycasting, and another for rendering- set the canvas to World Space, give it the raycast camera as the Event Camera, and set the rendering camera mask to only GUI. You can disable the camera component on the Event camera, it still works for raycasts without adding that extra rendering cost.

Some stuff (like mouse-related values in PointerEventData) loses sync, but the raycasts work and since the desynced values are just floats you can sort it out by factoring in the ratio of the screen to render texture sizes.
Alternately you can scale the canvas and the rendering camera to match the window size (and move them to sit in the corner of the event camera), but I guess that's just a "pick your flavor of fiddly" sort of situation.

 

Thank you, this bug had me up against a wall. I can live with this workaround for the meantime!

When you say some stuff loses sync, are there any common pitfalls that I should keep an eye out for? I'm assuming the big one really is just accounting for different aspect ratios.

 

Bug report number and link?

Place the link also on top

 

Oh my god I was in the exact same situation as you and just wasted 2 hours trying to debug this issue!!
Thanks for the tip!

 

I received an email back about this bug last week, with the response that this behavior is By Design.

I'm disappointed by this decision, as it feels like it's been hand-waved away. The way that this currently behaves is obviously not useful to anyone.

For now, I'll be sticking with the solution that CptOblivion described just above, as it is a far simpler fix for this issue. However, if anyone knows how to implement the conversion via BaseInput, I'd love to have more control over how canvas raycasting is scaled.