Search Unity

  1. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Bug Canvas Raycast Hitbox doesn't match Screen Resolution when using Camera with Render Texture

Discussion in 'UGUI & TextMesh Pro' started by HeyItsLollie, Dec 1, 2020.

  1. HeyItsLollie

    HeyItsLollie

    Joined:
    May 16, 2015
    Posts:
    68
    Edit: Bug is on the issue tracker!
    https://issuetracker.unity3d.com/is...n-camera-when-ui-camera-targets-rendertexture

    Bandaid workaround:
    Original post below.
    _____

    I'm trying to set up a UI that accepts mouse interactions, in a game with a low internal resolution. Our setup requires our UI and in-game cameras to output to a render texture, so that we can apply visual effects as a final pass over the entire screen via the main camera.

    The problem: The raycasts for the UI don't scale to fit the actual game window's resolution. Our internal resolution is 384x216, and that's the size of the region where all the UI raycasts are being calculated in the bottom-left of the window.

    Here's a gif demonstration:

    20201201_21-17-40-955.gif

    To replicate, just make a scene that features:
    • Two cameras - Main and UI
    • A render texture with a low resolution, to be attached to the UI cam
    • A canvas that renders to Screen Space - Camera using the UI camera, and uses a Graphic Raycaster and a Canvas Scaler component
    • A regular ol' Button gameobject in that canvas, added via GameObject > UI > Button
    • An Event System gameobject, to make that Button functional
    • And a tiny script that blits the render texture to the Main camera, with the render texture added.

    CanvasRaycastIssues.png

    Code (CSharp):
    1. using UnityEngine;
    2. using System.Collections;
    3.  
    4. [ExecuteInEditMode]
    5. public class BlitRenderTexture : MonoBehaviour
    6. {
    7.     public RenderTexture BlitTex;
    8.  
    9.     void OnRenderImage(RenderTexture src, RenderTexture dest)
    10.     {
    11.         if (BlitTex != null)
    12.             Graphics.Blit(BlitTex, dest);
    13.     }
    14. }
    Am I trying to attempt this the wrong way, or have I run into a bug?
    Is there a way to rescale the canvas raycast hitbox so that it matches the screen / an arbitrary region?

    If anyone knows a workaround or a way to control this, I'd greatly appreciate it.
     
    Last edited: Mar 15, 2021
  2. AlanMattano

    AlanMattano

    Joined:
    Aug 22, 2013
    Posts:
    1,500
    Just some ideas
    • It could be that there is one component on top of the button? on the hierarchy
    • You have 2X apply on the scale?
     
    Last edited: Dec 1, 2020
  3. HeyItsLollie

    HeyItsLollie

    Joined:
    May 16, 2015
    Posts:
    68
    There shouldn't be! In the GIF I posted, the button is inside a GameObject with a Vertical Group Layout component, but that shouldn't affect how the raycasts are scaled to fit the game window's resolution. In any case, even if the button is simply on its own inside a canvas, the problem persists.

    There's no extra components on the Button either. Every component on it is exactly what you get when a button is created from the GameObject menu.

    Unity_20201202_16-47-05-407.png Unity_20201202_16-45-29-537.png

    I'm not sure I understand the question. Have I altered the scale of the Canvas or Button gameobjects?
    If that's what you mean: No, I haven't. When the Canvas "Render Mode" is set to "Screen Space - Camera", the RectTransform is completely greyed out, so I can't alter any part of it. And the scale of the Button is (1,1,1).

    I will note, the UI works correctly when the Canvas "Render Mode" is set to "Screen Space - Overlay", but this bypasses the UI Camera, preventing the UI from being captured by the render texture. So unfortunately, the Overlay mode is functionally useless to me.
     
  4. AlanMattano

    AlanMattano

    Joined:
    Aug 22, 2013
    Posts:
    1,500
    In the editor, there is a scale slider to make zoom. And I notice that the zoom is a scale to 2x.(next to "Maximize On Play"). Can it be that that is somehow triggering problems?
    Does this happen also on the game build?
     
  5. HeyItsLollie

    HeyItsLollie

    Joined:
    May 16, 2015
    Posts:
    68
    Oh, no, that's not the cause. Unity automatically switches to 1x scale after I select 16:9. It also doesn't matter if it starts at 1x or 4x scale, the outcome is the same.

    And yes, it does occur in a build.
     
    AlanMattano likes this.
  6. AlanMattano

    AlanMattano

    Joined:
    Aug 22, 2013
    Posts:
    1,500
    Can you put a screenshot of your Hierarchy with the button selected?
    Usually, I put the anchors on the corner.

    Another tip for debugging the issue:
    • I use to make a copy of the project.
    • I open the copy as a new project
    • I stake out all that is not related
    • Until I hit the game object that triggers the bug
    • later I undo
    • delete the component that can trigger the bug
    In this way, I found that the game object that triggers the bug is far away in the hierarchy.
     
  7. HeyItsLollie

    HeyItsLollie

    Joined:
    May 16, 2015
    Posts:
    68
    BugExample.gif

    Here, I've created a wholly new project from scratch. This is the hierarchy of the entire scene.
    Unity_20201210_17-48-11-171.png

    I've attached a UnityPackage to this post, containing this entire project. This was created with Unity 2020.1.12f1, but the issue exists on earlier and later versions of Unity. You can import this package into a new project and see it for yourself.

    To reproduce the issue, using the package at the end of this post:
    • Create a custom game resolution, with the same dimensions as the render texture being used by UI Camera. In this project, the resolution is: 320px Wide, 180px High. (This is a 16:9 image.)
    Unity_20201210_17-49-12-925.png

    • Press Play with this custom resolution ticked.
    • Move the mouse over the button to check that it changes correctly.
    • Change the game resolution to Free Aspect.
    • Try the button again, as you see it on-screen.
    • Move your mouse around the bottom-left region of the screen, until the button reacts.
     

    Attached Files:

    Last edited: Dec 10, 2020
    AlanMattano likes this.
  8. AlanMattano

    AlanMattano

    Joined:
    Aug 22, 2013
    Posts:
    1,500
    Some ideas
    • Do you need to scale the render texture to fit the resolution?
    • One camera is 2D the second is 3D. And if both are 2D or 3D?
    • Draw on the background, a UI image texture showing pixels and lines
    • Get the actual resolution
    • Submit a bug report
     
  9. AlanMattano

    AlanMattano

    Joined:
    Aug 22, 2013
    Posts:
    1,500
    Make a bug report.
    I was not able to understand what is triggering this bug.
    The two cameras probably.
     
  10. HeyItsLollie

    HeyItsLollie

    Joined:
    May 16, 2015
    Posts:
    68
    Just submitted a bug report today.
    I've also updated the .unitypackage in my post above, with the exact scene that I've sent Unity.

    From what I can tell, the problem begins with the UI Camera: It is targeting a render texture, which uses custom dimensions.

    When the UI Canvas is set to "Screen Space - Camera", the canvas ends up using the dimensions of the UI Camera's target texture (the render texture) in order to perform raycasts. This raycast region is anchored to the bottom-left corner of the screen, (0,0) in screen space.

    For some reason, this raycast region is not being scaled to fit the resolution of the game window.
     
    AlanMattano likes this.
  11. CptOblivion

    CptOblivion

    Joined:
    Mar 8, 2015
    Posts:
    9
    Wow! I just ran against this problem and found this thread through google, I was about to ask if you ever found a solution before I saw the date on that last post!

    The hacky bandaid I have found is to use one camera for raycasting, and another for rendering- set the canvas to World Space, give it the raycast camera as the Event Camera, and set the rendering camera mask to only GUI. You can disable the camera component on the Event camera, it still works for raycasts without adding that extra rendering cost.

    Some stuff (like mouse-related values in PointerEventData) loses sync, but the raycasts work and since the desynced values are just floats you can sort it out by factoring in the ratio of the screen to render texture sizes.
    Alternately you can scale the canvas and the rendering camera to match the window size (and move them to sit in the corner of the event camera), but I guess that's just a "pick your flavor of fiddly" sort of situation.
     
    LazyEti and HeyItsLollie like this.
  12. HeyItsLollie

    HeyItsLollie

    Joined:
    May 16, 2015
    Posts:
    68
    Thank you, this bug had me up against a wall. I can live with this workaround for the meantime!

    When you say some stuff loses sync, are there any common pitfalls that I should keep an eye out for? I'm assuming the big one really is just accounting for different aspect ratios.
     
    Last edited: Dec 13, 2020
  13. AlanMattano

    AlanMattano

    Joined:
    Aug 22, 2013
    Posts:
    1,500
    Bug report number and link?

    Place the link also on top
     
  14. LazyEti

    LazyEti

    Joined:
    Sep 10, 2017
    Posts:
    7
    Oh my god I was in the exact same situation as you and just wasted 2 hours trying to debug this issue!!
    Thanks for the tip!
     
    HeyItsLollie likes this.
  15. HeyItsLollie

    HeyItsLollie

    Joined:
    May 16, 2015
    Posts:
    68
    I received an email back about this bug last week, with the response that this behavior is By Design.

    I'm disappointed by this decision, as it feels like it's been hand-waved away. The way that this currently behaves is obviously not useful to anyone.

    For now, I'll be sticking with the solution that CptOblivion described just above, as it is a far simpler fix for this issue. However, if anyone knows how to implement the conversion via BaseInput, I'd love to have more control over how canvas raycasting is scaled.
     
    Last edited: Mar 15, 2021
    LazyEti likes this.