Search Unity

  1. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

5.3 Multi-Display Feature Issues...

Discussion in 'Editor & General Support' started by Awesumo, Dec 8, 2015.

  1. karl_jones

    karl_jones

    Unity Technologies

    Joined:
    May 5, 2015
    Posts:
    8,142
    Yeah I'll bring it up with the team and try and take a look in January. Im on holiday now :)
    I can't guarantee that it can be fixed though. I'll know more once I have talked to the team.
     
    leutnant_kane likes this.
  2. karl_jones

    karl_jones

    Unity Technologies

    Joined:
    May 5, 2015
    Posts:
    8,142
    Hey. I have taken a look. It's quite tricky.

    The main display rescales all the coordinates which really messes things up.
    I have created a potential fix that seems to work. I have only tested it locally myself so far, once it goes to QA they may find it has problems I have missed.

    You can test this fix yourself, it requires a few changes to the UI package. If you are able to test it and let me know if it works for you I would appreciate it :)

    Code (csharp):
    1.  
    2. ---
    3.  .../EventSystem/InputModules/BaseInput.cs     |  2 +-
    4.  .../Raycasters/PhysicsRaycaster.cs            |  2 +-
    5.  .../Runtime/UI/Core/GraphicRaycaster.cs       |  2 +-
    6.  .../UI/Core/MultipleDisplayUtilities.cs       | 44 +++++++++++++------
    7.  4 files changed, 34 insertions(+), 16 deletions(-)
    8.  
    9. diff --git a/Packages/com.unity.ugui/Runtime/EventSystem/InputModules/BaseInput.cs b/Packages/com.unity.ugui/Runtime/EventSystem/InputModules/BaseInput.cs
    10. index a023e932c02c..053d8116e926 100644
    11. --- a/Packages/com.unity.ugui/Runtime/EventSystem/InputModules/BaseInput.cs
    12. +++ b/Packages/com.unity.ugui/Runtime/EventSystem/InputModules/BaseInput.cs
    13. @@ -72,7 +72,7 @@ namespace UnityEngine.EventSystems
    14.          /// </summary>
    15.          public virtual Vector2 mousePosition
    16.          {
    17. -            get { return MultipleDisplayUtilities.GetMousePositionRelativeToMainDisplayResolution(); }
    18. +            get { return Input.mousePosition; }
    19.          }
    20.  
    21.          /// <summary>
    22. diff --git a/Packages/com.unity.ugui/Runtime/EventSystem/Raycasters/PhysicsRaycaster.cs b/Packages/com.unity.ugui/Runtime/EventSystem/Raycasters/PhysicsRaycaster.cs
    23. index 3c9832bedc44..6d16f1b5ee04 100644
    24. --- a/Packages/com.unity.ugui/Runtime/EventSystem/Raycasters/PhysicsRaycaster.cs
    25. +++ b/Packages/com.unity.ugui/Runtime/EventSystem/Raycasters/PhysicsRaycaster.cs
    26. @@ -105,7 +105,7 @@ namespace UnityEngine.EventSystems
    27.              if (eventCamera == null)
    28.                  return false;
    29.  
    30. -            var eventPosition = Display.RelativeMouseAt(eventData.position);
    31. +            var eventPosition = MultipleDisplayUtilities.RelativeMouseAtScaled(eventData.position);
    32.              if (eventPosition != Vector3.zero)
    33.              {
    34.                  // We support multiple display and display identification based on event position.
    35. diff --git a/Packages/com.unity.ugui/Runtime/UI/Core/GraphicRaycaster.cs b/Packages/com.unity.ugui/Runtime/UI/Core/GraphicRaycaster.cs
    36. index 95c7a0e57a66..5505027db1a0 100644
    37. --- a/Packages/com.unity.ugui/Runtime/UI/Core/GraphicRaycaster.cs
    38. +++ b/Packages/com.unity.ugui/Runtime/UI/Core/GraphicRaycaster.cs
    39. @@ -140,7 +140,7 @@ namespace UnityEngine.UI
    40.              else
    41.                  displayIndex = currentEventCamera.targetDisplay;
    42.  
    43. -            var eventPosition = Display.RelativeMouseAt(eventData.position);
    44. +            var eventPosition = MultipleDisplayUtilities.RelativeMouseAtScaled(eventData.position);
    45.              if (eventPosition != Vector3.zero)
    46.              {
    47.                  // We support multiple display and display identification based on event position.
    48. diff --git a/Packages/com.unity.ugui/Runtime/UI/Core/MultipleDisplayUtilities.cs b/Packages/com.unity.ugui/Runtime/UI/Core/MultipleDisplayUtilities.cs
    49. index 6022fda7c23a..c12ca62a89a8 100644
    50. --- a/Packages/com.unity.ugui/Runtime/UI/Core/MultipleDisplayUtilities.cs
    51. +++ b/Packages/com.unity.ugui/Runtime/UI/Core/MultipleDisplayUtilities.cs
    52. @@ -16,7 +16,7 @@ namespace UnityEngine.UI
    53.              position = eventData.position;
    54.              #else
    55.              int pressDisplayIndex = eventData.pointerPressRaycast.displayIndex;
    56. -            var relativePosition = Display.RelativeMouseAt(eventData.position);
    57. +            var relativePosition = RelativeMouseAtScaled(eventData.position);
    58.              int currentDisplayIndex = (int)relativePosition.z;
    59.  
    60.              // Discard events on a different display.
    61. @@ -30,29 +30,47 @@ namespace UnityEngine.UI
    62.          }
    63.  
    64.          /// <summary>
    65. -        /// Adjusts the position when the main display has a different rendering resolution to the system resolution.
    66. +        /// A version of Display.RelativeMouseAt that scales the position when the main display has a different rendering resolution to the system resolution.
    67.          /// By default, the mouse position is relative to the main render area, we need to adjust this so it is relative to the system resolution
    68.          /// in order to correctly determine the position on other displays.
    69.          /// </summary>
    70.          /// <returns></returns>
    71. -        public static Vector2 GetMousePositionRelativeToMainDisplayResolution()
    72. +        public static Vector3 RelativeMouseAtScaled(Vector2 position)
    73.          {
    74. -            var position = Input.mousePosition;
    75. -            #if !UNITY_EDITOR && !UNITY_WSA
    76. -            if (Display.main.renderingHeight != Display.main.systemHeight)
    77. +            //#if !UNITY_EDITOR && !UNITY_WSA
    78. +            // If the main display is now the same resolution as the system then we need to scale the mouse position. (case 1141732)
    79. +            if (Display.main.renderingWidth != Display.main.systemWidth || Display.main.renderingHeight != Display.main.systemHeight)
    80.              {
    81. -                // The position is relative to the main render area, we need to adjust this so
    82. -                // it is relative to the system resolution in order to correctly determine the position on other displays.
    83. +                // Calculate any padding that may be added when the rendering apsect ratio does not match the system aspect ratio.
    84. +                int widthPlusPadding = (int)(Display.main.renderingHeight * (Display.main.systemWidth / (float)Display.main.systemHeight));
    85.  
    86. -                // Correct the y position if we are outside the main display.
    87. +                // Calculate the padding on each side of the screen.
    88. +                int padding = (int)((widthPlusPadding - Display.main.renderingWidth) * 0.5f);
    89. +                int widthPlusRightPadding = widthPlusPadding - padding;
    90. +
    91. +                // If we are not inside of the main display then we must adjust the mouse position so it is scaled by
    92. +                // the main display and adjusted for any padding that may have been added due to different aspect ratios.
    93.                  if ((position.y < 0 || position.y > Display.main.renderingHeight ||
    94. -                     position.x < 0 || position.x > Display.main.renderingWidth) && (Screen.fullScreenMode != FullScreenMode.Windowed))
    95. +                     position.x < 0 || position.x > widthPlusRightPadding) && (Screen.fullScreenMode != FullScreenMode.Windowed))
    96. +                {
    97. +                    // Adjust for border pixels
    98. +                    position.x += padding;
    99. +
    100. +                    float xScale = Display.main.systemWidth / (float)widthPlusPadding;
    101. +                    float yScale = Display.main.systemHeight / (float)Display.main.renderingHeight;
    102. +                    position.x *= xScale;
    103. +                    position.y *= yScale;
    104. +
    105. +                    return Display.RelativeMouseAt(position);
    106. +                }
    107. +                else
    108.                  {
    109. -                    position.y += Display.main.systemHeight - Display.main.renderingHeight;
    110. +                    // We are using the main display.
    111. +                    return new Vector3(position.x, position.y, 0);
    112.                  }
    113.              }
    114. -            #endif
    115. -            return position;
    116. +            //#endif
    117. +            return Display.RelativeMouseAt(position);
    118.          }
    119.      }
    120.  }
    121. --
    122.  
     

    Attached Files:

    leutnant_kane likes this.
  3. leutnant_kane

    leutnant_kane

    Joined:
    Apr 29, 2015
    Posts:
    40
    Hi @karl_jones and thx for the upload.
    I tried the fix, but it only works in certain situations. The issue reappears as soon as any of the windows is not rendered in fullscreen. Also there seems to be some weird interactions when moving panels with masks between displays, I'll have a closer look at that tomorrow
     
  4. karl_jones

    karl_jones

    Unity Technologies

    Joined:
    May 5, 2015
    Posts:
    8,142
    I think I have fixed the window issue.

    Code (csharp):
    1.  
    2.  
    3.  .../UI/Core/MultipleDisplayUtilities.cs       | 27 ++++++++++++-------
    4.  1 file changed, 18 insertions(+), 9 deletions(-)
    5.  
    6. diff --git a/Packages/com.unity.ugui/Runtime/UI/Core/MultipleDisplayUtilities.cs b/Packages/com.unity.ugui/Runtime/UI/Core/MultipleDisplayUtilities.cs
    7. index 101d322bf766..ba6ecd019471 100644
    8. --- a/Packages/com.unity.ugui/Runtime/UI/Core/MultipleDisplayUtilities.cs
    9. +++ b/Packages/com.unity.ugui/Runtime/UI/Core/MultipleDisplayUtilities.cs
    10. @@ -42,24 +42,33 @@ namespace UnityEngine.UI
    11.              if (Display.main.renderingWidth != Display.main.systemWidth || Display.main.renderingHeight != Display.main.systemHeight)
    12.              {
    13.                  // Calculate any padding that may be added when the rendering apsect ratio does not match the system aspect ratio.
    14. -                int widthPlusPadding = (int)(Display.main.renderingHeight * (Display.main.systemWidth / (float)Display.main.systemHeight));
    15. +                int widthPlusPadding = Screen.fullScreen ? Display.main.renderingWidth : (int)(Display.main.renderingHeight * (Display.main.systemWidth / (float)Display.main.systemHeight));
    16.  
    17.                  // Calculate the padding on each side of the screen.
    18. -                int padding = (int)((widthPlusPadding - Display.main.renderingWidth) * 0.5f);
    19. +                int padding = Screen.fullScreen ? 0 : (int)((widthPlusPadding - Display.main.renderingWidth) * 0.5f);
    20.                  int widthPlusRightPadding = widthPlusPadding - padding;
    21.  
    22.                  // If we are not inside of the main display then we must adjust the mouse position so it is scaled by
    23.                  // the main display and adjusted for any padding that may have been added due to different aspect ratios.
    24.                  if ((position.y < 0 || position.y > Display.main.renderingHeight ||
    25. -                     position.x < 0 || position.x > widthPlusRightPadding) && (Screen.fullScreenMode != FullScreenMode.Windowed))
    26. +                     position.x < 0 || position.x > widthPlusRightPadding))
    27.                  {
    28. -                    // Adjust for border pixels
    29. -                    position.x += padding;
    30. +                    if (!Screen.fullScreen)
    31. +                    {
    32. +                        // When in windowed mode, the window will be centered with the 0,0 coordinate at the top left, we need to adjust so it is relative to the screen instead.
    33. +                        position.x -= (Display.main.renderingWidth - Display.main.systemWidth) * 0.5f;
    34. +                        position.y -= (Display.main.renderingHeight - Display.main.systemHeight) * 0.5f;
    35. +                    }
    36. +                    else
    37. +                    {
    38. +                        // Scale the mouse position
    39. +                        position.x += padding;
    40.  
    41. -                    float xScale = Display.main.systemWidth / (float)widthPlusPadding;
    42. -                    float yScale = Display.main.systemHeight / (float)Display.main.renderingHeight;
    43. -                    position.x *= xScale;
    44. -                    position.y *= yScale;
    45. +                        float xScale = Display.main.systemWidth / (float)widthPlusPadding;
    46. +                        float yScale = Display.main.systemHeight / (float)Display.main.renderingHeight;
    47. +                        position.x *= xScale;
    48. +                        position.y *= yScale;
    49. +                    }
    50.  
    51.                      return Display.RelativeMouseAt(position);
    52.                  }
    53. --
    54.  
    Our QA also found this issue so ill send the fix to them to verify as well. What do you mean about dragging? I suspect this will be hard to handle, especially when going from a windowed screen.
     
    Last edited: Jan 11, 2022
  5. leutnant_kane

    leutnant_kane

    Joined:
    Apr 29, 2015
    Posts:
    40
    Hi @karl_jones, sry for the delayed reply, caught something mean.

    Tried your new fix, the offset sadly does reappear as soon as one of the windows is moved though.

    About the dragging: I actually meant reparenting UI objects from a canvas on one display to a canvas on another display (we are doing actual dragging from/to Unity applications as well, but that's another topic probably). What I noticed there is that masking breaks on the way back to the first display. So first reparenting is fine (Display 1 to Display 2), but the second reparenting (Display 2 back to Display 1) causes the masks to just hide everything. I'm using the Rect2D Mask but I'm guessing the regular mask might be affected as well.

    Also I noticed some scripts using Input.mouseposition and RectTransformUtility.RectangleContainsScreenPoint not working correctly on the second screen. Does the mouse position need to get queried differently here or might that be a bug as well?
     
  6. karl_jones

    karl_jones

    Unity Technologies

    Joined:
    May 5, 2015
    Posts:
    8,142
    I think we cache all the values when the application starts so moving a window is likely to break things. Can you file a bug report for this issue?


    That also sounds like a new issue. We would need a bug report so we can look into this as it sounds quite specific.

    No this should just be the normal mouse position. Our QA did find some issue with OnMouseDown in window mode which I need to look into.

    Ill go ahead with this fix now and address the other issues with followup bug fixes in the future.
     
  7. leutnant_kane

    leutnant_kane

    Joined:
    Apr 29, 2015
    Posts:
    40
    Hi @karl_jones here are the bug report numbers:
    Pointer Offset reappearing on window move/resize: 1395689
    RectangleContainsScreenPoint only working on first display: 1395691
    RectMask2D hiding all content when reparented from secondary display to primary: 1395695

    Your OnMouseDown issue might actually be another separate bug (or maybe connected to the second one). If you need anything else, just say the word, glad this is getting looked at :)

    PS: The repro project is the same for all three and also contains code to restore windows to different sizes/modes (as seen in WindowManager.PopOut)
     
    karl_jones likes this.
  8. leutnant_kane

    leutnant_kane

    Joined:
    Apr 29, 2015
    Posts:
    40
    Hi @karl_jones I do have an update and a couple of questions regarding the tickets:
    The RectMask2D bug (1395695) was reproduced => so far so good

    But the other two Bug Report Processes have left me a bit baffled.

    Pointer Offset reappearing on window move/resize: 1395689
    This was closed as being the same ticket as the one below. I think we can agree that this is absolutely not the case. This was the problem that pointer offsets regarding the primary window position/size/resolution aren't updated (properly) at runtime, leading to wrong interaction areas on secondary screens as soon as you move/resize the primary window. (only visible after applying your fix, because otherwise pointer positions are broken anyway if not everything is same resolution,scaling, size, etc).

    RectangleContainsScreenPoint only working on first display: 1395691

    First I received a message that I would need to calculate the proper mouse position myself by dividing it through the resolution of the target window. Not only is this contrary to your statement in post 156 (that the regular Input.mousePosition should be used for this and calculations should happen internally) but it is also wrong. The result is not in screen space anymore and it does nothing to account for the offset. I dug around a bit and found Display.RelativeMouseAt(Input.mousePosition) which should supposedly give me valid mouse coordinates (and the targeted display at the z axis). Still unsure why I should do an extra query for that, it appears better suited to be handled internally when calling Input.mousePosition, but I've tried anyway. Turns out this is buggy as well. The function seemingly can't handle a non fullscreen primary window, as it then only returns Vector3.zero for parts of all displays (related to size and position of the primary window, but not 1:1) and offset/wrong positions for all other parts.

    I've added these findings to the ticket and received the following response:
    "...input on any display other than Display One is not supported at this time. Any input that works on secondary displays is coincidental"

    I seriously don't quite know how to react to this. I know mouse input (and all kinds of other things) are broken on secondary displays, but I thought that's what this entire endeavour is about. Fixing bugs that happen during multi-display use. That's why I'm testing your fixes, why I'm writing these tickets and why I'm pushing this conversation. My company is doing multi-display applications, but unitys multi-display is very buggys and broken. Maintaining a heavily customized fork of parts of the engine is a possibility, but the projects are really long lived, which would make this a maintenance nightmare. So we're trying to get these fixes into official unity patches. I hope this is just a case of miscommunication and the goal is still to provide unity with proper multi-display support. Any kind of clearing this up would be much appreciated.
     
    cecarlsen and Noisecrime like this.
  9. karl_jones

    karl_jones

    Unity Technologies

    Joined:
    May 5, 2015
    Posts:
    8,142
    Hi,
    Looks like there's a lot to go through to understand all these issues and a lot of internal messages. I'll try and take a look later this week and see if I can understand what's going on.
     
    Noisecrime likes this.
  10. karl_jones

    karl_jones

    Unity Technologies

    Joined:
    May 5, 2015
    Posts:
    8,142
    I'm not sure why 1395689 was closed. The RectMask2D issue seems to be unrelated to multiple displays, if you change all the cameras to display 1 it still happens, so that's a whole different issue. Ill raise a separate bug for it. https://issuetracker.unity3d.com/product/unity/issues/guid/1420114/

    Ill try and dig around more in this example and see what is happening.

    1395691. There seems to be some miscommunication going on here. ugui does support input on multiple displays and we have been fixing issues over the years to ensure this. I have asked the team for some clarification.
    We may be starting to get into some territory that is not officially supported, resizing windows with different resolutions etc. If that is the case then we should make it clear what is and what is not supported. I know some areas do not currently have support, the new Input system does not.

    I noticed in your bug example you switch from screenspace overlay to screen space camera for the displays. Screenspace overlay seems to work but the camera mode would appear to be the problematic area. I suspect the camera is not doing some conversions somewhere.
     
    Last edited: Apr 14, 2022
    leutnant_kane and Noisecrime like this.
  11. karl_jones

    karl_jones

    Unity Technologies

    Joined:
    May 5, 2015
    Posts:
    8,142
    Hey. Last update ;)

    I have found some bugs in your code that will explain the issues you are having, it's possible that after fixing these issues it may work...

    I noticed that in your sample project you are showing a Screen Space - Overlay canvas in Display 1, you then switch to a Screen Space - Camera overlay in Display 2 and 3. This is fine however you are using the Screen Space - Overlay versions of RectTransformUtility.
    For example in your Hover.cs script you do:

    Code (csharp):
    1. void Update()
    2. {
    3.     Vector2 mousePos = Input.mousePosition;
    4.     if (previousMousePosition != mousePos)
    5.     {
    6.         inside = RectTransformUtility.RectangleContainsScreenPoint(rt, mousePos);
    7.         insideSignifier.text = inside ? "Inside" : "Outside";
    8.         previousMousePosition = mousePos;
    9.     }
    10. }
    There are 2 problems here.
    1. You don't use the correct version of RectangleContainsScreenPoint, you need to pass in the optional camera when using Screen Space - Camera.
    2. You need to convert the mouse to the other display using Display.RelativeMouseAt(or the modified version I provided earlier).

    Here is a quick hack I made that now works:

    Code (csharp):
    1.  
    2. void Update()
    3. {
    4.     Vector2 mousePos = Input.mousePosition;
    5.     if (previousMousePosition != mousePos)
    6.     {
    7.         var canvas = rt.GetComponentInParent<Canvas>();
    8.         Camera cam = null;
    9.         if (canvas != null && canvas.renderMode == RenderMode.ScreenSpaceCamera)
    10.             cam = canvas.worldCamera;
    11.  
    12.         inside = RectTransformUtility.RectangleContainsScreenPoint(rt, Display.RelativeMouseAt(mousePos), cam);
    13.         insideSignifier.text = inside ? "Inside" : "Outside";
    14.         previousMousePosition = mousePos;
    15.     }
    16. }
    Im sure you can find a cleaner way to pass the Camera into your scripts, this is just to show the issue. I think if you go through and fix the other scripts to do the same then you should solve most of the issues.


    The RectMast2D issue seems to be due to the OnCanvasHierarchyChanged message not being fired when you reparent it. It gets fired when you disable/enable a Canvas but you never disable the original canvas. Its a bit of a misleading name as changing the child objects hierarchy does not trigger this message, the OnTransformParentChanged however does. It looks like we need to clear the cached canvas inside of RectMask2D.OnTransformParentChanged. Ill fix this.

    Edit:

    Here is a patch for the fix if you need it now

    Code (CSharp):
    1. ---
    2. Packages/com.unity.ugui/Runtime/UI/Core/RectMask2D.cs | 1 +
    3. 1 file changed, 1 insertion(+)
    4.  
    5. diff --git a/Packages/com.unity.ugui/Runtime/UI/Core/RectMask2D.cs b/Packages/com.unity.ugui/Runtime/UI/Core/RectMask2D.cs
    6. index 94cee37c02c5..443acdb65458 100644
    7. --- a/Packages/com.unity.ugui/Runtime/UI/Core/RectMask2D.cs
    8. +++ b/Packages/com.unity.ugui/Runtime/UI/Core/RectMask2D.cs
    9. @@ -344,6 +344,7 @@ namespace UnityEngine.UI
    10.          protected override void OnTransformParentChanged()
    11.          {
    12. +            m_Canvas = null;
    13.              base.OnTransformParentChanged();
    14.              m_ShouldRecalculateClipRects = true;
    15.          }
    16. --
     
    Last edited: Apr 19, 2022
    leutnant_kane and Noisecrime like this.
  12. leutnant_kane

    leutnant_kane

    Joined:
    Apr 29, 2015
    Posts:
    40
    Hi @karl_jones and thank you for the investigation.
    I feel it's on me now to clear some things up.

    RectMask2D issue: This was the one that is on track. Interesting to see that the specific displays don't matter, but as far as I can see there is already a fix in review:
    https://issuetracker.unity3d.com/is...play-to-first-dislpay-in-the-game-view-window
    Edit: seems you have found it as well, thx for sharing the fix


    RectangleContainsScreenpoint issue:
    Thanks for the camera info, I did miss that. But the mouse position is still a problem. Neither Display.RelativeMouseAt, nor your internal MultipleDisplayUtilities.RelativeMouseAtScaled work across displays. Display.RelativeMouseAt only returns valid coordinates for the window on the main display (and certain directions going out of it). For the rest of the first display and large parts of the secondary display (seemingly linked to size/position/resolution) of the first display it just returns (0,0,0). RelativeMouseAtScaled does a bit better and provides coordinates for the full first display (including outside the window) but it also only returns (0,0,0) for parts of the second screen (also linked to size/position/resolution of the first it seems). I have attached a mockup of my test setup.

    Offsets breaking on move issue (1395689)
    That is the ticket that was closed without explanation. Of course this is a basic requirement of building proper multi-display (or even multi-window) applications in unity, so as you said it would be amazing to get a clear answer here wether this is considered bugged, planned or not a goal of unity at all.

    So basically my remaining issues are mouse positions not returning a value at all for some parts of multi display setups and when they return a value, it will be wrong as soon as a window changes in any way.

    Hope this helps clear things up and thanks for your time.
     

    Attached Files:

    Last edited: Apr 20, 2022
  13. karl_jones

    karl_jones

    Unity Technologies

    Joined:
    May 5, 2015
    Posts:
    8,142
    Yes the fix in review is the one I posted ;)

    I have gone through your sample project and all the issues seem fixed for me now. I had to make 1 small change to
    RectTransformSizeHandle.CalculateNewSizeFromMousePosition
    . It was also missing the camera.
    My hack
    Code (csharp):
    1. private Vector2 CalculateNewSizeFromMousePosition(Vector2 mousePosition, Vector2 newSize)
    2. {
    3.     var canvas = rectTransform.GetComponentInParent<Canvas>();
    4.     Camera cam = null;
    5.     if (canvas != null && canvas.renderMode == RenderMode.ScreenSpaceCamera)
    6.         cam = canvas.worldCamera;
    7.  
    8.  
    9.     //RectTransformUtility.ScreenPointToLocalPointInRectangle(rectTransform, Display.RelativeMouseAt(mousePosition), null, out var localPoint);
    10.     RectTransformUtility.ScreenPointToLocalPointInRectangle(rectTransform, mousePosition, cam, out var localPoint);
    11.     if (mayDragWidth)
    12.         newSize.x = localPoint.x * rectTransform.localScale.x * (invertX ? -1 : 1);
    13.     if (mayDragHeight)
    14.         newSize.y = localPoint.y * rectTransform.localScale.y * (invertY ? -1 : 1);
    15.     return newSize;
    16. }
    After this everything seems to work correctly
    - The Hover with inside/outside works.
    - The RectMask2D works with the above fix which will also land in Unity in the future.
    - The resize of the window works with my change above.
    - Dragging the window seems fine, I think this always worked for me.

    Can you walk me through a set of steps to reproduce the issue you have or maybe provide another example project? The window mouse position was correct after I did the RelativePositionAtScaled fix. If its not working I will need an example as I tested every possible combination I could when I did the fix.
     
    leutnant_kane likes this.
  14. leutnant_kane

    leutnant_kane

    Joined:
    Apr 29, 2015
    Posts:
    40
    I now realize I have probably made this whole discussion a lot more difficult than it needs to be by including classes in my sample project that are called windowmanager and which create windows. I used these because they show the issues nicely, were reducable for a sample project and allow dragging a panel around the screen to test input stuff. I didn't realize their names collide with the problems I'm trying to describe. I feel a bit daft for not noticing this sooner.

    So quick definitions from my side:
    Display -> The physical monitor/displays the application is running/showing on
    Window -> The application window (windows if multiples active - 1 per Display, can be dragged/resized/fullscreen)
    Panel -> The Panel in the Application that shows the errors and can be dragged/resized/reparented etc.

    Pointer offsets/positions break after resizing or moving a window (1395689)

    Display.RelativeMouseAt and its scaled variant returns (0,0,0) for parts of any display and secondary window. I suspect the area where this function returns valid results is for some reason depending on the size/resolution/position of the primary window on the primary Display.

    The functions on the panel "work fine" (after fixing the camera and changing to the scaled relative position) for as long as none of the windows is changed in any way after creation/activation and as long as the panels are placed somewhere in the middle of the windows (which is where RelativeMouseAt seems to work).

    I have attached two images to try and explain this better

    Sorry for confusing the issue, I hope this cleared things up.
     

    Attached Files:

  15. karl_jones

    karl_jones

    Unity Technologies

    Joined:
    May 5, 2015
    Posts:
    8,142
    I think a lot of these issues are down to using multiple displays in windowed mode. As far as I'm aware we don't support this, I had to check with myself here ;) https://forum.unity.com/threads/multi-display-windowed-mode.429757/#post-2801070
    Are you resizing the window whilst in multiple display mode? How are you doing this? When I switch I lose the windowing and have it fixed in the center without any resizing.

    I tried to reproduce the 0,0 issue but the scaled version is working for me.
    Here is a video of me trying


    The left is a resizable windowed view. On the right is display 2 which shows a fullscreen 3440x1440 view.
    The bottom is the window zoomed in so you can see the numbers of the relative at(broken) and the scaled version.