Search Unity

  1. We are migrating the Unity Forums to Unity Discussions. On July 12, the Unity Forums will become read-only. On July 15, Unity Discussions will become read-only until July 18, when the new design and the migrated forum contents will go live. Read our full announcement for more information and let us know if you have any questions.

Multi-Display Canvases Not Working in 5.4.2

Discussion in 'Editor & General Support' started by led_bet, Nov 3, 2016.

  1. led_bet

    led_bet

    Joined:
    May 5, 2015
    Posts:
    83
    Despite saying this issue was resolved in 5.4.2, I'm unable to register input across displays. I currently have 3 displays in one scene, with UI elements being rendered in camera space for each display. The UI Event system is only registering on one display. Switching to the other two displays in the game tab does not effect the UI Event System.

    For example, the Event System registers a button click on Display 2. When I switch the game tab to Display 3, the Event System still registers all clicks for Display 2.

    Any help/ work arounds would be greatly appreciated.

    EDIT: This users issue describes my problem exactly, if this helps illustrate the bug in question:

    http://answers.unity3d.com/questions/1111243/53-multiple-display-canvases.html

    EDIT 2: Changing the Canvas to screen space and specifying which display to render on does not resolve the issue. Changing the Canvas sorting layer to be the highest value sets that particular canvas to be "active" allowing for interaction.
     
    Last edited: Nov 3, 2016
  2. karl_jones

    karl_jones

    Unity Technologies

    Joined:
    May 5, 2015
    Posts:
    8,570
    What OS is this on? Can you file a bug report with an example project and post the number here?
     
  3. led_bet

    led_bet

    Joined:
    May 5, 2015
    Posts:
    83
    Absolutely. Case 849146. The game was developed on Mac OS X El Capitan 10.11.5, but built and deployed for Windows Stand Alone. When testing it on the Windows machine, it was only registering input clicks on one screen, and not intercepting button presses on the other screens, as seen in the editor. Slightly different bug, but presumably related.
     
  4. karl_jones

    karl_jones

    Unity Technologies

    Joined:
    May 5, 2015
    Posts:
    8,570
    Thanks. I suspect the issue is with Display.RelativeMouseAt. This is a platform specific function that is currently only implemented on Windows player. I do not believe we have an editor implementation of this currently. This bug report should help get the ball rolling.
     
  5. led_bet

    led_bet

    Joined:
    May 5, 2015
    Posts:
    83
    Glad to help. In the meantime, is there a workaround? Or will developing natively on the windows machine resolve the problem?
     
    Last edited: Nov 7, 2016
  6. karl_jones

    karl_jones

    Unity Technologies

    Joined:
    May 5, 2015
    Posts:
    8,570
    Developing natively on the windows machine should give you the best results, I know its a bit of a pain having to rebuild each time though. You could also try and create a wrapper around Display.RelativeMouseAt to do it in the editor. I'll have to have a think how this will work though, its likely how the final fix will work.
     
  7. led_bet

    led_bet

    Joined:
    May 5, 2015
    Posts:
    83
    Did a quick test on a Windows machine, placing a UI button in each of three displays. Activate said displays via script. The default event system still only registers clicks on one screen. I don't know if this is excepted behavior, but I thought I'd let you know. Otherwise, I'll write some input interception code using the relative coordinate method you mentioned earlier. As far as I can tell, the current multi-display implementation isn't documented thoroughly. I'm getting hung up on unspecified implementation requirements. Any additional information the Unity team could provide about how to get the internal event system to register on all three screens would be great. Thank you for your effort to fix this issue!
     
  8. karl_jones

    karl_jones

    Unity Technologies

    Joined:
    May 5, 2015
    Posts:
    8,570
    I just knocked up this editor emulation script. Its only had a few seconds of testing but worth trying
    It basically finds the active gameview and returns the display ID for it.
    Code (CSharp):
    1.     Vector3 RelativeMouseAt()
    2.     {
    3.         System.Reflection.Assembly assembly = typeof(UnityEditor.EditorWindow).Assembly;
    4.         Type type = assembly.GetType("UnityEditor.GameView");
    5.  
    6.         var lastGameViewField = type.GetField("s_LastFocusedGameView", BindingFlags.Static | BindingFlags.NonPublic);
    7.         var gv = lastGameViewField.GetValue(null);
    8.  
    9.         int displayID = 0;
    10.         if (gv != null)
    11.         {
    12.             var displayField = type.GetField("m_TargetDisplay", BindingFlags.NonPublic | BindingFlags.Instance);
    13.             displayID = (int)displayField.GetValue(gv);
    14.         }
    15.  
    16.         var pos = Input.mousePosition;
    17.         pos[2] = displayID;
    18.         return pos;
    19.     }
    I have been working on writing a blog post on the multiple display system and its finer points as well as helping to clean it up and get it into better shape.
     
  9. karl_jones

    karl_jones

    Unity Technologies

    Joined:
    May 5, 2015
    Posts:
    8,570
    That's strange. We re enabled multiple display support for UI in 5.4.1p2 so any version after that should work. Is RelativeMouseAt working correctly on the windows builds?
     
  10. led_bet

    led_bet

    Joined:
    May 5, 2015
    Posts:
    83
    Taking a look at it now. Having some compilation issues (assuming it's C#). Do I need to specify any additional API's for this code? Or is this JavaScript (it doesn't seem to compile as a JavaScript file either)?

    About the Windows issues, I'm currently using 5.4.2f2 on the windows machine. I have three cameras and three canvases. Each canvas is assigned to its unique camera, each camera is told to render to its own unique display. The actual display renders properly on each screen. When I bring the mouse to screen 1 or 3, no input is registered. Input only is received on the main screen itself.
     
  11. karl_jones

    karl_jones

    Unity Technologies

    Joined:
    May 5, 2015
    Posts:
    8,570
    The code is c#.
    Maybe you are missing a using?
    Code (CSharp):
    1. using System;
    2. using System.Reflection;
    3. using UnityEngine;
     
  12. karl_jones

    karl_jones

    Unity Technologies

    Joined:
    May 5, 2015
    Posts:
    8,570
    Here is an improved emulation script that does not require the GameView to be active

    Code (CSharp):
    1.     Vector3 RelativeMouseAt()
    2.     {
    3.         var mouseOverWindow = EditorWindow.mouseOverWindow;
    4.         System.Reflection.Assembly assembly = typeof(UnityEditor.EditorWindow).Assembly;
    5.         Type type = assembly.GetType("UnityEditor.GameView");
    6.  
    7.         int displayID = 0;
    8.         if (type.IsInstanceOfType(mouseOverWindow))
    9.         {
    10.             var displayField = type.GetField("m_TargetDisplay", BindingFlags.NonPublic | BindingFlags.Instance);
    11.             displayID = (int)displayField.GetValue(mouseOverWindow);
    12.         }
    13.  
    14.         var pos = Input.mousePosition;
    15.         pos[2] = displayID;
    16.         return pos;
    17.     }
     
    cecarlsen likes this.
  13. led_bet

    led_bet

    Joined:
    May 5, 2015
    Posts:
    83

    Yea I was missing the Reflection include. I'll give this a shot when I'm back near the windows machine and report back in about thirty minutes. Thank you Karl, you've been a great help.
     
    karl_jones likes this.
  14. led_bet

    led_bet

    Joined:
    May 5, 2015
    Posts:
    83
    Karl,

    Your code works great in-editor.

    For further detail on the event input bug when built: the event system registers input on Display One, but for input on the UI of Display Two. Very bizarre. Clicking on UI elements from Display Two or Three does not register any input whatsoever. The build is in 4.5.2 on Windows 7.
     
    Last edited: Nov 7, 2016
  15. Ecocide

    Ecocide

    Joined:
    Aug 4, 2011
    Posts:
    293
    Glad there is already a thread about this.

    I found out that multiple canvases on multiple displays work, but only if the canvas is set to Screen Space - Overlay. As soon set the canvas on the second (or third, etc...) display to World Space, the input on the non-primary display is not recognized at all.
    This only happens in a build, in editor it also works with World Space mode.

    Karl, is it intended, the input only works in Screen Space - Overlay?

    Thanks for your help in this matter.
     
  16. karl_jones

    karl_jones

    Unity Technologies

    Joined:
    May 5, 2015
    Posts:
    8,570
    No that sounds like a bug. Could you file a bug report please.
     
  17. Ecocide

    Ecocide

    Joined:
    Aug 4, 2011
    Posts:
    293
    Case number: 852409
     
    karl_jones likes this.
  18. Ecocide

    Ecocide

    Joined:
    Aug 4, 2011
    Posts:
    293
    Hi Karl,

    the bug was immediately confirmed on that day. Now, it says "Fixed in future release"... unfortunately, it's not fixed in the current 5.5.0f3.
    Is there any more detailed info on this? Is it easily fixable or do we have to wait another major release?
     
  19. karl_jones

    karl_jones

    Unity Technologies

    Joined:
    May 5, 2015
    Posts:
    8,570
    It should be possible for the fix to go into a patch. I will check tomorrow.
     
  20. Ecocide

    Ecocide

    Joined:
    Aug 4, 2011
    Posts:
    293
    Did you check? :)
     
  21. Ecocide

    Ecocide

    Joined:
    Aug 4, 2011
    Posts:
    293
    Hi Karl,

    unfortunately it's still not working and it is not part of the newest patch or even Beta version. Could you please check if this will be part of an upcoming patch?

    Thank you!
     
  22. karl_jones

    karl_jones

    Unity Technologies

    Joined:
    May 5, 2015
    Posts:
    8,570
    The fix is in both 5.5 and 5.6 beta.
    The developer tells me the fix is to use the camera target display in world and screen space camera.
     
  23. Ecocide

    Ecocide

    Joined:
    Aug 4, 2011
    Posts:
    293
    ? I don't really know what you mean. I have two cameras with each set to Display 1 and the next to Display 2. Then I have canvas 1 with event camera set to camera 1 that is targeting display 1 and canvas 2 with event camera set to camera 2 that is targeting display 2. If works perfectly in the editor, but not in a build.

    I tried the same project I uploaded as a bug report in both 5.5p1 and 5.6 beta. Not working.
     
  24. karl_jones

    karl_jones

    Unity Technologies

    Joined:
    May 5, 2015
    Posts:
    8,570
    What mode are the canvases set to?
     
  25. Ecocide

    Ecocide

    Joined:
    Aug 4, 2011
    Posts:
    293
    Well, this is strange... Once I set it to Screen Space - Overlay and back to World Space it is indeed working. Maybe the reason was I did not start a fresh project but imported and upgraded the old one I used for testing (the one from the bug report)?

    Thanks for your time, karl. Great support!!!
     
    karl_jones likes this.
  26. Ecocide

    Ecocide

    Joined:
    Aug 4, 2011
    Posts:
    293
    All the input handling on Multi Display setups is still so buggy... Sometimes it's working in the build, but not in the editor, sometimes it's working in the editor, but not in a build. I tried several different setups. It does not matter if it's World Space or Overlay, there are some serious issues. I know that you want me to file another bug report, but just take the one I already posted and play with it. Even the one I posted does not work in 5.5.0p3, yet you guys say it's fixed and should work... Well, it does not.
     
  27. ASIM-SENYUVA

    ASIM-SENYUVA

    Joined:
    Apr 29, 2013
    Posts:
    90
  28. Fab-London

    Fab-London

    Joined:
    Feb 6, 2013
    Posts:
    35
    Hi,

    I am having the same challenges, this is a shame this is not resolved in the editor, I am running 2018.4.14f1 and wasting a lot of time to build the solution each time.

    I do not quite understand where you need to attach the work around you provided in this forum

    Thanks
    Fabien
     
  29. danbg

    danbg

    Joined:
    May 1, 2017
    Posts:
    65
    Hi @karl_jones and @ledbetterman I'm having the same problem as @Fab-London I'm not sure how to use your workaround. Could you please explain what we should do with your code? I'm using 2019.4.0f1 and Windows 10. Thanks.
     
  30. karl_jones

    karl_jones

    Unity Technologies

    Joined:
    May 5, 2015
    Posts:
    8,570
    The workaround should be used in place of Display.RelativeMouseAt in the editor.
    If you want an existing component to work(such as UI) then you will need to go and edit the UI package code and replace uses of Display.RelativeMouseAt with this when running in the Editor (use the UNITY_EDITOR define)
     
    led_bet likes this.
  31. danbg

    danbg

    Joined:
    May 1, 2017
    Posts:
    65
    Hi @karl_jones thanks for your answer. I'm not very experienced with the UI internals in Unity, so forgive my ignorance if I say something silly here. Just in case someone wants to try this too, I found 3 appearances in the UI package (com.unity.ugui): PhysicsRaycaster.cs GraphicRaycaster,cs and MultipleDisplayUtilities.cs
    In the first 2 I could find:
    Code (CSharp):
    1.  
    2.             var eventPosition = Display.RelativeMouseAt(eventData.position);
    3.             if (eventPosition != Vector3.zero)
    4.             {
    5.                 // We support multiple display and display identification based on event position.
    6.  
    7.                 int eventDisplayIndex = (int)eventPosition.z;
    8.  
    9.                 // Discard events that are not part of this display so the user does not interact with multiple displays at once.
    10.                 if (eventDisplayIndex != displayIndex)
    11.                     return;
    12.             }
    13.  
    And in the third one in 2 places, first for dragging:
    Code (CSharp):
    1.  
    2.         public static bool GetRelativeMousePositionForDrag(PointerEventData eventData, ref Vector2 position)
    3.         {
    4. ...
    5.             var relativePosition = Display.RelativeMouseAt(eventData.position);
    6. ...
    7.  
    And then this one. Maybe this could be a better place to mod:
    Code (CSharp):
    1.  
    2.         public static Vector2 GetMousePositionRelativeToMainDisplayResolution()
    3.         {
    4.             var position = Input.mousePosition;
    5.             #if !UNITY_EDITOR
    6.             if (Display.main.renderingHeight != Display.main.systemHeight)
    7.             {
    8.                 // The position is relative to the main render area, we need to adjust this so
    9.                 // it is relative to the system resolution in order to correctly determine the position on other displays.
    10.  
    11.                 // Correct the y position if we are outside the main display.
    12.                 if (position.y < 0 || position.y > Display.main.renderingHeight ||
    13.                     position.x < 0 || position.x > Display.main.renderingWidth)
    14.                 {
    15.                     position.y += Display.main.systemHeight - Display.main.renderingHeight;
    16.                 }
    17.             }
    18.             #endif
    19.             return position;
    20.         }
    21.  

    In any case, I modified GraphicsRaycaster.cs this way and it seems to work, but it gives an error detecting the correct display window:
    Code (CSharp):
    1.  
    2. //Added at the beginning
    3. using UnityEditor;
    4. using System.Reflection;
    5. ...
    6.             var eventPosition = Display.RelativeMouseAt(eventData.position);
    7. //Added
    8. #if UNITY_EDITOR
    9.             eventPosition = RelativeMouseAt();
    10. #endif
    11. ...
    12. //Added at the end
    13.         Vector3 RelativeMouseAt()
    14.         {
    15.             var mouseOverWindow = EditorWindow.mouseOverWindow;
    16.             System.Reflection.Assembly assembly = typeof(UnityEditor.EditorWindow).Assembly;
    17.             Type type = assembly.GetType("UnityEditor.GameView");
    18.    
    19.             int displayID = 0;
    20.             if (type.IsInstanceOfType(mouseOverWindow))
    21.             {
    22.                 var displayField = type.GetField("m_TargetDisplay", BindingFlags.NonPublic | BindingFlags.Instance);
    23.                 displayID = (int)displayField.GetValue(mouseOverWindow);
    24.             }
    25.    
    26.             var pos = Input.mousePosition;
    27.             pos[2] = displayID;
    28.             return pos;
    29.         }
    30. ...
    31.  
    The problem is within "displayID = (int)displayField.GetValue(mouseOverWindow);" giving a NullReferenceException error. If I put manually a displayID it works great, and I checked that mouseOverWindow gives a value, but displayField seems to not work. Any tips? Thank you again.
     
  32. danbg

    danbg

    Joined:
    May 1, 2017
    Posts:
    65
    Just in case someone is still interested, the problem was using "UnityEditor.GameView" instead of "UnityEditor.PlayModeView". At least in Unity 2019.4.0f1 you have to change Runtime\UI\Core\GraphicRaycaster.cs and use this:
    Code (CSharp):
    1.  
    2. //Added at the beginning
    3. using UnityEditor;
    4. using System.Reflection;
    5. ...
    6.             var eventPosition = Display.RelativeMouseAt(eventData.position);
    7. //Added
    8. #if UNITY_EDITOR
    9.             eventPosition = RelativeMouseAt();
    10. #endif
    11. ...
    12. //Added at the end
    13. #if UNITY_EDITOR
    14.         Vector3 RelativeMouseAt()
    15.         {
    16.             var mouseOverWindow = EditorWindow.mouseOverWindow;
    17.             System.Reflection.Assembly assembly = typeof(UnityEditor.EditorWindow).Assembly;
    18.             Type type = assembly.GetType("UnityEditor.PlayModeView");
    19.  
    20.             int displayID = 0;
    21.             if (type.IsInstanceOfType(mouseOverWindow))
    22.             {
    23.                 var displayField = type.GetField("m_TargetDisplay", BindingFlags.NonPublic | BindingFlags.Instance);
    24.                 displayID = (int)displayField.GetValue(mouseOverWindow);
    25.             }
    26.  
    27.             var pos = Input.mousePosition;
    28.             pos[2] = displayID;
    29.             return pos;
    30.         }
    31. #endif
    32. ...
    33.  
     
  33. cecarlsen

    cecarlsen

    Joined:
    Jun 30, 2006
    Posts:
    874
    Hi @karl_jones

    Are there plans to integrate your neat fix into the engine itself? So that Display.RelativeMouseAt works both in player and in Editor?

    EDIT: Ah, damn. The workaround no longer works in 2020.1. The fields "s_LastFocusedGameView" and "m_TargetDisplay" no longer seem to exist in GameView.
     
    Last edited: Jul 17, 2020
    led_bet likes this.
  34. cecarlsen

    cecarlsen

    Joined:
    Jun 30, 2006
    Posts:
    874
    I've updated the @karl_jones workaround to 2020.1. I hope I can remove this hack in a soon to come version of Unity.

    Code (CSharp):
    1. using System;
    2. using System.Reflection;
    3. using UnityEngine;
    4.  
    5. #if UNITY_EDITOR
    6. [UnityEditor.InitializeOnLoad] // Request static constructor call in Editor
    7. #endif
    8. public static class DisplayExtensions
    9. {
    10.     #if UNITY_EDITOR
    11.     static FieldInfo lastFocusedFieldInfo;
    12.     static FieldInfo targetDisplayFieldInfo;
    13.  
    14.     static DisplayExtensions()
    15.     {
    16.         Assembly assembly = typeof( UnityEditor.EditorWindow ).Assembly;
    17.  
    18.         // Source: https://github.com/Unity-Technologies/UnityCsReference/blob/master/Editor/Mono/PlayModeView/PlayModeView.cs
    19.         Type playModeViewType = assembly.GetType( "UnityEditor.PlayModeView" );
    20.         lastFocusedFieldInfo = playModeViewType.GetField( "s_LastFocused", BindingFlags.NonPublic | BindingFlags.Static );
    21.         targetDisplayFieldInfo = playModeViewType.GetField( "m_TargetDisplay", BindingFlags.NonPublic | BindingFlags.Instance );
    22.     }
    23.     #endif
    24.  
    25.  
    26.     public static Vector3 RelativeMouseAt()
    27.     {
    28.         Vector3 pos = Input.mousePosition;
    29.  
    30.         #if UNITY_EDITOR
    31.         object playModeView = lastFocusedFieldInfo.GetValue( null );
    32.         int displayIndex = 0;
    33.         if( playModeView != null ) displayIndex = (int) targetDisplayFieldInfo.GetValue( playModeView );
    34.         pos.z = displayIndex;
    35.         #else
    36.         pos = Display.RelativeMouseAt( pos );
    37.         #endif
    38.  
    39.         return pos;
    40.     }
    41. }
     
  35. ikriz

    ikriz

    Joined:
    Dec 3, 2009
    Posts:
    99
    led_bet and karl_jones like this.
  36. dananqqdeqd

    dananqqdeqd

    Joined:
    Nov 17, 2022
    Posts:
    3
    Hi Karl, I wonder when will this method supports Android platform ?