Multi-Display Canvases Not Working in 5.4.2

Despite saying this issue was resolved in 5.4.2, I'm unable to register input across displays. I currently have 3 displays in one scene, with UI elements being rendered in camera space for each display. The UI Event system is only registering on one display. Switching to the other two displays in the game tab does not effect the UI Event System.

For example, the Event System registers a button click on Display 2. When I switch the game tab to Display 3, the Event System still registers all clicks for Display 2.

Any help/ work arounds would be greatly appreciated.

EDIT: This users issue describes my problem exactly, if this helps illustrate the bug in question:

http://answers.unity3d.com/questions/1111243/53-multiple-display-canvases.html

EDIT 2: Changing the Canvas to screen space and specifying which display to render on does not resolve the issue. Changing the Canvas sorting layer to be the highest value sets that particular canvas to be "active" allowing for interaction.

 

Absolutely. Case 849146. The game was developed on Mac OS X El Capitan 10.11.5, but built and deployed for Windows Stand Alone. When testing it on the Windows machine, it was only registering input clicks on one screen, and not intercepting button presses on the other screens, as seen in the editor. Slightly different bug, but presumably related.

 

Glad to help. In the meantime, is there a workaround? Or will developing natively on the windows machine resolve the problem?

 

Did a quick test on a Windows machine, placing a UI button in each of three displays. Activate said displays via script. The default event system still only registers clicks on one screen. I don't know if this is excepted behavior, but I thought I'd let you know. Otherwise, I'll write some input interception code using the relative coordinate method you mentioned earlier. As far as I can tell, the current multi-display implementation isn't documented thoroughly. I'm getting hung up on unspecified implementation requirements. Any additional information the Unity team could provide about how to get the internal event system to register on all three screens would be great. Thank you for your effort to fix this issue!

 

Taking a look at it now. Having some compilation issues (assuming it's C#). Do I need to specify any additional API's for this code? Or is this JavaScript (it doesn't seem to compile as a JavaScript file either)?

About the Windows issues, I'm currently using 5.4.2f2 on the windows machine. I have three cameras and three canvases. Each canvas is assigned to its unique camera, each camera is told to render to its own unique display. The actual display renders properly on each screen. When I bring the mouse to screen 1 or 3, no input is registered. Input only is received on the main screen itself.

 

Yea I was missing the Reflection include. I'll give this a shot when I'm back near the windows machine and report back in about thirty minutes. Thank you Karl, you've been a great help.

 

Karl,

Your code works great in-editor.

For further detail on the event input bug when built: the event system registers input on Display One, but for input on the UI of Display Two. Very bizarre. Clicking on UI elements from Display Two or Three does not register any input whatsoever. The build is in 4.5.2 on Windows 7.

 

Glad there is already a thread about this.

I found out that multiple canvases on multiple displays work, but only if the canvas is set to Screen Space - Overlay. As soon set the canvas on the second (or third, etc...) display to World Space, the input on the non-primary display is not recognized at all.
This only happens in a build, in editor it also works with World Space mode.

Karl, is it intended, the input only works in Screen Space - Overlay?

Thanks for your help in this matter.

 

Case number: 852409

 

Hi Karl,

the bug was immediately confirmed on that day. Now, it says "Fixed in future release"... unfortunately, it's not fixed in the current 5.5.0f3.
Is there any more detailed info on this? Is it easily fixable or do we have to wait another major release?

 

Did you check?

 

Hi Karl,

unfortunately it's still not working and it is not part of the newest patch or even Beta version. Could you please check if this will be part of an upcoming patch?

Thank you!

 

? I don't really know what you mean. I have two cameras with each set to Display 1 and the next to Display 2. Then I have canvas 1 with event camera set to camera 1 that is targeting display 1 and canvas 2 with event camera set to camera 2 that is targeting display 2. If works perfectly in the editor, but not in a build.

I tried the same project I uploaded as a bug report in both 5.5p1 and 5.6 beta. Not working.

 

Well, this is strange... Once I set it to Screen Space - Overlay and back to World Space it is indeed working. Maybe the reason was I did not start a fresh project but imported and upgraded the old one I used for testing (the one from the bug report)?

Thanks for your time, karl. Great support!!!

 

All the input handling on Multi Display setups is still so buggy... Sometimes it's working in the build, but not in the editor, sometimes it's working in the editor, but not in a build. I tried several different setups. It does not matter if it's World Space or Overlay, there are some serious issues. I know that you want me to file another bug report, but just take the one I already posted and play with it. Even the one I posted does not work in 5.5.0p3, yet you guys say it's fixed and should work... Well, it does not.

 

problem still continues. I have multi-display and multi canvas. If ı choose Screenspace overlay and set target display its looks working but event system makes the problem for UI event detection. My problem at the below link. On the other hand, I try to set WordSpace for using event camera for Display2. but its only works on display 1 attached camera.

https://forum.unity.com/threads/multiple-camera-with-multiple-canvas-ui-button-event-problem.747788/

 

Hi,

I am having the same challenges, this is a shame this is not resolved in the editor, I am running 2018.4.14f1 and wasting a lot of time to build the solution each time.

I do not quite understand where you need to attach the work around you provided in this forum

Thanks
Fabien

 

Hi @karl_jones and @ledbetterman I'm having the same problem as @Fab-London I'm not sure how to use your workaround. Could you please explain what we should do with your code? I'm using 2019.4.0f1 and Windows 10. Thanks.

 

Hi @karl_jones thanks for your answer. I'm not very experienced with the UI internals in Unity, so forgive my ignorance if I say something silly here. Just in case someone wants to try this too, I found 3 appearances in the UI package (com.unity.ugui): PhysicsRaycaster.cs GraphicRaycaster,cs and MultipleDisplayUtilities.cs
In the first 2 I could find:

Code (CSharp):

1.  
2.             var eventPosition = Display.RelativeMouseAt(eventData.position);
3.             if (eventPosition != Vector3.zero)
4.             {
5.                 // We support multiple display and display identification based on event position.
6.  
7.                 int eventDisplayIndex = (int)eventPosition.z;
8.  
9.                 // Discard events that are not part of this display so the user does not interact with multiple displays at once.
10.                 if (eventDisplayIndex != displayIndex)
11.                     return;
12.             }
13.  

And in the third one in 2 places, first for dragging:

Code (CSharp):

1.  
2.         public static bool GetRelativeMousePositionForDrag(PointerEventData eventData, ref Vector2 position)
3.         {
4. ...
5.             var relativePosition = Display.RelativeMouseAt(eventData.position);
6. ...
7.  

And then this one. Maybe this could be a better place to mod:

Code (CSharp):

1.  
2.         public static Vector2 GetMousePositionRelativeToMainDisplayResolution()
3.         {
4.             var position = Input.mousePosition;
5.             #if !UNITY_EDITOR
6.             if (Display.main.renderingHeight != Display.main.systemHeight)
7.             {
8.                 // The position is relative to the main render area, we need to adjust this so
9.                 // it is relative to the system resolution in order to correctly determine the position on other displays.
10.  
11.                 // Correct the y position if we are outside the main display.
12.                 if (position.y < 0 || position.y > Display.main.renderingHeight ||
13.                     position.x < 0 || position.x > Display.main.renderingWidth)
14.                 {
15.                     position.y += Display.main.systemHeight - Display.main.renderingHeight;
16.                 }
17.             }
18.             #endif
19.             return position;
20.         }
21.  

In any case, I modified GraphicsRaycaster.cs this way and it seems to work, but it gives an error detecting the correct display window:

Code (CSharp):

1.  
2. //Added at the beginning
3. using UnityEditor;
4. using System.Reflection;
5. ...
6.             var eventPosition = Display.RelativeMouseAt(eventData.position);
7. //Added
8. #if UNITY_EDITOR
9.             eventPosition = RelativeMouseAt();
10. #endif
11. ...
12. //Added at the end
13.         Vector3 RelativeMouseAt()
14.         {
15.             var mouseOverWindow = EditorWindow.mouseOverWindow;
16.             System.Reflection.Assembly assembly = typeof(UnityEditor.EditorWindow).Assembly;
17.             Type type = assembly.GetType("UnityEditor.GameView");
18.    
19.             int displayID = 0;
20.             if (type.IsInstanceOfType(mouseOverWindow))
21.             {
22.                 var displayField = type.GetField("m_TargetDisplay", BindingFlags.NonPublic | BindingFlags.Instance);
23.                 displayID = (int)displayField.GetValue(mouseOverWindow);
24.             }
25.    
26.             var pos = Input.mousePosition;
27.             pos[2] = displayID;
28.             return pos;
29.         }
30. ...
31.  

The problem is within "displayID = (int)displayField.GetValue(mouseOverWindow);" giving a NullReferenceException error. If I put manually a displayID it works great, and I checked that mouseOverWindow gives a value, but displayField seems to not work. Any tips? Thank you again.

 

Just in case someone is still interested, the problem was using "UnityEditor.GameView" instead of "UnityEditor.PlayModeView". At least in Unity 2019.4.0f1 you have to change Runtime\UI\Core\GraphicRaycaster.cs and use this:

Code (CSharp):

1.  
2. //Added at the beginning
3. using UnityEditor;
4. using System.Reflection;
5. ...
6.             var eventPosition = Display.RelativeMouseAt(eventData.position);
7. //Added
8. #if UNITY_EDITOR
9.             eventPosition = RelativeMouseAt();
10. #endif
11. ...
12. //Added at the end
13. #if UNITY_EDITOR
14.         Vector3 RelativeMouseAt()
15.         {
16.             var mouseOverWindow = EditorWindow.mouseOverWindow;
17.             System.Reflection.Assembly assembly = typeof(UnityEditor.EditorWindow).Assembly;
18.             Type type = assembly.GetType("UnityEditor.PlayModeView");
19.  
20.             int displayID = 0;
21.             if (type.IsInstanceOfType(mouseOverWindow))
22.             {
23.                 var displayField = type.GetField("m_TargetDisplay", BindingFlags.NonPublic | BindingFlags.Instance);
24.                 displayID = (int)displayField.GetValue(mouseOverWindow);
25.             }
26.  
27.             var pos = Input.mousePosition;
28.             pos[2] = displayID;
29.             return pos;
30.         }
31. #endif
32. ...
33.  

 

Hi @karl_jones

Are there plans to integrate your neat fix into the engine itself? So that Display.RelativeMouseAt works both in player and in Editor?

EDIT: Ah, damn. The workaround no longer works in 2020.1. The fields "s_LastFocusedGameView" and "m_TargetDisplay" no longer seem to exist in GameView.

 

I've updated the @karl_jones workaround to 2020.1. I hope I can remove this hack in a soon to come version of Unity.

Code (CSharp):

1. using System;
2. using System.Reflection;
3. using UnityEngine;
4.  
5. #if UNITY_EDITOR
6. [UnityEditor.InitializeOnLoad] // Request static constructor call in Editor
7. #endif
8. public static class DisplayExtensions
9. {
10.     #if UNITY_EDITOR
11.     static FieldInfo lastFocusedFieldInfo;
12.     static FieldInfo targetDisplayFieldInfo;
13.  
14.     static DisplayExtensions()
15.     {
16.         Assembly assembly = typeof( UnityEditor.EditorWindow ).Assembly;
17.  
18.         // Source: https://github.com/Unity-Technologies/UnityCsReference/blob/master/Editor/Mono/PlayModeView/PlayModeView.cs
19.         Type playModeViewType = assembly.GetType( "UnityEditor.PlayModeView" );
20.         lastFocusedFieldInfo = playModeViewType.GetField( "s_LastFocused", BindingFlags.NonPublic | BindingFlags.Static );
21.         targetDisplayFieldInfo = playModeViewType.GetField( "m_TargetDisplay", BindingFlags.NonPublic | BindingFlags.Instance );
22.     }
23.     #endif
24.  
25.  
26.     public static Vector3 RelativeMouseAt()
27.     {
28.         Vector3 pos = Input.mousePosition;
29.  
30.         #if UNITY_EDITOR
31.         object playModeView = lastFocusedFieldInfo.GetValue( null );
32.         int displayIndex = 0;
33.         if( playModeView != null ) displayIndex = (int) targetDisplayFieldInfo.GetValue( playModeView );
34.         pos.z = displayIndex;
35.         #else
36.         pos = Display.RelativeMouseAt( pos );
37.         #endif
38.  
39.         return pos;
40.     }
41. }

 

For the earlier code example Replace UnityEditor.GameView with UnityEditor.PlayModeView and bobs your uncle
See this revised gist forked off of karls gist: https://gist.github.com/ikriz/ad17a2c2d803d15b1777590fb29dc4d3

 

Hi Karl, I wonder when will this method supports Android platform ?