Question Implementing Multi-Window Setup for Windows UWP

We're currently working on multi monitor fitness simulation and require a multi-window configuration for the Windows UWP target platform which unfortunately doesn't support Multi-window natively. Our project relies onmultiple monitors and Bluetooth sensor integration with Bluetooth code that's tailored specifically for Windows UWP, and unfortunately, it doesn't function properly when applied to other target platforms.

I've explored Unity's official documentation on multi-display setups (https://docs.unity3d.com/Manual/MultiDisplay.html), but it appears that this method isn't compatible with Windows UWP, as discussed in this forum thread: https://forum.unity.com/threads/how-to-support-multiple-displays-uwp.425926/. According to the responses there, there appear to be three potential solutions and we found 2 more that we're considering:

1. One option is to acquire an asset designed for multi-monitor support in Windows UWP, such as the one available at (https://www.republicofhandball.com/assets/) as posted here. However, this solution raises some concerns since it's not on the asset store and could be a bit shady. (Questionable source.)
   
   
2. Another possibility is to create our own secondary monitor functionality using Windows API, a more hands-on approach that would require quite some coding. as described here. (Maintenance Challenge)
   
   
3. We're also contemplating the idea of rworking the Bluetooth code to function beyond the limitations of Windows UWP, potentially utilizing resources like https://github.com/adabru/BleWinrtDll or insights from this thread: https://forum.unity.com/threads/bluetooth-low-energy-on-windows-under-unity.509387/. (Strategic Shift)
   
   
4. Another approach is using Nvidia Surround alongside viewport rect with dual cameras to simulate a second monitor.
   
   
5. Lastly, we're considering the option of developing a separate application specifically designed as a secondary monitor, which could communicate with the main application via UDP or another suitable method. (Communication and Sync Issue)

Given that the most of the referenced threads above date back to 2013, there's a possibility that there's now a different, built-in solution available.

We need your guidance: Are we overlooking any potential approaches, or is there a recommended solution among these? Your insights would be greatly appreciated.

 

In my testing, I encountered some issues. The 2. Window stopped responding, and didn't allow for interaction such as resizing. However, it's worth considering that these problems might be stemming from other factors. I'm going to delve further into this further to identify the root causes.

 

Okay, doing some further testing, It seems to be a problem with my Bluetooth code script and the 2. Screen interacting. It worked fine on the main screen, but when the script is running on a 2. Screen it doesn't work anymore and I keep getting.

All times must be in 0-1 range
And
A breakpoint instruction (__debugbreak() statement or a similar call) was executed in SBP2.exe.

Into

UnityPlayer_UAP_x64_debug_il2cpp.pdb not loaded

I've tried about everything I can think of, but can't pinpoint the error.

 

This is the call stack. I sadly can't figure out what part of the code is the root cause of the crash. The problem is it's only crashing when the code runs on 2. Screen and my attempts of figuring out why haven't been very successful. Also, trying to use the symbol server just gives me “source not available” instead of UnityPlayer_UAP_x64_debug_il2cpp.pdb not loaded.

> UnityPlayer.dll!00007ffa24477401() Unknown
UnityPlayer.dll!00007ffa22fbe6ae() Unknown
UnityPlayer.dll!00007ffa2300ea2e() Unknown
UnityPlayer.dll!00007ffa23ebcaca() Unknown
UnityPlayer.dll!00007ffa281658c9() Unknown
UnityPlayer.dll!00007ffa2446108a() Unknown
UnityPlayer.dll!00007ffa2445e162() Unknown
UnityPlayer.dll!00007ffa2445e69b() Unknown
UnityPlayer.dll!00007ffa244640e0() Unknown
UnityPlayer.dll!00007ffa252e916f() Unknown
kernel32.dll!BaseThreadInitThunk() Unknown
ntdll.dll!RtlUserThreadStart() Unknown

this is how the call stack looks after setting the symbol server:

Code (CSharp):

1. >    UnityPlayer.dll!Gradient::EvaluateHDR<0>(struct math::_float4 const &)    Unknown
2.      UnityPlayer.dll!Gradient::Evaluate<0>(float)    Unknown
3.      UnityPlayer.dll!Gradient::Evaluate(float)    Unknown
4.      UnityPlayer.dll!Build3DLine<unsigned short>(unsigned char *,unsigned short *,struct LineParameters const &,struct math::float4x4 const &,struct math::float4x4 const &,struct math::float3_storage const *,float const *,unsigned __int64,unsigned __int64,unsigned __int64,bool,float,bool,float)    Unknown
5.      UnityPlayer.dll!LineRenderer::RenderGeometryJob(struct SharedGeometryJobData *,unsigned int)    Unknown
6.      UnityPlayer.dll!ujob_execute_job()    Unknown
7.      UnityPlayer.dll!lane_guts()    Unknown
8.      UnityPlayer.dll!lane_run_loop()    Unknown
9.      UnityPlayer.dll!worker_thread_routine()    Unknown
10.      UnityPlayer.dll!Thread::RunThreadWrapper(void *)    Unknown
11.      kernel32.dll!BaseThreadInitThunk()    Unknown
12.      ntdll.dll!RtlUserThreadStart()    Unknown
13.  

this is the whole script code, but i'm not entirely sure this in itself causes the crash or not. Removing it however does stop it from crashing. It sees to be able to do one loop then crash after if it's on 2. screen but work fine if I use the same script on the main screen

Code (CSharp):

1. using System.Collections;
2. using System.Collections.Generic;
3. using UnityEngine;
4. using Assets.Scripts.Bluetooth.Connection_and_Device_Management;
5. using System;
6. using Assets.Scripts.Bluetooth.BluetoothEventArgs;
7. using System.Linq;
8. using System.Collections.Concurrent;
9. using TMPro;
10.  
11. public class LineRendererSetter : MonoBehaviour
12. {
13. #if WINDOWS_UWP
14.     private ConcurrentQueue<float> hearthRateQueue;
15. #else
16.     private System.DateTime throttle; //test
17. #endif
18.     private LineRenderer lineRenderer;
19.     private float height;
20.     private RectTransform rectTransform;
21.     private int amountPositions = 50;
22.     private List<float> hearthRate; // this could be change to a threadsave list to get rid of the queue but that doesn't seem to exist currently
23.     private TextMeshProUGUI AvgText;
24.     private TextMeshProUGUI MaxText;
25.     private TextMeshProUGUI VNumberLow;
26.     private TextMeshProUGUI VNumberHigh;
27.     private TextMeshProUGUI CurrentText;
28.     void Start() // sets all the required values
29.     {
30.         //textSetup();
31.         hearthRate = new List<float>();
32.         lineRenderer = GetComponent<LineRenderer>();
33.         lineRenderer.positionCount = amountPositions; // position amount
34.         rectTransform = GetComponent<RectTransform>();
35.         height = rectTransform.sizeDelta.y;
36.         calculatePositions(); // start calculating positions
37. #if WINDOWS_UWP // setup bluetooth values
38.         hearthRateQueue = new ConcurrentQueue<float>();
39.         DeviceDataHandler.Instance.HeartRateSensors += HeartRateSensorEvent;
40. #else // setup editor values
41.         throttle = DateTime.Now; //test
42. #endif
43.     }
44.  
45.     private void textSetup()
46.     {
47.         AvgText = transform.parent.Find("AVG").Find("ValueHolder").Find("Value").GetComponent<TextMeshProUGUI>();
48.         MaxText = transform.parent.Find("MAX").Find("ValueHolder").Find("Value").GetComponent<TextMeshProUGUI>();
49.         VNumberLow = transform.parent.Find("VNumber Low").GetComponent<TextMeshProUGUI>();
50.         VNumberHigh = transform.parent.Find("VNumber High").GetComponent<TextMeshProUGUI>();
51.         CurrentText = transform.parent.Find("Current Value").Find("Value").GetComponent<TextMeshProUGUI>();
52.     }
53.     // Update is called once per frame
54.     void Update()
55.     {
56. #if WINDOWS_UWP //bluetooth code
57.         if (!hearthRateQueue.IsEmpty) // empty the queue if needed
58.         {
59.             float Rate;
60.             if (hearthRateQueue.TryDequeue(out Rate))
61.             {
62.                 HearthRate(Rate);
63.                 if (CurrentText)
64.                 {
65.                     CurrentText.SetText(Rate.ToString());
66.                 }
67.             }
68.             UpdateLineData();
69.         }
70. #else // editor code
71.         if (DateTime.Now > throttle.AddSeconds(1.0)){
72.             throttle = throttle.AddSeconds(1.0);
73.             float Rate = UnityEngine.Random.Range(0, 200);
74.             HearthRate(Rate);
75.     //        if (CurrentText){
76.     //            CurrentText.SetText(Rate.ToString());
77.      //       }
78.             UpdateLineData();
79.         }
80. #endif
81.     }
82. #if WINDOWS_UWP
83.     private void HeartRateSensorEvent(object sender, BLEushortEventArgs deviceData)
84.     {
85.         Debug.Log("HearthRateDebug Adding"+ deviceData.Value);
86.         hearthRateQueue.Enqueue(deviceData.Value);
87.     }
88. #endif
89.     private void HearthRate(float DataValue) // manages the list
90.     {
91.         {
92.             if (hearthRate.Count < amountPositions) // if still space add values
93.             {
94.                 hearthRate.Add(DataValue);
95.             }
96.             else
97.             {
98.                 float lastRate = 0;
99.                 for (int i = hearthRate.Count - 1; i >= 0; i--) // if no more free space iterate over the list in reverse and reorder values -> FIFO
100.                 {
101.                     if (lastRate == 0)
102.                     {
103.                         lastRate = hearthRate[i];
104.                         hearthRate[hearthRate.Count - 1] = DataValue;
105.                     }
106.                     else
107.                     {
108.                         float Rate = hearthRate[i];
109.                         hearthRate[i] = lastRate;
110.                         lastRate = Rate;
111.                     }
112.                 }
113.             }
114.         }
115.     }
116.  
117.     /// <summary>
118.     /// Calculates the Position for all lineRenderer positions. This assumes all data object have the same time inbetween them.
119.     /// </summary>
120.     private void calculatePositions()  // this assumes all data object have the same time in between them. If this is not the case for the bluetooth data provided this needs to be changed
121.     {
122.         float width = rectTransform.sizeDelta.x;
123.         for (int i = 0; i < lineRenderer.positionCount; i++) // build positions in x direction
124.         {
125.             Vector3 Position = lineRenderer.GetPosition(i);
126.             float x = i < amountPositions / 2 ? -((float)amountPositions / 2 - i) / (amountPositions / 2) : (i - amountPositions / 2) / ((float)amountPositions / 2);
127.             Position.x = x * width / 2;
128.             Position.z = 1;
129.             lineRenderer.SetPosition(i, Position);
130.         }
131.     }
132.  
133.     /// <summary>
134.     /// Updates the LineRenderer to show the provided Data. This assumes all data object have the same time inbetween them.
135.     /// </summary>
136.     public void UpdateLineData() // this assumes all data object have the same time in between them. If this is not the case for the bluetooth data provided this needs to be changed
137.     {
138.         /*
139.         if (lineRenderer.positionCount != data.Count){ // adjust amount of positions if neccesary (this looks visually more appealing but can create other problems
140.             lineRenderer.positionCount = data.Count;
141.             calculatePositions();
142.         }*/
143.         for ( int i = 0; i < hearthRate.Count; i++)
144.         {
145.             Vector3 Pos = lineRenderer.GetPosition(i);;
146.             float calculatedPosition = (hearthRate[i] - hearthRate.Min()) * height / (hearthRate.Max()-hearthRate.Min());
147.             Pos.y = calculatedPosition-height/2;
148.             lineRenderer.SetPosition(i, Pos);
149.         }
150.        //SetTexts();
151.     }
152.     /// <summary>
153.     /// Sets the texts adjacent to the LineRenderer
154.     /// </summary>
155.     private void SetTexts()
156.     {
157.         AvgText.SetText(Math.Round(hearthRate.Average()).ToString());
158.         MaxText.SetText(hearthRate.Max().ToString());
159.         VNumberLow.SetText(Math.Round(hearthRate.Max()/3).ToString());
160.         VNumberHigh.SetText(Math.Round(hearthRate.Max()/ 3*2).ToString());
161.     }
162.  
163.  
164. }
165.  

It also crashes if i change the code to the editor only code so it doesn't sem to be the bluetooth interaction that is the problem

 

Yes, it does get rid of the Crash. Oddly enough, it only happens if the line renderer script is on the 2. Screen however, which is super odd. If I have it on the main screen, it works.

 

Yes i'm able to reliably crash the game using this (hopefully) minimal example without any bluetooth code.

Code (CSharp):

1. using System;
2. using System.Collections;
3. using System.Collections.Generic;
4. using System.Linq;
5. using UnityEngine;
6. using UnityEngine.Animations.Rigging;
7.  
8. public class LineRendererMinimalTest : MonoBehaviour
9. {
10.     private LineRenderer lineRenderer;
11.     private DateTime throttle; //test
12.     private int amountPositions = 50;
13.     private List<float> hearthRate;
14.     private RectTransform rectTransform;
15.     private float height;
16.     // Start is called before the first frame update
17.     void Start()
18.     {
19.         hearthRate = new List<float>();
20.         lineRenderer = GetComponent<LineRenderer>();
21.         lineRenderer.positionCount = amountPositions;
22.         rectTransform = GetComponent<RectTransform>();
23.         height = rectTransform.sizeDelta.y;
24.         calculatePositions();
25.         throttle = DateTime.Now;
26.     }
27.  
28.     void Update()
29.     {
30.         if (DateTime.Now > throttle.AddSeconds(0.2))
31.         {
32.             throttle = throttle.AddSeconds(0.2);
33.             float Rate = UnityEngine.Random.Range(0, 200);
34.             HearthRate(Rate);
35.             UpdateLineData();
36.         }
37.     }
38.  
39.     private void HearthRate(float DataValue) // manages the list
40.     {
41.         {
42.             if (hearthRate.Count < amountPositions) // if still space add values
43.             {
44.                 hearthRate.Add(DataValue);
45.             }
46.             else
47.             {
48.                 float lastRate = 0;
49.                 for (int i = hearthRate.Count - 1; i >= 0; i--) // if no more free space iterate over the list in reverse and reorder values -> FIFO
50.                 {
51.                     if (lastRate == 0)
52.                     {
53.                         lastRate = hearthRate[i];
54.                         hearthRate[hearthRate.Count - 1] = DataValue;
55.                     }
56.                     else
57.                     {
58.                         float Rate = hearthRate[i];
59.                         hearthRate[i] = lastRate;
60.                         lastRate = Rate;
61.                     }
62.                 }
63.             }
64.         }
65.     }
66.     public void UpdateLineData()
67.     {
68.         for (int i = 0; i < hearthRate.Count; i++)
69.         {
70.             Vector3 Pos = lineRenderer.GetPosition(i); ;
71.             float calculatedPosition = (hearthRate[i] - hearthRate.Min()) * height / (hearthRate.Max() - hearthRate.Min());
72.             Pos.y = calculatedPosition - height / 2;
73.             lineRenderer.SetPosition(i, Pos);
74.         }
75.     }
76.  
77.     private void calculatePositions()  // this assumes all data object have the same time in between them. If this is not the case for the bluetooth data provided this needs to be changed
78.     {
79.         float width = rectTransform.sizeDelta.x;
80.         for (int i = 0; i < lineRenderer.positionCount; i++) // build positions in x direction
81.         {
82.             Vector3 Position = lineRenderer.GetPosition(i);
83.             float x = i < amountPositions / 2 ? -((float)amountPositions / 2 - i) / (amountPositions / 2) : (i - amountPositions / 2) / ((float)amountPositions / 2);
84.             Position.x = x * width / 2;
85.             Position.z = 1;
86.             lineRenderer.SetPosition(i, Position);
87.         }
88.     }
89. }
90.  

and this is the matching stack trace

Code (CSharp):

1. >    UnityPlayer.dll!Gradient::EvaluateHDR<0>(struct math::_float4 const &)    Unknown
2.      UnityPlayer.dll!Gradient::Evaluate<0>(float)    Unknown
3.      UnityPlayer.dll!Gradient::Evaluate(float)    Unknown
4.      UnityPlayer.dll!Build3DLine<unsigned short>(unsigned char *,unsigned short *,struct LineParameters const &,struct math::float4x4 const &,struct math::float4x4 const &,struct math::float3_storage const *,float const *,unsigned __int64,unsigned __int64,unsigned __int64,bool,float,bool,float)    Unknown
5.      UnityPlayer.dll!LineRenderer::RenderGeometryJob(struct SharedGeometryJobData *,unsigned int)    Unknown
6.      UnityPlayer.dll!ujob_execute_job()    Unknown
7.      UnityPlayer.dll!lane_guts()    Unknown
8.      UnityPlayer.dll!lane_run_loop()    Unknown
9.      UnityPlayer.dll!worker_thread_routine()    Unknown
10.      UnityPlayer.dll!Thread::RunThreadWrapper(void *)    Unknown
11.      kernel32.dll!BaseThreadInitThunk()    Unknown
12.      ntdll.dll!RtlUserThreadStart()    Unknown
13.  

 

Sure thing. Created a very minimal example in a new project that should ease your process of reproducing the issue. If you need anything else, please let me know.

 

IN-54251