Re-render GUI when main thread is busy

Hey,
I am calling python scripts which invokes C# functions internally. Some of these scripts take time and I want to be able to print to a text field in real time as the script is executed. However, since the main thread is occupied with executing the script, my GUI, and everything else for that matter stops rendering until the script is finished running. I want to re-render the GUI and / or the full scene explicitly without waiting for some render function to be called implicitly for a given frame.

Avoiding threads would be preferrable. I just want to call a function to re-render.

I have tried using Camera.main.Render(), and Canvas.ForceUpdateCanvases(), but these don't seem to do what I want. Any visual updates are deferred until the main thread is no longer blocked.

 

You can't force the engine to render through some simple function call like that. A lot more goes into rendering than that.

Instead you have to allow the frame to complete and the engine to enter it's render phase.

Threading obviously works because you're not on the main thread.

But to do this on the main thread you basically need Coroutines. Coroutines are unity's way of staying on the main thread while giving chances to wait a frame so that it can render. I don't know in what manner you're implementing your python engine (nor do I know why you're using python), but you can try and implement a method of letting your python do an equivalent of 'yield return null'.

Like maybe you have a class on the C# side that starts the python script in a coroutine. And then the python script can call a function back to the C# side that blocks execution of the python script, forces the coroutine to yield null, and then returns execution to the python script.

 

Thanks for the info. I understand. I will have to see if it is going to be possible to use coroutines for this. I am using IronPython and I am not sure if it is possible to pause execution of a script in IronPython. If it is, I could pause it whenever the print function is called and yield return, continuing until finished using a loop in the coroutine. Threading will probably not be possible since the Unity engine does not support that and I need to be able to call the Unity API from the scripts I'm running.

I have a feeling that I am out of luck though. I have a hard time imagining that IronPython would support pausing a script execution and continuing later.

Thanks

 

The script would 'pause' by blocking for the duration of a function call.

Basically you'd call a function into unity, and that function would not return immediately, but instead would return after the amount of time would pass. Since python will block until the function returns, you effectively have 'paused' the script.

If you could share a project with your implementation of the python script engine working in it. I could maybe take a look at it.

 

Yes that is true, but as soon as the function returns it returns to python. The python source code that I'm executing is of type Microsoft.Scripting.Hosting.ScriptSource and it does not return an IEnumerator when calling Execute so an enumerator can not really be returned back through the whole function call chain back to the coroutine call, or maybe I have a lack of understanding of the IEnumerator interface.

What I meant with pausing execution of the script was for the Execute function to return without having finished the execution and then being able to call something like "ContinueExecution" the next frame.

I'll see if I can strip down the Unity project to its bare minimum of dependencies and share it in case you are interested in having a look.

Thanks again

 

@lordofduct, in case you are interested in helping out I have a demo showing the issue I have up on github. It requires Windows since the IronPython version I have is compiled for .Net.
https://github.com/kbladin/IronPythonUnityDemo

Hopefully it demonstrates the issue.

 

The general principle here is you should only have critical functionality on the main thread. Like your GUI, input detection, rendering and so forth. Any long blocking methods should be put on their own thread. Its relatively simple to just poll to see if the task is done every frame. For a more sophisticated approach, you can build a set of call back queues.

If you make a hard and fast rule that only one thread can touch data at a time, multithreading in this way is fairly straight forward.

 

I'll probably look at this some time later.

I have to drive up to Orlando now, my partner just got back from Japan.

 

Thanks. I see what you mean. I need to be able to call the Unity API from the scripts though. I have read on several forum posts that the Unity API is not thread safe so I haven't even bothered trying it. For isolated scripts that do not require calls to the Unity API it makes sense however. Since something as simple as updating a text field for the output log (which is partly what I want to do) requires a Unity engine call I presume it will not be thread safe. I could update a string on the worker thread and read it on the main thread to get console output but it would require special handling for each case of a Unity API call, that's why I was looking for a simpler solution than to use threads.

 

Accessing the unity API from another thread is almost always forbidden. You need to get back onto the main thread to do anything useful with Unity.

I normally set things up like this (psuedo code)

Code (CSharp):

1. public class UnityClass : MonoBehaviour {
2.  
3.     List<Action> thingsToDo = new List<Action>();
4.  
5.     void Update () {
6.        lock (thingsToDo){
7.            foreach (Action action in thingsToDo){
8.                action();
9.            }
10.            thingsToDo.Clear();
11.        }
12.     }
13. }
14.  
15. public class RandomThread {
16.  
17.     UnityClass unityClass;
18.  
19.     void AsyncMethod () {
20.         // Do stuff
21.         lock (unityClass.thingsToDo){
22.             unityClass.thingsToDo.Add(SomeRandomAction);
23.         }
24.     }
25. }

Note this is really crude. There are better data structures and locking schemes designed exactly for this purpose.

 

I see what you mean. This could possibly be a solution. Maybe also useful without threads. If I decide to run everything on the main thread for simplicity I could just queue up a bunch of actions when I call my module from python, then execute them one by one each frame instead of having a foreach loop like you have there. It might require some awkward wrappers for each module call but it feels like it should work. I'll test it out to see.

 

Crude but the essential core of any inter-thread callback operation. Nothing wrong with your example at all.

 

@Kiwasi This works fine on a smaller scale and as a solution for job scheduling. If I would start having more functions that are not simple actions I need to return unfinished pointers to objects in the python world and then start enqueuing all dependencies which in the end feels awfully complex. When I run a function in python I would expect it to return a usable value but with this method I can't guarantee that the function is finished running when I get the return value.

 

So today I played around with this.

First and foremost... you can't use time.sleep if you're executing the script from the main thread because it's on the main thread. It's the equivalent of sleeping the thread in C# with Thread.Sleep. So only use time.sleep if you know you're on a different thread.

Next... python does have yield. It's called a generator function. I checked and ironpython does support it. Here is more on a generator function:
https://ironpython-test.readthedocs.io/en/latest/howto/functional.html

So what I did was create a scope that did a few things:

1) gotoUnityThread - allow you to enter the main thread of unity if you aren't on it
2) exitUnityThread - execute on a thread other than the unity thread
3) coroutine - start up a coroutine (using the generator function from python)
4) wait - get a WaitForSeconds object for use in a coroutine

I have 2 ways to start the script. On the main thread, and on async.

I also added in a way to reference the UnityEngine assembly... otherwise what's the point of being on the unity thread?

So the ScriptEngine from your github has changed a lot. First off I made it a MonoBehaviour so we can hook into the thing. Also, I moved the scope into its own class:

Code (csharp):

1.  
2. using UnityEngine;
3. using System.Collections.Generic;
4. using System.Threading;
5. using IronPython.Hosting;
6.  
7. /// <summary>
8. /// Python script engine wrapper which contain the Python scope used in an
9. /// application.
10. /// </summary>
11. public class ScriptEngine : MonoBehaviour
12. {
13.    
14.     #region Fields
15.  
16.     private Microsoft.Scripting.Hosting.ScriptEngine _engine;
17.     private Microsoft.Scripting.Hosting.ScriptScope _mainScope;
18.  
19.     private int _mainThreadId;
20.  
21.     private System.Action<string> _logCallback;
22.     private System.Action _joinMainThread;
23.     private object _joinLock = new object();
24.  
25.     #endregion
26.  
27.     #region CONSTRUCTOR
28.  
29.     private void Awake()
30.     {
31.         _mainThreadId = Thread.CurrentThread.ManagedThreadId;
32.     }
33.    
34.     public void Init(System.Action<string> logCallback)
35.     {
36.         _logCallback = logCallback;
37.         _engine = Python.CreateEngine();
38.  
39.         // Create the main scope
40.         _mainScope = _engine.CreateScope();
41.        
42.         // This expression is used when initializing the scope. Changing the
43.         // standard output channel and referencing UnityEngine assembly.
44.         string initExpression = @"
45. import sys
46. sys.stdout = unity
47. import clr
48. clr.AddReference(unityEngineAssembly)";
49.         _mainScope.SetVariable("unity", new ScriptScope(this));
50.         _mainScope.SetVariable("unityEngineAssembly", typeof(UnityEngine.Object).Assembly);
51.  
52.         // Run initialization, also executes the main config file.
53.         ExecuteScript(initExpression);
54.     }
55.  
56.     #endregion
57.  
58.     #region Properties
59.    
60.     public System.Action<string> LogCallback
61.     {
62.         get { return _logCallback; }
63.         set { _logCallback = value; }
64.     }
65.  
66.     #endregion
67.  
68.     #region Methods
69.  
70.     public void ExecuteScript(string script)
71.     {
72.         this.ScriptStart(script);
73.     }
74.  
75.     public void ExecuteScriptAsync(string script)
76.     {
77.         ThreadPool.QueueUserWorkItem(this.ScriptStart, script);
78.     }
79.  
80.     #endregion
81.  
82.     #region Private Methods For Engine
83.  
84.     private void Update()
85.     {
86.         System.Action a;
87.         lock(_joinLock)
88.         {
89.             a = _joinMainThread;
90.             _joinMainThread = null;
91.         }
92.  
93.         if(a != null)
94.         {
95.             a();
96.         }
97.     }
98.  
99.     private void ScriptStart(object token)
100.     {
101.         try
102.         {
103.             var script = _engine.CreateScriptSourceFromString(token as string);
104.             script.Execute(_mainScope);
105.         }
106.         catch (System.Exception e)
107.         {
108.             Debug.LogException(e);
109.         }
110.     }
111.  
112.     #endregion
113.  
114.     #region Special Types
115.  
116.     public class ScriptScope
117.     {
118.  
119.         public ScriptEngine engine;
120.  
121.         public ScriptScope(ScriptEngine engine)
122.         {
123.             this.engine = engine;
124.         }
125.        
126.         public void write(string s)
127.         {
128.             if (engine._logCallback != null) engine._logCallback(s);
129.         }
130.  
131.         public void gotoUnityThread(System.Action callback)
132.         {
133.             if (callback == null) return;
134.  
135.             if (Thread.CurrentThread.ManagedThreadId == engine._mainThreadId)
136.             {
137.                 callback();
138.             }
139.             else
140.             {
141.                 lock(engine._joinLock)
142.                 {
143.                     engine._joinMainThread += callback;
144.                     System.GC.Collect();
145.                 }
146.             }
147.         }
148.  
149.         public void exitUnityThread(System.Action callback)
150.         {
151.             if (callback == null) return;
152.  
153.             if (Thread.CurrentThread.ManagedThreadId != engine._mainThreadId)
154.             {
155.                 callback();
156.             }
157.             else
158.             {
159.                 ThreadPool.QueueUserWorkItem((o) =>
160.                 {
161.                     callback();
162.                 }, null);
163.             }
164.         }
165.  
166.         public WaitForSeconds wait(float seconds)
167.         {
168.             return new WaitForSeconds(seconds);
169.         }
170.        
171.         public void coroutine(object f)
172.         {
173.             if (Thread.CurrentThread.ManagedThreadId != engine._mainThreadId)
174.             {
175.                 this.gotoUnityThread(() =>
176.                 {
177.                     this.coroutine(f);
178.                 });
179.             }
180.             else
181.             {
182.                 var e = f as System.Collections.IEnumerator;
183.                 if (e == null) return;
184.  
185.                 engine.StartCoroutine(e);
186.             }
187.         }
188.  
189.  
190.     }
191.    
192.     #endregion
193.  
194. }
195.  

Your ScriptRunner was truncated down. And I made it so that it can take a TextAsset so swapping out scripts is easy:

Code (csharp):

1.  
2. using System.Collections;
3. using System.Collections.Generic;
4. using UnityEngine;
5.  
6. public class ScriptRunner : MonoBehaviour {
7.  
8.     #region Fields
9.  
10.     public TextAsset script;
11.  
12.     private ScriptEngine _scriptEngine;
13.  
14.     #endregion
15.  
16.     #region CONSTRUCTOR
17.  
18.     // Use this for initialization
19.     void Start () {
20.         _scriptEngine = this.gameObject.AddComponent<ScriptEngine>();
21.         _scriptEngine.Init(Debug.Log);
22.     }
23.  
24.     #endregion
25.  
26.     #region Methods
27.  
28.     // Update is called once per frame
29.     void Update () {
30.         if (Input.GetKeyDown(KeyCode.Return))
31.         {
32.             _scriptEngine.ExecuteScript(this.script.text);
33.         }
34.     }
35.    
36.     #endregion
37.  
38. }
39.  

And some example scripts.

First we show threading. The script defines to functions and then make sure it's on the unity thread to run foo where it accesses stuff on the unity thread. Then it exits the main thread to then do your sleep loop:

Code (csharp):

1.  
2. import UnityEngine
3. from UnityEngine import Vector3
4. import time
5.  
6. def foo():
7.     unity.engine.transform.position = Vector3(5,0,0)
8.     unity.exitUnityThread(foo2)
9.  
10. def foo2():
11.     for x in xrange(0, 5):
12.         print 'TestPrint'
13.         time.sleep(1)
14.  
15. unity.gotoUnityThread(foo)
16.  

This one demonstrates doing a coroutine in python. In it I start the coroutine where I wait a frame, print "1", wait 5 seconds (using our custom wait method), prints 2, and then sets the position of the engine transform.

Code (csharp):

1.  
2. import UnityEngine
3. from UnityEngine import Vector3
4.  
5. def DoWork():
6.     yield None
7.     print "1"
8.     yield unity.wait(5)
9.     print "2"
10.     unity.engine.transform.position = Vector3(5,0,0)
11.  
12. unity.coroutine(DoWork())
13.  

I created a pull request with my changes.

 

So the general idea here is that the script runs on the main thread from the get go (unless you want otherwise).

If you need to wait/sleep though. You should start coroutines or use threads... which requires callback functions to be defined.

So if you just want to run a script that modifies some values. Have at it, you don't need to def and functions, like so:

Code (csharp):

1.  
2. import UnityEngine
3. from UnityEngine import Vector3
4.  
5. unity.engine.transform.position = Vector3(5,0,0)
6. print 'Modified the transform'
7.  

But once threading/waiting/sleeping comes in. Yeah, you'll need callbacks. It's just like the C# side of things. You can't just willy nilly jump around threads and sleep with out some sort of structure to it.

 

@lordofduct Thanks a lot for taking the time! I'll have a look once I have the time. Btw, I am not sure what you are referring to regarding sleep on the main thread. I am well aware that sleep to wait for a second thread to finish would not make sense anywhere. I simply used sleep to simulate some work being done on the main thread in the purpose of demonstration.

 

In the context of doing work.

If you're attempting to simulate 1 second of work.

Well the same still applies.

Just like if you are in C#... doing a second worth of work is going to block for a second. This issue is independent of IronPython. Work is work, if it takes a lot of time, you need to mitigate that. Usually with threads or coroutines.

 

Thanks @lordofduct for the help on this issue. The solution to the problem was the use of yield and being able to do that from within Python itself was what I was looking for. Mitigating work to a different thread is a separate issue and I'm glad you covered that as well in your solution. Thanks again, very much appreciated that you took the time!