Hiding Threading in managed plugin

Hello,

I am writing a managed code plugin which downloads some resources in separate threads returning the result to unity. My problem is that I basically want to hide all the threading operation from unity so that only one function call with a callback is needed to download a huge chunk of data in a different thread and when finished the callback on the main thread should be called.

 

Coroutines?

 

Well using Coroutines requires a Monobehavior right?

 

If you're concerned about the need for a MonoBehaviour, it's not really that you want to hide it from unity, it's that you want to not have to consider unity in the process. There's no real "hiding" from unity, I'm not sure what would entail the act of hiding from unity.

That'll be hard, as you'll need some sort of reference back into unity to hook into its thread. And pulling the reference to the thread on call of the 'load' method isn't enough, because the thread has no way to merge back in. You have to rejoin the thread at time when the main thread is ready to accept you back in (usually during the 'Update' call).

Either the caller has to deal with it themself, or you need to take a reference.

I would create 2 versions of the method so the user gets to decide:

Code (csharp):

1.  
2. ////
3. // Perform the load on own thread, and returns on that thread.
4. // Maybe consider designing with the Task/AsyncState models defined by .Net
5. public void BeginLoad(System.Action callback)
6. {
7.     //... perform load on some thread, and callback via callback
8. }
9.  
10. ////
11. // Perform the load on own thread, return on main thread supplied via 'handle'.
12. // 'handle' MUST be an active MonoBehaviour as deactivating it will kill the coroutine.
13. public void BeginLoad(MonoBehaviour handle, System.Action callback)
14. {
15.     handle.StartCoroutine(this.LoadRoutine(callback));
16. }
17.  
18. private System.Collections.IEnumerator LoadRoutine(System.Action callback)
19. {
20.     bool complete = false;
21.     var thread = new System.Threading.Thread(() => {
22.         //... perform load
23.         //on complete set true
24.         complete = true;
25.     });
26.     thread.Start();
27.  
28.     while(!complete)
29.     {
30.         yield return null;
31.     }
32.  
33.     callback();
34. }
35.  

Be sure you do exception catching and everything, otherwise that coroutine could get stuck infinitely waiting for complete which will never happen because it never receives the signal.

You could also check if the thread is still alive in the while loop to ensure it didn't enter a bad state as well.

 

Thank you for the reply. I totally forgot that I also could use the UnityAPI in my plugin (when I add the unity dll as reference) but imho this approach is too error prone from the "user" point of view.

The first approach looks like the one I am looking for but I am not quite sure how that will work. When you say that the BeginLoad method should run on its own thread, so that the callback also returns on that thread, how do I communicate the response with the main thread?Isn't that the problem that I am having?

I also forgot to mention that I would like to use the .Net WebClient class with its Asyc methods. As far as I understood the those methods run on their own thread and the callback returns also on that thread. (so it should be no difference to your example with manually creating a thread right?)

 

I wasn't giving two examples. I was saying supply two overloads of the method for the user. So that way they get to choose if they want your API to return it to the main thread, or to return it to the main thread themselves.

Note, you can't get onto the main thread with out SOME reference to the unity api. As I said, it's not you hiding your library from the unity api, it's you hiding the unity api from your library. There's no way around it if you keep hiding it from yourself.

The main thread is a pump that the unity engine cycles through. There's only so many safe spots to insert yourself back into that pump, and the only way to get access to them is by referencing the unity API. The example I have is doing that.

If you don't like that the user has the ability to accidentally disable the MonoBehaviour they pass in. Then instead create a new GameObject from your end (on the main thread of course), flag it as not destroyed on load, and attach a custom MonoBehaviour to it, so as to hook back in yourself. This way you have control over the GameObject/MonoBehaviours life cycle, and the user can't break it (unless they sniffed out the GameObject and forced deleted it).

This is what I do with my 'GameLoopEntry' in my spacepuppy framework. I also have a 'Invoke' method that allows calling from any thread to inject a callback to the main thread on the next Update call:

https://github.com/lordofduct/spacepuppy-unity-framework/blob/master/SpacepuppyBase/GameLoopEntry.cs

Of course my GameLoopEntry is too heavy for your simple use case, and updates every frame, because it's used as the backbone of a lot of time sensitive things in my framework.

Furthermore, no matter what, you need to make sure that someone activates your method that creates this GameObject from the main thread the first time. Otherwise it'll fail.

In the end, all routes to hooking into the main thread come with caveats of breaking if the user of your code doesn't treat it correctly. This is what documentation is for, you outline the use case.

Otherwise... you use the first version of the BeginLoad and leave it to the user to deal with returning to the main thread and take all the burden off your library.

 

ok I get your point now I need a reference of the UnityAPI. I actually like your idea of creating a gameobject and attaching a MonoBehavior which runs the coroutines. The only problem I have with this approach is that I have no idea how to test this without unity. (I will need to use a unit testing framework and I have no clue how to mock this, but thats a different story)

What about the naive approach of having a thread-save datastructure? Whenever a BeginLoad is started the main thread creates an item and stores it in the thread-save datastructure.This item contains a "isCompleted" flag, a callback, and userData. When the BeginLoad on a different thread is ended this thread flags the item, and stores the result in the thread-save datastructure. The main thread polls every update (with one Coroutine - there I will need a monobehavior handle) if there is an item in the thread-save datastructure with flag "isCompelte==true" then it fires the callback.

I am not quite sure how this Datastructure should look like (I guess a List<> will do with a lock when reading/writing?) but I can't see why that should not work(or would be more error prone).

And thanks again for your detailed reply.

 

Have you considered the tasks framework? It's in System.Threading.Tasks (I think).

You simply return a task as a result if your ASync functions. The user can query the task to see if it's finished (typically in a coroutine). Tasks can also have call backs and results. Seems like it will fit your needs.

 

This needs .NET 4.0 unfortunately

 

Hmm, didn't think of that. Parse uses it in their unity sdk. I wonder how they managed it?

 

I am not sure how they managed it, but it's not possible out of the box. I am still wondering if my naive approach has any downsides?