Unit testing nested couroutines

I've created an IEnumerator extension method that executes an entire coroutine in one frame, making coroutines (mostly) compatible with NUnit:

https://gist.github.com/mstevenson/2a3acd75e7f7f6d2a0a52a72734202b2

My issue is that if yield return StartCoroutine is called inside an IEnumerator it yields a Coroutine object, but this type has no public members. I'm unable to manually advance the resulting coroutine or track its progress. When decompiled, Coroutine appears to be managed by the underlying C++ engine so this may be a dead end.

Is there any way to interact with a Coroutine object other than passing it into StartCoroutine or StopCoroutine? If not, is there perhaps a way to intercept calls to StartCoroutine from test code?

 

Let me guess how you ended up writing a regression tests framework for coroutines:

At some point in the development of your project, coroutine becomes the ideal tool to write some actions that takes several frames to be accomplished. This worked well since coroutines handles quiet well actions that can be splitted in small chunks that run in a sequence, plus that sequence is readable directly in the coroutine function (great!). Then running these sequential actions was not enough; there was a need to put some logic into the sauce to make some selection or branching. Then, there as a need to put some hierarchy to makes the whole thing more organized, that's where the nested coroutines idea has hit.

Some time later, after putting coroutines everywhere, the high-level logic seems to be diluted all other the code and it becomes hard to read and to understand. Furthermore a single modification somewhere seems to have an unpredicable impact somewhere else, which might brake stuffs that already works. Now, since the broken stuffs is not directly apparent after the modifications, we need to implement some regression tests so that we know what stuffs got broken.

And that's where you've stepped in.

Sorry, if that's not your experiences. But its the experience I had with coroutines, and I've reached the conclusion that coroutine is definitely not a tool to write complex logic that runs over several frames. So far, the best alternative I know is Behaviour Tree, which is a tool that has been designed specifically to describe complex logic in a frame-independent way. This technique is, in my humble opinion, a tool to add at any game programmer tool belt, even more for AI programmer.

I'm the author of Panda BT, a script-based Behaviour Tree engine which has a free (as well as a pro) edition on the Asset Store.

You can have more information about this tool here:

http://www.pandabehaviour.com/

 

I've thankfully managed to avoid complex coroutines, and am not currently using them for any game logic. I'm writing tests for an asset loader where there's a mix of threads and coroutines. Asset instantiation is essentially being treated as file IO, so I have to kick it over from a thread to a coroutine in order to safely interact with the Unity engine.

 

Panda BT can be applied to other domain than game logic. I've used it once to implement a stateful protocol over the network to synchronize data with the server. It serves well where you need to implement any logic that runs over long period time (several frames).

 

@ericbegue - I get wanting to share your approach to the problem. I do the same as well... but you may want to shy away from over selling yourself... OP does have a question and you're getting way off base from OP's question.

The iterator function that is used for the Coroutine is really just an IEnumerator.

Just call the function, get the enumerator, and step over it.

Code (csharp):

1.  
2. IEnumerator void MyRoutine()
3. {
4.     yield return null;
5. }
6.  
7. //else where
8.  
9. var e = MyRoutine();
10. while(e.MoveNext())
11. {
12.     var instruction = e.Current; //the yielded object
13. }
14.  

[edit]
ouch, I misread some of your post, sorry, this isn't a solution at all.

I've battled with this same issue before, and haven't come up with a solution either.

So much so that I wrote my own wrapper around Coroutine to both extend its functionality and store better state information for pausing/cancelling/resuming coroutines.

https://github.com/lordofduct/space...lob/master/SpacepuppyBase/RadicalCoroutine.cs

Note the 'manual override' functionality...

 

@lordofduct I did not indeed answer the OP question directly, sorry for that. However, I don't think I am out of context of this thread, I do think the solution I proposed is an excellent alternative to coroutines, (Using coroutines can become quickly difficult to manage, and I assumed that might have been the reason why @mstevenson designed a regression testing framework). I would excuse myself to @mstevenson if my answers are annoying him and are not bringing nothing constructive to its problem, but please, do not speak for him.

My general approach to problem solving, is to try to identify the very root of the problem and solve the problem there instead of trying to implement add-hoc solutions that would fix only the consequences. I've suggested that the root problem was the usage of coroutines, therefore not answering the OP question was consistent with that.

And I won't shy away from promoting my work, since I do believe Panda BT is a viable solution that would save users spending their valuable time into debugging, therefore creating more time for themselves to produce game content.

 

@OP - sorry, I had to edit my previous post, I completely misread a couple lines of it.

Lol, says the person making a 3 paragraph assumption about what led OP to write a regression test for coroutines.

I was mostly commenting on the fact that OP responded to you with what basically amounted to lack of interest in behaviour trees for dealing with what essentially amounts to thread hopping.

And you respond back with trying to sell your tool again.

Great, they write it with Panda BT... now they might need to unit test that... some people unit test not out of need, but out of standard.

You're welcome to promote your solution all you want... I still think it's a bit over zealous.

 

@lordofduct Why those flames?

@mstevenson has a problem, I've made an assumption (which could be right or wrong) about what led him to this problem and proposed/promoted Panda BT as a solution. Where exactly is the problem with that?

 

@mstevenson, I really don't think there's a way around this other than to wrap your StartCoroutine calls in something that switches behaviour based on test context.

It's going to be ugly, and it's going to leak test details into your production code, but as long as you're stuck with how StartCoroutine works, there's not much to do about it. I guess you could have your tests do a static-analysis pass over your code and rewrite the "yield return StartCoroutine(Foo())" to "yield return Foo()", recompile, and then run the tests, but don't actually do that.

Something like:

Code (csharp):

1. //on whatever base class you're using, or as an extension method or whatever.
2. public Coroutine RunCoroutine(IEnumerator coroutine) {
3.     if(in_test_context) {
4.         testFramework.RegisterCoroutine(coroutine);
5.     }
6.     return StartCoroutine(coroutine);
7. }

then lines 33-35 of RunCoroutineWithYields would be something like

Code (csharp):

1.  
2. } else if (activeEnum.Current is Coroutine) {
3.     enumStack.push(testFramework.lastRegisteredCoroutine);
4.     activeEnum = testFramework.lastRegisteredCoroutine);
5. }

It's a bit hard to read, and uses globals to achieve it's goal. But it should work. I'm assuming you're not running the tests in a multithreaded contexts (which would destroy a global-based solution), and your tear-down code needs to ensure that there's not garbage data from the last test in the set of registered coroutines.

You could also have a more engineered solution where you have a coroutine runner singleton that you swap for a test-based one in test time. If you already have a solution for doing stuff like static coroutines, that's where you'd want to put it.

Spoiler: FLAMES

Probably because you're advertising a freemium product in a thread where OP's problem is completely unrelated to what you're advertising. Panda is probably a great framework, but it can't test coroutines.

 

Completely unrelated is bit exaggerated. I've never said this tool is a framework to test coroutines. I am saying it is an alternative to using coroutines in the first place. It's just a suggestion.

 

@lordofduct Thanks for the link, there are some neat ideas in there that I may incorporate into the next version of our asset loader. Being able to jump between the main thread and a worker thread is really nice.

@Baste I was afraid that it'd be ugly, but thanks for the suggestion. Until I'm able to refactor this, I'm going to try running these tests through Unity Test Tools which seems to allow multi-frame execution.