TestStar - Make your games BugFree(tm)

We've decided to make our latest release of TestStar more available at a new, lower price ( $50 ). If the price tag was making you hesitate before, go for it!

I just wanted to mention that TestStar is now available! Finally, a testing framework completely integrated with the unity editor.

TestStar lets you build tests for your game functionality as you are building your game.

We use NUnit's assert functionality for nice, "literate" testing, and we try to work with the "unity style" of development.

Here's a quick intro video:

Questions, comments, and feedback most welcome!

 

cool

 

Why 100 dollars? Nunit framework is free.

Every c# developer must know this framework.

Are you using directly Nunit framework? So it Unity Pro feature only.

 

Sleglik: at what price would you find it worth investing in?

It's not free because it has more than 200+ hours of development to provide a real test framework in Unity3d. We tried using NUnit "by itself" - but if you've tried, you know it doesn't work well with the "Unity Way" of doing things.

The only thing we use from NUnit is the "Framework" - that is, the "Assert.That()" style expressions. We have a custom test runner and lots of custom Unity3d code.

We had to rewrite a little bit of code from NUnit in order to make it work right with unity's pseudo-null objects (and will be adding even more code to provide custom Constraints for Unity structs) and so we include straight source code (with license) from NUnit. Thus, no Pro only.

What you get for this is a nice UI, as well as tight integration with Unity - the ability to do testing that depends on physics, timing, etc. In other words, it's way more than just a Unit testing framework; it is closer to a functional-testing kit than a Unit testing one.

 

This afternoon I had the honor of giving TestStar a TestRun,

It took me a few minutes to figure out exactly how it works and how I am supposed to use it, but let me tell you, it's brilliant!

Let's say you wonder how the firing speed of your enemies weapons will affect the players chance to dodge them by running away. You set up several possible scenarios (player is surrounded, there is only one enemy, enemies are above him, enemies are behind him, etc) and state what has to be tested: "Can the player survive by running to the side?", which takes about 3 lines of code.

You can now simple change the firing speed of your enemies' weapons and see the changes reflected in several controlled test cases, immediately.

The time it took me between importing the package and setting up my first complete test, with several different test cases, was less then halve an hour. I'm sure that now I know how it works I could do it in several minutes.

The potential for this is incredible and I can't wait to use it on a larger scale. I'm currently in the middle of a large game, and being able to - with the push of a single button! - verify that all the changes I made did *not* break how platforms work under specific circumstances, how enemies react when some obscure event takes place or a million other things is incredible! I can't wait to set up a solid test which I can then make every member of my team run before dedicating their changes to the server, which will probably reduce the amount of bugs caused by accidental changes to something immensely.

 

Looks very cool!

I don't suppose it does any sort of action recording does it? In order to test some scenarios in our game you would have to drag and drop something, for example. I can't, in this case, put an object down and press play.

 

Action (or input) recording is high on our priority list. For the moment, you have to script the behaviour you want to replicate.

Rafes: I assume you need to recreate user input, not just recreate the consequences of the input? Because in some ways the latter is easier than the former.

 

We just made a 1.2 release with some improvements suggested by reviewers - and a 50% price cut! Get it while it's hot!

 

Can TestStar be used for testing complex Editor scripts (such as custom inspectors) or only Runtime code?

 

Right now it is targeted at testing runtime code.

 

Being able to test custom editor scripts (in addition to runtime code) would be a key feature for me... of course it's better to test only runtime scripts than not doing any unit testing at all but, before purchasing your product, I have to ask if you are planning to add support for custom editor scripts in a future release or if it is something that isn't in the roadmap.

 

david,

We are definitely going to add testing "code that runs in edit mode" in the not too distant future, but we are unlikely to be able to offer testing anything that runs in OnGUI for a while yet.

Automated testing of such code would require a selenium-like ui testing ability, or a rather complex mocking framework, both of which are not in our roadmap at this time.

Unity 3.5 might change things on that front.

If you could share your specific use cases with eye3ware directly, we might be able to accommodate your needs earlier.

Thanks for your interest!

 

Until now, I have used UUnit ( http://www.unifycommunity.com/wiki/index.php?title=UUnit ) to test my runtime code, but I suppose it would be possible to use the "ExecuteInEditor" attribute to test editor scripts. One of the problems of UUnit is that it displays the results of the tests in the Editor Console instead of having a custom interface, I know using the Console is a "less than ideal solution", but it would be good enough for me (at least for testing editor scripts) until Unity 3.5 is released (and you may find a better solution using the new GUI functions).


I know testing OnGUI code (or OnInspectorGUI code) can be difficult, but I usually do complex calculations in a separate function and I would like to be able to test that code. For example:

Code (csharp):

1.  
2. // Take into account that this code is executed in the "Unity Inspector"
3. // if the GameObject is selected, even if the game is paused or stopped...
4.  
5. protected void OnSceneGUI{
6.      // I want to test if this function returns the expected calculation
7.      Vector3 position = this.DoComplexCalculations();
8.  
9.      // I don't need to test GUI functions, I'm sure Unity Team did  a good job
10.      Handles.Label(position, "Label");
11. }
12.  

 

ok, we can make some small changes to TestStar to enable this kind of editing (mostly right now we don't look in the editor namespace for tests at all). If you buy it, we'll make sure the feature is in the next release

 

I've already bought TestStar. Even if it doesn't support editor scripts yet, I find TestStart much better than the testing framework I was using previously. However, I want to start using a "Test Driven Development" methodology with a "Continuous Integration" server (or equivalent), so I will need to test editor scripts in addition to runtime code...


I will be looking forward your next release

 

This is incredibly awesome! Thanks so much for creating this! I do TDD on all my projects so having the ability to do this in Unity is unreal. Now I won't be so scared when developing games in Unity.

Thanks

 

Is this plug-in still maintained? I bought it a while back and found out that the online manual and the whole TestStar web page is down and not working!!

Devs?

/trumps