Unity Test Tools

The latest versions of Test Tools are not up to date with the editor - there's some obsolete warnings, and one obsolete error.

If I import the tools, Unity wants to "upgrade the project", which means that it'll spend an hour scanning all files for upgradeable errors. Yay.

 

Hi, I've been using Test Tools and find it quite useful. However the lack of support for WSA and WP8 leaves a big hole in our automated testing. Particularly since these platforms seem to have many special rules/unity quirks... What are the plans for WSA/WP8 support?

 

Sounds like a good plan - looking forward to it!

 

I run with jenkins and use these command, but no xml output
please any help

/Applications/Unity/Unity.app/Contents/MacOS/Unity -quit -projectPath "${WORKSPACE}" -logFile unity_unittests_results.log -executeMethod UnityTest.Batch.RunUnitTests -resultFilePath="${WORKSPACE}/unity_unittests_results.xml"

 

Are there known imitations with WebGL integration tests?

Using a simple AssertionComponent testing gameObject.activeSelf
I get an InvalidPathException: Invalid path part gameObject

Same tests work on other platforms.

 

In the latest Unity version (5.1.1f1) it appears the -testscenes argument causes an overwrite of the scene selection in EditorBuildSettings.asset, overwriting our normal scene selection setup. This was not occuring on 5.0.x builds and the -testscenes remained temporary for that run.

Any work around? This is causing issues in our build process as we run tests first, then a regular build and the regular builds ends up with the test levels.

 

Hi, i was wondering if their is any way to call this function UnityTest.Batch.RunUnitTests from code. I have another pre build script where we set up various configs for our project and want to call the unit tests running from there, but it keeps telling me Batch does not exisit, i assume it's because it's a partial class and can't be called?

Any ideas? can you call multiple functions from execute method?

 

I'm a step further it will compile (thanks for that, editor folder...i was silly) but our builder (teamcity) returns with exit code 1, which is a polite way of it tell me the unity build crashed.

To make sure it wasn't a teamcity error, i wrote an autobuilder to run within the editor. Simulate the same process.

Code (CSharp):

1. /// <summary>
2. /// Auto Builder
3. ///
4. /// Editor: Kyle Hatch
5. ///
6. /// Description: For Building Debug builds of the product
7. /// </summary>
8.  
9. using UnityEditor;
10. using UnityEngine;
11. using UnityTest;
12. using System.Collections;
13.  
14. public static class AutoBuilder
15. {
16.     private const string FILENAME_EXT = "/Base.exe";
17.  
18.     private static string[] m_Levels = new string[]
19.         {"Assets/Main Project/Scenes/FrontEnd.unity" };
20.  
21.     [MenuItem("Tools/Windows Build With Higher Quality Settings &B")]
22.     public static void SingleBuild ()
23.     {
24.         BuildProject(false);
25.     }
26.  
27.     private static void BuildProject(bool bDev)
28.     {
29.         System.DateTime startTime = System.DateTime.Now;
30.  
31.         //Set Configs
32.         PreBuildScript();
33.  
34.         string sPath = EditorUtility.SaveFolderPanel("Select a folder", Application.dataPath + "/../Build", "");
35.  
36.         if(null != sPath && string.Empty != sPath)
37.         {
38.             Utils.Code.ClearFolder(sPath, ".gitignore", ".lnk");
39.  
40.             Texture2D[] icon = PlayerSettings.GetIconsForTargetGroup(BuildTargetGroup.Standalone);
41.             if(null != icon)
42.             {
43.                 icon[0] = Resources.Load("Icon") as Texture2D;
44.             }
45.             PlayerSettings.productName = "Basic Principle Switching Simulator";
46.             PlayerSettings.SetIconsForTargetGroup(BuildTargetGroup.Standalone, icon);
47.  
48.             BuildOptions Options = BuildOptions.None;
49.             if(true == bDev)
50.             {
51.                 Options =  BuildOptions.Development | BuildOptions.ConnectWithProfiler | BuildOptions.AllowDebugging;
52.             }
53.  
54.             BuildPipeline.BuildPlayer(m_Levels, sPath + FILENAME_EXT, BuildTarget.StandaloneWindows, Options);
55.         }
56.  
57.         System.DateTime endTime = System.DateTime.Now;
58.         System.TimeSpan span = endTime.Subtract (startTime);
59.      
60.         EditorUtility.DisplayDialog("Build Complete", "Build Completed at: " + endTime + " Build Time: " + span, "OK");
61.     }
62.  
63.     public static void PreBuildScript()
64.     {
65.         //Run Tests
66.         Batch.RunUnitTests();
67.  
68. //Other things i want pre build
69. //DoSomeChanges();
70.     }

The editor crashes when i run this script. i comment the line Batch.RunUnitTests() it builds fine. I've attached the Editor log from the crash.

Ideas welcome.

 

I am using Unity 5.1 and unity test tools for integrations tests
I am trying to simulate key pressing so that we can test the GUI responds correctly (dependant on screen size, desktop to wall size screens)
I can't seem to find anyway to simulate a key press I obviously could hook into the code and run the procedure that the key press would instantiate, however this is not what I want to do for code maintainability. I have tried looking at using system.windows.forms.sendkeys (on active window), I have looked at using the dlIImport methodology to import user or win32.dll to hook in at the Kernel level (again not something I really want to do both of these are not available in the current version of .net with Unity.
for example we have a private c# method in a game object that calls a screen shot and then places it in a specific folder under the users area. I would love just to be able to press "." (for example) and then wait to see fi we get a folder and then a screen shot. again for a system generated word document it goes into a documentation folder.

Is there anyway to simulate a user key press/ mouse click within the unity test tools.

cheers
D2

 

Excluding the test tools from builds by copying the folder away and back in the build script is very error prone, quite often copying the test stuff back just fails for some reason and then you have to manually sort the mess.

Is there some better way to exclude the test tools resources from builds?

 

hi all,
when i import the package,some error occurred:
unitytesttool the imported type mono.cecil.cil.sequencepoint is defined multiple times

empty project is ok!

thanks for help

 

I am trying to play around with integration tests (for the first time, yes... the shame).

One thing i've noticed is the DynamicTest attribute. I am not sure how to use it though. I've placed it over a MonoBehaviour class but it doesn't seem to be picked up by the test runner.

Also, it looks like it needs to run in conjunction with a scene. Tried that as well but still no go.

Is there any option to drive an integration test strictly from code? (e.g: the test runner will create a new temp scene and i will create all objects in code and run the assertions on whatever i want).

 

What does this workflow save me? why should i use it instead of the older one (e.g: create a scene, add objects, etc).

 

@Tomek Paszek I actually tried to use it but was unable to run the test...

I have a MonoBehaviour that i marked with that attribute.
For the tests i created a generic, empty scene (passed that name to the attribute). Since all objects will be created in code, i don't need anything in that scene, so i assume i can share the same (empty) scene between all my dynamic tests that are driven by code.

This test was not picked up by the runner though... I could not see it in the list (only when that empty scene was the loaded), but at that point i was still unable to run the test.

I wonder if i am not getting it, or if the way i attempted to use it was the correct way ....

 

I dont have any other integration test...so i guess i won't have the test runner object anywhere...

 

Same problem here with unity 5.2 on visual studio community 2015.

Edit: I was able to fix the problem, unity 5.2 uses Mono.cecil 0.9.6 and unity test tools 0.9.4, to make it work replace Mono.cecil and Mono.cecil.Mdb dll in yourProject\Assets\UnityTestTools\UnitTesting\Editor\NUnit\Libs by the ones in C:\Program Files\Unity\Editor\Data\Managed

 

I continued to develop one of my projects throughout the whole Unity 5.2 beta period and regularly updated. The last version I used was Unity 5.2.0f2 and I had no issues. Now with Unity 5.2.0f3, I am also getting the SequencePoint issue that was mentioned a few times. I don't understand why it only appeared in Unit 5.2.0f3 and not in previous versions.

Case 725779

 

Ditto to Dantus, You can comment out the function and the call to continue work.

 

There is a bug with version 1.5.6 in TestComponent.CreateTest on line 266. The use of Undo causes projects that contain this library to not build as Undo is unavailable outside the editor.

Assets/UnityTestTools/IntegrationTestsFramework/TestRunner/TestComponent.cs(266,25): error CS0103: The name `Undo' does not exist in the current context

I was running this on Unity 4.6.8p3 but don't think that should matter. A workaround is to wrap the offending line with #IF UNITY_EDITOR

 

Is there a way to run a single test from the command line on Android?

This will run the whole suite:

adb shell am start -a android.intent.action.MAIN -n com.unity.IntegrationTest/com.unity3d.player.UnityPlayerActivity

But I need to debug a single test in the suite. Are there any parameters I can provide to this command to run a specific test? Or is there anything I can add to the com.unity.IntegrationTest.xml file to specify a single test run?

Let me know if there is a more appropriate place to ask this question.

 

Sorry if this was posted before: I just tested with the latest version of UTT from the asset store (1.5.6) on Unity 5.2.1:


I suppose the "Passing tests" should pass, right ?

 

Just tried again and i also can't repro this.

 

Hi,

Are there any plans to have Unit Tests able to run from within Visual Studio? It would be great if we didn't have to switch between the IDE and the Editor each time we want to run the tests..

Mike

 

Tomek - the tests should still be executed within theeditor right? Only the results will be communicated back to VS.

 

Can i use integration tests on "real" game scenes?

For example - in case some of the game's logic cannot even be extracted into its own test scene since it's all a tangled mass of objects in the real scene (i didn't write that, of course !! haha).

Can i define a dynamic integration test (from code), set its scene to the real scene from our game, and execute the tests in the context of that scene ? all objects added, etc at runtime shouldn't affect the scene since they are in play mode, right ?

Has someone attempted this workflow before ?

 

The documentation in this page are out of date: https://bitbucket.org/Unity-Technologies/unitytesttools/wiki/IntegrationTestsRunner

This includes the instructions for some things (call Testing.Pass / Testing.Fail) and also the screenshots are not what it currently looks like.

 

I have some code based tests that ran fine in Unity 5.0 but are timing out in Unity 5.3.

It seems to timeout when using this pattern:

Code (CSharp):

1.         void Start()
2.         {
3.             StartCoroutine(Test());
4.         }
5.  
6.         IEnumerator Test()
7.         {
8.             yield return 0;
9.             DoTest();
10.         }

DoTest uses IntegrationTest.Assert

In Unity 5.3 (or maybe latest Test Tools since I had to import that for Unity 5.3 compatibility) this test times out, like the Assert is never registered...

Is this a known issue?

 

Actually it seems to be a more general problem with IntegrationTest.Assert.

Maybe I'm using it wrong...?

I was doing tests like this:

Code (CSharp):

1. public class Test_EaseActions : MonoBehaviour
2. {
3.     void FixedUpdate()
4.     {
5.         TestStartValues();
6.         enabled = false;
7.     }
8.  
9.     void TestStartValues()
10.     {
11.         var fsm = gameObject.GetComponent<PlayMakerFSM>().Fsm;
12.  
13.         var myFloat = fsm.GetFsmFloat("myFloat").Value;
14.         IntegrationTest.Assert(gameObject, Math.Abs(myFloat - 1.23f) < Mathf.Epsilon, "myFloat = " + myFloat);
15.  
16.         var myRect = fsm.GetFsmRect("myRect").Value;
17.         IntegrationTest.Assert(gameObject, myRect == new Rect(1f,2f,3f,4f), "myRect = " + myRect);
18.  
19.         var myColor = fsm.GetFsmColor("myColor").Value;
20.         IntegrationTest.Assert(gameObject, myColor == Color.red, "myColor = " + myColor);
21.  
22.         var myVector3 = fsm.GetFsmVector3("myVector3").Value;
23.         IntegrationTest.Assert(gameObject, myVector3 == new Vector3(1f,2f,3f), "myVector3 = " + myVector3);
24.     }
25. }

This used to work. But now in Unity 5.3 it only seems to work if I setup Assertion Components with custom comparers.

Setting up this test with AssertionComponents takes a lot longer than writing the above script.

Is there a recommended way of doing tests with scripts instead of AssertionComponents?

 

I've blogged about "What's new with Automated testing" (as of 5.3). Check it out here: http://www.tallior.com/whats-new-automated-testing-unity/

As part of the past, I'm running a short survey to see what other devs' testing habits look like, feel free to jump in: https://www.surveymonkey.com/r/Y9FYVHZ

 

@Tomek: just started playing with this and I think it is awesome. great job really!
i hope the whole thing gets integrated in the ide at some point, not only the unit test one

 

Would love to read your post but the link seems to be dead?

 

Just checked now, it's working fine ...

 

Perfect thanks!

 

Hey all,

Starting with Unity 5.3, there's a new feature called "multi scene editing" that allows having multiple scenes opened at the same time (http://docs.unity3d.com/Manual/MultiSceneEditing.html).

While I haven't tested this feature myself (yet), i thought of new scenarios this may open in terms of automated testing, and was wondering whether these were considered (to be included in a future UTT release).

Integration tests are currently defined in their own separate scene where test objects are placed and asserted against. This can be challenging for games that were not built with testing in mind (hard to separate prefabs and other game objects into a "clean scene").

The MSE feature can assist this scenario by allowing to load a normal game scene together with a "test scene" that contains test objects or script code that will assert against the real objects from the game scene. This can make it much easier to create tests that are executed in the real context of the game, without having to define a new integration test scene.

Is this something that is going to be implemented? anyone else thinks this can be a good idea? the test scene that will be loaded can also be dropped from the build, so it is essentially invisible to the actual game running on the device.

What are your opinions on this? I thought of actually trying to implement it and submit as a pull request to the UTT repo, if there's demand of course.

 

I just got the latest unity (5.3.3) and unity testing tools (1.5.7), and I have tons of warnings about obsolete methods, for example:

Aside from triggering a bad case of OCD, these are really obnoxious since they make it hard to find the real warnings in my code.