Search Unity

  1. Unity 2019.1 is now released.
    Dismiss Notice

Unity Unity Test Tools

Discussion in 'Testing & Automation' started by Tomek-Paszek, Dec 18, 2013.

  1. Tomek-Paszek

    Tomek-Paszek

    Unity Technologies

    Joined:
    Nov 13, 2012
    Posts:
    116
  2. kenshin

    kenshin

    Joined:
    Apr 21, 2010
    Posts:
    916
    Very interesting!

    Thanks a lot
     
  3. Pip-Robbins

    Pip-Robbins

    Joined:
    Oct 19, 2010
    Posts:
    213
    Very exciting news!
     
  4. kdunity

    kdunity

    Joined:
    Jan 3, 2014
    Posts:
    1
    Thanks Tomek, this is very useful. It looks like the example unit tests only deal with manipulating basic primitives. Are there any example unit tests that create and check game objects (and their properties) in a scene? By this I mean unit tests written in C# and run through NUnit, not integration tests.
     
    Last edited: Jan 3, 2014
  5. Tomek-Paszek

    Tomek-Paszek

    Unity Technologies

    Joined:
    Nov 13, 2012
    Posts:
    116
    The unit test framework allows you to instantiate a GameObject from the code. Simply call var go = new GameObject(). From there you can manipulate it and test it. Unfortunately, no frame-dependant operations will work since all tests are executed in the same frame.
     
  6. ijaeckel

    ijaeckel

    Joined:
    Jan 6, 2014
    Posts:
    3
    Hi,

    First of all, thanks for sharing this amazing tool! I am really excited about this after seeing the screenshots. Are there any step-by-step instructions on how to run the tests in the example scene? I opened the AngryBots TestScene.unity, but I am not sure how I can bring up the TestRunner. Even more importantly, how would I add this to an existing application and run tests every now and then?

    Ideally, I would like to include this into my continuous build system as well. Is there a way to include it in the build?

    Thanks,
    Ingo
     
  7. Tomek-Paszek

    Tomek-Paszek

    Unity Technologies

    Joined:
    Nov 13, 2012
    Posts:
    116
    Hi Ingo!

    You will find answers in the docs provided with Unity Test Tools :)

    To save you some trouble: you open the Integration Test Runner under Unity Test Tools menu bar (or press ctrl+alt+shift+t).

    Tomek
     
  8. ijaeckel

    ijaeckel

    Joined:
    Jan 6, 2014
    Posts:
    3
    Hi Tomek,

    Thank you very much! I can now execute the integration tests that come with the AngryBots example. I've also found the UnityTestTools.pdf file. Looks like to run the tests on a continuous integration server, I'd run them in headless mode as described on page 13/14.

    > Unity.exe PATH_TO_YOUR_PROJECT -batchmode -executeMethod BatchTestRunner.RunAllTests -testscene=IntegrationTestsExample

    This generates an XML file which I can parse to determine if they passed/failed. Do you recommend any specific plugins for running Unity tests on CI systems? For Jenkins there seems to be at least Unity3dBuilder.

    Do tests still have to be stripped from the compiled game? Or will they already not be part of the resulting application? I just want to make sure that none of the test dependencies increase the size of the released game.

    Thanks,
    Ingo
     
  9. Tomek-Paszek

    Tomek-Paszek

    Unity Technologies

    Joined:
    Nov 13, 2012
    Posts:
    116
    There is a little mistake there. The PATH_TO_YOUR_PROJECT should be succeeded with -projectPath. You will find some more information about command lines you can use with Unity and how to run it in a batch mode here: http://docs.unity3d.com/Documentation/Manual/CommandLineArguments.html
    I think you don't need any plugins for just running tests. Simply run Unity in batch mode as described in the docs and the read and parse result file. Nevertheless, I never used any plugins for CI.
    Unfortunately, not at the moment. You need to manually (or automatically in you build pipeline) strip them before building your production code. Unit tests, however, will not be included as long as you keep them in Editor folder.
     
  10. xlarrode

    xlarrode

    Joined:
    Nov 9, 2012
    Posts:
    18
    Very nice tool and more stable than TestStar...

    Unfortunately it seems that the XML output can't be validate by the NUnit XSD
    So my JenKins NUnit plugin don't want to display the test result...
    The test-suite element is missing.
     
  11. Tomek-Paszek

    Tomek-Paszek

    Unity Technologies

    Joined:
    Nov 13, 2012
    Posts:
    116
    Thanks for pointing it out! We'll give it a look :)

    Tomek
     
  12. xlarrode

    xlarrode

    Joined:
    Nov 9, 2012
    Posts:
    18
    Well i have added this
    Code (csharp):
    1.  
    2.   private void AddDefaultTestSuite(ResultSummarizer summaryResults)
    3.         {
    4.             xmlWriter.WriteStartElement("test-suite");
    5.             xmlWriter.WriteAttributeString("type",
    6.                                             "TestFixture");
    7.             xmlWriter.WriteAttributeString("name",
    8.                                             "ALLTests");
    9.             xmlWriter.WriteAttributeString("description",
    10.                                             "ALLTests");
    11.             xmlWriter.WriteAttributeString("executed",
    12.                                             "True");
    13.             xmlWriter.WriteAttributeString("result",
    14.                 summaryResults.Failures >0 ?  "Failure" : "Success");
    15.  
    16.             xmlWriter.WriteAttributeString("success",
    17.                summaryResults.Failures > 0 ? "Failure" : "Success");
    18.  
    19.             xmlWriter.WriteAttributeString("time", "0.0");
    20.  
    21.             xmlWriter.WriteAttributeString("asserts", "0");
    22.  
    23.             xmlWriter.WriteStartElement("results");
    24.         }
    25.  
    On InitializeXmlFile and i patched TerminateXmlFile()

    Code (csharp):
    1.  
    2. private void TerminateXmlFile()
    3.         {
    4.             //Add by XL
    5.                 xmlWriter.WriteEndElement(); // test-suite
    6.                 xmlWriter.WriteEndElement(); // result
    7.                
    8.             //
    9.                 xmlWriter.WriteEndElement(); // test-results
    10.                 xmlWriter.WriteEndDocument();
    11.                 xmlWriter.Flush();
    12.  
    13.                 xmlWriter.Close();
    14.         }
    15.  
    And it's working now...

    But adding the test-suite notion to organize the integration tests could be nice.
     
  13. ijaeckel

    ijaeckel

    Joined:
    Jan 6, 2014
    Posts:
    3
    Hi xlarrode,

    are you saying (1) you customized Unity Test Tools to fix the XML output and (2) the Jenkins plugin can now detect if the tests passed/failed?

    Thanks,
    Ingo
     
  14. xlarrode

    xlarrode

    Joined:
    Nov 9, 2012
    Posts:
    18
    Yes exactly...
     
  15. boinged

    boinged

    Joined:
    May 8, 2013
    Posts:
    16
    There is a problem when trying to build a WinPhone8 project because the integration test framework relies on System.Xml.XmlTextWriter, which is not present on this target platform.

    Could perhaps switch to System.Linq.Xml but that would break the web target.
     
  16. Tomek-Paszek

    Tomek-Paszek

    Unity Technologies

    Joined:
    Nov 13, 2012
    Posts:
    116
    Hi,

    We've rewritten the result writer not to depend on the xml library. We also made sure the xml is compatible with appropriate xsd. This should solve some of the problems with platforms and result parsing. It will come with next release.

    Tomek
     
  17. boinged

    boinged

    Joined:
    May 8, 2013
    Posts:
    16
    That's great!

    I think the dependency should also be fine with XmlTextWriter being included in 4.3.3 now for WP8.

    On a sidenote, we actually changed our codebase to use the Linq XML package so I wish Linq.Xml was added to WebPlayer for 4.3.3 instead of the missing System.Xml classes added to WP8 :D
     
  18. RC-1290

    RC-1290

    Joined:
    Jul 2, 2012
    Posts:
    635
    Any advice for testing custom editor code in isolation? For example with a custom PropertyDrawer, field info and attribute info are only available as read-only instance properties, which can't be set through the constructor of PropertyDrawer. And event data is given through the global current event property. So you can't easily mock that data either.

    I guess I could try to create the testing class as a subclass of the class I'm testing, and test the protected methods of the class instead of the public ones. But is that a good idea? Or would you recommend a different approach?
     
  19. Tomek-Paszek

    Tomek-Paszek

    Unity Technologies

    Joined:
    Nov 13, 2012
    Posts:
    116
    Hi RC-1290,

    Unfortunately, the global event state makes some things hard to test. Maybe if you shared some your code you want to test, we could find a different approach.

    Tomek
     
  20. RC-1290

    RC-1290

    Joined:
    Jul 2, 2012
    Posts:
    635
    Thanks for the offer Tomek. I guess it was a bit silly of me to ask for advice without posting any example code. But I'm a bit reluctant to post the code with which I'm having trouble automating the testing, because it's at the core of a product of which I'm not yet sure if I want to release it for free or not.

    I'll try to refactor the code for testing the protected methods (passing the required data as parameters, in stead of having the method accessing global state directly).

    If it turns out that I can't manage to figure it out on my own, I guess I might as well post the code here, because it won't be a really reliable product if I can't easily test it with future versions of Unity.
     
  21. mattias800

    mattias800

    Joined:
    Jan 22, 2014
    Posts:
    3
    Hello!

    I'm trying to make some unit tests.

    I created a C# file in MonoDevelop:

    using System;
    using System.Threading;
    using NUnit.Framework;

    class TextProcessorTests {
    }

    But I run into an error:

    The type or namespace 'NUnit' could not be found. Are you missing a using directive or an assembly reference?

    There are no other errors. The sample tests are running properly, with their 'using NUnit.Framework' statement.
    What am I missing?

    UPDATE:

    I noticed that the nunit-DLL:s are only available in Assembly-CSharp-Editor, not in Assembly-CSharp where my tests are located. Is this the problem? If so, how do I fix it?
     
    Last edited: Jan 22, 2014
  22. Tomek-Paszek

    Tomek-Paszek

    Unity Technologies

    Joined:
    Nov 13, 2012
    Posts:
    116
    Place your file with tests under 'editor' folder. Read more about special folders in Unity here: http://docs.unity3d.com/Documentation/Manual/ScriptCompileOrderFolders.html

    Tomek
     
  23. mattias800

    mattias800

    Joined:
    Jan 22, 2014
    Posts:
    3
    Thanks, it is working! I put it in Assets/Editor/Tests/ .. Can the Editor folder be placed anywhere?
    It doesn't really make sense :) Lets hope TDD becomes part of Unity in the future, and not just an asset.

    Also, the PDF contains no instructions on how to run the tests in the terminal on OSX.
     
  24. mark-gehan

    mark-gehan

    Unity Technologies

    Joined:
    Nov 15, 2012
    Posts:
    4
    I'm hitting an issue with running the tests from the command line. If I put the -quit in the command line arguments, the test results .xml file never gets output. This happens for unit tests and integration tests. Is this by design? I am trying to make this part of my build process and I need Unity to shut down between calls since you can't have two instances open at the same time.

    These both work if I take out the -quit command, but with the -quit I see the window pop open for a second and no output appears.

    Here is what I am trying to run for unit tests:

    "C:\Program Files (x86)\Unity\Editor\Unity.exe" -projectPath [ProjectPath] -testscene=IntegrationTests -batchmode -quit -executeMethod UnityTest.UnitTestView.RunAllTestsBatch

    And for integration tests:
    "C:\Program Files (x86)\Unity\Editor\Unity.exe" -projectPath [ProjectPath] -testscene=IntegrationTests -quit -executeMethod UnityTest.IntegrationTestsRunnerWindow.RunAllTestsMenu

    I also tried encapsulating these methods in a static Editor methods that call EditorApplication.Exit(0) after calling the run tests methods, that also did not work. Has anyone else tried this?
     
  25. Tomek-Paszek

    Tomek-Paszek

    Unity Technologies

    Joined:
    Nov 13, 2012
    Posts:
    116
    Yes, you can place it anywhere.
    Look here how to start unity from the terminal on MacOS. Just pass the same arguments.
     
  26. Tomek-Paszek

    Tomek-Paszek

    Unity Technologies

    Joined:
    Nov 13, 2012
    Posts:
    116
    The reason why the Editor quits and no results are produced is because the tests never get to run. It happens because the runner's start is asynchronous and happens in a certain frame. The -quit parameter will make the editor quit once the static method is executed and that happens almost instantly since the method will not wait for the runner to finish. However, the runner is designed to quit the application, the play mode (in non batch runs) or even the editor (in batch mode runs) by itself after it's finished.

    What you can do is to implement an asynchronous mechanism for detecting if the run has ended (the results are generated or the application is not running anymore) in your continuous integration framework.

    Tomek
     
  27. aoeuaoeu

    aoeuaoeu

    Joined:
    Nov 13, 2013
    Posts:
    2
    There is an error in the v1.1.1 batchmode integration test example in the documentation. The argument in the doc is "-testscene" but the BatchTestRunner looks for "-testscenes".

    I have a Unity project with multiple scenes using Unity Test Tools v1.1.1. I'm trying to run my integration test scene with the following command...

    > Unity -batchmode -projectPath /path/to/my/unity/project -executeMethod BatchTestRunner.RunIntegrationTests -testscenes=IntegrationTestScene

    Application.LoadLevel(1) in BatchTestRunner.Start() appears to load only the first scene configured in the build settings rather than my IntegrationTestScene. I can see in Editor.log that this is true.

    The following line in BatchTestRunner appears to have no effect. Debugging shows that the array of scenes it creates is correct, but that Application.LoadLevel() just ignores the new scenes array.

    EditorBuildSettings.scenes = sceneList.Select (s => new EditorBuildSettingsScene (s, true)).ToArray ();

    Any thoughts on a fix or workaround?
    Thanks.
     
    Last edited: Jan 31, 2014
  28. mark-gehan

    mark-gehan

    Unity Technologies

    Joined:
    Nov 15, 2012
    Posts:
    4
    My CI environment will know that Unity has exited and to continue on. I have the Unit Test runner working in my CI system with -batchmode, but integration tests are still giving me trouble.

    When I run the integration tests with -batchmode, I hit a weird issue where I see the Unit Test Runner Window pop up, but Unity never shows up. The tests are not run. I can see an instance of Unity running in the Process Monitor, but there is no editor window, just the Test Runner window. I have to go into the Process Monitor and actually kill the process manually.
     
  29. Tomek-Paszek

    Tomek-Paszek

    Unity Technologies

    Joined:
    Nov 13, 2012
    Posts:
    116
    testscene parameter was renamed to testscenes after we added the possibility to run multiple scenes... aaand we forgot to update the docs. Sorry about that!

    About your issue, which Editor version you are using?
     
  30. Tomek-Paszek

    Tomek-Paszek

    Unity Technologies

    Joined:
    Nov 13, 2012
    Posts:
    116
    Which OS and Unity version?
    The window pops up because the run is initiated 'through' the window, thus it's being opened (and it's visible because it's how Unity behaves, although it's in batch mode). The window popping up it's probably something that can be avoided.
    The issue with the Editor not closing is related to a failure with the test runner. If you provide log from the run I will be able to tell you more details. http://docs.unity3d.com/Documentation/Manual/LogFiles.html

    Tomek
     
  31. aoeuaoeu

    aoeuaoeu

    Joined:
    Nov 13, 2013
    Posts:
    2
    I'm using Unity for Mac. I've tried this on version 4.3.0f4 and 4.3.4f1. It behaves the same way in both versions. If I only want to run (in batch mode) my integration tests, I need to change my build settings so my IntegrationTestScene is first in the list and make it the only scene with its checkbox checked.
     
    Last edited: Jan 31, 2014
  32. xlarrode

    xlarrode

    Joined:
    Nov 9, 2012
    Posts:
    18
    Hi,
    I 've just updated the UnityTestTools package. ( i like the new GUI ;))
    But now all the tests are waiting the unity editor to get the focus in order to play them.
    It seems that you are making a pause ( i guess a time.scale =0) when the focus is lost...

    How can i remove that ?


    Thanks

    PS : You should also put Icons.cs on Editor/ or WebPlayer won"'t build....
     
    Last edited: Feb 3, 2014
  33. npruehs

    npruehs

    Joined:
    Jul 20, 2012
    Posts:
    67
    Importing Unity Test Tools makes building Windows Store Apps player fail with the following errors:

    Will there by a version for Windows Store Apps with #defines and Metro dlls?
     
  34. Tomek-Paszek

    Tomek-Paszek

    Unity Technologies

    Joined:
    Nov 13, 2012
    Posts:
    116
    It default behaviour of Unity. If you build a Standalone player, it it will also pause by default on lost focus. Go to Player Settings (Edit > Project Settings > Player) and check Run In Background. That should do the trick.

    PS. We go the icons to Editor folder now :)
     
  35. Dmytro Mindra

    Dmytro Mindra

    Joined:
    Jul 1, 2013
    Posts:
    4
    npruehs, we are working on it.
     
  36. AdamV.SG

    AdamV.SG

    Joined:
    Sep 20, 2013
    Posts:
    3
    Hi! The Test Tools are a great upgrade. Thanks!

    However, NUnit test inheritance does not work for me. When I'm trying to unit test interfaces, I create an abstract class testing the interface and a specialized class with just overridden instantiation code (override protected ISomething NewSomething() { return new Something(); }) for each class implementing the interface. As far as I can tell, it should be a pretty common practice, but it does not work for me in Unity.

    When there is no other test in the specialized class, I get a pretty ugly exception every time I try to run the tests:
    ArgumentOutOfRangeException: Argument is out of range.
    System.Linq.Enumerable.ElementAt[UnitTestResult] (IEnumerable`1 source, Int32 index)
    UnityTest.UnitTestView.FindTestResultByName (System.String name) (at Assets/UnityTestTools/UnitTesting/Editor/TestRunner/UnitTestView.cs:392)
    UnityTest.UnitTestView.UpdateTestInfo (ITestResult result) (at Assets/UnityTestTools/UnitTesting/Editor/TestRunner/UnitTestView.cs:386)
    UnityTest.UnitTestView+TestRunnerEventListener.TestFinished (ITestResult result) (at Assets/UnityTestTools/UnitTesting/Editor/TestRunner/UnitTestView.cs:547)
    UnityTest.UnitTestRunner.TestRunnerCallbackList.TestFinished (ITestResult fullName) (at Assets/UnityTestTools/UnitTesting/Editor/TestRunner/TestRunnerCallbackList.cs:22)
    UnityTest.NUnitTestEngine+TestRunnerEventListener.TestFinished (NUnit.Core.TestResult result) (at Assets/UnityTestTools/UnitTesting/Editor/TestRunner/NUnitTestEngine.cs:129)
    NUnit.Core.TestMethod.Run (EventListener listener, ITestFilter filter)
    NUnit.Core.TestSuite.RunAllTests (NUnit.Core.TestResult suiteResult, EventListener listener, ITestFilter filter)
    NUnit.Core.TestSuite.RunSuite (EventListener listener, ITestFilter filter)
    NUnit.Core.TestSuite.RunSuiteInContext (EventListener listener, ITestFilter filter)
    NUnit.Core.TestSuite.Run (EventListener listener, ITestFilter filter)
    NUnit.Core.TestFixture.Run (EventListener listener, ITestFilter filter)
    NUnit.Core.TestSuite.RunAllTests (NUnit.Core.TestResult suiteResult, EventListener listener, ITestFilter filter)
    NUnit.Core.TestSuite.RunSuite (EventListener listener, ITestFilter filter)
    NUnit.Core.TestSuite.RunSuiteInContext (EventListener listener, ITestFilter filter)
    NUnit.Core.TestSuite.Run (EventListener listener, ITestFilter filter)
    NUnit.Core.TestSuite.RunAllTests (NUnit.Core.TestResult suiteResult, EventListener listener, ITestFilter filter)
    NUnit.Core.TestSuite.RunSuite (EventListener listener, ITestFilter filter)
    NUnit.Core.TestSuite.RunSuiteInContext (EventListener listener, ITestFilter filter)
    NUnit.Core.TestSuite.Run (EventListener listener, ITestFilter filter)
    NUnit.Core.TestSuite.RunAllTests (NUnit.Core.TestResult suiteResult, EventListener listener, ITestFilter filter)
    NUnit.Core.TestSuite.RunSuite (EventListener listener, ITestFilter filter)
    NUnit.Core.TestSuite.RunSuiteInContext (EventListener listener, ITestFilter filter)
    NUnit.Core.TestSuite.Run (EventListener listener, ITestFilter filter)
    NUnit.Core.TestSuite.RunAllTests (NUnit.Core.TestResult suiteResult, EventListener listener, ITestFilter filter)
    NUnit.Core.TestSuite.RunSuite (EventListener listener, ITestFilter filter)
    NUnit.Core.TestSuite.RunSuiteInContext (EventListener listener, ITestFilter filter)
    NUnit.Core.TestSuite.Run (EventListener listener, ITestFilter filter)
    NUnit.Core.TestSuite.RunAllTests (NUnit.Core.TestResult suiteResult, EventListener listener, ITestFilter filter)
    NUnit.Core.TestSuite.RunSuite (EventListener listener, ITestFilter filter)
    NUnit.Core.TestSuite.RunSuiteInContext (EventListener listener, ITestFilter filter)
    NUnit.Core.TestSuite.Run (EventListener listener, ITestFilter filter)
    UnityTest.NUnitTestEngine.ExecuteTestSuite (NUnit.Core.TestSuite suite, ITestRunnerCallback testRunnerEventListener, ITestFilter filter) (at Assets/UnityTestTools/UnitTesting/Editor/TestRunner/NUnitTestEngine.cs:93)
    UnityTest.NUnitTestEngine.RunTests (System.String[] tests, ITestRunnerCallback testRunnerEventListener) (at Assets/UnityTestTools/UnitTesting/Editor/TestRunner/NUnitTestEngine.cs:38)
    UnityTest.UnitTestView.StartTestRun () (at Assets/UnityTestTools/UnitTesting/Editor/TestRunner/UnitTestView.cs:462)
    UnityEngine.Debug:LogException(Exception)
    UnityTest.UnitTestView:StartTestRun() (at Assets/UnityTestTools/UnitTesting/Editor/TestRunner/UnitTestView.cs:468)
    UnityTest.UnitTestView:Update() (at Assets/UnityTestTools/UnitTesting/Editor/TestRunner/UnitTestView.cs:400)
    UnityEditor.EditorApplication:Internal_CallUpdateFunctions()


    If I add some test to the specialized class, the exception goes away and the test runs OK. BUT still none of the inherited tests is executed.

    Has anyone else experienced this? Or is the inheritance working for you?

    Thanks for reply,
    Adam
     
  37. Tomek-Paszek

    Tomek-Paszek

    Unity Technologies

    Joined:
    Nov 13, 2012
    Posts:
    116
    Hi Adam!

    Thanks, we are happy you like the update!
    Could you please post some of the code that causes the exception and the OS you are using?

    Tomek
     
  38. AdamV.SG

    AdamV.SG

    Joined:
    Sep 20, 2013
    Posts:
    3
    I'm on Windows 8 Pro, 64 bit, using Unity 4.3.3f1, Test Tools 1.1.1.

    Well, when I have tried to reproduce the exception in a fresh project just with Test Tools installed, I didn't get the exception... And now when I have reloaded the old project, it does no appear even if I remove the additional tests. Strange... (The exception appeared when there was no TestOtherStuff() in the code below, but disappeared when this was uncommented.)

    However, I'm more concerned about the inheritance of the test methods. Should they work this way? (I have each class or interface in a separate file.)

    Code (csharp):
    1. public interface ISomething
    2. {
    3.     int ReturnOne();
    4. }
    5.  
    6. public abstract class TestISomething
    7. {
    8.     public abstract ISomething NewSomething();
    9.  
    10.     [Test]
    11.     void TestReturnOne()
    12.     {
    13.         ISomething s = NewSomething();
    14.         Assert.That(s.ReturnOne(), Is.EqualTo(1));
    15.     }
    16. }
    17.  
    18. public class Something : ISomething
    19. {
    20.     public int ReturnOne() { return 1; }
    21.     ... other stuff
    22. }
    23.  
    24. public class TestSomething : TestISomething
    25. {
    26.     public override ISomething NewSomething() { return new Something(); }
    27.  
    28.      [Test]
    29.      public void TestOtherStuff() { ... }
    30. }
    From what I have read, the NUnit should execute both TestReturnOne() and TestOtherStuff() on TestSomething. But only TestOtherStuff() is actually executed. TestReturnOne() does not produce any output to console, unit test runner, or XML.

    My main question is: Do I have something wrong? If yes, what's the best way to test multiple classes with the same interface without code duplication?

    Thanks!
     
  39. perlohmann

    perlohmann

    Joined:
    Feb 12, 2009
    Posts:
    221
    It states that it requires 4.0.0, but I am getting a bunch of errors on 4.0.1 (to begin with there where two which I fixed, but after that there where about 26 new compile errors). An example of one of the compile errors are:

    Assets/UnityTestTools/Assertions/Editor/AssertionComponentEditor.cs(38,59): error CS0103: The name `target' does not exist in the current context

    or

    Assets/UnityTestTools/Assertions/Editor/AssertionComponentEditor.cs(48,33): error CS1502: The best overloaded method match for `UnityTest.AssertionComponentEditor.DrawOptionsForAfterPeriodOfTime(UnityTest.AssertionComponent)' has some invalid arguments
     
  40. juanbatovi

    juanbatovi

    Joined:
    Jun 27, 2013
    Posts:
    1
    I'm having the same issue. The only way to get the integration tests running in batch mode is by manually selecting *only* the integration scenes in the editor build settings. Tested on OS X and Windows (Unity 4.3.4f1 and Unity Test Tools 1.1.1). Is there any workaround?

    No problem with unit tests though. Awesome tools, by the way. Thanks for sharing.
     
    Last edited: Feb 24, 2014
  41. Franzi

    Franzi

    Joined:
    Feb 28, 2014
    Posts:
    3
    I've been having an issue using Unity 4.3.4f1 and the Unity Test Tools. I ran into this initially trying to integrate the tools into my own project, but have tried with a new project after just importing the Unity Test Tools and running the sample tests and am getting the same error.

    When running the Unit Tests through MonoDevelop, I receive the error for C# tests:

    Code (csharp):
    1. Internal error
    2. FileLoadException: Could not load file or assembly 'nunit.framework, Version=2.6.2.0, Culture=neutral, PublicKeyToken=93811cf430eaf12a' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference.
    Boo tests give the error:

    Code (csharp):
    1. TestFixtureSetUp failed
    And UnityScript tests crash MonoDevelop when trying to access their failed output.

    This is using Windows 7 with Unity Free. I have tried cleaning up my Mono/Unity installation. Any ideas on what might be going on here?
     
  42. Franzi

    Franzi

    Joined:
    Feb 28, 2014
    Posts:
    3
    I should also note here that the tests seem to run fine when run through the Unit Test Runner inside of Unity itself, just not through MonoDevelop.
     
  43. Tomek-Paszek

    Tomek-Paszek

    Unity Technologies

    Joined:
    Nov 13, 2012
    Posts:
    116
    Hi Franzi,
    The unit test runner from MonoDevelop won't work unless you don't have any references to Unity. The assembly load is probably caused by the fact that we ship UTT with our custom build of nUnit and it's different from the one shipped with MonoDevelop. Use to test runner in the editor and it should be fine :)
    I wrote a simple test in Boo using TestFixtureSetUp and it works fine for me. Can you post the exact code you have?
     
  44. Mikeysee

    Mikeysee

    Joined:
    Oct 14, 2013
    Posts:
    155
  45. ValeBros

    ValeBros

    Joined:
    Feb 26, 2014
    Posts:
    10
    I guys,

    I try to download the free version of unity, but every time I try to start the download.. gives me error 404 not found .. what happens?

    http://unity3d.com/unity/download

    This is the link
     
  46. Franzi

    Franzi

    Joined:
    Feb 28, 2014
    Posts:
    3
    The code I'm using are the examples that came with the Unity Test Tools package. A new project, just importing the Unity Test Tools gives me these errors. There are no references to UnityEngine or UnityEditor in my tests.

    I have had previous projects work just fine with MonoDevelop, Unit Testing and the Unity Test Tools. This has only been an issue on a clean install of Unity 4.3.4f1 on a new machine.
     
  47. raphaelbaldi

    raphaelbaldi

    Joined:
    Mar 4, 2008
    Posts:
    23
    Is it possible to run unit tests, during batch mode, without quitting the Editor after the tests are done? Also, is it possible to know when the tests are done (as they are executed asynchronously)?

    Our current issue is that we have Jenkins set to run a custom builder for each of our projects and we wanted to run tests every time a new deploy occurs. What we've done so far was to set build steps to call both UnityTest.UnitTestView.RunAllTestsBatch and BatchTestRunner.RunIntegrationTests. It is really weird to have the Editor open thrice for a single build to run as both of the calls will quit the editor as soon tests are done.

    What we want to achieve:
    - Call our custom deploy method from batch mode: ProjectBuilder.CIDeploy (it will handle loading a lot of environment variables).
    - If there is an environment variable set to run unit tests - and integration tests - call them.
    - When tests are done, check if any of them failed. If so, exit the editor with some error code.
    - If all tests passed, deploy the player and perform post build tasks.

    This way the editor would be opened once to run tests and deploy the player, saving a lot of time.
     
  48. Tomek-Paszek

    Tomek-Paszek

    Unity Technologies

    Joined:
    Nov 13, 2012
    Posts:
    116
    @Mikeysee
    Check this thread for mirror: link

    @ValeBros
    I'm afraid it's a wrong thread.

    @Franzi
    We don't use TestFixtureSetUp in the examples so you had to add something yourself. Anyway, I double checked it and it seems to be fine for me.
     
  49. Tomek-Paszek

    Tomek-Paszek

    Unity Technologies

    Joined:
    Nov 13, 2012
    Posts:
    116
    Yes, it is possible. You can instiantiaze your own NUnitTestEngine and run the tests with it. The tests are not execute in the main thread, synchronously. You can set you own callback to handle runner's events where you can check the result.

    I hope that will solve your problem you are trying to achieve.
     
  50. rahuxx

    rahuxx

    Joined:
    May 8, 2009
    Posts:
    532
    Is there any good video tutorial for using this tool?