Search Unity

Unity Unity Test Tools

Discussion in 'Testing & Automation' started by Tomek-Paszek, Dec 18, 2013.

  1. Arkade

    Arkade

    Joined:
    Oct 11, 2012
    Posts:
    565
    Well, it's submitted! I got the documentation in Github but ran out of time for the source, asset store resources, etc. That'll probably now have to wait until after the UK Develop Conference next week (anyone here going?)

    I also worried that I was spamming and/or commandeering this thread so I've spawned a separate one here for CUTTE.

    As noted over there, contributions most eagerly sought. (caveat my finding a contributors agreement!)

    HTH, Rupert.
     
  2. karmahoos

    karmahoos

    Joined:
    Jul 9, 2014
    Posts:
    1
    I started a convo with Tomek, but it occurs to me that this may be something that other people have encountered and solved. Has anyone tried tying in the unit testing or integration testing pieces with code coverage tools? I almost have the unit tests going thru mono (though I'm still having some issues), but I'm worried that having something similar for the integration tests may be just plain infeasible.
     
  3. Arkade

    Arkade

    Joined:
    Oct 11, 2012
    Posts:
    565
    Not tried but would love both (or either) to be possible!

    Obvious ideal being that they can generate standard coverage report data file that can be graphed, used for highlighting retrograde steps, refusing integrations, etc. by build servers or whatever. (but anything better than nothing, huh)

    How does coverage usually work in CLR environment? If it's hooks in the VM (as I believe is used in some other platforms), I guess the question is whether they're already or Tomek can get them exposed for use?

    If it's something higher level (e.g. extra IL instructions by pre/post processing before running), then maybe there's no need.

    Enough hypothesizing in the dark from me ;-)
     
  4. Tomek-Paszek

    Tomek-Paszek

    Unity Technologies

    Joined:
    Nov 13, 2012
    Posts:
    116
    It's a big wild guess from me, since I never really dealt with code coverage from the technical side, but I think there are several approaches:
    • Code junks injection before compilation happens and removing them after the execution. Basically, you scan code files and write the reporting code before and after part you want to measure (begging of the method, if, while loops, return statements, etc.). Seems like a fairly easy solution but somehow doesn't seems professional for me :)
    • Weaving IL code to dll during compilation - probably requires compiler hooks. Seems like the way to go but probably not feasible in Unity.
    • Maybe it's possible to weave IL code in runtime with tools like Mono.Cecil but I didn't investigate it yet.
    If anyone had more experience, I'd love to hear about it :)
     
  5. Arkade

    Arkade

    Joined:
    Oct 11, 2012
    Posts:
    565
    I'd like to offer 2 tiny patches for UTT:

    1. Remove restriction that AssertionComponentEditor doesn't provide GUI for certain Comparer/ActionBase subclasses members (Vector3, etc).

    I.e. Create a Comparer with a public Vector3 member and it currently isn't shown in the GUI since typeof(Vector3).IsSerializable == false. The test that does this seems superfluous. Here's a gist...

    https://gist.github.com/Arakade/749a2d43e62dcf8beee2

    Removing doesn't seem to have caused any problems so far but, if it did, a method that properly checks Unity serializability could be written by checking both IsSerializable and all of the special cases listed http://docs.unity3d.com/ScriptReference/SerializeField.html

    2. AssertionComponentEditor should includeChildren so Comparer/ActionBase subclasses can include Arrays, etc.
    Here's another gist...
    http://forum.unity3d.com/threads/unity-test-tools.218287/

    HTH, Rupert.

    p.s. CUTTE got declined from the AssetStore because I included the BSD 3 license. I'm awaiting their response on how to free-license per the Provider terms section 6.2. Once fixed, I'll get the source and these patches into the GitHub repo.

    p.p.s. <funny> You know you've been testing enough when you typo Assert Store :D I wonder if Tomek's next work for Unity will be a new store where we sell truths! </funny>
     
  6. xuanyusong

    xuanyusong

    Joined:
    Apr 10, 2013
    Posts:
    45
    unity 4.5.1 import has this error

    Internal compiler error. See the console log for more information. output was:
    Unhandled Exception: System.Reflection.ReflectionTypeLoadException: The classes in the module cannot be loaded.
    at (wrapper managed-to-native) System.Reflection.Assembly:GetTypes (bool)
    at System.Reflection.Assembly.GetTypes () [0x00000] in <filename unknown>:0
    at Mono.CSharp.RootNamespace.ComputeNamespaces (System.Reflection.Assembly assembly, System.Type extensionType) [0x00000] in <filename unknown>:0
    at Mono.CSharp.RootNamespace.ComputeNamespace (Mono.CSharp.CompilerContext ctx, System.Type extensionType) [0x00000] in <filename unknown>:0
    at Mono.CSharp.GlobalRootNamespace.ComputeNamespaces (Mono.CSharp.CompilerContext ctx) [0x00000] in <filename unknown>:0
    at Mono.CSharp.Driver.LoadReferences () [0x00000] in <filename unknown>:0
    at Mono.CSharp.Driver.Compile () [0x00000] in <filename unknown>:0
    at Mono.CSharp.Driver.Main (System.String[] args) [0x00000] in <filename unknown>:0
     
  7. Alex-Chouls

    Alex-Chouls

    Joined:
    Mar 24, 2009
    Posts:
    2,294
    Starting to use Unity Test Tools and it's looking very promising!

    However one problem had me tearing my hair out for a few hours. I was calling IntegrationTest.Pass but the test would carry on and timeout. Eventually I realized it has something to do with Application.RegisterLogCallback, which is used by Test Tools, but also by ScriptInspector2 and PlayMaker both in my project.

    Apparently only one script can register this callback at a time:
    http://feedback.unity3d.com/suggest...gisterlogcallback-to-allow-multiple-callbacks

    But I'm not sure what the rules are. Sometimes Test Tools works fine with the other tools open... maybe it grabbed the callback first... or last...?

    Anyway, I'm investigating Test Tools to replace Test Star which doesn't seem to be maintained any more. I like the design of Test Tools and I love the ability to run in batch mode. However, I'm using it to test PlayMaker, so the RegisterLogCallback conflict is worrying... I can work around it, but hacking a test environment just feels like starting off on the wrong foot ;)

    Maybe you have some pull with the core unity team to bump the feature request up in priority? With the growth of the Asset Store lots of editor tools are going to be fighting over RegisterLogCallback and a lot of hair will be pulled out!
     
  8. Alex-Chouls

    Alex-Chouls

    Joined:
    Mar 24, 2009
    Posts:
    2,294
    I'm getting this error testing on Metro with Unity 4.2.2:
    MetroErrors.PNG
     
  9. Tomek-Paszek

    Tomek-Paszek

    Unity Technologies

    Joined:
    Nov 13, 2012
    Posts:
    116
    @雨松MOMO
    Can you write something more about the error? It's hard to tell anything since the callstack doesn't correspond the tools.

    @Alex Chouls
    Yes, sorry about that. RegisterLogCallback should definitely allow multiple callbacks to be registered. Now, every RegisterLogCallback will override anything that is registered there already without any warning or notification. I hope devs will fix it asap.

    About the metro issue, I'll give it a look but metro tends to give lots of problem because it's actual using it's own .Net (instead of Unity's Mono).
     
  10. Tomek-Paszek

    Tomek-Paszek

    Unity Technologies

    Joined:
    Nov 13, 2012
    Posts:
    116
    I'm not sure how it's implemented in the other tools but you could probably force it by settings the callback in Update method.
    PS. I already poked some guys and added a check for RegisterLogCallback in the next UTT version
     
  11. Alex-Chouls

    Alex-Chouls

    Joined:
    Mar 24, 2009
    Posts:
    2,294
    I hear you! It would be great if you could get this working. Being able to quickly run integration tests across all platforms is VERY useful. BTW, it works fine on WP8.
     
  12. Alex-Chouls

    Alex-Chouls

    Joined:
    Mar 24, 2009
    Posts:
    2,294
    I found a workaround: Make a new layout and add the Unity Test Tools to the layout first - it seems that the first script to grab the callback keeps it... at least in 4.2 so far...

    But obviously a real solution would be better :)
     
  13. Alex-Chouls

    Alex-Chouls

    Joined:
    Mar 24, 2009
    Posts:
    2,294
    Is there a recommended way to load levels in integration tests? Right now it breaks the test runner - the dialog hangs and can't be cancelled. I imagine because the test it was running was unloaded when the new level was loaded... I tried DontDestroyOnLoad on the test object, which keeps it around, but still hangs the test runner...
     
    McGravity likes this.
  14. Tomek-Paszek

    Tomek-Paszek

    Unity Technologies

    Joined:
    Nov 13, 2012
    Posts:
    116
    The runner wasn't designed to work with level loading. I've heard people modifying it and using it that way but I can't promise it will work. The DontDestroyOnLoad *might* do the job but you'd need to set it on the TestRunner object too. There might be also the runner window standing on the way. But as I said, no promises there. Noone tested this scenario (maybe it the future though ;)). If you manage to run it, i'd be happy to hear about it!
     
    maxrevilo likes this.
  15. xuanyusong

    xuanyusong

    Joined:
    Apr 10, 2013
    Posts:
    45
    @Tomek Paszek

    Internal compiler error. See the console log for more information. output was:
    Unhandled Exception: System.Reflection.ReflectionTypeLoadException: The classes in the module cannot be loaded.
    at (wrapper managed-to-native) System.Reflection.Assembly:GetTypes (bool)
    at System.Reflection.Assembly.GetTypes () [0x00000] in <filename unknown>:0
    at Mono.CSharp.RootNamespace.ComputeNamespaces (System.Reflection.Assembly assembly, System.Type extensionType) [0x00000] in <filename unknown>:0
    at Mono.CSharp.RootNamespace.ComputeNamespace (Mono.CSharp.CompilerContext ctx, System.Type extensionType) [0x00000] in <filename unknown>:0
    at Mono.CSharp.GlobalRootNamespace.ComputeNamespaces (Mono.CSharp.CompilerContext ctx) [0x00000] in <filename unknown>:0
    at Mono.CSharp.Driver.LoadReferences () [0x00000] in <filename unknown>:0
    at Mono.CSharp.Driver.Compile () [0x00000] in <filename unknown>:0
    at Mono.CSharp.Driver.Main (System.String[] args) [0x00000] in <filename unknown>:0


    I konw why has this error .

    beasuse I use a unity plugin . https://sites.google.com/site/apshaiarts/

    when I delete this plugin all is OK
     
  16. xuanyusong

    xuanyusong

    Joined:
    Apr 10, 2013
    Posts:
    45
  17. Alex-Chouls

    Alex-Chouls

    Joined:
    Mar 24, 2009
    Posts:
    2,294
    Using Platform Runner with BB10 I get this error:

    Error: Bar file must have .bar extension: IntegrationTests

    UnityEditor.BuildPipeline:BuildPlayer(String[], String, BuildTarget, BuildOptions)
    UnityTest.IntegrationTests.PlatformRunner:BuildAndRunInPlayer(BuildTarget, String[], String, String) (at Assets/UnityTestTools/IntegrationTestsFramework/TestRunner/Editor/PlatformRunner/PlatformRunner.cs:58)
    UnityTest.IntegrationTests.PlatformRunnerSettingsWindow:OnGUI() (at Assets/UnityTestTools/IntegrationTestsFramework/TestRunner/Editor/PlatformRunner/PlatformRunnerSettingsWindow.cs:83)
    UnityEditor.DockArea:OnGUI()
     
  18. Tomek-Paszek

    Tomek-Paszek

    Unity Technologies

    Joined:
    Nov 13, 2012
    Posts:
    116
    Hey, It's a bug in the code. It will be fixed in the next release but in the meantime you can just add an appropriate case in method GetTempPath in PlatformRunner.cs:
    case BuildTarget.BlackBerry:
    return path + ".bar";

     
    Last edited: Jul 29, 2014
  19. Alex-Chouls

    Alex-Chouls

    Joined:
    Mar 24, 2009
    Posts:
    2,294
    Cool - thanks!
     
  20. Alex-Chouls

    Alex-Chouls

    Joined:
    Mar 24, 2009
    Posts:
    2,294
    I'm trying to write test scripts that use IntegrationTest.Assert.
    I'm setting up a bunch of Asserts in OnEnable and a bunch in Update, but only the first set of asserts runs.
    Right now it only runs the OnEnable asserts. If I comment out the OnEnable tests it runs the Update asserts.
    How do I tell the test runner to check asserts in both OnEnable and Update?

    EDIT: Some code if it helps:
    Code (CSharp):
    1. using System;
    2. using UnityEngine;
    3. using System.Collections;
    4.  
    5. namespace HutongGames.PlayMaker.Tests
    6. {
    7.     [IntegrationTest.SucceedWithAssertions]
    8.     public class Test_SetProperty : MonoBehaviour
    9.     {
    10.         void OnEnable()
    11.         {
    12.             TestInitialState();
    13.         }
    14.  
    15.         void Update()
    16.         {
    17.             DoTest();
    18.         }
    19.  
    20.         void TestInitialState()
    21.         {
    22.             var target = GetComponent<CallMethodTests>();
    23.             IntegrationTest.Assert(gameObject, target != null, "Component not found!");
    24.             target.TestDefaultValues();
    25.         }
    26.  
    27.         void DoTest()
    28.         {
    29.             var target = GetComponent<CallMethodTests>();
    30.             IntegrationTest.Assert(gameObject, target != null, "Component not found!");
    31.             target.TestValues();
    32.         }
    33.     }
    34. }
    TestDefaultValues and TestValues use IntegrationTest.Asserts
     
  21. Alex-Chouls

    Alex-Chouls

    Joined:
    Mar 24, 2009
    Posts:
    2,294
    If I select a child of the test object and hit "run selected test", I get this error and the test runner hangs:

    KeyNotFoundException: The given key was not present in the dictionary.
    System.Collections.Generic.Dictionary`2[UnityTest.ITestComponent,System.Collections.Generic.HashSet`1[UnityTest.ITestComponent]].get_Item (ITestComponent key) (at /Applications/buildAgent/work/c514da0c8183631c/mcs/class/corlib/System.Collections.Generic/Dictionary.cs:150)
    UnityTest.IntegrationTestRunner.IntegrationTestsProvider.FindInnerTestGroup (ITestComponent group) (at Assets/UnityTestTools/IntegrationTestsFramework/TestRunner/IntegrationTestsProvider.cs:87)
    UnityTest.IntegrationTestRunner.IntegrationTestsProvider..ctor (IEnumerable`1 tests) (at Assets/UnityTestTools/IntegrationTestsFramework/TestRunner/IntegrationTestsProvider.cs:28)
    UnityTest.TestRunner.InitRunner (System.Collections.Generic.List`1 tests, System.Collections.Generic.List`1 dynamicTestsToRun) (at Assets/UnityTestTools/IntegrationTestsFramework/TestRunner/TestRunner.cs:110)
    UnityTest.IntegrationTestsRunnerWindow.Update () (at Assets/UnityTestTools/IntegrationTestsFramework/TestRunner/Editor/IntegrationTestsRunnerWindow.cs:250)
    System.Reflection.MonoMethod.Invoke (System.Object obj, BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) (at /Applications/buildAgent/work/c514da0c8183631c/mcs/class/corlib/System.Reflection/MonoMethod.cs:222)
    Rethrow as TargetInvocationException: Exception has been thrown by the target of an invocation.
    System.Reflection.MonoMethod.Invoke (System.Object obj, BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) (at /Applications/buildAgent/work/c514da0c8183631c/mcs/class/corlib/System.Reflection/MonoMethod.cs:232)
    System.Reflection.MethodBase.Invoke (System.Object obj, System.Object[] parameters) (at /Applications/buildAgent/work/c514da0c8183631c/mcs/class/corlib/System.Reflection/MethodBase.cs:115)
    UnityEditor.HostView.Invoke (System.String methodName, System.Object obj) (at C:/BuildAgent/work/ea95e74f6e5f192d/Editor/Mono/GUI/DockArea.cs:234)
    UnityEditor.HostView.Invoke (System.String methodName) (at C:/BuildAgent/work/ea95e74f6e5f192d/Editor/Mono/GUI/DockArea.cs:227)
    UnityEditor.HostView.SendUpdate () (at C:/BuildAgent/work/ea95e74f6e5f192d/Editor/Mono/GUI/DockArea.cs:284)
    UnityEditor.EditorApplication.Internal_CallUpdateFunctions () (at C:/BuildAgent/work/ea95e74f6e5f192d/Editor/MonoGenerated/Editor/EditorApplication.cs:227)
     
  22. Tomek-Paszek

    Tomek-Paszek

    Unity Technologies

    Joined:
    Nov 13, 2012
    Posts:
    116
    @Alex Chouls
    The runner check for assertion right before the test is enabled. Because of that, it doesn't 'consider' the assertion that are added dynamically afterwards. This can be fixed, I believe :)

    Thats a bug. Thanks!
     
  23. Arkade

    Arkade

    Joined:
    Oct 11, 2012
    Posts:
    565
    Does the latest version require Unity Pro for mobile (Android/iOS) intentionally?
    There are references to .NET Sockets in non-Editor code meaning Unity Indie mobile won't compile one's project if UTT is present.
    I'm hopping this was accidental mistake :)
    Thanks, Rupert.
     
  24. Alex-Chouls

    Alex-Chouls

    Joined:
    Mar 24, 2009
    Posts:
    2,294
    I have a similar problem building for WP8, where Sockets are not supported at all!
     
  25. Alex-Chouls

    Alex-Chouls

    Joined:
    Mar 24, 2009
    Posts:
    2,294
    I think the auto-discovery of tests in the project might need to be a bit more discerning...

    I made a simple Test behaviour (just to test something, not to write a test) and immediately got these errors in the console:

    Assets/UnityTestTools/UnitTesting/Editor/NUnit/Renderer/UnitTestRendererLine.cs(29,46): error CS1061: Type `Test' does not contain a definition for `TestName' and no extension method `TestName' of type `Test' could be found (are you missing a using directive or an assembly reference?)

    Assets/UnityTestTools/UnitTesting/Editor/NUnit/Renderer/UnitTestRendererLine.cs(30,50): error CS1061: Type `Test' does not contain a definition for `TestName' and no extension method `TestName' of type `Test' could be found (are you missing a using directive or an assembly reference?)

    Assets/UnityTestTools/UnitTesting/Editor/NUnit/Renderer/UnitTestRendererLine.cs(31,46): error CS1061: Type `Test' does not contain a definition for `TestName' and no extension method `TestName' of type `Test' could be found (are you missing a using directive or an assembly reference?)

    ...
     
  26. Alex-Chouls

    Alex-Chouls

    Joined:
    Mar 24, 2009
    Posts:
    2,294
    What about assertions in code? To simplify my example, this passes:

    Code (CSharp):
    1. public class MyCodedTest: MonoBehaviour
    2. {
    3.     void OnEnable ()
    4.     {
    5.         IntegrationTest.Assert(gameObject, true);
    6.     }
    7.    
    8.     void Update ()
    9.     {
    10.         IntegrationTest.Assert(gameObject, false);
    11.     }
    12. }
    Some tests are easier to write with IntegrationTest.Assert instead of making an AssertionComponent etc.

    Do you plan to support that workflow? Or is it already possible somehow...?

    Or perhaps if the AssertionComponent had the ability to call a method instead of just comparing properties...
     
  27. ikriz

    ikriz

    Joined:
    Dec 3, 2009
    Posts:
    85
    I was running into problems with NUnit plugin and resultFilePath variable in Jenkins because of invalid xml, turned out the xml file wasn't being replaced but overwritten leaving old data (read: larger previous files) intact.
    Minor fix in XmlResultWriter.cs creates a new file if it exists then it overwrites the file completely:

    Code (CSharp):
    1.  
    2. diff --git a/Assets/UnityTestTools/Common/ResultWriter/XmlResultWriter.cs b/Assets/UnityTestTools/Common/ResultWriter/XmlResultWriter.cs
    3. index abaedb5..326f7ab 100644
    4. --- a/Assets/UnityTestTools/Common/ResultWriter/XmlResultWriter.cs
    5. +++ b/Assets/UnityTestTools/Common/ResultWriter/XmlResultWriter.cs
    6. @@ -316,11 +316,7 @@ namespace UnityTest
    7.              {
    8.                  var path = System.IO.Path.Combine (resultDestiantion, resultFileName);
    9.                  Debug.Log ("Saving results in " + path);
    10. -                using (var fs = System.IO.File.OpenWrite (path))
    11. -                using (var sw = new System.IO.StreamWriter (fs, Encoding.UTF8))
    12. -                {
    13. -                    sw.Write (GetTestResult ());
    14. -                }
    15. +                System.IO.File.WriteAllText(path, GetTestResult(), Encoding.UTF8);
    16.              }
    17.              catch (Exception e)
    18.              {
    19.  
     
  28. Tomek-Paszek

    Tomek-Paszek

    Unity Technologies

    Joined:
    Nov 13, 2012
    Posts:
    116
    Hey guys, sorry for the delays. I'm a bit busy at Unite at the moment. I'll try to answer your questions whenever a get a chance to.

    PS. From now on we host the tool on bitbucket! https://bitbucket.org/Unity-Technologies/unitytesttools
    You are welcome to make pull requests!
     
    liortal likes this.
  29. ikriz

    ikriz

    Joined:
    Dec 3, 2009
    Posts:
    85
    Have done just that :)
     
  30. Arkade

    Arkade

    Joined:
    Oct 11, 2012
    Posts:
    565
    Congrats on getting it open-sourced! Really happy -- thanks! I'm a bit busy with LudumDare atm but hope to offer my CUTTE bits (and scrap CUTTE) once LD's finished :)
    To save a round of pull-request & fix, for an Integration Test Editor Extension helper, is that best to go in a new directory inside IntegrationTestFramework ?

    Thanks, Rupert.
     
  31. _Shockwave

    _Shockwave

    Joined:
    Sep 2, 2012
    Posts:
    21
    Hi there, I'm getting the exact same issue you had. My jenkins doesn't recognize the xml report because it thinks the xml is not formatted in a junit format that jenkins understand:



    I validated the outputted xml for my tests against the junit.xsd format used by jenkins and it shows that the output is not a valid junit xml format.



    I was wondering if you can share your XmlResultWriter.cs file or explain the changes you have done.
    P.S: I'm using Unity 4.5.3f and Jenkins 1.576

    P.P.S: I would really appreciate any help as I'm currently stuck at this step (getting jenkins to parse xml results generated from unity test tools).
     
    Last edited: Aug 24, 2014
  32. ikriz

    ikriz

    Joined:
    Dec 3, 2009
    Posts:
    85
    huge post there man... check the open source repository has a pull request with my changes
     
  33. _Shockwave

    _Shockwave

    Joined:
    Sep 2, 2012
    Posts:
    21
    @ikriz, thanks for the reply man. I will take a look at your pull request, cheers!
     
  34. liortal

    liortal

    Joined:
    Oct 17, 2012
    Posts:
    3,201
    Sorry for the "bump" - i believe most coverage frameworks use the .NET profiling API - http://msdn.microsoft.com/en-us/library/bb384493(v=vs.110).aspx

    This API allows to do all sorts of weird and wonderful things like manipulating IL code at runtime.

    I think this is also how some test mocking frameworks are built (e.g: TypeMock Isolator).
     
  35. Tomek-Paszek

    Tomek-Paszek

    Unity Technologies

    Joined:
    Nov 13, 2012
    Posts:
    116
    Whoa, the test tools repo has already 25 forks! That's amazing! I just got back from a week off after Unite and trying to get back on track so excuse me for the delays :) Since the tools are more 'public' now, i'm gonna try to create a general roadmap for development and try to get into discussion with you guys. We still have some cool features to be added there!

    To answer some of your questions:

    Probably yes. I don't know what extensions you have in there. Do you have a fork with your changes?

    @_Shockwave
    The results are in nUnit format, not jUnit. Try this plugin for Jenkins: https://wiki.jenkins-ci.org/display/JENKINS/NUnit+Plugin

    @Arkade @Alex Chouls
    It wasn't my purpose to disallow running tests no non-Pro version. Should be possible to fix by if-defing out some parts of the code. I'll take a look at it soon.

    @Alex Chouls
    I'm working now on an assertion library that will allow you to set up 'low level' assertion. The assertion component is more of a visual tool for setting assertion that should be verified in certain point of time.

    I know the amount of different testing framework/assertion is growing but each of them has it's own place ;) As always, I'm always happy to hear any comments!
     
    landon912 likes this.
  36. Arkade

    Arkade

    Joined:
    Oct 11, 2012
    Posts:
    565
    Welcome back @Tomek Paszek - hope you had a good hol!

    Clarification: it prevents building!
    Steps to reproduce:
    1. Install indie
    2. add test tools
    3. (optional: create test, run test = fine)
    4. create simple scene
    5. Build exe (windows, Web player, etc) = fine.
    6. switch to android
    7. (optional: Tests run in editor (IIRC))
    8. Build apk = failure.
    HTH
     
  37. Arkade

    Arkade

    Joined:
    Oct 11, 2012
    Posts:
    565
  38. Tomek-Paszek

    Tomek-Paszek

    Unity Technologies

    Joined:
    Nov 13, 2012
    Posts:
    116
  39. Alex-Chouls

    Alex-Chouls

    Joined:
    Mar 24, 2009
    Posts:
    2,294
    Cool :)
     
  40. Arkade

    Arkade

    Joined:
    Oct 11, 2012
    Posts:
    565
    If you haven't done this already, I have a start from my own workspace if that's be helpful? I'll clone the repo, push and offer in case so. I'll admit it was a potentially overly simple fix and one that may only work automatically on Unity 4.5+...

    I noticed the appropriate sections have code checking UNITY_METRO. Adding extra conditions there seems to suffice.

    I say "automatically" since the #define for Unity Pro was only introduced then. One could instruct Pro users of prior versions to set the define in the build settings for Android and iOS if they want net sending of tests. (seems a reasonable workaround for legacy support.)

    HTH, Rupert.
     
  41. Arkade

    Arkade

    Joined:
    Oct 11, 2012
    Posts:
    565
    Pull request offered for the Indie networking dependency (Issue #1).
     
  42. cesarpo

    cesarpo

    Joined:
    Jun 8, 2013
    Posts:
    98
    Which is the roadmap for the test tools? Are you guys planning to add some kind of automatic Dependency Injection framework? Seems to me that is one of the big things left. I saw the blog posts about doing DI by hand, and though it looks nice for short examples, it becomes unmanageable for mid to large projects.
     
    Last edited: Sep 8, 2014
  43. Tomek-Paszek

    Tomek-Paszek

    Unity Technologies

    Joined:
    Nov 13, 2012
    Posts:
    116
    I agree with that. Thanks for your pull request! I had similar solution ready but couldn't find the time to test it on Win8. Yours seems to be more comprehensive with the tests (I like that!).
    I will introduce a Development folder where I will put the tests (we have something similar internally) and other code not directly related to UTT package.

    We will probably use BitBucket's issue tracker as a 'roadmap'.
    About the DI frameworks, we are not planning on implementing one at the moment. There are, however, DI frameworks made by other people that suppose to work in Unity.
     
  44. adamhill

    adamhill

    Joined:
    May 30, 2008
    Posts:
    20
    There is Zenject and StrangeIOC for DI, both available from the Asset Store. There is also TinyIOC on Github for barebones, really lightweight IOC.

    Those are confirmed to work on desktop and most mobile (no clue about Blackberry) Other frameworks (Ninject) may or may not work on mobile.
     
    cesarpo and Arkade like this.
  45. Alex-Chouls

    Alex-Chouls

    Joined:
    Mar 24, 2009
    Posts:
    2,294
    Can you add unity-version and build-platform to the Environment attributes in the test results xml?

    I'm building a final report that combines all xml files into a single html file, so it's useful to have this info in the xml reports... BTW, it would be cool in the test tools shipped with an example XSLT/Stylesheet for nicely formatted results.

    Locally I've added these to XmlResultsWriter.WriteEnvironment:

    {"unity-version", Application.unityVersion},
    {"build-platform", Application.platform.ToString()},

    For some reason Application.platform always returns "WindowsEditor", even thought the test runs on the correct platform...? Any idea how to report the correct platform?
     
  46. Tomek-Paszek

    Tomek-Paszek

    Unity Technologies

    Joined:
    Nov 13, 2012
    Posts:
    116
    Sure, that's a good idea :)
    I never wrote XSLT, I need to see if I handle that ;)
    That's because the XML file is generated in the editor now. You should probably use the build target as the value or pass the target platform parameter to the result generator.
     
  47. liortal

    liortal

    Joined:
    Oct 17, 2012
    Posts:
    3,201
    Hey,

    quick question (not sure if it was asked before): should the package allow importing only some of its assets?

    It seems that there's some degree of coupling between its different parts.

    For example, in case i want to use only the unit testing capabilities, importing the Unit Testing folder, along with the Common folder did not work (TestResult class is defined inside IntegrationTestFramework). Adding that folder on its own also did not work, since it relies on the Assertion Component class.

    Basically i had to import the whole thing. Is there any plan to re-structure to allow importing only some functionality ?
     
  48. hyperspace_x

    hyperspace_x

    Joined:
    Mar 5, 2014
    Posts:
    2
    Hi,

    I'm using this tool with Unity Free. But it gives me error when I was building iOS project.
    "System.Net.Sockets is not supported"

    I found several files are using System.Net.Sockets in Unity Test Tools. Is there a workaround, or I have to use Unity Pro?
     
  49. Arkade

    Arkade

    Joined:
    Oct 11, 2012
    Posts:
    565
    @hyperspace_x hi. Yes, it will again support free -- it was a mistake that it stopped. The fix is in the src repository. I imagine @Tomek Paszek will push a new build soon to fix. Until then, you probably need to grab from the repo.

    HTH, Rupert.
     
  50. Tomek-Paszek

    Tomek-Paszek

    Unity Technologies

    Joined:
    Nov 13, 2012
    Posts:
    116
    Hi guys,
    @liortal Yes, Ideally it should be decoupled and it was until some point when it got broken :) I should look at it again and fix it.
    @hyperspace_x @Arkade
    Like Rupert said, it's fixed in the repo but I can't find time to update the package in the A$. I'll try to do it as soon as I can.