Search Unity

Help Wanted ArgumentException: Type provided to TestRunCallbackAttribute does not implement ITestRunCallback

Discussion in 'Testing & Automation' started by Buretto, Mar 24, 2020.

  1. Buretto

    Buretto

    Joined:
    Mar 23, 2015
    Posts:
    42
    I am encountering this exception while trying to run performance tests on Android.


    Here is my test script: https://gist.github.com/bureto/1c284fa7cdbe20c1350c722e4aa1f121

    Things to note:
    • The test runs fine in editor play mode.
    • I am running the test on Android via the "Run all in player " button located in the Test Runner editor window.
    • After the test is built on the device, and after the splash screen, the screen goes black when it encounters this exception and nothing is sent back (i.e. the test isn't run).
    I'm still unsure how this all works together; looking for help trying to get it to run on Android.
    Do I have to create a class that implements ITestRunCallback for it to work on Android? (I tried this but still encounter the exception, although maybe I didn't implement it properly?)

    Also, unrelated but would like to know if there is a solution: Is it possible to run the tests on a device without building every time? I.e. Am I able to just run the built app on the phone or w/e and it will perform the tests and give feedback? This would be much more convenient than having to build every time I press the "Run all in player" button.
     
  2. Buretto

    Buretto

    Joined:
    Mar 23, 2015
    Posts:
    42
    So adding a link.xml to prevent stripping resolved this first error.. but now I am running into another problem:

    StaticTextureTest (23.918s)
    ---
    AfterTest : System.NullReferenceException : Object reference not set to an instance of an object.
    ---
    --AfterTest
    at Unity.PerformanceTesting.PerformanceTest.EndTest (NUnit.Framework.Interfaces.ITest test) [0x00000] in <00000000000000000000000000000000>:0
    at Unity.PerformanceTesting.PerformanceAttribute+<AfterTest>d__2.MoveNext () [0x00000] in <00000000000000000000000000000000>:0
    at UnityEngine.TestTools.BeforeAfterTestCommandBase`1+<ExecuteEnumerable>d__9[T].MoveNext () [0x00000] in <00000000000000000000000000000000>:0
    at UnityEngine.SetupCoroutine.InvokeMoveNext (System.Collections.IEnumerator enumerator, System.IntPtr returnValueAddress) [0x00000] in <00000000000000000000000000000000>:0

    Any idea's for the cause of this error is greatly appreciated.
     
  3. LauraBriedyte

    LauraBriedyte

    Unity Technologies

    Joined:
    Jul 24, 2018
    Posts:
    2
    Hello, it would be great if you could submit a bug from Unity Editor with all the information on how to reproduce this issue, along with attaching the project you used to reproduce it. Write in the description that it should be assigned to laurab. I will investigate it further and get back to you.
     
  4. floky

    floky

    Joined:
    Oct 6, 2010
    Posts:
    260
    I can confirm I'm getting the same error when trying to run all tests on a Standalone player. Trying now to create a separate empty Unity project to reproduce the problem.
     
  5. floky

    floky

    Joined:
    Oct 6, 2010
    Posts:
    260
    I've managed to pinpoint what's causing this issue in a separate repro project.
    Apparently this error is being thrown, at least in my case, when marking a test method with the [Performance] attribute when using the "Performance Testing API".

    Using:
    - Unity 2019.3.11f1
    - Performance Testing API preview package v2.0.9
    - TestFramework package v1.1.14

    upload_2020-5-3_14-25-13.png
     
  6. floky

    floky

    Joined:
    Oct 6, 2010
    Posts:
    260
  7. cbauman

    cbauman

    Joined:
    Aug 29, 2011
    Posts:
    17
    Last edited: May 10, 2020
    floky likes this.
  8. floky

    floky

    Joined:
    Oct 6, 2010
    Posts:
    260
    Oh crap, so the issue is actually because some code from "Unity.PerformanceTesting" was being stripped out by the compiler.
    Good find!
    Didn't see that in their getting started docs or I must have missed it.
    Thanks!
     
unityunity