Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.

InputTestFixture

Discussion in 'Input System' started by jweidenbach, Jan 2, 2019.

  1. jweidenbach

    jweidenbach

    Joined:
    Dec 29, 2018
    Posts:
    54
    Hey there. I'm working on trying to integrate the new input system into my game, and since I tend to use TDD as my development approach my first thoughts for working on my input layer turned to Mocking the Hardware inputs. Thinking I'd look at the approach y'all were using, I went to the code for the InputSystem tests, and found (to my delight) a prebuilt test fixture that looks as though it should allow for this in a simple, definable way. Of course, when I tried to link to Unity.InputSystem.Tests, the asmdef doesn't actually link against it (as it's marked with TestAssemblies), so I can't access InputTestFixture to extend my test class from.

    Are there any plans to expose this in the future for developers consuming this API? Or is there another way we should be using?
     
  2. Rene-Damm

    Rene-Damm

    Unity Technologies

    Joined:
    Sep 15, 2012
    Posts:
    1,779
    ATM the fixture is only available from test assemblies. If you put your test code in its own assembly, mark it as test assembly, and reference Unity.InputSystem.Tests, you should be fine. The tests for the demo do that. You can find the asmdef for them here. Overall, the demo tests should be helpful in giving an example of how the input test infrastructure can be used in a game project.
     
  3. jweidenbach

    jweidenbach

    Joined:
    Dec 29, 2018
    Posts:
    54
    hmm, I have a test assembly that is trying to reference it, and it's not being seen at all. Refreshing to know it was intended that we should be able to use it :)

    Here's my asmdef:
    Code (CSharp):
    1. {
    2.     "name": "LSS.DCHL.Input.Tests",
    3.     "references": [
    4.         "LSS.DCHL.Input",
    5.         "Unity.InputSystem",
    6.         "Unity.InputSystem.Tests"
    7.     ],
    8.     "optionalUnityReferences": [
    9.         "TestAssemblies"
    10.     ],
    11.     "includePlatforms": [],
    12.     "excludePlatforms": [],
    13.     "allowUnsafeCode": false,
    14.     "overrideReferences": false,
    15.     "precompiledReferences": [],
    16.     "autoReferenced": true,
    17.     "defineConstraints": []
    18. }
    19.  
    It appears I've done as you said. I've also checked my assemblies, and they're all building as expected, but the .csproj for this assembly only references my Input module and Unity.InputSystem, not Unity.InputSystem.Tests. I could manually add the reference to the csproj for now, but of course that would get overwritten so it's not a manageable workflow.

    I did notice in my ScriptAssemblies folder that there is no Unity.InputSystem.Tests assembly. Could this be because I'm using the InputSystem preview 0.1.2 from the package manager? Would I be better off cloning the repo and putting it locally in my assets?
     
  4. jweidenbach

    jweidenbach

    Joined:
    Dec 29, 2018
    Posts:
    54
    Ok, moving the InputSystem out of the package manager and into my project directly seems to have fixed it. The Fixture is now available.

    As to development workflow, if the packages don't build Test modules and developers want to use them, is there a recommended process? Git submodules for the stable branch or some such thing?
     
  5. jweidenbach

    jweidenbach

    Joined:
    Dec 29, 2018
    Posts:
    54
    hmmm, one part of this, my cloud build runs the tests as part of the pass/fail specs, does anyone know if there's a way to specify (to account for the TODO tests in this package) to skip tests so I'm not having to turn off the CI testing?
     
  6. jweidenbach

    jweidenbach

    Joined:
    Dec 29, 2018
    Posts:
    54
    just trying to avoid modifying the package for my workflow. If I have to, then I have to :)
     
  7. Rene-Damm

    Rene-Damm

    Unity Technologies

    Joined:
    Sep 15, 2012
    Posts:
    1,779
    Ok, there is indeed something fishy here. There's been some recentish changes around how the Unity package manager handles tests contained in packages and I think that's messing things up.

    Trying to repro, I can see the Unity.InputSystem.Tests available as an assembly reference but the assembly isn't really compiled so I guess it shouldn't be in the list in the first place. Adding it as a reference does not cause an error but also doesn't make the DLL contents (and thus InputTestFixture) available (unsurprisingly as the DLL doesn't exist).

    I'll go and talk to the package management guys and see what the right steps here are. My guess is we'll have to move the tests into a separate package. I'll get back to you as soon as I know more.

    ////EDIT: Opened an issue here.
     
  8. jweidenbach

    jweidenbach

    Joined:
    Dec 29, 2018
    Posts:
    54
    Yeah, what you're seeing sounds precisely like what I'm seeing. Unity.InputSystem.Tests shows up in the inspector building the ASMDEF, but doesn't show up built in the ScriptAssemblies folder alongside Unity.InputSystem, there's no reference in the csproj, and no definitions can be found in Visual Studio or Rider for the InputTestFixture.

    When I bring in the project directly, everything works. So, what you say about splitting out the intended public test interfaces makes sense as a workaround, but I'd expect in packages that you could have systems that are exposed to developers but that don't get linked in to a player build. Always the fun part of exposing a development API, multiple levels everywhere. That's obviously a much more difficult system to implement though, so I understand if it gets pushed back on.
     
  9. jweidenbach

    jweidenbach

    Joined:
    Dec 29, 2018
    Posts:
    54
    Also as a note, I'm on Windows 10, GTX 1070, in case it's platform specific.

    Another note, I went ahead and moved the repo into my project as a local folder, and everything seems good so far. I commented out all of the TODO tests for now. Additionally, in order to make the whole thing work on cloud build, I had to disable the following tests:
    • PickerWillNotThrowError_WhenEscIsPressed
    • InputSystemSourceCodeCompilesWithoutErrors
    • InputSystemSourceCodeCompilesWithoutWarnings
    • FilteringByName
    • FilteringByGroup
    • FilteringByGroupAndDevice
    • FilteringByGroupDeviceAndName
    • NewBindingWithGroupFilter
    • NewBindingIsSelected
    • NewCompositeIsSelected
    • NewActionMapIsSelected
    • NewActionIsSelected
    My suspicion on these is that the cloud build doesn't handle Editor tests well. Shouldn't matter at all for fixing this issue, but it's good to note.
     
  10. Rene-Damm

    Rene-Damm

    Unity Technologies

    Joined:
    Sep 15, 2012
    Posts:
    1,779
    So, turns out that in our current packages + test runner setup, there isn't a good solution to this but there is work (targeted for 2019.2) under way that will resolve this properly. There is a workaround, however.

    If you put "com.unity.inputsystem" in "testables" in Packages/manifest.json like so

    Code (CSharp):
    1.     "com.unity.modules.wind": "1.0.0",
    2.     "com.unity.modules.xr": "1.0.0"
    3.   },
    4.   "testables" : [
    5.     "com.unity.inputsystem"
    6.   ]
    7.  
    then you should be able to reference Unity.InputSystem.Tests.dll and have access to the fixture. Downside is you get all the input system tests, too. So that problem is still the same as you've run into above where you had to disable some tests.

    The proper solution for this is targeted to hit 2019.2 but we'll consider splitting off the tests so that the workaround at least doesn't drag the tests into user projects.
     
  11. jweidenbach

    jweidenbach

    Joined:
    Dec 29, 2018
    Posts:
    54
    No worries, that's good news. For now I can stick with what I've got and when there's a new preview release I can start moving towards this.
     
  12. jweidenbach

    jweidenbach

    Joined:
    Dec 29, 2018
    Posts:
    54
    Ok, coming back after some time off on this. I've updated to 0.2.1-preview, seems to be much better with the manifest workaround you specified.

    I'll see how things go when I push it up to the Cloud build, but at this stage there's only 1 test I needed to disable (UITests/MouseActions_CanDriveUI). It seems to pass in standalone, but when all the tests run it still fails. Not sure if it'll be an issue or not for cloud build, as I don't know if the manifest will affect that.
     
  13. Rene-Damm

    Rene-Damm

    Unity Technologies

    Joined:
    Sep 15, 2012
    Posts:
    1,779
    Yup, looking at it :)

    NB: 2019.1 seems to have some new stuff for dealing with test fixtures in packages. We're looking at properly handling this so no hacking the manifest is required.
     
  14. jasonatkaruna

    jasonatkaruna

    Joined:
    Feb 26, 2019
    Posts:
    64
    Is there any update to this? I'm using Unity 2019.3.0b7 and I'm getting "InputTestFixture not found".

    Code (JavaScript):
    1. // RiggedXR.Tests.asmdef
    2. {
    3.     "name": "RiggedXR.Tests",
    4.     "references": [
    5.         "Unity.InputSystem",
    6.         "Unity.InputSystem.TestFramework",
    7.         "UnityEngine.TestRunner",
    8.         "RiggedXR"
    9.     ],
    10.     "includePlatforms": [],
    11.     "excludePlatforms": [],
    12.     "allowUnsafeCode": true,
    13.     "overrideReferences": true,
    14.     "precompiledReferences": [
    15.         "nunit.framework.dll"
    16.     ],
    17.     "autoReferenced": false,
    18.     "defineConstraints": [
    19.         "UNITY_INCLUDE_TESTS"
    20.     ],
    21.     "versionDefines": [],
    22.     "noEngineReferences": false
    23. }
    Code (CSharp):
    1. // InputSystemXRTest.cs
    2. using UnityEngine.InputSystem;
    3. public class InputSystemXRTest : InputTestFixture
    4. {
    5. }
    UPDATE:
    If you're stumbling on this issue the problem still exists in:

    UnityEditor 2019.3.0b7
    Input System 1.0.0-preview.1

    Use the workaround mentioned above.
     
    Last edited: Oct 19, 2019
    shoffing likes this.
  15. Rene-Damm

    Rene-Damm

    Unity Technologies

    Joined:
    Sep 15, 2012
    Posts:
    1,779
    @jasonatkaruna This is expected to work. Surprised it's still an issue. Could you file a ticket with the Unity bug reporter? We'll have a look.
     
    shoffing and jasonatkaruna like this.
  16. shoffing

    shoffing

    Joined:
    Oct 31, 2013
    Posts:
    9
  17. Rene-Damm

    Rene-Damm

    Unity Technologies

    Joined:
    Sep 15, 2012
    Posts:
    1,779
    Understand that the necessity for listing the input system in "testables" isn't great. We're looking at ways to get rid of that.

    Not following this part. The manifest is intended to be put under version control. Without sharing it between users of the project, how are you ensuring that the input system package itself is getting installed?
     
    jasonatkaruna likes this.
  18. YuriyVotintsev

    YuriyVotintsev

    Joined:
    Jun 11, 2013
    Posts:
    86
    Hi, I have exactly same issue in 2019.3.0f1
     
  19. shoffing

    shoffing

    Joined:
    Oct 31, 2013
    Posts:
    9
    Ah sorry, ours was ignored because I had copied the .gitignore from here: https://docs.microsoft.com/en-us/nuget/consume-packages/packages-and-source-control (we're using Nuget).

    I had to manually unignore it

    ```
    !**/[Pp]ackages/manifest.json
    ```
     
  20. KingRecycle

    KingRecycle

    Joined:
    Jul 20, 2013
    Posts:
    25
    I have this issue even after using the "fix" above using Unity 2019.3.0f1 RC

    Edit: Seems reloading the assemblies fixed it.
     
  21. GoldenImper

    GoldenImper

    Joined:
    Nov 4, 2019
    Posts:
    1
    Hi! I'm using Unity 2019.3.0f6 and still having this issue. Fixed with the workaround.
     
  22. danpeczek

    danpeczek

    Joined:
    Dec 14, 2020
    Posts:
    1
    Hello,

    I'm using the Unity 2019.4.16f1 and the issue still exist. The workaround mentioned above worked for me.
     
  23. Shikiko

    Shikiko

    Joined:
    Dec 8, 2019
    Posts:
    2
    Hello,

    Just chiming in . I'm on build 2020.2.5f1 and the issue still persists.
     
  24. Pavo76

    Pavo76

    Joined:
    Nov 23, 2016
    Posts:
    1
    I have the same issue in versions 2020.3.1f1 and 2020.3.22f1.

    Here is the test assembly:

    Code (CSharp):
    1. {
    2.     "name": "Tests",
    3.     "rootNamespace": "",
    4.     "references": [
    5.         "UnityEngine.TestRunner",
    6.         "UnityEditor.TestRunner",
    7.         "GameAssembly",
    8.         "Unity.InputSystem",
    9.         "Unity.InputSystem.TestFramework"
    10.     ],
    11.     "includePlatforms": [],
    12.     "excludePlatforms": [],
    13.     "allowUnsafeCode": false,
    14.     "overrideReferences": true,
    15.     "precompiledReferences": [
    16.         "nunit.framework.dll"
    17.     ],
    18.     "autoReferenced": false,
    19.     "defineConstraints": [
    20.         "UNITY_INCLUDE_TESTS"
    21.     ],
    22.     "versionDefines": [],
    23.     "noEngineReferences": false
    24. }
    manifest.json:

    Code (CSharp):
    1. {
    2.   "dependencies": {
    3.     "com.unity.2d.animation": "5.0.9",
    4.     "com.unity.2d.pixel-perfect": "4.0.1",
    5.     "com.unity.2d.psdimporter": "4.1.2",
    6.     "com.unity.2d.sprite": "1.0.0",
    7.     "com.unity.2d.spriteshape": "5.1.5",
    8.     "com.unity.2d.tilemap": "1.0.0",
    9.     "com.unity.cinemachine": "2.6.10",
    10.     "com.unity.collab-proxy": "1.13.5",
    11.     "com.unity.ide.rider": "2.0.7",
    12.     "com.unity.ide.visualstudio": "2.0.11",
    13.     "com.unity.ide.vscode": "1.2.4",
    14.     "com.unity.inputsystem": "1.0.2",
    15.     "com.unity.test-framework": "1.1.30",
    16.     "com.unity.textmeshpro": "3.0.6",
    17.     "com.unity.timeline": "1.4.8",
    18.     "com.unity.ugui": "1.0.0",
    19.     "com.unity.modules.ai": "1.0.0",
    20.     "com.unity.modules.androidjni": "1.0.0",
    21.     "com.unity.modules.animation": "1.0.0",
    22.     "com.unity.modules.assetbundle": "1.0.0",
    23.     "com.unity.modules.audio": "1.0.0",
    24.     "com.unity.modules.cloth": "1.0.0",
    25.     "com.unity.modules.director": "1.0.0",
    26.     "com.unity.modules.imageconversion": "1.0.0",
    27.     "com.unity.modules.imgui": "1.0.0",
    28.     "com.unity.modules.jsonserialize": "1.0.0",
    29.     "com.unity.modules.particlesystem": "1.0.0",
    30.     "com.unity.modules.physics": "1.0.0",
    31.     "com.unity.modules.physics2d": "1.0.0",
    32.     "com.unity.modules.screencapture": "1.0.0",
    33.     "com.unity.modules.terrain": "1.0.0",
    34.     "com.unity.modules.terrainphysics": "1.0.0",
    35.     "com.unity.modules.tilemap": "1.0.0",
    36.     "com.unity.modules.ui": "1.0.0",
    37.     "com.unity.modules.uielements": "1.0.0",
    38.     "com.unity.modules.umbra": "1.0.0",
    39.     "com.unity.modules.unityanalytics": "1.0.0",
    40.     "com.unity.modules.unitywebrequest": "1.0.0",
    41.     "com.unity.modules.unitywebrequestassetbundle": "1.0.0",
    42.     "com.unity.modules.unitywebrequestaudio": "1.0.0",
    43.     "com.unity.modules.unitywebrequesttexture": "1.0.0",
    44.     "com.unity.modules.unitywebrequestwww": "1.0.0",
    45.     "com.unity.modules.vehicles": "1.0.0",
    46.     "com.unity.modules.video": "1.0.0",
    47.     "com.unity.modules.vr": "1.0.0",
    48.     "com.unity.modules.wind": "1.0.0",
    49.     "com.unity.modules.xr": "1.0.0"
    50.   },
    51.   "testables": [
    52.     "com.unity.inputsystem"
    53.   ]
    54. }
    55.  
    And beginning of test script:
    Code (CSharp):
    1. using System.Collections;
    2. using System.Collections.Generic;
    3. using NUnit.Framework;
    4. using UnityEngine;
    5. using UnityEngine.TestTools;
    6. using UnityEditor;
    7.  
    8. public class GameplayTest : InputTestFixture {
    I get error that the InputTestFixture could not be found, and the above workaround doesn't solve the issue.
    Am I missing something?
     
  25. dougfy

    dougfy

    Joined:
    Nov 9, 2015
    Posts:
    3
    I am stuck at the same point as above. Currently on 2020.1.1f1
     
  26. dougfy

    dougfy

    Joined:
    Nov 9, 2015
    Posts:
    3
  27. ausmle

    ausmle

    Joined:
    Jun 17, 2022
    Posts:
    1
    The steps in the documentation which include the manifest.json work around are not working for me. I am not able to use the InputTestFixture class in my tests

    I am using Unity Version 2020.3.32f1 and input system version 1.3.0.

    I have referenced the Unity.InputSystem and Unity.InputSystem.TestFramework assemblies in my test assembly yet I still get the message that "The type or namespace 'InputTestFixture' cannot be found"

    Code (CSharp):
    1. using System.Collections;
    2. using System.Collections.Generic;
    3. using NUnit.Framework;
    4. using UnityEngine;
    5. using UnityEngine.EventSystems;
    6. using UnityEngine.TestTools;
    7. using UnityEngine.InputSystem;
    8.  
    9. [TestFixture]
    10. public class GameMenuTest
    11. {
    12.     private InputTestFixture input = new InputTestFixture();
    13.  
    14.     [OneTimeSetUp]
    15.     public void LoadScene()
    16.     {
    17.         GameSceneManager.Load(GameSceneManager.Scene.GameLevelScene);
    18.     }
    19. }

    EDIT: Nevermind, it started working. I happened to randomly add other DLLs to the test assembly references, maybe that caused it to start working. Or maybe there was an error somewhere and after running it was then able to work
     
    Last edited: Jun 17, 2022
  28. epratt

    epratt

    Joined:
    Oct 15, 2019
    Posts:
    14
    What got it working for me was after adding the "testables" entry in Packages/manifest.json I closed my editor and Visual Studio, deleted all the csharp files in my project (just in case. You may not have to do that), Opened my Unity project again and generated project files again and opened up my test script again. It was able to see InputTestFixture after that.