InputTestFixture

Hey there. I'm working on trying to integrate the new input system into my game, and since I tend to use TDD as my development approach my first thoughts for working on my input layer turned to Mocking the Hardware inputs. Thinking I'd look at the approach y'all were using, I went to the code for the InputSystem tests, and found (to my delight) a prebuilt test fixture that looks as though it should allow for this in a simple, definable way. Of course, when I tried to link to Unity.InputSystem.Tests, the asmdef doesn't actually link against it (as it's marked with TestAssemblies), so I can't access InputTestFixture to extend my test class from.

Are there any plans to expose this in the future for developers consuming this API? Or is there another way we should be using?

 

ATM the fixture is only available from test assemblies. If you put your test code in its own assembly, mark it as test assembly, and reference Unity.InputSystem.Tests, you should be fine. The tests for the demo do that. You can find the asmdef for them here. Overall, the demo tests should be helpful in giving an example of how the input test infrastructure can be used in a game project.

 

hmm, I have a test assembly that is trying to reference it, and it's not being seen at all. Refreshing to know it was intended that we should be able to use it

Here's my asmdef:

Code (CSharp):

1. {
2.     "name": "LSS.DCHL.Input.Tests",
3.     "references": [
4.         "LSS.DCHL.Input",
5.         "Unity.InputSystem",
6.         "Unity.InputSystem.Tests"
7.     ],
8.     "optionalUnityReferences": [
9.         "TestAssemblies"
10.     ],
11.     "includePlatforms": [],
12.     "excludePlatforms": [],
13.     "allowUnsafeCode": false,
14.     "overrideReferences": false,
15.     "precompiledReferences": [],
16.     "autoReferenced": true,
17.     "defineConstraints": []
18. }
19.  

It appears I've done as you said. I've also checked my assemblies, and they're all building as expected, but the .csproj for this assembly only references my Input module and Unity.InputSystem, not Unity.InputSystem.Tests. I could manually add the reference to the csproj for now, but of course that would get overwritten so it's not a manageable workflow.

I did notice in my ScriptAssemblies folder that there is no Unity.InputSystem.Tests assembly. Could this be because I'm using the InputSystem preview 0.1.2 from the package manager? Would I be better off cloning the repo and putting it locally in my assets?

 

Ok, moving the InputSystem out of the package manager and into my project directly seems to have fixed it. The Fixture is now available.

As to development workflow, if the packages don't build Test modules and developers want to use them, is there a recommended process? Git submodules for the stable branch or some such thing?

 

hmmm, one part of this, my cloud build runs the tests as part of the pass/fail specs, does anyone know if there's a way to specify (to account for the TODO tests in this package) to skip tests so I'm not having to turn off the CI testing?

 

just trying to avoid modifying the package for my workflow. If I have to, then I have to

 

Ok, there is indeed something fishy here. There's been some recentish changes around how the Unity package manager handles tests contained in packages and I think that's messing things up.

Trying to repro, I can see the Unity.InputSystem.Tests available as an assembly reference but the assembly isn't really compiled so I guess it shouldn't be in the list in the first place. Adding it as a reference does not cause an error but also doesn't make the DLL contents (and thus InputTestFixture) available (unsurprisingly as the DLL doesn't exist).

I'll go and talk to the package management guys and see what the right steps here are. My guess is we'll have to move the tests into a separate package. I'll get back to you as soon as I know more.

////EDIT: Opened an issue here.

 

Yeah, what you're seeing sounds precisely like what I'm seeing. Unity.InputSystem.Tests shows up in the inspector building the ASMDEF, but doesn't show up built in the ScriptAssemblies folder alongside Unity.InputSystem, there's no reference in the csproj, and no definitions can be found in Visual Studio or Rider for the InputTestFixture.

When I bring in the project directly, everything works. So, what you say about splitting out the intended public test interfaces makes sense as a workaround, but I'd expect in packages that you could have systems that are exposed to developers but that don't get linked in to a player build. Always the fun part of exposing a development API, multiple levels everywhere. That's obviously a much more difficult system to implement though, so I understand if it gets pushed back on.

 

Also as a note, I'm on Windows 10, GTX 1070, in case it's platform specific.

Another note, I went ahead and moved the repo into my project as a local folder, and everything seems good so far. I commented out all of the TODO tests for now. Additionally, in order to make the whole thing work on cloud build, I had to disable the following tests:

• PickerWillNotThrowError_WhenEscIsPressed
• InputSystemSourceCodeCompilesWithoutErrors
• InputSystemSourceCodeCompilesWithoutWarnings
• FilteringByName
• FilteringByGroup
• FilteringByGroupAndDevice
• FilteringByGroupDeviceAndName
• NewBindingWithGroupFilter
• NewBindingIsSelected
• NewCompositeIsSelected
• NewActionMapIsSelected
• NewActionIsSelected

My suspicion on these is that the cloud build doesn't handle Editor tests well. Shouldn't matter at all for fixing this issue, but it's good to note.

 

So, turns out that in our current packages + test runner setup, there isn't a good solution to this but there is work (targeted for 2019.2) under way that will resolve this properly. There is a workaround, however.

If you put "com.unity.inputsystem" in "testables" in Packages/manifest.json like so

Code (CSharp):

1.     "com.unity.modules.wind": "1.0.0",
2.     "com.unity.modules.xr": "1.0.0"
3.   },
4.   "testables" : [
5.     "com.unity.inputsystem"
6.   ]
7.  

then you should be able to reference Unity.InputSystem.Tests.dll and have access to the fixture. Downside is you get all the input system tests, too. So that problem is still the same as you've run into above where you had to disable some tests.

The proper solution for this is targeted to hit 2019.2 but we'll consider splitting off the tests so that the workaround at least doesn't drag the tests into user projects.

 

No worries, that's good news. For now I can stick with what I've got and when there's a new preview release I can start moving towards this.

 

Ok, coming back after some time off on this. I've updated to 0.2.1-preview, seems to be much better with the manifest workaround you specified.

I'll see how things go when I push it up to the Cloud build, but at this stage there's only 1 test I needed to disable (UITests/MouseActions_CanDriveUI). It seems to pass in standalone, but when all the tests run it still fails. Not sure if it'll be an issue or not for cloud build, as I don't know if the manifest will affect that.

 

Yup, looking at it

NB: 2019.1 seems to have some new stuff for dealing with test fixtures in packages. We're looking at properly handling this so no hacking the manifest is required.

 

Is there any update to this? I'm using Unity 2019.3.0b7 and I'm getting "InputTestFixture not found".

Code (JavaScript):

1. // RiggedXR.Tests.asmdef
2. {
3.     "name": "RiggedXR.Tests",
4.     "references": [
5.         "Unity.InputSystem",
6.         "Unity.InputSystem.TestFramework",
7.         "UnityEngine.TestRunner",
8.         "RiggedXR"
9.     ],
10.     "includePlatforms": [],
11.     "excludePlatforms": [],
12.     "allowUnsafeCode": true,
13.     "overrideReferences": true,
14.     "precompiledReferences": [
15.         "nunit.framework.dll"
16.     ],
17.     "autoReferenced": false,
18.     "defineConstraints": [
19.         "UNITY_INCLUDE_TESTS"
20.     ],
21.     "versionDefines": [],
22.     "noEngineReferences": false
23. }

Code (CSharp):

1. // InputSystemXRTest.cs
2. using UnityEngine.InputSystem;
3. public class InputSystemXRTest : InputTestFixture
4. {
5. }

UPDATE:
If you're stumbling on this issue the problem still exists in:

UnityEditor 2019.3.0b7
Input System 1.0.0-preview.1

Use the workaround mentioned above.

 

@jasonatkaruna This is expected to work. Surprised it's still an issue. Could you file a ticket with the Unity bug reporter? We'll have a look.

 

@Rene-Damm

Can confirm this is still an issue with 2019.2.11f1. Had to use the above mentioned fix (https://forum.unity.com/threads/inputtestfixture.606226/#post-4076920).

Which really stinks, because now we have to include `Packages/manifest.json` in our version control so everybody can run the tests.

 

Understand that the necessity for listing the input system in "testables" isn't great. We're looking at ways to get rid of that.

Not following this part. The manifest is intended to be put under version control. Without sharing it between users of the project, how are you ensuring that the input system package itself is getting installed?

 

Hi, I have exactly same issue in 2019.3.0f1

 

Ah sorry, ours was ignored because I had copied the .gitignore from here: https://docs.microsoft.com/en-us/nuget/consume-packages/packages-and-source-control (we're using Nuget).

I had to manually unignore it

```
!**/[Pp]ackages/manifest.json
```

 

I have this issue even after using the "fix" above using Unity 2019.3.0f1 RC

Edit: Seems reloading the assemblies fixed it.

 

Hi! I'm using Unity 2019.3.0f6 and still having this issue. Fixed with the workaround.

 

Hello,

I'm using the Unity 2019.4.16f1 and the issue still exist. The workaround mentioned above worked for me.

 

Hello,

Just chiming in . I'm on build 2020.2.5f1 and the issue still persists.

 

I have the same issue in versions 2020.3.1f1 and 2020.3.22f1.

Here is the test assembly:

Code (CSharp):

1. {
2.     "name": "Tests",
3.     "rootNamespace": "",
4.     "references": [
5.         "UnityEngine.TestRunner",
6.         "UnityEditor.TestRunner",
7.         "GameAssembly",
8.         "Unity.InputSystem",
9.         "Unity.InputSystem.TestFramework"
10.     ],
11.     "includePlatforms": [],
12.     "excludePlatforms": [],
13.     "allowUnsafeCode": false,
14.     "overrideReferences": true,
15.     "precompiledReferences": [
16.         "nunit.framework.dll"
17.     ],
18.     "autoReferenced": false,
19.     "defineConstraints": [
20.         "UNITY_INCLUDE_TESTS"
21.     ],
22.     "versionDefines": [],
23.     "noEngineReferences": false
24. }

manifest.json:

Code (CSharp):

1. {
2.   "dependencies": {
3.     "com.unity.2d.animation": "5.0.9",
4.     "com.unity.2d.pixel-perfect": "4.0.1",
5.     "com.unity.2d.psdimporter": "4.1.2",
6.     "com.unity.2d.sprite": "1.0.0",
7.     "com.unity.2d.spriteshape": "5.1.5",
8.     "com.unity.2d.tilemap": "1.0.0",
9.     "com.unity.cinemachine": "2.6.10",
10.     "com.unity.collab-proxy": "1.13.5",
11.     "com.unity.ide.rider": "2.0.7",
12.     "com.unity.ide.visualstudio": "2.0.11",
13.     "com.unity.ide.vscode": "1.2.4",
14.     "com.unity.inputsystem": "1.0.2",
15.     "com.unity.test-framework": "1.1.30",
16.     "com.unity.textmeshpro": "3.0.6",
17.     "com.unity.timeline": "1.4.8",
18.     "com.unity.ugui": "1.0.0",
19.     "com.unity.modules.ai": "1.0.0",
20.     "com.unity.modules.androidjni": "1.0.0",
21.     "com.unity.modules.animation": "1.0.0",
22.     "com.unity.modules.assetbundle": "1.0.0",
23.     "com.unity.modules.audio": "1.0.0",
24.     "com.unity.modules.cloth": "1.0.0",
25.     "com.unity.modules.director": "1.0.0",
26.     "com.unity.modules.imageconversion": "1.0.0",
27.     "com.unity.modules.imgui": "1.0.0",
28.     "com.unity.modules.jsonserialize": "1.0.0",
29.     "com.unity.modules.particlesystem": "1.0.0",
30.     "com.unity.modules.physics": "1.0.0",
31.     "com.unity.modules.physics2d": "1.0.0",
32.     "com.unity.modules.screencapture": "1.0.0",
33.     "com.unity.modules.terrain": "1.0.0",
34.     "com.unity.modules.terrainphysics": "1.0.0",
35.     "com.unity.modules.tilemap": "1.0.0",
36.     "com.unity.modules.ui": "1.0.0",
37.     "com.unity.modules.uielements": "1.0.0",
38.     "com.unity.modules.umbra": "1.0.0",
39.     "com.unity.modules.unityanalytics": "1.0.0",
40.     "com.unity.modules.unitywebrequest": "1.0.0",
41.     "com.unity.modules.unitywebrequestassetbundle": "1.0.0",
42.     "com.unity.modules.unitywebrequestaudio": "1.0.0",
43.     "com.unity.modules.unitywebrequesttexture": "1.0.0",
44.     "com.unity.modules.unitywebrequestwww": "1.0.0",
45.     "com.unity.modules.vehicles": "1.0.0",
46.     "com.unity.modules.video": "1.0.0",
47.     "com.unity.modules.vr": "1.0.0",
48.     "com.unity.modules.wind": "1.0.0",
49.     "com.unity.modules.xr": "1.0.0"
50.   },
51.   "testables": [
52.     "com.unity.inputsystem"
53.   ]
54. }
55.  

And beginning of test script:

Code (CSharp):

1. using System.Collections;
2. using System.Collections.Generic;
3. using NUnit.Framework;
4. using UnityEngine;
5. using UnityEngine.TestTools;
6. using UnityEditor;
7.  
8. public class GameplayTest : InputTestFixture {

I get error that the InputTestFixture could not be found, and the above workaround doesn't solve the issue.
Am I missing something?

 

I am stuck at the same point as above. Currently on 2020.1.1f1

 

For those that end up here. What is needed was to install the InputSystem from the package manager. Description can be found here: https://docs.unity3d.com/Packages/com.unity.inputsystem@1.2/manual/Installation.html

 

The steps in the documentation which include the manifest.json work around are not working for me. I am not able to use the InputTestFixture class in my tests

I am using Unity Version 2020.3.32f1 and input system version 1.3.0.

I have referenced the Unity.InputSystem and Unity.InputSystem.TestFramework assemblies in my test assembly yet I still get the message that "The type or namespace 'InputTestFixture' cannot be found"

Code (CSharp):

1. using System.Collections;
2. using System.Collections.Generic;
3. using NUnit.Framework;
4. using UnityEngine;
5. using UnityEngine.EventSystems;
6. using UnityEngine.TestTools;
7. using UnityEngine.InputSystem;
8.  
9. [TestFixture]
10. public class GameMenuTest
11. {
12.     private InputTestFixture input = new InputTestFixture();
13.  
14.     [OneTimeSetUp]
15.     public void LoadScene()
16.     {
17.         GameSceneManager.Load(GameSceneManager.Scene.GameLevelScene);
18.     }
19. }

EDIT: Nevermind, it started working. I happened to randomly add other DLLs to the test assembly references, maybe that caused it to start working. Or maybe there was an error somewhere and after running it was then able to work

 

What got it working for me was after adding the "testables" entry in Packages/manifest.json I closed my editor and Visual Studio, deleted all the csharp files in my project (just in case. You may not have to do that), Opened my Unity project again and generated project files again and opened up my test script again. It was able to see InputTestFixture after that.