New BDD Utility for Unity

I've recently been embracing a more BDD-like style in high level integration and system tests, so I got inspired to write some utilities for this:

• The first is new methods in (my test framework) Responsible, which simply enforce using BDD-style keywords and giving clear descriptions for the top level of tests.
• The second is a Gerkin to Responsible code generator. Note: this is not designed to support editing the Gherkin files after the code has been generated in any way, only to kick-start your tests. Also, tags and Scenario Templates are not supported (yet).

If you aren't already familiar with Responsible, it's a test framework (mainly) for asynchronous code, focusing on clear messages on failures, and safe composability of test operations. I admit there's a bit of a learning curve, but if you're fluent with a more functional programming style in C# (think Linq or Rx), it shouldn't be too hard to pick up.

Disclaimer: While Responsible itself is highly stable and being actively used, I still consider the new BDD keywords and the Gherkin tool experimental, and thus not under semantic versioning.

Disclaimers out of the way, let's get to an example from the package sample:

Using the following feature file:

Code (CSharp):

1. # responsible-namespace: ResponsibleGame.PlayModeTests
2. # responsible-base-class: SystemTest
3.  
4. Feature: Restarting the game
5.   It should be easy to restart the game under all conditions,
6.   so that it doesn't become a common churning point.
7.  
8.  Scenario: Restarting after failure should be easy
9.    Given the player has failed
10.    When the player presses the trigger key
11.    Then the game should be restarted

I can use the dotnet tool

responsible-gherkin

to convert the feature file into the following stub. Note that this step is optional, and you could also write the stub tests manually to being with!

Code (CSharp):

1. using Responsible;
2. using System.Collections;
3. using UnityEngine.TestTools;
4. using static Responsible.Bdd.Keywords;
5.  
6. namespace ResponsibleGame.PlaymodeTests
7. {
8.     // It should be easy to restart the game under all conditions,
9.     // so that it doesn't become a common churning point.
10.     public class RestartingTheGame : SystemTest
11.     {
12.         [UnityTest]
13.         public IEnumerator Scenario_RestartingAfterFailureShouldBeEasy() => this.Executor.YieldScenario(
14.             Scenario("Restarting after failure should be easy"),
15.             Given("the player has failed", Pending),
16.             When("the player presses the trigger key", Pending),
17.             Then("the game should be restarted", Pending));
18.     }
19. }

The

Pending

keywords will make the tests bail out early, but not fail. This means you can write multiple scenario and feature specifications, and then implement them piece by piece as you develop the features.

Once I have some helper methods available in my system test base class set up, I can easily replace all the pending steps with real implementations:

Code (CSharp):

1. using Responsible;
2. using System.Collections;
3. using System.Linq;
4. using NUnit.Framework;
5. using UnityEngine.TestTools;
6. using static Responsible.Bdd.Keywords;
7. using static Responsible.Responsibly;
8.  
9. namespace ResponsibleGame.PlayModeTests
10. {
11.     // It should be easy to restart the game under all conditions,
12.     // so that it doesn't become a common churning point.
13.     public class RestartingTheGame : SystemTest
14.     {
15.         [UnityTest]
16.         public IEnumerator Scenario_RestartingAfterFailureShouldBeEasy() => this.Executor.YieldScenario(
17.             Scenario("Restarting after failure should be easy"),
18.             Given("the player has failed", this.FailTheGame()),
19.             When("the player presses the trigger key", this.SimulateTriggerInput()),
20.             Then("the game should be restarted", AssertTheGameHasBeenRestarted()));
21.  
22.         private ITestInstruction<object> FailTheGame()
23.         {
24.             var miss = this
25.                 .TriggerHit(false)
26.                 .ExpectWithinSeconds(2)
27.                 .ContinueWith(WaitForFrames(1));
28.  
29.             return Enumerable.Repeat(miss, Status.StartingLives).Sequence();
30.         }
31.  
32.         private static ITestInstruction<object> AssertTheGameHasBeenRestarted() => Do(
33.             "Assert the game has been restarted",
34.             () => Assert.IsTrue(ExpectStatusInstance().IsAlive));
35.     }
36. }

And for those not already familiar with Responsible, this is the output we get if we simulate a failure on the second miss (I also replaced some absolute path components with ...).

Code (CSharp):

1. Failure context:
2. [!] Scenario: Restarting after failure should be easy (Failed after 0.04 s ≈ 2 frames)
3.   [!] Given the player has failed (Failed after 0.04 s ≈ 2 frames)
4.     [✓] Trigger miss CONDITION EXPECTED WITHIN 2.00 s (Completed in 0.02 s ≈ 0 frames)
5.     [✓] WAIT FOR 1 FRAME(S) (Completed in 0.01 s ≈ 2 frames)
6.     [!] Trigger miss CONDITION EXPECTED WITHIN 2.00 s (Failed after 0.00 s ≈ 0 frames)
7.       WAIT FOR
8.         [✓] Wait for main components (Completed in 0.00 s ≈ 0 frames)
9.         [✓] Player object is within target area: False (Completed in 0.00 s ≈ 0 frames)
10.       THEN RESPOND WITH
11.         [!] Simulate trigger input (Failed after 0.00 s ≈ 0 frames)
12.           Failed with:
13.             System.Exception: 'Fake exception'
14.           Test operation stack:
15.             [Do] SimulateTriggerInput (at Assets/Samples/Responsible/X.Y.Z/ResponsibleGame/PlayModeTests/SystemTest.cs:74)
16.             [ExpectWithinSeconds] FailTheGame (at Assets/Samples/Responsible/X.Y.Z/ResponsibleGame/PlayModeTests/RestartingTheGame.cs:26)
17.             [ContinueWith] FailTheGame (at Assets/Samples/Responsible/X.Y.Z/ResponsibleGame/PlayModeTests/RestartingTheGame.cs:27)
18.             [Sequence] FailTheGame (at Assets/Samples/Responsible/X.Y.Z/ResponsibleGame/PlayModeTests/RestartingTheGame.cs:29)
19.             [Scenario] Scenario_RestartingAfterFailureShouldBeEasy (at Assets/Samples/Responsible/X.Y.Z/ResponsibleGame/PlayModeTests/RestartingTheGame.cs:17)
20.             [ToYieldInstruction] YieldScenario (at .../Responsible/com.beatwaves.responsible/Runtime/BddExtensions.cs:37)
21.     [ ] WAIT FOR 1 FRAME(S)
22.     [ ] Trigger miss CONDITION EXPECTED WITHIN 2.00 s
23.     [ ] WAIT FOR 1 FRAME(S)
24.   Failed with:
25.     System.Exception: 'Fake exception'
26.   Test operation stack:
27.     [Scenario] Scenario_RestartingAfterFailureShouldBeEasy (at Assets/Samples/Responsible/X.Y.Z/ResponsibleGame/PlayModeTests/RestartingTheGame.cs:17)
28.     [ToYieldInstruction] YieldScenario (at .../Responsible/com.beatwaves.responsible/Runtime/BddExtensions.cs:37)
29.   [ ] When the player presses the trigger key
30.     [ ] Simulate trigger input
31.   [ ] Then the game should be restarted
32.     [ ] Assert the game has been restarted
33. Failed with:
34.   System.Exception: 'Fake exception'
35. Test operation stack:
36.   [ToYieldInstruction] YieldScenario (at .../Responsible/com.beatwaves.responsible/Runtime/BddExtensions.cs:37)
37. Error: System.Exception: Fake exception
38.   at ResponsibleGame.PlayModeTests.SystemTest.<SimulateTriggerInput>b__12_0 () [0x0001a] in .../Responsible/ResponsibleUnity/Assets/Samples/Responsible/X.Y.Z/ResponsibleGame/PlayModeTests/SystemTest.cs:78
39.   at ...

• You can find more information about Responsible at the documentation site: https://www.beatwaves.net/Responsible/
• The

  responsible-gherkin

  command line tool can be installed with

  dotnet tool install --global Beatwaves.ResponsibleGherkin

If you have any questions, comments, or feedback, I'd be happy to hear them!