Search Unity

We need input automation

Discussion in 'Testing & Automation' started by laurentlavigne, Jul 26, 2020.

  1. laurentlavigne

    laurentlavigne

    Joined:
    Aug 16, 2012
    Posts:
    6,320
    And input recorder
    To auto play games for days
    Plug and play, no code necessary
     
    laurentlavigne, Jul 26, 2020
    #1

  2. superpig

    superpig

    Drink more water! Unity Technologies

    Joined:
    Jan 16, 2011
    Posts:
    4,653
    Input recording/replay tests are very fragile, which limits their usefulness quite significantly - if you record a play through of your level, then every time a designer moves a platform or tweaks an enemy position, you have to redo the recording. Each recording is also only testing one specific path through the game, which is usually pretty poor state space coverage for the time investment.

    So, can you explain more about your use case? What problem do you think input recording and playback would solve for you?
     
    superpig, Jul 26, 2020
    #2

  3. laurentlavigne

    laurentlavigne

    Joined:
    Aug 16, 2012
    Posts:
    6,320
    These tests happen after level designs are locked so this dimension of fragility does not exist.
    Behavior is deterministic enough that play sessions should be similar in duration.
    If it's too nondeterministic I add 
    [Test]
    or 
    [TestTrigger("cheat")
    in front of the class that turns on infinite money and invulnerability. Due to the nature of the game the enemies will be in the same areas so if I record the play sessions without moving the camera I will get relevant framerate information and since it's wave based memory leaks will show up if they exist.
    So the purpose of this testing is check memory leaks, major game logic regression and framerate regressions.
     
    laurentlavigne, Jul 26, 2020
    #3

  4. laurentlavigne

    laurentlavigne

    Joined:
    Aug 16, 2012
    Posts:
    6,320
    oh and it's to test on consoles.
     
    laurentlavigne, Jul 26, 2020
    #4

  5. superpig

    superpig

    Drink more water! Unity Technologies

    Joined:
    Jan 16, 2011
    Posts:
    4,653
    OK, I see.

    Are you using the new input system?
     
    superpig, Jul 26, 2020
    #5

  6. laurentlavigne

    laurentlavigne

    Joined:
    Aug 16, 2012
    Posts:
    6,320
    Not at the moment but input is decoupled so I could use it if such record/playback mechanism is planned or available on it.
     
    laurentlavigne, Jul 26, 2020
    #6

  7. superpig

    superpig

    Drink more water! Unity Technologies

    Joined:
    Jan 16, 2011
    Posts:
    4,653
    So the new input system actually comes with a sample package that demonstrates input recording and playback:
    upload_2020-7-26_20-41-59.png

    There's a bit of documentation about how the technique works. So I think this would be the best place to start - it's not "no code necessary" because you'll still need to figure out how you want to integrate it into your project and workflow, but almost all the heavy lifting is done for you already.

    I think it should also be totally possible to use the InputEventTrace/ReplayController types from inside a [UnityTest], if you want to do that - you can start playback and then just yield in a loop until the .finished property returns true.
     
    superpig, Jul 27, 2020
    #7
    laurentlavigne likes this.

  8. laurentlavigne

    laurentlavigne

    Joined:
    Aug 16, 2012
    Posts:
    6,320
    Any ETA for when the new input system is in 2018.LTS?
     
    laurentlavigne, Jul 27, 2020
    #8

  9. superpig

    superpig

    Drink more water! Unity Technologies

    Joined:
    Jan 16, 2011
    Posts:
    4,653
    It's not coming to 2018 LTS - it depends on core engine changes which were made in the 2019 codeline, and back-porting them is too risky for an LTS branch.
     
    superpig, Jul 27, 2020
    #9
    laurentlavigne likes this.

  10. laurentlavigne

    laurentlavigne

    Joined:
    Aug 16, 2012
    Posts:
    6,320
    Thanks for the info.
    I'll see what the impact of witching to 2019 is on the framerate, as you know 2019 regressed in performance by quite a wide margin so if that's too much for good old switch I'll roll in my own input recorded.
     
    laurentlavigne, Jul 27, 2020
    #10