input recording/playback

I've been watching old videos on Unity's YouTube channel and there's a video from Unite 2012 where reps from The Creative Assembly give a postmortem on the development of Shogun: Total War Battles.

In the video, one of the developers gives a very impressive demo of a custom input recording and playback system they developed to help reproduce bugs in their game.

It looks invaluable.

I've searched the forums here and there are a few old threads where an input recorder is asked about but there does not seem to be a definitive answer on how to implement it or if one is available for purchase.

I haven't played Shogun: Total War Battles but, from the video, it does not look like they used physics; that probably saved a lot of complexity in their implementation.

I need physics for my games though.

Does anyone have any more info on implementing such a system? Is there an asset available for purchase? Is everyone just rolling their own system?

If there's really nothing out there, maybe I'll work on creating such an asset.

 

Hello,

We didn't use physics for Shogun, but that's not the heart of this problem.
The key thing is that you have to build a simulation that is 100% deterministic, so that it will replay exactly the same.

You can't really use an external plugin to do so, you need to engineer your game code for that purpose.
This is a wide subject but here are the main ideas:

> Your simulation needs to be separated from your rendering/display
So that your simulation always run at a fixed time interval even if your display update (Frame rate) varies from one run to the other.

> You only record the player's action and at which simulation frame they occurred (not render frame, simulation frame)

> You make very sure you have a tag of the exact build/version of your code that generated this replay
If you use Perforce or the likes you can just dump the exact revision number at the start of the replay file

> You eliminate any 'random randomness' in your simulation
Key point: you need to store the seed of your random number generator if you have one and the seed (initial state) of any side system that can influence your simulation.

> Then, to repro any bug, you revert your game data/project to the exact version then run the game using not the player inputs but injecting the recording inputs in the simulation.

Your simulation (so the game) will occur exactly the same as when the bug occurred, and then, when you reach that bug you can effectively debug it (trace it) as your entire game is in the exact same state as when it occurred.

There is a last catch but that will only impact on multi-platform games: to replay the simulation exactly the same you also need to be sure that both platforms compute exactly the same. That's a problem you also get when you try to get exact same simulations in sync across various hardware and OS and use different compilers. Here you can't trust any Float calculation nor any supporting function (sin, cos, ordering an array, anything...) and have to rely on fixed point calculations, basically only dealing with integers internally as only integer operations are always the same from one computer system to another.

You will see that creating an input replay system is quite some work and that you will also have to maintain it through the course of development, as it's very easy to "break" it while developing new features.

That said it is an INVALUABLE tool for debug, once you have used it to debug a game you will never ever try to do it again without. It basically allow you to always be able to repro and fix that one elusive nasty bug that only happens once every 1000 playthroughs...

Hope that will help,

Renaud

 

Wow, what a great response. Thank you for taking the time to provide such a detailed break down.

It's given me a lot to consider and makes what your team accomplished that much more impressive.