Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Assets [Released] Ultimate Replay 2.0 - State based replay system

Discussion in 'Works In Progress - Archive' started by scottyboy805, Jan 31, 2020.

  1. scottyboy805

    scottyboy805

    Joined:
    Apr 10, 2013
    Posts:
    1,192
    Ultimate Replay 2.0 is currently in beta.
    You can download the latest beta version here.​



    You may or may not already be aware of our publish asset Ultimate Replay which allows you to record and replay objects in the scene using state infomormation to create killcams, ghost vehicles and more. We have recieved alot of feedback and suggestions for improvements to Ultimate Replay and now it is time to make those suggestions a reality.

    This is where Ultimate Replay 2.0 comes in. We have decided to develop version 2.0 as a standalone asset instead of an update because in order to implement many of the features and suggestions you have provided, major change and redesign of the code base is required. Essentially we will be starting from scratch and upgrading to 2.0 will cause many breaking changes.

    Here are some of the features we plan to support as well as all features already included in Ultimate Replay:
    • Complete Support for replaying and recording multiple instances simultaneously
    • Complete Support for simulating missed frames during playback. Currently in Ultimate Replay, if the playback speed is high or the game fps is low then it is possible for replay frames to be skipped which may contain per frame data such as events. Ultimate Replay 2.0 will simulate any missed frames in its next update cycle so that important data is not skipped.
    • Complete Support for auto compression of object state data if an object has not moved or changed state.
    • Complete Support for recording method calls.
    • Complete Better control over which replay objects will be recorded. This will be essential to support multiple replay instances properly.
    • Complete Serialization support for all primitive types. Previoulsy only Unity supported primitves were supported meaning that types like short and double could not be used.
    • Complete Better control over recording per instance (record fps, behaviour method to use for recording etc)
    • Complete Better control over playback per instance (playback fps, playback end behaviour etc)
    • Complete Better recording of the Animator component
    • Complete Better memory management to reduce unnecessary allocations.
    • Complete An example scene demonstrating how a ghost vehicle can be setup
    • Complete An example scene demonstrating how a killcam can be setup
    • WIP API's for accessing stored replay data such as transform position without requiring recorder components for deserialization.
    • Complete Support for saving and loading replay data using a stream.




    We will likley be increasing the price for Ultimate Replay 2.0 (although this is not confirmed yet) but existing customers will receive a heavily reduced price upgrade.

    If you have any suggestions or questions then please feel free to post a reply.
     
    Last edited: Sep 30, 2020
    petey likes this.
  2. scottyboy805

    scottyboy805

    Joined:
    Apr 10, 2013
    Posts:
    1,192
    We now have the foundations of the replay system setup which now supports recording and replaying multiple instances simultaneously. Using these API's we have been able to setup a basic test scene for a racing game ghost vehicle which is working nicley. Here is the code we created to record and replay a ghost vehicle using the new replay system so you can see how it might work:

    Code (CSharp):
    1.  
    2. public class StartFinishLine : MonoBehaviour
    3.     {
    4.         // Private
    5.         private ReplayScene recordScene = null;
    6.         private ReplayScene playbackScene = null;
    7.         private ReplayStorageTarget recordStorage = new ReplayMemoryTarget();
    8.         private ReplayStorageTarget playbackStorage = new ReplayMemoryTarget();
    9.         private ReplayHandle recordHandle;
    10.         private ReplayHandle playbackHandle;
    11.         private int targetTriggerIndex = 0;
    12.  
    13.         // Public
    14.         public ReplayObject playerCar;
    15.         public ReplayObject ghostCar;
    16.  
    17.         // Methods
    18.         public void Start()
    19.         {
    20.             // Create a recordable scene containg the player car only
    21.             recordScene = new ReplayScene(playerCar);
    22.             playbackScene = new ReplayScene(ghostCar);
    23.         }
    24.  
    25.         public void OnTriggerEnter(Collider other)
    26.         {
    27.             // Stop replaying
    28.             if (ReplayManager.IsReplaying(playbackHandle) == true)
    29.                 ReplayManager.StopPlayback(ref playbackHandle);
    30.  
    31.             // Stop recording
    32.             if(ReplayManager.IsRecording(recordHandle) == true)
    33.             {
    34.                 // Stop recording
    35.                 ReplayManager.StopRecording(ref recordHandle);
    36.  
    37.                 playbackStorage = recordStorage;
    38.                 recordStorage = new ReplayMemoryTarget();
    39.  
    40.                 // Enable the ghost car
    41.                 ghostCar.gameObject.SetActive(true);
    42.  
    43.                 // Clone identities - This allows the ghost car to be replayed as the player car
    44.                 ReplayObject.CloneReplayObjectIdentity(playerCar, ghostCar);
    45.  
    46.                 // Start replaying
    47.                 playbackHandle = ReplayManager.BeginPlayback(playbackStorage, playbackScene);
    48.  
    49.                 // Add end playback listener
    50.                 ReplayManager.AddPlaybackEndListener(playbackHandle, OnGhostVehiclePlaybackComplete);
    51.             }
    52.  
    53.             // Start recording
    54.             recordHandle = ReplayManager.BeginRecording(recordStorage, recordScene);
    55.         }
    56.  
    57.         private void OnGhostVehiclePlaybackComplete()
    58.         {
    59.             // Hide ghost car
    60.             ghostCar.gameObject.SetActive(false);
    61.         }
    62.     }
    63.  
     
  3. firstuser

    firstuser

    Joined:
    May 5, 2016
    Posts:
    147
    very exciting
    any idea on when it may be ready?

    might be able to help beta testing too if you need it! just pm :)
     
  4. scottyboy805

    scottyboy805

    Joined:
    Apr 10, 2013
    Posts:
    1,192
    Its hard to give a deadline at the moment because we are prioritizing an update to one of our other assets but we would hope to have many or the core features impemented by the end of the month. Then the real work will begin with finishing, testing, documenting, creating examples etc.

    We may need beta testers when nearing completion and will update this thread with details.
     
    firstuser likes this.
  5. firstuser

    firstuser

    Joined:
    May 5, 2016
    Posts:
    147
    excellent, thank you!
    can't wait :) !
     
  6. scottyboy805

    scottyboy805

    Joined:
    Apr 10, 2013
    Posts:
    1,192
    We have been working on Ultimate Replay 2.0 on and off over the past few weeks and have now got many of the core systems in place. We have been working spcifically on a new feature that we wanted to support which was saving and loading replay data using a stream. This will allow you to store the replay data almost anywhere and was requested by a number of users. We also decided that we should provide API's for accessing the deserialized object state data without the need to use a recorded component such as ReplayTransform to decode the data.
     
    firstuser likes this.
  7. scottyboy805

    scottyboy805

    Joined:
    Apr 10, 2013
    Posts:
    1,192
    Ultimate Replay 2.0 is getting close to a usable system now so we have started doing some stress testing and performance testing. We have made many improvements to the storage and compression aspects of the asset and thought we would share some of the results:

    One of the tests that we were suprised by was a file size comparison between Ultimate Replay 1.0 and 2.0. We setup a test scene with 300 dynaically spawned objects (Constantly moving around in the scene) in both assets and set the recording to be saved to a file. We then left the recording to run for 1 hour on each asset (Time duration measured in code for accuracy) and here were the results:

    Ultimate Replay 1.0 generated a replay file with the size of: 2.06GB
    Ultimate Replay 2.0 generated a replay file with the size of: 385MB

    ReplayFileTest.png

    That works out to be an improvement of almost 550% and even we did not expect improvements of that magnitude.

    It is also worth noting that if the objects were not constantly moving around in the scene then Ultimate Replay 2.0 could offer even better results due to the new way it stores object state data.
     
    Last edited: Mar 5, 2020
    firstuser likes this.
  8. firstuser

    firstuser

    Joined:
    May 5, 2016
    Posts:
    147
    Regarding "Support for simulating missed frames during playback" can you share a bit about what the strategy will be? The word simulate makes me a tad nervous. Will we be able to do true capture of each frame and replay? Or will we have to assume things might be interpolated and important events snap to the next frame?
     
  9. scottyboy805

    scottyboy805

    Joined:
    Apr 10, 2013
    Posts:
    1,192
    Simulating missed frames is a feature we have implemented in 2.0 to address an underlying design problem with the way playback worked in the current Ultimate Replay version. When in playback mode and the playback speed is high or the game frame rate is low, it is possible that the amount of time passed betwen updates it large enough to completley skip over one or more frames. This is where the missed frame simulation will simulate these missed frames by running them thought the replay system so that objects are deserialized are restored, replay events fire, Replay behaviour callbacks run etc. This simulation stage will occur on the same frame as the current frame update if previous frames have been missed. It is really intended to ensure that single frame playback data such as replay events do not go missed during playback.

    I hope that makes sense. Let me know if you have any questions.
     
  10. petey

    petey

    Joined:
    May 20, 2009
    Posts:
    1,817
    Wow, just saw this! This is great news.
    I'm especially interested in the Animator controller support as that is what has stopped me from making replays in the past. Out of interest, would anything like this be able to run on mobile?
     
    scottyboy805 likes this.
  11. scottyboy805

    scottyboy805

    Joined:
    Apr 10, 2013
    Posts:
    1,192
    Hi, We had experimental Animator recording in the original Ultimate Replay asset but as you may or may not be aware, it would sometimes struggle to replay complex Animator state machines properly. We hope to be able to accuratley record and replay any Animator state machine in Ultimate Replay 2.0 no matter how complex.
    It should run only mobile paltforms just fine. The only potential issue I could see is maybe some performance issues due to the amount of CPU work required to record a snapshot (Mainly due to compression and storage optimization techniques) but if and when that becomes a problem we can look into that.
     
  12. petey

    petey

    Joined:
    May 20, 2009
    Posts:
    1,817
    Sounds cool! My scene is pretty straight forward really the only thing that is complicated is the animator, so I’ll definitely give it a try :)
     
    scottyboy805 likes this.
  13. scottyboy805

    scottyboy805

    Joined:
    Apr 10, 2013
    Posts:
    1,192
    If you are willing/able to, then feel free to send us the animated object to info(at)trivialinteractive.co.uk and we will use it in our testing to make sure it is recorded and replayed correctly. We are currently looking for testing/validation assets like this.
     
  14. petey

    petey

    Joined:
    May 20, 2009
    Posts:
    1,817
    Thanks!
    Actually thinking about it my animator isn't overly complicated. There are a lot of clips but most them use either animator.CrossFadeInFixedTime or animator.PlayInFixedTime to control if they are playing and they are mostly on the one layer. So in that sense it might not bee too challenging for you.
    I'll try and make a little separated sample scene for you to check out though.
     
  15. scottyboy805

    scottyboy805

    Joined:
    Apr 10, 2013
    Posts:
    1,192
    If there are lots of animations and transitions then that would be a useful test asset for us. We have already tested the animator recording with a number of basic animator setups and everything seems to be working correctly but the more testing we can do on a variety of assets the better.
     
  16. scottyboy805

    scottyboy805

    Joined:
    Apr 10, 2013
    Posts:
    1,192
    We are now pleased to anounce that Ultimate Replay 2.0 beta is now available. The beta is open to everyone and we would encourage users to test out features and report any bugs or issues via our website or here on the forum. No registration is required and the beta package can be downloaded here and will be updated as bugs and features are fixed or added. Source code will not be available during the beta period and building a player will also not work.
    Documentation is mostly non-existant at the moment although many of the API's have XML comments. If you have any quesitons about usage then we will be happy to answer them. Also if you have any feature requests then feel free to suggest them either here or via the feature request form on our website.
     
    firstuser likes this.
  17. scottyboy805

    scottyboy805

    Joined:
    Apr 10, 2013
    Posts:
    1,192
    Here are some test scenes we are working on at the moment. Nothing too impressive yet but it gives an idea of what UItimate Replay can do or be used for:

    A ghost vehicle system setup by recording the player car and replaying the data onto the ghost car.


    The old replay cubes example used in the previous version ofl Ultimate Replay. It is much more efficient this time around.


    Full playback support for particle systems. Something which was missing from the previous version.
     
    Last edited: Aug 25, 2020
    firstuser likes this.
  18. firstuser

    firstuser

    Joined:
    May 5, 2016
    Posts:
    147
    @scottyboy805

    Scenario: I record an event on frame 1 and another event on frame 2. During playback frame 1 gets skipped. Does frame 1's event 1 get triggered before frame 2's?

    And would the same thing happen for a recorded function?

    Thank you.
     
  19. scottyboy805

    scottyboy805

    Joined:
    Apr 10, 2013
    Posts:
    1,192
    Hi,
    Technically the first frame can never be skipped when entering playback mode as it is is restored on the same frame as the playback call is made. If you apply the example to frames 2 and 3 though, frame 2 will be simulated (if missed) before frame 3 is resotred meaning that events, functions and variables for frame 2 will be restored before frame 3. This will occur on the same frame as frame 3 though just with different playback timings plugged into the playback update system to simulate the frame as if it was played back normally.

    I hope this helps you. Let me know if you have any more questions.
     
    firstuser likes this.
  20. scottyboy805

    scottyboy805

    Joined:
    Apr 10, 2013
    Posts:
    1,192
    It has been a little while since we updated this thread but we have been working on bug fixes and new features for Ultimate Replay 2.0 in between other projects. We have been improving file compression even further to produce even more compact replay files. We have also been working on a killcam example scene which is coming along nicley and we will be able to show that in the near furture. Just a few things need finishing up and adding before we will add that to the beta. It is a simple call of duty style killcam example where when the player is killed, you get to watch the replay from the perspective of the player that killed them. It works quite nicley and we also added ragdolls for when players are killed to show how Ultimate Replay 2.0 is able to record skeleton setups.

    We have also had quite a few requests for human readable file formats for replay files so we have been working on adding json support for saving replay files. We have this mostly working at the moment and it is able to produce a json file containing all the necessary data but with the cost of file size and performance. At the moment, an equivilent json file is more than 27 times larger than the binary version for the same scene so will likley not be for everyone. Currently we only have support for writing replay files in this format and it may have to stay that way. Initial testing indicates that the amount of loading and processing required to read the json data causes the streaming thread to fall behing which results in stutters and lag during playback as the playback system is waiting for the data to be read. We will try to find a way around this but it may take some time.
    In the meantime we have added an example json file generated from one of out test scenes so you can get a picture of how the data is stored. This test scene contains 300 dynamically spawned cubes that are spawned in mid air and interact using physics. The record duration is 5 seconds and the equivilent binary file is roughly 114KB in size.
     

    Attached Files:

    Last edited: Jun 9, 2020
    firstuser and petey like this.
  21. petey

    petey

    Joined:
    May 20, 2009
    Posts:
    1,817
    Wow this all sounds amazing!
    I have a few questions if you have a moment.
    Does this support animations created in the playable graph? I was using Animancer a bit lately so just wondering.
    Also how would you manage sound effects in a replay, I'm guessing it gets kinda tricky.
    Thanks!
    Pete
     
  22. scottyboy805

    scottyboy805

    Joined:
    Apr 10, 2013
    Posts:
    1,192
    Hi,
    Ultimate Replay 2.0 can support any animation that you create using the Animator component and also things like Ik animation and ragdolls but that requires a difrferent approach. The ReplayAnimator component is used to record and replay a specific Unity Animator.
    Sound is also supported but at the moment there are a few issues that need to be fixed but it is working for the most part. Again we have a ReplayAudio component which is able to record and replay an audio source component by storing data like the audio time stamp and other related values.
     
    petey likes this.
  23. Mikle7

    Mikle7

    Joined:
    Apr 29, 2020
    Posts:
    1
    Hey guys, just wondering if there's any update on the progress of this beta?
    Really keen for it
     
  24. scottyboy805

    scottyboy805

    Joined:
    Apr 10, 2013
    Posts:
    1,192
    Hi,
    Good timing. We were just about to update this thread with some more progress that we have made.

    We have almost finished the killcam demo scene that we started a little while ago and it is working nicley. There are just a few finishing details to sort out like dropping the weapon on death as well as maybe adding some props to the scene to make it more interesting.

    Killcam.gif

    We have also added a new tool to the asset to help the setup of humanoid characters for recording things such as inverse kinematics, ragdolls or bone based animation recording. The tool is used to add the necessary replay components to the correct transforms of a character which could be a long or error prone process if done manually. The tool requires an Animator component to be attached and it must also have a valid avatar configured as a humanoid.

    ReplayHumanoidConfigurator.png

    Other than that we have started working on the documentation of the asset so we are getting closer to releasing onto the Unity asset store.
     
    firstuser, Mikle7 and petey like this.
  25. petey

    petey

    Joined:
    May 20, 2009
    Posts:
    1,817
    Hey that all sounds really cool! Just wondering would it be possible to set up a generic skeleton also? I’m guessing it would have to be manually?
    Thanks!
     
  26. scottyboy805

    scottyboy805

    Joined:
    Apr 10, 2013
    Posts:
    1,192
    Generic skeletons will have to be setup manually unfortunatly. We use Animator.GetBoneTransform to get the assigned bones and I don't think there is a way to do this for generic rigs. If someone knows a way to access the bones of a generic rig the let me know and we could add support then but we couldn't find an obvious way looking through the API.
     
  27. petey

    petey

    Joined:
    May 20, 2009
    Posts:
    1,817
    Yeah that sounds fine. I’m excited now :D
     
    scottyboy805 likes this.
  28. scottyboy805

    scottyboy805

    Joined:
    Apr 10, 2013
    Posts:
    1,192
    We have updated the latest beta version which is available here. This version includes many bug fixes and new features along with the killcam demo which we showcased a previously. Please note that some of the demos and example scripts are not yet completed 100% but work well enough to demonstrate some of the key aspects.
     
    petey, Mikle7 and firstuser like this.
  29. scottyboy805

    scottyboy805

    Joined:
    Apr 10, 2013
    Posts:
    1,192
    Just a quick update. We are moving closer to the release of Ultimate Replay 2.0 but still have things to finish up and refine. We have now finished the first draft of the user guide which can be view here. We have also added a 'How do I...' section which we hope will answer many of the common questions that users might face. If you have a question about how to do something with the asset that is not already listed, then let us know and we can add it into the user guide for all to benefit.
     
    petey likes this.
  30. petey

    petey

    Joined:
    May 20, 2009
    Posts:
    1,817
    Wow the docs look great, really thorough!
     
    scottyboy805 likes this.
  31. scottyboy805

    scottyboy805

    Joined:
    Apr 10, 2013
    Posts:
    1,192
    Thanks! Glad you like it. It took alot of work as there are a many features to cover in the new version.
     
  32. scottyboy805

    scottyboy805

    Joined:
    Apr 10, 2013
    Posts:
    1,192
    The asset promotion and support thread has now been created which can be found here. Still quite a bit of work to do before we can think about submiting to the asset store but we are working hard :).
     
    petey and firstuser like this.
  33. scottyboy805

    scottyboy805

    Joined:
    Apr 10, 2013
    Posts:
    1,192
    Ultimate Replay 2.0 has now been submitted to the asset store. More information on the main thread here.
     
    julianr, petey and firstuser like this.