Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

[Released] Ultimate Replay 2.0 - Next generation state-based replay system

Discussion in 'Assets and Asset Store' started by scottyboy805, Sep 8, 2020.

  1. lorewap3

    lorewap3

    Joined:
    Jun 24, 2020
    Posts:
    58
    To anyone reading this in the future, I provided these guys with a stripped down version of my project and within a day they identified a bug and provided a fix. My weird use case brought up another, and even though a much harder to debug issue dealing with the underlying compression library, they implemented a workaround and got it to me yesterday. And I submitted the project to them only 5 days ago including a weekend!

    These guys have a strong dedication to improving their products and fixing any issues you encounter. It is not lost on me and is greatly appreciated!

    I'm currently working on integrating it with the Unity Timeline. The goal is to be able to record play sessions and then be able to scrub the timeline to visually see the replay in the edit mode for the purpose of compositing elements around them. I knew my use case would push the limits of any replay system and these guys have definitely delivered on a system that can handle it!

    -Will
     
  2. scottyboy805

    scottyboy805

    Joined:
    Apr 10, 2013
    Posts:
    1,192
    Ultimate Replay 2.1.4 has been submitted to the asset store and will be available for download within a day or two. This version includes the following:
    • Fixed an issue in the replay file target where a CorruptedData exception could be thrown in some rare cases. The problem was caused by a bug in the mono compression library that we use and a suitable workaround is now in place.
    • Fixed an issue in the replay file target where the streamer thread could sometimes report that it had failed to start. This was a simple logic timing issue casued by a race condition and has now been fixed.
    • Fixed a bug in the ReplayAnimator component where some parameters would not be recorded correctly.
    • Fixed an issue where moving the root asset folder into a folder named 'Plugin' would cause script compilation errors.
    • A few othr minor bug fixes.
     
    julianr likes this.
  3. lorewap3

    lorewap3

    Joined:
    Jun 24, 2020
    Posts:
    58
    Hey @scottyboy805 ,

    I'm making progress on the timeline integration, but I have 2 main questions regarding implementing in Edit mode.

    I made my ReplayControls component into a Singleton that executes in both Edit/Play mode. I implemented it such that when the timeline is scrubbed, it starts a coroutine to restore the ReplaySnapshot for the specified time. I'm using coroutines so that in the event I scrub too fast, the SeekToTime operations will be added to the coroutine queue. I'm trying to minimize reading and disposing storage targets excessively, so after a scrub is complete, the storage target is disposed of (as to not incur possible issues later). So with every scrub, the StorageFileTargets are read in. But if there is already a coroutine in progress, the first won't dispose of the storage target. Only the last one will do the disposing.

    Anyway, that's the approach I'm trying to take. Cause I'm not 'Playing' any replay, just scrubbing to a certain time. Maybe there will be a way to integrate with the preview feature of timeline, but that's far down the line and not really necessary.

    Question 1
    What I don't know how to do is how to restore the snapshot directly. I used the code you provided from Jan 11 as a baseline, but it shows how to read the data itself. I really just want to restore it. This is the function so far:
    Code (CSharp):
    1. public IEnumerator SeekToTime(double time)
    2. {
    3.     if (!Application.isPlaying)
    4.     {
    5.         LoadStorageTargets();
    6.         for (int i = 0; i < storageTargets.Count; i++)
    7.         {
    8.             ReplaySnapshot snapshot = storageTargets[i].FetchSnapshot((float)time);
    9.             replayScenes[i].RestoreSnapshot(snapshot, storageTargets[i].InitialStateBuffer);
    10.             yield return null;
    11.         }
    12.     }
    13.     yield return true;
    14. }
    I think I'm using the right method in RestoreSnapshot. But I have no idea what I'm supposed to provide for the 2nd parameter given I'm not using the ReplayManager or in Play mode. I plugged storageTargets.InitialStateBuffer in there but I get null exceptions.

    Question 2
    I'm also interested in preserving the initial state before I enter 'Timeline Editing' mode. Here's the basic 'Directors Chair' window I created. It has 3 states, and shows the replays created. The replay highlighted green is currently loaded.
    upload_2021-3-3_16-44-53.png
    Rehearsal - Nothing happens. No recording and no replay scrubbing.
    Recording - Nothing in Edit mode, but in Play mode records for all loaded scenes.
    Performance - Timeline scrubbing in Edit mode. In Play mode, plays the replay normally.

    So ideally I'd like to save the state of all ReplayObjects before entering 'Performance' mode and restore back to that state after leaving 'Performance' mode. I'm not sure how best to save the state when not using ReplayManager or in Play mode.

    Q1 is really the roadblock at the moment, though. Or perhaps I'm going about this the wrong way and there's a better way to implement this. I did initially try using ReplayManager, but instantly encountered issues as I'd expected. ReplayManager.Awake() gives exceptions and any loading tries to utilize the DoNotDestroyOnLoad scene which Unity is not happy about when in Edit mode. Any advice you have is most appreciated!

    Thanks again!
    Will
     
    Last edited: Mar 3, 2021
  4. scottyboy805

    scottyboy805

    Joined:
    Apr 10, 2013
    Posts:
    1,192
    Hi again,

    1. That code looks fine to me and should work as expected for the most part. The only other thing you need to do (which I am not sure if you are doing in the 'LoadStorageTargets' method) is to prepare the storage target for read operations. It is an important step and can be done by simply calling the following method:

    Code (CSharp):
    1. myStorageTarget.PrepareTarget(ReplayTargetTask.PrepareRead);
    The behaviour of a storage target when attempting to read from a non-prepared target is undefined and would likley explin the NullReferenceException you mention. The initial state buffer data is likley not created at that stage.

    Also, is there any reason why you do not want to use the main replay API for this? It can be done manually if needed, but it should also work fine when called from edit more with a small addition. Replay services wil not be updated in edit mode but you can update them manually if needed like so:

    Code (CSharp):
    1. class Example
    2. {
    3.     float lastTime = 0;
    4.  
    5.     void EditorUpdate()
    6.     {
    7.         float current = EditorApplication.timeSinceStartup;
    8.         float delta =  current - lastTime;
    9.         lastTime = current;
    10.        
    11.         ReplayManager.UpdateState(delta);
    12.     }
    13. }
    Edit - I see now that you tried this but had exceptions from the Awake method? Are you sure it is 'ReplayManager.Awake' and not 'ForceAwake', because the former method has an empty body in the full version of the asset? If force awake is the issue, then I assume the stack trace points back to ReplayControls.Awake? If so, you should be able to simply comment out the call to 'ReplayManager.ForceAwake' and just use the manual update method I suggested. We will add editor defines in the next update so that these issues cannot occur.

    2. Yes this is possible, and infact, the replay system does exactly this to restore the scene to its original state when playback has finished. You can just store a ReplaySnapshot in memory for a give frame and then restore it at a later date when playback has ended.

    Code (CSharp):
    1. class Example
    2. {
    3.     ReplayScene myReplayScene = ...
    4.     ReplaySnapshot snapshot;
    5.     ReplayInitialStateBuffer stateBuffer = new ReplayInitialStateBuffer();
    6.    
    7.     public void BeforeReplay()
    8.     {
    9.         snapshot = myReplayScene.CaptureSnapshot(0, 0, stateBuffer);
    10.     }
    11.    
    12.     public void AfterReplay()
    13.     {
    14.         myReplayScene.RestoreSnapshot(snapshot, stateBuffer);
    15.     }
    16. }
    If you don't see any changes after calling the 'AfterReplay' method, then you may also need to trigger a ReplayUpdate event, which is slightly problematic. The methods used to trigger replay events are marked as internal. You can try the following code to send an update but the compiler may not recognise it if you are using assembly definitions:

    Code (CSharp):
    1. ReplayBehaviour.InvokeReplayUpdateEvent(myReplayScene.ActiveReplayBehaviours, ReplayTime.startTime);
    I hope that makes things a bit clearer for you. Let me know if you have any more questions.
     
  5. lorewap3

    lorewap3

    Joined:
    Jun 24, 2020
    Posts:
    58
    Hey @scottyboy805 , I haven't tried anything yet, but I wanted to respond to this one and give you the error I get when I call ReplayManager.ForceAwake() . (Yes, it was ForceAwake, I mistakenly typed Awake)
    Code (CSharp):
    1. InvalidOperationException: The following game object is invoking the DontDestroyOnLoad method: ReplayManager. Notice that DontDestroyOnLoad can only be used in play mode and, as such, cannot be part of an editor script.
    2. UltimateReplay.ReplayManager.ForceAwake () (at Assets/Plugins/Recording Plugins/Ultimate Replay 2.0/Scripts/ReplayManager.cs:1174)
    3. LorewaPlay.ReplayController.OnEnable () (at Assets/Scripts/Main Assembly/MonoBehaviours/Replay/ReplayController.cs:92)
    I will try the manual update method using ReplayManager as you suggest. Maybe that will allow me to keep the storage targets opened while in the 'Performance' mode and only dispose of them when exiting instead of every scrub operation. I will also try capturing a snapshot before and restoring it. That was the part I was missing:
    Code (CSharp):
    1. myReplayScene.CaptureSnapshot(0, 0, stateBuffer);
    I was trying to call RestoreSnapshot but didn't know how to create the InitialStateBuffer that the RestoreSnapshot was asking for:
    Code (CSharp):
    1. replayScenes[i].RestoreSnapshot(snapshot, storageTargets[i].InitialStateBuffer);
    It's not obvious from the code that CaptureSnapshot is populating the InitialStateBuffer it's taking in as a parameter. I mean it is if you read the function itself but I wouldn'tve guessed otherwise.

    Will try these out and get back to you. Thanks!
     
  6. scottyboy805

    scottyboy805

    Joined:
    Apr 10, 2013
    Posts:
    1,192
    Thanks for checking that.
    We have now fixed the problem on our end by not running that code while in edit mode, but you can just remove the call to 'ForceAwake' for the time being to get around that problem.

    We will add some additional code comments about the initial state buffer so that it is easier to understand what to pass to the snapshot methods.

    Let me know how you get on and if you have any further issues.
     
  7. scottyboy805

    scottyboy805

    Joined:
    Apr 10, 2013
    Posts:
    1,192
    We have just setup a discord server for Ultimate Replay 2.0 and a few other assets of ours so that we can offer better and faster support. We would love for you to come and join and hang our with other users of the asset and maybe share any cool projects you are working on.
     
    Last edited: Mar 21, 2021
    julianr likes this.
  8. lorewap3

    lorewap3

    Joined:
    Jun 24, 2020
    Posts:
    58
    Hey @scottyboy805 ,

    Good news for us about the discord! Thank you for that! As I've told other asset developers, you opened a can of worms you may regret ;).

    I need some more clarification on how to use ReplayManager in Edit mode, if it's possible as you suggest. I'm not calling ReplayManager.ForceAwake() anymore, but I'm still getting the same error using the other ReplayManager methods. I don't want to overload you with unnecessary code, but here's the function calls when entering Performance from the Rehearsal/Recording state:

    Code (CSharp):
    1.  
    2. if (VerifyReplayLoaded())
    3. {
    4.     LoadStorageTargets();
    5.     CaptureInitialState();
    6.     StartPlaybackFrame();
    7. }
    VerifyReplayLoaded ensures that valid replay files exist.
    LoadStorageTargets loads (and prepares) them

    CaptureInitialState: captures Snapshot/InitialState
    Code (CSharp):
    1. private void CaptureInitialState()
    2. {
    3.     initialDataBuffers.Clear();
    4.     initialSnapshots.Clear();
    5.     for (int i = 0; i < replayScenes.Count; i++)
    6.     {
    7.         ReplayInitialDataBuffer replayInitialDataBuffer = new ReplayInitialDataBuffer();
    8.         initialSnapshots.Add(replayScenes[i].CaptureSnapshot(0, 0, replayInitialDataBuffer));
    9.         initialDataBuffers.Add(replayInitialDataBuffer);
    10.     }
    11. }
    and finally:
    StartPlaybackFrame:
    Code (CSharp):
    1.  
    2. private void StartPlaybackFrame()
    3. {
    4.     replayHandles = new ReplayHandle[storageTargets.Count];
    5.     for (int i = 0; i < storageTargets.Count; i++)
    6.         replayHandles[i] = ReplayManager.BeginPlaybackFrame(storageTargets[i], replayScenes[i], true, false, null);
    7.     return;
    8. }
    9.  
    It errors out on
    Code (CSharp):
    1. replayHandles[i] = ReplayManager.BeginPlaybackFrame(storageTargets[i], replayScenes[i], true, false, null);
    with the same "InvalidOperationException: The following game object is invoking the DontDestroyOnLoad method: ReplayManager." error I get from ForceAwake()

    All of those functions execute a single time on an event triggered by switching the mode to Performance. The following SeekToTime function is called externally by the PlayableDirector after entering Performance mode and when the Timeline is scrubbed to a new time.
    Code (CSharp):
    1. public IEnumerator SeekToTime(double time)
    2. {
    3.     if (!Application.isPlaying)
    4.     {
    5.         for (int i = 0; i < replayHandles.Length; i++)
    6.         {
    7.             if (ReplayManager.IsReplaying(replayHandles[i]))
    8.                 ReplayManager.SetPlaybackTime(replayHandles[i], (float)time, PlaybackOrigin.Start);
    9.             yield return null;
    10.         }
    11.         ReplayManager.UpdateState(0.5f); // Edit: Added this late at the start of Edit: at the bottom
    12.     }
    13.     yield return true;
    14.  
    15. }
    I included the SeekToTime function to help understand my thought process but it never gets to that point because the ReplayManager errors out.

    You're right, I don't have ReplayManager.UpdateState() in there, and I'm sure I need to, but I haven't gotten there yet. I believe I understand what you meant by your previous example, but I was a little confused at first by the deltaTime. This doesn't need to run every frame so I don't want to run UpdateState on every editor update as it's unecessary. So please correct me if I'm wrong but in my case I should just be able to do this?
    Code (CSharp):
    1. ReplayManager.UpdateState(0);
    Since deltaTime makes no sense in this context, I'm assuming that UpdateState just needs to be called so it updates all ReplayObjects in the scene and it doesn't matter what you pass it. To be safe I'd just pass it 0, though.

    Do you still think I can use the ReplayManager with some minimal tweaking? I would really prefer to use the built-in ReplayManager methods for all this, but I have no concept of how big of a task it will be to make it Edit mode friendly. Although looking into BeginPlayback it just calls ForceAwake() at the end of the function.

    Edit: (More like I tested this while I was typing and felt it worthwhile to have the history)
    So just for kicks, I tried commenting out the DontDestroyOnLoad(go) line in ReplayManager.ForceAwake(). I don't get any exceptions anymore, but nothing is getting restored either. I've traced the debugging all the way through the SeekToTime, and everything gets executed without errors. The replayHandles are valid, the SetPlaybackTime doesn't error, neither does the UpdateState(), but nothing gets updated in the scene. I also verified the replayScenes were valid as well. I tried 0.5 instead of 0 as a parameter to UpdateState as shown above but that didn't change anything.

    Thanks Scotty!
    Will
     
    Last edited: Mar 6, 2021
  9. scottyboy805

    scottyboy805

    Joined:
    Apr 10, 2013
    Posts:
    1,192
    Hi again,
    Thanks for the detailed report of what you have tried.
    Your setup all looked OK to me so I decided it would be best to put a test project together to run replays in edit mode. After setting up a simple test project, it became clear that there were a couple of issues preventing edit time replays from working correctly. Here are the issues we found:
    • Calling 'ForceAwake' while in edit mode causes issues as you discovered already. We have now added additional checks to prevent that code from running when called in edit mode.
    • ReplayObjects did not have the necessary setup code run while in edit mode. This caused replay components to not be deserialized or updated by the replay system, resulting in static objects with no movement during playback. We modified the code slightly so that initialization code can run once in edit mode if required so that replay components will be updated correctly.
    • Dynamic replay objects that need to be destroyed during playback will generate an error about calling the 'Destroy' method at edit time. We have now added support for the correct method 'DestroyImmediate' at edit time only.
    After discovering and fixing these issues, we can now view a replay file in edit mode with no apparent issues. Replay transfom animations are updated smoothly depending upon how often you call 'UpdateState', and dynamic replay objects work fine also.

    I think you may have misunderstood the 'deltaTime' value for the 'UpdateState' method. It is a required parameter and simply setting it to '0' will cause the function to return without doing anything at all. You should pass a positive value representing the amount of time in seconds since the last update. The replay system uses this value to drive all record and replay timers. if you pass a value of '3' to the method, the replay system will update any recordings or replays by adding '3' seconds to it's internal timers (assuming time scale is set to '1'). For replays this will cause the playback time to be advanced '3'. You can call this as little or as often as you need, but you must still pass ther amount of time in seconds that you want to advance the replay system. I hope that makes sense.

    It is also worth mentioning that you may not need to call UpdateState at all if you are not interested in viewing the replay in sequence. You should just be able to call 'BeginPlaybackFrame' and then use 'SetPlaybackTime' to move to specific frames.

    There were quite a few changes made to the code to get this working which means that we will need to submit an update. We will do that shotly but it will likley take a day or two to be verified and published. If you want the update right away, feel free to send us an email with your invoice number to (info@trivialinteractive.co.uk) or come and message me on the discord server.
     
    Jmonroe likes this.
  10. lorewap3

    lorewap3

    Joined:
    Jun 24, 2020
    Posts:
    58
    Ah this is what I thought. I didn't think it should be necessary to call UpdateState if I was directly setting the playhead with SetPlaybackTime.

    All that makes sense about the deltaTime. That's how I'll do it down the line when I implement being able to 'Preview' a timeline while in edit mode, but right now direct positioning is all I really want.

    I sent you an email with my invoice #. I don't know how long it takes the asset store to approve things but as long as I've been working on this I'm very eager to get it in a working state! If you have the code ready I'd love to get my hands on it to try out.

    Thanks Scott!
    Will
     
  11. scottyboy805

    scottyboy805

    Joined:
    Apr 10, 2013
    Posts:
    1,192
    It usually takes a day or two for updates to be verified in our experience.
    I have sent you the update direct via email though so you have access to it straight away.

    Let me know if you have any more issues or questions.
     
  12. petey

    petey

    Joined:
    May 20, 2009
    Posts:
    1,817
    Hiya! Pretty close to having another play with Ultimate Replay 2. Last time I had a look I ran into a bit of trouble with sounds and particle systems. I realise they could be tricky components to wrangle. Just wondering if you have any tips or best practices for using them in a more replay friendly way?
    Thanks!
    P.
     
  13. scottyboy805

    scottyboy805

    Joined:
    Apr 10, 2013
    Posts:
    1,192
    Hi again,
    The ReplayAudio and ReplayParticle components are still a little limited in their features. We looked into adding better support for things like rewiding audio, moving partice systems, etc. but we could not get a solid working solution with out some tradeoffs. There would always be small (but noticable) audio glitches in the Replay Audio component, and the Replay Particle System component would not always simulate nicely. As a result, we decided to stick to a failry simple implementation, and users can implement a custom more advanced component if needed that is more tailored to their use case. With this in mind, the built in components will work for simple cases, but don't expect perfection in compex use cases. I hope that helps you.
     
  14. petey

    petey

    Joined:
    May 20, 2009
    Posts:
    1,817
    Yeah that’s totally understandable, I think I’ll just have to make a few custom things here and there.
     
  15. lorewap3

    lorewap3

    Joined:
    Jun 24, 2020
    Posts:
    58
    Hey @scottyboy805 ,

    Were the changes made to the codebase that you sent me via email added to the official release yet? I think I clobbered my code. I presumed you guys had published the updates, and went in and updated Ultimate Replay via the package manager. When it gave me the "DoNotDestroyOnLoad was called in Edit mode" error I could tell it wasn't, so I tried reapplying the patch you sent me over email, and the edit previewing no longer works.

    What is the ETA on publishing those changes? I'm not even sure what order to reapply the code changes. I'm working on other things right now, but it does tend to bug me when things are in a non-working state.

    Thanks Scotty,
    Will
     
  16. scottyboy805

    scottyboy805

    Joined:
    Apr 10, 2013
    Posts:
    1,192
    Hi,
    We have not yet submitted an official update to fix that issue. We are currently working on some optimizations that we want to bundle with the same update, and also some other minor bug fixes. There is also a bug going on with the publisher portal where we submit updates, so it may take a little longer than usual for the update to go though. I will see if I can get an update submitted before the weekend.
     
  17. lorewap3

    lorewap3

    Joined:
    Jun 24, 2020
    Posts:
    58
    Ah I see. If you guys are still working on the code I don't want to rush you guys into publishing something you're not comfortable with. I have plenty of work to do elsewhere it's not a huge rush. I probably won't circle back around to this until mid-May.

    I just would like to be notified whenever the changes have been published. So if you could @me here or email me when those updates are published I'd much appreciate it. I do get asset update notifications but those aren't often very prompt.

    Thanks Scotty!
    Will
     
  18. scottyboy805

    scottyboy805

    Joined:
    Apr 10, 2013
    Posts:
    1,192
    Ok, no worries.
    There is not too much work left to do on the update I believe, just a few more minor fixes and testing.
    I always post on the forum anyway when a new update has been submitted, so hopefully you will get a notification for when the update has been submitted. Hopefully the publisher portal bug will not be an issue for us.
     
  19. Fibonaccov

    Fibonaccov

    Joined:
    May 5, 2020
    Posts:
    58
    Hi,

    I couldn't find the answer here so I am posting the issue I am facing with the Killcam demo (which I downloaded through your website).

    I did not update anything but running it will cause the exception below as the scene attempts to replay.

    Let me know if you need more info. Thanks

     
  20. scottyboy805

    scottyboy805

    Joined:
    Apr 10, 2013
    Posts:
    1,192
    Hi,
    Thanks for reporting this error.
    I did a quick test in a clean project (2020.2), and apart from some broken materials (which I will upload a fix for), the killcam demo appeard to function correctly and I was not able to reproduce the exception you posted. Would it be possible to send us the Unity project you are using if you are willing/able to, and I can look into it further. It could be that something is messed up or missing after import which may be causing you problems. Email us at info(at)trivialinteractive.co.uk if you can share the project.
    Apart from that, could you try using the latest version of Ultimate Replay 2.0 if you are not already (2.1.5). Also, if you can let me know which Unity version you are using, I can retest on my end to see if the problem is specific to a Unity version.
     
  21. scottyboy805

    scottyboy805

    Joined:
    Apr 10, 2013
    Posts:
    1,192
    Hello Ultimate Replay 2.0 users!

    Are there any features or improvements you would like to see added to the asset? We are due to start work on the next major update of Ultimate Replay 2.0 shortly and want to make sure we are adding features that are needed/wanted, or improvements that will make it easier even to use the asset. We will consider any requests at this stage, although the scale of the feature must be reasonable and not outside the scope of the asset.

    We would love to hear your thoughts :)
     
    firstuser likes this.
  22. julianr

    julianr

    Joined:
    Jun 5, 2014
    Posts:
    1,212
    Hi. Maybe expand on the Demos a bit more, adding in examples of switching on/off replay with a saved replay list GUI of each time you click the play button (by date), so you can load up replays to watch, delete or protect from deletion. Example Demo of two replays, one for game play one for reversing time on specific objects?
     
    scottyboy805 likes this.
  23. scottyboy805

    scottyboy805

    Joined:
    Apr 10, 2013
    Posts:
    1,192
    Thanks for the feedback.
    Do you mean a replay viewer gui where the user can select a replay to watch? We can certainly add a demo for this.
    Rewinding only certain objects is an interesting idea. It is not someting that is really supported natviley (The whole replay must rewind instead of specific objects) but can be achieved using multiple storage targets. I will see what can be done to support this use case.

    Let me know if you think of anything else.
     
  24. julianr

    julianr

    Joined:
    Jun 5, 2014
    Posts:
    1,212
    Yes, a replay viewer GUI. This will be really useful, time saving too! Should add some extra value to the asset as this feature seems like a must have.

    Nice on the multiple storage targets. One process for replay, the other for recording specific rewinds, and perhaps syncing them into the core replay through timed event triggers? So you could have the reversal of time for a set number of objects. I've not really looked into it much, and not sure how you do things in the asset. It would be a bit like an alternate timeline. If it can't be sync'd in, then maybe it can prevent objects from being originally replayed on the main replay, and use an group replay file for specific objects for the new timeline.

    I have plenty of ideas :)
     
    Last edited: Jun 10, 2021
    scottyboy805 likes this.
  25. scottyboy805

    scottyboy805

    Joined:
    Apr 10, 2013
    Posts:
    1,192
    Ok, we will certainly add a demo for that.

    Yes, you have the right idea. Multiple replays can be used to achieve this feature. I will see if we can get a demo setup for that too.

    Thanks again for the valuable feedback.
     
    julianr likes this.
  26. julianr

    julianr

    Joined:
    Jun 5, 2014
    Posts:
    1,212
    Awesome! Look forward to seeing the results from this, which will be good I am sure!!
     
    scottyboy805 likes this.
  27. TSBKIMSANGYEOB

    TSBKIMSANGYEOB

    Joined:
    Dec 21, 2018
    Posts:
    1
    Hi,
    I want to know that it can save replay data(e.g. replay transform) as a file.
    And can I replay what I did with the file whenever I want?
     
  28. scottyboy805

    scottyboy805

    Joined:
    Apr 10, 2013
    Posts:
    1,192
    Hi,
    Yes you can. There is no restrictions for when or where you can replay a file, so you can choose to do it whenever you like. Ideally though you would ensure that the same scene is loaded that was used to record, otherwise you may see some playback inaccuracy warnings for objects that cannot be found. You can eliminate this issue completley by using replay prefbs for all recorded objects. using this approach, you can record in one scene and then replay in a completely different scene without issue.

    I hope that helps you. Let me know if there is anything else.
     
  29. scottyboy805

    scottyboy805

    Joined:
    Apr 10, 2013
    Posts:
    1,192
    Ultimate Replay 2.1.6 has been released on the asset store. This version includes:
    • Fixed a bug where playback accuracy warnings could be reported falsely.
    • Fixed a bug where the replay system would sometimes fail to find a replay object in the scene based on its replay id value.
    • Fixed a bug in the ReplayTransform component where an exception could be thrown in some cases during Awake.
    • A few other minor bug fixes.
     
    firstuser, petey and julianr like this.
  30. lorewap3

    lorewap3

    Joined:
    Jun 24, 2020
    Posts:
    58
    Hey @scottyboy805 !

    Been a while since I touched this code, so I thought I'd give it another shot with the recent updates you guys pushed out. It didn't work out the gate, but I didn't really expect it to given my unique use case. To expedite this process I repackaged the same test project I sent to you guys last time. Only this time it includes the ReplayBridge SO and edit-mode Timeline scrubbing functionality. It includes all the functionality I plan to use, just stripped down, so if we can get all of this working together and keep it working amidst future updates I'd be a very happy camper :) . I'm emailing the zipped project to you guys directly, I just wanted to include here as well for posterity and to help others.

    Here's a quick project overview.
    upload_2021-8-4_18-27-39.png
    Director Component - The Director component handles switching of modes. There's Rehearsal/Recording/Performance.
    Rehearsal - does nothing, does not invoke creating/replaying
    Recording - Upon entering Play mode, automatically creates a ReplayBridge SO for each scene (Scene1/Scene2 above) and .replay file for each and initiates recording. Upon exiting Play mode (in OnDisable) it ends recording.
    Performance - This serves multiple functions. In edit mode, it activates the edit mode timeline scrubbing. Upon clicking it it loads the replays and seeks to the time set by the Timeline playhead. If you enter play mode with this option selected, it will play the entire replay from the beginning (no matter where it was in edit mode)

    ReplayController Component - While the Director component is decider, this component handles all those decisions. It only uses OnEnable/OnDisable for the purpose of hooking into events and disposing of storage targets upon exiting 'Performance' state in edit mode. It contains the other functions called by the Director such as creating the SO/replay upon entering play mode. It contains the functions for starting and ending the replays. The custom PlayableBehaviors used in Timeline are very minimal and really only call the seek functions in this component.

    There's a few issues going on.

    1) Exiting playmode in Recording state - It appears to record the replays, but upon exiting play mode (causing the recording to end) I'm getting the infinite import loop error again. Actually, sometimes it happens in the middle of play mode as well.
    Code (CSharp):
    1. An infinite import loop has been detected. The following Assets were imported multiple times, but no changes to them have been detected. Please check if any custom code is trying to import them:
    2. Assets/ReplayFiles/Scene2_2021-08-04_181507.replay
    3. Assets/ReplayFiles/Scene1_2021-08-04_181551.replay
    4. Assets/ReplayFiles/Scene2_2021-08-04_181551.replay
    5. Assets/ReplayFiles/Scene1_2021-08-04_181507.replay
    The other error I get is, but not 100% of the time like the import error, is:
    Code (CSharp):
    1. ObjectDisposedException: Cannot access a disposed object.
    2. Object name: 'File target has been disposed'.
    3. UltimateReplay.Storage.ReplayFileBinaryTarget.CheckDisposed () (at Assets/Ultimate Replay 2.0/Scripts/Storage/File/ReplayFileBinaryTarget.cs:111)
    4. UltimateReplay.Storage.ReplayFileBinaryTarget.get_InitialStateBuffer () (at Assets/Ultimate Replay 2.0/Scripts/Storage/File/ReplayFileBinaryTarget.cs:50)
    5. UltimateReplay.ReplayManager.StopRecording (UltimateReplay.ReplayHandle& handle) (at Assets/Ultimate Replay 2.0/Scripts/ReplayManager.cs:582)
    6. LorewaPlay.ReplayController.EndRecording () (at Assets/Scripts/Replay/ReplayController.cs:201)
    7. LorewaPlay.Director.OnDisable () (at Assets/Scripts/Director.cs:138)
    8.  
    2) It's creating two for each Scene - Honestly I'm not convinced this is caused by your code or simply is a byproduct of the infinite import issue or a bug in my code. I'd be the first to admit if it was a bug in my code but I did have this system working perfectly in April with no duplications. If my code is the cause I can't see it.

    3) Performance mode - This doesn't work at all right now, though I suspect it's because of the recording issues above. I'm including here just so you guys remember to test it. In edit mode, clicking it should jump the characters to the position of the timeline playhead and allow scrubbing.

    If I'm doing anything dumb in my code please let me know. I thought I had it worked out pretty well, at least it used to work, but if you guys have any recommendations I'm all ears. This is the best approach I came up with to allow switching in edit mode and keeping the code relatively contained within only a few components.

    Note: I didn't apply that custom fix you provided me back in Feb. I assumed you guys had rolled it into the main codebase and published it by now.

    Thanks Scotty!
    Will
     
  31. scottyboy805

    scottyboy805

    Joined:
    Apr 10, 2013
    Posts:
    1,192
    Hi again,
    Thanks for sharing.
    I will certainly take a look at the project to see if those issues can be sorted.

    I am not sure about the importing loop, but I guess that maybe Unity does not like that the file will be adding data continuously. I guess this would trigger a refresh and reimport by Unity since the file has changed, but it will happen constantly while the replay system is recording. Perhaps the file should be saved outside of the 'Assets' folder (Maybe in '<projectfolder>/Temp), and then copied into assets when recording has finished. I think Unity would have less issues with that approach, since the file will not be continuously growing. I think it might not be a problem normally, but Ultimate Replay is using a background streaming thread to write the file, so there is no syncing between the import thread (unity) and the streaming thread that is continuously updating the replay file. I can see how that could be a potential issue and cause import issues like you mention, but further testing/debugging will hopefully tell for sure.

    As for the disposed exception: The replay system will automatically dispose of any non-disposed file targets when exiting play mode so that file handles are not left hanging causing locked files. My guess is that the automated dispose code is running before your manually cleanup code, so when you come to release the target, it has already been disposed. That might explain why it does not happen every time too, because Unity is not guaranteed to run scripts in the same order each time. I will look into it and test it out but I suspect that might be the issue.
     
  32. julianr

    julianr

    Joined:
    Jun 5, 2014
    Posts:
    1,212
    Good fixes! Any news on what we discussed previously with new demos?
     
  33. scottyboy805

    scottyboy805

    Joined:
    Apr 10, 2013
    Posts:
    1,192
    Hi,
    We are currently recreating the killcam demo using original assets so that we can include it directly in the package (Some of the assets used in that demo are licensed so we cannot include it on the asset store directly). Once we have finished that, we will then be working on the other demos we discussed, so hopefully not too long to wait.
     
    julianr likes this.
  34. jjobby

    jjobby

    Joined:
    Nov 28, 2009
    Posts:
    160
    Hello, does this plugin support WebGL?
     
  35. scottyboy805

    scottyboy805

    Joined:
    Apr 10, 2013
    Posts:
    1,192
    Hi,
    Yes, WebGL is indeed supported. The only thing to note is that replay files are not supported on that platform since it relies on IO and threading, which are problematic on WebGL. All other features work as expected though.
     
  36. julianr

    julianr

    Joined:
    Jun 5, 2014
    Posts:
    1,212
    Awesome! No hassle - just planning my time and these demos will save me a ton of time, as you know your product inside and out. Once I work on how it all works, I can modify it to my needs.
     
    scottyboy805 likes this.
  37. lorewap3

    lorewap3

    Joined:
    Jun 24, 2020
    Posts:
    58
    Hey @scottyboy805 ,

    I just wanted to check back in with you and see if you had a chance to look at my project. If you want any clarification on anything let me know. I don't want you to feel like I'm trying to offload the troubleshooting.

    I'll definitely modify my code to remove trying to dispose of the target upon exiting playmode. That will solve that issue. But that won't affect the import issue or the other errors I'm getting.

    Anyway, just wanted to check in. There's no huge rush, I just don't want it to fall into the cracks.

    Thanks!
    Will
     
  38. scottyboy805

    scottyboy805

    Joined:
    Apr 10, 2013
    Posts:
    1,192
    Ultimate Replay 2.1.7 has been released on the asset store. This version includes:
    • ReplayBehaviour will now derive from NetworkBehaviour when Mirror is installed.
    • Fixed a bug where disabled replay components could disappear from the observed components collection of the managing replay object when entering play mode.
    • Fixed a bug where an exception could be thrown when replay variables of type quaternion are used.
     
    julianr and firstuser like this.
  39. scottyboy805

    scottyboy805

    Joined:
    Apr 10, 2013
    Posts:
    1,192
    Hi Will,
    Yes, I had a quick look at the project and have some things to note. I wanted to take a more in-depth look though when I have chance, but here are some things I noticed:
    1. I think, but am not 100% sure that the naming of the assets inside the ReplayFiles folder may be causing issues relating to the import error. I was able to reproduce the import error you mentioned quite reliably to start with, but then I changed the save name of the 'ReplayBridge' scriptable asset to not share an identical name as the replay file (I just appended '-asset' to the end of the file name). After that I only got the import error once randomly, and have not seen it since. I think the importer may be struggling with identically named files, even though they have a different extension. So that is something you might like to try on your end.
    2. The dispose exception does indeed look like it is caused by the auto-cleanup system running before your game code has chance. You can see this because the cleanup system will log a warning when it disposes of a 'hanging' file stream, and this message appears in the console before the exception when you try to dispose manually. I think this is something we will need to fix on our end.
    3. I have noticed the ThreadAbortException on occasion when entering play mode (Appears to be very random). I have not looked into that much yet, but I think it can only be caused by one of two things: A file target being disposed while the streaming thread has a very high workload, or the Unity editor aborting threads due to an assembly reload or similar. I think the latter is probably more plausible, and probably more tricky to fix/workaround too.
     
  40. lorewap3

    lorewap3

    Joined:
    Jun 24, 2020
    Posts:
    58
    Hmm, that's interesting. I thought the GUIIDs were what Unity mostly went off of that's weird that a name clash would cause that. Regardless if that fixes the problem I will happily adapt.

    I know in the past I caused this issue with how I was 'inspecting' the replay files from within the SO itself. I had an editor window for inspecting ReplayBridges, and part of that was reading the replay file to display some general data such as replay object IDs, etc. This was definitely causing issues as I was doing this in editor mode and not properly disposing of them in a timely manner. But I intentionally gutted all of that code from the test project I sent you cause I wanted to ensure it wasn't the cause.

    That's easy enough, I'll just remove the disposal logic since it's doing it for me.

    Hmm, when you say assembly reload I think mainly recompilations but I'm inexperienced in that regard. In general I'm guessing all this has to do with how I'm entering/exiting playmode to start/stop recordings instead of doing it all in game like the normal use case. I'm not too knowledgable about when/why Unity does assembly reloads but I suppose entering/exiting playmode would be an appropriate time.

    Everytime I troubleshoot this I usually have to force a recompilation before I test again. I enter play mode, get the error, exit play mode, but then won't see the remaining exceptions (such as thread abort) until I force another recompilation. I assumed this was just the threads not being closed properly and them hanging around until another recompile forced their closure.

    At any rate, I appreciate your time looking at it. In the end I suppose I could add logic to start/stop record within play mode, but I was hoping to avoid it since my project is very editor focused. Again, no rush, as long as this just doesn't fall into the abyss.

    Thanks Scotty!
    Will
     
  41. scottyboy805

    scottyboy805

    Joined:
    Apr 10, 2013
    Posts:
    1,192
    No problem.
    I will see if I can look into it a bit more when I have chance.
    You are right that Unity uses guid's for everything so I cam not sure why the name would cause such issues. I just had a feeling when I noticed the identical names that it could potentially be causing an issue. My on guess is that the importer checks if files have changed by their file name, and since there were 2 files with identical names, it treated them as the same. I could be way off but it is all I have at the moment.
    Assembly reloads do indeed occur when entering and exiting play mode. There are also a few other things that cause it like changing some settings etc. I think Unity now has an option to disable domain reloads when entering play mode, so maybe that might be worth a try to make sure it is indeed the cause of the problem.
     
  42. scottyboy805

    scottyboy805

    Joined:
    Apr 10, 2013
    Posts:
    1,192
    Ultimate Replay 2.1.8 has been released on the asset store. This version includes a number of performance improvements relating to file streaming:
    • Optimized the file streaming thread to create less allocations.
    • Optimized file streaming chunks to be more performant during loading.
    • Optimized file streaming code to deserialize common replay types much more quickly (About 40% improvement indicated from tests)
    • Fixed a bug where the streaming thread could cause the main thread to wait for a small period of time even though no data was read.
     
    julianr and firstuser like this.
  43. jjobby

    jjobby

    Joined:
    Nov 28, 2009
    Posts:
    160
    Hi, I'd like to save replay into file and upload it to my server. But I would like to add some custom filed into it such as seed number or some start items as array or may be json object. Is this easy to do?
     
  44. scottyboy805

    scottyboy805

    Joined:
    Apr 10, 2013
    Posts:
    1,192
    Hi, There is no easy way to add custom metadata at the moment. It is possible, but I would call all of the solutions workarounds really. It may be better if you are able to upload a corresponding meta file with such information along with the replay file. If that is not possible, then here are a couple of ways you can do it:
    1. Create a custom replay component that serializes this custom data. A custom replay component is simply a script that derives from ReplayRecordableBehaviour, and implements the serialize and deserialize methods. Your script would need to be a little bit different though, because you would not want to serialize the same data on every sample, which can occur multiple times per frame depending on record FPS.
    2. Use the ReplayStreamTarget instead of the ReplayFileTarget and insert some data at the start of the stream. This is a little more advanced but allows you to package any custom data nicely into the stream. For example: You could open a file stream using a binary writer, serialize your data in some way, perhaps as json like you say, then pass the stream object to the replay system via the ReplayStreamTarget where it will happily append the rest of the recorded data.
    I can go into more detail and give examples of the above options if they would be useful to you. Just let me know.

    I hope that helps you. Let me know if you have any more questions or if there is anything else.
     
  45. jjobby

    jjobby

    Joined:
    Nov 28, 2009
    Posts:
    160
    Thank you for your answer. I'd like to ask your suggestion. My game is a Flappy Bird clone. There are some random elements like pipe height or position. I've used fix random seed so that the random number will be the same on that game session. But it also have a usable item. So, I will have to record when the player use each item too. Do you think that record the time since when the game has started is good enough? My game also modify Unity timescale to manipulate game speed. Will this have any problem with saving replay?
     
  46. scottyboy805

    scottyboy805

    Joined:
    Apr 10, 2013
    Posts:
    1,192
    Hi again,
    You will probably want to store the seed value then so that you can generate the same level in the replay. You can use any of the approaches I mention before to do that, but it is probably easiest to have a separate .json file or similar that is paired with your replay file.
    For usable items, you might want to take a look at replay events, as they will probably be the easiest way to record and replay when the player collects/uses an item. Essentially you will just call 'RecordEvent' method during your game when an item is used, and pass in a unique event id that you can allocate. Then when in playback mode, you can listen for events and react to them through code however you need to. There is more information in the user guide, but I can also provide examples if needed.
    The replay system does indeed use 'Time.deltaTime' for recording and replaying updates. If you reduce the time scale, it will record in slow motion and increasing has the opposite effect. I guess you will just be gradually increasing the value over time to increase game speed? In this case, I don't think it should be so much of an issue, but it is probably best to test it out.
     
  47. jjobby

    jjobby

    Joined:
    Nov 28, 2009
    Posts:
    160
    Back to this answer, my game is a WebGL. But I don't intend to replay the record on that platform. I only want to record player transform and upload the replay to the server. Is this still not supported?
     
  48. scottyboy805

    scottyboy805

    Joined:
    Apr 10, 2013
    Posts:
    1,192
    In that case, you may be able to use the RepayStreamTarget instead to store the data in a memory stream object. You can then upload that later on without needing to create a file, as that can be a bit of an issue on webgl I believe. I should also add that threading will be disabled on webgl since it is not supported, so the recording process may add lag to the game as it needs to perform the file operations on the main thread. It may not make for the best experience, but it would probably be best to try it out to see if it is manageable.
     
  49. hauwing

    hauwing

    Joined:
    Dec 13, 2017
    Posts:
    69
    Hi, I tried the ghost car demo, it works fine in both editor and builds. But as the scene get a little bit more complex by adding some rocks, trees etc or switching to HDRP, the playback started to struggle as the car keep jittering even in build. However, it moves smoothly during recording. The frame rate drops a bit during playback but still it can maintain 200+, however, jittering still occurs. Is there any way to improve the playback? I tried different settings but still no luck, thanks.
     
  50. scottyboy805

    scottyboy805

    Joined:
    Apr 10, 2013
    Posts:
    1,192
    Hi, Thanks for reporting that.
    I will look into it as soon as I can to see what the issue might be. I guess it could be related to the update method used (Fixed Update would be preferred since the player car has a rigid body) but I will check that on my end. I assume interpolation is still enabled for the player car and the ghost car when in playback mode?