Search Unity

  1. We are migrating the Unity Forums to Unity Discussions. On July 12, the Unity Forums will become read-only. On July 15, Unity Discussions will become read-only until July 18, when the new design and the migrated forum contents will go live. Read our full announcement for more information and let us know if you have any questions.

Bug Timeline Markers Do Not Fire Reliably in Low FPS Situations

Discussion in 'Timeline' started by Ferazel, Nov 21, 2020.

  1. Ferazel


    Apr 18, 2010
    I've run into a problem that I'd like to see if I can get visibility on that appears to be a timeline bug. We have customized markers in our game that signal a timeline to pause so that we can display narrative bubbles that the user has unlimited time to read and dismiss before the timeline continues.

    We ran into a problem though where even though we followed the recommendation here: Where we scan the marker's time field and set the duration of the playable to a hard value.

    Unity 2020.1.14f1
    TimelinePackage 1.4.4

    I submitted bug# 1294256 but I'm wondering if anyone has experienced this? It appears that this primarily appears in low frame rate situations, but we need timelines to be deterministic even in low frame rate situations. It basically destroys our cutscene system because we cannot deterministically know how many stop points there are going to be in a conversation.

    @seant_unity if you have any insight I'd be happy to discuss aspects of the bug report here.
    Last edited: Nov 21, 2020
  2. seant_unity


    Unity Technologies

    Aug 25, 2015
    I just had a peek at the bug. I haven't gone though it in depth, so I may have missed something obvious. However, I noticed that you are using the dialog markers on nested timelines. I suspect that's where the problem lies. A control track creates a playable that tries to keep the clip and the nested playable director in sync, which may be conflicting with trying to hold the time.

    Do you know if this is an issue if you place all dialogue markers on the master timeline?
  3. Ferazel


    Apr 18, 2010
    You are correct that it works correctly if all of the markers are on the master timeline.

    However, that is not the workflow that our animators wanted. We split our cutscenes into separate directors for the different "chunks" so that they can retime sequences that sometimes have dozens of tracks and a reduction of conflicts as they can work on the same cutscene at the same time (as long as they work on different chunks). So they want to keep the markers on the lowest-level timeline as that is closest to the individual curves/tracks. They also still want the ability to scrub the entire timeline so that is the purpose of the parent timeline.

    Is this child control track marker skipping something that you can fix?
  4. seant_unity


    Unity Technologies

    Aug 25, 2015
    We probably can't fix that unfortunately, or least not without breaking something else that relies on subtimelines staying in sync with where the parent needs them to be.

    However, it may be something that you can account for. If you gather all the markers relative to the root level timeline, and only control time using the master timeline.

    Here's a quick example of how to get all the markers with their time relative to the root that should get you started if you decide to go that route.

    Code (CSharp):
    1.        struct GlobalMarker
    2.         {
    3.             public IMarker marker;
    4.             public double time;
    5.         }
    7.         static void GetAllMarkers(PlayableDirector director, List<GlobalMarker> markers)
    8.         {
    9.             var timeline =  director.playableAsset as TimelineAsset;
    10.             if (timeline == null)
    11.                 return;
    13.             // grab all the markers on the marker track
    14.             // if all timeline markers are required, loop over each track and do the same
    15.             if (timeline.markerTrack != null)
    16.             {
    17.                 foreach (var m in timeline.markerTrack.GetMarkers())
    18.                 {
    19.                     markers.Add(new GlobalMarker() {marker = m, time = m.time});
    20.                 }
    21.             }
    23.             // search control tracks for nested timelines
    24.             foreach (var track in timeline.GetOutputTracks().OfType<ControlTrack>())
    25.             {
    26.                 foreach (var clip in track.GetClips())
    27.                 {
    28.                     var playableAsset = (ControlPlayableAsset)clip.asset;
    29.                     // may also have to check the prefab gameObject
    30.                     var gameObject = playableAsset.sourceGameObject.Resolve(director);
    31.                     if (gameObject == null)
    32.                         continue;
    34.                     // assumes searchHierarchy is off and controlDirectors is on
    35.                     var subDirector = gameObject.GetComponent<PlayableDirector>();
    36.                     if (subDirector == null)
    37.                         continue;
    39.                     var childMarkers = new List<GlobalMarker>();
    40.                     GetAllMarkers(subDirector, childMarkers);
    42.                     // copy to the parent timeline, transforming time from the child timeline to
    43.                     // the parent timeline
    44.                     foreach (var childMarker in childMarkers)
    45.                     {
    46.                         // remove markers not in range of the clip
    47.                         var newTime = (childMarker.time - clip.clipIn) / clip.timeScale + clip.start;
    48.                         if (newTime < clip.start || newTime >= clip.end)
    49.                             continue;
    51.                         markers.Add(new GlobalMarker()
    52.                         {
    53.                             marker = childMarker.marker,
    54.                             time = newTime
    55.                         });
    56.                     }
    57.                 }
    58.             }
    59.         }
    This assumes you are not trying to pause only the nested timeline but the entire sequence. To pause only the nested sequence would require new track, clip, and behaviours to replace control tracks.
  5. Ferazel


    Apr 18, 2010
    As a user, I expect that the timeline markers to be able to be deterministic regardless if the marker is located in a control track or at the master track level regardless of framerate. If I'm understanding correctly, you're saying that this is a bug that you will not fix and that I should take ownership of firing all the notifications by parsing the notifications manually by doing a query of all of the markers from the master/control tracks.

    If true, I'm disappointed that this is your recommended fix. I feel bad for the users who will miss this thread and will depend on this behavior and end up shipping buggy software as a result. I'd recommend investing the time to have the timeline manage their markers deterministically or at least notify the users that markers at the control level will not fire reliably. Leaving this unintended behavior of the timeline sometimes firing markers is not why I spent hours of my day to track down the issue and to submit a bug report.
  6. seant_unity


    Unity Technologies

    Aug 25, 2015
    Triggering of markers is deterministic. Even in subTimelines. The issue isn't that. The problem is having two separate agents trying to manipulate time on the nested timeline playableDirector. Control tracks run sub-timelines by controlling time on the nested PlayableDirector. The workaround to not make time jump past the marker conflicts with that. When the playable director doesn't advance due to a marker, the parent timeline pulls the time ahead anyway to keep them in sync. That is the root cause. There isn't a bug to be fixed in the control track - it is doing exactly what it is supposed to.

    So the suggestion is to not manipulate the playableDirector of the nested timeline, but only to ever manipulate the root playableDirector time (or hold state).

    However, if I understand the problem, you'd like markers on the nested timelines to cause the master timeline to pause. So the suggestion I gave is to only control the master timeline, but gather all the markers with their times relative to that.

    If I misunderstood the problem, feel free to clarify.

    The initial workaround to hold the time is exactly that, a workaround. The suggestion is simply a way to work within those constraints.
  7. Ferazel


    Apr 18, 2010
    Well, I guess we're both saying it's a condition of the code failing, but you're saying it's working as intended and I'm saying it's a bug. Maybe there's a terminology difference, but a subtimeline is an embedded timeline via a control track right? So in my bug, a subtimeline's markers are not deterministically firing. It is skipping the firing of the markers when the parent director pulls the subdirector forward unless there's a frame before the marker on that sub-timeline. Is there some way of setting up an embedded timeline without a director target via control track that you're saying would work that is exposed via the GUI that animators can use?

    Since you seem reluctant to acknowledge this as an issue you want to fix. I went through and queried the markers and am firing them myself from the master director and that seems to work. However, I'm disappointed that this is yet another hoop that you expect your users to jump through to get around timeline code that isn't handling low framerate cases correctly. This issue on top of the behavior of letting a timeline director go past markers that should pause the timeline are two areas I wish I didn't have to manually handle in our code. Both of these scenarios IMO are bugs and can cause a lot of unintended bugs in user code that rely on the markers firing in a specific way.
  8. seant_unity


    Unity Technologies

    Aug 25, 2015
    First off, I'm happy you were able to get it working.

    Not at all. I suggest solutions that you can apply immediately to unblock you and your team without waiting for a fix. Especially if a fix requires adding an API, checkbox or anything that is not strictly a bug fix, because those are required to go in the next version and do not get backported. In your case, a change in behaviour is required and, in order to not break existing users, it would need to be opt-in.

    As well, our plans contain better solutions to this issue. For example
    • The APIs required to implement nested timelines already exists. Where control tracks are too generic or don't work for specialized use cases, it is possible (albeit not trivial) to implement different solutions.
    • That was a precursor to allow us to split control tracks into more specialized tracks (nesting, particles, etc...) that solve the specified use cases better. Your case requires a better nesting solution for timelines, one that has a single director driving everything.
    • Adding better PlayableDirector Update modes. More/better options for control over the source clock/time driving the timeline is a much better solution here. This goes to the heart of the issue here - the hold time workaround and giving better solutions.

    Anyway, I hope that helps. Dialogue systems are a very common thing users try to do with Timeline - given that timeline was designed as more of a cut-scene/cinematic tool, it does take some creative license to bend Timeline to that use case.
  9. GrooveJones


    Aug 19, 2016
    8 months later I can confirm this is still an issue. We are working with nested timelines much the way you would with prefabs. The master timeline runs the whole game and the sub timelines process smaller sections of the game. We have signal receivers on the sub timelines that are not firing, particularly if they are on or close to frame 0 in the sub timeline.

    Now all of our timelines are updating on GameTime and none of them are set to play on awake. The strange thing is this isn't random or occasional, its happening every time we play through the game. The signals on the first few frames of the timelines are not firing.
  10. Oneiros90


    Apr 29, 2014
    I want to share my solution to this problem:
    1. Attach the provided TimelinePausesManager to the PlayableDirector
    2. Change your signals to be PauseSignals in your timeline
    The timeline will pause automatically and accurately to those signals.
    Code (CSharp):
    1. using System.Collections.Generic;
    2. using System.Linq;
    3. using UnityEngine;
    4. using UnityEngine.Events;
    5. using UnityEngine.Playables;
    6. using UnityEngine.Timeline;
    8. namespace Utils
    9. {
    10.     // Special signal used to pause the timeline precisely
    11.     public class PauseSignalEmitter : SignalEmitter { }
    13.     // Attach this component to the PlayableDirector in order to correctly setup the pauses
    14.     [RequireComponent(typeof(PlayableDirector))]
    15.     public class TimelinePausesManager : MonoBehaviour
    16.     {
    17.         private PlayableDirector _director;
    18.         private List<PauseSignalEmitter> _markers;
    20.         private void Awake()
    21.         {
    22.             _director = GetComponent<PlayableDirector>();
    23.             _markers = GetAllMarkers<PauseSignalEmitter>(_director);
    25.             if (_markers.Count > 0)
    26.                 SetupPauses();
    27.         }
    29.         private void SetupPauses()
    30.         {
    31.             // Update timeline duration now and on any play/pause events
    32.             UpdateTimelineDuration();
    33.             _director.played += _ => UpdateTimelineDuration();
    34.             _director.paused += _ => UpdateTimelineDuration();
    36.             // Make sure that the signal receiver exists
    37.             var signalReceiver = GetComponent<SignalReceiver>();
    38.             if (signalReceiver == null)
    39.                 signalReceiver = gameObject.AddComponent<SignalReceiver>();
    41.             // Pause the timeline automatically on each pause marker
    42.             foreach (var marker in _markers)
    43.             {
    44.                 var reaction = signalReceiver.GetReaction(marker.asset);
    45.                 if (reaction == null)
    46.                 {
    47.                     reaction = new UnityEvent();
    48.                     signalReceiver.AddReaction(marker.asset, reaction);
    49.                 }
    50.                 reaction.AddListener(_director.Pause);
    51.             }
    52.         }
    54.         private void UpdateTimelineDuration()
    55.         {
    56.             // Find the next pause marker that will be hit
    57.             var nextMarker = _markers
    58.                 .Where(m => m.time > _director.time)
    59.                 .FirstOrDefault();
    61.             // Force the timeline duration to end exactly on that marker
    62.             // (or to the original duration if there are no pause markers ahead)
    63.             var nextMarkerTime = nextMarker != null ? nextMarker.time : _director.duration;
    64.             _director.playableGraph.GetRootPlayable(0).SetDuration(nextMarkerTime);
    65.         }
    67.         /// Find all markers of a specified type
    68.         private static List<T> GetAllMarkers<T>(PlayableDirector director) where T : IMarker
    69.         {
    70.             List<T> markers = new();
    72.             var timeline = director.playableAsset as TimelineAsset;
    73.             if (timeline == null)
    74.                 return markers;
    76.             if (timeline.markerTrack == null)
    77.                 return markers;
    79.             foreach (var marker in timeline.markerTrack.GetMarkers())
    80.                 if (marker is T tMarker)
    81.                     markers.Add(tMarker);
    83.             return markers;
    84.         }
    85.     }
    86. }
  11. daveMennenoh


    May 14, 2020
    Has anyone tried this solution from @Oneiros90
    I'm not using nested timelines but the playhead is sometimes stopping up to 8 frames after the pause signal. I'm hoping this will work but I don't get step 2 - Change your signals to PauseSignals. Like name them PauseSignals? I don't see where the code looks for anything named PauseSignal... and I have many other signals. If anyone has some clarification on implementing this, it'd be appreciated.
    inteligen likes this.