Search Unity

Question Extrapolated clips and getting normalized time

Discussion in 'Timeline' started by dvuk89, May 18, 2020.

  1. dvuk89

    dvuk89

    Joined:
    May 11, 2020
    Posts:
    3
    There were talk in the past about pushing Clip from CreatePlayable on, so you can access proper clip start time and duration in ProcessFrame. Now CreatePlayable was reworked and I can't find a way to access those values.

    Was this simplified now? Or is it removed?

    The issue is with Extrapolated clips which returns playable.GetDuration() as Infinity and Start as 0;
     
  2. seant_unity

    seant_unity

    Unity Technologies

    Joined:
    Aug 25, 2015
    Posts:
    1,516
    It was? In what way? I don't think this has changed since the public release of timeline, but I could be wrong.

    We haven't made changes to push the clip down to the playable asset however. The workaround of assigning clips to the clip playable by overriding TrackAsset.CreateTrackMixer should still work, and in the case of extrapolation, you can use the track duration as a replacement for the end value of the last clip where the extrapolation gives you +infinity.
     
  3. havchr

    havchr

    Joined:
    Jun 18, 2009
    Posts:
    75
    I have been struggling with the same thing myself :
    I have found two different approaches - both feel like work-arounds , and I am quite new to the Timeline system, so I don't know if I'm doing unnecessary workarounds, or if this is the way you have to do what I am trying to do.

    In short - what I want to do :
    While playHead is inside a clip - I want a value t in range 0 to 1 , based on clip start and clip end.
    Code (CSharp):
    1. var myVal = behaviour.animCurve.Evaluate(t)*behaviour.myData;
    If playHead is past the clip end, I want to grab the value as given by
    Code (CSharp):
    1. var myVal = behaviour.animCurve.Evaluate(1.0)*behaviour.myData;

    Here is a video with both workarounds



    That far cleanest solution I have found - when it comes to MixerBehaviourCode is having access to the clip start and clip end : which I had to do by using the work-around described here :


    The mixer behaviour code becomes like this :
    Code (CSharp):
    1.    
    2. public void ProcessFrameClipPassthroughAndPlayHeadWorkAround(Playable playable, FrameData info, object playerData)
    3.     {
    4.         ScoreTallyMaster trackBinding = playerData as ScoreTallyMaster;
    5.  
    6.         if (!trackBinding)
    7.             return;
    8.  
    9.         int inputCount = playable.GetInputCount ();
    10.         double speed = 1.0;
    11.         float totalEndScore = 0f;
    12.         float currentAccumulatedScore = 0f;
    13.      
    14.         for (int i = 0; i < inputCount; i++)
    15.         {
    16.          
    17.             ScriptPlayable<ScoreTallyBehaviour> inputPlayable = (ScriptPlayable<ScoreTallyBehaviour>)playable.GetInput(i);
    18.             ScoreTallyBehaviour input = inputPlayable.GetBehaviour ();
    19.          
    20.             totalEndScore += input.GetScore();
    21.      
    22.             double playHeadTime = playable.GetGraph().GetRootPlayable(0).GetTime();
    23.             double duration = input.Clip.end - input.Clip.start;
    24.             double playHeadRelativeToClip = playHeadTime - input.Clip.start;
    25.             double tForClip = playHeadRelativeToClip / duration;
    26.             float floatingScore = input.GetScore() * input.easing.Evaluate((float)tForClip);
    27.  
    28.             if (playHeadTime > input.Clip.end)
    29.             {
    30.                 input.ScoreTallyPart.myScore.text = $"{Mathf.Floor(input.GetScore())}";
    31.                 currentAccumulatedScore += (input.GetScore());
    32.             }
    33.             else if (playHeadTime < input.Clip.start)
    34.             {
    35.                 input.ScoreTallyPart.myScore.text = $"0";
    36.             }
    37.             else
    38.             {
    39.                 speed = SpeedFromScore(input.EditorPreviewScore);
    40.                 currentAccumulatedScore += (floatingScore);
    41.                 input.ScoreTallyPart.myScore.text = $"{Mathf.Floor(floatingScore)}";
    42.             }
    43.  
    44.         }
    45.      
    46.         if (trackBinding.Total!=null)
    47.         {
    48.             trackBinding.PublishScore(currentAccumulatedScore,totalEndScore);
    49.         }
    50.         playable.GetGraph().GetRootPlayable(0).SetSpeed(speed);
    51.     }
    52.  


    The other solution I found to get this working foregoes the clip start/end workaround, but uses a
    "workAroundForHold" boolean in the behaviour - this is turned on on clips that has to fill the gaps between the real clips.
    The mixerBehaviour for this method is weirder by magnitudes...


    Code (CSharp):
    1.  
    2. public void ProcessFrameHoldWorkAround(Playable playable, FrameData info, object playerData)
    3.     {
    4.         ScoreTallyMaster trackBinding = playerData as ScoreTallyMaster;
    5.  
    6.         if (!trackBinding)
    7.             return;
    8.  
    9.         int inputCount = playable.GetInputCount ();
    10.         float totalWeight = 0.0f;
    11.         float floatingScore = 0f;
    12.         double speed = 1.0;
    13.         float totalEndScore = 0f;
    14.         float currentAccumulatedScore = 0f;
    15.  
    16.         /*
    17.          * We are counting the clips backwards for the reasons below..
    18.          * if we encounter clip 4 with weight > 0 , we
    19.          * want all clips prior to clip 4 to be at considered as weight 1
    20.          * When we are inside a clip - we want the local time t in 0-1
    21.          * to use with our own AnimationCurve to set easing for our clips.
    22.          */
    23.      
    24.         int playHeadPastClipIndex = -1;
    25.         for (int i = inputCount-1; i >= 0; i--)
    26.         {
    27.          
    28.             ScriptPlayable<ScoreTallyBehaviour> inputPlayable = (ScriptPlayable<ScoreTallyBehaviour>)playable.GetInput(i);
    29.             ScoreTallyBehaviour input = inputPlayable.GetBehaviour ();
    30.          
    31.             bool pastClip = false;
    32.             float inputWeight = playable.GetInputWeight(i);
    33.             if (inputWeight > Double.Epsilon)
    34.             {
    35.                 playHeadPastClipIndex = i;
    36.             }
    37.             if (i < playHeadPastClipIndex)
    38.             {
    39.                 pastClip = true;
    40.             }
    41.  
    42.             /* I was unable to get previous values to hold and still get a t=0-1 range with GetTime and GetDuration
    43.              * when there were gaps in the timeline track.
    44.              *
    45.              * I ended up with a pragmatic workaround where a "hold"-clip is inserted instead of gaps in the timeline
    46.              * If the playhead is at a hold clip, just continue the loop
    47.              */
    48.             if (input.workAroundForHold)
    49.             {
    50.                 continue;
    51.             }
    52.             if (inputWeight > Double.Epsilon)
    53.             {
    54.                 speed = SpeedFromScore(input.EditorPreviewScore) * inputWeight;
    55.             }
    56.             totalEndScore += input.GetScore();
    57.             totalWeight += inputWeight;
    58.          
    59.             double timeForInput = playable.GetInput(i).GetTime();
    60.             double durationForInput = playable.GetInput(i).GetDuration();
    61.             float t = (float) (timeForInput/durationForInput);
    62.             floatingScore = input.GetScore() * input.easing.Evaluate(t);
    63.  
    64.             if (pastClip)
    65.             {
    66.                 input.ScoreTallyPart.myScore.text = $"{Mathf.Floor(input.GetScore())}";
    67.                 currentAccumulatedScore += (input.GetScore());
    68.             }
    69.             else if (inputWeight > Double.Epsilon)
    70.             {
    71.                 currentAccumulatedScore += (floatingScore);
    72.                 input.ScoreTallyPart.myScore.text = $"{Mathf.Floor(floatingScore)}";
    73.             }
    74.             else if (inputWeight < Double.Epsilon)
    75.             {
    76.                 input.ScoreTallyPart.myScore.text = $"0";
    77.             }
    78.  
    79.         }
    80.      
    81.         if (trackBinding.Total!=null)
    82.         {
    83.             if (totalWeight > Mathf.Epsilon || playHeadPastClipIndex!=-1)
    84.             {
    85.                 trackBinding.PublishScore(currentAccumulatedScore,totalEndScore);
    86.             }else if (totalWeight < Mathf.Epsilon)
    87.             {
    88.                 trackBinding.PublishScore(0f,totalEndScore);
    89.             }
    90.         }
    91.      
    92.         playable.GetGraph().GetRootPlayable(0).SetSpeed(speed);
    93.     }

    I would love to know if there is a better way to do things?
    If there isn't a better way, I think I would go for the the Clip-passthrough technique, because that code is just so much easier to read and understand.
     
  4. seant_unity

    seant_unity

    Unity Technologies

    Joined:
    Aug 25, 2015
    Posts:
    1,516
    There are some built-in timeline features that may help here. Animated parameters and extrapolation,

    Instead of storing an animation curve, if you use an animated parameter like in the script below, your animated value is automatically updated before ProcessFrame is called.

    This will also expose the extrapolation options in the clip editor, which will allow the user to say how the gap gets filled (hold, loop, etc...) like the animation track.

    Code (CSharp):
    1. using System;
    2. using UnityEngine;
    3. using UnityEngine.Playables;
    4. using UnityEngine.Timeline;
    5.  
    6. [Serializable]
    7. public class MyPlayableAsset : PlayableAsset, ITimelineClipAsset
    8. {
    9.     [Serializable]
    10.     public class MyBehaviour : PlayableBehaviour
    11.     {
    12.         public float AnimatedValue = 0;
    13.     }
    14.  
    15.     // this allows the editor to animate any parametes in the behaviour
    16.     public MyBehaviour template = new MyBehaviour();
    17.    
    18.     public override Playable CreatePlayable(PlayableGraph graph, GameObject go)
    19.     {
    20.         return ScriptPlayable<MyBehaviour>.Create(graph, template);
    21.     }
    22.    
    23.     // this tells timeline to support extrapolation in gaps
    24.     public ClipCaps clipCaps
    25.     {
    26.         get { return ClipCaps.Extrapolation; }
    27.     }
    28. }

    upload_2020-5-29_11-13-6.png
     
  5. havchr

    havchr

    Joined:
    Jun 18, 2009
    Posts:
    75
    But once you get to the next clip after the extrapolation - you would no longer get the extrapolated values from the previous clip - or am I mistaken?
     
  6. ayellowpaper

    ayellowpaper

    Joined:
    Dec 8, 2013
    Posts:
    52
    So I'm reviving this thread because the concrete answer to the question is still missing.
    I would like to get the normalized time from the playable when using extrapolation, but for that I need the "actual duration". GetDuration() is already nicely adjusted accordingly, returning looping and wrapping values, so the "actual duration" has to exist somewhere. How can I get the normalized value?
     
  7. havchr

    havchr

    Joined:
    Jun 18, 2009
    Posts:
    75
    I think your best bet is this workaround - https://forum.unity.com/threads/how-to-access-the-clip-timing-start-end-time-in-playablebehaviour-functions.494344/#post-3861688