Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Video Video - seeking (scrubbing) performance on Windows

Discussion in 'Audio & Video' started by ccklokwerks, Apr 9, 2018.

  1. ccklokwerks

    ccklokwerks

    Joined:
    Oct 21, 2014
    Posts:
    62
    Hi all,

    I haven't been able to find any posts about this but I'm wondering if anyone else has encountered this.

    I have a paused video with input to scrub back and forth. In the editor or in a build in OSX, performance is all I could ask for. In Windows, on the other hand, scrubbing performance is really, incredibly slow

    To be specific, the scrubbing works by responding to a drag (every frame) which ends up setting "frame" on the video. I assume it's possible for a new assignment to occur before the previous seek completed.
     
  2. bgUnity1

    bgUnity1

    Joined:
    Nov 15, 2017
    Posts:
    32
    What exactly is the issue on Windows? It doesn't update very frequently?
     
  3. ccklokwerks

    ccklokwerks

    Joined:
    Oct 21, 2014
    Posts:
    62
    VERY infrequently. On OSX, a drag produces a nice smooth transition through the video. On Windows I would be surprised to see an update more than once second, if that.

    I was thinking of making a test project to demonstrate the issue.
     
  4. bgUnity1

    bgUnity1

    Joined:
    Nov 15, 2017
    Posts:
    32
    Code (CSharp):
    1. if (progressBarSlider)
    2.         {
    3.             //if the progress bar is not being adjusting then set the value to the current video's time.
    4.             if (!adjustingProgressBar)
    5.             {
    6.                 //Constantly update the value of the slider to be exact with the current frame of the video
    7.                 progressBarSlider.value = (float)videoPlayer.time;
    8.             }
    9.             else
    10.             {
    11.                 //Otherwise if it is being adjusted then set the video's time to the value of the Progress Bar
    12.                 videoPlayer.time = progressBarSlider.value;
    13.             }
    14.             //Only run this block of code if the progress bar is togglable during runtime
    15.  
    16.         }
    Is the code that I use in my update function that is activated whenever I am currently seeking using the progress bar. Maybe it is me setting the Time instead of the Frame? I have not tested on other OS aside from Windows so I don't know if it will be better or worse. For me on Windows, I mean it isn't exactly "frequent" but it is not exactly "Infrequent" also. Definitely isn't bad enough to be noticable.

    Could it also be that you only update the video on letting go of the scrubber? Mine runs anytime that I am currently seeking using the scrubber.

    The other thing to check for is how are you transcoding your videos?
    I know that for my MP4s, they take an extra second to load when I start the video, but if I transcode them to H264 then they load a lot quicker.

    If none of those work then feel free to upload a sample project and I'll check it out for you!
     
  5. jestermaximusJr

    jestermaximusJr

    Joined:
    Jan 12, 2013
    Posts:
    14
    Was a solution ever found for this? I don't know much about video, but I'm using .mov/H264 videos and I'm experiencing the same "lag" as the OP. Setting the player.time parameter updates the displayed video frame on osx immediately, but that same update on Windows takes well over a second.
     
  6. Vincent_Lo

    Vincent_Lo

    Joined:
    Sep 20, 2013
    Posts:
    3
    I have the same problem. The video is not updating immediately. Frames are dropped while scrubbing. I'm using Windows 10 and Unity 2017.3.1f1.

    I've tried:
    1. Adding the video into the Preloaded Assets array in the Player Settings under Other Settings
    2. Using the method Prepare() on Start
    3. Transcoding the video
    4. All sorts of combinations of above

    My code is identical to Bcsproperties, and I've tried setting the video player's .time and .frame properties to see if that made any difference. No luck.

    I would think Prepare() would do the trick as documentation says "After this is done, frames can be received immediately," but frames were still dropped when scrubbing.
     
  7. Edgyr

    Edgyr

    Joined:
    Nov 20, 2013
    Posts:
    5
    I had something similar and I found out (with the help of a collegue) that when you do this:

    Code (CSharp):
    1. videoPlayer.time = progressBarSlider.value;
    It doesn't automaticaly set the time at that position. It needs to load that frame first, then show it. If you keep changing that value, it never finish loading a frame and it's never able to show one. Until you stop moving your scrubber, then it loads and show it.

    Here's how I fixed this:

    Code (CSharp):
    1.  
    2.     // Have a boolean that blocks time changes until one is loaded
    3.     private bool m_Lock = false;
    4.  
    5.     private void OnScrubberValueChanged(float aValue)
    6.     {
    7.             if (!m_Lock)
    8.             {
    9.                 // I use frames not time, sorry
    10.                 long tFrameIndex = (long)(aValue * m_VideoPlayer.frameCount);
    11.                 StartCoroutine(SeekVideoAtFrame(tFrameIndex));
    12.             }
    13.     }
    14.  
    15.     private IEnumerator SeekVideoAtFrame(long aFrame)
    16.     {
    17.         // That coroutine will lock other calls until the frame/time is set for real.
    18.         m_Lock = true;
    19.         m_VideoPlayer.frame = aFrame;
    20.  
    21.         yield return new WaitUntil(() =>
    22.         {
    23.             if (m_VideoPlayer.frame == aFrame)
    24.             {
    25.                 return true;
    26.             }
    27.             else
    28.             {
    29.                 return false;
    30.             }
    31.         });
    32.  
    33.         m_Lock = false;
    34.     }
    This improved my video controls a lot.
     
    romi-fauzi, dpotuznik and ccklokwerks like this.
  8. ccklokwerks

    ccklokwerks

    Joined:
    Oct 21, 2014
    Posts:
    62
    Well, I finally decided to take a look at this again and I can't say that I really I got better results.

    I didn't actually have luck with Edgyr's idea of using a coroutine - I used the callback from the video player to detect when the seek completed. And while seeking/scrubbing, I stored any additional request to scrub in a separate variable to execute a new seek at the end of the active one.

    HOWEVER, with a 1920x1080 video the performance was just unacceptably slow, and I'm not sure why at the moment.

    One other thing I'll note is that the video player started reporting back a frameCount of 0 which was messing up subsequent seeks. That's why I tried storing the frameCount in numFrames.

    Code (CSharp):
    1.  
    2.     private void VideoPlayerStarted(VideoPlayer source)
    3.     {
    4.         videoPlayer.started -= VideoPlayerStarted;
    5.         this.numFrames = (float)videoPlayer.frameCount;
    6.     }
    7.  
    8.     private float numFrames;
    9.     private bool scrubbing = false;
    10.     private bool pausedBeforeScrub = false;
    11.     private long targetScrubFrame;
    12.     private long nextScrubFrame = -1;
    13.  
    14.  
    15.     public void Seek(float position)
    16.     {
    17.         if (this.numFrames <= 0f)
    18.         {
    19.             // still not set.
    20.             this.numFrames = (float)videoPlayer.frameCount;
    21.         }
    22.  
    23.         long newFrame = (long)(this.numFrames * position);
    24.  
    25.         if (newFrame == this.targetScrubFrame)
    26.         {
    27.             // nothing to do
    28.             return;
    29.         }
    30.  
    31.         if (this.scrubbing)
    32.         {
    33.             this.nextScrubFrame = newFrame;
    34.             return;
    35.         }
    36.        
    37.         this.targetScrubFrame = newFrame;
    38.         this.scrubbing = true;
    39.         this.videoPlayer.Pause();
    40.  
    41.         videoPlayer.frame = newFrame;
    42.     }
    43.  
    44.     void VideoPlayer_SeekCompleted(VideoPlayer source)
    45.     {
    46.         if (this.nextScrubFrame >= 0)
    47.         {
    48.             videoPlayer.frame = this.nextScrubFrame;
    49.             this.nextScrubFrame = -1;
    50.             return;
    51.         }
    52.  
    53.         this.scrubbing = false;
    54.     }
    55.  
     
  9. kittrick

    kittrick

    Joined:
    Jul 10, 2019
    Posts:
    3
    I had decent luck with Edgyr's solution. There were certain situations when the frame lock got stuck, especially when the user adjusted the time value frequently. This was my solution. It's not short but it worked pretty well and had fairly smooth scrubbing. Hopefully this helps someone out.

    Code (CSharp):
    1. using System.Collections;
    2. using UnityEngine;
    3. using UnityEngine.Video;
    4.  
    5. public class VideoTimeControl : MonoBehaviour
    6. {
    7.     public  VideoPlayer vp;
    8.     [Range(0, 1)] public float targetTime;
    9.     private float targetTimeCache;
    10.     [Range(0, 1)] public float seekTime;
    11.     [Range(0, 1)] public float actualVideoTime;
    12.     private float velocity;
    13.     public bool seeking = false;
    14.     private Coroutine currentCoroutine;
    15.     private Coroutine lockTimeout;
    16.     public bool timeout;
    17.  
    18.     public bool whileLoopRunning = false;
    19.  
    20.     // Have a boolean that blocks time changes until one is loaded
    21.     public bool locked = false;
    22.  
    23.     // Start is called before the first frame update
    24.     void Start()
    25.     {
    26.         vp = GetComponent<VideoPlayer>();
    27.         vp.Prepare();
    28.         vp.skipOnDrop = true;
    29.     }
    30.  
    31.     // Update is called once per frame
    32.     void Update()
    33.     {
    34.         if (Mathf.Abs(targetTime - targetTimeCache) > 0.01f)
    35.         {
    36.             seeking = true;
    37.             targetTimeCache = targetTime;
    38.             if (currentCoroutine != null)
    39.             {
    40.                 StopCoroutine(currentCoroutine);
    41.                 locked = false;
    42.             }
    43.         }
    44.        
    45.         if (seeking)
    46.         {
    47.             // I use frames not time, sorry
    48.             seekTime = Mathf.SmoothDamp(seekTime, targetTime, ref velocity, 1f);
    49.             long seekFrame = (long) (seekTime * vp.frameCount);
    50.             if (!locked)
    51.             {
    52.                 currentCoroutine = StartCoroutine(SeekVideoAtFrame(seekFrame));  
    53.             }
    54.  
    55.             if (Mathf.Abs(targetTime - seekTime) < 0.01f)
    56.             {
    57.                 seeking = false;
    58.             }
    59.         }
    60.         else
    61.         {
    62.             if (!vp.isPlaying)
    63.             {
    64.                 vp.Play();
    65.             }
    66.         }
    67.  
    68.         actualVideoTime = (float)(vp.time / vp.length);
    69.     }
    70.  
    71.     private IEnumerator SeekVideoAtFrame(long aFrame)
    72.     {
    73.         // That coroutine will lock other calls until the frame/time is set for real.
    74.         locked = true;
    75.         vp.frame = aFrame;
    76.         timeout = false;
    77.         if (lockTimeout != null)
    78.         {
    79.             StopCoroutine(lockTimeout);  
    80.         }
    81.         lockTimeout = StartCoroutine(LockTimeoutRoutine(.1f));
    82.         yield return new WaitUntil(() =>
    83.         {
    84.             whileLoopRunning = true;
    85.             if (vp.frame == aFrame || timeout)
    86.             {
    87.                 return true;
    88.             }
    89.             else
    90.             {
    91.                 return false;
    92.             }
    93.         });
    94.         whileLoopRunning = false;
    95.         locked = false;
    96.     }
    97.  
    98.     private IEnumerator LockTimeoutRoutine(float timer)
    99.     {
    100.         yield return new WaitForSeconds(timer);
    101.         timeout = true;
    102.     }
    103. }