Question Most efficient way to seek & render the first VALID frame of a video? In fewest frames possible...

I am trying to return a paused VideoPlayer to the first valid (renderable) frame of a video and output it to a RenderTexture. I have just been testing with MP4's downloaded from YouTube and TikTok. I am streaming them from a URL on my server in my test.

Perhaps a quirk is those videos have warnings in Unity like:

Code (csharp):

1. First video frame not zero: 2 (0.066667s). Result may be out of sync. Please make sure tracks all start at 0 in (videoname)

I have noticed some of the videos I have made myself with FFMPEG can also end up having quirks like this. So I need my method to work irrespective of this (can't depend on the videos being "perfect").

Here is the simplified code in a coroutine I am testing:

Code (csharp):

1.                         //ensure "playing"/"paused"
2.                         if (!videoPlayer.isPaused) {
3.                             videoPlayer.Play();
4.                             videoPlayer.Pause();
5.                             yield return null;
6.                         }
7.                             videoPlayer.targetTexture = renderingTexture;
8.  
9.               //SEEK TO FIRST FRAME
10.                         Debug.Log("CAN SET FRAME " + setVideoPlayerClass.videoPlayer.canSetTime + videoClass.videoName); //returns true always
11.                         bool seekDone = false;
12.                         videoPlayer.seekCompleted += delegate {
13.                             seekDone = true;
14.                         };
15.                         videoPlayer.time = 0;
16.  
17.                         while (!seekDone && !setVideoPlayerClass.videoPlayer.isPlaying) {
18.                             Debug.Log("WAITING FOR VIDEO PLAYER TO BE TIME 0 ");
19.                             yield return null;
20.                         }
21.  
22.                         //STEP FORWARD ONE FRAME
23.                         Debug.Log("STEP FORWARD START FRAME " + videoPlayer.frame + " is playing " + videoPlayer.isPlaying);
24.                         int stepForwardCount = 0;
25.                         while (stepForwardCount < 2) { //===CANNOT REDUCE THIS TO ONE - WILL FAIL SOMETIMES...
26.                             stepForwardCount++;
27.                             setVideoPlayerClass.videoPlayer.StepForward();
28.                             Debug.Log("STEP FORWARD" + setVideoPlayerClass.videoClass.videoName);
29.                             yield return null;
30.                         }
31.                         Debug.Log("STEP FORWARD FINISH FRAME " + videoPlayer.frame + " is playing " + setVideoPlayerClass.videoPlayer.isPlaying);
32.  

At all points the code seems to do what it's trying to but it is wasting at least one frame on stepping forward. For some reason, I must step forward twice. If I only step forward once, I get a near zero frame output at the end on MOST videos, but some will still leave this code with a frame of say 36 or any random higher number. Ie. the frame setting fails?

Without stepping forward at all, I don't get the RenderTexture updated or the output frame listed as correct.

I am trying to get this done obviously in the least lines of code possible.

Is the yield return null also needed after Play(); Pause(); at the beginning?

Any more correct or frame efficient way to do this? Right now, stepping forward twice solves the issue but if I get a more unusually coded video (I presume this is a side effect of some videos not starting at zero), maybe that won't. Maybe I'd need three or five or 10 steps.

If that is the likely issue, is there any reliable way of finding what the first frame is of a prepared video is so I can just go to that?

I also notice even with this method when it outputs that it has returned to say frame 2 or 3, the render texture is not updated until it starts truly playing later. This is very frustrating. It should still always be updated while paused and stepped forward. But yet it is still not in some cases.

Any solutions or ideas for any of this?

Thanks.

 

I am also having this issue with all the videos I have in Unity 2023.1.3f1. I tried toggling 'VideoPlayer.waitForFirstFrame' but the problem still persists.