[Released] AVPro Video - complete video playback solution

Thanks for reporting this. This isn't something we've seen before, and you're right that changing the scene should have the same effect as destroying it manually...one would think.

We will try to reproduce it so that it can be fixed. Which version of Unity were you using?

Thanks,

 

Does this only happen with a particular video, or do you get the same thing if you use one of our sample videos?

Which version of Unity and AVPro Video are you using?

Does it never get the unstalled event? Does it throw the stalled event continuously, or intermittently? Perhaps you could give more detail on the exact behaviour.

The logic for this is in BasePlayer.cs, around line 212 in a method called IsPlaybackStalled. Here it will consider playback stalled if it hasn't received a new frame while playing for 0.75 seconds.

I can't replicate this behaviour so it would help if you could provide answers to all fo these questions.

Thanks,

 

I think this may be because your Unity has the colorspace set to "Linear". Try set it back to "Gamma", this can be doing in Player Settings.

I hope this solves it for you. Otherwise perhaps you could send screenshots and a copy of your video (www.renderheads.com/contact/).

Thanks,

 

So if I understand correctly: it's working fine in the Unity Editor (Windows), but when you build for Android then it is not able to continue buffering during playback?

And this seems to be happening because you have two MediaPlayers - one for audio and one for video? Or do you have one MediaPlayer which plays and audio file, and then once it has completed it then loads a video?

Thanks,

 

Hmm I'm not sure why you're getting that problem. But try adding the MediaPlayer framework to the build phases. It seems like somehow this dependency hasn't been picked up.

Thanks,

 

Thanks for reporting this. Yes, this is because the OES shader only support OES textures, which Unity Textures are not. OES is a special case optimisation so it doesn't support all the usual things.

In this case you would have to do something like hide the sphere and display your texture yourself manually, until the video has loaded, and then show the sphere and hide the object showing your texture. Or disable the ApplyToMesh and just apply your texture manually to the mesh using a different material/shader. Then once the video has loaded, switch the material/shader back to the OES one and enable ApplyToMesh.

 

Hi,

That sounds very strange. Do you get any audio playback from the blank videos? It there anything in the console window at all? It should at least log that it is trying to open the videos....

Or perhaps try creating a new blank Unity project and reimporting the package, in case something strange happened during the last import.

Let me know how it goes - we haven't heard any similar reports.

Thanks,

 

I've just thought of another possibility...Are you on a Macbook Pro by any chance? If you're still not getting any visuals, you could try switching the graphics API from Metal to OpenGL (in Player Settings > Automatic graphics api). We've had some rare reports of Metal problems on macOS with the trial watermarking....
If that fixes it please could you let us know your machine model and OS version.

Thanks,

 

You should make your build with Developer Mode enabled, and then after the run, open up the log file (in Users folder) to see what errors there were. This should give you a clue to the problem.

I'm not sure what is happening here, but your code:

mp.Control.Play();

looks a bit dangerous, because mp.Control could be null, so you need to check for this:

if (mp.Control != null)
{
mp.Control.Play();
}

Actually you should use the safer shortcuts:

mp.Play();

and

mp.Pause();

I hope this helps you. Let me know if you're still having trouble.

Thanks,

 

Did you try creating a new blank Unity project, reimporting the AVPro Video package in and then making a build for desktop/hololens UWP?

I think this is something you should.

Thanks,

 

I would also recommend upgrading to the latest version of AVPro Video (1.9.8) as there have been many improvements since your version.

Thanks,

 

I tried meshes of higher quality before using the HIGH_QUALITY option on the shader, and it was never good enough, or at least as good as with the high quality shader.

Any updates on the memory leak issue?

I tried it with ExoPlayer, and it seems to be working fine, and GetCurrentTimeMs is not getting stuck. However, I cannot use ExoPlayer due to the memory leak issue I mentioned above and in my other posts.

It is always the same videos that have this issue with MediaPlayer. It never works fine with these videos. With other videos it is just fine. I have not yet found any videos with which sometimes works and sometimes doesn't.

Resolution, fps, codec and bit-rate vary a lot with the videos I use, but mostly it is mp4, 4K, H.264 and around 10-30 mbps. It does not seem to not work for only demanding videos, because it does work fine for some.

unfortunately, I am not allowed to send you any videos since they belong to our clients, and we are only allowed to use them for internal testing.

 

I've attached here an updated OES shader that now support the high quality option. It does look a lot better, so I hope it helps. Let me know if it works.

We're working on it, but any information you could provide that would help us to isolate the problem would greatly help us to solve it. Do you have any information that shows that it is actually a leak? Or do you have any logs with error messages?

Doh..well that makes it quite difficult for us to investigate...

I would say try to lower the bit-rate. If the video is encoded with variable bit-rate, then the rate might peak very high during some scenes (usually with lots of detail / motion) which can cause decoders to choke and do strange things.

Thanks,

 

Actually i got only one MediaPlayer. After more tests, i find that when using exo player instead of media player on android it can buffer and play at the same time, problem solved.

But then i found another problem. The mp4 format video can get correct buffering progress, but m3u8 cant. it seems the progress is always 0 when playing, but a mp4 video shows buffering progress 1 when playing (tested on windows unity editor). in this situation, i cant use isbuffering function to get the playing status cause the buffering progress is 0 and isbuffering is true but still in playing.

i am confused, when playing online videos, even though the video pauses for buffering, IsPlaying() is still true. how can i get the real playing status when using online videos, epecially m3u8 for streaming?

 

And i found that the buffering progress seems to always jumps back a little, i mean now it's 0.16 for example, next Update() log show that the progress is 0.14, which decrease a little. so what is the problem with buffering?

when using mp4 file, it pauses for buffering, but isPlaying is true, at least i can use buffering progress != 1 to check is it really in playing status. like the log shows:

isPlaying = True
buffering progress = 0.55
isPlaying = True
buffering progress = 0.54
isPlaying = True
buffering progress = 0.54
isPlaying = True
buffering progress = 0.54

but m3u8 get buffering progress is always 0 and isPlaying() is true when buffering, cant get the real playing status

 

Thanks a lot for the shader, it looks really good now. However, I run into a small issue. I use an "Apply to Mesh" component, and manually modify the offset and scale values. This worked great so far without the OES shader, but with this shader it does not seem to affect it at all. Is this not supported for OES?

I do not have any logs right now, since we switched to MediaPlayer to make the app usable. But I can tell you exactly what happened. So every time a video start playing, the log looked something like this:

Code (csharp):

1.  
2. Opening /storage/emulated/0/Videos/example.mp4 (offset 0)
3. Using playback path: ExoPlayer (3840x4320@0.00)
4. Texture ID: 84
5. Texture ID: 86
6.  

Notice that two texture IDs were assigned. I am not sure if this is how it is supposed to work, but it happened to a lot of videos (Not all). Also, every time another video started playing (even if it was the same video being loaded again), the texture ID was always increased (by 2). Well, over time, the videos started stalling and lagging more and more, to the point were the app became unresponsive.

None of the above happened in the Editor, and it does not seem to happen on Gear VR, only on Oculus Go.
Switching to MediaPlayer (nothing else changed in the app) seems to fix the problem.

We asked permission from some client to allow us to send you videos for further investigation. Hopefully, they will allow it, and I will send you example videos.

 

Hi Andrew,

Thanks for your response.

Yeah i also suspected the error isn't caused by the plugin, but wanted to make sure. Still for reference here is the full stack trace:

Code (CSharp):

1. AndroidPlayer(ADB@127.0.0.1:34999) WrteDataToFilestream::IOException: System.IO.IOException: Unable to write data to the transport connection: Connection reset by peer. ---> System.Net.Sockets.SocketException: Connection reset by peer
2.  at System.Net.Sockets.Socket.Send (System.Byte[] buffer, System.Int32 offset, System.Int32 size, System.Net.Sockets.SocketFlags socketFlags) [0x00000] in <00000000000000000000000000000000>:0  
3. at System.Net.Sockets.NetworkStream.Write (System.Byte[] buffer, System.Int32 offset, System.Int32 size) [0x00000] in <00000000000000000000000000000000>:0   at System.Net.ResponseStream.InternalWrite (System.Byte[] buffer, System.Int32 offset, System.Int32 count) [0x00000] in <00000000000000000000000000000000>:0  
4. at System.Net.ResponseStream.Write (System.Byte[] buffer, System.Int32 offset, System.Int32 count) [0x00000] in <00000000000000000000000000000000>:0  
5. at ServerManager.WriteDataBackToFileStream (System.String path, System.String fileName) [0x00000] in <00000000000000000000000000000000>:0
6. at ServerManager.WriteDataToFile (System.Byte[] dataToWrite, System.Int32 commandSequenceNumber) [0x00000] in <00000000000000000000000000000000>:0  
7. at ServerManager.ReadResponseMessage () [0x00000] in <00000000000000000000000000000000>:0  
8. at ServerManager.GetResponseMessage () [0x00000] in <00000000000000000000000000000000>:0  
9. at ServerManager.ResponseListenerThread () [0x00000] in <00000000000000000000000000000000>:0  
10. at System.Threading.ThreadStart.Invoke () [0x00000] in <00000000000000000000000000000000>:0
11. at System.Threading.ThreadHelper.ThreadStart_Context (System.Object state) [0x00000] in <00000000000000000000000000000000>:0  
12. at System.Threading.ContextCallback.Invoke (System.Object state) [0x00000] in <00000000000000000000000000000000>:0  
13. at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x00000] in <00000000000000000000000000000000>:0  
14. at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x00000] in <00000000000000000000000000000000>:0  
15. at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state) [0x00000] in <00000000000000000000000000000000>:0  
16. at System.Threading.ThreadStart.Invoke () [0x00000] in <00000000000000000000000000000000>:0    --- End of inner exception stack trace ---
17. at System.Net.Sockets.NetworkStream.Write (System.Byte[] buffer, System.Int32 offset, System.Int32 size) [0x00000] in <00000000000000000000000000000000>:0  
18. at System.Net.ResponseStream.InternalWrite (System.Byte[] buffer, System.Int32 offset, System.Int32 count) [0x00000] in <00000000000000000000000000000000>:0  
19. at System.Net.ResponseStream.Write (System.Byte[] buffer, System.Int32 offset, System.Int32 count) [0x00000] in <00000000000000000000000000000000>:0  
20. at ServerManager.WriteDataBackToFileStream (System.String path, System.String fileName) [0x00000] in <00000000000000000000000000000000>:0
21. at ServerManager.WriteDataToFile (System.Byte[] dataToWrite, System.Int32 commandSequenceNumber) [0x00000] in <00000000000000000000000000000000>:0  
22. at ServerManager.ReadResponseMessage () [0x00000] in <00000000000000000000000000000000>:0  
23. at ServerManager.GetResponseMessage () [0x00000] in <00000000000000000000000000000000>:0  
24. at ServerManager.ResponseListenerThread () [0x00000] in <00000000000000000000000000000000>:0  
25. at System.Threading.ThreadStart.Invoke () [0x00000] in <00000000000000000000000000000000>:0  
26. at System.Threading.ThreadHelper.ThreadStart_Context (System.Object state) [0x00000] in <00000000000000000000000000000000>:0  
27. at System.Threading.ContextCallback.Invoke (System.Object state) [0x00000] in <00000000000000000000000000000000>:0  
28. at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x00000] in <00000000000000000000000000000000>:0  
29. at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x00000] in <00000000000000000000000000000000>:0  
30. at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state) [0x00000] in <00000000000000000000000000000000>:0  
31. at System.Threading.ThreadStart.Invoke () [0x00000] in <00000000000000000000000000000000>:0
32. #0 0xc7d7cec0 (libunity.so) GetStacktrace(int) 0x3c
33. #1 0xc755455c (libunity.so) DebugStringToFile(DebugStringToFileData const&) 0x22c
34. #2 0xc77f5c14 (libunity.so) DebugLogHandler::Internal_Log(LogType, LogOption, core::basic_string<char, core::StringStorageDefault<char> >, Object*) 0xa4
35. #3 0xc77f5b08 (libunity.so) DebugLogHandler_CUSTOM_Internal_Log(LogType, LogOption, ScriptingReferenceWrapper<Il2CppString*>, ScriptingReferenceWrapper<Il2CppObject*>) 0xe4
36. #4 0xc5039d80 (libil2cpp.so) ? 0x938d80
37. #5 0xc5042368 (libil2cpp.so) ? 0x941368
38. #6 0xc5361740 (libil2cpp.so) ? 0xc60740

Do you think this is something that can be fixed on my end, or that it needs a patch from unity. If it does do know the best way to get in touch with them? Thanks for your support.

Kind regards,

Remy.

 

Glad to hear that is it solved.

You should use the methods:

IsPlayBackStalled() to determine when playback has stopped while it is rebuffering

and

GetBufferedTimeRangeCount()
GetBufferedTimeRange()

To get the ranges of the timeline currently buffered.

Thanks,

 

Yes the GetBufferingProgress() probably shouldn't be used. We need to define what that is and currently it varies based on media type and platform. Better to use the GetBufferedTimeRange() method to get a true picture of buffering, though we currently don't have this exposed on Android.

Thanks,

 

Glad to hear that it worked. Yes we're not currently apply texture transforms in the OES shader. This is something you could add, or we will try to see whether we can add it soon.

Yes that definitely doesn't sound right. There should only be a single texture allocated per video....
Which version of AVPro Video are you on again? Was it 1.9.8?

Great

 

Hmm it doesn't look like it has anything to do with us....

 

Thanks, any updates on the fix would be greatly appreciated or any prerelease access to AV Video 2.0

 

Hi AndrewRH, I am always try to search the cause of the crash on my OculusGo when I reload the video.

Here is my settings :
- Unity 2017.4.14f1
- AvPro 1.9.8
- Oculus Utilities v1.30.0, OVRPlugin v1.30.0

- Player Settings
- - XR Settings : Stereo Rendering Method : Multi Pass

- AvPro Settings
- - I use Exo Player
- - I use "Fast OES Path"

My Video :
3480 x 1920
Bitrate : 1914kbps
Frame rate : 30 fps

This time I get an error about ExoPlayer on the adb logcat (see attachments)

 

We have an update (1.9.10) coming out any day now which hopefully will fix this issue.

Thanks,

 

I think I found a bug in the DisplayUGUI component:
- using the Scale Mode "Scale And Crop", adjusting the UVRect has no effect.
- adjusting the UVRect on Modes "Stretch To Fill" and "Scale To Fit" works as expected.

I'm trying to develop a zoom function for this component, and have to use Scale And Crop. So this is pretty much essential...

 

Is there a way to set the log level? In Debug mode, we are getting a lot of logs on Android. I know we could specify some sort of filter with logcat as you recommend in the documentation, but I would like to also be able to set the log level directly in code with the API. Something like what SmartFoxServer has with Debug, Info, and Error. http://docs2x.smartfoxserver.com/GettingStarted/log-configuration

 

Hey,

I have a question with 360 playback on galaxy S9's. We have been trying to play different resolution 360 videos, both stereo and mono. Once we pass 3840x2160 on either stereo or mono, the player blacks out. We then try to play these higher res videos through Samsung's native player and its fine. Is there a resolution cap for AVpro or could it be down to Unity? Or any way to get them playing?

Thanks

 

I'm afraid the amount of logging is not something that is controlled by AVPro Video. Internally we try to log as little as possible, but on Android there are tons of system logs that just happen automatically.

Unity does have some settings to control it's own log level (in Player Settings), and our plugin has to option to disable its own logging in the MediaPlayer > Global Settings.

Thanks,

 

That's strange...We don't have an S9 to test with. Sometimes manufacturers keep some special tricks for their own software... There is no resolution cap in AVPro Video.. Did you try with the OES mode enabled? Any clues in the adb log cat?

Thanks,

 

Good find. I've attached the fixed script here, and the fix will be included in the next version.

Thanks,

 

I've attached an updated shader here with transform supports.

Thanks,

 

Hi All,

AVPro Video version 1.9.10 has just been released!

You can find the updated version on the Asset Store and the free trial version on our website.

Changes include:

Android

• Fixed a crash bug with ExoPlayer that would happen randomly when loading or closing a video
• Added HIGH_QUALITY option to OES shader which fixes the issue of equirectangular videos having distortion at the poles
• Added texture transform to the OES shader

General

• Fixed bug where UVRect didn't work in DisplayGUI when in ScaleAndCrop mode

Thanks to everyone that reported bugs that were fixed in this release

Please report any issues here on the forum, or to our Github Issues

Thanks,

 

Hi Andrew, the hasVideo, GetVideoWidth() and GetVideoHeight() are all returning False and 0 respectivly. However when I run play() the video plays fine. If I check GetVideoWidth() and height afterwards it still returns 0. Any idea why this is happening?

 

Weve had this problem with a few of the S phones (6-9 i think) and also the galaxy tablets. Only error we get is:

[AVProVideo] Error: Loading failed. File not found, codec not supported, video resolution too high or insufficient system resources."

With the error, we pretty much ruled everything out apart from the resolution since the same video, very similar size (sometimes bigger), same path, name etc and it works perfectly with lower res.

We did also try OES mode and it make no difference at all

 

Hi,

I finally got a stable version for my project to works with OculusGo.

Here is my configuration on my thread :
https://forum.unity.com/threads/oculus-go-video-360-freeze-crash-after-reload.596956/#post-4089118

Thank you again AndrewRH for your help and your update

 

Any news on the fix for Android Vimeo Adaptive Streaming?

 

What's the issue?

 

I still think most of the Galaxy S phones report 3840x2160 as their maximum video decoding resolution...It's only the Oculus Go that I know of that can go higher. I only have an S6 so I can't be sure about the S9...

What resolutions are you using that are failing?

 

Umm, this is very strange. Nobody else is reporting this.
Which version of AVPro Video are you using? Have you tried importing it into a blank project to see whether the same thing happens? Do our sample scenes do the same thing? Generally those fields will only return valid values after HasMetaData() return true, or you can subscribe to the MetaData event.

 

Hi @AndrewRH the Vimeo issue is this one which was mentioned earlier on last week.
Thanks

 

Hello AndrewRH! I am planning to buy AVPro video, but sadly , the 450$ seems pretty expensive - so I was wondering as I only need IOS and Android, am I able to buy both of them (so 300$ instead of 450$) and use both at the same time? thanks!

 

Don't know if this has been covered (it's really difficult finding relevant data on a topic in a single thread). I'm getting the Error: loading failed on Oculus Go using the AVPro 360SphereVideo video prefab. Plays fine in Unity editor on Windows and also fine on Go using the Oculus video player. I've haven't had trouble before with AVPro on mobile VR. So I'm wondering if there's something about the video it doesn't like It's about 1.25GB and I'm loading it explicitly from an external folder on the device (not StreamingAssets). Here are the details:

Stream 0:
Codec:H264 - MPEG-4 AVC (part 10) (avc1)
Video resolution: 3840x2160
Frame rate: 25
Projection: Equirectangular

Stream 1:
Codec: MPEG AAC Audio (mp4a)
Channels: Ambisonics
Sample rate 40k

Is there any obvious reason why the AVPro player rejects this?

 

Investigating...

 

Yes you can buy both and combine them. See the PDF documentation where we have a section at the beginning about installing multiple platform versions.

Thanks,

 

Hi MarkHenryC,

I suspect the problem is caused by the audio. In order to play ambisonic videos in AVPro Video, you need to:
1) Have the audio encoded as Opus
2) Set the audio output to Facebook Audio 360 (in MediaPlayer > Platform Specific > Android > Audio Output)
3) Set the channel layout to the appropriate ambisonic order (1st, 2nd, 3rd).

I hope that helps.

Thanks,

 

OK, thanks. So if everything else is OK except (1), where the encoding is AAC instead of Opus, this will cause the error?

UPDATED

Video was sourced from Garmin Virb 360. Generates audio format 48 kHz Sampling, AAC Compression, 4-Channel Ambisonic Spatial Output Planar, 4-Channel First-Order Ambisonic (ACN Ordering, SN3D Normalization).

(Video format: MPEG-4 ID mp41 AVC, High@L5.1 CABAC / 1 Ref Frames)

So does this mean the output video would need to have the audio re-encrypted to Opus? Last time I looked, the tools supplied by Facebook involved a lot of post-production work. If you have any tips on how to manage this source of video I'd appreciate it. It will probably come up a lot since these sorts of devices are getting pretty popular.

Also, I remember having an issue with AVPro where the video needed to be MKV. Is this still the case? All a bit confusing...

 

Hi,

In our PDF documentation, take a look at section 4.4.1, we explain how you can convert an existing video with AAC audio into a video with Opus encoding (this is required). I'll paste here what it says:

Converting existing ambisonic videos

It is also possible to convert existing ambisonic videos so they are compatible. For example if you have an existing MP4 file with 4-channel 1st order ambisonic audio, then it is possible to convert this into the above format (MKV container with Opus audio) using a tool like FFMPEG. Simple put the following command in a .BAT file and then drag your MP4 into the batch file:

ffmpeg -y -i input.mp4 -c:v copy -acodec libopus -mapping_family 255 output.mkv

This should then generate an MKV file that you can play with AVPro Video. All that remains is to set the channel mapping in the MediaPlayer component to AMBIX_4.

I hope this helps,

 

Thanks. I missed the batch command tip in the manual.

UPDATED:

That was a head-slap thing for me. I realised I had not had external write set so AVPro wouldn't load videos from a file.

 

Anything over 3840x2160 is not working. So 4096x2160, 4096x4096 are the mains ones we want to get working.

Max resolution weve tried on the phones is 5968x5968 - h265/HEVC, bit rate unknown. That worked fine on the Samsung native player.

 

Probably the best video player on Asset Store.

I find it weird that only iOS platform can use GetBufferedTimeRange(). Any plan of getting GetBufferedTimeRange() supported on Android in the future ?