[Released] AVPro Video - complete video playback solution

Man your plugin is really the best out there, I got it to work by tweaking my ffmpeg command.
If anyone is wondering, I had to change pixel format in the output side of ffmpeg.

Really well done to you and your team, I now have 360 live streaming ambisonic working on Windows and going to my HTC Vive, I will try to do the same on Android now, it should be fairly easy with Facebook Audio integration.

 

Hi

I had the previous problem relating to Media foundation not playing a video that it previously did. I gave some more info in my previous post here.

The reason I'm messaging again is to ask if there is any way to download the previous version on the plugin?

All the best

Phil

 

Hi Phil,

Sure, email me (you should have my address), or use http://www.renderheads.com/contact and we can send you an older version. Just let us know which version you want.

I'm also curious to see whether there is a regression... We will test the Use Unity Audio option on our end too.

We do have an update coming out very soon, so it would be nice to include a fix for this if possible.

Thanks,

 

Hi, I have a weird issue using AVPro for Windows and a Leap Motion sensor that tracks your real hands. The moment I enable the Leap Motion service, the AVPro player stops playing videos. I have to restart the computer to get the videos running again. This may very well be a Leap Motion issue, but I wanted to see if anyone had run into anything similar? Any thoughts? Thank you.

 

Hey,

I was wondering if there is a get/set frame function for videos? I am looking at syncing videos and making sure they all pause at the same frame, and if not setting them all to a specific frame. Cheers

 

Hi,

When I use only mobile data (3G, 4G) to play video from Vimeo (Streaming), this not work. Video are not played.
But videos works fine with Wifi, on Android and IOS

Data usage section in Android Settings is activated for my app.
Tested on Samsung S6 - S8 - S8+ Android 7
Unity 5.6.4p1, Avprovideo 1.7

Works fine on Iphone, Ipad with mobile data or Wifi...

Any idea ?

Thanks

 

Hi Yorick,

OpenVideoFromBuffer requires you to have the entire file available to load and doesn't handle loading fragments as they come into the buffer, so I'm not sure if it is suitable for you. We currently have no plans to add support for Android or iOS but we will add it to the list to investigate.

I received your email and will reply shortly.

Thanks,

 

Hi Johan,

For iOS I don't think this will be possible in the near future.
For Android we already support 360 audio in the latest version of the plugin, via the Facebook 360 Audio system, which supports their propitiatory format, and also some forms of ambisonics. Currently this requires the videos to be MKV files with Opus audio.
I hope that helps answer your question.

 

This bug should now be fixed in 1.7.2

Thanks,

 

Strange..perhaps you could show us how you're opening them through code?

Also you could try loading them with your prefabs set to not using hardware decoding, as with h/w decoding there is a limit as to how many videos it can decode simultaneously.

Perhaps you could also give the specs of your videos, or send us a sample project to our unity support email address?

Thanks,

 

Hi Yorick,

Hmm that's strange. It may be that the codec isn't supported by the OpenVideoFromBuffer path. How is your video encoded? Perhaps send us over a copy to test with?

Also I would personally recommend trying the LAV Filters pack instead of K-Lite.

Thanks,

 

Cool!
What pixel format did you change it to? Mind sharing your ffmpeg comand?

 

That's very strange. We have not tested with Leap Motion sensor, but can't imagine why it would do that.

Which version of AVPro Video and Unity are you using?

Does it make a difference if you disabled hardware decoding (in MediaPlayer > Platform Specific > Windows)?

Thanks,

 

Yup, you can do

MediaPlayer mp;

mp.Control.GetCurrentTimeMs(); // returns time in milliseconds
mp.Control.Seek(3000); // Jump to time 3 seconds

If you work want to work with frames, then you need to knwo the frame rate of the video and you can use this helper function:

mp.Control.Seek(Helper.ConvertFrameToTimeSeconds(90, 30)); // Jump to frame 90 in a 30fps video.

Thanks,

 

With Android are you using the MediaPlayer or ExoPlayer API? Maybe try both?

Thanks,

 

Hi All,

AVPro Video version 1.7.2 has just been released!

You can find the updated version on the Asset Store and the free trial version / demos on our website.

Changes include:

Android

• Fixed two major GLES bugs which sometimes prevented the video appearing. This mostly affected Galaxy S8 devices, Lenovo tablets and anyone using the Vuforia plugin
• Added support for accurate seeking when using MediaPlayer API - requires Android 8.0 Oreo
• Added support for HTTP/HTTPS redirects when using ExoPlayer API

macOS, iOS and tvOS

• Added support for switching between multiple audio and video tracks
• Added support for querying audio and video track bitrate

Windows Desktop

• Added the ability to set a list of preferred decoders for DirectShow. This makes it easier to predict which decoder installed on the user's system will be used. For example they could have many codec packs installed, but you prefer to use the Microsoft decoders for stability
• When using the Use Unity Audio option, the video will no longer fail to load if the audio is not able to pass through to Unity
• Fixed a bug where Facebook Audio 360 channel mode wouldn’t be passed through to the plugin
• Fixed a shader bug causing Main10 videos in DirectShow to display too dark

General

• Added new HLSParser class to help parse HLS m3u8 manifests. This allows for the discovery of all of the streams available, their resolutions and bitrates
• Improved Facebook Audio channel mode enumeration UI

Thanks,

 

Hi Andrew,

Thanks for your quick reply. I think we have to use fragmented mp4 or mpeg-dash for this in the end. We wanted to load it from a buffer while the video was being downloaded. Our plan was to open one connection and send everything through that connection: Video, database data and any other type of data. Then sort that data on the client side using the headers found in the transfered data. The part with the video stream would be forwarded to the buffer of the video and other data would be forwarded to their respective places. The problem with the mpeg-dash and fragmented mp4 method is that they open their own connections and thus don't work together with our sorting and network load balancing system. We will have to investigate how we will still be able to make this system as fast as we wished it would be. Thanks for your clarification.

To answer on your mail: A network load balancing and data sorting system is the reason we want to do it this way.
We have a couple of reasons we want this system:
A: Opening only one connection and keeping this connection open helps transfering data when required without first having to open a new connection.
B: When the internet is slow(minimum 4G speed) we will send only one video and every now and then a bit of other data, when the internet is quick, we will send the current video and more other data. In this case we also send the beginning of the future videos so there is no pre-loading time when switching to a new video(which can happen anytime based on the user's choices).
C: We can send the large data blocks(mainly videos) over UDP and use the header information to make sure it's the correct follow up data and request a resend when UDP has failed(which might happen every once in about 3000 packets).
D: We want complete power and responsiblility over what is sent, why and when, relying on mpeg-dash and ExoPlayer would invalidate this.

We use H.265 decoding but I'm not sure if that's the issue thanks to your previous post. You clarified that it requires you to have the file available which might also cause the problem because the file is being downloaded at the time of playing them using the buffer. It uses WebClient.DownloadData(url); and passes that as a byte[] buffer to MediaPlayer.OpenVideoFromBuffer();

However it's odd that the sound does play.
EDIT: Also tried using WWW but causes the same black screen and only audio playing, also gives an error when the file is not completely downloaded yet like you've said. Also tried LAV filters but same issue, when opening a video(H.265) from buffer the screen stays black but audio plays. When playing that same video the normal way(url/absolute path) it does play.

I will install the LAV Filters pack and try again,

Thank you very much,

Yorick

 

Yep tried Mediaplayer and Exoplayer, tested with OES Path too :-(

 

Thanks for the follow up, I'm using AVPro 1.7.2 and Unity 2017.1.1, disabling the hardware decoding didn't do anything. It's working with a VLC based player, so there's a conflict there I don't understand.

 

Awesome. Prompt response as always Andrew.

Edit: After some tests, I am having an issue that may be a bug? For example:

mp.Control.Pause()
float frameTime = mp.Contol.GetCurrentTimeMs();
mp.Control.Seek(frameTime);

Once Seek goes through, the video skips back a few frames. Even if i delay the Get Time and Seek through Invoke, i still get the same issue. Any ideas what the cause of this is?

Cheers.

 

Thank you for the reply.

I've found interesting m3u8 specification that seems to indicate client(AVPro in our case) can choose which resolution of file from the playist to load at first. Below is the article I've found.

Question 1: Does this mean that version 1.7.2 might have options to choose preferably higher resolution of video?

Question 2: Or, would it work if we change order of files in the playlist so that higher resolution goes first so that AVPro might choose the first in the list preferably?

* Here's the link.
https://tools.ietf.org/html/draft-pantos-http-live-streaming-08#section-6.3.3

* Here's the part I wanted to point out.(Bold lines)
NOTE: later on, I noticed this is about selecting segment files not the 'resolution' files.

The client SHALL choose which media segment to play first from the
Playlist when playback starts. If the EXT-X-ENDLIST tag is not
present and the client intends to play the media regularly (i.e. in
playlist order at the nominal playback rate), the client SHOULD NOT
choose a segment which starts less than three target durations from
the end of the Playlist file. Doing so can trigger playback stalls.

* To achieve regular playback, media segments MUST be played in the
order that they appear in the Playlist file. The client MAY present
the available media in any way it wishes, including regular playback,
random access, and trick modes.*

 

@AndrewRH

I was creating a windows 10 UWP application and upon building it the videos didn't play to my shock, as I have used and love your plugin. So I got the log and wanted to tell you about this error as your plugin currently will not work for UWP applications. Let me know if I can change anything in the interim about it as I would like to use this plugin but am in a time crunch.

TypeLoadException: Unresolved P/Invoke method 'Init!AVProVideo' in assembly 'Assembly-CSharp, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null' because it is not available in UWP applications. Please either use an another API , or use [DllImport(ExactSpelling=true) to indicate that you understand the implications of using non-UWP application APIs.
at RenderHeads.Media.AVProVideo.WindowsMediaPlayer.Native.Init(Boolean linearColorSpace, Boolean isD3D11NoSingleThreaded)
at RenderHeads.Media.AVProVideo.WindowsMediaPlayer.InitialisePlatform()
at RenderHeads.Media.AVProVideo.MediaPlayer.CreatePlatformMediaPlayer()
at RenderHeads.Media.AVProVideo.MediaPlayer.Initialise()
at HomePage.Initalize()
at HomePage.Open()
at PageManager.switchToPage(Int32 index)
at PageManager.Start()
at PageManager.$Invoke20(Int64 instance, Int64* args)
at UnityEngine.Internal.$MethodUtility.InvokeMethod(Int64 instance, Int64* args, IntPtr method)
(Filename: <Unknown> Line: 0)

Thank you in advance,
Dan

 

hi again, so it seems its behaving the same through code and just dropping them on, DirectShow is more consistent in showing them but plays back not quite as good, but I am just getting videos not showing from time to time, I have just noticed the new update to 1.7.2, ill try that now, and also I have installed the LAV Filters too

 

so with this

• Added the ability to set a list of preferred decoders for DirectShow. This makes it easier to predict which decoder installed on the user's system will be used. For example they could have many codec packs installed, but you prefer to use the Microsoft decoders for stability

how do you specify a new codec, and what codec would you suggest for H.265? and also would it fall back to a standard codec if the specified one was not installed?

 

Hi,
We're trying to make a VR app on oculus rift with AVPRO. One thing I noticed is that when the app goes to background or Oculus universal menu, the video keeps playing instead of pause. Does anyone know how to stop playing video in background? Thanks!

 

Andrew suggested me to use the LAV filter pack for H.265, works very good. I uninstalled my other codec packs like K-lite etc. and just installed the LAV filters pack. Did you also uninstall all other codec packs?

Kind regards,

Yorick

 

Thanks, just for the record we're handling this via email and Skype now. Cheers,

 

Hmm....Are you able to perhaps call Play() again to resume it?
Do you get the Stalled event at all, on either MediaPlayer or ExoPlayer Android APIs?

Thanks,

 

This is probably happening because the media player is seeking to the closest keyframe. Some platforms don't support seeking to the exact frame and instead seek to the closest keyframe. You can add more frames to your video to improve this.

Thanks,

 

So the segments are the chunks that the video is split up into. I don't believe the clients have to play a particular bandwidth stream - it is up to the client player, not the order of the streams listed in the manifest.

In 1.7.2 we have included a HLSParse class which you can use to parse the root m3u8 file, and then extract the URL and bandwidth for each stream. You can then pass these manually to the MediaPlayer to play, this way you can force a particular bandwidth/resolution.

Thanks,

 

Unfortunately I don't have a Leap Motion here to test with...I did install the Leap software but I couldn't get the issue to happen.

Is it still the same, or did you manage to find a fix?

Which version of Windows are you using?

Thanks,

 

Hmm that's strange - it's working for us.

Which version of AVPro Video are you using?
Which platform are you building for inside Visual Studio (x86, x86_64 or ARM)?
Which version of Unity are you using?
Are you able to replicate this using a blank project with just AVPro Video installed and one of our demo scenes?

Thanks,

 

You just add the name of the codec to the array in MediaPlayer > Platform Specific > Windows

Common codec names are:

LAV Video Decoder
Microsoft DTV-DVD Video Decoder

And yes, it will fall back to the standard system codecs if it doesn't find the one you specify.

Thanks,

 

You should be able to add OnApplicationPause(bool) to your app to handle this case, and just manually pause/resume the video.

Thanks,

 

Thank you so much again for the quick reply. We tried recently released 1,17.2.

It's great to parse m3u8 root and select specific video resolution to load but we are concerned that if we pick the highest resolution to load first, it would be taking too long for users with low network bandwidth until they start seeing the video play. I guess by selecting specific resolution of URL from the HLSParser result, we lose to make it re-adjusted later on to other resolution.

Is there a way to keep using root m3u8 URL but pick the resolution of the first video URL to start buffering so that users with higher bandwidth could see high-resolution video but also ones with lower bandwidth to have an option to switch to lower resolution video just like the normal m3u8 behaviour?

We thought about using the highest resolution among the sub m3u8 URL from HLSParser and check if it's taking too long time before it's ready then going back to root m3u8 so that it would choose lower bandwidth. This would actually take more time to show the video since this initial 'trying' time for higher bandwidth URL would add up the buffering time for lower bandwidth users.

Does this plan sound reasonable or do you have a suggestion for us to handle this issue?

 

Yes that's true - you would just be selecting a single resolution - manual control.

No I don't think this is possible. In Apple's HLS documention they say:

"The first entry in the master playlist will be played at the initiation of a stream and is used as part of a test to determine which stream is most appropriate. The order of the other streams is irrelevant. Therefore, the first bit rate in the playlist should be the one that most clients can sustain."

So I guess you should do this and hopefully then it will start with this stream, as long as they follow Apples guidelines.

I think it might be better to do like Apple say - pick a some of medium resolution to start with, then you can go up or down from there. You could also try to download a file from the server initially just using WWW class, and work out approximate speed from that to try to pick a starting resolution.

Hope this helps you! Hopefully in an upcoming version we can add better support for stream specification.

Thanks,

 

Hi All,

AVPro Video version 1.7.3 has just been released!

You can find the updated version on the Asset Store and the free trial version / demos on our website.

Changes include:

Windows

• Fixed a bug when using D3D9 that would cause the DirectShow video to either render black or cropped. This was a regression since version 1.7.0
• Fixed DirectShow loading from buffer support for 32-bit builds

WebGL

• Added support for error events
• Fixed issue where resampler could be enabled when it is not supported for WebGL

General

• Optimised SRT subtitle file parse to be much faster

Thanks,

 

Urgent!
Any way to get frame count aside from using GetTextureFrameCount()? The problem being that if you rewind/stop and then play the video, GetTextureFrameCount does not reset.

Thanks,
Bryan

 

Hi,

i use avprovideo in ios application video play back... face problem when i suspend the application and open the application again... sometimes the video will stuck , sometimes only sound is playing , image freezes.... And other when i unplug audio jack from ipad... the video playback will pause... how to solve this? i using mac ,unity 5.6.2 xcode 9.1

thanks

 

Hi Bryan,

So GetTextureFrameCount() is just a count of the number of times we've updated the texture, and not necessarily which actual frame of the video it is displaying. This number doesn't reset to zero until a new video is loaded. If you want to get the frame number, then I recommend using this function:
Helper.ConvertTimeSecondsToFrame(mp.Control.GetCurrentTime() / 1000f, mp.Info.GetVideoFrameRate());

Hope that helps.

 

Which version of Unity are you using?
What resolution is your video?
Which model of iPad is this happening on and how much memory does it have?
Are you using the YCbCr iOS option in the MediaPlayer > Platform Specific > iOS settings?

Thanks,

 

Hello,
I am trying to read Metadata embedded into a mp4 video file. How can i read them using AVPro as a player?
thank you

 

thanks for reply,

resolution 1024 768 ..ipad gen 5 2017....2gb ram.... nope i not using the ycbcr setting....

 

Can you please add mipmap options to Android? We are using 4k videos and they look very pixelated in some cases. We think mipmapping will solve this issue

 

We need to loop through multiple videos. It is important that we use different videos as they can change dynamically depending on the state of the machine. There is only one prefab containing a video at any given time. The problem is there is a white flash when the video loads (which a couple of folks have mentioned in this thread). I'm a little uncertain on how I would use the FirstFrameReady event to mitigate this ... could you provide a little guidance on what the process would be? i.e., do I open, then wait for the event, then play?

In the meantime I am opening and loading the video off-screen then moving it back after calling Play(). Thanks for any help you can provide!

 

We're experiencing an issue with our AVPro Android deployment, where in our project we utilize the pause control function on video playback to display a quiz modal that we designed. We have noticed that there is a second or so of playback in the audio after we initiate the pause, and then when we resume the video playback, the video never starts at the same place we paused, it usually resumes a second or so before the pause timestamp. Is there a better way to pause video playback on Android to prevent such latency?

 

We currently don't offer any way to extra metadata. There are different types of metadata that can be embedded into video files, eg per-frame metadata, metadata about the video for example the orientation, a custom video track containing metadata, etc. What kind of embedded meta data were you hoping to extract?

Thanks,

 

Maybe try with the YCbCr option enabled? Or perhaps you could send us an example project?

Are you using any 3rd party plugins in your project, eg Vuforia?

Can you try changing your video API from Metal to GLES (in Unity Player Settings)?

Thanks,

 

Okay we've added it to the list. Hopefully it will come out soon. Thanks

 

Yeah, if your 4K video is 360 then mip-map won't really help, as it is being stretched on the sphere. But if you are playing a 4K video on the whole screen, and the screen resolution is less than 4K, then mip-map can certainly help - or actually what might be better is a custom bicubic downsampling..
We'll try to get the Android mipmap generation into the next version.

Thanks,