[Released] AVPro Video - complete video playback solution

Thanks for your response.
"Have you tried re-encoding using VP9?"
=> I didn't tried that yet because I was still using the default Unity VideoPlayer that couldn't handle VP9. Will try that asap.

"What is it like when just looping at a static speed?"
=> No matter which speed I set and let it stay at that speed it had a short delay between loops.

"Are the audio and video the same length?"
=> Yes.

"What range of playback speeds are you wanting to use and what rate are you wanting to change them?"
=> 0.5 to 2. After manual user input. At the moment it's controlled with a slider. I need to check wether it sets the speed on mouse up or each frame, if that matters.

"How are your videos encoded?"
=> VP8 with Vorbis for audio codex, otherwise the default Unity VideoPlayer couldn't play webm files.

 

Hi @MarvellousLunatic

How did the VP9 test go?
Have you made sure that your video adheres to the container guidelines (https://www.webmproject.org/docs/container/)?

Kind regards,
Chris

 

Hi All,

AVPro Video version 2.5.5 has just been released!

Changes include:

Windows

• Fixes
  • #1228 Fixed transparent NotchLC videos displaying with red-blue channels swapped

Android

• Improvements
  • Added option to change final blit texture filtering (Point|Bilinear|Trilinear). Note this will only take affect when using none-OES playback
• Fixes
  • #1227 Fixed OES GLSL shader regression introduced in 2.5.4 that caused shaders to not compile on some Adreno GPUs
    
  • Fixed cloud server playback regression introduced in 2.5.4 that caused exception to be thrown

Download Links

Editions

On the Unity Asset Store we've made several editions of AVPro Video available so you can pick the one that's best for your project:

• Core Edition

• Core Windows Edition
• Core Android Edition
• Core macOS/iOS/tvOS Edition​

• Ultra Edition
• Enterprise Edition (contact us for multi-site licensing / support options)​

You can read this page to learn more about the features, pricing and upgrades for the different editions.

DOWNLOAD THE FREE TRIAL VERSION HERE

Thanks to everyone who reported bugs or feature requests! We're looking forward continuing to support this product into the future and adding more features.

Thanks,

 

Hello,
We're trying to stream a video from the StreamingAssets directory on a Quest 2. The issue is that it's an .m3u8 file which means I think that we need to use the StreamParser class first - and the StreamParser only takes as input a url - and I'm not sure how to define that so that it reads from the StreamingAssets?
Application.streamingAssetsPath + "/lobby/lobby.m3u8" doesn't seem to work.... (but does in the editor) any clues?!

 

Hi @rosssssss
I'm presuming that you're using version 1?
You could try this:
path = "jar:file://" + Application.dataPath + "!/assets/";
(from https://docs.unity3d.com/Manual/StreamingAssets.html)

 

@Chris-RH, TerryRH, hello, i have a bug like this: my video source is streaming from server with m3u8 extention, when i slide the silder to seek video, the "_segmentsProgress" in MediaPlayerUI.cs (we use like the demo scene) did'nt work, the segments progess did not follow the handle, but when i let go the handle it did seek video. it happen on ios but work fine on android with same setup. Can you help me with this. i use 2.3.0 avpro video ultra.

 

Hi @SuperTung
Are you playing a livestream? There isn't really a meaningful begin and end times to show it relative to.
We've tested and it works fine for a vod.

Kind regards,
Chris

 

Is there possibility to detect which particular codec is missing?
I.e. instead of this error:
"File not found, codec not supported, video resolution too high or insuficient system resources"

could it be:
"Please install H.265 / HVEC codec"?


Or maybe there is a way to get information, which codec is needed for particular video file?

 

Hi @LoopIssuer,

Unfortunately the API's don't give us a lot of useful feedback so it's not really possible to be more specific with the error message.

Kind regards,
Chris

 

thank you for reply, yes, it's a streaming video (Video specs: m3u8).

 

Hi @SuperTung
Yes, then it sounds like it is behaving as expected. It is just to do with the way in which different platforms handle things.
Kind regards,
Chris

 

thank you,to a avoid misunderstanding eachother, Could you please take a look of this video and confirm it's the normal behavior of the slider and segment ? Thank you very much.
https://drive.google.com/file/d/1R53b5N17AX61NNUkIcK1QcLjeC-pv9pL/view?usp=sharing

 

Hi @SuperTung
We have checked this and we believe that the slider position is based off of the current media time. It looks like in Android this is updated pre-seek and in IOS it is updated post-seek. So it is normal, yes.
Kind regards,
Chris

 

Any plan on supporting Linux Desktop? It would be the best solution for all Unity Server apps

 

Hi @CONGOBILL

Unfortunately, there are currently no plans for linux support as it is difficult to ensure compatibility across such a wide range of distributions.

Kind regards,
Chris

 

Thank you, Is there any way to get the ios update pre-seek like the android, it's pretty weird that it's not acting the same on both platform of the app.

 

AVPro Video version 2.5.6 has just been released!

Changes include:

Windows

• Fixes
  • #1235 Fixed NotchLC pixelated decode bug in DX11 (regression introduced in 2.3.2)

Android

• Added
  • #1236 Added maximum bitrate and resolution limitations to media caching (Android only)
    
  • Added HLS Decryption (AES) 'Auth Token' and 'Key Override' support
• Fixes
  • #1044 Fixed JNI overflow issue
    
  • #1234 More reliable source video framerate when using the MediaPlayer API path

Download Links

Editions

On the Unity Asset Store we've made several editions of AVPro Video available so you can pick the one that's best for your project:

• Core Edition

• Core Windows Edition
• Core Android Edition
• Core macOS/iOS/tvOS Edition​

• Ultra Edition
• Enterprise Edition (contact us for multi-site licensing / support options)​

You can read this page to learn more about the features, pricing and upgrades for the different editions.

DOWNLOAD THE FREE TRIAL VERSION HERE

Thanks to everyone who reported bugs or feature requests! We're looking forward continuing to support this product into the future and adding more features.

Thanks,

 

Hi @AndrewRH

Thanks for all the hard work.

When playing clips on the timeline the audio level is overwritten by each clip. This is not the behaviour I hoped for as I need to control the audio volume per player dynamically through scripting. I see no other way than modifying your source scripts. Or am I missing a setting? Ideally this would be a setting on the Renderheads.Media.AVProVideo.Playables track; override audio volume per clip or don't.

 

Hi @cecarlsen ,

We're not quite sure what you mean by the audio level being overwritten. Could you show us a screenshot of your timeline please so we can see how the tracks are set up?

Kind regards,
Chris

 

When you set up clips on the timeline for AV Pro Media each clip has a volume setting. When played, these values will override the volume setting in the player. That is a problem because I need to control the audio volume of the player from a script. At the moment I've just removed the lines of code that does this in your source files.

 

Hi @Chris-RH

I have a another issue that is actually more pressing.

I have difficulties playing back HAP videos beyond 4K resolution. Specifically, I need to play back videos at 15360x8640 (16K) and below. I get this error:

Code (CSharp):

1. [AVProVideo] Error: Loading failed.  File not found, codec not supported, video resolution too high or insufficient system resources.

Curiously, it plays back fine in Keijiros KlakHap on the same machine. The trouble is that I need some of the features provided by AV Pro Media, such as getting the current frame number.

Is this resolution limitation introduced on purpose? I am testing using the trial version 2.5.6.

 

Hi @cecarlsen,

I am told that it should be able to handle that res.

Have you made sure both to tick both the "enable Hap/NotchLC" box and the "use custom MOV parsers" box and tried changing the framecount sliders?

We have further information on Hap over here: https://www.renderheads.com/content/docs/AVProVideo/articles/feature-hap-codec.html

If its still not working, could you perhaps send a sample video to unitysupport@renderheads.com? Please included the issue detail in the subject line (eg high-res HAP files won't load).

Kind regards,
Chris

 

AVPro Video version 2.5.7 has just been released!

DOWNLOAD THE FREE TRIAL VERSION HERE

Changes include:

Windows

• Fixes
  • #1237 Fixed issue with DirectShow not displaying videos at the correct aspect ratio when DAR didn't match SAR

Android

• Fixes
  • #1238 Fixed null exception related to SetOverrideDecryptionKey()

macOS / iOS

• Fixes
  • #1239 Fixed issue with thumb not tracking input when scrubbing a HLS video

Download Links

Editions

On the Unity Asset Store we've made several editions of AVPro Video available so you can pick the one that's best for your project:

• Core Edition

• Core Windows Edition
• Core Android Edition
• Core macOS/iOS/tvOS Edition​

• Ultra Edition
• Enterprise Edition (contact us for multi-site licensing / support options)
​

You can read this page to learn more about the features, pricing and upgrades for the different editions.

DOWNLOAD THE FREE TRIAL VERSION HERE

Thanks to everyone who reported bugs or feature requests! We're looking forward continuing to support this product into the future and adding more features.

Thanks,

 

That was it. Thank you.

I am now encountering another issue. I have a lookup array with data that needs to be applied to each frame in a post process. I use player.Control.GetCurrentTimeFrames() to get the current frame number, but when I apply the data I can see on the resulting image that the frame number I get is not precise. I am playing back Hap video.

Should I be able to count on the number being returned by GetCurrentTimeFrames() to match the actual frame currently being displayed (being available in TextureProducer.GetTexture())?

EDIT 1: I get the best results (best match) when I place my post process first in the Script Execution Order and process in Update(). But it still does not match precisely.

EDIT 2: I also found that Control.GetCurrentTimeFrames() returns the exact same as (int) ( Control.GetCurrentTime() * Info.GetVideoFrameRate() ). But that dosen't really help me.

EDIT 3: This seems to work. I have to check whether TextureProducer.GetTextureFrameCount() changed, THEN use Control.GetCurrentTimeFrames().

Code (CSharp):

1. int textureUpdateCount = _player.TextureProducer.GetTextureFrameCount();
2. if( textureUpdateCount != _lastTextureUpdateCount )
3. {
4.     int frameNum = _player.Control.GetCurrentTimeFrames();
5.     // Post process using frameNum here ...
6.     _lastTextureUpdateCount = textureUpdateCount;
7. }

EDIT 4: Crap. Apparently that trick only works for some of my videos. I guess it depends on the work load.

EDIT 5: I tried both Direct Show and Media Foundation. I also tried messing with the Parallel and Preroll Frame Count settings. No luck.

 

Hi @cecarlsen,

You could try using GetTextureTimeStamp()
https://www.renderheads.com/content...ProVideo_ITextureProducer_GetTextureTimeStamp

That should give you the timestamp for the current frame that is being displayed. Then you just need to convert it from HNS to seconds or frames.
int timeFrame = Helper.ConvertTimeSecondsToFrame(media.TextureProducer.GetTextureTimeStamp() / Helper.SecondsToHNS, media.Info.GetVideoFrameRate());

Kind regards,
Chris

 

Thanks Chris. This seems to work ok when I put the code in LateUpdate in a script that is moved last in Script Execution Order. I still get a frame offset glitch once in a while, but I think it may be a work load issue. I'll get back if it persists.

 

Great to hear that its working

 

Hi again @Chris-RH

When playing back very large HAP videos I am noticing low framerate during the first playthrough. Once it starts looping it runs smoothly. When I load other videos into the same player the same thing happens - but when I load videos that I have previously loaded (within the same Play session) they run smoothly from the very beginning. It seems as if the MediaPlayer is caching/preloading something during first playthrough. If that is the case, is it possible to cache before playing? Or force caching? At the moment I call Play on the MediaPlayer once I receive MediaPlayerEvent.EventType.ReadyToPlay.

EDIT1. Aha. I had to wait for the event MediaPlayerEvent.EventType.FirstFrameReady. Runs smoothly from the beginning now.

EDIT2: Nope. See later post =(

 

Oh crap. I thought that MediaPlayerEvent.EventType.FirstFrameReady fixed the problem. But no. Every time I reopen the Unity Editor, or run a build, the first play through struggles. The reason I did not see this before is because whatever is cached or preloaded lasts between Play sessions.

What exactly is slowing down MediaPlayer during the first playthrough? Is there a way to force the caching/preloading before calling Play?

 

Hi @cecarlsen

How low framerate are you getting and is it lasting for the duration of the first playthrough? What is your target framerate?
What is the fileseize? Video length? Bitrate? Chunk size?

Have you disabled the following, if possible
Use Text Tracks : Disabled
Use Stereo Detection : Disabled
Use Facebook Audio 360 : Disabled
Audio Output : None

Can you try enabling "Use Low latency"?

You could try increasing parallel/preroll frame count.

You could try encoding with faststart.

What are you CPU,GPU and memory usage like
1. Before first playthrough
2. During first playthrough
3. On second playthrough

AVPro will be decoding through the first playthough. Windows will naturally cache, hence the improved second playthrough.
You could try pausing between mediaPlayer.OpenMedia(mediaPath, autoPlay: false) and mediaPlayer.Play()

 

Hi @Chris-RH

Framerate I am getting during the first playthrough: 1-5 fps
Target framerate, which is also what I am getting after first playthrough: 29.97.
File size: 3.9 Gb.
Bitrate: 6.501.759 kb/s
Resolution: 5016x5760
Chuk size (if you mean the Hap chunks option): 4 (equal to the number of cores in my CPU)

I tired all of the below.
- Use Text Tracks : Disabled
- Use Stereo Detection : Disabled
- Use Facebook Audio 360 : Disabled
- Audio Output : None
- Enabling "Use Low latency"
- Increasing parallel/preroll frame count.
- Pausing between mediaPlayer.OpenMedia(mediaPath, autoPlay: false) and mediaPlayer.Play().

The Hap codec does not have a faststart option.

CPU,GPU and memory usage
1. Before first playthrough: CPU: ~610 Mb, GPU: 26Mb
2. During first playthrough: CPU: ~960 Mb, GPU: 26Mb
3. On second playthrough: CPU: ~960 Mb, GPU: 26Mb

Is there no way to force preload a Hap video?

 

Hi @Chris-RH

So far I've been using 2.5.6f1-trial, but now that I bought and imported 2.5.7f1-core (for Windows) I run into a strange issue. With the same video and same settings (see my previous post) I now get the error "Loading failed. File not found, codex not supported, video resolution too high or insufficient resources." (a quite useless error message btw).

Any clues what might be going on?

EDIT: Same settings, so "Use Custom MOV Parser" is enabled, as suggested earlier.

EDIT2: I am using Media Foundation. It works using DirectShow, but I cannot get the current frame index when set to DirectShow. Using the method suggested earlier:

Code (CSharp):

1. int frameIndex = Helper.ConvertTimeSecondsToFrame( _player.TextureProducer.GetTextureTimeStamp() / Helper.SecondsToHNS, _player.Info.GetVideoFrameRate() );

 

The only difference is that the trial version is watermarked and does not support Android OES textures. The Core Windows edition should still support HAP codec, albeit watermarked as HAP codec support is a feature of the Ultra version. Nevertheless, the videos should still play as previously.
We have an update going out hopefully later today which has a fix for some issues that were occurring playing some HAP files in Media Foundation (related to the only Windows fix in 2.5.7), so that might fix it.

 

It has to be the MediaFoundation issue then. Great news with the update. Thank you Chris =)

Meanwhile, I found out how to make _player.TextureProducer.GetTextureTimeStamp() behave in DirectShow like it does in MediaFoundation. Perhaps something to look at.

Code (CSharp):

1. if( frameIndex != 0 ) frameIndex++;
2. if( frameIndex >= _player.Info.GetDurationFrames() ) frameIndex = 0;

Currently, I am not getting a watermark on the Hap video ... so perhaps I stick with the current version

I still have low framerate in first play-through which is pretty unsatisfying.

 

AVPro Video version 2.6.0 has just been released!

DOWNLOAD THE FREE TRIAL VERSION HERE

Changes include:

Windows

• Fixes
  • #1237 Fixed issue with pixel aspect ratio not being respected when using the MediaFoundation API

macOS / iOS / tvOS

• Improvements
  • Added new audio mode "System Direct With Capture". This allows access to the PCM data via `MediaPlayer.Control.GrabAudio()` for analysis or for generating your own visualisations

tvOS

• Fixes
  • #1248 Fixed compile issue with UpdateMultiPassStereo script

Download Links

Editions

On the Unity Asset Store we've made several editions of AVPro Video available so you can pick the one that's best for your project:

• Core Edition
• Core Windows Edition
• Core Android Edition
• Core macOS/iOS/tvOS Edition
• Ultra Edition
• Enterprise Edition (contact us for multi-site licensing / support options)
You can read this page to learn more about the features, pricing and upgrades for the different editions.

DOWNLOAD THE FREE TRIAL VERSION HERE

Thanks to everyone who reported bugs or feature requests! We're looking forward continuing to support this product into the future and adding more features.

Thanks,

 

Unfortunately 2.6.0f1-core (Windows) didn't solve any of my problems.

1) Hap plays slowly during first playback because of decompression. No way to decompress before playback.
2) MediaFoundation not working for Hap videos. It worked for me in the trial version. And the error message I get is useless.
3) I am using DirectShow as fallback, but it seems less performant and I cannot get a accurate current frame index (which was possible with MediaFoundation).

I am suspecting that issue 2 is simply because I have the Core version, not Ultra.

EDIT:
I just bought Ultra and Hap now works both with MediaFoundation and DirectShow. Actually DirectShow turns out to be more performant. However, I am still experiencing frame index issues.

 

Hi @cecarlsen

It sounds like things are slowly improving.
What code are you using to get the current frame index and in what way is it incorrect?

Kind regards,
Chris

 

Hello. We are playing HLS streams with multiple resolutions - running on android devices (mostly VR) and it doesn't seem to go down resolution enough. The 8k video for example can be really stalling badly - unusably but still it doesn't go down most of the time. Are there some controls we can set that control how quickly it will decide to drop down a resolution step? - or are there some steps to make sure that the adaptive code is enabled? Thanks

 

I am using the code you provided earlier.

Code (CSharp):

1. int frameIndex = Helper.ConvertTimeSecondsToFrame( _player.TextureProducer.GetTextureTimeStamp() / Helper.SecondsToHNS, _player.Info.GetVideoFrameRate() );

And for DirectShow, I am using the workaround I posted earlier:

Code (CSharp):

1. if( frameIndex != 0 ) frameIndex++;
2. if( frameIndex >= _player.Info.GetDurationFrames() ) frameIndex = 0;

The odd thing right now is that it works perfectly for videos with a frame rate at 29,97, but not for ones with 25. GetVideoFrameRate() does return the correct frame rate. I really am out of clues.

 

@cecarlsen Would you be able to send us a sample of the HAP file you are using so that we can see if there is anything in it that might be causing issues? unitysupport@renderheads.com

Kind regards,
Chris

 

Hi @rosssssss

You could try changing your HLS manifest to have each stream on it's own as well as an adaptive version.
HLS selection is one thing that we are looking into improving in the future.

Kind regards,
Chris

 

Hi @Chris-RH, I'd like to make a feature request for a new shader.

Since a mesh can have 4 UV maps, you could use those 4 UV maps for Left-RGB, Right-RGB, Left-Alpha, Right-Alpha. This would allow some creative packing possibilities. Right now there's the InsideSphere - Custom UV shader but AFAIK it doesn't support transparency, plus it flips normals as well.

I'm working on a VR project where there's stereo 360 video with stereo motion graphics that are following your gaze, and right now I'm playing two 4k x 4k videos at the same time (on Quest 2) because there's no other way. It works, I just don't know yet if it will keep sync properly over its 20 minute runtime. Can't use OES either at the moment, since there doesn't seem to be a OES shader that supports alpha + stereo packing at the same time.

 

@AndrewRH - I've been getting back into AV Pro, looks like many of the issues I had ~3yrs ago are now totally solved, but Timeline still seems to be a bit of a mess. I'm thinking that the best approach to what I want to do (trigger a bunch of different things synched to a single video playing back in AV Pro) is to sync the timeline to the AVPro Media Player, i.e. continuously seek that timeline to the MediaPlayer's current time. Is there a recommended way to do this currently? I've been searching this massive thread here and everywhere online but have only found advice on how to drive a MediaPlayer from Timeline, not the other direction.

BTW, the Timeline Playables samples from Unity include a VideoPlayer that lets you scrub, which is kinda critical for finding exact timings for Timeline triggers, it might be worth looking at that to expand the AVPro Timeline Playable

 

EDIT: this is for Android/VR if that helps narrow down the issue and potential options...

 

Hi @SpatialEffects

That is certainly something that I can add to the enhancements list and the devs might do it if they have the time.

Kind regards,
Chris

 

Hi @avclubvids

Yes, our timelines implementation is only very basic. It might be improved at a later date, but we are only a very small team so have to focus our time accordingly.

Kind regards,
Chris

 

When will h265 pull flow be supported？

 

Hi @xy849288321

Do you mean RTMP/RTSP pull? On which platform?

Kind regards,
Chris

 

Yes, the RTSP monitoring stream does not seem to support h265 streams on the windows platform

 

AVPro Video version 2.6.1 has just been released!

DOWNLOAD THE FREE TRIAL VERSION HERE

Changes include:

General

• Fixes
  • Fixed issue with certain properties (Loop, AudioVolume, AudioMuted, etc.) on MediaPlayer not reflecting changes made directly on the IMediaControl interface

macOS / iOS / tvOS

• Fixes
  • #1261 Fixed issue where playback would not resume after an audio interruption such as receiving a call
  • #1268 Fixed issue with looping when looping hasn’t been enabled before OpenMedia is called

Android

• Fixes
  • #1278 SetPlaybackRate behaves like other platforms now

Windows

• Known Issues
  • HAP playback of videos with a width greater than 8192 might currently fail. We hope to fix this in the next update. Advise using v2.5.7 until then. Contact us if you do not have this version available to you

Download Links

Editions

On the Unity Asset Store we've made several editions of AVPro Video available so you can pick the one that's best for your project:

• Core Edition
• Core Windows Edition
• Core Android Edition
• Core macOS/iOS/tvOS Edition
• Ultra Edition
• Enterprise Edition (contact us for multi-site licensing / support options)
You can read this page to learn more about the features, pricing and upgrades for the different editions.

DOWNLOAD THE FREE TRIAL VERSION HERE

Thanks to everyone who reported bugs or feature requests! We're looking forward continuing to support this product into the future and adding more features.

Thanks,