[Released] AVPro Video - complete video playback solution

Hi
I am playing 360 video as background in a Gear VR app. The app has nothing else in scene except UI elements. When number of UI elements increases the video fps is dropped to 5 fps on Samsung Galaxy S6 phone. Can you please fix this. Or is there any workaround to maintain 30 fps for video. This is quite urgent as this issue is occuring at last moment of one of our project.

Thanks

 

More information please. Standard things:
1) Unity version
2) AVPro version
3) Android OS version

Other things:
4) How many UI elements are you rendering when it drops to 5fps? Is it possible to see a screenshot of the UI before/after and how this affects the Unity profiler?
5) Have you tried to profile the app in Unity?
6) Have you made sure that you don't have something doing alpha blending covering a large area of the screen (fill-rate)?
7) Are you using the OES path? (see Android Platform Specific settings in the Media Player) This can give a nice performance boost. See the docs for more information about how to set this up if you're not already using it.
8) What resolution and format is your video? Is it stereo?
9) Is it possible to send us your project or a minimal example project?

Feel free to email us with this information - unitysupport@renderheads.com

Thanks,

 

Hello,

When I play the video in a period of time, I use Rewind(false) or Seek(0.0f) to replay the video. It will stop at start position. How can I replay the video from first frame.

Thanks.

 

Which platform is this on, and which version of the plugin?

Thanks,

 

Dear iOS, tvOS and macOS users,

Please note that v1.5.4 (the latest) is broken. Sorry about this mistake!
We're working on the fix now and it should be on the store soon.

Thanks,

 

Hi All,

AVPro Video version 1.5.4b has just been released!

You can find the updated version on the Asset Store and the free trial version / demos on our website.

Changes include:

General

• Fixed iOS, tvOS and macOS plugin bug in previous 1.5.4 release

WebGL

• Fixed "null id" error in multiple video sample

Thanks,

 

Hi All,

AVPro Video version 1.5.5 has just been released!

You can find the updated version on the Asset Store and the free trial version / demos on our website.

Changes include:

Windows

• Fixed issue on old Windows versions without D3D11 which caused the plugin to not load

WebGL

• Fixed width and height being zero when MetaDataReady event fires

Thanks,

 

plugin: 1.5.2
platform: ipad air 9.3.3 and Mac 10.12

 

@AndrewRH
After looking at this https://twitter.com/unity3d/status/793525901988200448
How do you intend to make current plugin to attractive to both existing and new customers? Can you share what can come in AVPro which will never be in Unity 5.6

 

Hi,

I've been using AVPro Quicktime for quite some time and got the AVPro Video plugin as well.

I noticed everything now is time-based, and there is no apparent Frame seeking support on the docs. Is there a way to go to a specific frame instead of a specific time?

I need frame accuracy and I can't seem to be able to get it on this new plugin.

Thanks!

 

Hey @AndrewRH! Really impressed with your work. Congrats on the new version today.

I am looking to set up a 360 video as a skybox, so that when I look around in my Vive everything in my scene is floating above the video. I've gone through the example scenes carefully and several come close (a sphere feels one baby step from a skybox, right?) but none of them really nail it.

What's the best practice, here?

I looked to see if this had been answered and found this post but it didn't really answer my question.

 

Okay, thanks - we're investigating.

 

If you know the frame rate of the video (you can read it via script) then you can make a function to convert frame number to time, eg:

Code (CSharp):

1. // Convert a frame number to time in seconds
2. float FrameToTimeSeconds(int frame, float frameRate)
3. {
4.      float frameDurationSeconds = 1f / frameRate;
5.      return ((float)frame * frameDurationSeconds) + (frameDurationSeconds * 0.5f);
6. }

This will only work if you video is encoded with a fixed frame rate. I'll see if we can add a helper function in there to do this. The problem is that sometimes people are playing videos with a variable frame rate, which makes frame related functions not work.

Thanks,

 

Hi,

Yes the sphere acts just like a skybox - there is really no difference except that the sphere takes in a single equi-rectangular video instead of taking 6 textures... What is missing that causes it to not "nail it"?

Thanks,

 

Oh wow, cool announcement, it's about time!

I don't have any more details about this than you do, so I'm not sure on what features they plan to have or not.. We use AVPro Video internally for our projects and I'm sure there will still be many cases where it will be useful, especially since any new features we can easily add. Are they going to support streaming? What about different file formats? (Formats such as Hap let you play super resolution videos for video walls). Will it support files external to the Unity project? I think that the need for a basic video player will certainly be diminished once Unity 5.6 comes out, but we'll still continue to offer AVPro Video on the Asset Store as long as it has useful features.

Thanks

 

Hi, Sounds Really Good
Especially support for Gif and other formats will keep it unique, especially once its out you can see how can u add more features so pple still come back to AVPro, i really want it to be #1

 

@AndrewRH do you know how can I set a loading sprite that appears the first time the video is loaded and everytime I seek and then starts loading again? I mean what "MediaPlayerEvent.EventType" values I need to use to "show" and "hide" that sprite?

Thanks!

 

Hi Andrew,

Got a new feature request:
MediaPlayerEvent.EventType.Stopped and MediaPlayerEvent.EventType.Pauzed

I tried the sound system, It works but i kinda need directshow support for it to be truly usable on Windows. Haven't tried but would it degrade gracefully? in a sense that it would just play the video sound normally on the other platforms?

Thanks!

 

Hi good gods. I have maybe trivial problem but I do not know ...maybe not..
my video is on this url: http://vid909.photobucket.com/albums/ac294/stoparr/PearlJam-JeremyPinkpop92.mp4
WEBGL application on my serverhosting http://37.157.197.34/. In Unity editor is all ok,




but in browser no:


Please help me... what is bad?
thank you..

 

The way the web works is that if you're accessing content from domain A on domain B then domain B needs to have permission set up for domain A to access it.

This is called CORS

You would need to set up CORS permission on the photobucket server, or host the content on the same server as the web page, or set up another server that you can set CORS permission on.

I hope that makes some sense. I'm not an expert on this subject so that's pretty much all I know on the subject. I'm just guessing that this is the problem...

 

Yup we plan to add DirectShow support to the AudioOutput system very soon. Currently yes, it degrades gracefully - if you select the AudioOutput option for Media Foundation, then when it switches to DirectShow (on older systems) the audio will just play in the normal manner.

I'll add those events to our TODO list

 

Hi @AndrewRH

I got a problem with android player. I can hear the video playing even if I App loses focus by pressing home button. Am I missing any settings or is this a bug?

 

Hi,

Yes it should pause playback when the app loses focus.

It works here on my device so I suspect that it's either a bug/quirk/difference in your version of Android, or in your version of Unity.

Please could you tell me your versions of:
1) Unity
2) Android OS
3) AVPro Video

Then I can do some further testing on our end.

Thanks,

 

Hi,

I regularly experience either a crash or a black screen with audio when trying to load a video using latest version. The same app does not crash when using 1.5.2 (crashes begin to occur with no changes other than to update the asset). This does not always happen, it seems to be random as sometimes I'll run into trouble the first time I load a video, sometimes I can load a few before anything goes wrong. I would really like to be able to play audio through Unity and am hoping this gets resolved soon.

I am running Unity 5.4.1p1 on Windows 10 with an NVIDIA 1070 driver version 368.71.

I suppose this could be the problem PeteS encountered.

Let me know if I can provide any other information that would be helpful.

 

Using the fast OES playback path we experience some weird "mirror reflect in mirror" effect on some android devices.
namely the Nexus 5X and Nexus 6P. The Moto G 2014 and the Samsung S6 have no issues. (attached a screenshot, everywhere the video should be showing we see some sort of reflection of the alpha transparant stuff in front of it)

The videos are FullHD h264 MP4. (example found here)
Using version: 1.4.8 (we dare not to update this late in the project and choosing to turn off the OES path for now) so not sure if this has been known before and fixed or not. We're rushing so no time to check now, just thought i'd let you guys now.

 

hello,

Has this problem been solved yet? When I set the PlaybackRate to more than 1, it will happen more frequent. I will pay for it after you've changed . Hope to get your reply as soon as possible.

Thanks.

 

I have tried it on Unity 5.4.0 and 5.4.2
android 6.0.1
AVPro Android 1.4.4t (scripts v1.5.2)

It happens sometimes, and disappears next day. I tried to debug IsPlaying value and its false at start. If I pause it for first time it wont pause. then i press play button on my UI it will set IsPlaying true and then i can pause it for second time. So this same behaviour is applied on onApplication pause code I guess. As leaving the app wont stop the player at first time and behaves normal at second time.

 

How do I resolve this error:

I try to play a VR Video (top/bottom 3840x2048) in the editor on Windows 10.
I switched Video api to DirectShow to get this error. For Media foundation the loading fails without error message. Do you know how i switch the codec (in windows?)?
Playback on android works.
Edit: codec should be H.265 and it seems that windows 10 can't play H.265 without hardware support

 

[EDIT]
Aaaaaand solved it the moment I sent the request. The player is deactivated at start because we have several players in our scene that may be activated sometime later. This causes Unity to crash when all are activated at start. I will activate the Audiooutput at the same time I activate the player.
[/EDIT]


Hi,
in our current project I want to locate the audio of the video in 3D. When I try to use AudioOutput as described in the manual (Stack Audiooutput referencing the player before an AudioSource component) I cannot set a clip in the AudioSource. I assumed that the AudioOutput sets this parameter at runtime. But unity shows me an error every frame:

The audio of the video is playing but in Stereo mode, the same way it does without AudioOutput. Is there something I missed?

Thank you!

 

Hi Mandelboxed,

I'm pretty sure that v1.5.4 was the first version with support for piping audio to Unity. v1.5.4 also had a bad crash bug (which is perhaps what you're experiencing) so I would recommend using the latest version (v1.5.5).
I hope this solves your problem.

Thanks,

 

Thanks! Yes the OES path is a very special case and it does have some issues. Unity doesn't really support it so there can be issues especially when trying to use multiple OES videos. We find it works well when there is just one video at a time. This is something we really want to improve. I will have to test on the Nexus 6P (just got one recently) and get back to you. Could you tell me how you're using it? It's quite difficult for me to work out what's happening in the screenshot - is it simply mapping one of the textures to the sphere, or something more complicated? Something to do with reflection probes?

Thanks!

 

Ahhh, I think this is something we have fixed in v1.5.4:
"Fixed bug where IsPlaying would return false during playback"
This could very much affect its ability to pause when losing focus. I would recommend upgrading to v1.5.5 - should fix the problem I hope.

Thanks,

 

Hi Andrew,

We're using it on a sphere as the only video playing, I think it's mapping the current viewport on to the sphere. We're using the Cardboard SDK (0.7) no idea if that is interfering somewhere. There are no reflection probes or lights.
Sorry for not having the time to do some more extensive testing ourselves now.

Thanks!
Tom

PS: For the audioSource stuff, would be awesome if we could pipe the left or right channel of a stereo audio file to seperate audioSources, I have no experience with 5.1 of more channel sound but it would be really cool if we could use it to do some positional sound in a 360 video that way.

 

Hey Andrew,

Unfortunately this is happening for me with 1.5.5. I upgraded directly from 1.5.2 to 1.5.5.

 

Why are you trying to use the legacy DirectShow decoder when you're running Windows 10? DirectShow doesn't natively support modern codecs like H.265 or 4K resolution H.264 and so you would need to install a 3rd party codec (such as LAV Filters) if you want to use DirectShow. If you're not able to use GPU decoding then LAV Filters certainly has faster H.265 CPU decoders compared to the default Microsoft Media Foundation ones.

For Windows 10 I would recommend that you stick to the default Media Foundation as it does have H.265 support. Hardware decoding for H.265 will very much depend on your GPU capabilities. For Nvidia only cards with the "GM206" chipset or above have hardware H.265 decoding. If you look at our PDF documentation (page 21) we show a table of which cards support which codecs in hardware.

If your video is failing to load in Media Foundation and not producing and error, then this is a bug. Perhaps you could send us your video so we can debug this?

Thanks,

 

Glad it's working now

 

Ahh rats...

Any chance you could send over a sample scene that we can run to get it to crash? Or do you know if there is a way to get one of the sample scenes to do this crash?

Thanks,

 

Okay great, that's very useful information - I'll give it a try and see if there is something we can do. It sounds like a RenderState isn't being reset correctly somewhere. It could very well be related to Cardboard SDK

Yup, that is a great suggestion and definitely what we're working on next for the AudioOutput component.

Thanks,

 

We're working on it. Turns out to be a tricky problem...Hopefully we'll have a fix soon.

Thanks,

 

Well I've reimported the latest (I rolled back after the crashes) and I don't seem to be able to reproduce the problem anymore which is great! I don't know if it's the changes I made since I tried last or if I got mixed up and was using 1.5.4b but either way it seems to be working now. I'll let you know if it happens again. Thank you!

 

Scratch that, still having trouble in a standalone build.

I'm getting this in the output_log.txt

d3d11: failed to create vertex buffer of size 24720 [0x887A0005]
d3d11: failed to create index buffer of size 1236 [0x887A0005]
d3d11: attempt to lock null vertex buffer
d3d11: attempt to lock null index buffer
d3d11: failed to create vertex buffer of size 24720 [0x887A0005]
d3d11: failed to create index buffer of size 1236 [0x887A0005]
d3d11: attempt to lock null vertex buffer
d3d11: attempt to lock null index buffer
d3d11: failed to create vertex buffer of size 24720 [0x887A0005]
d3d11: failed to create index buffer of size 1236 [0x887A0005]
d3d11: attempt to lock null vertex buffer
d3d11: attempt to lock null index buffer
d3d11: failed to create vertex buffer of size 24720 [0x887A0005]
d3d11: failed to create index buffer of size 1236 [0x887A0005]
d3d11: attempt to lock null vertex buffer
d3d11: attempt to lock null index buffer

I'll email you guys a link to the same build and a link to a test file with which I am able to consistently reproduce the problem. A quick round of tests suggests that it only happens with VR enabled. The test file is HVEC but it does happen sometimes with h264 as well.

 

That is the case. I can't send the video but it is a 3840x2048 h265 Video on a pc without h256 gpu hardware support.

 

Hi Pete,

Do you have a VR headset (eg Rift / Vive) on your system?

Thanks,

 

@Andrew H Can you answer me?

Also I get this error only on UWP platform (Pressing play from Visual Studio in debug mode):

[AVProVideo] Error: LoadFailed

It's a streamed MP4 video (http://) encoded in "H264 - MPEG-4 AVC (part10) (avc1)".

My problem it's that the error is not really descriptive and I don't know if my video it's supported looking in your docs link:

https://msdn.microsoft.com/library/windows/apps/ff462087(v=vs.105).aspx#BKMK_VideoSupport

 

Hi, sorry I missed your message.
There isn't any event that fires when the video is buffering or seeking. For this you would need to poll the media player:
mediaPlayer.Control.IsSeeking();
mediaPlayer.Control.IsBuffering();

For the streaming videos - make sure you have set the permissions in your app to allow access to the internet. You could try the included sample videos to see if they stream from your website as they're pretty simple videos.
If you send me your URL I can do some testing here and let you know. Please email me at unitysupport@renderheads.com.

Thanks,

 

Hi All,

AVPro Video version 1.5.6 has just been released!

You can find the updated version on the Asset Store and the free trial version / demos on our website.

Changes include:

General

• Added “Scale and Crop” scaling mode to uGUI component (replaces old “keep aspect ratio” option)
• Added helper function for seeking to a specific frame
• Improved MediaPlayer event handling code
• Fixed null exception when unloading scene from the event callback

Windows

• Fixed major issue using hardware decoding which caused videos to crash or not load
• Improved DirectShow colour reproduction by using more accurate linear colour-space and YUV conversion functions
• Added ability to get the presentation timestamp of the current video frame - useful for more accurate frame syncing: TextureProducer.GetTextureTimeStamp()
• Fixed issue with HLS streaming crashing when changing stream quality and resizing texture
• Fixed AudioOutput issue where playing back multi-channel videos would cause the video not to load
• Fixed AudioOutput issue where some videos would play back at the wrong pitch

Thanks,

 

Hi @AndrewRH , i am having problems with a VR video playback. I`m using the 360 sphere prefab and my video is getting too blurry sometimes, is this related to the video encoding, or can be something else?
Thanks

 

Thanks, I've sent you an email.

 

Thanks, we solved this issue. The InternetClient capability needed to be selected in the UWP Player Settings.

 

Hi, I'm not sure - perhaps you could show us a screenshot of this blurring versus how it's meant to look? Perhaps you could also tell us which platform you're testing on, the version of Unity you're using, the specs of your video etc..
The more information you give the better we can help