[Released] AVPro Video - complete video playback solution

Hi JohnGate,

We support HLS natively on: iOS, tvOS, macOS (OS X), Windows 10, Android - so most operating systems.

Olders version of Windows would require a 3rd party codec (such as LAV Filters). WebGL support will depend on the browser - we've only seen Chrome with native HLS support so far with other browsers requiring JavaScript support for HLS.

Thanks,

 

Could you provide us with more information, or an example of this? Feel free to email unitysupport@renderheads.com with your specifics (server URL, Unity version, AVPro version etc).

Thanks,

 

Strange...could you provide more details such as:
1) AVPro Video version
2) Unity version
3) Android device model and Android operating system version
4) The URL of the file you're playing

Feel free to email these details to us at unitysupport@renderheads.com

Thanks,

 

I recommend that you upgrade to the latest version of the plugin (v1.5.2) as there have been many improvements for iOS since v1.3.9

Thanks,

 

1) AVPro Video version : 1.5.2t(sctipts v1.5.2) trial version with watermark
2) Unity version : 5.3.5p6
3) Android device model and Android operating system version : meizu mx5 with Android 5.1
4) The URL of the file you're playing : I use my local http server,the mp4 has dealt with "ffmpeg -movflags faststart"

the mp4 video on my local http server take about 20 seconds for buffering and play.


I have sent the e-mail to you,my mailing address is shibo.wang@pisofttech.com, please see the attachment mp4 video for your reference.thx.

 

Thanks, email received - we are looking into it.

 

Hey Andrew, did you got my email about the Playmaker action?

 

Thanks for sending me the video and information.

I had a look and I could also replicate the same issue here on my Android Samsung Galaxy S6. The issue seems to be related to the audio track. As soon as I remove the audio track the video plays fine.

I ran this command in ffmpeg:
ffmpeg -i %1 -vcodec copy -an %1-mute.mp4
Then the streaming worked much better. I think there might be some issue with the AAC audio track....

Please try this command.. I hope it helps you.

Perhaps there is a chance that you can change how your audio is encoded?

Thanks,

 

I did, I'm replying to it now

 

Minor issue:
In the docs you suggest that we rename videos with the *.bin extension to avoid Unity's auto video conversion. I've done this as well in all my projects and I have a PostBuild script that copies the videos to the build folder and removes the *.bin. The problem is that when I try to load and play the video in the Editor for testing, I get an "[AVProVideo] Error: LoadFailed" error. If I remove the *.bin, Unity does it's conversion and MediaPlayer loads and plays the video in the Editor correctly.

Like I said minor issue, but would be great to support this, specially when you mention it as a workflow solution in the docs.

AVPro Video 1.5.2
Unity 5.4.1f1

Thanks,
Dustin

 

Hi Dustin,
Surely the solution here is to simply use the StreamingAssets folder?
I'll edit the documentation to make a note that this is the preferred approach.
Whether the player can open a .bin file will depend on the platform and the video API used...
Thanks,

 

We have a folder in the Unity project "Assets/CopyToBuildFolder/*.*", the post build script takes everything in this folder and copies it to the root of the build folder.

This is very specific to our projects because we have a lot of assets that need to be in this folder structure, right next to the exe. (We have remote update systems that can push out updates into these folders)

When playing in the Editor I have the MediaPlayer set to:
FileLocation = MediaPlayer.FileLocation.RelativeToProjectFolder
Path = "CopyToBuildFolder/Assets/Media/*.mp4"

and when playing in the build it is set to:
FileLocation = MediaPlayer.FileLocation.RelativeToDataFolder
Path = "../Assets/Media/*.mp4"

Also for the API we are using "OpenVideoFromFile". Should we use a different API call? Do some methods allow playback of videos with .bin added and others do not? Also the platforms are for Standalone Win and Mac.

Thanks,
Dustin

 

Ahh I see. I would recommend that you put your folder CopyToBuildFolder actually relative to the project folder instead (so on the same level as /Assets instead of within it). You can then always have your file location set to RelativeToProjectFolder (in build this will be EXE folder) and your path can also be fixed.

So for example this folder structure:

• /Assets/
  • /Scripts/
  • /Plugins/
  • /etc...
• /MyMedia/
  • /MyVideo.mp4
• /ProjectSettings/

Then have all videos set to: RelativeToProjectFolder
And the path would be "Media/MyVideo.mp4"

Then when you build, you post build script would simply need to copy the MyMedia folder to the folder of the build.

I hope that makes sense. I don't see any need to have the video files inside the /Assets folder, unless they are in the /StreamingAssets folder. This way you don't have to have your files as .bin too.

OpenVideoFromFile is the only method you can use to open the files. It's actually the low-level API in the operating system that fails to load videos without the correct extension. I believe that MacOS is one platform that doesn't allow loading media files with .bin extensions (unfortunately).. Testing on Windows and it works fine, so blame Apple...

Thanks,

 

Thanks!

 

“ffmpeg -i %1 -vcodec copy -an %1-mute.mp4” worked！thank you very much！！！

 

Hi,

First, I'd like to thank you for your continuing support of the plugin, it's great.

We've ran into a performance problem lately when using Media Foundation as the playback path on Windows 10, our videos freezes slightly every time there's a lot of changes (or so it appears) in the images.
Those videos were encoded using ffmpeg with libx264 using profile high level 5.1.
The problem doesn't seem to occur when using the Direct Show playback path.

We also have a set of encoded videos intended for playback on mobile platforms, those videos are encoded using profile main level 4.1. When playing those videos using Media Foundatation, they play a lot smoother (no noticeable freeze).

I think (but didn't do extensive testing) that this problem did not occur before version 1.5 but since I also upgraded ffmeg version that could be the cause.

Thanks in advance for your response

 

Hi,

Any chance you could send as a snippet of one of these videos so that we can see the issue? You can email unitysupport@renderheads.com

Some questions:
1) Which version of AVPro Video are you using?
2) Which version for Unity are you using?
3) What is your GPU?
4) What resolution is your video?

Thanks for reporting the issue, I'm sure we can resolve it together

Thanks,

 

I just sent you a mail with all the info I could think of.
Thanks in advance for your support.

 

Hello,
we've tried to use your plugin in our video-heavy (lots of videos with transparency simultaneously on one scene) project. It works well on PC, but we're experiencing huge perfomance drop on Android. Previously we used Mobile Movie Texture (wrapper for libtheora) plugin, but it had serious memory leaks, when overall performance was acceptable, so we decided to find another plugin suitable for our needs.
We made test project with 10 different looped videos on scene, and nothing more on it. Video resolution was different, from 128*128 to approx. 1024*512. We had around 20FPS on this scene, but sometimes it was dropping to 5-7FPS for several seconds, then framerate going back to 20FPS. This temporary performance drop is our main problem, what has caused it?
Can it be an issue with your plugin or it is we have just put too much video to play on device?
To test your plugin we had to reconvert all of our videos to h264 using ffpmpeg. Maybe we converted our videos with improper settings? We used

Code (CSharp):

1. --profile baseline --level 3.1 --no-deblock --ref 1 --vbv-bufsize 3000 --vbv-maxrate 3000 --vbv-init 0.7 --rc-lookahead 10

Testing device: Samsung Galaxy A5 500
Android version: 6.0
Unity versoin: 5.4.1
Last demo version of plugin.

 

It's coming We fixed the issue with seeking so this feature will definitely be in the next release.

Thanks,

 

Thanks, we got your email. Looking into it now. The first thing I noticed is that the problem video has a reference frame count of 16, which is very high. You may want to limit this to 3-4. I will let you know what else we find..
Thanks,

 

Okay, we'll look into this and get back to you. I suspect the drop in FPS happens when the videos loop? Or perhaps it could be the garbage collector running? Did you try to hook the Unity profiler up to see if there are any clues there?

Thanks,

 

Hi,

I tested your video playback on a similiar system - Nvidia GT 750 but I didn't have any problems with the video turning blocky during fast motions. I had a look to see what the difference between the videos was:

ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L5.1
Format settings, CABAC : Yes
Format settings, ReFrames : 16 frames
Codec ID : avc1
Codec ID/Info : Advanced Video Coding
Duration : 1mn 3s
Bit rate : 1 511 Kbps
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 25.000 fps
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.029

The major difference is that the video is using 16 reference frames while the other is using 3. 16 Reference frames can add quite a bit of complexity to the decoder so I recommend that you limit this to 2-4. I hope this solves your playback problem.

We will also be releasing a new version of the AVPro Video plugin next week that fixes an issue with the GPU decoding path, so this may also improve things for you.

Another tip is to render your video at 30fps instead of 25. This should result in slightly smoother playback as it is a multiple of the display frame rate(60). 25fps is better for TV, but if it's not possible to render at 30 then it should still be okay.

Thanks,

 

Hello, thank you for your answer,
we tried to run test application with attached profiler. We caught this issue again. Just after application launch, FPS was around 20, then it falls to relatively stable 13-15, but after some time it drops with no visible reason, then it goes back again to 13-15. It could be GC, but i think it lasts too long for single GC call, isn't it? Also: if I minimize application and open it again on Android, FPS grows again to 20 for a short time.
We tried to play videos with loop setting turned off, starting it again after receiving a "video finished" event, but everything remains the same.
Profiler screenshots of this stutter in attachment.
If You need, we can send you test project, which reproduces our issue.

 

Hi,

Thanks for the information! Yes, that slow down certainly doesn't look like it's caused by GC.. I wonder if the device could be throttling itself? The fact that performance comes back after minimise is possibly a clue for this theory.

The Android chip only has a certain amount of silicon dedicated to video decoding, so I wonder if you try to play too many videos whether it's getting overloaded. I presume when you use less videos or lower resolution videos that this effect improves?

You should also test using Unity 5.3.6 as I've seen some evidence that performance has reduced in 5.4.x but I haven't yet proved it.

If you're willing to share your videos / scene we can run some tests here. Email unitysupport.renderheads.com

Thanks,

 

Hi!
This plugin sounds amazing, just what I need.
Is it possible to purchase the Mac version now and later upgrade it to the all platforms package?

 

Hi Andrew!

We've got the perfect project coming up for the audio source feature!
Hat off to you sir for replying to all these question and giving this awesome support and still getting features out of the door.

You guys totally deserve the asset store award so i voted here:
https://unity3d.com/awards/2016

 

Yes that's totally possible. You can find the Mac version here (it includes all other platfoms, just watermarked). You can upgrade later for the cost of the difference. Also you can test it out by downloading the free trial version on our website.

Thanks,

 

Thanks Tom-Goethals

 

Hey there!

1st Question
I'm using a Sphere to render a 360 video. The sphere is of course... white when I'm in the scene view. Is there a way to preview the video on that sphere while I am in the Scene view?

2nd Question
I'm using stereoscopic videos that I'm projecting on that sphere... the manual says to set up the Packing to "TOP BOTTOM" in the Media Properties. I havn't done this... yet everything seem fine... am I doing something differently?

3rd Question
I will have a couple of videos. I have each video in a different game object with a Media Player Script. I have left out the "Auto Open" enabled to all of the videos. Is this something crucial to performance? Should I disable this and enable the "Auto Open" when I also do the "Play"??



Thank You <3

 

No there isn't. The ApplyToMesh / ApplyToMaterial component only run when the editor is playing...

That is strange. You should always need to set the stereo packing mode for it to display correctly. The only thing I can think of is that your 360 video already have its stereo packing information encoded into the video as metadata, and that the video decoder is realizing this and only outputting a single eye. I've seen this happen on Windows before. The way to fix this is to remove the 360 stereo meta data from the video file.

Disabling auto open will save you memory, as each open video takes up memory. This can be important on mobile platforms, but isn't critical for desktops.

 

Hello,
Tried to launch test project built on Unity 5.3.6, instead of FPS drop it now crashes.
In a complex scene with all other contents, some time after playing videos once, FPS also drops for a short time, even when there's only few currently playing videos (2 or 3 small and short videos). If all videos are turned off, FPS remains stable.
I've e-mailed you our test project, will be glad if you check it.

 

I have received your project files and will get back to you soon.
Thanks,

 

Hello,

Is there a way to control buffering settings when streaming videos? We're working with high resolution 360 videos and are struggling to get things working on bad internet connections. Essentially I'm looking for a pause until a good chuck of the video has been loaded option.

Thanks!

 

Hi,
No there isn't a way to control the buffering I'm afraid. In the next version (1.5.4) we have some new functions that show you how much of the video has buffered, and also have a demo that shows how to use this. So you could use this yourself manually to see how much is buffered and decide if you want to pause and "buffer". The update should be out tomorrow.
Thanks,

 

I'm having an issue in (v1.5.2) where, using a variation of your controls demo mapped onto a 360 sphere, I cannot pause a video in GearVR until it's fully played (or seeked to the end) and then is playing again. On the Oculus, however, it works just fine. I wonder if, perhaps, it's related to the video not fully being buffered as noted with presencePG's issue.

 

I have tried to replicate this issue but I can't. It always pauses and plays fine for me. Perhaps you could share with us a complete sample project that shows this bug? Please email unitysupport@renderheads.com.

Thanks!

 

Hi,
I have emailed you the solution. Basically it is a bug in the current Android version and the bug will be fixed in the next release. Thanks.

 

Hey guys,

Our plugin AVPro Video has been nominated in the Unity Awards 2016 for the "Asset Store" and "Golden Cube" awards! We're delighted

The voting will close in a few days so please vote for your favourite nominees:
https://unity3d.com/awards/2016

Thanks,

 

Hi, i just bought the whole plugin solution,

I have errors when building :

- Shader error in 'AVProVideo/UI/Stereo': failed to open source file: 'AVProVideo.cginc' at line 59 (on d3d9)
- Shader error in 'AVProVideo/UI/Stereo': failed to open source file: 'AVProVideo.cginc' at line 59 (on d3d11)
- Shader error in 'AVProVideo/Lit/Diffuse (texture+color+fog+stereo support)': failed to open source file: 'AVProVideo.cginc' at line 25 (on d3d9)
- Shader error in 'AVProVideo/Lit/Diffuse (texture+color+fog+stereo support)': failed to open source file: 'AVProVideo.cginc' at line 25 (on d3d11)

....

and many many others...

Yet the videos work fine, but was wondering how to get rid of thoses annoying errors...


And the demo "10_Demo_Transparency" does not work, videos won't display (pink rendering) and I have this error :

Shader error in 'AVProVideo/IMGUI/Texture Transparent': failed to open source file: 'AVProVideo.cginc' at line 30 (on d3d11)


My last question :

we'll release our game on pc / PS4 / XboxONE, have you already tried videos on those platforms ?


Thanks.

 

Hi

I have a quick question, hopefully. I wondered what the best way was to detect if a video has failed to load. I'm currently testing to see if the video has returned a duration but as live streaming videos don't have a duration it's not practical to do that anymore.

Also are you still planning to release the next update this week? I'm really looking forward to implementing the positional sound

Regards

Phil

 

Yup, it'll have to be tomorrow now. We keep trying each day but issues get in the way... Definitely tomorrow! (Friday)

 

Some questions:
1) Which platform are you building for when you get thos errors?
2) Which version of Unity and AVPro Video are you using?
3) Have you checked that there is AVPRoVideo.cginc in your /Assets/AVProVideo/Resources folder?
4) Could you email the complete log to unitysupport@renderheads.com?

Thanks,

 

Really good to hear it's coming today. Regarding my other question about testing if a film is loaded. It would be good to have a more robust way to determine that the video has loaded. Another issue I've run into is that when I load a video I sometimes need to jump to a point in the video. So load video, does it have a duration over 0, if so jump to set location (say 4000 seconds). This works well in Directshow but doesn't work in Media foundation. I think that media foundation needs a little more time after reading the duration before it can be used. A way to say, this video is ready to use, seek etc would be good. Like mymovie.Videoloaded function.

This might already be in there but I'm not sure what it is. Any help would be appreciated.

Phil

 

Yes it's coming today, but still needs a few more hours of testing.
Your point about adding a function like IsLoaded is a good one and noted. We already have something for this in the todo list Also the seeking to a position on start is something we have on the list. In theory you should be able to use Control.CanPlay()....
Thanks,

 

Any chance for linux support in the future? Looking for a solution to replace the (barely usable) video texture support in unity, but I'd need to support all desktop platforms with my game.

 

Hmm potentially! We haven't had much request for this.. I'll put it on the list at the bottom, but unless we get more requests it won't happen any time soon. Thanks,

 

Hi All,

AVPro Video version 1.5.4 has just been released!

You can find the updated version on the Asset Store and the free trial version / demos on our website.

Changes include:

General

• 02_Demo_imGui demo scene updated to show network streaming progress
  Improved how destroyed MediaPlayer components are shut down
• Fixed alpha packing in background transparent shader
• Fixed some bugs in three of the sample scene scripts that would cause a crash due to null values or invalid textures left behind when destroying MediaPlayer
• Added documentation about hardware GPU decoding

Android

• Fixed bug where IsPlaying would return false during playback
• Fixed a shader compiler error for newer Android phones with Snapdragon chipset when using the OES shaders
• Optimised code to generate less garbage
• Some internal JNI caching for speed
• Added new unlit OES shader

macOS

• Added support for Linear colour space
• Improved feedback when progressive and streamed movies are buffering
• Added support for macOS 10.12 streaming auto wait behaviour

iOS and tvOS

• Fixed a case where some HLS streams wouldn’t play
• Improved feedback when progressive and streamed movies are buffering
• Added support for iOS/tvOS 10 streaming auto wait behaviour

Windows

• Added new AudioOutput component so that audio can be played directly through a Unity AudioSource node. This allows Unity audio effects to be used, spatial 3D movement with falloff and doppler, and most importantly allows sound to move with the head in VR!
• Fixed a visual glitch when using the hardware decoder path, most noticeable at the end of videos
• Streaming can now return multiple buffered ranges using method GetBufferedTimeRange()
• Windows watermark improved for GPU decoding path
• Potential fix for DirectShow seeking issue causing hanging on some systems

VR

• Support for 3D audio via the new AudioOutput component (Windows only, supports Oculus Rift and HTC Vive)
• InsideSphere transparency shader now supports transparency packing

Thanks,

 

Hi I'm crashing while using this plugin on PC so i made a sample project with just this plugin. I have videos in 2 different scenes, I'm using the Canvas UI to play the videos, when i load the next level with the second video UNITY crashes after going into the next scene. I'm using the stop function and closevideo to make sure its not running then a 1 sec delay before loading the next scene. This happens in the editor and build. I'm using auto-play and load.

Win 10 Unity 5.4.2f2 (64bit) AVpro 1.5.4 nvidia driver 375.70.

I tried on 3 different PCs same result. Using the stop and close video helped making the crash less frequent.
Here is the sample project just add the 1.5.4 AVPro Package to it to run it.
https://drive.google.com/file/d/0B8g5Jns72DLieXFGeEM1WE82dVU/view?usp=sharing

There is a button that loads then next scene it goes from scene 1 to 2 to 1 after a few loops it crashes. sometimes it takes 43 scenes loads, sometimes 5 to crash.

To replicate click the button to go between scenes there is a 1 second delay before going in to the next scene clicking faster does crash it faster.

This will be an interactive video app with videos in different scenes.

====UPDATE====
I loaded an old version 1.3.0 of AvPro its stable no crashes so far. hopefully this will help others if they run in to this.

 

Thanks PeteS, we'll check it out and let you know!