[Released] AVPro Video - complete video playback solution

Hi,
That is very odd. Perhaps you could email us one of your videos (unitysupport@renderheads.com). It sounds like the video may be using an audio codec that isn't supported on Android. I presume our sample videos play audio?
Thanks,

 

Hi, I bulit a video apk with AVpro Video 1.4.8 to demonstrate my new way of VR video building.
https://forums.oculus.com/community/discussion/43260/a-new-technology-to-build-vr-video/p1?new=1
Without the help of AVpro Video, it is impossible for me to convert this idea into reality. Thank you so much!
And two small questions.

One is that the frist frame still don't show when using auto open, so I use play and pause as you suggested, that's all right.

The other thing is it seems that playing two small size videos is harder than playing one big size video. At first I used 33 media players to construct this video(all 60 fps), and it can not be played smoothly. At last I used 6 +1 media players and problem solved. Is it possible to optimize that?

Thanks~

 

Hello,

I just bought your Video Player for our 360 work, and it works great.
Can I ask You what can I do, what kind of event to use, to load another scene after video finish playing ?

AVPro for Windows, H.264 MPEG-4, 360video

Thank You

 

Hi

I have a question stroke request. We are set to release a film soon on Steam (This is me under the NSC Creative guise). We have a media foundation version of the film but also have a Directshow version of the film. The Driectshow version is twice the resolution double the frame rate and just all round much better. The only problem is having to get people to install and set to preferred the Lav codecs. The issue mostly here is the set to preferred. The LAV installer which we will be pointing people too just installs the codecs and you have to use another program such as tweaktools (I only know of tweaktools apart from editting the registry yourself) to set them to prefered. What would be really good is if you could make the player use the LAV codecs by default without having to set them to preferred. Is there a way of doing this and if not is there any chance of you implementing this in the future or can you think of a better way of getting LAV to work without having to install a third party codec pack?

Any ideas would be gratefully received..

Phil

 

You are correct, the audio codec wasn't supported on Android. I used HandBrake and re-saved it using their Universal device profile, and it is working great now. Thanks!

 

Figured it out. It turned out to be the canvas renderer and googleVR not playing nicely.
AVPro just happened to be in the middle!

 

Hi Duoliang,

Very interesting work! I was wondering - are you using the new Android OES texture feature that we recently exposed? This could give you extra performance on Android and it certainly helps with memory usage too. You may want to investigate this feature.

I'm pleased to report that the bug with first frame not showing on Android will be fixed in the next release (v1.5.0)

You used 33 media players on Android!? That's very brave. Yes the Android hardware only has a limited number of dedicated decoding engines so it's much better at playing fewer videos. Each video also has a bunch of memory (reference frames) that it needs to keep in memory/cache, so having more videos playing fragments this and performance is lost.

I don't think there is a way to optimise this. But please look at the OES feature - it is a much more optimal way to play videos. If you do test it I would love to hear if you find any improvements.

Thanks,

 

There is the FinishedPlaying event that is fired when the video ends You can see the IMGUI demo which uses the SimpleController.cs script and shows how to listen for this event. After that it's just the usual Unity code to changes scenes.
If you are playing videos on different scenes though I recommend reusing MediaPlayers between scenes instead of destroying and recreating them.

 

Do you really still need to use the Tweaktools? When I've been using LAV Filters recently (on Win10 + Win8.1) I'm pretty sure I didn't have to use the Tweaktools..... Please let me know about this.
But yes, we should be able to make the DirectShow player prefer the LAV Filters (and expose an option for this), so that is always tries to use LAV if it finds it. I will just have to find out the GUID for the LAV Splitter+Decoder, but that should be quite easy. This is a good idea so we can try to get this in for the version after v1.5.0. But again, I was pretty sure that it already worked this way...

 

AVPro Video v1.5.0 is almost done. Just putting on the finishing touches and doing the last round of testing

 

Thanks for the reply.

I read the duration and the currentTime every frame, at the moment (which is not very efficient... I know).
The video is loaded from the device itself (This PC\Galaxy Tab A (2016)\Tablet\Android\data\nl.kissthefrog.someClient.someApp.challenge\files\somePrettyVideo.mp4)

The debug overlay that comes with the plugin shows the same (false) duration on the Android device. The duration of the first video loaded is retrieved correctly, all other video's that I load after that, show the duration of the first video.

Cheers,

 

@Andrew

Ok I've looked installing the LAV codecs on a system without using the Tweak tools to set as preferred and you're right it doesn't seem to need tweaktools to use the LAV codecs. However I find this very odd as I am 100 percent sure this was a continuing problem I was having relating to people not being able to use Whirligig. It was defiantly a problem when using Windows 7 and I had continuous inquiries when people were going to Windows 10 and wanted to use the player to playback videos. The problem that people had on Windows 10 was that Klite (a codec pack I recommend at the time, not so much now) installed Tweaktools and registered LAV as the preferred codec but the version of tweak tools that they included was compatible with Windows 10 and it always failed to do so. I ended up creating my own installer with just LAV and the latest version of tweaktools and then register it in the installer to overcome this problem.

I have to admit this has baffled me. This has been a consistent problem for years but now doesn't seem to exist. Although I now no longer have windows 7 to test on so maybe it's still an issue on those systems. Having a way that would still use the LAV codecs as proffered regardless of your setup is still desirable though as I could fully confirm that you don't need to set as preferred on any system.

If you can shed any light of the need to not set prefered now I'd very greatful as I want to know if my battle with this is over

Regards

Phil

 

@Andrew

Ok I think I've found the reason. On a clean system the without any codecs installed things are set to Windows as prefered. You then install the latest LAV codecs and they are still set to Windows. In your old plugin (AVpro for Windows) the player does not run the installed LAV codecs but with your new plugin (AVPro Video) it does. I guess the update you made has changed the way the codecs are accessed. Does this sound right?

Phil

 

I had a look through the code, and I think I see the reason So when DirectShow is decidign on which filter to use for decoding, it has a preference for decoders that support NV12 pixel format. The Microsoft H.264 decoder has a strange quirk where a video with resolution higher than 1920x1080 initially gets assigned a resolution of 720x480. In the latest code if the decoder has a width of 720 and it's trying to enumerate NV12 I failed it so that the Microsoft H.264 decoder will only decode to RGBA. In this case if LAV is installed it's filter will not return a width of 720 and so it will not fail when trying to enumerate NV12 format, and it will be selected.
This is a bit of a hack, but it works well for videos over 1080p. Ideally though we should have an option where it first tries to find the LAV filters... I've put it on the list, but I hope that the above explanation helps.

 

Thanks, I think I understand. As our video is at least 2048 by 2048 it should always default to lav if installed which is good to know. I look forward to having the LAV prefered by default over Windows. The more things implemented that prevent people using the wrong codecs the better

 

Hey there, guys,

Firstly, thanks for the amazing plugin!

I have run into what I believe is a bug. On Android, I am unable to load videos (I get the Error EventType with LoadFailed) when my mp4 file (H264 + AAC) has AAC as the first stream (Stream 0) and h264 as the second stream (Stream 1). This happened on android phones (Ex: Nexus5X and Sony Xperia), but not on the Tablet Nexus 9.

Exchanging the order of the streams fixes the problem, but doing this is annoying/time-consuming, and hard to debug. Is this really a bug? I searched the manual but found nothing on the subject.

For other people who might end up having the same problem:

To check the streams:
ffprobe -show_streams -i video.mp4

To exchange the stream order: (From http://superuser.com/questions/5396...audio-streams-in-an-mkv-in-a-linux-based-syst)
ffmpeg -i input.mp4 -map 0:1 -map 0:0 -c copy output.mp4

Thanks

Edit: Not sure if this affects anything, but I have 2 MediaPlayers playing concurrently.

Edit2: Just realized that exchanging the order of the streams broke the duration of my mp4 videos. I'll just ask the artist to re-export the videos

 

Agreed! Expect the LAV preference feature in the next version. I'm also looking at whether I can include LAV DLLs directly....
v1.5.0 of the plugin is just about to be released, and it contains the Windows DirectShow multiple audio track feature you requested. Cheers,

 

Interesting We'll have to test this out. The plugin doesn't currently support videos with multiple video tracks anyway, that is it doesn't let you switch video tracks..
If you could email me (unitysupport@renderheads.com) a copy of one of these videos that crashes, that would greatly speed up the time for us to investigate this. Thanks,

 

I still can't replicate this. Did you try this in the IMGUI demo scene?

 

Done!

 

Hi Everyone,

AVPro Video version 1.5.0 has just been released!

You can find the updated version on the Asset Store and the free trial version / demos on our website.

Changes include:

General

• Added new shader for uGUI to support stereo videos
• Added global option to disable logging
• Added audio track selection to IMGUI demo
• Fixed editor bug where “\” character could be used in file path
• Updated documentation

Android

• Added display of initial “poster” frame for videos set to not auto-play
• Fixed bug that prevented files with escape characters from loading
• Fixed bug causing audio files to not play
• Fixed issue where GetCurrentTimeMs could report values greater than Duration

macOS

• Fixed no error being returned for loading videos without correct plist permissions set

Windows

• Hardware decoding enabled by default
• Hardware (GPU) decoding no longer require command-line parameter
• Better video API selection in editor
• Added support for multiple audio tracks to DirectShow playback path via IAMStreamSelect.
• Added Media Foundation multiple audio track code back in
• Fixed some minor resources leaks

Windows Phone / UWP

• Added option for hardware decoding (enabled by default)

WebGL

• Added support for multiple audio tracks
• Fixed frame rate and count not displaying
• Fixed issue where multiple videos wouldn’t unload cleanly
• Cleaned up plugin code, removing unneeded code

Thanks,

 

RTSP stream doesn't seems to work well. any recomendation? thanks!

windows 10 x64

 

Simple answer, the plugin doesn't have native support for RTSP on all platforms
Actually I think the only platform that supports RTSP well is Android. For Windows you can add support by installing a 3rd party codec, but there is no native support.

 

what kind of coded would you recommend to install to play rtsp stream specified in your demo? (rtsp://rtmp.infomaniak.ch/livecast/latele)

I tried many other streams and no one seems to be working at windows with AVProVideo .

 

Well the first two streams in that demo should work
Stream 1 is just a straight streaming MP4 file from a server, this is supported on all platforms
Stream 2 is an HLS stream and is supported in Windows 10, Android, iOS, tvOS, macOS
Stream 3 is RTSP stream, which only Android has native support for

On Windows you can install a 3rd party filter such as LAV Filters. You can select which protocols it supports:


Then in the AVPro Video MediaPlayer component set the video API to DirectShow.



It should work then

 

Hi Everyone,

AVPro Video version 1.5.0 has just been released!

You can find the updated version on the Asset Store and the free trial version / demos on our website.

Changes include:

General

• Added new shader for uGUI to support stereo videos
• Added global option to disable logging
• Added audio track selection to IMGUI demo
• Fixed editor bug where “\” character could be used in file path
• Updated documentation

Android

• Added display of initial “poster” frame for videos set to not auto-play
• Fixed bug that prevented files with escape characters from loading
• Fixed bug causing audio files to not play
• Fixed issue where GetCurrentTimeMs could report values greater than Duration

macOS

• Fixed no error being returned for loading videos without correct plist permissions set

Windows

• Hardware decoding enabled by default
• Hardware (GPU) decoding no longer require command-line parameter
• Better video API selection in editor
• Added support for multiple audio tracks to DirectShow playback path via IAMStreamSelect.
• Added Media Foundation multiple audio track code back in
• Fixed some minor resources leaks

Windows Phone / UWP

• Added option for hardware decoding (enabled by default)

WebGL

• Added support for multiple audio tracks
• Fixed frame rate and count not displaying
• Fixed issue where multiple videos wouldn’t unload cleanly
• Cleaned up plugin code, removing unneeded code

Thanks,

 

Thanks for the update it's really good to be able to change tracks in Directshow now and I've already implemented it.

I do have one question. One of the reasons I've not been using the Media foundation part of your plugin is that difficult I've had with getting gpu decoding to work properly. Now that you've implemented this as default I'm very interested to switch my support over to your plugin. The one thing I need to do is find a way to switch from Directshow to media foundation at runtime. During would be ideal but at the beginning of the level is fine, I don't mind having to reload the level. Is there a way of doing this and if so could you give some tips on achieve this.

The second second question I've got is is there any way to retrieve the names of the audio tracks selected. Currently I'm just naming the track 1,2,3 etc but being able to name them based on what they've been named in the video file would be really useful.

One thing I've noticed is the gamma is set differently between directshow and Media foundation. It's not a problem for me as I've got a way of changing it but I thought I'd mention it incase you didn't know.

Regardless of whether these things are possible it's really good to see your continued development of the plugin. It stands at the core of my program and for good reason.

All the best

Phil

 

You should be able to switch between video API at run-time, using something like this:

Code (CSharp):

1. _mediaPlayer.PlatformOptionsWindows.videoApi = Windows.VideoApi.DirectShow;
2. _mediaPlayer.PlatformOptionsWindows.videoApi = Windows.VideoApi.MediaFoundation;

Currently there's no way to retrieve the name of the audio track, but I'm sure we can add this in the next update.

Gamma/Linear should be handled automatically correctly in all cases, except one: using MediaFoundation + Linear. In this case an extra correction is required in any shader that renders the texture (simply adding col.rgb = pow(col.rgb, 2.2)) - see the bottom of the InsideSphere shader for an example of this;

 

Hi Everyone,

AVPro Video version 1.5.1 has just been released!

You can find the updated version on the Asset Store and the free trial version / demos on our website.

Changes include:

Windows

• Fixed crash bug when playing audio files in DirectShow path
• Fixed visual flicker bug in Unity editor in DirectShow path

Thanks,

 

AndrewRH, hello!
A am only started to use AvProDecklink (trial version), I use Unity 5.4.0f3 (32-bit).
End when I play any Demo, I have this mistakes (file attached).
Can you help me?

Thx

 

Hi, I've replied to your question in the AVPro DeckLink support thread
Thanks,

 

Hi Andrew, I have an issue with 1.5.1 where i use the new uGUI shaders. Basically I can only describe the effect as the video bouncing one of the top / bottom channels. This happens on both T/B stereo and Alpha.

Cheers

Nick

 

Ah, I see this is the new watermark system... My error -)

 

Correct - now that GPU decoding is default this is the only way we can watermark this easily. It's weird I know - we'll see if we can improve it in the future
Thanks,

 

Hi Everyone,

Thanks for your feedback on the 1.5.0 release. We're working on v1.5.2 now which will contain some nice improvements Please feel free to post feature requests here.

Thanks,

 

Hi Andrew,
I've been trying to play a video on a WebGL build. I download the video with WWW() and save the bytes to Application.temporaryCachePath. I am unable to make the AVPro player open the file, however, and get a "Failed to open /tmp/video2.mp4" (video2.mp4 is the filename).

I was also unable to load directly from the URL, getting the following error:
SecurityError: Failed to execute 'texImage2D' on 'WebGLRenderingContext': The video element contains cross-origin data, and may not be loaded.,Error: Failed to execute 'texImage2D' on 'WebGLRenderingContext': The video element contains cross-origin data, and may not be loaded.

Do you have any tips?

Thanks,
Gian

Edit: Our app will play several videos, one after the other, so our idea was to pre-download a few videos so that the time between videos is minimal. By using the URL directly on the OpenVideoFromFile, we are unable to cache the next videos

 

@AndrewRH

Thanks for the explanation regarding changing the video path. I'll be implementing it today hopefully.

You asked for future requests and I know this has been requested before but it is pretty much the one that I've wanted since year dot. The feature is having audio go to audio sources so they can be used for positional audio. It's a request that I know isn't easy to implement but it's the one that I very much want. I know that the guy behind popMovie has been able to achieve it so it's defaintly possible. Maybe get in touch with him. I've always find him willing to help.

An ability to play internal subtitles would be good as well although I'm aware that requires a lot more than just being able to access them.

Currently that's the two I can think off.

Either way I'm looking forward to your next update.

Phil

 

hi, I've downloaded the windows demo build and the trial version.
On playing back a video I'm seeing a slow moving horizontal stripe superimposed on top, is this part of the trial version's watermark?

thanks in advance

 

Hello, I am about to purchase your product and just trying out the trial version. It is great but I have one question, I want to play a 3D stereoscopic video with an alpha channel in my scene. Is this possible? I was thinking a possible solution would be to create a video divided into four parts with with stereo left and right and their respective alpha channels below, then perhaps trying to tweak one of the shader scripts... but perhaps there may be another way already available in your package?

thanks very much!

 

Hi,

I noticed our App is very slow on Surface Pro 4 whenwe call the seek method.

When we navigate the video with the mouse scroll button, which calls a multiple seeks, it can take a while before it resolves.

I'm trying to reduce the number of seeks as much as possible but I'm still wondering if there is a setting I could use to improve the seek performance on the Surface 4. Could the latest AVPro release help?

I tried SeekFast and it didn't make a noticeable difference.

Specs :
Windows 10
Surface 4 (2.4GHz Intel Core i5-6300U, SSD 256 GB, 8 GB RAM, 128MB Intel HD Graphics 520)
Unity 5.4.0
AVPro 1.4.4

Thanks!

 

Cross-origin is a factor when using the web that one has to be aware of. This isn't something specific to AVPro Video and happens when try to load to content from another domain. This is usually something that needs to be set up on the server to allow an alternative domain to access content on it.

 

Audio source support is coming At least on Windows...the other platforms we'll have to see. I'm hoping to have it in for the next version. Subtitles are a bit further off...
Thanks for the support, Cheers!

 

3D and alpha at the same time is definitely possible, but we don't yet include any shaders that will support this so it would be up to you to alter the shaders for this case. Alternatively you could look at using a codec that already supports transparency, such as HAP (supported on macOS and Windows). Otherwise we can add this idea to the list and try to add support for it in the future.
Thanks,

 

This is usually caused by the way the video is encoded. Try for example the included video BigBuckBunny720p.mp4. This video should seek very quickly because it has a GOP of 2, so it has many keyframes (I-frames) which makes it fast to seek. I hope that helps you.
Thanks,

 

Yes, that's the watermark

 

Awesome updates again, congratulations!
Feature requests:

• +1 for audio source support.
• uGUI shader that supports the new Android OES decoding path.

 

Hello Renderheads team + community.

i'm using avpro to play a stereoscopic 360 spherical video. using google VR to split camera
the video is a 4k h265 .mp4
i also tried a 2k h264 .mp4

the videos are top bottom stereoscopic (edit - i believe this is the problem now)

both these run fine in player.

however i am building to android so when i install the APK and run - the app only gets to split the screen but no video plays back.
its just a white screen.

i'm thinking its got something to do with where android/unity saves the video out on build.

i've had the video in the streaming assets folder and tried a build with the video flat on the assets folder with no luck.

anyone got any ideas?

many many thanks

+++ update +++

i've tried a build with the sample spherical video in the demo folder - and still no luck

+++ update 2 +++

i've also tried building your demo scene and that doesn't play back either - when i click stereo packing onto none - the video plays fine on android build)
am going to now load a completely new project with JUST the avpro pack installed. (wihout the google VR unity package) and see how that goes

 

@AndrewRH

That's really good to hear . The audio source support will be a major update for me.

I have a bug (not really a bug). You've updated the AVpro Video Windows to 1.51 and have reportedly update the AVpro Video full pack but when I download it it still appears to be 1.4.8.

Thanks for the great support. I'm very much looking forward to the Audio source update, I'll be happy to test betas for that if you want.

Regards

Phil

 

Hi Phil,
Could you check and try to upgrade again (reimport package)?
Sometimes Unity locks the DLL files and this prevents them from updating.
To make sure this doesn't happen you should open a fresh Unity, without clicking on any GameObjects, then try the upgrade.
Thanks,

 

HI
I've got it working. I had also tried a fresh install on an empty scene but it still wasn't working. I then deleted the Renderheads folder in C:\Users\philipd\AppData\Roaming\Unity\Asset Store-5.x and re-imported and it work. Must of been some sort of permission thing on my system.

Thanks for the advice

Phil