[Released] AVPro Video - complete video playback solution

Hi,

We don't have any event that can be triggered on a frame, so you would need to poll for the current frame and then either invoke your own event or just run your code directly. You can poll for a specific frame like:

float frameRate = mediaPlayer.Info.GetVideoFrameRate();
float seconds = mediaPlayer.Control.GetCurrentTimeMs() / 1000f;
int frame = Helper.ConvertTimeSecondsToFrame(seconds, frameRate);

I hope this helps you.

Thanks,

 

Hi, what is easiest way to play over 4gb video size on Avpro Android locally? Thanks in advance!

 

Hello,

We are experiencing issues with URL CloudFront streamed videos not playing in the editor on windows OS. Videos are playing on all other platforms no problem, but it seems Windows is having some trouble playing HLS videos specifically as local videos are working fine.

It seems its an encoding issue with windows as this error appears after videos are found and opened:

[AVProVideo] Error: Loading failed. File not found, codec not supported, video resolution too high or insufficient system resources.

Is there something we can do to re-enforce the Windows decoding or support for HLS videos?
Some information of our setup:

• Unity version: 2019.3.15
• AVPro Video version: v1.11.5
• Codec: H264 - MPEG-4 AVC (part 10) (avc 1)
• Video Resolution = 1920 x 1080
• Buffer Dimensions = 1920 x 1088
• Frame rate = 30
• File type = .m3u8

 

HLS should work fine in Windows 10...Could you give us a URL to test with?

 

Copy/download the file to some location on the file system (perhaps your PersistantData folder) and then play it from that location.

 

Hi there
I'm develop a simple player for 360 video in Unity 2019.4.3, using last version of AVPro plugin/exoplayer on Oculus Quest
And sometimes my files are freezing on first frame, sometimes not
Videos in H256 codec 5760x2880 35mbps 30 minutes long with main profile and audio in wav 9ch ambisonics + 2 stereo, then i muxing it in MKV container with Facebook 360 encoder
Downscaling video to 5120x2560 and 4096x2048 and changing codec to H264 not helps
In Oculus gallery this files works fine!

Logcat gives me some issues

Somebody knows what could be wrong?

 

Thanks for posting this to github - we'll follow up there

 

Hi,

Thanks for sending the URL. We tested it and were able to replicate the problem. The video seems okay. If I download it locally it plays fine, so it seems to be some sort of network issue. Perhaps CloudFront has a log that will show why a connection attempt failed? Perhaps it's particular about the user-agent or something. I can't see anything on my side that would cause it to fail. Interesting on AVPro Video 2.0 (still in beta) it plays using the WinRT API, but not the Media Foundation API.. It may be that cloudfront is using some more modern form of HTTPS perhaps that isn't supported by the older Media Foundation API..

 

You may have to try changing the CloudFront settings to see if you can find one that is compatible:


Thanks,

 

Hi All,

AVPro Video version 2.0.0-BETA-5 has just been released!

You can download this version and view the release notes here:
https://github.com/RenderHeads/UnityPlugin-AVProVideo/releases/tag/2.0.0beta5

We will continue to release beta versions of 2.0 until we have all the features we're aiming for.

Please report any issues here on the forum, or preferably to our Github Issue tracker

Thanks,

 

Hello RenderHeads. We've used AVPro for a while on a couple projects, but this is the first time something like this came up. The videos appear to be washed out dramatically compared to their normal appearance (shown below, one is a snapshot from VLC, the other is via the 'Save PNG' button) other than specifically using directshow on windows, which is not our target platform (Oculus Quest). The videos have the same result whether using 264 out of adobe media encoder or VP9 in mp4 container via XMediaRecode, and they play correctly in every program other than AVPro's decoder.

Additionally, a similar problem not related to AVPro was fixed by changing the color space, but in this case switching the color space to Gamma from Linear made the entire project appear washed out on Quest, clearly not the solution.

Any help would be appreciated.

Left: Snapshot from VLC, correct colors.
Right: Snapshot taken from AVPro's Save PNG button, incorrect colors.

Edit: Seems to be something with how AVPro renders it inside the Linear color space, which we are too fargone to switch away from. I'm gonna try to do some shader level corrections on it, but I'm sure it's not a good idea. Up for suggestions.

Edit 2: The fix was further up in the thread, you need to convert the color space. Unity actually has a nice node in shadergraph called Colorspace Conversion, RGB -> Linear fixed it up nicely for me. Would recommend to anyone else having this problem, sorry if this was asked and answered.

 

Hi, may I ask a maybe noob question? Is AVPRO capable of having surrounded sound for 360 video on Android cardboard? What I mean is the sound will play on left or right side based on my current orientation.

 

Hi,
I have 10 videos in attached VCR script. While playing on some event I want to jump to 3rd video at specified location and play from there.
I tried
if (Input.GetKeyUp(KeyCode.X))
{
PlayingPlayer.OpenVideoFromFile(_location, System.IO.Path.Combine(_folder, _videoFiles[3]), _AutoStartToggle.isOn);
_videoSeekSlider.value = 0.5f;
PlayingPlayer.Control.Seek(_videoSeekSlider.value * PlayingPlayer.Info.GetDurationMs());
}

In this it loads third video but plays from beginning. It is not jumping to specified location.
I don't know what I am missing.

 

Thanks for the ideas. Trying some of these now, and I'm hopeful!

Can you explain GetBufferedTimeRangeCount? When would you have more than one?

 

You're right - usually there is just 1. In fact I don't think any of the APIs we use internally currently provide more than 1, so we were just future proofing it. You could have more than one if you were streaming a video and then decide to seek to a random position, it would then start to download a new section of the video, so if it retained the first section buffer you would have 2 buffers, and in theory you could have many buffered regions if you did a bunch of seeking to unbuffered positions. It seems all the APIs that we use though will just trash any old buffers if you seek to a new position - hence why it'll never return more than 1 right now.

 

Seek commands are not buffered, so you would need to poll the video while it loads by calling either
PlayingPlayer.Control.HasMetaData() or PlayingPlayer.Control.CanPlay() and then seek, waiting for an event like
MetaDataReady or Started and then seek.

You could also consider joining all of your videos into a single video and then simply seeking within that video instead of loading separate videos.

I hope that helps you.

Thanks,

 

Yes this is totally possible. Currently we support the Facebook Audio 360 standard - which means you need to use their tools to convert the audio into the correct format (MKV with Opus audio). Then you need to enable FB Audio 360 option in the Platform Specific > Android section in the Audio Output mode. There is a section in the documentation about this. Currently we support this sort of 360 surround audio on Android and Windows desktop only.

Thanks,

 

Hi,

The Save to PNG is just a sort of debugging tool we use, so it is not guaranteed to have the correct gamma behaviour. (we have however fixed this in version 2.0 which will be released soon).

But I think you're saying that you have a gamma issue anyway in the video display? Can I ask how are you displaying the video (eg DisplayUGUI, ApplyToMesh etc), or are you using your own shaders (gamma correction requires use of our AVPro Video shaders)?

Also are you able to replicate this problem using our included demo scenes? This would indicate a bug in our system.

Thanks,

 

Hi! Im setting up the http header using the add function httpheaders using the format

LoadingPlayer.PlatformOptionsAndroid.httpHeaders.Add(
new MediaPlayer.PlatformOptions.HTTPHeader("Cookie",
"CloudFront-Policy=XXXXX; " +
"CloudFront-Key-Pair-Id=XXXXX; " +
"CloudFront-Signature=XXXXX; "));

Im getting the " Error: Loading failed. File not found, codec not supported, video resolution too high or insufficient system resources." error.

Im currently using

• Exo Player (default)
• m3u8
• Unity Editor 2019.4.3 (Target Android)
• AVPro Video 1.11.5

Any suggestions? Thanks in advance!

Trying building to Android, and changing the video extension:

• Exo Player (default)
• mpd
• Android 9
• AVPro Video 1.11.5

But still no luck. Im getting the same error. I ve doubled checked, and the address is working fine. Is there any way to see which is being returned in the request?

 

I've been using AVPRO for my Unity Project VR Project using Windows configuration and XR input system. It works well about 90% of the time. However once in a while, quite randomly, the video playing starts stuttering (or fps drops significantly). When I simply reload the SAME video, stuttering stops. I've been trying to identify what causes this as it is very intermitent but frequent enough, but I'm having problems finding how to debug this.

I'm using a Mesh Renderer on a octahedron sphere who has the Applyto Mesh script and the Update Stereo Material script. The player itself is using the Media Foundation video API with hardware decoding.

Have you had any recent stuttering reported? Is there any way to see the fps, memory usage or other parameters that may cause this?

 

Hi,

I'm not sure what the issue could be. Perhaps you could post the full android log (via adb log cat)? Also if you post the details to our GitHub issues then our support team can investigate:
https://github.com/RenderHeads/UnityPlugin-AVProVideo/issues

Thanks,

 

Hi,
Thanks for reporting this issue. No I haven not heard of any issues with stuttering lately. Make sure you don't have the 'low latency' option enabled in Platform Specific > Windows. It would also be useful to know the specs of your video and also your Unity version, AVPro Video, system specs etc.

If you post the details to our GitHub Issues then our support team will be able to investigate:
https://github.com/RenderHeads/UnityPlugin-AVProVideo/issues

Thanks,

 

I'm trying to figure out if AVPro Video Android will do what we need. It does a LOT of stuff and this thread is very, very long, so it's hard to tell for sure. Also tips on how to do it would be helpful. I have the demo version to try things out on.

What we want to do is live stream the output from a consumer PC-connected webcam (or potentially other cameras) and send it into the Quest VR headset (android) and put it on a monitor there. Can AVPro do this? I think the answer is yes. So, how? What flavor of different settings would be best. Any recommendations on the software to use on the PC side that will generate the kind of feed AVPro would like? We would like to keep the latency as low as possible, and to work on both intranet and internet applications. The stream can be unicast, if that simplifies things at all.

Thanks! I'm so glad to see an actively maintained component. I researched this all day yesterday and there's so many that half work, are abandoned, etc.

(Edit: Right now I'm using VLC as a tester to stream the video. That doesn't have to be permanent, though. OBS is an option as well. I haven't had a lot of luck getting AVPro to open the stream VLC is producing, even though I can get a VLC client to read it. I've tried having VLC stream the webcam, streaming a file, and streaming other streams like the apple one in the AVPro manual. I've tried it with transcoding turn on and off in VLC. The VLC client can see them, but AVPro always says file not found, codec not supported, video resolution too high or insufficient system resources. All this testing has been done on the PC, none on the Quest yet.)

 

Hi,

Thanks, yes maintaining software for a long time is difficult and not such a popular trend

AVPro Video might be able to do what you want. On Android it can play back HLS, MPEG-DASH and RTSP streams. Do you know what sort of stream VLC or OBS is creating? If it's an RTSP stream then you will need to change the Video API setting (in MediaPlayer > Platform Specific > Android) to MediaPlayer, as the default ExoPlayer API doesn't have RTSP suppport.

I hope that helps but if you could give more details then we can make some more suggestions.

Thanks,

 

Hello, can you tell me how to adjust volume at runtime? I tried setting MediaPlayer.m_volume, but it doesn't seem to change the volume of the track playing. Is it an issue that I'm using System Direct?

Thanks!

Update: Never mind, I found the MediaPlayer.Control.SetVolume method

 

[AVProVideo] Browser permission prevented video playback
I don't know what it really means

 

Could you tell us which platform and which browser you have this on?

Some browsers do not allow videos to auto play (eg Safari), unless the site is added to a list of allowed sites for autoplaying media.

 

Hi!

So, I reported the error using the httpHeaders a few days ago. I also find this error reported in the github:

https://github.com/RenderHeads/UnityPlugin-AVProVideo/issues/56

Can I downgrade my AVProVideo version to see if it works in that aprticular one? Im using version 1.11.5 with no luck, I wanted to try version 1.10.1

Thanks!

 

Thanks for the reply. I wound up getting it to work with OBS-> Nimble Streamer with RTMP. Of course, that doesn't help for AVPro since it doesn't do RTMP. I tried doing RTSP but on two different broadcasting computers and it made OBS go wonky (wouldn't actually start the stream, and stopping it would make it hang). I just wound up playing my RTMP stream back with VLC to verify it worked.

One thing it did highlight was lag. For what we're doing, a lag of several seconds would make it pretty excruciating. What we're doing is more akin to Zoom, so that kind of lag would just kill it. I'm going to try to experiment with HLS now, which apparently AVPro/Nimble Streamer likes but for OBS you have to jump through some hoops.

I also wonder if some sort of peer-to-peer solution might be better, perhaps using AVPro Live Camera. If you have any recommendations on that I'd be interested. We can control everything on the computer that records and sends the video, so we can pick our solution. Our main issue is that all this stuff seems to be incredibly pricey, simply because there are some big players in the market. So there seems to be super expensive monthly payment stuff and free stuff, with not much in between. We just want to get video from one single person to another person (inside VR on a Quest).

 

Unity asset store unfortunately doesn't keep old versions.

But you can download the old trial versions here:
1.10.1:
http://downloads.renderheads.com/2019/UnityPlugin-AVProVideo-v1.10.1-Trial.unitypackage

1.10.2:
http://downloads.renderheads.com/2020/UnityPlugin-AVProVideo-v1.10.2-Trial.unitypackage

Yes if this is a regression bug then it would be good to narrow down where it was introduced. Our Android developer is a away for a few days now but we should be able to pick this up soon. Let's try to keep all comms in the GitHub issue so the support guys can track it.

Thanks,

 

How can I play a sound effect at an exact frame or a rough estimate in the video playback.

For now I hacked together the VCR.cs example to play it with these variables in update to do a playMeTime - 0.1 < d && d < playMeTime + 0.1 to check its in between

Code (CSharp):

1. float time = PlayingPlayer.Control.GetCurrentTimeMs();
2.             float duration = PlayingPlayer.Info.GetDurationMs();
3.             float d = Mathf.Clamp(time / duration, 0.0f, 1.0f);
4.  
5.             Debug.Log(string.Format("time: {0}, duration: {1}, d: {2}", time, duration, d));

 

Hey RenderHeads,

We are currently experiencing a problem with stereo VR playback on Mac. We are using the shader AVProVideo/VR/InsideSphere Unlit (stereo+fog) Stereo UV for our player. When we run the application on our windows editor / build there are no issues with the playback and everything is as expected. But when we run a Mac editor or build, the stereo player plays the video's with 2 separate eyes instead of merging the 2 eyes for VR playback.

Note we updated AVPro as we purchased licensing for Mac recently. Prior to this we were just using Windows license with watermarks, but the same shader.

Thanks

 

We have the same problem. In editor, the 360 view is wrong. On another device (Oculus Go) it's alright.

We're using the same shader (AVProVideo/VR/InsideSphere)

 

@RenderHeads, what is the recommended way to encode/play 360 videos with ambisonic audio with MediaPlayer?

 

I suspect the issue is actually with Windows. The Mac should work fine.

Please check that you have set the Stereo Packing mode correctly on the MediaPlayer component (in Visual section). Also make sure that you have a UpdateStereoMaterial component in your scene set up correctly.

On Windows I suspect it looks like it's working, when it actually isn't. If your video has stereo metadata embedded then our Windows decoder will only render 1 eye (left). You need to strip (or not embed in the first place) the stereo metadata from the video fine.

If you do both of the above then it should work.

Thanks,

 

I'm not sure which platform your editor is running on, nor what 'wrong' means. If it's a stereo issue and you're on Windows in the editor then I suspect your video contains stereo metadata. On Windows we don't support videos containing stereo metadata as it causes only one of the eyes (left) to render. If you remove the stereo metdata (or not embed it in the first place) then the issue should be solved.

Thanks,

 

Yes, we have included a demo scene that demonstrates this. Currently we only support 360 audio on Windows and Android. The plugin doesn't support ambisonic audio, only Facebook Audio 360 format. There are tools to convert from ambisonic to FB360 audio, and the AVPro Video documentation has instructions on how to do this.

Thanks,

 

Your code looks correct.

 

Hey Nathan,

All the fields are correctly set in Mac with top down packing, UpdateStereomaterial is fine as well. I also tried the test scene and had the same issues.

Sorry i forgot to mention that we are using Force eye mode to render one eye, my apologies.

 

Hi! @AndrewRH I just bought AvPro,it's a nice Plugin, but there's a weird problem when I use user32.dll functions in Standalone build, once I called "SetForegroundWindow" or "SetFocus" something like that, I'm unable to call OpenVideoFromFile(), otherwise the application will freeze(not Crash, just no responding), it seems that as long as I use user32.dll, the MediaPlayer(WindowsMediaPlayer in my case) stop working. I'm confusing and I hope you can help me with that.

 

@NathanRH

 

Hi,

That sounds very strange.

Could you paste the code that causes this issue?
Also which version of AVPro Video are you using, and on which version of Windows?

Thanks,

 

Ahh, I see.
Perhaps to help debug this you could use the included top-bottom alpha-packed video in StreamingAssets - since that doesn't contain any stereo metadata - see if that behaves correctly.

Otherwise if that doesn't help to work out what's wrong, perhaps you could share a screenshot to show the problem you're seeing on the Mac and your setup.

Thanks,

 

Code (CSharp):

1. [DllImport("user32.dll")] static extern bool SetForegroundWindow(IntPtr hWnd);
2.  
3.     [DllImport("user32.dll")]
4.     private static extern System.IntPtr FindWindow(String lpClassName, String lpWindowName);
5.  
6.     public void Foreground()
7.     {
8.         IntPtr hWnd = FindWindow(null, "MyApp");
9.         SetForegroundWindow(hWnd);
10.     }

After called this little function, I can't call OpenVideoFromFile(). if I set the video path of MediaPlayer in Editor everything is fine, but if I open video via script after Foreground(), built and run, it will freeze.
I'm using the latest version AvPro Video 1.11.5.
and I'm using Windows10.

 

Hello,
Avpro mp4 player works well in Unity 2019 (Webgl1).
But I am having problems with the same configuracion in UNITY 2018.3.13f
Please help us.
Best regards,

 

Hi,

Happy to help. We would need more details. Such as:
1) What exactly is the problem?
2) Which platform and browser?
3) Which version of AVPro Video?
4) What does the browser developer console logs report?

Thanks,

 

I suspect that there is some problem with the call to those native functions.

SetForegroundWindow() for example requires the return value to be marshaled from unmanaged bool type:

[DllImport("user32.dll")]
[return: MarshalAs(UnmanagedType.Bool)]
static extern bool SetForegroundWindow(IntPtr hWnd);

FindWindow() seems correct, though perhaps the string types need marshalling type set..

I hope the above bool marshaling solves the issue...

Thanks,

 

Thank you for your reply, It seems that FindWindow() string type do need marshalling, but I'm pretty much a newbie at this, I have tried to marshal string type to every possible option but none of them works, the application still freeze.

 

Will you be adding Direct3D 12 support?

 

Hi,
When I use URL mode to play video on IOS，unity just crashed。 I've debugged it many times，The discovery was caused by “ _mediaPlayer.OpenVideoFromFile(_location, _mediaPlayer.m_VideoPath, true)”; What is the reason。
My AVProVideo version is 1.11.1

Thanks,